Hyrax Release Testing Subgroup

Purpose:

• Working with Samvera community, develop a process for ensuring Hyrax releases are stable and tested.

Goals:

• To understand all of the features in Hyrax releases and document them. 

Google Hangout: https://hangouts.google.com/hangouts/_/c4dvatljlzeofeapgzlljsok2ie

Members:

@Chris Diaz

@Steve Van Tuyl

@Sherry Lake

@jrudder

@Harsh Parekh

Consults:

@Michael Joseph Giarlo - dev team

@Jenn Colt - ux team

Michael Tribone and Kate Deibel said they were willing to consult on process for accessibility testing. 

Documents: 

• UI interactions

• Hyrax testing process proposal

• UI testing tracking document (Example of a potential model, only)

Questions:

Why not automate, why human testing?

What about UX and accessibility?

Meeting notes:

We will produce two documents:

UI Interactions = Call it "Hyrax Feature Guide"

UI Testing Tracking = Call it "Release testing template" and make a new version of it with every release. 

Complete Hyrax Feature Guide document - for Hyrax 2.0 

• How do we do this before sandbox? 

• Decide Audience: -We want to make this a manual that explains all the features, and admin. 

• 1.3. walkthrough - Chris (mark features that are not in 1.3, add missing interactions) by Sept 1st. 

• Regarding Format: categories seem good, the repeating the word interaction is distraction. Needs more examples, that address "Use this for" examples. Ask Steve what he has in mind.

• What is in Hyrax 2 and when can we get a list.  

Release Testing Process Notes: 

• Need bug reporting template. 

• What is a time frame that is acceptable for returning results of test? (2-3 weeks) 

• The turn around time for updating the 2 important documents is of concern. 

• We think we need 3 institutions to commit to this work for each release ahead of time. Would be good to rotate these institutions. 

• We would love for 2 institutions to commit to upgrading in-production systems before wide-release but realize this is hard. 

• Minor release should have less process? 

  • update the 2 documents

  • new features should be tested

  • how thoroughly do all features need to be tested? 

• Accessibility testing should be included in tab, but needs some expertise. What is the minimal that we support? Should automated testing be done as a minimal, consult with Michael and Kate. 

  • http://wave.webaim.org/

  • Use Firefox for NVDA testing with a screen reading. Either Windows or Mac

  • Identify what is minimal support (stated?) 

Before the Release Testing Template is complete, we need to complete UI interaction and answer these questions: 

• What browsers and platforms should be included, what are supported? 

  • After asking the UX group and the Repo-managers here is a suggestion about how we decide what browsers. 

    • Include browsers that the majority of users used based on data from http://caniuse.com/usage-table

    • Include support for high-use browsers in different areas of the world. http://gs.statcounter.com/

    • Are there browser versions to consider for accessibility?

    • Admin and staff tools may consider reduced amount of support for certain platforms. 

    • Browserstack is a good tool for testing. 

    • The list of browser support needs to be considers for each release. 

  • List to test for Hyrax 2.0 release will include the following: (browser version popularity)

    • Chrome on Windows 

    • Chrome on Mac OS X 

    • Chrome on Android (for public interactions) (most popular android versions)

    • Safari on iPhone (for public interactions)

    • Firefox and Safari should be added (for public interactions?)

    • Use http://wave.webaim.org/ (any browser?)

    • Use Firefox for NVDA testing with a screen reading. Either Windows or Mac

  • The testing document will include what versions of OS and browsers. Institutions are welcome to test additional browsers to indentify gaps and report issues. 
    
    
    

To do's: by Aug 18th

Ask repo-managers what platforms and OS should be tested/supported

Ask Michael/Kate testing for accessibility

Develop a list of features to be included in 2.0 

Chris does 1.3 walkthrough 

Finish a template for the testing document,

• Add instructions for how to use the document -Sherry 

• Mirror the structure and organization of the UI Interactions documents.

Bug reporting template. Would the new issue helper text work or do we need something else from students? 

Open