Hyrax Release Testing Subgroup

Owned by jrudder
Last updated: Sept 19, 2017 by Harsh Parekh


Purpose:

  • Working with Samvera community, develop a process for ensuring Hyrax releases are stable and tested.

Goals:

  • To understand all of the features in Hyrax releases and document them. 

Google Hangout: https://hangouts.google.com/hangouts/_/c4dvatljlzeofeapgzlljsok2ie


Members:

Chris Diaz

Steve Van Tuyl

Sherry Lake

jrudder

Harsh Parekh

Consults:

Michael Joseph Giarlo - dev team

Jenn Colt - ux team

Michael Tribone and Kate Deibel said they were willing to consult on process for accessibility testing. 

Documents: 


Questions:

Why not automate, why human testing?

What about UX and accessibility?



Meeting notes:

We will produce two documents:

UI Interactions = Call it "Hyrax Feature Guide"

UI Testing Tracking = Call it "Release testing template" and make a new version of it with every release. 


Complete Hyrax Feature Guide document - for Hyrax 2.0 

  • How do we do this before sandbox? 
  • Decide Audience: -We want to make this a manual that explains all the features, and admin. 
  • 1.3. walkthrough - Chris (mark features that are not in 1.3, add missing interactions) by Sept 1st. 
  • Regarding Format: categories seem good, the repeating the word interaction is distraction. Needs more examples, that address "Use this for" examples. Ask Steve what he has in mind.
  • What is in Hyrax 2 and when can we get a list.  


Release Testing Process Notes: 

  • Need bug reporting template. 
  • What is a time frame that is acceptable for returning results of test? (2-3 weeks) 
  • The turn around time for updating the 2 important documents is of concern. 
  • We think we need 3 institutions to commit to this work for each release ahead of time. Would be good to rotate these institutions. 
  • We would love for 2 institutions to commit to upgrading in-production systems before wide-release but realize this is hard. 
  • Minor release should have less process? 
    • update the 2 documents
    • new features should be tested
    • how thoroughly do all features need to be tested? 
  • Accessibility testing should be included in tab, but needs some expertise. What is the minimal that we support? Should automated testing be done as a minimal, consult with Michael and Kate. 


Before the Release Testing Template is complete, we need to complete UI interaction and answer these questions: 


  • What browsers and platforms should be included, what are supported? 
    • After asking the UX group and the Repo-managers here is a suggestion about how we decide what browsers. 
      • Include browsers that the majority of users used based on data from http://caniuse.com/usage-table
      • Include support for high-use browsers in different areas of the world. http://gs.statcounter.com/
      • Are there browser versions to consider for accessibility?
      • Admin and staff tools may consider reduced amount of support for certain platforms. 
      • Browserstack is a good tool for testing. 
      • The list of browser support needs to be considers for each release. 
    • List to test for Hyrax 2.0 release will include the following: (browser version popularity)
      • Chrome on Windows 
      • Chrome on Mac OS X 
      • Chrome on Android (for public interactions) (most popular android versions)
      • Safari on iPhone (for public interactions)
      • Firefox and Safari should be added (for public interactions?)
      • Use http://wave.webaim.org/ (any browser?)
      • Use Firefox for NVDA testing with a screen reading. Either Windows or Mac
    • The testing document will include what versions of OS and browsers. Institutions are welcome to test additional browsers to indentify gaps and report issues. 


To do's: by Aug 18th

Ask repo-managers what platforms and OS should be tested/supported

Ask Michael/Kate testing for accessibility

Develop a list of features to be included in 2.0 

Chris does 1.3 walkthrough 

Finish a template for the testing document,

Draft template (see comment below on contents)

  • Add instructions for how to use the document -Sherry 
  • Mirror the structure and organization of the UI Interactions documents.

Bug reporting template. Would the new issue helper text work or do we need something else from students?