Overview

• Date: Thursday, October 02, 2008
• Time: 10:30 am to 12:00 pm
• Location: T22 Boardroom
• Topic: Catalyst GradeBook / Student Web Services testing collaboration (Dan Walker / Kerry Lamb)
• Attendees: Alin Hunter, Jeani Wells, Kerry Lamb, Mary Syre, Dan Walker, Marcy Tufarolo, Chris Heiland, Linda Braziel, Frank Williston,, Alexis Raphael, Larai Wushishi, Ahuvah Reese

Catalyst testing:

• Due to scheduling, the Catalyst GradeBook was deployed using LDAP instead of web services, which turned out to be better for the testers, since they could focus on the user interface instead of testing both the UI and a new data source (multiple variables can cloud test issues).
• The catalyst team has a test tool which allows them to impersonate users in order to take advantage of 'real' test data, but it can be difficult to locate the right data to use.
• The Catalyst team has a testing process that includes a code freeze, so that the only changes allowed are resolutions to security, data loss or critical problems. 
• Catalyst deploys new releases every Wednesday morning, but there is also always a rollback plan, and QA has the final call.  If QA has determined that a release isn't ready for production, the code is rolled back.
  

Catalyst/Web Services testing:

• Both teams have their own bug tracking systems (Bugzilla for Catalyst, TestTrack Pro for Student Web Services).  Team members log problems in 'their' tool but have access to view entries in both tools. Where necessary, problems are manually transferred from one tracking system to the other (since the anticipated number is small).
• Triage is performed to determine the source of a bug (CGB or service).
  

Continuous testing and cost of running the mainframe:

Catalyst runs continuous tests, a process which highlights problems early.  On the mainframe, the processing power is paid for using a 'pre-paid phone card' model, which means every ping against the mainframe costs something.  We had a discussion about the best way to use the resources we have given the costs:

Things to think about:

• Risk management (e.g., more test cycles are worth the higher cost for high risk / high churn projects)
• Schedule test cycles instead of full system tests for every change (pro: lower cost, con: more difficulty isolating sources of problems - more variables with a batch of changes vs. few variables every change)
• How much testing?
• What's it cost?
• Where can we offload (e.g., do compares on windows vs. mainframe)
• Could we put something between .net and COBOL (loop back with dummy data)?
  

Possible future topics:

• Bug tracking demos
• Test Plans / Test plan template to use for new projects
• Anne Hopkins - Risk Assessment / Threat Analysis
• Quality planning & project start up
• Common agreement regarding acceptable quality levels and how to measure (internal and vender packages - what are we willing to accept?  to risk?
  

Next Meeting (TBD)