A summary of the management/administrative activities that relate to testing. This process was created based on activities performed during the Integration Test Bed project. See the Test Planning Checklist on the Project Management Portal for creating a test plan, as well as the Test Plan Template on this site.
Prepare a list of sources that are touched by this project; these To-Be-Changed and AffectedBys will become the Project Items list.
Identify the list of sources changing for the project (e.g., programs, copylibs, wfls, stored procedures, web pages). Include new and obsoleted items in this list. This list will be easier to work with if you include the 'title' of each source (a one-line description of the source) in addition to the source name.
Get a 1-line description of the change for each source in your list. This should indicate the magnitude of the change.
Provide the Project Items list to the accepting team (e.g., Production Support).
For MCP sources, the 'Reminder' function in SURE is one way to indicate these (flagged as 'To Be Changed By X Project').
Identify the affected items - the sources (pages, programs, modules, wfls) that use or otherwise are affected by the items being changed (e.g., the programs using each changed copylib or format, or report programs using new or changed data).
Put the Project Items in context
Identify the subsystems (Online, PM, Calc) or WFLs that are affected by each Project Item.
For each Project Item, gather additional info such as type of source (e.g., batch program or driver module or web page), whether it's an updater, when it runs (e.g., Calc, Timesheet night).
Identify needed environmental components (MCP, web, SQL, vitcos) for each Project Item.
Rank the Project Items
For each Project Item, add a ranking based on why it's on the list (e.g., an AffectedBy which calls an IO Mod with a copylib change gets a lower rank, while an AffectedBy which uses a changed copylib gets a higher rank). Rank changed items the highest. (This is referred to as the UsesRank in the TestBed project examples.)
Do a risk analysis
If not already done, determine the risk (if it fails in production) for each Project Item (High, Medium, Low). This should be done with a representative of the production team. This risk is independent of the type of change being made to the item. Questions to consider when determining risk:
If this fails in production, who is potentially affected, for example, a few central users, all staff, the president?
If this fails in production what is the impact, for example, a report of data available elsewhere is not printed, paychecks could be calculated incorrectly?
If this fails in production, how many other or downstream items could be affected? (e.g., one standalone program, or a copylib used in 100 sources?). How 'important' are they?
Is this an updater?
Based on the answers to these & other questions, assign the risk. This may be a subjective process, especially the first time around.
If some deliverables will be removed from production, 'Obsolete' can also be used as a risk. These items would be verified separately (e.g., references removed from all calling items).
Plan the test coverage
The combination of the Project Item Rank, Risk and Context will help you decide how much testing is 'enough' for each item.
Look at your project context (see Put the Project Items in context). If the bulk of your high-risk project items are in one subsystem, plan on hitting that area the hardest, but make sure you plan to cover at least one or two items from every subsystem.
Look at each project item and consider:
(Anticipated) Changed Lines of Code (for new items it's all of the code)
Is the change (based on loc) Small (1-5)? Med (6-20)? Large (20-100)? Huge (>100)?
Is it Obsolete? New? Changed?
What are the changes?
What's the Rank?
What's the Risk? (High risk items should have more testing)
What is the AffectedBy Count?
What Cycle?
SubSystem?
What general test types provide coverage? e.g.
Special Batch Test
Post-Deploy
Checks
Conversion
Downstream
Retirement Screens
HEPPS Screens
Special Tests for New items
Special Client Tests
OPUS testing
Batch Parallel Testing
WFL tests
See CoveragePlanning.xls for an example spreadsheet used to see where the tests fall (e.g., 30% in calc, 25% in OPUS).
Determine acceptance criteria for the tests (e.g., is the test considered successful if there are open high level bugs found for a given project item?)
Review the prioritization and the acceptance criteria with the accepting team (e.g., production support).
Plan the tests and the test environment
For each Project Item, write down a brief description of how to test it (e.g., parallel run, no differences except "ABCD" is changed to "EFGH", or the screen info for a changed module).
For projects that require more detailed testing, create sets of tests for each Project Item, group of Project Items, functional areas, etc. (e.g., system interfaces, security tests).
Think about what's needed in the environment to do these tests (e.g., batch programs need a batch environment in a certain state with specific test data, web modules need a web environment with a driver up, etc.)
Prioritize the tests based on the considerations above. (Tests for high-risk drastically changed items should have a higher priority than tests for low-risk, minimally changed items.)
Review the test notes and prioritization with the accepting team.
Decide how (and by whom) tests will be run and tracked.
Decide how (and by whom) the test environments will be managed.
Decide how (and by whom) defects will be captured, prioritized, and tracked (e.g., in the project tracking database or in a separate bug tracking database like RT or TestTrack); decide what to do with outstanding bugs at the end of the project.
Set up a project mailman list (e.g., CSRTest@u.washington.edu) for email notifications from the bug tracking system, plus other testing notifications. (Used to replace emails to clients (e.g., perms, file availability notifications) to avoid confusion with actual production. Note: the HEPPS team have the following emails set up for each SURE environment (hrpdev@u, hrpeval@u, hrprc@u and hrptrain@u) for this purpose.
Capture any environment-dependent changes made for testing (e.g., hardcoded emails) for later removal.
Map the tests to Requirements
A Baseline Traceability Matrix is a useful tool to determine if a test has been designed for every requirement (it documents the relationships between Requirements and Tests). This mapping can be useful for all test types, but is crucial for User Acceptance Testing and Functional Testing. Examples of a Traceability Matrix.
Map the tests to Project Items
For projects with more detailed tests, create a mapping of tests to the Project Items (e.g., Testcases A, B, C & D together verify Project Items 001 and 002, Testcase E verifies Project Item 003). This will help determine if a deliverable has been adequately covered during the test execution.
Several iterations of testing should be expected as defects are found and resolved, and as subsequent testing phases are implemented.
Track the Tests and Test Results
Create a spreadsheet, database, or some other file with the Project Items and test information. See what the Testbed project included in its Tracking Database.
Keep track of each Project Item status (e.g., Coded, Reviewed, UnitTested, SystemTested; test execution success/failure) by updating your Tracking Database regularly.
If new items are added to the project, add them to the database. If something isn't going to be changed after all, then update the tracking database accordingly (including its AffectedBys)
Manage the defect tracking database, and reflect those issues in the status, based on the acceptance criteria.
At the end of testing, prepare a summary of the test results (pass/fail) and outstanding issues.
After deployment, monitor the production defect tracking database to learn where improvements could be made in future testing.
Page Operations
Browse Space