Pages

Saturday, June 30, 2012

Software Quality Assurance (QA) checklist


Initial quality of released software contributes to reputation, customer retention, bug/issue fixing cycles and overall cost of ownership. Validating and ensuring “Ultimate Quality” using “Quality Assurance Practices” has become an essential step of development life-cycle.


Think of following as a high level glance at components of a “Successful QA Practice” or a “QA Checklist”... May be even "Designing good Quality in". 
[This is not a guide on how to create individual test cases or a guide on what type of tests to create].



[Start of Project] Setup Core Roles of QA-Practice
Appoint a Test (Quality Assurance) lead; that is responsible for validating an issue free solution.
Appoint a test lead from the design/development team [development team has testing too].
Identify Business-Analysts [that are responsible for defining proper requirements; so, QA can test for expected quality of delivery].
Identify Business Product Owners [that is responsible for articulating the change, confirming the understanding and validating final solution].
Depending on the project, you may also identify various “Subject Matter Experts”; that must contribute to final quality [from both IT and Business].
[This will ensure that there are lead roles responsible for quality form each area (Business/BA/Dev/QA). For very large projects, (for each area) there could be feature-leads reporting to an overall lead.]

[During Project] Core processes needed for solution quality
Identify all stakeholders and a strategy to engage them [part of quality is what they say it is].
Establish a process for requirement gathering and validating gathered requirements.
QA team is responsible for defining tests to validate requirements.
Agree on types of testing to be done by development team [which is different from QA team testing].
Agree on types of testing to be done by QA team.
Agree on types of testing to be done by business [product owners].
Agree on types of testing or validating to be done by identified subject matter experts.
Identify a “User Experience” lead [that is chasing user satisfaction. Such as Documentation, Training, usability etc.]
[You may also identify any tools that will be used for bug/issue recording and trend analysis.]

Business Analyst responsibilities related to Quality
Business Analysts are to clearly document and present Vision, Scope and Requirements.
Understand acceptance criteria [avoid the cost of unnecessary features, development, testing etc.]
Take the time to clearly communicate requirements to designers, developers, testers etc.
[Involve non business analysts in client meetings early]
Validate solution against original requirements.

Design/Development Teams Quality related Responsibilities
Solution Architect should identify and maintain an open list of planned deliverable [documents, code, deployment scripts etc.].
Development must test deliverables thoroughly prior to others testing.
[Unit Tests – testing individual software components (automating will help testing consistency).]
[Integration Testing – making sure complete integrated solution works (automating will help testing consistency).]
Communicate changes in the latest release to QA.
Setup needed testing environments.
Sign off by Design/Development lead; as release is verified.

Test Team Responsibilities
Test lead should create a “Test Plan” for funding and resource assigning (dates, people, locations etc.).
QA team members to define test-cases based on collected requirements [functional and non-functional].
[Collect a list of all feature classes to be testing… all features, deployment, security, usability, accessibility, various platforms, current standard adherence etc.]
Record Bugs/issues and pass it back to development
Provide periodic test status reports
Record known issues that will not be fixed [document any workarounds].
Complete “Requirement Traceability Matrix”
Produce a “Release-Quality” report prior to approval [not all releases are perfect].
Sign off by QA lead.

Test Preparation
Decide on tools needed for testing… install.
Decide a process for bug/issue recording, communication and analysis.
[Development Team] Setup needed test environments
Create test cases and get approval.
Clearly specify test platforms used.
Setup “Requirement Traceability Matrix”.

User Experience
[“Microsoft Solutions Framework” defines a responsibility chain to ensure ultimate user experience]
User experience specialists are to track down all possible user-categories and ensure satisfaction.
User specialists are to chase end-user experience, such as documentation, training, usability etc.
Don’t forget all types of users… operations, help-desk, regulatory, auditors, specific subject-matter-experts etc.
[Business Analysts often play this role]

User Acceptance Testing
Enlist needed business users and get commitment
Assist business in creating test cases
Facilitate UAT
Conduct a post-test survey to gather sentiment
Sign off by business prior to release.

Post Deployment and Quality
Where possible, test system in production.
Record post production bugs/Issues and resolution for later analysis.
Analyze post production bugs/issues for root causes; and implement corrective project lifecycle processes.
[Production-Support should conduct regular meetings to discuss current-production-problems, planning for pending changes (need more system space, need more support staff in six months etc.)].

Help Desk Processes related to continued Quality
Left for you :-)

Various “Development Lifecycle Gates” contributing to final quality
“Vision and Scope” (and charter) review by IT and approval by client.
Report pending change to operations early.
Approve business requirements [IT and Client].
Approve Functional Specification.
Approve Solution Architecture.
Approve planned deliverables [code, solution, documentation, training etc.].
Review [IT] and approve [client] Project Plans [Various Plans including test plan].
“Scope to Solution” comparison by QA and Business Analysts.
IT to validate and approve final development deliverable as “Scope Completed”.
Approval by QA Team.
Business to approve UAT.
IT to approve “Release readiness” of all deliverables.
Client to accept final release and roadmap of next-steps.
Post deployment, verification and approval of “Solution Stability” in production.
All future maintenance funding is planned for [licenses etc.]
Final release to be signed-off by each lead.
Support team to accept product from project team.


Here are some manager type questions [be it a single project or the whole enterprise]

  • Who is responsible for quality?
  • Are people rewarded for contributing to higher quality?
  • What are we doing to educate staff?
  • Are there quality measuring/monitoring mechanisms setup?
  • Do quality gap findings lead to corrective actions?
  • Do we know the cost of running QA processes?  
  • Can the cost of QA processes justify the benefits?



Seems like it takes the village to raise Quality.