A checklist of “Solution Qualities”

When building systems... we often come across various operational characteristics the solution should have. One system may require a system failure-recovery-time to be less than 15 minutes, while another system may require real-time monitoring.

If such non-functional requirements and expectations are not formally collected and published, each team member is left to assume such details of final deliverable. One member may decide to build a self-healing core... while another deciding on a failure alerting and managing system.

Here is a formal list of such potential “Solution Qualities” published with “Microsoft Solutions Framework” [MSF].

Following should be considered for every effort and implemented as appropriate

Performance-Oriented Qualities of Service
Responsiveness  - Allowable amount of delay when responding to an action, call, or event. Concurrency - Ability to perform well when operated concurrently with other aspects of a solution and with its environment. Efficiency - Capability to provide appropriate performance, relative to the resources used, under stated conditions. Fault tolerance - Capability to maintain a specified level of performance irrespective of faults or of infringement of system resources. Scalability - Ability to handle simultaneous operational load. Extensibility - Ability to extend a solution without significant rework.
Trustworthiness-Oriented Qualities of Service
Security - Ability to prevent access and disruption by unauthorized users, viruses, worms, spyware, and other agents. Privacy - Ability to prevent unauthorized access or viewing of data.
User Experience-Oriented Qualities of Service
Accessibility - Extent to which individuals with disabilities have access to and use of information and data that is comparable to the access to and use by individuals without disabilities. Attractiveness - Measure of visual appeal to users. Compatibility - Conformance to conventions and expectations. Discoverability - Ability of a user to find and learn features of a solution. Ease of use - Cognitive efficiency with which a user can perform desired tasks. Localizability - Ability to be adapted to conform to the linguistic, cultural, and conventional needs and expectations of a specific group of users.
Manageability-Oriented Qualities of Service
Availability - Degree to which it is operational and accessible when required for use. Often expressed as a probability. Reliability - Capability to maintain a specified level of performance when used under specified conditions, commonly stated as mean time between failures (MTBF). Installability and uninstallability - Capability to be installed in a specific environment and uninstalled without altering the environment’s initial state. Maintainability - Ease of modification to correct faults, improve performance or other attributes, or adapt to a changed environment. Monitorability - Extent to which health and operational data are automatically collected while operating. Operability - Extent to which a solution is controlled automatically while operating. Portability - Capability to be transferred from one environment to another. Recoverability - Capability to reestablish a specified level of performance and recover the data directly affected in the case of a failure. Testability - Degree to which the solution facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met. Supportability - Extent to which operational problems are corrected. Reusability - Ability to repurpose solution components with little rework. Conformance - Adherence to applicable rules, regulations, policies, and standards. Interoperability - Capability to interact with other solutions in the environment.

To be Practical

1. Pay attention to required “Solution Qualities” from the beginning of a project.
2. Keep a generic list of solution qualities that you can compare your solution requirements to.
3. Publish the agreed upon qualities (and value) that must be present in the delivery.
4. Get sign-off for high-cost items [i.e. 99.999% uptime is an expensive requirement].
5. Don’t forget to estimate building and support cost of each “Quality” [need should be balanced against implementation and maintenance cost].
6. Post-production "stability and bug" issues can also be categorized and monitored against original "Qualities" as “issues due to non-functional-complexity X”.