CISQ Guidance is Complementary to Existing Frameworks
Select from this list or scroll down
ISO - International Organization for Standardization
The ISO/IEC 25000 series of standards, also known as SQuaRE (System and Software Quality Requirements and Evaluation), contains a framework for evaluating software product quality. ISO/IEC 25010 defines a set of eight software quality characteristics, including Security, Reliability, and Maintainability. ISO/IEC 25023 describes how to measure software product quality using these characteristics, and largely describes quality at the behavioral level. For example, ISO/IEC 25023 measures Reliability as availability rather than as source code weaknesses that cause reliability problems. In general, ISO/IEC 25023 does not address the underlying issues in source code affecting the reliability or other quality characteristics of software-intensive systems.
To supplement ISO, CISQ has enumerated specific vulnerabilities in system source code and architecture that underlie the quality characteristics of Security, Reliability, Performance Efficiency and Maintainability and cause unwanted behavior. The Reliability standard developed by CISQ is true to ISO/IEC 25010 definition and is composed from critical violations of architectural and coding practice that affect the availability, fault tolerance, recoverability, and data integrity of an application. ISO has defined product quality and measured aspects of system behavior, while CISQ measures quality at the source code level by defining and measuring structural weaknesses in software underlying each characteristic. This level of standardization advances the automation of measuring software structural quality. Dr. Bill Curtis, Founding Executive Director of CISQ, is participating as a co-editor in the revision of ISO 25010. For more on CISQ and ISO, click here.
Common Weakness Enumeration
The Common Weakness Enumeration (CWE) repository is maintained by The MITRE Corporation on behalf of the U.S. Department of Homeland Security. The CWE repository contains a list of over 800 known software weaknesses that hackers have exploited to gain unauthorized entry or cause malicious behavior. Each CWE is labelled with a unique identifier and description. The standards developed by CISQ for automating the measurement of Security, Reliability, Performance Efficiency and Maintainability are composed of the most critical and impactful weaknesses in the CWE. See the set of CISQ coding rules here, mapping back to the CWE.
ITU - International Telecommunication Union
ITU-T, or the Telecommunication Standardization Sector of ITU, has standardized the MITRE Common Weakness Enumeration (CWE). See Recommendation ITU-T X.1524. The CISQ standards for automating the measurement of Security, Reliability, Performance Efficiency and Maintainability are composed of the most critical and impactful weaknesses in the CWE. See the set of CISQ coding rules here, mapping back to the CWE.
NIST - National Institute of Standards and Technology
NIST is an agency of the U.S. Department of the Commerce. The Information Technology Laboratory (ITL) at NIST develops and deploys standards, tests, and metrics to make U.S. information systems more secure, usable, interoperable, and reliable. The NIST Cybersecurity Framework is used by government and industry to map their cybersecurity, software assurance, and risk management practices to various levels of capability. Standards from CISQ are actionable metrics for software risk assessment that can be used within an organization’s implementation of NIST’s Cybersecurity Framework. For more on CISQ and the NIST Cybersecurity Framework, click here.
CMMI - Capability Maturity Model Integration
CMMI is a process-based improvement framework developed at the Software Engineering Institute (SEI) at Carnegie Mellon University with support from the U.S. Department of Defense. CMMI is an expansion to product systems of the original Capability Maturity Model for Software (CMM) developed by SEI in 1991. CMMI is currently administered by the CMMI Institute, a subsidiary of ISACA. CMMI is a framework for measuring the maturity of an organization’s software and system development processes. The framework consists of five maturity levels, each representing a plateau in an organization’s transition from inconsistent, undisciplined development activities to standardized, quantitatively managed, and innovative processes. CMMI measures software processes, but not the products built through them.
CISQ supplements CMMI by measuring the quality of a software product. Both practices can be used together to improve an organization’s ability to develop and maintain high quality software. Dr. Bill Curtis, Founding Executive Director of CISQ, led development of CMMI in the 1990’s. CMMI is referenced in U.S. Department of Defense and government contracts. CISQ measures are starting to be cited in these contracts as well.
SPICE - Software Process Improvement and Capability Determination
SPICE is an ISO/IEC 15504 standard for process assessment. The SPICE framework is currently being transitioned to the ISO/IEC 33000 family of standards. The SPICE/ISO 33000 standard is structured around a reference model divided into two dimensions: a process dimension and a capability dimension. SPICE is similar to CMMI in that it’s used to measure the maturity of an organization’s individual software development processes. CISQ supplements SPICE/ISO 33000 by providing standards to measure software products developed through these processes.
COBIT - Control Objectives for Information and Related Technologies
COBIT is an ISACA framework for the governance and management of enterprise IT. The framework includes a set of IT controls mapped to processes and business risks. COBIT is commonly referenced in audits and regulatory compliance reports. COBIT 5 was released in 2012. COBIT is a process standard that does not provide product measures. Thus, CISQ supplements COBIT in the same way it supplements CMMI and SPICE by providing product-level measures for software applications.
ITIL - Information Technology Infrastructure Library
ITIL is a collection of IT Service Management practices that focus on aligning IT services with the needs of business. The British standard 15000 and the subsequent ISO/IEC 20000 Service Management standard are based on the ITIL framework. ITIL was developed by the British government’s Central Computer and Telecommunications Agency (CCTA) in the 1980s. Since 2013, ITIL is owned by Axelos which licenses businesses to use the ITIL framework in their consulting practice, while managing updates and process changes. However, ITIL is free for internal use. CISQ supplements ITIL in the same way it supplements CMMI, SPICE, and COBIT by providing product-level measures for software applications.
IFPUG - International Function Point Users Group
Function Points are used for software sizing. IFPUG is an international organization of function point analysts. IFPUG develops and manages a standard set of counting rules for Function Point Analysis. ISO/IEC 20296:2009 specifies the definitions, rules, and steps for applying the IFPUG’s functional size measurement method.
Function Points are traditionally measured manually by IFPUG-certified function point counters. CISQ developed a standard for Automated Function Points (AFP) that can be computed by software tools. The AFP standard stayed as close to IFPUG counting guidelines as possible while eliminating the ambiguity to support automation. The benefits of an automated measure include speed, cost-efficiency, and repeatable, consistent counts that eliminate the ± 10% variance among counters. The AFP measure from CISQ is an OMG standard and also ISO/IEC 19515:2019.
Nesma - formerly an acronym for Netherlands Software Metrics Users Association
Nesma is a software industry organization in the Netherlands focused on software metrics and measurement. Similar in scope to IFPUG, Nesma promotes Function Point Analysis for measuring the functional size of a piece of software. Nesma manages the COSMIC method which is an alternative to the IFPUG method for functional size measurement. The COSMIC method has been approved by ISO as ISO/IEC 19761:2011.
MISRA - Motor Industry Software Reliability Association
MISRA develops secure coding guidelines for programming languages used in embedded control systems and software in motor vehicles and other industries. MISRA publications include MISRA C, MISRA C++, MISRA SA, and MISRA Autocode. MISRA C is a set of software development guidelines and coding rules for the C programming language. Its aims to facilitate code safety, security, portability and reliability in the context of embedded systems in critical environments. While MISRA is focused on embedded systems, CISQ supplements MISRA with rules primarily designed for non-embedded IT business systems.