Current Projects - Source Code Quality Measurement

Objective

Measure the quality of production source code and the development processes that produce that source code

Approach

  • Apply measurement theory to source code review and analysis to create a reproducible process that supplies quantitative estimates of source code and programming practice quality
  • Employ a fixed set of well-formed evaluation criteria to identify programming anomalies in a source code sample where each anomaly is described by the location in the source file where some aspect of that source code contradicts an assertion of an evaluation criterion and the contradicted evaluation criterion; these criteria are specific to the source code programming language and are drawn from and traceable to current best programming practices for that language
  • Use a software quality model developed from the programming technical literature to map the programming anomalies observed in a source code sample into 4 software quality categories: security, reliability, efficiency and maintainability
  • Normalize the total number of anomalies assigned to each quality category by the total sample size (in SLOC) to produce anomaly densities (in anomalies/SLOC) for each quality category as well as for the entire source code sample
  • Assess the quality of the programming practices applied when developing a source code sample by comparing source code quality measurements with the baseline values for the programming language and by measuring the adherence to the applicable organizational programming practices to software quality standards
  • Document each review as if reporting the results from experiments and with enough detail that a skilled independent observer can repeat the review process and obtain the same results within the process uncertainties

Results

  • Constructed sets of evaluation criteria for the C, C++, Java and PL/SQL programming languages from surveys of the technical literature on programming best practices, secure programming and reliable computing
  • Executed this process for > 39 reviews of production source code
  • Computed the statistics of these quality measures from the results from these reviews to define software quality baselines for the C, C++, Java and PL/SQL programming languages; The table below summarizes the mean anomaly densities measured for the source code samples in these programming languages
Programming Language Total No. of Reviews Performed Total SLOC Reviewed Mean Anomaly Density (anomalies/ SLOC)
Java 27 37564 0.339 ± 0.107
C 1 1586 0.283
C++ 4 6774 0.287 ± 0.041
PL/SQL 7 8497 0.445 ± 0.103
Totals/Mean 39 54421  

Conclusions

This project has demonstrated the ability of a measurement-based technique for source code review to produce meaningful quantitative assessments of source code and programming practice quality.