Naohiko Tsuda, Hironori Washizaki, Kiyoshi Honda, Hidenori Nakai, Yoshiaki Fukazawa, Motoei Azuma, Toshihiro Komiyama, Tadashi Nakano, Hirotsugu Suzuki, Sumie Morita, Katsue Kojima, Akiyoshi Hando, “WSQF: Comprehensive Software Quality Evaluation Framework and Benchmark based on the SQuaRE,” Proceedings of the 41st ACM/IEEE International Conference on Software Engineering (ICSE 2019), Industry Track, pp. 1-10, Montreal, QC, Canada, 25 May – 31 May 2019. (CORE Rank A*)(acceptance rate 30/118=25%)

Conventional quality evaluations of software concentrate on specific quality characteristics. Moreover, the measurement data are limited to specific products and organizations. Consequently, the present state of product quality and quality-in-use characteristics are not fully understood, preventing effective decision-making for software stakeholders. To alleviate this problem, ISO/IEC defined international standards called the SQuaRE (Systems and software Quality Requirements an Evaluation) series for comprehensive quality measurement and evaluation. However, these standards remain rather general and abstract, making them difficult to apply. In this paper, we establish a SQuaRE-based comprehensive software quality evaluation framework, Waseda Software Quality Framework (WSQF), which concretizes many product quality and quality-in-use measurement methods originally defined in the SQuaRE series. By applying the WSQF to 21 commercial ready-to-use software products, we revealed the status of software product quality. A resulted comprehensive benchmark includes trends of the quality measurement values, relationships among quality characteristics, relationship between quality-in-use and product quality, and relationship between the quality characteristics and product contexts within the limits of an application.