Last update: December 18th, 2021
Practitioners and researchers study best practices to design machine learning (ML) application
systems and software to address quality and constraint problems. Such practices are often
formalized as design patterns. In this study, a multi-vocal literature review identified 15 software engineering design patterns for ML applications. A questionnaire survey inquired about ML developers’ use of the ML design patterns to validate them in practice. 118 ML developers responded to our survey. Results show that developers were unfamiliar with most of the patterns, although there are several major patterns already used by 20+% of the respondents. For all patterns, most of the respondents would consider using them in future designs. As the respondents became more consistent in their approach to design problems by reuse, the pattern usage ratio increased. These findings suggest that there are opportunities to increase the patterns’ adoption in practice by raising awareness of such patterns within the community.
Project members
- Hironori Washizaki, Dept. of Computer Science and Engineering, Waseda University, Tokyo, Japan, National Institute of Informatics, Tokyo, Japan
- Yann-Gael Gueheneuc, Ptidej Team, DGIGL, Ecole Polytechnique de Montreal, Quebec, Canada
- Foutse Khomh, Ptidej Team, DGIGL, Ecole Polytechnique de Montreal, Quebec, Canada
- Hironori Takeuchi
- Naotake Natori
- Takuo Doi
- Satoshi Okuda
- Nobukazu Yoshioka
Funds and Grants
This work was supported by JST-Mirai JPMJMI20B8 Engineerable AI Project, JSPS JPJSBP 120209936, and KAKENHI 21KK0179.
Publications
- Hironori Washizaki, Foutse Khomh, Yann-Gael Gueheneuc, Hironori Takeuchi, Naotake Natori, Takuo Doi, Satoshi Okuda, “Software Engineering Design Patterns for Machine Learning Applications,” IEEE Computer, Vol. 55, No. 3, pp. 1-9, 2022. (preliminary preprint)
- Nobukazu Yoshioka, Hironori Washizaki, Eduardo B. Fernandez, Tomoko Kaneko, Shuichiro Yamamoto, Fuyuki Ishikawa, Foutse Khomh, Giuliano Antoniol, Proceedings of the International Workshop on Evidence-based Security and Privacy in the Wild and the 1st International Workshop on Machine Learning Systems Engineering, CEUR Workshop Proceedings, Vol-2809, 2021.
- Hnin Thandar Tun, Jati H. Husen, Nobukazu Yoshioka Yoshioka, Hironori Washizaki and Yoshiaki Fukazawa, “Goal-Centralized Metamodel Based Requirements Integration for Machine Learning Systems,” 3rd International Workshop on Machine Learning Systems Engineering (iMLSE 2021), December 6, 2021, Virtual – in Conjunction with APSEC 2021
- Nobukazu Yoshioka, Jati H. Husen, Hnin Thandar Tun, Zhenxiang Chen, Hironori Washizaki and Yoshiaki Fukazawa, “Landscape of Requirements Engineering for Machine Learning-based AI Systems,” 3rd International Workshop on Machine Learning Systems Engineering (iMLSE 2021), December 6, 2021, Virtual – in Conjunction with APSEC 2021
- Jomphon Runpakprakun, Sien Reeve Ordonez Peralta, Hironori Washizaki, Foutse Khomh, Yann-Gael Gueheneuc, Nobukazu Yoshioka, Yoshiaki Fukazawa, “Software Engineering Patterns for Machine Learning Applications (SEP4MLA) – Part 3 – Data Processing Architectures,” 28th Conference on Pattern Languages of Programs (PLoP 2021), pp. 1-11, 2021.
- Jati H. Husen, Hnin Thandar Tun, Nobukazu Yoshioka, Hironori Washizaki and Yoshiaki Fukazawa, “Goal-Oriented Machine Learning-Based Component Development Process,” ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS), Poster Track, pp. 1-2, online, Sun 10 – Fri 15 October 2021.
- Yasuhiro Watanabe, Hironori Washizaki, Kazunori Sakamoto, Daisuke Saito, Kiyoshi Honda, Naohiko Tsuda, Yoshiaki Fukazawa, Nobukazu Yoshioka, “Preliminary Literature Review of Machine Learning System Development Practices,” 45th IEEE Computer Society Signature Conference on Computers, Software and Applications (COMPSAC 2021), Fast Abstract, pp. 1-2, 2021.
- Hironori Washizaki, Hironori Takeuchi, Foutse Khomh, Naotake Natori, Takuo Doi, Satoshi Okuda, “Practitioners’ insights on machine-learning software engineering design patterns: a preliminary study,” 36th IEEE International Conference on Software Maintenance and Evolution (ICSME 2020), Late Breaking Ideas track, Sep 26 – Oct 3, 2020.
- Hironori Washizaki, Hironori Takeuchi, Yann-Gaël Guéhéneuc, Foutse Khomh, Naotake Natori, Naohisa Shioura and Takuo Doi, “Machine-Learning Software-Engineering Design Patterns: Literature Review and Practitioners’ Insights,” Poster, 2nd International Workshop on Machine Learning Systems Engineering (iMLSE 2020), December 1, 2020, Virtual – in Conjunction with APSEC 2020
- Hironori Washizaki, Foutse Khomh, Yann-Gaël Guéhéneuc, Hironori Takeuchi, Satoshi Okuda, Naotake Natori, Naohisa Shioura, “Software Engineering Patterns for Machine Learning Applications (SEP4MLA) – Part 2”, the 27th Conference on Pattern Languages of Programs in 2020 (PLoP’20), October 12-16, 2020
- Hironori Washizaki, Foutse Khomh, Yann-Gael Gueheneuc, “Software Engineering Patterns for Machine Learning Applications (SEP4MLA),” 9th Asian Conference on Pattern Languages of Programs (AsianPLoP 2020), March 4-6, 2020, Taipei, Taiwan
- Hironori Washizaki, Hiromu Uchida, Foutse Khomh and Yann-Gaël Guéhéneuc, “Studying Software Engineering Patterns for Designing Machine Learning Systems,” The 10th International Workshop on Empirical Software Engineering in Practice (IWESEP 2019), Tokyo, Japan, on December 13-14, 2019.
Others and Data
- Valliappa Lakshmanan, Sara Robinson, Michael Munn著, 鷲崎弘宜, 竹内広宜, 名取直毅, 吉岡信和 訳, “機械学習デザインパターン – データ準備、モデル構築、MLOpsの実践上の問題と解決”（原著 “Machine Learning Design Patterns”）, オライリージャパン, 2021.
- Hironori Washizaki, List of documents and patterns identified by multivocal literature review of software engineering patterns for machine learning applications, zenodo, 2021
- List of patterns 機械学習応用システムのアーキテクチャ・デザイパターン (under construction)
Contact
Project leader: Hironori Washizaki, washizaki [at] waseda.jp
