Analysis and evaluation of collaborative modeling processes

Abstract

Analysis and evaluation of collaborative modeling processes is confronted with many challenges. On the one hand, many systems design and re-engineering projects require collaborative modeling approaches that can enhance their productivity. But, such collaborative efforts, which often consist of the “modeling process” and the “modeling products” – the models – that are generated thereof, are facilitator-driven. Analysis of such collaborative efforts requires us to have a glimpse inside the modeling process to know what takes place therein. Evaluation requires us to determine not only the quality of the models, but also that of the “modeling artifacts” that are used in, and produced from, such a collaborative modeling effort. However, the process of modeling is still a gray area – a black box that needs breaking the seal so that we can study, analyze and understand what takes place during collaborative modeling. Moreover, studying, analyzing and understanding what takes place during collaborative modeling still lacks a well-structured analysis and evaluation methodology. In our research, we offer not only the well-structured methodology for analysis and evaluation, but also an explanatory and descriptive theory that can be used to study, analyze and understand what takes place during collaborative modeling. Using communication theory as a basis, we develop the “Rules-Interactions-Models (RIM)” framework for the study and analysis of the collaborative modeling process. We also develop the “Collaborative Modeling Evaluation (COME)” framework, based on the Analytic Hierarchy Process (AHP), for the evaluation of the different modeling artifacts used in, and produced from, such a collaborative modeling process. The two frameworks are integrated in a meta-model that can serve as a template or blueprint for deriving the actual analysis and evaluation structures for the construction of a support-tool that incorporates analysis and evaluation concepts.