Quantitative evaluation of crowd intelligence innovation system health: An ecosystem perspective

Crowd intelligence innovation (CII) is an open problem-solving model, typically performs on a crowdsourcing-based online innovation platform, which is highly dependent on the engagement and high-quality work of participants. On an online platform, many demanders publish tasks, and innovators submit schemes, which forms a complex CII system. To ensure such a CII system operates sustainably, it is critical for the platform operators to know the current system's health statuses before taking proper management actions. However, the large influx of participators with varied behaviors makes it extremely difficult to evaluate the health statuses of the system and capture its characteristics. Practicable method for quantitatively evaluating the health status of a CII system is still missing. Therefore, this study proposed a new quantitative health status evaluation method from an ecosystem perspective. First, the crowd intelligence innovation ecosystem (CIIE) was established by mapping the CII system onto the natural ecosystem based on ecological theory. Second, the health status of CIIE was described by its internal health status and external health status using a mixed method. The internal health status of the CIIE was defined and evaluated based on its health characteristics in terms of stability, ascendency, development capacity, and redundancy measurements using ecological network analysis. The external health status was evaluated using the ecological entropy index system. As the primary verification of the feasibility of our proposed approach, a case study of Zhubajie was conducted. These results confirmed the feasibility of our evaluation method. With this method, the health status of CIIEs could be accurately measured, which supported the management and organization the CII systems. Future work will focus on adapting suitable measures to improve the operation of CII.