Open Access

Quantitative Evaluation Model in Large Scale Innovation Competition Based on Statistical Theory

PDF

Mengyao Zhang ¹ , Ling Deng ¹ , Jiaxing Wang ¹ , Bin Zhao ¹ ,

⁴ School of Science, Wuhan University of Science and Technology, Wuhan, Hubei, China

⁴ School of Science, Hubei University of Technology, Wuhan, Hubei, China

Abstract

Large-scale innovation contests are becoming increasingly popular globally, with many innovators and enterprises actively participating. Work allocation optimization and fair judging are two significant concerns in innovation contests. This academic paper aims to recommend an optimized "cross- assignment" program by leveraging the optimal objective function method to improve the comparability of scores given by various judges. We conducted data-based descriptive statistical analysis and concluded that the two-stage and weighted evaluation schemes are more beneficial than the traditional judging scheme. Nonetheless, there are still some shortcomings that require addressing. To enhance fairness, we propose an improved two-stage evaluation scheme. In the first stage, we normalize the scores with a normal distribution. In the second stage of the process, we implement a system utilizing the Borda sorting technique to categorize submissions into five distinct groups for judges to evaluate based on their perceptions. We also detail a method for weighting tied scores to determine the final rankings. Testing indicates that this approach yields a Normalized Discounted Cumulative Gain (NDCG) of 0.8667, implying greater fairness and precision in the assessment of submissions.

Keywords

Evaluation program, Normalization of a normal distribution

References

Chesbrough H W, Vanhaverbeke W, West J, et al. Open Innovation: Researching a New Paradigm[J]. Wim Vanhaverbeke, 2006, 84(1):1259-1262.

Poetz M K, Schreier M. The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas [J]? Journal of Product Innovation Management, 2012, 29(2):245-256.

You Qinggen. Research and Implementation of Expert Selection Model for WISCO Research Project Review. Huazhong University of Science and Technology, 2014,32(7):56-60 (in Chinese)

Kim Y, Kim Y, Kim J, et al. Boosting on the functional ANOVA decomposition[J]. Statistics & Its Interface, 2009, 2(3):361-368.

Kannemann K. The Exact Evaluation of 2-way Cross-classifications: An Algorithmic Solution[J]. Biometrical Journal, 2010,24(2):157-169.

Song Yonghao, Shi Xiao, Hu Bin, et al. Team Building Algorithm Based on Multi-Objective Greedy Strategy for Gain Maximization[J]. High Technology Communications, 2018, 28(4): 279-290. (in Chinese)

Mielik Inen T, Ukkonen E. The Complexity of Maximum Matroid-Greedoid Intersection and Weighted Greedoid Maximization[J]. Discrete Applied Mathematics, 2006, 154(4):684-691.

Jia Tianli. Hypothesis Analysis Model of the Causes of Mean Differences[J]. Mathematics Practice and Understanding, 2005,64(9): 212-215. (in Chinese)

Daoji S. An analysis of variance test for the extreme value distribution[J]. Tianjin Daxue Xuebao, 1988, 1988(2):116-121.

Shapiro S S, Wilk M B. An analysis of variance test for normality (complete samples)[J]. Biometrika, 1975, 67(3):215-216.

Guohong S, Zhenhui Z. Application of Gray Correlation Analysis in Fault Tree Diagnosis[J]. China Safety Science Journal, 1999, 30(2):1505-1507.

Liu G, Yu J. Gray correlation analysis and prediction models of living refuse generation in Shanghai city[J]. Waste Management, 2007, 27(3):345-351.

Victor G, D. J T M. The Eurovision Song Contest: voting rules, biases and rationality[J]. Journal of Cultural Economics, 2022,47(2):247-277.

Looney, Marilyn A. Evaluating Judge Performance in Sport[J]. Journal of Applied Measurement,2004,5(1):31-47.

Margaret A. Cliff & Marjorie C. King A proposed approach for evaluating expert wine judge performance using descriptive statistics, Journal of Wine Research, 1996, 7(2), 83-90.

Wang Yonglin. Subjective Indicators in Educational Evaluation and Factors Affecting Their Judgment[J]. Education Science, 2017, 33(3): 14-19. (in Chinese)

Xu Kai. Exploration and Research on Retrospective Evaluation Selection Method Based on Peer Review Experts[J]. Beijing University of Chinese Medicine, 2017,7(4):47-50 (in Chinese)

Ryo I, Kazumasa O. Borda Count Method for Fiscal Policy: A Political Economic Analysis[J]. The Institute of Comparative Economic Studies, Hosei University, 2022, 36(7):25-40.

Chen Yuan. Research on Decision-Making Methods for Scientific Fund Project Review and Selection. Northeastern University, 2011,9(5):23-26 (in Chinese)

Lee S M, Kim K H, Kim E J. Selection and Classification of Bacterial Strains Using Standardization and Cluster Analysis[J]. Journal of Animal Science and Technology, 2012,32(6):54-56.

Zheng Rui. Ranking Risk with Borda Method. Enterprise Reform and Management, 2015,32(11): 154-156. (in Chinese)

Orouskhani M, Shi D, Cheng X. A Fuzzy Adaptive Dynamic NSGA-II With Fuzzy-Based Borda Ranking Method and its Application to Multimedia Data Analysis[J]. IEEE Transactions on Fuzzy Systems, 2021,29(8):118-128.

Tang Huanling, Wang Jingdong, Lu Yuchang. A New Weight Allocation Strategy to Reduce Similar Topic Classification Errors. Computer Engineering and Applications, 2004,58(13): 185- 188. (in Chinese)

Wang Zeyan, Gu Hongfang, Yi Xiaoxin, Zhang Shenru.A Linear Combination Weighting Method Based on Entropy. Systems Engineering Theory and Practice, 2003, 69(3): 112-116. (in Chinese)

Kilgour, Marc D, Grégoire , et al. Correction to: Weighted scoring elections: is Borda best?[J]. Social Choice and Welfare,2021,58(2):1-2.

Pierrat L, Samuel K. Standardization of the Logistic Distribution based on Entropy[J]. International Journal of Performability Engineering, 2013, 9(3):352-354.

Jian W, Qianqian M, Hu-Chen L. A meta-evaluation model on science and technology project review experts using IVIF-BWM and MULTIMOORA[J]. Expert Systems With Applications,2021,36(3),68-70.

Ke Nianri, Zhang Chaomo, Wang Yingwei. Application of Normal Distribution Method in Well Logging Curve Standardization in M Oilfield. Journal of Yangtze University (Natural Science Edition), 2010, 7(4): 76-78. (in Chinese)

Aroian, Leo A. Handbook of the Normal Distribution[J]. Technometrics, 1996, 25(1):112-115.

Taylor M. ABSTRACT SoftRank: Optimising Non-Smooth Rank Metrics[J]. Inproceedings, 2008(Volume 7):77-86.

Most read articles by the same author(s)

<< < 1 2 3 4 5 6 > >>

Critique Open Research & Review

Quantitative Evaluation Model in Large Scale Innovation Competition Based on Statistical Theory

Abstract

Keywords

References

Most read articles by the same author(s)

Similar Articles