A code reviewer assignment model incorporating the competence differences and participant preferences

• Autorzy: Wang Y. , Wang X. , Jiang Y. , Liang Y. , Liu Y.

Identyfikatory

DOI

10.1515/fcds-2016-0004

• Języki publikacji: EN

Abstrakty

EN

A good assignment of code reviewers can effectively utilize the intellectual resources, assure code quality and improve programmers’ skills in software development. However, little research on reviewer assignment of code review has been found. In this study, a code reviewer assignment model is created based on participants’ preference to reviewing assignment. With a constraint of the smallest size of a review group, the model is optimized to maximize review outcomes and avoid the negative impact of “mutual admiration society”. This study shows that the reviewer assignment strategies incorporating either the reviewers’ preferences or the authors’ preferences get much improvement than a random assignment. The strategy incorporating authors’ preference makes higher improvement than that incorporating reviewers’ preference. However, when the reviewers’ and authors’ preference matrixes are merged, the improvement becomes moderate. The study indicates that the majority of the participants have a strong wish to work with reviewers and authors having highest competence. If we want to satisfy the preference of both reviewers and authors at the same time, the overall improvement of learning outcomes may be not the best.

Słowa kluczowe

EN

peer code review; reviewer assignment problem (RAP); code review assignment model; preference matrix; subtour-size constraint; mutual admiration society

• Wydawca: Wydawnictwo Politechniki Poznańskiej
• Czasopismo: Foundations of Computing and Decision Sciences
• Rocznik: 2016
• Tom: Vol. 41, No. 1
• Strony: 77--91
• Opis fizyczny: Bibliogr. 24 poz., rys. tab.

Twórcy

autor

Wang Y.

• School of Management, Harbin Institute of Technology, No.13, Fayuan St., Nangang Dist., Harbin 150001, P. R. China

autor

Wang X.

• School of Management, Harbin Institute of Technology, Harbin 150001, P. R. China

autor

Jiang Y.

• Department of Information Systems, College of Business Administration, California State University Long Beach, CA 90840, USA

autor

Liang Y.

• Department of Information Systems, College of Business Administration, California State University Long Beach, CA 90840, USA

autor

Liu Y.

• Department of Information Systems, College of Business Administration, California State University Long Beach, CA 90840, USA

Bibliografia

• [1] Chen Y., Fan Z.P., A Method for proposal-reviewer assignment in proposal review based on the match degree of research discipline, Chinese Journal of Management Science, 19, 2, 2011, 169-173 (in Chinese).
• [2] Cook W.D., Golany B., Kress M., Penn M., Raviv T., Optimal allocation of proposals to reviewers to facilitate effective ranking, Management Science, 51, 4, 2005, 655-661.
• [3] Devito Da Cunha A., Greathead D., Does personality matter? An analysis of code-review ability, Communications of the ACM, 50, 5, 2007, 109-112.
• [4] Drexl M., Irnich S., Solving elementary shortest-path problems as mixed-integer programs, OR spectrum, 36, 2, 2014, 281-296.
• [5] Fagan E., Design and code inspections to reduce errors in program development, IBM System Journal, 3, 1976, 182-211.
• [6] Karimzadehgan M., Zhai C.X., Integer linear programming for constrained multi-aspect committee review assignment, Information Processing and Management, 48, 4, 2012, 725-740.
• [7] Li X., Using peer review to assess coding standards–a case study, In Frontiers in education conference, 36th annual, San Diego, CA, USA, 2006, 9-14.
• [8] Li X., Incorporating a code review process into the assessment, In the 20th Annual Conference of the National Advisory Committee on Computing Qualifications, Nelson, New Zealand, 2007, 125-131.
• [9] Li X., Watanabe T., Automatic paper-to-reviewer assignment based on the matching degree of the reviewers, Procedia Computer Science, 22, 2013, 633-642.
• [10] Long C., Wong R.C.W., Peng Y., Ye L., On good and fair paper-reviewer assignment, In IEEE 13th International Conference on Data Mining (ICDM’2013), 2013, 1145-1150.
• [11] Martello S., Pulleyblank W.R., Toth P., de Werra D., Balanced optimization problems, Operations Research Letters, 3, 5, 1984, 275-278.
• [12] Meyer B., Design and code reviews in the age of the internet, Communications of the ACM, 51, 9, 2008, 66-71.
• [13] Saaty T.L., How to make a decision: the analytic hierarchy process, European journal of operational research, 48, 1, 1990, 9-26.
• [14] Sun Y.H., Ma J., Fan Z.P., Wang J., A hybrid knowledge and model approach for reviewer assignment, Expert Systems with Applications, 34, 2, 2008, 817-824.
• [15] Tayal D.K., Saxena P.C., Sharma A., Khanna G., Gupta S., New method for solving reviewer assignment problem using type-2 fuzzy sets and fuzzy functions, Applied intelligence, 40, 1, 2014, 54-73.
• [16] Topping K., Peer Assessment Between Students in Colleges and Universities, Review of Educational Research, 68, 3, 1998, 249-276.
• [17] Tsang E.W.K., Is this referee really my peer? A challenge to the peer-review process, Journal of Management Inquiry, 22, 2, 2013, 166-171.
• [18] Turner S.A., Peer review in CS2: the effects on attitudes, engagement, and conceptual learning, Doctoral Dissertation of Virginia Polytechnic Institute and State University, 2009, Retrieved from http://scholar.lib.vt.edu/theses/available/etd-08272009-003738/unrestricted/Turner_SA_D_2009.pdf.
• [19] Wang F., Shi N., Chen B., A comprehensive survey of the reviewer assignment problem, International Journal of Information Technology and Decision Making, 9, 4, 2010, 645-668.
• [20] Wang Y.Q., Li H., Feng Y.Q., Jiang Y., Liu Y., Assessment of programming language learning based on peer code review model: Implementation and experience report, Computers & Education, 59, 2, 2012, 412-422.
• [21] Wang Y.Q., Li H., Sun Y.N., Jiang Y., Yu J., Learning outcomes of programming language courses based on peer code review model, In the 6th International Conference on Computer Science & Education, August 3-5, SuperStar Virgo, Singapore, ThC 5.47, 2011, 751-754.
• [22] Wang Y.Q., Li Y.J., Collins M., Liu P.J., Process improvement of peer code review and behavior analysis of its participants, ACM SIGCSE Bulletin, 40, 1, 2008, 107-111.
• [23] Wang Y.Q., Yang F., Liu P.J., Collins M., Quality assurance of peer code review process: A computer science based strategy, Zhongshan Daxue Xuebao/Acta Scientiarum Natralium Universitatis Sunyatseni, 46 (suppl), 2007, 116-120.
• [24] Xu Y.H., Ma J., Sun Y.H., Hao G., Xu W., Zhao D.T., A decision support approach for assigning reviewers to proposals, Expert Systems with Applications, 37, 10, 2010, 6948-6956.

Uwagi

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę.