Modeling Biological Systems Using Crowdsourcing

• Autorzy: Wasik S.

Identyfikatory

DOI

10.1515/fcds-2018-0012

• Języki publikacji: EN

Abstrakty

EN

Crowdsourcing is a very effective technique for outsourcing work to a vast network usually comprising anonymous people. In this study, we review the application of crowdsourcing to modeling systems originating from systems biology. We consider a variety of verified approaches, including well-known projects such as EyeWire, FoldIt, and DREAM Challenges, as well as novel projects conducted at the European Center for Bioinformatics and Genomics. The latter projects utilized crowdsourced serious games to design models of dynamic biological systems, and it was demonstrated that these models could be used successfully to involve players without domain knowledge. We conclude the review of these systems by providing 10 guidelines to facilitate the efficient use of crowdsourcing.

Słowa kluczowe

EN

computer game; crowdsourcing; dynamic system; modeling; serious game; systems biology

• Wydawca: Wydawnictwo Politechniki Poznańskiej
• Czasopismo: Foundations of Computing and Decision Sciences
• Rocznik: 2018
• Tom: Vol. 43, No. 3
• Strony: 219--243
• Opis fizyczny: Bibliogr. 74 poz., rys., tab.

Twórcy

autor

Wasik S.

• Institute of Computing Science, Poznan University of Technology, Poznan, Poland; Institute of Bioorganic Chemistry, Polish Academy of Sciences, Poznan, Poland; European Center for Bioinformatics and Genomics, Poznan University of Technology, Poznan, Poland

Bibliografia

• [1] An G., Mi Q., Dutta-Moscato J., and Vodovotz Y. Agent-based models in translational systems biology. Wiley Interdisciplinary Reviews: Systems Biology and Medicine, 1(2):159-171, 2009.
• [2] Ansari S., Binder J., Boue S., Fabio A.D., Hayes W., Hoeng J., Iskandar A., Kleiman R., Norel R., O’neel B., Peitsch M.C., Poussin C., Pratt D., Rhrissorrakrai K., Schlage W.K., Stolovitzky G., and Talikka M. On crowd-verification of biological networks. Bioinformatics and Biology Insights, 7:BBI.S12932, 2013.
• [3] Bae J.A., Mu S., Kim J.S., Turner N.L., Tartavull I., Kemnitz N., Jordan C.S., Norton A.D., Silversmith W.M., Prentki R., et al. Digital museum of retinal ganglion cells with dense anatomy and physiology. bioRxiv, page 182758, 2018.
• [4] Barone J., Bayer C., Copley R., Barlow N., Burns M., Rao S., Seelig G., Popovic Z., Cooper S., and Players N. Nanocrafter: Design and evaluation of a dna nanotechnology game. In Proceedings of the 10th International Conference on the Foundations of Digital Games, pages 1-5, 2015.
• [5] Barrett T., Wilhite S.E., Ledoux P., Evangelista C., Kim I.F., Tomashevsky M., Marshall K.A., Phillippy K.H., Sherman P.M., Holko M., et al. Ncbi geo: archive for functional genomics data sets-update. Nucleic acids research, 41(D1):D991-D995, 2012.
• [6] Bladek I., Chojnacki T., Miazga K., and Szwachla J. Platform for evaluation of algorithms estimating real paremeters in biological systems (Platforma do oceny algorytmow wyznaczania paramterow rzeczywistych w systemach biologicznych). Master’s thesis, Faculty of Computing, Poznan University of Technology, 2014.
• [7] Bonabeau E. Decisions 2.0: The power of collective intelligence. MIT Sloan management review, 50(2):45, 2009.
• [8] Bruggeman F.J. and Westerhoff H.V. The nature of systems biology. Trends in Microbiology, 15(1):45-50, 2007.
• [9] Cookson C. Online gaming yields first results for alzheimer’s research. Financial Times, 2017.
• [10] Cooper S., Khatib F., Treuille A., Barbero J., Lee J., Beenen M., Leaver-Fay A., Baker D., Popović Z., et al. Predicting protein structures with a multiplayer online game. Nature, 466(7307):756, 2010.
• [11] Costello J.C. and Stolovitzky G. Seeking the Wisdom of Crowds Through Challenge-Based Competitions in Biomedical Research. Clinical Pharmacology & Therapeutics, 93(5):396-398, feb 2013.
• [12] Dahari H., Ribeiro R.M., and Perelson A.S. Triphasic decline of hepatitis C virus RNA during antiviral therapy. Hepatology, 46(1):16-21, 2007.
• [13] Dawson R. and Bynghall S. Getting Results From Crowds: The definitive guide to using crowdsourcing to grow your business. Advanced Human Technologies Incl, 2012.
• [14] Deng N., Galliers R., and Joshi K. Crowdworking-a new digital divide? is design and research implications. In ECIS Conference Proceedings, 2016.
• [15] Derry J.M.J., Mangravite L.M., Suver C., Furia M.D., Henderson D., Schildwachter X., Bot B., Izant J., Sieberts S.K., Kellen M.R., and Friend S.H. Developing predictive molecular maps of human disease through community- based modeling. Nature Genetics, 44(2):127-130, jan 2012.
• [16] DiStefano III J. Dynamic systems biology modeling and simulation. Academic Press, 2015.
• [17] Doan A., Ramakrishnan R., and Halevy A.Y. Crowdsourcing systems on the world-wide web. Commun. ACM, 54(4):86-96, Apr. 2011.
• [18] Eiben C.B., Siegel J.B., Bale J.B., Cooper S., Khatib F., Shen B.W., Stoddard B.L., Popovic Z., and Baker D. Increased Diels-Alderase activity through backbone remodeling guided by foldit players. Nature biotechnology, 30(2):190, 2012.
• [19] Estellés-Arolas E. and González-Ladrón-De-Guevara F. Towards an integrated crowdsourcing definition. J. Inf. Sci., 38(2):189-200, Apr. 2012.
• [20] Fratczak F. Design of the algorithm for estimating values of parameters in biological models (Opracowanie algorytmu wyznaczajacego wartosci parametrow w modelach biologicznych). Master’s thesis, Faculty of Computing, Poznan University of Technology, 2016.
• [21] Fries P.A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends in Cognitive Sciences, 9(10):474-480, 2005.
• [22] Frings M., Berkels B., Behr M., and Elgeti S. Comparison of optimization algorithms for the slow shot phase in hpdc. AIP Conference Proceedings, 1960(1):110005, 2018.
• [23] Galton F. Vox populi (the wisdom of crowds). Nature, 75(7):450-451, 1907.
• [24] Good B.M. and Su A I. Crowdsourcing for bioinformatics. Bioinformatics, 29(16):1925-1933, 2013.
• [25] Haklay M. and Weber P. Openstreetmap: User-generated street maps. Ieee Pervas Comput, 7(4):12-18, 2008.
• [26] Helmstaedter M., Briggman K.L., Turaga S.C., Jain V., Seung H.S., and Denk W. Connectomic reconstruction of the inner plexiform layer in the mouse retina. Nature, 500(7461):168, 2013.
• [27] Hill S.M., Heiser L M., Cokelaer T., Unger M., Nesser N.K., Carlin D.E., Zhang Y., Sokolov A., Paull E.O., Wong C.K., et al. Inferring causal molecular networks: empirical assessment through a community-based effort. Nature methods, 13(4):310, 2016.
• [28] Hirschman L., Fort K., Boue S., Kyrpides N., Islamaj Doğan R., and Cohen K.B. Crowdsourcing and curation: perspectives from biology and natural language processing. Database, 2016:baw115, 2016.
• [29] Ho C.C. and Ting C.-Y. Measuring crowd sourced analytics: A review. International Information Institute (Tokyo). Information, 19(10B):4891, 2016.
• [30] Hodgkin A.L. and Huxley A.F. A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol, 117(4):500- 544, Aug 1952.
• [31] Horowitz S., Koepnick B., Martin R., Tymieniecki A., Winburn A.A., Cooper S., Flatten J., Rogawski D.S., Koropatkin N.M., Hailu T.T., et al. Determining crystal structures through crowdsourcing and coursework. Nature communications, 7:12549, 2016.
• [32] Howe J. The rise of crowdsourcing. Wired magazine, 14(6):1-4, 2006.
• [33] Kawrykow A., Roumanis G., Kam A., Kwak D., Leung C., Wu C., Zarour E., Players P., Sarmenta L., Blanchette M., and WaldispÃ(Ehl J. Phylo: A citizen science approach for improving multiple sequence alignment. PLOS ONE, 7(3):1- 9, 03 2012.
• [34] Kim J.S., Greene M.J., Zlateski A., Lee K., Richardson M., Turaga S.C., Purcaro M., Balkam M., Robinson A., Behabadi B.F., et al. Space-time wiring specificity supports direction selectivity in the retina. Nature, 509(7500):331, 2014.
• [35] Kitano H. Foundations of Systems Biology. The MIT Press, 2001.
• [36] Kittur A., Chi E.H., and Suh B. Crowdsourcing user studies with mechanical turk. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 453-456. ACM, 2008.
• [37] Lee J., Kladwang W., Lee M., Cantu D., Azizyan M., Kim H., Limpaecher A., Gaikwad S., Yoon S., Treuille A., Das R., and . Rna design rules from a massive open laboratory. Proceedings of the National Academy of Sciences, 111(6):2122-2127, 2014.
• [38] Lee J. and Seo D. Crowdsourcing not all sourced by the crowd: An observation on the behavior of wikipedia participants. Technovation, 55:14-21, 2016.
• [39] Liang J., Qu B., Suganthan P., and Hernández-Díaz A.G. Problem definitions and evaluation criteria for the CEC 2013 special session on real-parameter optimization. Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China and Nanyang Technological University, Singapore, Technical Report, 201212:3-18, 2013.
• [40] Logas H., Whitehead J., Mateas M., Vallejos R., Scott L., Shapiro D., Murray J., Compton K., Osborn J., Salvatore O., and others. Software Verification Games: Designing Xylem, The Code of Plants. In Foundations of Digital Games 2014 (FDG2014), pages 1-8. Center for Games and Playable Media, 2014.
• [41] Luengo-Oroz A.M., Arranz A., and Frean J. Crowdsourcing malaria parasite quantification: An online game for analyzing images of infected thick blood smears. J Med Internet Res, 14(6):e167, Nov 2012.
• [42] Marbach D., Costello J.C., Kiiffner R., Vega N.M., Prill R.J., Camacho D.M., Allison K.R., Aderhold A., Bonneau R., Chen Y., et al. Wisdom of crowds for robust gene network inference. Nature methods, 9(8):796, 2012.
• [43] Mavandadi S., Feng S., Yu F., Dimitrov S., Yu R., and Ozcan A. Biogames: A platform for crowd-sourced biomedical image analysis and telediagnosis. Games for Health Journal, 1(5):373-376, 2012. PMID: 23724363.
• [44] Mesarovic M. Systems Theory and Biology. Springer Verlag, 1968.
• [45] Meyer P., Cokelaer T., Chandran D., Kim K.H., Loh P.-R., Tucker G., Lipson M., Berger B., Kreutz C., Raue A., Steiert B., Timmer J., Bilal E., Sauro H.M., Stolovitzky G., and Saez-Rodriguez J. Network topology and parameter estimation: from experimental design methods to gene regulatory network kinetics using a community based approach. BMC Systems Biology, 8(1):13, Feb 2014.
• [46] Mietchen D., Wodak S., Wasik S., Szostak N., and Dessimoz C. Submit a topic page to plos computational biology and wikipedia. PLOS Computational Biology, 14(5):1-4, 05 2018.
• [47] Nurse P. Reductionism: The ends of understanding. Nature, 387(6634):657, 1997.
• [48] Peer E., Samat S., Brandimarte L., and Acquisti A. Beyond the turk: An empirical comparison of alternative platforms for crowdsourcing online behavioral research. NA - Advances in Consumer Research, 43:18-22, 2016.
• [49] Pico A.R., Kelder T., van Iersel M.P., Hanspers K., Conklin B.R., and Evelo C. Wikipathways: Pathway editing for the people. PLOS Biology, 6(7):1-4, 07 2008.
• [50] Powell M.J. A view of algorithms for optimization without derivatives. Mathematics Today-Bulletin of the Institute of Mathematics and its Applications, 43(5):170-174, 2007.
• [51] Prejzendanc T., Wasik S., and Blazewicz J. Computer representations of bioinformatics models. Current Bioinformatics, 11(5):551-560, 2016.
• [52] Prelec D., Seung H.S., and McCoy J. A solution to the single-question crowd wisdom problem. Nature, 541(7638):532, 2017.
• [53] Saez-Rodriguez J., Costello J.C., Friend S.H., Kellen M.R., Mangravite L., Meyer P., Norman T., and Stolovitzky G. Crowdsourcing biomedical research: leveraging communities as innovation engines. Nature Reviews Genetics, 17(8):470-486, jul 2016.
• [54] Schmidt M. and Lipson H. Distilling Free-Form Natural Laws from Experimental Data. Science, 324(5923):81-85, apr 2009.
• [55] Schrier K. Investigating typologies of games as research environments. In Proceedings of International Conference on the Foundations of Digital Games, pages 1-4, 2017.
• [56] Szostak N., Synak J., Borowski M., Wasik S., and Blazewicz J. Simulating the origins of life: The dual role of RNA replicases as an obstacle to evolution. PLOS ONE, 2017.
• [57] Szostak N., Wasik S., and Blazewicz J. Hypercycle. PLOS Computational Biology, 12(4):e1004853, apr 2016.
• [58] Szuba T.T., Polański P., Schab P., and Wielicki P. On efficiency of collective intelligence phenomena. In Transactions on computational collective intelligence III, pages 50-73. Springer, 2011.
• [59] Tellioglu U., Xie G.G., Rohrer J.P., and Prince C. Whale of a crowd: Quantifying the effectiveness of crowd-sourced serious games. In 2014 Computer Games: AI, Animation, Mobile, Multimedia, Educational and Serious Games (CGAMES), pages 1-7, July 2014.
• [60] Tomita, Hashimoto, Takahashi, Shimizu, Matsuzaki, Miyoshi, Saito, Tanida, Yugi, Venter, and Hutchison. E-CELL: Software environment for whole cell simulation. Genome Inform Ser Workshop Genome Inform, 8:147-155, 1997.
• [61] Tyagi M., Hashimoto K., Shoemaker B.A., Wuchty S., and Panchenko A.R. Large-scale mapping of human protein interactome using structural complexes. EMBO reports, 13(3):266-271, 2012.
• [62] Van Pelt C. and Sorokin A. Designing a scalable crowdsourcing platform. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, SIGMOD ’12, pages 765-766, New York, NY, USA, 2012. ACM.
• [63] Von Ahn L., Maurer B., McMillen C., Abraham D., and Blum M. recaptcha: Human-based character recognition via web security measures. Science, 321(5895):1465-1468, 2008.
• [64] Wang Z., Monteiro C.D., Jagodnik K.M., Fernandez N.F., Gundersen G.W., Rouillard A.D., Jenkins S.L., Feldmann A.S., Hu K.S., McDermott M.G., et al. Extraction and analysis of signatures from the gene expression omnibus by the crowd. Nature communications, 7:12846, 2016.
• [65] Wasik S., Antczak M., Badura J., and Laskowski A. Evaluation as a service architecture and crowdsourced problems solving implemented in optil. io platform. arXiv preprint arXiv:1807.06002, 2018.
• [66] Wasik S., Antczak M., Badura J., Laskowski A., and Sternal T. Optil.io: Cloud Based Platform For Solving Optimization Problems Using Crowdsourcing Approach. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion, pages 433-436, San Francisco, CA, USA, 2016. ACM, ACM Digital Library.
• [67] Wasik S., Antczak M., Badura J., Laskowski A., and Sternal T. A survey on online judge systems and their applications. ACM Comput. Surv., 51(1):3:1-3:34, Jan. 2018.
• [68] Wasik S., Fratczak F., Krzyskow J., and Wulnikowski J. Inferring Mathematical Equations Using Crowdsourcing. PLOS ONE, 10(12):e0145557, dec 2015.
• [69] Wasik S., Jackowiak P., Figlerowicz M., and Blazewicz J. Multi-agent model of hepatitis C virus infection. Artificial Intelligence in Medicine, 60(2):123-131, feb 2014.
• [70] Wasik S., Prejzendanc T., and Blazewicz J. ModeLang - a new approach for experts-friendly viral infections modeling. Computational and Mathematical Methods in Medicine, 2013:8, 2013.
• [71] Wawrzyniak P. Methods of integrating micro-tasking websites with crowdsourced serious games (Metody integracji serwisow mikro-zadaniowych z grami komputerowymi adresowanymi do tlumow). Master’s thesis, Faculty of Computing, Poznan University of Technology, 2018.
• [72] Wulnikowski J. and Krzyskow J. Design and implementation of the game utilizing crowdsourcing to model dynamic systems (Opracowanie gry wykorzystujacej crowdsourcing do modelowania systemow dynamicznych). Master’s thesis, Faculty of Computing, Poznan University of Technology, 2016.
• [73] Xintong G., Hongzhi W., Song Y., and Hong G. Brief survey of crowdsourcing for data mining. Expert Systems with Applications, 41(17):7987-7994, 2014.
• [74] Zhan C. and Yeung L.F. Parameter estimation in systems biology models using spline approximation. BMC Systems Biology, 5(1):14, Jan 2011.

Uwagi

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2018).