Optimum Large Sensor Data Filtering, Networking and Computing

• Autorzy: Berlińska Joanna , Drozdowski Maciej , Robertazzi Thomas

Identyfikatory

DOI

10.15439/2023F5436

• Języki publikacji: EN

Abstrakty

EN

In this paper we consider filtering and processing large data streams in intelligent data acquisition systems. It is assumed that raw data arrives in discrete events from a single expensive sensor. Not all raw data, however, comprises records of interesting events and hence some part of the input must be filtered out. The intensity of filtering is an important design choice because it determines the complexity of filtering hardware and software and the amount of data that must be transferred to the following processing stages for further analysis. This, in turn, dictates needs for the following stages communication and computational capacity. In this paper we analyze the optimum intensity of filtering and its relationship with the capacity of the following processing stages. A set of generic filtering intensity, data transfer, and processing archetypes are modeled and evaluated.

Słowa kluczowe

EN

processor; scalability; distributed databases; filtering algorithm; data transfer; software

PL

procesor; skalowalność; rozproszone bazy danych; algorytm filtrowania; transfer danych; oprogramowanie

• Wydawca: Polskie Towarzystwo Informatyczne
• Czasopismo: Annals of Computer Science and Information Systems
• Rocznik: 2023
• Tom: Vol. 35
• Strony: 431--440
• Opis fizyczny: Bibliogr. 25 poz., wz., wykr.

Twórcy

autor

Berlińska Joanna

• Adam Mickiewicz University, Poznan, Poland

• https://orcid.org/0000-0003-2120-2595

autor

Drozdowski Maciej

• Poznan University of Technology Poznan, Poland

• https://orcid.org/0000-0001-9314-529X

autor

Robertazzi Thomas

• Stony Brook University Stony Brook NY. USA

• https://orcid.org/0000-0002-2382-1843

Bibliografia

• 1. M. Moges and T. Robertazzi, “Wireless sensor networks: Scheduling for measurement and data reporting,” IEEE Transactions on Aerospace and Electronic Systems, vol. 42, no. 1, pp. 327–340, 2006. http://dx.doi.org/10.1109/TAES.2006.1603426
• 2. J. Berlińska, “A comparison of priority rules for minimizing the maximum lateness in tree data gathering networks,” Engineering Optimization, vol. 54, pp. 218–231, 2022. http://dx.doi.org/10.1080/0305215X.2020.1861263
• 3. J. Berlińska, “Scheduling in data gathering networks with background communication,” Journal of Scheduling, vol. 23, pp. 681–691, 2020. http://dx.doi.org/10.1007/s10951-020-00648-5
• 4. J. Berlińska, “Heuristics for scheduling data gathering with limited base station memory,” Annals of Operations Research, vol. 285, pp. 149–159, 2020. http://dx.doi.org/10.1007/s10479-019-03185-3
• 5. J. Berlińska, “Scheduling for data gathering networks with data compression,” European Journal of Operations Research, vol. 246, pp. 744–749, 2015. http://dx.doi.org/10.1016/j.ejor.2015.05.026
• 6. T. Colombo, “Trigger & DAQ at the LHC, filtering data from 50 TB/s to 1 GB/s,” https://indico.cern.ch/event/825688/attachments/1872900/3082664/trigger_daq_at_lhc.pdf, CERN EP/LBC, July 2019, accessed 31/3/2022.
• 7. Committee on U.S.-Based Electron-Ion Collider Science Assessment, An Assessment of U.S.-Based Electron-Ion Collider Science. Washington D.C.: The National Academy of Science, Engineering and Medicine, The National Academy Press, 2018.
• 8. C. Toth and C. Jóźków, “Remote sensing platforms and sensors: A survey,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 115, pp. 22–36, 2016. http://dx.doi.org/10.1016/j.isprsjprs.2015.10.004
• 9. Y.-C. Cheng and T. Robertazzi, “Distributed computation with communication delay,” IEEE Transactions on Aerospace and Electronic Systems, vol. 24, no. 6, pp. 700–712, 1988. http://dx.doi.org/10.1109/7.18637
• 10. V. Bharadwaj, D. Ghose, V. Mani, and T. Robertazzi, Scheduling Divisible Loads in Parallel and Distributed Systems. Los Alamitos, CA: IEEE Computer Society Press, 1996.
• 11. T. Robertazzi, “Ten reasons to use divisible load theory,” IEEE Computer, vol. 36, no. 5, pp. 63–68, 2003. http://dx.doi.org/10.1109/MC.2003.1198238
• 12. M. Drozdowski, Scheduling for Parallel Processing. London: Springer, 2009.
• 13. H. Casanova, A. Legrand, and Y. Robert, Parallel Algorithms. London, UK: CRC Press, Taylor and Francis, 2009.
• 14. I. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, “A survey on sensor networks,” IEEE Communications Magazine, pp. 102–114, 2002. http://dx.doi.org/10.1109/MCOM.2002.1024422
• 15. P. Pereira, A. Grilo, and F. Rocha, End-to-End Reliability in Sensor Networks: Survey and Research Challenges in P. Pereira (ed), EuroFGI Workshop in IP Qos and Traffic Control. Academia, 2007.
• 16. T. Muhammed and A. Shaikh, “An analysis of fault detection strategies in wireless sensor networks,” Journal of Network and Computer Applications, vol. 78, pp. 267–287, 2017. http://dx.doi.org/10.1016/j.jnca.2016.10.019
• 17. M. Dener, “Security analysis in wireless sensor networks,” International Journal of Distributed Sensor Networks, vol. 10, no. 10, p. 303501, 2014. http://dx.doi.org/10.1155/2014/303501
• 18. W. Meng, L. Xie, and W. Xiao, “Optimality analysis of sensorsource geometries in heterogeneous sensor networks,” IEEE Transactions on Wireless Communication, vol. 12, pp. 1958–1967, 2013. http://dx.doi.org/10.1109/twc.2013.021213.121269
• 19. L. Cao, Y. Cai, and Y. Yue, “Swarm intelligence-based performance optimization for mobile wireless sensor networks: Survey, challenges, and future directions,” IEEE Access, vol. 7, pp. 161 524–161 553, 2019. http://dx.doi.org/10.1109/access.2019.2951370
• 20. W. Luo, B. Gu, and G. Lin, “Communication scheduling in data gathering networks of heterogeneous sensors with data compression: Algorithms and empirical experiments,” European Journal of Operational Research, vol. 271, pp. 462–473, 2018. http://dx.doi.org/10.1016/j.ejor.2018.05.047
• 21. W. Luo, Y. Xu, B. Gu, W. Tong, R. Goebel, and G. Lin, “Algorithms for communication scheduling in data gathering network with data compression,” Algorithmica, vol. 80, pp. 3158–3176, 2018. http://dx.doi.org/10.1007/s00453-017-0373-6
• 22. C. Li and W. Luo, “Exact and approximation algorithms for minimizing energy in wireless sensor data gathering network with data compression,” American Journal of Mathematical and Management Sciences, vol. 41, no. 4, pp. 305–315, 2022. http://dx.doi.org/10.1080/01966324.2021.1960226
• 23. J. Berlińska and M. Drozdowski, “Scheduling divisible mapreduce computations,” Journal of Parallel and Distributed Computing, vol. 71, no. 3, pp. 450–459, 2011. http://dx.doi.org/10.1016/j.jpdc.2010.12.004
• 24. J. Berlińska and M. Drozdowski, “Comparing load-balancing algorithms for mapreduce under zipfian data skews,” Parallel Computing, vol. 72, pp. 14–28, 2018. http://dx.doi.org/10.1016/j.parco.2017.12.003
• 25. Wikipedia contributors, “Lambert W function,” https://en.wikipedia.org/wiki/Lambert_W_function, [Online; accessed 5-August-2022].

Uwagi

1. Thematic Tracks Regular Papers

2. Opracowanie rekordu ze środków MEiN, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).