Using Redis supported by NVRAM in HPC applications

• Autorzy: Malinowski A.

Identyfikatory

DOI

10.7494/csci.2017.18.3.2077

• Języki publikacji: EN

Abstrakty

EN

Nowadays, the efficiency of a storage systems is a bottleneck in many moern HPC clusters. High performance in traditional approach – processing using files – is often difficult to obtain because of model complexity and its read/write patterns. Alternative approach is applying a key-value database, which usually has low latency and scales well. On the other hand, many key-value stores suffer from limitation of memory capacity and vulnerability to serious faiures, which is caused by processing in RAM. Moreover, some research suggests, that scientific data models are not applicable to storage structures of key-value databases. In this paper, the author proposes resolving mentioned issues by replacing RAM with NVRAM. Practical example is based on Redis NoSQL. The article contains also a three domain specific APIs, that show the idea bhind transformation from HPC data model to Redis structures, as well as two micro-benchmarks results.

Słowa kluczowe

EN

high performance computing; storage systems; NoSQL; NVRAM

• Wydawca: Wydawnictwa AGH
• Czasopismo: Computer Science
• Rocznik: 2017
• Tom: Vol. 18 (3)
• Strony: 287--300
• Opis fizyczny: Bibliogr. 23 poz., rys., wykr., tab.

Twórcy

autor

Malinowski A.

• Faculty of Electronics, Telecommunications and Informatics, Gdansk University of Technology, Gdansk, Poland

Bibliografia

• [1] Bosagh Zadeh R., Meng X., Ulanov A., Yavuz B., Pu L., Venkataraman S., Sparks E., Staple A., Zaharia M.: Matrix Computations and Optimization in Apache Spark. In: Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD'16, pp. 31-38. ACM, New York, 2016. http://dx.doi.org/10.1145/2939672.2939675.
• [2] Butler D.M.: Scientific Computing Doesn't Need noSQL. In: Proceedings of the 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, SCC'12, pp. 1301-1302. IEEE Computer Society, Washington, DC, USA, 2012. http://dx.doi.org/10.1109/SC.Companion.2012.158.
• [3] Carlini E., Dazzi P., Esposito A., Lulli A., Ricci L.: Balanced Graph Partitioning with Apache Spark. In: Euro-Par 2014: Parallel Processing Workshops: Euro-Par 2014 International Workshops, Porto, Portugal, August 25{26, 2014, Revised Selected Papers, Part I, pp. 129-140. Springer International Publishing, Cham, 2014. http://dx.doi.org/10.1007/978-3-319-14325-5_12.
• [4] Dutka L., Słota R., Wrzeszcz M., Król D., Kitowski J.: Uniform and Efficient Access to Data in Organizationally Distributed Environments. In: Bubak M., Kitowski J., Wiatr K. (eds.), eScience on Distributed Computing Infrastructure: Achievements of PLGrid Plus Domain-Specific Services and Tools, pp. 178-194. Springer International Publishing, Cham, 2014. http://dx.doi.org/10.1007/ 978-3-319-10894-0_13.
• [5] Foong A., Hady F.: Storage as Fast as Rest of the System. In: 2016 IEEE 8th International Memory Workshop (IMW), pp. 1-4, 2016. http://dx.doi.org/ 10.1109/IMW.2016.7495289.
• [6] Forum M.P.I.: MPI: A Message-Passing Interface Standard Version 3.1, 2015. http://www.mpi-forum.org/docs/mpi-3.1/mpi31-report.pdf.
• [7] Gao S., Xu J., Hrder T., He B., Choi B., Hu H.: PCMLogging: Optimizing Transaction Logging and Recovery Performance with PCM, IEEE Transacti- ons on Knowledge and Data Engineering, vol. 27(12), pp. 3332-3346, 2015. http://dx.doi.org/10.1109/TKDE.2015.2453154.
• [8] Han J., Haihong E., Le G., Du J.: Survey on NoSQL database. In: Pervasive Computing and Applications (ICPCA), 2011 6th International Conference on, pp. 363-366. 2011. http://dx.doi.org/10.1109/ICPCA.2011.6106531.
• [9] Hanlon M.R., Dooley R., Mock S., Dahan M., Nuthulapati P., Hurley P.: A Case Study for NoSQL Applications and Performance Benefits: CouchDB vs. Postgres. In: Proceedings of the 2011 TeraGrid Conference: Extreme Digital Discovery. ACM, New York, NY, USA, 2011.
• [10] Intel Corporation: Introducing Breakthrough Memory Technology, 2015. http: //www.intel.com/content/www/us/en/architecture-and-technology/ non-volatile-memory.html.
• [11] Kryder M.H., Kim C.S.: After Hard Drives. What Comes Next? IEEE Tran- sactions on Magnetics, vol. 45(10), pp. 3406-3413, 2009. http://dx.doi.org/ 10.1109/TMAG.2009.2024163.
• [12] Li T., Verma R., Duan X., Jin H., Raicu I.: Exploring Distributed Hash Tables in HighEnd Computing, SIGMETRICS Performance Evaluation Review, vol. 39(3), pp. 128-130, 2011. http://dx.doi.org/10.1145/2160803.2160880.
• [13] Mateescu G., Gentzsch W., Ribbens C.J.: Hybrid Computing{Where fHPCg meets grid and Cloud Computing, Future Generation Computer Systems, vol. 27(5), pp. 440-453, 2011. http://dx.doi.org/http://dx.doi.org/10. 1016/j.future.2010.11.003.
• [14] Mohan C.: History Repeats Itself: Sensible and NonsenSQL Aspects of the NoSQL Hoopla. In: Proceedings of the 16th International Conference on Extending Database Technology, EDBT'13, pp. 11-16. ACM, New York, 2013. http://dx.doi.org/10.1145/2452376.2452378.
• [15] Monnerat L., Amorim C.L.: An E_ective Single-hop Distributed Hash Ta- ble with High Lookup Performance and Low Tra_c Overhead, Concurrency and Computation: Practice and Experience, vol. 27(7), pp. 1767-1788, 2015. http://dx.doi.org/10.1002/cpe.3342.
• [16] Oral S., Dillow D.A., Fuller D., Hill J., Leverman D., Vazhkudai S.S., Wang F., Kim Y., Rogers J., Simmons J., Miller R.: OLCFs 1 TB/s, Next-Generation Lustre File System. In: Proceedings of Cray User Group Conference (CUG 2013), 2013.
• [17] Patterson D.: Past and Future of Hardware and Architecture. In: SOSP History Day 2015, SOSP '15, pp. 9:1-9:63. ACM, New York, 2015. http://dx.doi.org/ 10.1145/2830903.2830910.
• [18] Qiu M., Ming Z., Li J., Gai K., Zong Z.: Phase-Change Memory Optimization for Green Cloud with Genetic Algorithm, IEEE Transactions on Computers, vol. 64(12), pp. 3528-3540, 2015. http://dx.doi.org/10.1109/TC.2015. 2409857.
• [19] Strohmaier E., Dongarra J., Simon H., Meuer M.: TOP 10 Sites for June 2016. https://www.top500.org/lists/2016/06/.
• [20] Wrzeszcz M., Trzepla K., Słota R., Zemek K., Lichoń T., Opioła L., Nikolow D., Dutka L., Słota R., Kitowski J.: Metadata Organization and Management for Globalization of Data Access with Onedata. In: Wyrzykowski R., Deelman E., Dongarra J., Karczewski K., Kitowski J., Wiatr K. (eds.), Parallel Processing and Applied Mathematics: 11th International Conference, PPAM 2015, Krakow, Poland, September 6{9, 2015. Revised Selected Papers, Part I, pp. 312{321. Springer International Publishing, Cham, 2016. http://dx.doi.org/10.1007/ 978-3-319-32149-3_30.
• [21] Zhao B.Y., Huang L., Stribling J., Rhea S.C., Joseph A.D., Kubiatowicz J.D.: Tapestry: A Resilient Global-scale Overlay for Service Deployment, IEEE Journal on Selected Areas in Communications, vol. 22(1), pp. 41-53, 2006. http://dx.doi.org/10.1109/JSAC.2003.818784.
• [22] Zhao B.Y., Kubiatowicz J.D., Joseph A.D.: Tapestry: An Infrastructure for Fault-tolerant Wide-area Location, Technical report, Berkeley, CA, USA, 2001.
• [23] Żmuda M., Opioła L ., Dutka L ., Słota R., Kitowski J.: Kademlia with Consistency Checks as a Foundation of Borderless Collaboration in Open Science Services. In: Boukhanovsky A., Bubak M., Balakhontceva M. (eds.), Procedia Computer Science. 5th International Young Scientist Conference on Computational Science, YSC 2016, 26-28 October 2016, Krakow, Poland, vol. 101, pp. 304-312, 2016. http://dx.doi.org/http://dx.doi.org/10.1016/j.procs.2016.11.036.

Uwagi

PL

Opracowanie ze środków MNiSW w ramach umowy 812/P-DUN/2016 na działalność upowszechniającą naukę (zadania 2017).