Cloudová infrastruktura pro zpracování velkých dat

Cloud Infrastructure for Big Data Analytics

Typ dokumentu
diplomová práce
master thesis
Autor
Maroš Popovič
Vedoucí práce
Pulc Petr
Oponent práce
Vondra Tomáš
Studijní obor
Softwarové inženýrství
Studijní program
Informatika
Instituce přidělující hodnost
katedra softwarového inženýrství


Práva
A university thesis is a work protected by the Copyright Act. Extracts, copies and transcripts of the thesis are allowed for personal use only and at one?s own expense. The use of thesis should be in compliance with the Copyright Act http://www.mkcr.cz/assets/autorske-pravo/01-3982006.pdf and the citation ethics http://knihovny.cvut.cz/vychova/vskp.html
Vysokoškolská závěrečná práce je dílo chráněné autorským zákonem. Je možné pořizovat z něj na své náklady a pro svoji osobní potřebu výpisy, opisy a rozmnoženiny. Jeho využití musí být v souladu s autorským zákonem http://www.mkcr.cz/assets/autorske-pravo/01-3982006.pdf a citační etikou http://knihovny.cvut.cz/vychova/vskp.html
Metadata
Zobrazit celý záznam
Abstrakt
This masters thesis aims to investigate deployment options for an Apache Flink application to a cloud infrastructure on AWS. We deployed the application across three fundamental parallel computation modes: Amazon Managed Service for Apache Flink, Amazon Elastic MapReduce on EC2, and Amazon Elastic Kubernetes Service. Additionally, we conducted a cost-efficiency analysis, comparing the deployment modes. The results showed that the Amazon Managed Service for Apache Flink is the most cost-efficient deployment mode for the given application. For more complex applications with higher computational requirements or faster processing speed, Amazon Elastic Kubernetes Service (EKS) proved to provide a more economical solution. We used Terraform for the deployment of the Flink application, as well as other components for additional infrastructure. This approach allows for the easy reproducibility of our findings and customizing deployments further to suit the readers needs. We also implemented tests to validate the Terraform deployment and conducted tests to verify the output consistency. This thesis contributes to providing deep insights for users intending to deploy Big Data processing applications to the cloud, including actual cost analysis and comparing the three deployment modes. All of this should simplify otherwise complex decision-making and cost-estimation processes.
 
This masters thesis aims to investigate deployment options for an Apache Flink application to a cloud infrastructure on AWS. We deployed the application across three fundamental parallel computation modes: Amazon Managed Service for Apache Flink, Amazon Elastic MapReduce on EC2, and Amazon Elastic Kubernetes Service. Additionally, we conducted a cost-efficiency analysis, comparing the deployment modes. The results showed that the Amazon Managed Service for Apache Flink is the most cost-efficient deployment mode for the given application. For more complex applications with higher computational requirements or faster processing speed, Amazon Elastic Kubernetes Service (EKS) proved to provide a more economical solution. We used Terraform for the deployment of the Flink application, as well as other components for additional infrastructure. This approach allows for the easy reproducibility of our findings and customizing deployments further to suit the readers needs. We also implemented tests to validate the Terraform deployment and conducted tests to verify the output consistency. This thesis contributes to providing deep insights for users intending to deploy Big Data processing applications to the cloud, including actual cost analysis and comparing the three deployment modes. All of this should simplify otherwise complex decision-making and cost-estimation processes.
 
URI
http://hdl.handle.net/10467/121020
Zobrazit/otevřít
Kolekce