Review of performance evaluation benchmarks of Apache hadoop

With the Internet and data growth increasing trends, big data is becoming an extremely important and challenging problem for Data Centers. Many platforms and frameworks are working to bring a cutting edge technology to this problem. Apache Hadoop is a software framework addressing the big-data processing and storing on clusters, providing reliability, scalability and distributed computing. Hadoop has a distributed file system to store vast amount of data in distributed environments, and uses MapReduce algorithm to perform the computations and process large amount of data, by parallelizing the workload and storage. In comparison to other relational database systems, Hadoop works well with unstructured data. Our work is focused on performance evaluation of benchmarks of Hadoop, which are crucial for testing the infrastructure of the clusters. Taking into consideration the sensitiveness and importance of data, it’s inevitable testing the clusters and distributed systems before deploying. The benchmark results can lead to optimizing the parameters for an enhanced performance tuning of the cluster. This thesis covers the necessary related topics of Hadoop and a comprehensive listing of benchmarks used to test Hadoop, while providing detailed information for their appliance and procedures to run them. We tested benchmarks in a virtual environment, with different parameters and options which yielded results that led to the conclusion of this thesis.