Application-Level Benchmarking of Big Data Systems

The increasing possibilities to collect vast amounts of data—whether in science, commerce, social networking, or government—have led to the “big data” phenomenon. The amount, rate, and variety of data that are assembled—for almost any application domain—is necessitating a re-examination of old technologies and development of new technologies to get value from the data, in a timely fashion. With increasing adoption and penetration of mobile technologies, and increasing ubiquitous use of sensors and small devices in the so-called Internet of Things, the big data phenomenon will only create more pressures on data collection and processing for transforming data into knowledge for discovery and action.
A vibrant industry has been created around the big data phenomena, leading also to an energetic research agenda in this area. With the proliferation of big data hardware and software solutions in industry and research, there is a pressing need for benchmarks that can provide objective evaluations of alternative technologies and solution approaches to a given big data problem. This chapter gives an introduction to big data benchmarking and presents different proposals and standardization efforts.