Over 50 recipes on Spark, covering Spark Core, Spark SQL, Spark Streaming, MLlib, and GraphX Libraries, for machine learning and data analysis algorithms

About This Book

  • Become an expert at graph processing using GraphX
  • Use Apache Spark as your single big data compute platform and master its libraries
  • Learn with recipes that can be run on a single machine as well as on a production cluster of thousands of machines

Who This Book Is For

If you are a data engineer, application developer, or data scientist who would like to leverage the power of Apache Spark to get better insights from big data, this is the book for you.

What You Will Learn

  • Install and configure Apache Spark with various cluster managers
  • Set up development environments
  • Perform interactive queries using Spark SQL
  • Get to grips with real-time streaming analytics using Spark Streaming
  • Master supervised learning and unsupervised learning using MLlib
  • Build a recommendation engine using MLlib
  • Develop a set of common applications or project types and solutions that solve complex big data problems

In Detail

The success of Hadoop as a big data platform had users asking for more. Apache Spark emerged as one standard, rather than a combination of tools, which solves all these challenges in one shot. By introducing in-memory persistent storage, Apache Spark eliminated the need to store intermediate data in filesystems, thereby increasing the processing speed up to 100 times. It provides a single-computer platform that takes care of various computer needs such as machine learning and real-time streaming.

This book will focus on how one can analyze large and complex sets of data. Starting with installing and configuring Apache Spark with various cluster managers, you will cover setting development environments. You will then cover various recipes to perform interactive queries using Spark SQL and then real-time streaming with various sources such as Twitter Stream and Apache Kafka. You will then focus on machine learning including supervised learning, unsupervised learning, and recommendation engine algorithms. Once you master graph processing using GraphX you will cover various recipes for cluster optimization and troubleshooting will also be discussed.

Rishi Yadav

Rishi Yadav is the President and CEO of InfoObjects, Inc. a leading big data consulting company he founded 9 years ago. InfoObjects has been given the Inc. 5000 fastest growing company award in America for 4 years in a row and was awarded the best place to work in Bay Area in 2014.

Rishi has 17 years of experience designing and developing open source applications. He is an open source software expert and advises Silicon Valley companies on big data trends. Rishi Yadav has been honored as Silicon Valley’s 40 under 40 in 2014 by Silicon Valley Business Journal. He finished his bachelor’s degree from the prestigious Indian Institute of Technology (IIT) Delhi in 1998.