Big Data in combination with Machine Learning are transforming the world around us. This talk will cover the following topics:
We will highlight key concepts and programming paradigms in this exciting technological frontier, including Hadoop, NoSQL Databases, Kafka and Spark, and how they can be used to store, process and analyze massive data sets.
Until recently it was practically impossible to teach big data courses at IDC. We will review a new project which created a Big Data Infrastructure for IDC on AWS, with the following key achievements:
Supports creating and scaling resources on demand.
Enables global and per-course credit and budget monitoring, alerting, and reuse.
Automates many manual and error prone tasks and mitigates potential budget wasting.
Using the new infrastructure we will demonstrate how we can run a scalable computation on 10 or even 100 servers with just a few mouse clicks.
Dr. Shaul Dar is a technologist, data scientist, manager and lecturer with extensive experience in research and software development, in areas related to big data, machine learning and AI. Shaul has worked in leading research institutions (UW-Madison, AT&T Bell Labs), start-ups (Mercado, Savantis) and large corporations (McKesson, AGT, PayPal / eBay, Check Point, Intuit, Dell).
Mr. Zuriel Levi is an M.Sc student at IDC, also working as a teaching assistant in Dr. Shaul Dar's course - "Big Data Platforms"
Zuriel is an officer in the intelligence corps, specializing in the fields of Big Data and AI Infrastructure.
He has been leading two major projects around Hadoop as a Service and AI infrastructure in Unit 8200.