Implement MapReduce
map, shuffle, reduce with combiners
Challenge 1 of Data PipelinesIntermediate~50 minFree
Process a dataset too big for one machine by splitting the work into two simple functions. Map turns each input into key/value pairs; the framework shuffles pairs together by key; reduce collapses each key's values into a result. Build the three phases, wire them into a word-count job, and add a combiner that pre-aggregates locally, the programming model that powered Google's index and inspired Hadoop and Spark.