RSS

Elastic Map Reduce tutorial error (The Word Count thing)

11 Apr

So I started working on a Map Reduce project for work and came across an irritating error:

WARN Step failed with bad retval ‘1’

Took me quite a while to find out that the aggregate reduce function (built-in Hadoop lib) expects a key/val pair in a certain format which makes perfect sense. Also what makes it even trickier is that there were no clues in the step, controller or task logs. 

So if you are going to change the tutorial from…

print “LongValueSum:” + word.lower() + “\t” + “1”   

…then make sure you have a custom reduce function that can handle the new format.

 

 

Advertisements
 
Leave a comment

Posted by on April 11, 2013 in Uncategorized

 

Tags: , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: