So I started working on a Map Reduce project for work and came across an irritating error:
WARN Step failed with bad retval ‘1’
Took me quite a while to find out that the aggregate reduce function (built-in Hadoop lib) expects a key/val pair in a certain format which makes perfect sense. Also what makes it even trickier is that there were no clues in the step, controller or task logs.
So if you are going to change the tutorial from…
print “LongValueSum:” + word.lower() + “\t” + “1”
…then make sure you have a custom reduce function that can handle the new format.