> Whether the naive implementation of map can be easily parallelized across multiple machines/threads/processes is not relevant. In fact, he even says that you need a 'super genius' to write the scaling map and reduce operations.
I think this brings out why I wasn't just "nitpicking." Even though you're presumably pretty savvy, Joel's article has confused you. You're thinking that the easy parallelization is an attribute of the implementation of map, and that a sufficiently smart implementor could make it work.
In fact, the implementation of parallel map is pretty irrelevant here. The important thing is that the function provided to map create no inter-loop dependencies. And given that, you don't need to be a super-genius to implement MapReduce. The super-genius was in coming up with the interface constraints in the first place and recognizing that they were applicable to a large class of problems.
So there are two problems with these "details that are not relevant to the article".
One is that Joel is presenting these details in order to convince you of his point of view. But since they aren't actually true, any confidence they inspire in you is misplaced. You'd be better off with a bald assertion of the importance of functional programming, backed up with "trust me, I know better than you." Admittedly, that's not as much fun to read.
The second problem is that, although Joel has achieved his goal of convincing many people of his point of view, he has done damage to their models of the world in the process. I fully expect to see newbies for years believing that you could never possibly implement MapReduce in C or C++, or that Java requires you to create a whole new file instead of an anonymous function, or that Google Search works by invoking a MapReduce job. And these false beliefs will do real damage to their lives.
I think this brings out why I wasn't just "nitpicking." Even though you're presumably pretty savvy, Joel's article has confused you. You're thinking that the easy parallelization is an attribute of the implementation of map, and that a sufficiently smart implementor could make it work.
In fact, the implementation of parallel map is pretty irrelevant here. The important thing is that the function provided to map create no inter-loop dependencies. And given that, you don't need to be a super-genius to implement MapReduce. The super-genius was in coming up with the interface constraints in the first place and recognizing that they were applicable to a large class of problems.
So there are two problems with these "details that are not relevant to the article".
One is that Joel is presenting these details in order to convince you of his point of view. But since they aren't actually true, any confidence they inspire in you is misplaced. You'd be better off with a bald assertion of the importance of functional programming, backed up with "trust me, I know better than you." Admittedly, that's not as much fun to read.
The second problem is that, although Joel has achieved his goal of convincing many people of his point of view, he has done damage to their models of the world in the process. I fully expect to see newbies for years believing that you could never possibly implement MapReduce in C or C++, or that Java requires you to create a whole new file instead of an anonymous function, or that Google Search works by invoking a MapReduce job. And these false beliefs will do real damage to their lives.