Really an unfair article. The premise, title, and bulk have nothing to do with what actually happened.
With his custom software he's solving a completely different problem with a different algorithm that was only possible based on observations unique to his dataset:
I originally started with code very similar to the C++ STL set intersection code and was dissatisfied with the results. By observing that the longer sequences will skip more items than the shorter sequences, I wrote a 2-level loop that skips over items in the longer sequence before performing comparisons with the shorter sequence. That got me roughly a 2x performance advantage over the STL variant.
Let's call this apples. Then he talks about how using C++ STL wouldn't do the trick since he doesn't care about the data, only the count (oranges). And how bloom filters applied naïvely to the raw data would still take longer (zebras).
He's solving a completely different problem, and lambasting perfectly viable technologies for taking longer to solve a completely different problem.
Now obviously there's a good blog post and moral in there: don't solve the wrong problem. Generic algorithms can't be both generic and special at once - they're not always going to give you what you're looking for and only what you're looking for, and that's a cost to be taken into consideration at any time that you're trying to choose a solution. But don't criticize them for being slow, and then in a postscript say "and I don't need the actual results of this algorithm, anyway."
I don't understand your response. Whom or what was he unfair to? What technology was he 'lambasting'?
But don't criticize them for being slow, and then in a
postscript say "and I don't need the actual results of
this algorithm, anyway."
I don't see him criticising any solution for 'being slow' in general. Only for 'being too slow for this specific problem'.
It's very common to use a (already available) algorithm to calculate a result related to what you actually want and then infer the answer from that result. E.g. calculate the intersection of two sets and then count the members, instead of intersecting and counting simultaneously and never having the intersection available. It's usually a wise decision to use such the first approach: the performance penalty is acceptable and the advantage of code reuse (and not having to write and test a new algorithm) is larger. His point was that in this case, it wasn't.
I didn't find it unfair at all. Comparing some general purpose approaches to a more specialzed one is not an apples and oranges comparison. It's the stuff that many orders of magnitude type optimizations are made of.
Also, keeping cache friendliness in mind in the design and choice of datastructures and algorithms is a good idea, which is the main point of his article.
What he says about vtables is fluffy though. The STL doesn't use polymorphism much.
The way I understand it, the article was not meant as a general criticism of bloom filters, but a reply to some people commenting that he should have used . bloom filters when he described his custom algorithm in an earlier article.
It isn't undue or unfair criticism to say "I looked at these solutions to my problem, here are the issues with those solutions with respect to my problems".
If I have been using a screwdriver deck screws to attach things to my project, and some guy says "hey why don't you just use duct tape", and I explain that duct tape is very useful, but in this case its ugly and doesn't hold stuff on right, am I being unfair to duct tape? What about some guy saying "just use a hammer and nails?" and again, I mention that nails are a decent solution, but screws solve a problem introduced by the nails, am I being unfair to the hammers?
In both cases, the answer is "No". When explaining technical choices, there must be some discussion of why solutions are better or worse. When one of the criteria is speed, calling a rejected solution too slow is not unfair, just fact. It doesn't even speak to other applicability of the solution. Heck, in this case he even explicitly states he likes those tools, meaning that pointing to specific case where they aren't good is probably the opposite of "unfair".
Let's say that you need to solve problem X. So you solve it with solution Z.
People then come by later and say, "you should use A", or "B is better in that case", or even "C solves everything".
The article was pointing out why A, B, or C would not solve problem X as well as solution Z. A, B, and C are generic solutions for similar problems, but Z was purpose built for X.
Also, at no point did I say that the algorithms were slow, only not as fast as what we needed.
With his custom software he's solving a completely different problem with a different algorithm that was only possible based on observations unique to his dataset:
I originally started with code very similar to the C++ STL set intersection code and was dissatisfied with the results. By observing that the longer sequences will skip more items than the shorter sequences, I wrote a 2-level loop that skips over items in the longer sequence before performing comparisons with the shorter sequence. That got me roughly a 2x performance advantage over the STL variant.
Let's call this apples. Then he talks about how using C++ STL wouldn't do the trick since he doesn't care about the data, only the count (oranges). And how bloom filters applied naïvely to the raw data would still take longer (zebras).
He's solving a completely different problem, and lambasting perfectly viable technologies for taking longer to solve a completely different problem.
Now obviously there's a good blog post and moral in there: don't solve the wrong problem. Generic algorithms can't be both generic and special at once - they're not always going to give you what you're looking for and only what you're looking for, and that's a cost to be taken into consideration at any time that you're trying to choose a solution. But don't criticize them for being slow, and then in a postscript say "and I don't need the actual results of this algorithm, anyway."