I got an email from one of our regular commenters, a nice guy who contributes to our blog discussions when not busy at his day job. He wrote a comment that did not show up on the blog, and emailed me:
This is not at all important, but since something similar happened recently I wonder if something is up with your comment form.
I explained that the blog has a spam filter that catches hundreds of items per day, so many that it’s impossible to go through them and check for legitimate comments. If you email and tell me you have a comment that didn’t show up, I can search through the spam folder for your name and approve the comment. In this case I found one of this guy’s comments but not the other, as I’d emptied the spam folder a few days earlier.
Also every day we get a few comments that are in some intermediate state—not approved and not going straight into spam, they’re held in the main comment folder waiting for me to decide whether to approve them or send them into spam. About half of these are legit and about half are spam, for example here’s one that recently came in:
Was looking for some takes regarding this topic and I found your article quite informative. It has given me a fresh perspective on the topic tackled. Thanks!
This one’s obviously spam. Other times the comment seems to have been written by a computer program that pulls phrases from the post or from published comments, or maybe it’s an actual human who’s trying to write a comment that will be approved. In this case the commenter had a link to a homemade site selling some crap. The only thing I don’t know is if it’s written by the site owner in a desperate attempt to get some web traffic, or if he paid some sleazy online company to promote his site, and this is what they’ve come up with.
In any case, it’s horrible, not just cos it wastes my time but also because it creates the conditions under which a fun and serious legitimate commenter can’t always get through. Not the worst thing going on, just annoying vandalism. Can’t these people find some better things to do, like publishing fake psychology studies, going on NPR, and giving Ted talks?
This is one of those things that’s so ripe for a proof of work system. It would be totally worth my time to pay say 5 cents every time I posted a blog comment. However that would totally bankrupt a assembly line spam engine.
Not sure why this sort of solution hasn’t caught on.
Even if 5 cents is an acceptable cost for a user, the overhead of setting up a payment method would probably stop 99% of legitimate commenter before they got started. Not to mention the work for the blogger to set up the system. You’d have to set up all kinds of security to safely store payment information, and I can’t imagine credit card companies are going to go out if their way to help set up such small transactions.
I didn’t necessarily mean a physical payment.
E.g. get you CPU to perform 10 mins of hard work to compute an eligible hash for each comment. Legitimate users won’t mind but spammers will be handicapped. Similar to the Bitcoin proof of work idea.
From everything I’ve heard, bitcoin is even more cumbersome to use on transactions than regular money, so….
It wouldn’t be literal bitcoin. The only reward is posting, so there’s no unstable speculation. The system is already used with success in emails.
I disagree. This is a horrible idea that only leads to environmental devastation by wasting CPU time. Also, your basic assumptions are wrong. If there is anyone who doesn’t have to pay for cpu time it’s spammers. Who else is running all those insanely large botnets we are seeing in recent years?
Isn’t there some system where you are shown a trivial arithmetic operation and need to answer it to post, like you see a picture showing 13-5 and you have to enter 8 in a box. A computer would have to do image recognition but a human can do it easily?
@Daniel Lakeland: I imagine such a system wouldn’t work well with people who use screen readers.
Wait, what would be the point of leaving a spam message “Was looking for some takes regarding this topic and I found your article quite informative. It has given me a fresh perspective on the topic tackled.”
Did it also have a link to a herbal supplements website? Or was it an initial probe of your defenses?
I’m trying to imagine someone writing a charitable spambot that posts nice but innocuous comments all over the internet, and having trouble doing it.
https://gist.github.com/shanselman/5422230?WT.mc_id=-blog-scottha
I thought that answer is hilarious, no one ever thinks of all the benefits spammers contribute to society. There was probably a link in the homepage field though. Id guess Andrew could remove it if he wanted but some people on here do put legit links in there.
I don’t know if its simple, but one way to improve this blogs filter would be to whitelist a few domains like pubmed that are very unlikely to be spam. Then the comment could have 2+ whitelisted links without requiring manual approval.
Dmitri, Anon:
It’s rare, but every one in a while there’s a high-quality comment with a link to some spam webpage. I guess in this case someone who happens to be a spammer also had something interesting to say. In those cases I keep the comment but remove the link. Usually, though, the spam is contentless, either pure bland verbiage or a cut-and-paste of some part of the post that it’s responding to.
I don’t have a solution for spammers, but I think there should be a Donations Button for the sake of all the work involved in maintaining a great blog, not the least of which is spam patrol.
There’s probably something wrong with this idea too, but how about a Federal Bureau of Spam Investigation? (After the necessary regulations are developed, reviewed, and approved by Congress.) Okay, that wouldn’t work either given our current USA politics.