Trapped in the spam folder? Here’s what to do.

[Somewhat-relevant image]

It seems that some people’s comments are getting trapped in the spam filter.

Here’s how things go. The blog software triages the comments:

1. Most legitimate comments are automatically approved. You write the comment and it shows up right away.

2. Some comments are flagged as potentially spam. About half of these are legitimate comments and about half are actually spam. I go through the main comment folder about once a day (or more often if I’m trying to procrastinate) to (a) read the new comments, and (b) check the comments that were flagged as possibly spam and either approve them or send them to the spam folder. Sometimes I see already-approved comments that are spam, and I send them to the spam folder too, but that’s rare. And sometimes the classification is difficult: a comment looks real, but the identifying url is spam, and then I classify the whole comment as spam.

3. Lots and lots of comments are identified by the software as spam and sent directly into the spam filter, where I never see them. We get thousands and thousands of these, and there’s no way I could go through them. (Just to get a sense of this problem, the spam folder currently has 30,000 comments, all since 22 July (the last time I emptied it, I guess), as compared to 100,000 comments in the entire history of the blog. So it seems that we’d get more spam in 2 months than we received real comments in 15 years.

But . . . after hearing about the recent comments caught in spam, I went into that spam filter, searched on Anonymous, and found a few legitimate comments that got trapped there.

Unfortunately, I don’t know how to whitelist people, and it seems that various regular commenters still have been having this problem.

As a stopgap, try this: If you’re worried that your comment might be going straight to the spam filter, include the following bit of html in your comment:

<!-- notspam -->

Then, every once in awhile I can search the spam folder for the string “notspam” and fish these out.

An actual spammer could read all this and then spam me, but there’d be no real point, as I won’t be automatically approving these new comments; I’ll check them first in any case.

P.S. The whole thing reminds me of this story, hence the above image.

9 thoughts on “Trapped in the spam folder? Here’s what to do.

  1. Thank you professor Gelman for your reply and this post! (related to comment(s) here: http://statmodeling.stat.columbia.edu/2018/08/03/replication-crisis-political-process/#comment-835679). I see my posts have now been posted (e.g. http://statmodeling.stat.columbia.edu/2018/07/26/think-accelerating-string-research-successes/#comment-831212).

    I always use a temporary mail-address which has always worked for me in the last few years i have been commenting here. I prefer to keep doing this, but i also don’t want to contribute to making you have to go through the spam-box because of that. I did not realize using a temporary mail-address possibly makes things harder for you (?). I am still not sure if that is the case, but if me using a temporary mail-address is making things unnecessarily hard for you, please feel free to possibly let me know so i could then decide what to do with that information (e.g. keep using a temporary mail-address but then take the “risk” that my comments won’t get published or get published much later, or use my “real” mail address, or stop commenting because i don’t want to use my “real” mail address, etc.)

    For now, i assume temporary mail addresses are not the only reason real/valid comments get stuck in the spam-box and/or you will keep checking the spam-box. I will try and remember to use the “” thing (if i indeed am using it correctly in this post above, because i don’t know anything about programming/html). And i will try to remember to use the new temporary mail provider which seems to work for me/manages to escape the spam-box.

    (Side note: as a result of this post i am now also more mindful and thankful for all your reading of comments and/or replying to comments and/or fishing out spam-trapped comments in the spam-box. I really appreciate you giving everyone a change to try and join the conversation and try and contribute to (improving) science. Thank you again for all you do).

  2. Andrew writes: “Most legitimate comments are automatically approved. You write the comment and it shows up right away.”

    This is certainly not my experience. The time lag between hitting the enter key and the appearance of the innocuous comment is usually hours. Sometimes there is an indication, “awaiting moderation,” but lately there has been no warning at all.

    • Paul:

      If it takes hours, it’s because your comment is in moderation. (This comment of yours took only a few minutes to appear, because a couple minutes ago I felt like avoiding work so I went to check blog comments, saw yours, and approved it.) Again, I have no idea what it takes for some comments to be approved automatically and for some to go into moderation.

      • Andrew: As an experiment, I am using a different email address of mine to see if the spam filter is address dependent. Maybe it does not like the number “9” in my usual email address. Unfortunately, the alternate email address also has a “9” which is symbolic of the lack of a middle name. However, some web sites incorrectly insist I have a middle initial, a “T.”

      • Until a few weeks or a month or two ago, every time I came to the blog, my browser would already have my name and email and website filled in, the blog would know who I am already, and if my comment went to moderation, *i would see it there with a note about how it was in moderation so no one else could see it*.

        Around about the time that I started getting all those stupid notifications about the new General Data Protection laws (some time early june?) the blog also stopped tracking who I was. Now *every time* I post I have to fill in my name, email, and website address. EVERY SINGLE TIME. And of course, the blog doesn’t know who I am so it doesn’t show me my held moderated comments.

        I gotta say, it’s a real pain and very error prone to have to re-enter all this info each time I post anything. If theres something you can do to turn on some kind of cookies that help the blog remember us and pre-fill that info, I think regular posters would appreciate it. Of course, there are privacy issues there too, so if you’ve turned that off on purpose, I understand. But if it’s turned off basically just by default and you weren’t aware of it or something, perhaps you can get some help to turn it back on so it works the way it did for the first 12 years ;-)

      • 1. Andrew said, “I have no idea what it takes for some comments to be approved automatically and for some to go into moderation.”

        My experience is that if I include two links in a comment, it goes into moderation.

        2. Daniel said, “Until a few weeks or a month or two ago, every time I came to the blog, my browser would already have my name and email and website filled in, the blog would know who I am already”

        This happened to me as well — except I think the change was more recent, only a week or two ago.

Leave a Reply

Your email address will not be published. Required fields are marked *