Apologies to anyone who’s comment hasn’t made it onto this site in recent months. I’ve spotted a few false positives in my Akismet spam list lately, which makes me wonder how many I’ve missed in the past (because I usually just ‘delete all’). The inability to sort by “spamminess”, as Techcrunch puts it, is a glaring omission in Akismet’s functionality. It would be OK if I had 10 spam comments a day, but with hundreds of messages in the queue, I could never have time to check them all.
I used to filter e-mail with SpamAssassin. Any mail with a “spam score” of between 5 and 8 (higher probability of being spam) was held for moderation, and anything above 8 was just automatically deleted. A score of 5.01 means there’s a (relatively) good chance that the email is legit, while a score of 7.9 is almost certainly spam. Sorting by spam score meant I could quickly and easily identify false positives, and 90% of them would have a score of 5.x.
Matt et al are very secretive about the way Akismet operates behind the scenes, but if there is some fundamental reason why future releases won’t have this functionality, then I would see that as a fatal flaw. I might try out Defensio this week. If their spam filtering can get anywhere close to Akismet’s accuracy, then the ability to easily find false positives will make all the difference.
2 Comments
The thing with making a comment’s spamminess score available is it makes it possible for a comment spammer to game the system.
Scenario: A comment spammer sets up a test blog with Defensio installed (only used here as the example as it’s the subject of this post) and then proceeds to hit it with his own spam. The spammer can then monitor the spamminess of each comment in order to learn which elements of each comment contribute to the spamminess score being higher.
In theory this would allow a comment spammer to develop spam comments with a low spamminess score, and then proceed to hit live blogs with them.
Obviously over time Defensio’s database of spam and it’s algorithm will evolve to eliminate such comments as they’re reported by blog admins, but the theory and the possibility is still there. Akismet’s approach of not revealing a comment’s spamminess eliminates the possibility of gaming the system.
It’s unfortunate that false positives occur. It would be interesting to see if Defensio and Akismet have similar false positive rates.
Good point, although I’m sure some people will mumble about security through obscurity. SpamAssassin make their rules public and give their reasoning behind that here, but maybe that will only work well when it is open source and widely used. I can’t say much without knowing exactly how Akismet works.
It doesn’t matter much on this site if somebody’s comment about World of Warcraft doesn’t make it through, but as Wordpress gets more and more widely used, with spam traffic always increasing, then the issue is only going to grow. Based on the assumption that there will always be false positives, the ability to easily identify them will be critical.
One Trackback/Pingback
[...] solutions such as Defensio. I haven’t tried that one yet myself, but I’ve seen some pretty good buzz about [...]
Post a Comment