The CRAP Test Doesn’t Work?

Mike Caulfield (@holden on Twitter) delivered a keynote a long while back which I only caught on twitter by way of others posts. It’s interesting that one of his comments was that the CRAP Test Doesn’t work in a world filled with misinformation. Now, the fact that the CRAP Test isn’t a widely taught technique, nor that the real components that one needs to do (including such archaic things as WHOIS searching, and name searches geolocation of posts by IP and cross referencing with known authorities) something that the general public doesn’t​ do. It’s too much work. So sure, that aspect of it “doesn’t work”, but the underlying skills founded in skepticism work. If you use them. I think that’s an important distinction. To say the tool doesn’t work because the person doesn’t use it is a bit disingenuous.

Most people don’t, because they take a short cut, because it was a parent, in-law or someone they know and trust who forwarded it. Or it already fits their point of view. Or it passed a superficial sniff test. And that’s where the conspiracy theory peddlers make money. And much like advertising where it’s not effective the first time you see the ad and think you need a new sofa, it’s the thirtieth time when you actually need a sofa, that your brain brings up the ad you saw repeatedly.

The public has been primed for this for years. It’s essentially advertising that has replaced truth. It’s bullshit that’s replaced truth. It’s repetition that’s replaced truth.

And with that, the Internet has ended. Well, the Internet that held hope for a lot of educators, for a lot of activists, for a lot of free thinkers and artists. For marginalized folks. If we take this as a parrallel to punk rock – which exploded in 1976 with diversity of sound and voice, and quickly marginalized people by 1979 by getting harder, faster and more straightforward, we can see what the future of a “free” Internet will look like. Lots of the same, but with pockets of interesting, but marginal works. Except this time, we got close to 10 years of the web before it became an advertising platform for bullshit, junk and waste, with some good information out there.

This is the end of truth:

What conspiracy theorists, bullshit peddlers, technology hacks, blackmailers, and those who value lies over truth want has won. Technology has helped them get there faster than we can manage. And no one is working on a way to fight this bullshit, because there’s no money in a truth serum.

An Alternative Approach To Crap Detection

I haven’t written a lot about what Howard Rheingold calls critical consumption of information on the web, mostly because I’m paid to teach about it and somehow I feel that giving things away when I’m paid to do that is a violation of “trade secrets” or something. After finishing a video for Alec Couros, I feel that I should put this part out there because it’s more of an essential skill and less of a key feature of any course I teach. In fact, it’s a key feature of every course I teach… neither here nor there.

So I mentioned the five questions approach rather than crap detection or CRAP test. The five questions (who, what, where, when, why) comes from the journalistic inquisitiveness that we all intuitively ask when we smell something fishy. I’ve brought those questions to bear on looking at the websites we view daily. Here are some questions you might want to ask:


Who is writing this stuff? Does the author identify themselves? Can you find an author through tools like if they don’t? Generally if a person signs a name to an article or column, it is more reliable. If you find a  name and an e-mail, do you get a response if you send them something? Can you further find a phone number looking at an online white pages? On top of those questions, ask yourself who this person is and what credentials they have. Is that important? Sure, being a lawyer in California is an impressive credential, but if you’re a lawyer who only practices in California with California law, does that make you credible to talk about issues in other jurisdictions? In some cases, law might be the same everywhere. In others, maybe not. Who links to this author, or writes about them? In many cases, links are from like-minded people. If it is difficult to determine people’s intentions, this is a good barometer of that person’s viewpoint.


What are they saying? Does it ring true with your internal sensibilities? Does it sound reasonable? There’s many cases where a reasonable statement is false, only because it’s close to the truth, but not quite. Check with Snopes to see if it’s an urban legend. We also know how the masses can augment this effect, where we all come to a consensus of what truth is – if we say it often enough and loud enough and repeat it enough, it becomes truth. “What” attempts to combat this with evidence and fact checking. See what other people say about the subject. See if they agree, or is it cut and paste agreement? Is this person trying to sell you something? What is their motivation (which is quite often tied to who they are)?


When was this written? Without a date, you can use the last time Google accessed the site, available on the results page through the Cached link. That gives you an idea of at least the last time Google saw it. When is tricky, because it’s importance depends on the context of the subject matter. Does it matter if information about the war of 1812 was written in 1995 or 2005? Only if something has been discovered in that time period. You can use the WayBack Machine to see if the website or webpage has changed in that period. Now ask if it matters if information on hip replacement surgery was written in 1995 or 2009?


Where was the article written? Does it matter? Where really only contributes to the case you’re building, rarely does location destroy credibility. It might play a part if liability and copyright are at issue. If you have an article written in a country that does not have strong libel laws, perhaps it might be less credible. You can determine where the site is hosted by the whois registration, and a brief glance at the domain name might tell you something. Of course, there are some obscure domain extensions (.tk anyone?) so refer to this handy list of domain extensions. Is the site hosting for free? If someone is willing to pay for hosting, chances are they aren’t just kidding around. While the cost of hosting is very inexpensive, certainly not as imposing as it once was, it is not a magic bullet to kill the claims of a website. Websites with the extension .org, .edu and .gov (depending on your view of the government) tend to be good sources of information.


After you’ve put together all the other pieces, you now know a little more about the author. Of course, you can never know why someone writes something, but you can examine what they have to gain from writing the piece. Are they trying to sell something? Is it a blog post or review about a product by an employee of the same company? These sorts of testimonials are hard to decipher because they can be very well written.

If you take into consideration the five questions whenever you are faced with confirming information on the Internet, you should be able to build a case to justify the website’s inclusion into your work.