crap detection

The CRAP Test Doesn’t Work?

Posted on by Jon K.

Mike Caulfield (@holden on Twitter) delivered a keynote a long while back which I only caught on twitter by way of others posts. It’s interesting that one of his comments was that the CRAP Test Doesn’t work in a world filled with misinformation. Now, the fact that the CRAP Test isn’t a widely taught technique, nor that the real components that one needs to do (including such archaic things as WHOIS searching, and name searches geolocation of posts by IP and cross referencing with known authorities) something that the general public doesn’t​ do. It’s too much work. So sure, that aspect of it “doesn’t work”, but the underlying skills founded in skepticism work. If you use them. I think that’s an important distinction. To say the tool doesn’t work because the person doesn’t use it is a bit disingenuous.

Most people don’t, because they take a short cut, because it was a parent, in-law or someone they know and trust who forwarded it. Or it already fits their point of view. Or it passed a superficial sniff test. And that’s where the conspiracy theory peddlers make money. And much like advertising where it’s not effective the first time you see the ad and think you need a new sofa, it’s the thirtieth time when you actually need a sofa, that your brain brings up the ad you saw repeatedly.

The public has been primed for this for years. It’s essentially advertising that has replaced truth. It’s bullshit that’s replaced truth. It’s repetition that’s replaced truth.

And with that, the Internet has ended. Well, the Internet that held hope for a lot of educators, for a lot of activists, for a lot of free thinkers and artists. For marginalized folks. If we take this as a parrallel to punk rock – which exploded in 1976 with diversity of sound and voice, and quickly marginalized people by 1979 by getting harder, faster and more straightforward, we can see what the future of a “free” Internet will look like. Lots of the same, but with pockets of interesting, but marginal works. Except this time, we got close to 10 years of the web before it became an advertising platform for bullshit, junk and waste, with some good information out there.

This is the end of truth:

What conspiracy theorists, bullshit peddlers, technology hacks, blackmailers, and those who value lies over truth want has won. Technology has helped them get there faster than we can manage. And no one is working on a way to fight this bullshit, because there’s no money in a truth serum.

No comments yet

Howard Rheingold’s Crap Detection 101

Posted on by Jon K.

This may be one of the most important thing that will determine success or failure in the future. Not just to determine who is telling us bullshit, but what their motivations might be in telling us bullshit. Otherwise, we’ll be exposed to sending personal information to Nigeria to help out princes, which is this generation’s prime real estate in Florida and bridge in Brooklyn.

1 Comment

Reflections on My Use of Wikis in the Classroom

Posted on by Jon K.

Wikipedia has fundamentally and finally altered epistemology itself—our commonly held ideas about knowledge. For the academy at large, the significance of Wikipedia is roughly equivalent to that which the Heisenberg uncertainty principle had in the sciences in the 1920s—stating what is not possible rather than what is. It is no longer possible to plan, tax, and budget for universities as if their model of knowledge creation is the only epistemological path. No matter how improbable it might seem that a Web page that anyone can edit would lead to valuable knowledge, Wikipedia makes clear that there is now another model for knowledge creation. And it also recasts the comments of the diplomatic chancellor in a supremely ironic light: here is the leader of a massive state system for knowledge creation stating that “when every one is responsible no one is responsible,” while he, and certainly everyone in that audience, has probably relied upon a knowledge acquisition path—from Google to Wikipedia—for which everyone is responsible and no one is responsible at once. — Robert E. Cummings, Wiki Writing: Collaborative Learning in the College Classroom (online book) (link to quote)

I’ve written previously about my wiki problems, assessing the wiki work, but never really assessed the impact of my decision to turn the Searching the Internet course into a guided research course. Now is a good time to do this as the second iteration of the course is done, and I’m handing it off to someone else. For the most part, people embraced the technology once they understood the purpose of using the wiki – which was hard to explain to some people. It was important to understand that user-created content needs some critical consumption before you trust it. It’s constantly amazing that many people don’t think to question broadcast news, newspapers or media in general, which really is the main goal that I hoped to get out there to people. I think in some ways I failed, more on that later.

One major hurdle that I still am not sure about how to get around (through?) is student expectations of what should go on in the classroom. Using the wiki for everything was conceptually difficult for those who attended lectures in the face-to-face offering. They wanted to discuss the issues in class – and I didn’t persuade them otherwise. It’s the thing I love about classrooms – the discussions therein. I should’ve made a better attempt at summarizing these in class discussions in the wiki, that way there would be a digital record of what was discussed, what the decisions were and where to go post-discussion. Of course, having the discussions robbed them of a crucial piece of the collaborative work – the discussions on talk pages. This discussion serves two purposes. The first being the revelation that the general public have a democratic say in the content published. The second being that hopefully the fact that they’re editing the content means that other non-experts are also editing content, and that means you have to take everything written with a grain of salt (sometimes a pound).

Another classroom expectation that I had trouble with was a small minority of students were just not comfortable doing their own research. They wanted specific instructions from me as to what to do. I was clear in that this course would be unlike other courses they may have taken. I didn’t want the authority of the teacher (and considering the subject matter, let’s face it, people have to get over this authority complex they have – it’s decentralized just like the Internet) and was looking for ways to bust ye olde teacher as authority. I tried telling people that I was not an expert and that my role was as a guide through the material laid out before them. Yes, I wrote it and yes, I researched it. Yes, it could be wrong too. I tried telling people that it wasn’t a course, and they weren’t students and they weren’t doing assignments they were doing exercises. Of course, the exam at the end was real. I tried telling students that I only know this stuff because I looked it up on the Internet. That didn’t work out so well, and I never repeated that one. Nothing will devalue the course than telling the truth. In the end I didn’t try to break this power structure, and it’s one of the reasons I won’t be teaching after this semester.

I disagree with Cumming’s assertion that everyone and no one is responsible for the content. It’s neither. It’s you who is responsible for assessing the information you consume. I think that’s where I’ve failed, not getting this point through, that every piece of information you consume has a bias, a history and a reason. Nobody publishes a story in the newspaper or on a blog without a reason. Some are transparent, some are difficult to read. While I’ve given the students of the Searching the Internet over the last seven years the tools and some experience in using them, I’m not sure anyone stayed with it.

With that said, it wasn’t an all-around failure. I did become a better teacher, more confident in the skills I do have (and able to improve the ones that I’m lacking). The content on the wiki was well crafted, well thought out and showed that when students would engage with the subject, they could become subject matter experts on their own.

No comments yet

An Alternative Approach To Crap Detection

Posted on by Jon K.

I haven’t written a lot about what Howard Rheingold calls critical consumption of information on the web, mostly because I’m paid to teach about it and somehow I feel that giving things away when I’m paid to do that is a violation of “trade secrets” or something. After finishing a video for Alec Couros, I feel that I should put this part out there because it’s more of an essential skill and less of a key feature of any course I teach. In fact, it’s a key feature of every course I teach… neither here nor there.

So I mentioned the five questions approach rather than crap detection or CRAP test. The five questions (who, what, where, when, why) comes from the journalistic inquisitiveness that we all intuitively ask when we smell something fishy. I’ve brought those questions to bear on looking at the websites we view daily. Here are some questions you might want to ask:

Who?

Who is writing this stuff? Does the author identify themselves? Can you find an author through tools like whois.net if they don’t? Generally if a person signs a name to an article or column, it is more reliable. If you find a  name and an e-mail, do you get a response if you send them something? Can you further find a phone number looking at an online white pages? On top of those questions, ask yourself who this person is and what credentials they have. Is that important? Sure, being a lawyer in California is an impressive credential, but if you’re a lawyer who only practices in California with California law, does that make you credible to talk about issues in other jurisdictions? In some cases, law might be the same everywhere. In others, maybe not. Who links to this author, or writes about them? In many cases, links are from like-minded people. If it is difficult to determine people’s intentions, this is a good barometer of that person’s viewpoint.

What?

What are they saying? Does it ring true with your internal sensibilities? Does it sound reasonable? There’s many cases where a reasonable statement is false, only because it’s close to the truth, but not quite. Check with Snopes to see if it’s an urban legend. We also know how the masses can augment this effect, where we all come to a consensus of what truth is – if we say it often enough and loud enough and repeat it enough, it becomes truth. “What” attempts to combat this with evidence and fact checking. See what other people say about the subject. See if they agree, or is it cut and paste agreement? Is this person trying to sell you something? What is their motivation (which is quite often tied to who they are)?

When?

When was this written? Without a date, you can use the last time Google accessed the site, available on the results page through the Cached link. That gives you an idea of at least the last time Google saw it. When is tricky, because it’s importance depends on the context of the subject matter. Does it matter if information about the war of 1812 was written in 1995 or 2005? Only if something has been discovered in that time period. You can use the WayBack Machine to see if the website or webpage has changed in that period. Now ask if it matters if information on hip replacement surgery was written in 1995 or 2009?

Where?

Where was the article written? Does it matter? Where really only contributes to the case you’re building, rarely does location destroy credibility. It might play a part if liability and copyright are at issue. If you have an article written in a country that does not have strong libel laws, perhaps it might be less credible. You can determine where the site is hosted by the whois registration, and a brief glance at the domain name might tell you something. Of course, there are some obscure domain extensions (.tk anyone?) so refer to this handy list of domain extensions. Is the site hosting for free? If someone is willing to pay for hosting, chances are they aren’t just kidding around. While the cost of hosting is very inexpensive, certainly not as imposing as it once was, it is not a magic bullet to kill the claims of a website. Websites with the extension .org, .edu and .gov (depending on your view of the government) tend to be good sources of information.

Why?

After you’ve put together all the other pieces, you now know a little more about the author. Of course, you can never know why someone writes something, but you can examine what they have to gain from writing the piece. Are they trying to sell something? Is it a blog post or review about a product by an employee of the same company? These sorts of testimonials are hard to decipher because they can be very well written.

If you take into consideration the five questions whenever you are faced with confirming information on the Internet, you should be able to build a case to justify the website’s inclusion into your work.

No comments yet