blackboard

Blackboard Thinks They Know How You Teach

Posted on by Jon K.

In November of last year a blog post by Blackboard was making the rounds. A Blackboard study documented how instructors use their system which was somehow conflated as being equivalent to knowing how teachers teach. We could talk about how terrifying this blog post is, or how it’s devoid of solid analysis. Instead I’d like to turn over a new leaf and not harp on about privacy, or a company using the aggregate data they’ve collected with your consent (although you probably weren’t aware that when you agreed to “allow Blackboard to take your data and improve the system” that they’d make this sort of “analysis”), but just tear into the findings.

Their arbitrary classifications (supplemental? complementary?) are entirely devoid of the main driver of LMS use: grades, not instructor pedagogical philosophy – students go to the tools where they are going to get marked on.  Social? I guarantee if you did an analysis of courses that value use of the discussion tool (note, that’s not about quality) as greater than 30% of the final grade, you’ll see a  hell of a lot more use of discussions. It’s not that the instructor laid out a social course, with a student making an average of 50 posts throughout the course (which if you break that down to 14 weeks of “activity” that works out to 3.5 posts per week – which is strangely similar to the standard “post once and respond twice” to get your marks – it’s that discussions are the only tool that give some power to students to author.

Time spent is also a terrible metric for measuring anything. Tabs get left open. People sit on a page because there’s an embedded video that’s 20 minutes long. Other pages have text which is scannable. Connections are measured how? Is it entry and exit times? What if there’s no exit, how does the system measure that (typically it measures for a set amount of activity and then if none occurs the system assumes an exit)? Were those values excluded from this study? If so, how many were removed? Is that significant?

Now Blackboard will say that because of the scale of the data collected that those outliers will be mitigated. I don’t buy that for a second because LMS’s are not great at capturing this data from the start (in fact server logs are not much better), because there’s very little pinging for active window or other little tricks that can only be really assessed using eye tracking software and computer activity monitoring. Essentially garbage-in, garbage-out. We have half baked data that is abstractly viewed and an attempt is made at some generalizations.

Here’s a better way to get at this data: ask some teachers how they use the LMS. And why. It’s that simple.

If you look at Blackboard’s findings, you should be frankly scared if you’re Blackboard. Over half of your usage (53% “supplemental” or content-heavy) could be done in a password protected CMS (Content Management System). That’s a system that could cost nothing (using Joomla or any of the millions of free CMS software available) or replicated using something institutions already have. The only benefit institutions have is that they can point at the vendor when stuff goes wrong and save money on outsourcing expertise into an external company.

If you take the findings further, only 12% of courses (“evaluative” and “holistic”) get at the best parts of the LMS, the assessment tools. So 88% of courses reviewed do things in the system that can be replicated better elsewhere. Where’s the value added?

No comments yet

LMS Review

Posted on by Jon K.

So our current contract with our LMS vendor is complete in October 2018. I can’t say what we’ll do, but the landscape has changed somewhat since the last LMS review was done at work, so we’re starting to form a committee to look at our options. Ultimately, we’re doing this so far in advance because you should take at least a year to transition, which would put us at fall 2017 for starting to roll out the new system for users, which means that we’d need to know in the spring of 2017. Maybe the cost of change will scare us into another contract. As I see it, there’s four options available to us.

1. Stay with D2L. This is the easiest answer, and least complicated. We’ve been relatively happy with them as a company so there’s really no need to change, and despite the minor blips (the great outage of 2012) it’s been pretty smooth so far. Maybe continuous delivery will be awesome, and really a review of the LMS will be a mere formality.

2. Move to another hosted LMS. This is the second easiest answer – if the campus decides that D2L isn’t the choice for us, then we choose another vendor, enter into protracted negotiations, and go through the formal process of getting vendors to submit to review, tender offers, and go through the motions with selecting a new partner for the next five years. I’m not sure if the campus will think this is an option at all. Blackboard was our previous LMS, and didn’t work – essentially leaving the campus without a functional LMS for a whole semester. By the time it did start working, the relationship was in trouble, and well, that’s why we have D2L. That’s according to faculty who were here during the time it happened, which was prior to my time. Another seismic shift may not be in the cards. Another factor is that we’ve become a PeopleSoft school for our various systems across campus. That implementation has been rough for the campus. I’m not sure they have the appetite for another system to learn so quickly.

3. Go back to self-hosting LMS software. This allows us to look at open source solutions, and rely on our own IT group to take server maintenance, infrastructure and all the other associated risks back under our roof. It’s unlikely that we would do this due to the human cost of running a mission critical server – and we’d have to look at hiring back expertise that was relocated to other groups on campus or into industry. Those costs, are not insignificant. The complexity of running Moodle or Sakai at scale for 25,000 to 30,000 users, isn’t lost on me. It’d be a great challenge. I don’t know that this will be palatable to the campus either as we’ve had people who were running their own Moodle install come over to use the institution’s provided install of D2L. Maybe that’s the path of least resistance? Maybe it’s the students pushing for one platform? Who knows.

4. Do away with the LMS. This is an entirely radical idea, but what if we just left it up to instructors to do it themselves? I’d be ok in this scenario, despite having this a huge part of my job description, because there’s always going to be technology to use to teach. I’d have to adjust. Would this even fly? Probably not. Imagine the headlines: “first University to do away with the LMS”… would be useful to put on my tombstone after everyone lynches me because they need a secure webpage to link their stuff to.

As a teaching and learning centre, we’ll be interested in finding something flexible to teach not only in the modes that people currently teach in, but also in the pedagogy that people want to teach in. All LMS’s say they can do constructivist style setups, but really they require changes globally to do so. No one gives the instructor the power to turn on or off student control of a slice of content, or a discussion, or even a collaborative space for document sharing. I’ll go out on a limb and suggest that all LMS’s are designed as content delivery tools, not knowledge construction tools. And to that end, the choice of tools that can be used is often controlled by LMS administrators, not the instructors. Now, there’s great reasons for structuring things in such a way; theoretically administrators have subject matter expertise in selecting partners to connect to the LMS and have experience with vetting vendors. Right? I hope so. I know I’ve tried my best to make sure I’ve done right for student’s privacy, intellectual property and general safe digital space. I don’t know what I don’t know though. I guess, through the next three years, I’ll start to find out.

2 Comments

Answers for 2012

Posted on by Jon K.

Every year I try to do a Questions for the year – themes that I think will be interesting to explore and think about. At the end of the year, I go back and see how wrong I was.

For 2012, the Questions are here.

For those questions here’s some answers:

1. Pearson LMS? No big deal. I think the Blackboard free LMS is more important in the LMS space, but Pearson may be doing some things, but nothing big or earth shattering. Of course, a lot of faculty I work with don’t use Pearson texts.

2. Web mining useful? Ultimately yes, but increasingly difficult to do. With Twitter becoming more walled off, Instagram way more walled off and Facebook increasingly walled off, it’s much more difficult to use something like Ifttt to get something cool to mashup. It’ll be interesting to see how open data sources survive, and whether APIs will wither. I’d like to see more open data – I think it’s where we’ll see growth and interesting possibilities emerge. From an economics standpoint, these sorts of niche areas will be tremendous economic generation in the future.

3. MITx? In and of itself is not that big, but EdX, Udacity, Coursera and the others are making MOOCamania running wild on you. Credentials is still a big thing, but I suspect that’s the gateway and where these startups will make their money – partnering with a school who will rubber stamp their findings – or partially rubber stamping credit.

4. Android tablets in Education? Big fart of air. iPads still rule. Android will suffer for the hundred of crappy tablets and lag of killer apps on the platform. For phones, it’s fine; for tablets, not so great.

5. Learning Technologists? Still play their/our marginal role.

2 Comments

Answers for 2011

Posted on by Jon K.

Well, I guess a year’s time is as good as any to have some answers – even if the answer may very well be no answer. For the original post see: Questions for 2011. Yes, there will also be a Questions for 2012.

1. What makes anyone think that the video games push (mostly by the iOS platform devices, but Xbox, Playstation and Wii) has anything to do with formal education? 

Well, I don’t know if gamification gained any traction, but things like achievements in video games have lent themselves to things like badges. I suspect that my original assertion that it will be marginalized, will remain until someone can quantify and measure the whole process, much like they’ve tried to do with standardized testing.

2. Why haven’t educational institutions really pushed for a mobile learning environment? 

I think there’s been some motion here – certainly the open courses are structured so that they are mobile friendly, and the big two LMS vendors (Desire2Learn and Blackboard) are both becoming more mobile friendly, I suspect the resistance comes from the institution’s inability to control and verify that a potentially mobile student may not be that student, and the only way to assess a person is still in-person. I don’t think it matters anymore, in work most people will use the Internet to research a possible solution to whatever problem they face, so knowing something isn’t as crucial as it once was. Knowing something however does allow you to find a solution sooner – making you a more efficient worker – which is what capitalism wants.

3. Will the consolidation of the web conferencing tools that education typically use (Wimba and Elluminate) mean that new companies with new models will arise? 

Well, they haven’t arisen yet, but there’s a plethora of tools out there to replace Blackboard Collaborate or whatever it’s called this week. However, no one has put together the killer app – which I hope is the form the web conferencing takes – mobile native, low bandwidth friendly, and most of all, accessible.

4. Wither edupunk? 

Yup. edu-post-punk should be interesting.

5. What will Pearson as a publishing giant and accredited University mean? 

Turns out, not much. Unless you consider an extremely walled off garden of textbooks in a proprietary LMS with Google Doc integration something.

No comments yet

LMS Review

Posted on by Jon K.

I’ve spent 19 hours at work the past two days, so I come home, plop down in front of the computer and blog about work.

As with all Colleges, we’re in the final days of preparation for the first classes next week. For us at Mohawk, that means manually (well, through input scripts) creating classes, enrolling students and creating instructor accounts. Until last year, Mohawk has never had a policy that said “we as an institution will use one LMS”. So the e-Learning department (of which I am a small part) has been very very lucky to be able to experiment with several different platforms. FirstClass is one of the longest running options at the College – having been in use for at least 9 years, probably more. If you’ve never used it, you’ve missed out. It’s a standalone client, so it’s not a web browser based solution, and while it’s quirky, it has lots of options for collaboration. In fact, having used it for around 8 years, I think it’s still heads and shoulders above the other LMS’s in that aspect.

WebCT is still running at Mohawk as well, and I never really had to use it but the instances I’ve had to develop content and media for it, it’s fine. I never liked the view options (switching from designer to student view to see how things look and function), and it actually plays nice with others (sort of). I mean you can export something and import it somewhere else and it kind of works fine.

Mohawk’s also running an instance of Blackboard CE 6, which as I understand it is some hybrid between WebCT and Blackboard. Like WebCT, it’s ok. Fairly locked in, and creating user accounts on it is a real pain, involving a custom hacked Perl script, XML massaging and a CSV. Getting students in shouldn’t be this difficult should it? Of course Blackboard offered to integrate the system with our Student Information System, for a fairly large (to me anyways) sum. No thanks, how about creating a way to bulk import students that doesn’t take half an hour?

We also have an installation of Moodle. Which was pretty daunting for faculty to use as there’s not a large support component for them. That’s not to say that Moodle doesn’t have a large support community, it most certainly does. Our faculty are not the most e-learning adventurous, and the ones that are, are already using one of the 5 systems in place (FirstClass, WebCT, Blackboard, Desire2Learn and the Portal CourseTools, which e-Learning doesn’t have any control over). So the ones who might’ve been interested in trying Moodle were probably scared away by the lack of immediate help with the system. Which is too bad, because the flexibility it can deliver is really nice. It’s the MySQL of osCommerce (or WordPress?) of Learning Management Systems.

Last year, e-Learning (I was only peripherally involved with the department as I was working with technology for second language learners at the time) went through a nine month review process, to look at acquiring one system to replace the six. Desire2Learn came out on top. The final paper is on the LMS Review blog.

Having worked with Desire2Learn for about four months, I’m disappointed with the collaborative tools (blogs specifically, but online documents were something I had hoped they would be developing).  I guess I shouldn’t be, my expectations of a modern system is far and above what the average instructor or user would expect or need. And it’s perfect for that beginner user – I just hope that we don’t end up down the garden path and find out that it’s not quite robust at that level. Of course, I know it sort of is. Barry Dahl and Kyle Mackie are always posting about interesting things to do with D2L and I hope that our faculty can get to that point. I guess it’s part frustration that we haven’t used it before, and even though our admin has been using a lot since February, he still feels he has no mastery of it. That to me suggests there’s some depth to the system. Maybe we’re all a bunch of self-deprecating navel gazers? It is an exciting time at Mohawk. I talked to two faculty members today who were genuinely excited to use the system, which is all I need to get through the day I suppose.

2 Comments