CSG Fall 2013 – Is the Grass Greener? Exploring the New Wave of Course Management Systems, Part 4

Washington – Use of Canvas

Wanted a flexible, extensible elearning platform; cloud-based and built for the Internet; speed to innovation; open standards and SOA (fits with Oren’s definition :); Modern user experience

What would make the Canvas LMS successful at UW? Pilotded during Autumn 2011 & Winter 2012; Wanted to collect informtaion about engineering and support in addition to faculty reaction. Found that both faculty and students rated Canvas very highly. Faculty who used it multiple times liked it even more. Faculty think using Canvas has made teaching more efficient. Repeat users started using new features, because easy stuff was easier.

Engineering – no technical impediments uncovered (auto-provisioning 10k course sections for Winter today)

Support – Found it easy to support. Some features were missing at beginning, but Canvas responded, doing releases every 3 weeks, and by spring problems were ironed out.

Have integrated enterprise groups within Canvas, so can create courses and course groups. Integrated with thei Astra role management. Doing social ID login. Now have 21000 students and 1200 instructors. 

Moving to Canvas – Jenn Stringer, UC Berkeley

The decision of what LMS to choose ends up being political as much as anything. Berkeley has recommendation to move from Sakai to Canvas. In the midst of a pilot phase. 

Upgraded Sakai this year so they have a stable platform to support a migration. Sakai has been the only free collaboration system at Berkeley, so they have 8,500 project sites that aren’t courses. Canvas is truly a learning management system, so won’t port those project sites over.

Motivation for change: faculty frustration with perceived lack of functionality within Sakai (could be more perception than reality); Campus commitment to select a next generation teaching and learning tool – initial plan to have a joint course/collaboration environment using Sakai OAE; withdrawal from OAE project in Fall of 2012; UC Online and UC Berkeley Extension commitments to Canvas. 

Limited Pilot – 10 courses – broad use cases with ~400 students.

Faculty findings – navigation was easy and intuitive; upload of images is faster and simpler; assignments tool was easy; quiz tool made the creation of formative, informal assessments much easier; modules tool made organizing content much faster; release cycle (every 3 weeks) confused some faculty with tool updates. 

Students – 66% agreed that canvas helped learn course content more effectively.

Identified issues: not great (yet) for large school administration, doesn’t have the kinds of tools for sections that Sakai has; Communication tools for subgroups are not great; Accessibility  – core student tasks acceptable – Berkeley has identified some issues, but not show-stoppers. Have given issues back to Instructure and they’ve been responsive – will that change as they grow?

Technical – SIS and SSO integration done – very easy. Local CSS & Javascript customizations including skinning/branding. LTI interop integrations with local tools; Leverage APIs to localize user experience. Capability of only showing photo rosters to faculty is not in Canvas, so they built and integrated a tool for that – took 2 weeks. 

CalCentral portal rolled out this fall – very lightweight Ruby on Rails app (moved away from Java). Easy to build integrations from Canvas into portal – mashup of Google task list and Canvas assignments for students. Mashed up activity streams from Canvas and the SIS. Announcements took .5 day to integrate with Canvas, but a week with Sakai. Canvas APIs “a dream to work with.”

Looking at Net+ licensing; 24×7 support is available; Cloud licensing is hard at UC. 

One issue is who has the right to take down content – Instructure has some language reserving their right to take down content – having discussions about how to deal with intellectual freedom and copyright. 

Next steps if recommendation goes through governance: Phased rollout with Opt-in. 5000 students this fall, 15k in spring, final rollout in Spring 2015.



CSG Fall 2013 – Is the Grass Greener? Exploring the New Wave of Course Management Systems, Part 3

Aisha Wood Jackson – University of Colorado.

Teaching and Learning Applications program. Supports iTunes U, Kaltura, VoiceThread, as well as LMS. They have Google Apps, and Coursera, as well as Desire2Learn. 

Came to D2L after being on WebCT. Two committees formed to come up with selection criteria for replacement – functional stakeholders and technical stakeholders. Criteria included Experience, background, and qualifications; features and usability; LMS admin; Product environment; product roadmap; financial. D2L won on everything except financial and Experience, background, and qulifications.

Using hosted D2L.

Implementation schedule: Spring 2011 pilot; Fall 2011 opt-in period; D2L default Spring of 2012; D2l All-in Fall 2012. 

2291 courses this fall (out of about 5000 total courses) on D2L. 

Tools used – Content (43%), Homepages (28%), Discussion (12%), Quizzes (11%), Grades (2%), dropbox (3%)

Had series of outages at beginning of spring semester – upgraded to new interface in January, started having outages and access issuesin mid-January, leading to a 61 hour outage on January 29 – Feb 1. D2L said failure while migrating CU to new file storage system. Yes but – D2L didn’t consider risk and potential impact and lack of communication about change: weren’t informed of work or given a scheduling choice. 

Managing after outage – transparency and communication was key (used lots of social netowrks); emphasized that OIT was not D2L; CEO and COO of D2L sent communication to the campus; Provost and CIO kept communicating to campus. Communicated alternatives to D2L during outage – how to email classes, share docs, etc. 

Afterwards had Faculty review committee look at options: staying with hosted D2L, self-hosting D2L, or middle-ground with a redundant system on campus. Did an audit and on-site visit to D2L. Decided to stay with hosted D2L. Saw that D2L was putting processes in place to do a better job. Changing agreement to require consideration of risk and impact when making changes, and requiring notification of changes. This year D2L implemented a change freeze during school start. 

CSG Fall 2013 – Is the Grass Greener? Exploring the New Wave of Course Management Systems, Part 2

Moving NYU from Blackboard to Sakai – Kitty Bridges.

NYU turned off Blackboard on August 15. On November 14, 2011 the Provost announced plan to replace Blackboard with Sakai, with goal of completing transition by August 2013.

Why Sakai?

Recommendation made by Teaching Technology Committee. BB would have required a substantial upgrade; Open source providing framework for integration (integrating things with Blackboard was painful); Gives NYU a voice in the development of Sakai; Positive experiences with Sakai at Med School; Lots of new capabilities for loose or tight integrations.

Phased approach – phased by school, so had students living in two LMS systems during transition. Phase 1 – exploratory group of 20 faculty, integration with SIS; Phase 2 – fully transition two mid-sized schools, begin migrating Bb content to Sakai; Phase 3 – All schools.

Migrating Bb Content – mostly good news. Leveraged import code from Sakai community. some issues with tests and surveys (data structures not quite the same), and some issues with really big files. Automatically migrated five semesters of content, and archived older content.

Project website: http://www.nyu.edu/nyuclasses-project/

Communications – short and actionable. Can directly see uptick in visits to training and support site on days when email to faculty went out.

Communications and online training resources – high degree of satisfaction.

Did a self-service course site creation. Puts decision to cobine sections into one course site in the hands of faculty and admin staff. 75% of faculty feel confident in ability to create and publish course sites.

Anticipated 6500 course sites to be created last winter, but had 6800 – anticipated numbers were based on Bb usage, so might have new users who didn’t use Bb. Huge number of visits to training and support site. No increase in help tickets from previous year.

Lessons – governance model and academic leadership was key; early outreach to deans, chairs, and IT partners; early training with videos, faqs, and in-person options were key; self-service site creation (the sooner the better); 18 month project lenth was right and phasing was right; Early and thorough testing of the migration of most complext test and survey content. Prepare team or instructors to deal with question types that differ, requires some manual reworking; Make sure you have a methodology and enough staff to deal with migration of large sites.

Staff resources: Bb 2.5 FTE faculty support; 1 FTE for accounts; 1.75 FTE for systems. Transition: 5.8 FTE technical; .5 FTE communications ramped up to 7 FTE + 4 students during height of transition; New system steady state: 3 FTE faculty support, .25 FTE for account management, 1.5 FTE systems.

Have new group of 16 instructional technology staff, centrally funded. Dotted-line report to teaching technology committee, deans very heavily involved. Central intake point, leveraging Service Now for requests from faculty for help.


CSG Fall 2013 – Is the Grass Greener? Exploring the New Wave of Course Management Systems, Part 1

Survey Results: Kelly Doney – AVP for Enterprise Applications, Georgetown

Started with a significant skepticism about Blackboard – wanted to understand if new players on the block had anything new to offer.

The landscape – On average, each institution has 2.88 LMSes. Harvard has 6, Penn State has 1. Great diversity of platforms being used.

50% use Sakai for the primary LMS, 24% Blackboard, 12% Moodle, 8% Desire2Learn, 12% Moodle. On average have been on their primary LMS 8 years.

36% of institutions migrated within last 5 years (almost all from Blackboard), 38% considering changing their primary LMS. Most popular candidate people are considering is Canvas, next is Blakboard.

Most people feel that their systems are reliable, but are dissatisfied with user experience, regardless of product. 

Features most used; grade book, photo reoster, faculty-student communication features, uploading course documents, assessments. Least popular: wiki, blogs, maybe because there are better products being used for those features.

Integrations – Kaltura, Echo, Voicethread, WordPress, Coursera, Edx, Google apps

Training and support: Average number of staff supporting primary LMS: 5.86 FTE. Nobody trains students, but targeted or 1:1 trainings are generally more popular than mass trainings. NYU, Duke have office hours for LMS support. Online documentation is popular. 52% offer around-the-clock support for LMS. Georgetown and Berkeley reported 2 FTE, and at least Georgetown that’s just the technical staff. Michigan reported 20 people, but that includes help desk staff.

On average, volume of support calls did not change with transitions to new system. Notable exception: UVa – more calls because of more capabilities and features in new system.

Student needs and satisfaction: Students care most about reliability. Students want faculty to use the LMS more, and use more of the LMS. Average of 70% of faculty and courses using the primary LMS.


Level of LMS Use – Basic users – 60% of faculty, moderate, 25%, Advanced 15%. Common obstacles – Grade book, integration with SIS. But at the same time grade book is the most popular feature. 

Reliability and update cycle: system reliability is generally high. Most schools update annually or biannually. USC avoids all even number releases of Blackboard.

84% host on premise. In future 67% of those thinking about changing LMS would host Off-premise, 33% undecided. Nobody said they wanted to host on premise in the future.  Duke hosts off-premise – went their because it was the fastest way to switch their system.

East of migration to new system: on a 0-100 scale, average score 48.5. NYU – Bb-> Sakai at 90.

Many people seeing that the future is likely to be an integration layer – Gartner sees an environment of a middleware layer as a learning platform connecting many pieces. If that’s true, is this the time to be changing LMS to a new monolithic platform? 

Discussion – one of the values of the LMS is providing a consistent home for each course. Also the integration of assessment tools with gradebook capable of complex calculations.

CSG Spring 2013 – Campus Computing Sites, Public Labs, Computing Clusters in a BYOD World

CMU, Duke, Princeton, Michigan

Now used as collaboration and social spaces, quick access to kiosks, access to fasterm machines, large screens, printing, special apps, adaptive tech, electricity. Access for the have-nots. Parallel service: virtual access to specialized applications

Magnitude of spaces – most have shrunk the “rows” model, most have shrunk the the computer classrooms.

At CMU some departments have shut down labs, putting more pressure on central labs. Offering over 100 applications. Also using VDI. Strategy is to continue to decrease physical devices iteratively. Have a new site without computers but lots of connections for people and good videoconferencing. Will compare the use of that with sites that still have some devices. There are licenses that don’t work very well in this playing field, especially Adobe. Students prefer to carry iPads over laptops, and VDI doesn’t work well on tablet devices. 

Duke has had a hard time running the remaining Unix desktop labs due to short expertise on Unix desktops, so they’ve subjected the desktops to change management, but that frustrates faculty. They have VCL, but it gets very little use. May end up pushing labs back into the departments. 

Princeton has 1/3 the number of machines in labs that they had at the peak, but the amount of space they’re using is pretty much the same. Making spaces into collaborative spaces with big monitors, KVM switches, etc. In residential spaces they are defining lab spaces as preferred study spaces. They also observe that people aren’t carrying laptops but are carrying iPads. They run into labs between classes when they have to compose things for a 5-10 minute period. 

What they shall be: Collaboration and social space with access to large displays and keyboards, very large displays (wall size) for collaboration, printing, poster, 3D, adaptive technology, tools, electricity. Specialized labs with access to video/audio production. 

Labs evolving that are more specialized – more like chemistry or dissection labs. Chicago is working with a faculty member on a “maker” style lab with 3d printers, laser cutters, and space to do Arduino like projects. 

Global learning presents challenges, especially around software licensing. Some vendors have been reasonable, some impossible. 

VDI presents problems for software where people expect really fast screen refreshes, so doesn’t get well used in creative and design environments. Where sound and image need to be synchronized can present problems. 

Wisconsin has a restaurant that has a booth with a large screen that students can plug into while they eat. 

People are hearing requests for sharing a large screen over wireless. AppleTV is too idiosyncratic for large-scale rollout, but generates higher expectations. 

And with that… we are adjourned!


CSG Spring 2013 – MOOC Workshop, continued – Future challenges

UCSD – Emily Deere

In trying to create a central online course for the UC campuses, they can’t pass records among the campuses to award credit for a course taken at another campus without a paper process. Registrar’s office, and departments of majors and minors are a challenge – will they accept credit for courses from another UC campus? They’re going to try to create a clearinghouse to get some interoperability among campuses. 

There’s a bill in the state legislature that will mandate that if a student can’t get into a course in their major that the campus will have to accept credit for an online course that is somehow equivalent.

Bruce Vincent – Stanford

How much do we want to project infrastructure into MOOCs? We’re trying to respond as support organizations – our response will be appropriate or not. We need to quickly learn lessons and accept a range of things like weak binding in identity space to assurance of identity for credit. We need to be able to support the world of the cloud platforms that these things run on. These all make our existing infrastructure look a lot less shiny. The parts of the organization dedicated to supporting LMS may be struggling for relevance. 

CSG Spring 2013 – MOOC Workshop – Panel continued

Tracey Futhey, Duke

Duke Digital Initiative, goes back to the issuing of iPods to freshmen. Been trying things to get faculty involved in experiments. MOOCs are inspiring interest in innovation. Also driven by Duke’s commitment of service to society. Approach has a couple of components (try multiple things): got involved with Coursera, have now done 11 courses pushing close to a million registrations. Took significant effort. Looking at for-credit courses internally (the 2U effort). Having an issue with some faculty who do not want to do online for-credit courses, which they’re working through. 

Materials reuse and access not just in the class, but components across multiple classes.

Sooner is better – do courses quickly rather than high production values. Two of the eleven courses have been offered a second time to date. 

Most popular of the eleven courses is a philosophy course (185k people). 2/3 outside US.  Professor describes situation with two people who are really active – turn out to be 9 and 11 year old brother and sister in Pakistan.

Just finished an astronomy course with a physicist. Eight week long course, with students doing between 10-15 hours of work per week. He found that in the course, which was offered simultaneous with an on-campus version, the on-campus students only got through six of the eight weeks that the MOOC students did. 

Asbed Bedrossian, USC

President’s vision is that the focus for online programs is on post-undergraduate and lifelong learning. Online degrees will use normal USC admission process and charges, with the same expectations for rigor on the part of students. USC will not offer online degree at the undergraduate level. In the next few years they expect all 18 USC schools to offer online post-graduate programs. 

John Krogman –UW Madison

Offering four MOOC courses this coming fall. “It’s the instructor, stupid.” Partly marketing, partly research, part of the bigger picture of grappling with disruptive change. Not qualitatively different than what we’ve done for years with innovation – lab sections, field trips, internships, simulations, web sites, etc. Need to give students soft skills as well as subject matter expertise. What measures will determine success of MOOCs? Which brings up the issue of how we measure learning outcomes at all. 

What experiences will generate loyal alumni? Will MOOCs do that or are they just PR for the institution?

UCSF will be offering continuing medical education online through MOOC platforms. Also doing courses to advance world-wide health. Three courses: Nutrition, contraception, and diagnoses. Saw students as young as 13 taking the nutrition course.