MOOCS – Oren Sreebny

CSG Spring 2013 – MOOC Workshop, continued – Future challenges

UCSD – Emily Deere

In trying to create a central online course for the UC campuses, they can’t pass records among the campuses to award credit for a course taken at another campus without a paper process. Registrar’s office, and departments of majors and minors are a challenge – will they accept credit for courses from another UC campus? They’re going to try to create a clearinghouse to get some interoperability among campuses.

There’s a bill in the state legislature that will mandate that if a student can’t get into a course in their major that the campus will have to accept credit for an online course that is somehow equivalent.

Bruce Vincent – Stanford

How much do we want to project infrastructure into MOOCs? We’re trying to respond as support organizations – our response will be appropriate or not. We need to quickly learn lessons and accept a range of things like weak binding in identity space to assurance of identity for credit. We need to be able to support the world of the cloud platforms that these things run on. These all make our existing infrastructure look a lot less shiny. The parts of the organization dedicated to supporting LMS may be struggling for relevance.

CSG Spring 2013 – MOOC Workshop – Panel continued

Tracey Futhey, Duke

Duke Digital Initiative, goes back to the issuing of iPods to freshmen. Been trying things to get faculty involved in experiments. MOOCs are inspiring interest in innovation. Also driven by Duke’s commitment of service to society. Approach has a couple of components (try multiple things): got involved with Coursera, have now done 11 courses pushing close to a million registrations. Took significant effort. Looking at for-credit courses internally (the 2U effort). Having an issue with some faculty who do not want to do online for-credit courses, which they’re working through.

Materials reuse and access not just in the class, but components across multiple classes.

Sooner is better – do courses quickly rather than high production values. Two of the eleven courses have been offered a second time to date.

Most popular of the eleven courses is a philosophy course (185k people). 2/3 outside US. Professor describes situation with two people who are really active – turn out to be 9 and 11 year old brother and sister in Pakistan.

Just finished an astronomy course with a physicist. Eight week long course, with students doing between 10-15 hours of work per week. He found that in the course, which was offered simultaneous with an on-campus version, the on-campus students only got through six of the eight weeks that the MOOC students did.

Asbed Bedrossian, USC

President’s vision is that the focus for online programs is on post-undergraduate and lifelong learning. Online degrees will use normal USC admission process and charges, with the same expectations for rigor on the part of students. USC will not offer online degree at the undergraduate level. In the next few years they expect all 18 USC schools to offer online post-graduate programs.

John Krogman –UW Madison

Offering four MOOC courses this coming fall. “It’s the instructor, stupid.” Partly marketing, partly research, part of the bigger picture of grappling with disruptive change. Not qualitatively different than what we’ve done for years with innovation – lab sections, field trips, internships, simulations, web sites, etc. Need to give students soft skills as well as subject matter expertise. What measures will determine success of MOOCs? Which brings up the issue of how we measure learning outcomes at all.

What experiences will generate loyal alumni? Will MOOCs do that or are they just PR for the institution?

UCSF will be offering continuing medical education online through MOOC platforms. Also doing courses to advance world-wide health. Three courses: Nutrition, contraception, and diagnoses. Saw students as young as 13 taking the nutrition course.

CSG Spring 2013 – MOOC Workshop – Panel

Confessioans of a MOOC Dilettante – Alan Crosswell, Columbia

Has traken a few MOOCs, including AI-Class.com from Stanford, MITx 6.002x Circuits from MIT, NLP from Columbia, Big data class from U Washington.

Attributes – not classroom lecture capture. Not 48 minutes of straight video. It’s time-shifted yet time-bound (not a set lecture time but assignments due each week); Community participations (replaces in-class experience); Massive and open.

Only some courses feature talking head video – does that add value?

Progress indications in some MOOC platforms are very useful.

Downloads of marked up lecture slides are good.

Not all the features (like discussion forum) are good in all platforms.

Recitations, worked problem sets and supplementary tutorials in MIT’s circuit class are good.

Immediate exam feedback (per-question vs. entire quiz) is good.

Instructors and TAs don’t sleep during the course because they have to constantly monitor the content and performance of the course. How many of the MOOC courses have run a second section?

Georgetown U and the Initiative for Technology-Enhanced Learning (ITEL) – Kelly Doney

What is the ITEL initiative – designed to address the broad spectrum of technology enhanced learning. Allows them to very quickly redesign elements and look at effects, to experiment with different approaches. Made an $8 million investment over 3 years. Money goes to fund instructional designers, faculty grants, and infrastructure.

Why do ITEL? Responding to disruptors in higher education. Initiative led by Provost, who previously was Obama’s head of the Census Bureau. Extremely data-driven. “Believe that the undergraduate experience cannot be replaced by MOOCs, but we need the data to prove it.”

MOOC platform selection process – Analyzed options and developed a funding strategy (including faculty grants). MOOC Partner decision matrix – felt EdX shared the concern for the Georgetown brand; platform allowed reuse of materials outside the MOOC; lots of data and analytics; interested in being able to shape direction. Selected edX.

Released initial call for conceptual proposals in February2013 – received 55 proposals, then received 43 full proposals, and gave 5 Level 3 grants on May 1. Demonstration grants – 1-2 semesters to transform existing materials. Level 2 grants – Level 3 – Design and implementation grants – take 2-4 semesters, design new or significantly changed experiences. Transformation grants – up to three years to completely bring a course online or develop a MOOC. Adding 2 MOOCs for Fall 2013 – Bioethics, Glboalization: Winners and Losers.

Trying to drive online learning architecture – IT has a seat at the table overall.

Current architecture: “pastel spaghetti” – trying to simplify, with the LMS at the center. Working directly with business units, and driving common requirements, forcing vendors to do business to work within their technology stack. Business school wanted a platform with richer UI than Blackboard could provide, but they (IT) are working with Blackboard to enrich the UI, even as they launch the program with another product.

Architecture – develop architectural strategy driven by smart IT. ITEL, MOOCs, and distance Learning – leadership from President and Provost essential; Transparent processes and involvement of key non-faculty stakeholders are important.

MOOC Landscape at Penn State – Cole Camplese

96k students, 23k World Campus enrollments, 24k full and part-time faculty.

Decided to go with Coursera.

Why participate? PSU has established revenue from World Campus, most of which is returned to the programs that give the courses.

World Campus – more than 90 existing programs with 50k annual tuition paying enrollments. Goal is to get to 45k full time students. Investing $20 million to do that.

40% of residential students take at least one web course per semester.

Started MOOC discussions in 11/12 to first PSU MOOC delivery in 5/13 – extremely fast motion for campus.

Invested in a Center for Online Innovation in Learning – awarding research innovation grants – like one to look at whether talking head vieeo in MOOCs is useful.

Built a fovernance model – Subcommittee of PSU Online Steering Committee (administrative); MOOC strategy group – trends, design, assessment; MOOC reporting group to report to the provost.

Multi-tiered strategy – PSUE Coursera courses for total external MOOC audiences; MOOCs coupled with existing PSU world campus course; Large scale internal courses (e.g. English 15 which enrolls 15k students annually); PSU MOOCs designed under Penn State brand.

First five courses: Intro to Art (~47k enrolled), Maps and the Geospatial Revolution (~15k); Creativity, Innovation and Change, Energy the environment and our future, Epdiemics, the Dynamics of Infectious Diseases. Two are from faculty who have never taught online, which presents challenge.

Sociology 119 – making a television program that will be distributed by PBS, along with an app, and a MOOC. The program is called “You Can’t Say That”. National TV and web audience combined with a mobile experience. Mobile app posts to twitter, which then get interested in the MOOC, and curated into the course.

All courses except Art are taught by teams of faculty. For Art hiring undergrads to monitor discussion boards.

CSG Spring 2013 – MOOC Workshop – Survey and discussion

This morning’s workshop is a workshop on MOOCs.

Chuck Powell begins by asking where we are on MOOCs in the Gartner hype cycle – approaching the peak of hype, or heading into the trough of disillusion.

The CSG MOOC survey shows 60% of respondents have already begun to offer MOOCs, most (50%) using external providers. 4% have explicitly decided not to offer MOOCs now.

Number of courses range from 1 to 20, with an average of just over 7, growing slightly in the next year.

Why are they offering MOOCs – 30% desire to support global education, 18% for marketing and outreach, 25% are driven by faculty desire, 14% for competitiveness. It’s noted in a comment that lots of this is driven by schools not wanting to be the last in the game, driven by fear, uncertainty, and doubt. Maybe it’s better for us (universities) to be driving the bus rather than have it done to us by outside forces (like happened in scholarly publishing). One institution notes that it’s the best teachers at the institution who want to participate. Another says that desire for growth at global campuses is part of what’s driving them to participate. One flagship public institution says that they feel a mission to provide resources to the other institutions in the state.

The calamity factor – provosts want perception that they have it thought through before they are advised to (see the Virginia incident).

To what extent is this necessary to recruit and/or retain faculty – one institution says that this is definitely seen as a factor by one new center.

MOOCs don’t usually pay faculty, but can result in an uptick in textbook sales.

One idea is institutions using MOOCs for finding new faculty to recruit.

45% of respondents are discussing the possibility of granting credit. Georgia Tech online CS program for $7k. The big question (for the institutions in this room) isn’t granting credit, but accepting it. Will we grant credit if our own students have taken our own MOOC? Some states are talking about how to take the first year out of the college cost structure.

Some of this is a question of what the value added is when we grant a degree. Is it just a certification of an assemblage of experiences, wherever they come from? Imagine if undergraduate courses are taught more like graduate seminars.

33% are considering monetizing your offerings. Could get revenue from reaching new audiences, either outside the country or in new vertical markets. Total costs for offering a course on a MOOC can be upwards of $100k (not everyone agrees that they’re spending that, citing more like $20k out of pocket). How is that sustainable? Consider Khan academy – not so high fidelity yet very effective. It can be a long tail model.

It’s early days. Like we saw with mobile and social networks, these will get monetized and it’s important to be in the game at the beginning.

MOOCs can be an important marketing tool for smaller schools.

There’s a good opportunity if we can see that we shouldn’t compete on whole courses, if we can dis-aggregate content and reuse it in pieces in our institutions. There are discussions of inter-institutional arrangements (CIC CIOs are talking, as are the state systems). Can we build a repository of learning materials – we’ve done that before, but haven’t gotten used. Some of those materials already exist out on YouTube and get used in our institutions – hard to beat free, but there could be some kickass materials with a per-use fee.

Are students really learning in these experiences?

Are MOOCs going to replace brick and mortar? Probably not for top tier residential institutions. We have a long tradition of credentialing our faculty as experts – how is quality determined online? MOOCs don’t replace the connections with alumni, careers, etc. that people get at online courses.

There’s an opportunity for IT groups to be involved in helping people run infrastructure.

One school – we don’t know where this is going to go, so let a thousand flowers bloom. Wide open for experimentation. They want to gather data, so they can then evaluate what works and what doesn’t. People are excited about being able to get learning metrics which they couldn’t get before.

One of the key returns may be the effect of MOOCs in the residential instruction model. This will change how our campuses operate in the next five years – what does this do to the model of semesters and years when we’re freed from the physical constraints? One person thinks that it won’t have an effect for twenty years, while another thinks that it’s already starting to change with the effect of flipped classrooms.

Faculty are inviting staff (whether IT or others) in to help create the courses for MOOCs.

How do we archive the materials that come out of MOOCs? What’s the role of libraries?

One campus is rejecting online education for undergraduates, but embracing it for graduate education.

MOOCs allow students to audition subjects and faculty.

Why are we still building large lecture halls? Part of the investment in MOOCs may be recouped in not having to build those. Buildings are 30 year decisions, while MOOCs are 1-3 year decisions. How many of us are concerned about the changes in physical models due to this?

This is similar to discussions we had five years ago about Second Life – what’s different now? Are MOOCs a distraction to concentrating on the changes in pedagogy? The real impact of MOOCs could be competency based education – we should pay attention to what’s happening in Europe with testing of skills.

Physical changes could be in building labs and meting spaces instead of lecture halls.

Success may be based on agility – look at Amazon.

Some institutions are looking at online degree programs to replace diminished funding from states.

MOOCs are driving a new conversation about teaching and technology in ways we’ve been trying to get people thinking about for twenty years. Faculty who teach MOOCs learn new things about their teaching.

Faculty who are teaching MOOCs are taking parts of them back into their on campus courses.

CSG Winter 2012 – Policy Discussion on MOOCs

Policy discussion – MOOCs

Courses are all very large enrollments (100k +), based on institutional content. Also seeing specific modules being deployed for residential learning.

What are the challenges for institutional IT?

When we get to the point where it’s not just freebies given to anonymous users, and want to use them for our students, we are facing big identity management issues – do we need federated identity for MOOCs?

Issues around support – is there funding? Some are focusing on very heavily produced – one figure is that for every hour of MOOC course time it takes 9-12 hours of staff time. $8-12k per course. Faculty will want to redo courses periodically, so it becomes a compound interest type of curve. In a few years you will be doing a whole lot of courses.

MOOCs have generated faculty interest in flipped courses, and that’s creating demand which can’t always be supported.

Not all the courses are so highly produced – the AI course from Stanford is the professor in front of his web cam.

Some schools are making self-production facilities available for faculty. It’s taking a lot of faculty time to rethink material into smaller chunks and tailored for online. Some faculty want very highly produced material – this will be all over the map for some time. For some faculty it’s very difficult to teach without a class of students in front of them.

How do we evaluate the quality of courses? This gives schools of education an opportunity to do research.

Retention rates for many courses are in the single digits. Many of the students are international, so it’s a different kind of student body.

Wall Street Journal did an article on business models for MOOCs, and there’s an Educause Executive Briefing – What Campus Leaders Need to Know About Moocs.

Some of the courses have experienced a high degree of mentorship among the students from people with a lot of professional experience in the field – but will that persist when the novelty wears off? But it will be sustainable if they replay the course in subsequent iterations.

Dan Russell at Google has been running several rounds of a MOOC course on how to search well (http://www.google.com/insidesearch/landing/powersearching.html ) – it’s about an eight hour course. He makes a point that a lot of the potential is that with these enormous enrollments you get a good deal of feedback about what works and what doesn’t. When you combine a commitment to capitalize on that feedback with the fact that these courses are dealing with technical areas that evolve rapidly it means that you’ll be doing a lot of revision on these courses. The more you invest in production values the more you’re taking a do it once and done approach that makes it hard to revise.

You want to be careful about your digital asset management – e.g. keep the talking head and the slides as separate streams so you can revise them separately. When you think of these as living things you can structure them in ways that allow for continual revision.

Some people think of these more as next generation textbooks, which is highly produced as a product that people will treat as definitive. That approach may be antithetical to the ways millennials want to learn.

You have to be careful about how you set up the course, in terms of what you do if people fail quizzes, etc.

Who gets any revenue generated from these courses? Is it more like textbooks (revenue goes to faculty), or class materials (institution owns).

What are the reasons institutions are participating? Brand enhancement, a sense that there’s something new and important starting and they want to part of the conversation.

In some fields employers will be setting up their own training environments and hiring star teachers to teach skills to employees, e.g. Cisco University

CNI Fall 2012 Meeting: HarvardX: Developing Communities of Practice for Innovation in Online Learning

HarvardX: Developing Communities of Practice for Innovation in Online Learning
Samantha Earp, Susan Fliss, Harvard University

Most in the audience think MOOCS are at the height of inflated expectations on the Gartner hype cycle.

Early work in progress at Harvard (in startup mode).

EdX was launched in May, following on MITX courses. Harvard and MIT are founding institutions and funding initial work. Slowly growing in a deliberate, measured pace. Evolving to a software + services organization.

HarvardX is the Harvard engagement manifesting as a set of courses on EdX.

High level institutional goals: Improve education on campus, bring education to the world, conduct research on how to deliver on the first two goals. Alignment with educational mission at Harvard, priorities should align with the educational goals of the individual schools. Starting with intent that everything recirculates back to campus.

A very faculty-driven effort. Hoping to push the envelope of using the platform – many examples of online and distance learning at Harvard already. Not looking for a cookie-cutter approach, as opposed to the templated approach in older LMS systems. Looking to explore what’s possible, and willing to tolerate some mistakes along the way. Not just a publication effort – not just taking a course and crafting for publication like a bronze in a museum, but trying to make courses available and learn and adapt.

Faculty have established two high-level criteria – focus on quality and impact. Beyond that focus is on extending and sustaining. Not interested in a series of one-offs. The idea is to foster communities of practice. An important strategy in how we learn from each other.

EdX is governed by board of four members each from MIT and Harvard. Those four (at Harvard) form the core leadership team. THere’s a faculty course team, working on major policy questions, what a course looks like, etc, from faculty perspective. There’s a research committee looking at what is important for Harvard to learn from this.

Course development – have a small core team. Most critical is a HarvardX fellow role – deep pedagogical background along with project management and tech skills. Coordinators of communities of practice and a number of students and contract staff. Will eventually will have research fellows. Will work as data scientists on research questions.

Looking for pedagogy to drive platform development. Slated to become open source platform, which implies a governance and community process that don’t exist yet.

What does it mean to teach in this flavor of online space, and what does it mean for the campus? Leads to a rejuvenated discussion of pedagogy on campus. What pedagogies might be out there already that we should adopt locally?

Intellectual property, copyright, content is an issue.

The opportunity to Collect and analyze data on the librarian participation in courses is exciting to Harvard librarians. Librarians are beginning to plan to support EdX both locally and across the participating institutions. Forming two groups: one on copyright process one on research.

Main issues are use of copyright materials in use, assignment of copyrighted readings, assignment of copyright for original content, applicability of notice and takedown provisions of DMCA, and accessibility. Group is working on suggesting best practices in these areas, using use cases.

Research skills working group in participating libraries – Determining best how to support learners in information seeking tasks and information literacy.

Two groups will collaborate with faulty, academic colleagues, and each other.

Two courses so far. 56,000 people signed up for epidemiology course and roughly 20% are sticking with it. Adding four more courses in spring – law, greek lit, government.

Interested in engaging faculty in developing experimental instructional modules – hoping to quickly ramp up experiments with faculty. May get used in future learning experiences that may or may not be a course. Interested in how this interacts with residential education.

For one course Elsevier allowed page images from the textbook to be used in the MOOC and then sold out the entire press run of that book – they’re interested in exploring more use.

Some materials are clearly available openly, some clearly need licensing, but there’s a large body in the middle that might be covered by fair use.

CNI 2012 Fall meeting – Opening Plenary

I came in a bit late to this plenary, so forgive me for the incomplete notes.

Cliff Lynch, CNI

MOOCS – Machine grading will be important. Masses of data coming out of MOOCS – who controls it and who gets to do what with it? Nobody seems to have asked the students. What’s the pact around learning data? Google decided it would be fun to build a MOOC platform and ran a couple of instances of how to search using Google course. They did it in order to drive use to their product – an example of applying this technology outside of academia, which will be increasingly more common. An affordable way of doing consumer education.

It’s clear that globally promiscuous admission to MOOCs doesn’t map well to the way institutions license content. Will drive greater use of open access materials. Will also drive broader community licensing of materials. Finally we’re seeing some serious work on alumni access to licensed materials – JSTOR is one among players working on this. There will be pressure to move towards more rational personal licensing schemes, but these have high transactional costs.

E-textbooks – Seeing some attempts to do licensing at scale. May have some economic payoff. Completely remap the relationships between faculty, presses, bookstores, etc. One of the messages to take away from MOOCs and e-texts is we need a more deliberate strategy around licensing of instructional materials. A lot of this work has taken place in the IT community, who’ve been able to make some progress. Libraries have stayed away from licensing e-texts at the same time as they’ve developed sophisticated understandings of licensing of other materials that could be brought to bear.

About a month ago there was a short piece in the Chronicle about a new textbook platform. The distinctive feature was that it would report to the teacher whether you’d done the readings. Quoted a few faculty who thought this was wonderful. Do you find this at all creepy? Could ask the same question about MOOCs or LMS systems. Do the students even know? This is an issue waiting to hit the front pages. Need some conversations about privacy and informed consent around these platforms.

Debates going on about how vocational higher education should be, should we still do humanities, are there really decent jobs for STEM graduates, etc. These things are under debate with a new intensity that deserves some serious considerations. The role of employers in training as opposed to universities in teaching – revisit the comments on MOOCS in teaching outside of academia.

Science is under a great deal of pressure. There is a crisis about reproducibility of results bubbling up – some attempts to reproduce results aren’t going very well. If we don’t get this under control it will affect the public support and funding of science.

The rise of PLOS ONE – publishing a measurable part of the scholarly literature. Vetting for correctness rather than ranking. Offers a level of predictability not found in many major journals.

Public libraries are being largely cut out of ebooks, particularly mass market ebooks. Very good example of where licensing can take us, and the extraordinary power that licensing rather than first sale gives publishers.

We’ve had some encouraging court judgements supporting the principle of fair use. At the same time there are some very troubling things in the first sale area suggesting that we may see an increasing limitation to first sale. Broad populace is starting to wake up to this – is someone going to be able to inherit your ebooks?

One of touchstones of CNI’s work has been understanding the changes in scholarly practice. That’s worth continual revisiting. Can identify a number of new developments over the past few years. One is one that would be easy to mis-classify as “big data” – we’ve moved into a world where there is an abundance of evidence, whether that’s an historian visiting records or an archaeologist seeking to understand urn making. We have lots of examples – we want to look at outliers, make sense of millions of email messages, etc. We’re seeing automatic search and clustering tools, analysis of social networks, etc. Different (and predate) big data tools.

Some new scholarly environments – Math Overflow – mathematicians and upper level grad students can post questions and get answers. Has an elaborate system of ratings and rankings – similar to Stack Overflow. Very slce-able scholarly practice communities using these tools in their work. Wolfram Alpha – a new class of information system that has some capabilities for encoding computational knowledge. We need to be very open to recognizing these kinds of new systems showing up in scholarly communities. Scholarly practice does not stay still – change continues to ripple.

Data curation and research data management. CNI has been very active for a decade now, trying to look at what was coming. We’ve seen NSF and NIH requirements, other funding agencies are moving along this path. While we’ve changed the regulations we’re flying mostly blind. We know very little (collectively) about what’s being proposed, what effect it’s having on funding decisions, and whether people do what they say they’re going to do. There’s a tremendous need to collect data so we understand what’s working or not.

Some specific problematic areas – Individually identifiable data: reusing this is very hard We need to think about research continuity and risk management. See Hurricane Sandy.