csg-Spring-2013 – Oren Sreebny

CSG Spring 2013 – Campus Computing Sites, Public Labs, Computing Clusters in a BYOD World

CMU, Duke, Princeton, Michigan

Now used as collaboration and social spaces, quick access to kiosks, access to fasterm machines, large screens, printing, special apps, adaptive tech, electricity. Access for the have-nots. Parallel service: virtual access to specialized applications

Magnitude of spaces – most have shrunk the “rows” model, most have shrunk the the computer classrooms.

At CMU some departments have shut down labs, putting more pressure on central labs. Offering over 100 applications. Also using VDI. Strategy is to continue to decrease physical devices iteratively. Have a new site without computers but lots of connections for people and good videoconferencing. Will compare the use of that with sites that still have some devices. There are licenses that don’t work very well in this playing field, especially Adobe. Students prefer to carry iPads over laptops, and VDI doesn’t work well on tablet devices.

Duke has had a hard time running the remaining Unix desktop labs due to short expertise on Unix desktops, so they’ve subjected the desktops to change management, but that frustrates faculty. They have VCL, but it gets very little use. May end up pushing labs back into the departments.

Princeton has 1/3 the number of machines in labs that they had at the peak, but the amount of space they’re using is pretty much the same. Making spaces into collaborative spaces with big monitors, KVM switches, etc. In residential spaces they are defining lab spaces as preferred study spaces. They also observe that people aren’t carrying laptops but are carrying iPads. They run into labs between classes when they have to compose things for a 5-10 minute period.

What they shall be: Collaboration and social space with access to large displays and keyboards, very large displays (wall size) for collaboration, printing, poster, 3D, adaptive technology, tools, electricity. Specialized labs with access to video/audio production.

Labs evolving that are more specialized – more like chemistry or dissection labs. Chicago is working with a faculty member on a “maker” style lab with 3d printers, laser cutters, and space to do Arduino like projects.

Global learning presents challenges, especially around software licensing. Some vendors have been reasonable, some impossible.

VDI presents problems for software where people expect really fast screen refreshes, so doesn’t get well used in creative and design environments. Where sound and image need to be synchronized can present problems.

Wisconsin has a restaurant that has a booth with a large screen that students can plug into while they eat.

People are hearing requests for sharing a large screen over wireless. AppleTV is too idiosyncratic for large-scale rollout, but generates higher expectations.

And with that… we are adjourned!

CSG Spring 2013 – MOOC Workshop, continued – Future challenges

UCSD – Emily Deere

In trying to create a central online course for the UC campuses, they can’t pass records among the campuses to award credit for a course taken at another campus without a paper process. Registrar’s office, and departments of majors and minors are a challenge – will they accept credit for courses from another UC campus? They’re going to try to create a clearinghouse to get some interoperability among campuses.

There’s a bill in the state legislature that will mandate that if a student can’t get into a course in their major that the campus will have to accept credit for an online course that is somehow equivalent.

Bruce Vincent – Stanford

How much do we want to project infrastructure into MOOCs? We’re trying to respond as support organizations – our response will be appropriate or not. We need to quickly learn lessons and accept a range of things like weak binding in identity space to assurance of identity for credit. We need to be able to support the world of the cloud platforms that these things run on. These all make our existing infrastructure look a lot less shiny. The parts of the organization dedicated to supporting LMS may be struggling for relevance.

CSG Spring 2013 – MOOC Workshop – Panel continued

Tracey Futhey, Duke

Duke Digital Initiative, goes back to the issuing of iPods to freshmen. Been trying things to get faculty involved in experiments. MOOCs are inspiring interest in innovation. Also driven by Duke’s commitment of service to society. Approach has a couple of components (try multiple things): got involved with Coursera, have now done 11 courses pushing close to a million registrations. Took significant effort. Looking at for-credit courses internally (the 2U effort). Having an issue with some faculty who do not want to do online for-credit courses, which they’re working through.

Materials reuse and access not just in the class, but components across multiple classes.

Sooner is better – do courses quickly rather than high production values. Two of the eleven courses have been offered a second time to date.

Most popular of the eleven courses is a philosophy course (185k people). 2/3 outside US. Professor describes situation with two people who are really active – turn out to be 9 and 11 year old brother and sister in Pakistan.

Just finished an astronomy course with a physicist. Eight week long course, with students doing between 10-15 hours of work per week. He found that in the course, which was offered simultaneous with an on-campus version, the on-campus students only got through six of the eight weeks that the MOOC students did.

Asbed Bedrossian, USC

President’s vision is that the focus for online programs is on post-undergraduate and lifelong learning. Online degrees will use normal USC admission process and charges, with the same expectations for rigor on the part of students. USC will not offer online degree at the undergraduate level. In the next few years they expect all 18 USC schools to offer online post-graduate programs.

John Krogman –UW Madison

Offering four MOOC courses this coming fall. “It’s the instructor, stupid.” Partly marketing, partly research, part of the bigger picture of grappling with disruptive change. Not qualitatively different than what we’ve done for years with innovation – lab sections, field trips, internships, simulations, web sites, etc. Need to give students soft skills as well as subject matter expertise. What measures will determine success of MOOCs? Which brings up the issue of how we measure learning outcomes at all.

What experiences will generate loyal alumni? Will MOOCs do that or are they just PR for the institution?

UCSF will be offering continuing medical education online through MOOC platforms. Also doing courses to advance world-wide health. Three courses: Nutrition, contraception, and diagnoses. Saw students as young as 13 taking the nutrition course.

CSG Spring 2013 – MOOC Workshop – Panel

Confessioans of a MOOC Dilettante – Alan Crosswell, Columbia

Has traken a few MOOCs, including AI-Class.com from Stanford, MITx 6.002x Circuits from MIT, NLP from Columbia, Big data class from U Washington.

Attributes – not classroom lecture capture. Not 48 minutes of straight video. It’s time-shifted yet time-bound (not a set lecture time but assignments due each week); Community participations (replaces in-class experience); Massive and open.

Only some courses feature talking head video – does that add value?

Progress indications in some MOOC platforms are very useful.

Downloads of marked up lecture slides are good.

Not all the features (like discussion forum) are good in all platforms.

Recitations, worked problem sets and supplementary tutorials in MIT’s circuit class are good.

Immediate exam feedback (per-question vs. entire quiz) is good.

Instructors and TAs don’t sleep during the course because they have to constantly monitor the content and performance of the course. How many of the MOOC courses have run a second section?

Georgetown U and the Initiative for Technology-Enhanced Learning (ITEL) – Kelly Doney

What is the ITEL initiative – designed to address the broad spectrum of technology enhanced learning. Allows them to very quickly redesign elements and look at effects, to experiment with different approaches. Made an $8 million investment over 3 years. Money goes to fund instructional designers, faculty grants, and infrastructure.

Why do ITEL? Responding to disruptors in higher education. Initiative led by Provost, who previously was Obama’s head of the Census Bureau. Extremely data-driven. “Believe that the undergraduate experience cannot be replaced by MOOCs, but we need the data to prove it.”

MOOC platform selection process – Analyzed options and developed a funding strategy (including faculty grants). MOOC Partner decision matrix – felt EdX shared the concern for the Georgetown brand; platform allowed reuse of materials outside the MOOC; lots of data and analytics; interested in being able to shape direction. Selected edX.

Released initial call for conceptual proposals in February2013 – received 55 proposals, then received 43 full proposals, and gave 5 Level 3 grants on May 1. Demonstration grants – 1-2 semesters to transform existing materials. Level 2 grants – Level 3 – Design and implementation grants – take 2-4 semesters, design new or significantly changed experiences. Transformation grants – up to three years to completely bring a course online or develop a MOOC. Adding 2 MOOCs for Fall 2013 – Bioethics, Glboalization: Winners and Losers.

Trying to drive online learning architecture – IT has a seat at the table overall.

Current architecture: “pastel spaghetti” – trying to simplify, with the LMS at the center. Working directly with business units, and driving common requirements, forcing vendors to do business to work within their technology stack. Business school wanted a platform with richer UI than Blackboard could provide, but they (IT) are working with Blackboard to enrich the UI, even as they launch the program with another product.

Architecture – develop architectural strategy driven by smart IT. ITEL, MOOCs, and distance Learning – leadership from President and Provost essential; Transparent processes and involvement of key non-faculty stakeholders are important.

MOOC Landscape at Penn State – Cole Camplese

96k students, 23k World Campus enrollments, 24k full and part-time faculty.

Decided to go with Coursera.

Why participate? PSU has established revenue from World Campus, most of which is returned to the programs that give the courses.

World Campus – more than 90 existing programs with 50k annual tuition paying enrollments. Goal is to get to 45k full time students. Investing $20 million to do that.

40% of residential students take at least one web course per semester.

Started MOOC discussions in 11/12 to first PSU MOOC delivery in 5/13 – extremely fast motion for campus.

Invested in a Center for Online Innovation in Learning – awarding research innovation grants – like one to look at whether talking head vieeo in MOOCs is useful.

Built a fovernance model – Subcommittee of PSU Online Steering Committee (administrative); MOOC strategy group – trends, design, assessment; MOOC reporting group to report to the provost.

Multi-tiered strategy – PSUE Coursera courses for total external MOOC audiences; MOOCs coupled with existing PSU world campus course; Large scale internal courses (e.g. English 15 which enrolls 15k students annually); PSU MOOCs designed under Penn State brand.

First five courses: Intro to Art (~47k enrolled), Maps and the Geospatial Revolution (~15k); Creativity, Innovation and Change, Energy the environment and our future, Epdiemics, the Dynamics of Infectious Diseases. Two are from faculty who have never taught online, which presents challenge.

Sociology 119 – making a television program that will be distributed by PBS, along with an app, and a MOOC. The program is called “You Can’t Say That”. National TV and web audience combined with a mobile experience. Mobile app posts to twitter, which then get interested in the MOOC, and curated into the course.

All courses except Art are taught by teams of faculty. For Art hiring undergrads to monitor discussion boards.

CSG Spring 2013 – MOOC Workshop – Survey and discussion

This morning’s workshop is a workshop on MOOCs.

Chuck Powell begins by asking where we are on MOOCs in the Gartner hype cycle – approaching the peak of hype, or heading into the trough of disillusion.

The CSG MOOC survey shows 60% of respondents have already begun to offer MOOCs, most (50%) using external providers. 4% have explicitly decided not to offer MOOCs now.

Number of courses range from 1 to 20, with an average of just over 7, growing slightly in the next year.

Why are they offering MOOCs – 30% desire to support global education, 18% for marketing and outreach, 25% are driven by faculty desire, 14% for competitiveness. It’s noted in a comment that lots of this is driven by schools not wanting to be the last in the game, driven by fear, uncertainty, and doubt. Maybe it’s better for us (universities) to be driving the bus rather than have it done to us by outside forces (like happened in scholarly publishing). One institution notes that it’s the best teachers at the institution who want to participate. Another says that desire for growth at global campuses is part of what’s driving them to participate. One flagship public institution says that they feel a mission to provide resources to the other institutions in the state.

The calamity factor – provosts want perception that they have it thought through before they are advised to (see the Virginia incident).

To what extent is this necessary to recruit and/or retain faculty – one institution says that this is definitely seen as a factor by one new center.

MOOCs don’t usually pay faculty, but can result in an uptick in textbook sales.

One idea is institutions using MOOCs for finding new faculty to recruit.

45% of respondents are discussing the possibility of granting credit. Georgia Tech online CS program for $7k. The big question (for the institutions in this room) isn’t granting credit, but accepting it. Will we grant credit if our own students have taken our own MOOC? Some states are talking about how to take the first year out of the college cost structure.

Some of this is a question of what the value added is when we grant a degree. Is it just a certification of an assemblage of experiences, wherever they come from? Imagine if undergraduate courses are taught more like graduate seminars.

33% are considering monetizing your offerings. Could get revenue from reaching new audiences, either outside the country or in new vertical markets. Total costs for offering a course on a MOOC can be upwards of $100k (not everyone agrees that they’re spending that, citing more like $20k out of pocket). How is that sustainable? Consider Khan academy – not so high fidelity yet very effective. It can be a long tail model.

It’s early days. Like we saw with mobile and social networks, these will get monetized and it’s important to be in the game at the beginning.

MOOCs can be an important marketing tool for smaller schools.

There’s a good opportunity if we can see that we shouldn’t compete on whole courses, if we can dis-aggregate content and reuse it in pieces in our institutions. There are discussions of inter-institutional arrangements (CIC CIOs are talking, as are the state systems). Can we build a repository of learning materials – we’ve done that before, but haven’t gotten used. Some of those materials already exist out on YouTube and get used in our institutions – hard to beat free, but there could be some kickass materials with a per-use fee.

Are students really learning in these experiences?

Are MOOCs going to replace brick and mortar? Probably not for top tier residential institutions. We have a long tradition of credentialing our faculty as experts – how is quality determined online? MOOCs don’t replace the connections with alumni, careers, etc. that people get at online courses.

There’s an opportunity for IT groups to be involved in helping people run infrastructure.

One school – we don’t know where this is going to go, so let a thousand flowers bloom. Wide open for experimentation. They want to gather data, so they can then evaluate what works and what doesn’t. People are excited about being able to get learning metrics which they couldn’t get before.

One of the key returns may be the effect of MOOCs in the residential instruction model. This will change how our campuses operate in the next five years – what does this do to the model of semesters and years when we’re freed from the physical constraints? One person thinks that it won’t have an effect for twenty years, while another thinks that it’s already starting to change with the effect of flipped classrooms.

Faculty are inviting staff (whether IT or others) in to help create the courses for MOOCs.

How do we archive the materials that come out of MOOCs? What’s the role of libraries?

One campus is rejecting online education for undergraduates, but embracing it for graduate education.

MOOCs allow students to audition subjects and faculty.

Why are we still building large lecture halls? Part of the investment in MOOCs may be recouped in not having to build those. Buildings are 30 year decisions, while MOOCs are 1-3 year decisions. How many of us are concerned about the changes in physical models due to this?

This is similar to discussions we had five years ago about Second Life – what’s different now? Are MOOCs a distraction to concentrating on the changes in pedagogy? The real impact of MOOCs could be competency based education – we should pay attention to what’s happening in Europe with testing of skills.

Physical changes could be in building labs and meting spaces instead of lecture halls.

Success may be based on agility – look at Amazon.

Some institutions are looking at online degree programs to replace diminished funding from states.

MOOCs are driving a new conversation about teaching and technology in ways we’ve been trying to get people thinking about for twenty years. Faculty who teach MOOCs learn new things about their teaching.

Faculty who are teaching MOOCs are taking parts of them back into their on campus courses.

Ken K – Conservation of APIs. Interfederation is

Ken K – Conservation of APIs. Interfederation is driving even further multiplication of APIs. What are the pain points? Marshaling parameters – what do you call it? What are the attributes being passed? Normalization of release of information. Building interfaces that will work. Reconcile that over 90% of users of Google don’t know what information is being released. APIs are going to invoke privacy management, so we’re going to need conservation of paradigms around that. It’s going to mean multiprotocol. What happens if I want to interact with my colleagues in a cloud app? Re-emphasizes bilateral federation vs. multilateral federation. Feds will announce federal portal soon. MLS services have done Shib, as is health care. APIs haven’t been worrying about strength of authentication. In the future you’ll be able to buy attribute verification services – e.g. “user has put in this street address, do they really live there?”. Presumably the more you spend the better the strength of the verification. You’ll be able to buy from the Postal Service, Google, credit card services, etc. through APIs. Reinforces the privacy aspect – will need to make this tractable. There are a bunch of vendors that want values of “studentness” – we will be in a position to sell that to them. We pass it among each other with no charge. What does this do to our social fabric? Steven Carmody – Brown University – Models for Managing Privacy and Attribute Release Fair Information Practice Principles – passed by congress in 1974 – Transparency, individual participation, purpose speification, data minimization, use limitation, data quality and integrity, security, accountability and auditing So why are so many places asking for so much personal data to be released? Research from Penn State – when faced with a note that says “This program will erase your hard disk, click yes to continue”, 90% of people click it. One model that might help is if the attribute release process is not part of going to actually use a service. Abc4trust/Idemix – You download identity cards from your identity providers. Will produce trusted assertion without revealing details. You choose what source you will release the information from. Process occurs in your desktop, so the iDP doesn’t know where the credential is being used. Privacy manager properties: Accurate; Design consistency; describe who will use the data, and what they will do with it; understandable (balance, summary vs. detail, information model), encourage reasonable answers (position in flow, timing). Leveraging Social Identities Allowing access to the other 90%. An InCommon pilot to support access by people with either Federated or Social identities. Provide application owners with a single auth framework for both types of identiies. Provide info to the application about the user with a singel interface regardless of identity type. application owner can choose which social identity providers to permit. Why? We already work with people outside our communities – applicants, parents, continuing ed/MOOCs. Other areas showing interest in working with people outside the traditional communities. All of those people have identities at one of the social/personal providers. This may be preferable to issuing campus identities to those people. However, there is NO guarantee about who is using a social account. How – Web based authentication gateway, translates authentication responses from popular “social” ID services into regular SAML 2 Assertions (consumable by Shibboleth). Allows downstream applications which only understand SAML to easily utilize social IDs. Maps attributes. Works great for guest authentication. Typical use is “pick and choose” among the external services. Very powerful combined with an invitation service like that in Grouper. Consent screen at Social Providers akss users to release attributes to the gateway. Pilot gateway available since Fall 2012, but will end. 2nd Pilot underway, run by Cirrus Identity. Can be used to access I2 Spaces Wiki and InCommon Federation Manager App, currently only supports Google. Next phase – looking to expand use and use cases, require definition, testing during summer 2013, campus participants being identified, hope to have services available to InCommon members for Fall 2013. USC’s OAuth Recipe: oAuth + Enriched Identity Data + Central Authorization – Brendan Belina Benefits of Using OAuth (Social Providers) – extend USC services to greater populations; password related issues addressed by OAuth providers; reduces barriers to adoption. Chalenges: Different versions of OAuth with different capabilities; Inconsistent and unpredictable attribute release; Attributes required for applications may be missing; Identity is self-asserted – potential risk to applications; user may use multiple OAuth providers, leading to login confusion and multiple identifiers; OAuth providers come and go, leading to potential ;loss of identifier persistence; how to revoke an OAuth Login?; Authentication without Authorization What is needed: Allow multiple OAuth providers per identity and the provider should be transparent to the service – addresses problem of user using multiple OAuth providers, addresses problem of deprecated OAuth providers; Deliver a standard attribute set regardless of OAuth provider or version for compatibility with applications; Provide consistent user attribute values to services; Externalize authorization to apps to reduce risk and allow revocation; support for both Just-In-Time provisioning and ETL provisioning.

Benefits of self-registration: registry provides single place for meantenance of user attributes; opportunity to enrich data released by OAuth providers to meet requirements and provide consistency; allow creation of persistent identifiers for use across institutional services; Opportunity to provide linking to multiple OAuth providers to address continuity; ability for user to unlink an OAuth provider or credential; Registry entries can be used for ETL provisioning and authorization to services.

Scotty says you should use OpenID Connect for social authentication instead of OAuth.

CSG Spring 2013 – OAuth & Web Services

University as Platform – Jim Phelps, U Wisconsin Madison

In conversation with Undergraduate Advising, found that they have to use an average of 7 tools for every task they want to accomplish when talking with a student. They spend 40% of time navigating systems to perform a task, 40% teaching students how to use the systems, and 20% of their time doing high value advising.

Platform principles: resusable, transparent, leveraged, consistent, fiscally efficient, drives agility and innovation.

Second use case – the curricular hub. Spent a lot of up front time building it, but then requires very little reinvestment afterwards. Now have 27 different consumers, all getting the same answers.

Web Services, Authorization and the API Economy – Scotty Logan, Stanford

Data extracted from programmableweb.com – which doesn’t have an API.

About 9k APIs registered – straightline log chart since 2005. Mostly REST, SOAP coming back a bit. API Billionaires Club: twitter, google, facebook, netflix, etc.

90% of Expedia’s business is through APIs, half of Salesforce’s access is through the APIs. 1.1 million requests/second on S3.

Drivers: Mobile, cloud, social, desire to be open, competitor has an API

We have a dysfunctional relationship with the cloud – we’re consumers of APIs, and sometimes we produce APIs, but not consistently or on a huge scale. Who allows the cloud to push and pull data? Who allows mobile apps to push and pull? The mobiel app economy relies on the API economy. How well do your apps work without a network? The Internet of Things relies on the API economy.

API Value Chain – App user->App Store->App->App developer-> world of APIs-> API-> API Team-> Internal systems. App developers are kingmakers – so they’re the ones you have to get APIs to.

Web Services – SOA is dead – long live services (Anne Thomas Manes, Gartner).

SOAP or REST? REST is more popular now, means more tools and libraries, examples, design/dev experience. SOAP uses HTTP, REST is HTTP.

Authorization – how does a person allow one appt to access their data in a different app? Shibboleth (SAML) is great if there’s a browser involved. How do you authenticate if there’s no app? OAuth + REST – Designed for REST-like APIs. Like REST, OAuth leverages HTTP features and response codes.

Allows a client application to ask a user for restricted permissions to act on their behalf / access their data in another system without exposing credentials.

OAuth authenticates apps, not people. OpenID Connect authenticates people and is built on top of OAuth 2.

OAuth 2.0 vs 1.0/1.0a

OAuth 1 required signatures – harder than it looked. OAuth 2.0 relies on SSL/TLS – additional RFCs for signatures, JOSE specs for signed/encrypted tokens.

API Evangelism and support at Stanford – Goal is make it easier to create and deploy APIs and to find and user APIs “the right way”.

In the beginning – helped a few groups create basic APIs. Scott & Bruce as API evangelists – please make APIs, please make them RESTful, please use OAuth.

Med School CAP System – Community Academic Profiles – expanding to all faculty and staff. Other apps want data from CAP – “we’d like help with an API!” – two day REST design session with vendor. Investigated their API management tool – concerns about integration, cost. They wanted SAML – AS, token attribute passing, scope approved, rate limiting, API console, distributed ownership/control, workgroup integration (groups).

Found a few open source API management tools – big bloated unfamiliar framework or obscure implementation language or no OAuth 2.0 support.

They’re rolling their own – three pieces: OAuth Authoriztion Server; API Proxy/Gateway;

Using CloudFoundry’s UAA, CLient Credentials flow available now.

Developer portal – probably Drupal, wil integrate with OAuth AS.

Federated OAuth is possible.

Web Services Gone Wild! Mojgan Amini, UCSD

Users were happy with their green screens – then came web applications, first with Perl then with Java. Had users using web apps and others using green screens. They centralized common services, giving birth to their middleware. Every new technology got wrapped into the middleware, which got very bloated. Attended 2006 Gartner Web Services Summit – SOA is the wave of the future. Churned out 2000+ web services! Lesson learned: a little more effort on governance up front would be a good thing. Then created a good set of web services that were reusable that were heavily used – starting with commonly used public services. Then started talking with admin users who wanted real time data over web services – provided a select set of authorized services for their purposes. Put them in a good place for launching mobile apps when they were needed.

Still have Cobol, Perl, and Java apps along with 2000 base web services. Moving to a Red Hat middleware solution. Hoping to address self-service campus web services.

Student Developers, Web Service APIs, and OAuth – Mark McCahill, Duke

Two cautionary tales about what can happen with student developers, web APIs and how OAuth can maybe help.

Duke launched Innovation CoLab – effectively a long format “hackathon” – student dev teams compete to identify and address campus needs, in hopes that useful apps will emerge along with more insight how to support the institution. Has pushed a bunch of changes within OIT because students want to move fast.

Planning for CoLab – how can we enable student dev innovaation – infrastructure (easy): on-demand VMs with pre-installed application stacks for student servers (60 different images using BitNomi); self-service Git repository + bug tracking. Enterprise data in easy-to-use form for mobile apps and alternate user interfaces (harder). What data to they want? In which formats? SOAP+WDLs? XML-RPC? REST + JSON? The last won – students didn’t touch the others.

Impedance mismatch between what students want and what IT staff are used to doing. Enterprise developers are used to servers and app stacks hand crafted by artisan system administrators. Enterprise devs use Java frameworks etc. Student developers want on-demand PPAS services such as Heroku and software ecosystems such as Node.js, Ruby-on-Rails, JSON, REST, curl, screen scraping.

CoLab; Note Cheese – Sharing class notes is a good thing; you don’t know everyone in your class and you don’t know who took good notes. Might end up with something liek Quora meets Piazza with class notes. Why make users type in their class schedule? Devs can’t get class schedules via API, so they asked for NetID and PW, then screen scraped the classes.

Lesson: OAuth opt-in and informed consent to access individual class enrollment? Allow students at least some access to Shibb SP signup tool – policy now is that students can’t get SPs, so need to change that. Provide pre-built shib along with API stacks.

CoLab: Hack Duke – Enterprise via REST – read-only REST access to everything. Live drill-down navigation of the APIs: classes, terms, course evaluations, departments. Event calendar, map coordinates, LDAP directory via REST. Available on GitHub. Backend is MongoDB, nodeJS front end. a fork of openworld. Some concerns are expressed – controlled access to sensitive data? All students can view course evaluations, but should the public see this? Frequency of updates? Data validity? departments offering courses <> all departments.

Lessons: if you don’t provide enterprise data in easy-to-use forms, the students will do it anyway. read-only REST + JSON is the way to roll. Realtime browsable catalog of APIs (requiring an API key). OAuth for non-toxic user opt-in for release of data to applications, Expectation of live data for realtime course registration decision making.

http://www.hackduke.com

CSG Spring 2013 – Data Warehousing and Data Governance

We’re at Brown University in Providence, Rhode Island. The first workshop is about data warehousing and governance.

Business Intelligence survey –

What are the most important business drivers for BI? Top 3: Have data consistent & reliable across organization (this is the therapy we provide as IT professionals, to get the business offices to agree on data); make better data-informed decisions – people expect data at their fingertips, and it’s not; easier access to data – access is easy, but trusting that people will make decisions based on correct interpretations of data is not uniform. What is the role of IT in making people come together on data?

What are the biggest barriers to using BI? Cooperation and coordination across depts – business offices don’t have the time or intellectual investment to help figure the issues out; No common source of data – data pops up everywhere, how do we corral it to have a system of record?; no consistency in definitions or terms; Customers knowing what they want in a BI solution – we know this might not be an IT problem, but the rest of campus doesn’t always.

What business Intelligence related initiatives are you interested in? Self-service – you give people access to tools and the take it and put it in Excel.; Leaerning analytics and student outcomes (predictive analyses); Big data

BI – Just Do It – Suneetha Vaitheswaran, U of Chicago

There is a roadmap for doing BI the right way, as expressed by professional organizations. The state of universities don’t always support that approach – economic downturn, continued aggressive growth, faculty-driven, bottoms-up institution with lack of standardization around data, but executives are demanding data for decision making. Hundreds of legacy and siloed source systems, erratic data stewardship.

Took over management of systems and consultants to coordinate reporting approaches into an enterprise wide solution. Began to revamp the training program, recognizing that not everyone should be a user of the ad hoc reporting tools. Began using consistent methods and tools. Started a Data Stewardship Council. Had a new registrar that championed the Student data warehouse project, and got reporting considered as part of the new research admin system.

The emerging vision is to login to one place for reporting with a similar look/feel across subject areas, to support different user skill levels with training as a means to an end. High performance and availability and integration points are important. Had to commit to multiple projects at a team, focused on standards and process including coordinating with the PMO, and brought in SME’s earlier. Had to acknowledge surprises and new priorities as they come up – need to work them into the strategy as they come up. They do a fair amount of resource loading to know where they have capacity.

Marketing opportunities come up in new ways, so be ready. Planning happens all the time.

MIT has managed data services where they maintain servers and databases for local units and integrate tables with local data.

Managed self-service BI – Jerry Singer, UCSD

Definition – Managed facilities that empower BI users to become more self-reliant and less dependent on central IT

Primary objectives: Agility to respond to changing requirements; quick to market; lower costs; clearer definition of roles with emphasis on individual strengths; improve access to source data; facilitate data governance and collaboration; centrally control tool set and enterprise data; reduce toolset learning curve; provide function righ BI tools with current state capabilities.

Key success factors: make BI tools easy to use; make BI results easy to consume and enhance; make DW solutions fast to deploy and easy to manage; make it easy to access source data.

Roles – BI/DW Builder and info producer (Access); BI/DW builder; Information producer; information consumer and collaborator

Organizational considerations: governance; IT role (monitoring, oversight); Departments (access and security definition, requirements and design).

Recommendations: Just installing a tool does not get you there. Support collaborative BI, starting with friendly collaborators with current needs. Establish governance, reuse deliverables, release toolset functionality incrementally by creating a starter set of reports, analyses and widgets. Provide training and sandbox environments, enable data federation to include DW source.

Framework for starting or restarting a BI program – Todd Hill, Notre Dame

It was taking far too long to deliver business value to customers, and BI projects are incredibly complicated. Came up with a metaphor: BI is like a fine restaurant: variety of options; well-prepared (easy to use); consistent; timely; great service.

Looked more like: long time to deliver (15-18 moths), customer frustration, poor morale, internal conflict, role confusion.

Data prepartion is like the kitchen. Presentation is like the dining room experience. No matter how good your kitchen is, that’s not what the customers come for.

Step 1: Get serious about BI – create a sense of urgency. Build a guiding coalition with a strong sponsor – it’s a business problem not a technology one.

Who’s driving the bus? Make sure you know. Is it central IT, or the business?

Step 2 – Drive accountability

Step 3 – Remember that BI is about much more than technology: Data governance, functional requirements, technology.

Data input integrity is important – push changes back into the source systems, don’t change in the DW.

80-90% of customers use pivot tables in Excel to analyze data – so why not support it?

Step 4: Begin with the end in mind. Mapped requirements by stakeholders. Shows commonality of requirements across organizations. EVP required prioritization of functions.

Step 5 – … but start small – pick bite size chunks that deliver value, don’t spend all your time in the kitchen. Did a 4 month project in HR, which resulted in the customers demoing to the EVP by themselves.

Step 6 – Embrace change – you’re not going to get it right at first. Embraced agile method – typical project is 4 months. Business value delivered in one month increments. Think about the spectrum of BI from personal, through team, to enterprise. Hard to start with etnerprise. Personal BI is enabled by in memory tools that encourage iteration, then you want to share. Jump to team BI – get more feedback, iterate some more till it’s good for your department. They build in a burn-in period of 4 months. Then elevate to the enterprise, which is where you can invest in infrastructure, fine grained access controls, etc.

This is not a prototype approach, but each step involves delivery of business value. Finding issues in the enterprise level requires about 8x the time to fix than the personal space, so it’s important to find them early.

Guiding principles: keep it simple (don’t over-engineer a solution); be agile (flex on scope, not timeline); speak common language; Work smarter; celebrate successes (and failures) – they demo after the 2nd and 4th sprints to all customers.

Data security

One case (from Suneetha) – legacy systems s DW must build custom security; each model driven by steward; department pressure for additional granularity; substantial administration and workflow; data usage requests to document feeds; data stewardship council surfaces issues. Model deals with culture and systems at each insittution.

Data Governance Approaches – How do you secure and manage access to data? Anja Canfield-Budde, University of Washington.

Data Management Committee, appointed by Provost. Created a Data Map, aligning business and data perspectives. Seven main subject areas further classified into data domains, each of which has a steward identified. Developed a framework for managing access to data. A finite set of fourteen roles has been identified across all areas. Access granted by role across subject areas. Data Access Control, controls data access across systems. Data stewards classify users into the fourteen roles, and then apply security classifications to the data. DAC consumes data from DAC. Security views dynamically show only data user is allowed to see.

The implementation of data security has resulted in a rapid growth for BI. This was not anticipated. By not assuming that all data was sensitive and being able to trust classification of data people were more comfortable. Have a portal that not only provides access to data, but also metadata about security.

Data governance: http://www.washington.edu/uwit/im/dmc/

UW Data Warehouse and BI: http://decisionsupport.washington.edu

Effective Practices for Data Governance – Mike Chapple, Notre Dame

Difficulty in defining terms like “student”. Overarching goal is to provide access to data. Model of pillars, with five principles; Quality & consistency; policies & standards; security & privacy; compliance; retention & archiving.

Each project preceded by a data governance review that defines terms in those five principles.

Governance model – Exec sponsors (EVP & CIO), Campus data steward, unit data stewards, coordinating committees (info governance committee, data-driven decision making steering committee).

Building a data dictionary – definition, source, what’s the query, revision history, etc. Definition has to be in plain english understandable by average admins. Using Google docs because it’s easy to do collaborative editing, but it’s a bad repository – plan to move to Sharepoint for the repository. Defining each term requires 1.5 hour meeting with 6-8 people plus an hour prep and an hour cleanup. Have currently defined around 100 terms.

Give the group a starting point – don’t start with a blank sheet. Develop simple guiding principles (e.g. put jargon in its place).

Business Intelligence Functional Approach – Chris Frederick, Notre Dam

Customers were unsatisfied, and deliveries were late and incomplete.

Key issues: Insufficient collaboration, low accountability, user doesn’t know all the requirements up front, bad data, no shared data definitions, overly complex and hard to maintain (we were enamored of the technology), tools are too difficult.

Waterfall doesn’t work – takes too long and you don’t get the tight feedback loops you need. Be agile. Got visual with their scrum boards. Priorities change.

Breakthrough – In-memory BI tools. Column-oriented databases instead of row-oriented. Can mash up data from disparate soources, can easily fit 10s of millions of rows into memory, reduces need to optimize databases, quickly create rich interactive dashboards. Security needs to be first in concern, because data sits on your laptop. Usually requires 64-bit architecture. Watch out for “spread marts”. Hot tools: microsoft powerpivot for Excel, Tableau, QlikView.

Work with partners who are “shovel ready”. Motivation, availability, knowledge of business problem. Co-locate – get out of the server room. Exec product owner came to the daily standups.

Schedule live BI demos. Learn in the least costly manner.