CSG Fall 2012 – Projecting Infrastructure into the Cloud

Tom Barton from Chicago and Michael Gettes from Carnegie Mellon are leading a discussion on Projecting Infrastructure into the Cloud.

Identity Federation & Attribute Release – Federated Access anyone? Release directory info! In InCommon identity providers get into the federation, but not always service providers. – get your SP into the Federation. At CMU they release directory information – For everyone- eduPersonPrincipalName (which for them is an email address), and eduPersonScopedAffiliation. For non-students: givenName, surname, commonName, email. Allows for very quick integration of cloud providers. Will this work for others? Ken notes that projects such as Vivo have lots of data with no access control.

Contracts – we spend lots of time on compliance and security, but not on functionality and defining the relationship. CMU and PSU are requiring their vendors to join InCommon. One comment is that vendors are increasingly resistant to joining InCommon.

There’s a bunch of discussion about things that are beyond identity – how do we deprovision users, how do we communicate limitations, where things are easier or harder in the cloud. Kitty notes that in some contract negotiations with cloud vendors they are requiring targets about load and latency testing from different points in the world.


CSG Fall 2012 – Future of the IT Organization pt. 2

Now we’re having a series of point-counterpoint arguments about some issues around the future of IT organizations in higher ed.

The first is about the pace of change. Bernie Gulachek from Minnesota is arguing the point that change in higher education comes slowly and will continue to do so. Applications are up and enrollment is being managed, and until that is threatened our institutions have little impetus for change. Our institutions are not architected for change and we have a hard time affecting outcomes. We’ve had lots of opportunities for change in the last 10-15 years, but we’ve made conscious decisions to not bring about change. We’re now talking about commodity solutions in the cloud because we weren’t able to come together and create our own solutions collaboratively over the last ten years. Change will come incrementally.

Ted Dodds from Cornell  takes the position that we’re in denial – people believe we can duck and cover and the old ways will serve us into the future. The notion that somehow we can protect and insulate our community from the change doesn’t make sense. What he’s seeing for the first time ever is that academic leadership is provoked and engaged by MOOCs. These will undeniable change and affect the business models of our institutions. If we as IT leaders aren’t out in front of and participating in this change we won’t survive.

Charlie Leonhart from Georgetown and Bill Clebsch from Stanford are arguing about efficiency in IT organizations. Charlie starts by stating that the era of big IT is over. When he walks around campus he hears complaints about “you’re too big, you’re too fat, you’re too slow, you have too many people, you’re in my way – you guys suck.” We have basic facts – decimated budgets, needs to cut spending. We require bold leadership. Five point plan – cut spending, do more with less, cut staff positions, let’s virtualize. Get rid of desktop devices. Encourage innovation that drive down costs. Build strategic partnerships. Reduce central services and let users vote with dollars.

Bill starts saying there are 3 tsunamis: Research computation and big data (not building wet labs anymore, but building to support cyberscience); online learning; cloud and mobility. All this takes more money. The dollar savings aren’t there – but advantages in scaling, speed, and response to user. Mobility is a cost, not a cost saver. This is becoming a larger part of our lives – we can’t stop spending more. This is a part of the critical mission of the University – can’t stop spending now.

Elazar thinks we can do both – achieve efficiencies in our organization and use the savings to invest.

Elazar Harel from UCSF and Ron Kraermer are arguing about Bring Your Own Device. Elazar starts by saying that he believes that everybody should be able to bring whatever devices they want and they should work well in our environments. It also means BYOA (apps), BYOC (connectors), BYOP (printers), etc. When you think about this, there are some solutions – we need to be in the cloud with our infrastructure and applications.

Ron asks what business are we in? We’re in the business of delivering an optimal educational experience. We can compromise that technology by trying to support everything. What he plans to do at Notre Dame is lower the cost of textbooks by delivering them on standard devices (iPads), and they’ll train faculty on those devices to develop and deliver content. iPads will dominate the education market, and they can use it to improve the experience.

There’s a general discussion of the emerging roles of vendor relationship managers, service and product managers and technical integrators. The crowd isn’t clear yet as to how much or how quickly the old roles will go away.

CSG Fall 2012 – Future of the IT Organization

Ron Kraemer from Notre Dame and Steve Fleagle from Iowa are leading a workshop on the future of the IT organization – what’s changing in IT and how will it affect us?

Steve starts by talking about drivers of change in higher ed. We live at the intersection between higher ed and IT – there are things that happen on each side that will drive our organizations to change.

IT drivers of change include cloud computing, the consumerization of technology, personal cloud (more than just storage, but the glue that links devices, information, people, and services), identity management, migration from the PC as the most common method of accessing IT, widespread availability and value of large data sets,  high demand for skilled IT staff, business-driven IT, the success of disruptive technology (encouraging innovation), the rate of change will continue to increase. Access to click-stream and other data logs and how and whether to share them will also be interesting. Social network tools are increasingly important.

Higher ed challenges include the growing scrutiny and criticism of higher ed, financial pressures, growing enrollments, increased competition for research funding, likely market disruptions, and other challenges including a culture averse to change. Startups don’t have the same constraints, and are starting to receive funding from venture capitalists and some states. Students are flocking to these alternatives and some faculty are starting their own companies in this space. Some universities are responding (e.g. Coursera and edX), but alternatives are gaining traction – StraighterLine, Western Governors University, Excelsior College, Udacity. What happens when universities start accepting transfer credits from StraighterLine or MOOCs, or employers start taking those credits in place of degrees? There are emerging corporate universities that are training employees. Greg notes that higher education isn’t just one sector, it’s multiple sectors – community colleges are different from universities.

Ron Kraemer notes that states are decreasing their funding for higher education, but want their influence to increase. The difference in private institutions is that funds come from the benefactors that support us – they want influence too, but they bring resources along with it.

What Ron thinks about: Am I burning up my staff? Are we as efficient as we can be? Can we maintain quality as we are spread more thinly? Are we taking enough time for professional development? Does the current organization structure meet our needs? – for how long? How can we create a more comprehensive view outside IT? Why do most outside IT think this is easy?

Several folks note that our needed skill sets are changing and we need more skills in procurement, contracts, and business analysis.

One comment is about how computer science departments aren’t training people for our industry – where are the new people going to come from?

Mairead Martin is leading off a series or presentations on organizations. She asks to what extent do our organizational structures allow us to respond to change?  Traditionally stability and job security brought and kept people in our organizations – that’s no longer the case. Process management has become big – project management, ITSM, etc. It still takes us a long time to recruit and hire people. Investing in staff well-being and organizational resilience is important – how do we help our organizations through tough times? Diversity is an issue too. Kitty notes that our younger staff aren’t necessarily seeking security – they’re far more willing to move on if they’re not getting what they’re looking for. There’s some talk about career ladders and Ron notes that perhaps we shouldn’t have traditional ladders but flatter and flatter structures where the technology contributors can have influence on decision making processes. Shel notes that there’s more job stagnation in higher-ed IT than any other IT organization, which leads to people defending old technologies that are no longer the right ones to champion. Ten years with one skill set is no longer a viable skill set in IT.

Shel Waggener from Internet2 now is talking about cloud services and the impact they are having. At the scale of Google, Amazon, or Microsoft the scale changes how we approach our roles. “This is a tsunami… the potential upside is so big its hard for me to imagine any large research university that wouldn’t want to be involved” – Richard Demillo. Education is scaling similarly.

Our constituents don’t care about our organization. What it’s like today to get it: campus identifies a need, contacts IT, makes a case on why it’s needed, begin requirements process, realize project is bigger and more expense, cut scope, scrape more resources, finally see early prototype, start testing, fix bugs, deploy. Takes months or years to do that. What they really want from IT – the magic iPhone app! – Google idea to find app, install on device, start using new app – takes minutes. You can’t download an enterprise app, but what is an enterprise app now? An aggregation of many of those small apps.

We were too slow to adapt individually – we don’t have the time anymore. We have to find ways to work together to influence vendors. Our current organizations aren’t adequate to make this change. We have to find a way to involve our staff in the business changes that are happening in our institutions. In the past you could stop procurements of new waves of technologies – they are going to get steamrolled by the wave.

Major investment is flowing into cloud services for commercial offerings that can be adapted to higher ed use. 70% of higher ed IT spend is on people. The only leverage you get from Moore’s law is on the 30% left.

Shel outlines the Net+ process of aggregating demand for cloud services.

Sharif from Notre Dame is talking about the Cloud Compromise. He uses Box as an example – it doesn’t handle groups well. What kind of compromises are we willing to take to use cloud services? Another example – Notre Dame turned on Google Sites for everyone – started getting errors when people started uploading content into Sites. Found out from Google that there was an undocumented limit to uploads for an enterprise into Sites. Google raised the limit for Notre Dame, but is that good enough? A comment is that we should get used to things not being perfect. Options – Wait and watch and see if the problem recurs or intentionally fill up a site until it hits the limit to make the point or pay Google for better service. Bruce notes that different providers respond to different pressure points – perhaps we can find the way to collectively stuff the ballot box with specific vendors. There’s an observation that we can influence smaller vendors but not the larger ones like Google.

CSG Fall 2012 – Global hiring, staffing, and procurement

Steve Huth from Carnegie Mellon is talking about this topic.

Want to think about the nature of staff you’re sending over. They need to be adaptable. Think “Start Up” rather than a mature organization. Are your staff prepared for that? In one of the locations the guy building out the network and servers also helped put in the automatic gate openers. Some of the best staff had never travelled internationally – don’t want to miss out on a good prospect because they don’t meet your mental model.

Is your organization ready to do this? We deal with a lot of process and constraints built up over a lot of time – in these situations what you might need is a bag of cash in the souk – will drive procurement folks crazy.

Need people who are capable of interacting with the ambassador or the government – a lot of non-technical skills.

Dealing in a global environment offset by a fair amount of time. Even the work week might differ. Maintaining communications takes a lot of flexibility. Technology can help, but nothing takes the place of people going and having the remote people come back. Censorship can be a big deal in some of these countries. If we’re dealing with academic materials, they come in ok, but when you’re dealing with staff their materials may not. Share the good and bad parts.

People need more help than they would on a business trip to navigate in these new societies. Find local people who can help with things like getting health care set up, or if you’re in a car accident. Need to understand the culture when you’re doing business. Odd sorts of situations that won’t resolve in any meaningful way in any meaningful amount of time.

Things happen back home that global staff will have a hard time dealing with. Can be a high personal cost – if something needs to happen quickly you need to be prepared to get people on a plane so they can get home to deal with emergencies.

International assignments provide an unparalleled opportunity for growth and development. CMU worked on an international work assignment program – have people submit ideas for work at international locations. Gets people thinking about the campus globally.

CSG Fall 2012 – Global IT Services & Environments

Bob Johnson from Duke is introducing the Global Networking panel – we’re all faced with the same networking issues. Bandwidth availability, political restrictions, latency, jitter. Asia Pacific activities underway – Duke has a medical school in Singapore, building a million square foot campus in Shanghai, NYU has presence in Shanghai, Chicago has a presence in Singapore and Beijing. NYU is opening a presence in Sidney.

Working on building an International Network Exchange Point and Co-location site.  – provide common point for connection of regional R&E networks. Benefits are cost savings, building on R&E networks instead of leased lines to the US. Co-lo space is just cost sharing. Having a neutral location for data storage, hosting computer services (lower latency).

Where to put this? Reviewed three sites. The best ended up being TATA in Singapore.

Dale from Internet2 goes over the process and the support services required. There will be a telepresence and HD video exchange under the Internet2 Commons. Network Performance monitoring will be available.  Multiple functions for this facility: Co-location (initially capable of 10 racks, which can grow); Layer 3 capability; an instance of an Advance Layer 2 Services exchange – support OpenFlow, SDN, Dynamic Layer 2 circuits; Exchange will operate as a GLIF Open Lightpath Exchange; Essentially policy free – if you can get a circuit in and pay the fees, you’re welcome.

Some sites might bring in their own address space and router, others might use shared space. 1 Gb physical link to commodity Internet. Initially provision 200 meg on 1 gig circuit (with some burst capabilities). 1 Gb link to global switching building for peering – TEIN3, Gloriad (which ends up in Seattle). 1 gig link to Hong Kong light to meet CERNET and CSTnet.

Timelines: Sept 14 Tat agreements to be signed and in place and equipment ordered; Nov 1 – equipment delivered to Singapore; Dec 15- everything in place to beging testing; Jan 1 – fully operational.

Kitty – what’s worked and what hasn’t?

NYU has been testing, focusing on latency and user experience – acceptable, tolerable, or frustrating.

Common issues – network bandwidth, amount and does it match contracted bandwidth? Response times are highly variable, Some apps aren’t tuned for latency. Latencies range from around 80 ms to over 300 ms depending on sites. Focused on two forms of testing/monitoring – latency simulator and actual testing from different locations. Implemented a tool to understand user experience for web-based applications.

Implemented a long distance performance simulatore to create profiles. Implemented a tool call TrueSight that’s a Web App performance tool – allows clear understanding of what happens in a web app. An appliance connected to the F5 Span port – captures all the traffic and analyzes. Performance metrics of http and https web traffic. Able to track usage over time then drill down into specific sessions. Service leads get daily or weekly reports. Anonymized data being moved to data warehouse for trend analysis.

Remediation – optimizing webpages, applications; tuning network; WAN acceleration

It’s hard for app builders and owners to think of applications this way. Network folks haven’t really understood how applications perform on networks. Most app builders assume their users are on the LAN, not across the world.

Aspire to do testing before going live, setting watch points on end-user app tool to watch how performance is doing. Working with cloud vendors on how they test instances before selecting.

CSG Fall Meeting 2012 – Big Data – open discussion

Raj opens the discussion by asking about who’s responsible for big data on campus, and do we even know who makes that decision?

In talking about repositories, are there campuses that are trying hard to fill their repositories? A few raise their hand, but most don’t.

Moving to the cloud for storage – will there still be a need for local storage expertise? John says the complexity and heterogeneity is increasing and won’t go away any time soon. Bruce would like to see us positioned to provide expertise and advice on advanced data handling like Hadoop. Mark notes that we should be using those technologies to mine our own data sets – e.g. security mining application and network logs.

In a discussion of data management and the need to engage faculty on planning for data management Kitty notes that she’s had some success hiring PhDs who didn’t want to go into faculty positions but had a lot of experience with data. They could talk to faculty in ways that neither librarians nor IT people can. Curt notes that last year’s data management workshop noted the need for grad students to be trained in data management as part of learning research methods.

John asks whether people are planning central services for systems that store secure data that cannot be made public. Bill notes that Stanford is definitely building that as part of their repository, including ways to store data that cannot be released for a certain number of years. Carnegie Mellon is also planning to do this. Columbia has a pilot project in this area.

Raj notes that on the admin side, Arizona State is doing a lot of mining of their student data and providing recommendations for students on courses, etc.

Mark notes that we don’t have a good story to tell about groups in controlling access. Michael says we do have a fairly good story on group technology (thanks to Tom’s work on Grouper), but we still need to work across boundaries and to develop new standards, such as O-AUTH and SCIM.

Mark also postulates that the volume of data generated by the Massively Online Courses would be a really interesting place to think about big data.

There’s some general discussion about the discoverability of data in the future and the ability to understand and use data in the future when the creators might not be available. That’s where data curation becomes important.  Data curation should include standards for metadata and annotation, and also processes for migrating data forward as formats change. Ken quotes a scientist as saying “I would rather use someone else’s toothbrush than their ontology”.

On to lunch.

[CSG Winter 2011] Higher ed from both sides now

Greg Jackson (Educause)

Collaboration – we don’t do it very well across our organization.
– We sign NDAs for No Benefit
– We let vendors pick us off
– We keep our cake (we hold on to resources we really should be sharing)

Battles – we fight those we can’t win. Prevalence will sometimes win out over quality.
– Google is going to win
– The CFO is going to win
– Verizon/AT&T/Sprint are going to win
– Oracle is going to win – not everything, but everything it cares about
We don’t engage very well if we characterize them as evil

– Being different from peers isn’t the same as being ahead of peers. No competitive advantage to how we use IT at our institutions.
– Being ahead of peers isn’t the same as winning.
– Distinctiveness yields value, but it also consumes it
– It doesn’t matter what computer you use, because standardization has largely been achieved
– When standardization fails, idiosyncrasy accelerates

Tracy notes that we’re different because our environments demand us to be.
Greg – we don’t want to aspire to mediocrity. We shouldn’t innovate in different directions just for the sake of different directions.

– We reject cost accounting
– We prefer tactics to strategies
– We send good money after bad
– We prefer right to timely
– We eat (or alienate) our seed corn
– We mistake users for customers

– We squabble (especially in public)
– We waste too much time on governance
– We spread ourselves too thinly
– We obsess