CSG Winter 2015 – Integration Workshop

Beth Ann –
Integration getting more complex and it’s an area where central IT has always struggled. Demand is rising.
Survey results: 
Do you have an integration strategy? 11 do 2 don’t
What’s driving it – Innovation, security, new ERPs, BI/CRM, Other: diverse, distributed cloud systems
Where does integration responsibility reside? –  Central IT / Middleware – 6, distributed – 5, virtual group or hybrid – 2, shotgun – 1
Do you have an esb? WSO2 – 4, Mulesoft – 1, Oracle – 2, None yet – 2.
Biggest app challenges – Workday, mainframe, ERPs, departmental apps, apps that require data transformation
Manage integration performance? Most don’t or have limited perf management
Security? certs, “service” accounts, custom tools, policy, platform (Boomi, MuleSoft), Nginx, IDS, Kibana, encryption firewalls.
Critical staff skill sets: Data architecture, advanced programming, basic programming, business analyst, config management
Jim Choate – UPenn – Riding the Enterprise Service Bus
Starting point: Myriad of point-to-point integrations, not well documented, fragile, difficult to be agile,
Drivers for change: New student system, SAAS solutions gaining traction: Concur, Canvas, Success Factors (used in health system and HR training), MIR3
New Approach – SOA, ESB
Benefits – Flexibility, Scalability, reasonable cost, availability, centralized management and monitoring
ESB selection criteria – Services: pub-sub, synchronous messaging, synch messaging, transaction support, transformations, web service generation, guaranteed delivery
Deployment Environment: Scalability, avaialbility, load balancing, clustering
Governance and Deployment: versioning, deployment, upgrades, support roles
Runtime
Security
Three finalists: WSO2, Mulesoft, Oracle
30 applications in production
Open Data Initiative – sparked by undergraduate assembly resolution to open up access to non-confidential dat sets, implemented as restful APIs. Deployed APIs: map item filter service, map item filter parameters service, courses catalog search service, course section search service, course section search parameters service, dining service, directory search service, directory person details, news/events/map search service, transit data service.
Used by departmental web apps, student developed web apps, and (soon) student developed mobile app
Development and Slumni Relations – Key system is Peoplesoft app, real time sync of key biographic and contact data with IModules online community; high volume; fire and forget
Canvas – Real time enrollments; very well received by students, faculty, and staff.
Lessons learned: Don’t get too far in front of vendors (every release of one product breaks APIs), Mulesoft – support generally good, but long wait times for some fixes; Mulesoft – upgrades more difficult than expected; Mulesfot – development environment wasn’t set up for how they liked to work, had to tune; Instructor – API issues… questionable design choices, bugs, throughput problems, documentation; Instructure – overburdened test environment – ended up paying for a dedicated test system
API Infrastructure – Scotty Logan – Stanford
In the beginning – standalone APIs, buildings/ Campus maps, Project Task Awards check – hard to find, moved around; Other random APIs – SOAP, RPC< rematch, RMI, RESTish
“SOA is Dead; Long Live Services” – Anne Thoimas Mane, Burton Group, January 2009
CAP – Community Academic Profiles – Med School Faculty Profiles. Was Java + Oracle; Dept Drupal Sites that wanted data – not java, not oracle, very common, need profile data. Common answer: RESTful API, JSON, OAuth 2.0.
API “Gang of Four” forms. On campus Apigee workshop validate approach. CAP API using Spring and MongoDB – generate JSON versions on profile changes; OAuth AS using Spring. API gateway using node.js. Minimal feature set
Early 2013 – CAP API design & implementation, client/server only, API gateway implementation.
Early 2014 Planning – API gateway enhancements, OAuth 2.0 AS enhancements (user (3 legged) tokens, SAML integration), Thin client + API for accounts system; developer portal
Mid 2014 – Reorg interrupted things but still moving forward
API Discussions: Real-time bus location; device compliance system; AED locations; environmental health & safety; staff & faculty training; directory data.
Common Concerns: opening up data, lack of desks, APIs and OAuth are new and scary; system load and support; infrastructure development
Still to do: Developer portal; “3 legged” Oauth; New gateway features: caching, token verification, rate limiting; UI for managing tokens
Susan Kelley – Yale
Lot of interest in  SOA and ESB from a lot of people for a very long time.
Software solutions in varying degrees of use.
2013 IT Strategic Plan included a section on Integration – still needed to figure out implementation.
Hired a Director of Systems and Data Integration. Workday, opportunity to redo many integrations. CIO very interested in exposing APIs. Infrastructure becomes “exostructure”.
Systems and data integration – created a rationale, scope, and vision document. Central integration competency center – 2 people located with the DBAs, integration community of practice, training for all app teams on new tools.
Defined a three year time horizon in constraining discussion about tools. Had advocates for Fuse/Service Mix, Dell Boomi, IBM Cast Iron, WebMethods. Determined that WebMethods would work well (they already had it) for short term.  Focused on gaps and ended up selecting Layer7 for API management. Uses Telend for easy ETL work.
Merged IDM with systems and data development. Developers, platform admins.
Yale Data API service – being socialized by web team for reuse of data on sites. Data governance – quietly beginning to get some traction; Workday integrations.
Value, Business Drivers, and Challenges – Beth Ann Bergsmark – Georgetown
Current business and use cases for better integration – New ERPs, mobile experience, achieving the “connected campus” means breaking the ERP silos, engagement layer will create many more integrations; seamless intuitive user experiences – drivers for better application integration of extension tools.
Mobile experience – students expect realtime rich data. Use cases: registration, mobile access (ID cards) and identity, innovation.
Emerging and future use cases: increased importance of BI and CRM – engagement layer for experience, aggregation of data across multiple points, actionable data and outcomes; Internet of Things; Agility in service adoptions (cloud) means data integration must be agile too.
Challenges: Technology – systems that were never designed to share data; vendor lok in issues, lack of APIs; fragile integrations that break; performance and monitoring. Business process: data elements defined in isolation. Greater complexity: no longer just point to point; new skills for staff; new situational awareness.
Path forward: Gain situational awareness by mapping and inventory of integrations across stacks, understand the gap between use cases and capabilities. Track or adopt iPaaS solution with standardized connectors; Build integration center of excellence, shift away from artisanal handcrafted integrations, unified voice with vendors to push for modern integration capabilities, integration capability needs to be a heavier incluence on priorities for service/system modernization and product service selection; align with data governance.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s