Tag: #IT-security

Tom Vachon, Harvard

Harvard’s cloud at a glance

  • 684 applications targeted for migration by 7/18, 300+ migrated already
    • Shutting down one on-prem data center
  • 1 VPC per account on average
    • Centrally Billed: 131 Accounts
    • 45 Accounts/VPCs on Direct Connect
    • Looking to make Cloud a University-wide strategic program
  • Cloud Shield – physical firewall
    • Kicked off 7/15 in response to a security breach
    • POC – 11/15 – 2/16
    • Started automation code 3/16
    • 15,000 lines of code
    • Production ready 7/16
    • Design goals
      • provide highly available and highly redundant AWS network access
      • Provide visibility of traffic into, out of, and between cloud applications
      • Provide next-gen firewall protections
      • Inline web filtering to simplify server configuration
      • Provide multicloud connectivity
    • Tech details
      • Diverse paths and POPs – Boston has 2 direct connects, and a POP in Equinix in Virginia with private network connection to campus
      • Primarily done for visibility
    • Actively discourage host-based firewalls
      • Use security groups instead
      • Don’t use Network ACLs
  • Will provision services with public IPs
    • They have overlapping private address spaces
  • Design manager of managers in Python
    • Create an ops & maintenance free architecture in Lambda
    • Provide REST API through AWS API Gateway
    • Isolate changes by segregating integrations in AWS Lambda
  • Leverage AWS DynamoDB for
    • Schemaless session cache
    • Dynamic reconfiguration
  • Challenges
    • Static DNS names
      • use ELB or ALB for applications
    • Everyone needs to be on Harvard IP space
      • Delegates six /16s for AWS
    • Legacy application stacks
      • Java has a “mostly hate” relationship with DNS
        • Lots of apps cache DNS forever
    • Reduced S3 visibility
    • Inability to do app-by-app identification
      • Grouping by data classifications
    • Items which are unknowingly locked down to AWS IP space
      • eg doing a yum update to AWS Linux from a non-AWS ip space
  • Virtual firewalls per VPC were going to cost >$4 million over three years, this model costs $1.6 million over five years
  • Most applications got faster when distributed across this model
    • Less switching in the way

Panel Discussion

  • Biggest technical challenges so far?
    • Georgetown  – have to run virtual firewalls in HA. Looking at replacing with TrendMicro
    • Harvard – lack of visibility in AWS
    • UNL – Vast offerings from vendors – how to wrap heads around it?
    • How to support on prem and burst out, especially for research instruments?
    • Cornell – Keeping up with the technology. Having people to manage and implement solutions. Encouraging lack of consistency in an effort to use the best new technology to solve problems.
    • Wisconsin – Have to worry about security a whole new paradigm in the cloud.
    • Notre Dame – pace of innovation. Do we prepare for a more rapid pace of change (and those costs) or learn to live with not implementing the latest?

 

 

Advertisement

CSG Fall 2016 – Security and Configuration in the Cloud, pt 1

Sarah Christen is introduces the workshop.

Bob Winding and Sharif Nijim – Notre Dame

  • Cloud first – even distributed groups going cloud first
  • VPCs: Share Services VPC with peering to Central Applications or Departmental VPCs; VPN Tunnels over I2 to campus; Be wary of implicit peering through campus routers.
  • 80% central IT, 20% distributed
  • Pauses to assess progress are built into the plan, with sprints to address issues. Inviting Mandian to campus to help establish 5 year security roadmap.
  • Export controlled data
    • 22 projects on campus dealing with this kind of data
    • Gov Cloud new initiative to support research
    • NIST 800-171 DFAR-7012 – looks a lot like PCI DSS
      • AWS covers 1/3 of security controls in GovCloud
      • Talked to a half-dozen PIs – experiments generate lots of data, then they move data to a local spot for analysis, or design work that happens locally with specific apps.
      • Developed a compliance matrix and quick start template in Cloud Formation
        • Quick start builds shared services and multi-tenant project VPC
      • Want to create an environment in GovCloud that is cloistered for the work until it goes back to the sponsor.
  • VDI – Using graphics intensive applications in the cloud
    • Looked at frame – delivers screen from remote desktop over video streams. Running pilot in US East
  • Look at the RDP gateway as the audit boundary – doesn’t include the end user device
  • Least privileges in IAM
  • Working with Purdue to look at SaaS providers for security monitoring and log analysis
  • AWS Security
    • Flipped IAM from least privilege to explicit deny of dangerous operations
      • Separation of control on IAM policy creation and application
      • Writing Lambda functions to undo changes that aren’t permitted
  • Organizing security groups
    • Setting standards for common functions, like sysadmin access
    • Engineers have a hard time keeping things simple
    • Databases use security groups for access control, which simplifies auditing
  • `Data security
    • Using Tripwire tuned precisely on systems with confidential information
    • Encryption at rest and backups
    • Replication of backups/snapshots to a separate account and region. If a credential is compromised can’t destroy both operational data and backup
  • Future
    • Cloudfront WAF
      • Want to fully leverage Amazon’s tools to gain advantage
      • Realize that this increases lock-in with the vendor
    • Host IDS for selected sensitive systems – looking for things that don’t cause choke points
    • Comment from Bruce – “we’re on the verge of a post-firewall world”
      • At AWS have to use IP-address based controls across VPCs and shared services
  • https://oit.nd.edu/cloud-first

Bob Turner – Wisconsin

  • Somewhere between cloud experimentation and cloud aware.
  • Trying to not yet deal with sensitive and restricted data in the cloud
  • security requirements for accounts and VPCs
    • Working off script based on risk management framework
    • Using it for onboarding people into cloud environments
    • Working on audits and attestations
  • Enforcing cloud controls (will also use for on campus environments)
    • Provisioning/De-provisioning
    • Going to try to use FEDRAMP checklist as a guide
    • Approval of risk by Executive able to accept on behalf of University
  • Automated Templates (consultancy model)
    • Create a new account or migrate existing account under master
    • Pre-provisioned equipment templates with logging enabled
    • Configured for Shibboleth
    • Moving towards Duo for MFA
    • Activate AWS Config
    • Use (future) cloud security tool for initial verification and continuous monitoring
  • Things to be concerned about
    • Holding on to root accounts and credentials
    • Challenges of CDM
    • Usual tools are not necessarily available
    • AWS tools have charges
    • Challenge of cloud vendors that don’t support SAML or federation
  • Account management
    • Group email per department, including Office of Cybersecurity Rep
  • Researcher accounts
    • must know their expected data (at present no Restricted or Sensitive data)
      • Google as a government service that has been pretty well vetted by US agencies

Sarah Christen – Cornell

  • Cloud first according to IT Strategic plan written in 2013
  • 54 accounts under master contract, hundreds outside
  • Cloudification services has been an opportunity for central IT to partner with campus
  • Requirements for being on master contract
    • Onboarding discussion
      • How billing works; unit responsibilities – how is this different than the data center?; Security and configuration requirements; Benefits; Discussion about joining tech community; central services available – Container service (will containerize and run code for fee), DevOps service
    • Attestation
      • Explicit agreement to policies
    • Shibboleth
    • Duo for MFA for console access
    • Activation for AWS Config and CloudTrail
    • CloudTrail logs sent to Security Office
  • Onboarding – create account, configure Shib and Duo, lockdown root account, standard AD groups (admin, cloud group, security), activate Config and CloudTrail and configure CloudTrail logs to be sent to Security office as well as the VPC owner; activate Cloudcheckr and schedule review of how to use.
  • CloudCheckr – allows those with accounts to see usage data; makes recommendations on how to save money; sends monthly invoices; runs continuous vulnerability scan; gives Security a view into all accounts
  • Standard VPC setup – blogs.cornell.edu/cloudification/2016/04/08
  • What about reseaerch accounts?
    • Easy onboarding without a lot of steps or complication
    • No intereference with research, no cost of performance overhead
    • Solutions for export controlled data and othe rcompliance requirements
    • Standard network config not always a good fit
    • Consultation and services – Docker, Data Storage, Training, Devops support

Mark Debonis – VaTech

  • Cloud Aware -> Moving into CLoud Experiment
  • One production VPC in AWS, five pre-production
  • Moving towards both AWS and Azure offerings
  • Manual provisioning process
    • Customer contacts CCS via Service Catalog for Cloud brokerage discussion
    • Difference in Azure (upfront) and AWS billing models – In Azure if you don’t use your commitment in a year you lose it
  • Logins to Azure portal with VT AD account, Redirect to VT ADFS, Login and use Duo, Primary contact manages other users through Azure Admin portal with VT AD accounts

Kevin Murphy – UNL Lincoln

  • Cloud first for SaaS
  • Experimentation for PaaS and IaaS: Rackspace, Azure, AWS
  • On VPC in Azure for disaster recovery (domain controllers, ADFS)
  • VPC in progress for AWS
  • Central IT is pushing cloud strategies, very little departmental participation. Research computing run by CS faculty, not interested in cloud computing.
  • Security requirements: Federated logins (ADFS with Duo) for Azure. Shipping everything from IaaS to Splunk on campus
  • Security requirements – manually creating accounts; No PII data in the cloud
  • Been doing Azure StoreSimple device – hybrid solution.
  • Moving PCI environment to the cloud with a managed service provider who will take the liability and run on AWS. “not extremely expensive”
  • Challenges: Moving current architecture to IaaS can be prohibitively expensive – people build for peak loads, need to use elastic capabilities. Exploring PaaS options such as Azure Web Apps and DB services. Billing is a challenge.

Bereket Amdemichael, Daniel Tamiru, Georgetown

  • Based their AWS cloud architecture on the work done at the CSG Cloud Architecture Working Group
  • Added a proxy layer.
  • IPSec VPN – Cisco
  • Users only have access to specific VMs – have to access across the VPN
  • VPC and group architecture is a “spirited discussion”
  • When do they (security) need to be alerted when something isn’t right?
  • Using Equinix for high speed transfer to AWS

CSG Spring 2015 – Security 3.0: Physical meets cyber (IDS meets GIS)

Randy Marchany – VA Tech

All security is local; empower the local departmental IT staff; Business Process trumps the Security Process if there’s a conflict; learn the business process before imposing security requirements; restrictive security practices cause worse problems overall.

Three main business processes at Universities _ Academic, Administrative, Research.

Continuous Monitoring: Keeping someone from getting inside has failed miserably. Firewalls are not effective proteection devices – they are effective detection devices. Change the strategy – assume they are in so go hunt for compromised hosts; monitor outbound traffic; prevent their command and control communication; inbound monitors server side attacks; outbound monitors client side attacks.

Map developed tool that displays estimate of number of people occupying general-use classrooms and dining facilities, hour by hour throughout the week. They have a gameday app that keeps track of concentrations of people within the football stadium. Married GIS with IDP sensors so they can see where machines are that are being attacked or compromised.

Challenges: Funding, Training, Process, Technology.

CSG Spring 2015: Security 3.0 – SCADA Tales of Horror!

After the break there’s discussion about risk management, and business impact analysis. How do we know what’s worth protecting for what kind of investment? There’s a point made that we’ve focused on protecting devices, not data. Should be focusing on what is your data and where does it reside? Can we partner with audit teams to look at these risks?

When Worlds Collide: Security, the physical world & IoT – Bill Allison, UC Berkeley

Consider what’s coming: media, environmental monitoring, infrastructure management, manufacturing, energy management, medical and healthcare systems, building and home automation, transportation, large scale deployments (e.g. smart cities).

More is not just more – it’s different.

Security at Berkeley began around WWI with three guys with flashlights, guns, and sticks. 1986 – online intrusion was the Cuckoo’s Egg and the 1988 Morris Worm.

SCADA _ Supervisory Control and Data Acquisition – generations: First generation, monolithic; second generation, distributed; third generation, networked; fourth generation, Internet of Things.

Most institutions have SCADA systems but they’re not controlled in IT.

SCADA AND BAS – Jim Jolkl, UVa

SCADA: Focus: industrial process automation, utilities, gas pipelines; BAS – Building Auotmation Systems. Same technologies, by and large.

Monitor and Control: Heating and cooling systems including our data centers, hospitals and clinics, animal studies areas, biosafety rooms, HAZMAt areas; Power systems, distribution, generators and transfer switches; often fire and security systems, sometimes door locks. How secure are these systems?

Systems arenot small – 200 buildings; 90,500 physical BAS points used for monitoring and/or control; 15,000 BAS controllers, at UVa.

BAS Network Technology: Common protocol, BACnet, supports services beyond HVAC. Security: security was not a focus, but standards now exist. But deployment use of BACnet security is limited. Multiple transport forms supported: RS-485, ARCNET, ethernet, BACnet over IP.

SCADA – what is different? Scada networks perform critical functions: temperature, pressure, valves, generators, chillers, etc. Constructed with old technology with a very long refresh cycle (15-30 years). Intrinsic security generally lacking; Expensive ($1m for a moderate building); limited CPU power in devices, so hard to do crypto or mutual authentication. Firmware update facilities are good, allowing to push anything to it.

Typically campuses hire control vendors whose knowledge of networking comes from dedicated dialups. See that in things like video surveillance systems too.

Decentralized SCADA – Much SCADA gear is outside the control of facilities: freezers; lab equipment; door systems; classroom controls; cameras. Many of us have no knowledge of what types of control and data acquisition equipment departments place on the network.

Protection strategy: IT Security: Firewalls, etc; Physical security; Monitoring; Cryptography – work towards being able to consider the SCADA network as an untrusted network.

We’re back in the 90s again, sort of: important equipment that can’t protect itself. Protofols are open, widely deployed and insecure; large installed base of old equipment on a slow refresh cycle. But our ability to add external protection is much better, active monitoring is generally in place; system owners generally understand the problem and want to fix; main control software runs on modern platforms.

CSG Spring 2015 – Security 3.0: The CISO’s Empty Cooking Pot, Part 1

Stefan Wahe, Madison:The CISO’s Empty Cooking Pot

Goals: Describe the baseling of Cyber Security Strategif Plan; Learn how to gain participation in achieving the plan; identify how you may help Cyber Security on your campus.

Background: If a strategy’s posted on a website does it make a sound? UW Madison 2011 IT Security Strategy.

People forgot about the strategy – no reporting, no accountability. Positive outcomes: Consolidated two competing groups, elevated security to report through CIO’s office. New CISO with risk-based methodology. Created a 100 day plan including drafting a cyber-security plan. Hired a Chief Data Officer – brings governance groups together to talk about data.

Baseline strategy will: have a commonly agreed to purpose; be understood by the community; establish a governance model; assign accountability; have a communications plan; be flexible or adaptable to change

Cyber Security Baseline: Identifies current and emerging threats to support the strategy; identifies the responsibilities of the CISO and IT Security org; identify and empower governance groups to participate in and evolve the strategy; Identify goals, assign accountability and timeframes; Align with the campus and IT strategies.

Strategic Elements: Complete data governance and information classification plan; establish risk management framework to reduce cybersecurity risk; build a community of experts; consolidate seccurity operations; improve cyber threat intelligence analysis, dissemination, and remidation; optimize services, establish metrics, promote compliance. Each element has SMART goals.

Enabling Objectives: Tactical things that need to be done. Establish restricted data environments; centralize data collection, etc.

Governance: Identify governance groups to empower community to meet goals.

CSG Spring 2015 – Security 3.0: Card Access Security using Grouper

Charlie Kniefel – Duke

Legacy environment: Prior to 12/14 was using Blackboard Uptim to manage door access; independently managed buildings; no consistent rules for building access on campus; Would have to ask each building owner for access; deactivation of cards not as quick as desired.

Implementation of Blackboard Transact: Started in early 2014 with planning starting in 2013. Covers both financial systems and access control; access control services are separated from financial transactions as part of implementation; lots of cleanup/preparation prior to transition.

Management of access plans: seems to be a lot like groups; based on who you are, your role, what classes you take.

Community plans: groups created for faculty, staff, students; needed to agree on standard business hours.

Individual building plans: access for students can be based on major or classes; challenges – financial system based on who pays you, not where you work; local coordinators trained to use tools to manage membership; goal is to get to role-based access.

Future steps: Wrapping grouper seems to be a common trend: toolkits for instruction; research toolkits for research group services management; access control membership; share? Additional technology enablement: contactless but with legs? Apple’s plans?; Roles, roles, roles.

Built access for students to get the data about their card activity.

Lack of good space management data hinders usefulness of roles.

CSG Spring 2015 – Security 3.0: Multifactor authentication

Passwords are dead: Richard Biever, Duke

What got us here?: Breaches, breaches, breaches; account & password sharing; tension between strong password policies and user acceptance; time it takes to crack a password.

How did we start?: Pre-2011 faculty concern about access to benefit information; 2010-2011 evaluation of existing technologies; 2011-12 evaluation of integration with Shib; 2013 pilot with Duo (600 IT people); 2013 IT rollout; 2014 Direct deposit phishing incident – individuals lost pay due to falling for scheme on bank routing; 2014-15 voluntary adoption, currently about 14k accounts across campus.

Approach: Focus on shib sites but don’t forget other technologies (SSH, RDP, VPN); allow strength checking for multiple factors in shib; build our own self-service interface to mimic what users were used to; make it easy to recover with temporary passcodes.

Four-pronged rollout: evangelaize across campus for voluntary enrollment; make mandatory for specific services (e.g. protected network); Make mandatory for certain groups (e.g. Finance, IT, School of Nursing); Duke Medicine implementing mandatory for remote access Aug 1.

Tone from top is important – memo from EVP to campus, memo from Medicine leadership.

At about 1/3 participation now.

What’s next: mandatory for Duke Medicine remote access; mandatory for HR system; solve the “thick client” problem for SAP and Peoplesoft; Test how to accept “MFA” attribute from federation partners for shib logins.

InCommon is working on creating a MFA working group.

CSG Spring 2015 – Security 3.0: Learning to live with an advanced persistent threat.

We’re at Penn State University for the Spring CSG meeting. The first workshop is titled Security 3.0

John Denune – UCSD – Learning to live with an advanced persistent threat.

What is an APT? Not an opportunistic attack – they’re after something you have. Targeted, skilled, and won’t stop till they reach their goals. Can take years to break into your systems. Can be technical means (0 days, custom malware), or social engineering. Can be theft for financial information, corporate espionage, state sponsored.

APT Lifecycle: External recon (looking at projects, org charts, etc), initial compromise, then establish a foothold, escalate privileges until they get what they want.

Initial detection started in June 2012.

Tried to drop malware on a departmental machine – not all that unusual. Came in the same way the following night. Came in on separate VPN accounts to VPN concentrators, and logged in to servers with OU admin credentials. Over next several nights reset passwords, rebuilt machines, etc.

Lesson learned: Really pay attention to anti-virus alerts, but don’t (completely) rely on your AV product – only one caught this and it only caught one out of several instances.

Where possible, track IPs instead of blocking them.

Initial Recon was traced back in February 2012 – scoured departmental web servers. Initial compromise happened in April. Found a dozen compromised machines they didn’t know of.

Called in help: Make your local FBI agent your new best friend. They knew why hackers would be interested in the international studies department. Also were very helpful technically.

One piece of malware was a custom version of Gh0st RAT. Another technique was Dynamic DNS Beaconing. Talking to different servers every hour or day. Makes it difficult to track IPs. Had to turn up logging as high as they could bear, especially authentication, netflow (on VPN concentrators), and DNS. Found another dozen systems that had been compromised.

All attacks took place Sunday – Thursday between 6pm and 3am Pacific: 9-5 Monday-Friday in Beijing.

You don’t need to rely on a lot of malware when you’ve already got a long list of credentials. You don’t need to crack passwords when you can just pass a hash. Can get the hashes from compromising a client, and if it’s an admin can then get access to servers and domain controllers.

Mitigations: change passwords multiple times per day; fast track 2FA; Compartmentalize passwords; separate user and admin credentials; minimize lateral trust – host based rules to prevent system-system access; scan entire domain for scheduled tasks; rebuild domain controllers.

Emergency Action – September 2013

Hadn’t forced password change in a dozen years. Effectively and securely communicating a password change is hard. Now doing it on a yearly cycle.

Reengagement – July 2013

Hackers kept trying to get access with stored credentials. After a week of failure they disappeared for a while, and then tried passing old hashes for all upper level management. Failed at attempts so far.

Infrastructure changes: Yearly PW changes; monitoring the network for pass-the-hash (not easy because that’s the normal Microsoft way for getting access to file servers, so looking for hashes that don’t correspond to direct client logins); implement 2-factor for OU admins; additional “bastion hosts”, limit lateral access; more logging and splunk analysis; security clearances for some personnel so they can talk to the FBI; Windows 8.1 and Server 2012 R2 features: RDP use without putting the credentials on the remote computer, addition of a new Protected Users group whose credentials cannot be used in remote PtH attacks.