Cloud Platform Support for API Governance Chandra Krintz Hiranya Jayathilaka, Stratos Dimopoulos Alex Pucher, Rich Wolski, Tevfik Bultan Dept. of Computer Science UC Santa Barbara March 2014

WEB AND MOBILE APP DEVELOPMENT o  “Service-ize” digital assets and IP" o  Accessible everywhere, all the time (BYOD)" o  Platforms-as-a service (Paas) simplify this process" o  Facilitates deployment for software/data as-a-service" o  Automates configuration, deployment, container management, monitoring, …"

DIGITAL ASSETS ENCAPSULATED AS SERVICES o  Code and data of value to the organization" o  Application Programming Interface (API)" o  Common entry point for access & control"

DA SW Env

o  By client apps and browsers"

o  Functional boundary "

o  Web service software environment (SW Env)! o  Storage technologies: SQL, NoSQL, bucket/object stores, HDFS" o  Computation technologies: Hadoop, tasks, event handlers" o  Load balancers, app servers"

DIGITAL ASSETS ENCAPSULATED AS SERVICES o  Code and data of value to the organization" o  Application Programming Interface (API)"

DA SW Env

o  Common entry point for access & control" o  By client apps and browsers"

o  Functional boundary "

o  Web service software environment (SW Env)! o  Storage and computational technologies" Develop

Retire

o  API lifecycle"

Deprecate

o  Evolves with that of digital assets (DAs)" o  Software environment can/does evolve separately! o  Can change without impacting API clients"

Deploy Manage

PROLIFERATION OF APIS o  Popular development model" o  Service-ize digital assets, exposing them via APIs" o  Reuse extant APIs to construct new APIs, mashups, applications"

o  For public consumption" o  Pay-per-use and free" o  Google, Facebook, Twitter, Yahoo!, …"

PROLIFERATION OF APIS 123ContactForm 123ShopPro 12secondstv 140Proof 18amail 1Map 21FortyMedicalDistrictSlideShowD 23 30Boxes 3dCart 411Sync 43Places 43Things 4Shared 500px 5min 7digital 8coupons 8tracks A9 Abbreviations Acapela AccuWeather ActBlue ActivaLiveChat Active Activecom ActivFinancial ActualReports AddThisAnalytics AddThisMenu AddThisServices AddThisSharingEndpoints AdenForshaw039sTheCat Adility AdobeOnAir AdobeShare AdobeSocial AevumObscurum AftertheDeadline AgendizeAction AgentRank aideRSS AIM AIMPhoneline Airbrake Akismet AlchemyAPI AlchemyAPIKeywordandTermExtracti AlchemyAPITextCategorization Alexa AlexaThumbnail AlexaTopSites AlexaWebInfo Alibris AllforGood AllocinMovie Allogarage AlternativeTo Amazonca AmazonCloudWatch AmazonDynamoDB AmazonEC2 AmazonElasticMapReduce AmazonFulfillmentWebService AmazonHistorical AmazonMarketplaceWebService AmazonPayments AmazonProductAdvertising AmazonQueue AmazonRDSRelationalDatabaseServi AmazonRedshift AmazonS3 AmazonSES AmazonSimpleDB AmazonSNS Ambassador AMEE Amplify AngelList AngularJS AOLOpenAuth AOLOpenMail AOLPictures AOLVideo AOLVideoUpload AOLWebAIM AonawareDictionary ApiculturWordLemmatizer APIfy AppFog AppHarbor Appnet ArcWeb ARKive ArtBeat ArtistData arXiv Assembla AtlassianBitbucket AustralianBusinessNumberLookup AustraliaPost AuthenticJobs AuthorityLabsAccount AuthorityLabsPartner Authorize.Net AvantLink AviaryEffects AviarySuite AviaryWeb Avvo AwardWallet Awesm BabyNames Backpack BackTweets BackType Baidu Bandcamp Bandsintown BarcelonaBicing BART Basecamp BatchBook BBC BBCMusic Beatport Bebo BeenVerified BeerMapping Behance BeliefNetworks BestBuyBBYOpenProducts Betfair BibleGateway Bibliacom BibSonomy BigCommerce BigHugeThesaurus BigTribe Billboard BilleoActiveeWallet Billomat Bing BingMaps BingMapsGeocode BingMapsGeocodeDataflow BingMapsRoutes BingMapsSearch BingTraffic BingTranslator BiodiversityHeritageLibrary BioIDWebServices Bit.ly BitcoinChartsMarkets Bitcurex Bitext BitPay Bitrix24 BitStamp BlankSlate bLaugh Blekko Blinksale Blinkx Blip.tv Blipfm BlockAvenueNeighborhoodReviews BlockchainBlockExplorer BlogamaIPInfoDB Blogger Bloglines Blogmarks Blue BlueDot Bolcom BookingMarkets BookMooch BooRahRestaurantSearch Box Boxcar BoxnetEmbedit BreweryDB Brightcove BrighterPlanetEmissionEstimates Brightkite BroadsoftXtended BrooklynMuseum BrownPaperTickets BTCe Bter BTWeb21C Buffer Buildasearch Bump Burstn BusinessAlerts Businessgov BusinessProfiles buySAFE BuzzData Buzznet CafePress Calais CampaignMonitor CampBX

From programmableweb.com

DemocracyInAction DeviantART Dezrez DHL Digg DigitalBucket DigitalNZ DigitalPodcast Diigo Dipity DirectedEdge DirectTextbook Discogs Disqus Dlvrit DNSimple DNSTools Doba Docstoc DocuSignEnterprise DoIt Domain DomainTools DonorsChoose Doodle dopplr Drawloop Dribbble Dronestream Dropbox Dropio Dwolla Earth911Search EarthTools EasyUtil EBay EBayFinding EBayMerchandising EBayProductServices EBayShopping ECGridOSEDI EchoNest EchoSign EditGrid EdmundscomVehicle Educationcom EEADiscomap Egnyte Elance ElderCareLocator EliLillyClinicalOpenInnovation Embedly EmporaEvergreen EndiciaLabelServer Enthusem Entrez Envato EPAStationCatalog EPAWatershedSummary Ergast eSideWalk ESPN ESRIArcGISJavaScript ESV Etsy EvatureTravelSearch Eventbrite Eventful EveOnline Evernote EveryTrail Evoca Evri ExchangeRate Exfm Expedia Expono Extractiv EyeEm Facebook FacebookAds FacebookChat FacebookCredits FacebookGraph FacebookRealtimeUpdates FacebookSocialPlugins Facecom Faces Factual FanarttvMusic FanBridge FanFeedrSportsNews FanSnap FantasyFootballNerd FAROO FatSecret fav.or.it Faviconz FCC FedEx FedSpending FeedBurner FeedMap Feedzilla ffwd Figoconnect FilePicker FilesAnywhere FilesTube FilmCrave Findory FireEagle Fitbit FizberNeighborhoods Flattr Flickr FlightStats FlightView Floobs FloristOne FollowTheMoney Forecast Formstack Forrst Forvo Fotolia Foursquare Foxrate Framey Freebase FreebieSMS Freewheel FreeYourID FreshBooks FriendFeed Friendster Fring FullContactPerson FUTEFWikipedia Fwix FwixLocation GamePro GamesRadar Garmin Gatekrash GeckoLandmarks Geeklist Genability GengoHumanTranslation GeoAdmin geocoder geocoderCanada Geocubes Geograph GeoGratis GeoIQ Geolenz Geoloqi GeoNames GeoNB GeoPlugin Geoportail GetGlue GetMapping GetSatisfaction Gigablast GigJunkie Gigya GitHub GlobalBiodiversityInformationFac GlobeXplorer GoMoTextSMSGateway Goodreads Goodsie Google GoogleAdSense GoogleAdWords GoogleAffiliateNetwork GoogleAjaxFeeds GoogleAJAXLanguage GoogleAJAXLibraries GoogleAjaxSearch GoogleAnalytics GoogleAppEngine GoogleAppsEmailMigration GoogleAppsMarketplace GoogleAppsScript GoogleBase GoogleBooks GoogleBookSearchBookViewability GoogleBuzz GoogleCalendar GoogleChart GoogleCheckout GoogleCivicInformation GoogleClientAuth GoogleClosureCompiler GoogleCloudPrint GoogleCodeSearch GoogleContacts GoogleCustomSearch GoogleDesktop GoogleDirections GoogleDistanceMatrix GoogleDocsList GoogleDrive GoogleEarth GoogleEmailSettings GoogleFinancePortfolio GoogleFont

PROLIFERATION OF APIS o  Popular development model" o  Service-ize digital assets, exposing them via APIs" o  Reuse extant APIs to construct new APIs, mashups, applications"

o  For public consumption" o  Pay-per-use and free"

o  Within organizations! o  Leverage development across organization through reuse! o  Internal and external development"

o  Agile processes and DevOps change SW environment frequently"

o  API becomes point of strategic business decisions! o  Common entry point into org (security, access control, activity)" o  Focuses development, DevOps, and IT" o  Can influence business value"

A NEW IT RESPONSIBILITY o  HW/SW infrastructure now commoditized by cloud" o  Experiencing a shift toward management of " o  Software environment (directly controllable by DevOps)" o  APIs" o  Control, maintain, facilitate reuse, and secure" o  Provision resources and software environment"

o  Requires new tools and a system-wide framework" o  For API Governance: combined policy, implementation, and deployment control of APIs for IT-managed services and DAs " o  Unify and automate API management processes" o  Facilitate efficiency and scale" o  In number of APIs, API clients (service users), developers" o  That accounts for input from business concerns"

PAAS’S NEXT BIG THING o  PaaS is the perfect infrastructure for API Governance" o  Deployment automation, elasticity, fault tolerance, high availability, logging and monitoring, on-premise and/or public"

o  Requires additional support for" o  API cataloging, search, and registration" o  This already exists in the numerous API management platforms!"

o  Unified and automated policy support! o  Specification (a language)" o  Verification, analysis, and feedback (developer tools)" o  Enforcement" o  Deployment time" o  Runtime"

API GOVERNANCE FOR IT SW environment

DA

API API API

API

DA DA API

DA

API API

API

Runt Enfor ime ceme nt

Deplo y Enfor ment ceme nt

Developers

IT Managed Infrastructure Service consumers and clients

Dev Tools

•  Dependency checks •  APIs •  SW infrastructure •  Provenance tracking

•  Static analysis •  Policy verification •  Automated testing •  Autogen of enforcement logic

•  Runtime policy enforcement •  Access and rate control •  AB testing •  Auditing & feedback gathering •  SLA & QOS checks

EAGER -- Enforced API GovernancE for REST

EAGER IMPLEMENTATION: APPSCALE o  PaaS platform that decouples innovation from common services" o  Automatically manages and scales apps + service ecosystem" o  Access scalable services via well-defined de facto standard APIs!

User Credentialing

Data storage & Processing (NoSQL, SQL, …)

Monitoring & Logging

API

Security & Authentication

API

Developer Innovation

Messaging & Communications

Web Hosting & Serving

EAGER IMPLEMENTATION: APPSCALE o  PaaS platform that decouples innovation from common services" o  Automatically manages and scales apps + service ecosystem" o  Access scalable services via well-defined de facto standard APIs! o  Starting point: Google App Engine " User Credentialing

Data storage & Processing (NoSQL, SQL, …)

Monitoring & Logging

API

Security & Authentication

API

Developer Innovation

Messaging & Communications

Web Hosting & Serving

WRITE-ONCE, RUN-ANYWHERE CLOUD APPS

o  On-premise" o  Behind your firewall" o  Everywhere"

NO CODE REWRITE

EAGER: APPSCALE EXTENSIONS o  Enforced API GovernancE for REST" o  Policy language " o  Restricted subset of Python" o  Policies: per-operation, per-API, system-wide"

o  Developer Tools" o  API analysis! o  Static policy verification" o  Automatic generation of" o  Functional tests from policies" o  Policies from unit tests" o  Deployment enforcement checks" o  Runtime deployment checks"

o  Deployment enforcement" o  Runtime enforcement"

EAGER DEVELOPER TOOLS: EXAMPLE o  API Similarity Tool" o  Evaluate the “porting effort” associated with changing an application " o  That is using one API, to use a similar API"

o  Helps developers and IT managers reason about " o  How hard it will be to change to use a similar API (reuse code)" o  How similar two APIs are (for policy enforcement @ code reuse)" o  How APIs evolve over time (and how to enforce change control)"

o  Describe API behavoral and functional semantics" o  Using the EAGER language: as axiomatic semantics" o  Translate to ASTs (per operation)" o  Employ DICE coefficient (Hoare’s Rule of Consequence)" o  To compute an AST similarity (porting effort) score"

EEMPIRICAL MPIRICALEEVALUATION VALUATION: PORTING EFFORT

SUMMARY o  API Governance is increasingly important for IT " o  Vast proliferation of API development/deployment" o  Lacking management, control, and automation" o  API Governance is unified and automated API policy specification, analysis, auditing, and control "

o  PaaS is the ideal foundation for providing API Governance solution" o  Elasticity, fault tolerance, scale, distribution, portability"

o  EAGER extends PaaS (AppScale in particular) with " o  Policy specification, verification, and enforcement" o  Developer tools (analysis, feedback, autogeneration of tests and enforcement checks)" o  Automatic deployment and runtime enforcement"

THANKS! o  Recent Student Researchers and Visitors!" o  Current: Stratos Dimopoulos, Geoffrey Douglas, Adam Ehrlich, Chris Horuk, Hiranya Jayathilaka, Alex Pucher" o  Past: V. Arora, M. Baranski, C. Bunch, N. Canumalla, J. Chohan, N. Chohan, A. Gupta, S. Hedge, M. Hubert, J. Kupferman, P. Lakhina, Y. Li, Y. Nomura (Fujitsu), K. Prakasam, S. Sundaram"

o  Collaborators" o  Linda Petzold (CSE/UCSB), Andreas Hellander (Uppsala U), Rich Wolski (UCSB/Eucalyptus)"

o  Support" o  Google, IBM Research, NSF, NIH" " "http://www.cs.ucsb.edu/~ckrintz "[email protected]! " " "http://www.appscale.com (AppScale Systems)"