I/O conference page - desiging for scale and reliability -- web stack: 3 tier --- frontent --- applayer --- api --- persistence layer -- lamp stack evolution --- scalable? no. shared machine for db and webserver. --- split machines for db and server ---- 2 spof --- multi-web servers, single db machine --- dns round robin router ---- pretty scalable --- reverse proxy routing ---- cache static content ---- app-level load balencing; least-loaded instead of round robin ---- requires more machines ---- eg proball --- master-slave db ---- gets better read throughput ---- invisible to app ---- scalable? scales read rate w/ # of servers, but not write ---- spof for writes ---- master may die before replication --- partitioned db ---- requires re-arch of data model: no joins --- app engine stack ---- reverse proxy ---- app & static fiule serving ---- data store ---- memchache ---- api
- design motivations -- considerations --- build on existing google tech, --- integrated env --- small per-req footprints --- isolation btwn apps --- statelessness & specialization --- require partitioned data model
- life of a request -- routed to nearest data center using existing google infrastructure -- load-balancing and routing at reverse proxy layer -- static content --- served, if necessary, using existing google cache. static content is specified in app.yaml in the app -- dyn content --- app server ---- the app engine web server ---- serve many apps ---- concurrent ---- enforces isolation btwn apps and google, and statelessness --- flow ---- checks for cached runtime ---- executes the request ---- caches the runtime ---- app master: manages and schedules ops on app servers -- api requests --- app issues api call --- server accetpts --- blocks runtime --- app server issues call --- server returns resposne
- apis -- memcahceg --- distributed in-memory cache --- optimistic cache --- db queries, results of internet data fetch, etc. --- very specialized: just in-mem cache -- app engine data store --- built in big table (whitepaper avail) --- partitioned --- explicit indexes --- instant reads --- slower writes --- replicated >= 3 machines --- built on gfs (white paper available) -- mail api --- uses same routing as gmail -- the majority of the apis app engine uses are built on other apis used by much larger services = reliable
- recap -- built on existing google tech --- years, lots of money, much talent spent on optimization for scalable tech --- integrated env so ---- best practices ---- some restrictions ---- google tools easily avaolable ---- all logs in one place ---- no machine config ---- ez deployment -- small per-req footprints --- better utilization of app servers --- less mem usage --- limited cpu -- fast requests --- fairness to other apps --- agile routing and scheduling --- runtime caching --- request deadlines --- better use of resources -- isolation between apps --- reasons: safetly and predictability --- certain sys calls are unavailable -- statelessness & specialization --- how? use api calls --- why? performance, load balanced, fault tolerant -- partitioned data model --- indexes for all queries --- no schema --- super-fast reads; writes are a bit slower --- great for read-intensive apps, which include most apps
- metrics -- 80k apps -- 140m pageviews per day -- 200k developers -- whitehouse “open for questions” app --- handled 100k questions and 3.6M votes at 700 requests/sec peak --- instance of google moderator running on whitehouse servers
- questions -- moderator --- it’s an opensource project --- whitehouse engineers tweaked it for performance and security --- google provided support -- api for full-text search? --- they’re working on it -- loading big libraries? --- use runtime caching to store runtime for subsequent requests -- one-click magic for python like w/ gwt? --- not at this time -- how is static serving priced? --- by bandwidth -- data portability in/out bigtable? --- bulk uploader/downloader --- check out “app engine nitty-gritty” talk tomorrow -- differences btwn java and python capabilities? --- equivalence is important -- how many req/sec are req’d to maintain runtime in cache? --- no -- can we pin bigtable replication geographically? --- no, but they’re working on it