1 🔥🔥 The Fire Down Below! 🔥🔥Interoperable Convergence of Storage, Networking and Processing at Layer 2 🔥🔥 The Fire Down Below! 🔥🔥 Micah Beck Electrical Eng. & Computer Sci. Terry Moore & Piotr Luszczek Innovative Computing Lab. University of Tennessee,Knoxville Dept of Energy Germantown 8/2/2017

2 Motivation and Background20 yrs: Web caches to prestage content (TN Cachebox) 15 yrs: Logistical Networking as async. communication Design methodology: follow patterns of Internet and Unix Architectural analysis: very confused, much controversy Experiment, deploy, describe… try, try, try to understand! Slow realization: storage, processing implement layer 3 Last year: some formal insight into the hourglass model

3 Interoperation and Innovation“What is lacking … is a common, open, underlying ‘platform’, analogous to … the Internet or Web, allowing applications and services to be developed as modular, extensible, interoperable components. To achieve the level of interoperation and innovation… that we have seen in the Internet will require investment in the basic research and development of an analogous open platform for intelligent infrastructure….” [CCC Smart Cities whitepaper]

4 Historical Context: Divergence of IT silosStarting with gates, wires and cores Modeled by Boolean operators and variables Divergence of communication, transformation, persistence Giving rise to networking, computation and storage silos But optimization & service creation work across silos Active Networking, Web Caching, Content Delivery Grid, Software Defined Networking This talk proposes an architectural strategy for convergence

5 Legacy Silos and Overlay Convergence

6 The Hourglass vs. The Anvil

7 Publications ”On the Hourglass Model, The End-to-End Principle and Deployment Scalability”, Micah Beck, Submitted to CACM in 2016 (http://philsci-archive.pitt.edu/id/eprint/12626) “Interoperable Convergence of Storage, Networking and Processing”, Micah Beck, Terry Moore and Piotr Lusczcek, Submitted to Usenix Middleware in (https://arxiv.org/abs/ )

8 Our Proposal: A common abstraction of the IT nodeConvergence can occur at many levels Operating system kernel interface (eg. file systems, sockets) Application overlay environments (eg. PVM, Grid, Workflows) Convergence at a high level preserves silo abstractions Reliability, performance, fairness, permanence, identity, accountability Convergence at a high level limits interop. and ubiquity Resources at the bottom of the stack are accessed through the top We propose a converged common abstraction of buffer management, transformation and transfer at layer 2.

9 Buffer Interoperability: Linux sendfile() system call

10 Interoperability: The spanning layerThe spanning layer as the basis of interoperability “Certain protocols are designed with the specific purpose of bridging differences at the lower layers… Such a layer is called a ‘spanning layer’...” David. Clark, Interoperation, Open Interfaces, and Protocol Architecture, 1997 N interfaces require O(N2) translations, perhaps imperfect A common spanning layer enables “network effects” The “thin waist” of the hourglass “Interoperable convergence”: full generality, performance

11 Deployment Scalability: Simple, generic, limitedOvercoming barriers: technological, geographical, admin. Successful infrastructure interfaces scale “by design” Not regulation or market dominance The strategy of the Internet: converge at a lower layer Expose underlying resources Enable flexibility and heterogeneity at higher layers Unix also followed a “lowering” design strategy Saltzer, Clark, Reed, Thompson & Ritchie all worked on MULTICS Internet and Unix are the two examples of deployment scalability

12 Multics and Unix sure, but the Internet? End-to-end?“A similar problem exists for the magnetic tapes containing backup copies of segments. … These tapes should probably be encrypted, using per-segment keys known only by the operating system.” “Protection and the Control of Information Sharing in Multics”, Jerome Saltzer, Massachusetts Institute of Technology Communications of the ACM, July 1974

13 Logical Weakness: The Hourglass Theorem (Beck 2016)Competing notions of “thinness” at the spanning layer “Logical weakness” may exert a controlling effect A weak spanning layer implies more implementations But a weak spanning layer implies fewer applications! Recipe for deployment scalability: Select “necessary application requirements” Design the weakest spanning layer that satisfies them! Best effort, bounded lifetime, cycles, memory, block size, MTU, …

14 Heterogeneity: Lowering the spanning layerOssification: The dark side of spanning layer ubiquity Necessary uniformity at the spanning layer Resources poorly exposed are not accessible IT nodes are computers: processors, storage, transfer In digital computer systems high level flow/process/file abstractions are implemented as sequences of buffer-to- buffer steps Minimal abstraction: operations on common buffer model

15 The Hourglass vs. The Anvil

16 Exposed Buffer ProcessingFundamental Buffer Management Local or in neighbor Weak: best effort with limits imposed by node owner Allocate/delete buffer lease Write/Read to/from/xfer-between buffer(s) Apply operation to transform buffer(s) Semi-static set of (weak, limited) operations available An operation can be a VM (taking code as input) “

17

18 Application: Content DeliveryUnicast datagram delivery and point-to-multipoint services Redundant point-to-point transmissions Bottlenecks overload local network and server resources Poor user performance or denial of service (“Slashdot effect”) Content Delivery employs caching & server replication A proprietary converged distributed platform “at application layer” Specialized, expensive, difficult to deploy and operate Layering violation – an overlay built out of pieces of the Internet stack

19 Point-to-Multipoint Communication

20 Persistent processes vs. EBP opsmanager data movement ops

21 Application: Edge processingData deluge at the edge Sensors and many other distributed data producers Application-specific data volume reduction is required Thin clients require shared infrastructure Administration & policy are major roadblocks Trust, fair contention, cooperative sharing are enablers Remember: It works with LAN and OS resources

22 Example: Record filteringFILTER consider input buffer as an array of records of width bits with a field starting at bit position of width eliminate all records whose value in that field is not equal to Records generated by sensors at edge nodes, filling buffers Continuously issued FILTER operations implement filtering “process”

23 Persistent processes vs. EBP opsfilter ops manager data movement ops

24 What about high level “requirements?”Security: moats and bridges Low latency transmission Highly synchronized computation Permanent, corruption-free storage Identity and accountability, metering and billing Overlay implementation: The Data Logistics Toolkit, Lead PI Martin Swany, Indiana University (data-logistics.org).

25 Broader Implications and Generalized ConvergenceCommunicating with poor infrastructure Digital divide: rural, remote, urban, developing world, … Disaster response: wildfire, floods, earthquakes, war, … Can other resources & services inteoperate at layer 2? Power and transportation Smart Cities, Cyberphysical Systems, Critical Infrastructures, …

26 WildfireDLN (Data Logistics Network)"Resilient System Solutions for Data Sharing for Wildland Fire Incident Operations," NIST Public Safety Accelerator Program Award #70NANB17H174 Lead PI Nancy French, Michigan Technical University

27 Conclusions Divergence is a historical artifact, not inevitableOptimization across silos is necessary but challenging Convergence at layer 2 is a path to generality, innovation Heterogeneity is key to flexible global services (layer 3) Locality of access enables control and flexibility

28 Thank you!