1 Augmented Reality in a 4G WorldApplications for Higher Education February 15, 2011 This presentation is primarily focused on augmented reality for Android mobile devices.
2 What is Augmented Reality?A technology…and confluence of technologies A field of research A vision of future computing An emerging commercial industry A new medium for creative expression Source: Gene Becker, Lightning Laboratories
3 Definition of AR Augmented reality (AR) refers to the addition of a computer-assisted contextual layer of information over the real world, creating a reality that is enhanced or augmented. ELI Horizon Report, February 2011 Trying to imagine how augmented reality will be used is like trying to forecast the future of the web in 1994. The Economist Technology Quarterly, September 2009
4 Definition of AR - ExampleSource:
5 How Will Your Campus Use AR?We will be running a Poll during this presentation and ask you to respond how you will be using AR on your campus. Three ways to respond: Text Web: poll4.com Twitter Tweet @poll Campus Tour
6 What AR Applications Exist on the Market?483 AR apps exist in the Apple Store out of 300,000+ total iOS apps 258 AR apps exist in the Android Market out of 130,000+ total Android apps Browsers allow you to view augmented content – Junaio, Layar, and Wikitude World Browser Editing tools allow you to augment your own reality – Tag What, Artags, and Sekai Camera Educational tools help you learn by interacting with the environment around you – Google Goggles and Google Sky Map Sources: AndroidZoom, Augmented Planet
7 Browser App Demo Wikitude World Browser
8 Editing App Demo Tag What
9 Educational App Demo Google Sky Map
10 Are Current AR Apps Really AR?No, not really The biggest category of apps is AR browsers They just use the device’s GPS and compass to determine where you are and which way you are facing It is doing pretty much the same thing as Google Maps, except overlaid on a real-time digital video image It is a good start, but there is still a lot of work to do to get to real AR
11 So, What Does a Real AR App Look Like?It would have the ability to take advantage of contextually aware information The user can tailor their interests and experience the content based on their interests It would have the ability to add content The user can add, remove, or change the openness of their content in the AR space It would have the ability to immerse the user into the scene without the user being aware they were in the scene The user can be tracked as an object amongst other objects to become part of a seamless environment
12 Types of AR Marker-based (Visual Metaphor)Markerless (Spatial Positioning/Gravimetric) Sensor dependent (i.e. GPS, compass, accelerometer) Natural feature/image recognition Natural feature/ Image recognition Marker Sensor dependent
13 Marker-based Demo SSTT 3D box demo with Galaxy Tab Example Screenshot
14 Sensor Dependent ExamplePollution visualized in Manhattan Sean White and Steve Feiner of Columbia University developed an app that shows levels of carbon monoxide in Manhattan based on collected CO sensor data (red = locally sensed, green = remote EPA sensor reading). The height of each ball reflects concentrations of the pollutant. Source:
15 Image Recognition DemoGoogle Goggles on the Samsung Transform
16 AR Limitations Constrained audience Immature technologyComplex app authoring Limited use cases
17 Constrained Audience 83% of U.S. population has a mobile phone25% of those who have mobile phones have smartphones 67% of those with smartphones don’t have a data plan That means only 1 in 7 of the U.S. population has the technical capability to access your AR application Source: BillShrink, InfoWeek
18 Smartphone Penetration at Notre DameSource: The Nielsen Company
19 Immature Technology Sensor accuracyGPS only accurate within 10 meters; accelerometer drifts over time and can get confused Indoor/relative positioning and object tracking GPS needs clear line-of-sight to satellites; Skyhook can place you by wi-fi triangularization Graphics/screen size Not well suited for outdoor use, low light situations, or rapid movement/screen refresh events; small screen size Quality and quantity of bandwidth Network latency (~400 ms) causes many delays and errors Battery/Power Usage GPS, other sensors, and processor drain battery quickly
20 Complex App Authoring Expensive to develop AR apps due to:No single standard/spec or AR engine Platform centric authoring required Android Too many versions to support: 1.5, 1.6, 2.0, 2.1, 2.2, 3.0 Inconsistent applications across carriers iOS Apple reluctant to make all sensors available Unwilling to open up all hardware/software for AR development Open source Too many choices, no real leader (Gamaray, Mixare, openAR, Argon, KHARMA, ARToolKit, FLARToolKit, OpenMAR, ARML) Develop for web app or native app or both? Too many options creates scalability and cost issues Need an open, distributed, multi-user AR framework fully integrated with the Internet
21 Limited Use Cases Conventional desktop UI environment doesn’t workWindows, icons, menus, and pointing – six degrees of freedom needed rather than 2D Reliance on single-person, socially disconnected AR user experiences Very few AR apps exist to socially connect people Ergonomics – Unnatural physical position/point only Having to hold up phone to view is awkward Overload or over-reliance UI must not overload user while also preventing the user from over relying on the AR app such that they miss important cues from the environment
22 Benefits of 4G Higher potential bandwidth Minimal latencyBetter throughput Better coverage/signal penetration Simultaneous user support Reliability Interoperability
23 Mobile Data Traffic GrowthThe Mobility Revolution = How People Work and Play 3G Traffic in Mature Markets to Increase 20 Fold by End of 2014 September 1, 2009 Machine to Machine (M2M) Smartphone adoption Embedded laptops and data card growth Consumers adopting mobile apps Enterprise mobility into every product, service, and process 4G Source: Chetan Sharma Consulting
24 High Capacity Requires SpectrumExpect no more than 22 MHz FDD per operator nationwide 198 MHz (1) (2.5 Ghz) 85 MHz (1) 60 MHz (1) 700 MHz Most of these are allocated to voice 3G Extensive enough to deliver next generation broadband content & services 4G (1) Spectrum band shared by multiple operators
25 Sprint 4G Spectrum AdvantageWith large spectrum holdings, Sprint offers faster speeds and serves more users, while competitors must choose between number of users versus high data rates. In summary: More Spectrum = Greater Capacity and Fewer Potential Bottlenecks for Customers Spectrum Position in Market for high speed data Competitors Concern and potential customer impacts… “rapidly growing demand for bandwidth-intensive wireless services is placing excessive demands on current networks and spectrum In order to migrate to LTE on a broad scale, wireless companies will need a “wide band of clean spectrum.” – Colin S. Stretch, Counsel for AT&T Inc. Comments of AT&T Inc., WT Docket No , September 30, 2009 120 MHz Sprint 2.5 GHz WiMAX “I'm not ready to put a number on it, but I'll say in the five-plus year timeframe, I'd like to have north of -- I'd like to be in a position where I could acquire north of 100 megahertz. “ – Bill Stone, Executive Director Network Strategy, Verizon Wireless National Broadband Plan Workshop, Spectrum, September, 700 MHz and AWS 30-46 MHz Verizon “We’re going to have to consider pricing structures that allow us to sell packages of bytes, and at the end of the day the concept of a flat-rate infinitely expandable service is unachievable.” – Verizon Chief Technology Officer Dick Lynch Metered Mobile Broadband 700 MHz and AWS 12-24 MHz AT&T “AT&T CEO Randall Stephenson says wireless customers who are heavy Internet users will eventually pay more than those who use less.” -MarketWatch March ,2010 Source: Macquarie, The Wireless 4G Evolution, November 2009
26 Sprint Download Speeds4G Compared to 3G 4G enables productivity with speeds up to 10 times faster than 3G Sprint Download Speeds Sprint Upload Speeds Peak Up to 10 Mbps 10 Mbps 8 Mbps 6 Mbps 4 Mbps 2 Mbps Downloads up to 10x faster than 3G 8 Mbps 6 Mbps 4 Mbps 2 Mbps Average 3-6 Mbps Kbps Peak Up to 1 Mbps Sprint 4G Sprint 3G Sprint 4G Sprint 3G
27 4G Can Complement or Replace Wi-fi4G shares much in common with typical public WiFi… high bandwidth and performance broadly supported by IT, Computer and Consumer Electronics companies superb technologies for embedding low-cost radio chipsets …but overcomes some challenges businesses/institutions experience with WiFi Sprint 4G Wide area coverage (Anywhere on the Campus) Mobility (walking, riding, exercising) No capacity constraints Full security (FERPA) Dedicated spectrum (no interference)
28 Green Campus SustainabilityCore Enablers 4G and Custom Network Solutions Mobile Learning Convergence Stadium Coverage Street-level Research Center Covered Parking Campus Safety Courtyard Lecture Hall Operations Premise Applications Subway Admin Buildings Library Underground Coverage Residence Halls Green Campus Sustainability
29 The Sprint Id Opportunity in EducationSprint ID lets users, on select Sprint devices, get a feature-packed bundle of digital content (apps, widgets, ringers, wallpapers and more) all at once in a single download to build a total personalized experience for their phone. Campus administrators can take advantage of Sprint Id to create unique, branded campus experiences that foster community building and engagement ; while leveraging the existing resources at their disposal. Through a variety of experience packets, universities can extend their campus reach, build affinity, and provide access to key, campus resources.
30 Can AR Take Advantage of 4G?Unfortunately, not right now The limitations of the device, both hardware and software means we cannot fully take advantage of 4G yet As the devices mature, you will see more and more AR apps becoming true AR apps where they take advantage of: Cloud-based content/data flows from sensors Mashups Shared augmented realities Registration and display of virtual objects
31 Today’s Smartphone AR LimitationsSensor accuracy (GPS, compass, etc.) Display (small screen) Form factor/appearance (ergonomics) Weight of device Battery life Processor speed and battery usage Outdoor use (weather, sunlight) Cost Memory/storage capacity
32 Are there any 4G AR Apps Available Now?No, but there are other 4G apps available Current 4G apps take advantage of higher speeds and specialize in transferring large files that you wouldn't want to download over a 3G network Here are some of the winners of Sprint’s 4G app challenge in 2010 NASAImages – this image app gives users access to large, high-quality pictures Say What? – this gaming app requires reliable connectivity and low latency to be played properly MediaFly – this is a video and radio streaming app that detects when there's a 4G connection nearby and it can switch to that and provide more high-quality videos and a less blocky experience
33 NASAImages Demo NASAImages – this image app gives users access to large, high-quality pictures
34 MediaFly Demo MediaFly – this is a video and radio streaming app
35 AR Now and in the Future Now Future “Point” interactionLive, interactive experience Involvement of limited senses in apps Multi-sensed apps Single, individual viewer experience Multi-user, collaborative, social experience Local, fixed content Cloud-based contextual data integration AR engine fragmentation Common AR engine across all platforms
36 Health Instruction/Knowledge DatabaseExample Screenshot Source:
37 Classroom Control ConceptWhen the professor walks into their classroom, their mobile phone senses they are in the room and pops up a virtual control panel for them to control all aspects of the classroom with hand gestures such as turning the projector on, setting the room volume, changing the light settings, etc.
38 Roman Forum Example
39 Arch of Septimius Severus
40 Roman Forum Example
41 Roman Forum Augmented ExampleTo the Emperor septimus severus , Son of Marcus, Pius, Pertinax, Pater Patriae, Parthicus Arabicus,Parthicus Adiabenicus, Pontifex Maximus, having held the tribunician power 11 times, acclaimed emperor 11 times, Consul 3 times, Proconsul,and Emperor Marcus Aurelius (caracalla), Son of Lucius, Antoninus, Augustus Pius, Felix, having held the auspicious tribunician power 6 times, Consul, Proconsul, Pater Patriae, Highest and Strongest Princes for having restored the State and enlarged the Empire of the Roman people, by their visible strengths at home and abroad, the Senate and People of Rome [made this]
42 Show Poll Results Show Poll results for “How will your campus be using AR”
43 Questions?
44 Higher Education AR Example #1
45 Higher Education AR Example #2
46 Higher Education AR Example #3