1 Virtual Reality with Azure and UnityAzure IoT - Stream Analytics - SignalR - Unity FuseThru

2 FuseThru

3 About Me Jared Rhodes MVP for Microsoft Azure CTO FuseThru Inc.– Blog – Qimata.com GitHub – Qimata Twitter FuseThru

4 Virtual Reality with Azure and UnityA Windows IoT Core device with sensors streaming to Microsoft Azure Azure IoT Hub forwarding the data to Stream Analytics Stream Analytics collecting the data and moving to a service bus Topic. Topic to a ASP.NET SignalR instance Unity engine consuming the SignalR data for realtime data feed in a game. FuseThru

5 FuseThru

6 FuseThru

7 $1,500 Sony VAIO PCG-SRX99 (circa 2001), weighed just 2$1,500 Sony VAIO PCG-SRX99 (circa 2001), weighed just 2.76lbs and made do with a 10-inch screen. Inside, it had plenty of power, an 850MHz Pentium III-M, 256MB of RAM, and b for wireless networking (if you could find a wireless network, that is). Plus it had 20GB of storage, FireWire and even one USB port. he Raspberry Pi 3 is the third generation Raspberry Pi. It replaced the Raspberry Pi 2 Model B in February Compared to the Raspberry Pi 2 it has: A 1.2GHz 64-bit quad-core ARMv8 CPU 802.11n Wireless LAN Bluetooth 4.1 Bluetooth Low Energy (BLE) Like the Pi 2, it also has: 1GB RAM 4 USB ports 40 GPIO pins Full HDMI port Ethernet port Combined 3.5mm audio jack and composite video Camera interface (CSI) Display interface (DSI) Micro SD card slot (now push-pull rather than push-push) VideoCore IV 3D graphics core FuseThru

8 FuseThru Echo Bands - FitBit Cameras – Nest Amazon Dash ButtonsiBeacons Kinect / Hololens Blood Pressure Monitors Myriad of sensor devices FuseThru

9 Vive Rift OSVR Playstation VR Hololens Project Alloy FuseThru

10 FuseThru Uses: Games Treating PTSD Medical TrainingArchitectual Modeling / Real estate Vacation Planning Military training FuseThru

11 FuseThru

12 FuseThru

13 FuseThru

14 FuseThru

15 Azure IoT Hub Device twins. Using Device twins, you can store, synchronize, and query device metadata and state information. Device twins are JSON documents that store device state information (metadata, configurations, and conditions). IoT Hub persists a device twin for each device that you connect to IoT Hub. Per-device authentication and secure connectivity. You can provision each device with its own security key to enable it to connect to IoT Hub. The IoT Hub identity registry stores device identities and keys in a solution. A solution back end can add individual devices to allow or deny lists to enable complete control over device access. FuseThru

16 Azure IoT Hub Route device-to-cloud messages to Azure services based on declarative rules. IoT Hub enables you to define message routes based on routing rules to control where your hub sends device-to- cloud messages. Routing rules do not require you to write any code, and can take the place of custom post-ingestion message dispatchers. Monitoring of device connectivity operations. You can receive detailed operation logs about device identity management operations and device connectivity events. This monitoring capability enables your IoT solution to identify connectivity issues, such as devices that try to connect with wrong credentials, send messages too frequently, or reject all cloud-to-device messages.

17 Azure IoT Hub An extensive set of device libraries. Azure IoT device SDKs are available and supported for various languages and platforms--C for many Linux distributions, Windows, and real-time operating systems. Azure IoT device SDKs also support managed languages, such as C#, Java, and JavaScript. IoT protocols and extensibility. If your solution cannot use the device libraries, IoT Hub exposes a public protocol that enables devices to natively use the MQTT v3.1.1, HTTP 1.1, or AMQP 1.0 protocols. You can also extend IoT Hub to provide support for custom protocols by: Creating a field gateway with the Azure IoT Gateway SDK that converts your custom protocol to one of the three protocols understood by IoT Hub. Customizing the Azure IoT protocol gateway, an open source component that runs in the cloud. Scale. Azure IoT Hub scales to millions of simultaneously connected devices and millions of events per second. FuseThru

18 Azure IoT analytics patternsDevice Collection Presentation & Action Storage & Batch Analysis Stream Analysis Ingestion IoT Hub Search and query Data analytics (Power BI) Web/thick client dashboards RTOS, Linux, Windows, Android, iOS Devices Stream Analytics Storage Blobs / Tools Device Twin SQL DB /SQL DW Machine Learning Gateway Filter Send Ingest Document DB Event Hub Routing Service Bus Data Lake IoT supports multiple patterns of analytics as you can see. For simplicity’s sake lets look at each phase in an IoT solution where analytics can be applied. Let’s start with the one on the left. The first is on the edge where you can add modules to filter, aggregate, log, route data and more. This phase is prevalent in many gateway scenarios. Even during In-Cloud Ingestion-Analytics can be supported. Such as querying the twin within Azure IoT Hub, filtering and routing message at this level etc. There are two main patterns that can be applied to the data for IoT. The first one is hot path: Where data is actually analyzed in real time. In this path real time event monitoring whether simple or complex becomes critical. The incoming data is analyzed based on a set of rules and the data can then be transformed and insights generated. Typically looking at this data through the time series lens can make it even more important to generate accurate predictions. This data can also be applied to machine learning models to better produce the desired output or prediction. There are many different services that can support hot path analytics some of which we will discuss today. Another path we see is the cold path where data is moved into some sort of storage and then different analytic services dependent on your needs can be used to generate insights. Casandra DB Apache Storm/Spark Device Twin Devices to take action On-Device App Analytics On-Gateway Analytics In-Cloud Analytics In-Cloud Hot-Analytics In-Cloud Cold-Analytics In-Cloud Analytics FuseThru

19 Batch Analytics (Cold Path)DATA SOURCES INGEST PREPARE ANALYZE PUBLISH CONSUME Ex: SQL Server, Oracle, DB2, MySQL On-Prem Storage Ex: Azure Data Factory, Oozie Orchestration, Movement, Scheduling, Monitoring Ex: Custom Code, Azure Data Lake Analytics, Hadoop Transform, Combine, Clean, etc. Ex: Azure Data Lake Analytics, Hadoop, Azure Machine Learning, Mahout Data Aggregation, Data Science, etc. Batch Data Movement Presentation, Dashboarding Ex: SQL Server, Oracle, DB2, MySQL SaaS Data Sources Ex: SSIS, Azure Data Factory, Sqoop, IoT Hub (via File Upload) Traditional analytics first moves data (Archived, historical, logs, etc) into some sort of storage and then different analysis is applied to it to generate insights. This is still a common pattern that we must at least talk about briefly as many non software companies still understand this scenario better. Plus when you talk about kicking off POC’s it is very simple to start with making sense of historical data first. As you can see Azure IoT Hub can push cold data directly into blob using file upload feature, or telemetry data using Event Hubs can be pushed into different storage types whether it is Blob or Data Lake or something else. Then machine learning can be applied to this data after it has been cleaned, prepped and transformed. Azure IoT suite and Cortana Intelligent suite components can be leveraged here and finally the insights from this data can be visualized, fed back into business systems or actions back on devices can be taken as well. Ex: Azure Data Lake, Azure SQL DB, Azure SQL DW, HDFS, Cassandra, HBase Cloud Storage Ex: File Upload via IoTHub) IoT Hub FuseThru

20 Near Real Time Analytics (Hot Path)DATA SOURCES INGEST PREPARE ANALYZE PUBLISH CONSUME Ex: Azure Stream Analytics, Storm, Spark Stream Processing Ex: Azure IoT Hubs, Kafka, Kinesis Scalable device data ingestion Devices Ex: Azure Machine Learning, Mahout Machine Learning Archival Presentation, Dashboarding Hot path or real time analytics or streaming analytics is becoming extremely popular due to its usefulness in IoT scenarios. Real time analytics refer to analyzing fast moving data streams and events, while the data is still on the wire. As seen in the previous slide, In traditional data analytics you first store the data in a database and then run analytics on the data to uncover insights. But, you do not always have the luxury of time to do that – as many insights have a very short shelf life for you to act on. For example, you have less than 5 seconds to stop an overheated engine , detect a power failure, detect a fire in a building – so that you can promptly address such situations and avoid catastrophic damage. This real time analysis when coupled with machine learning and visualization is gaining a whole lot of momentum in the IoT space. This combination of analytics on data on the wire can lead us from proactive monitoring to predictive and prescriptive analytics. The data through real time analysis can be fed through machine learning which will determine the probability of predicting a certain outcome based on the algorithm and historical data. This outcome can then be visualized graphically and a predictive action can be taken saving millions of dollars in efficiency and giving way to new business models. Ex: Azure Data Lake, Azure SQL DB, Azure SQL DW, HDFS, Cassandra, HBase Storage Ex: Azure Data Factory, Azure Function, Oozie Orchestration, Movement, Scheduling, Monitoring FuseThru

21 Scenario examples FuseThru Real-time Fraud Detection Streaming ETLPredictive Maintenance Call Center Analytics IT Infrastructure and Network Monitoring Customer Behavior Prediction Log Analytics Real-time Cross Sell Offers Fleet monitoring and Connected Cars Real-time Patient Monitoring Smart Grid Real-time Marketing As you can see there are many scenarios where stream analytics is applicable. From a practical perspective a lot of the stream analytics is looking at telemetry data in real time and getting insights whether this is looking at credit card data from Point of Sale devices or looking at data in real time from a truck for fleet management for route optimization to making real time power related decisions based on data from power grids. Real time analytics is becoming key in IoT scenarios where the telemetry data streaming from the devices need to be acted upon in a very short time frame for a big impact on corporations. The example is credit card transactions from Point of Sales and data analysis from Power Grids to determine anomalies for appropriate actions. Azure Stream Analytics makes all of it possible!! FuseThru

22 Stream Analytics Reference data: Stream Analytics provides users the ability to specify and use reference data. This could be historical data or simply non- streaming data that changes less frequently over time. The system simplifies the use of reference data to be treated like any other incoming event stream to join with other event streams ingested in real time to perform transformations. User Defined Functions: Stream Analytics has integration with Azure Machine Learning to define function calls in the Machine Learning service as part of a Stream Analytics query. This expands the capabilities of Stream Analytics to leverage existing Azure Machine Learning solutions. FuseThru

23 Stream Analytics Connectivity: Stream Analytics connects directly to Azure Event Hubs and Azure IoT Hubs for stream ingestion, and the Azure Blob service to ingest historical data. Results can be written from Stream Analytics to Azure Storage Blobs or Tables, Azure SQL DB, Azure Data Lake Stores, DocumentDB, Event Hubs, Azure Service Bus Topics or Queues, and Power BI, where it can then be visualized, further processed by workflows, used in batch analytics via Azure HDInsight or processed again as a series of events. When using Event Hubs it is possible to compose multiple Stream Analytics together with other data sources and processing engines without losing the streaming nature of the computations. FuseThru

24 Stream Analytics IoT Edge (Private Preview) – Write once in the cloud and push down to IoT Gateways….. FuseThru

25 FuseThru

26 FuseThru

27 FuseThru

28 FuseThru

29 FuseThru

30 FuseThru

31 Email – [email protected] Blog – Qimata.com GitHub – Qimata Jared Rhodes – Blog – Qimata.com GitHub – Qimata Twitter Slides - qimata.com/?p=8461 FuseThru