Search and Analytics Platform for Text and Rich Media

1 Search and Analytics Platform for Text and Rich MediaHP...
Author: Jordan Harrison
0 downloads 0 Views

1 Search and Analytics Platform for Text and Rich MediaHPE IDOL Search and Analytics Platform for Text and Rich Media

2 Major trends transforming the marketIndustry trends Challenge 1 Content explosion Traditional processes break down in an era of content explosion and rich formats How do I get a handle on my data in all its forms to drive value in the business moment? Regulatory pressures New regulation increasing cost and risk Requires timely cross-repository access 2 How do I comply with compliance & regulations cost effectively without impacting productivity? Major industry trends are leading to major challenges. Diverse content explosion is outstripping the capabilities of traditional analytics tools. Competitive advantage depends upon how businesses tap into the massive volumes of text, video, image and audio data for critical insights With increasing regulatory requirements – how can we look into the data to proactively mitigate risks and address non-compliance issues such as data loss and security concerns? Mobility is one of the major driving forces behind big data. While the opportunity to leverage the data is huge, there is significant security exposure due to the nature of the mobile environment Employee productivity plays a key role in the success of an organization. It has become critical that knowledge workers can easily find the right information that they are authorized to access quickly, and can identify relevant knowledge sources to facilitate collaboration across the organization. Mobility and Consumer data Convergence of files sharing, backup 3 How do I enable mobile computing, drive adoption while maintaining data security? Borderless Enterprise Enabling borderless and secure collaboration 4 How can I enable seamless and secure information collaboration within and across the enterprise?

3 Big Data Shift: The landscape is radically changingMore connected people, apps and things generating more data in many forms Human data Technology gap Mobile apps System logs Data centers Compliance archives Internet of Things Sensors Social networking Photo sharing Wearable devices What’s making it difficult to realize this opportunity, is a fundamental technology gap that’s become acute as all these new sources of data have proliferated. Most of the technologies designed for managing and analyzing data were designed for an era when the data we cared about – the business data from our systems of record – neatly fit into rows and columns. So relational databases, enterprise data warehouses and business intelligence tools served us for many years. The challenge is that the fastest growing segments of Big Data are Human Data and Machine Data, with a new set of challenges: Human Data includes all the content we create: some of which is highly regulated for compliance purposes, like contracts, legal documents, medical or business records, or customer communications and other content that may not be regulated but is still valuable, including marketing documents, s, social media posts, images, voice recordings and video. You can’t put that into rows and columns, so how do you uncover what matters in all that information to gain insight and act on it quickly? Think about how calls and recorded and monitored in a call center. Traditionally we would only be able to automate and capture the structured data like the time & length of a call, but what matters most to us is the content of the conversation itself – what was the customer trying to achieve, what was their experience like, how well did the CSR’s responses satisfy the customer? This is a goldmine for marketing, customer service and even product development, but a call center supervisor cannot possibly listen to every call and traditional data architectures don’t offer much help. Machine Data is the complete opposite of Human Information. It’s the high-velocity information generated by the computers, networks, security devices and sensors embedded in just about everything—the Internet of Things. Think system log files, click streams, IT monitoring feeds, temperature reads, energy usage stats and building access alerts. How do you make sense of the millions of events generated every second, and find the anomalies to rapidly identify a system outage or security threat? It’s the proverbial needle in a haystack and our traditional data architectures weren’t built to handle the speed and volume of machine data. Together, Human Data and Machine Data are growing 10x faster than traditional Business Data and this growth has created a technology gap. Traditional data vendors are trying to extend the architectures designed for Business Data, but our customers are telling us those architectures aren’t scaling. Machine data Business data

4 Why is processing human data different?Human Information is made up of ideas, is diverse and has context Ideas don’t exactly match like data does; they have distance. Human Information is not static – it’s dynamic and lives everywhere. Legacy techniques have all fallen short. Social Media Video Audio Texts Mobile Human data, unlike structured data, is diverse, context sensitive and nuanced. It is scattered everywhere and changes rapidly. For example, the hot topics regarding a major sporting event (e.g. World Cup) may change with entertainment (e.g. celebrity sports stars) news and/or business/political news (e.g. FIFA investigations). The dynamic patterns, trends and relationships existing within the data are critical but cannot be easily and quickly uncovered with legacy analytics techniques. Documents Search Engine Images IT/OT Transactional Data

5 HPE IDOL : Understand & act on human dataData Enrichment Cloud The IDOL platform is one single, secure, scalable, language agnostic platform for processing and understanding data from virtually any source and of any format. It can connect to a wide range of enterprise repositories and cloud based sources as well. This gives users the ability to leverage all the data available to address diverse search and analytics requirements. The four key capabilities are data enrichment, advanced enterprise search, knowledge discovery an rich media analytics. I will explain what these capabilities are and how they are used in rea-life customer implementations. Data Enrichment -Augment data with other relevant data Example - Extract company names from tweets and make tweets searchable by company names Advanced Enterprise Search -Context sensitive search across internal and external sources Example -Search for HP and get results related to HP, IBM, Dell Knowledge Discovery - Uncover trends, patterns & relationships without explicit queries Example- Uncover root causes of customer attrition with social media and call center data Rich Media Analytics -Recognize and analyze image, video and audio Example -Logo/object/text recognition and speed-to-text transcription in broadcast media Advanced Enterprise Search HPE IDOL Diverse Data Sources Enterprise Knowledge Discovery Rich Media Analytics

6 What is IDOL technology?6 Cloud Use patterns to monetize data At a very high level, IDOL technology consists of three main components: -A very rich array of data ingestors enabling IDOL to monitor and index contents across hundreds of repositories both on premise and in the cloud. Data is managed in place so there is no relocation of data required. The indices are stored in the XML database for fast processing and access. The pattern recognition engine uses statistical techniques to pick up patterns, trends and relationships. Unlike typical database queries, IDOL users do not need to know what questions to ask in questions. They can easily explore insights with advanced search and personalization functions. Library of data ingesters IDOL Data Store Enterprise

7 IDOL’s core capabilitiesRich Media Analytics Knowledge Discovery Advanced Enterprise Search Data Enrichment What is it? Augment data with other relevant data Example Extract company names from tweets and make tweets searchable by company names Context sensitive search across internal and external sources Search for Samsung and get results related to Samsung, Apple iPhone, Huawei Let’s look at the four key capabilities more closely. Date enrichment is about augment data with other relevant data such as metadata. For example, we can extract company names from tweets, associate the tweets with the extracted names, and make the tweets searchable by company name. Advanced enterprise search is about providing search results based upon relevant concepts associated with the search terms. This goes beyond simple keyword search. For example, if you search for HP, you may see results associated with, HP, IBM and Dell because IDOL understands that all these companies are related in that they are in the same industry and address very similar markets Knowledge discovery liberates users from having to know what questions to ask beforehand. IDOL can recognize patterns, trends and relationships hidden within the data and lets the data tell the story. For example, IDOL can analyze customers’ tweets and call center logs to reveal root causes as to why the product has not been selling now. Rich media analytics allows users to incorporate video, image and audio data in gaining more complete insights. For example, in addition to say, text analytics of social media, a marketer can also monitor and analyse broadcast media for logos, on-screen text and speeches. Uncover trends, patterns & relationships without explicit queries Uncover root causes of customer attrition with social media and call center data Recognize and analyze image, video and audio Logo/object/text recognition and speech-to-text transcription in broadcast media

8 IDOL Capability - Data EnrichmentKey capabilities: Capture key attributes such as names, numbers, places, languages 1000+ data formats (text, video, audio) supported Sentiment extractions Automatic association of extracted entities with data Native viewing of files Value: Increase findability, insight robustness, and search accuracy Accelerate high fidelity rendition of search results Data enrichment increases information finadalibity, search accuracy and insight robustness by extracting important entities from the data and associate the entities with the data. Here is an example of what entity extraction looks like. As you can see, entities such as names and places are automatically spotted and extracted from the digital content. These metadata can be associated with the original document to enrich the overall information about the document. Organization −National Security Agency Places −Moscow −St. Petersburg −Washington −Syria −Russia Names −Vladimir Putin −President Obama −Edward Snowden

9 IDOL Capability - Advanced Enterprise SearchKeyword: Clinton Key capabilities: Conceptual search unconstrained by keywords Search federation across multiple repositories Integrated security entitlement based search Personalization and knowledge management Value: Increase relevancy and contextual applicability Elevate search efficiency and effectiveness Protect sensitive information Enable extensive knowledge sharing and development The ability to search based upon concepts liberates uses from the constraints of keywords. IDOL analyses the data and derives concepts from it so relevant information that may not contain the exact search keyword can be provided to the user. When a query is executed, it searches across all repositories simultaneously to accelerate the delivery of all relevant information while preserving security entitlements so only authorized users can view sensitive data. It also learns from users’ actions to develop implicit profiling for custom knowledge discovery and sharing. Results include Bernie Sanders

10 IDOL Capability - Knowledge DiscoveryKey capabilities: Monitor and analyze 400+ repositories and formats Index data where it resides Uncover trends, patterns and relationships Support 150 languages Value: Leverage virtually any data (inside and outside the enterprise) to gain comprehensive information Eliminate copying requirements, storage costs, and hand-off risks Get holistic intelligence and reduce blind spots Derive insights regardless of language barriers IDOL has an extensive library of connectors and broad support for diverse data types so you can tap into virtually any data any where for insights. It indexes data where it is located so you can eliminate copying requirements, storage costs, and hand-off risks. It leverages proven statistical techniques and natural language processing to understand the data and uncover trends, patterns and relationships within the data. This is especially important for the discovery of new insights where no known queries can be used to unearth. With the support 150 languages, you can easily support operations where multiple languages (including multi-byte) are involved.

11 IDOL Rich Media AnalyticsKey capabilities: High Performance Video analytics Recognition – Character, Objects, License Plate, Vehicle, Clothing, People Counting, Face Analyses – Scene, Demographics, Color, Clothing Classification Deep Neural Network Powered Audio analytics Speech-to-text, transcript alignment Recognition – speaker, language, fingerprint Segmentation, phrase search Seamless Integration with IDOL’s advanced search and text analytics Data Fusion - Uncover patterns or relationships among events to reveal complete picture Open system for easy 3rd party integrations such as VMS and PSIM Value: Capitalize on rich media to deliver unique insights unavailable from text Accelerate better decisions with speed and accuracy Protect investment with proven, modular and scalable architecture The ability to capture and analyze video and audio can provide unprecedented insights. It covers wide range of applications including people and object recognitions, scene and demographic analysis and rich media data classification. We also leverage Deep Neural Network in audio analytics for performance and accuracy. The robust capabilities allow users to process and analyze speech data to facilitate search and knowledge discovery. Rich media analytics is seamlessly integrated with IDOL where rich media data can be directly indexed into IDOL. One of the key capabilities is data fusion where patterns and relationships hidden within seemingly disparate data can be uncovered. For example, a car passing through an airport terminal may not be much of a threat. However, if the following events occur, data fusion can connect the “dots” and issue an alert: o Scene analysis detects the same car passing through the terminal multiple times in a short timeframe o License plate and vehicle make/model recognition detects that the license plate does not match the vehicle make and model o Audio recognition detects a glass breaking noise at a parking lot near the terminal We have an open system that integrates with major 3rd vendors in Video Management System (VMS) and Physical Security Information Management (PSIM) to enable easily integration of analytics into video data rich environments.

12 HPE IDOL - Market leadershipGartner Magic Quadrant for Enterprise Search 2015 For the 2nd consecutive year, Gartner has positioned HPE as a leader in its Enterprise Search Magic Quadrant 2015 based on ability to execute and completeness of vision. HPE, for the 2nd year in a row, is positioned in the leaders quadrant. The 2015 Gartner MQ report for enterprise search is available from - This sustained and consistent level of leadership speaks to our the strengths of our vision and execution. In the 2015 Garter MQ on enterprise search, Gartner particularly cited our mapped security, our ability to reveal patterns in unstructured data and cloud-based offering as our key strengths. This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from HPE. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose. Source: Gartner (August 2015)

13 Customers across industries around the globe“ … gain value from unstructured data, which has not been effectively utilized until now, and enable a truly comprehensive healthcare analytics approach to uncover trends, patterns and relationships.” As you can see, IDOL is entrenched in many industries across the world. Public sector, media, telecom and healthcare providers. IDOL is a highly competitive and robust platform. In a recent customer survey, 83% of our customers who bought IDOL over Google would recommend IDOL. “83% of surveyed IT organizations who bought HP IDOL over Google would recommend HP IDOL … “ “HP ... integrates across diverse data sources and uncovers actionable intelligence ..” 2015 Survey Beijing Future Advertising

14 HPE IDOL for higher Big Data returnHolistic Proven Versatile Integrate data silos & unlock hidden insights Sustained market leadership One platform for diverse use cases IDOL is unique for the following combination: - Enable a holistic approach to search and analytics where virtually any data from any where and of any language can be leveraged to derive a complete picture of insights. Organizations no longer have to make decisions in partial vacuum and take full advantage of their data assets - Consistently validated as a market leader by analysts (Gartner MQ, Forrester Wave - ) and is proven in diverse industries. -Unified platform with hundred of advanced analytics functions under one roof to support diverse use cases. Open and scalable system for easy embedding and 3rd integrations. Built on proven world-class technology rooted in Machine Learning, Natural Language Processing, Deep Neural Networks. For more information, please visit

15 Thank you