Drive Business Innovation With a Modernized Data Warehouse Environment

1 Drive Business Innovation With a Modernized Data Wareho...
Author: Francis Rice
0 downloads 0 Views

1 Drive Business Innovation With a Modernized Data Warehouse EnvironmentUnleash your unstructured data, and marry it with the structured data to innovate.

2 ANALYST PERSPECTIVE Unleash the potential of your data warehouse. The vast majority of your corporate data is not being leveraged and this translates into untapped opportunities. While data can be structured or unstructured, the majority of the data warehouse is managing only structured data. With modernization technologies, however, you can harness the unstructured portion of your data. By default, a data warehouse is a meeting place for data. If IT can create a modernized data warehouse to act as a meeting place where both structured and unstructured data converge, unlimited business opportunities can be realized. Daniel Ko, Advisory and Research Manager, Information Management Info-Tech Research Group

3 Our understanding of the problemCIO or CDO, who is tasked with modernizing an existing data warehouse to meet the ever changing business demand. VP of Data Management, who coordinates the effort of the data warehouse modernization project. Assess your current data warehouse environment from both business and IT perspectives. Identify business-driven opportunities for data warehouse improvement and match the opportunities with modernization technologies. Construct a compelling business case to get the necessary investment. Chief Technology Officer and Enterprise Architects who need to produce a solution architecture for the modernized data warehouse. Business leaders who want to understand more about modernization technologies and how to leverage them in the business. Align business and IT in terms of defining business-driven modernization initiatives. Understand the emerging technologies that complement the data warehouse and how they can be used. Build architecture diagrams that appropriately describe the current and future states of the data warehouse.

4 Executive summary Many organizations have implemented enterprise data warehouses as a stepping stone to creating a single-source-of-truth for business intelligence and analytics purposes. These data warehouses are traditionally constructed with a rigid relational or dimensional model, which does not support emerging data trends or technologies. The vast majority of your corporate data is not being properly leveraged. Modernize the data warehouse to get value from the 80% of unused unstructured data. Avoid rip and replace. Develop a future state that complements your existing data warehouse with emerging technologies. Be flexible in your roadmap. Create an implementation roadmap that’s incremental and adapts to changing business priorities. Relational data warehouses were never intended to handle today’s business demand for data in terms of volume, velocity, and variety. IT departments are under immense pressure to discover and implement new technologies that will give business users greater agility and insight, in a market where the benefits of these tools seem endless but there is no tried and tested route to achieve them. Establish both the business and IT perspectives of today’s data warehouse environment. Explore the art-of-the-possible. Don’t get stuck trying to gather technical requirements from business users who don’t know what they don’t know. Use Info-Tech’s interview guide to discuss the pains with the current environment, and more importantly, where stakeholders want to be in the future. Build an internal knowledgebase with respect to emerging technologies. The technology landscape is constantly shifting and often difficult for IT staff to keep track of. Use Info-Tech’s DW Modernization Technology Education Deck to ensure that IT is able to appropriately match the right tools for the business’ use cases. Create a compelling business case to secure investment and support.

5 The data warehouse is a staple in nearly every organizationAs a centralized point of storage for current and historical data, the data warehouse has become synonymous with creating a “single source of truth.” It has provided organizations with a method to consolidate disparate databases and data stores to: Create a more comprehensive view of corporate data Facilitate reliable analyses that support data-driven decision making Despite all of the technology disruptions in the data world since the introduction of the data warehouse … 99% of organizations still view their data warehouse as integral to business operations. Source: “The State of the Data Warehouse” 2015

6 The corporate data you currently use 70-80% of the data they ingest.Business innovation opportunities are sitting untapped in your unstructured data assets Fact: The vast majority of data that businesses ingest comes in unstructured formats that relational data warehouses cannot handle in raw form: Audio, image, video, audio files Documents with free text Sentiment data on social media The corporate data you currently use Conventional wisdom estimates that most organizations are unable to use 70-80% of the data they ingest. The corporate data you’re not leveraging, resulting in untapped opportunities

7 Both business units and IT are itching to modernize, but significant barriers remainA resounding majority of professionals, from both IT and the business, view modernization as critical to getting continued value out of the data warehouse and realizing success. 91% of businesses feel their data warehouse is lagging behind. 50% Yet… Source: TDWI, 2016 Poor data stewardship/governance Inadequate data warehouse staffing Cost of implementing new technologies Inadequate skills for new technologies Lack of business case or sponsorship Lack of big data/analytics experience Top 6 Challenges to Data Warehouse Modernization Source: TDWI, 2016 There are a number reasons why modernization projects are held back:

8 Modernization is a prime opportunity for CIOs to differentiate themselves as strategic leadersAs data rapidly accumulates and evolves, the pressure grows on the CIO and the rest of IT to move beyond providing basic IT services, and instead be seen as enabling business innovation. Updating the data warehouse with new capabilities: Addresses the lament of siloed, inaccessible data. Evolves the company into a data-driven organization, moving business users away from gut-feel decisions. Clearly aligns IT with strategic initiatives along the lines of customer intimacy, product innovation, or operational excellence. The ultimate gain of data warehouse modernization is that the CIO becomes the person business people collaborate with, instead of dictate to.

9 Rip and replace isn’t the only way to modernize; add complementary systems as an effective alternative A relational data warehouse is a tough sell on its own, given the large upfront monetary investment, as well as dedicating person-hours to maintenance after the implementation is complete. Making the case to overhaul the existing platform in favor of new technology is unrealistic for most organizations, from both an IT resource capacity standpoint and in terms of getting the most value for business users. therefore… The number of businesses with multiple data platforms around the data warehouse is expected to double in the next three years, while half as many will be relying on a data warehouse alone within the same time frame. Source: TDWI, 2016

10 Key Pitfalls EncounteredModernizing too much and too quickly can trip up IT and leave business units more frustrated than before CASE STUDY Industry Source Finance Info-Tech Research Group Professional Services Situation An enterprise-level company specializing in online payment series recently moved into the big data world by running a pilot intended to support new analytics for business users on Hadoop. Challenge The pilot project showed some promise, which heightened the business’ expectations for what could be done with modernization technology. The business wanted to push for cutting edge and move the majority of data operations and analysis to Hadoop. Results The business’ desire for new technology outpaced IT’s ability to enable new analytics. The technical team found that it was not as easy to spin up analytics in the Hadoop environment as promised by vendors. Specifically, ETL processing times took longer to manually code on Hadoop than on traditional platforms. Key Pitfalls Encountered Lack of clear business use case Inadequate data warehouse staffing Inadequate skills for new technologies Scaling of data management activities

11 Info-Tech’s Data Warehouse Modernization FrameworkSuccessful data warehouse modernization demands tight IT-business collaboration Modernization initiatives fail when communication between the business and IT is too sporadic, or when one perspective dominates the conversation. Info-Tech’s phased approach to data warehouse modernization incorporates input from both groups at every step: Current state evaluation, combining user and technical team feedback Exploring the art-of-the-possible with business ideas and new technology Creating new initiatives as part of a compelling business case and roadmap 1 2 3 Info-Tech’s Data Warehouse Modernization Framework

12 Pick a path for enhancing your data warehouse environmentBroader Faster The rigid structure of a relational data warehouse provides reliability and consistency, but also limits what can be ingested. Emerging technologies provide a separate environment to pool a greater variety of data for later use. The relational data warehouse was not meant to handle the speed of data movement today, nor the pace at which business users want to conduct analysis. New streaming applications and parallel processing databases move data quicker than ever. Deeper Cloud When it comes to unstructured data that does not conform to the strict fields present in a data warehouse, technologies that can extract and organize data elements are useful in separating the signal from noise. As more applications and services are hosted in the cloud, your data warehouse needs to adapt to new flexibilities (i.e. increased storage capacity) and constraints (integration patterns, security guidelines) presented by cloud architecture.

13 Top Modernization BenefitseBay drives customer-insight generation by diversifying their data processing capabilities CASE STUDY Industry Source E-Commerce DataStax Situation eBay continues to be a powerhouse as an e-commerce marketplace because it is able to process massive amounts of data generated by their customers’ interactions and turn it into actionable insight. Conducting billions of read-write functions per day, eBay’s data centers need to process data at speeds and volumes that would easily overwhelm relational databases. Solution eBay expanded its data environment with Apache Cassandra™, a NoSQL database that offers linear scale performance with increase in data volumes, along with some real-time analytics. Results eBay was able to handle over 6 billion writes and 5 billion reads per day, as well as capture all the information about the products their users like, want, or own for display on a personalized “Your Favorites” page. The real-time capability enables mobile notification logging/tracking and fraud detection. Top Modernization Benefits Expanded data storage Reduced stress on core commodity servers Cost-effective scaling as data processing needs increase Enhanced capture of customer transactions Enhanced capture of customer transactions

14 Educational MaterialsUse Info-Tech’s education resources and use cases to hone in on where modernization can maximize your benefits Don’t get wrapped up in headlines and stuck with technology you don’t need. In the rush to acquire the latest and greatest tools, many organizations end up with solutions that over-promise but under-deliver because clear use cases have not been defined. Along with a clear approach, Info-Tech provides the following resources to clarify the direction of your modernization project: Info-Tech’s use case templates provide you with a starting point for your modernization initiatives, by matching popular business needs with emerging technologies in an easy-to-consume architecture diagram. Each sample comes with high-level deployment steps that can be customized to get your initiative off the ground and producing value. Educational Materials Use Case Templates Understanding the technology landscape in today’s marketing jargon is difficult. Info-Tech’s educational deck provides clear summaries of emerging technologies so your IT department can discern which solutions will best support business ideas.

15 Innovate on structure and unstructured dataInsight Tree Overarching Insight: The vast majority of your corporate data contains untapped business and innovation opportunities. Harness your unstructured data, and marry it with the structured data to generate insights that are contextual and relevant. Phase 1 Understand the known Master the known, structured portion of the data warehouse. Don’t just rip and replace. Complement the existing data warehouse by incorporating modernization technologies. Phase 2 Explore the unknown Explore the opportunities powered by the unstructured data. Combine innovative business ideas and innovative technologies to lead innovation in your industry. Phase 3 Innovate on structure and unstructured data Define solutions that use unstructured data to enhance structured data. Architect fast, current, and historical data holistically with an integration end goal in mind.

16 Use these icons to help direct you as you navigate this researchUse these icons to help guide you through each step of the blueprint and direct you to content related to the recommended activities. This icon denotes a slide where a supporting Info-Tech tool or template will help you perform the activity or step associated with the slide. Refer to the supporting tool or template to get the best results and proceed to the next step of the project. This icon denotes a slide with an associated activity. The activity can be performed either as part of your project or with the support of Info-Tech team members, who will come onsite to facilitate a workshop for your organization.

17 Info-Tech offers various levels of support to best suit your needsGuided Implementation “Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track.” DIY Toolkit “Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful.” Workshop “We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place.” Consulting “Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project.” Diagnostics and consistent frameworks used throughout all four options

18 Drive Business Innovation With a Modernized Data Warehouse Environment – project overview1. Assess the Current Data Warehouse Environment 2. Define Modernization Drivers 3. Create the Modernization Future State Best-Practice Toolkit 1.1 Understand the Current Data Warehouse Perception 1.2 Review the Current Technical Environment 2.1 Explore Business Opportunities 2.2 Review the Technology Landscape 3.1 Define Modernization Solutions 3.2 Establish a Modernization Roadmap Guided Implementations Discuss Info-Tech’s approach to modernization and the importance of business input Review business user survey results and discuss how to define the current state from an IT perspective Review IT’s current state assessment and discuss conducting business user workshops/interviews Review modernization ideas from the business; create approach for building internal IT knowledge for modernization technology Match modernization technologies to business ideas and discuss future state diagrams Review finalized initiatives, discuss costs/benefits, and build modernization roadmap Onsite Workshop Module 1: Assess the Current Environment Module 2: Explore Business Opportunities Module 3: Review the Technology Landscape Module 4: Define Modernization Solutions Module 5: Establish a Modernization Roadmap Phase 1 Outcome: Business and IT perspectives of current state Decision to move forward with modernization Phase 2 Outcome: Business-driven modernization ideas IT education on modernization technology Phase 3 Outcome: Consolidated modernization initiatives Business case and roadmap for deployment

19 Workshop overview Contact your account representative or for more information. Workshop Day 1 Workshop Day 2 Workshop Day 3 Workshop Day 4 Workshop Day 5 Activities Assess the Current Data Warehouse Environment 1.1 Discuss business user perception of the current data warehouse 1.2 Review data warehouse project history 1.3 Evaluate data warehouse maturity 1.4 Draw architecture diagrams 1.5 Review supporting data management practices Explore Business Opportunities 2.1 Review general trends and drivers in your industry 2.2 Identify primary business frustrations, opportunities, and risks 2.3 Identify business processes to target 2.4 Capture business ideas for future state Review the Technology Landscape 3.1 Identify modernization advisors 3.2 Run an internal education session for modernization technology Define Modernization Solutions 4.1 Match technologies to business ideas 4.2 Group ideas to build modernization initiatives 4.3 Create conceptual diagrams for the future state Establish a Modernization Roadmap 5.1 Validate initiatives with business 5.2 Define tangible and intangible benefits 5.3 Identify enablers and barriers 5.4 Determine preliminary activities 5.5 Evaluate costs of initiatives 5.6 Calculate modernization ROI Deliverables Data Warehouse Maturity Assessment Tool (structured + unstructured data) Data architecture diagrams Business ideas for modernization Data Warehouse Modernization Initiative Building Tool Modernization advisors appointed Data Warehouse Modernization Technology Education Deck Data Warehouse Modernization Technology Architectural Template Future state diagrams Modernization initiatives Finalized modernization initiatives Modernization business case and roadmap Initiative RACI