1 Scoping Meta Data ManagementMark W. Mosley Principal Consultant Enterprise Data World 2010 Enterprise Warehousing Solutions, Inc. © 2010 All Rights Reserved Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
2 About the Speaker Mark Mosley, CDMP, is a Principal Consultant with EWSolutions and a leading expert in enterprise information management (EIM). As a consultant, manager and data architect, Mark has established successful data governance programs, developed enterprise data models, and implemented data warehouses, BI environments and managed meta data environments (MMEs). Mark served as Development Editor for The DAMA Guide to the Data Management Body of Knowledge (DAMA-DMBOK Guide) and the DAMA Dictionary of Data Management. He is the author of the DAMA-DMBOK Framework white paper (available at and articles published by the EIM Institute and Real World Decision Support electronic newsletters. Mark is the 2009 recipient of the DAMA International Community Award. You can Mark at Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
3 EWSolutions Enterprise Information Management (EIM) strategic consulting partner and full-service systems integrator Meta Data Management, Information Architecture, Data Governance, Information Quality, Business Intelligence/Data Warehousing (BI/DW) David Marco, founder and president Large commercial, non-profit & federal government clients Award-winning implementations at the FBI and Dept. of Defense Business-driven, iterative methodologies Thought leadership: books, articles, electronic newsletters, workshops, certification training, industry surveys, professional associations Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
4 EWSolutions Clients FBI FBI Corning Cable Systems DepartmentUnited States Of Defense Transportation Command Driehaus Capital Management Thomson Consumer Electronics (RCA) Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
5 Agenda Introduction Meta Data Subject AreasMeta Data Management Drivers Meta Data Sources Repository Support Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
6 What Is Meta Data? Metadata or Meta-data or Meta Data?“Data about data” “Data about information and technology assets” Data about business data meaning Data about technical data structure, heritage and usage Data about business processes Data about information systems and applications Data about information technology infrastructure Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
7 Meta Data is Data About …Data structures Business semantics Data stewardship Reference data values Data quality metrics Data security Data movement and transformations Business intelligence reporting and analysis Application data access Documents and their information content Business processes Applications and software Legacy systems Project portfolio Technology infrastructure Enterprise architecture Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
8 Why Manage Meta Data? Meta data provides context that enables people to turn data into information, guiding analysis and decisions Meta data is the key to managing data assets Meta data is the foundation of data governance Meta data is the glue that enables enterprise integration Meta data is also a valuable enterprise asset! Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
9 Meta Data Management Standardizing, consolidating, reconciling, controlling and delivering easy access to high quality meta data Part of a comprehensive Enterprise Information Mgmt. (EIM) program Document & Content Management Data Warehousing & Business Intelligence Management Reference & Master Data Management Data Security Management Data Development Meta Data Management Data Quality Management Data Architecture Management Database Operations Management Data Governance Copyright © 2008 DAMA International DAMA-DMBOK Framework Data Stewardship Database Management Unstructured Data Reference & Master Data Warehousing & Business Intelligence Information Quality Data Governance Data Architecture Meta Data Management Information Security Management Transaction Data Business Intelligence Data Unstructured & Semi-Structured EIM Services Copyright © 2008 EWSolutions, Inc. EIM Framework Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
10 Managed Meta Data Environment (MME)Managed Meta Data Environment (MME): the architectural components, people and processes required to properly and systematically gather, retain, and disseminate meta data throughout the enterprise. David Marco Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
11 Managed Meta Data Environment (MME)Meta Data Creation Meta Data Usage Meta Data Integration Meta Data Delivery DBMS Catalog Modeling Tools ETL Tools BI Tools Sys Dev Tools Sys Mgmt Tools Spreadsheets Documents XML Msgs / Xctns Applications External Data Direct Entry Query & Browse Reports Spreadsheets Documents Web Pages Meta Data Marts BI Tools Modeling Tools Sys Dev Tools ETL Tools Sys Mgmt Tools XML Applications Meta Data Repository Meta Data Model Meta Data Control Strategy / Policy Roles & Responsibilities Organization Resources & Skills Standards Architecture Meta Model Integrity Procedures Quality Assurance Security Training Metrics Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
12 Defining Your MME RoadmapYou can’t address everything at once! Too many different kinds of meta data Too many different sources Too many different user constituencies Too many different uses You need to take a phased, incremental and iterative approach You need to prioritize and manage scope Focus first on what provides the best return Then gradually broaden your focus and services Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
13 The Zachman Framework DATA What FUNCTION How NETWORK Where PEOPLE Who TIME When MOTIVATION Why Sub - Contractor FUNCTIONING ENTERPRISE Function = Business Process Proc = Business Process I/O = Business Resources Node = Business Location Link = Business Linkage Work Flow Model People = Organization Unit Work = Work Product Cycle = Business Cycle Business Plan End = Business Objective Means = Business Strategy Proc = Application Function I/O = User Views Human Interface Architecture People = Role Work = Deliverable Time = System Event Cycle = Processing Cycle End = Structural Assertion Means = Action Assertion System Design Presentation Architecture End = Condition Means = Action Program Proc = Language Statement I/O = Control Block Node = Addresses Link = Protocols Security Architecture People = Identity Work = Job Timing Definition Time = Interrupt Cycle = Machine Cycle Rule Design End = Sub-Condition - Means = Step Function Network Organization Schedule Strategy SCOPE (CONTEXTUAL) Planner MODEL (CONCEPTUAL) Owner SYSTEM MODEL (LOGICAL) Designer TECHNOLOGY (PHYSICAL) Builder DETAILED REPRESENTATIONS (OUT – OF- CONTEXT) (OUT - OF - CONTEXT) Processes Performed Business Process Model Business Logistics System Master Schedule Important Organizations People = Major Organization Business Locations Events Significant To the Business Time = Major Business Event Business Goals and Strategy Ends/Means = Business Goal Application Architecture Distributed System Node = IS Function Link = Line Characteristics Processing Schedule Business Rule Model Technology Architecture Time = Execute Cycle = Component Cycle People = User Work = Screen Format Control Structure Ent = Business Entity Rel = Business Relationship Entity = Business Thing Ent = Data Entity Rel = Data Relationship Proc = Computer Function I/O = Data Elements/Sets Node = Hardware/Software Link = Line Specifications Ent = Segment/Table Rel = Pointer/Key Ent = Field Rel = Address Network Architecture Node = Location Site Time = Business Event Things Important To the Business Semantic Model Logical Data Model Physical Data Model Data Definition Data THE ZACHMAN FRAMEWORK FOR ENTERPRISE ARCHITECTURE TM © Copyright 1986, 2008 John Zachman, Zachman International Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
14 A Meta Data Subject Area FrameworkData Stewardship Stewards, Roles, Teams, Councils, Representation, Responsibility Assignments Data Models Entities, Relationships, Attributes, Subject Areas, Terms, Business Definitions Process Models Functions, Activities, Tasks, Steps, Roles, Inputs, Outputs, Workflow, Business Rules Business Models Divisions, Departments, Locations, Resources, Goals, CSFs, Strategies Reference Data Values Internal & External Codes, Labels, Domain Values, Meaning, Usage Rules Data Structures Databases, Files, Tables, Columns, Views, Indexes, Performance, Security Application Software Requirements, Design, Testing, UML, Java, Web Services, SOA, Wrappers, Reuse Portfolios Applications, Projects, Programs, Costs, Benefits, Status, Issues Data Quality Rules, Defects, Metrics, Key Performance Indicators, Service Level Agreements Data Integration Sources, Targets, Movement, Transformations, Lineage, Performance, Errors, EAI, Migration / Conversion Appl. Data Access Views, Queries, Screens, Reports, Messages, SQL , XML, MQ Technology Models Standards, Protocols, Products, Architecture Unstructured Data Documents, Content, Taxonomies, Ontologies, File Plans, Records Retention Business Intelligence Views, Business Definitions, Queries, Reports, Analytics, Users, Performance Legacy Systems COBOL, Restructuring, Reuse, Componentization Technology Inventory Hardware, Software, Networks, Configurations, Licenses Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
15 Data Structures Meta DataSourced from: DBMS System Catalogs Physical Data Models SQL DDL Specifications COBOL VSAM File Layouts XML Specifications Used by: Database administrators Application programmers Quality assurance testers Business professionals Data Structures Databases, Files, Tables, Columns, Views, Indexes, Performance, Security Data about: Database objects Files and fields Technical specifications --- type & length Business names and definitions Instance statistics Users, roles, groups, privileges Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
16 Data Modeling Meta DataSourced from: Modeling tools Business glossaries Used by: Business professionals Data administrators Application programmers Data Models Entities, Relationships, Attributes, Subject Areas, Terms, Business Definitions Data about: Business entities and data attributes Business relationships and rules Business concepts, terms and definitions Subject areas Semantic models (ontologies) Model versions Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
17 Data Stewardship Meta DataSourced from: Spreadsheets Extended Data Modeling Tools Direct Entry into Repository Used by: Data stewards Data administrators Data Stewardship Stewards, Roles, Teams, Councils, Representation, Responsibility Assignments Data about: Stewardship roles Stewardship appointments Data governance organizations and participants Subject area and business entity stewardship responsibility assignments Tenure (start date / end date) Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
18 Reference Data Values Meta DataSourced from: Spreadsheets Extended Data Modeling Tools Master Data Management Applications Direct Entry into Repository Used by: Business professionals Data stewards Data administrators Reference Data Values Internal and External Codes, Domain Values, Labels, Meaning, Usage Rules Data about: Internal reference data External reference data Domain values Codes, labels, business definitions Code cross-references Synonyms Usage rules Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
19 Data Integration Meta DataSourced from: ETL Tools Spreadsheets Extended Data Modeling Tools Code Direct Entry into Repository Used by: IT Professionals Data administrators Business professionals Data stewards Data Integration Sources, Targets, Transformations, Lineage, Performance, Errors, EAI, Migration / Conversion Data about: Sources and target mappings Transformations Data Lineage ETL job performance and errors EAI procedures Migrations and conversions Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
20 Business Intelligence Meta DataSourced from: BI Tools Spreadsheets Used by: Business professionals DW/BI administrators Business Intelligence Data Views, Definitions, Queries, Reports, Usage, Performance Data about: Business user views of data Business data definitions Saved queries Reports Data access and usage Query performance Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
21 Data Quality Meta Data Sourced from: Used by: Data about: Data QualityData quality profiling tools Spreadsheets Used by: Data quality administrators Database administrators Data stewards Data Quality Rules, Defects, Metrics, Key Performance Indicators, Service Level Agreements Data about: Data quality requirements and rules Data quality standards / SLAs Data quality defect statistics Data quality management metrics Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
22 Unstructured Data Meta DataSourced from: Enterprise content management (ECM) Records management tools XML Spreadsheets Used by: Data quality administrators Database administrators Data stewards Unstructured Data Documents, Content, Taxonomies, Ontologies, File Plans, Records Retention Data about: Documents - text, reports, diagrams, images, photos, video, audio recordings Paper and electronic documents / files Document “envelope” meta data – format, size, location, Document information content meta data – classifications, keywords Ontologies, taxonomies and file plans Record retention schedule Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
23 Process Modeling Meta DataSourced from: Process modeling tools Workflow design tools Application development tools Spreadsheets Used by: Business professionals Enterprise architects System analysts Process Models Functions, Activities, Tasks, Steps, Roles, Inputs, Outputs, Workflow, Business Rules Data about: Functions, activities, tasks, steps Roles and responsibilities Inputs and outputs Predecessors and antecedents Workflow logic and business rules Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
24 Application Software Meta DataSourced from: Application development tools Object oriented design tools Code libraries / SCM tools Testing tools Used by: Application developers Application Development Requirements, Design, Testing, UML, Java, Web Services, SOA, Wrappers, Reuse Data about: Business requirements UML specifications Application design specifications SOA standards Software components Component reuse Test plans, scripts, run results Defects and issues Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
25 Application Data Access Meta DataSourced from: Application development tools Object oriented design tools XML design tools Used by: Application developers Database administrators Application Data Access Views, Queries, Screens, Reports, Messages, SQL, XML, MQ Data about: Application data views Database calls and queries Application screens and reports XML file specifications Bus Messages Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
26 Legacy Systems Meta DataSourced from: Reverse engineering tools Code libraries Used by: Application developers Legacy Systems COBOL, Restructuring, Reuse, Componentization Data about: Code and application logic Program design Impact analysis Restructuring and reuse Components and wrappers Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
27 Business Modeling Meta DataSourced from: Enterprise modeling tools Spreadsheets Used by: Business leadership Business planners Enterprise architects Business Models Organizations, Roles, Locations, Resources, Goals, CSFs, Strategies Data about: Data, data usage, data sources Business functions, activities and events Roles, organizations and locations Goals, objectives, policies, standards Relationships Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
28 Portfolio Management Meta DataSourced from: Enterprise modeling tools Portfolio management tools Project management tools Spreadsheets Used by: Business planners and managers IT governance programs Program and project managers Portfolios Applications, Projects, Programs, Costs, Benefits, Status, Issues Data about: Applications Projects and programs Costs and benefits Project status Project issues Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
29 Technology Models Meta DataSourced from: Spreadsheets Used by: IT architects and managers Technical Architecture Standards, Protocols, Products, Architecture Data about: Technology standards and protocols Technology products and tools Technology infrastructure design configurations Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
30 Technology Inventory Meta DataSourced from: System management tools Spreadsheets Used by: IT architects IT infrastructure service managers Technology Inventory Hardware, Software, Networks, Configurations, Service Levels, Problem tickets Data about: Installed hardware and software Network configurations Desktop and server configurations Service levels and performance data Problem tickets Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
31 Prioritizing Meta Data Subject AreasData Stewardship Stewards, Roles, Teams, Councils, Representation, Responsibility Assignments Data Models Entities, Relationships, Attributes, Subject Areas, Business Definitions Process Models Functions, Activities, Tasks, Steps, Roles, Inputs, Outputs, Workflow, Business Rules Business Models Divisions, Departments, Locations, Resources, Goals, CSFs, Strategies Reference Data Values Internal & External Codes, Labels, Domain Values, Business Definitions Data Structures Databases, Files, Tables, Columns, Views, Indexes, Performance, Security Application Software Requirements, Design, Testing, UML, Java, Web Services, SOA, Wrappers, Reuse Portfolios Applications, Projects, Programs, Costs, Benefits, Status, Issues Data Quality Rules, Defects, Metrics, Key Performance Indicators, Service Level Agreements Data Integration Sources, Targets, Movement, Transformations, Lineage, Performance, Errors, EAI, Migration / Conversion Appl. Data Access Views, Queries, Screens, Reports, Messages, SQL , XML, MQ Technology Models Standards, Protocols, Products, Architecture Unstructured Data Documents, Content, Taxonomies, Ontologies, File Plans, Records Retention Business Intelligence Views, Business Definitions, Queries, Reports, Analytics, Users, Performance Legacy Systems COBOL, Restructuring, Reuse, Componentization Technology Inventory Hardware, Software, Networks, Configurations, Licenses Phases: 1 2 3 4 5 6 7 Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
32 Some Meta Data Business DriversBusiness Intelligence & Data Warehousing Data Stewardship & Governance Application Change Impact Analysis Application Integration & Reuse Enterprise Architecture Master Data Management Content Management & Records Management Portfolio Management Systems Management Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
33 Data Warehousing & Business IntelligenceData Stewardship Stewards, Roles, Teams, Councils, Representation, Responsibility Assignments Data Models Entities, Relationships, Attributes, Subject Areas, Business Definitions Reference Data Values Internal & External Codes, Labels, Domain Values, Business Definitions Data Structures Databases, Files, Tables, Columns, Views, Indexes, Performance, Security Data Quality Rules, Defects, Metrics, Key Performance Indicators, Service Level Agreements Data Integration Sources, Targets, Movement, Transformations, Lineage, Performance, Errors, EAI, Migration / Conversion Business Intelligence Views, Business Definitions, Queries, Reports, Analytics, Users, Performance Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
34 Impact Analysis, Integration & ReuseData Models Entities, Relationships, Attributes, Subject Areas, Business Definitions Process Models Functions, Activities, Tasks, Steps, Roles, Inputs, Outputs, Workflow, Business Rules Data Structures Databases, Files, Tables, Columns, Views, Indexes, Performance, Security Application Software Requirements, Design, Testing, UML, Java, Web Services, SOA, Wrappers, Reuse Data Integration Sources, Targets, Movement, Transformations, Lineage, Performance, Errors, EAI, Migration / Conversion Appl. Data Access Views, Queries, Screens, Reports, Messages, SQL , XML, MQ Business Intelligence Views, Business Definitions, Queries, Reports, Analytics, Users, Performance Legacy Systems COBOL, Restructuring, Reuse, Componentization Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
35 Enterprise ArchitectureData Stewardship Stewards, Roles, Teams, Councils, Representation, Responsibility Assignments Data Models Entities, Relationships, Attributes, Subject Areas, Business Definitions Process Models Functions, Activities, Tasks, Steps, Roles, Inputs, Outputs, Workflow, Business Rules Business Models Divisions, Departments, Locations, Resources, Goals, CSFs, Strategies Portfolios Applications, Projects, Programs, Costs, Benefits, Status, Issues Technology Models Standards, Protocols, Products, Architecture Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
36 Business Drivers Strategic Partner & Systems IntegratorIntelligent Business Intelligencesm
37 Meta Data Sources Strategic Partner & Systems IntegratorIntelligent Business Intelligencesm
38 Repository Product SupportStrategic Partner & Systems Integrator Intelligent Business Intelligencesm
39 A Meta Data Subject Area FrameworkData Stewardship Stewards, Roles, Teams, Councils, Representation, Responsibility Assignments Data Models Entities, Relationships, Attributes, Subject Areas, Business Definitions Process Models Functions, Activities, Tasks, Steps, Roles, Inputs, Outputs, Workflow, Business Rules Business Models Divisions, Departments, Locations, Resources, Goals, CSFs, Strategies Reference Data Values Internal & External Codes, Labels, Domain Values, Business Definitions Data Structures Databases, Files, Tables, Columns, Views, Indexes, Performance, Security Application Software Requirements, Design, Testing, UML, Java, Web Services, SOA, Wrappers, Reuse Portfolios Applications, Projects, Programs, Costs, Benefits, Status, Issues Data Quality Rules, Defects, Metrics, Key Performance Indicators, Service Level Agreements Data Integration Sources, Targets, Movement, Transformations, Lineage, Performance, Errors, EAI, Migration / Conversion Appl. Data Access Views, Queries, Screens, Reports, Messages, SQL , XML, MQ Technology Models Standards, Protocols, Products, Architecture Unstructured Data Documents, Content, Taxonomies, Ontologies, File Plans, Records Retention Business Intelligence Views, Business Definitions, Queries, Reports, Analytics, Users, Performance Legacy Systems COBOL, Restructuring, Reuse, Componentization Technology Inventory Hardware, Software, Networks, Configurations, Licenses Strategic Partner & Systems Integrator Intelligent Business Intelligencesm
40 Questions Strategic Partner & Systems IntegratorIntelligent Business Intelligencesm
41 Enterprise Warehousing Solutions, Inc.EWSolutions, Inc. 15 Spinning Wheel Road, Suite 330 Hinsdale, IL 60521 Office Fax Mark Mosley Cell Enterprise Warehousing Solutions, Inc. © 2010 All Rights Reserved Strategic Partner & Systems Integrator Intelligent Business Intelligencesm