1 PowerShell and DSC Empowers DevOps and the CloudDan Stolts Chief Technology Strategist US DX North East – Audience Blog: ITProGuru.com Title: PowerShell and DSC Empowers DevOps and the Cloud Abstract: Dan Stolts from Microsoft and ITGuru.com comes with a full session on Desired State Configuration! DSC is a new management platform in Windows PowerShell that enables deploying and managing configuration data for software services and managing the environment in which these services run. DSC provides a set of Windows PowerShell language extensions, new Windows PowerShell cmdlets, and resources that you can use to declaratively specify how you want your software environment to be configured. It also provides a means to maintain and manage existing configurations.Practical applications Following are some example scenarios where you can use built-in DSC resources to configure and manage a set of computers (also known as target nodes) in an automated way: Enabling or disabling server roles and features Managing registry settings Managing files and directories Starting, stopping, and managing processes and services Managing groups and user accounts Deploying new software Managing environment variables Running Windows PowerShell scripts Fixing a configuration that has drifted away from the desired state Discovering the actual configuration state on a given node Name of Presenter: Dan Stolts Bio of Presenter (max 500 spaces): Dan Stolts “ITProGuru” is a technology expert who is a master of systems management and security. He is Chief Technology Strategist for Microsoft, owns several businesses and is a published author. Reach him on his primary blog or He is proficient in many datacenter technologies (Windows Server, System Center, Virtualization, Cloud, Etc) and holds many certifications including MCT, MCITP, MCSE, TS, etc. Dan is currently specializing in system management, virtualization and cloud technologies. Dan is and has been a very active member of the user group community. Dan is an enthusiastic advocate of technology and is passionate about helping others. See more at: aka.ms/ book Specializations: Cloud, Virtualization, Windows Server @ITProGuru
2
3 The consequences of inefficiencyBuild 2015 12/20/2017 9:22 PM The consequences of inefficiency IT drives business success! High IT performance correlates with strong business performance, helps boost productivity, market share and profit. 40 % … of implementations end up getting reworked because they don’t meet the users’ original requirements Outperforming teams are collaborate extensively with their counterparts 54 % more likely to … of development budgets for software, IT staff and external professional services will be consumed by poor requirements 41 % 26.7% No executive support 56.7% Cultural inhibitors 43.3% Fragmented processes Collaboration blockers … for companies that try to adapt their existing tools for DevOps practices 80 % failure rate … IT Ops Developers CIOs 70 % to reduce IT costs Would increase risk and accelerate business agility o f Business IT decision makers is still unfamiliar with the term DevOps 6 1 in In the real world, there are real consequences if you are unable to deliver high-quality software quickly or build the wrong thing to begin with: 40% of implementations end up getting reworked because they don’t meet the users’ original requirements The average cost of one hour downtime of a customer-facing app is calculated at dollars per hour – and this does not take into account the damage to reputation, which can be even greater. Fixing such production issues takes on average 200 minutes per incident Three quarters of development teams have adopted Agile methodologies today, enabling them to develop faster. While this is a great number, it does not help if a development team is Agile but deployment still takes weeks or months because IT Ops is perceived as not being Agile These are just 3 very high-level examples but all the data we have today points toward the same conclusion – this is about more than just frustration or minor delays. Lack of collaboration between dev and ops can have substantial impact on a company’s bottom line and success It takes on average 200 minutes to diagnose and repair a production issue Agile methodologies have adopted 3/4 of teams DevOps was being initiated by more development teams than IT Ops teams by about a 40% to 33% margin Responding to ongoing needs for efficiency and growth Always keeping all systems safe and secure dual goals The average hourly cost of infrastructure failure is $100,000 per hour A bug caught in production ends up costing than if the same bug was found earlier in the development cycle 100x more © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
4 List of DevOps PracticesOne Marketing Template 12/20/2017 List of DevOps Practices Infrastructure as Code (IaC) Continuous Integration Automated Testing Continuous Deployment Release Management App Performance Monitoring Load Testing & Auto-Scale Availability Monitoring Change/Configuration Management Feature Flags Automated Environment De-Provisioning Self Service Environments Automated Recovery (Rollback & Roll-Forward) Hypothesis Driven Development Testing in Production Fault Injection Usage Monitoring/User Telemetry © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
5 Why DSC VS GP? (Better than GP)Declaring a DSC configuration is PowerShell based to help admins define configuration in the way they want. Reusability of Configuration Data Can be used anywhere - on-premise, in a public or in a private Cloud environment. Detects and fixes Configuration Drifts Support for integration with other tools Declaring a DSC configuration is PowerShell based - You define configuration programmatically(script) which comes to admins naturally and help them define configuration in the way they want. Reusability of Configuration Data - You can separate the configuration data from the logic of your configuration so that you can reuse your configuration data for different resources, nodes, and configurations, see Can be used anywhere -DSC can be used on-premise, in a public or in a private Cloud environment. You just need either Windows Server 2012 R2 or Windows 8.1 and local administrator permissions to execute the DSC PowerShell scripts Support for integration with other tools -You can integrate DSC with any Microsoft or non-Microsoft solutions as long as you can execute a PowerShell script on the target system. E.g. using DSC within the Windows Azure Pack portal in conjunction with SMA.
6 PowerShell Command-Line Tool for Azurehttps://azure.microsoft.com/downloads/ Windows Install (under PowerShell) CLI is there too if you want to play with it
7 Windows Management Framework 5.0Windows Management Framework 5.0 includes updates to Windows PowerShell, Windows PowerShell Desired State Configuration (DSC), Windows Remote Management (WinRM), Windows Management Instrumentation (WMI). Updates… Just Enough Administration (JEA) Creating Custom Types using PowerShell Classes Improvements in PowerShell Script Debugging Improvements in Desired State Configuration (DSC) Audit PowerShell Usage using Transcription and Logging Software Discovery, Install and Inventory with PackageManagement PowerShell Module Discovery, Install and Inventory with PowerShellGet PowerShell Script Discovery, Install and Management with PowerShellGet New and updated cmdlets based on community feedback Information Stream Generate PowerShell Cmdlets based on OData Endpoint Network Switch Management with PowerShell Software Inventory Logging (SIL) https://www.microsoft.com/en-us/download/details.aspx?id=50395
8 DSC Resource Kit – Anniversary ReleasexActiveDirectory xAdcsDeployment xCertificate xChrome xComputerManagement xDFS xDhcpServer xDnsServer xDscDiagnostics xDSCResourceDesigner xExchange xFailOverCluster xFirefox xNetworking xPhp xPSDesiredStateConfiguration xRobocopy xSqlPs xSQLServer xStorage xTimeZone xWebDeploy xWindowsUpdate To see a list of all released DSC Resource Kit modules, go to the PowerShell Gallery … display all modules tagged as DSCResourceKit. You can also enter a module’s name in the search box in the upper right corner of the PowerShell Gallery to find a specific module. # To list all modules (WMF 5.0)of the DSC Resource Kit Find-Module -Tag DSCResourceKit # To list all DSC resources (WMF 5.0) from all sources Find-DscResource https://blogs.msdn.microsoft.com/powershell/2016/05/18/dsc-resource-kit-anniversary-release/
9 Tip: Must Read Book Amazon… The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win Paperback – October 16, 2014 The book is available on Audio Book; you can likely get it for free - if it’s your first time accepting a book from a friend. If you do not have an Audible Account, you get two free books when signing up for your free account. This is the BEST IT Book I have ever read.
10 What is DevOps… http://www.itproguy.com/devops-practices/Infrastructure as Code (IaC) Continuous Integration (CI) Automated Testing Application Performance Monitoring/Management (APM) Continuous Deployment (CD) Release Management Configuration Management DevOps Fundamentals Advanced Monitoring Capacity Management Feature Flags Self-Service Environments Automated Recovery (Rollback & Roll- Forward) Hypothesis Driven Development Testing in Production – partial user base Fault Injection Usage Monitoring/Telemetry A/B Testing (aka canary testing) Gene Kim’s “3 ways” The First Way emphasizes the performance of the entire system, as opposed to the performance of a specific silo of work or department — this as can be as large a division (e.g., Development or IT Operations) or as small as an individual contributor (e.g., a developer, system administrator). The Second Way is about creating the right to left feedback loops. The goal of almost any process improvement initiative is to shorten and amplify feedback loops so necessary corrections can be continually made The Third Way is about creating a culture that fosters two things: continual experimentation, taking risks and learning from failure; and understanding that repetition and practice is the prerequisite to mastery.
11 Traditional vs modern dev-opsCategory Traditional Modern DevOps Quality of code check-ins Unknown Validated through unit tests Environment Creation/Configuration Manual Automated Deployment Frequency 1-2 months (or less frequent) Deploy whenever needed, including several per day App Deployment Process Requires meetings and planning Push-button deployment Deployment validation Monitoring Minimal to none Health and Performance monitoring Dev and Ops relationship Blame culture Culture of trust
12 DSC – Definition “Standard Space management to manage heterogeneous networks containing different types of Servers connected with Clients of different form factors.” Windows/Linux etc. Servers Windows/iOS etc Clients PC/Servers/Tablet/Phones etc. Form Factors
13 Why DSC? Need of an agent to “Make It So”Life in the Cloud… Rapid change, at scale with, constant failures Life with non-Registered (mobile) Devices Different (including non-Windows) type of devices needs to be handled Requirement Need of an agent to “Make It So” Want to compare Actual and Expected States easily Need a language to express custom desired state easily Need
14 What is “Make it so”? With DSC we just define in configuration –“Hey, you are a web browser, this is what you should look like. Get to it, stay that way.” Logic of making configuration in that way lies with ‘Local Configuration Manager’ on Client Machines. Making logic on client side enable us to control device specific behavior implementation
15 One Stop Solution to Manage EnterpriseHow DSC help? One Stop Solution to Manage Enterprise Option of correcting configuration drift when it occurs, or just report on configuration drift, to ley it know to admins that it has occurred Cloud BYOD - non-Win Devices Laptop (VPNs) Desktop/ Server
16 One Stop Solution to Manage EnterpriseHow DSC help? One Stop Solution to Manage Enterprise Allows you to describe the desired state of your environment by using this new power shell syntax Option of correcting configuration drift when it occurs, or just report on configuration drift, to ley it know to admins that it has occurred Cloud BYOD - non-Win Devices Laptop (VPNs) Desktop/ Server
17 One Stop Solution to Manage EnterpriseHow DSC help? One Stop Solution to Manage Enterprise Allows you to describe the desired state of your environment by using this new power shell syntax Option of correcting configuration drift when it occurs, or just report on configuration drift, to ley it know to admins that it has occurred Cloud BYOD - non-Win Devices Laptop (VPNs) Desktop/ Server
18 Already Supported in GPWhere DSC can be used? Install or remove server roles and features Manage registry settings Manage files and directories Start, stop, and manage processes and services Manage local groups and user accounts Install and manage packages such as .msi and .exe Manage environment variables Discovering the actual configuration state on a given node Fix a configuration that has drifted away from the desired state Already Supported in GP
19 How DSC works? 3 Phase Model Authorizing Phase Staging Phase“Make It So” Phase 3 Phase Model
20 How DSC works? Authoring PhaseAdvanced task dependencies – talk about “Dependency Type” options enabled only for multi-machine job when Dependencies is clicked in the Task tab. Studio Help -> Contents -> “Working With Jobs” -> “Key Concepts for WTT Jobs” -> Parameters In this phase Admin creates the DSC configuration locally, through PowerShell or by third party languages/tools. The output is one or more MOF (Management Object Format) files, the format which is consumable by DSC.
21 How DSC works? Authoring Phase Staging PhaseDC / Pull Server Authoring Phase Staging Phase Advanced task dependencies – talk about “Dependency Type” options enabled only for multi-machine job when Dependencies is clicked in the Task tab. Studio Help -> Contents -> “Working With Jobs” -> “Key Concepts for WTT Jobs” -> Parameters In this phase DSC data (MOF files) is staged for Deployment – Push Model – On DC Pull Model – Dedicated Web Server (IIS)
22 DSC data is either pulled or pushed to the “Local Configuration Store” and contains the current, previous and the desired (DSC) state configuration. How DSC works? DC / Pull Server Authoring Phase Staging Phase “Make it So” Phase Advanced task dependencies – talk about “Dependency Type” options enabled only for multi-machine job when Dependencies is clicked in the Task tab. Studio Help -> Contents -> “Working With Jobs” -> “Key Concepts for WTT Jobs” -> Parameters The configuration then gets parsed and the relevant (WMI) provider implements the change and “makes it so”. Enterprise
23 DSC data is either pulled or pushed to the “Local Configuration Store” and contains the current, previous and the desired (DSC) state configuration. How DSC works? DC / Pull Server Authoring Phase Staging Phase “Make it So” Phase Advanced task dependencies – talk about “Dependency Type” options enabled only for multi-machine job when Dependencies is clicked in the Task tab. Studio Help -> Contents -> “Working With Jobs” -> “Key Concepts for WTT Jobs” -> Parameters The configuration then gets parsed and the relevant (WMI) provider implements the change and “makes it so”. Enterprise
24 Power Shell Extension Support type definitions for eg . Configuration types New language/constructs specific to authoring of the Configuration scripts. Eg. configuration MyWebsite { node ("WebServer1", "WebServer2") WindowsFeature IIS Ensure = "Present" Name = "Web-Server“ }
25 Configuration Staging AreaDSC Push Model Authoring Phase Staging Phase “Make it So” Phase Local Configuration Store Configuration Staging Area (Contains DSC data) Parser and Dispatcher Imperative Providers
26 DSC Pull Model Pull Server (Contains DSC data and Modules)Authoring Phase Staging Phase “Make it So” Phase Local Configuration Store Pull Server (Contains DSC data and Modules) Parser and Dispatcher Imperative Providers
27 When to use which model? Pull Model Push ModelRecommended Pull Model Push Model Suitable for new enterprises where network is dynamic in nature Suitable for legacy enterprises where network is static in nature Scalable Solution – Dedicated server ensure that response can be scaled specially in case of Cloud where VMs can be created on demand All files resides in DC, increased DSC activity may increase load on already burdened DC. Non-Windows Devices – Pull Server being just webserver, all devices can pull configuration from Web URL. No current support for non-Windows Device
28 As Dan the developer, I can easily author a DSC script reusing DSC resources from a public or private gallery, so that I can quickly and easily write DSC scripts with very little custom code.
29 Author a DSC script WITHOUT DSC Resource
30 … reusing DSC resource/s - II can Find DSC resources from Gallery by Search Term/Name/Tag I have a rich set of resources to use from Day 1 that replace my RM actions [see next slide]
31 Rich set of DSC resources replacing RM actionsNo. RM Tool Tag RM Action RM Customer Usage Analog DSC Resource 1 Application Pool IIS Create/Config/Start/Remove App Pool 1. Very Frequent IISWebConfiguration 2 Web Application Create/Configure/Remove Web Application WebDeploy 3 Website Create/Configure/Remove/Start/Stop/Restart Website 4 Database Deployer SQL Create/Drop Database SQLQuery 5 Execute SQL Script 6 Windows System Common Windows IO File 7 XCopy Deployer Copy File or Folder 8 MSI Deployer Deploy MSI Package 9 Virtual Directory Create/Configure/Remove Virtual Directory 2. Frequent 10 Remove Web Application 11 Create/Delete File or Folder 12 Command Line Runner Run Command Line [as User] Process 13 Windows Services Manager Create/Start/Stop/Config/Restart Service Service 14 MTM Automated Tests Manager Test Run Tests ?
32 Existing PS Modules/DSC resources ported to galleryI can port PS modules & DSC resources from Technet Script center to the gallery.
33 reusing DSC resource/s - IIWITH DSC Resource I can install multiple resources & modules that my DSC script depends on before deploying to a node in push mode.
34 PowerShell Gallery
35 … from a public or private gallery - II can install DSC resources from the Gallery
36 … from a public or private gallery - III can choose the URI of the public/private gallery or the path [like a build drop path] that I want to install the module from.
37 … from a public or private gallery - IIIUser can choose to not install module from the public/private gallery but to directly copy all the required bits directly from a build drop path.
38
39 Deep dive into Script Authoring phaseGenerate MOFs using Modules+Script+StaticConfig along with app bits DSC Pull Server Checkin Modules+Script+ StaticConfig along with app source Source Control Build Nodes Author DSC script Azure VMs Azure VMs Copy “Package” DTL Push Server Uses modules from different sources P0 – MOFs need to be machine agnostic. P1 – MOFs need to be tag/role based. OR P0 – Pull needs to have DSC files and generate MOFs in the nodes. Private Gallery Public Gallery Self-Authored
40 Deep dive into Script Authoring phaseGenerate MOFs using Modules+Script+StaticConfig along with app bits DSC Pull Server Checkin Modules+Script+ StaticConfig along with app source Source Control Build Nodes Author DSC script Azure VMs Azure VMs Copy “Package” DTL Push Server Uses modules from different sources Private Gallery Public Gallery Self-Authored Apply Dynamic Config? [“Property Bags”]
41 What does a Script look like?“MyConfiguration.webserver” Configuration Block… Within a Configuration block, you can do anything that you normally could in a PowerShell function Resource Block… (AKA: Compilation Job) Node block… https://azure.microsoft.com/en-us/documentation/articles/automation-dsc-overview/
42 DSC configuration Step-by-StepPrerequisites An Azure Automation account. see Azure Run As Account. An Azure Resource Manager VM (not Classic) running Windows Server 2008 R2 or later. see Create your first Windows virtual machine in the Azure portal Prerequisites (above) Create Test Config Importing a configuration into Azure Automation Viewing Configuration Compile Configuration (MOF file) Viewing a compilation Viewing Node Configuration Onboard Azure ARM VM for Mgmt Assign Node Configuration Save the file as TestConfig.ps1. https://azure.microsoft.com/en-us/documentation/articles/automation-dsc-getting-started/
43 Azure Automation DSC LifeCycle
44 Session VS VM (or VDI) Massive VM Overhead Single OSMicrosoft Ignite 2015 Session VS VM (or VDI) 12/20/2017 9:22 PM Single OS All Workloads OS, Apps, Config, Mgmt EACH Workload OS Apps Apps Apps Apps Massive VM Overhead (Disk, Disk IO, Memory, CPU, OS, Licenses, Density, The works) Apps Apps Apps Apps workloads Apps Apps Apps Apps Apps Apps Apps Apps Application(s) Operating System Hardware Virtualization Application(s) Operating System Hardware Virtualization VM or VDI Based Computing Parent OS Hardware / Hypervisor Level Session Based Computing © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
45 All Advantages of VM & SessionMicrosoft Ignite 2015 Containers 12/20/2017 9:22 PM Single OS All Workloads EACH Platform OS App App App App App App App App App All Advantages of VM & Session Isolation, Little overhead App App workloads App App Container / Image App Container Engine App App Linux VM Container / Image Container Container Application(s) Operating System Hardware Virtualization Container Engine Container Engine Operating System Hardware Virtualization Parent OS Hardware / Hypervisor Level Container Based Density Advantages Container Based Computing VM Isolation © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
46 Container Run-time On-PremisesMicrosoft Ignite 2015 12/20/2017 9:22 PM Container Run-time On-Premises Windows Server Container(s) Docker Container Hyper-V Container(s) Docker Engine Host Operating System Virtual machine(s) Linux Hyper-V Hypervisor Physical Server Windows Host Operating System © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
47 Container Run-time CloudMicrosoft Ignite 2015 12/20/2017 9:22 PM Container Run-time Cloud Windows Coming Docker Container (Linux Now) Windows Server Container(s) Hyper-V Container(s) Azure Container(s) ACS … Containers As A Service Docker Compatible Virtual machine(s) Docker Engine Hyper-V Hypervisor Linux Windows Windows Hyper-V Hypervisor Cloud Provider Windows Host Operating System © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
48 Azure is an open cloud DevOps Management Applications App FrameworksClients DevOps Management Orchestration Applications PaaS App Frameworks & Tools Databases & Middleware Infrastructure
49 Free Resources for DevOps Practices12/20/2017 9:22 PM Free Resources for DevOps Practices Optimize your DevOps practices & tools: Get started on your DevOps journey: aka.ms/devops Download the Forrester Infrastructure-as-Code whitepaper: Complexity kills. Automate with Infra as code: aka.ms/iac_tlp Accelerate your application delivery lifecycle Technical resources for Practitioners: Get access to free online training, evals and HOLs: aka.ms/devopsmva Join the Community conversations: Use #TalkDevOps on Twitter © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
50 Resources Group Policy vs DSC DSC Blog
51 What’s Next? How can you stay up to date?Microsoft Ignite 2015 12/20/2017 9:22 PM What’s Next? Preview of Windows Server Containers Preview of Hyper-V Containers How can you stay up to date? Follow me on Follow my Blog Windows Container’s site https://msdn.microsoft.com/virtualization/windowscontainers © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.