vSphere Design Best Practices
Apply industry-accepted best practices to design reliable high-performance datacenters for your business needs
Brian Bolander Christopher Kusek
professional expertise distilled
P U B L I S H I N G BIRMINGHAM - MUMBAI
vSphere Design Best Practices Copyright © 2014 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: May 2014
Production Reference: 1200514
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-78217-626-8 www.packtpub.com
Cover Image by Abhishek Dhir (
[email protected])
Credits Authors Brian Bolander
Project Coordinator Melita Lobo
Christopher Kusek Proofreader Reviewers
Simran Bhogal
Andy Grant Muhammad Zeeshan Munir Prasenjit Sarkar Commissioning Editor Rebecca Youe Acquisition Editor
Indexers Mariammal Chettiyar Tejal Soni Graphics Valentina Dsilva Abhinash Sahu
Meeta Rajani Production Coordinator Content Development Editor
Alwin Roy
Sharvari Tawde Cover Work Technical Editors Rosmy George Manal Pednekar Copy Editor Laxmi Subramanian
Alwin Roy
About the Authors Brian Bolander spent 13 years on active duty in the United States Air Force. A
veteran of Operation Enduring Freedom, he was honorably discharged in 2005. He immediately returned to Afghanistan and worked on datacenter operations for the US Department of Defense (DoD) at various locations in southwest Asia. Invited to join a select team of internal consultants and troubleshooters responsible for operations in five countries, he was also the project manager for what was then the largest datacenter built in Afghanistan. After leaving Afghanistan in 2011, he managed IT operations in a DoD datacenter in the San Francisco Bay Area. His team was responsible for dozens of multimillion dollar programs running on VMware and supporting users across the globe. He scratched his adrenaline itch in 2011 when he went back "downrange", this time directing the premier engineering and installation team for the DoD in Afghanistan. It was his privilege to lead this talented group of engineers who were responsible for architecting virtual datacenter installations, IT project management, infrastructure upgrades, and technology deployments on VMware for the entire country from 2011 to 2013. Selected as a vExpert in 2014, he is currently supporting Operation Enduring Freedom as a senior virtualization and storage engineer. He loves his family, friends, and rescue mutts. He digs science, technology, and geek gear. He's an audiophile, a horologist, an avid collector, a woodworker, an amateur photographer, and a virtualization nerd.
Christopher Kusek had a unique opportunity presented to him in 2013, to take
the leadership position responsible for theater-wide infrastructure operations for the war effort in Afghanistan. Leveraging his leadership skills and expertise in virtualization, storage, applications, and security, he's been able to provide enterprise-quality service while operating in an environment that includes the real and regular challenges of heat, dust, rockets, and earthquakes. He has over 20 years' experience in the industry, with virtualization experience running back to the pre-1.0 days of VMware. He has shared his expertise with many far and wide through conferences, presentations, #CXIParty, and sponsoring or presenting at community events and outings whether it is focused on storage, VMworld, or cloud. He is the author of VMware vSphere 5 Administration Instant Reference, Sybex, 2012, and VMware vSphere Performance: Designing CPU, Memory, Storage, and Networking for Performance-Intensive Workloads, Sybex, 2014. He is a frequent contributor to VMware Communities' Podcasts and vBrownbag, and has been an active blogger for over a decade. A proud VMware vExpert and huge supporter of the program and growth of the virtualization community, Christopher continues to find new ways to outreach and spread the joys of virtualization and the transformative properties it has on individuals and businesses alike. He was named an EMC Elect in 2013, 2014, and continues to contribute to the storage community, whether directly or indirectly, with analysis and regular review. He continues to update his blog with useful stories of virtualization and storage and his adventures throughout the world, which currently include stories of his times in Afghanistan. You can read his blog at http://pkguild.com or more likely catch him on Twitter; his twitter handle is @cxi. When he is not busy changing the world, one virtual machine at a time or Facetiming with his family on the other side of the world, he's trying to find awesome vegan food in the world at large or somewhat edible food for a vegan in a war zone.
About the Reviewers Andy Grant works as a technical consultant for HP Enterprise Services. Andy's
primary focus is datacenter infrastructure and virtualization projects across a number of industries including government, healthcare, forestry, financial, gas and oil, and international contracting. He currently holds a number of technical certifications including VCAP4/5-DCA/DCD, VCP4/5, MCITP:EA, MCSE, CCNA, Security+, A+, and HP ASE BladeSystem. Outside of work, he enjoys backcountry camping, playing action pistol sports (IPSC), and spending time being a goof with his son.
Muhammad Zeeshan Munir is a freelance ICT consultant and solution architect. He established his career as a system administrator in 2004 and since then has acquired and executed many successful projects in multimillion ICT industries. With more than 10 years of experience, he now provides ICT consultancy services to different clients in Europe. He also works as a system consultant for Qatar Computing Research Institute. He regularly contributes to different wikis and produces various video tutorials mostly about different technologies, including VMware products, Zimbra e-mail services, OpenStack, and Red Hat Linux, which can be found at http://zee.linxsol.com/system-administration. When he is doing nothing, he likes to travel around and speak languages such as English, Urdu, Punjabi, and Italian.
Prasenjit Sarkar is a senior member of the technical staff in VMware Service
Provider Cloud R&D where he provides architectural oversight and technical guidance to design, implement, and test VMware's Cloud datacenters. You can follow him on Twitter at @stretchcloud. He is an author, R&D guy, and a blogger focusing on virtualization, cloud computing, storage, networking, and other enterprise technologies. He has more than 10 years' expert knowledge in R&D, professional services, alliances, solution engineering, consulting, and technical sales with expertise in architecting and deploying virtualization solutions and rolling out new technology and solution initiatives. His primary focus is on the VMware vSphere infrastructure and public cloud using VMware vCloud Suite. One of his other areas of focus is to own the entire life cycle of a VMware-based IaaS (SDDC), especially vSphere, vCloud Director, vShield Manager, and vCenter Operations. He was one of the VMware vExperts in 2012 and 2013 and well known for his acclaimed virtualization blog http://stretch-cloud.info. He holds certifications from VMware, Cisco, Citrix, Red Hat, Microsoft, IBM, HP, and Exin. Prior to joining VMware, he has served other fine organizations such as Capgemini, HP, and GE as a solution architect and infrastructure architect. I would like to thank and dedicate this book to my family. Without their endless and untiring support, this book would not have been possible.
www.PacktPub.com Support files, eBooks, discount offers and more
You might want to visit www.PacktPub.com for support files and downloads related to your book. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at
[email protected] for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks. TM
http://PacktLib.PacktPub.com Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can access, read and search across Packt's entire library of books.
Why Subscribe? •
Fully searchable across every book published by Packt
•
Copy and paste, print and bookmark content
•
On demand and accessible via web browser
Free Access for Packt account holders
If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate access.
Instant Updates on New Packt Books
Get notified! Find out when new books are published by following @PacktEnterprise on Twitter, or the Packt Enterprise Facebook page.
Table of Contents Preface 1 Chapter 1: Virtual Data Center Design 5
Virtual Data Center design principles 6 Best practices 7 Designing Virtual Data Center 8 VMware vCenter components 10 Choosing a platform for your vCenter server 11 Using the vCenter server appliance 12 Sizing your vCenter server 13 Choosing your vCenter database 14 vSphere clustering – HA and DRS 16 Host considerations 16 Network considerations 17 Storage considerations 19 Cluster considerations 19 Admission control 20 Summary 21
Chapter 2: Hypervisor Design
23
ESXi hardware design 23 CPU considerations 24 Memory and NUMA considerations 26 Virtual NUMA (vNUMA) considerations 28 Considerations for Java virtual machines (JVMs) 28 Network interface card considerations 29 Hypervisor storage components 31 Stateless host design 31 Scale-Up and Scale-Out designs 33 Summary 35
Table of Contents
Chapter 3: Storage Design
37
Chapter 4: Network Design
49
Chapter 5: Virtual Machine Design
59
Storage protocols 37 Fibre Channel (FC) 38 iSCSI 38 Fibre Channel over Ethernet (FCoE) 39 NFS 39 Virtual machine filesystems 40 VMFS 40 RDM 41 NFS 42 Designing storage 43 Local storage considerations 43 Datastore and cluster design 44 VMware storage features 45 Configurable storage features 46 SIOC 46 Configuring the multipathing path selection policy 47 Provisioning 47 Summary 48 Introducing vSphere switching 50 vNetwork Standard Switches (vSS) 50 vNetwork Distributed Switches (vDS) 50 Common networking design considerations 52 Best practices to implement in your design 53 Implementing Network I/O Control (NIOC) 53 Providing network redundancy, resiliency, and throughput 54 Physical switches and adapters 55 ESXi host hardware 56 Considerations when using IP storage 56 Separating your traffic 57 Validating your storage configurations 57 Summary 57 Virtual machine resources and templates Best practices for virtual machines and templates Over-allocation can be a painful proposition
Physical to virtual migrations Multi-vCPU considerations Shares, limits, and reservations
[ ii ]
59 60
61
61 62 62
Table of Contents
Configuring shares 63 Limits 63 Reservations 63 Causes of virtual machine performance problems 64 CPU performance issues 64 Memory performance issues 65 Storage performance issues 66 Network performance issues 68 Summary 68
Chapter 6: Business Critical Applications
69
Chapter 7: Disaster Recovery and Business Continuity
81
Appendix A: vSphere Automation Tools
93
Special considerations for tier 1 applications Starting simple Workload profiles Designing for Microsoft Exchange Considering Microsoft SQL server HA, DRS, and vMotion considerations Resource considerations Ensuring high availability and application resiliency Enabling CPU Hot Add in vSphere client The vCenter operations manager dashboard Other business critical application resources Summary The benefits of disaster avoidance Designing backup strategies Designing for RPO and RTO requirements Choosing replication technologies VMware vSphere Replication Storage-based replication Mixing replication solutions Planning and testing the DR strategy Summary
Introducing VMware vCenter Orchestrator Introducing VMware vCloud Automation Center Unified IT service catalog Advanced Service Designer Multivendor interoperability Designing for automation and XaaS Summary [ iii ]
69 70 71 71 72 73 75 76 77 78 79 79 82 83 85 86 86 87 88 89 91 93 95 96 96 97 98 98
Table of Contents
Appendix B: Certification Resources
VMware Certification Roadmap Associate exams Professional exams Advanced professional exams Expert-level defense VMware vExpert program Evangelist path Customer path VMware Partner Network (VPN) path Recommended reading, blogs, and podcasts Exam and study tips Summary
99
100 101 101 101 102 102 102 103 103 103 104 104
Index 105
[ iv ]
Preface Welcome to vSphere Design Best Practices, an easy-to-read guide full of hands-on examples of real-world design best practices. Each topic is explained and placed in the context of virtual datacenter design. This book is all about design principles. It isn't about operations or administration; instead, we focus on designing virtual datacenters to meet business requirements and leveraging vSphere to get us to robust and highly available solutions. In this book, you'll learn how to utilize the features of VMware to design, architect, and operate a virtual infrastructure using the VMware vSphere platform. Readers will walk away with a sense of all the details and parameters there are for a design that has been well thought out. We'll examine how to customize your vSphere infrastructure to fit business needs and look at specific use cases for live production environments. Readers will become familiar with new features in Version 5.5 of the vSphere suite and how these features can be leveraged in design. Readers will walk away with confidence as they will know what their next steps are towards accomplishing their goals, whether that be a VCAP-DCD certification or a sound design for an upcoming project.
What this book covers
Chapter 1, Virtual Data Center Design, gives you an insight into the design and architecture of the overall datacenter, including the core components of vCenter and ESXi. Chapter 2, Hypervisor Design, dives into the critical components of the datacenter by providing the best practices for hypervisor design.
Preface
Chapter 3, Storage Design, explains one of the most important design points of the datacenter—storage. This chapter focuses attention on the design principles related to the storage and storage protocols. Chapter 4, Network Design, peels back the layers of networking by focusing on designing flexible and scalable network architectures to support your virtual datacenter. Chapter 5, Virtual Machine Design, covers what it takes to inspect your existing virtual machines and provides guidance and design principles to correctly size and deploy VMs. Chapter 6, Business Critical Applications, breaks through the barriers and concerns associated with business critical applications, allowing business requirements to translate into a successful and stable deployment. Chapter 7, Disaster Recovery and Business Continuity, considers the many parameters that make up a DR/BC design, providing guidance to make sound decisions to ensure a well-documented and tested disaster recovery policy. Appendix A, vSphere Automation Tools, briefly discusses the value of automation tools such as vCenter Orchestrator (vCO) and vCloud Automation Center (vCAC), their differences, and use cases for your design and implementation. Appendix B, Certification Resources, gives guidance on the VMware certification roadmap and lists resources for the Data Center Virtualization (DCV) track.
What you need for this book
As this book is technical in nature, the reader should possess an understanding of the following concepts: • Storage technologies (SAN, NAS, Fibre Channel, and iSCSI) • Networking—both physical and virtual • VMware vSphere products and technologies such as: °°
Hypervisor basics
°°
vMotion and Storage vMotion
°°
Cluster capabilities such as High Availability (HA) and Distributed Resource Scheduler (DRS)
[2]
Preface
Who this book is for
This book is ideal for those who desire a better understanding of how to design virtual datacenters leveraging VMware vSphere and associated technologies. The typical reader will have a sound understanding of VMware vSphere fundamentals and would have been involved in the installation and administration of a VMware environment for more than two years.
Conventions
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning. Code words in text are shown as follows: "VMware Communities Roundtable podcast hosted by John Troyer (@jtroyer)." New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: "You can adjust your guests' vNUMA configuration on a per-VM basis via Advanced Settings in order to adjust vNUMA to meet your needs." Warnings or important notes appear in a box like this.
Tips and tricks appear like this.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of. To send us general feedback, simply send an e-mail to
[email protected], and mention the book title via the subject of your message. If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.
[3]
Preface
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub. com/submit-errata, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.
Piracy
Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy. Please contact us at
[email protected] with a link to the suspected pirated material. We appreciate your help in protecting our authors, and our ability to bring you valuable content.
Questions
You can contact us at
[email protected] if you are having a problem with any aspect of the book, and we will do our best to address it.
[4]
Virtual Data Center Design The datacenter has quite simply become one of the most critical components to organizations today. Some datacenters, either owned by large enterprises or leased by large service providers, can contain great amounts of bleeding-edge technologies, tens of thousands of servers, huge network pipes from multiple carriers, and enough contingency mechanisms to keep their systems running for months in case of a disaster. Other datacenters can consist of just a small handful of servers and off-the-shelf networking gear stuffed in a closet. Every organization is a bit different and has its own unique set of requirements in order to provide IT services to its users. As system administrators, engineers, and architects, we need to be able to take those requirements, sometimes even search for or define them, and design a solution to meet or exceed those requirements. As datacenters have evolved, we've seen a paradigm shift towards virtualization. This is due to a number of factors including performance, availability, management, and recoverability. This book aims to help call attention to these factors and explains how to address them in your Virtual Data Center designs. The Virtual Data Center has several key components such as compute, storage, networking, and management. We will be covering the following topics in this chapter: • Core design principles for the Virtual Data Center • Best Practices for Virtual Data Center design • How best practices change over time • Virtual Data Center design scenarios • vCenter design including the Linux-based vCenter Server Appliance (vCSA) • vSphere clustering, HA, and DRS • A consideration of the other components of the vCloud Suite
Virtual Data Center Design
Virtual Data Center design principles
VMware vSphere is a leading platform for virtualization with many components that make up the VMware vCloud Suite including vCenter Server, ESXi hypervisor, and Site Recovery Manager (SRM). Through these products, the vSphere platform, with proper design characteristics, enables an IT infrastructure to provide services, availability, flexibility, recoverability, and performance that its customers require. Apart from knowledge of the aforementioned VMware products (or perhaps any third-party solutions being considered), there are a few key principles to get started with our designs. While there are numerous methodologies out there, this is a framework on which you can base your process consisting of three basic phases—Conceptual Design, Logical Design, and Physical Design, as shown in the following diagram:
Logical Design
Physical Design
Conceptual Design
It is important to remember that while the process is important, the focus of this book is about the design decisions. If you typically subscribe to a different framework or methodology, that is OK, as the design decisions discussed throughout the book will hold true regardless of your process and methodology in most cases. First, create a Conceptual Design by gathering and analyzing business and application requirements, and then document the risks, constraints, and assumptions. Once all of the information is gathered and compiled, you should be able to formulate a high-level end goal, which is ultimately a vision of the solution. For example, you could have requirements to provide 99.99 percent uptime to an application, guarantee 2500 IOPS via shared storage, 25 GHz CPU, and 48 GB of RAM. You could be constrained by a budget of $100,000 among various risks and assumptions. So, conceptually, you'll have a high-level idea that you'll need a certain class of compute, storage, and network to support that type of application.
[6]
Chapter 1
You certainly don't need to know the specifics at this point but you should have a vision and direction of the goal. Next, building from the conceptual design, we'll begin formulating the Logical Design. To do this, we'll begin laying down the logical representations of the infrastructure components. Why just logical representations? Continuing with the above example, we don't know what servers we need or how many, we just know that we need servers. We don't know what storage array we need or what size, we just know that we need shared storage, and so on. We need to map each of the requirements to a logical solution within the complete design. This also begins to flush out dependencies between components and services. While it would be great if we could jump straight from the conceptual design right to the complete solution, we should be taking smaller steps towards our goal. This step is all about putting ideas down on paper and thinking through this Logical Design before we start procuring hardware and building up the datacenter by trial and error. Finally, we transition our Logical Design into a Physical Design. While the previous steps allow us to be somewhat theoretical in our work, this phase absolutely requires us to do our homework and put specific parameters around our design. For example, we determine we need six servers with dual 8-core CPUs, 128 GB RAM, 15 TB of array-based fiber channel attached storage, 4 x 10 Gbe NICs, 2 x 8 Gb Fiber Channel HBAs, and so on. In most cases, you'll have this detail down to the manufacturer of each component as well. This physical design should consist of all the infrastructure components that will be required to support the requirements gathered in the Conceptual Design phase. For example, a complete Physical Design might include a vSphere Network Design, Storage Design, Compute Design, Virtual Machine (VM) Design, and Management Design. As you can imagine, the documentation for a physical design can get quite large and time consuming, but this is a critical step. In summary, remember that the three design phases are additive and dependent upon one another. In order to have a successful Physical Design, both the Conceptual and Logical Designs must be solid. Document as much as is reasonable in your designs and even include reasoning behind choices. Many choices seem obvious while making them; however, when it is time to review and deploy a design months later, it may not be as obvious as to why certain choices were made.
Best practices
We hear the term Best practices everywhere but what does it really mean? Do we always adhere to best practices? What happens to best practices as technology evolves, or as your business evolves? These are great questions and ones that many people don't seem to ask. There are three scenarios when addressing a design. [7]
Virtual Data Center Design
You may ignore best practices, strictly follow best practices, or employ a combination of the two. The third scenario is the most common and should be your target. Why? The simplified answer is that there are certainly times when best practices aren't in fact best for your design. A great example is the general best practice for vSphere where the defaults for just about everything in the environment are acceptable. For example, take storage multipathing. VMware default is for fixed path and if best practice is default, you may consider not modifying this; experience shows that often you'll see performance gains switching to Round Robin (see VMware KB1011340 for details). While changing this default may not be a best practice, then again, "Best Practice" might not fit your needs in the real world. Choose what works for you and the constraints of your environment. Best practices also seem to have a shelf life much longer than intended. A favorite example is that the best size for vSphere datastore is 500 GB. While this sizing best practice was certainly true at one time, vSphere has improved how it handles datastore sizing using features such as SCSI Reservation, which enables much higher VM per datastore density. Since vSphere 5.0, we've been able to create 64 TB datastores and the best practice is more about evaluating parameters such as number of VMs, number of VMDKs per VM, required IOPS, SLAs, fault domains, and restore capabilities. Many a times the backend disk capabilities will determine the ideal datastore size. Yet, many customers continue to use 500 GB as their standard datastore size. There are many more examples but the bottom line is that we should use best practices as a guideline and then do our homework to determine how to apply them to our designs. Along with the best practices and design framework we've already discussed, it is also important to define and adhere to standards throughout your Virtual Data Center. This includes items such as naming conventions for ESXi hosts and VMs as well as IP address schemes, storage layout, network configurations, and security policies. Adhering to these standards will make understanding the design much easier as well as help as changes are introduced into the environment after deployment.
Designing Virtual Data Center
As previously mentioned VMware's Virtual Data Center (vDC) is composed of several key components: vCenter, ESXi hypervisor, CPU, storage, networking, and virtual machines. Many datacenters will have additional components but these form the foundation of typical vDC. Note that it is possible to operate, albeit in a limited capacity, without vCenter in the management layer. We find that it is really the core of the vDC and enables many features we seem to take for granted, such as vMotion, DRS (Distributed Resource Scheduling), and HA (High Availability).
[8]
Chapter 1
Our vDCs will also consist of several constructs that we will leverage in our designs to provide organization and operability. These include the following: • Datacenters • Clusters • Resource pools • Folders • Tags By using these constructs, we can bring consistency into complex designs. For example, by using folders and tags, we can organize VMs, hosts, and other objects so we can easily find them, report on them, or even perform operations such as backup based on folders or tags. Tags are a new feature in vSphere 5.1 and above that work much like tagging in other applications. It allows us to put arbitrary tags on objects in our vDC for organizational purposes. Folders present a challenge when we have objects that could belong in multiple folders (which isn't possible). Many administrators use a combination of folders and tags to both organize their inventory as well manage it.
The following screenshot is a specific example of how folders can be used to organize our vDC inventory:
It is important to define a standard folder hierarchy as part of your design and include tags into that standard to help organize and manage your infrastructure.
[9]
Virtual Data Center Design
Another facet to our designs will be how we might employ multiple vCenters and how we manage them. Many organizations utilize multiple vCenters because they want to manage Production, QA, and Test/Dev, separately. Others have vCenters distributed geographically with a vCenter managing each local infrastructure. In any case, with vSphere 5.5, multiple vCenter management has become much easier. We have what is called Linked Mode as an option. This allows us to provide a single pane of access to resources connecting to multiple vCenters—think shared user roles/groups and being able to access objects from any vCenters that are linked through a single console. However, with the vSphere web client included with vSphere 5.1 and above, we are able to add multiple vCenters to our view in the client. So, if your requirements are to have less roles and groups to manage, then Linked Mode may be the best route. If your requirements are to be able to manage multiple vCenters from a single interface, then the standard vSphere web client should meet your needs without adding the complexity of Linked Mode. Next, we have Single sign-on (SSO). This is a new feature introduced in vSphere 5.1 and improved in vSphere 5.5. Previously, SSO had a difficult installation process and some difficult design decisions with respect to HA and multisite configurations. With SSO in vSphere 5.5, VMware has consolidated down to a single deployment model, so we're able to much more easily design HA and multisite configurations because they are integrated by default. Finally, there is the subject of SSL certificates. Prior to vSphere 5.0, it was a best practice (and a difficult task) to replace the default self-signed certificates of the vSphere components (vCenter, ESX, and so on) with CA-signed certificates. That remains the case, but fortunately, VMware has developed and released the vCenter Certificate Automation tool, which greatly reduces the headaches caused when attempting to manually generate and replace all of those SSL certificates. Although not a steadfast requirement, it is certainly recommended to include CA-signed certificates in your designs.
VMware vCenter components There are four primary components to vCenter 5.1 and 5.5:
• Single sign-on: This provides identity management for administrators and applications that interact with the vSphere platform • vSphere web client: This is the service that provides the web-based GUI for administrators to interact with vCenter and the objects it manages • vCenter inventory service: This acts as a cache for vCenter managed objects when accessed via the web client to provide better performance and reduce lookups to the vCenter database [ 10 ]
Chapter 1
• vCenter server: This is the core service of vCenter, which is required by all other components There are also several optional components and support tools, which are as follows: • vSphere client: This is the "legacy" C#-based client that will no longer be available in the future versions of vSphere (although still required to interact with vSphere Update Manager) • vSphere Update Manager (VUM): This is a tool used to deploy upgrades and patches to ESXi hosts and VMware tools and virtual hardware version updates to VMs • vSphere ESXi Dump Collector: This is a tool used mostly to configure stateless ESXi hosts (that is, have no local storage) to dump VMkernel memory to a network location and then pull those logs back into vCenter • vSphere Syslog Collector: This is a tool similar to the Dump Collector that allows ESXi system logs to be redirected to a network location and be accessed via vCenter • vSphere Auto Deploy: This is a tool that can provision ESXi hosts over the network and load ESXi directly into memory allowing for stateless hosts and efficient provisioning • vSphere Authentication Proxy: This is a service that is typically used with Auto Deploy so that hosts can be joined to a directory service, such as MS Active Directory, without the need to store credentials in a configuration file At the onset of your design, these components should be accounted for (if used) and integrated into the design. For small environments, it is generally acceptable to combine these services and roles on the vCenter while larger environments will generally want to separate these roles as much as possible to increase reliability by reducing the fault domain. This ensures that the core vCenter services have the resources they need to operate effectively.
Choosing a platform for your vCenter server
Now that we've reviewed the components of vCenter, we can start looking at making some design decisions. The first design decision we'll need to make relative to vCenter is whether you'll have a physical or virtual vCenter. VMware's own recommendation is to run vCenter as a VM. However, just as with best practices, we should examine and understand why before making a final decision. There are valid reasons for having a physical vCenter and it is acceptable to do. However, if we make the appropriate design decisions, a virtual vCenter can benefit from all the flexibility and manageability of being a VM without high amounts of risk. [ 11 ]
Virtual Data Center Design
One way to mitigate risks is to utilize a Management Cluster (sometimes called a management pod) with dedicated hosts for vCenter, DNS, Active Directory, monitoring systems, and other management infrastructure. This benefits us in a few ways such as being able to more easily locate these types of VMs if vCenter is down. Rather than connecting to multiple ESXi hosts via Secure Shell (SSH) or using the vSphere Client to connect to multiple hosts in order to locate a domain controller to be manually powered on, we're able to identify its exact location within our management pod. Using a management cluster is sometimes not an option due to the cost of a management pod, separate vCenter server for management, and the cost of managing a separate cluster. In smaller environments, it isn't such a big deal because we don't have many hosts to go searching around for VMs. But, in general, a dedicated cluster for management infrastructure is recommended. A second way to mitigate risks for vCenter is to use a product called vCenter Heartbeat. VMware vCenter Heartbeat provides automated and manual failover and failback of your vCenter Server and ensures availability at the application and service layer. It can restart or restore individual services while monitoring and protecting the Microsoft SQL Server database associated with vCenter Server, even if it's installed on a separate server. For more information on vCenter Heartbeat, you can go to the product page at http://www.vmware.com/products/vcenterserver-heartbeat/. In general, if choosing a virtual vCenter, do everything possible to ensure that the vCenter VM has a high priority with resources and HA restarts. An option would be to set CPU and memory resource shares to High for the vCenter VM. Also, do not disable DRS or use host affinity rules for the vCenter VM to try to reduce its movement within the vDC. This will negate many of the benefits of having a virtual vCenter.
Using the vCenter server appliance
Traditionally, this decision hasn't really required much thought on our part. The vCenter Server Appliance (vCSA or sometimes called vCVA or vCenter Virtual Appliance) prior to vSphere 5.5 was not supported for production use. It also had a limit of managing five hosts and 50 VMs. However, with vSphere 5.5, vCSA is fully supported in production and can manage 1000 hosts and 10000 VMs. See vSphere 5.5 Configuration Maximums at http://www.vmware.com/pdf/vsphere5/r55/ vsphere-55-configuration-maximums.pdf.
[ 12 ]
Chapter 1
So why would we want to use an appliance versus a full Windows VM? The appliance is Linux-based and comes with an embedded vPostgres, which will support up to 100 hosts and 3000 VMs. In order to scale higher, you need to use an external Oracle database server. Environments can save on Windows and MS SQL licensing by leveraging the vCSA. It is also a much simpler deployment. There are a few caveats, however, in that VMware Update Manager (VUM) still requires a separate Windows-based host with an MS SQL database as of vSphere 5.5. If you are running Horizon View, then you'll need a Windows server for the View Composer service and a similar situation exists if you are running vCloud Director. The general idea, though, is that the vCSA is easier to manage and is a purpose-built appliance. While vCSA may not be for everyone, but I would encourage you to at least consider it for your designs.
Sizing your vCenter server
If you're using the vCSA, this is less of an issue but there are still guidelines for disk space and memory as shown in the following table. Note that this only applies to vCSA 5.5 and above as the vCSA prior to 5.5 is only supported with a maximum of five hosts and 50 VMs. VMware vCenter Server Appliance Hardware
Requirement
Disk storage on the host machine
The vCenter Server Appliance requires at least 7 GB of disk space, and is limited to a maximum size of 80 GB. The vCenter Server Appliance can be deployed with thin-provisioned virtual disks that can grow to the maximum size of 80 GB. If the host machine does not have enough free disk space to accommodate the growth of the vCenter Server Appliance virtual disks, vCenter Server might cease operation, and you will not be able to manage your vSphere environment.
Memory in the VMware vCenter Server Appliance
• Very small (≤ 10 hosts, ≤ 100 virtual machines): This has a size of at least 4 GB. • Small (10-100 hosts or 100-1000 virtual machines): This has a size of at least 8 GB. • Medium (100-400 hosts or 1000-4000 virtual machines): This has a size of at least 16 GB. • Large (≥ 400 hosts or 4000 virtual machines): This has a size of at least 24 GB. Source: VMware KB 2052334
[ 13 ]
Virtual Data Center Design
If you are choosing a Windows-based vCenter, you need to size your machine appropriately based on the size of your environment. This includes CPU, memory, disk space, and network speed/connectivity. The number of clusters and VMs within a vCenter can absolutely affect performance. This is due to DRS calculations that must be made and more objects mean more calculations. Take this into account when estimating resource requirements for your vCenter to ensure performance and reliability.
It is also recommended to adjust the JVM (Java Virtual Machine) heap settings for the vCenter Management Web service (tcServer), Inventory Service, and Profile-driven Storage Service based on the size of deployment. Note that this only affects vCenter 5.1 and above. The following VMware KB articles have up-to-date information for your specific vCenter version and should be referenced for vCenter sizing: vCenter version
VMware KB article number
5.0
2003790
5.1
2021202
5.5
2052334
Choosing your vCenter database
The vCenter database is the location where all configuration information, along with some performance, task, and event data, is stored. As you might imagine, the vCenter DB can be a critical component to your virtual infrastructure. But it also can be disposable—although this is becoming a much less adopted strategy than it was a couple of years ago. In some environments that weren't utilizing features like the Virtual Distributed Switch (VDS) and didn't have a high reliance on the stored event data, we saw that vCenter could be quickly and easily reinstalled from scratch without much of an impact. In today's vDC's, however, we see more widespread use of VDS and peripheral VMware technologies that make the vCenter DB much more important and critical to keep backed up. For example, the VDS configuration is housed in the vCenter DB, so if we were to lose the vCenter DB, it becomes difficult and disruptive to migrate those hosts and their VMs to a new vCenter.
[ 14 ]
Chapter 1
There are two other components that require a DB in vCenter and those are VUM and SSO. SSO is a new component introduced in vSphere 5.1 and then completely rewritten in vSphere 5.5. If you haven't deployed 5.1 or 5.5 yet, I'd strongly encourage you to go to 5.5 when you're ready to upgrade due to this re-architecture of SSO. In vSphere 5.1, the SSO setup requires manual DB configuration with SQL authentication and JDBC SSL, which caused many headaches and many support tickets with VMware support during the upgrade process. We have the following choices for our vCenter DB: • MS SQL Express • MS SQL • IBM DB2 • Oracle The different versions of those DBs are supported depending on the version of vCenter you are deploying, so check the VMware Product Interoperability Matrixes located at http://partnerweb.vmware.com/comp_guide/sim/interop_ matrix.php. The MS SQL Express DB is used for small deployments—maximum of five hosts and 50 VMs—and is the easiest to set up and configure. Sometimes, this is an option for dev/test environments as well. Next, we have probably the most popular option, that is, MS SQL. MS SQL can either be installed on the vCenter itself, but it is recommended to have a separate server dedicated to SQL. Again, sometimes in smaller deployments, it is OK to install the SQL DB on the vCenter server, but the general best practice is to have that dedicated database server for better protection, flexibility, and availability. With vCenter 5.5, we now have broader support of Windows Failover Clustering (formerly Microsoft Cluster Services or MSCS), which enhances our options for the availability of the vCenter DB. Last, we have Oracle and IBM DB2. These two options are common for organizations that already have these databases in use for other applications. Each database type has its own benefits and features. Choose the database that will be the easiest to manage while providing the features that are required. In general, any of the database choices are completely capable of providing the features and performance of vCenter.
[ 15 ]
Virtual Data Center Design
vSphere clustering – HA and DRS
Aside from vCenter itself, HA and DRS are two features that help unlock the true potential of virtualization. This combination of automatic failover and workload balancing result in more effective and efficient use of resources. The mechanics of HA and DRS are a full topic in and of itself, so we'll focus on how to design with HA and DRS in mind. The following components will be covered in this section: • Host considerations • Networking design considerations • Storage design considerations • Cluster configuration considerations • Admission control
Host considerations
Providing high availability to your vSphere environment means making every reasonable effort to eliminate any single point of failure (SPoF). Within our hosts, we can make sure we're using redundant power supplies, ECC memory, multiple I/O adapters (such as NICs and HBAs), and using appropriate remote monitoring and alerting tools. We can also distribute our hosts across multiple racks or blade chassis to guard against the failure of a rack or chassis bringing down an entire cluster. So not only is component redundancy important, but it is also important to look at the physical location. Some other items related to our hosts are using identical hardware. Although not always possible, such as hosts that have different lifecycles, we should make an effort to be consistent in our host hardware within a cluster. Aside from simplifying configuration and management, hardware consistency reduces resource fragmentation. For example, if we have hosts with Intel E5-2640 CPUs and E5-2690 CPUs, the cluster will become unbalanced and result in fragmentation. HA prepares for a worst-case scenario and plans for the largest host in a cluster to fail resulting in more resources being reserved for that HA event. This results in a lower efficiency in resource utilization. The next consideration with HA is the number of hosts in your design. We typically look for an N+1 configuration for our cluster. N+1 represents the number of hosts we need to service our workload (N) plus one additional host in case of a failure. Some describe this as having a "hot spare", but in the case of vSphere, we're typically using all of the hosts to actively serve workload. [ 16 ]
Chapter 1
In some cases, it can be acceptable to actually reserve a host or hosts for failover. This is truly a hot spare scenario where those designated hosts sit idle and are used only in the case of an HA event. This is an acceptable practice, but it isn't the most efficient use of resources.
VMware vSphere 5.5 supports clusters of up to 32 hosts and it is important to understand how HA reserves resources based on cluster size. HA essentially reserves an entire hosts' worth of resources and then spreads that reservation across all hosts in the cluster. For example, if you have two hosts in a cluster, 50 percent of the total cluster resources are reserved for HA. To put it another way, if one host fails, the other host needs to have enough resource capacity to service the entire workload. If you have three hosts in a cluster, then 33 percent of the resources are reserved in the cluster for HA. You may have picked up on the pattern that HA reserves )N1) resources for the cluster where N is equal to the number of hosts in the cluster. As we increase the number of hosts in a cluster, we see a smaller percentage of resources reserved for HA that result in higher resource utilization. One could argue that larger clusters increase complexity but that is negated by the benefits of higher utilization. The important takeaway here is to remember that these reserved resources need to be available to the hosts. If they are being used up by VMs and an HA event occurs, it is possible that VMs will not be restarted automatically and you'll experience downtime. Last, with respect to hosts, is versioning. It is absolutely recommended to have all hosts within a cluster at the same major and minor versions of ESXi, along with the same patch level. Mixed-host clusters are supported but because there are differences in HA, DRS, and other features, functional and performance issues can result. If different host versions are required, it is recommended to separate those hosts into different clusters, that is, a cluster for 5.0 hosts and a cluster for 5.5 hosts. For a VMware HA percentage Admission Control Calculator see
http://thinkingloudoncloud.com/calculator/vmware-ha-admissioncontrol-calculator/.
Network considerations
Specific to HA, there are several design considerations regarding the network. Even though the dependency HA had on DNS has been removed in vSphere 5.0 and above, it is still a good practice to ensure all hosts and VMs can be resolved through DNS. Also, Spanning Tree Protocol (STP) on network switches can take a significant amount of time to start passing traffic on a port. [ 17 ]
Virtual Data Center Design
STP is a protocol that runs on network switches to prevent network loops.
For example, if a host is rebooted, STP takes approximately 50 seconds to recalculate the topology on its connected switch ports, and while that calculation is occurring the host does not have network connectivity. This can cause HA to report the host as dead and VMs will not start up on that host. This can also trigger an isolation response due to the datastore heartbeat. In either case, this can be prevented by enabling PortFast on those ports on the connected switches. Some switch vendors may call the feature something different, but the idea of PortFast is that STP will allow traffic to pass on those ports while it completes its calculations. HA relies on the management network of the hosts. This brings with it several design considerations. Each host should be able to communicate with all other hosts' management interfaces within a cluster. Most of the time, this is not an issue because our network designs typically have all of our hosts' management VMkernel ports on the same layer 2 subnet, but there are scenarios where this is not the case. Therefore, it is important to make sure all hosts within a cluster can communicate properly to ensure proper HA functionality and prevent network partitions. Along these same lines, we should be designing our networks so that we have the fewest possible number of hops between hosts. Additional hops, or hardware segments, can cause higher latency and increase points of failure. Because of the criticality of the management VMkernel port for proper HA functionality, it is imperative that we design in redundancy for this network. This means having more than one physical NIC (pNIC) in our hosts assigned to this network. If there is a failure in the management network, we still may get host isolation since HA won't have a means to test connectivity between vSphere servers. When standardizing your network configuration, you should use consistent names for objects such as Port Groups, VLANs, and VMkernel ports. Inconsistent use of names across our hosts can result in VMs losing network connectivity after a vMotion or even preventing vMotion or an HA restart. This, of course, results in increased downtime.
[ 18 ]
Chapter 1
Storage considerations
HA, as previously mentioned, also relies on datastore heartbeats to determine if an HA event has occurred. Prior to vSphere 5.0, we only had the network to determine whether a host had failed or become isolated. Now we can use datastores to aid in that determination enhancing our HA functionality. Datastore connectivity is inherently important—after all, our VMs can't run if they can't connect to their storage, right? So, in the case where we are utilizing a storage array, practices such as multipathing continue to be important and we need to make sure our hosts have multiple paths to our storage array whether that is iSCSI, NFS, FC, or FCoE. vCenter will automatically choose two datastores to use for heartbeats. However, we do have the capability to override that choice. A scenario where this makes sense is if we have multiple storage arrays. It would be recommended to have a heartbeat datastore on two different arrays. Note that we can have up to five heartbeat datastores. Otherwise, the algorithm for choosing the heartbeat datastores is fairly robust and normally does not need to be tampered with. Another consideration is that if we are using all IP-based storage, we should try to have separate physical network infrastructure to connect to that storage to minimize the risk of the management network and storage network being disrupted at the same time. Last, wherever possible, all hosts in a cluster should have access to the same datastores to maximize the ability of isolated hosts to communicate via the datastore heartbeat.
Cluster considerations
Of the many configurations we have within a cluster for HA, Host Isolation Response seems to be often neglected. While we previously talked about how to make our management networks robust and resilient, it is still important to decide how we want our cluster to respond in the event of a failure. We have the following three options to choose from for the Host Isolation Response: • Leave Powered On: VMs will remain running on their host even though the host has become isolated • Power Off: VMs will be forcibly shutdown as if pulling the power plugs from a host • Shutdown: Using VMware tools, this option attempts to gracefully shutdown the VM and after a default timeout period of 300 seconds will then force the power off
[ 19 ]
Virtual Data Center Design
In many cases, the management network may be disrupted but VMs are not affected. In a properly architected environment, host isolation is a rare occurrence, especially with a fiber channel SAN fabric as datastore heartbeats will be out-ofband of the network. If the design requires IP storage, which uses the same network infrastructure as the management network, then the recommended option is to shut down during isolation as it is likely that a disruption in the management network will also affect access to the VM's datastore(s). This ensures that another host can power on the affected VMs. Another HA feature is VM and application monitoring. This feature uses VMware tools or an application agent to send heartbeats from the guest OS or application to the HA agent running on the ESXi host. If a consecutive number of heartbeats are lost, HA can be configured to restart the VM. The recommendation here is to utilize this feature and decide ahead of time how tolerant of heartbeat failures you wish to be.
Admission control
Admission Control (AC) is another great feature of HA. Because AC can prevent VMs from being powered on during a normal operation or restarted during an HA event, many administrators tend to turn AC off. HA uses AC to ensure that sufficient resources exist in a cluster and are reserved for VM recovery during an HA event. In other words, without AC enabled, you can overcommit your hosts to the point that if there was to be an HA event, you may find yourself with some VMs that are unable to power on. Let's take another look at a previous example where we have three hosts in a cluster. HA reserves 33 percent of that cluster for failover. Without AC enabled, we can still utilize that 33 percent reserve which puts us at risk. With AC, we are unable to utilize those reserved resources, which mitigates that risk of overprovisioning and ensures we'll have enough resources to power on our VMs during an HA event. When Admission Control is enabled, we can choose from the following three policies: • Host Failures Cluster Tolerates (default): This reserves resources based on the number of hosts for which we want to tolerate failures. For example, in a 10 host cluster, we could want to tolerate two host failures, so instead of the default one host failures the cluster tolerates (remember the )N1) formula), we would be allowing two host failures to tolerate in the cluster for failover. HA actually uses something called a "slot size" when determining resource reservations. Slot sizes are calculated on a worst-case basis; thus, if you have VMs with differing CPU and memory reservations, the slot size will be calculated from the VM with the highest reservations. This could greatly skew the slot size and result in more resources being reserved for failover than necessary. [ 20 ]
Chapter 1
• Percentage of Cluster Resources Reserved: This reserves the specified percentage of CPU and memory resources for failover. If you have a cluster where VMs have significantly different CPU and memory reservations or hosts with different CPU and memory capacities, this policy is recommended. This setting must be revisited as the cluster grows or shrinks. • Specify a Failover Host: This designates a specific host or hosts as a hot spare. The general recommendation is to use the percentage of cluster resources policy for Admission Control because it offers maximum flexibility. Design the percentage reserved to represent the number of host failures to be tolerated. For example, in a four host cluster, if you want to tolerate one host failure your percentage of failures tolerated is 25 percent, and if you wanted to tolerate two host failures it is 50 percent. If there will be hosts in the cluster of different sizes, be sure to use the largest host(s) as your reference for determining the percentage to reserve. Similarly, if the Specify a Failover Host policy is used, we must specify hosts that have CPU and memory equal to or larger than the largest nonfailover host in the cluster. If the Host Failures Cluster Tolerates policy is to be used, then make every reasonable attempt to use similar VM resource reservations across the cluster. This is due to the slot size calculation mentioned earlier.
Summary
In this chapter, we learned about the design characteristics of the Virtual Data Center and the core components of the VMware vSphere platform. We also discussed best practices and design considerations specifically around vCenter, Clustering, and HA. In the next chapter, we'll dive into the best practices of design of the ESXi hypervisor.
[ 21 ]
Hypervisor Design This chapter begins a series of chapters regarding the architecture of specific virtual datacenter components. The hypervisor is a critical component and this chapter will examine various best practices for hypervisor design. We will cover the following topics in this chapter: • ESXi hardware design including CPU, memory, and NICs • NUMA, vNUMA, and JVM considerations • Hypervisor storage components • Stateless host design • Scale-Up and Scale-Out designs
ESXi hardware design
Over the past 15 years we've seen a paradigm shift in server technology from mainframe and proprietary systems such as PowerPC, Alpha, and Itanium to commodity x86 architecture. This shift, in conjunction with x86 virtualization, has yielded tremendous benefits and has allowed us both flexibility and standardization with our hardware designs. If we take a look at the vSphere Hardware Compatibility List (HCL), we only see two CPU manufacturers: Intel and AMD (Advanced Micro Devices). The VMware HCL can be found at http://www.vmware.com/resources/compatibility/search.php. Between these two manufacturers, however, there are a wide range of CPUs to choose from and VMware does a great job of supporting new CPUs as they are released and developing new technologies that further enhance the capabilities of x86 virtualization.
Hypervisor Design
CPU considerations
Naturally, one of the first decisions we find ourselves making when designing our hosts is which chip manufacturer and which CPU will go into our hosts. As mentioned in Chapter 1, Virtual Data Center Design, standardization goes a long way and our host designs are no different. However, there are certainly instances in which we may choose different CPUs for our hosts. It remains a best practice to keep the same CPU model consistent throughout a cluster. If we find ourselves in a situation where we need different CPU models within a cluster, such as adding new hosts to an existing cluster, we have a great feature called Enhanced vMotion Compatibility (EVC). As long as we're running CPUs from the same chip manufacturer (that is all Intel or all AMD), we can mask CPU features to a lowest common denominator to ensure vMotions can occur between different chip models. For example, without EVC, we could not vMotion a VM from a host with an Intel Xeon 5500-series CPU to a host with the latest Intel Xeon E5-2600 CPU. With EVC this is completely possible and works very well. For more information on the EVC requirements, see VMware KB 1003212. CPUs have many parameters to look at when we're choosing the best one(s) for our host design. The following parameters are the most common factors in determining the appropriate CPU for a host: • Cores/threads • Clock speed • Cache • Power draw (watts) • vSphere feature support • Memory support • Max CPUs supported (2-way, 4-way, 6-way, and so on) Each of these parameters can be integral to a design while we may not use the others. We still need to consider the effects each one may have on our designs. Provided we have a good conceptual design, and thus proper requirements, we should be able to determine how many CPU resources we will need. If we have a requirement for eight cores per socket or 12 GHz CPU per host or maybe more commonly a density of 60 VMs per host is specified. We can then profile the workload to determine the proper range of CPUs to use. A range is used because cost is typically always an influencing factor. Initially, we may come up with a range and then research cost to narrow down our selection.
[ 24 ]
Chapter 2
Continuing with the example where we require 60 VMs per host, there is an important metric to look at when considering consolidation ratios: the ratio of vCPUs to pCPUs denoted as vCPU:pCPU. A vCPU is a virtual CPU that is presented to a VM while the pCPU represents a physical CPU in the host system. Furthermore, with technologies such as hyperthreading, pCPU can represent a logical processor. For instance, an Intel Xeon E5-2640 CPU has eight cores and hyperthreading enabled, which yields 16 pCPUs because each core can execute two threads. Each thread represents a logical processor (pCPU) and thus 8 x 2 = 16 logical processors or 16 pCPUs. VMware Best Practices for Performance with Hyper-threading can be read about in more detail at http://www.vmware.com/pdf/Perf_Best_Practices_ vSphere5.0.pdf. Sometimes we have project requirements that allow us to start with a specific goal of an overall vCPU:pCPU ratio or specific ratios for different parts of the infrastructure. From there, we can choose a CPU and server platform that can accommodate those requirements. Let's get back to that 60 VMs per host example, and for this we will assume that these are all general purpose VMs with no special requirements and an average of 1.5 vCPUs per VM because of a mixture of one vCPU and two vCPU machines. Our design targets a vCPU: pCPU ratio of 2:1. The math is as follows: 1.5 x 60 = 90 vCPUs. Therefore, we'll require at least 45 pCPUs per host to achieve our 2:1 target (90:45). An eight-socket server with six-core CPUs would yield 48 pCPUs and would accommodate the needs for this design. A quad socket configuration with 12-core CPUs would also yield 48 pCPUs. There are many ways to make the math work and other considerations such as using high availability configurations, additional capacity for growth that we will discuss as we continue our design. For general purpose predictable workloads that don't have special requirements (such as SQL, Exchange, SharePoint, and VDI), we typically see a vCPU:pCPU ratio of 2:1 to 4:1 inclusive with higher ratios for test and development environments. But this is certainly a situation where your mileage may vary, so you need to know your applications and workload in order to intelligently choose the appropriate ratio for your environment. Some of the other factors, such as cache and clock speed, are a bit more straightforward. More cache and more speed equal higher performance in most cases. If you are power conscious, then you'll want to take into account the power draw of each CPU (wattage). Power consumption modeling is often used to determine the TCO (total cost of ownership) of systems in a datacenter, especially when using co-location facilities where you are charged for power consumption. Unless we're specifying refurbished or otherwise dated CPUs, new ones are typically going to support all of VMware's feature set. If you are using older equipment, it is vital to review the HCL to validate that your infrastructure is supported.
[ 25 ]
Hypervisor Design
Memory and NUMA considerations
While CPUs provide the raw processing power for our VMs, memory provides the capacity for our applications and workload to run. Also important are High Availability (HA) admission controls. If strict admission controls are enabled, then guests that violate HA controls won't come back online after a failure. Instead, by employing relaxed admission controls, you'll see performance degradation but won't have to deal with the pain of HA causing VMs to power off and not come back. By considering the speed, quantity, and density of our host-based RAM, we're able to avoid those issues and provide predictable performance to our virtual infrastructure. This means that your design depends on sizing memory and calculating HA minimums to ensure you don't cause problems. At the highest level, the total amount of host memory is pretty straightforward. As we take into consideration our workload and application requirements, we can start by simply dividing the total amount of RAM required by the number of hosts to get the amount of memory we want in each host. Just as before, there are tools to help us determine the amount of memory we need. VMware Capacity Planner is one of those tools and is available to VMware partners. VMware has some free flings such as vBenchmark and ESX System Analyzer. Other tools such as Dell Quest's vFoglight and SolarWinds Virtualization Manager are commercially available and can provide detailed reporting and recommendations for properly sizing your virtual environment. In most cases we're designing an N+1 architecture, we'll then add an extra host with that amount of RAM into the cluster. For example, if our overall workload requires one TB of RAM, we may design for four hosts each with 256 GB of RAM. To achieve the proper failover capacity required for HA and our N+1 architecture, we'll add a fifth host for a total of 1.25 TB. NUMA (non-uniform memory access) is a feature where the system memory (RAM) is divided up into NUMA "nodes". These nodes are then tied to a pCPU (remember, this also could be a CPU core) and represent the set of RAM (or address space) that can be accessed quickest by a given pCPU.
[ 26 ]
Chapter 2
Each NUMA node is situated the closest to its associated pCPU and results in a reduced time to access that memory. This reduces memory access contention as only a single pCPU can access the memory space at a time. pCPUs may simultaneously access their NUMA nodes resulting in dramatically increased performance over non-NUMA, or UMA systems. The following diagram shows the NUMA architecture: Intersocket connection
Node 1
CPU
CPU
1
2
1
2
4
3
4
3
Local access
Local Memory node 1
Local Memory node 0
Node 0
Remote access
NUMA Architecture
Some server manufacturers include settings in the server's BIOS, which will impact NUMA. One such setting is called Node Interleaving and should be disabled if we want to take advantage of NUMA optimizations. For more information, have a look at the White Paper from VMware at https://www.vmware.com/files/pdf/ techpaper/VMware-vSphere-CPU-Sched-Perf.pdf.
The other consideration with NUMA is vCPU. If we can identify the number of vCPUs required to support our workload, we can size our hosts so that our number of cores per NUMA node is greater than or equal to the number of vCPUs in our VMs. For example, if we have four cores in a NUMA node, then we'll see the best performance for VMs with four vCPUs or less. This is because those VMs will be assigned to cores within a NUMA node and thus get quicker, local access to system RAM. If we were to increase the vCPUs in this example VM to six, then we could see an increase in memory access times because the VM will span NUMA nodes and will need to access "remote memory" or memory outside of its NUMA node.
[ 27 ]
Hypervisor Design
Virtual NUMA (vNUMA) considerations
ESXi hosts create a virtual NUMA topology that closely matches the server's underlying physical NUMA layout. This vNUMA topology ensures that guests have optimal access to compute and memory resources. vNUMA for VMs is calculated for the host on which the VM starts and will not adjust if a vMotion operation takes the VM to another host; this is another good reason to employ hosts with the same hardware configurations. Additionally, vNUMA ensures a guest's memory and processing power is provisioned and in line with the physical NUMA layout that lives underneath, even if that topology spans more than one NUMA node. vNUMA has a few caveats to bear in mind. Your VMs have to be VM Version 8 and running on vSphere 5.0 (or higher). Guests with eight vCPUs enjoy automatic vNUMA configuration. It's very likely that monster VMs with many vCPUs will access multiple NUMA nodes when running, so this makes sense from a default settings point of view. Defaults aren't always best when large VMs have unique requirements. You can adjust your guests' vNUMA configuration on a per-VM basis via Advanced Settings in order to adjust vNUMA to meet your needs. Do this to meet a particular VM's demands, or when tailoring vNUMA for better performance in vApps with multiple guests.
Considerations for Java virtual machines (JVMs)
When designing JVMs, some special considerations exist. Bear these in mind while planning and deploying your Java VMs. When determining the number of vCPUs to allocate to a new JVM, test performance until you arrive at a number of vCPUs that provide the performance you need under varying loads. If you have access to a testbed, do the performance tests in that environment. If you don't have access to a lab environment or you can't test in your production environment, be prepared to adjust the number of vCPUs allocated to the JVM. You want to arrive at a vCPU allocation that doesn't inhibit the JVM under a variety of loads. If your Java code runs multiple garbage collector threads, ensure that your vCPU allocation matches the number of threads. Memory allocation is also critically important to JVM performance. Ensure you leave adequate memory overhead for the Java heap, any other Java code requirements, and other operations on your JVM. Be sure to test this memory consumption under a variety of loads. Once you've sized the memory, set a reservation for that memory to avoid swapping to disk. Memory swapping, either physical or virtual, will have a negative performance impact on the Java heap. This impact is felt most during garbage collection operations within the JVM. Ensure the balloon driver is running on your JVMs, especially if your environment is overcommitted. [ 28 ]
Chapter 2
Lastly, JVM processes should be configured on a one-for-one VM basis. Don't load up one JVM with multiple Java processes. You'll avoid resource contention and make your life much easier when monitoring your JVMs.
Network interface card considerations
While CPU and memory are integral components of our ESXi hosts, we won't have an environment without network connectivity. NICs provide network connectivity to our hosts, VMs, and may also provide storage connectivity if our designs that utilize iSCSI, NFS, or fiber channel over ethernet (FCoE). In this section, we'll examine some scenarios and discuss guidelines for quantity, type, and configuration of NICs. Let's start by looking at the most common deployments of 1 Gbps Ethernet (1 GbE) NICs. Dedicating NICs to different traffic types is typically ideal. In other words, at a minimum, we see a pair of NICs for management traffic, vMotion, Fault Tolerance (FT) (if used), IP Storage (if used), and VMs for a total of ten NICs. This may seem like quite a few, but quad-port NICs (NIC cards with four 1 GbE ports onboard) are quite common and cost effective nowadays. If we're not using FT- or IP-based storage, that immediately drops us down to six NICs. In either case, it is a good idea to have multiple physical NIC cards in the system. This, in conjunction with using NIC teaming, can provide protection against the failure of an entire NIC card. NIC teaming is used to provide resiliency to our hosts' network uplinks. For instance, to provide high availability to our hosts' management network, we can bond multiple NICs to that network. If one NIC fails, we will still have management network functionality as long as one NIC remains operational.
To fully realize the benefits of NIC teaming with multiple NICs, we need to be sure to team NICs that are not on the same piece of hardware. For example, let's take another look at our management network. If our server has four NICs built in on the motherboard, we should still have a PCI-based NIC card in the system. Then, our management network can have a NIC from the on-board card and a NIC from the PCI card teamed together. If either of those NICs fail, we'll still be operational. We can extend this concept and include more than just pairs of NICs for resiliency. Perhaps our workload requires bandwidth above and beyond the 2 Gbps we would get with a pair of NICs, so we can increase the number of NICs in the team to gain additional bandwidth.
[ 29 ]
Hypervisor Design
In general, we'll see a minimum of six to ten NICs in an ESXi host with NIC teaming being used to spread bandwidth and connectivity across multiple physical NIC cards for the highest possible resiliency. With respect to bandwidth, there are tools that can be used to monitor and/or estimate the required bandwidth needed to support our workload. VMware's Capacity Planner is one of those tools but there are other tools. Based on the output of those tools, we can determine the quantity of NICs required for VM connectivity. Typically, a pair of 1 GbE NICs is sufficient for management, and a pair of 10 Gb NICs for vMotion is recommended. VMware vSphere 5 allows four concurrent vMotion operations over 1Gb NICs and up to eight with 10 Gb links. We have similar considerations when utilizing 10 GbE NICs in our designs. However, instead of ten or more 1 GbE NICs, we usually see two to four 10 GbE NICs per host. Multiple adapters and ports provide adapter and connectivity redundancy allowing a spread of NIC teams across different physical adapters and switches. There are also Converged Network Adapters (CNAs) that are connected via the Data Center Bridging (DCB) protocol. This allows both ethernet and fiber channel (via Fiber Channel over Ethernet (FCoE)) to be transmitted over the same cable and port. While 10 GbE adapters can greatly reduce the amount of cabling, CNAs can go a step further by consolidating fiber channel Host Bus Adapters (HBAs) and fiber cables.
LAN
Server Fibre Channel Driver
Lossless Ethernet CNA
Networking Driver
With FCoE
FCoE Switch
SAN
Converged Network Adapter Architecture
There are a few things to look at no matter what speed and type of adapters will be used in our designs. If at all possible, be sure to use NICs that support TCP checksum offload, Jumbo Frames, and NetQueue. These services may enhance performance and throughput while reducing the amount of CPU cycles in the host that would be used to process IP traffic. Our designs may not initially utilize all of these features, for example, Jumbo Frames, but having adapters with these features will ensure flexibility down the road should the design requirements or workloads change. [ 30 ]
Chapter 2
Hypervisor storage components
In many cases, one of the last things we consider in our host designs is the storage. To gain access to some of the best features of vSphere, we typically use shared storage. That could be a storage array or a NAS (Network Attached Storage) device. But what about the hypervisor itself? ESXi has a very small disk footprint—144 MB in vSphere 5. However, a minimum of 5.2 GB of disk space is required when booting from local disk or a SAN/iSCSI LUN. This is because a 4 GB scratch partition is required for ESXi. If that 4 GB of space doesn't exist in the host, then a RAM disk is utilized for that 4 GB. For best performance and memory optimization, it is recommended that we do not utilize the host's RAM disk for the scratch partition. The bottom line for ESXi is that we need a relatively small amount of space for the hypervisor. One of the most common deployments has been to use a pair of hard drives in a RAID-1 set for the hypervisor. With hard drive density increasing, it may be tough to justify 300 GB or larger drives just for 5.2 GB of space. We're starting to see the hypervisor installed either on an SD card or a USB stick. Newer server hardware even allows us to install multiple SD cards in a RAID-1 set to provide resiliency, which was a concern when a single SD card could be a single point of failure. Of course, if our storage design requires us to have local data stores, then we'll have to have storage to accommodate that, but those are for specific use cases such as VDI, VSA, and VSAN deployments. There are third-party software packages that can take advantage of local disks and turn them into shared storage. But, for the hypervisor itself, we can safely reduce costs by leveraging SD cards or USB devices for boot. Add additional storage if there is a requirement for local data stores on the host.
Stateless host design
Stateless hosts have been a supported vSphere deployment option since vSphere 5.0 but are already gaining ground a preferred deployment methodology. In general, stateless hosts are hosts that receive their "state" from another source. That state, which could also be thought of as their personality, is downloaded or streamed from the boot source each time the host boots. Stateless hosts allow us to have the ability to swap out host systems (think hardware upgrades or failures) without the need to reinstall the hypervisor. That should sound pretty exciting! There are a couple of ways in which we can accomplish this type of deployment—Boot-from-SAN and vSphere Auto Deploy.
[ 31 ]
Hypervisor Design
Booting from SAN, either via Fiber Channel, iSCSI, or even FCoE, has become more common in recent years. This provides some advantages but can also increase complexity and cost. Booting from SAN allows us to have zero storage in our hosts causing them to be stateless. Having stateless hosts allows us to leverage the availability and resiliency of our SAN and storage arrays without adding cost to each of our hosts. One of the downsides, however, is that our boot LUNs now reside on a storage array. If we need to migrate to a new storage array, that migration can become tricky. Another disadvantage is that in order to communicate with our storage arrays, we need some stateful information that is going to reside on our hosts in the cases where we're using FC or FCoE storage. Our HBAs (host bus adapters) contain information much like that of a MAC address for an ethernet adapter. These WWNNs (world wide node names) and WWPNs (world wide port names) are granted access to our storage arrays through a mechanism called zoning. If we swap out hosts, or the HBAs, we'll have to do some legwork to adjust our zones for the new HBAs because they'll have different WWNNs and WWPNs. This is at the detriment to our whole stateless strategy, as even though we're removing the hypervisor's stateful information from the host, there is still other stateful information that will remain with the host. Some server manufacturers have been providing mechanisms to abstract this stateful information from the hosts. Cisco's UCS and HP's BladeSystem are two of those systems that allow us to get to a completely stateless host.
Auto Deploy is another mechanism that allows us to achieve stateless hosts. Auto Deploy uses a boot process called PXE (Pre-boot eXecution Environment) and Host Profiles to provision and customize ESXi hosts over the network. Here, again, we gain the benefits of stateless hosts in our design and this seems like a really great technology if we're not utilizing an FC, FCoE, or iSCSI SAN. However, Host Profiles continue to add some complexity and unpredictability, so be careful. Host Profiles don't seem to function 100 percent as they should; while these issues don't typically lead to production impacts, they could cause additional manual steps to get hosts online. In any Auto Deploy design, there should be a management cluster that uses a stateful boot mechanism for vCenter and other dependencies. In summary, stateless hosts yield some fantastic advantages. Auto Deploy is going to continue to mature and perhaps will be the leading method to achieve stateless hosts, but boot-from-SAN seems to be the most effective. If we want to extend our stateless hosts beyond the hypervisor, there are manufacturers who have solutions that further abstract stateful information and enable a truly flexible and stateless host environment.
[ 32 ]
Chapter 2
Scale-Up and Scale-Out designs
When creating the design for our hosts, it is important to take a look at the density of our virtual infrastructure. Specifically, we need to consider the size of our hosts from a resource perspective and the percentage of VMs that will run on a single host. Let's go back to the previous example where we have a workload that requires 1 TB of RAM. Previously, we configured an N+1 cluster with five hosts each with 256 GB RAM. Are there other combinations we can use to accommodate that workload? Absolutely! With modern hardware, we could use a single host to support this 1 TB workload but then we'll have a single point of failure. So what is the minimum number of hosts recommended for a cluster? Although, technically, two is the minimum to satisfy HA, we want to leverage three hosts. The reason for this is that with a two-host cluster we could be at risk while we have a host in maintenance mode. If we have a host in maintenance mode, and we have a failure on the other host, where are our VMs going to go? With three hosts, we'll have that extra failover capacity if we've sized for two host failures, or if we happen to just have the spare capacity. Now that we've identified the minimum number of hosts we can talk about Scale-Up and Scale-Out architectures. Scale-Up design is when we add more resources to the existing hosts in order to increase capacity while Scale-Out means that we add additional hosts in order to increase capacity.
Scale-Up
Scale-Out
Scale-Up versus Scale-Out
[ 33 ]
Hypervisor Design
Scaling up has been common in scenarios where rack space is a premium. Another use case to have fewer, larger hosts is where organizations have a policy to minimize the number of hosts to be managed. With technologies such as Cisco's Unified Fabric and 10 GB CNAs, we could drastically reduce the amount of cables, ports, and port licenses to pay for and manage by using a Scale-Up strategy. The more we pool our resources, the more efficient we can be. However, how long does it take to put an ESXi host into maintenance mode when it is running 100+ VMs? These VMs will need to be vMotioned to another host and there are a finite number that can run in parallel. With high consolidation ratios, there are instances where it could take hours to evacuate a host. What about an HA event? A host failure would cause those 100+ VMs to go down and then be rebooted on other hosts. There is an adjustable maximum of 32 VMs restarted per host, which could increase the time it takes to bring those VMs back up during an HA event. The Scale-Out strategy is becoming more common. With new automation and orchestration tools becoming more prevalent, some of which are already included with vSphere such as vCenter Orchestrator, we're less concerned about increasing the amount of managed objects. Scale-Out allows us to spread our resources and VMs across more, smaller hosts. If a host fails, then it impacts far fewer VMs. Replacing and scaling by adding hosts can be less time consuming. But this strategy is also not without its weaknesses, such as cost and environmental factors. For instance, with Scale-Out, we have a need for additional ports, port licenses, cables, and environmental factors. Perhaps, we're not using any orchestration or automation in our environment so a Scale-Out strategy is going to have a dramatic impact on our ability to manage our resources and infrastructure. In many cases, the key to sizing our designs is to strike a balance between Scale-Up and Scale-Out. Let's imagine a design where we have four large hosts in a Scale-Up strategy and then adjust the design for Scale-Out, which may have twelve hosts. Due to licensing and infrastructure costs, we may find the "sweet spot" for cost per VM, which may be seven hosts. This is just an example and there are other considerations that go into this decision. The point is that there is an entire design spectrum we can focus on and we need to identify the parameters that are important, adjust them, and determine the areas of that spectrum that best fit our requirements.
[ 34 ]
Chapter 2
Summary
In this chapter, we explored the ESXi hypervisor and the design decisions that go into the hardware of our ESXi hosts. We discussed CPU, memory, network, and storage components and considered design characteristics within each component. Lastly, we discussed Scale-Up and Scale-Out and when we might utilize those concepts in our own designs. In the next chapter, we dive into principles of storage design and how to prepare your infrastructure for its next steps.
[ 35 ]
Storage Design A virtual datacenter has a number of critical components and storage is a major part of any datacenter or design. It is the proverbial lifeblood of every virtual infrastructure. Without a functioning storage infrastructure, the ability for virtual machines to write data and operate, as we know it, would not be possible. This chapter delves into some of the principles of how storage is leveraged in a virtualization environment and how you can go about improving your existing investment. In this chapter, we will cover the following topics: • Storage protocols (such as Fibre Channel, iSCSI, FCoE, and NFS) and their uses • Virtual machine filesystems and their utilization • Storage, datacenter, and cluster design best practices and examples • VMware storage features and configurable options
Storage protocols
Let's talk about storage. In a virtual datacenter, storage is a critical bit that is often overlooked. In a perfect world, we're given unlimited budget and a clean slate to do what we want with the storage backend. Usually, however, we don't have a clean slate or an unlimited budget and often fall in on an already existing storage area network (SAN). Many vDCs will have a SAN infrastructure built around a preferred storage vendor's product family where the virtualization team has no direct access to the storage platform. This is not a debate about which vendor is best, whether or not fiber channel is the be-all and end-all solution, whether iSCSI rocks, or whether NFS is the best. Let's start with a discussion about the different types of storage available to us.
Storage Design
Fibre Channel (FC)
Fibre Channel (FC) got its start as the gold standard for both connectivity and performance, with speeds initially at 1 Gbps evolving to increase to 2 Gbps, 4 Gbps, 8 Gbps, and today 16 Gbps. Originally designed, as the name implies, to use fiber optic cabling, in the past several years, the standard has allowed the use of copper cabling. In a world with limited storage capabilities and few options for scalable access, FC was designed to provide access to disks or storage. Fibre Channel is usually deployed through a switch, a set of switches, or preferably a Director. Switches are typically smaller semi-modular, less-redundant devices used to support the storage infrastructure, whereas Directors consist of high port count, modular slot-based chassis with no single point of failure. Hosts, adapters, switches, and the storage combine together to form a fabric, which provides the infrastructure for your storage backbone. Let's say you have a physical server with a Fibre Channel Host Bus Adapter (HBA) with a unique World Wide Name (WWN) and the ubiquitous orange cable (or cables) connecting that server to a switch. The switch port you're plugged in to has its own World Wide Port Name (WWPN). The switch will also have a connection to a storage controller, with its own unique World Wide Node Name (WWNN). All of these components combine to form a meshed network with connections to servers, switches, and storage. Solutions leveraging FC, while often complicated and expensive, provide high levels of availability and performance which depending upon your business needs, may be the best solution.
iSCSI
iSCSI is the Internet extension of an even older protocol, the Small Computer Systems Interface (SCSI) protocol from the days long before x86 virtualization. There are initiators and targets here similar to how we discussed FC working previously. One cool VMware feature in ESXi is a software adapter that allows you to access your storage via your existing network adapters without having to make investments in additional interfaces if your budget is limited. While hardware is more efficient and can provide a fabric similar to FC leveraging dedicated Ethernet adapters or Converged Network Adapters (CNAs), you're not required to make that investment for iSCSI to function. Another benefit associated with iSCSI (if you are running iSCSI on CNAs) is the ability to leap above and beyond traditional Ethernet networking and provide 10 GbE connectivity to your infrastructure. One of the biggest selling points of IP storage has been that it is inherently cheaper than Fibre Channel by leveraging your existing network infrastructure and adapters. There are some tradeoffs you'll need to address should you choose an IP-based storage if you do not have [ 38 ]
Chapter 3
administrative control over the network infrastructure. While there may be reduced acquisition costs to support and engineer the environment, there are risks associated with an infrastructure shared with end users, network connected printers, and other nonvirtualization based servers, which should be mitigated by dedicating networking resources to your virtual environment and storage networking. In other words, a dedicated network for your core virtual infrastructure is a must do, not a "nice to have".
Fibre Channel over Ethernet (FCoE)
FCoE marries some of the flexibility of iSCSI with the capabilities of the Fibre Channel protocol's priority-based flow control among other capabilities. The decision to adopt FCoE over traditional FC or iSCSI is often pursued to reduce the number of adapters required to connect to storage and IP networks, and reduce the number of cables routing from servers to switches, while also reducing power and cooling costs. Servers deploying FCoE will commonly use CNAs, which combine a Fibre Channel HBA and an Ethernet NIC card. Adopting FCoE in an environment will often require coordination and cooperation of the storage and networking teams as the overlap of the infrastructure can blur the lines. While the benefits of adopting FCoE are numerous, there are many considerations that need to be addressed in order for its adoption to be a success.
NFS
Network File System (NFS)was developed as a network filesystem for UNIX to provide access to network files the same as if they were local. NFS is arguably the cheapest to set up and the simplest to manage. Some storage vendors even optimize deduplication on NFS to take your virtual footprint and extend it even further. NFS access is file-based and thus does not provide block access to SAN resources; the implications of this impacts your ability to leverage RDMs if required. The following table shows how these storage protocols interact with physical connectivity and also gives information about the types of access: Technology
Protocol
Speed (GBps)
Access
Interface
Fibre or FCoE
FC/SCSI or FCoE/SCSI
1/2/4/8/10/16
Block
FC HBA or CNA w/ FCoE
iSCSI
IP/SCSI
1/10
Block
CNA or NIC w/ iSCSI
NAS
IP
1/10
File
NIC or CNA
Now that you've got a table of how these protocols connect, how about another one that touches on important vSphere features per storage type? [ 39 ]
Storage Design
The following table is the vSphere features support matrix: Storage type
Boot from storage
vMotion support
Datastore type
RDM support
VM clustering
VMware HA & DRS
Local
Yes
No
VMFS
No
Yes
No
FC/ FCoE
Yes
Yes
VMFS
Yes
Yes
Yes
iSCSI
Yes
Yes
VMFS
Yes
No*
Yes
NAS
Yes
Yes
NFS
No
No
Yes
*Supported in 5.5 (KB2052238) and using in-guest iSCSI pre-vSphere 5.5. VMware is agnostic with regards to protocols and connectivity—they all work. What matters are features supported on the different storage technologies; in other words, if you need RDM support, native NFS isn't going to do it, so don't build a design around it. Design to requirements and select a storage technology that meets the needs of the business. Select a protocol or a set of protocols that support the features of VMware you wish to leverage in the design. Your design should be driven by the requirements of the businesses needs and your applications. So whether you choose FC, iSCSI, NFS, or a mixture of the protocols, datacenters (large or small) are evenly split on which protocol they choose based on business, architecture, and budgetary decisions.
Virtual machine filesystems
Now that we've gotten over the difficult decisions of figuring out how our ESXi hosts will communicate to the backend storage, we now need to decide how our VMs will be laid out on the disk. VMware has several options available for laying data down on the disk, be it VMFS, RDMs, or NFS. We'll go into the details of each and how this may influence your storage architecture.
VMFS
Virtual Machine File System (VMFS) is designed as a clustered filesystem, allowing simultaneous access from multiple hosts. VMFS handles file locking natively, protecting data and ensuring VMs work with ESXi hosts in concert without stepping on each other or more importantly, the data. VMFS 5, the latest version, brings new features to the table and makes it an even more palatable option than it was in version 3. New features added in VMFS 5 are: • A maximum size of 64 TB for VMFS datastores • A maximum of 256 VMFS stores per host [ 40 ]
Chapter 3
• A maximum of 62 TB size for virtual compatibility RDMs • A maximum of 64 TB size for physical compatibility RDMs • Improved file locking mechanism allowing 2048 VMs per VMFS volume • Support for Virtual Machine Disk (VMDK) larger than 2 TB There are some caveats to VMs with VMDKs larger than 2 TB; see KB 2058287 for details. VMFS was also designed to support a virtual infrastructure. As such, VMware features will all work on VMFSs. The following are some best practices to keep in mind when deploying VMFS: • Limit VMFS volume size; just because you can go to 64 TB doesn't mean you should • Determine the number of VMs per VMFS datastore based upon the performance capabilities of the datastore and not only its available capacity • Try and manage your datastore sizes to concur with your preferred backup solution • Avoid mixing storage workloads for high performance apps (SQL, Oracle, and SAP), and instead consider dedicated VMFS spindles or RDMs to support the performance requirements
RDM
Raw Disk Mapping (RDM) provides virtual machines direct access to SAN resources. In other words, your VMs are able to bypass the virtual layers in your infrastructure and hit the disks directly. RDMs aren't a panacea though, and some important drawbacks exist that must be considered when planning to use them. Perhaps the most common use for RDMs is when you need to cluster MS applications using Microsoft Clustering Services (MSCS). Another plus is the ability to provide direct access to your VMs when you want to provide max performance for SQL and other applications that require direct I/O. RDMs were particularly useful in versions older than vSphere 5.5 when access to LUNs larger than 2 TB was needed in situations where it was not possible to use multiple VMDK files on VMFS. Let's have a look at RDMs—use cases, some benefits, and some drawbacks. Some of the benefits of RDM are as follows: • Direct access to disk for MS clustering • Applications need direct access for best performance (SQL, Oracle, and OLTP)
[ 41 ]
Storage Design
• SAN management applications that require access to SCSI commands Some of the drawbacks of RDM are as follows: • No snapshots in physical mode RDM (available in virtual mode) • Can't clone an RDM, and no conversion of an RDM to a VM template The use-cases requiring RDMs are often very specific and unique. If you're unable to support your applications capabilities through VMFS or VMDKs, then RDMs may be the appropriate solution for you. RDMs should only be used where they're required as some benefits of virtualization are restricted when they're used.
NFS
One thing that often leads people to choose NFS storage for their virtualization is the ease of provisioning and maintaining NFS shared storage pools. NFS storage is often the cheapest to setup and maintain, than FC storage. For this reason, NFS tends to be the choice adopted by small and medium businesses. Some of the benefits of NFS are as follows: • Management ease • Great for shared repositories (ISO stores) • Deduplication from storage vendors • Thin provisioned by default Some of the drawbacks of NFS are as follows: • Can't work with RDMs natively • No VM clustering on NFS • Management overhead; something has to manage NFS and ESXi doesn't • No "control" from VMware for things such as file locks and contention See the best practices for NFS on vSphere at http://www.vmware.com/files/pdf/ techpaper/VMware-NFS-BestPractices-WP-EN.pdf. Now that we've explored storage protocols and filesystems, it's time to discuss storage design.
[ 42 ]
Chapter 3
Designing storage
When talking about how best to design storage, consider how ESXi will be deployed. Will it be on commodity servers? A converged solution? How many HBAs do you have per host? What sort of controllers and disk will be procured, based on what you need your storage solution to do? It's not necessarily about what storage protocol you employ, it is more about what you want your storage backend to do. That drives how it's built, configured, and deployed. Ensure that you build hardware redundancy for fault tolerance from the ground up. Sufficient redundancy means having multiple CNAs, HBAs, or NICs per host, multiple paths, and connection to multiple redundant switches. This type of configuration is valid in both FC and IP storage environments. ESXi provides the ability to house multiple adapters in a single chassis and aggregate bandwidth for storage. If iSCSI is your chosen solution, it is preferred to have 10 GB connectivity for storage over 1 GB, and if you're doing FC, then 16 GB is your limit. Employing FC means that your storage networking is a closed loop; you won't be sharing any of this with network traffic. Should you decide on iSCSI, care must be exercised to prevent sharing of resources or a QoS policy interfering with storage traffic. Wherever possible, dedicate networking and physical paths to storage for best performance. Build redundancy into your switching fabric and eliminate single points of failure by ensuring you have redundant hardware, CNAs, NICs, shelves, and controllers.
Local storage considerations
So what is the use of local storage in today's virtual datacenter? Most often it's a place to land your ESXi install and boot your hosts. An ESXi installation is small these days, and will easily fit on a SD card or a thumb drive as previously discussed. Many server manufacturers are offering internal SD or USB slots for the sole purpose of booting ESXi. This isn't to say that spinning disks don't have a place; they certainly do. Should you elect to boot from a spinning disk or array of disks, ensure that you select a fault tolerant solution; many instances are using a RAID 1 mirror to boot ESXi. Even better is to provide a hot spare in standby, so you're not having to replace disks to keep your hosts alive all the time.
[ 43 ]
Storage Design
Another option for local storage is solid state drives. SSDs have come a long way in reliability and price point. With the ability of vSphere 5 to detect a locally installed SSD and use it for host swap cache and vSphere Flash Read Cache, it's not a bad idea to have at least one in each host. That said, if your design starts swapping to disk, even SSD, you may wish to consider taking a hard look at why this is happening. SSDs are great, but they're not a panacea. In an environment where you're booting from SAN, you wouldn't be able to swap at the hosts without some local storage. Finally, let's touch on VMware's latest offering, Virtual Storage Area Network (VSAN). VSAN provides a scalable, highly available, and redundant method to leverage local spinning disks and SSDs to create shared storage providing the full suite of vSphere capabilities. There's an excellent design consideration document for VSAN at http://www.vmware.com/files/pdf/products/vsan/ VSAN_Design_and_Sizing_Guide.pdf.
Datastore and cluster design
There are some basic pitfalls you can avoid by employing some of VMware's recommended best practices as they relate to storage. Bear in mind that the following are recommendations and won't fit all designs, but we can avoid some issues with intelligent design and smart decisions about storage design: • Minimize the number of datastores; don't create one VMFS per VM as it increases LUN count unnecessarily, making management more complicated than it has to be. Remember vSphere 5.5 has a limit of 256 LUNs per server. • Keep datastores reasonably sized; you can always grow your datastore. You can make them larger within the 64 TB limit if your design requires it. • Thin provision where you can. A majority of VMs left to their own devices go under-utilized on space usage. Take advantage of their under-use and reclaim the space. • Separate production LUNs from test LUNs, and store ISO files, templates, and backups on their own dedicated LUNs. • Disk alignment can and will impact performance. Ensure your VMDKs are properly aligned after OS provisioning. • Be cautious with multiple snapshots of VMs in the same LUN; you run risk of running out of disk space if not monitored.
[ 44 ]
Chapter 3
• Extents should avoid being used. With a 64 TB maximum limit on LUN size, you can always expand your LUN without requiring extents. • Avoid grouping several LUNs to form a VMFS because different environments cannot be separated (such as the production environment, test environment, and templates).
VMware storage features
This section will discuss what VMware does in terms of storage and how this plays into your design. First up is Storage DRS. VMware vSphere 5 introduced Storage Distributed Resource Scheduler (SDRS). Storage DRS can automatically balance I/O loads via ongoing monitoring and Storage vMotion of VMs off datastores with high I/O loading to preserve performance across your datastore cluster. It allows for affinity rules, which enable you to pin a high I/O demanding VM to a certain store, or prevent sharing of spindles amongst VMs that have different requirements, and affords an easy way to place datastores in maintenance mode. Just like DRS, this lets you put hosts in the same sort of "I'm still working with you but I'm going to take a break for a while" mode. Some components of the VMware Storage APIs are as follows: • VAAI: VMware Array Aware Integration (VAAI) is a set of hardware APIs that offload some of the storage workload from your ESXi hosts to storage controllers. Features such as provisioning, cloning, vMotion operations, xcopy, and ATS are handled by storage primitives that direct workload from ESXi hosts to storage controllers. The performance benefits include faster provisioning, less work for CPUs, and less I/O on ESXi nodes meaning more cycles and I/O for the VMs on your hosts. • VAMP: vStorage API for Multipathing (VAMP) facilitates I/O path selection from hosts to storage devices and provides storage path failover. Storage I/O throughput is optimized and VMware makes intelligent choices beyond default pathing policies.
[ 45 ]
Storage Design
• VASA: vStorage APIs for Storage Awareness (VASA) enables compliant storage to communicate with vCenter and report storage topology, capability, and state information. This information supports policy-based management, operations and Distributed Resource Scheduler functions. VASA providers can identify trends in the way a VM consumes storage facilitating the troubleshooting of storage issues. In addition, performance indicators for VMs with storage health reporting provides invaluable insight into the storage backend from vSphere and can help with decisions about storage locations for VMs. VASA aware storage providers are invaluable in the vDC as they provide insight into what the backend is doing and provide metrics used in vSphere policy and operations management.
Configurable storage features
There are some features you'll want to be aware of and potentially leverage in a vDC. You may use some, all, or none of these, but in the real world, it's highly likely that you'll leverage them all. Even if you don't, be aware of what they are so that design decisions leverage all the possibilities available.
SIOC
First up is Storage I/O Control (SIOC). Introduced in Version 4.1, SIOC's purpose is to prevent VMs from hogging I/O on storage. In Version 5 and later, you can apply SIOC to NAS devices as well as block storage. SIOC controls access to storage through the shares concept, and provide I/O minimums to VMs based on the policies you control. The congestion threshold for SIOC is set at the datastore, with a default latency value of 30 ms. This is changeable up or down based on the demands of your VMs and the capabilities of your storage, for example, adjust to 20 ms for SAS and 15 ms for SSD. SIOC doesn't do anything unless the defined latency threshold is exceeded. This is true across all VMs, nodes, and datastores in a cluster. VMs can be assigned a priority; this works on the shares concept, configured at the VM. When contention arises, SIOC ensures that VMs that need I/O receive their configured share of I/O. Other things to note are that SIOC works when it has visibility to all the workloads on a datastore. Should you have a condition where external, non-VMware loads are taking I/O, then you may see an alarm along the lines of non-VI workload detected on the datastore. SIOC works best when you eliminate external actors, hence the case for "don't share your datastores with things that aren't VMware"!
[ 46 ]
Chapter 3
SIOC also is critical to another feature, Storage Distributed Resource Scheduler (SDRS). Essentially, SDRS calculates I/O loading and balances that load across storage. This is especially useful in scenarios where you have disks with different performance characteristics; think SSD, SAS, SATA, and so on. Unless defined, SDRS doesn't play a part; however, when you create a storage cluster and populate with datastores, you can segregate by performance profile, and balance loads automatically. SDRS will storage vMotion VMs based on I/O to ensure that efficient I/O is maintained across the storage cluster. You can also place datastores in maintenance mode just like an ESXi host, making storage maintenance a much simpler process.
Configuring the multipathing path selection policy
Your path selection policy consists of three options: Fixed, Most Recently Used (MRU), and Round Robin. If you have multiple path paths to your storage, you'll want to take advantage of some kind of path selection policy. It's best to refer to your vendor's best practices, though Round Robin tends to be a fairly common configuration to provide additional performance and resiliency ahead of the default of Fixed. You can learn more about multipathing policies in general in KB 1011340.
Provisioning
Provisioning is often overlooked. The following three types exist: • Thick Provision Eager Zero: Blocks are allocated and zeroed on provisioning. This is important when provisioning fault tolerant VM. This provisioning process takes longer as blocks are allocated and zeroed when creating the VMDK. This improves first time write performance. • Thick Provision Lazy Zero: Blocks are allocated upon creation but not zeroed until the VM first writes to the VMDK. This process happens quickly in comparison to an eager zero operation. Blocks are allocated but space isn't used until the VM needs to write beyond the initial footprint. • Thin Provisioned: In this type of provisioning, you only use as much space as required by the initial write. You cannot convert to a thin provisioned disk, but you can perform a storage vMotion to a thin provisioned disk and "convert" it that way. When assigning storage to VMs, select the disk type that meets your applications needs.
[ 47 ]
Storage Design
Summary
In this chapter, we learned about the design characteristics of storage in a virtual datacenter and the core components of the VMware storage platform. We discussed the best practices and design considerations specifically around storage protocols, virtual machine filesystems, storage, datacenter, and cluster design best practices. You should be able to better assess the best storage protocol and filesystem to design and map your architecture to meet your business requirements. In the next chapter, we'll discuss principles of network design.
[ 48 ]
Network Design In datacenter networking, design is broken into two parts; these are virtual and physical networking considerations. If storage is the heart of VMware vDC design, then networking is the arteries. Virtual networking is critically important in order for your virtual infrastructure to operate and function. Your physical network infrastructure needs to be built on top of the requirements that your underlying virtual infrastructure will call for. We'll discuss how your virtual requirements can help determine what physical infrastructure will support it. That said, design principles aren't going to change here; there's a recurring theme of speed, redundancy, high availability, and survivability. Design your vSphere networking so that when (not if) something fails, your vDC doesn't come to a grinding halt. It might limp along a bit, but will continue to operate, regardless of whether we're talking a three-node deployment supporting 10 virtual machines all the way out to massive datacenters with hundreds of hosts and thousands of VMs. We want a design that's high performance, provides reliable 24/7 availability, is secure, and as easy to maintain as possible. Let's have a look at the supported technologies available to us as we talk design and the best way to network our environment. The following are the topics we'll be covering in this chapter: • An Introduction to vSphere Standard Switches (vSS) and vSphere Distributed Switches (vDS) • Common considerations when designing your network • How to provide appropriate network redundancy, resiliency, and throughput • Networking considerations regarding IP storage
Network Design
Introducing vSphere switching
There are many ways to skin a cat and networking in ESXi isn't any different. Let's take a look at the two main ways in which we achieve our goals with VMware networking.
vNetwork Standard Switches (vSS)
The simplest and often first virtual switching technology that folks see is the vSS. A standard Ethernet switch is emulated in software and this configuration is managed individually and stored on each of your ESXi hosts. A deep dive into how vSSes work is well documented by VMware and since we're discussing design best practices, let's dive straight in and talk about them: ESXi host 1 Management traffic
vMotion traffic
VM
ESXi host 2 VM
VM
VM
VM
VM
VM
VM
vMotion traffic
Management traffic
vNIC Virtual port
Network production
Management
vMotion
Test environment
Production
Port groups
Production
Test environment
Uplink port group
uplink port 0
uplink port 1
vMotion
vmknic
Management
Uplink port group
uplink port 2
uplink port 0
uplink port 1
uplink port 2
Physical network adapters vminc0
......
vminc1
vminc0
vminc3
vminc1
vminc3
Physical Switch
vNetwork Distributed Switches (vDS)
Another switching technology native to VMware is the vNetwork Distributed Switch or vDS. Again, very briefly, a vDS is configured on your vCenter server and propagates to your ESXi hosts providing centralized management and administration. One point that differentiates the vDS from the vSS is that configuration is performed at the cluster level and propagated to the hosts instead of managed on each host individually. A vDS will do all the things that a vSS will do with some notable improvement over vSS. [ 50 ]
Chapter 4
Let's take a look at the similarities first. Standard and distributed switches support VLAN segmentation and 802.1q encapsulation for VM traffic, provide outbound traffic shaping, allow for NIC teaming, and are able to forward L2 frames. vDS also offers the ability to shape inbound traffic, support private VLANs, and as previously mentioned, supports central management. Improvements to vDS capabilities in VMware 5.x saw native support for NetFlow, the ability to monitor and mirror ports via DVMirror, improvements in link aggregation that provide hashing algorithm choices as well as an increase in the number of supported link aggregation groups. Also, new for 5.x is the support for Link Layer Discovery Protocol (LLDP), traffic filtering within a vDS, and the ability to shape inbound traffic. Other bonuses are support for single-root I/O virtualization and 40 GB NIC support. The following is a depiction of a Distributed vSwitch: VM
VM
VM
VM
Distributed port groups
vNIC
Production
Management
vMotion
VM
Test environment
VMkernal adapter (vmknic)
Management
vMotion
vSphere Distributed Switch Uplink port group vCenter Server Uplink0
Uplink1
Uplink2
ESXi host 1 Production
Test environment
vMotion
ESXi host 2 Management
Production
Test environment
Uplink port group
uplink port 0
uplink port 1
vMotion
Management
Uplink port group
uplink port 2
uplink port 0
uplink port 1
uplink port 2
Physical NICs vminc0
......
vminc1
vminc0
vminc3
Physical Switch
[ 51 ]
vminc1
vminc3
Network Design
You might be wondering why the comparison? The similarities and differences are important to decisions about what technology to employ and what considerations you need to make when implementing.
Common networking design considerations
In this section, we will discuss some common considerations regarding networking that are independent of virtual switch technologies. Again, these are potential pitfalls that an intelligent vSphere design will try to avoid from the start. First up are traffic types. These break down as shown in the following table: Traffic type
I/O requirements
Other requirements
Management
Low
Secure and HA
vMotion
High
High I/O at times
Fault tolerance
Medium to high
HA low latency
iSCSI/NFS
High
HA, fast, and low latency
VM
Depends!
Depends!
The items in the preceding table are explained as follows: • Management: This has ESXi to vCenter traffic, HA traffic, and vCenter to host communications. • vMotion: Moving VMs can consume all of a 10 GB link, which may impact high-I/O VMs. Not sensitive to throttling, vMotion may be an ideal candidate for shaping. Consider though that throttling may make the vMotion operation take longer. • Fault Tolerance: This replicates the state of active VM to the secondary FT VM, can be bursty and high-consumption as all I/O and memory-state info is passed continuously. • iSCSI/NFS: This has access to the disk. Contention can cause storage latency. 10 GB links are recommended. Also avoid contention issues as they will impact storage performance. (Jumbo Frames are your friend here.) • VM: This depends on what the VM is doing. You want to avoid contention for high I/O demands from VMs.
[ 52 ]
Chapter 4
Best practices to implement in your design
The best practice to implement in your design is to segregate traffic with an eye towards type and the unique requirements of each category. For example, consider a design with hosts that possess 2x 1 GB NICs and 2x 10 GB NICs. In this scenario, it would not be advised to run iSCSI on 1 GB links and VM traffic only on both of your 10 GB links. Deciding how best to structure your uplinks and traffic flows are critical to a well-performing design.
Implementing Network I/O Control (NIOC)
NIOC can allocate bandwidth to defined resource pools. There are a number of defaults you may choose from or you may create and implement your own; usually you'll end up with a mix of some sort. The default resource pools are ESXi management, iSCSI, vMotion, Fault Tolerance (FT), vSphere Replication (VR), NFS, and VM traffic. Resource pools are linked with port groups and shares or limits are used to shape traffic. Additionally, you may assign specific 802.1Q priorities to pools. NIOC can guarantee bandwidth for specific needs and can prevent any one resource pool from impacting the others. To implement NIOC, we need to perform the following steps: 1. Select a Load Balancing and Failover Policy that best suits your needs and design. Select what type you wish to employ to balance traffic over adapters and prevent NIC or physical networking failures from taking your environment down. 2. Implement TCP Segmentation Offload (TSO) and jumbo frames on your switches. TSO is enabled at the VMkernel interface by default, but you have to enable it at the VM as well in order to take advantage of it. Jumbo frames increase the maximum transmission unit (MTU) size to greater than the default of 1500 to typically 9000, and enables your switches to pass larger packets. Larger packets with less fragmentation means greater throughput. Ensure you do this on both the virtual switches and the physical switches you are connected to, and remember to account for any packet encapsulation to avoid fragmenting large frames. This is especially important when doing IP-based storage such as iSCSI. Consider physical adapters that speak NetQueue. When you have adapters that support NetQueue, you can leverage multiple receive queues, and offload reassembly from the host CPU and perform it on the adapters. [ 53 ]
Network Design
Shape traffic where appropriate. In broad strokes, average peak and burst bandwidth define shaping policy. VMware allows you to create and implement shaping policies for each vSS port group or vDS distributed port or distributed port groups. Remember that vSS allows outbound shaping only, while vDS allows you to shape traffic both ways. In addition, you can define a max burst rate to allow for short periods where you need to pump packets at a higher rate. VMware 5 saw the advent of vMotion with multiple adapters. Implement this where possible as the VMkernel will transparently load balance vMotion traffic over all of the vMotion-enabled VMknics to saturate all of the connections. In situations where there is a single vMotion, VMkernel will use all of the available network adaptors to spread the vMotion traffic. Consider separating network services. Do this by creating separate vSS or vDS switches for each service or implementing port groups for each service and segregating traffic with VLANs. This creates a more secure environment with associated performance benefits. If security is of great concern in your design, separate your VMs with firewalls between your VLANs.
Providing network redundancy, resiliency, and throughput
At this point, you should have a good understanding of the capabilities of your ESXi configuration and best practices. There are a few things you will want to do to ensure that you're able to maximize your investment. This starts with providing the best solution on the virtual side, but also extends to the physical; it makes zero sense to provide a fantastic vDS design that connects to an inadequate physical backend. We've discussed the virtual side, but before we get into the physical, be sure that you're thinking maximum redundancy in your virtual and physical network. This can be achieved with multiple adapters in hosts, hosts with multiple connections to multiple independent switches, switches that provide redundancy to provide connectivity, and so on. As we have discussed earlier, you can be assured that something will fail at some point and if you've built this infrastructure with redundancy as your foremost concern and speed as a close second, vSphere can, will, and is designed to continue to work even with multiple failures. We're on the 10 GB gravy train at this point, but there are some drawbacks to just loading up a host with network cards. Modern servers utilize the PCIe architecture. Keep in mind that there are limitations of the bus. Basically, it breaks down to the number of ports on a 10 GB card and the PCIe lanes the slot you're plugging that card into has available for bus-level communication.
[ 54 ]
Chapter 4
Single port 10 GB adapters require a PCIe x8 or better slot (PCIX 266 will also work), while dual port 10 GB adapters require a PCIe x16 or better slot. The so-called "bridge" chips, that take PCIe and downrev it to an earlier standard like PCIx are "bad" and should be avoided. These offload processing to CPU and others and are a performance hit and can be easily avoided. Also, beware of (and avoid!) any adapters that do this on-board, same story here with host CPU performance hits.
Physical switches and adapters
Multiple vendors supply datacenter switches—again, this (as it was the storage discussion) is not a sales pitch for a particular vendor, so we're not going to debate vendor preference, instead let's discuss major pitfalls we can avoid up front. Review your switch configurations to ensure that they're set up correctly with regards to link speeds, VLANs, port channel configurations, and so on. Often bad networking configs will cause networking anomalies; for example, on a port with link speed set at 100 MB or half duplex, or a port channel configured as PAgP when expected to be LACP. Be sure to compare your virtual switch configurations align with your physical configurations. There's no headache like troubleshooting performance for hours to find that a physical switch port MTU value is set to 1500 and your virtual switches are running jombo frames. It happens! If you don't have physical switch access, ask for the configuration files and parse them for errors. You'll save yourself those headaches with a little work up front. In addition, ensure that you have nothing that will cause traffic bottlenecks. Bad cables are often the culprit when diagnosing connectivity or speed issues. Be sure to check them first. Wherever possible, use 10 GB adapters. They're flat out more capable than their 1 GB counterparts. Use the fastest available to you, and ensure that the corresponding ports on physical switches are configured to support the speed of the adapters. VMware recommends the following features when selecting network adapters: • TCP Checksum offload (TCO): This offloads data verification of packets from ESXi to the network interface card • TCP segmentation offload (TSO): This allows large frames (up to 64 KB) regardless of MTU size by breaking the frame down into MTU sized frames with adjusted headers
[ 55 ]
Network Design
• Direct Memory Access (DMA): There should be 64-bit Direct Memory Access adapters • Multiple scatter gather elements per Tx frame • Jumbo frame support: This supports MTU in excess of 1500 (Typically 9000) • Large receive offload (LRO): This aggregates incoming packets into single streams On some 10 Gigabit Ethernet hardware network adapters, ESXi supports NetQueue, a technology that significantly improves performance of 10 Gigabit Ethernet network adapters in virtualized environments. Refer to KB 1004278 for details. It goes without saying that physical network adapters connected to the same vSphere standard switch or vSphere distributed switch should also be connected to the same physical hardware.
ESXi host hardware
At the host level, be aware of some specific concerns as they relate to the hardware itself when considering storage and storage networking. CPU utilization plays a significant role in network throughput. To process higher levels of throughput, more CPU resources are needed. The effect of CPU resource availability on the network throughput of virtualized applications is even more significant. Because insufficient CPU resources will limit maximum throughput, it is important to monitor the CPU utilization of high-throughput workloads. ESXi hosts' BIOS should be adjusted to max performance. When set to conserve power, you may run into performance issues caused by the BIOS "suspending" your NICs. Easy to fix, hard to diagnose; do yourself a favor and avoid this from the start.
Considerations when using IP storage
In Chapter 3, Storage Design, we discussed using iSCSI and NFS as a form of IP storage. When it comes to the network configuration of that storage, there are a few key areas you want to hone in on, which will ensure that your environment provides a healthy stability for years to come.
[ 56 ]
Chapter 4
Separating your traffic
This seems pretty straightforward but is often overlooked. This can lead to performance issues that will impact you when you least expect it. Storage traffic loads such as iSCSI and NFS should be separated from your VM traffic. This is best accomplished with separate interfaces (when using 1 GbE interfaces) or via VLAN tagging when using 10 GbE interfaces. Separation helps ensure that one workload will not impact another workload which pays dividends when it matters most.
Validating your storage configurations
Whether you're using iSCSI or NFS, you will want to ensure that your configurations across your ESXi hosts are identical. This is especially more pressing if using vSS as the configuration is set individually per host. What this really comes down to is walking through each of your configurations one-by-one to check for anything that may be inconsistent. Some common problems found here are MTU mismatches, speed, and duplex mismatches, and subnet mask misconfigurations. A misconfiguration in one of these areas can mean the difference between your storage performing and it failing to provide data access in the event an interface fails.
Summary
In this chapter, we learned about design principles of vSphere networking to provide further resiliency at this core part of the Virtual datacenter. We discussed the best practices and design considerations specifically around networking, virtual switches, IP storage, and networking design best practices. By the end of this chapter, you should be able to better assess your network's configuration to design and map your architecture to meet your business needs. Next up is virtual machine design; let's get into how to best architect your virtual machines.
[ 57 ]
Virtual Machine Design By now, you should have a fair understanding of the design principles that make up the core of your virtual datacenter. Virtual machines are the reason we've built such a comprehensive and resilient infrastructure to operate and support. Without these virtual machines and applications, there'd be no purpose to operate and support this environment. This chapter helps to peel back the covers and delve directly into the concepts of the virtual machines themselves. In this chapter, we'll be covering the following topics: • Virtual machine resources, templates, and best practices • Virtual machine migrations, including physical to virtual or virtual to virtual • Situations where you should allocate more than one vCPU and when you shouldn't • Shares, limits, reservations, and maximizing your infrastructure investment • Causes of virtual machine performance problems
Virtual machine resources and templates A virtual machine provides a layer of abstraction between compute, storage, and networking providing a consistent and uniform experience for the operating systems and applications running within them. Physical servers have numerous unique attributes associated with them such as different processor types, video cards, and network adapters. Virtual servers on the other hand, provide consistent hardware so that each virtual machine is as capable and portable as the next. These consistencies tend to be most noticed in the CPU-type, video card, hard disk, and network controllers.
Virtual Machine Design
What this means is that when you deploy a virtual machine, whether it is your first or your hundredth, there will be some pretty uniform and consistent steps you'd go through. Deploying your first virtual machine can be exciting, assigning various resources and installing an operating system, which you can dynamically resize and repurpose with minimal effort on your part. After your second or third VM, this can start getting mundane, which prompts the investment in learning about setting up templates. A template is basically a master image of a virtual machine that you can use to create new virtual machines. You start by creating the gold standard of a virtual machine you want in your organization, including the operating system, applications, and configuration settings. Once your master image is complete, you can convert that image into a template that allows you to deploy that image over and over again. In the past, setting up and creating virtual machines and templates was often an arduous process; these days it's often a simple click or right-click-type operation. With a wealth of knowledge on how to actually configure virtual machines and templates, we'll now focus on some best practices you'll want to apply when it comes to configuring these resources.
Best practices for virtual machines and templates
In this world of high speed, multicore CPUs, and large volumes of memory, the inclination is towards over-provisioning your VMs. Don't! Seriously, don't! The average physical server has two CPUs, which are utilized less than 5 percent, 8 GB of memory with an average of 2 GB memory used, and approximately 20 IOPS. That is the average physical server, which has all the benefits of a wasteful architecture while nothing else is in contention for its resources. It will be up to you to analyze your organization's needs and requirements based upon their utilization. A good rule of thumb is to start small and grow from there as needed. After analyzing an environment with less than 10,000 virtual machines with every workload under the sun (such as high performance SQL, Microsoft Exchange, file servers, print servers, and network monitoring applications), we found that 99 percent of these VMs were using less than 10 percent of each vCPU; approximately 2 GB of memory of an allocated 4 GB and that 50 GB of thin provisioned SATA storage was more than efficient. Outliers do and will exist, and we'll cover those in detail in the next chapter on business critical applications. But when it comes to the majority of applications, they tend to fall within this configuration template.
[ 60 ]
Chapter 5
The resource battle will always begin and end with you as the architect and operator of the virtual infrastructure. The most important part to emphasize here is that your virtual infrastructure is dynamic and flexible. You may have applications that require more than your default template; in fact, they may require far greater resources. The benefit of VMware virtualization is that without much thought on your part, you can simply expand those resources. If you over-allocate resources to your application owners, you'll be in a far more difficult position to reclaim those resources should they be found to not actually be using them.
Over-allocation can be a painful proposition
If resources have been over-allocated to your end-users, they may complain or state they can never shut down the server to have you reclaim them. By not allocating resources in the first place, there is nothing to reclaim. If you give people "just enough" resources and they need more, you can always expand them. When you go to reclaim resources, even if there is no actual performance implication, users will often claim any change in performance is due to you stealing their resources. It is for this reason that you should be encouraged not to over-provision resources in the first place. And lastly, while vCPUs and memory can very easily be rescinded, disk space cannot. While you can Storage vMotion a host to thin provision a 2 TB disk, which is only consuming 10 GB on disk, you won't be able to "shrink" the disk without deleting and recreating the disk, or taking it through a lengthy, disruptive, and painful VMware Converter migration.
Physical to virtual migrations
A physical to virtual (P2V) migration is the process of taking your physical infrastructure and instantiating it as a virtual machine, levering the uniform and consistent resources discussed earlier in this chapter. This is important as you're able to pick up your physical infrastructure and transport it into a virtual machine with minimal effort on your part. It is worth noting that there is a similar process called a virtual to virtual (V2V) migration, which allows you to move a VM from one infrastructure to another. When it comes to P2V, V2V, and disk resizing, there are many solutions on the market. Use of the VMware Converter standalone tool is advised. One of the primary reasons is that this tool is free, and the second is that it works quite well for the task. There are numerous types of machines that can be converted using this tool, from a powered-on server, an existing VMware Infrastructure VM, VMware Workstation, third-party VMs, Hyper-V, or even a backup image.
[ 61 ]
Virtual Machine Design
While this tool will convert a server or workstation into a virtual machine image that you can use in your virtual infrastructure, you are not required to replace or shut down the original one. That is to say that you can test out your physical server as a VM by converting it into a virtual machine and running it in a testbed to ensure that the end-state environment will meet your needs. This not only provides you a fully running and operating backup of your server, it also allows for portability that you're not likely to experience in a physical server. One other section that is overlooked when using this tool is the ability to resize the end destination to fall in line with any templates or best practices subscribed within your organization. Just because the source server had eight CPUs allocated with 32 GB of memory and a 2 TB hard disk doesn't mean that the end state of this server needs to have those requirements, especially if this is a print server or similar low-utilization application server.
Multi-vCPU considerations
When it comes to deciding whether to allocate virtual CPUs, you should always initially consider allocating fewer vCPUs. When it comes to vCPUs and CPUs in general, they are a finite resource not easily added or replaced. The total number of vCPUs assigned to a VM cannot exceed the total number of physical cores on the host. When you over-allocate vCPUs, you can cause issues that will ultimately impact performance on all of your virtual machines, not merely the one which has been over-provisioned. Why this becomes important in multi-vCPU consideration is that, when you assign more vCPUs than are required to a virtual machine, even if those resources are left idle, they will still compete for system resources, which can cause resource retention. This is in no way a judgment to say to avoid allocating vCPU resources where they are legitimately needed. As discussed before, start off small, as you can always increase resources with minimal impact; the same cannot be said so easily about reclaiming resources when misappropriated.
Shares, limits, and reservations
Shares, limits, and reservations always influence performance. If a virtual infrastructure should reach a point of resource contention, these settings should be implemented to reduce strain and burden in your environment. In a perfect world, with an infinite amount of resources, you will never experience any resource contention or problems. In those not-so-perfect worlds or tightly resourced environments, it is important to understand how shares, limits, and reservations will impact your organization. [ 62 ]
Chapter 5
Configuring shares
Resource allocation initially starts off by configuring shares, which is divided by default into four settings of Low, Normal, High, and Custom. These four share designations default to counts of 1000, 2000, 4000 (or 1:2:4 ratio), and the default configurable and customizable setting to whatever previous share setting was configured. These settings can be configured across your CPU, memory, and disk. Reasons you might go about configuring shares is when you are leveraging resource pools or have an otherwise contentious infrastructure; this allows you to establish a means of prioritization or Quality of Service (QOS). Should your infrastructure not experience any contention, the shares, while configured, will impact dynamic entitlement in order to time-slice resources. If there are extra resources, shares help influence who gets more of the extra. In practice, vSphere typically works well with resources when they are abundant, so leaving configurations at defaults is desirable unless you have a specific use case you are configuring for.
Limits
Resource allocation hits another threshold here when it comes to setting limits. Unlike shares, which have no meaning until you experience contention, limits are just that. Limits by default are set to Unlimited, which really seems supportive of how shares are handled, but when configured can be set as high as the total consumable resource available. CPU limits are measured in MHz, memory limits are measured in MB, and disk limits are measured in IOPS. Limits may be the most appropriate decision for some organizations, and is a way to help ensure that shares configurations are kept in order while ensuring a finite amount of resources allocated to some applications. In practice, this is usually dedicated to extremely resource-constrained environments and is very popular when your infrastructure equipment is rather outdated. VMware generally recommends that you avoid the use of limits. While there are some environments that have a use case, most do not. You may want to avoid this in production unless you happen to have a specific use case which supports this.
Reservations
Reservations are the last bastion of hard and fast resource enforcement. Unlike shares and limits, this can only be configured on CPU and memory. The purpose behind reservations is to guarantee that a certain amount of resources will be provided to the virtual machine, and thus, unlike shares and limits, there is no chance for another application to consume those resources or cause contention with them. When you over-commit resources to allow the applications to consume more than available, reservations help to ensure that applications that have a critical need will be guaranteed those resources. The majority of your servers and services will not require their resources to be guaranteed. [ 63 ]
Virtual Machine Design
In addition, enforcing or setting reservations can cause difficulties with vMotion and HA depending on admission control policy, which should be closely monitored if you choose to use them. In practice, the decision to leverage reservations typically involves scenarios where you want to guarantee resources of mission-critical applications; a topic discussed in Chapter 6, Business Critical Applications.
Causes of virtual machine performance problems In a perfect virtual infrastructure, you will never experience any performance problems and everything will work well within the budget that you allocated. But should there be circumstances that happen in this perfect utopian datacenter you've designed, hopefully this section will help you to identify and resolve the problems easier.
CPU performance issues
The following is a summary list of some the common CPU performance issues you may experience in your virtual infrastructure. While the following is not an exhaustive list of every possible problem you can experience with CPUs, it can help guide you in the right direction to solve CPU-related performance issues: • High ready time: When your ready time is above 10 percent, this could indicate CPU contention and could be impacting the performance of any CPU-intensive applications. This is not a guarantee of a problem; however, as applications which are not nearly as sensitive can still report high values and perform well within guidelines. Whether your application is CPU-ready is measured in milliseconds to get percentage conversion; see KB 2002181. • High costop time: The costop time will often correlate to contention in multi-vCPU virtual machines. Costop time exceeding 10 percent could cause challenges when vSphere tries to schedule all vCPUs in your multi-vCPU servers. • CPU limits: As discussed earlier, you will often experience performance problems if your virtual machine tries to use more resources than have been configured in your limits.
[ 64 ]
Chapter 5
• Host CPU saturation: When the vSphere host utilization runs at above 80 percent, you may experience host saturation issues. This can introduce performance problems across the host as the CPU scheduler tries to assign resources to virtual machines. • Guest CPU saturation: This is experienced on high utilization of vCPU resources within the operating system of your virtual machines. This can be mitigated, if required, by adding additional vCPUs to improve the performance of the application. • Misconfigured affinity: Affinity is enabled by default in vSphere; however, if manually configured to be assigned to a specific physical CPU, problems can be encountered. This can often be experienced when creating a VM with affinity settings and then cloning the VM. VMware advises against manually configuring affinity. • Oversizing vCPUs: When assigning multiple vCPUs to a virtual machine, you would want to ensure that the operating system is able to take advantage of the CPUs, threads, and your applications can support them. The overhead associated with unused vCPUs can impact other applications and resource scheduling within the vSphere host. • Low guest usage: Sometimes poor performance problems with low CPU utilization will help identify the problem existing as I/O or memory. This is often a good guiding indicator that your CPU being underused can be caused by additional resources or even configuration.
Memory performance issues
Additionally, the following list is a summary of some common memory performance issues you may experience in your virtual infrastructure. The way VMware vSphere handles memory management, there is a unique set of challenges with troubleshooting and resolving performance problems as they arise: • Host memory: Host memory is both a finite and very limited resource. While VMware vSphere incorporates some creative mechanisms to leverage and maximize the amount of available memory through features such as page sharing, memory management, and resource-allocation controls, there are several memory features that will only take effect when the host is under stress.
[ 65 ]
Virtual Machine Design
• Transparent page sharing: This is the method by which redundant copies of pages are eliminated. TPS, enabled by default, will break up regular pages into 4 KB chunks for better performance. When virtual machines have large physical pages (2 MB instead of 4 KB), vSphere will not attempt to enable TPS for these as the likelihood of multiple 2 MB chunks being similar is less than 4 KB. This can cause a system to experience memory overcommit and performance problems may be experienced; if memory stress is then experienced, vSphere may break these 2 MB chunks into 4 KB chunks to allow TPS to consolidate the pages. • Host memory consumed: When measuring utilization for capacity planning, the value of host memory consumed can often be deceiving as it does not always reflect the actual memory utilization. Instead, the active memory or memory demand should be used as a better guide of actual memory utilized as features such as TPS can reflect a more accurate picture of memory utilization. • Memory over-allocation: Memory over-allocation will usually be fine for most applications in most environments. It is typically safe to have over 20 percent memory allocation especially with similar applications and operating systems. The more similarity you have between your applications and environment, the higher you can take that number. • Swap to disk: If you over-allocate your memory too high, you may start to experience memory swapping to disk, which can result in performance problems if not caught early enough. It is best, in those circumstances, to evaluate which guests are swapping to disk to help correct either the application or the infrastructure as appropriate. For additional details on vSphere Memory management and monitoring, see KB 2017642.
Storage performance issues
When it comes to storage performance issues within your virtual machine infrastructure, there are a few areas you will want to pay particular attention to. Although most storage-related problems you are likely to experience will be more reliant upon your backend infrastructure, the following are a few that you can look at when identifying if it is the VM's storage or the SAN itself: • Storage latency: Latency experienced at the storage level is usually expressed as a combination of the latency of the storage stack, guest operating system, VMkernel virtualization layer, and the physical hardware. Typically, if you experience slowness and are noticing high latencies, one or more aspects of your storage could be the cause. [ 66 ]
Chapter 5
• Three layers of latency: ESXi and vCenter typically report on three primary latencies. These are Guest Average Latency (GAVG), Device Average Latency (DAVG), and Kernel Average Latency (KAVG). °°
Guest Average Latency (GAVG): This value is the total amount of latency that ESXi is able to detect. This is not to say that it is the total amount of latency being experienced but is just the figure of what ESXi is reporting against. So if you're experiencing a 5 ms latency with GAVG and a performance application such as Perfmon is identifying a storage latency of 50 ms, something within the guest operating system is incurring a penalty of 45 ms latency. In circumstances such as these, you should investigate the VM and its operating system to troubleshoot.
°°
Device Average Latency (DAVG): Device Average Latency tends to focus on the more physical of things aligned with the device; for instance, if the storage adapters, HBA, or interface is having any latency or communication backend to the storage array. Problems experienced here tend to fall more on the storage itself and less so as a problem which can be easily troubleshooted within ESXi itself. Some exceptions to this being firmware or adapter drivers, which may be introducing problems or queue depth in your HBA. More details on queue depth can be found at KB 1267.
°°
Kernel Average Latency (KAVG): Kernel Average Latency is actually not a specific number as it is a calculation of "Total Latency - DAVG = KAVG"; thus, when using this metric you should be wary of a few values. The typical value of KAVG should be zero, anything greater may be I/O moving through the kernel queue and can be generally dismissed. When your latencies are 2 ms or consistently greater, this may indicate a storage performance issue with your VMs, adapters, and queues should be reviewed for bottlenecks or problems.
The following are some KB articles that can help you further troubleshoot virtual machine storage: • Using esxtop to identify storage performance issues (KB1008205) • Troubleshooting ESX/ESXi virtual machine performance issues (KB2001003) • Testing virtual machine storage I/O performance for VMware ESX and ESXi (KB1006821)
[ 67 ]
Virtual Machine Design
Network performance issues
Lastly, when it comes to addressing network performance issues, there are a few areas you will want to consider. Similar to the storage performance issues, a lot of these are often addressed by the backend networking infrastructure. However, there are a few items you'll want to investigate within the virtual machines to ensure network reliability. • Networking error, IP already assigned to another adapter: This is a common problem experienced post V2V or P2V migrations, which results in ghosted network adapters. VMware KB 1179 guides you through the steps to go about removing these ghosted network adapters. • Speed or duplex mismatch within the OS: Left at defaults, the virtual machine will use auto-negotiation to get maximum network performance; if configured down from that speed, this can introduce virtual machine limitations. • Choose the correct network adapter for your VM: Newer operating systems should support the VMXNET3 adapter while some virtual machines, either legacy or upgraded from previous versions, may run older network adapter types. See KB 1001805 to help decide which adapters are correct for your usage. The following are some KB articles that can help you further troubleshoot virtual machine networking: • Troubleshooting virtual machine network connection issues (KB 1003893) • Troubleshooting network performance issues in a vSphere environment (KB 1004097)
Summary
With this chapter, you should be able to inspect existing VMs while following design principles that will lead to correctly sized and deployed virtual machines. You also should have a better understanding of when your configuration is meeting your needs, and how to go about identifying performance problems associated with your VMs. In the next chapter, we dive into the applications that your business is dependent on and how to make them perform optimally.
[ 68 ]
Business Critical Applications Business critical applications are usually treated with a very wary and cautious eye and your environment should be no different. We'll discuss how to architect application solutions to ensure performance, resiliency, and high availability for business critical applications such as SQL and Exchange. You will learn how to gather requirements and work with business stakeholders to translate those requirements into a successful and stable deployment. We will be covering the following topics in this chapter: • • • • • •
Special considerations for tier 1 applications Workload profiles HA, DRS, and vMotion considerations Resource considerations regarding CPU, Storage, Memory, and Network Ensuring high availability and application resiliency Other business critical application resources
Special considerations for tier 1 applications
First and foremost, let us address the fact that most often people consider virtualization as the platform for running nonessential, nonmission-critical applications. Sure this platform is fine for a print server or running an application, but certainly not our e-mail, let alone the core services that run our business. If you're of the opinion that tier 1 applications cannot be virtualized, allow this chapter to help dispel that belief and guide you into the future of "virtualization first" to support all of your applications. When it comes to the traditional mission-critical applications people think of Exchange, SQL, and other workloads. These are not only capable of being virtualized but also are able to perform in ways that your traditional environment may be unable to realize today. The first steps of virtualizing this infrastructure need not be complex; in fact, it should be handled with the exact opposite attitude.
Business Critical Applications
Starting simple
The inclination to deploy or architect a business critical application is for the effort to be painful and complex. It doesn't have to be! Start simple and remember that a lot of the work has already been done. These applications have been around for a while and a lot of others have been virtualizing them for years, which allows us to take advantage of those benefits. The following are a few simple rules that will save you a lot of time and complexity: • Refer to support policies, recommendations, and best practices • Architect for the application, not for the virtualization solution • Defaults are your friend until you tune and optimize These are not the end-all and be-all, but tend to work pretty reliably in most organizations. Refer to the specific application vendor's guidance, but remember our discussion of when to apply best practice recommendations and make intelligent choices about what will work for you—guidance is, after all, guidance and not requirements. If you find that your requirements act considerably above or below what is being recommended, then act accordingly. That is to say, just because your vendor's best practices call for 64 GB of memory per instance does not mean you need that much memory if it goes unused, and equally if your needs far exceed, that you should limit your environment. This leads directly into architecting for the application, which should have been defined by the users and their actual requirements and not the perceived ones. Then apply those principles into how you'd go about designing as though it were physical and applying virtually. Avoid throwing resources at a design problem—we're talking virtual datacenters and applications built on VMware. We don't have the hard and fast constraints that a purely physical infrastructure imposes upon us. We can add memory, CPU, disk and networking, as needed, easily in virtual datacenters. The point here is to understand the application's requirements and the right size of your resources to meet the needs and not over engineer in the hopes that 16 CPUs and 256 GB of RAM will keep you out of trouble! And the last and the most critical is the defaults until you tune. That is not to say stick with defaults; quite the opposite. It implies that your architecture initially should leverage defaults, so you can get into a pattern of defining what your baseline is and where your environment starts to wear level. After such a time as you get a good sense of what your peaks and valleys are, you should go in and tune aspects of the architecture that apply to the application. Some particular areas of interest here are tuning your application's memory and CPU utilization, so they do not conflict with the general operation of the system. Changing defaults just for the sake of changing defaults does no one any good, so be particular about the changes that you make for the benefit of your architecture. [ 70 ]
Chapter 6
In addition, beyond the application, this lends itself as the perfect opportunity to resize your application based on its needs, whether that includes increasing or decreasing resources based upon its actual use instead of its perceived use.
Workload profiles
Every business critical application carries with it a function, workload, and profile; consider a business critical application such as a unique and beautiful snowflake. Putting that analogy aside for a moment, all snowflakes are ice crystals and carry with them more common characteristics than they do unique ones, which allows us to profile our workloads enabling a consistent architecture to design and deploy. All applications will require CPU, memory, storage, and network resources; the application will help to define exactly how much of each resource is required.
Designing for Microsoft Exchange
Over time, the Exchange server has made significant modifications to its architecture shifting which resources are required over that time. In the beginning, Exchange was memory light and storage IOPS heavy, but as Exchange 2003 was staged out in favor of Exchange 2007, 2010, and 2013 a fundamental shift in its architecture moved it to being memory heavy. While at the same time that the storage IOPS decreased, the storage consumption requirements increased with the loss of Single Instance Storage (SIS). With these changes in Exchange, you'll want to consider whether you'll be building on an existing deployment or starting fresh with a brand new Exchange infrastructure. Irrespective of which path you choose, you'll want to consider some of the following best practices for that architecture: • Do not P2V your Exchange servers, it is cleaner and easier to build new and move your users mailboxes • If you have existing Physical Servers, analyze their performance characteristics before and after you migrate users to ensure your performance stays the same or improves • Split your roles and size their CPU/Memory on a role-by-role basis and by their actual utilization, not simply what a sizing spreadsheet says should be allocated • If resource constrained, do not over-commit resources unless absolutely required • Size your Exchange VMs to fit within NUMA nodes for best performance [ 71 ]
Business Critical Applications
The Exchange server is typically one of the easiest business critical applications to stand up and virtualize and may require the least amount of effort and configuration. Following the best practice guidance for Exchange can save you a lot of effort and complexity, with a wary eye kept on the resources required versus those actually used. The scalability of virtualization may enable you to use fewer deployed servers to handle the same workload than you had experienced physically, so be sure to take that into account and not simply re-create one for one physical to virtual.
Considering Microsoft SQL server
SQL has actually a less contentious history with changes to its capabilities as Exchange does, but still carries with it a little baggage of drawing the distinction between its function and utilization. That is to say what the assumption typically is, is that if it is a SQL server it will immediately require extensive memory, CPU, and disk both in IOPS and storage. This is unfortunately not always the case and can lead to an oversized SQL architecture. In the previous chapter, a generic analysis of environments was referenced and more specifically an analysis of over 100,000 SQL workloads with the VMware capacity planner tool has identified that the average physical SQL server used two CPUs at 6 percent utilization, 3 GB of memory at 60 percent utilization, and about 20 IOPS of storage utilization. This is not to say that business critical SQL servers do not exist, in fact they exist in droves; this expresses the importance to architect for your application's usage and not simply having SQL on the label. Using the following formulae can allow for sample SQL application deployments: • Light workloads: One vCPU with 4 GB of RAM • Medium workloads: Two vCPUs with 8 GB of RAM • Heavy workload: 4 vCPUs with 16 GB of RAM • Very heavy workload: Assign the "maximum": and analyze its peaks and assign accordingly With a little effort on your part and analysis over the life of the application, you'll be able to ensure that your application environment runs better and faster. You'll also be allocated the most appropriate resources at the right time instead of over-assigning resources.
[ 72 ]
Chapter 6
Some special considerations that you'll want to ensure are part of your best practices are as follows: • Database and log LUNs need enough spindles. If your application calls for disk performance, ensure that the appropriate performance is backed by your SAN. • Evaluate workloads for SQL-intensive operations and optimize as appropriate. • Consider scaling out for high-end deployments, if you require more resources than a single physical host can provide. • Refer to best practices from your storage vendor and Microsoft and assign liberally. Remember, best practices don't always apply to your application's needs. • If clustering, follow best practices for SQL clustering. Most applications will not require clustering and can take advantage of VMware HA, but if an application requires 99.999 percent uptime, apply considerations as required. There are countless other items to reference, it is highly advised to read through the best practices for SQL on VMware's site and Microsoft SQL server and VMware virtual infrastructure best practices. The criticality of your application will require special attention to ensure it provides the availability, capabilities, and services that earn it a business critical designation.
HA, DRS, and vMotion considerations
Here's where some of the previously discussed features of vSphere really start to shine. By nature, a well-designed vSphere implementation is highly available and robust, but when we're talking about business critical applications, the following are some specifics to be aware of as they relate to application availability: • VMware HA: This isn't application aware! First and foremost, don't rely on this feature for application HA. Ensure your design has sufficient capacity for faultless vSphere HA operations. Consider the constraints of HA as they relate to VM admission control and plan for enough overhead to prevent HA problems, should you lose a host. In an environment where availability is critical, consider the cost of compute resources against the loss of revenue as multiple host failures could be costly in the long run. This is one area where it doesn't hurt to over-engineer a bit.
[ 73 ]
Business Critical Applications
• DRS and DRS rules: This can do a number of things for application availability. You can "pin" VMs to hosts with affinity rules, ensure loads are split with anti-affinity rules that ensure VMs run on different hosts, and automatically balance loads across your hosts. For example, if you were running an application with web frontends and SQL servers on the backside, DRS rules that pin the web servers and SQL VMs to the same host may give you increased performance by allowing the VMs to communicate via the server backplane. But, it might hurt CPU, memory, and inbound/outbound network bandwidth on the ESXi hosts. One potential mistake is "required" DRS rules, it's almost always better to use a "should" ruleset in a shared environment. • vMotion: This allows for automated migrations from overloaded or failed hosts in conjunction with DRS and HA. Additionally, mission-critical applications hosted on your VMs can be moved with the VM causing no interruptions in service to users of those apps. Be aware of factors such as storage consumption, bandwidth availability in your vSphere stack, and any DRS rules you have in place when doing vMotion or maintenance. Protect these critical apps and be aware that the even seemingly innocuous migrations may impact availability. • Fault Tolerance (FT): This is a feature of vSphere that provides VM level failure protection. There are unique planning requirements and capacity concerns when implementing FT VMs and you must be aware that we're talking VM fault tolerance—not application fault tolerance. FT isn't application aware. If you implement, be aware that there is another offering from VMware that gives you application fault tolerance. • Application HA (App HA): This is a VMware offering as part of the vCenter operations management suite. It currently affords application HA for some versions of Microsoft's SQL and Internet Information Server (IIS) offerings, Apache and Apache Tomcat, and SharePoint 2007 and 2010. New for Version 5.5 is App HA's ability to host and VM monitors with service monitoring. App HA can restart supported applications if it detects a failure. In order to use App HA, you must deploy two appliances that run vSphere App HA and vFabric Hyperic. • Also available is the VMware vSphere Guest SDK that allows for third-party monitoring solutions such as Symantec and NeverFail—and if you're inclined or have the budget, you can write your own monitors! For more information, please refer to: http://pubs.vmware.com/appha-1/topic/com.vmware. ICbase/PDF/vsphere-app-ha-install-guide-1.pdf.
[ 74 ]
Chapter 6
Resource considerations
Every application has its own resource considerations to be aware of, and every workload has its own needs. The following are some key areas you'll want to consider when it comes to resource assignment in general. Special considerations are treated on a per-application basis. • CPU: If best practices from the vendor calls for multiple vCPUs for your application, look at the utilization first before arbitrarily assigning them. The note we saw earlier about overassigning to see where your environment starts to level out in utilization applies here. Thus, only allocate vCPUs that are being used, as idle vCPUs will compete for system resources. If your workload is unknown, size for fewer vCPUs or intentionally overallocate and analyze to appropriately right-size later, after running your production loads to see their operational effectiveness. If this is a truly business critical application, ensure that the total number of vCPUs assigned is less than or equal to the total number of cores on the host. When you exceed that number, you'll introduce performance problems and slow your application down instead of speeding it up. • Memory: When it comes to your business critical workloads it has never been more important to allocate your memory based upon your application workload. Certain applications will have shared memory, which will be very common, duplicate, and respectively will dedupe very well. Unfortunately, many business critical applications that are unique will not have duplicate operations. That being the case, you may want to consider ensuring you do not oversubscribe your mission-critical workloads. Unique case applications specific to your business or data warehouses typically do not dedupe very well. So you will want to ensure that the memory allocated is available, setting reservations if needed. Doing this helps to avoid issues of memory contention or resource starvation when nonessential applications start to consume your primary resources. • Storage: Business critical applications' usage of storage is best treated in two ways. First, you focus on the IOPS required by the application; secondly, you ensure you have adequate capacity to support those requirements. With the evolution of flash storage and high-speed storage solutions on the market, the ability to supply IOPS to your applications has been getting easier than ever. However, to the points earlier about over-architecting the solution just because you can assign an application 10,000 IOPS doesn't mean anything if it is averaging less than 100. This is where ensuring that the right workload is running on the right tier of storage comes into play.
[ 75 ]
Business Critical Applications
• Network: We discussed this earlier in Chapter 4, Network Design, but ensure that you separate your workloads of storage and management traffic such as iSCSI, vMotion, and others, while leveraging VLAN tagging as appropriate. To get the highest throughput and bandwidth you should use a paravirtualized vNIC for high performance workloads. Each of these actions will require effort on the part of the operating system and configuration but certain applications do require high network load and networks do have hard limitations in how much throughput they can handle per interface. Lastly, if you have co-located applications that generate excessive network load between each other, consider hosting them adjacent to each other as network transactions, which traverse the same vSwitch operate inline as memory operations not going at NIC speed and can improve performance significantly.
Ensuring high availability and application resiliency Once you have an established business critical application, the sole function of your infrastructure beyond this point is not only to ensure that it operates within the performance thresholds you designed it for; it is to ensure that it remains up at all times. VMware has multiple features to help ensure that your service continues to operate even when things start to fail, starting with VMware vSphere High Availability (HA), which is a feature that protects against OS or hardware failure. That means, should your VM fail or the ESXi host it resides on fail, the virtual machine will be restarted on another host. This is similar to how Microsoft Clustering is used to protect against hardware failure without all of the complex configurations associated. To advance this further, VMware vSphere HA offers a VM monitoring function that provides application insight if the VMware tools application heartbeat should exceed a certain threshold. This requires integration with a third-party product such as App-Aware or NeverFail.
If an application requires a zero-downtime mirroring instances, you can leverage VMware vSphere FT, which has an identical instance of a virtual machine running in tandem. Details on how to leverage VMware FT are detailed in KB1013428 and should you look to leverage this extensively, refer to VMware's Whitepaper Protecting Mission-Critical Workloads with VMware Fault Tolerance at http://www.vmware.com/files/pdf/resources/ft_virtualization_wp.pdf.
[ 76 ]
Chapter 6
In addition to the mentioned functions, VMware vSphere DRS will be a critical member of ensuring availability of your business critical applications. In the past, DRS was not supported by application vendors such as Microsoft to support the migration of virtual machines between servers, but this is no longer an issue plaguing environments and can be fully leveraged. With DRS leveraged, if you should experience any resource faults with resource starvation, the virtual machines can be relocated to other hosts to limit these challenges.
Enabling CPU Hot Add in vSphere client
One feature that is advised not to be run on noncritical applications, but can have a significant impact on some business and mission critical, unable to accept even the smallest downtime is VMware vSphere Hot Add. VMware Hot Add is a capability that lets you assign additional virtual machine memory or Hot Plug virtual CPUs without having to incur an outage to increase the resources. There are some considerations when using this feature such as the guest VM has to be shut down to enable this feature, and not all guest operating systems support this feature. The previous screenshot (Enable CPU Hot Add in vSphere Client) shows how this is configured via the vSphere web client. Some applications have native functions intended to be used for resiliency of the service. These functions should be determined and architected for if their functionality supports your business needs, especially if these capabilities exceed those found within the native functionality of vSphere. Some examples of that being if your organization is leveraging Microsoft Exchange DAGs to replicate a copy of your database to a remote datacenter that provides a benefit above what HA is providing locally. In situations like that, Microsoft Exchange DAGs and LAGs should be leveraged over implementing storage replication and using the features of VMware Site Recovery Manager (SRM). In another circumstance, if you're leveraging Microsoft Clustering services for Microsoft SQL, but are not using Database Mirroring, you'll find more value for your dollar by simply allowing HA and DRS to handle the server in the event of a failure as you're gaining no additional resiliency with this implementation. [ 77 ]
Business Critical Applications
There are plenty of solutions for resiliency and capabilities native to the applications that we've discussed in this chapter and also refer to cited in some of the links within here that should be reviewed and determined whether your production environment will call for these. Lastly, the most often overlooked and quite possibly the most important aspect of ensuring application availability and resiliency of your environment falls on what your monitoring solutions are and how they're leveraged. There are numerous tools to monitor your infrastructure and your application environment, two of those are Microsoft System Center Operations Manager (SCOM) and VMware vCenter Operations Manager (vCOPS). These tools allow you to monitor the applications, their historical usage over time, as well as down to event-driven heartbeats and application availability.
The vCenter operations manager dashboard It is outside the scope of this book to detail every possible situation with which to leverage SCOM and vCOPS. However, remember that these tools will allow you to monitor your application and infrastructure performance and ensure that the reliability you expect is being achieved.
[ 78 ]
Chapter 6
Other business critical application resources
VMware has a pretty extensive and detailed white paper covering Virtualizing Business-Critical Applications on vSphere covering Microsoft Exchange, SQL, SharePoint, Oracle, SAP, Java, and Hadoop. This resource should be leveraged in addition to this chapter to help address your business critical application needs as it addresses in detail how licensing, support, and other items are handled. Refer to http://www.vmware.com/files/pdf/solutions/VMware-VirtualizingBusiness-Critical-Apps-on-VMware_en-wp.pdf for more details.
Summary
By the end of this chapter, you should be able to identify which of your VMs are business critical applications and ensure that their deployment meets your architectural considerations. You should also have a better understanding of what best practices are and when better practices apply to meet your business critical workload. In the next chapter, we look into what disaster recovery is and business continuance to keep your infrastructure running even in the worst of situations.
[ 79 ]
Disaster Recovery and Business Continuity By now you have what it takes to build a solid, stable, and enduring architecture that is resilient, scalable, and capable of handling your business requirements. However, up until this point, this architecture has been focused on local and adjacent resources; being able to survive an infrastructure failure by mitigating single points of failure does not eliminate catastrophic site failure. This final chapter deals with the disaster recovery (DR) and business continuity (BC) planning, processing, testing, and execution. This chapter provides guidance on how to extend local site resiliency to scale outside the walls of your datacenter. In this chapter, we'll be covering the following topics: • The benefits of disaster avoidance • Designing backup strategies • Designing for RPO and RTO requirements • Choosing replication technologies • Planning and testing the DR strategy
Disaster Recovery and Business Continuity
The benefits of disaster avoidance
When it comes to establishing a disaster recovery or business continuity plan, you have to consider what solution is appropriate for your environment. This has a direct impact and correlation in determining what your disaster avoidance threshold is. In the earlier chapters, we discussed local site disaster avoidance technologies such as High Availability (HA) and Distributed Resource Scheduler (DRS) and how they will actively monitor your environment for the failure of a host or virtual machine and move them around as applicable. A disaster avoidance solution provides similar features and capabilities but is extended across datacenters in metropolitan or geographic areas. The decision on where and how you implement this directly correlates to the business criticality of the application. Not all solutions need to take advantage of virtualization to achieve this as some native applications have application resiliency you can leverage (for example, Microsoft Active Directory, Microsoft Exchange Database Availability Group (DAG), Microsoft SQL mirroring, and Oracle Real Application Clusters (RAC)). However, there may be applications that do not have any native replication or distribution capabilities and rely upon some function of VMware vSphere replication, storage-based replication, and VMware Site Recovery Manager (SRM). To achieve an optimal implementation of a disaster avoidance strategy, it is best to categorize your application's tolerance for data loss and the impact on business operations with unavailability of the application. Most businesses have a few critical functions, which if nonfunctional will impact a business's bottom line. Each environment is unique in this regard, which is why it is important to understand and identify your specific business's needs and requirements. Some common applications, which if failed can impact a business's ability to function, are Active Directory and DNS, SQL databases, billing applications, and e-mail especially if customer-facing. This is just a small list but what most environments find is that the more technologically integrated they are, the more dependent they are on the majority of applications and services functioning, especially during an outage. There are steps that should be followed to ensure continued availability of services in the event of an outage or full recovery of services should you experience catastrophic failure.
[ 82 ]
Chapter 7
Designing backup strategies
The first step in ensuring that your business will be able to continue to function in case an epic failure strikes your organization is through the development of a solid backup and recovery program. VMware has numerous methods and means for backing up, which we will discuss, but it is important to establish some acceptable metrics for backup before we delve into that. Some of the methods are as follows: • Business data should be backed up and retained as long as the business requires recovery: Often, there is a disconnect when it comes to IT's implementation of backup policies and the requirements of the business. I'm not sure if that is due to confusion over the function of IT and the expectations of the business, but this is common in organizations of all sizes. This is to say, if there is an expectation that you can restore data from 90 days ago, that data needs to be present in some backed up format and should be supported by the business. If the business and IT are not aligned in this regard, the impact to an environment can be catastrophic for expectations of recovery. • All data that is backed up should be confirmed as restorable: When you back up data, if it is never restored or tested to confirm that restore is possible, it has no more value than if you hadn't taken a backup in the first place. This may seem like common sense but this is also fairly common place for restores of backups to fail due to never having been tested or configured incorrectly in the first place. Irrespective of the method or solution that you choose to back up data, you need to ensure that it can be restored in an easy and reliable fashion. • Backups should complete in a timely fashion ensuring that restores are timely: Organizations often have an expectation of service-level agreements (SLA) as to the availability of services and information. These will be put in metrics of seconds, minutes, hours, or days. If the organization requires that data be restorable within 15 minutes of a successful backup and the backup and recovery process each take hours to complete, the ability to achieve this SLA would be unreachable. This goes back to the two earlier points of ensuring data is backed up, retained as the business requires, and is made available in a timely fashion. When the demands of the business exceed the capabilities of IT, this will often require a modification in how IT handles recovery. Investments in infrastructure may be required to facilitate these needs.
[ 83 ]
Disaster Recovery and Business Continuity
• Back up with vSphere Data Protection (VDP): Backups are a subject in and of itself; instead of writing another book on this well-covered and well-understood subject, let's focus on VMware's offering for backups, vSphere Data Protection. There are two flavors available: vSphere Data Protection (VDP) and vSphere Data Protection Advanced. The following table gives us a quick look at the differences: Capability
VDP support
VDPA support
VMs per VDP appliance
100
400
Max datastore size
2 TB
8 TB
Datastore expansion
No
Yes
Support image-level backups
No
Yes
Microsoft SQL support
No
Yes
Microsoft Exchange support
No
Yes
Support for file-level restores
Yes
Yes
Obviously, the Advanced offering is the more capable. VDP and VDPA deploy as appliances, and care should be exercised when sizing them; once you roll them out, you can't expand them later. Size your VDP appliance(s) from the start to avoid performance problems or unnecessary VDP proliferation later. VDP integrates directly into the vSphere web client and allows control, scheduling, and restoring operations from the client. You're able to perform backups on VMs without installing an agent. Change Block Tracking (CBT) allows the kernel to track changes to VMs over time and vStorage API's for Data Protection (VADP) offloads processing from your ESXi hosts. Deduplication is built in and you're able to back up running and powered off VMs. There's even a facility for end users to restore single files from a web-based interface. As you can see, VDP offers a number of advantages over third-party or storage vendor solutions. That's not to say that these options don't have their place; they certainly do and VDP can't do it all (yet). Consider what works best in your design, be it VDP, third-party, or a combination of both.
[ 84 ]
Chapter 7
Designing for RPO and RTO requirements
Now that we have discussed how backups are performed and some technologies you can leverage, the key is to identify how those can, should, and are aligned to your business. Every business should have some defined Recovery Point Objective (RPO) and Recovery Time Objective (RTO) and these will directly impact what kind of backups you perform, where and how you store your data, and your business's tolerance for data loss and availability of services.
Recovery Point Objective (RPO)
Recovery Time Objective (RTO) Syncronous Real-time Replication
Best $$$$
WARM
Replication
Better $$$
$$ Traditional
$$$
Backup with Offsite Storage
Good
COLD $$
Backup to tape local only
$
$
No Backup Solution
Bad
Weeks
HOT
$$$$
Days
Hours
Minutes
Time
Minutes
Hours
Days
Weeks
Cost of investment measured in Time and Money as RPO and RTO approach zero
As you can see in the preceding chart, the closer your solution gets to a recovery point of zero data loss and a recovery time of zero seconds, the more expensive the solution gets. This is where the business needs to make the hard and fast decisions of what acceptable recovery points and recovery times are. One benefit of establishing an RPO and RTO plan is different applications can establish and be addressed with different RPO and RTO solutions. Thus, less critical services that do not change as often can be set to have data to be recovered within 24 hours, being backed up once a day, whereas a critical database application that handles customer records could be set to back up all changes—every transaction synchronizing every minute allowing the impact of loss to be minimized as close to zero as possible. When an organization does not have an established RPO or RTO, they also will typically not have any established SLAs. Organizations like these are less integrated with IT and will often see them as a cost center; the impact of this usually translates into the expectation of "best effort recovery" and the respective availability.
[ 85 ]
Disaster Recovery and Business Continuity
It can be difficult to translate best effort services as value back to the organization or be able to achieve any reliable figures for availability and recovery. This is why it is important to engage business requirements with IT capabilities and align the two in order to establish what reliable and deliverable capabilities to the business are. These RPO and RTO requirements will directly translate into what capabilities should be expected to be delivered by IT and will identify deficiencies that need to be corrected in order to achieve these metrics.
Choosing replication technologies
A fundamental part of protecting your infrastructure relies on the resiliency and robustness to protect your application stack. Once you've established what data you intend to back up, how often you intend to back it up, and aligned this backup schedule to your best effort RPO and RTO, the question now becomes whether this will require data replication as part of the solution. There are numerous ways you can go about replicating data, whether handled by an application, virtual machine, or as a backend operation handled by your storage. We'll be discussing the capabilities that exist in native vSphere replication and general storage type replications.
VMware vSphere Replication
VMware vSphere Replication is a new feature of the VMware vSphere platform designed to augment and extend the recovery capabilities, providing a native ability to replicate virtual machines within the cluster or to a remote site. This solution can be executed on a per virtual machine basis and can be integrated with orchestration solutions for disaster recovery, including integration with vCenter Site Recovery Manager. In addition, VMware vSphere Replication has automatic native integration with Microsoft Volume Shadow Copy Service (VSS) allowing for consistency and the ability to quiesce applications such as Microsoft Exchange and Microsoft SQL. If leveraging capabilities such as Exchange DAG or SQL Mirroring, ensure this doesn't conflict or impact scheduled RPOs for those applications.
[ 86 ]
Chapter 7
VMware vSphere Replication is a policy-driven protection system integrated within vCenter as a means of setting and complying with established points of recovery for your virtual machines. As you can see in the following screenshot, you are not only able to configure what your RPO is but you're also able to establish multiple copies to keep, allowing for the ability to recover to different points in time. It is this level of granularity that supports your organization being able to extend your backup implementation down to the virtual machine enabling for multiple layers of recovery and availability. Set your virtual machine Recovery Point Objective and retention of instances as shown in the following screenshot:
As you can see, the native power and capabilities within the vCenter suite inclusive of vSphere Replication provide even the smallest implementation of VMware to be able to replicate, recover, and protect their organization's investment in virtualization.
Storage-based replication
With a solution like vSphere Replication why would anyone want or even need a storage-based solution may leave you wondering. There are some situations where you'd want to leverage storage-based replication instead of VM based replication.
[ 87 ]
Disaster Recovery and Business Continuity
In circumstances like the preceding, you're put into a position to leverage your storage solution in order to achieve this replication. If left natively up to the storage entirely, you'll end up with what is called a "crash consistent" type of replication and recovery of the data. Crash consistent data is at a point in time without any regard for consistency, transaction, or application awareness. While the data would be at a remote site, there is no guarantee that the applications would be functional. In order to achieve application awareness and have some consistency, the virtual environment and the applications would need to interact with the storage with some kind of adapter to ensure replication happens correctly. If you're leveraging VMware Site Recovery Manager (SRM) in conjunction with storage-based replication, you will need to install a Storage Replication Adapter (SRA), which is specific to the storage vendor that you're leveraging. Each storage vendor creates and supplies these SRAs that are available on VMware's site. Some considerations when using SRAs is that not all vendor capabilities are supported. These are treated on a vendor-by-vendor basis and should be addressed accordingly in compliance with their best practices and recommendations. VMware vSphere Replication does not require an SRA as it does not leverage your storage array's replication capabilities.
Mixing replication solutions
One of the benefits of vSphere is its incredible flexibility and ability to support a mixture of application replication, VM-based replication with vSphere, storage-based replication, and different RPO and RTO schedules for each. Fortunately, unless you have very different business requirements on an application-by-application basis, you can reasonably choose one or a small mixture of solutions and implement it for your environment if your RPO tolerates this. Reasons you truly must separate the two loads will often be associated with a high volume of data rate in your more critical applications, and the respective bandwidth required to address the replication in compliance with your RPO/RTO. Even when supporting a mixture of applications your infrastructure does not only support a catastrophic disaster of your environment, you'll be able to leverage these solutions to enable high availability and business continuity of your virtual machines and applications across datacenters. The determination of how, when, and why you would choose to mix or separate those workloads and how you establish their SLAs and RPO/RTO should be a critical part of your disaster recovery and business continuity plan.
[ 88 ]
Chapter 7
Planning and testing the DR strategy
In order to achieve warm site or better capability, you will need a disaster recovery plan that can be implemented, tested, and executed in as automated a fashion as possible. VMware vCenter Site Recovery Manager (SRM) is the perfect candidate to provide a disaster recovery solution with automated orchestration and nondisruptive testing for your virtual machines and applications. Some of the key capabilities provided with vCenter Site Recovery Manager are as follows: • Centralized recovery plans managed directly within vCenter server • Failover of servers including reconfiguration of IP addresses • Automated failback to restore services to the host site after recovery has been executed • Nondisruptive testing allowing for DR testing without impacting production • Support disaster recovery SRM to a remote VMware service provider in the cloud In order to effectively leverage and implement SRM, this starts with setting up a site that is composed of resource, folder, and network mappings. Each of these attributes are tied to a local-side resource, which would then be mapped to a resource located at the destination. An example of this is if you have multiple network VLANs present at the source location, you'd need to map or assign where the virtual machines would point to at the destination for network connectivity to function. Once sites have been established, "array managers" are configured in order to identify what your local and remote arrays are. How these are configured is specific to the vendor and type of storage you have selected for your design. But, applicable to SRM is that this is where SRM knows the location of data at each location so that replication leveraging the Storage Replication Adapter (SRA) knows where, when, and how it goes about executing a replication as part of the tool. Optionally, if using vSphere Replication as we discussed, this section allows you to configure the relationship between the datacenters to support the configuration and the replication of the virtual machines. The next essential part is to configure protection groups that are combinations of one or more datastore groups. Protection groups are usually used in alignment with your SLAs and replication schedules to help identify which datastores and their respective virtual machines will replicate and their priority of starting up when configured in the recovery plan.
[ 89 ]
Disaster Recovery and Business Continuity
At this point, your implementation can leverage SRM to set up recovery plans that meet the requirements of your RPO and RTO discussed earlier. A recovery plan is made up of all of the components we just discussed, all brought together under a single umbrella. While setting up your recovery plan, you identify a site to use, which protection groups you wish to leverage, and confirm the configuration of the test networks used while staging a test DR plan. At this point, you should run a DR test to ensure that if you need to execute a recovery, all of the components will function and operate. One additional feature is the ability to configure the virtual machines in the recovery plan individually. This is especially important if certain VMs have special requirements that apply at the remote site, or unique characteristics you'd leverage in a failover. Some examples of this would be choosing to not turn on certain machines in a failover or changing the boot order of some VMs to have them come up only after another VM had already started. The test mode feature lets you practice this operation seamlessly and all of the automation that exists behind the scenes, so you're able to ensure should you have to recover, you have a tested example of exactly what your outcome will be. The following screenshot shows the output of a recovery:
After a test has been executed, you can simply clean up your environment destroying the entire test environment. VMware SRM exposes all elements of the testing, clean up, recovery, and reprotect state, so you can be confident in the outcome when it matters most during a catastrophic event.
[ 90 ]
Chapter 7
Summary
At this point, you should have a better understanding of what parameters and components are involved in disaster recovery and business continuity to ensure your business critical applications are able to take resilience and availability to the next level. Organizational alignment of business with an IT strategy can help ensure that your environment is able to be well documented with well-tested disaster recovery policies.
[ 91 ]
vSphere Automation Tools This appendix will talk briefly about the value of automation tools such as vCenter Orchestrator (vCO) and vCloud Automation Center (vCAC), their differences, and various use cases. The reader will also learn how to incorporate these tools into their vSphere designs. In this appendix, we will be covering the following topics: • VMware vCenter Orchestrator • VMware vCloud Automation Center • Designing for automation and XaaS
Introducing VMware vCenter Orchestrator
Every organization typically starts out the same with their virtualization footprint, often consolidating and performing the kinds of actions they would from a physical environment, now in a virtual one. An area often overlooked because the capability did not exist previously, whether physically or virtually, is automated configuration and orchestration. VMware vCenter Orchestrator helps to bridge that gap by providing capabilities to integrate with IT processes, which were either difficult or impossible in the past. VMware vCenter Orchestrator has a workflow engine that is used to help in the automation of bulk operations and create workflows that you can integrate with components of vCenter and can also plug in to third-party applications. This component exists as a standalone product installable on Windows or one that can be installed as a Virtual Appliance (vApp).
vSphere Automation Tools
A few reasons why people most often initially start to leverage vCenter Orchestrator are as follows: • Minimizing human error by automating a task performed for every VM deployment • Orchestrating a repetitive task that may be redundant or time consuming • Ensuring consistency in your virtual machine deployments through workflows There are numerous other reasons that may or may not impact your own business and organization. Some areas where this tool can save you a lot of time and heartache is identifying what matters most to your organization and ensuring that an orchestration workflow guarantees a consistent outcome. In addition, if you find your organization is error prone in one area, through the creation of a process, you can ensure that these errors are eliminated and improve the quality of service in your virtual environment. As you can see in the following figure, this workflow is written with intelligence while performing a virtual machine disk request to check for storage space. In this example, if the space is available, the workflow continues to operate and provision the space. In the event that the requested space is not available, the workflow changes gears entirely and submits a change request for additional space to be provisioned. How the workflow handles that is entirely up to the organization or maintainer. This could be handled by putting a request out to the storage team to provision additional space, or even integrate with a third-party API to perform the provisioning activity on behalf of the workflow. The power of vCenter Orchestrator is only limited by your imagination and your skills to automate and integrate all of your redundant activities.
[ 94 ]
Appendix A
CMDB
VM Disk Reques
Check Storage space
Enough free space?
Add Disk
updateCMDB
Notify User
Create Change Req
An example of a vCenter Orchestrator workflow
If you are just getting started with vCenter Orchestrator, there is a wealth of knowledge by vCenter Orchestrator expert, Cody Bunch, at http://professionalvmware.com. In addition, VMware Education Services has a series of instructional and useful videos to help you get familiar and comfortable with using this vital tool for your organization's growth and evolution. It is available at http://vmwarelearning.com/orchestrator/.
Introducing VMware vCloud Automation Center
With vCenter Orchestrator providing orchestration and automation functions, what is the purpose of vCloud Automation Center? This question, although often posed, is quickly answered. While vCO does provide you with automation capabilities typically driven through workflows and process automation, vCAC allows you to take your organization numerous steps further. VMware vCloud Automation Center is a suite of components, which can and will leverage vCenter Orchestrator workflows. Although you are able to take advantage of vCO standalone without vCAC, when you combine the two, you're able to extend your capabilities even further.
[ 95 ]
vSphere Automation Tools
Unified IT service catalog
VMware vCloud Automation Center supports a unified service catalog where users can deploy virtual machines from a prebuilt menu of offerings. Just like in a restaurant where you can select an item from a menu, this service catalog lets you choose items and customize as the offerings allow. As you can see in the following screenshot, you can build your own customized menu experience for your end users to leverage. In organizations with numerous similar requests, you can leverage a capability like this service catalog to provide a self-service experience to your users by putting the power into their hands.
An example of a vCAC service catalog
Advanced Service Designer
With vCAC, there is an Advanced Service Designer that is a wizard-driven approach providing an end-to-end process delivery of custom services within your service catalog. Taking the deployment of just virtual machines and service to another level, some of the use cases typically performed in a vCAC automated delivery are as follows: • New employee on board • Active Directory management tasks [ 96 ]
Appendix A
• Software updates and software upgrades • Password and user profile management • Backup and restore of user data or services Essentially, anything that can be done manually can often be configured and delivered as a service. This takes what was just a virtual infrastructure to run your servers and extends those capabilities outward from IT and into the community user space empowering the user. An audit of regular helpdesk tickets and calls is a good place to start in order to find areas to empower your user community and take the burden off of manual administrative tasks.
Multivendor interoperability
With the growing trend of integrated and extended networks, there has been an increased need to be able to scale, grow, or extend your environment within and throughout clouds. VMware vCloud Automation Center takes that request and provides the ability for interoperability and connectivity to a myriad of cloud service providers. The following figure shows integration with hybrid cloud providers, VMware vCloud Hybrid Service, vCloud Providers, and OpenStack and Amazon Web Services, to name a few: vCloud Automation Center Self-Service Catalog Automated Delivery
Physical
Policy-Based Governance
Intelligent Resource Management
Virtual
Cloud
cloud Provider
vCAC Hybrid Service Providers
[ 97 ]
vSphere Automation Tools
Designing for automation and XaaS
With tools such as VMware vSphere and VMware vCenter Server integrating into workflows with VMware vCenter Orchestrator and pulling it all together in a vCloud Suite, leveraging vCloud Automation Center allows you to take your organization far beyond the next step. XaaS or Everything as a Service is the next generation in organizational independence, allowing you to maximize your investment in technology and spend more time designing for the future and business requirements with less time fighting fires and performing mundane tasks. Just as we discussed in depth in the previous chapters, the key message throughout this book has been determining what your business requirements are, for example, what is it we are trying to accomplish and why are we doing it? With those few questions answered in depth, you'll have the answers you need to know where to focus your attention and harness the power of this technology to take your organization well into the future.
Summary
This appendix touched on a few very powerful tools available from VMware within the vSphere and vCloud suite. While most users will barely scratch the surface of the capabilities of vCO and vCAC, every journey begins with a first step. We became familiar with some of the tools and the ease and simplicity of their deployment. Through the use of tools such as these, you'll be able to further guarantee that your architectural plans and designs are executed enabling a solid and stable infrastructure that can self-maintain and self-heal.
[ 98 ]
Certification Resources This appendix will give guidance on the VMware certification roadmap and list some of the available resources for each certification track in Data Center Virtualization (DCV). Through this appendix, the reader will have a wealth of resources in order to pursue VMware certification. In this appendix, we will be covering the following topics: • VMware vSphere Datacenter Virtualization Certification roadmap • VMware vExpert program • Recommended reading—podcasts, blogs, and additional books • Exam and study tips
Certification Resources
VMware Certification Roadmap
VMware has a number of certification tracks and designations, which are differentiated by their delivery, difficulty, and expectation of knowledge required to pass them. In the following figure, the entire VMware roadmap is laid out for test-based certifications: Certification tracks
Data Center Virtualization
cloud
Expert
Engineer & Adminstrator
VCDX-Cloud
Advanced Professional Professional Progression Associate
Governance & Operations
Architect
Engineer & Administrator
Architect
Engineer & Administrator
Architect
VCDX-Data Center Virtualization
VCDX-Desktop
VCAPVCAPData Center Data Center Administation Design
VCAPDesktop Administration
VCP-Cloud
VCP5-Data Center Virtualization
VCP-Desktop
VCA-Cloud
VCA-Data Center Virtualization
VCA-Workforce Mobility
VCDX-Cloud VCDX-Cloud Infrastructure Infrastructure Administration Design
VCDX-Cloud Governance
Engineer & Administrator
VCAPDesktop Design
Available today
What's next?
Network Virtualization
End User Computing
VCA-Network Virtualization
New release
Coming soon
learn more and get started
vmware.com/certification
EDUCATION SERVICES
VMware Certification: Be exceptional
VMware certification track roadmap
With the focus of this book on Data Center Virtualization (DCV), we will put our attention on this area as we break down the exam types and their progression as shown in the following figure:
CERTIFIED ASSOCIATE DATA CENTER VIRTUALIZATION
CERTIFIED PROFESSIONAL DATA CENTER VIRTUALIZATION
CERTIFIED
CERTIFIED
CERTIFIED
ADVANCED PROFESSIONAL
ADVANCED PROFESSIONAL
ADVANCED PROFESSIONAL
DATA CENTER ADMINISTRATION
DATA CENTER DESIGN
DATA CENTER DESIGN
VMware certification track roadmap
[ 100 ]
Appendix B
Associate exams
The associate-level exam was introduced in late 2013 as an introduction and entry-level certificate to VMware virtualization. The intended audience for this entry-level certificate is those not familiar with or first getting their feet wet with virtualization. Included as a benefit with this track is a free, self-paced e-learning course that prepares you to take the exam. While not required, if you have minimal or no virtualization experience, this helps establish a foundation and improve your comfort and confidence with virtualization. In addition, the delivery of this exam can be entirely self-proctored via the Internet without requiring you to go to an official Pearson Vue testing center.
Professional exams
The professional-level exam differs from the associate exam in a number of ways. The knowledge and expertise required to pass this exam raises the bar higher; this exam is not for those without real-world VMware knowledge. Secondly, this exam must be taken at a Pearson Vue testing center. Finally, you are required to complete an appropriate VMware training course to receive credit for having passed the exam. There are numerous benefits associated with completing the VMware Certified Professional exam, a few of which are discounts on VMware Press, VMware events, and a complimentary license of VMware Workstation. The VMware Certified Professional is highly sought after by hiring managers and IT recruiters alike and will help to fortify your position in career advancement.
Advanced professional exams
If the VCP was that notch on your belt to differentiate you from your peers, the VMware Certified Advanced Professional (VCAP) takes it one step further. The VCAP differs substantially from the VCP in a number of ways. First, the administration version of this exam is lab-based where you perform actual tasks on equipment, validating your skills and knowledge at configuring and administering complex, large-scale virtualized environment. Second, as this is a lab-based exam, this exam is only offered at Pearson Vue professional centers, which may not be available at all locations or countries. Another consideration here is as the environment is a remote lab and if the link speed is slow, it can impact your ability to complete the exam on time. Preparation for this exam differs from simply knowing the correct answer to knowing how to perform tasks that may or may not be familiar.
[ 101 ]
Certification Resources
Expert-level defense
The VMware Certified Design Expert (VCDX) is the highest level of VMware certification. This certification is leaps and bounds above what you know and takes you to being able to demonstrate beyond a lab scenario, instead requiring you to design, submit, and defend an architectural design. The process consists of submitting a unique production-ready VMware architectural design. Once accepted, a date is selected whereby you will go before a veteran panel of VCDX holders where you will be required to explain and effectively defend your architectural design. It should be noted that not all submissions are accepted the first time. The VCDX defense panels are traditionally done at major conferences such as VMware Partner Exchange (PEX) and VMworld.
VMware vExpert program
The VMware vExpert designation, while not a certification, should not go unmentioned nor unnoticed. VMware introduced the VMware vExpert program several years ago and shares similarity to the Microsoft Valued Professional (MVP) program and EMC Elect Program. The vExpert title is effectively an "award" based upon your contributions to the virtualization community, society, industry, and beyond. Contributions are recognized by three distinct and separate paths— evangelist, customer, and VMware partner. vExperts are awarded the title of vExpert and are authorized to display the following logo:
Evangelist path
The evangelist path includes book authors, bloggers, tool builders, public speakers, VMTN contributors, and other IT professionals who share their knowledge and passion with others with the leverage of a personal public platform to reach many people. Employees of VMware can also apply via the evangelist path. A VMware employee reference is recommended if your activities weren't all in public or were in a language other than English.
[ 102 ]
Appendix B
Customer path
The customer path is for leaders from VMware customer organizations. They are internal champions in their organizations, or have worked with VMware to build success stories, acted as customer references, given public interviews, spoken at conferences, or were VMUG leaders. A VMware employee reference is recommended if your activities weren't all in public.
VMware Partner Network (VPN) path
The VPN path is for employees of partner companies who lead with passion and by example; those who are committed to continuous learning through accreditations and certifications and to make their technical knowledge and expertise available to many. This can take shape of event participation, video, IP generation, or even public-speaking engagements. A VMware employee reference is required for VPN path candidates. Nominations and applications for the VMware vExpert program were handled on a yearly basis until 2014 when quarterly nominations were introduced. This designation is not for everyone and is based on your involvement and contributions to virtualization and the community or your organization.
Recommended reading, blogs, and podcasts
Whether pursuing a specific certification path, rounding out your knowledge base or just looking to be aware of what is going on with and involving yourself in the community, the following are a small cross section of available resources from blogs, books, and podcasts you should be aware of: • http://professionalvmware.com/, which includes weekly vBrownBag online webinars • VMware Communities Roundtable podcast hosted by VMware • Virtualization Security Roundtable podcast hosted by Edward Haletky (@Texiwill) • http://vsoup.net, which includes a monthly podcast and blog posts • http://blog.scottlowe.org/, which is a blog by the infamous IT Pro and Virtualization guru Scott Lowe (@scott_lowe)
[ 103 ]
Certification Resources
• http://pkguild.com, which is author Christopher Kusek's (@cxi) blog focused on virtualization and storage • http://www.yellow-bricks.com/, which is a blog by Duncan Epping, Principal Architect R&D for VMware
Exam and study tips
There are a number of ways to advance your skills with VMware and this is the important bit here; whether VCP, VCAP, or VCDX, VMware is interested in practical experience and demonstrable skills. Build your own lab, experiment with the technology, or apply your skills in a real-world job setting. At the end of the day, passion for the technology will drive your progress, not cramming. VMware has structured the certification tracks to prevent certification without practical experience. There's a vast array of resources out there to leverage, to learn from, and to apply. That's the difference in VMware certifications. You have to know the technology inside out.
Summary
We've had a look at a part of VMware's certification roadmap, talked about basic, advanced, and expert-level certifications, discussed the vExpert program, recommended some of our favorite blogs and online resources, and also have given a few short pointers on how to study for the exams.
[ 104 ]
Index A
C
adapters 55 Admission Control (AC) about 20, 21 policies 20 Advanced Micro Devices (AMD) 23 Advanced Service Designer 96, 97 Application HA (App HA) considerations, for business critical application 74 associate-level exam 101 Auto Deploy 32
Change Block Tracking (CBT) 84 cluster design considerations, for HA 19 Conceptual Design about 6 creating 7 configurable storage features multipathing path selection policy, configuring 47 provisioning 47 SIOC 46 considerations, for IP storage usage about 56 storage configurations, validating 57 traffic, separating 57 considerations, network design 52 Converged Network Adapters (CNAs) 30, 38 CPU considerations, for ESXi hardware design 24, 25 performance issues, of virtual machine 64, 65 customer path 103
B backup strategies designing 83, 84 Booting from SAN 32 business continuity (BC) 81 business critical application Application HA (App HA), considerations 74 architecting 70, 71 DRS, considerations 74 Fault Tolerance (FT), considerations 74 HA, considerations 73 HA, ensuring 76, 77 other resources 79 resiliency, ensuring 76, 77 resource, considerations 75 virtualizing, considerations 69 vMotion, considerations 74 workload profile 71
D database availability group (DAG) 82 Data Center Bridging (DCB) 30 Data Center Virtualization (DCV) 99, 100 datastore design consideration 44
Device Average Latency (DAVG) 67 Direct Memory Access (DMA) 56 disaster avoidance benefits 82 disaster recovery (DR) 81 Distributed Resource Scheduler (DRS) about 82 considerations, for business critical application 74 DR strategy planning 89, 90 testing 89, 90
E EMC Elect Program 102 Enhanced vMotion Compatibility. See EVC ESXi hardware design about 23 CPU considerations 24, 25 memory considerations 26, 27 NUMA considerations 26, 27 vNUMA considerations 28 ESXi host hardware 56 evangelist path 102 EVC 24 Everything as a Service. See XaaS
F Fault Tolerance (FT) about 54 considerations, for business critical application 74 fiber channel over ethernet (FCoE) 29, 30, 39 Fibre Channel (FC) 38
considerations, for business critical application 73 ensuring, for business critical application 76, 77 ensuring, with CPU Hot Add 77 ensuring, with vCenter operations manager dashboard 78 host considerations 16, 17 network considerations 17, 18 storage considerations 19 HCL about 23 URL 23 High Availability. See HA host design considerations, for HA 16, 17 Host Bus Adapter (HBA) 30, 38 Host Isolation Response, options Leave Powered On 19 Power Off 19 Shutdown 19 hypervisor storage considerations, for JVM 31
I IBM DB2 15 Internet Information Server (IIS) 74 iSCSI 38 iSCSI/NFS 52
J
Guest Average Latency (GAVG) 67
Java virtual machine. See JVM jumbo frame support 56 JVM about 28 design considerations 28, 29 hypervisor storage considerations 31 NIC considerations 29, 30
H
K
HA about 26, 82 Admission Control (AC) 20, 21 cluster considerations 19
Kernel Average Latency (KAVG) 67
G
L Large receive offload (LRO) 56 [ 106 ]
latency about 66 DAVG 67 GAVG 67 KAVG 67 limits about 62 setting 63 Link Layer Discovery Protocol (LLDP) 51 local storage design considerations 43, 44 Logical Design about 6 creating 7
M management pod (Management Cluster) 12 maximum transmission unit (MTU) 53 memory considerations, for ESXi hardware design 26, 27 performance issues 65, 66 Microsoft Clustering Services (MSCS) 41 Microsoft Exchange designing 71, 72 Microsoft SQL server considerations 72, 73 Microsoft Valued Professional (MVP) program 102 MS SQL 15 MS SQL Express 15 multipathing path selection policy configuring 47 multi-vCPU considerations 62
N NetQueue 56 network best practices, for implementation 53 considerations 52 design considerations, for HA 17, 18 performance issues 68 Network Attached Storage (NAS) 31 Network File System. See NFS
Network interface card. See NIC Network I/O Control. See NIOC network redundancy providing 54 NFS about 39-42 benefits 42 drawbacks 42 URL, for best practices 42 NIC considerations, for JVM 29, 30 NIOC about 53 implementing 53 non-uniform memory access (NUMA) about 26 considerations, for ESXi hardware design 26, 27
O Oracle 15
P P2V migration 61, 62 Physical Design about 6 creating 7 physical switches 55 physical to virtual migration. See P2V migration professional-level exam 101 provisioning about 47 Thick Provision Eager Zero 47 Thick Provision Lazy Zero 47 Thin Provisioned 47
Q Quality of Service (QOS) 63
R Raw Disk Mapping (RDM) about 42 benefits 41, 42 [ 107 ]
drawbacks 42 Real Application Clusters (RAC) 82 Recovery Point Objective. See RPO Recovery Time Objective. See RTO replication technologies mixture of replication solutions 88 selecting 86 storage-based replication 87, 88 VMware vSphere Replication 86, 87 reservations about 62 configuring 63 resiliency ensuring, for business critical application 76, 77 ensuring, with CPU Hot Add 77 ensuring, with vCenter operations manager dashboard 78 providing 54 resource, business critical application CPU considerations 75 memory considerations 75 network considerations 76 storage considerations 75 RPO about 85 requirements, designing 85 RTO about 85 requirements, designing 85
S SAN 37 Scale-Out design 33, 34 Scale-Up design 33, 34 SDRS 45 service-level agreements (SLA) 83 shares about 62 configuring 63 Single Instance Storage (SIS) 71 single point of failure (SPoF) 16 Single sign-on (SSO) 10 SIOC 46 Site Recovery Manager (SRM) 77 about 82, 88, 89
key capabilities 89 stateless host design about 31 Auto Deploy 32 Booting from SAN 32 storage configurable storage, features 46 design considerations 43 design considerations, for HA 19 local storage, design considerations 43, 44 performance issues 66, 67 storage area network. See SAN storage-based replication 87, 88 Storage Distributed Resource Scheduler. See SDRS Storage I/O Control. See SIOC storage protocols about 37 FCoE 39 Fibre Channel (FC) 38 iSCSI 38, 39 NFS 39, 40 Storage Replication Adapter (SRA) 88, 89
T TCP Checksum offload (TCO) 55 TCP segmentation offload (TSO) 55 Thick Provision Eager Zero 47 Thick Provision Lazy Zero 47 Thin Provisioned 47 throughput providing 54 traffic type Fault Tolerance 52 iSCSI/NFS 52 management 52 VM 52 vMotion 52
U unified service catalog 96
V V2V migration 61, 62 VAAI 45 [ 108 ]
VAMP 45 VASA 46 vCenter platform, selecting for vCenter server 11, 12 vCenter database, selecting 14, 15 vCenter server appliance, using 12 vCenter server, sizing 13, 14 vCenter, components Single sign-on (SSO) 10 vCenter inventory service 10 vCenter server 11 vSphere web client 10 vCenter DB (vCenter database) IBM DB2 15 MS SQL 15 MS SQL Express 15 Oracle 15 vCenter Heartbeat about 12 URL 12 vCenter inventory service 10 vCenter, optional components vSphere Auto Deploy 11 vSphere client 11 vSphere ESXi Dump Collector 11 vSphere Syslog Collector 11 VUM 11 vCenter Orchestrator (vCO) about 93-95 incorporating 98 vCenter server about 11 platform, selecting for 11, 12 sizing 13, 14 vCenter server appliance using 13 vCloud Automation Center (vCAC) about 93, 95 Advanced Service Designer 96, 97 incorporating 98 multivendor interoperability 97 unified IT service catalog 96 vDS 50, 52 Virtual Appliance (vApp) 93 Virtual Data Center design best practices 7 components 8
Conceptual Design 6, 7 constructs 9 Logical Design 6, 7 Physical Design 6, 7 scenarios 9, 10 virtualization, business critical application considerations 69 Virtual Machine Disk (VMDK) 41 Virtual Machine File System. See VMFS virtual machine filesystems NFS 42 RDM 41, 42 VMFS 40, 41 virtual machine performance issues CPU performance issues 64 memory performance issues 65, 66 network performance issues 68 storage performance issues 66, 67 virtual machine resources about 59, 60 best practices 60 over-allocation 61 virtual machine templates about 59, 60 best practices 60 virtual NUMA. See vNUMA virtual to virtual migration. See V2V migration VM 52 VMFS about 40, 41 deploying, best practices 41 VMFS 5 features 40, 41 vMotion considerations, for business critical application 74 VMware recommended features, for network adapters selection 55 VMware Array Aware Integration. See VAAI VMware Certified Advanced Professional (VCAP) 101 VMware Partner Exchange (PEX) 102 VMware Partner Network path. See VPN path
[ 109 ]
VMware storage features 45 VMware storage API VAAI 45 VAMP 45 VASA 46 VMware vExpert program about 102 customer path 103 evangelist path 102 VPN path 103 VMware vSphere Replication 86, 87 vNetwork Distributed Switches. See vDS vNetwork Standard Switches. See vSS vNUMA about 28 considerations, for ESXi hardware design 28 Volume Shadow Copy Service (VSS) 86 VPN path 103 VSAN URL, for design consideration 44 vSphere 5.5 Configuration Maximums URL 12 vSphere Authentication Proxy 11 vSphere Certification blogs 103 podcasts 103, 104 recommended reading 103 study tips 104 vSphere certification roadmap about 100 associate-level exam 101 professional-level exam 101 VMware Certified Advanced Professional (VCAP) 101 VMware Certified Design Expert (VCDX) 102 vSphere client 11
vSphere clustering with DRS 16 with HA 16 vSphere Data Protection (VDP) 84 vSphere ESXi Dump Collector 11 vSphere Hardware Compatibility List. See HCL vSphere switching about 50 vDS 50, 52 vSS 50 vSphere Syslog Collector 11 vSphere Update Manager. See VUM vSphere web client 10 vSS 50 vStorage API for Multipathing. See VAMP vStorage APIs for Data Protection (VADP) 84 vStorage APIs for Storage Awareness. See VASA VUM 11
W workload profile about 71 Microsoft Exchange, designing 71, 72 Microsoft SQL server, considerations 72, 73 World Wide Name (WWN) 38 World Wide Node Name (WWNN) 32, 38 World Wide Port Name (WWPN) 32, 38
X XaaS 98
Z zoning 32
[ 110 ]
Thank you for buying
vSphere Design Best Practices About Packt Publishing
Packt, pronounced 'packed', published its first book "Mastering phpMyAdmin for Effective MySQL Management" in April 2004 and subsequently continued to specialize in publishing highly focused books on specific technologies and solutions. Our books and publications share the experiences of your fellow IT professionals in adapting and customizing today's systems, applications, and frameworks. Our solution based books give you the knowledge and power to customize the software and technologies you're using to get the job done. Packt books are more specific and less general than the IT books you have seen in the past. Our unique business model allows us to bring you more focused information, giving you more of what you need to know, and less of what you don't. Packt is a modern, yet unique publishing company, which focuses on producing quality, cutting-edge books for communities of developers, administrators, and newbies alike. For more information, please visit our website: www.packtpub.com.
About Packt Enterprise
In 2010, Packt launched two new brands, Packt Enterprise and Packt Open Source, in order to continue its focus on specialization. This book is part of the Packt Enterprise brand, home to books published on enterprise software – software created by major vendors, including (but not limited to) IBM, Microsoft and Oracle, often for use in other corporations. Its titles will offer information relevant to a range of users of this software, including administrators, developers, architects, and end users.
Writing for Packt
We welcome all inquiries from people who are interested in authoring. Book proposals should be sent to
[email protected]. If your book idea is still at an early stage and you would like to discuss it first before writing a formal book proposal, contact us; one of our commissioning editors will get in touch with you. We're not just looking for published authors; if you have strong technical skills but no writing experience, our experienced editors can help you develop a writing career, or simply get some additional reward for your expertise.
VMware View 5 Desktop Virtualization Solutions ISBN: 978-1-84968-112-4
Paperback: 288 pages
A complete guide to planning and designing solutions based on VMware View 5 1.
Written by VMware experts Jason Langone and Andre Leibovici, this book is a complete guide to planning and designing a solution based on VMware View 5.
2.
Secure your Visual Desktop Infrastructure (VDI) by having firewalls, antivirus, virtual enclaves, USB redirection and filtering and smart card authentication.
VMware vCloud Security ISBN: 978-1-78217-096-9
Paperback: 106 pages
Make your datacenter secure and compliant at every level with VMware vCloud Networking and Security 1.
Take away an in-depth knowledge of how to secure a private cloud running on vCloud Director.
2.
Enable the reader with the knowledge, skills, and abilities to achieve competence at building and running a secured private cloud.
3.
Focuses on giving you broader view of the security and compliance while still being manageable and flexible to scale.
Please check www.PacktPub.com for information on our titles
VMware ThinApp 4.7 Essentials ISBN: 978-1-84968-628-0
Paperback: 256 pages
Learn how to quickly and efficiently virtualize your applications with ThinApp 4.7 1.
Practical book which provides the essentials of application virtualization with ThinApp 4.7.
2.
Learn the various methods and best practices of application packaging and deployment.
3.
Save money and time on your projects with this book by learning how to create portable applications.
Implementing VMware Horizon View 5.2 ISBN: 978-1-84968-796-6
Paperback: 390 pages
A practical guide to designing, implementing, and administrating an optimized Virtual Desktop solution with VMware Horizon View 1.
Detailed description of the deployment and administration of the VMware Horizon View suite.
2.
Learn how to determine the resources your virtual desktops will require.
3.
Design your desktop solution to avoid potential problems, and ensure minimal loss of time in the later stages.
Please check www.PacktPub.com for information on our titles
