How to Deploy Resources to VMware vCenter through Cloud Provisioning and Governance using Terraform

[Ben Carlsten -]

With the Orlando release of ServiceNow, creating new Blueprints manually has been deprecated leaving the only method to create new catalog items through templates.  That’s great for AWS and Azure because they are made to deploy using their own template technologies (CloudFormation and ARM templates respectively) but where does this leave endpoints like VMware vCenter?  That is where Terraform comes in!  The preferred method for deploying to environments like VMware vCenter beginning in Orlando is through Terraform.  In this post, I will go over the basics of setting up a server to use for Terraform, some configuration, connection to ServiceNow, and using this connection to deploy vCenter resources through CPG.

My Setup:  vCenter 6.7, ServiceNow Paris, Terraform Server (CentOS 8), Windows MID Server.

1: Activate Terraform Plugin for Cloud Management

To begin with, I have a ServiceNow Paris developer instance with Cloud Management activated along with the plugin ‘Cloud Management: Terraform Connector’ activated as well.  This plugin is necessary to expose the Terraform provider options when creating a new Cloud Catalog item.  The Terraform plugin can be activated by:

• Search ‘plugins’ in the left navigator in the main ServiceNow menu
• Search ‘Cloud Management’ in the Search Bar
• Click ‘Install’ on the pane for ‘Cloud Management: Terraform Connector

2:  Deploy/Configure a Terraform server (Skip this step if one is already configured)

I’m using a Terraform server deployed on a CentOS 8 virtual machine running in the same vCenter that I will be accessing.  The VM must be configured for SSH Key authentication to connect to ServiceNow.  Assuming a VM has already been provisioned:

• Install Terraform on your selected linux distro:
  • https://phoenixnap.com/kb/how-to-install-terraform-centos-ubuntu
• Configure SSH Key authentication to server:

I used Puttygen to create the key pair necessary for SSH Key authentication and will go over that process here

• Open Puttygen
• Choose ‘RSA’ and type 4096 for the ‘Number of bits and click ‘Generate’
• Move your cursor to create randomness

Once the key is created there are 2 things that need to be saved, the public and private keys.  This generates a .ppk private key that can be save and used for SSH into the machine but we need a different type of key for ServiceNow.

• In the box on the top, it says ‘Public key for pasting into OpenSSH authorized_keys file:’. Save that text in a notepad for use later.  This is the public key that will be referenced on the Terraform VM.

• On the top, click ‘Conversions’, ‘Export OpenSSH key’ and save it to your local machine. This is the private key that will be used when creating the new credential inside of ServiceNow for Terraform.  When the file is open, it should start ‘-----BEGIN RSA PRIVATE KEY-----’

Now that the public and private keys are generated, the public key has to be added to the VM hosting Terraform to allow authentication using the generated Private Key.  There are a few methods to do so but I’ll cover the manual method here

• SSH into the VM in which Terraform is installed, I used putty
• If not created already create the .ssh directory and add the ‘authorized_keys’ file
  • mkdir -p ~/.ssh
  • touch ~/.ssh/authorized_keys
• Copy the contents of the ‘public’ key that you saved earlier from PuTTygen to the ‘/.ssh/authorized_keys’ file. I used nano and just copied directly from my Windows machine into the CentOS 8 VM through PuTTy using a right-click.
  • nano /.ssh/authorized_keys (opens the newly created blank file)
  • right-click in the file after copying from Windows
  • ctrl-x, then ‘y’ to save the file

You should now be able to SSH into the VM without using password authentication when using the Private Key generated.

3:  Create/Activate a VMware vCenter Terraform Configuration Template

Now that you have a VM with Terraform installed and configured and that VM is setup to be accessed remotely using an SSH Key, we can setup the Configuration Template that will be used to deploy resources into vCenter through ServiceNow. 

It’s important to consider that CPG will discover the templates based on the folder structure that you give it to search.  We hand CPG a base folder when configuring a Terraform Provider and there must be at least another layer of folders beneath it that contain these templates.  A good way to think about it is that you create an environmental folder (vcenter,aws,azure, etc) and possibly another folder layer underneath if you want to organize the templates into categories (basevms, templates,dev,test, etc).  For this example, I’m creating a master folder (/var/terraform), an environmental folder underneath for vCenter (/var/terraform/vcenter) and I’m going to create my template holding folders off of the environmental folder.

• SSH into the Terraform VM
• Create the folder structure
  • mkdir /var/terraform
  • mkdir /var/terraform/vcenter (this is the base directory we will reference in CPG)
  • mkdir /var/terraform/vcenter/basevm
• Go to the newly created folder
  • cd /var/terraform/vcenter/basevm
• Create the Configuration file

provider "vsphere" {
  user           = var.vsphere_user
  password       = var.vsphere_password
  vsphere_server = var.vsphere_server

  # If you have a self-signed cert
  allow_unverified_ssl = true
}

data "vsphere_datacenter" "dc" {
  name = "_YOUR_DATACENTER_NAME"
}

data "vsphere_datastore" "datastore" {
  name          = "_YOUR_DATASTORE_NAME"
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_resource_pool" "pool" {
  name          = "_YOUR_RESOURCE_POOL_NAME_"  #Resources by default 
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_network" "network" {
  name          = "_YOUR_PORTGROUP_NAME_"
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_virtual_machine" "template" {
  name          = "_YOUR_TEMPLATE_NAME_"
  datacenter_id = "${data.vsphere_datacenter.dc.id}"
}

resource "vsphere_virtual_machine" "vm" {
  name                       = var.name
  datastore_id               = data.vsphere_datastore.datastore.id
  resource_pool_id           = data.vsphere_resource_pool.pool.id
  num_cpus                   = 2
  memory                     = 2048
  guest_id                   = "centos7_64Guest"
  wait_for_guest_net_timeout = -1
  network_interface {
    network_id = data.vsphere_network.network.id
  }
  disk {
    label = "disk0"
    size  = 20
  }
  clone {
    template_uuid = "${data.vsphere_virtual_machine.template.id}"
    customize {
      linux_options {
        host_name = var.name
        domain    = "_YOUR_DOMAIN_NAME_"
      }
      network_interface {
        ipv4_address = var.ip
        ipv4_netmask = 24
      }
    }
  }
}
variable "vsphere_server" {
  type    = "string"
  default = "_VCENTER_FQDN_OR_IP_"
}

variable "vsphere_user" {
  type    = "string"
  default = "_VCENTER_USER_"
}

variable "vsphere_password" {
  type    = "string"
  default = "_VCENTER_PASSWORD_"
}

variable "name" {
  type    = "string"
  default = "test-1"
}

variable "ip" {
  type    = "string"
  default = "192.168.1.1"
}

This example file is made to be expanded upon, but in short, it will clone a VM from an existing VM template inside your vCenter.  It has an input for ‘name’ and ‘ip’.  Adjust the data with your own environmental configuration.  Further down the line you may add multiple various inputs based off of data discovered in your ServiceNow environment.

• nano baseVM.tf

Copy and paste the modified configuration file into this and:

• ctrl-x, then ‘y’ to save.

• Run the following commands
  • terraform init (this downloads the vcenter modules terraform needs)
  • terraform fmt (this makes the code formatted correctly)
  • terraform validate (validates the configuration of the file is good)

• You can test the file now by running
  • terraform apply (this will create the desired infrastructure)

4: Configure Terraform in CPG

Now that we have the Terraform plugin installed in ServiceNow, a Terraform Server running in our environment and configured with a working vCenter template, we can connect Terraform into CPG and start consuming the template as a Cloud Catalog Item.

• Login to your ServiceNow instance and access the Cloud Admin Portal
• Go to ‘Credentials’ in the ‘Manage’ tab
• New
• Type: SSH Private Key
  • Give it a name
  • username of the logged in user (root for my environment)
  • no password required
  • if you configured a passphrase for the exported SSH Key credential enter it
  • Copy the contents of the RSA Key into the SSH private key field
  • Click Submit
• Go to ‘Config Management’ under the ‘Manage’ Tab
  • Under ‘Config Providers’ select ‘New’
    • Give it a Name (vCenter Servers)
    • Provider (Terraform Environment – this is for Linux machines)
    • URL (enter the IP address of the VM, FQDN doesn’t seem to work)
    • Base Directory: /var/terraform/vcenter (or wherever the base directory is)
    • Server Type: Linux (should only be 1 option)
    • Credential (the credential you just created, must be done prior)
    • Version (only 1 option)

• Next, click on the newly created Config Provider tile and click ‘Discover Now’ in the top-right hand corner of the new config provider record.  Successful Discovery of the VM will look something like below with the child folders discovered.

• Each Folder can be selected which will show the discovered Configuration Templates (.tf files). Important to note here is that discovery only goes one layer deep, so if the base is ‘/var/terraform’ and your configuration file is ‘/var/terraform/vcenter/basicVM/baseVM.tf’ it won’t be located.  That’s why we set the base folder to /var/terraform/vcenter.

Now that we’ve discovered a folder with a Terraform Configuration Template (.tf file) we can create the Cloud Catalog Item

• Go to ‘Cloud Catalog Items’ in the ‘Design’ tab
• Click ‘New’
  • Give it a Name
  • Source: Change to ‘Configuration Management Template’
  • Provider Type: ‘Terraform Environment’
  • Provider: {{The provider you just created in the previous step}}
  • Click Submit

• Select the newly created catalog item from the tile:
  • In the related links at the bottom, under Cloud Templates, click ‘New’
  • Configuration Installable: Choose the discovered Configuration Template
  • Click Submit

• This brings you back to the Cloud Catalog Item
  • In the related links for Cloud Templates, a new Version should have been created in ‘Draft’ State, click the number under Version to open it
  • Here you can see the Terraform Configuration Template in the body. Assuming it looks good, click ‘Activate’ on the record.  This does a few important things.  It goes through the Template and creates any inputs that are listed (with the exception of vCenter user/password,url) and makes them visible on the request form, it also creates a Blueprint that backs the cloud catalog item and allows it to be Launched in the portal.  (We can no longer create blueprints in the portal but they are still very much a part of the deployment infrastructure.  If a blueprint is unable to be created from the given template, you cannot launch a catalog item.  This will be noticeable on the Cloud Catalog Item record, if the ‘Blueprint’ field is empty, you cannot activate the item or deploy it)
  • After pressing Activate, it should bring you back to the Cloud Catalog Item with the ‘Active’ box now selectable (b/c of the previously mentioned blueprint item). Check the box and click ‘Update’ to activate the item.
    • You can also look at the Variable Sets that are created. The additional Name and IP inputs will be under the ‘Provision’ Variable Set.  As we know from our training, tabs in the provisioning form are in fact Variable Sets.

Now that we have successfully created the Catalog Item and activated it, head on to the User Portal, find it, and launch it into vCenter!