Build a vGPU enabled VM using Packer

Packer has support for NVIDIA vGPU in its vsphere-iso builder. You simply specify the vGPU profile in your template. Packer takes care of adding the PCI Device. Make sure to reserve all RAM, as required by the shared PCI Device. For example

"vgpu_profile": "grid_p4-2q",
"RAM_reserve_all": true

Next, we need to install the driver onto the image. I am using the PowerShell scripts provisioner to accomplish this. The NVIDIA installer reboots when it is done, which Packer wasn’t happy with (it’s not expecting a disconnect). I wasn’t able to figure out a way to prevent that reboot. I added a windows-restart provisioner which seemed to clear this up.

"provisioners": [
{
  "scripts": [
    "{{template_dir}}/setup/nvidia.ps1"
  ],
  "type": "powershell"
},
{
   "type": "windows-restart",
   "check_registry": true
}
]

The nvidia.ps1 script pulls the driver install package from an internal web server, along with the 7za command line utility to extract it to a temporary directory. You can find 7za.exe on 7-zip.org, it’s included in the extras bundle.

The script also adds the registry keys that point to your license server. My lab only has one, if you are running an HA setup, add the appropriate lines to the script.

$ErrorActionPreference = "Stop"

$webserver = "192.168.51.55"
$url = "http://" + $webserver
$7za = "7za.exe"
$installer = "452.96_grid_win10_server2016_server2019_64bit_international.exe"
$7zaArgs = "x C:\$installer -oC:\NVIDIA\"
$licenseServer = "192.168.51.55"
$licenseServerPort = "7070"
$listConfig = "/s"

# Verify connectivity
$connTestResult = Test-NetConnection -Computername $webserver -Port 80
if ($connTestResult.TcpTestSucceeded){
  # Get 7za to extract the installer
  Invoke-WebRequest -Uri ($url + "/downloads/packer/" + $7za) -OutFile C:\$7za
  
  # Get NVIDIA Driver package
  Invoke-WebRequest -Uri ($url + "/downloads/packer/" + $installer) -OutFile C:\$installer

  # Unblock files
  Unblock-File C:\$7za -Confirm:$false
  Unblock-File C:\$installer -Confirm:$false

  # Extract and install GRID Driver
  Try 
  {
	Start-Process C:\$7za -ArgumentList $7zaArgs -PassThru -wait
	New-Item -path "HKLM:\Software\NVIDIA Corporation\Global" -name GridLicensing -force
	New-ItemProperty -path "HKLM:\Software\NVIDIA Corporation\Global\GridLicensing" -name "ServerAddress" -value "$licenseServer" -propertytype string -force
	New-ItemProperty -path "HKLM:\Software\NVIDIA Corporation\Global\GridLicensing" -name "ServerPort" -value "$licenseServerPort" -propertytype string -force
	Start-Process C:\NVIDIA\setup.exe -ArgumentList $listConfig -PassThru -Wait
  }
  Catch
  {
    Write-Error "Failed to install the NVIDIA GRID Drivers"
    Write-Error $_.Exception
    Exit -1 
  }

  # Cleanup on aisle 4...
  Remove-Item C:\$installer -Confirm:$false
  Remove-Item C:\$7za -Confirm:$false
  Remove-Item C:\NVIDIA -Confirm:$false -recurse
}

After the script run, the VM is rebooted, and you can no longer see the console (as expected from installing the NVIDIA drivers).

My build environment is based on this blog post by @virtualhobbit, check it out if you are just getting started with packer.

You can view my windows-10-vgpu GitHub repository to see how this all fits together.