vSphere - NetWorker SQL backups fail with error: "Unable to find VSS metadata files in directory"

In this post I want analyze a curious behavior of NetWorker SQL backup in vSphere enviroment:

Symptoms

An examination of the MSVMAppAgent logs on the SQL server show the following errors:

C:\Program Files\DPSAPPS\MSVMAPPAGENT\logs\msvmcatsnap.log

171634:msvmagentcatsnap:

===Command Line Arguments===: 

C:\Program Files\DPSAPPS\MSVMAPPAGENT\bin\msvmagentcatsnap.exe -i -Extracted VSS metadata file 'C:\Program Files (x86)\EMC\vProxy FLR Agent\tmp\695ccb0f-b0f3-43a5-942f-e0c83fcafe26\vm-180966-vssManifest.zip' to directory 'C:\Program Files\DPSAPPS\MSVMAPPAGENT\tmp\VSSMetadata.2021-07-30_09-08-58'168886:

msvmagentcatsnap: Unable to find VSS metadata files in directory C:\Program Files\DPSAPPS\MSVMAPPAGENT\tmp\VSSMetadata.2021-07-30_09-08-58.102333:msvmagentcatsnap:

 Exiting with failure.

Cause

When VMware takes a snapshot of a Windows VM with the Quiesce option enabled it uses the installed VMware tools to query theMicrosoft Volume Shadow Copy service.

Then creates a file called VM_name-vss_manifest#.zip in the VM's datastore foldercontaining the VSS configuration metadata.

That file is copied into the VM during the vProxy based SQL backup and parsed for the specific configuration details. If that file is notpresent or not complete the backup will fail with the error noted above.

You can manually verify the contents of this file using the procedure below:


1. Take a Snapshot of the SQL server in VMware and specify the Quiesce option:

2. From the Data Store Browser download the "VM-Name-vss_manifest##.zip file and examine the contents.

An intact example of the file contents similar to those shown below:

An example of an incomplete version of the file is shown below:

Resolution

After a deep analysis the error was due to the missing VM parameter disk.EnableUUID = “TRUE” inside the .VMX.

The curious fact is that happen only with Windows Server 2016 VMs and is described on this KB https://kb.vmware.com/s/article/52815.

If you add manually disk.EnableUUID = “TRUE” parameter in the VM configuration the error will be resolved and the backup job compete successfully.

Thanks for reading!