You may see an error in SGMon debug logs such as, "Agent does not support SSL. Not attempting SSL connection." and possibly, "ConnectToVirtualMachine: failed to connect to RAWS in virtual machine, error: 0xe0000383".
There will likely be a final error of, "ConnectToVirtualMachine: returning 0xe000974a"
A VM backup may complete with a failed status. But a restore attempt against this data will show the VMDK file, as well as file and folder GRT data, available for restore. However, it is when Application GRT (AD, Exchange, SQL, or SharePoint) is selected in the job, will there be an attempt to utilize the Remote Agent on that VM.
Ensure that the Remote Agent is fully up to date and matches the Backup Exec server and that the Backup Exec Service Account (BESA) has adequate permissions in the ESX or vCenter permissions tab.