vmm, ch-remote: Allow VFIO fd substitution at restore

A VFIO device restored onto a different host has a device path and
iommufd that are meaningless there, and an FD backed device cannot
serialize a live descriptor into the snapshot at all. Restoring one
therefore needs fresh descriptors supplied with the request.

RestoreConfig gains vfio_fds, pairing each device id with a cdev FD,
and iommufd_fd for the backing iommufd. Both arrive over SCM_RIGHTS on
the restore request. vm_restore swaps each named device's stale path
or FD for the received one and installs the iommufd before the VM is
built, so the device comes up FD backed.

The request is rejected when a substituted device lacks the iommufd
backend, when an id is unknown or repeated, or when an FD backed device
names no replacement.

ch-remote gains the vfio_fds and iommufd_fd options and forwards the
descriptors through the SCM_RIGHTS pool.

Signed-off-by: Saravanan D <saravanand@crusoe.ai>

cloud-hypervisor/src/bin/ch-remote.rs

@@ -933,14 +933,22 @@ fn snapshot_config(url: &str) -> String {
 fn restore_config(config: &str) -> Result<(String, Vec<i32>), Error> {
     let mut restore_config = RestoreConfig::parse(config).map_err(Error::Restore)?;
     // RestoreConfig is modified on purpose to take out the file descriptors.
-    // These fds are passed to the server side process via SCM_RIGHTS
-    let fds = match &mut restore_config.net_fds {
+    // These fds are passed to the server side process via SCM_RIGHTS, in the
+    // order net_fds, then vfio_fds, then the iommufd FD, matching the
+    // server's split.
+    let mut fds: Vec<i32> = match &mut restore_config.net_fds {
         Some(net_fds) => net_fds
             .iter_mut()
             .flat_map(|net| net.fds.take().unwrap_or_default())
             .collect(),
         None => Vec::new(),
     };
+    if let Some(vfio_fds) = restore_config.vfio_fds.as_mut() {
+        fds.extend(vfio_fds.iter_mut().filter_map(|v| v.fd.take()));
+    }
+    if let Some(iommufd_fd) = restore_config.iommufd_fd.take() {
+        fds.push(iommufd_fd);
+    }
     let restore_config = serde_json::to_string(&restore_config).unwrap();
 
     Ok((restore_config, fds))