pci: vfio: Implement save path state transitions

Wire a migratable VFIO device's migration state to the VM lifecycle so
the device's internal state survives snapshot and restore. A device such
as a ConnectX VF bound to mlx5_vfio_pci would otherwise come back blank,
because a plain snapshot saves only the PCI configuration Cloud
Hypervisor owns, not the device's own state.

On save, pause moves the device to STOP and snapshot() drives it through
STOP_COPY to extract the opaque state blob, attached to the device
snapshot as a base64 encoded child. resume() returns it to RUNNING.

All new behavior is gated on migration_flags.is_some(), so devices
without migration support (including vfio-user) retain their previous
snapshot behavior.

If the data read fails after STOP_COPY was entered, the device is
returned to STOP before the error is bubbled, since the STOP_COPY
to STOP arc stays valid. A failed transition into STOP_COPY returns
immediately because a STOP from the resulting ERROR state cannot
help. Full recovery including device reset is deferred.

Since the non BAR write path goes directly to the VFIO device and not
the shadow, the PciConfiguration shadow can get stale. Mirror every
non BAR, non MSI config write into the shadow via write_byte /
write_word / write_reg so snapshot() can capture PCI_COMMAND. Without
this the shadow keeps the values set at device init and snapshot()
encodes PCI_COMMAND as zero.

Use the raw write_byte, write_word, and write_reg helpers rather than
PciConfiguration::write_config_register, which would otherwise drain
pending_bar_reprogram, consumed by the BAR block below, and rerun
MSI-X set_msg_ctl, already done by update_msix_capabilities.

Signed-off-by: Saravanan D <saravanand@crusoe.ai>

Cargo.toml

@@ -83,6 +83,7 @@ serde_with = { version = "3.19.0", default-features = false }
 
 # other crates
 anyhow = "1.0.102"
+base64 = "0.22.1"
 bitflags = "2.11.1"
 byteorder = "1.5.0"
 cfg-if = "1.0.4"