pass

Configure physical devices or data blob types passed through from the host to a guest

Synopsis:

[blob_type] pass [loc options] [intr guest_intr[,u][=vector_number]]

Options:

blob_type
The blob_type setting affects only the loc option. If blob_type is specified, then all locations specified by the loc options that follow (until the next context) provide data of the type blob_type to the guest that will be hosted in the VM being configured.
If blob_type precedes a load option, the qvm process copies the contents of the specified file into the guest system address space (see load in this chapter).
intr guest_intr[,u][=vector_number]
Pass the guest_intr interrupt through to the guest. This argument is associated with a host vector number, either vector_number if this value is specified, or the value specified by the hostvector vdev option for the PIC identified by guest_intr.
The host system uses an interrupt service thread (IST) to deliver guest_intr into the guest system. In this release, there's one thread for each passed-through interrupt.

The following attributes specify special behaviors:

u
The u attribute permits immediate unmasking of interrupts to a pass-through device, eliminating the need for a guest exit in order to unmask the EOI.
Warning:

This attribute may be used only if your system has been configured to meet certain conditions, including:

  • Interrupts in the hypervisor host are edge-triggered.
  • Interrupts in the guest are edge-triggered.
  • The guest's device driver won't create new interrupt conditions, including when it is clearing the interrupt.

Incorrect implementation of this feature might create a race condition and result in the loss of the device from the system. Contact your QNX representative before attempting to implement this feature.

loc [{mem:|io:}]region,length,{+|-|r|w|x|c|e|m|d|n|s|t}[=host_location]

The host resource is made directly available to the guest starting at region and continuing for length bytes in guest-physical memory. The newly accessible region is typically memory, which is optionally prefixed with mem: (because this is the default resource type), but on the x86 you can prefix the location with io: to pass I/O space through.

The length argument is a 64-bit value.

CAUTION:
When passing through memory, the host-physical starting address of the memory (which gets mapped to region in the guest) must be aligned to a page boundary, and the length value must be a multiple of the page size on the host system. If not, the guest may experience I/O failures.

These restrictions apply also when using $asinfo_start{name} and $asinfo_length{name}. For more information, see VM configuration syntax in the Configuration chapter.

The length setting can be followed by access type attributes, which may be a combination of the following symbols:
Argument Description
+ Attributes following are added to the region (this is the initial state).
- Attributes following are removed from the region.
r Read
w Write
x Execute
c Cachable
e Report exception
m Usable as system memory (implicitly specifies +rwxcstn). If you specify this attribute, you must not specify d.
d Device uses DMA. If you're using smmuman to manage DMA, you must specify either this attribute or n. You may, however, choose to manage DMA without smmuman; for more information, read the note about the d semantics just below.

If you specify d, you should specify at least the r and w access types.
n Device doesn't use DMA. If you're using smmuman to manage DMA, you must specify either this attribute or d.
s Region can be the source of a DMA request.
t Region can be the target of a DMA request.

If no access types are specified, rw is assumed.

The last part of the loc mem: arguments string may be an equal sign followed by the host-physical address of the device on the system. For some but not all devices, you can provide the guest with an address that is different from the host-physical address (i.e., the device is at host location A, but the guest sees it at location B).

For example: 
pass loc mem:0x2000,r=0x1000
specifies a read-only location at guest-physical address of 0x2000 for a device at the host-physical address of 0x1000.

If you want the guest location to be identical to the host location, don't specify this last part.

CAUTION:
You can use the pass loc option to pass RAM (memory) through to a guest, but the RAM doesn't get zeroed, neither before the guest is given access to it, nor afterwards (e.g., after a VM termination). If you pass through RAM to which another guest system had access, remember that, unless you zero it, this memory may contain vestigial data from a previous guest system.
For the io: case, the location is a port number. For example: 
io:4
specifies an x86 I/O device on port 4. You can provide a length and specify access type flags, but no host address is needed.
Note:

If you set the d attribute to specify that the device uses DMA, the qvm process by default expects to use the smmuman service to ensure that the DMA configuration is safe and secure. If this service isn't running, qvm outputs a fatal error message and aborts without starting the VM.

If, however, the smmu-externally-managed variable has been enabled (set to on), the d attribute has no effect and qvm outputs an information message saying so. Enabling this variable tells qvm to trust that the DMA configuration normally addressed by smmuman has been taken care of in some other way. Note that in this case, the VM configuration must not contain the smmu vdev or qvm reports a fatal error and aborts.

For more details about the smmu-externally-managed variable, refer to the Configuration variables reference.

loc pci:vid/did[/idx]
The host PCI function provided by the given vendor ID (vid) and device ID (did) is made available to the guest. Sometimes these two values aren't enough to uniquely identify a PCI function. In this case, an index for the matching function (idx) can be specified. Because the PCI enumeration order is deterministic, specifying a given index value for a given vendor ID and device ID ensures the same function is passed through every time.
Note:
To support pass-through PCI functions, you must include pci-connector and its dependencies in your hypervisor host image. For information on defining the host image, see Configuring the hypervisor host. Also, for each VM that will provide these functions, you must set the pci-mux-path configuration option if you plan to use a non-default location for pci-connector. For information about this service and the other components it depends on, see the pci-connector entry in the PCI Server User's Guide.
loc pci:bus:dev[.func][=pci:pci_spec]

The PCI function at PCI bus bus, PCI device dev, and PCI function number func (if this parameter is given) on the host is made available to the guest. If no PCI function number is specified, it's assumed to be function 0, which is always present.

If no equals sign follows, the guest location will be identical to the host location. If there's an equals sign followed by another pci: prefix, then the pci_spec after this prefix is either vid/did[/idx] or bus:dev[.func], which specifies the host vendor/device ID (and possibly function index) or the host bus/device/function to be passed through.

For example: 
pass loc pci:0:12.0=pci:0x8086/0x1234
passes a device with vendor ID 0x8086 and device ID 0x1234 to the guest at bus 0, device 12, and function 0 (multi-function). At present, you may choose the device and function numbers in the guest, but the bus number must be 0.
Note:
To support pass-through PCI functions, you must include certain PCI components in your hypervisor host image, and set the pci-mux-path configuration option if you plan to use a non-default location for the PCI connector service. For more details, see the note in the preceding section about the other loc pci: form.
Per the PCI specification, a multi-function device must implement function 0. You can use a vdev pci-dummy instance as a placeholder to implement a function 0 when you want to define a PCI function with a non-zero function number for the guest to use with a given device number. For example, the following configuration uses a pci-dummy vdev to implement function 0 for device 12 on bus 0. This allows the same configuration to pass through function 3 to the guest for the same device 12 on bus 0: 
vdev pci-dummy loc pci:0.12.0
pass loc pci:0:12.3=pci:1:2:0
devid:guest_base,length[=host_base]
Make one or more Device IDs available to the guest starting at guest_base and continuing for length. When the guest uses a Device ID in this range, it actually targets a physical device on the host. Specifying this option is necessary when you want to support devices capable of using Message Signaled Interrupts (MSIs). In this case, your host system must have an Interrupt Translation Service (ITS) and you must set the its-client-config VM configuration option.

The arguments string can include an equal sign followed by the base location on the host to translate a guest Device ID into a host Device ID. This argument extension is mandatory when there is more than one Device ID domain on the host. The upper 32 bits of the host_base value indicate the domain while the lower 32 bits indicate the Device ID. This translation makes it possible to pass through overlapping ranges of Device IDs from different host domains into the unique domain of the guest.

Suppose you are passing through two PCIe controllers that live in different Device ID domains. Both controllers will consume 0x10000 bytes in Device ID space (256 buses, 32 devices per bus, 8 functions per device). Your configuration might then look like this: 
# pass through memory and interrupts for the first controller
pass devid:0,0x10000
# pass through memory and interrupts for the second controller
pass devid:0x10000,0x10000=0x100000000

Here, the second devid: specification defines a host_base value because the Device IDs in the guest are meant to be different than those in the host. However, if you want the guest Device IDs to match those in the host, as in the first devid: specification, or the host has only one Device ID domain, you don't need to specify the host location extension.

sched priority

Set the priority of the pulses the host system uses to inform the qvm process instance of the pass-through interrupts for the current pass-through device (i.e., the one specified after this option in the current pass context).

For example, the following sets the pulse priority to 67 for interrupt 89 for the current device: 
pass
    sched 67 intr gic:89
Similarly, the following sets the pulse priority to 44 for interrupts for the current device: 
pass
    sched 44 loc pci:0:3.0

In this last example, no interrupt is specified because for PCI devices interrupts can be implicit (see Common vdev options in the Virtual Device Reference chapter).

For more information about scheduling priorities, see Scheduling in the Understanding QNX Virtual Environments chapter, and Thread scheduling in the QNX OS System Architecture guide.

Description:

The pass option specifies that one of the following should be passed through from the hypervisor host domain to the guest:

  • a physical device, such as a PCI audio device
  • a data blob, such as an initial RAM disk (initrd) for a Linux guest

Until the next pass, vdev, or reserve option is encountered in the configuration, all intr and loc options that follow a pass option specify interrupts and locations for this pass-through device or component.

CAUTION:

In general, only one OS may have direct control over a physical hardware resource such as a device or region of memory. In a system that supports virtualization, this means either the host OS or a guest OS (but not both) and requires important considerations when passing physical devices or memory through to a guest.

If the host owns a device that a guest requires pass-through access to, the host must terminate its driver for the device before the guest can start a driver for the device in its virtual environment. Similarly, if one guest owns a device as a pass-through device, it must terminate the device driver in its virtualized space before another guest can use the device in its space.

In short, you should never pass a DMA device through to more that one guest, and only in exceptional designs should you pass a non-DMA device through to more than one guest. If you believe that your design requires the use of a non-DMA device in this way, contact your QNX representative.

The same restriction applies to RAM. When you pass through host-physical memory to a guest, the host must have been configured, via options passed to the startup bootstrap program, to not allocate from that same memory region. Otherwise, it might try to use this memory and, thus, the host and guest OSs might end up corrupting each other's memory, leading to bad behavior or system crashes.

The qvm process recognizes the following data types (blob_type) specified before a pass option:

acpi
acpi pass
Only available for x86. Any loc options following will be interpreted as providing additional ACPI tables to the guest system.
data
Recognized, but not relevant for pass-through.
fdt
fdt pass
Any loc options following will be interpreted as providing an FDT binary blob to the guest system (see System information tables in the Configuration chapter).
guest
guest pass
Any loc options following will be interpreted as providing a guest OS boot image to the guest system. For example, if you use this option, you can have the bootloader pre-load a Linux kernel image to RAM at the location specified by loc, then point the qvm process instance to this location rather than to a file to start its guest.
initrd
initrd pass
Any loc options following will be interpreted as providing an initial RAM disk (initrd) image to the Linux guest system.
raw
Recognized, but not relevant for pass-through.

For general information about pass-through devices, see Pass-through devices in the Understanding QNX Virtual Environments chapter.

Note:
On ARM platforms, devices that are clock-dependent (e.g., eMMC devices) can't be passed through (see Passing through clock-dependent devices in the same chapter).

DMA status of pass-through devices

When you are implementing a QNX hypervisor system, you should specify the DMA status of any pass-through device you add to your system; that is, include either d (DMA device) or n (not DMA device) in the access type following the length argument. For example, this configuration: 
pass loc mem:0x2000,0x1000,rn=0x1000
specifies that the device is not a DMA device.

If you're using smmuman for DMA configuration checking (which is the default assumption), and you specify neither d nor n or you specify both for the access type, the qvm process instance assembling the VM prints a fatal error message and aborts without starting the VM.

CAUTION:
A DMA device may not be used by more than one OS, whether it's multiple guests or the host and a guest. For example, if you configure a VM to give a guest access to a DMA device that has been already allocated to another VM, when the newly configured VM attempts to access the device, this will either succeed or fail depending on whether an external manager (e.g., smmuman) is controlling access to the device. If the access fails, qvm aborts and hence, the guest terminates immediately. Therefore, it's important to manage system-wide contention for devices independently of each VM.

Examples:

Linux RAM disk image

The following provides an initrd RAM disk image to a Linux system from host system memory: 
initrd pass loc 0xB1234000,0x4000000,rc=0xB7654000

The above creates a RAM disk of 0x4000000 bytes located in the host-physical memory at address 0xB1234000, accessible by the guest at address 0xB7654000 in guest-physical memory. This region is cacheable (c) and read-only (r).

Note:
It is your responsibility to locate the data in the guest system in accordance with the guest OS's rules. You may prefer to use the load option to pass a Linux initrd RAM disk to a guest. For example: initrd load ./myinitrd.gz (see load in this chapter).

Audio devices

The following configuration passes through the audio-related devices: 
pass pci:0:14.0 pass pci:0:23.0

Since these are PCI devices, the configuration uses BDF (bus/device/function) numbers rather than memory addresses. The example assumes a board with the specified physical devices at the specified locations.

Graphics device

Assuming that you have a screen driver and the other components you need to run graphics on the board, you could do the following to pass through a graphics device:

  1. Modify your VM configuration file to create a PCI pass-through device: 
    pass loc pci:0:2.0=pci:0:2.0 
vdev pci-dummy 
    clone pci:0:31.0
  2. Start the guest with the new VM configuration.
  3. From your desktop host, start Screen in the guest, using the -c option to point screen to the DRM manager: 
    screen -c /usr/lib/graphics/intel-drm/graphics.conf

In step 1 we use the 0:2.0 BDF location, which Intel has designated for integrated graphics:

  • The value to the left of the equals sign passes the host BDF through to the guest.
  • The value to the right of the equals sign ensures that the device will appear at the specified location on the guest's PCI bus.

We specify the BDF value in both places to ensure that the qvm process instance presents the same location (0:2.0) to the guest, in case code in the guest assumes this is an Intel-designated BDF.

The pci-dummy vdev is not strictly required. However, the device driver usually uses this vdev to identify the display adapter, so using the pci-dummy vdev allows us to expose the correct vendor and device IDs without providing any further (and unneeded) functionality.

The example assumes a board with the specified physical devices at the specified locations.

See also loc above, and pci-dummy in the Virtual Device Reference chapter.

Page updated: