Add the kernel boot parameter to enable Intel IOMMU.
Edit /etc/default/grub and insert intel_iommu=on as an argument on the GRUB_CMDLINE_LINUX_DEFAULT line - separate other arguments with space. Save the file and run update-grub
Make sure that the Proxmox linux installation does not attempt to use the nVidia card by blacklisting the drivers and updating initramfs afterwards:
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf update-initramfs -u
Configure vfio - add the following to /etc/modules:
vfio vfio_iommu_type1 vfio_pci vfio_virqfd
Now we need to attach the PCI devices to the vfio driver. First find your PCI device with lspci.
Now scroll up to find your GPU - my output has this section:
You'll be able to scroll back with shift+pgup/pgdown in most terminals.
00:1f.6 Signal processing controller: Intel Corporation C600/X79 series chipset Thermal Management Controller (rev 06) Subsystem: Super Micro Computer Inc C600/X79 series chipset Thermal Management Controller Flags: bus master, fast devsel, latency 0, IRQ 5, NUMA node 0 Memory at fed08000 (64-bit, non-prefetchable) [size=4K] Capabilities:  Power Management version 3 04:00.0 VGA compatible controller: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] (rev a1) (prog-if 00 [VGA controller]) Subsystem: NVIDIA Corporation GP107 [GeForce GTX 1050 Ti] Flags: bus master, fast devsel, latency 0, IRQ 62, NUMA node 0 Memory at dd000000 (32-bit, non-prefetchable) [size=16M] Memory at c0000000 (64-bit, prefetchable) [size=256M] Memory at d0000000 (64-bit, prefetchable) [size=32M] I/O ports at 8000 [size=128] Expansion ROM at de000000 [disabled] [size=512K] Capabilities:  Power Management version 3 Capabilities:  MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities:  Express Legacy Endpoint, MSI 00 Capabilities:  Virtual Channel Capabilities:  Latency Tolerance Reporting Capabilities:  Power Budgeting <?> Capabilities:  Advanced Error Reporting Capabilities:  Vendor Specific Information: ID=0001 Rev=1 Len=024 <?> Capabilities:  #19 Kernel driver in use: vfio-pci Kernel modules: nvidiafb, nouveau 04:00.1 Audio device: NVIDIA Corporation Device 0fb9 (rev a1) Subsystem: NVIDIA Corporation Device 11bf Flags: bus master, fast devsel, latency 0, IRQ 63, NUMA node 0 Memory at de080000 (32-bit, non-prefetchable) [size=16K] Capabilities:  Power Management version 3 Capabilities:  MSI: Enable- Count=1/1 Maskable- 64bit+ Capabilities:  Express Endpoint, MSI 00 Capabilities:  Advanced Error Reporting Kernel driver in use: vfio-pci Kernel modules: snd_hda_intel
Note the Nvidia device on 04:00.0, but also a matching sound device on 04:00.1. We want to pass both to our vm.
We can find the device ID of both with lspci by just referring to the top part of the address 04:00 In my case I can find the IDs here:
lspci -n -s 04:00 04:00.0 0300: 10de:1c82 (rev a1) 04:00.1 0403: 10de:0fb9 (rev a1)
Add the device IDs in /etc/modprobe/vfio.conf as options vfio-pci ids=XX:XX,XX:XX In my case:
echo "options vfio-pci ids=10de:1c82,10de:0fb9" > /etc/modprobe.d/vfio.conf
Reboot the Proxmox host before using the device on a vm.
Not sure why, but now I need the following in the [vmid].conf args: -no-hpet -rtc driftfix=slew -global kvm-pit.lost_tick_policy=discard -cpu host,hv_time,kvm=off,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_vendor_id=proxmox -machine kernel_irqchip=on
Also, only seems to work after fresh DDU removal
Alternatively, read up at: https://forum.proxmox.com/threads/fix-for-pci-passthrough-and-issues-with-proxmox-6-gui-regex-that-needs-resolving-after-upgrade.56507/