Monday, March 9, 2015

VGA Passthrough in a VM (Experimental) with Proxmox 3.x

It has been awhile, but a new post.
Been busy trying to get passthrough working on Proxmox.
Finally i managed to get it working.

 
Purpose:

Creating a VGA Passthrough to a VM.
The VM will see the Hardware VGA Card instead of the emulated VGA Card.
This will increase GPU Performance.

I tested this only with an Intel CPU.

Note that this is only for testing purpose.
If you put it in production, the risks are all yours !!!
I can't be held responsible in anyway.





Used Software:

Intel CPU Based Server. 
Find hardware specification system here

Proxmox VE 3.3

Windows 7 SP1 64bit as guest VM 
(make sure you can access it by remote desktop after this you can't access it by proxmox console)

Club3D CGNX-GS728 with NVIDIA 309.08 driver.


Caution:

You can't access your proxmox console local anymore. Because the Graphic Card is relayed to your VM.

Make sure you can access your vm by remote desktop after this you can't access it by proxmox web console. Only by monitor locally attached to your graphic card.



Installation:


Install Proxmox and a Windows VM.
Use standard display driver for the Windows VM.

Make sure all your vm's are down. You'll have to reboot your proxmox server.

Open the Proxmox Shell.

We need to install the 3.10 kernel.

type nano /etc/apt/sources.list

Add the following lines at the bottom:
deb http://download.proxmox.com/debian wheezy pve-no-subscription

type  apt-get update
type  aptitude search pve-kernel

Now you will see a list of the available kernels.
Choose the latest one.
In my case when i wrote this, it is: pve-kernel-3.10.0-7-pve

type apt-get install pve-kernel-3.10.0.7-pve

Reboot your server after the installation is completed.
Now when the server is starting up again you see proxmox is starting up with the installed kernel.

Open the Proxmox Shell again

To locate your VGA Card

type lspci

You will see a number in front of VGA Compatible Controller
in my case it is: 06:00.0

type lspci -n
You will get a list.
Locate your vga card by the number you just found (in my case 06:00.0)

It will look something like this:
06:00.0 0300: 10de:01d3

type nano /etc/default/grub


change the following line:
GRUB_CMDLINE_LINUX_DEFAULT="quiet"

to

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1"

Press CTRL+X to Exit
Then press Y to save
The press ENTER.

type update-grub

Now reboot your server:

type reboot

When the server is rebooted open the proxmox shell again:

type dmesg | grep -e DMAR -e IOMMU

When there is no output, something didn't work.
If you have an output it works.

Now it's time the add the VGA Card to the desired VM.
type nano /etc/pve/qemu-server/VM_ID.conf

Add the following lines at the end of your VM config file:
machine: q35
hostpci0: 06:00.0,x-vga=on,pcie=1,driver=vfio

The 06:00.0 number (Graphic Card) is what you got before, with lspci command.

Start your VM.
Once it's started it will detect your VGA card and you can go ahead and install the drivers.

Tested Graphic Cards:

Properly working Graphics Cards

Model:                                                Driver Tested:     Remarks:         

Club3D CGNX-GS718                           309.08
Club3D CGNX-GS728                           309.08
       

Not properly working Graphics Cards

Model:                                                 Driver Tested:     Remarks:  

Asus EN8600GT Silent/HTDP/512M/A     332.21               Code 43 in VM
Asus EN8600GT Silent/HTDP/512M/A     335.25               Code 43 in VM
Asus EN8600GT Silent/HTDP/512M/A     337.88               Code 43 in VM
Asus EN8600GT Silent/HTDP/512M/A     340.52               Code 43 in VM
Asus EN8600GT Silent/HTDP/512M/A     341.44               Code 43 in VM
Asus HD7770-2GD5                                                    VM hangs during boot

MSI R5450-MD1GD3H/LP                                             Server crash when
                                                                                  VM starts.

Conclusion:

It works, but as you can see many what now mainstream cards are not working yet.

Asus EN8600GT Silent/HTDP/512M/A should probably work when the right drivers are found.
The newest drivers are not working, maybe more older drivers will work.
When i have time i will test older drivers.
 
If you have a card that works let me know and i will put it in the list.

Thanks! 

8 comments:

Unknown said...

Hello, nice job !

How do you access to the VM, in case of trying to play games on it for example.
If you access with a remote desktop, is it going to be slow (LAG) ?
I guess the only way is to plug monitor directly on the proxmox server ?

Thks !

V.Kamphuis said...

Well,

You should be able to play games when you also connect your keyboard and mouse to the server.
Pass this also to the VM.

This setup is intended more for use with Microsoft RemoteFX and so on.

trentoy said...

Hi,

We are planning to virtualize our three windows 7 pc's that output S-video. I am wondering if we install three video card and assign a video card per vm. Please your thought would be very appreciated.

Thanks.

V.Kamphuis said...

Hey,

In theory it should work.
Each VGA Card would get a different hardware number which you can add to each vm.

But your motherboard probably should have support 3 vga cards.

Please let me know the outcome.
Good luck!

KIK said...

thanks for article, i'm using proxmox VE 3.4 with AMD processor, if command 'dmesg | grep -e DMAR -e IOMMU' with this output '[ 0.000000] Please enable the IOMMU option in the BIOS setup,[ 0.953971] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40', so i only enable IOMMU option in the BIOS and then do the rest, right?

Thanks

V.Kamphuis said...

Hey

Its been awhile since a made this how to.
If I remember right for and cpu only iommu part is different.

So when you enable this in the bios you can follow the rest of this how to.

In the proxmox wiki you can find more info how to get it to work with and cpu if above doesn't work.

Please keep us updated on your findings.

Good luck and if you have more questions let me know.

Unknown said...

Do you need a minimum of 2 graphics cards for this to work?

V.Kamphuis said...

No, 1 vga card is enough to get it to work, but be aware if you have 1 vga card that you also use for passthrough then you won't have local console of the proxmox.