My experience with virtualization and the linux kvm

Agenda

  • Why did I dive into this?

  • Trouble with virtual machines

  • Measures to detect presence of virtual machine

  • Hacking the linux kernel as countermeasure

  • Compiling QEMU and custom UEFI

  • Conclusion

Why did I dive into this?

  • Bypass anti-cheat of online games

  • Hide virtual machine to guest

  • Learning

  • How far will I get? (achievement)

  • Privacy?

BUT WHY?

I got banned...

  • Hardware identifiers blacklisted
  • Permanently banned

The enemy

  • Kernel driver (ring 0)

  • Usermode service

  • Multiple cheat detection mechanisms

  • Virtual machine detection

... so I came up with an idea!

  • phsyical hardware = €€€

  • virtual hardware = free

  • use a virtual machine!

  • but what about the GPU?

virtual machines

  • other people have done it before

  • pass host gpu to guest

  • linux kernel modifications

For amusement, watch
https://youtu.be/L1JCCdo1bG4

(for gaming)

libvirt, qemu and the kvm

  • libvirt

    • api/daemon/cli

  • qemu

    • software virtualization, standalone (slow stuff)

  • kvm

    • kernel virtual machine (fast stuff)

    • vm exit = transition from execution to emulation!

  • qemu can use kvm

the good...

  • new hardware identifiers for the anti-cheat

  • leverage gaming graphics card in virtual machine

  • time travel through snapshots of vm

  • save ram to disk (fast stuff!)

... and the bad

  • generic hardware = hint for vm presence

  • guest occupies graphics card... and the host?

  • not very reliable setup

    • crashes, freezes, stuck in boot loop

  • heavy resource consumption

the worst

  • gpu passthrough = no virtual screen
  • 1 screen for host and vm
  • lots of HDMI cable plugging

(not me, I borrowed a screen from the office!)

how to detect presence of a virtual machine?

  • commonly used virtual hardware identifiers
  • hardware survey
  • lack of thermal/voltage/... pci adapters
  • cpuid
  • cpu timing checks
  • vcpu implementation quirks

the "easy" stuff

  • PCI devices by RedHat, Inc.
  • MAC address vendor ID exclusive to QEMU
  • i9-9900k with 7 cores instead of 8
  • no thermal/fan/voltage information available
  • hard disk vendor "qemu"
  • bios strings contain "qemu"

CPUID and virutal machines

  • special cpu instruction
  • gives details about the processor
  • kvm traps cpuid
    • leaks hypervisor presence bit
    • implementation quirk
  • feature (ecx) bit 31 = hypervisor bit

Obviously a virtual machine

Not so obvious

Windows Task Manager gives it away

actually...

timing checks

  • read time
  • force vm exit
  • read time again
  • compare time diff with "real" systems

tools to automatically check presence of vm

Time for counter measures

Thanks to a reddit post for inspiration

rdtsc and cpuid

offsetting timing check

  • rdtsc = read time stamp counter
    (cpu cycles since reset)
  • cpuid is handled/trapped by kvm
  • cpuid => force vm exit
  • vm exit takes time... overhead!
hardware intercepted
cpuid 30-100 ops 800+ ops
... ~ x 10

vmexit

vmenter

intercept the cpu instruction

normal execution

kernel interception

cpuid

...

rdtsc and cpuid

offsetting timing check

  • rdtsc can be trapped as well!
  • emulate rdtsc in kvm handler
  • remember that cpuid was called and offset timestamp to appropriate value
static int handle_rdtsc_interception(struct vcpu_svm *svm) 
{
	u64 differece;
	u64 final_time;
	u64 data;
	
	differece = rdtsc() - svm->vcpu.last_exit_start;
	final_time = svm->vcpu.total_exit_time + differece;

	data = rdtsc() - final_time;

	svm->vcpu.arch.regs[VCPU_REGS_RAX] = data & -1u;
	svm->vcpu.arch.regs[VCPU_REGS_RDX] = (data >> 32) & -1u;

	svm->vcpu.run->exit_reason = 123;
	return nop_interception(svm);
}

// offset time!!!

what now?

compile and run!

  • compile linux kernel
  • install it
  • hope that you did not **** up
  • for me it works
    • until I noticed I did not switch kernel
      • but then it really worked

only for this to turn green!

the qemu/uefi part

  • manufacturer / bios information still show "qemu"

the qemu/uefi part

  • also the hardware ids are a hint!

the qemu/uefi part

  • but its all in the source code ;)

the qemu/uefi part

the qemu/uefi part

  • takes some time...

the qemu/uefi part

  • it crashes/glitches 50% of the time
  • but if not, then it works!

Conclusion

Conclusion

  • anti-cheat strong
  • felix weak
  • still get kicked from games
  • but i learned alot
    • how to compile the linux kernel
    • context switch execution -> interception
  • i will continue my journey

I hope you liked it

as much as I did

questions and feedback now? :)

my experience with virtualization and the linux kvm

By amazingturtle

my experience with virtualization and the linux kvm

  • 76