Muffinresearch Labs by Stuart Colville

VMWare: Fixing Time Drift in AMD Dual-core Systems | Comments (2)

Posted in Code, Linux/Unix on 22nd July 2008, 3:57 pm by Stuart

I’ve been having some really nasty time drift on VMware servers guests of late. This is a big pain because so much relies on accurate date and time.

The problem I was seeing was that the time on the guest was running very fast so I took the steps of setting the max cpu on the host machine’s vmware configuration. To do this you run cat /sys/devices/system/cpu/cpu1/cpufreq/cpuinfo_max_freq and set this in /etc/vmware/config as follows:

host.cpukHz = 2300000
host.noTSC = "TRUE"
ptsc.noTSC = "TRUE"

Once you’ve updated the configuration you then have to shutdown all of your VMs and then restart Vmware. /etc/init.d/vmware restart and then boot back all of your VMs.

This resulted in the VM’s clocks running really slow (as revealed by running the date command) and so I spent some time looking into why this was. I found a clue looking at /proc/cpuinfo as this was showing that both processors were running at 1Ghz.

I found various posts on forums talking about the cpufrequtils package which are a set of utilities to aid checking and setting the cpu configuration. They are installed with the following apt-get incantation:

sudo apt-get install cpufrequtils

The next step is to check where you settings are currently at:

muffin@winandawsome /usr/bin $ cpufreq-info
cpufrequtils 0.2: cpufreq-info (C) Dominik Brodowski 2004
Report errors and bugs to linux@brodo.de, please.
analyzing CPU 0:
  driver: powernow-k8
  CPUs which need to switch frequency at the same time: 0 1
  hardware limits: 1000 MHz - 2.30 GHz
  available frequency steps: 2.30 GHz, 2.20 GHz, 2.00 GHz, 1.80 GHz, 1000 MHz
  available cpufreq governors: conservative, ondemand, userspace, powersave, performance
  current policy: frequency should be within 1000 MHz and 2.30 GHz.
                  The governor "userspace" may decide which speed to use
                  within this range.
  current CPU frequency is 1000 MHz.
analyzing CPU 1:
  driver: powernow-k8
  CPUs which need to switch frequency at the same time: 0 1
  hardware limits: 1000 MHz - 2.30 GHz
  available frequency steps: 2.30 GHz, 2.20 GHz, 2.00 GHz, 1.80 GHz, 1000 MHz
  available cpufreq governors: conservative, ondemand, userspace, powersave, performance
  current policy: frequency should be within 1000 MHz and 2.30 GHz.
                  The governor "userspace" may decide which speed to use
                  within this range.
  current CPU frequency is 1000 MHz.

As you can see thie meant for me we are currently running at 1000Mhz (1Ghz)

Setting the cpu frequency requires swtiching to the userspace governor and then setting the value you want in kHz.

sudo cpufreq-set -g userspace
sudo cpufreq-set -f 2300000

Testing the value is set correctly is as follows:

muffin@winandawesome /usr/bin $ cpufreq-info
cpufrequtils 0.2: cpufreq-info (C) Dominik Brodowski 2004
Report errors and bugs to linux@brodo.de, please.
analyzing CPU 0:
  driver: powernow-k8
  CPUs which need to switch frequency at the same time: 0 1
  hardware limits: 1000 MHz - 2.30 GHz
  available frequency steps: 2.30 GHz, 2.20 GHz, 2.00 GHz, 1.80 GHz, 1000 MHz
  available cpufreq governors: conservative, ondemand, userspace, powersave, performance
  current policy: frequency should be within 1000 MHz and 2.30 GHz.
                  The governor "userspace" may decide which speed to use
                  within this range.
  current CPU frequency is 2.30 GHz.
analyzing CPU 1:
  driver: powernow-k8
  CPUs which need to switch frequency at the same time: 0 1
  hardware limits: 1000 MHz - 2.30 GHz
  available frequency steps: 2.30 GHz, 2.20 GHz, 2.00 GHz, 1.80 GHz, 1000 MHz
  available cpufreq governors: conservative, ondemand, userspace, powersave, performance
  current policy: frequency should be within 1000 MHz and 2.30 GHz.
                  The governor "userspace" may decide which speed to use
                  within this range.
  current CPU frequency is 2.30 GHz.

After setting the time via ntp you can test that drift is reduced. To sync with an ntp server simply run:

sudo ntpdate ntp.ubuntu.com

There are other things you can do to minimise time drift on VMWare images. For machines whose clocks are appearing to run slowly you can use VMWare tools “tools.syncTime” property in your .vmx file e.g:

tools.syncTime = "TRUE"

Do bear in mind that the VMware syncTime process cannot slow down fast clocks it can only help slow ones.

The main thing here is make sure the host machine is not doing anything which can cause CPU frequency to change dynamically - which also includes anything related to powersaving modes etc.

For further reference it’s worth looking at VMWare’s own paper on ways to correct time drift in VM guests which is a great starting point for working out how to attack time drift problems.

Post Tools

Comments: Add yours

1. On August 5th, 2008 at 2:05 pm Lee Kelleher said:

Hi Stuart, Good post.

I’ve been experiencing the same problem, (but on a Windows Vista host), trying to find out how to resolve this has been a challenge!

The “host.cpukHz” setting and disabling AMD’s PowerNow! has helped, but the seconds keep speeding up every now and then.

Luckily, I’m not using my VMs for anything that requires accurate timestamps, like emails, etc.

Thanks again, I’m glad I’m not the only one with this problem!

Cheers,
- Lee

2. On August 20th, 2008 at 2:51 pm Ubuntu: BUG: soft lockup - CPU#0 stuck for 11s! by Stuart Colville said:

[...] What’s not clear to me is what is the cause of these issues as they seemed to appear out of no where though I suspect they may have been related to the cpufrequency changes to prevent time drift. [...]







XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>



Inspiring a Sense of Ownership|(0)

Former colleague Mike West talks about how inspiring a team’s sense of ownership around a project is the key to great things happening: http://mikewest.org/2008/11/the-inspiration-of-ownership. Quality stuff.

VMware Server: Convert Fixed Disk-images to Growable|(0)

Quick tip if you ever want to convert from a fixed disk image to an expandable one then the following command should do it:

sudo vmware-vdiskmanager -r source.vmdk -t 0 expandable.vmdk

Just replace “source” and “expandable” with your disk image file names. For more on what vmware-vdiskmanager can do for you type vmware-vdiskmanager -h

Photos on Flickr

© Copyright 2004-08 Stuart Colville, all rights reserved. May contain traces of Muffin. Powered by WordPress. Hosting by 1&1 This page was baked in 1.251s.