Updated about 8 yrs, 7 mths ago (April 8, 2016). Know a better answer? Let me know!
Correcting Linux Clock Drift on Hyper-V on Server 2008 R2
How to correct clock drift on a CentOS 5 virtual machine guest running on a Server 2008 R2 Hyper-V host.
I recently had a problem where my CentOS 5 virtual machine running on Hyper-V on Server 2008 R2 was gaining time quite dramatically.
Initially, I turned off “Time synchronization” in the Integration Services section of the Hyper-V settings for the CentOS virtual machine. This reduced the clock drift from more than a minute per minute, down to only seconds per minute, however this is still far too inaccurate for NTP or similar to correct.
Here is what I did to correct the clock inaccuracy problem:
- Modify the kernel boot options by editing /boot/grub/grub.conf and adding the line “divider=10 clocksource=acpi_pm” (for 32-bit) or “notsc divider=10” (for 64-bit) after the appropriate kernel line, as below. Clearly, this will vary depending on your exact install.:
# grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00 # initrd /initrd-version.img #boot=/dev/hda default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title CentOS (2.6.18-164.6.1.el5) root (hd0,0) kernel /vmlinuz-2.6.18-164.6.1.el5 ro root=/dev/VolGroup00/LogVol00 hda=noprobe hdb=noprobe notsc divider=10 initrd /initrd-2.6.18-164.6.1.el5.img title CentOS (2.6.18-164.el5) root (hd0,0) kernel /vmlinuz-2.6.18-164.el5 ro root=/dev/VolGroup00/LogVol00 initrd /initrd-2.6.18-164.el5.img
- Configure ntpd, as below:
- The following is a sample /etc/ntp.conf. Note that you should normally change 0.vmware.pool.ntp.org to something specific to your country, for example, 0.au.ntp.pool.org, and the same for the other ntp servers here, and in /etc/ntp/step-tickers in item 2 below. See www.pool.ntp.org/en for more information on the NTP pool and which servers you should use.
tinker panic 0 restrict 127.0.0.1 restrict default kod nomodify notrap server 0.vmware.pool.ntp.org server 1.vmware.pool.ntp.org server 2.vmware.pool.ntp.org driftfile /var/lib/ntp/drift
- The following is a sample /etc/ntp/step-tickers:
0.vmware.pool.ntp.org 1.vmware.pool.ntp.org
The configuration directive tinker panic 0 instructs NTP not to give up if it sees a large jump in time. This is important for coping with large time drifts and also resuming virtual machines from their suspended state. Note: The directive tinker panic 0 must be at the top of the ntp.conf file. - It is also important not to use the local clock as a time source, often referred to as the Undisciplined Local Clock. NTP has a tendency to fall back to this in preference to the remote servers when there is a large amount of time drift. An example of such a configuration is:
server 127.127.1.0 fudge 127.127.1.0 stratum 10
Comment out both lines.
- The following is a sample /etc/ntp.conf. Note that you should normally change 0.vmware.pool.ntp.org to something specific to your country, for example, 0.au.ntp.pool.org, and the same for the other ntp servers here, and in /etc/ntp/step-tickers in item 2 below. See www.pool.ntp.org/en for more information on the NTP pool and which servers you should use.
- Synchronise the time manually:
ntpd -q
- Set the hardware clock to the newly synchronised time:
hwclock --systohc
- Set up ntpd to automatically run and synchronise the time, as below:
chkconfig ntpd on
- Reboot the machine. I have not had any further time issues.
More Information
For more information about:
- Windows Server 2008 R2, see microsoft.com/windowsserver2008/en/us/default.aspx
- Hyper-V, see en.wikipedia.org/wiki/Hyper-V
- CentOS, see www.centos.org
- Adding “divider=10 clocksource=acpi_pm”, and configuring ntpd for use on virtual machines, see kb.vmware.com/kb/1006427
- ntpd, see linux.die.net/man/1/ntpd
- hwclock, see linux.die.net/man/8/hwclock
- pool.ntp.org, see www.pool.ntp.org/en
Updated about 8 yrs, 7 mths ago (April 8, 2016). Know a better answer? Let me know!
Related categories
.User submitted comments:
Mardolf, about 14 yrs, 9 mths ago
Monday February 1, 2010 9:09 AM
Best!
Romansmm, about 14 yrs, 8 mths ago
Monday March 8, 2010 8:16 AM
Thanks! It's very useful and it works!
grasomega, about 14 yrs, 6 mths ago
Monday May 3, 2010 2:08 PM
Thank you very much for this article! I found it after some search and it looks lke it's the only method that works flawlessly. Keep up the good work!
hyperblah, about 14 yrs, 6 mths ago
Monday May 17, 2010 1:10 PM
Thank you very much for that kernel line. I would buy you a beer if I could :)
Urban Alho, about 14 yrs, 5 mths ago
Thursday June 3, 2010 8:28 PM
Thank you VERY MUCH, i usually google things pretty fast and nothing worked until i found this guide. THANK YOU
Leonardo Rodrigues, about 14 yrs, 3 mths ago
Thursday July 29, 2010 11:36 PM
after LOTS of research on this subject/problem, i have found lots of articles asking to change kernel parameters to minimize the clock drift problems. Basically all articles suggested a mix of 3 kernel arguments:
divider=10
clocksource=acpi_pm
notsc
after some researchs, i've found that the combination of 'divider=10 notsc' is the best one, even better that the 'divider=10 clocksource=acpi_pm' proposed here.
with 'divider=10 clocksource=acpi_pm' my clock was still drifting about 12 seconds in 600 seconds. With 'divider=10 notsc' the drift was about 1.5-2 seconds in 600 seconds. That's the lowest i've acchieved with LOTS of combinations settings i've tried. NO option i've found could SOLVE the drift problem, but these could take it to more acceptable levels, so ntpd could be able to correct it.
hope this helps ..... and remember YMMV ... take your tests and find what's best for you.
Leonardo Rodrigues, about 14 yrs, 3 mths ago
Thursday July 29, 2010 11:45 PM
i forgot to mention on my comment above ..... my tests were done on CentOS 5.5 x86_64 machines running over HyperV 2008-R2
Ned, about 13 yrs, 11 mths ago
Thursday November 25, 2010 6:50 AM
Thanks Leonardo, I have updated the article to reflect your findings. Based on some recommendations from VMWare, it appears that “notsc divider=10” is recommended for x64, whereas “divider=10 clocksource=acpi_pm” is recommended for x86.
Jeff Russell, about 12 yrs, 8 mths ago
Friday March 16, 2012 12:32 AM
Thank you so much for this! I've been having this issue with a Centos VM running Zimbra Collaboration Suite and was fed up with my email registering times in the future! This seems to have resolved it at long last! Thank you again!
Pedro Gordo, about 12 yrs, 5 mths ago
Thursday June 21, 2012 9:31 AM
Perfect Clean Explanation.
Kirill, about 12 yrs, 2 mths ago
Thursday August 23, 2012 2:06 PM
Hey, pretty helpful.
But Leonardo Rodrigues is right. Took me a while to get different CentOS versions to achieve acceptable time deviations.
Esxi user, about 12 yrs, 19 days ago
Friday November 2, 2012 3:49 AM
So this is a direct copy off of VMware w/o any credit to them? Solve a hyperV issue originally solved by its main competitor. Classy.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006427
Ned, about 12 yrs, 19 days ago
Friday November 2, 2012 3:54 AM
Esxi user, your link is already referenced in the More Information section above.
Nate, about 11 yrs, 11 mths ago
Thursday November 29, 2012 9:44 PM
Thank you very much for this tip. I was getting frustrated by Linux clock drift on Hyper-V.
Ian Brown, about 11 yrs, 9 mths ago
Monday February 4, 2013 9:11 AM
Finally, a solution that works, thanks!!
Ian
Griff, about 11 yrs, 8 mths ago
Monday March 11, 2013 2:03 PM
Brilliant write up, I've used this a few times now and it has worked a treat.
Thanks Very Much!!!
Griff
Josh, about 11 yrs, 7 mths ago
Thursday April 18, 2013 10:32 PM
Clock drift was driving me nuts on my CentOS VM hosting Cacti on Hyper-V, this worked perfectly. Thank you!
Comment on this article (no HTML, max 1200 characters):
Katrina, about 14 yrs, 10 mths ago
Friday January 8, 2010 5:37 PM
A huge thank you on this! 24 hours after following these instructions, the VM is still keeping perfect time. This has solved so many problems.