date is 5 hours behind...

[Beastmanh]

I have tried everything I know and I can't get one of my machines to display the right time... I have turned off ntpd so it won't interfere with my settings.. I set the time zone to America/Los_Angeles and the time is 5 hours behind what it should be. I have scp'd a "Los_Angeles" file over from a machine that displays the right time and sym linked to it from /etc/localtime and no change. I have installed system-config-date and used it to set the timezone to Los_Angeles with no change. When I run ntpdate sync.server. it gives me an offset number that is exactly 5 hours off and then later fails because the time is further off than 1000 seconds from the real time.

Here is the result of: tail -150 /var/log/messages |grep ntpd

Sep  4 10:14:25 Server1 ntpd[21063]: ntpd exiting on signal 15
Sep  4 10:27:16 Server1 ntpd[21624]: ntpd 4.2.2p1@1.1570-o Tue Jun 10 00:07:14 UTC 2008 (1)
Sep  4 10:27:16 Server1 ntpd[21625]: precision = 1.000 usec
Sep  4 10:27:16 Server1 ntpd[21625]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Sep  4 10:27:16 Server1 ntpd[21625]: Listening on interface wildcard, ::#123 Disabled
Sep  4 10:27:16 Server1 ntpd[21625]: Listening on interface lo, ::1#123 Enabled
Sep  4 10:27:16 Server1 ntpd[21625]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 10:27:16 Server1 ntpd[21625]: Listening on interface lo, 127.0.0.1#123 Enabled
Sep  4 10:27:16 Server1 ntpd[21625]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 10:27:16 Server1 ntpd[21625]: Listening on interface eth0:0, 192.XXX.XXX.XXX#123 Enabled
Sep  4 10:27:16 Server1 ntpd[21625]: kernel time sync status 0040
Sep  4 10:27:16 Server1 ntpd[21625]: frequency initialized 0.000 PPM from /var/lib/ntp/drift
Sep  4 10:27:28 Server1 ntpd[21625]: ntpd exiting on signal 15
Sep  4 10:27:28 Server1 ntpd[21648]: ntpd 4.2.2p1@1.1570-o Tue Jun 10 00:07:14 UTC 2008 (1)
Sep  4 10:27:28 Server1 ntpd[21649]: precision = 1.000 usec
Sep  4 10:27:28 Server1 ntpd[21649]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Sep  4 10:27:28 Server1 ntpd[21649]: Listening on interface wildcard, ::#123 Disabled
Sep  4 10:27:28 Server1 ntpd[21649]: Listening on interface lo, ::1#123 Enabled
Sep  4 10:27:28 Server1 ntpd[21649]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 10:27:28 Server1 ntpd[21649]: Listening on interface lo, 127.0.0.1#123 Enabled
Sep  4 10:27:28 Server1 ntpd[21649]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 10:27:28 Server1 ntpd[21649]: Listening on interface eth0:0, 192.XXX.XXX.XXX#123 Enabled
Sep  4 10:27:28 Server1 ntpd[21649]: kernel time sync status 0040
Sep  4 10:27:28 Server1 ntpd[21649]: frequency initialized 0.000 PPM from /var/lib/ntp/drift
Sep  4 10:27:31 Server1 ntpd[21649]: ntpd exiting on signal 15
Sep  4 10:27:31 Server1 ntpd[21670]: ntpd 4.2.2p1@1.1570-o Tue Jun 10 00:07:14 UTC 2008 (1)
Sep  4 10:27:31 Server1 ntpd[21671]: precision = 1.000 usec
Sep  4 10:27:31 Server1 ntpd[21671]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Sep  4 10:27:31 Server1 ntpd[21671]: Listening on interface wildcard, ::#123 Disabled
Sep  4 10:27:31 Server1 ntpd[21671]: Listening on interface lo, ::1#123 Enabled
Sep  4 10:27:31 Server1 ntpd[21671]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 10:27:31 Server1 ntpd[21671]: Listening on interface lo, 127.0.0.1#123 Enabled
Sep  4 10:27:31 Server1 ntpd[21671]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 10:27:31 Server1 ntpd[21671]: Listening on interface eth0:0, 192.XXX.XXX.XXX#123 Enabled
Sep  4 10:27:31 Server1 ntpd[21671]: kernel time sync status 0040
Sep  4 10:27:31 Server1 ntpd[21671]: frequency initialized 0.000 PPM from /var/lib/ntp/drift
Sep  4 10:30:05 Server1 ntpd[21671]: ntpd exiting on signal 15
Sep  4 10:45:55 Server1 ntpd[22130]: ntpd 4.2.2p1@1.1570-o Tue Jun 10 00:07:14 UTC 2008 (1)
Sep  4 10:45:55 Server1 ntpd[22131]: precision = 1.000 usec
Sep  4 10:45:55 Server1 ntpd[22131]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Sep  4 10:45:55 Server1 ntpd[22131]: Listening on interface wildcard, ::#123 Disabled
Sep  4 10:45:55 Server1 ntpd[22131]: Listening on interface lo, ::1#123 Enabled
Sep  4 10:45:55 Server1 ntpd[22131]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 10:45:55 Server1 ntpd[22131]: Listening on interface lo, 127.0.0.1#123 Enabled
Sep  4 10:45:55 Server1 ntpd[22131]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 10:45:55 Server1 ntpd[22131]: Listening on interface eth0:0, 192.XXX.XXX.XXX#123 Enabled
Sep  4 10:45:55 Server1 ntpd[22131]: kernel time sync status 0040
Sep  4 10:45:55 Server1 ntpd[22131]: frequency initialized 0.000 PPM from /var/lib/ntp/drift
Sep  4 10:49:17 Server1 ntpd[22131]: synchronized to LOCAL(0), stratum 10
Sep  4 10:49:17 Server1 ntpd[22131]: kernel time sync disabled 0001
Sep  4 10:50:14 Server1 ntpd[22131]: synchronized to 69.10.36.3, stratum 2
Sep  4 10:58:09 Server1 ntpd[22131]: ntpd exiting on signal 15
Sep  4 11:05:13 Server1 ntpd[22706]: ntpd 4.2.2p1@1.1570-o Tue Jun 10 00:07:14 UTC 2008 (1)
Sep  4 11:05:13 Server1 ntpd[22707]: precision = 1.000 usec
Sep  4 11:05:13 Server1 ntpd[22707]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Sep  4 11:05:13 Server1 ntpd[22707]: Listening on interface wildcard, ::#123 Disabled
Sep  4 11:05:13 Server1 ntpd[22707]: Listening on interface lo, ::1#123 Enabled
Sep  4 11:05:13 Server1 ntpd[22707]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 11:05:13 Server1 ntpd[22707]: Listening on interface lo, 127.0.0.1#123 Enabled
Sep  4 11:05:13 Server1 ntpd[22707]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 11:05:13 Server1 ntpd[22707]: Listening on interface eth0:0, 192.XXX.XXX.XXX#123 Enabled
Sep  4 11:05:13 Server1 ntpd[22707]: kernel time sync status 0040
Sep  4 11:05:13 Server1 ntpd[22707]: frequency initialized 0.000 PPM from /var/lib/ntp/drift
Sep  4 11:05:17 Server1 ntpd[22707]: ntpd exiting on signal 15
Sep  4 11:05:17 Server1 ntpd[22721]: ntpd 4.2.2p1@1.1570-o Tue Jun 10 00:07:14 UTC 2008 (1)
Sep  4 11:05:17 Server1 ntpd[22722]: precision = 1.000 usec
Sep  4 11:05:17 Server1 ntpd[22722]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Sep  4 11:05:17 Server1 ntpd[22722]: Listening on interface wildcard, ::#123 Disabled
Sep  4 11:05:17 Server1 ntpd[22722]: Listening on interface lo, ::1#123 Enabled
Sep  4 11:05:17 Server1 ntpd[22722]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 11:05:17 Server1 ntpd[22722]: Listening on interface lo, 127.0.0.1#123 Enabled
Sep  4 11:05:17 Server1 ntpd[22722]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 11:05:17 Server1 ntpd[22722]: Listening on interface eth0:0, 192.XXX.XXX.XXX#123 Enabled
Sep  4 11:05:17 Server1 ntpd[22722]: kernel time sync status 0040
Sep  4 11:05:17 Server1 ntpd[22722]: frequency initialized 0.000 PPM from /var/lib/ntp/drift
Sep  4 11:05:29 Server1 ntpd[22722]: ntpd exiting on signal 15
Sep  4 16:34:44 Server1 ntpd[23945]: ntpd 4.2.2p1@1.1570-o Tue Jun 10 00:07:14 UTC 2008 (1)
Sep  4 16:34:44 Server1 ntpd[23946]: precision = 1.000 usec
Sep  4 16:34:44 Server1 ntpd[23946]: Listening on interface wildcard, 0.0.0.0#123 Disabled
Sep  4 16:34:44 Server1 ntpd[23946]: Listening on interface wildcard, ::#123 Disabled
Sep  4 16:34:44 Server1 ntpd[23946]: Listening on interface lo, ::1#123 Enabled
Sep  4 16:34:44 Server1 ntpd[23946]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 16:34:44 Server1 ntpd[23946]: Listening on interface lo, 127.0.0.1#123 Enabled
Sep  4 16:34:44 Server1 ntpd[23946]: Listening on interface eth0, XXX.XXX.XXX.XXX#123 Enabled
Sep  4 16:34:44 Server1 ntpd[23946]: Listening on interface eth0:0, 192.XXX.XXX.XXX#123 Enabled
Sep  4 16:34:44 Server1 ntpd[23946]: kernel time sync status 0040
Sep  4 16:34:44 Server1 ntpd[23946]: frequency initialized 0.000 PPM from /var/lib/ntp/drift
Sep  4 16:37:59 Server1 ntpd[23946]: synchronized to 64.202.112.75, stratum 1
Sep  4 16:37:59 Server1 ntpd[23946]: time reset +17984.996046 s
Sep  4 16:37:59 Server1 ntpd[23946]: kernel time sync disabled 0001
Sep  4 16:41:43 Server1 ntpd[23946]: synchronized to LOCAL(0), stratum 10
Sep  4 16:42:04 Server1 ntpd[23946]: synchronized to 64.202.112.75, stratum 1
Sep  4 16:42:04 Server1 ntpd[23946]: time correction of 17985 seconds exceeds sanity limit (1000); set clock manually to the correct UTC time.

Please help if you have ANY info...

Cheers,

~Mike

[JeoSaurus]

Hi Mike,

That's certainly an odd problem... Let's compare the output of these commands:

$ date "+%a %d %b %Y %X %Z"; hwclock; ntpdate pool.ntp.org

That'll tell us if there are any differences between the system clock and the hardware clock, and give us a baseline from ntp. Do you know if the hardware clock is set to local time or UTC?

[Beastmanh]

I think I might have an idea what the prob is, but here is my output of your commands:

Ran this command at 10:58pm

date "+%a %d %b %Y %X %Z"

got:

Fri 05 Sep 2008 17:58:52 PDT

ran: hwclock

Cannot access the Hardware Clock via any known method.
Use the --debug option to see the details of our search for an access method.

ran at 11:02pm: ntpdate pool.ntp.org

5 Sep 18:02:50 ntpdate[22338]: step time server 66.187.233.4 offset 17982.912856 sec

(note the offset is exactly 5 hours off)

Ok, now for my hypothesis. The machine that has this problem is running as a virtual machine on a xen hypervisor... The problem is I can't log into the hypervisor. I'm guessing the hypervisors hardware clock is 5 hours off of normal. But I won't know until I am able to log into that machine. (which i should be able to do sometime this next week). If you can see some other obvious problem then please let me know, otherwise I will give you more feedback when I log into the hypervisor... :)

Thanks so much for your help,

~Mike

[JeoSaurus]

Yeah, haven't worked with Xen much myself. That could be it! Let us know what you find :)

-G

[efmesch]

Have you tried setting the time zone to one that is 5 hours ahead of Los Angeles? Does the time change then?

[jbennet]

Nah that wont work. Dont UNIX machines operate using UTC offsets so that it doesnt cause problems with timestamps on WANs?

[Beastmanh]

Sorry it took me so long to get back to this thread... I solved the problem and forgot to update. Yes the problem was that the hypervisor hardware clock was 5 hours off.

Cheers,'

~Mike

[jbennet]

marked as solved