Troubleshooting OpenVMS Time zone Issues
Sandeep Mathew, sandeep.mathew@hp.com
This white paper outlines the time zone feature of OpenVMS time, common issues faced and troubleshooting tips and techniques. The following topics are covered:
- Introduction to Time zone
- Building the Time Zone rules
- Introduction to ZIC compiler
- How Daylight Saving Shift take effect
- Time synchronization services
- DTSS
- NTP
- Time Zone trouble shooting techniques
- Common Problems and Solution
- Reference Documentation
Introduction to Time zone
If we were to ask what time is it on earth right now, it would be sufficient to give the Greenwich Mean Time (GMT) as the answer. GMT is reference time from which other times are measured. Coordinated Universal Time (UTC) can, for all the practical purposes, be considered as equivalent to GMT. However UTC is more accurate than GMT (discussion is beyond the scope of this article).
The time of a particular area depends on the time zone and the daylight saving rule as applicable to the area. The time zone determines the offset from GMT whereas daylight saving rule determines the date and time of a temporary time advances done in some time zones typically between spring and autumn. Time zone Differential Factor (TDF) is the difference between the local time and the UTC. The various TDF zones are depicted in the diagram above.
Configuring Time Zones in OpenVMS: The command procedure @SYS$MANAGER:UTC$TIME_SETUP is used to configure the time zone in a single system. Note: If you run UTC$TIME_SETUP to set a new time zone with a different time differential or change the clock (with the option to adjust +/- hours) then DTSS and NTP should be shut down and restarted.
Building Time Zone rules
Time zone rule files contains the current time zone name and when the next daylight saving shift will take place.
Time zone rule files are typically found in SYS$COMMON:[SYS$ZONEINFO.SYSTEM.SOURCES].
A rule file consists of the following:
- Rule Lines
- Zone Lines
- Link Lines
A rule line has the form:
# Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S
Example:
# Rule NAME FROM TO TYPE IN ON AT SAVE LETTER/S Rule Turkey 2011 only - Mar 28 3:00 1:00 S
The fields that make up a rule line are:
- NAME Gives the (arbitrary) name of the set of rules this rule is part of.
- FROM Gives the first year in which the rule applies. Any integer year can be supplied; the Gregorian calendar is assumed. The word minimum (or an abbreviation) means the minimum year representable as an integer. The word maximum (or an abbreviation) means the maximum year representable as an integer. Rules can describe times that are not representable as time values, with the un- representable times ignored. This allows rules to be portable among hosts with differing time value types.
- TO Gives the final year in which the rule applies. In addition to minimum and maximum (as above), the word only may be used to repeat the value of the FROM field.
- TYPE Gives the type of year in which the rule applies. If TYPE is - then the rule applies in all years between FROM and TO inclusive.
- IN Names the month in which the rule takes effect. Month names may be abbreviated.
- ‘s’ to indicate local “standard” or "wall clock" time example 2:00s
- ‘u’ or ‘g’ or ‘z’ to indicate universal time example 2:00u
- Backup your existing binary and source files
- Compile the new rules using ZIC compiler
- Configure your time zone again by running @SYS$MANAGER:UTC$TIME_SETUP
Note: If you are running a time synchronization service such as NTP or DTSS, then it is recommended to shut them down and then restart them after configuration is done.
- Verify that the new time zone rules are in effect by running. @SYS$MANAGER:UTC$TIME_SETUP SHOW Alternatively by taking a look at the logical SYS$TIMEZONE_RULE
- Clerks - runs on local system and synchronizes time on the node
- Local Servers – gives time through the Local Area Network
- Global Servers – provides time to clerks and local servers
- Time Service Provider – provides time using an external accurate clock source
- Couriers – servers that sync between the Local and Global servers
- $ SHOW LOGICAL *TIME*
- $ @SYS$MANAGER:UTC$TIME_SETUP SHOW
- $ PRODUCT SHOW HISTORY
- EST5EDT4: This means that standard time is called EST and it has a time differential of 5 hrs with respect to GMT and EDT is the time in daylight saving time period and it has a time differential 4 hrs with respect to GMT.
- M3.2.0/02: This means that EST to EDT transition will occur at Month 3 , 2nd week ,0 th day (Sunday) at 2.00 AM. It means “Change EST to EDT on the Second Sunday of March (13-Mar-2011) at 02:00”
- c. M11.1.0: This means that EDT to EST transition will occur on Month 11, 1st week 0 th day ( Sunday) at 2.00 AM. It means “Change EDT to EST on the First Sunday of November (6-Nov-2011) at 02:00”
- a. Last boot time
- Time when last daylight saving shift occurred
- Time when you last configured time (got by doing a date on dir /date SYS$MANAGER:SYS$TIMEZONE.DAT).
- Find the time zone rule installed in SYS$STARTUP:TDF$UTC_STARTUP.COM<
$ TYPE SYS$STARTUP:TDF$UTC_STARTUP.COM - Find the time zone rule using the logical SYS$TIMEZONE_RULE
- See if there is a mismatch, if yes then reconfigure time zone and reboot the system to resolve the problem.
- We recommend the customer to upgrade to latest time zone patches (VMS83A/I_TDF-V0200, VMS831H1I_TDF-V0200)
- IDASM – OpenVMS Internals and Data Structures – Ruth Goldenberg
- OpenVMS System Managers Manual
- DECdts Manual
- ZIC man page
ON Gives the day on which the rule takes effect. Recognized forms include:
5 the fifth of the month
lastSun the last Sunday in the month
lastMon the last Monday in the month
Sun>=8 first Sunday on or after the eighth
Sun<=25 last Sunday on or before the 25th
Names of days of the week may be abbreviated or spelled out in full. Note that there must be no spaces within the ON field.
AT Gives the time of day at which the rule takes effect. Recognized forms include:
2 time in hours
2:00 time in hours and minutes
15:00 24-hour format time (for times after noon)
1:28:14 time in hours, minutes, and seconds
- represents 0
Where hour 0 is midnight at the start of the day, and hour 24 is midnight at the end of the day. Any of these forms may be followed by a letter with meaning as below,
In the absence of an indicator, wall clock time is assumed.
SAVE Gives the amount of time to be added to local standard time when the rule is in effect. This field has the same format as the AT field (although, of course, the w and s suffixes are not used).
LETTER/S Gives the "variable part" (for example, the "S" or "D" in "EST" or "EDT") of time zone abbreviations to be used when this rule is in effect. If this field is -, the variable part is null.
A zone line has the following form
Zone NAME GMTOFF RULES/SAVE FORMAT UNTIL
Example:
# Zone NAME GMTOFF RULES FORMAT [UNTIL]
Zone Europe/Istanbul 2:00 Turkey EE%sT 2012
The fields that make up a zone line are:
NAME The name of the time zone. This is the name used in creating the time conversion information file for the zone.
GMTOFF
The amount of time to add to UTC to get standard time in this zone. This field has the same format as the AT and SAVE fields of rule lines; begin the field with a minus sign if time must be subtracted from UTC.
RULES/SAVE
The name of the rule(s) that apply in the time zone or, alternately, an amount of time to add to local standard time. If this field is - then standard time always applies in the time zone.
FORMAT
The format for time zone abbreviations in this time zone. The pair of characters %s is used to show where the "variable part" of the time zone abbreviation goes. Alternately, a slash (/) separates standard and daylight abbreviations.
UNTILYEAR [MONTH [DAY [TIME]]]
The time at which the UTC offset or the rule(s) change for a location. It is specified as a year, a month, a day, and a time of day. If this is specified, the time zone information is generated from the given UTC offset and rule change until the time specified. The month, day, and time of day have the same format as the IN, ON, and AT fields of a rule; trailing fields can be omitted, and default to the earliest possible value for the missing fields.
The next line must be a "continuation" line; this has the same form as a zone line except that the string "Zone" and the name are omitted, as the continuation line will place information starting at the time specified as the "until" information in the previous line in the file used by the previous line. Continuation lines may contain "until" information, just as zone lines do, indicating that the next line is a further continuation.
A link line has the following format
Link LINK-FROM LINK-TO.
Example:
Link Europe/Istanbul Asia/Istanbul # Istanbul is in both continents.
In the OpenVMS implementation, Link is interpreted as a copy. Thus, the previous line copies the information from Europe/Istanbul to Asia/Istanbul.
You can have a peek at one of the rule files provided in SYS$COMMON:[SYS$ZONEINFO.SYSTEM.SOURCES] and get an understanding of how rule files are written. Typically you need to make changes only to rule lines and zone lines so as to effect the required changes in case a new government policy has been put into place.
The latest time zone rule files are generally available from fromftp://elsie.nci.nih.gov/pub/ and http://www.iana.org/time-zones
Introduction to the ZIC compiler
ZIC stands for Zone Information Compiler. ZIC can be used for compiling the time rule binary files from sources. Time zone binary files are typically found in SYS$COMMON:[SYS$ZONEINFO.SYSTEM].
Steps for installing new rule files are as follows
All these can be conveniently written as a command procedure. For example
! This file is officially unsupported and is provided as is.
! Description:
! This command procedure will backup the current timezone
! binaries and sources into a saveset in the current
! directory. From the europe timezone rules present in
! the current directory , it will compile and install
! the new time zone rules
! Perquisites:
! Make sure that EUROPE. file exists in the current
! directory
! Examples:
! In a directory with .EUROPE file is provided, run
! @INSTALL_TZ
!
$WRITE SYS$OUTPUT “Backing up current time zone files”
$BACKUP SYS$COMMON:[SYS$ZONEINFO.SYSTEM...] TZSRCBIN.SAV/SAVE_SET
$WRITE SYS$OUTPUT "Compiling new europe time zone rule source"
$zic -v -d SYS$COMMON:[SYS$ZONEINFO.SYSTEM] EUROPE.;
$WRITE SYS$OUTPUT "Compiling existing asia rules source”
$zic -v -d SYS$COMMON:[SYS$ZONEINFO.SYSTEM] SYS$COMMON:[SYS$ZONEINFO.SYSTEM.SOURCES]ASIA.;
$WRITE SYS$OUTPUT “Compiling backward compatibility rules”
$zic -v -d SYS$COMMON:[SYS$ZONEINFO.SYSTEM] SYS$COMMON:[SYS$ZONEINFO.SYSTEM.SOURCES]BACKWARD.;
$WRITE SYS$OUTPUT “Time zone rule binaries have been updated
, please reconfigure the time zone”
How Daylight Saving Shifts take effect
The job controller and TDF work together so that daylight saving shift takes place. For auto daylight saving shift to occur automatically the SYSGEN parameter AUTO_DLIGHT_SAV should be set to 1. In OpenVMS V8.3A/I and OpenVMS V8.3-1H1, from the patch kit TDF-V0200 onwards, this SYSGEN parameter has been enhanced to be dynamic i,e no reboot is required for the new setting to take effect. Job controller, during its initialization, sets an AST to be fired when a TDF change occurs. When a TDF change occurs it runs the command procedure JBC$DST_COMMAND.COM which in turn runs the read time zone and daylight saving executables ([SYSEXE]SYS$READ_TIME_ZONE_RULE.EXE & [SYSEXE]SYS$DAYLIGHT_SAVING.EXE respectively). Daylight saving executable defines the logical SYS$DST_DELTA_TIME and shifts the time backward or forward depending on the current time. The logical SYS$DST_DELTA_TIME contains the time interval for the next shift to happen. Job controller then uses the current time value and the value of SYS$DST_DELTA_TIME to schedule the next run of the JBC$DST_COMMAND.COM command procedure. This can be verified by the Timer Queue Entries (TQE) present in the job controller. The JBC$DST_COMMAND.COM is also always scheduled to run on Jan1, 2 AM of next year.
Time synchronization services
The time in the system clock might drift due to slight variations in clock frequency or system interrupt activity. To correct this various time synchronization services are available so that the Operating System can synchronize the system clock from more accurate sources. Most commonly used services are DTSS and NTP. It is recommended to use only a single time synchronization service at a time.
DTSS
DTSS is started by invoking @SYS$STARTUP:DTSS$STARTUP command procedure. Time zone configuration when DTSS is enabled, is done using the NET$CONFIGURE command procedure. Following are the important DTSS entities
The clerks periodically check with the servers and obtain the time. Local time is monotonically adjusted to reflect the new time. This means that time is only drifted forward never backward. Time is synchronized by using small time drifts over a period of time rather than an abrupt shift all at once. DECdts manual contains detailed information about configuring and using DTSS.
NTP
NTP is a TCP/IP based time synchronization protocol. NTP uses a hierarchical and somewhat layered set of clock sources. Each node in the hierarchy is assigned a stratum and they provide time to the stratum above them. Time is synchronized using small time drifts over a period of time, causing the local time to pass slightly faster or slower to gradually adjust it to the reference time. Before you configure NTP, run the command procedure @SYS$MANAGER:TCPIP$DEFINE_COMMANDS which defines the symbols used in the NTP configuration. After that run SYS$MANAGER:TCPIP$CONFIG to enable NTP service. Then edit the SYS$SPECIFIC:[TCPIP$NTP]TCPIP$NTP.CONF to perform NTP related settings. After the settings are done, start the NTP service using @SYS$MANAGER:TCPIP$CONFIG. Please refer “HP TCP/IP Services for OpenVMS Management” for more information about NTP.
Time Zone troubleshooting techniques
First thing to do before solving any time zone issue is to collect the data about the system in which the issue occurs. Following are important set of information one should gather
How to interpret the collected data? Here is the sample output of the above commands.
$ SHOW LOGICAL *TIME*
(LNM$PROCESS_TABLE)
(LNM$JOB_89509740)
(LNM$GROUP_000001)
(LNM$SYSTEM_TABLE)
"SYS$DST_DELTA_TIME" = "ffff6dc6bb34a0f0"
"SYS$LOCALTIME" = "SYS$SYSROOT:[SYS$ZONEINFO.SYSTEM.US]EASTERN."
"SYS$TIMEZONE_DAYLIGHT_SAVING" = "1"
"SYS$TIMEZONE_DIFFERENTIAL" = "-14400"
"SYS$TIMEZONE_NAME" = "EDT"
"SYS$TIMEZONE_RULE" = "EST5EDT4,M3.2.0/02,M11.1.0/02"
"TCPIP$BIND_TIMEOUT" = "...."
(LNM$SYSCLUSTER_TABLE)
(DECW$LOGICAL_NAMES)
Now let us see what each of the time related logicals means (Simplest first).
SYS$LOCALTIME
This logical gives the current local time zone in the system under consideration.
SYS$TIMEZONE_DIFFERENTIAL
This value is the difference between UTC time and current time zone time. The value here is in seconds. To find out the value in hours, divide by 3600.
SYS$TIMEZONE_DAYLIGHT_SAVING
This indicates whether the current time is within the daylight saving window or not. This is different from the AUTO_DLIGHT_SAV SYSGEN parameter which decides auto daylight switch over.
The value of this logical will be 1 when the current time is in the daylight saving window otherwise it will be set to zero.
SYS$TIMEZONE_RULE
This is one of the most important logical. Let us interpret and understand the above output for this parameter.
SYS$TIMEZONE_NAME
This gives the name of the currently active time zone. In this case it is EDT, since SYS$TIMEZONE_DAYLIGHT_SAVING is set to one.
SYS$DST_DELTA_TIME
This is another important logical. Its value is an offset from any of the following to the next daylight saving shift.
As you can see, the above 3 references are based on the occurrence of an event. Depending on the event that occurred last, one of the above will be chosen. For example, if the system reboot is the latent event among the three (other two being daylight shift and user configuring time), (a) is chosen as the base value of the time shift. If you know any of the above values and the value of this logical (whichever comes later), you can calculate when the next daylight saving shift occurs. This is done by adding the reference starting time given above and the offset. The offset value can be found out by giving EV/TIME in SDA.
Example:
SDA> EVALUATE/TIME ffff6dc6bb34a0f0
186 01:57:46.49
Add 186 days to the reference value to be chosen as per above guideline to get the time when next shift happens. Note that if time evaluates to a time past Jan1 of next year, the calculation will again happen on Jan1 of next year. This is done to re read the time zone rule files again in case rule files got changed.
Example, if boot time is the reference time:
SDA> SET FETCH QUAD SDA> EVAL/TIME @EXE$GQ_BOOTTIME+FFFF6DC6BB34A0F0 06-NOV-2011 02:00:00.00
To generate more OPCOM messages on TDF events and other diagnostic information, it is recommended to define the logical TDF$DEBUG to any value between 1 and 100 to get more information messages from TDF. For example
$DEFINE /SYSTEM/EXEC TDF$DEBUG 21
Setting this logical will give the value of SYS$DST_DELTA_TIME each time it is changed, it also gives additional logs. The logs are generated into this file:
SYS$SPECIFIC:[SYSMGR]TDF$SET_TIMEZONE_ERROR.LOG
$ @SYS$MANAGER:UTC$TIME_SETUP SHOW
AUTO_DLIGHT_SAV is set to "1". OpenVMS will automatically change to/from Daylight Saving Time.
(in time zones that use Daylight Saving Time)
LOCAL TIME ZONE = US / EASTERN -- DAYLIGHT TIME
LOCAL SYSTEM TIME = 4-MAY-2011 00:49:10.78 (EDT)
TIME DIFFERENTIAL FACTOR = -4:00
TIME ZONE RULE = EST5EDT4,M3.2.0/02,M11.1.0/02
Change EST to EDT on the Second Sunday of March (13-Mar-2011) at 02:00
Change EDT to EST on the First Sunday of November (6-Nov-2011) at 02:00
If DTSS is running, you will see a message “DTSS is in use and will make changes to/from Daylight Saving Time. (in time zones that use Daylight Saving Time)”.
$ PROD SHOW HISTORY is used to find out whether the customer has the latest patches installed. If latest patches are not installed we would recommend the customer to install the latest patches (VMS732_TDF-V0600, VMS83A/I_TDF-V0200, VMS831H1I_TDF-V0200).
If all the above looks correct then it would be good to verify the following
Verification of time zone rule binary files
It can be done in following way, Change SYS$TIMEZONE_RULE to some invalid time zone rule.
$Tz:=$”SYS$SYSDEVICE:[VMS$COMMON.SYSEXE]SYS$READ_TIME_ZONE_RULE.EXE”
$Tz
Now verify that correct rule is reflected in the logical SYS$TIMEZONE_RULE logical. Alternatively the publicly available utility zdump may be used to verify the time zone binary files.
Verification of SYS$TIMEZONE.DAT and SYS$SYSROOT:[SYSEXE]DTSS$TIMEZONE_DIFFERENTIAL.DAT
$DUMP [SYSEXE]SYS$TIMEZONE.DAT and see if it contains a valid TDF.
000C0A14 0001E240 FFFFFFDE 78EE6000 00000000 00000000 00000000 00000000
SDA> EVALUATE/TIME FFFFFFDE.78EE6000
0 04:00:00.00
$ TYPE SYS$SYSROOT:[SYSEXE]DTSS$TIMEZONE_DIFFERENTIAL.DAT, and verify that it contains the right values.
1 -14400 EDT 1 EST5EDT4,M3.2.0/02,M11.1.0/02
Time Related System Cells
EXE$GQ _TDF – Current System TDF
EXE$GQ_SYSTIME – Current system date and time in 100-nanosecond intervals from midnight, November 17, 1858
EXE$GL_ABSTIM - System uptime in seconds
EXE$GL_ABSTIM_TICS -System uptime in elapsed soft ticks (10-millisecond units)
EXE$GQ_SAVED_HWCLOCK - Time of last adjustment of EXE$GQ_SYSTIME in system time format
EXE$GQ_1ST_TIME -Expiration time in system time format of the most imminent time-dependent request
EXE$GQ_BOOTTIME - Time of last bootstrap in system time format
EXE$GL_DTSSFLAGS – Least Significant bit set if DTSS is enabled
Common Problems and Solutions
Note: Abruptly setting time forward and backward using SET TIME = VALUE can have a cascading impact that can lead to lots of other problems. The best thing to do in such situations is to reconfigure time and reboot the system. Given below are some of the problems reported in recent times and the solutions we recommend:
Problem: It is observed that time is shifting by 1 hour after each reboot and time needs to be adjusted manually.
Solution:
Problem: Some of the time zone logicals are missing.
Solution: Ensure that TDF$UTC_STARTUP.COM is getting executed at startup. If DTSS is not enabled then it is recommended to define the logical NET$DISABLE_DTSS to TRUE.
Problem: TQE Entry for the next change is found to be missing.
Solution: Verify that you have all the other factors noted in the Troubleshooting section are correct, like having latest patches, whether SYSGEN parameter AUTO_DLIGHT_SAV is set to 1 etc. If all seems right, then try stopping and restarting the job controller which may help in fixing the issue.