From 0319e1dd69f1a1289b6bfe7ebb1cd73d2e7461db Mon Sep 17 00:00:00 2001 From: Stephen Wadeley Date: Aug 01 2013 19:35:16 +0000 Subject: Adding chapter "Configuring PTP using ptp4l" in file Configuring_PTP_using_ptp4l.xml --- diff --git a/en-US/Configuring_PTP_using_ptp4l.xml b/en-US/Configuring_PTP_using_ptp4l.xml new file mode 100644 index 0000000..3c63228 --- /dev/null +++ b/en-US/Configuring_PTP_using_ptp4l.xml @@ -0,0 +1,423 @@ + + + + + Configuring PTP using ptp4l +
+ Introduction to PTP + + The Precision Time Protocol (PTP) is a protocol used to synchronize clocks in a network. When used in conjunction with hardware support, PTP is capable of sub-microsecond accuracy, which is far better than is normally obtainable with NTP. PTP support is divided between the kernel and user space. The kernel in Red Hat Enterprise Linux 6 now includes support for PTP clocks, which are provided by network drivers. The actual implementation of the protocol is known as linuxptp, a PTPv2 implementation according to the IEEE standard 1588 for Linux. + + + The linuxptp package includes the ptp4l and phc2sys programs for clock synchronization. The ptp4l program implements the PTP boundary clock and ordinary clock. With hardware time stamping, it is used to synchronize the PTP hardware clock to the master clock, and with software time stamping it synchronizes the system clock to the master clock. The phc2sys program is needed only with hardware time stamping, for synchronizing the system clock to the PTP hardware clock on the NIC. + +
+ Understanding PTP + + The clocks synchronized by PTP are organized in a master-slave hierarchy. The slaves are synchronized to their masters which may be slaves to their own masters. The hierarchy is created and updated automatically by the best master clock (BMC) algorithm, which runs on every clock. When a clock has only one port, it can be master or slave, such a clock is called an ordinary clock (OC). A clock with multiple ports can be master on one port and slave on another, such clock is called a boundary clock (BC). The top- level master is called the grandmaster clock, which can be synchronized by Global Positioning System (GPS). By using a GPS-based time source, disparate networks can be synchronized with a high-degree of accuracy. + +
+
+ Advantages Of PTP + + One of the main advantages over the Network Time Protocol (NTP) is hardware support present in various network interface controllers (NIC) and network switches. This specialized hardware allows PTP to account for delays in message transfer, and greatly improves the accuracy of time synchronization. While it is possible to use non-PTP enabled hardware components within the network, this will often cause an increase in jitter or introduce an asymmetry in the delay resulting in synchronization inaccuracies, which add up with multiple non-PTP aware components used in the communication path. To achieve the best possible accuracy, it is recommended that all networking components between PTP clocks are PTP hardware enabled. Time synchronization in larger networks where not all of the networking hardware supports PTP might be better suited for NTP. With hardware PTP support, the NIC has its own on-board clock, which is used to time stamp the received and transmitted PTP messages. It is this on-board clock that is synchronized to the PTP master, and the computer's system clock is synchronized to the PTP hardware clock on the NIC. With software PTP support, the system clock is used to time stamp the PTP messages and it is synchronized to the PTP master directly. Hardware PTP support provides better accuracy since the NIC can time stamp the PTP packets at the exact moment they are sent and received while software PTP support requires additional processing of the PTP packets by the operating system. + +
+
+ + + +
+ Using PTP + + In order to use PTP, the kernel network driver for the intended interface has to support either software or hardware time stamping capabilities. + + +
+ Checking for Driver and Hardware Support + + In addition to hardware time stamping support being present in the driver, the NIC must also be capable of supporting this functionality in the physical hardware. The best way to verify the time stamping capabilities of a particular driver and NIC is to use the ethtool utility to query the interface as follows: + ~]# ethtool -T eth3 +Time stamping parameters for eth3: +Capabilities: + software-receive (SOF_TIMESTAMPING_RX_SOFTWARE) + software-system-clock (SOF_TIMESTAMPING_SOFTWARE) +PTP Hardware Clock: none +Hardware Transmit Timestamp Modes: none +Hardware Receive Filter Modes: none + + Where eth3 is the interface you wish to check. + + + For software time stamping support, the parameters list + should include: + + + + SOF_TIMESTAMPING_SOFTWARE + + + + + + + +SOF_TIMESTAMPING_TX_SOFTWARE + + + + + + + +SOF_TIMESTAMPING_RX_SOFTWARE + + + + + + + +For hardware time stamping support, the parameters list +should include: + + + + SOF_TIMESTAMPING_RAW_HARDWARE + + + + + + + +SOF_TIMESTAMPING_TX_HARDWARE + + + + + + + +SOF_TIMESTAMPING_RX_HARDWARE + + + + + + +
+ +
+ Installing PTP + + The kernel in Red Hat Enterprise Linux 6 now includes support for PTP. User space support is provided by the tools in the linuxptp package. To install linuxptp, issue the following command as root: + ~]# yum install linuxptp + This will install ptp4l and phc2sys. + +
+ +
+ Starting ptp4l + +The ptp4l program tries to use hardware time stamping by default. To use ptp4l with hardware time stamping capable drivers and NICs, you must provide the network interface to use with the option. Enter the following command as root: ~]# ptp4l -i eth3 -v Where eth3 is the interface you wish to configure. Below is example output from ptp4l when the PTP clock on the NIC is synchronized to a master: +~]# ptp4l -i eth3 -v +selected eth3 as PTP clock +port 1: INITIALIZING to LISTENING on INITIALIZE +port 0: INITIALIZING to LISTENING on INITIALIZE +port 1: new foreign master 00a069.fffe.0b552d-1 +selected best master clock 00a069.fffe.0b552d +port 1: LISTENING to UNCALIBRATED on RS_SLAVE +master offset -23947 s0 adj +0 path delay 11350 +master offset -28867 s0 adj +0 path delay 11236 +master offset -32801 s0 adj +0 path delay 10841 +master offset -37203 s1 adj +0 path delay 10583 +master offset -7275 s2 adj -30575 path delay 10583 +port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED +master offset -4552 s2 adj -30035 path delay 10385 + +The master offset value is the measured offset from the master in nanoseconds. The s0, s1, s2 strings indicate the different clock servo states: s0 is unlocked, s1 is clock step and s2 is locked. Once the servo is in the locked state (s2), the clock will not be stepped (only slowly adjusted) unless the option is set to a positive value in the configuration file (described in the ptp4l(8) man page). The adj value is the frequency adjustment of the clock in parts per billion (ppb). The path delay value is the estimated delay of the synchronization messages sent from the master in nanoseconds. Port 0 is a Unix domain socket used for local PTP management. Port 1 is the eth3 interface (based on the example above.) INITIALIZING, LISTENING, UNCALIBRATED and SLAVE are some of possible port states which change on the INITIALIZE, RS_SLAVE, MASTER_CLOCK_SELECTED events. In the last state change message, the port state changed from UNCALIBRATED to SLAVE indicating successful synchronization with a PTP master clock. + + +The ptp4l program can be also started as a service by running: +~]# service ptp4l start +When running as a service, options are specified in the /etc/sysconfig/ptp4l file. More information on the different ptp4l options and the configuration file settings can be found in the ptp4l(8) man page. + + +By default, messages are sent to /var/log/messages. However, specifying the option enables logging to standard +output (useful for debugging purposes.) + + + To enable software time stamping, the option needs to be + used as follows: + ~]# ptp4l -i eth3 -v -S + + +
+ Selecting a Delay Mechanism + + There are two different delay measurement mechanisms and they can be selected by means of an option added to the ptp4l command as follows: + + + - peer-to-peer (P2P) delay measurement mechanism + + + The P2P mechanism is preferred as it reacts to changes in the network topology faster and may be more accurate in measuring the delay, but it must be supported and used by all hardware, including transparent clocks, on the communication path. + + + + + - end-to-end (E2E) delay measurement mechanism. This is the default. + + + The E2E mechanism is also referred to as the delay request-response mechanism. + + + + + - Automatic selection of the delay measurement mechanism. + + + The automatic option starts ptp4l in E2E mode. It will change to P2P mode if a peer delay request is received. + + + + + + + + All clocks on a single PTP communication path must use the same mechanism to measure the delay. A warning will be printed when a peer delay request is received on a port using the E2E mechanism. A warning will be printed when a E2E delay request is received on a port using the P2P mechanism. + + + + +
+ + +
+
+ +
+ Specifying a Configuration File + + The command line options and other options, which cannot be set on the command line, can be set in an optional configuration file. + + + No configuration file is read by default, so it needs to be specified at runtime with the option. For example: + ~]# ptp4l -f /etc/ptp4l.conf + + + + A configuration file equivalent to the -i eth3 -v -S options shown above would look as follows: +~]# cat /etc/ptp4l.conf +[global] +verbose 1 +time_stamping software +[eth3] + + +
+ +
+ Using The PTP Management Client + + The PTP management client, pmc, can be used to obtain additional information from ptp4l as follows: + + + ~]# pmc -u -b 0 'GET CURRENT_DATA_SET' +sending: GET CURRENT_DATA_SET + 90e2ba.fffe.20c7f8-0 seq 0 RESPONSE MANAGMENT CURRENT_DATA_SET + stepsRemoved 1 + offsetFromMaster -142.0 + meanPathDelay 9310.0 + + + ~]# pmc -u -b 0 'GET TIME_STATUS_NP' +sending: GET TIME_STATUS_NP + 90e2ba.fffe.20c7f8-0 seq 0 RESPONSE MANAGMENT TIME_STATUS_NP + master_offset 310 + ingress_time 1361545089345029441 + cumulativeScaledRateOffset +1.000000000 + scaledLastGmPhaseChange 0 + gmTimeBaseIndicator 0 + lastGmPhaseChange 0x0000'0000000000000000.0000 + gmPresent true + gmIdentity 00a069.fffe.0b552d + + + Setting the option to zero limits the boundary to the locally running ptp4 instance. A larger boundary value will retrieve the information also from PTP nodes further from the local clock. The retrievable information includes: + + + +stepsRemoved is the number of communication paths to the grandmaster clock. + + + + +offsetFromMaster and master_offset is the last measured offset of the clock from the master in nanoseconds. + + + + +meanPathDelay is the estimated delay of the synchronization messages sent from the master in nanoseconds. + + + + +if gmPresent is true, the PTP clock is synchronized to a master, the local clock is not the grandmaster clock. + + + + + gmIdentity is the grandmaster's identity. + + + + + + For a full list of pmc commands, type the following as root: + ~]# pmc help + Additional information is available in the pmc(8) man page. + +
+ +
+ Synchronizing The Clocks + + The phc2sys program is used to synchronize the system clock to the PTP hardware clock (PHC) on the NIC. To start phc2sys, where eth3 is the interface with the PTP hardware clock, enter the following command as root: + ~]# phc2sys -s eth3 -w + The option waits for the running ptp4l application to synchronize the PTP clock and then retrieves the TAI to UTC offset from ptp4l. + + + Normally, PTP operates in the International Atomic Time (TAI) timescale, while the system clock is kept in Coordinated Universal Time (UTC). The current offset between the TAI and UTC timescales is 35 seconds. The offset changes when leap seconds are inserted or deleted, which typically happens every few years. The option needs to be used to set this offset manually when the is not used, as follows: +~]# phc2sys -s eth3 -O -35 + + + Once the phc2sys servo is in a locked state, the clock will not be stepped. This means that the phc2sys program should be started after the ptp4 program has synchronized the PTP hardware clock. + + +The phc2sys program can be also started as a service by running: +~]# service phc2sys start +When running as a service, options are specified in the /etc/sysconfig/phc2sys file. More information on the different phc2sys options can be found in the phc2sys(8) man page. + +
+
+ Verifying time synchronization + + When PTP time synchronization is working properly, new messages with offsets and frequency adjustments will be printed periodically to the ptp4 and phc2sys (if hardware time stamping is used) outputs. These values will eventually converge after a short period of time. These messages can be seen in /var/log/messages file. An example of the output follows: + +master offset -111684 s2 adj -40746 path delay 19461 +master offset -99129 s2 adj -61697 path delay 19461 +master offset -62940 s2 adj -55246 path delay 16458 +master offset -36080 s2 adj -47268 path delay 16458 +master offset -15833 s2 adj -37845 path delay 14862 +master offset -5035 s2 adj -31797 path delay 13412 +master offset -1574 s2 adj -29847 path delay 13412 +master offset 632 s2 adj -28113 path delay 12443 + + + + For ptp4l there is also a directive, , to reduce the output and print only statistics, as normally it will print a message every second or so. For example, to reduce the output to every 1024 seconds, add the following line to the /etc/ptp4l.conf file: + summary_interval 10 + To reduce the output from the phc2sys, it can be called it with the option as follows: + ~]# phc2sys -u summary-updates + Where summary-updates is the number of clock updates to include in summary statistics. + +
+ +
+ Serving PTP time with NTP + + The ntpd daemon can be configured to distribute the time from the system clock synchronized by ptp4 or phc2sys by using the LOCAL reference clock driver. To prevent ntpd from adjusting the system clock, the ntp.conf file must not specify any NTP servers. The following is a minimal example of ntp.conf: +~]# cat /etc/ntp.conf +server 127.127.1.0 +fudge 127.127.1.0 stratum 0 + +
+ + +
+ Serving NTP time with PTP + + NTP to PTP synchronization in the opposite direction is also possible. When ntpd is used to synchronize the system clock, ptp4 can be configured with the option (or other clock options included in the best master clock algorithm) to be the grandmaster clock and distribute the time from the system clock via PTP: +~]# cat /etc/ptp4l.conf +[global] +priority1 127 +[eth3] +# ptp4l -f /etc/ptp4l.conf + + +With hardware time stamping, phc2sys needs to be used to synchronize the PTP hardware clock to the system clock: +~]# phc2sys -c eth3 -s CLOCK_REALTIME -w + + + To prevent quick changes in the PTP clock's frequency, the synchronization to the system clock can be loosened by using smaller (proportional) and (integral) constants of the PI servo: +~]# phc2sys -c eth3 -s CLOCK_REALTIME -w -P 0.01 -I 0.0001 + +
+ +
+ Improving accuracy + + Test results indicate that disabling the tickless kernel capability can significantly improve the stability of the system clock, and thus improve the PTP synchronization accuracy (at the cost of increased power consumption). The kernel tickless mode can be disabled by adding to the kernel boot option parameters. + +
+ + + +
+Additional Resources + + The following sources of information provide additional resources regarding PTP and the ptp4l tools. + +
+ Installed Documentation + + + + ptp4l(8) man page — Describes ptp4l options including the format of the configuration file. + + + + +pmc(8) man page — Describes the PTP management client and its command options. + + + + +phc2sys(8) man page — Describes a tool for synchronizing the system clock to a PTP hardware clock (PHC). + + + + +
+ +
+ Useful Websites + + + + + + + +The Linux PTP project. + + + + + + +The IEEE 1588 Standard. + + + + + +
+ +
+ + +