BL Linux - Ch. 18 - Logging and Time Services

For the *date* command, what is this format string? %Y

The four digit year, such as 2018

How do you ensure that a centralized server can receive logs from remote hosts?

*-r* flag You must start the syslog server with the *-r* flag. Then you need a rule that forwards the desirable messages to the remote servery by supplying the destination with an @ symbol. Example: *.info @logserver.example.com This will send all info messages and above to logserver.example.com for central analysis.

What flag do you use to review the order of the log file?

*-r* flag will reverse the order to show the admin the newest files.

Which file shows the calibration that has been done on the hardware clock?

*/etc/adjtime*

Where is the logrotate's configuration file stored? Please give the full path.

*/etc/logrotate.conf* and */etc/logrotate.d* The main *logrotate.conf* file specifies any default settings and system files to be reotated and each file in *logrotate.d* is used to add additional settings or override the default settings for specific files.

What does the following log file contain? */var/log/cron*

*/var/log/cron* Logs of schedule job activity

What does the following log file contain? */var/log/maillog*

*/var/log/maillog* All logs relating to email

What does the following log file contain? */var/log/messages*

*/var/log/messages* General purpose log messages that aren't in one of the other files.

What does the following log file contain? */var/log/secure*

*/var/log/secure* Security logs, such as records of connection attempts and failures

What does the following log file contain? */var/log/xferlog*

*/var/log/xferlog* Logs of local File Transfer Protocol (FTP) server activity

What is the command to view the new logs being generated/added to the *journald*?

*journalctl -f*

What is *klogd*?

*klogd* listens specifically for kernel logs and forwards them to *syslogd*. It can also do processing on the message before sending it along, such as to translate memory addresses into names that would be helpful to kernel developers.

What does *ntpd* set?

*ntpd* gradually brings the system clock into alignment. You can also use it to set a specific clock time if the difference is greater than 1,000 seconds at which point, *ntpd* will exit as unable to update the time clock. *ntpd* also enables a mode in the Linux kernel where the hardware clock is set every 11 minutes. Thus a system running *ntpd* has both clocks in sync.

What is *syslog*?

*syslog* is a standardized system that can receive messages from local applications or remote servers and write them to disk or send to another server.

Before you can pass the *date* command a format string, what symbol needs to precede the format string?

+ date +formatstring

Please provide the full path for the ntpd configuration file.

/etc/ntp.conf

Where is *sylogd*'s configuration file?

/etc/syslog.conf This file defines a set of rules, one per line. Each rule is evaluated, and if the log matches that rule, the log is sent to the destination specified in the rule.

What is the systemd journal's configuration file? Please specify the whole path.

/etc/systemd/journald.conf

What are some ways that a message can be logged with *syslog*?

1.) From within an application using the *syslog* library call 2.) From the command line using the *logger* command 3.) Remotely by connecting to a syslog server over the network

6.) Type the full path to the directory where systemd stores it log files:

/var/log/journal

What are the 4 aliases that you can use with the *pool.ntp.org*?

0.pool.ntp.org 1.pool.ntp.org 2.pool.ntp.org 3.pool.ntp.org

What are some advantages that *journald* has over *syslogd*?

1.) Looking at the logs is the same no matter which distribution you are using. 2.) You can query on metadata such as the level, facility, or message, and by date and time. 3.) You have access to more metadata than possible in a text file, such as the full command line, process id, binary, and security privileges of the process. 4.) You can annotate the logs with your own metadata, such as source file, line number, or customer id.

What are the 3 common solutions when dealing with open files that are logging and need to be rotated?

1.) Move the log files and then send the application a signal to close its logs and reopen them. This is ideal, but the application must support doing this. 2.) Restart the application after moving the files over and it should start logging in a new file. The *create* keyword can also be used to create the new file after rotation if special permissions are needed or the application doesn't do it itself. This is good as long as restarting the application doesn't impact service. 3.) Copy the logs instead of moving them and then truncate the old file in place, using the *copytruncate* keyword. This involves a lot more disk operations than the other two options and may lose log entries written after the copy operation started.

What can you do with *log rotation*?

1.) Start a new log file on a schedule, such as daily, weekly, or monthly. 2.) Compress old log files to save disk space. 3.) Prune old archives so that you only keep a certain number of old logs. 4.) Rename old log files with a date stamp so you know where to look for older logs. 5.) Run commands before or after a set of logs is rotated.

What are the two different logging systems used by Linux?

1.) syslog 2.) systemd's journal

How many clocks does your Linux machine have? What are they?

2 clocks 1.) The hardware clock is on the motherboard on your computer and is implemented as a separate chip. 2.) The system clock is part of the Linux kernel. When you ask Linux for a time such as a timestamp for a log entry, you get the system clock.

%%

A literal %

What is a *stratum 2 server*?

A server that gets its time from a stratum 1 server is called a *stratum 2 server*.

What is a *stratum 3 server*?

A server that gets its time from a stratum 2 server is called a *stratum 3 server*.

What does this command do? logger -i Starting script processing

Apr 20 19:55:01 bob sean[9054]: Starting script processing bog = hostname sean = user name * -i, --id * log the process ID too So in this instance, the process ID is 9054

What does this command do? date +%s

This shows the system time which is kept as the number of seconds since January 1, 1970 (Unix epoch time).

What does the *peers* command do? ntpq> peers

The *peers* command gives details about each of the servers defined with the *server* keyword in the configuration.

What does this command do? date +/var/backup/website.%Y%m%d%H%M%S

This will back up a file with the timestamp attached: website.20150418111854

Define: Syslog severity *level 1*

Level = 1 Tag = alert Description: Alert: Immediate action is needed to prevent a failure.

Define: Syslog severity *level 2*

Level = 2 Tag = crit Description = Critical: The system has reached a critical level, such as disk space almost running out

What does this command do? hwclock -w *OR* hwclock --systohc

Either of these commands will write the time in the system clock to the hardware clock.

Define: Syslog severity *level 3*

Level = 3 Tag = err Description = Error: Some part of the system encountered an error

Define: Syslog severity *level 4*

Level = 4 Tag = warn Description = Warning: Something happened that may have been processed incorrectly.

Define: Syslog severity *level 5*

Level = 5 Tag = notice Description = Notice: Not an error condition, but may need special handling.

How does ntpd set the clock?

If the clock is out of sync, ntpd gradually brings it back into alignment by speeding up the local clock or slowing it down as necessary. If the time difference is too great, which is defined as 1,000 seconds by default, ntpd will exit and you will need to set the clock with *ntpdate* before continuing.

What does this command do? date +%s

If you need a timestamp, the current time since epoch can be displayed using this command.

Define: Syslog severity *level 6*

Level = 6 Tag = Info Description = Information: A normal log entry about a routine event that happened successfully.

Define: Syslog severity *level 7*

Level = 7 Tag = Debug Description = Debug: A message about internal state that is for debugging problems.

What is *ntpd*?

It is a long running daemon that continually adjusts the local clock, as opposed to *ntpdate*'s one-off behavior. *ntpd* runs in the background and continually queries remote servers for their time while learning about how the network conditions are affecting the communications and compensating for local time drift. Thus, a server running *ntpd* ALWAYS has the correct time with no need for a periodic job to reset the clock.

What is *syslog-ng*?

It is a next generation syslog server that offers both open source and commercial versions. It has many of the same features of rsyslogd, but the configuration syntax has been reworked.

What is *rsyslogd*?

It is meant to be a "rocket-fast" implementation of a syslog daemon with support for plugins, alternate storage mechanisms, and more flexible rules processing. You can use it to store logs in a database, filter the logs based on keywords, or keep statistics.

What is the *ntpq* command used for?

It is used to query an NTP server for its statistics and connects to the local host by default. When you use the *ntpq* command, it opens a separate prompt denoted by *>* and you can type in help to see all of the commands and set up the configuration, etc....

Define: Syslog severity *level 0*

Level = 0 Tag = emerg Description: Emergency: The system is unusable

What does this command do? ntpdate

This will set the clock to use NTP. You will need to be root because you are setting the system clock.

Why do you want redundancy of the drift file and when using NTP?

More than one server results in more accurate clock settings and redundancy should one server go out of service or become unreliable.

What are *stratum 1 servers*?

NTP servers that get their time directly from a reference clock are called *stratum 1 servers*. These servers are considered to have the most accurate time in the NTP hierarchy. Even with this accuracy, there is no guarantee the servers have the same time.

What is *NTP*?

Network Time Protocol (NTP) Is a protocol that enables a computer to get its time from a remote server and to be accurate within milliseconds even if the network is unreliable. Coupled with NTP is a globally distributed network of freely usable servers that participate in the protocol.

What does this command do? date + "Today is %A"

Output: *Today is Saturday* %A gives you the day as a word instead of a number - so it depends on what date it is - the output will change accordingly.

What is RTC?

Real Time Clock (RTC) This is on the motherboard. It is usually only used when the computer boots so that Linux has something from which to set the system clock.

If you made changes to the /etc/systemd/journald.conf file, what do you have to do to ensure they are enabled?

Restart journald with *systemctl restart systemd-journald* after making any changes.

What does this command do? hwclock -s *OR* hwlock --hctosys

Sets the system clock using the hardware clock's time.

What is *systemd journal*?

The *systemd journal* is a newer form of logging for distributions that have moved to systemd. Fortunately, it is backward compatible so that applications that can log to syslog can also log to systemd, and it can also forward messages to a syslog server for centralized logging.

What does the *association* command do? ntpq> association

The *association* command gives more details on each sever, including how well the remote server is performing.

What is the *driftfile*?

The *driftfile* tracks local clock drift over time, and allows ntpd to compensate. This can be added to */etc/ntp.conf* - - Example on page 506

What does this command do? hwclock

The *hwclock* command manipulates the hardware clock by talking to the motherboard. you need to be root to run the command.

What is the main difference with *syslogd* versus *journald*?

The *journald* main difference from *syslogd* is that it logs to a binary file and annotates each log entry with metadata rather than using a text format. This gives you a lot of advantages over *syslogd*.

What does this command do? *logger*

The *logger* command is helpful for both testing your logging configuration and for logging within scripts. The easiest way to use it is just to pass the log message on the command line.

For the *date* command, what is this format string? %H

The current hour in 24 hour time such as 15 for 3pm

For the *date* command, what is this format string? %M

The current minute

For the *date* command, what is this format string? %S

The current second Capital *%S* is for seconds in the date command Lowercase *%s* is for time since epoch time - be careful!

For the *date* command, what is this format string? %s

The current time since epoch

What is the difference in the output between *hwclock* command and the *date* command?

The difference is that there is an extra value at the end, expressed in seconds. This value calculates how long it was between the time the command was started and the clock was read, which is used inside the *hwclock* command to help correct *drift*.

What is "drift" as it relates to clocks?

The hardware and system clocks do differ from each other and from the real time, in a process known as drift. A clock drifts because it runs slightly faster or slower, and therefore needs periodic synchronization.

Where is the journald log file stored and in what format?

The journal's log is stored in a binary format under */var/log/journal*, so you need special tools to read the log.

What does this command do? logger Starting script processing

This is an example of passing the *logger* command a message. If you were to look at the */var/log/messages* file, you would see the following in response to your command: Apr 20 19:55:01 bob sean: Starting script processing bog = hostname sean = user name

For the *date* command, what is this format string? %d

The two digit day, such as 13

For the *date* command, what is this format string? %m

The two digit month, such as 01 for January

What does this command do? ntpdate ntp1.torix.ca

This asks the provided remote server (ntp1.torix.ca) for the time and sets your local time to that value.

What does this command do? date +Y%m%d%H%M%S

This can be a user friendly way of providing a timestamp. Output would look like 20150418111854

What is *pool.ntp.org*?

This is a group of time servers behind a DNS alias which people can use to set their time.

What are *reference clocks*?

Throughout the world are various clock sources, such as government agencies broadcasting the time over radio, high-quality cesium clocks, or devices that read the current time from satellites. These clocks are called *reference clocks* in the NTP system.

How do you access the journal?

Using the *journalctl* command. By default you see the entire contents of the journal file on your screen. Depending on your configuration, such as the contents of the *PAGER* environment variable, the output may be paginated.

How can you filter logs with journald?

You can use the *grep* command or you can use the name of the service such as *sshd.service* or you can use the identifier such as *SYSLOG_IDENTIFIER* Example journalctl -u sshd.service | tail -5

9.) Which of the following syslog configuration rules ignores anything from the mail facility but captures info messages from other places? a.) *.debug;mail.none b.) *.alert;mail.* c.) *.notice;mail.none d.) *.info;-mail.*

a.) *.debug;mail.none A is correct because it captures everything at debug level and greater, which includes info, and removes mail with the mail.none keyword.

2.) If you ran *date "+%Y %m %d %H %M"* on April 26, 2015 at 8:19 p.m., which of the following would you see? a.) 2015 04 26 20 19 b.) 2015-04-26 8:19 c.) 1430097577 d.) Sun Apr 26 8:19:45pm CDT 2015

a.) 2015 04 26 20 19 A is correct because the format string given separates all elements by spaces and specifies year, numeric month, day of month, hour in 24 hour time, and minute.

1.) A Linux computer typically maintains two clocks. What are they? (Choose two). a.) Hardware clock b.) System clock c.) NTP clock d.) cron clock

a.) Hardware clock b.) System clock A is correct because the hardware clock is the real-time clock on the computer's motherboard. B is correct because the system clock is the clock maintained inside the Linux kernel.

4.) If your syslog.conf file contained a linue such as *.notice;mail.none /var/log/foo which of the following messages would make it into /var/log/foo? (Choose two). a.) kernel.warning b.) cron.info c.) mail.err d.) cron.err

a.) kernel.warning d.) cron.err The rule matches anything of notice priority or more important unless it is mail.

5.) Which command will log a message to syslog from the command line? a.) logger This is a message b.) log This is a message c.) syslog "This is a message" d.) echo "This is a message" | syslogd

a.) logger This is a message A is correct because the *logger* command takes a message on the command line and sends it to syslog.

7.) Which of the following is an alternative syslog implementation? a.) syslog-ng b.) rsyslog-ng c.) syslog++ d.) journald

a.) syslog-ng A is correct because the syslog-ng, along with rsyslog, are alternative syslog implementations.

Log Facilities: auth

auth = Security logs that can be public

Log Facilities: authpriv

authpriv = Security logs that need to be private

8.) Log rotation is triggered: a.) From the log rotation daemon b.) From cron on anacron c.) By the kernel d.) Manually

b.) From cron on anacron B is correct because anacron runs the logrotate program, which performs log rotation.

3.) if you ran *hwclock -w*, what would happen? a.) The hardware clock's time would be displayed in the local time zone. b.) The software clock time would be copied to the hardware clock. c.) The hardware clock time would be copied to the software clock. d.) The hardware clock's time would be displayed in UTC.

b.) The software clock time would be copied to the hardware clock. B is correct because the -w option write the current system clock to the hardware clock.

3.) You are about to reboot your computer but want to make sure that the computer boots with the correct time. How do you set the hardware clock using the value of the system clock? (Choose two.) a.) hwclock --hctosys b.) hwclock --systohc c.) hwclock -w d.) hwclock -r

b.) hwclock --systohc c.) hwclock -w B and C are correct because they are the long and short ways of writing the system clock to the hardware clock.

8.) Which of the following syslog severities are most servere than notice? (Choose two). a.) debug b.) warning c.) info d.) alert

b.) warning d.) alert B and D are correct because the severities, in order of least to most severe, are debug, info, notice, warning, err, crit, alert, emerg.

1.) The source of the Linux system clock is: a.) The motherboard's real-time clock b.) The Network Time Protocol daemon c.) The Linux kernel d.) A GPS clock

c.) The Linux kernel C is correct because the Linux kernel maintains the system clock.

7.) You have configured logrotate to rotate your custom application's log file, but the logs do not seem to be written to the new file --- they are still going to the timestamped file. What could be needed? (Chose two.) a.) The application is not a candidate for *logrotate* b.) logrotate is not running c.) You need the *copytruncate* option d.) You need to restart your *logrotate* daemon

c.) You need the *copytruncate* option d.) You need to restart your *logrotate* daemon C is correct because the *copytruncate* option rotates a log file by copying it and truncating the original. This allows you to rotate a log file without touching the process that owns it. D is correct because restarting the daemon gets it to reopen the log files with the nontimestamped names.

6.) How do you watch the systemd journal logs in real time a.) tail -f /var/log/journal/* b.) journalctl | tail -f c.) journaltcl -f d.) journalctl --continuous

c.) journaltcl -f C is correct because the *-f* flag puts *journalctl* in follow mode.

Log Facilities: cron

cron = Scheduled jobs such as cron and at

10.) You find out that your server is a stratum 2 NTP server. What does this mean? a.) There are two levels of servers getting time from your server. b.) It is getting its time from two different servers and averaging the results. c.) The time is within two standard deviations of normal. d.) The server that is getting its time from has a reference clock.

d.) The server that is getting its time from has a reference clock. D is correct because a stratum 2 server gets its time from a stratum 1 server, which must have a reference clock.

2.) Which of the following sets the clock to 6:30 p.m. on January 21, 2016, if the current year is 2015? a.) date 012120161830 b.) date 201601211830 c.) date +201601211830 d.) date 012118302016

d.) date 012118302016 D is correct because it uses the correct format - the year comes last, and the items before it are month, day, hour, and minute.

5.) Your server's clock is badly out of sync. How could you set the clock from a remote time source? a.) ntpdate pool.ntpd.com b.) ntpdate pool.ntp.com c.) ntpdate 0.pool.ntpd.org d.) nptdate pool.ntp.org

d.) nptdate pool.ntp.org D is correct because the pool.ntp.org is the name of the public NTP pool.

4.) Which command is most helpful in troubleshooting NTP problems? a.) ntp --debug b.) date c.) ntptool d.) ntpq

d.) ntpq D is correct because the *ntpq* command is used to query the ntp daemon and find out the status of peers.

Log Facilities: daemon

daemon = Other system daemons

What does this command do? date -u

date -u Displays the time in Universal Coordinated Time (UTC) Example: Wed Apr 15 19:36:05 UTC 2015

What does the syslog daemon do?

it is responsible for the collection and distribution of log messages. You can think of it as a pipeline that takes messages from various sources and directs them to the appropriate destination.

How can you ask for logs generated relative to a specific binary?

journalctl /usr/sbin/sshd This will only show logs generated for the sshd binary.

How do you see all of the details related to the log file?

journalctl SYSLOG_IDENTIFIER=ssh -o verbose

Log Facilities: kern

kern = kernel messages

Log Facilities: local0-7

local0-7 = Eight different user-definable facilities

If you want to send the message that you are logging to a different facility, which flag would you use?

logger *-p* mail.info This example will send it to the mail.info facility. The default log location is the *user* facility, but in this instance, the *-p* tells logger to send it to the *mail* facility. kern | user | mail | daemon | syslog | lpr | cron | auth | authpriv | local0-7

Log Facilities: lpr

lpr = Printing

Log Facilities: mail

mail - Email server

What are two other options instead of the *ksyslogd* package?

rsyslogd syslog-ng

Log Facilities: syslog

syslog = Internal messages for syslog

What is *systemd*'s logging system called?

systemd journal *OR* journald

how do you follow a log in real time?

tail -f logfilename i.e. tail -f /var/log/secure

Log Facilities: user

user = Random user level messages