RHEL - Chapter 13 - Archiving and Transferring Files

The following command creates an archive named archive.tar with the contents file 1, file 2, file 3 gin the user's home directory.

$ tar -cf archive.tar file1 file2 file3 -- or -- $ tar --file=archive.tar --create file1 file2 file3

Overview of tar options - 3 options

-c, --create ---> Create a new archive -x -extract ---> Extract from an existing archive -t, --list ---> List table of contents for an archive

To preserve the permissions of an archived file, use the ____ option when extracting and archive.

-p [must be root]

Options Enabled with rsync -a (Archive Mode)

-r, --recursive synchronize recursively the whole directory tree -l, --links synchronize symbolic links -p, --perms preserve permissions -t, --times preserve time stamps -g, --group preserve group ownership -o, --owner preserve the owner of the files -D, --devices synchronize device file

option to list contents of a .tar file

-t ---> t = 'Table of Contents' ------ tar -tf /root/etc.tar

- Decompressing single files -

The corresponding commands to decompress are gunzip, bunzip2, and unxz. For example, the gunzip /tmp/etc.tar.gz command results in the etc.tar uncompressed tar file, while the bunzip2 abc.tar.bz2 command results in the abc.tar uncompressed tar file, and the unxz myarchive.tar.xz command results in the myarchive.tar uncompressed tar file

EXTRACTING A COMPRESSED ARCHIVE

The first step when extracting a compressed tar archive is to determine where the archived files should be extracted to, then create and change to the target directory. ------------ The tar command determines which compression was used and it is usually not necessary to use the same compression option used when creating the archive To extract the contents of a gzip compressed archive named /root/etcbackup.tar.gz in the /tmp/etcbackup directory: [root@host ~]# mkdir /tmp/etcbackup [root@host ~]# cd /tmp/etcbackup [root@host etcbackup]# tar -tf /root/etcbackup.tar.gz etc/ etc/fstab etc/crypttab etc/mtab ...output omitted... [root@host etcbackup]# tar -xzf /root/etcbackup.tar.gz

Command for transferring files securely between systems (via ssh)...

SCP - (Secure Copy)

Selected tar general options

-v, --verbose ---> Verbose. Shows which files get archived / extracted. -f, --file ---> File-name. this option must be followed by the file name of the archive to be used or to create -p, --preserve-permissions ---> Preserve the permissions of files and directories when extracting an archive, without subtracting the umask.

options to choose a compression method

-z or -- gzip ---> gzip (filename.tar.gz or filename.tgz) -j or --bzip2 ---> bzip2 (filename.tar.bz2) -J or --xz ---> xz(filename.tar.xz)

Overview of tar compression options

-z, --gzip ---> Use gzip compression (.tar.gz) -j, --bzip2 ---> Use bzip2 for compression (.tar.bz2) Bzip typically achieves a better compression ratio than gzip. -J, --xz ---> Use xz compression (.tar.xz). The xz compression ratio typically achieves a better compression ratio than Bzip.

The archives will NOT include files that the archiving user does dot have permissions to read.

...

Tar file

A tar archive is a structured sequence of file data mixed in with metadata about each file and index so that individual files can be extracted. The archive can be compressed using gzip, bzip2, or xz compression. The tar command can list the contents of archives or extract their files to the current system.

Extracting tar file

A tar should usually be extracted in an empty dir, as it will overwrite existing files. To restore files from the /root/etc.tar archive to the /root/etcbackup dir, run: # mkdir /root/etcbackup # cd root/etcbackup # tar -tf /root/etc.tar # tar -xf /root/etc.tar

- Compressing single files -

Additionally, gzip, bzip2, and xz can be used independently to compress single files. For example, the gzip etc.tar command results in the etc.tar.gz compressed file, while the bzip2 abc.tar command results in the abc.tar.bz2 compressed file, and the xz myarchive.tar command results in the myarchive.tar.xz compressed file.

Using sftp

The interactive sftp session accepts various commands that work the same way on the remote file system as they do in the local file system, such as ls, cd, mkdir, rmdir, and pwd. The put command uploads a file to the remote system. The get command downloads a file from the remote system. The exit command exits the sftp session.

rsync

The rsync command is another way to securely copy files from one system to another. The tool uses an algorithm that minimizes the amount of data copied by synchronizing only the changed portions of files. It differs from scp in that if two files or directories are similar between two servers, rsync copies only the differences between the file systems, while scp would still copy everything. An advantage of rsync is that it can copy files between a local system and a remote system securely and efficiently. While an initial directory synchronization takes about the same time as copying it, subsequent synchronizations only require the differences to be copied over the network, substantially speeding updates. An important option of rsync is the -n option to perform a dry run. A dry run is a simulation of what happens when the command gets executed. The dry run shows the changes rsync would perform when the command is run normally. Perform a dry run before the actual rsync operation to ensure no important files get overwritten or deleted

TRANSFERRING FILES USING THE SECURE FILE TRANSFER PROGRAM

To interactively upload or download files from a SSH server, use the Secure File Transfer Program, sftp. A session with the sftp command uses the secure authentication mechanism and encrypted data transfer to and from the SSH server. Just like the scp command, the sftp command uses [user@]host to identify the target system and user name. If you do not specify a user, the command will attempt to log in using your local user name as the remote user name. You will then be presented with an sftp> prompt. [user@host ~]$ sftp remoteuser@remotehost remoteuser@remotehost's password: password Connected to remotehost. sftp>

rsync -v or rsync --verbose ----------------------------------- rsync -a or rsync --archive

Two common options when synchronizing with rsync are the -v and -a options. The -v or --verbose option provides more detailed output. This is useful for troubleshooting and to view live progress. The -a or --archive option enables "archive mode". This enables recursive copying and turns on a large number of useful options that preserve most characteristics of the files. Archive mode is the same as specifying the following options:

Rsync - Trailing Slashes

When typing the source directory in the rsync command, it is significant whether a trailing slash is present on the directory name. It determines whether the directory or just the contents of the directory are synchronized to the target. Bash Tab-completion automatically adds a trailing slash to directory names

Format for remote location in SCP command:

[user@]host:/path The user@ portion of the argument is optional. If it is missing, your current local username will be used. When you run the command, your scp client will authenticate to the remote SSH server just like ssh, using key-based authentication or prompting you for your password.

To copy a whole directory tree recursively with scp, use -r

[user@host ~]$ scp -r root@remoteuser:/var/log /tmp

copy the local /etc/yum.conf and /etc/hosts files on host, to the remoteuser's home directory on the remotehost remote system:

[user@host ~]$ scp /etc/yum/conf /etc/hosts remoteuser@remotehost:/home/remoteuser

Rsync - to synchronize contents of the /var/log directory to the /tmp directory...

[user@host ~]$ su - Password: password [root@host ~]# rsync -av /var/log /tmp receiving incremental file list log/ log/README log/boot.log ...output omitted... log/tuned/tuned.log sent 11,592,423 bytes received 779 bytes 23,186,404.00 bytes/sec total size is 11,586,755 speedup is 1.00 [user@host ~]$ ls /tmp

Compression methods compared

gzip - fastest, oldest, most widely available. bzip2 - creates smaller archives than gzip, but is less widely available. xz - relatively new, but offers the best compression ratio of the methods available.

You can also use scp to copy files in the other direction, from remote to local. In this example, the file /etc/hostname on remotehost is copied to the local directory /home/user:

scp remoteuser@remotehost:/etc/hostname /home/user

To upload the /etc/hosts file on the local system to the newly created directory /home/ remoteuser/hostbackup on remotehost:

sftp> mkdir hostbackup sftp> cd hostbackup sftp> put /etc/hosts Uploading /etc/hosts to /home/remoteuser/hostbackup/hosts /etc/hosts 100% 227 0.2KB/s 00:00 sftp>

To create a xz compressed archive named /root/sshconfig.tar.xz, with contents from the /etc/ssh dir on host:

tar -cJf /root/sshconfig.tar.xz /etc/ssh

To create the tar archive named, /root/etc/.tar, with the /etc directory content as user root...

tar -cf /root/etc.tar /etc * Will remove the leading '/' from tar filename.

To create a bzip2 compressed archive named /root/logbackup.tar.bz2, with contents from the /var/log dir on host:

tar -cjf /root/logbackup.tar.bz2 /var/log

To create a gzip compressed archive named /root/etcbackup.tar.gz with the contents from the /etc directory on host:

tar -czf /root/etcbackup.tar.gz /etc