Additional Linux notes

What is the difference between dmesg and syslog

- dmesg: writes specific events(kernel ring buffer) - syslog.conf - mail errors - application errors -

Maximum number of logical volumes on LVM

-l, --maxlogicalvolumes MaxLogicalVolumes Sets the maximum number of logical volumes allowed in this volume group. The setting can be changed with vgchange. For volume groups with metadata in lvm1 format, the limit and default value is 255. If the metadata uses lvm2 format, the default value is 0 which removes this restriction: there is then no limit.

mtab vs fstab

/etc/fstab contains a list of filesystems that CAN be mounted and it may include specific commands (options) that will be applied automatically when the filesystems are mounted. The "mount" command uses fstab to decide how to mount a filesystem, except if you explicitly specify all the mount parameters in the command. /etc/mtab on the other hand is where the "mount" command keeps a record of the filesystems that it HAS mounted. So mtab shows the filesystems that are currently mounted.

What is saved when creating a user

/etc/passwd /etc/skel /etc/group

.bashrc, .bash_profile, .profile, /etc/profile, etc ...

/etc/profile A global configuration script that applies to all users. ~/.bash_profile A user's personal startup file. Can be used to extend or override settings in the global configuration script. ~/.bash_login If ~/.bash_profile is not found, bash attempts to read this script. ~/.profile If neither ~/.bash_profile nor ~/.bash_login is found, bash attempts to read this file. This is the default in Debian-based distributions, such as Ubuntu. ~/.bashrc A user's personal startup file. Can be used to extend or override settings in the global configuration script

Broadcast Domain vs Collision Domain

A collision domain is, as the name implies, a part of a network where packet collisions can occur. A collision occurs when two devices send a packet at the same time on the shared network segment. The packets collide and both devices must send the packets again, which reduces network efficiency. Collisions are often in a hub environment, because each port on a hub is in the same collision domain. By contrast, each port on a bridge, a switch or a router is in a separate collision domain. A broadcast domain is a domain in which a broadcast is forwarded. A broadcast domain contains all devices that can reach each other at the data link layer (OSI layer 2) by using broadcast. All ports on a hub or a switch are by default in the same broadcast domain. All ports on a router are in the different broadcast domains and routers don't forward broadcasts from one broadcast domain to another.

What is a kernel

A kernel is the part of the operating system that mediates access to system resources. It's responsible for enabling multiple applications to effectively share the hardware by controlling access to CPU, memory, disk I/O, and networking. An operating system is the kernel plus applications that enable users to get something done (i.e compiler, text editor, window manager, etc).

What is a bootloader

A program that loads an operating system when a computer is turned on. "Linux's GRUB bootloader can be used to select which operating system you'd like to boot into"

Apache logs

Apache error and access logs: error log (/var/log/httpd/error_log The server error log, whose name and location is set by the ErrorLog directive, is the most important log file. This is the place where Apache httpd will send diagnostic information and record any errors that it encounters in processing requests. It is the first place to look when a problem occurs with starting the server or with the operation of the server, since it will often contain details of what went wrong and how to fix it. access log(/var/log/httpd/access_log) The server access log records all requests processed by the server. The location and content of the access log are controlled by the CustomLog directive. The LogFormat directive can be used to simplify the selection of the contents of the logs. This section describes how to configure the server to record information in the access log. /etc/httpd /etc/httpd/conf/httpd.conf

Backing up the MBR

Backing up the MBR The MBR is stored in the the first 512 bytes of the disk. It consist of 3 parts: - The first 446 bytes contain the boot loader. - The next 64 bytes contain the partition table (4 entries of 16 bytes each, one entry for each primary partition). - The last 2 bytes contain an identifier To save the MBR into the file "mbr.img": # dd if=/dev/hda of=/mnt/sda1/mbr.img bs=512 count=1

Correct permission for authorized_keys, home directory file

Change the file permissions of the .ssh directory to 700 (this means only the file owner can read, write, or open the directory). Important This step is very important; without these exact file permissions, you will not be able to log into this account using SSH. [newuser ~]$ chmod 700 .ssh Create a file named "authorized_keys" in the .ssh directory. [newuser ~]$ touch .ssh/authorized_keys Change the file permissions of the authorized_keys file to 600 (this means only the file owner can read or write to the file).

Kernel.sem

Contains semaphore parameters to see your value: #sysctl -A | grep kernel.sem /etc/sysctl.conf Semaphore can be described as counters used to control access to shared resources by multiple processes, They are most often used as a locking mechanism to prevent processes from accessing a particular resource while another process is performing operations on it. Semaphore can be used when number of processes try to access the shared resource or same file,Semaphore stored in kernel, so that it can be accessed by all the processes,

Header/Payload/Footer

Fundamental Message Elements While the format of a particular message type depends entirely on the nature of the technology that uses it, messages on the whole tend to follow a fairly uniform overall structure. In generic terms, each message contains the following three basic elements (see Figure 3): Header: Information that is placed before the actual data. The header normally contains a small number of bytes of control information, which is used to communicate important facts about the data that the message contains and how it is to be interpreted and used. It serves as the communication and control link between protocol elements on different devices. Data: The actual data to be transmitted, often called the payload of the message (metaphorically borrowing a term from the space industry!) Most messages contain some data of one form or another, but some actually contain none: they are used only for control and communication purposes. For example, these may be used to set up or terminate a logical connection before data is sent. Footer: Information that is placed after the data. There is no real difference between the header and the footer, as both generally contain control fields. The term trailer is also sometimes used.

NAT

Instances that you launch into a private subnet in a virtual private cloud (VPC) can't communicate with the Internet. You can optionally use a network address translation (NAT) instance in a public subnet in your VPC to enable instances in the private subnet to initiate outbound traffic to the Internet, but prevent the instances from receiving inbound traffic initiated by someone on the Internet. NAT serves three main purposes: Provides a type of firewallby hiding internal IP addresses Enables a company to use more internal IP addresses. Since they're used internally only, there's no possibility of conflict with IP addresses used by other companies and organizations. Allows a company to combine multiple ISDN connections into a single Internet connection.

LILO vs GRUB

LILO was the default boot loader for most Linux distributions in the years after the popularity of loadlin. Today, most distributions use GRUB as the default boot loader. LILO has no interactive command interface, whereas GRUB does. LILO does not support booting from a network, whereas GRUB does. LILO stores information regarding the location of the operating systems it can to load physically on the MBR. If you change your LILO config file, you have to rewrite the LILO stage one boot loader to the MBR. Compared with GRUB, this is a much more risky option since a misconfigured MBR could leave the system unbootable. With GRUB, if the configuration file is configured incorrectly, it will simply default to the GRUB command-line interface.

/etc/security.conf

Limiting user processes is important for running a stable system. To limit user process, you have just to set shell limit by adding: a user name or group name or all users to /etc/security/limits.conf file and impose then process limitations. Example of /etc/security/limits.conf file

PAM

Q1: What exactly is PAM? A1: PAM = Pluggable Authentication Modules Basically, it is a flexible mechanism for authenticating users. Since the beginnings of UNIX, authenticating a user has been accomplished via the user entering a password and the system checking if the entered password corresponds to the encrypted official password that is stored in /etc/passwd . The idea being that the user *is* really that user if and only if they can correctly enter their secret password. That was in the beginning. Since then, a number of new ways of authenticating users have become popular. Including more complicated replacements for the /etc/passwd file, and hardware devices Smart cards etc.. The problem is that each time a new authentication scheme is developed, it requires all the necessary programs (login, ftpd etc...) to be rewritten to support it. PAM provides a way to develop programs that are independent of authentication scheme. These programs need "authentication modules" to be attatched to them at run-time in order to work. Which authentication module is to be attatched is dependent upon the local system setup and is at the discretion of the local system administrator. additional info about PAM Programs that grant users access to a system use authentication to verify each other's identity (that is, to establish that a user is who they say they are). Historically, each program had its own way of authenticating users. In Red Hat Enterprise Linux, many programs are configured to use a centralized authentication mechanism called Pluggable Authentication Modules (PAM). PAM uses a pluggable, modular architecture, which affords the system administrator a great deal of flexibility in setting authentication policies for the system. PAM is a useful system for developers and administrators for several reasons: PAM provides a common authentication scheme that can be used with a wide variety of applications. PAM provides significant flexibility and control over authentication for both system administrators and application developers. PAM provides a single, fully-documented library which allows developers to write programs without having to create their own authentication schemes.

SFTP

SFTP (Secure File Transfer Protocol) runs over SSH protocol on standard port 22 by default to establish a secure connection. SFTP has been integrated into many GUI tools (FileZilla, WinSCP, FireFTP etc.). sftp stands for "secure FTP". It is a command-line program for transferring files securely over a network connection. sftp performs all operations over an encrypted ssh session. It uses many of the features of ssh, such as public key authentication and data compression.

LNC-CSMACD

Short for Carrier Sense Multiple Access / Collision Detection, a set of rules determining how network devices respond when two devices attempt to use a data channel simultaneously (called a collision). Standard Ethernet networks use CSMA/CD to physically monitor the traffic on the line at participating stations. If no transmission is taking place at the time, the particular station can transmit. If two stations attempt to transmit simultaneously, this causes a collision, which is detected by all participating stations. After a random time interval, the stations that collided attempt to transmit again. If another collision occurs, the time intervals from which the random waiting time is selected are increased step by step. This is known as exponential back off. CSMA/CD is a type of contention protocol.

NTP

Short for Network Time Protocol, an Internet standard protocol (built on top of TCP/IP) that assures accurate synchronization to the millisecond of computer clock times in a network of computers. Based on UTC, NTP synchronizes client workstation clocks to the U.S. Naval Observatory Master Clocks in Washington, DC and Colorado Springs CO. Running as a continuous background client program on a computer, NTP sends periodic time requests to servers, obtaining server time stamps and using them to adjust the client's clock. Runs over UDP port 123, and is used to sync time between machines on a network.

ethtool vs miitool(deprecated)

Since mii-tool is deprecated, I'd go for ethtool. notice that mii-tool can only report up to 100MBit, while ethtool is aware of the gigabit concept, and knows how to look at a greater range of hardware. Ethtool utility is used to view and change the ethernet device parameters 4. Display Ethernet Driver Settings ethtool -i option displays driver version, firmware version and bus details as shown below. # ethtool -i eth0 driver: bnx2 version: 2.0.1-suse firmware-version: 1.9.3 bus-info: 0000:04:00.0

SMP Kernel

Symmetric Multi Processing. In human-language, it is the option that allow the kernel to run two or more processes running concurrently at the same time (this is different from multithreading on single core, single-CPU system, which two or more processes taking turns to use the single CPU). Enable it if you have multi-core or multi-CPU system. If you have a multi-core or multi-CPU system, and this option is disabled, the kernel willl only use one core/CPU at a time (which is a big loss). On a single core, single CPU system, leave the option disabled as it may slow down the kernel.

What type of files are in /etc/init.d

System startup scripts for every single program in the Linux system. Shell scripts, can find out the type of file by typing the command. file sshd netfs: Bourne-Again shell script, ASCII text executable

/password and /shadow file

The /etc/passwd file Well, this is the file in Linux system that contains all the relevant information related to user login. The /etc/passwd file also contains information like user ID's and group ID's that are used by many system programs. Therefore, the /etc/passwd file must remain world readable. The /etc/shadow file As discussed earlier, this file contains encrypted password entries for users in the system. Beside containing encrypted passwords, an entry in this file also contains ageing and expiration information of password. An entry in this file can be classified into following information: Login name The corresponding Encrypted password Number of days since 1st Jan 1970, that password was last changed Number of days before password may be changed Number of days after which password has to be changed Number of days before password expiry warning starts popping up Number of days after password expires that account is disabled Number of days since 1st Jan 1970, that account is disabled Reserved field for further use. The /etc/shadow file is set so that it cannot be read by just anyone. Only root will be able to read and write to the /etc/shadow file. When a user picks or is assigned a password, it is encoded with a randomly generated value called the salt. This means that any particular password could be stored in 4096 different ways. The salt value is then stored with the encoded password. When a user logs in and supplies a password, the salt is first retrieved from the stored encoded password. Then the supplied password is encoded with the salt value, and then compared with the encoded password. If there is a match, then the user is authenticated. Basic info: Short answer: passwd stores general user info and shadow stores user passwd info. Somewhat longer answer: passwd is the file where the user information (like username, user ID, group ID, location of home directory, login shell, ...) is stored when a new user is created. shadow is the file where important information (like an encrypted form of the password of a user, the day the password expires, whether or not the passwd has to be changed, the minimum and maximum time between password changes, ...) is stored when a new user is created.

What is grub configuration file

The GRUB menu interface configuration file is /boot/grub/grub.conf. The commands to set the global preferences for the menu interface are placed at the top of the file, followed by stanzas for each operating kernel or operating system listed in the menu.

Iterated DNS

The goal of DNS is to resolve a fully qualified domain name (FQDN) to an IP address. This work can either be done by the DNS server or the DNS client. These approaches are referred to as Recursive or Iterative, respectively. In the Recursive approach, a client sends a query to the server. Assuming recursion is enabled, the server then looks for resolution first locally in its own database, then in its local cache, finally by going through the DNS tree until it finds a server that can give an authoritative answer to the query. In this model, the client is referred to as a Stub Resolver. Typically, Stub Resolvers are implemented on devices with limited resources such as embedded systems or Personal Computers. In the Iterative approach, the client sends a query to the server. If recursion is disabled, and the server cannot answer the query, the server will responds with a Referral answer. The client will then use that information to query another DNS server. This process will continue until a server responds with an Authoritative response.

kernel ring buffer

The kernel ring buffer is a data structure that records messages related to the operation of the kernel. A ring buffer is a special kind of buffer that is always a constant size, removing the oldest messages when new messages come in. dmesg examines or controls the kernel ring buffer.

syslog.conf

The syslog.conf file is the main configuration file for the syslogd(8) which logs system messages on *nix systems. This file specifies rules for logging

Kernel Parameter

There are three ways to pass options to the kernel and thus control its behaviour: - When building the kernel. See Kernel Compilation for details. - When starting the kernel (usually, when invoked from a boot loader). - At runtime (through the files in /proc and /sys). See sysctl for details. Kernel parameters can be set either temporarily by editing the boot menu when it shows up, or by modifying the boot loader's configuration file. *Parameter list* root= Root filesystem. ro Mount root device read-only on boot (default1). rw Mount root device read-write on boot. initrd= Specify the location of the initial ramdisk. init= Run specified binary instead of /sbin/init (symlinked to systemd in Arch) as init process. init=/bin/sh Boot to shell. systemd.unit= systemd.unit=multi-user Boot to a specified runlevel. systemd.unit=rescue Boot to single-user mode (root). nomodeset Disable Kernel Mode Setting. zswap.enabled Enable Zswap. https://wiki.archlinux.org/index.php/kernel_parameters

Dig command

Using dig command you can query DNS name servers for your DNS lookup related tasks. dig -t NS redhat.com dig -t MX dig -t A... dig -t ANY redhat.com reverse DNS lookup dig -x <IP> http://www.thegeekstuff.com/2012/02/dig-command-examples/

X11/XOrg

Xorg is the public, open-source implementation of the X window system version 11. Since Xorg is the most popular choice among Linux users, its ubiquity has led to making it an ever-present requisite for GUI applications, resulting in massive adoption from most distributions. To launch the X server and clients, run: $ startx http://wiki.gentoo.org/wiki/Xorg/Configuration

What is a volume label, and when is it used for?

You might have seen labeled partitions if you have opened and viewed the /etc/fstab file. There you can see that the 'root', 'home', 'boot, and other system partitions are labeled and are mounted using the label rather than referring the device name. The advantage is that, the root partition will be the same even if the device name got changed in an unlikely event during the system startup. The volume labels make the partition retain a consistent name regardless of where they are connected and what else are connected. And for your information such an unlikely event of changing the device name is quiet often if you are mounting mounting multiple iSCSI drives to your system. http://foralllinux.blogspot.com/2010/03/labeling-linux-partition-volume-labels.html

alias

alias =cd /etc/var/log.... unalias to remove The alias command makes it possible to launch any command or group of commands (inclusive of any options, arguments and redirection) by entering a pre-set string (i.e., sequence of characters). That is, it allows a user to create simple names or abbreviations (even consisting of just a single character) for commands regardless of how complex the original commands are and then use them in the same way that ordinary commands are used.

Number of CPU on an instance

cat /proc/cpuinfo lscpu nproc -all

Cannot resolve hostname

check safari books for troubleshooting

Chmod chusr chroot

chmod - modify file access rights su - temporarily become the superuser chown - change file ownership chgrp - change a file's group ownership

Curl command

curl -I www.yahoo.com I, --head (HTTP/FTP/FILE) Fetch the HTTP-header only! HTTP-servers feature the command HEAD which this uses to get nothing but the header of a document. When used on an FTP or FILE file, curl displays the file size and last modification time only.

Where is the bootloader stored?

grub.conf, what is grub.conf? term of Ubuntu system, the process is as following: After you turn on your PC, BIOS (which is stored in ROM) automatically runs and initializes various parts of PC hardware. Then it checks for a specific sector in the defined first Boot device (typically Hard disk). This sector is boot-sector, which is 512 bytes in size. The program in the boot-sector is loaded in memory (1st stage). This tiny program has information which programs should it load into memory next and where that program is located in Disk or Boot device. It loads that program. In Ubuntu, it is /boot/grub/core.img. In the 2nd stage, The OS-Loader, GRUB loads Ubuntu by loading kernel and initial ram disk into the memory and hand-over control to the kernel. Then kernel runs and loads all of the necessary programs such as display manager, Gui's etc. So, We can clearly say, Boot-loader neither stored in ROM, nor in RAM, It is actually stored on Hard disk (or other Boot device, such as bootable CDROM, USB drives etc) , precisely speaking the first sector of the hard disk, which is of size 512 bytes and often referred as boot-sector. And this bootloader loads OS-loader (in Ubuntu, it is grub) which is also resides in Hard disk (i.e /boot/grub/ folder) , and it's task is loading the OS (say, Ubuntu).

Using grep to sort file/output

http://unix.stackexchange.com/questions/138318/how-do-i-sort-using-the-grep-and-sort-commands

Whats in the root directory, important directories

http://www.thegeekstuff.com/2010/09/linux-file-system-structure/

Troubleshooting logs: var/log/*

http://www.thegeekstuff.com/2011/08/linux-var-log-files/

i386/i686

i386 and i686 are both part of the x86 family of processors. They just refer to the specific age of the processor platform. i386 is an older platform (early 90s?) used back when 386 processors were used in machines. Then this was upgraded to 486 processors, which was the same basic instruction set as 386 just faster and newer. 586 was another upgraded and was when the term Pentium started floating around. Eventually all of these got encapsulated into the x86 architecture name. i686 just refers to the 6th generation of x86 architecture. For all intents and purposes i386 and i686 are the same thing, just that i686 is a lot newer. They will have some additional instruction sets, but will be backwards compatible with i386. I would venture a guess that all of the processes you find today that use x86 would be classified as i686 processors. x86 refers to a processor family. x86_64 is different than x86 (it is the 64 bit brother of x86). There are also Alpha processors, Sparc, ARM, PPC. These all refer to different processor families.

insmod/modinfo/modprobe

insmod - install loadable kernel module , insmod installs a loadable module in the running kernel. modinfo - program to show information about a Linux Kernel module modprobe - high level handling of loadable modules used to add a loadable kernel module (LKM) to the Linux kernel or to remove a LKM from the kernel. It is commonly used indirectly: udev relies upon modprobe to load drivers for automatically detected hardware.

lspci

lspci - list all PCI devices lspci is a utility for displaying information about PCI buses in the system and devices connected to them. By default, it shows a brief list of devices. Use the options described below to request either a more verbose output or output intended for parsing by other programs.

/etc/nsswitch.conf

nsswitch.conf - Name Service Switch configuration file The Name Service Switch (NSS) configuration file, /etc/nsswitch.conf, is used by the GNU C Library to determine the sources from which to obtain name-service information in a range of categories, and in what order. Each category of information is identified by a database name. The /etc/nsswitch.conf file is used to configure which services are to be used to determine information such as hostnames, password files, and group file

/etc/resolv.conf

resolv.conf - resolver configuration file http://man7.org/linux/man-pages/man5/resolv.conf.5.html

Permissions

rwx = 111 in binary = 7 rw- = 110 in binary = 6 r-x = 101 in binary = 5 r-- = 100 in binary = 4 ex: -rwxr-xr-x - means regular file if the - was replaced by d, it would mean its a directory. owner - The Owner permissions apply only the owner of the file or directory, they will not impact the actions of other users. group - The Group permissions apply only to the group that has been assigned to the file or directory, they will not effect the actions of other users. all users - The All Users permissions apply to all other users on the system, this is the permission group that you want to watch the most.

ss (sockets in linux)

ss is used to dump socket statistics. It allows showing information similar to netstat. It can display more TCP and state informations than other tools. The ss command is capable of showing more information than the netstat and is faster. The netstat command reads various /proc files to gather information. However this approach falls weak when there are lots of connections to display. This makes it slower. The ss command gets its information directly from kernel space. The options used with the ss commands are very similar to netstat making it an easy replacement. ss - socket statistics the netstat command has long been deprecated and replaced by the ss command from the iproute suite of tools. List all connections The simplest command is to list out all connections. $ ss | less 2. Filter out tcp,udp or unix connections To view only tcp or udp or unix connections use the t, u or x option. $ ss -t udp is ss -ua 4. Show only listening sockets This will list out all the listening sockets. For example apache web server opens a socket connection on port 80 to listen for incoming connections. $ ss -ltn http://www.binarytides.com/linux-ss-command/

stateful vs stateless

stateful= security group stateless = ACL http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Security.html

How to flush iptables? How to save iptables permanently

stop iptables: sudo service iptables stop status: sudo service iptables status iptables -L IPTables Rules are stored in /etc/sysconfig/iptables Temporarily delete all the firewall rules Use 'iptables -flush' option to delete all the rules temporarily. Permanently remove all the default firewall rules # cat /etc/sysconfig/iptables # iptables --flush # service iptables save

sysctl, /etc/sysctl.conf

sysctl - configure kernel parameters at runtime sysctl is used to modify kernel parameters at runtime. The parameters available are those listed under /proc/sys/

tempfs/shm

tmpfs is a temporary filesystem that resides in memory and/or your swap partition(s), depending on how much you fill it up. Mounting directories as tmpfs can be an effective way of speeding up accesses to their files, or to ensure that their contents are automatically cleared upon reboot. Some directories where tmpfs is commonly used are /tmp, /var/lock and /var/run. Do not use it on /var/tmp, because that folder is meant for temporary files that are preserved across reboots. By default, a tmpfs partition has its maximum size set to half your total RAM, but this can be customized. 31 down vote accepted /dev/shm is a temporary file storage filesystem, i.e. tmpfs, that uses RAM for the backing store. Since RAM is significantly faster than disk storage, you can use /dev/shm instead of /tmp for the performance boost, if your process is I/O intensive and extensively uses temporary files. Generally, I/O intensive tasks and programs that run frequent read/write operations can benefit from using a tmpfs folder. Some applications can even receive a substantial gain by offloading some (or all) of their data onto the shared memory.

traceroute how does traceroute work?

traceroute -- print the route packets trace to network host traceroute tracks the route packets taken from an IP network on their way to a given host. It utilizes the IP protocol's time to live (TTL) field and attempts to elicit an ICMP TIME_EXCEEDED response from each gateway along the path to the host.

time in linux

use date command

Creating user and pass and viewing users

useradd test passwd test123 view users: cat /etc/passwd | grep home