Build Your Own
How To Videos
Quick Reference
View Order Pad

About Us
Contact Us
Linux News

Acceptance Mark
Amazon Pay


Linux Quick Reference

This is a collection of useful information when working with Linux. If you are new to Linux, some of this may be invaluable. If you are experienced, this reference may also be helpful in getting the syntax correct, or performing specific tasks.


This is a work in progress, and it is highly biased towards the specific needs of the author. If you have any input or feedback, or wish more details on any particular subject, please feel free to contact us.

For details on the history of Linux, UNIX, and the BSDs, there are numerous resources (Wikipedia - History of Linux is a good start). This is all about using these operating systems. The fact that these operating systems were written by developers, for developers, and have a history dating back to when bandwidth and storage space was valuable, means that not everything is intuitive or fits in well with the preconceived notions of different aged users or different cultures. More importantly, this means that not everything is necessarily obvious to any particular person. In fact, certain things only make sense when you understand the history and context of the decisions made during the design of a particular portion of the entire system. The reality is that not everyone has time to learn everything they need to know when a problem presents itself. The only choice then is to do what needs to be done.

This is not intended to be a comprehensive reference, nor a replacement for man or info pages. This is all about using Linux, and a place to find things that you might have used once, but that was years ago, and you need to do it NOW!

If you are new to Linux, you may want to refer to some of our How To Videos...

Here are a few sites that have some useful information and a good place to search for details:

Here are useful/interesting tools and applications:

  • Flathub - Flathub has hundreds of apps which can be easily installed on any Linux distribution.
  • Linux From Scratch - Linux From Scratch (LFS) is a project that provides you with step-by-step instructions for building your own customized Linux system entirely from source.
  • OSAlt.com - Open Source Alternatives to well-known commercial software
  • SourceForge.net - thousands of software projects
  • VirtualBox.org - Open Source, GPL virtual machine solution

Here are some sites for those interested in developing software:

Here are online options for productivity apps/tools:

  • In addition to Microsoft Office 365 and Google G-Suite, there are alternatives.
  • Notes/Docs: Quip, Notejoy, Slite, Zenkit, Notion and Agenda.
  • Spreadsheets: Bellevue, Smartsheet, Airtable, Coda and Trello.

Reference and resource links:

Quick Start

If you are brand new to Linux, and not sure where to start, here is a guide (with some insights)

  • Start with a distribution, any recent release is fine (shameless plug - try the Popular Collection)
  • Find an unused or older system - if you don't have one, check with family, friends, associates. It is best to start with a clean system, empty hard drive, set to boot from DVD (if you leave the side off of your PC, it makes it fairly easy to swap out hard drives - just a couple of cables. You can buy hard drives by the lot on eBay, and then swap out an installed OS quickly & easily by swapping the hard drive. It also is sometimes helpful to delete all partitions on the hard drive with fdisk before starting (so the Install sees it as a "new" drive) - boot to a rescue prompt, run fdisk /dev/sda and d to delete partifion(s), then w write - then reboot & do install. We would recommend a separate / unused hard drive (or separate system) to learn Linux on - it gives you freedom to do anything, and even if something goes horribly wrong, you can just re-install.)
  • Step through install process, accept defaults (until you know otherwise, use the whole disk when presented as an option)
  • After install, boot & log in (get in the habit of using a default user name and password when asked during install - either your name & regular password, or "user" and "password" - for non-secured/play around installs, use a password of "password" or the user name, e.g. "user" and "user")
  • Verify internet connection (this depends on your network, DHCP, settings, etc., but you want to have internet working right away)
  • Find the way to get to the terminal (Utilities or System or Accessories or right-click, etc. - also, most X Windows supports Ctrl-Alt-F1 for a terminal screen (Ctrl-Alt-F7 typically to get back to graphical interface))
  • Now explore, run tutorials, read, play, try things, basically (as the old SuSE said) Have a Lot of Fun!

The Beginning - Boot sequence summary      Boot loaders
Boot from USB Drive, USB Version of USB Drive      fdisk notes
fsck notes      file basics - cp, mv, rm, rename, basename, dirname, rsync
Bash Scripts (various)      Network File Basics (ftp/scp/wget/smbclient)
Network Basics (route, ifconfig, netstat, etc.)      Web Basics (dns/html/url encoding)
Process Basics      System Information
utility basics - man, info, file, ps2pdf, pdftk, mutool, fromdos, iconv, tr & UTF notes      utility basics - sed, grep
utility basics - find, which, whoami, id, w      Linux directories
Integrate scripts with X      X server Notes
VNC - Virtual Network Computing      split, tar, gz (file size limits)
date - How to set the date / time      Accessing a CD/DVD ISO image
Obtaining MD5 Sum from CD/DVD/USB      Controlling Users
passwd notes      Controlling Processes
smb:// or smbclient unable to connect by name      Manipulating images
Sound & Video      Manipulating mail box files
Scanners      Debian exim configuration
APT on CD      Postfix
DNS Tools      Debian package management
Debian Packages - Create Local Repository      Debian Configuration Notes
Mouse Configuration      MySQL Statements Reference

The Beginning - Boot sequence summary


Load boot sector from one of: Floppy, CDROM, Hard drive
The boot order can be changed from within the BIOS. BIOS setup can be entered by pressing a key during bootup. The exact key depends varies, but is often one of Del, F1, F2, or F10.

Master Boot Record (MBR)

BIOS loads and executes the first 512 bytes off the disk (/dev/hda). This is very limited code, but typically enough to check for bootable partition and load the next stage (LILO/GRUB)


  • does not understand filesystems
  • code and kernel image to be loaded is stored as raw disk offsets
  • uses the BIOS routines to load
Loading sequence
  • load menu code, typically /boot/boot.b
  • prompt for (or timeout to default) partition or kernel
  • for "image=" (ie Linux) option load kernel image
  • for "other=" (ie DOS) option load first 512 bytes of the partition


  • Understands file systems
  • config lives in /boot/grub/menu.lst or /boot/boot/menu.lst

Startup splash notes

  • The opening "splash" display is typically unique and a special aspect of a distribution.
  • bootsplash - requires kernel modification for framebuffer (video) interaction - be sure to review all options available
  • Splashy (user space, supercedes bootsplash)
  • Usplash (user space)
  • Red Hat Graphical Boot


  • initialize devices
  • (optionally loads initrd, see below)
  • mounts root filesystem
    • specified by lilo or loadin with root= parameter
    • kernel prints: VFS: Mounted root (ext2 filesystem) readonly.
  • runs /sbin/init which is process number 1 (PID=1)
    • init prints: INIT: version 2.76 booting
    • can be changed with boot= parameter to lilo, eg boot=/bin/sh can be useful to rescue a system which is having trouble booting.


Allows setup to be performed before root File System is mounted
This ram disk is system dependent and typically must be generated when changes to the kernel or hardware affects the existing initrd. This is a tool that provides an environment for the kernel and system devices to prepare for loading other file system modules, and other pieces the kernel will need to operate the live system, prior to mounting the root file system.

  • lilo or loadlin or grub loads ram disk image
    • load modules
    • initialize devices
  • "real" root is mounted
  • kernel runs /sbin/init


  • reads /etc/inittab which specifies the scripts below
    • Run boot scripts (system dependent /etc/init.d, /etc/rcS.d, /etc/rc.boot, /etc/rc3.d, etc.)
    • switches to default runlevel (e.g. 3)

Run Levels

  • 0 halt
  • 1 single user
  • 2-4 user defined (3 is typically multi-user, text console)
  • 5 X11 only (0 or 1 text console) (GUI environment)
  • 6 Reboot
  • Default is defined in /etc/inittab, eg:
    • id:3:initdefault:
  • The current runlevel can be changed by running /sbin/telinit # where # is the new runlevel, eg typing telinit 6 will reboot.

Boot loaders

GRUB Problem: You've just imaged a drive, moved it to a new system, and it won't boot. (/dev/hda vs. /dev/sda issue, ide vs. sata)

You can edit the Grub menu and change the root drive, e.g. /dev/sda3 to /dev/hda3. This may allow a boot or partial boot, and allow you to get to a login prompt where you can login as root and fix other items (see below). Alternatively, you can boot with a rescue disc via CD/DVD.

Issues you may run into:
fstab - look for mount points that need to be updated, e.g. /dev/sda1 should be /dev/hda1, etc.
could not stat resume device file - could be the swap partition:
- Try swapoff /dev/sdb2 to make sure it's not in use, then mkswap /dev/hda2
- uswsusp.conf - if there's any line like resume /dev/sda2, change to /dev/hda2
- initramfs - update it with update-initramfs -u -k all
GRUB Menu - edit / update /boot/grub/menu.lst or entry with /dev/sda? to /dev/hda?

GRUB Problem: You've just imaged a drive, and it won't boot.

There are various things that can happen with an imaged disk - if the hard drive is a different size or not an exact clone, grub may have trouble finding its stage 1/stage 1.5/stage 2 pieces (perhaps because they are oriented differently on newly imaged drive). Unfortunately there is no specific sequence to offer here because there are various situations you may be facing if it doesn't boot - it could be stuck in a reboot loop, it could drop to a grub> prompt, or it the grub menu items may not work. Also, grub has changed somewhat over the years (now grub2), and what is your situation? Are you running a new system or an old system? Which version of grub? What exactly has happened on your system? So here is a collection of things that may give you some options, but it is best to try and identify and answer the above questions to try and determine what exactly is your problem and the best solution.

If your system is in a reboot loop, you must get into a rescue system (and best to use as close to the installed operating system on the hard drive as possible) - boot a live linux disc/USB and get to command prompt/terminal. As super user (or via system GUI) mount the drive. Note that if your system can't see or mount the drive, you may be having problems other than a misconfigured grub(!) Here is a sequence to take a mounted drive from a rescue environment and configure it so it acts as if it was booted directly, so you can run grub-install:

As superuser (sudo su, su/[password]):
# mount /dev/sdaN /mnt (where sda could be something else if not first hard drive, N is main partition (usually 1), and /mnt exists)
# mount --rbind /dev /mnt/dev (makes rescue dev as if imaged drive's /dev)
# mount --rbind /sys /mnt/sys
# mount --rbind /proc /mnt/proc
# chroot /mnt (possibly 'chroot /mnt bash' to specify shell)
At this point, this is more or less like mount /dev/sdaN / - if you try this here, it will say already mounted
# grub-install /dev/sdaN - this will udpate/install/setup grub on the imaged drive - watch for success or errors to determine situation / what else may be needed. If blocklist errors, you may need to review partitions on drive, or force(?)

If you drop into the grub> menu, there are various options, and may be worth learning what is possible within grub> environment. You can try the grub options below (root, setup, reboot). You can also do the following steps to specify the kernel/initrd options: kernel /boot/vmlinuz-somedistro-vv-vv-vv-plat (this should load kernel/indicate something)
initrd /boot/initrd-somedistro-vv-vv-vv-plat (this should load and indicate something)

boot - will boot the kernel/initram environment specified.

If the above doesn't work or gives you errors, see if you can determine what is missing. It could be root drive, or file system type - here are some options:
kernel /boot/vmlinuz-somedistro-vv-vv-vv-plat root=/dev/sda1 (where sda1 is correct boot partition)
initrd /boot/initrd-somedistro-vv-vv-vv-plat rootfstype=ext4 (where ext4 is correct file system type for boot drive)

If you do boot, and all seems well, how to fix grub so system is back to normal? Try these steps as superuser:

# update-grub
# grub-install /dev/sda (note sda and not sda1 to install boot loader on drive)

This is another option with grub prompt (some older grub versions had a grub command), boot a live linux disk or 1st disk of installation, get to a rescue (or root) prompt - use these commands grub - gets you to grub> prompt

grub> find /boot/grub/stage1
indicates drive/partition grub stage1 is on, e.g. (hd0,1), or (hd0,0) Select root for grub operations:

grub>root (hd0,1)
Then setup this partition for grub booting:

grub> setup (hd0,1)
A list of "successful" operations should be shown.

Reboot the system.

When cloning a SUSE 9.3 workstation onto a different size drive, this will fail at GRUB - boot Knoppix (or rescue disc), mount /dev/sda1 to /boot, mount /dev/sda3 /mnt/sda3 then run grub --batch --device-map=/boot/grub/device.map < /mnt/sda3/etc/grub.conf

Then for good measure, run grub and to the above steps (root (hd0,1), setup(hd0,1), then reboot

Original SUSE notes:

grub --batch --device-map=/boot/grub/device.map < /etc/grub.conf
or edit /usr/sbin/grub-install script, and change (line 30):

Boot from USB Drive, USB Version of USB Drive

Boot from USB Flash Drive/USB Stick/USB Thumb Drive/USB Jump Drive/USB Pen Drive

If you have never ventured into your system's configuration, do not know what BIOS is or what it does, and need help on getting your computer to boot from USB, please read this Blog post before reading any of the following: First Steps - How To Boot from USB

Most computers that support USB have some option to boot from a USB device, but because the USB interface supports many different types of hardware, external discs, etc. there may need to be special configuration options so you can boot from a Linux distro delivered on a USB Flash Drive.

Best option To boot from a USB Flash Drive, check with Computer manufacturer, or motherboard/BIOS manufacturer for correct configuration.

Here are other helpful hints culled from working with different computers/bioses:

You may need to Enable Booting, e.g. USB Boot, Boot from USB, Boot USB Device First, Boot Mode set to Legacy+UEFI (MSI)

You may need to select boot USB device as USB-ZIP or USB-CDROM or USB-HDD

You may need to enable Legacy handling / Legacy Function, Storage Function (may be in peripherals or USB Configuration, i.e. a different part of BIOS) For newer Ubuntu versions, Legacy USB may work for some options and may not if system has UEFI - you may need to try several variations to find what works for particular system

You may need to enable CSM, Compatibility Support Module, (MSI) Windows OS set to CSM

You may need to select the drive as a Hard drive (Award bioses, F12 to select boot device, Select "+ Hard Disk", then select & then Select USB HDD device), or Fixed Disk Order / USB Hard Disk (MSI)

You may need to select it as a Hard Drive within BIOS to boot first (device listed before internal hard drive) (American Megatrends)

Here is a specific outline from a customer for an American Megatrends BIOS: Had to disable internal hard drive. The External Device was enabled and the Network Boot disabled. The three boot options were 1. external device, 2. optical drive, 3. network boot. The BIOS reported “No OS Found” with a blank screen. Press Ctrl - Alt - Delete, then booted to USB menu.

You may need to change priority of boot devices (Hard Disk Boot Priority)

You may need to update your BIOS to allow USB booting

If presented with option for a specific partition or the device itself, select the device

On MSI Motherboards, the Settings-Boot-USB Hard Disk BBS Priorities, Boot Option 1 should be Partition 1 on the USB

You may need to select the EFI (UEFI) bootable partition

You may need to select correct (or safe) video option if video card/monitor not supported by particular distro (or enter command line options for video mode or option)

You may need to edit grub menu and add root=/dev/sdb1, or as appropriate device/partition booting from. Typically this is done with 'e' to edit, for the linux= line add the new item root=/dev/sdb1, then F10 to boot. Note you may be able to permanently update this (depending on grub menu configuration on bootable USB).

You may need to learn specific steps needed for your particular BIOS/System.

Realize it may not be possible for a specific computer to boot from USB - this is a function of the system hardware & BIOS, not the USB device.

Understanding what a bootable USB Flash Drive is

In general, the ISO that can generate the disc (CD or DVD) is copied to the USB Flash Drive directly, so the computer BIOS must be able to handle/read this structured layout from the USB device and have the ability to "Boot" from this ISO data. In general, if you are already running an operating system, and insert the USB drive, you should be able to see files, etc. The mount type will most likely be iso9660 (standard CD/DVD iso data layout), and may also have other file systems (which may be loaded separately).

The USB Collection USB drives use an UEFI compatible version of GRUB2 and present a Boot menu, where you can select which option to boot. This is distinctly different from the single ISO bootable USB options (these use the ISO format and booting selection (typically isolinux), while the GRUB version uses GRUB options for USB devices. You can configure via GRUB and can also modify the grub.cfg to set default/timeout options (e.g. [USB_Boot]/boot/grub/grub.cfg) - here are example entries (default position starts at 0, no timeout setting or timeout=-1 mean wait forever):
set default="3"
set timeout=10

UEFI General notes / Booting recent Ubuntu family USBs specifically (2020+)

As more & more systems phase in with UEFI (Unified Extensible Firmware Interface) vs. the older BIOS (Basic Input/Output System), distros are making the assumption this is what you are booting on. Most current UEFI (sometimes referred to as UEFI BIOS) based systems are geared towards secure boot (for example, Windows 11 requires this). This can be referred to as TPM (Trusted Platform Module) and there is also CSM (Compatibility Support Module) that may need to be enabled (or modified to allow booting from a generic bootable USB). In simple terms, if the system's authentication settings are configured for secure boot, then only a properly authenticated operating system will boot on the specific PC or Laptop. So if you try to boot a LinuxCollections.com USB, it may fail because the UEFI based BIOS is configured specifically to secure the system and prevent this. Because there are different manufacturers that take different approaches, and the whole concept of UEFI (the Extensible part) means the manufacturer can configure/setup a system for various uses, it is difficult to state a definitive solution that applies to all systems. So as a general approach, you will want to access the UEFI menus/BIOS/system options prior to booting the operating system - often this is done with pressing the Delete key or F1 or F2 or F10 or Esc right after powering on the system, then finding the area in Boot, or Advanced, or TPM/CSM settings to disable secure boot or enable the compatibility mode that allows non-UEFI devices. The strongest recommendation is that you should check with the manufacturer for settings that are needed to enable booting from a generic USB.

The most recent releases of the Ubuntu family (since 2020) exhibit strange operation when booting from non UEFI systems or with UEFI systems booting in a compability/legacy mode. These can take as long as 10 minutes to get to the desktop(!) We have not tested this aspect, but it also may expect a live internet connection. Because all our test units do not have automatic internet connections, we wonder if some of the delay could be related to this, as it makes no sense that a boot process should take this long. Meanwhile, if you boot with the CSM disabled, it can get to startup screen background much quicker, although there is a lot of waiting before the Try/Install options (timed out waiting for internet?).

Boot Options

Many distros have methods of adding boot options to the default boot method. Some provide a rescue option, some have this available, but not necessarily obvious. If there are boot problems (hang/error message/etc.), you should check with the distro source for any notes, forums, suggestions, etc. Some common options are:
These can be added to the boot options to disable certain functions that may be problematic on a specific system. Again, the distro source is the best place to find a booting solution, but these are options that can be tried to see if it alleviates the issue.

Booting USB on a HP ProBook

As of 2022, a customer working with the 256GB USB Debian Developer Complete Collection found that when booting in Legacy mode it would fall into a limited "grub rescue" prompt. There appears to be some incompatibility with the version of grub used and the HP BIOS. It turns out that the 128GB USB Debian Complete Collection (smaller USB/no source ISOs) would boot and work correctly. The only major differences are the size of the USB, and subsequently the size of the boot partition (USB_Boot) where grub is located, so it seems like either the BIOS or GRUB is running into difficulties handling this 256GB USB drive. This has been the only known instance of this situation.

Booting USB on a MacBook

This is an anecdote from a customer that bought the Debian USB Complete Collection: "Just to follow up, thought you might be interested to know that I've investigated the rEFInd boot manager for my Macbook, and using that, I've finally gotten the USB stick to work! " Note that Apple's Macs have used different processors - this would be an Intel 64-bit processor in a Macbook during the time Apple transitioned to Intel - now thay are transitioning to their own processor. Please be aware of the actual processor in your system when choosing a Linux platform.

Advantages of USB Flash Drive

Speed - once you can get a system to boot from the USB Flash Drive, you typically will be in exactly the same place as if you had booted from a disc. However, the USB Flash Drive will be much faster than optical media - you will be able to run Live Linux distros or do a normal install. If the system does not have an optical drive, or there is a problem with the optical drive, a USB boot option may be extremely helpful. The only disadvantage would be the time (& sometimes exasperation) of trying to get a system to boot from USB (if this situation even applies).

USB Version of USB Drive

To find the USB version of a particular USB drive (e.g. USB 2.0 or USB 3.0, etc.), you can use lsusb. Here are the steps:

1) With the USB drive inserted, at a terminal, type lsusb[Enter]. This will list USB Bus/Device info, e.g.
Bus 002 Device 004: ID abcd:1234 Unknown
Bus 002 Device 005: ID 058f:6387 Alcor Micro Corp. Flash Drive

Note you can also see some items using dmesg. The last device inserted will have a line like this:
usb-storage: device found at 5

2) Identify the particular device - if unsure, use lsusb with a remove/insert approach to identify the device in motion. Now you can use the Bus/Device to get details, e.g. lsusb -v -s 2:5[Enter] -v = verbosity, -s [bus]:[device] to identify which USB to list (with verbose info).

3) The item that shows the USB version is bcdUSB. You can use grep to isolate this line, e.g.:
lsusb -v -s 2:5 | grep bcdUSB[Enter]
bcdUSB 2.00

fdisk notes

fdisk is a partition modifier - it lets you create, modify, and delete partitions on a hard drive. IMPORTANT: You really want to understand what you are doing before you play around on an existing hard disk. Learn on a new hard drive to get some experience!

fdisk /dev/hda opens the partition of the first hard disk. Typically IDE hard drives / CD/DVD drives are hda, hdb, hdc, hdd. /dev/hda1 is the first partition on /dev/had, /dev/hda2 is second partition, etc. Typically sata & usb drives are /dev/sda, /dev/sdb, etc.
p - prints the current partition table
n - creates new partition
t - sets type (L lists types)
d - d deletes a partition
w - writes partition table to disk

fsck notes

fsck is the file system check tool. Learning about system maintenance is important, and not covered here. Most systems have a counter, and will force a check after every so many boot-ups or days. Generally everything is fine, until a forced check at boot up finds some problem, and halts the boot up process.

When this happens, here is a quick run-down on how to fix it, IF there isn't a serious problem with your drive, hardware, or system.

  • Review the boot messages and screen to determine which drive (partition) was being checked. In this example it is hda4.
  • If prompted, use your root password to log on into a maintenance mode. If not, restart as a single-user, or in rescue mode.
  • At the root prompt, type fsck /dev/hda4 (where the device is the device that failed the fsck)
  • Answer any questions, see what the problem is. If its serious, and this doesn't help you resolve it, refer to man fsck, fix the hardware/system problem, or start looking for your backups.

Also note that fsck is often a front end to a more specific version, e.g. fsck.ext3, fsck.ext4, fsck.msdos, fsck.vfat, etc. a specific version that knows about the specific file system a drive is formatted with.

There are a few other utilities to be aware of:
(All should be run as root/superuser or with sudo and /dev/sda or /dev/sda1 should be replaced with your specific device)
hdparm - displays information about specified drive, e.g. hdparm /dev/sda
badblocks - runs and indicates any bad blocks on disc, e.g. badblocks -s -v /dev/sda

Example (for an ext? formatted drive) to mark bad blocks (say after formatting an old drive to re-use), you can use:
umount /dev/sda1
badblocks -v -s /dev/sda1 > badblocks.log
e2fsck -l badblock.log /dev/sda1

file basics - cp, mv, rm, rename, basename, dirname, rsync

To Copy a file, use cp ExistingFileName NewFileName[Enter]
To Move or Rename a single file, use mv ExistingFileName NewFileName[Enter]
To rename multiple files (on Linux) use rename ExistingNamePart NewNamePart Files, e.g. rename .c .cpp *.c[Enter]
For more on rename man rename


Copy foo.txt to foo.bar:
cp foo.txt foo.bar[Enter]

Copy foo.txt to foo.bar with folders:
cp /tmp/foo.txt /home/user/Desktop/foo.bar[Enter]

Copy folder and all its files:
cp -R /home/groupleader/data /home/user/Desktop/data[Enter]

Remove a file:
rm foo.bar[Enter]
rm /home/user/file.txt[Enter]

Script to remove all hidden mp3 files:
#script for root of flash drive with hidden mp3 files (copied from an Apple)
find ./ -name ".*.mp3" | while read item;
echo "$item"
rm "$item"

Move (Rename single file) foo.txt to foo.bar:
mv foo.txt foo.bar[Enter]

Rename all .txt files to .c files:
rename .txt .c *.txt[Enter]
(Newer versions may require expression, e.g
Rename all .txt files to .c files:
rename 's/.txt/.c/' *.txt[Enter]

Change case of extension - .jpg files to .JPG files:
rename 'y/jpg/JPG/' *.jpg[Enter]

Change case of files - lower to upper:
rename 'y/[a-z]/[A-Z]/' *.fil[Enter]

Do something fancy with a bunch of files (bash script) - prepend header.c to text files and rename to .c using base file name:
vim script.sh[Enter] -or- nano script.sh[Enter]
#add header & rename files
for item in `ls *.txt`
  #fancy bash syntax to get filename without extension
  cat header.c $item > $newitem".c"
## Note if you have long file names with spaces, the for loop also marches through parts of the file name. Since this isn't what you want, you can do this:
ls *.txt | while read item;
  #use sed to strip out file extension
  newitem=`echo "$lineitem" | sed -e 's/.txt//'`
  cat header.c $item > $newitem".c"
[Save file]

Change script to be executable:
chmod 755 script.sh[Enter]
-or- chmod +x script.sh[Enter]
Then run the script:

Other tools - basename, dirname
Strip off directory info:
for item in `find ./ -name *.txt`
  directory=`dirname $item`
  filename=`basename $item`
  echo $directory
  echo $filename

This is bash syntax to strip off directory, and then get base file name:
(where $item is a /long/path/stuff/we/do/not/want/somefile.ext)

Single line for loop in bash:
for item in `ls`; do echo $item; md5sum $item >> md5sums.txt; done
(This will list all items in the current folder, echo its name, and create an md5sum listed in md5sums.txt)

rsync is "remote synchronize", a way to duplicate files/folders locally or across a network. There are many options, but a common usage is to duplicate a folder from one device to another. The simplest command is the -a option (archive, which copies files/sub folders):
rsync -a /path/to/source /path/to/destination
You can also use rsync -av for verbose / details while running the rsync command.

Bash Scripts (various)

This includes some from the File Basics, but has other example scripts that do specific things. This is a collection of useful scripts that do specific things and show off certain common structures (loops/conditionals/etc.) so there is a good chance one of these will be a good starting point for the script you want...

myfirstscript.sh text from Introduction to Bash Scripts video


echo "Hello World!"

#for loop

ls Pictures/

for pic in `ls Pictures/`
echo "Pictures/"$pic
echo "${pic%.*}"
mv Pictures/$pic Pictures/$newpic.JPG

exit 0

echo "Num Parameters: "$#

#Conditional (if)

if [[ $# < 2 ]];then
echo "Need at least 2 Parameters!"
exit 1

echo $1
echo $2
echo $3

exit 0

#Example Variable

echo "Today is "$THEDATE

#Variable counting

echo $COUNT


echo $COUNT


echo $COUNT


echo $COUNT

Here are some logical constructs as a quick reference

# This checks number of parameters provided - if none (0), exit
if [ $# -eq 0 ]; then
   echo "No arguments supplied'
   exit 1

# This checks to see if parameter 1 has been passed in or not (zero string check)
if [ -z "$1" ]; then
   echo 'No argument supplied'
   exit 1

Some useful logical checks for Bash scripts (Note: there are more)

[ -a FILE ] True if FILE exists.
[ -d FILE ] True if FILE exists and is a directory.
[ -e FILE ] True if FILE exists.
[ -f FILE ] True if FILE exists and is a regular file.
[ -r FILE ] True if FILE exists and is readable.
[ -s FILE ] True if FILE exists and has a size greater than zero.
[ -w FILE ] True if FILE exists and is writable.
[ -x FILE ] True if FILE exists and is executable.
[ -L FILE ] True if FILE exists and is a symbolic link.
[ -z STRING ] True of the length if "STRING" is zero.
[ -n STRING ] True if the length of "STRING" is non-zero.
[ STRING1 == STRING2 ] True if the strings are equal.
[ STRING1 != STRING2 ] True if the strings are not equal.

Some string examples:
if [ "$1" == "somefile.ext" ]; then
   echo 'Ready to process somefile.ext'

if [ "$1" != "somefile.ext" ]; then
   echo 'Incorrect file name provided'
   exit 1

Convert avi file to mp4 file with ffmpeg

#This checks to see if a parameter has been passed in or not (zero string check)
if [ -z "$1" ]; then
  echo "Please provide avi file name!"
  exit 1
#This verifies the file exists, otherwise indicates error
if [ -f "$1" ]; then
  #OK, file passed in and it exists - force to lower case for basename strip
  #send file name to tr and force upper to lower
  lcfile=`echo $1 | tr '[:upper:]' '[:lower:]'`
  #now use basename utility to strip off .avi suffix
  newfile=`basename $lcfile .avi`
  #and put .mp4 extension for new file
  echo "Will convert $1 to $newfile"
  #this is ffmpeg convert to take input file (.avi) and convert to h.264 mp4 file
  ffmpeg -i $1 -vcodec libx264 -pix_fmt yuv420p $newfile
  #file exists check fails, report as error
  echo "$1 file does not exist!"
  exit 1

#leave script with success code
exit 0

prepend header.c to text files and rename to .c using base file name
#add header & rename files
for item in `ls *.txt`
  #fancy bash syntax to get filename without extension
  cat header.c $item > $newitem".c"
## Note if you have long file names with spaces, the for loop also marches through parts of the file name. Since this isn't what you want, you can do this:
ls *.txt | while read item;
  #use sed to strip out file extension
  newitem=`echo "$lineitem" | sed -e 's/.txt//'`
  cat header.c $item > $newitem".c"

Obtain lines from a long list of name/e-mail address with a subset list of only e-mail addresses
#pull out name/e-mail address from failed e-mail addresses for resend
for item in `cat subsetlist.txt`
  #echo $item
  cat fulllist.txt | grep $item >> newlist.txt

Drop signature image PNG onto PDF file - this merges / places a signature image file onto a PDF

### This merges PDFs using pdftk background
### Uses graphicsmagick and pdftk
### Created transparent 8-1/2x11 page png in gimp to drop sig.png onto

#first, place sig on page at appropriate place

#if you use ! you force resize as specified (if sig.png needs to scale)
#position as needed with +xpos+ypos
#gm composite -geometry 352x62!+400+2200 sig.png transpage.png out.png
#gm composite -geometry 176x31+400+2200 sig.png transpage.png out.png
gm composite -geometry 452x92!+440+2710 sig.png transpage.png out.png

#second, convert PNG to PDF
gm convert out.png signed.pdf
rm out.png

#third, merge PDFs
pdftk needsigform.pdf background signed.pdf output signedform.pdf

#Now signedform is PDF with dropped in signature!

##for 3 page file, wanting sig only on page 1 use
##split into pages using pdftk
#pdftk ThreePageForm.pdf burst
##update pg_0001.pdf to signedform.pdf (script above, i.e. step 3)
##concatenate pages pack to single form
#pdftk signedform.pdf pg_0002.pdf pg_0003.pdf cat output SignedThreePageForm.pdf

Compare 2 directory trees by files (file name only - not size/contents)

if [ "$#" != "2" ]; then
  echo "Please enter 2 directories, e.g. compdirs.sh /some/path/one/ /compared/to/path/two/[Enter]"
  exit 0

if [[ "$1" != */ ]]; then

if [[ "$2" != */ ]]; then

#fastest way to get slash count - echo string through tr to echo only slashes and new line
#use awk for count
slashes1=`echo $dir1 | tr -d -c '/\n' | awk '{ print length }'`
slashes2=`echo $dir2 | tr -d -c '/\n' | awk '{ print length }'`

#cut will need to be increased by 1 for correct column
let cut1=$slashes1+1
let cut2=$slashes2+1

if [ -d "$dir1" ]; then
  echo "Listing files for $dir1..."
  #use find to list files, cut removes leading directory, sort for drive/file system list differences
  find $dir1 -type f | cut -d/ -f$cut1- | sort > compdir1.txt
  if [ -d "$dir2" ]; then
   echo "Listing files for $dir2..."
   find $dir2 -type f | cut -d/ -f$cut2- | sort > compdir2.txt
   echo "Comparing lists..."
   diff compdir1.txt compdir2.txt > compdirs.txt
   rm compdir1.txt
   rm compdir2.txt
   if [ -s compdirs.txt ]; then
    echo "Full listing in compdirs.txt:"
    echo "'<' $dir1..."
    echo "'>' $dir2..."
    head -n 12 compdirs.txt
    rm compdirs.txt
    echo "No Differences! Directory files / file names all match! (size / data NOT checked)"
    exit 0
   echo "$dir2 is NOT a directory!"
  echo "$dir1 is NOT a directory!"
  exit 1

Some integer representations for Pi

22 / 73.1429
355 / 1133.14159292

Network File Basics (ftp/scp/wget/smbclient)

This is a general list for transferring files between 2 networked computers, with the basic approaches outlined below - if looking for other approaches or need something else, you can dig into this list: ssh, sshfs, scp, ftp, sftp, nfs, samba (smbclient), giver, wget, nc.

FTP (File Transfer Protocol) is a simple, straightforward way of getting at files, and is perfect & easy for public / non-secure file transfers. It is easy, there are numerous approaches one can use, it is ubiquitous and still commonly used. A working FTP server is assumed for these notes.

SCP is a Secure cp (secure copy), e.g. scp - it does require a ssh server and configured client (i.e. with ssh/scp software available). There are numerous security/password/configuration issues if used. It is assumed you have a server with ssh setup and just want to copy files with scp.

wget (Web get) is a command line utility that can get a web page or pages, links, etc. Uses the http protocol, but can also be used to get ftp based files.

smbclient is a command line utility for accessing a Samba (SMB) server or Windows based servers. Samba servers are typically used with Linux boxes on a Microsoft Windows network - allowing Windows to access the Linux system.
FTP file access from command prompt (pretty much the same in Windows/Mac/etc.)

Sample FTP session to get file example.txt at ftp://someftpserver.com - as anonymous, with name as anonymous, blank (empty) password

ftp> open someftpserver.com
ftp> name: anonymous
ftp> password:
ftp> get example.txt
ftp> disc
ftp> quit

You can use ls to list files, cd to change directories.
Use open to open a connection, disconnect (or just disc) to close/end a connection.
To retrieve a file, use get. To send a file, use put
To retrieve multiple files, use mget. To send multiple files, use mput
Other commands - prompt toggles prompts for mget. help lists all ftp commands. To delete a file, use delete

Sample FTP session to get all files in sample folder at ftp://someftpserver.com

ftp> open someftpserver.com
ftp> name: anonymous
ftp> password:
ftp> cd sample
ftp> prompt
ftp> mget *
ftp> disc
ftp> quit

scp Examples

Retrieve myfile.zip from someserver.com
scp user@someserver.com:myfile.zip myfile.zip
Send myfile.zip to someserver.com
scp myfile.zip user@someserver.com:myfile.zip

Retrieve myfile.zip from someserver.com with custom port 1234
scp -P 1234 user@someserver.com:myfile.zip myfile.zip
Send myfile.zip to someserver.com with custom port 1234
scp -P 1234 myfile.zip user@someserver.com:myfile.zip

wget Examples

Get example.txt from FTP server (same as FTP session above!)
wget ftp://someftpserver.com/example.txt

Get index.htm from Web server
wget http://www.somewebserver.com/index.htm

Get a whole website from Web server (use with care!)
wget -m http://www.somewebserver.com

smbclient Example

smbclient //smbserver/sharedfolder
smbclient //

If security is not an issue (i.e. on a private local network), this is a simple way to use the nc (netcat) utility to manage the transfer.
On the machine that has the source file(s) (all in the same folder, current working directory), use this command:
tar -cz . | nc -q 10 -l -p 45454
On the machine that will receive the files at the current working directory (where $REMOTE_HOST is IP address of source file machine):
nc -w 10 $REMOTE_HOST 45454 | tar -xz
e.g. nc -w 10 45454 | tar -xz

Quick explanation of commands (see manual pages for more details): On the source host, tar (Tape Archive) will create & compress (-cz) the file data, and pipe to the nc command, which will use port 45454 (-p), will listen (-l) and quit 10 seconds after end of data (EOF) (-q). On the receiving system, nc will connect to the entered IP address / port 45454 and received data will be piped to tar, which will extract and uncompress (-xz). After data stops, nc will end (-w).

Network Basics (route, ifconfig, netstat, etc.)

The various details and meanings of these can span volumes. The basic issue is you need a network interface (i.e. eth0 (ethernet 0)) that can connect to other systems. Your local network is often,, or You can connect to the internet through a gateway (router or cable/dsl modem).
As an overview of what needs to happen, you can refer to this checklist (as user root):
- Your system needs a network interface (ifconfig)
- Your interface needs address & routing information (/etc/network/interfaces) "man interfaces"
- The interface needs to be up (ifup eth0) "man ifup"
- You need a physical connection to router/modem/local network
For internet access:
- Your network needs a connection to the internet (cable/dsl modem/router/gateway)
- Your system needs name server addresses to look up URL/human names, e.g. linuxcollections.com /etc/resolv.conf
To see network information on your system, as root, run ifconfig - this shows details on your network hardware, i.e. network interfaces (InterFace CONFIG)
You can use netstat -rn to view the same info (run as root) as route -n. This outlines details on your network and pathways to other networks (e.g. Gateway, destinations, etc.)
As root, you can use route add and route delete to manage options - this usage requires knowledge and experience to manipulate your network configuration. Typically you are better off using system supplied tools to configure IP and gateway addresses.
Once you have a path to the internet, you need a way to resolve names to IP addresses. This is the Domain Name System (or DNS), and can be referred to as DNS, NS, or name servers. Again, system tools should be used, but most systems support name servers in /etc/resolv.conf - use "man resolv.conf" for details on this file. There is also a /etc/hosts that can store known hosts/names/address information - use "man hosts" for details on this file.
A quick & simple way to check for internet access is to ping a website, e.g ping linuxcollections.com. You can also use traceroute for more detailed information.

The most common internet issues are:
- You don't have a working internet connection, or the internet is down at the provider (test with ping)
- You don't have good name server (DNS) addresses (test with ping)
- You have firewall issues (ping local, gateway, then internet addresses)
- You have routing issues (ping gateway address)

Web Basics (dns/html/url encoding)

When you enter a website into your browser, a whole sequence of events occur. This walks through the main steps for a simple website.

  1. You enter a website, e.g. example.com
  2. Your system queries a nameserver with this human readable name. The nameserver is found based on your network configuration (this is the DNS or Domain Name Server system). Your system can be configured manually or by DHCP (Domain Host Configuration Protocol).
  3. The nameserver returns an IP address based on the human readable website.
  4. The URL is sent to this returned IP address on port 80 - the standard HTTP (HyperText Transfer Protocol) port
  5. The web server serves up the page and sends back a well formed response based on HTTP. The data portion of this is typically HTML (HyperText Markup Language) based page data.
  6. Your browser takes this HTML based data and layouts the page. Portions of HTML are NOT displayed, as they are markup items and indicate different things.

Each step of this process covers a wide range of activities that have been built up over decades, and the details of each step can be extremely complicated. The above is a simplistic explanation to cover the basics of how entering a website results in you seeing a page on your browser.

URL Encoding

The following is a basic reference of allowed characters in a URL:

There are also "Escaped" characters which use a % character followed by 2 characters using hexadecimal.

A common example is the space character which would be encoded %20

The following table covers just 1 character that is NOT a standard URL character - so if you type this character, it will be encoded in this escaped format. As an example, http://www.example.com/Rückzug would be sent as http://www.example.com/R%C3%Bcckzug in UTF-8
Example CharacterWindows-1252 encodingUTF-8 encoding

Process Basics

Unless you are a programmer, you often will just install or add packages from repositories, and won't need to do much more. However, if you are migrating systems, copying files, or other tasks, you may run into compatibility issues or need more details on running a program.


A file that can run as a process is typically compiled from source code. If you have the source code available (or can obtain it), you can rebuild the executable file by building (or "make") the process executable on the system you want to run it on. This ensures that it is built with the correct processor architecture and system/kernel configuration. If this is not possible, there are various things that can be done if there is a required library missing, or there are compatibility issues. A library often has a .so extension (shared object code), and is typically pre-built compiled code saved in a library file which is then available for any program running on the system. Object Code is code compiled for the processor architecture, but not linked together with system or startup code to make it a stand-alone executable. It is an interim step as part of the make process (or compile/link steps in creating an executable or library file). Typically source code is pre-processed, compiled, then linked. The pre-processor replaces human readable source code with more computer oriented code. The compiler creates object code, or computer instructions organized in particular ways. The linker puts together object code in the proper format for a stand-alone, runnable executable process file, or into a library file.

If you are trying to build from source, always look for a README.txt or CONFIG.TXT or other available information which will outline steps needed to build the source and other requirements. It is recommended that you look at all or at least most of the files in the main folder to ensure you find any important information. Often a C or C++ program will require make config and make commands to build the program. Sometimes a make install will install the program, but may require root/super user access.

If you are trying to run a program and it reports errors or won't run, and complains about versions or libraries, there are a few things to check. You can use ldd to list dependencies, and this may help show what is needed. Sometimes installing the missing components is all that required. For even more details info, you can use readelf -V [binary], e.g. readelf -V myprogram[Enter]. If there are library version issues, sometimes symbolic links can be used. For example, you may have a version of libpng, but the program/executable is looking for a generic entry in this example, libpng.so vs. the detailed version listing - to create the generic symbolic link file, use "ln -s /usr/lib/libpng.so. /usr/lib/libpng.so[Enter]" (root/superuser required)). Sometimes there may be multiple versions available, and the main / generic symbolic link file may be pointing to the newer (or older) version - you can use the same approach to point to the correct file. Note that this may "break" other programs in the system, so be aware of any changes made to external files/libraries/configuration files.

System Information

Various ways to get OS information in Linux

At the command line, there are various ways to get OS / Distribution information

You are pretty much guaranteed to have this available. Use uname -a to view all available system information that uname can provide. Depending on the system, the kernel may have distribution info available, but it may not. You should be able to find the processor, the kernel, OS type, etc., but this may not have distribution details that are useful.

As more and more linux distributions adhere to the Linux Standards Base, this should provide the distribution information. Use lsb_release -a to view details on the running distribution release.

You can find various release information in the /etc folder. Use ls /etc/*release* to see what is available - here are some common distribution files:

  • /etc/os-release
  • /etc/lsb-release
  • /etc/redhat-release
  • /etc/centos-release
  • /etc/fedora-release
  • /etc/system-release
  • /etc/issue
Use cat /etc/os-release (for example) to see contents of file.

systemctl / systemd systems
For systems that moved to systemd (vs. old/original Unix based init), here are a few helpful commands & equivalents:

systemctl rebootinit 6
systemctl poweroffinit 0
systemctl start [service]/etc/init.d/[service] start
systemctl stop [service]/etc/init.d/[service] stop

"ls" commands
There are several "lists" that can useful (packages may need to be installed, e.g. pciutils, usbutils, util-linux (or distribution specific)):
ls - list file system contents (many options, "man ls" for help)
Example: ls -lrt = list current directory files with details in reverse time order (most recent file at bottom)

lspci - List PCI Bus Devices
Example: lspci -v = list PCI bus devices with details (verbose)

lsusb - List USB buses and device details
Example: lsusb -v = list USB bus and device details (verbose)

lsblk - List block devices (storage devices)
Example: lsblk -l -o +UUID = list all block devices with their UUID (unique identifier)
Note: There is also a blkid command that will echo the UUID

Other options/info
Use cat /proc/version for other version information.
Use cat /proc/cpuinfo for processor details / information.
Use dmesg | grep "Linux" for Linux details.
Use yum info nano shows details on nano and where it was sourced (Redhat family).
Use yum repolist shows repositories, typically connected to distro (Redhat family).
Use dnf info nano shows package info on nano (Redhat family).
Use rpm -q nano shows package info on nano (Redhat family).
Use apt-cache policy nano for package info on nano (Debian family)
Use dpkg -l nano for package info on nano (Debian family)
Use cat /etc/apt/sources.list for apt package sources (repositories) (Debian family)

utility basics - man, info, file, ps2pdf, pdftk, mutool, fromdos, iconv, tr & UTF notes

Command reference, system manual, info pages, conversion utilities

man = manual pages command, used to look up reference & details on system commands, system calls, etc., anything documented
man grep[Enter] displays the options and overview of the grep command
man sed[Enter] displays the options and overview of the sed command

info = info pages - the GNU update/improvement to man - man is the original UNIX command for the manual, info is a GNU system preferred replacement. In practice, often man and info essentially display the same things, however (dependent on the system and command), there may be differences.

file = file details - this shows off details of executables, scripts, text files, etc. If you aren't sure what a file is/does, start with file unknown.fil

Using the ps2pdf utility, you can easily create PDF documents for specific commands - which may make it easier to print/save/view:

ps2pdf = outputs postscript to PDF format
man -t grep | ps2pdf - grep.pdf[Enter] outputs the man page to postscript (-t = default formatter groff for postscript) which is then piped into the ps2pdf utility (postscript to pdf) using stdin as input (-) and the output file specified (here, grep.pdf)

pdftk = formats/manipulates PDF files

Split a multi-page PDF into individual pages
pdftk multipage.pdf burst - splits multiple pages into pg_0001.pdf, pg_0002.pdf, pg_0003.pdf, etc.
pdftk pg_0002.pdf pg_0004.pdf cat output newmultipage.pdf - concatenates individual pages into a multipage pdf
Also can merge, rotate, watermark, etc. See Batch Scripts for merging pdf example

To turn a text file into a PDF on the command line, you can use the enscript utility (apt-get install enscript[Enter]):
enscript -p text.ps text.txt
ps2pdf text.ps text.pdf

mutool = pdf utilty for inspection and manipulation

This can do clean (rewrite), extract (font/image resources), info (details), poster (split into tiles), show (pdf details). Poster is useful to split a page in half (or smaller).

Here are steps how to take the top from 1 pdf and the bottom from another pdf and merge into single page pdf:
Step 1: make single file 2 page PDF: pdftk pdf1.pdf pdf2.pdf cat output twopage.pdf[Enter]
Step 2: split into half pages horizontally: mutool poster -y 2 twopage.pdf split4page.pdf[Enter]
Step 3: create individual files with each half page: pdftk split4page.pdf burst[Enter]
Step 4: Merge top from original page 1 with bottom from original page 2: pdfjam pg_0001.pdf pg_0004.pdf --nup 1x2 --outfile merged.pdf[Enter]
Now you will have merged.pdf as a single page pdf with top/bottom together. This can be useful if you scan the front and back of an ID card or have documents that need to be a single page vs. 2 pages.

fromdos - converts DOS/Windows based text files into Unix files, e.g. CR-LF to LF line delimiters. (apt-get install tofrodos)

todos - converst Unix files to DOS/Windows text files, e.g. LF to CR-LF line delimiters

Notes: CR is carriage return, ASCII control character 13, used to return text caret to beginning of line, LF is Line Feed, ASCII control character 10, used to move down a line. For certain older printers, the dual CR-LF was required so the next line would start at the beginning (vs. immediately below the last character of the previous line). Certain Web technologies chose CR-LF, while Unix went with the single LF character. However, for certain uses, the assumption had to be made that LF also meant go to the beginning of the line, to save lots of bytes in large files (not needing the extra CR byte).

flip in older debian, you can use flip -u filename.txt to go to Unix text (LF) and flip -m filename.txt to go to DOS text (CRLF)

iconv -f UTF-16 -t ASCII//TRANSLIT//IGNORE unicode.txt > ascii.txt - iconv will convert between various formats - see man iconv - the above can be used to convert Windows Unicode Text files to something a bit more useful in Linux

To use iconv do to multiple files, here is an example of converting Unicode *.TXT files to Ascii *.txt files: for f in *.TXT; do iconv -f UTF-16 -t ASCII//TRANSLIT//IGNORE $f > $f.txt; done

tr - translate e.g. upper to lower case: tr '[:upper:]' '[:lower:]' < input.txt > output.txt
or getting a guid all uppercase: cat /proc/sys/kernel/random/uuid | tr '[:lower:]' '[:upper:]'

UTF-8 / UTF-16 notes:
UTF = Unicode (or Universal Coded Character Set) Transformation Format, and it can be in byte format (8) or word/double byte format (16) (also UTF-32, UTF-7, UTF-1, but 8/16 most common in normal usage). Some UTF encoded files have a BOM (Byte Order Mark) which indicates the byte order (high/low or low/high or "endian") In UTF-8 the characters (first 3 bytes) are 0xEF, 0xBB, 0xBF, while in UTF-16 it is 0xFEFF - if it appears as 0xFFFE, then it is little-endian (vs. big-endian). Because UTF-8 matches ANSI except for embedded encoding, the document may appear with 3 strange characters at the very front. To remove the BOM you can use sed or awk or other utilities. If the vim (VI Improved) is availale, you can edit the file, use : then "set nobomb", and then save the file.

utility basics - sed, grep

Lots of capabilities, these are just some useful basics

sed = stream editor. Conceptually this streams text through a filter, and you can make changes or perform operations on the text as it goes by. Most common usage would be to search and replace, either first occurrence, or globally.
Take a file and replace first occurrence of some text ("OldValue") with something else ("NewValue"):
cat myfile.txt | sed -e 's/OldValue/NewValue/g' > mynewfile.txt[Enter]
Explanation: cat outputs myfile.txt to standard output, which is pipelined (filtered) through sed. The -e (expression, or 'edit script') option says use the script command given to perform your operation. The s///g syntax says search for the first value (between slashes), and replace with the second value. Output is modified text. This is redirected to a new file called mynewfile.txt. So mynewfile.txt gets the modified version of myfile.txt (which is left intact in this example). The g ensures the operation is global, or for every pattern matched - this is a basic global search and replace using sed.

grep = line matching (prints lines matching pattern). Conceptually this streams text through a filter, and will print (echo back) any line that has a match of pattern.
Take a file and find all lines with ("OldValue"):
cat myfile.txt | grep OldValue > mynewfile.txt[Enter]
Explanation: Only lines that have the OldValue text in them will be placed in new file mynewfile.txt.
cat myfile.txt | grep -i OldValue > mynewfile.txt[Enter]
Explanation: (-i option - case Insensitive) Only lines that have the OldValue text (case Insensitive search, i.e matching text, case does not matter) in them will be placed in new file mynewfile.txt.
cat myfile.txt | grep -c OldValue[Enter]
Explanation: (-c option - count option) Only lines that have the OldValue text in them will be counted - result will be a number, e.g. 0, 12, 44, etc.
grep can be useful for working with logs, e.g. a mail.log: cat mail.log | grep emailtoyou@example.com[Enter]
Explanation: This will filter log and show any mail events for the e-mail address specified.

utility basics - find, which, whoami, id, w

find = multi-faceted utility that can find files

find always wants a path first, followed by options

Find a specific file (missinglib.so) somewhere on the system:
find / -name missinglib.so[Enter]

Find a certain files (QuarterlyReport*.*) in the home area:
find /home -name QuarterlyReport*.*[Enter]

Find all files larger than 500M and less than 1GB:
find / -type f -size +500M -size -1G[Enter]

Find all files in home directory modified within last 60 days (between a second ago and 60 days ago):
find /home -mtime -60[Enter]

This will display files with 0 bytes in current directory
find ./ -type f -size 0b -print[Enter]

This will display files with 0 bytes of certain type (here .txt) in current directory
find ./ -type f -size 0b -name "*.txt" -print[Enter]

You can use -delete instead of -print to delete the files rather than print them on the screen
find ./ -type f -size 0b -delete[Enter]

which = utility to locate commands / find executable files

which grep[Enter] - shows /bin/grep
which which[Enter] - shows /usr/bin/which

whoami = shows current user. This can be useful especially for remote users and system administrators.

whoami[Enter] - shows current logged on user

id[Enter] - shows id and group ids for current user

w[Enter] - shows who is doing what on the system

Linux directories

Use "man hier" for specifics about your particular system/distro. Even this might not cover all the root folders, such as dos, initrd, sys, etc., etc.

This is the reference: Filesystem Hierarchy Standard (FSSTND)


bin contains the system binary files that are essential for the general operation of your computer. These executable files are critical to actually use your system. Some of these include: cp, ls, bash, mount, etc.


All Linux devices are represented as a directory. This makes it easier for other programs to interact with them. For example mount - You can use the system binary mount in the /bin folder to interact with a removable hard drive for example (e.g. mount /dev/sda1).


For ext2/ext3/ext4 file systems. This is where files recovered after a system crash are placed (by the Linux file system checker (fsck)).


opt is reserved for additional software you install; although, most providers do not use it. This is conceptually the same as Windows' Program Files for linux.


sbin is similar to /bin with the exception that these ready to run binaries are reserved for root users. That is they typically will only work when executed by root or sudo. Examples would include: mke2fs, ifconfig, fdisk


tmp is a temporary storage folder. Anything that is to be temporarily stored goes here. It is recommended that you do not delete these manually. If you want to delete them you usually add this to boot up or shutdown procedure since they can contain useful information for programs that are already running.


This folder contains only the files that are absolutely necessary to get a basic Linux system up and going. This folder is not used for programs to run on startup or other user scripts. This folder usually contains Grub information and the Linux kernel.


This folder is the config folder for your entire operating system. Almost everything about configuring your system can be done in here. The general rule of thumb is that these files are static and text based. No binaries should be placed in this folder.

Common config files in here are:

  • /etc/X11 - For configuring X (gui)
  • /etc/apt/sources.list - Configuring apt for Debian based systems
  • /etc/samba/smb.conf - Configuring Samba (making a Linux box visible to a Windows system)
  • /etc/cups - Printer configuration
  • /etc/fstab - Device mounting
  • /etc/httpd (/etc/apache) - Apache webserver


Traditionally all mounts were stored in the /mnt directory but out of controversy of where to stored removable mounts this directory was born. /media should be used to store mounts for removable devices only, like: CD-Rom, DVD-Rom, Floppy, Flash Disks


proc is a special virtual directory like /dev that contains process information. It contains runtime system info like: system memory, devices mounted, hardware configuration, etc.


srv is a services folder. It holds system specific data to be served by the system for protocols such as, ftp, rsync, www, cvs etc. To be compliant distributions include this folder but it doesn't seem to be used much.


usr houses all the binaries, documentation, libraries, and header files for all the user applications. Most user binaries will be installed into this folder making this one of the largest folders in the Linux directory.


CD-ROMs can be mounted to this folder but it is not in the official Linux Filesystem Hierarchy. CD-ROMs should be mounted under /media.


home is where you will store all user specific documents and settings. This is similar to Windows "Documents and Settings" or "Users". On most Linux distributions /home will contain a directory with each user's name or group of users. e.g. /home/user /home/guests.


lib contains system library binaries that are required to run the system. Linux uses files named with a .so extension (shared object). (In Windows this would be the system folder with .DLLs.)


According to FSSTND version 2.3, This directory is provided so that the system administrator may temporarily mount a filesystem as needed. The content of this directory is a local issue and should not affect the manner in which any program is run.


root is the home directory for the root user. If this directory does not exist it defaults back to /. Hence the username root.


This is similar to /proc - it is used for plug and play configuration.


This stands for variable. This stores files that vary as the system runs. For example, log files, backups, mail, cache, etc.

Integrate scripts with X

The 2 most used X environments are GNOME and KDE. When scripting in a system with these environments available, there are 2 utilities to create dialogs and messages at the X window level

GNOME = zenity
zenity --warning "This is a warning!"

KDE = kdialog
kdialog --error "This is an error!"

Refer to the man pages for all the options (there are many!) - text input, directory lists, list boxes, combo boxed, open / save dialogs, yes/no/cancel, etc., etc.

X server Notes

in Debian, from the command prompt, you can use: dpkg-reconfigure xserver-xorg - on older versions, it may be dpkg-reconfigure xserver-xfree86

Debian Squeeze - to recreate the xorg.conf, without the x-server running, use Xorg -configure

Xnest is a tool that provides an X server that runs inside a standard X Window (for running different environments, etc.)

xwininfo is a tool that provides window info on the command line

Use xwininfo -root -tree[Enter] to list all windows, child windows, etc. currently on the system within X

xprop is a tool that provides property info for X objects on the command line

Use xprop[Enter] then click on a window to display its properties

To run X programs from the root user (i.e. you see a "cannot connect to X server :0.0"), you need to establish a host user. For a default account named user, do the following:
At your home directory (e.g. /home/user), run:
xhost local:root
Then login as root, and modify /root/.bashrc - add the following to the .bashrc file with your favorite editor:
export XAUTHORITY=/home/user/.Xauthority

VNC - Virtual Network Computing

VNC is both used as a specific term and a generic term. It was originally developed as Virtual Network Computing which is a way to separate the Graphical User Interface (GUI) from the computer it is running on over a network, allowing remote access of a system by sending graphical and input data between a host system and a remote client.

There are numerous free and commercial approaches to this technology, with various compatibility issues and capabilities. In a general sense, it is cross platform, and can be expected to work more often than not. At a basic level, you need a Server running on your host system (the system you want to connect to from a remote location), and a Viewer running on your remote client. If running on a LAN (Local Area Network), it can be setup very quickly and easily - install the server on one system, then connect using a viewer on a second system by entering the IP address of the host system. If behind a firewall, security can be managed within the LAN, and as an example, this can provide a way for support personnel to quickly & easily connect to any system on the LAN.

Common Ports used in VNC
5900 - default port, display 0
5901, 5902, 5903, ... - ports for display 1, 2, 3, etc.
5800 - default port for Java interface to VNC server
5801, 5802, 5803, ... - ports for java interfaces to display 1, 2, 3, etc.

Here are some packages/names that are a staring point:

vncserver - vnc4server - has command line level support
X11vnc - x11vnc - X11 Desktop VNC server, many, many options
Vino - vino - VNC Server for GNOME
TightVNC - tightvncserver - server component of TightVNC
linuxvnc - linuxvnc - vnc server for tty
Plus many more...

gvncviewer - gvncviewer - GNOME / GTK VNC viewer
gtkvncviewer - gtkvncviewer - Small GNOME VNC viewer
directvnc - directvnc - VNC viewer using RFB Remote Framebuffer Protocol
xtightvncviewer - xtightvncviewer - X Viewer component of TightVNC
krdc - krdc - KDE Remote Desktop Client
Plus many more...

Free/Commercial Solutions
Plus many more...

Some other support packages
xserver-xorg-video-dummy - used to create a dummy desktop for a vnc server on a headless server
Xvfb - a virtual frame buffer for X for a vnc server on a headless server

When operating over the internet, a common way to provide a secure connection is to use SSH tunneling, or port forwarding within an SSH connection. In general, when you enter an IP to connect to a VNC server, it connects to port 5900 at the server IP. To protect things, you can configure the server to ONLY open a connection from the localhost (i.e. itself). Then, you can ssh (Secure Shell) to the server, and by doing port forwarding (or tunneling), you can specify the port (e.g. 5900 or 5901, etc.), and then when you connect, you also specify localhost - but now that port is forwarded/tunneled through ssh to the server - so the client thinks it is connecting locally, and the server also sees the connection as local, with data communications to the VNC port 5900 going throught the ssh tunnel. This can get complicated and there are various options and security concerns. As a starting point, here is a command for the ssh connection:

ssh -L 5900:localhost:5900 -N -f -l thevncuser IP.of.the.server
e.g. "ssh -L 5900:localhost:5900 -N -f -l user"
e.g. "ssh -L 5900:localhost:5900 -N -f -l someuser myserver.example.com"

-L 5900:localhost:5900 : port on the local (client) host is to be forwarded to the given host and port on the remote system
-N : Do not execute a remote command i.e. just forward ports
-f : Requests ssh to go to background just before command execution.
-l user : login as user
IP.of.the.server : actual IP address or host name to connect to

Viewer configured to connect to localhost and server configured to only allow localhost connections

Conceptually this is:

If you have a firewall issue, and need an intermediate system, e,g:
You can setup the viewer to connect to the CLOUD_INTERMEDIATE, and use tunneling to connect the CLOUD_INTERMEDIATE to the VNC_SERVER_HOST. You also can chain tunnel systems and essentially duplicate the port forwarding for each system in the chain.

If you have a situation where the VNC_SERVER_HOST needs to create the connection OUT to the CLOUD_INTERMEDIATE, e.g.:
You can use Reverse tunneling, e.g. ssh -R 5900:localhost:5900 (instead of -L), and the sequence would be to connect via ssh from VNC_SERVER_HOST to CLOUD_INTERMEDIATE, and then connect from REMOTE_CLIENT using VNC Viewer to CLOUD_INTERMEDIATE. To verify port is open on CLOUD_INTERMEDIATE, as superuser, you can use netstat -aon, so to see if 5900 is open:

root@server# netstat -aon | grep 5900
tcp 0 0* LISTEN off (0.00/0/0) tcp6 0 0 :::5900 :::* LISTEN off (0.00/0/0)

This shows that port 5900 is open and is listening for a connection. If you do not see this, then your reverse tunnel is not up and running, and there is no VNC port to connect to on the CLOUD_INTERMEDIATE system.

split, tar, gz (file size limits)

On 32-bit linux, there is a 2GB file limit. If the files is larger than a DVD (or any set of files), then it might help to compress/zip them. However, if the zip will be greater than 2GB (probable), then just a zip won't do. Here are some options (Plus for Windows) using:

tar cz * | split -b1000m -d - fileprefix.s
will generate 1GB fileprefix.s00, fileprefix.s02, fileprefix.s03, etc.
This uses tar to create an archive with all the files to stdout, then split into 1GB files (- is stdin) named with fileprefix.s??

To restore from these files, on linux:
cat file.s00 file.s01 file.s02 | tar xz

To split an existing archive into 1GB files (say for backing up on a DVD without running into 2GB file size limit - 4 files fit comfortably per DVD)
split -d -b1000M archive_2013_03_18.tar.gz archive_2013_03_18.tar.gz.s
So a 9GB file will make .s00, .s01, .s02 --- .s08 files, e.g. archive_2013_03_18.tar.gz.s00

This sends the files thru the standard input to tar, which eXtracts and unZips

In Windows, the UnxUtils.zip will do the job, but not directly - the tar doesn't support zip, etc. So here is how to use these utils to restore
1) rebuild the unsplit file
cat file.tar.gz.s00 file.tar.gz..s01 file.tar.gz.s02 file.tar.gz.s03 > file.tar.gz
2) now unzip the file
gunzip file.tar.gz
3) now untar th archive
tar xf file.tar

date - How to set the date / time

The quickest and easiest way to set the date from the command line is using the date command (system privileges required, i.e. super user (root)). Also see timedatectl notes below for more control options.
To set date, use the -s option with sufficient privileges:
(Refer to the format displayed, then enter the new time/date following the indicated format with the -s option)
date -s "Mar 9 10:39"[Enter]
For quick date, use date -s 4/2/2009
For quick time, us date -s 1400 (2:00 pm)
Alternatively, you can specify the format, e.g.
date +%Y%m%d -s "20070309" for Mar 9 2007
date +%H%M -s "1054" for 10:54 am

To use the date in a script/file name, you can save the date output to a variable, e.g.:

... bash script excerpt ...
CURRENTDATE=`date +%Y-%m-%d`
cp mycurrentbackup.fil backup_$CURRENTDATE.fil

timedatectl may be used to query and change the system clock and its settings, and enable or disable time synchronization services.
To see current system status, just use timedatectl[Enter]

If you want to set the system date to the past or the future, you may need to disable the Network Time Protocol (NTP) service. You can do this with timedatectl's set-ntp, e.g.: timedatectl set-ntp false[Enter]
Note: the Boolean option can be true/false, 1/0, on/off, yes/no, e.g. set-ntp off

To set the time, use timedatectl set-time yyyy-mm-dd hh:mm:ss[Enter]

Accessing a CD/DVD ISO image

If you have an iso image file, and want to access it from the file system, you can "mount" the image file just as you would mount a burned disc.

su - (the su with the dash drops you into the home directory for root)
mkdir ~/temp
mount -o loop -t iso9660 /somepath/toiso/disc.iso ~/temp

Now the iso files will be accessible in the ~/temp directory. Just be sure to point the /somepath/toiso/disc.iso to point the the iso file you want to use, and use temp as above, or use a /mnt folder, or wherever you want to have it mounted. When finished, do the following to unmount the cd/dvd image:

su -
umount ~/temp-

Obtaining MD5 Sum from CD/DVD/USB

This can be useful to verify a burned CD/DVD from an ISO image when you already know the MD5 sum of the ISO image (e.g. downloaded file). For customers, we can provide ISO MD5 checksums/file size of original media if requested. See below for notes on verifying USB media. Also note that some distros may use SHA sums, e.g. sha1sum or sha256sum utilities.

Your system needs isoinfo, dd & grep, and md5sum, and this assumes your CD is in the CD drive tagged as /dev/cdrom in your system. Some systems may have DVD devices as /dev/sr0, but be sure to verify/know what the device is known as for the system in use, and make appropriate changes below if necessary.

First, obtain relevant information about the CD/DVD:
isoinfo -d -i /dev/cdrom | grep "size is"[Enter]
The 2 important lines are "Logical block size is" (bs) and "Volume size is" (count)- use these numbers for bs and count in the dd command.
Example results:
Logical block size is: 2048
Volume size is: 356540
Now run the md5 calculation from the disc data:
dd if=/dev/cdrom bs=2048 count=356540 conv=notrunc,noerror | md5sum[Enter]
Then the md5 sum will be displayed - this should match the md5 sum of the ISO used to create the CD (which typically would be provided from the source). To calculate the md5 of an ISO, use:
md5sum /home/user/mydisc.iso[Enter]
openssl md5 /home/user/mydisc.iso[Enter]

When working with USBs, there is no similar option to determine "burned" data, as the media is writable, incorporates partitions, etc. It is not in the same realm as a burned disc. Also it is writable, so actual use can affect the original image, especially if there is a writable/user-updatable partition. Because of the nature of checksums, things only match if and only if all the data is exactly the same, so actual use or adding data to a writable partition means there is no way a checksum will match. There may be ways to find the "read-only" partitions, but more details on the original ISO and the way the disk is laid out may be required (beyond file size/checksum). Note that some distros have options to "check this media" and that (if available) can be used, as once booted into the distro, the environment the operating system is running in "knows" about the structure of its own data.

Note that some distros may automatically mount one or more partitions when you insert a USB. To do any operation on the USB, it is recommended that you make sure there are no mounted partitions (e.g. umount /dev/sdb1). Also, due to the nature of the caching, you may not be actually comparing the USB, rather what is in memory. To flush the cache, this command: blockdev --flushbufs /dev/sdb, eject the USB, re-insert it, then do any comparison (where /dev/sdb matches the actual device)

One option is to read in the data from the USB, and then check the iso. Note, however, that if you read in the data from a drive, you will get a much larger set of data than the actual ISO originally written. So if you had a 2.5GB ISO on the USB, but used dd if=/dev/sdb of=checkthis.iso bs=4M, the file checkthis.iso will be closer to the size of the total USB vs. just the originally written ISO. One way to handle this is to use the utility head to read in only the data that corresponds to the original ISO. So if you have the file size (example 12345678), you can use a command like head -c 12345678 checkthis.iso | md5sum to obtain the checksum from the ISO portion of the data read from the USB.

Controlling Users

Add a user: useradd -m -c "First Last" newusername[Enter]
Password for user: passwd newusername[Enter]
New: newpassword[Enter]>
Confirm: newpassword[Enter]
Remove a user: userdel newusername[Enter]
Add existing user to an existing group: usermod -G groupname username[Enter]
Add a user with specific directory (here /home/ftp) as home: useradd -m -d /home/ftp -c "First Last" newusername[Enter]
Show which groups a user belongs to: groups newusername[Enter]
Show which users are in a group (example ftpusers): getent group ftpusers[Enter]

To list all groups on the system, use getent group
For just group names, use getent group | cut -d: -f1
To sort group names, use getent group | cut -d: -f1 | sort
For numbered sorted group names, use getent group | cut -d: -f1 | sort | cat -n

For actual system files, see /etc/passwd for user info and /etc/group for group info. Use man pages for details, e.g. man 5 passwd and man 5 group (where 5 indicates section)

These commands allow you to control users on your system, or force a user off if there is some problem

How to stop and halt (pause) the user
# skill -STOP -u username

How to continue the halted user
# skill -CONT -u username

How to Kill and make them logout from the computer
# skill -KILL -u username

How to kill all the users in the system
# skill -KILL -v /dev/pts/*

passwd notes

passwd changes your (or (if you are root/superuser) another user's) password.

Will ask for current password, new password, then retype new password (to verify) - if all is correct, it will update and change your password (or indicate a problem if one occurs).

as root/superuser
passwd someuser[Enter]
Will ask for new password, then retype new password (to verify) - if all is correct, it will update and change someuser's password.

A few passwd options:
-d, --delete, e.g. passwd -d someuser[Enter] - Delete a user's password (make it empty). This is a quick way to disable a password for an account. It will set the named account passwordless.
-S, --status, e.g. passwd -S someuser[Enter] - Display account status information. The status information consists of 7 fields. The first field is the user's login name. The second field indicates if the user account has a locked password (L), has no password (NP), or has a usable password (P). The third field gives the date of the last password change. The next four fields are the minimum age, maximum age, warning period, and inactivity period for the password. These ages are expressed in days.

Kerberos problem?
If this is on a new Debian (or other) system, and you get “Enter Kerberos password”, and you don't need/want Kerberos, you can remove the security library with:
apt-get purge libpam-krb5[Enter]

Controlling Processes

This is a complex issue. Here is enough to become dangerous…
Usually if something hangs or is causing some problem, you want to be able to kill it so you don't have to reset the system - this is how to do so (must be super user / root).

ps aux - lists all processes
ps aux | grep someprog - shows processes for someprog
kill 1234 - sends kill signal to process indicated by process ID 1234 (as shown in ps commands)
kill -s 9 1234 - terminate process ID 1234

smb:// or smbclient unable to connect by name

This is related to the machine being able to resolve machines/domains via the windows machine name. By updating /etc/hosts with an entry for the machine, it can work. You may also just browse by IP address instead of name, e.g. smb:// Or use smbclient //

/etc/hosts changes (example) somemailserver hpserver

Manipulating images

Want to create thumbnails on the fly - reference GraphicsMagick

Debian: apt-get install graphicsmagick

Examples for thumbnails:
gm convert -size 120x120 cockatoo.jpg -resize 120x120 thumbnail.jpg
gm convert -size 100x100 somepic.png -resize 100x100 thumb_somepic.png

Resize a picture example:
gm convert screenshot.png -resize 640x512\! -quality 100 websiteimg.png

For those looking at picture editing, or other tools for working manually with images, refer to this list:

Sound & Video

Sound and Video is an extremely large topic - this is for some people who are trying to do something, but don't know where to start. Just some basic info and a particular starting point for various categories.


Speakers - way to test speakers from command line:
speaker-test -t sine -f 1000 -c 2 -s 1 &
speaker-test -t sine -f 1555 -c 2 -s 2 &
speaker-test -c 5 -l 1 -t wav &

alsa - sound library (there are others)
Some command line tools:
aplay soundsound.wav


Audio Editors
Audacity, Jokosher, Ardour, Kwave, Qtractor, LMMS, Traverso


mplayer / mencoder
Huge number of options, command line examples:
mplayer SomeVideo.m4v
mplayer SomeVideo.avi
Convert from one Video format to another (with options)
mencoder SomeVideo.m4v -o SomeVideo.avi -oac pcm -ovc lavc -lavcopts vcodec=wmv1

Media Player
Audio: Amarok, Rhythmbox, Kaffiene, JuK
Video: VLC, Dragon Player, mplayer

DVD ripper/video converters
You never know when you'll have to get that family video onto a website...
Rip: HandBrake, Thoggen DVD ripper
converter: VLC, WinFF, ffmpeg

Disc burner
K3B, Brasero
mkisofs, cdrecord

WebCam/Camera/Tuner access
xawtv, video4linux
/usr/bin xawtv -hwscan
#/home/user/.xawtv has config settings /usr/bin/xawtv -device /dev/video0 &
Open a web cam on the secondary display
/usr/bin/mplayer tv:// -tv driver=v4l2:width=400:height=300:device=/dev/v4l/by-id/usb-Vimicro_Corp._PC_Camera-video-index0 &

Video Editors
OpenShot, Blender, Cinelerra, kdenlive, Avidemux, Flowblade, Jahshaka, Lightworks, LiVES, Pitivi, Shotcut

Video Renderer
Blender, slowmovideo

Video File Notes
You can run across all types of video files - MOV, MPG, MPEG, MP2, MP4, AVI, DV, RAW, VOB, MKV, ASF, etc., etc. Audio files can be WAV, WMA, MP3, RAW, AIFF, etc., etc. You will run into containers, and codecs - containers can have both audio & video streams in them, and then each can have a codec (coder/decoder, i.e. for digital file storage and compression).

The following Debian packages can be helpful in working with audio/video:
ffmpeg - powerful command line tool for manipulating video files
mencoder - utility for converting/manipulating media files
dvgrab - utility to pull video from a digitial camera
devede - basic tool to create a Video DVD
exiftool - perl tool for reading and writing meta information in media files
extract - utility to extract meta information from medial files
libav-tools - compatibility tools including ffmpeg, ffplay, ffprobe (avconv, avplay, avprobe)
mediainfo - utility for reading information from audio/video files

For example, if you want to find the original record date from an AVI file, you can use mediainfo to extract details on the AVI file.

If you want to split an AVI file, the following are possibilities:

Split a 4 minute video file.avi into two 2 minute video files file1.avi, file2.avi
ffmpeg -i file.avi -vcodec copy -acodec copy -t 00:02:00 file1.avi
ffmpeg -i file.avi -vcodec copy -acodec copy -ss 00:02:00 file2.avi

Same example with mencoder (seems to work better if later concatenating files with mencoder)

mencoder file.avi -o file1.avi -oac copy -ovc copy -endpos 00:02:00
mencoder file.avi -o file2.avi -oac copy -ovc copy -ss 00:02:00

Note that these utilities have a huge number of options and capabilities, so there can be a steep learning curve to become proficient when working with these types of files.

Manipulating mail box files

mbox type files are single files that concatenate messages separated by each other with a From[space] format line. Ignoring the details - see Wikipedia mbox for background - if you want to split this file into separate text files, the formail utility can help

Assume your mailbox file (mbox) is found in /var/mail/user (where user is the user who's mail you want to access). Have root (or sufficient rights user) copy the file into your home directory, i.e. cp /var/mail/user /home/user/mbox.txt[Enter], and perform the following steps:

mkdir mailfiles[Enter]
cp mbox.txt mailfiles
cd mailfiles[Enter]
cat mbox.txt | formail -ds sh -c 'cat > msg.$FILENO.txt'[Enter]

The above will split each mail message into msg.000.txt, msg.001.txt, etc.


In general, scanners are fairly well supported in Linux. For details and info on specific hardware, see these links:
USB Scanner Support (linux-usb.org)

The SANE project (Scanner Access Now Easy)
SANE - Supported Devices
SANE - Manufacturer list

Here are some manufacturers that have Linux drivers:

Brother drivers
Epson drivers
HP Linux Imaging and Printing
Kodak drivers

Debian exim configuration

This is default mail transfer agent (MTA), and if not setup correctly, it will hang for a long time at system boot up.

So, choose the "Local mail delivery only" option.

Try "dpkg-reconfigure exim4-config" on sarge and "eximconfig" on woody

Also, you can choose the "DNS on demand" dial-up setting for local servers or modem based systems

A handy ncurses GUI frontend for dpkg-reconfigure is "configure-debian" (and there's also another GUI frontend to be used in X, "gkdebconf"), which saves you from having to remember the exact names of all the packages that are handled by DebConf. Just install configure-debian, run it (as root), and you'll find your MTA listed under "mail" category


For Debian and Ubuntu, APTonCD is a utility that will preserve the packages installed on a system on disc, allowing you to quickly & easily save this, move to another system, or carry around for convenience.

APTonCD at SourceForge


Running postfix, and you need to delete a group of mail messages from a mail queue (e.g. a mail script gone bad, or a domain not accepting mail, etc.)

Here is a combination command that will delete a group of messages from the queue - it dumps the mail queue, strips out the messages of interest, obtains the mail ID, and finally ask postsuper (postfix administration command tool) to delete them.

mailq | grep advertmail@ | cut -f 1 -d ' ' | postsuper -d -
mailq | grep advertmail@ | awk '{print substr($1,1,10)}' | postsuper -d -

Command explained:
The commands are piped through each other using the | (pipe) symbol - this means the output of the first is fed as input to the second, the output of the second is fed as input to the third, etc.
mailq - this is the command that lists the mail queue, or items waiting to be mailed. This lists various details about each mail item.
grep advertmail@ - after this command is processed, it will only show the first line of the mail queue command for the mail items of interest. In this example, advertmail@ is the user that has sent the mail we want to delete. You might use a domain, e.g. "grep yahoo.com" to list mail going to the yahoo.com domain. You will want to use the correct text for the mail items of interest, and test at this point (i.e. mailq | grep someuser[Enter] to verify you are listing the mail items you want to delete).
cut -f 1 -d ' ' - this item "cuts" the first field [-f 1] (delimited by space, e.g. a [-d ' '] (single-quote,space,single-quote)), which is the mail ID. The awk approach [awk '{print substr($1,1,10)}'] is more exacting, but it helps if you are familiar with awk syntax - in this example, it takes the text of each line, and outputs the first 10 characters (in this case, the mail ID is 10 characters).
postsuper -d - - finally, the postsuper command has various options, and the -d option is to delete the mail (by ID). In this case the trailing -, means to take the input from the standard input (vs. a file or via command line).

When testing, build up the command by testing mailq and grep portions, then add the "ID stripping" command of cut or awk, before finally using the full command that will delete the mail items. If you build step-by-step, the last command (before postsuper) will show a line by line listing of mail IDs.

Other useful commands for working with IDs:
postqueue -i MAILIDCODE
The postqueue -I will force a resend of the specific MAILIDCODE (vs. sendmail -q to flush the entire queue)
postcat -vq MAILIDCODE
The postcat -vq will dump the full content of the e-mail specified by the MAILIDCODE - note there is some additional system information and detail (this is not the raw e-mail data)

If for some reason, your mail server is unable to send mail due to internet connection issues, this outlines a way to grab deferred messages and view mail contents and/or send mail from a different (connected) server.

The following bash script will get all deferred e-mails into a single text file for perusal, or other manipulation. Note that some manual steps are required (building list.txt, cleaning out sub-folders prior to script run, etc.):
#File: get_deferred_email.sh
#This uses list.txt with "find . > list.txt" in the
#/var/spool/postfix/deferred folder, removing plain directory entries
#(e.g. ./D)
#so get_deferred_email.sh would be copied to
#create list.txt as above
#make sure mail.txt is removed
#then run ./get_deferred_mail.sh

for item in `cat list.txt`;
echo $item
postcat $item >> mail.txt
echo "====####====" >> mail.txt

If you want these listed in timestamp order, you can use the following script:

#File: sort_by_time.sh

for item in `cat list.txt`;
#echo $item
THEDATE=`postcat $item | grep message_arrival | sed -e 's/message_arrival_time://' `
#echo $THEDATE
TIMESTAMP=$(date --date="$THEDATE" +"%s")

echo $TIMESTAMP"_"$item
echo $TIMESTAMP"_"$item >> mail.txt


cat mail.txt | sort > mailbytime.txt

for item in `cat mailbytime.txt`
THEFL=`echo $item | cut -f 2 -d "_"`
echo $THEFL >> ordlist.txt

Then you would have to modify get_deferred_mail.sh to use ordlist.txt instead.

If you had some other server configured to send mail, using sendmail -t < mail.txt approach could be used to strip out the actual mail message, edit if necessary, then save this as mail.txt, and use the above sendmail command to send it off. As an example, this is what mail.txt might look like:

To: Contact.Person@support.isp.com
Subject: Our main mail server is down!
From: webmasterdude <webmaster@mail.example.com>
Content-type: text/html
X-Sender: <normalmail@example.com>
X-Mailer: PHP
X-Priority: 3
Message-Id: <20220912183003.501AC1840F8@example.com>
Date: Mon, 12 Sep 2022 14:30:03 -0400 (EDT)

Our main server is down, and this message never went out - I've used my incredible skills as a mail
server admin to extract this e-mail from deferred e-mail, and am sending it from our backup server.
You will see the original message when the fiber optic dudes fix the severed cable.<br>

DNS Tools

DNS stands for Domain Name System, sometimes just referred to as nameserver(s). /etc/resolv.conf shows name servers to be queried to resolve a domain name for mail, ping, etc. utilities. For example, when you type ping linuxcollections.com[Enter], the sequence to ping the server starts with trying to resolve the IP address for linuxcollections.com - by referencing the /etc/resolv.conf file, the name servers listed can be used to find out what address should be used by the TCP/IP protocol layer to communicate with.

ping linuxcollections.com[Enter] verifies name server lookups are working
nslookup[Enter] interactive tool to display name server or ip lookups
nslookup goes into a > prompt - type the domain name to see response, and type the IP address to see the reverse DNS result
dig[Enter] command line tool to display name server records for a domain (e.g. A or MX entries)
type dig mymaildomain.com[Enter] to see basic entries (A), or use dig mymaildomain.com MX[Enter] to see Mail Exchanger entries
telnet mymaildomain.com 25[Enter] telnet is an interactive communication tool - here, can be used to verify a mail server is responding - 587 is becoming a more common port, e.g. telnet mymaildomain.com 587[Enter]

Debian package management

Apt notes / configuration

apt / apt-get, dpkg are managers for debian based packages (various distributions use this, including Ubuntu) (Other package managers are rpm (Red Hat Package Manager), yum, and others - common packages are .deb, .rpm, .tgz (.tar.gz))

These are command line tools for installing, removing, and managing packages. There are various graphical package tools, like Synaptic that can make some of this easier (aptitude is a text based tool).

Finding the names for a specific package or utility isn't always easy. If you know the file name you are looking for, you can search in the files included for each package. If you want a specific function, it can get a bit more difficult. Reference packages on debian.org.

These are reference notes for basic package management in a debian type system.

  • apt-get install somepackage - installs somepackage
  • apt-get remove somepackage - removes somepackage, but leaves configuration files
  • dpkg --purge somepackage - removes everything for somepackage
  • apt-get upgrade somepackage - upgrades the specific package somepackage
  • apt-get upgrade - upgrades installed packages
  • apt-get dist-upgrade - upgrades the distro you are on (handles other file type dependencies for the system)
  • apt-mark hold somepackage - tags package so it will not automatically update
  • apt-mark showhold - shows all packages marked as held
  • apt-mark unhold somepackage - removes tag on a held package
  • dpkg -l - lists installed packages
  • dpkg -l '*' - lists all packages available and status
  • Problems with GPG / No Public Key available
  • apt-get install debian-archive-keyring - add older/archive keyring
  • apt-get install debian-keyring - add current keyring
  • or just do both apt-get install debian-keyring debian-archive-keyring
  • General Steps (overview) for upgrading major release (e.g. Wheezy to Jessie, etc.)
  • apt-get update - update your system (from sources.list for current release)
  • apt-get upgrade - upgrade your system (make sure everything is fully updated for current release)
  • Change /etc/apt/sources.list - check debian.org, make sure you are moving to next distribution major release (never jump more than 1)
  • apt-get update - read in new dist sources
  • apt-get dist-upgrade - do major upgrade (watch for prompts/questions/configuration changes during process)
  • Adding a CD/DVD to /etc/apt/sources.list
  • Make sure that your CD is inserted to your CD-ROM drive and enter command: apt-cdrom add
  • This will add something like:
  • deb cdrom:[Debian GNU/Linux VERSION NAME DISC DETAILS]/ NAME main
  • line into your /etc/apt/sources.list file

/etc/apt has the apt configuration folders and files.

apt-config is the APT Configuration Query program.
apt-config dump shows the configuration.

/etc/apt/sources.list : Locations to fetch packages from.
/etc/apt/apt.conf : APT configuration file.
/etc/apt/apt.conf.d/ : APT configuration file fragments .
/etc/apt/preferences : version preferences file. This is where you would specify "pinning", i.e. a preference to get certain packages from a separate source or from a different version of a distribution.
/var/cache/apt/archives/ : storage area for retrieved package files.
/var/cache/apt/archives/partial/ : storage area for package files in transit.
/var/lib/apt/lists/ : storage area for state information for each package resource specified in sources.list
/var/lib/apt/lists/partial/ : storage area for state information in transit.

Debian Packages - Create Local Repository

There are 4 steps necessary to add your own local repository to the apt-get/aptitude/Synaptic package management tools. You might want to do this if you are working with non-free type packages, your own packages, or customizing a specific type of install.

  1. Create a folder to hold all .deb files, and then copy all .deb files into the folder, e.g.
    mkdir /home/user/repository[Enter]
    cp /home/user/Desktop/*.deb /home/user/repository[Enter]
  2. Create a packages list for use by apt using dpkg-scanpackages - dpkg-scanpackages . /dev/null | grep -9c > Packages.gz[Enter], e.g.
    cd /home/user/repository[Enter]
    dpkg-scanpackages . /dev/null | grep -9c > Packages.gz[Enter]
  3. As superuser/root, update the /etc/apt/source.list file, e.g.
    #nano /etc/apt/source.list[Enter]
    then add a few lines to end of file:
    #This is our local repository
    deb file:/home/user/repository ./
    Then save the file.
  4. Finally (as superuser/root) run apt-get update - now your packages will be available for install using apt-get, or via Synaptic, etc.

Debian Configuration Notes

Quick Reference for Debian (duplicates info elsewhere)

dpkg-reconfigure console-data
dpkg-reconfigure locales
dpkg-reconfigure debconf

dpkg-reconfigure exim4-config

dpkg-reconfigure xserver-xfree86
dpkg-reconfigure xserver-xorg
Xorg -configure
modify /etc/X11/xorg.conf

Network skipped during install, quick fix:
Add this to /etc/network/interfaces:

# The primary network interface
auto eth0
iface eth0 inet dhcp

or (if static ip)

auto eth0
iface eth0 inet static

If necessary, name servers can be added to /etc/resolv.conf, e.g.

Note: IP addresses are examples - be sure to use your correct values!

then /etc/init.d/network restart

Mouse Configuration

Background: Had problems with an old ball mouse, so bought a Logitech Optical Mouse PS/2 (model M-SBF96), and plugged it in to the KVM, rebooted, and the mouse went crazy wild. Many hours later, this was the solution.

Debian Squeeze, kernel 2.6.32-5-amd64. Started with the xserver-xorg settings, but that was a dead end. Needed to update/override the mouse driver options (specifically the driver protocol). Verified that without the psmous.conf file, the mouse did not work. Believe the proto=bare was key piece, but this was not verified (other proto settings did not work). Here is approach:

Created /etc/modprobe.d/psmous.conf file with these settings
options psmouse proto=bare rate=60 resolution=200 smartscroll=Y

Notes: Saw other notes that this file is psmouse.conf - did not check to see if it matters, but used psmous.conf
Played with resolution, not sure how sensitive/important this value is here
Also not sure how important rate and smartscroll are
Other entries for proto resulted in unusable mouse
Mouse wheel still not working - not sure if bare entry is cause
Run xev from terminal to see raw mouse events

Update: Due to other systems connected to KVM, switched to a Dell/Microsoft ball mouse with wheel - the wheel never worked in Linux/Debian/KDE until... Many months later, after some Debian updates, occasionally the mouse would go crazy and do all kinds of random events. Saw in dmesg | grep psmouse "psmouse.c: Mouse at isa0060/serio1/input0 lost synchronization, throwing 2 bytes away" and did some searching. Tried not using psmous.conf for a while, still had issues. Saw several items that suggested using the imps protocol, and created a /etc/modprobe.d/psmouse.conf file with the line: options psmouse proto=imps. Now the mouse works AND the wheel works. Unaware of specific system update that caused problem (or fixed problem?). Note when original configuration option was used, imps was tried, and did not fix wheel or mouse issue. Now with recent update, a good old ball mouse with wheel, the imps works like a charm.

MySQL Statements Reference

Creating tables via SQL “script”

CREATE TABLE contacts (
`ID` varchar(10),
`Customer ID` varchar(20)
) Type MyISAM;
//Type InnoDB;

MySQL quick notes

Show databases;
Use somedb;
Show tables;
Show columns from contacts;
describe contacts;
select count(*) from Contacts; -or- select count(*) as RecordCount from Contacts;

MySQL transfer updated Product database to live Product database

#This quickly moved all the data from an updated table to live table - can work with #productsnew as needed until ready for update, then run script below # (works for modest amount of products) #RootPwd = mysql root password
#ProductsDB = Product database
#products = live products table
#productsnew = updated products table

LOGDATE=`date +%Y-%m-%d`

#Save off current products data, copy with date in file name
mysqldump -pRootPwd ProductsDB products > /SQL/productscurrent.sql
cp /SQL/productscurrent.sql /SQL/products$LOGDATE.sql
#Save off updated products data
mysqldump -pRootPwd ProductDB productsnew > /SQL/productsnew.sql
#now update productsnew table references to current products table name
cat /SQL/productsnew.sql | sed -e 's/productsnew/products/g' > /SQL/products.sql
#drop and insert from modified sql dump data
mysql -pRootPwd ProductsDB < /SQL/products.sql

Show all data in table

SELECT * from contacts;

Add new records

INSERT INTO contacts ( ID, [Customer ID]) VALUES (‘1234’,’SHO0001’);

Update record

UPDATE contacts SET ID = ‘33221’ WHERE CustomerID = ’SHO0001’;

Change column name

Alter table contacts change ID RecordID varchar(10);

Change column type

Alter table contacts change ID ID varchar(50);

Dropping Table

Drop table contacts;

Drop column from table

Alter table contacts drop RecordID;

Adding column/index

Alter table contacts add RecordID varchar(30);
Alter table contacts add index `sortid` (`sortid`);

Copies from one table and inserts into another table (same structure)

All Columns:
INSERT INTO table2 SELECT * FROM table1 WHERE custid='4321';

Specified Columns:
INSERT INTO table2 (column1, column2, column3, ...) SELECT column1, column2, column3, ... FROM table1 WHERE userid='1234';