Text Manipulating Commands in Linux

# cat example.txt | awk 'NR%2==1'
remove all even lines from example.txt

# echo a b c | awk '{print $1}'
view the first column of a line

# echo a b c | awk '{print $1,$3}'
view the first and third column of a line

# cat -n file1
number row of a file

# comm -1 file1 file2
compare contents of two files by deleting only unique lines from 'file1'

# comm -2 file1 file2
compare contents of two files by deleting only unique lines from 'file2'

# comm -3 file1 file2
compare contents of two files by deleting only the lines that appear on both files

# diff file1 file2
find differences between two files

# grep Aug /var/log/messages
look up words "Aug" on file '/var/log/messages'

# grep ^Aug /var/log/messages
look up words that begin with "Aug" on file '/var/log/messages'

# grep [0-9] /var/log/messages
select from file '/var/log/messages' all lines that contain numbers

# grep Aug -R /var/log/*
search string "Aug" at directory '/var/log' and below

# paste file1 file2
merging contents of two files for columns

# paste -d '+' file1 file2
merging contents of two files for columns with '+' delimiter on the center

# sdiff file1 file2
find differences between two files and merge interactively alike "diff"

# sed 's/string1/string2/g' example.txt
replace "string1" with "string2" in example.txt

# sed '/^$/d' example.txt
remove all blank lines from example.txt

# sed '/ *#/d; /^$/d' example.txt
remove comments and blank lines from example.txt

# sed -e '1d' exampe.txt
eliminates the first line from file example.txt

# sed -n '/string1/p'
view only lines that contain the word "string1"

# sed -e 's/ *$//' example.txt
remove empty characters at the end of each row

# sed -e 's/string1//g' example.txt
remove only the word "string1" from text and leave intact all

# sed -n '1,5p' example.txt
print from 1th to 5th row of example.txt

# sed -n '5p;5q' example.txt
print row number 5 of example.txt

# sed -e 's/00*/0/g' example.txt
replace more zeros with a single zero

# sort file1 file2
sort contents of two files

# sort file1 file2 | uniq
sort contents of two files omitting lines repeated

# sort file1 file2 | uniq -u
sort contents of two files by viewing only unique line

# sort file1 file2 | uniq -d
sort contents of two files by viewing only duplicate line

# echo 'word' | tr '[:lower:]' '[:upper:]'
convert from lower case in upper case

Install proftd in linux

1. Download proftpd rpm package, download it from http://rpm.pbone.net

# wget ftp://ftp.pbone.net/mirror/centos.karan.org/el5/extras/testing/x86_64/RPMS/proftpd-1.3.1-3.el5.kb.x86_64.rpm

2. Install rpm package

# rpm -i proftpd-1.3.1-3.el5.kb.x86_64.rpm

3. Use ftpasswd to create user and group for ftp login. Complete manual click here.

Add users

# mkdir /etc/proftpd
# ftpasswd –passwd –file=/etc/proftpd/passwd –name=bob –uid=1001 –home=/home/bob –shell=/bin/false

Add group

# ftpasswd –group –file=/etc/proftpd/group –name=group-name –gid=group-id –member=user-member1 –member=user-member2 … –member=user-memberN

4. Edit /etc/proftpd.conf file

AuthUserFile /etc/proftpd/passwd
AuthGroupFile /etc/proftpd/group

#Disable PAM authentification
#AuthPAMConfig proftpd
#AuthOrder mod_auth_pam.c* mod_auth_unix.c

AuthPAM off

5. Restart proftpd service and put proftpd service in startup list.

# /etc/init.d/proftpd start
# chkconfig proftpd on

Mount Linux partition in Windows

Ext2Fsd free software to mount linux partition into my Windows system. It’s so easy to install and use. Just install it and with their friendly navigation we can mount it painlessly.

Package Auto Update Notifications

Install apticron

Type the following command at a shell prompt:

# apt-get update
# apt-get install apticron

Configure apticron to send email notifications

The default coniguration file is located at /etc/apticron/apticron.conf. Open file using text editor:

# vi /etc/apticron/apticron.conf

You need to set email address to email the notification as follows:

EMAIL="your_email@domain.com"

================================================================

sample configuration file

# apticron.conf
#
# set EMAIL to a list of addresses which will be notified of impending updates
#

EMAIL="admin@myhost.com"

#
# Set LISTCHANGES_PROFILE if you would like apticron to invoke apt-listchanges
# with the --profile option. You should add a corresponding profile to
# /etc/apt/listchanges.conf
#
# LISTCHANGES_PROFILE="apticron"

#
# Set SYSTEM if you would like apticron to use something other than the output
# of "hostname -f" for the system name in the mails it generates
#
# SYSTEM="foobar.example.com"

#
# Set IPADDRESSNUM if you would like to configure the maximal number of IP
# addresses apticron displays. The default is to display 1 address of each
# family type (inet, inet6), if available.
#
# IPADDRESSNUM="1"

#
# Set IPADDRESSES to a whitespace seperated list of reachable addresses for
# this system. By default, apticron will try to work these out using the
# "ip" command
#
# IPADDRESSES="192.10.2.1 2001:db8:1:2:3::1"

Save and close the file. /etc/cron.daily/apticron is the cron script for executing apticron daily and it will send you notfication when updates available.


SSH Manipulations

SSH Banner Message

Login as root and edit ssh config file

# vi /etc/ssh/sshd_config

Find this variable in the config file

# Banner /some/locations/file

Uncomment it and save the file

Restart openssh server

# /etc/init.d/ssh restart

SSH Timeout

echo “TMOUT=300 >> /etc/bashrc

echo “readonly TMOUT” >> /etc/bashrc

echo “export TMOUT” >> /etc/bashrc

Extract a single file from single tar ball

Extracting Specific Files

Extract a file called etc/default/sysstat from config.tar.gz tarball

#tar -ztvf config.tar.gz
#tar -zxvf config.tar.gz etc/default/sysstat
#tar -xvf {tarball.tar} {path/to/file}

This is also valid

#tar --extract --file={tarball.tar} {file}

Extract a directory called css from cbz.tar
##tar --extract --file=cbz.tar css

Wildcard based extracting

You can also extract those files that match a specific globbing pattern (wildcards). For example, to extract from cbz.tar all files that begin with pic, no matter their directory prefix, you could type:

#tar -xf cbz.tar --wildcards --no-anchored 'pic*'

To extract all php files, enter

#tar -xf cbz.tar --wildcards --no-anchored '*.php'

Where,

-x: instructs tar to extract files.
-f: specifies filename / tarball name.
-v: Verbose (show progress while extracting files).
-j : filter archive through bzip2, use to decompress .bz2 files.
-z: filter archive through gzip, use to decompress .gz files.
–wildcards: instructs tar to treat command line arguments as globbing patterns.
–no-anchored: informs it that the patterns apply to member names after any / delimiter.


Tar listing

Tar command provides the option to list files inside compressed tar ball. However mtools includes command called lz which gunzips and shows a listing of a gzip’d tar’d archive without extracting files.

For example, display listing of file called backup.tar.gz type command:

#lz backup.tar.gz

As you see lz provides a listing of a gzip’d tar’d archive, that is a tar archive compressed with the gzip command. It is not strictly necessary on Debian GNU/Linux (or other Linux/BSD/Solaris oses), because the GNU tar(1) program provides the same capability with the command:

#tar -tzf backup.tar.gz

Locking and Unlocking User Accounts in Linux


To lock, you can use the follow command

# passwd -l username

To Unlock the same account

# passwd -u username

Creating command Alias in Linux

Creating aliases is very easy. You can either enter them at the command line as you're working, or more likely, you'll put them in one of your startup files, like your .bashrc file, so they will be available every time you log in.

I created the l alias above by entering the following command into my .bashrc file:

alias l="ls -al"

As you can see, the syntax is very easy:

   1. Start with the alias command
   2. Then type the name of the alias you want to create
   3. Then an = sign, with no spaces on either side of the =
   4. Then type the command (or commands) you want your alias to execute when it is run. This can be a simple command, or can be a powerful combination of commands.

Sample aliases example

To get you going, here is a list of sample aliases I use all the time. I've pretty much just copied them here from my .bashrc file:

alias l="ls -al"
alias lm="ls -al|more"
alias html="cd /web/apache/htdocs/devdaily/html"
alias logs="cd /web/apache/htdocs/devdaily/logs"
alias qp="ps auxwww|more"
alias nu="who|wc -l"
alias aug="ls -al|grep Sep|grep -v 2010"

Ubuntu: Very useful Commands

Command privileges

    sudo command - run command as root
    sudo su – root shell open
    sudo su user – open shell as a user
    sudo -k – forget your password sudo
    gksudo command – sudo visual dialog (GNOME)
    kdesudo command – sudo visual dialog (KDE)
    sudo visudo – edit / etc / sudoers
    gksudo nautilus – root file manager (GNOME)
    kdesudo konqueror – root file manager (KDE)
    passwd – change your password

Command Network

    ifconfig – displays information network
    iwconfig – displays information from wireless
    sudo iwlist scan – scan wireless networks
    sudo /etc/init.d/networking restart – reset the network
    (file) /etc/network/interfaces – manual configuration
    ifup interface – bring online interface
    ifdown interface – disable interface

Commands Display

    sudo /etc/init.d/gdm restart – reset X (Gnome)
    sudo /etc/init.d/kdm restart – reset X (KDE)
    (file) /etc/X11/xorg.conf – show Configuration
    sudo dpkg-reconfigure - reconfigure xserver-xorg-phigh - reset configuration X
    Ctrl+Alt+Bksp – X display reset if frozen
    Ctrl+Alt+FN – switch to tty N
    Ctrl+Alt+F7 – switch back to X display

Commands Service System

    start service – service to start work (Upstart)
    stop service – service to stop working (Upstart)
    status service – check if service is running (Upstart)
    /etc/init.d/service start – start service (SysV)
    /etc/init.d/service stop – stop service (SysV)
    /etc/init.d/service status – check service (SysV)
    /etc/init.d/service restart – reset service (SysV)
    runlevel – get current runlevel

Commands for Firewall

    ufw enable – turn on the firewall
    ufw disable – turn off the firewall
    ufw default allow – allow all connections by default
    ufw default deny – drop all connections by default
    ufw status – current rules and
    ufw allow port – to allow traffic on port
    ufw deny port – port block
    ufw deny from ip – ip block

Command System

    lsb_release -a – get the version of Ubuntu
    uname -r – get kernel version
    uname -a – get all the information kernel

Commands for Package Manager

    apt-get update – refresh updates available
    apt-get upgrade – update all packages
    apt-get dist-upgrade – version update
    apt-get install pkg – installing pkg
    apt-get remove pkg – uninstall pkg
    apt-get autoremove – removing packages obsotletos
    apt-get -f install – try to fix packages
    dpkg –configure -a – try to fix a broken package
    dpkg -i pkg.deb – install file pkg.deb
    (file) /etc/apt/sources.list – list of repositories APT

Special Packages For commands

    ubuntu-desktop – Setting the standard Ubuntu
    kubuntu-desktop – KDE Desktop
    xubuntu-desktop – desktop XFCE
    ubuntu-minimal – core earnings Ubuntu
    ubuntu-standard – the standard utilities Ubuntu
    ubuntu-restricted-extras – not free, but useful
    kubuntu-restricted-extras – ditto KDE
    xubuntu-restricted-extras – ditto XFCE
    build-essential – packages used to compile
    linux-image-generic – latest generic kernel image
    linux-headers-generic – latest headlines

Applications commands

    nautilus – File Manager (GNOME)
    dolphin – File Manager (KDE)
    konqueror – Web browser (KDE)
    kate – text editor (KDE)
    gedit – text editor (GNOME)


Installing GRUB using grub-install


In order to install GRUB under a UNIX-like OS (such as gnu), invoke the program grub-install as the superuser (root).

The usage is basically very simple. You only need to specify one argument to the program, namely, where to install the boot loader. The argument has to be either a device file (like ‘/dev/hda’). For example, under Linux the following will install GRUB into the MBR of the first IDE disk:

# grub-install /dev/hda

Likewise, under GNU/Hurd, this has the same effect:

# grub-install /dev/hd0

But all the above examples assume that GRUB should put images under the /boot directory. If you want GRUB to put images under a directory other than /boot, you need to specify the option --boot-directory. The typical usage is that you create a GRUB boot floppy with a filesystem. Here is an example:

# mke2fs /dev/fd0
# mount -t ext2 /dev/fd0 /mnt
# mkdir /mnt/boot
# grub-install --boot-directory=/mnt/boot /dev/fd0
# umount /mnt

Some BIOSes have a bug of exposing the first partition of a USB drive as a floppy instead of exposing the USB drive as a hard disk (they call it “USB-FDD” boot). In such cases, you need to install like this:

# losetup /dev/loop0 /dev/sdb1
# mount /dev/loop0 /mnt/usb
# grub-install --boot-directory=/mnt/usb/bugbios --force --allow-floppy /dev/loop0

This install doesn't conflict with standard install as long as they are in separate directories.

Note that grub-install is actually just a shell script and the real task is done by grub-mkimage and grub-setup. Therefore, you may run those commands directly to install GRUB, without using grub-install. Don't do that, however, unless you are very familiar with the internals of GRUB. Installing a boot loader on a running OS may be extremely dangerous.