Quick Linux Commands

#lsb_release -a :     to get the OS release and version.
(to install – package is “lsb” or for debian user – #apt-get install sudo wget lsb-release)

A script runs normally in commandline but it doesnt in a cron job why ?

Different environment
Cron passes a minimal set of environment variables to your jobs. To see the difference, add a dummy job like this:
* * * * * env > /tmp/env.output
Wait for /tmp/env.output to be created, then remove the job again. Now compare the contents of /tmp/env.output with the output of env run in your regular terminal.
A common “gotcha” here is the PATH environment variable being different. Maybe your cron script uses the command somecommand found in /opt/someApp/bin, which you’ve added to PATH in /etc/environment? cron does not read that file, so runnning somecommand from your script will fail when run with cron, but work when run in a terminal.
To get around that, just set your own PATH variable at the top of the script. E.g.
#!/bin/bash
PATH=/opt/someApp/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# rest of script follows
Some prefer to just use absolute paths to all the commands instead. I recommend against that. Consider what happens if you want to run your script on a different system, and on that system, the command is in /opt/someAppv2.2/bin instead. You’d have to go through the whole script replacing /opt/someApp/bin with /opt/someAppv2.2/bin instead of just doing a small edit on the first line of the script.
You can also set the PATH variable in the crontab file, which will apply to all cron jobs. E.g.
PATH=/opt/someApp/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

15 1 * * * backupscript –incremental /home /root

Interesting Command on Ubuntu – brings quotes with pics

#fortune -a | fmt -80 -s | cowsay -$(shuf -n 1 -e b d g p s t w y) -f $(shuf -n 1 -e $(cowsay -l | tail -n +2)) -n

#free -m

outputs – Buffered and cached

Buffers is for remembering the FS like dir structure (ls -R . will boost the buffr)

Cache is storing the file content, so processes can access file in cache instead read it over and over from the disk.

Buffers and cache are considered low priority, they are using large mem when none uses it. when apps using it, they will be decreased.

The -/+ buffers/cache line shows how much memory is used and free from the perspective of the applications. Generally speaking, if little swap is being used, memory usage isn’t impacting performance at all.
Notice that I have 512 MB of memory in my machine, but only 503 is listed as available by free. This is mainly because the kernel can’t be swapped out, so the memory it occupies could never be freed.
There may also be regions of memory reserved for/by the hardware for other purposes as well, depending on the system architecture.

#echo 3 > /proc/sys/vm/drop_caches
This command will clear the buffer and cache.

How to disable a response to PING ?
Add the following line to your init script for the network (the name depends on the distribution you use):

#echo 1 >/proc/sys/net/ipv4/icmp_echo_ignore_all
This disables ping responses.
To reenable, use the following command:

#echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_all
To make this permanent set the following into /etc/sysctl.conf (if you have such a file)
net.ipv4.conf.icmp_echo_ignore_all = 1

cat /proc/cpuinfo
check out the “flags”
This unit has considerably more flags, but the flag that denotes 64-bit CPUs is lm,
which stands for Long mode. We can also find out other information from this line
alone, like the CPU being produced by AMD because of the 3dnow flag—which
represents an AMD specific instruction set.

One Liner to Find CPU Arch from cpuinfo file:

[[ “$(grep lm /proc/cpuinfo)” ]] && echo 1 || echo 0

This bash specific one-liner will look for the string lm in /proc/cpuinfo and print

“1” if found, and “0” if not found. You can test it right away to see whether the

outcome is as expected.

Command To know the RAM Clockspeed
#dmidecode –type 17 | more

To list folder and file size

#du -hs /* | grep -v 0

Alternative Command to find Disk usage

#ncdu

To Save disk space

There’s 160GB of instance storage available on /mnt (for small instance), but you can just extend the partition into it in place, so you can do something that is very close.

I recommend you move over the directories that are filling up like this:

#cp -a /path/to/big /mnt/big
#mount --bind /mnt/big /path/to/big

Since Linux 2.4.0 it is possible to remount part of the file hierarchy somewhere else. The call is
#mount –bind olddir newdir
or fstab entry is:
/olddir /newdir none bind

After this call the same contents is accessible in two places.  One can also remount a single file (on a single file).

This call attaches only (part of) a single filesystem, not possible submounts. The entire file hierarchy including submounts is attached a second place using
mount –rbind olddir newdir
or shortoption
mount -R olddir newdir

Screen
#screen -S amibundle               – to get into a screen
#screen -ls                                    – to list screen
#screen -r [screenid]                 – to reattach screen
ctrl + A d                                        – to get out screen.
#screen -X -S [screenid] kill     – To terminate a screen session

To Share terminal using Screen:
1.SSH into the server
2.Start a screen , #screen -S test
3.Ask the other party to join the session by using the command, #screen -x test (the other party has to be on the same machine as well)

Command to know the Domain Registrar of a Domain – whois
#whois youtube.com
sample output :
Domain Name: youtube.com
Registrar Name: Markmonitor.com
Registrar Whois: whois.markmonitor.com
Registrar Homepage: http://www.markmonitor.com

Find out IP address and its Owner
#host -t a youtube.com
sample output:
youtube.com has address 74.125.236.166
youtube.com has address 74.125.236.160
youtube.com has address 74.125.236.163

To get IP Address Information
#whois 74.125.236.166

Top
c -to show absolute path of the running tasks
z – to highlight running jobs
k – to kill a process by providing its PID
r – to renice an process
shift + p – sort according to CPU utilization
shift + m – to sort memory wise

lsof
One of the main reason for using this command is when a disk cannot be unmounted and displays the error that files are being used or opened. With this commmand you can easily identify which files are in use.

To list large files in the filesystem
#find / -xdev -type f -size +100000k -exec ls -lh {} \;

To move old log files and save disk space
#find /var/log/ -mmin +180 -exec mv {} /tmp/date-dir/  \;
3hrx60min=180

To record terminal activities
script -t -a 2> /tmp/time.txt /tmp/reply.txt
run some commands then ..
scriptreplay /tmp/time.txt /tmp/reply.txt

Find average from a list in a file
awk ‘{s+=$1} END {print “Average: ” s/NR}’  filename

Change EC2 timezon to IST (Amazon Linux)
#mv /etc/localtime /etc/localtime.old
#ln -s /usr/share/zoneinfo/Asia/Kolkata /etc/localtime

Command to search the package which contains a specific command
#yum install yum-utlis
Example – partprobe
#repoquery -f */partprobe
Output:
parted-0:2.1-19.17.amzn1.x86_64
parted-0:2.1-19.17.amzn1.i686

Commands for Apache Access Log
Fetching the whole day logs – grep “19/Apr/2013” access.log
Fetching the hourly log – grep “19/Apr/2013:01” access.log
Fetching the whole day count – grep “19/Apr/2013” access.log | wc -l
Fetching the hourly count – grep “19/Apr/2013:01” access.log | wc -l
Fetching the IP Address count for whole day – grep “19/Apr/2013” access.log | awk ‘{print $1}’ | sort | uniq -c | sort -n
Fetching the IP Address count for hourly log – grep “19/Apr/2013:01” access.log | awk ‘{print $1}’ | sort | uniq -c | sort -n

Testing SMTP Server
#telnet smtp.sendgrid.net 25
>helo unnitest.com
>auth login
>usernam-base64 encoding
>password-base64 encoding
>mail from:unnisathya88@gmail.com
>rcpt to:unnisathya88@gmail.com
>data
>this is a test mail
>.

Get Website load time

 curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} \n" -s https://privatedock.wordpress.com

Transfer Files using netcat

nc -l -p 1234 | uncompress -c | tar xvfp -

And on the sending server run:

tar cfp - /some/dir | compress -c | nc -w 3 [destination] 1234

Now you can transfer directories without FTP,SCP and without needing root access.

Find and Replace a word using SED

find /home/apache/site-enabled -type f -exec sed -i 's/ugly.com/beautiful.com/g' {} \;

 

Find my Public IP

curl ifconfig.me

 

Unable to umount a filesystem:
we have 2 options
1.Use lazy umount option

#umount -l /example

This is not recommended because the file system does not actually unmount it instead it just removes the name from the hierarchy. Connections to the files by the process exist and there is no guarantee that existing references will ever close on their own.In addition the semi-mounted state can provide inconsistent filesystem semantics to the program that are using it, they can read and write through existing file handles but cannot open new files or perform other filesystem operations.
2.Use fuser before unmount [Ideal choice]

#fuser -cv /example
#kill -9 [pid]
#umount /example

This is safe and maintains a consistent filesystem.

STRACE

Any process that is monitored using strace should be ready for restarting it.

To find the files accessed by apache
Get the process ID : ps -ef | grep apache (take the PID of www-data instead of ROOT)
Use the command : strace -e file -p 17198

Trace the execution of a command

#strace ls

Trace a Specific System Calls in an Executable Using Option -e

#strace -e open ls
#strace -e trace=open,read ls /home

Save the Trace Execution to a File Using Option -o

strace -o output.txt ls

Print Timestamp for Each Trace Output Line Using Option -t

strace -t -e open ls /home

Generate Statistics Report of System Calls Using Option -c

strace -c ls /home

Find PHP config file

strace -e open php 2>&1 | grep php.ini

Installing FABRIC on AMAZON LINUX

# yum install gcc python-devel python-setuptools
# easy_install pip
# pip install fabric
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s