Ubuntu Server Utilities

SSH using key

ssh -i ~/.ssh/synology admin@192.168.2.1 "cd /volume1/inode/ocal;ls -la"

SSH - for remote command execution

Executing commands

ssh user@hostname "uname;hostname;date"
#This will execute those 3 commands

The following SSH command can be used to create a file remotely.

$ ssh user@hostname "cd ~/Desktop;touch myfile.txt"

This example will make a local copy of a remote /etc/passwd file to /tmp/passwd:

$ ssh user@username "cat /etc/passwd > /tmp/passwd"

This example will execute a script on the remote server. Of course, this will only work if the script already exists and has the proper executable permissions.

$ ssh user@hostname "cd ~/Desktop;./myscript.sh"

In this example we make a bzip2 local copy of the remote server's /var/log/auth.log file to a local file in the /tmp/ directory:

$ ssh user@hostname "cp /var/log/auth.log /tmp/; cd /tmp/ && tar -jcvf - auth.log  > /tmp/auth.tar.bz2"

Linux DIG Command Examples for DNS Lookup

dig redhat.com - dns lookup
dig redhat.com  MX - mx records lookup
dig redhat.com NS

View ALL DNS Records Types Using dig -t ANY

dig -t ANY redhat.com

DNS Reverse Look-up Using dig -x

dig -x 209.132.183.81

Use a Specific DNS server Using dig @dnsserver

dig @ns1.redhat.com redhat.com

Extract Images from PDF Files With PDFimages

apt-get install poppler-utils

pdfimages --help for options

eg

pdfimages -j filename.pdf prefixforallfiles

if the image doesn't convert to jpg. then use the mogrify to convert them. mogrify is part of imagemagick

mogrify -format jpg *.pcm // this will convert all *.pcm files to jpg

Imagemagic

Installation instructions

sudo apt-get install build-essential

Download ImageMagick Source Filesm

 wget https://www.imagemagick.org/download/ImageMagick.tar.gz

Once the download is complete, extract its content and move into the extracted directory.

 $ tar xvzf ImageMagick.tar.gz
 $ cd ImageMagick-7.0.8-26

mageMagick Compilation and Installation
Now it’s time to configure and compile ImageMagick by running the ./configure command to perform a compilation configuration.

 $./configure 
 $ make

Once the compilation is successful, install it and configure the dynamic linker run-time bindings as follows.

$ sudo make install 
$ sudo ldconfig /usr/local/lib

Finally, verify that ImageMagick 7 has been installed on your system by checking its version.

$ magick -version

OR

$ identify -version

Some ImageMagick commands

convert rose.jpg rose.png

convert rose.jpg -resize 64x64 thumb_rose.jpg

Only shrink larger images

convert rose.jpg -resize 64x64\> rose.jpg

Percentage Resize

convert rose.jpg -resize 50% rose.jpg

Rotate

convert rose.jpg -rotate 90 rose.jpg

Rotate and overite

mogrify -rotate 90 img.jpg

Batch Convert from one format to another

mogrify -format jpg *.png 

this will convert all png files to jpg

Using Rsync

rsync local mac to synology

rsync -zvah demamazon/ admin@192.1.1.1:/volume1/sharedfolder/directory

- z option is compresssed - v option is verbose - a archive // copy with permissions - h human readable

rsync options source destination

Rsync Local Dir to Amazon AWS servers

rsync azvr --delete --progress -e "ssh -i /fordertopemfile/pemfile.pem" localfoldertosync ubuntu@ipaddress:/var/tmp/

Synchronize Two Directories in a Local Server: To sync two directories in a local computer, use the following rsync -zvr command.

$ rsync -azvr /var/opt/installation/inventory/ /root/temp

building file list ... done sva.xml svB.xml

sent 26385 bytes received 1098 bytes 54966.00 bytes/sec total size is 44867 speedup is 1.63 $ In the above rsync example: -a archive mode (which preserves timestamp) -z is to enable compression -v verbose -r indicates recursive Synchronize Only One File: To copy only one file, specify the file name to rsync command, as shown below

$ rsync -v /var/lib/rpm/Pubkeys /root/temp/

Pubkeys

sent 42 bytes received 12380 bytes 3549.14 bytes/sec total size is 12288 speedup is 0.99

Rsync Over SSH

With rsync, we can use SSH (Secure Shell) for data transfer, using SSH protocol while transferring our data you can be ensured that your data is being transferred in a secured connection with encryption so that nobody can read your data while it is being transferred over the wire on the internet. Also when we use rsync we need to provide the user/root password to accomplish that particular task, so using SSH option will send your logins in an encrypted manner so that your password will be safe.

Copy a File from a Remote Server to a Local Server with SSH

To specify a protocol with rsync you need to give “-e” option with protocol name you want to use. Here in this example, We will be using “ssh” with “-e” option and perform data transfer.

rsync -avzhe ssh root@192.168.0.100:/root/install.log /tmp/

root@192.168.0.100's password:

receiving incremental file list install.log sent 30 bytes received 8.12K bytes 1.48K bytes/sec total size is 30.74K speedup is 3.77>/dd>

Copy a File from a Local Server to a Remote Server with SSH

rsync -avzhe ssh backup.tar root@192.168.0.100:/backups/

root@192.168.0.100's password:

sending incremental file list backup.tar sent 14.71M bytes received 31 bytes 1.28M bytes/sec total size is 16.18M speedup is 1.10

Use of –include and –exclude Options

These two options allows us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred. Here in this example, rsync command will include those files and directory only which starts with ‘R’ and exclude all other files and directory.

rsync -avze ssh --include 'R*' --exclude '*' root@192.168.0.101:/var/lib/rpm/ /root/rpm
root@192.168.0.101's password:

receiving incremental file list created directory /root/rpm

Vi Text Editor

Main command mode commands

i puts you in insert mode where you can enter text.

ESC key takes you out of insert mode.

w -> go to the beginning of the next word
b -> go to the beginning of the previous word
e -> go to the end of the next word
n| -> go to the 'n' columns of the current row
$ -> go to the end of the current row
^ -> go to the first character of the current row
G -> go to the end of the file
gg -> go to the beginning of the file
Gn -> go to 'n' row of the file
i -> insert mode (before the cursor)
a -> insert mode (after the cursor)
o -> insert mode (under the current row)
x -> delete one character
nx -> delete 'n' characters
dd -> delete the whole row
ndd -> delete 'n' rows
yy -> copy the current row
p -> paste the copied row
Rmystring -> replace next characters with the string 'mystring'
/string -> find the next string 'string'
?string -> find the string 'string' backwards
n -> repeat the search again
u -> undo the last command

Main row editing mode commands

w -> write the file

w myfile -> write the file as 'myfile'

q -> quit the file

wq -> write and quit the file

q! -> quit without saving the file

help -> open the help file (:q to return editing the file)

help topic -> open the help file showing the subject 'topic' (:q to return editing the file)

rs/string1/string2/options -> replace string1 with string2 withing the 'r' range; options may be: g (replace all), c (ask before any replacement)

set number -> set rows numbering

* to move down:press j
* to move up:press k
* to move to the left:press h
* to move to the right:press l

You will have to tap each key for the number of lines or characters you want the cursor to move. In this case, I want to move down seven lines and then to the right. So I would hit j seven times, and hold down the l key until I got to the word “friend.”

Not every file you’re going to work in will be so small; some will be a couple hundred lines long. If you look at the last line of our file, you’ll see that the editor tells you how many lines there are in the file. If you want to jump down, you can type :(line number), where the line number is the actual numeral. So if I wanted to jump down to line 16 (the line beginning with “q”), I’d type:

16

hit Enter, and there I’d be.

Deleting Text

You can delete text one character at a time simply by pressing x. The character that gets deleted will be the one the cursor highlights.

If that seems too tedious, you can type dw, and delete the text a word at a time. Note that vi counts the spaces in between words as words too, so I’d have to type dw nine times to delete the line.

Or you can just wipe out the entire line of text. To do that, just type dd. Since this is the fastest way to do it, I’m going to type:

dd

and the line that my cursor sits in is gone.

Replacing text

type r (this tells the editor I want to replace the highlighted character). If I wanted to change the letter c to s, I’d type r, then s. If I wanted to change it to – well, you get the picture:Type r and then the replacing character.

Adding Text

So far, we’ve been working in edit mode. In this mode, we can move around the file with impunity, deleting or replacing existing text without worry. To add text, we have to switch to the insert mode.Fortunately, this isn’t hard:Just move the cursor to the spot in the document you want to edit, hit i, and begin typing.

The first thing you should know is that vi takes the insert command rather literally. Wherever your cursor is when you hit i will be the first point at which you can begin typing. So if you’re looking at a sentence like this:

w means to overwrite the last version

and the cursor is on the v in ” overwrite,” you’re going to start inserting text where the v used to be. So if you try to type the words “saved file” you’ll end up with a sentence that looks like this:

w means to saved file overwrite the last version

The easiest way around it is to position the cursor where you want it before you start typing. So, if I wanted to insert the phrase “saved file” after the word last, I would hit Escape to make sure I’m out of insert mode, move the cursor to the space between “last” and “version,” hit i for insert, and type in a space and then ” saved file.”

Note:You cannot move around using the h, j, k, and l keys, or edit anything, while you’re in insert mode! To snap out of insert mode and back into edit mode, hit the Escape key. Any time I’m in vi and forget which mode I’m in (it can happen), I hit Escape before I do anything else.

Undoing Mistakes

So let’s say you haven’t quite got the hang of inserting text and you’ve just typed something very wrong. Will you have to delete all the text and start over?

Nope – if you catch your errors quickly (like, right after you make them), you can always undo them. To undo any inserted text, hit the Escape key, then type u. It will undo the last action you did.

If you’re already in edit mode, just type u to undo your last action.

Webmin

 Webmin is a web-based interface for system administration for Unix. Using any modern web browser, you can setup user accounts, Apache, DNS, file sharing and much more. Webmin removes the need to manually edit Unix configuration files like /etc/passwd, and lets you manage a system from the console or remotely

 http://www.webmin.com/

After Installation: https://localhost:10000/ 

username root with your current password

Log watch

Logwatch is a modular log analyser that runs every night and mails you the results. It can also be run from command line.The output is by service and you can limit the output to one particular service. The subscripts which are responsible for the output, mostly convert the raw log lines in structured format. Logwatch generally ignores the time component in the output, that means, you will know that the reported event was logged in the requested range of time, but you will have to go to the raw log files to get the exact details. Install logwatch using the following comamnd

sudo apt-get install logwatch

This will install all the required packages

Configuring Logwatch

First you need to make sure you server is able to send the mails outside you can do this using postfix with SMTP server configuration

Now you need to edit logwatch.conf file

sudo nano /usr/share/logwatch/default.conf/logwatch.conf

Change the following information

Output = mail
Format = html
MailTo = test@gmail.com

Save and exit the file

If you want to know more available options check logcheck manpage using the following command from your terminal

man logcheck

Now edit the 00logwatch file

sudo nano /etc/cron.daily/00logwatch

add the following line

/usr/sbin/logwatch --mailto test@gmail.com

Save and exit the file

Using Tar Linux Shell Command

First we'll start with 'tar'. This program assembles various files into one package, commonly called a "tarball". Let's say you have some files - notes that you've taken during this course.

You have:

notes_1.txt notes_2.txt notes_3.txt notes_4.txt notes_5.txt

and you've placed them in a directory called /linux_course. You want to back them up and keep them on a floppy, let's say. You would type the following command to package them in a tarball.

tar -cvf linux_notes.tar notes*.txt

First, you have tar, the name of the program. Then you have the options, c (--create) v (--verbose-show what files they are) (f--file -make a file - should always be the last option) Then you have the name of the file you want to create ( linux_notes.tar) and the files you want to backup (notes*.txt).

This presupposes that you may have other files in the directory that you don't want to include. If you want to include ALL files in a directory, just substitute notes*.txt for *.*.

back up whole directories along with their corresponding subdirectories. Enter in the directory, let's say /home/bob/ and issue the command:

tar -cvf bob_backup.tar *

With one asterisk, you will include directories and files without extensions (my_file as opposed to my_file.txt). Be prepared to get a fairly voluminous tarball.

This is the first step in the backup process. Now let's look at the second step; the compression of these files.

Using 'gzip'

As we mentioned, 'tar' just assembles the files together into only one file. There is no reduction in the size of these files (the tarball might even be bigger!) Now we would have to do one more thing in order to reduce this file into a more manageable size: use 'gzip'.

gzip is the preferred compression tool for Linux. To reduce the size of your tar file, you would issue the following command:

gzip your_tar_file.tar

and the tar file would be compressed. You can also compress a regular file using the same command, but gzip is used primarily with tarballs.

The result would be a file like this: your_tar_file.tar.gz

The two file extensions show us that the file is a tarball and it is compressed with the 'gzip' format.

Putting it all together

tar

tar has an option built into it to use 'gzip' to zip the file at the same time you make the tarball. If you add z to the options, and change the name of the file to create to a .gz extension, you have the whole shebang in one step. Our previous example would be modified to this:

tar -czvf bob_backup.tar.gz *

Remember f should always be the last option.

UnTar

Using 'tar' and 'gzip' sort of supposes that you're going to want to "untar" and "unzip" these files at one point or another.

The easiest way for doing this is to use 'tar' for the whole process. You would locate the zipped tarball in question and then ask yourself a question:

Did I make any changes to the files inside the tarball after I made it? If you did, then you've got an old tarball. If you untarred it in the same directory, you'd overwrite the existing ones. If you would like a copy of the old file, untar it in a different directory. If you don't want the old files, then you should make a new tarball. It's pretty standard backup practice.

When you've decided what you want to do, to proceed with the "untarring", issue this command:

tar -zxvpf my_tar_file.tar.gz

I've used my preferred options. I'll explain them:

-z - unzip the file first -x - extract the files from the tarball -v - "verbose" (i.e tar tells you what files it's extracting) -p - preserves dates, permissions of the original files -f - use the file in question (if you don't specify this, tar just sort of sits around doing nothing)

The files are extracted and your original tarball is preserved (my_tar_file.tar.gz).

You can also untar the file and then use gzip separately. Just leave the z option out of the previous example and type:

gzip -d my_tar_file.tar.gz or

gunzip my_tar_file.tar.gz

(gunzip runs gzip -d "automagically"!)

These commands are good if you've just zipped a regular file (not a tarball).

ssh with key, run remote scripts