‘m a new Linux system user. How can I redirect command error output /dev/null on a Linux or Unix-like system using Bash shell?

Your shell comes with three file descriptors as follows:

stdin – 0 – Standard Input (usually keyboard or file)

stdout – 1 – Standard Output (usually screen)

stderr – 2 – Standard Error (usually screen)

What is a null (/dev/null) file in a Linux or Unix-like systems?

/dev/null is nothing but a special file that discards all data written to it. The length of the null device is always zero. In this example, first, send output of date command to the screen and later to the /dev/null i.e. discards date command output:

Show on screen ###

date

Discards date command output ###

date > /dev/null

Syntax: Standard Error (stderr -2 no) to a file or /dev/null

The syntax is as follows:

command 2>/dev/null

command arg1 arg2 2>/dev/null

date bar 2>/dev/null

ls -foo 2>/dev/null

In this example, send output of find command to /dev/null:

$ find /etc -type f -name  *  2>/dev/null

The following example will cause the stderr ouput of a program to be written to a file called errors.txt:

$ find /etc/ -type f -name  *  2> errors.txt

Linux and Unix redirect all output and error to file

The syntax is:

send command output to output.txt and error message to error.txt ##

command > output.txt 2> error.txt

command  -arg1 -arg2 > output.txt 2> error.txt

If you want both stderr and stdout in same file, try:

command > log.txt 2>&1

Use cat command to display log.txt on screen:

cat log.txt

See man pages for more information – bash(1),ksh(1).

foo

bar

foobar

How can I delete all leading and/or trailing blank spaces, tab from each line using sed command?

You can use sed command to delete all white (blank) spaces in a text file. You can also use other text processing utilities such as:

Perl.

Python.

Awk and friends.

Perl example

The syntax is:

perl -lape  s/\s+//sg  input > output

Sample outputs:

foo

bar

foobar

Or use -pie syntax to update file:

cat input

perl -lapi -e  s/\s+|^\n//sg   input

cat input

See perl(1) for more information.

Sed example

The syntax is:

sed -e  s/^[ \t]*//  -e  s/[ \t]*$//  input > output

OR updated file in a single go with the -i option:

sed -i -e  s/^[ \t]*//  -e  s/[ \t]*$//  input

See sed(1) for more information.

Awk example

The syntax is

awk  {$1=$1}{ print }  input > output

You can also use gsub() substring matching the regular expression function. See awk(1) for more information.

Xcode includes command line development tools such as gcc and friends.

Step #1: Install Xcode on a Apple Mac OS X

First, make sure Xcode is installed. If it is not installed on OS X, visit app store and install Xcode.

Fig.01: Make sure Xcode developer tools are install OS X

Step #2: Install gcc/LLVM compiler on OS X

Once installed, open Xcode and visit:

Xcode menu > Preferences > Downloads > choose  Command line tools  > Click  Install  button:

Fig.02: Installing gcc compiler on Mac OS X

Xcode will download package and install copies of the core command line tools and system headers into system folders, including the LLVM compiler, linker, and build tools.

Step #3: Verification

Open a terminal app and type the following commands:

$ gcc –version

$ whereis gcc

$ whereis make

Sample outputs:

Fig.03: Verify gcc compiler installation on Mountain Lion OS X

Testing sample “Hello world” C program

Create a text file called a.c as follows using a text editor such as vi or cat command:

/* a.c – demo for os x */

#include<stdio.h>

int main(void){

printf( Hello world\n );

return 0;

}

To compile, enter:

$ make a

Run it as follows:

$ ./a

Sample outputs:

Fig.04: Compiling and running sample “Hello world” C program on Mountain Lion 10.8.4

See also

And, there you have it, the gcc version 4.2.1 installed and working correctly on the latest version of Mac OS X 10.8.4. In Apple’s version of GCC, both cc and gcc are actually symbolic links to the llvm-gcc compiler. Similarly, c++ and g++ are links to llvm-g++. For more information and examples see the following man pages:

$ gcc(1)

$ make(1)

This entry is 5 of 13 in the Linux GNU/GCC Compilers Tutorial series. Keep reading the rest of the series:

Ubuntu Linux Install GNU GCC Compiler and Development Environment

Debian Linux Install GNU GCC Compiler and Development Environment

CentOS / RHEL 7: Install GCC (C and C++ Compiler) and Development Tools

Download and Install C, C++ Compiler on Red Hat Enterprise Linux 5 (RHEL)

Mac OS X: Install GCC Compiler with Xcode

Where is My Linux GNU C or GCC Compilers Are Installed?

HowTo: Compile And Run a C/C++ Code In Linux

RHEL / CentOS Linux Install Core Development Tools Automake, Gcc (C/C++), Perl, Python & Debuggers

HowTo Compiling C Program And Creating Executable File Under a Linux / UNIX / *BSD

How To Install ncurses Library on a Linux

Linux Find Out What Compilers Are Installed or Available On The System

Linux Find Out GNU gcc Compiler Version Used To Compile Running Kernel

Howto see output of C program in Linux or UNIX

You need to use the wakeonlan Perl script that generates and transmits a Wake-On-LAN (WOL) “Magic Packet”, used for restarting machines that have been soft powered-down (ACPI D3-warm state).

Method #1: Install wakeonlan using Homebrew

Open the Terminal app and type the following command:

brew install wakeonlan

Sample outputs:

Fig.01: OS X brew install wakeonlan client

Method #2: Download and install wakeonlan Perl script

Open a terminal and type the following curl command:

$ mkdir -p $HOME/bin

$ curl https://raw.githubusercontent.com/jpoliv/wakeonlan/master/wakeonlan -o ~/bin/wakeonlan

$ chmod +x ~/bin/wakeonlan

How do I send WOL on a OS X?

The syntax is:

$ ~/bin/wakeonlan server-mac-address-here

For example, if nas01 server has 00:08:9b:c4:30:30 mac address, enter:

$ ~/bin/wakeonlan 00:08:9b:c4:30:30

Sample outputs:

Sending magic packet to 255.255.255.255:9 with 00:08:9b:c4:30:30

Other options

-i ip_address

set the destination IP address

default: 255.255.255.255 (the limited broadcast address)

-p port

set the destination port

default: 9 (the discard port)

-f file

uses file as a source of hardware addresses

Apple computer wake for network access (WOL) setting

If you want other users to be able to access your Apple OS X based computerâ€s shared resources, such as shared printers/files/folders or iTunes playlists, even when your computer is in sleep mode. Open System Preferences > choose “Energy Saver preferences“. This set options that control your computerâ€s energy use including WOL for all Mac Based server and client systems:

Fig.01: OS X setting WOL

Make sure you select the option “Wake for network access” so that other users can wake up your computer using WOL magic packet.

References

wakeonlan(1) for more information.

Download Wakeonlan: Perl script for waking up computers via Wake-On-LAN magic packets

HowTo: Wake Up Computers Using Linux Command [ Wake-on-LAN ( WOL ) ]

WI-FI allows an electronic device such as Apple laptop, ipad, iPhone and other iDevices to exchange data or connect to the internet wirelessly using radio waves. You can see the current WI-FI speed using any one of the following tools in Mac OS X:

Network Utility.

System Information.

Wireless Network Preferences Menu Bar Icon Shortcut.

airport Command Line Utility.

Method #1: Checking the wireless connection speed using network utility

First, click on Spotlight and type “network utility“. OR press command + space-bar and type “network utility“.

Fig.01: Finding network utility

Make sure you select network utility. Select Info tab and choose Wi-fi (en0 or en1) on the drop-down options. Please note that the Link Speed is your wireless connection speed. In this example, my connection speed is set to 450 Mbit/s (Mega bits per seconds).

Fig.02: Finding the current wireless connection speed using network utility. The WI-FI speed will be displayed as the Link Speed.

Method #2: Checking the wireless connection speed using system information

Click on Spotlight and tye “system information“. OR press cmd+space-bar and then type “system information“. Finally, choose Network and select Wi-Fi. The WI-FI connection information will be displayed on the right.

Fig.03: Finding My Mac Mini’s WI-FI speed using system information. Please note that you may need to scroll down the right side to see all info.

Method #3: Finding wireless status using option key

First, hold down the Option key.

Next, Click on Wireless icon (Airport icon) and you will see information as follows:

Fig.04: Finding extended information about your wi-fi connection

Method #4: Command line utility

Open a terminal. Type the following command:

/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -I

Sample outputs:

agrCtlRSSI: -62

agrExtRSSI: 0

agrCtlNoise: -86

agrExtNoise: 0

state: running

op mode: station

lastTxRate: 150

maxRate: 450

lastAssocStatus: 0

802.11 auth: open

link auth: wpa2-psk

BSSID: 50:46:5d:d2:36:8c

SSID: very5_G

MCS: 7

channel: 161,-1

I strongly recommend that you use network utility for ease of use.

grep ‘foo’ 2012-2013.txt | awk ‘BEGIN{ sum=0.0}{ sub(“,”,””,$7); sum +=$7}END{ print “$” sum}’

$682444

I want $682444.57 as output. How can I force “awk” to do floating point math?

Floating-point numbers or “real” numbers are that have a fractional part. awk uses double-precision floating-point numbers to represent all numeric values. In other words, all numbers in awk are floating-point numbers i.e. all calculation done using floating-point numbers.

Example: Awk floating point calculation

The following example uses a file called trade.txt, which contains a list of week names as well as four profit values per week:

week1 12.5 12.5 13.5 18.5

week2 11.5 11.10 12.10 13.70

week3 8.5  8.10 8.5 12.5

week4 9.5 11.5 13.5 16.5

week5 8 7 13 17

The following awk program takes the file trade.txt and prints the sum of all four values:

awk  { sum = $2 + $3 + $4 + $5; print $1, sum }  trade.txt

Sample outputs:

week1 57

week2 48.4

week3 37.6

week4 51

week5 45

The following awk program takes the file trade.txt and prints the average of all four values:

awk  { sum = $2 + $3 + $4 + $5; avg = sum/4; print $1, sum , avg}  trade.txt

Sample outputs:

week1 57 14.25

week2 48.4 12.1

week3 37.6 9.4

week4 51 12.75

week5 45 11.25

To avoid surprises use printf to format text to make your output more beautiful and meaningful:

awk  { sum = $2 + $3 + $4 + $5; avg = sum/4; printf  %s: $%.2f ($%05.2f)\n ,$1, sum, avg}  trade.txt

OR

awk  { sum = $2 + $3 + $4 + $5; avg = sum/4; printf  %s: $%.2f ($%5.2f)\n ,$1, sum, avg}  trade.txt

Sample outputs:

week1: $57.00 ($14.25)

week2: $48.40 ($12.10)

week3: $37.60 ($ 9.40)

week4: $51.00 ($12.75)

week5: $45.00 ($11.25)

To fix your problem, replace the following awk code

grep  foo  2012-2013.txt  | awk  BEGIN{ sum=0.0}{ sub( , ,  ,$7); sum +=$7}END{ print  $  sum}

with:

grep  foo  2012-2013.txt  | awk  BEGIN{ sum=0.0}{ sub( , ,  ,$7); sum +=$7}END{  printf  $%.2f\n , sum}

You can skip the grep command and use awk as follows to match and perform sum of all $7:

awk  /foo/{ sub( , ,  ,$7); sum = old + $7; old=sum}END{ printf  $%.2f\n , sum}  2013-2014.txt

Recommended readings

See awk(1) for more info.

How do I redirect all mobile users to sub-domain using regex based rules?

And How can I automatically redirects visitors on mobile devices to its mobile version at http://m.example.com/ and also allow mobile devices to the desktop website at www.example.com if visiting via http://www.example.com/?desktop=true?

You can easily redirect all mobile users using nginx as follows:

m.example.com – Mobile domain name. Make sure app is configured to display same page as served on www.example.com

All desktop clients need to use www.example.com. However, m.example.com i.e. all mobile phone user can browser desktop version if visiting via www.example.com/?desktop=true

Make sure line Disallow: /*? added to /robots.txt on www.example.com

/robots.txt – It is a good idea to block all bots on m.example.com. This ensures that mobile users will see lightweight page; but all bots refer to your main site. This is SEO feature. Sample /robots.txt for m.example.com

User-agent: *

Disallow: /

From the robots.org page:

The “User-agent: *” means this section applies to all robots. The “Disallow: /” tells the robot that it should not visit any pages on the site. Bad robots can ignore your /robots.txt.

Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention. However, good robots such as Googlebot will follow your /robots.txt file.

Sample /robots.txt for www.example.com:

User-agent: *

Disallow: /*?

Nginx configurations

Edit the nginx.conf file and append the following after server directive:

set $mobile_rewrite do_not_perform;

chi http_user_agent for mobile / smart phones ##

if ($http_user_agent ~*  (android|bb\d+|meego).+mobile|avantgo|bada\/|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|ip(hone|od)|iris|kindle|lge |maemo|midp|mmp|netfront|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\/|plucker|pocket|psp|series(4|6)0|symbian|treo|up\.(browser|link)|vodafone|wap|windows (ce|phone)|xda|xiino ) {

set $mobile_rewrite perform;

}

if ($http_user_agent ~*  ^(1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\-(n|u)|c55\/|capi|ccwa|cdm\-|cell|chtm|cldc|cmd\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\-s|devi|dica|dmob|do(c|p)o|ds(12|\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\-|_)|g1 u|g560|gene|gf\-5|g\-mo|go(\.w|od)|gr(ad|un)|haie|hcit|hd\-(m|p|t)|hei\-|hi(pt|ta)|hp( i|ip)|hs\-c|ht(c(\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\-(20|go|ma)|i230|iac( |\-|\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\/)|klon|kpt |kwc\-|kyo(c|k)|le(no|xi)|lg( g|\/(k|l|u)|50|54|\-[a-w])|libw|lynx|m1\-w|m3ga|m50\/|ma(te|ui|xo)|mc(01|21|ca)|m\-cr|me(rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\-2|po(ck|rt|se)|prox|psio|pt\-g|qa\-a|qc(07|12|21|32|60|\-[2-7]|i\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\-|oo|p\-)|sdk\/|se(c(\-|0|1)|47|mc|nd|ri)|sgh\-|shar|sie(\-|m)|sk\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\-|v\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\-|tdg\-|tel(i|m)|tim\-|t\-mo|to(pl|sh)|ts(70|m\-|m3|m5)|tx\-9|up(\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|yas\-|your|zeto|zte\-) ) {

set $mobile_rewrite perform;

}

redirect to m.example.com ##

if ($mobile_rewrite = perform) {

rewrite ^ http://m.example.com$request_uri? redirect;

break;

}

Adding exceptions

You can allow user to browse and view desktop version of your site if url has www.example.com/?desktop=true. You can set cookie as follows:

set $force_dt_cookie    ;

if ($args ~  desktop=true ) {

set $mobile_rewrite do_not_perform;

set $force_dt_cookie   desktop=true ;

}

add_header Set-Cookie $force_dt_cookie;

if ($http_cookie ~  desktop=true ) {

set $mobile_rewrite do_not_perform;

}

Save and close the file. Restart or reload the nginx server, enter:

# /usr/sbin/nginx -s reload

OR

# /etc/init.d/nginx reload

Test it

Use the curl command as follows to see redirection:

curl -I -A  UserAgentString  http://www.example.com

curl -I -A  Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543a Safari/419.3  http://www.example.com

curl -I -A  Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543a Safari/419.3   http://www.example.com/?desktop=true

References

Mobile browser detection with Nginx.

Mobile redirection with Nginx.

an you give me a simple loop example in csh shell in Linux or Unix like operating systems?

The C shell (csh) or the improved version, tcsh is a Unix shell that was originally created by Bill Joy at University of California, Berkeley in the late 1970s.

Syntax

The syntax is as follows:

while ( condition )

# do something

# command 1

# command 2

end

OR

set i = 1

while ( i < 5 ) # do something till i < 5 # command 1 # command 2 @ i++ end

OR

foreach n ( 1 2 3 4 5 )

#command1

#command2

end

Examples

The following csh code will print welcome message five times on screen:

#!/bin/csh

# demoloop.csh – Sample loop script

set j = 1

while ( $j <= 5 )

echo  Welcome $j times

@ j++

end

Save and close the file. Run it as follows:

chmod +x demoloop.csh

./demoloop.csh

Sample outputs:

Welcome 1 times

Welcome 2 times

Welcome 3 times

Welcome 4 times

Welcome 5 times

csh foreach example

#!/bin/csh

echo  Setting name servers….

foreach i ( ns1.cyberciti.biz ns2.cyberciti.biz  )

echo $i

end

Sample outputs:

Setting name servers….

ns1.cyberciti.biz

ns2.cyberciti.biz

You can use wild card with foreach as follows:

#!/bin/csh

foreach i (*)

if (-f $i) then

echo  $i is a file.

endif

if (-d $i) then

echo  $i is a directory.

endif

end

Sample outputs:

mycal.pl is a file.

skl is a directory.

x is a file.

x.pl is a file.

y is a file.

Please note that csh was popular for many innovative features but csh has never been as popular for scripting. If you are writing system level rc scripts avoid using csh. You may want to use /bin/sh for any scripts that might have to run on other systems.

Recommended readings

tcsh(1)

You need to use the kldload command. It loads file.ko (kernel module or device drive) into the kernel using the kernel linker. The syntax is:

kldload module-name

OR

kldload module-name-1 module-name-2

OR

kldload [option] module-name

Please note that if multiple modules are specified then an attempt will be made to load them all, even if some fail. The .ko extension name is not mandatory when loading a given module using kldload. It does not hurt to specify it though.

/boot/kernel directory

/boot/kernel# ls -l /boot/kernel/

# ls -l /boot/kernel/ | more

Examples

Open a terminal or login using ssh. You must login as root user:

# kldload foo

# kldload drm

To load by file name within the module path:

# kldload drm.ko

To load by relative path in the current dir:

# kldload ./name.ko

To load by full path:

# kldload /boot/kernel/drm.ko

To see all loaded modules, enter:

# kldstat

To remove or unload module, enter:

# kldunload drm

ow can I fetch HTML web page content from bash and display on screen using shell utilities?

You can use any one of the following tool or combination of all of them to get the contents of a webpage in a shell:

[1] curl command – It is a tool to transfer data from or to a server using http/https/ftp and much more.

[2] lynx command – It is a fully-featured World Wide Web (WWW) client/browser for users running terminals.

[3] wget command – It is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

[4] w3m command – It is a text based Web browser and pager.

Installation

Above tools may not be installed on your Linux or Unix like operating systems.

Note: You need to login as root user to install required tools.

Debian / Ubuntu Linux install curl, wget, lynx, and w3m

Open a terminal and and then type:

$ sudo apt-get install curl wget lynx w3m

Fedora / RHEL / CentOS Linux install curl, wget, lynx, and w3m

Open a terminal and and then type:

$ sudo yum install curl wget lynx w3m

FreeBSD Unix install curl, wget, lynx, and w3m (binary package)

Open a terminal and and then type:

$ sudo pkg_add -v -r curl lynx w3m wget

Examples

You can use curl command to download the page:

curl http://www.cyberciti.biz/

curl http://www.cyberciti.biz/faq/bash-for-loop/

Use curl and store output into a variable as follows:

page= $(curl http://www.cyberciti.biz/)

page= $(curl http://www.cyberciti.biz/faq/bash-for-loop/)

To display content use echo or printf command as follows:

echo  $page

printf  %s  $page

lynx command examples

Use the lynx command as follows:

lynx -dump www.cyberciti.biz

lynx -dump www.cyberciti.biz/faq/bash-for-loop/

The -dump option dumps the formatted output of the default document or those specified on the command line to standard output. Unlike interactive mode, all documents are processed.

wget command examples

The syntax is as follows:

wget -O – http://www.cyberciti.biz

wget -O – http://www.cyberciti.biz/faq/bash-for-loop/

OR use wget command to grab the page and store it into a variable called page:

page= $(wget -O – http://www.cyberciti.biz)

display page ##

echo  $page

or pass it to lynx / w3m ##

echo  $page  | w3m -dump -T text/html

echo  $page  | lynx -dump -stdin

w3m command examples

The syntax is as follows to dump web page content in terminal using the w3m command:

w3m -dump http://www.cyberciti.biz/

w3m -dump http://www.cyberciti.biz/faq/bash-for-loop/

OR use w3m command to grab the page and store it into a variable called page:

page= $(w3m -dump http://www.cyberciti.biz/)

echo  $page

Practical examples

Get the definition of linux from a dictionary:

$ curl dict://dict.org/d:linux

Sample outputs:

220 pan.alephnull.com dictd 1.12.0/rf on Linux 3.0.0-14-server  <21853866.27331.1375614736@pan.alephnull.com>

250 ok

150 1 definitions retrieved

151  linux  wn  WordNet (r) 3.0 (2006)

Linux

n 1: an open-source version of the UNIX operating system

.

250 ok [d/m/c = 1/0/30; 0.000r 0.000u 0.000s]

221 bye [d/m/c = 0/0/0; 0.000r 0.000u 0.000s]

Backup your del.icio.us bookmarks:

$ wget –user=Your-Username-Here –password=Your-Password-Here https://api.del.icio.us/v1/posts/all -O my-old-bookmarks.xml

$ more my-old-bookmarks.xml

Grab all .mp3s from url:

mp3=$(lynx -dump http://server1.cyberciti.biz/media/index.html  | grep  http://  | awk  /mp3/{print $2} )

for i in $mp3

wget $i

done

See also

See man pages for more info – curl(1),w3m(1),lynx(1),wget(1)