Unlearning Windows

Here's a collection of information necessary for rudimentary interaction with the commandline (assuming bash).

What's a shell

A shell is a program specifically designed to be a command line interface into the operating system, as well as an interface for complex programs to be written that are composed of multiple smaller programs. The shell essentially acts as the glue that allows a unix-like operating system to string multiple unrelated programs together to accomplish tasks (remember, everything is a file, afterall).

Traditionally, boot systems have been created using the shell, as well as small scripts, and huge complex systems.

See here to learn a bit more: http://www.lispcast.com/the-heart-of-unix

Common Notation

Most documents online will use the notation
$ command1 ...........
to indicate a command being run as a user account, and
# command1 ...........
to indicate a command being run as a superuser account.

If you see a command online that you know you want to run (but do be careful to understand what it'll do before you run it...), and it has a # in front of it, the person telling you to run it probably intends for you to run it as root.

Along the same lines, if you see a # on a command prompt, it probably means that you're logged in as root.

For example: ITDA-PC-41L ~ #
Versus: m_jones@ITDA-PC-41L ~ $

Home

Each user account on a linux system has a specific directory in the filesystem tree that is that user's home directory. The directory can be any location, and it doesn't have to be unique to the user account in question. Typically this will be in /home/USERNAME, but for some services it might be somewhere under the /usr directory. In the terminal, in many instances, you can represent the current home directory with the tilde symbol (~). For instance, you might wnt to list all of the files in your home directory when your current working directory is somewhere else. To do so you only need to type "ls -lah ~" and the shell you are using will automatically replace ~ with your actual home directory before passing it to the ls program.

History

All of the commands typed into the terminal are stored in the history. The file that stores this history is in your user account's home directory, if you use the BASH shell, this file is named ".bash_history". Other shells will use different naming schemes.

Standard Inputs and Outputs

Each program can have multiple input and output streams. In the linux world, everything is a file / stream of bytes, and so each program receives information through streams of bytes, and each program exports information through streams of bytes. The standard behavior of newly launched programs is to be provided 3 active streams to communicate through.

The standard input stream is automatically set up by the linux environment, and in the case of launching a program from a shell (instead of from another custom program), the standard input of the newly launched program is linked to the user's typing in the shell. Any characters typed into the shell after the program is launched (But before termination) will be available for the program to read and act on.

The standard output stream, like the standard input, is automatically set up by the linux environment. When launching from a shell, the standard output of a program is automatically forwarded to the shell, and then displayed on the screen. Typically programs that in turn call other programs will have the output of the child program forwarded as well (though, this isn't necessarily automatic, and more of a convenience provided by the program author).

Standard error behaves almost identically to standard output, and by convention is separated to allow a calling program to identify whether the program output is an error or simply information by checking which output stream it came from.

See more info here: http://www.linfo.org/stdio.html

Redirection

A logical consequence of having standard inputs and outputs is the desire to be able to choose a difference source or destination for those tools than what you type or read. Maybe you already have the desired input written out in a file. Maybe you want to save the contents for viewing later. Either way, redirecting the standard input or output is fairly easy.

Redirecting standard input

command < inputfile.txt

Redirecting standard ouput

command > outputfile.txt

Redirecting standard error

command 2> errorfile.txt

Merging standard output with standard error

command > mergedoutputfile.txt 2>&1
Alternatively,
command 2> mergedoutputfile.txt >&2

Telling linux to ignore output

command > /dev/null

Giving a command a random input

command < /dev/random

Pipes

Ok, so we have programs that accept input from standard in, and report results on standard out. What if we want a program to act on the results from another command? One way would be to have the first program print the results to a file, and then pass that file as the input to a second program. But there is a more efficient way to do that called a pipe.

The | character (Shift \, above the enter key on most keyboards) connects the standard output of the program to it's left directly to the standard input of the program on it's right.

For instance, maybe the list of files in a directory is huge, thousands of files and folders. The results of the ls command might be too big to be easily readable on the screen. I can redirect the result of ls to a program designed to make it easy to read larges amounts of text on the terminal "less" like this

ls -lah | less

Or maybe I want to print a file to the screen, filter out all lines that have the word "bob" in them, and then sort the remaining lines alphabetically before putting them in an easy reading program?

cat inputfile.txt | grep -v bob | sort | less

Boolean Logic and Conditional Execution

This is a tricky subject because bash has such a difficult syntax. The meaning of checking for success or failure changes depending on where you're asking for a boolean check.

But here are some basic examples:

Logical And

command1 && command2 && command3

If and only if command1 reports success, then command2 will be run. If and only if command1 and command2 report success, then command3 will be run.

Logical OR

command1 || command2 || command3

(Note that that's the same character as the pipe operator. Tread carefully not to mistakenly put the wrong number of them).

If and only if command1 returns false, execute command2. If and only if command1 AND command2 return false, execute command3.

So, it might be more descriptive to call this the logical XOR, since the second command will be executed only if the first fails.

Part of the reason for this behavior is that you're asking for the shell to evaluate a logical expression. It obviously needs to run the first command to determine the true/false value of it, but if it returns true, evaluating the other 2 commands aren't necessary, because TRUE || anything || anything is always true. So as a shortcut it skips executing the other commands.

Executing commands in sequence

If you don't care about the success or failure of the commands being run, you can just use the ; operator.

command1 ; command2 ; command3

Will be executed by running each command in order regardless of if any of them succeeding or failing. But it's important to consider that the commands will be run one after another, not in parallel. If one of them never finishes, the ones after it won't run.

Exiting a script on failure

Sometimes you'll see scripts written so that they will stop if a specific function fails to operate correctly. Typically this is handled like this

command1 || exit -1

If the command1 succeeds, then exit never gets run and the script continues as normal.

Its important to note that this is only checking for ONE command. If you want a script to run so that it'll error out when ANY of the commands fail, then you need to use the command "set -e" at the beginning of your script.

See this for more information: http://www.faqs.org/docs/bashman/bashref_56.html

Loops

A while loop looks like this:

while [ condition ]
do
command1;
command2;
command3;
done

The condition can be any command, or it can be one of a list of possible special evaluation commands built into bash. See this as a reference : http://www.cyberciti.biz/faq/bash-while-loop/

for loops are similar

for file in "$1"/*; do
[[ $file -nt $latest ]] && [[ $file == $1/Back* ]] && latest=$file;
done

This command goes through each file in the directory passed in as the first argment, and finds the one with the newest timestamp. The "file" symbol after the "for" is treated as a variable name, with the value changing each time through the loop. See here for more information: http://www.cyberciti.biz/faq/bash-for-loop/

And here's another reference : http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-7.html

Checking Execution Success

To check the execution result of the last program run by bash, you can use the special variable $?. If this has a value of 0, then the previous command was a success, and any other value has a special meaning based on the program returning the value. For more information on a specific program, you'll have to consult that programs documentation.

Another special variable is $!, which returns the process ID number of the last program executed. This can allow you to run a program in the background, and then keep it's ID number around for later interacting with the program or halting the execution of it

Execution in the background

To execute a command in the background so that it runs in parallel to other commands that come after it, just put a single ampersand at the end, like so.

Command1 &

However, I haven't figured out the perfect trick for telling bash to execute multiple commands in parallel on the same line. So I normally just do them one after another.

To send a command that's currently in the foreground (the active command that you're interacting with), use CTRL+z , and it'll be paused and moved to the background, giving you a command prompt again. To make the backgrounded command resume executing in the background, run the command "bg", and to move a backgrounded command (paused or unpaused) back to the foreground, use "fg"

Tab Completion

Bash has a special feature built into it to help you write commands. Whenever you press the tab button, bash will look at all of the text you have written so far, (including if you've written nothing) and attempt to type the rest for you based on a variety of factors.

The default behavior is that bash will look at the contents of the current working directory, and all of the directories that are on the PATH environment variable, and try to match what you've written so far with what's available to match.

But some programs have more complex completions, that allow them to give bash some insight into their inner workings. These completions can be installed into bash, and act as functions that are called whenever bash detects that you're asking for a completion. The functions are given the entire sequence of characters that have been typed so far, and some other data, and then the functions can return any arbitrary list of possible results. For programs that are designed specifically to be used from the commandline, having custom completions can be a significant time saver.

See this article for some more information: http://fahdshariff.blogspot.com/2011/04/writing-your-own-bash-completion.html

The first thing that needs to be understood about the design philosophy of linux is that everything in the system can be represented as a file. Not necessarily everything IS represented this way, but everything can be. So, in a very real sense, everything in linux is a file. This applies to actual files, directories/folders, sections of active memory, configurable parameters of the kernel, actual hardware devices, processes and information about them, network connections, everything. This page goes into a bit more depth: http://wiki.freegeek.org/index.php/Everything_is_a_File

This design philosophy is an offshoot of the unix philosophy, where if a program is worth writing, that program will do one thing, and it will do that one thing very very well. If you need it to do a second thing, write a second program to handle that second thing. If you *really* need a program to do two things, write three programs. One do to thing one, another to do thing two, and a final one to call the other two. Having small single purpose programs necessitates that programs be capable of communicating with each other, and where there is communication there must also be an agreed upon format for that communication. Unix chose to use the concept of a stream of bytes as that form of communication, and files are in a literal sense a stream of bytes. Thus, everything is a file, and all programs should do one thing, do that one thing well, and be capable of communicating through files. See here for more information on the unix philosophy: http://en.wikipedia.org/wiki/Unix_philosophy

Unlearning Windows

Friday, April 11, 2014

Installing SLOCCount in Cygwin

Installing the ack commandline search utility in Windows via Cygwin.

Wednesday, October 23, 2013

How to set default user session in >=gdm-3-8

Thursday, September 12, 2013

Commandline foo and bash scripts