Using The Linux Operating System

(Parts of this guide were adapted from the CSC 161 Basic Linux Commands guide from Grinnell College.)

The Linux operating system has evolved from Unix, an operating system developed at AT&T Bell Labs in 1969. At the time, Bell Labs was a major research and development enterprise, averaging approximately a patent a day over its first 40 years of existence. With this impressive track record, the company was delighted to encourage development of a computing environment that would support research and development; common computing tasks needed to be easy and quick. As a result, the original Unix was extremely efficient, although sometimes cryptic. While professionals found Unix wonderfully powerful, beginners sometimes found that it took some time to master.

The current Linux operating system maintains the efficiency and power of the original Unix, with many extensions and revisions for modern needs. At the same time, Linux also has interfaces that make it much easier to use than the original Unix—although it sometimes still reflects the original cryptic style.

The Terminal Window

When Unix was initially developed, users entered information by typing on a keyboard, and results were printed on paper or a screen. Almost all work was character based, with very little use of graphics. (Images require extensive memory resources that were beyond the capabilities of most early computers.) With this history, many of the early Unix capabilities depended upon typing commands into a terminal. On current machines, the terminal window continues to provide a wide range of powerful capabilities for many processing needs.

Although you may be accustomed to a graphical user interface (GUI) from your past computer usage, you also need to become comfortable with a command-line interface. Historically, commands were typed into windows (actually, the entire terminal had just one screen—the only window). Unix/Linux provides several powerful tools for this type of interaction. To run various commands, you must invoke them by name. The computer program that reads and responds to such invocations is called the shell, and your interactions with the shell take place in a window generated by a program called a terminal emulator.

You may already have a terminal window open on your screen. If not, you can start one at any time by moving the pointer onto the small monitor icon on the bottom row of the front panel, and click with the left mouse button. Shortly a window will appear, displaying the shell prompt—the name of the workstation on which the shell is running, followed by a percentage sign (%) or dollar sign. This prompt indicates that the shell is ready to receive instructions.

You enter such instructions using the keyboard. Move the mouse pointer into the terminal window to make it active. (You many need to click on the window, once your mouse is in the desired area.) The window frame changes color, indicating that the window has become active.

Terminal Utilities

Although the basic running of programs within a terminal window may seem quite straightforward, a few nifty shortcuts can simplify your work considerably.

Stopping a Long or Infinite Loop

Suppose you wrote a program that contained an infinite loop, and you ran it from the terminal window. How could you stop the program?

One way is to close the window, but a better way is to type ctrl+c (i.e., hold down the ctrl key while you type c). This should kill the program without making you re-open the window and re-navigate to the directory you were working in.

The Arrow Keys

Suppose you want to run a command that you ran a short while ago? If the command is complicated, it would be nice not to have to re-type it, and you don’t have to! Within the terminal window, this is accomplished using the ↑ (up arrow) key to retrieve previous commands. Once you have viewed some earlier commands, note that the ↓ (down arrow) also works, which can be helpful if you go too far.

The History Command

But what if the command you want was several commands ago, say 10? Who wants to type that many arrows? To see another option, first type the following to get a list of the commands you have used most recently.

history

Typically, a terminal window remembers the last 50 commands (although a user can change this number). At the start, the history list might look something like this.

505 sleep 10
506 history

In this listing, each command is numbered for future reference. For example, with this listing, if you want to re-issue the sleep command, you could do so by typing !505 to repeat command number 505. Better yet (since you don’t need to know the command number), you can repeat the command with !s. This will re-issue the most recent command that begins with the prefix “s”. Note that the prefix can be of any length; it is not limited to one character.

cat

It is often convenient to look at the contents of a text file without having to open it in an editor. The simplest command for this purpose is cat. For example, cat .bashrc would display the contents of the .bashrc file in your terminal window. Although cat works well for short files that can be viewed all on one screen, we will see shortly that other commands may work better for longer files. As an aside, note that the command name cat stands for “concatenate” for reasons we will explore later.

Auto-completion

Another very useful keyboard shortcut is called auto-completion. Suppose you want to do something with a file called my-test-file in your home directory. That file name seems a bit long to type very often, so it’s nice that you don’t have to.

To use auto-completion, type the command and the first part of the file name (e.g., cat ~/my-t), followed by pressing the tab key. You will find that when you press tab, the system completes the (unique) file name for you.

If there isn’t a unique completion, the system may add a few letters and beep. This tells you that there are multiple files that have these beginning letters. On the other hand, if no files begin with the given file name, the system likely will just beep at you.

Background Processes

Sometimes you want to start a task in a terminal window and the continue with other work. For example, suppose you want to read the contents of your .bashrc file; type:

emacs ~/.bashrc

This will open emacs like normal, but also switches back to your terminal window. You will notice that the terminal window is unavailable for further use (i.e., you won’t get another command prompt) until emacs is closed.

Now close the editor, and then re-open it from the terminal window, but this time add an ampersand character (&) to the end of the command:

emacs ~/.bashrc &

Now when you return to your terminal window, a prompt is waiting for you, making it easy to do multiple tasks at once. Adding the ampersand character to a command causes the command to be launched as a “background process”—separate from the window process, allowing you to continue working with your terminal window.

Closing the Terminal Window

One way to shut down the terminal window is to type exit. However, a quicker (and more common) approach is just to type ctrl+d—that is, hold down either of the keys marked ctrl, just below the shift keys, and simultaneously press the d key. (On our workstations’ keyboards, the keys marked ctrl (“control”) and alt (“alternate” or “meta”) are somewhat like shift keys, in the sense that they modify the effect of other keys that are pressed simultaneously.) The shell program interprets ctrl+d as a signal that you have no more instructions for it and halts, and the terminal terminal emulator closes the window automatically once the shell stops running.

Although the shortcut ctrl+d is convenient, note that this key combination is used in multiple contexts in the Linux system to indicate the “end-of-input” character. Thus, when using a simple terminal window, pressing ctrl+d at a command prompt indicates that there will be no more commands forthcoming, and the terminal window should close. However, if you are using another application, the ctrl+d once may exit the application and a second ctrl+d will close the window.

The Manual

Linux includes an on-line help system, called the manual. There is a “man page” describing each Linux command. Don’t worry if you don’t understand everything you see in a man page. They are often long and cryptic, but you can learn a lot without having to understand everything.

For example, try typing man cat to read about cat. You should see the command name and a quick synopsis of how to use the command. For example, the synopsis cat [OPTION] [FILE] ... tells you that cat (optionally) takes a file as its input, and it also can take (optional) options as parameters. A list of the options follows the synopsis. To exit a man page, type q.

Flags

The manual for a program will also describe all the options available. Options are usually prefixed by a - (dash) and referred to as a flag. If a flag is a single letter it only needs one dash. However, longer flags are prefixed by two dashes. This convention allows multiple single letter flags to be smashed together behind one single dash. For example, you can type:

ls -l -a -h

to list all the files in a directory in the long-listing human-readable format. Or, you can simply type:

ls -lah

Directory and File Commands

The Linux Directory/File Hierarchy

Linux maintains directories and files in a hierarchical structure also called a tree structure. This section explores this organization.

Pathnames

When you first open a terminal window, the shell expects that you are in your home directory. At the start, this is called your current working directory (i.e., your position within the directory tree).

A relative pathname for a file is a name that is given “relative” to your current working directory. For example, if your current working directory is esap, then labs/hello.py could be a relative pathname for a file named hello.py located in a directory named labs that was itself inside esap.

An absolute pathname, such as /home1/p/posera/esap/labs/hello.py, includes the file’s complete path starting with the system’s “root” directory, which is always named / on a Linux system. Just like it sounds, the root directory is the top-most directory in the file system tree.

Each directory in a Linux system contains two special files . (dot) and .. (dot dot) that can be useful when constructing relative pathnames. The file named . (dot) means “the current directory,” and the file named .. (dot dot) means “the parent directory.” For example, ../mydir could be a directory that is a sibling of your current working directory (i.e., mydir could be a directory that has the same parent directory your current working directory does). Then ../mydir/foo/bar.txt could refer to a file farther down that branch of the tree. Similarly, ../../anotherdir/baz could be a directory that is a cousin of your current directory in the file system tree.

The tilde character (~) is also useful for specifying pathnames, but it works a little differently. Used alone, it specifies your home directory, so for me, ~/esap/labs is a short name for /home1/p/posera/esap/labs. In general, on the SEAS Unix machines, your home directory is located at /home1/<first letter of username>/<username>.

Root Directory and its Subdirectories

While we are poking around the Linux file system, take a look at the files in the root directory /. You should see directories with names like /bin, /home, /lib, and /usr. Again, the ls command can list the files in each of these directories, e.g., ls /bin. They contain many, many files, organized as follows.

Sometimes locating a program can be something of a challenge, but the commands which and whereis can help you solve this problem. For example, both

whereis emacs

and

which emacs

will give you the absolute path name of the emacs program.

Search Paths

When you type the name of a command in a terminal window, the operating system must determine the appropriate program to run. If you type a full path name, the operating simply uses that program. For example, if you type /bin/ls, the operating system will run the ls program located in the /bin directory.

Similarly, you can clarify that the operating system should use a relative path by starting a command with a period (. represents the current directory). For example, ./test tells the operating system to run the program test in the current directory.

However, if you type a command without specifying its location, the operating system must determine where to find the relevant program. To facilitate this process, the operating system maintains a variable PATH for each terminal window. For example, the variable might be set to

/usr/bin:/bin:/usr/X11R6/bin:/usr/local/bin:/net/bin:.

This string is interpreted as a sequence of directories, separated by a colon :. For example, suppose you type pwd in a terminal window. The operating system first tries to find a program pwd in directory /usr/bin. However, no such program exists there, so the operating system looks at the next directory, /bin. In this case, a program pwd is found, so the operating system uses this program to respond to the command in the terminal window. (If file pwd was not present in /bin, the operating system would continue its search with /usr/X11R6/bin, etc.)

In this example, note that the last option is . (the abbreviation for a program in the current directory). Thus, the operating would run a program in your current directory—but only if there was not a program of the same name in one of the earlier directories listed in the PATH variable. Note that the practice of putting your working directory in your default search path is considered a security risk in many circles. You would do well also to avoid this configuration. (Your default configuration on the SEAS servers does not include dot in PATH).

File Utilities

Some common file management commands are listed in the table below. When you have the chance you should try each of these to determine just how they work.

The following variants can be particularly handy. (Although some details may not be obvious now, we will see shortly how to find out more information about such commands.)

cd ..
cp -p
ls -l
mkdir -p
pushd and popd

Two more commands that can be quite useful for moving around the file system are pushd and popd. They let you jump back and forth between distant directories quickly and easily. For example, suppose your current working directory is esap/labs/ and you want to jump to a directory in another branch of your file system tree, say public_html/ and afterward you want to return to your original directory.

The following command will push the name of your current directory onto a stack of directory names that Linux maintains behind the scenes, and then change your current directory to the one named in the command:

pushd public_html/

When you are ready to return to your previous directory, you simply type popd. This pops the most recent directory name off the stack and then makes it your current working directory.

Go ahead and give these command a try. Of course, if you like, you can use pushd several times in a row (pushing multiple directory names onto the stack), and then backtrack through the sequence in reverse order.

Displaying Text Files

It is often convenient to look at the contents of a text file without having to open it in an editor. Previously in this reference, we saw that cat can be used for this purpose, but it is most useful for short files that can be viewed all on one screen.

GNU/Linux provides several other utilities that are useful for “paging” through text files (i.e., for viewing files one page at a time). Several of these commands are outlined in the following table.

Directory and File Permissions

From a user’s perspective within Unix or Linux, the world of files and directories is divided into three categories:

For each of these categories, users in these categories can have three types of permissions:

To clarify how these permissions work, we consider a long listing of files in one of my folders:

dertouzos> ls -l
total 1787004
-rw-r--r--  1 osera mathfac        895 Nov 20  2015 notes.txt
-rw-r--r--  1 osera mathfac         52 Aug 31  2015 pc-client.properties
drwxr-xr-x  2 osera mathfac       4096 May  4  2015 Pictures
-rw-r--r--  1 osera mathfac       2808 Dec  4  2015 PriorityQueue.java
drwxr-xr-x  2 osera mathfac       4096 May  4  2015 Public
drwxr-xr-x 14 osera mathfac       4096 Jul  1 23:36 public_html
drwxr-xr-x  4 osera mathfac       4096 Jan 31 10:17 Repositories
drwxr-xr-x 11 osera mathfac       4096 May 15 11:38 scratch
drwx------  3 osera mathfac       4096 Jan 31 10:22 shared
-rwxr-xr-x  1 osera mathfac       6952 Mar 18 13:49 sim

In the listing, the first character indicates whether the item is a directory (d) or a regular file (-). Thus, in this directory, Pictures is a directory (the line starts with d), and notes.txt is a regular file.

The next characters list specific file permissions (r=read, w=write, x=execute) for the user, group, and world, in that order. Thus, immediately after the d character for the Pictures directory, the sequence rwx indicates that user osera has all three permissions for these capabilities. However, both the group and any others have permissions r-x, indicating that they can read and execute the directory but cannot modify (write) the directory.

The rest of the line gives the username and groupname of the file’s owner (osera and mathfac), the file’s size, the date and time the file was last modified date, and file name. You may belong to more than one group on the SEAS Unix servers. To see a list of all of your groups, use the command groups. If you get really confused, you can ask for your own username with the command whoami.

Setting Permissions

You can set the permissions of the files you own using the chmod command. The simplest approach is to assign numbers to each capability (4 for read, 2 for write, 1 for execute) and then to use addition when combining numbers. Thus, 6 = 4 + 2 (read plus write permission), and 7 = 4 + 2 + 1 (all three permissions added together).

Within this framework, you set permissions for a file by specifying the desired capabilities for the user, group, and world (in that order). Thus, when she set up her directory for Pictures above, I might have issued the command:

chmod 755 Pictures

Here, the user (osera) has full permissions (7 = read + write + execute), while the group and others can read and execute, but not write (5).

Setting Terminal Defaults

Bash Runtime Configuration:

As you may have noticed, the system sets up many elements for you whenever you open a window. This setup is accomplished by the .bashrc file located in your home directory. As you will see, .bashrc is just a regular file that can be edited, and from time to time you may want to tailor this file to meet your special needs or interests.

For now, we mention only two special commands that you may find within .bashrc.

umask 077

As noted, this statement yields the same result as using chmod to give new files the permissions 700.

ssh osera@eniac.seas.upenn.edu

Although this command does the right task, it is awkward to type. Thus, in the .bashrc file of my home computer, I include a line of the form:

alias my-ssh="ssh osera@eniac.seas.upenn.edu"

With this command defined, I can just type my-ssh into a terminal window to connect over the Internet to eniac.

Remote Access

You may wish to access the SEAS Unix servers outside of a SEAS machine, e.g., to write code in an preset environment or test things out that require Unix support. SEAS provides a server that exposes a secure shell that you can use to log into remotely. The URL of this server is:

eniac.seas.upenn.edu

We can interact with this server with two key programs—a ssh client which allows you to log into the server remotely and scp which acts like cp but can be used to copy to and from a remote server.

Secure Shell Access

Depending on your platform, a number of ssh clients are available to you:

Use the client of your choice to connect to the server above using your SEAS username and password as credentials. Upon logging in, you can work as if you were present at a (terminal-only) workstation.

PuTTy provides a nice GUI for doing this. The ssh command is more barebones; you provide the address of the server you wish to connect to along with your username in the following format:

  <username>@eniac.seas.upenn.edu

For example, if I wanted to log in remotely, I would type the following command: ssh osera@eniac.seas.upenn.edu.

Remote Copying

Instead of logging into Eniac, you may want to simply copy files to and from your SEAS Unix account. To do this, use the scp program which acts identically to cp (described above) except that the source and/or destination could be ssh servers. The syntax of such a path is:

<username><server>:<path relative to home directory>

For example, if I wanted to copy a file file.txt from the current directory to the root of my Unix account, I would type:

scp file.txt posera@eniac.seas.upenn.edu:.

Note that the relative path is simply . because we want to copy file.txt to the root of my home folder.

In contrast, I may want to copy a file from my SEAS account by issuing the following command:

scp posera@eniac.seas.upenn.edu:work/test.py .

This command copies work/test.py (relative to the root of my home directory on the SEAS Unix servers) to my current directory (locally).