CS304: Unix Skills

CS304: Unix Skills

This course requires you to work on a Linux server, using a command line, and your life will be much easier if you have some Unix skills. (Linux is one of many descendants of Unix, an operating system that predates DOS and Windows. Mac OS X is another descendant, so many of the commands and concepts below will work on a Mac as well.)

Motivation

You already have a lot of experience working with computers, but with a GUI. A GUI is a graphical user interface, meaning icons, menus, and even drag-and-drop — heaven help us. Although Mac and Windows are rather different, they both use GUIs that rely crucially on those techniques to get work done. GUIs are intuitive, user-friendly, and easy to work with. I'm asking you to give all that up in favor of a CLI (command line interface), so I'd better have a really good reason.

The main reason is that the easiest way to connect to the server is through a relatively narrow tunnel, through which it's quick and easy to send textual commands and get textual responses. (It's possible to set up an X11 tunnel and use a Linux GUI, but that would require you to become familiar with a Linux GUI, so I think it's better to learn some skills that are more universally applicable.)

Another reason, almost as important, is that these skills will make you more efficient both now and in the future, in ways that you can't yet foresee. That's because programming languages are (for the most part) textual, and there's only a very blurry line between a program to accomplish some task and a series of commands to accomplish something. In short, commands can be automated in a way that is hard or impossible for a GUI.

Consider a quick example. Using a GUI, changing the permissions on a file takes about 5 mouse clicks (not including all the navigating around to find the file). Changing the permissions on a file using a CLI takes about 20 keystrokes. If you have to change the permissions on 100 files, it'll take you 5×100=500 mouse clicks, but about 20 keystrokes will do the trick using a CLI, with judicious use of wildcards. (Plus, if you're a good typist, those 20 keystrokes can take less time than those 5 mouse clicks.)

A software developer named Lawrence D'Oliveiro talks about this in his Tumblr post entitled CLI versus GUI Deathmatch!. You should also read a recent post about Linux philosophy. (I'll bet you didn't know an operating system could have a philosophy!)

Directories

Like all filesystems, Unix is organized as a tree of directories. To be able to refer to another directory or file, you have to understand the notation of the filesystem. Here are some to know about:

/
The directory whose name is the slash character is the root of the tree. Every directory and file is a descendant of this directory. Assuming there are no permissions issues, you can uniquely specify a directory or file by starting at the root. For example, this file is:
/home/cs304/public_html/unix.html

The preceding is called an absolute pathname, since it works from anywhere.

You'll notice that the different directories in the tree of directories are separated from each other with slashes. So, a slash plays two roles: it separates parent from child and also stands for the root of the ancestry tree. A side-effect of this notation is that you cannot name a directory or file with a slash in the name. (On Mac OS, the directory separator is a colon; on Windows, it's a backslash.)

.
The directory whose name is a single period (pronounced dot) is like the pronoun me: it stands for the directory you are in. Each process on a Unix system has a "current working directory" (CWD) and all relative pathnames implicitly start at the CWD. For example, the follow are two equivalent ways to say "the file named foo.text in the current working directory:"
./foo.text
 foo.text

Relative pathnames are very useful and important because they allow code to be relocatable, meaning that a directory subtree can be copied to another location, possibly even on another machine, and all relative pathnames that stay within the subtree will still work!

..
The directory whose name is a double period (pronounced dot dot) is like the word mom: it stands for the unique parent directory of a directory. For example, the following says "the file named bar.text in the the parent of the current working directory:"
../bar.text

The dot-dot syntax can be very useful in relative pathnames, to address a file that is related via an ancestor (say an uncle, or a second cousin, once removed).

~/
The directory whose name is a tilde is another kind a pronoun: it means your login directory, referring to a kind of "database" called the password database. For example, each of you has a public_html directory in your home directory. Here is how you would address a file in that directory:
~/public_html/index.html

Remember that for each of you, that will be a different file!

Also, because it refers to the password database, the tilde is typically not available in programs, but it is a nice feature of the shell, so you will often use it in commands.

~user/
Tilde has a second usage, where it is immediately followed by the name of a user: it means the login directory of that user. For example, the following path would be address a file in the CS 304 course account:
~cs304/public_html/home.html

Most of the pathname concepts above may be familiar to you from URLs, since the syntax of the pathname in a URL derives from Unix pathnames.

With these concepts in mind, hopefully the following sections will be more clear. I will be more terse in these sections, so if you find a command confusing, I encourage you to make use of one of those thousands of web tutorials on unix and linux. At the end of this document, I have links to the man pages for these commands.

Conventions and Prompts

In the following sections, I will give sample input and output from interactions with a Linux machine (actually, Tempest, the CS department server).

When you are logged into a Linux machine (directly via the console or or across the network via ssh), you will be running a "shell" (a shell is just a program that allows you to run commands). When the shell is ready for your command, it will print a "prompt." That prompt is wildly customizable. On Tempest, the default prompt is like this:

[user@host cwd]

That is, the shell prints three pieces of information, enclosed in square brackets: the username you're logged in with, the name of the machine you're logged into, and the name of your current working directory. In the examples below, I will usually be logged into the "Wendy Wellesley" test account, so the prompt will look like this:

[wwellesl@tempest ~] 

You will never type that part!

Also note that all these examples have a typographic convention that the stuff you're supposed to type is in bold monospace and the responses and other output is in regular monospace. Any tutorials you read on Linux will probably have occurrences of a prompt (possibly very terse, such as a dollar sign or a percent sign), and may have conventions to help you distinguish what you type from the computer's response.

man

From the very beginning, Unix machines have had online "manuals" for use by everyone from novices to experts. Probably only one (unix) person in a thousand remembers more than a handful of the options for the "ls" command. So, when you're logged in, don't hesitate to use the "man" command to learn more about a command you're unfamiliar with:

[wwellesl@tempest ~] man ls

To exit from man, type "q".

Of course, these online man pages are on the web as well; I give some links at the end of this page.

Navigating

As I mentioned, the shell always puts you "in" a directory, your "current working directory" (CWD, also called "." or dot). Commands to know:

ls
lists the files and directories in the given directory. With no arguments, lists the contents of the CWD.
cd
changes the CWD to the given directory. With no arguments, changes to your home directory.
pwd
prints the absolute pathname of the CWD, in case you forget where you are.

Examples:

[wwellesl@tempest ~] ls
course-info.pdf  cs789        foo.html          Pictures      saved.sql
cs111            dead.letter  fred              pset1         Templates
cs230            Desktop      HelloWorld.class  Public        tmp
cs232            Documents    HelloWorld.java   public_html   Videos
cs304            Download     laptop_key.pub    read-only
cs307            firstclass   Music             renamed-file
[wwellesl@tempest ~] pwd
/students/wwellesl
[wwellesl@tempest ~] cd public_html/
[wwellesl@tempest public_html] ls
cgi-bin         cs110-assignments  other-page.part  protected   wendy.html~
copy-of-bashrc  dot-bashrc2        pictures.html    wendy.html  wendy.shtml
[wwellesl@tempest public_html] pwd
/students/wwellesl/public_html
[wwellesl@tempest public_html] ls ..
course-info.pdf  cs789        foo.html          Pictures      saved.sql
cs111            dead.letter  fred              pset1         Templates
cs230            Desktop      HelloWorld.class  Public        tmp
cs232            Documents    HelloWorld.java   public_html   Videos
cs304            Download     laptop_key.pub    read-only
cs307            firstclass   Music             renamed-file
[wwellesl@tempest public_html] ls ~
course-info.pdf  cs789        foo.html          Pictures      saved.sql
cs111            dead.letter  fred              pset1         Templates
cs230            Desktop      HelloWorld.class  Public        tmp
cs232            Documents    HelloWorld.java   public_html   Videos
cs304            Download     laptop_key.pub    read-only
cs307            firstclass   Music             renamed-file
[wwellesl@tempest public_html] ls ~wwellesl
course-info.pdf  cs789        foo.html          Pictures      saved.sql
cs111            dead.letter  fred              pset1         Templates
cs230            Desktop      HelloWorld.class  Public        tmp
cs232            Documents    HelloWorld.java   public_html   Videos
cs304            Download     laptop_key.pub    read-only
cs307            firstclass   Music             renamed-file
[wwellesl@tempest public_html] 

Note that all three of the last "ls" commands were listing the same directory, using different ways to specify that directory.

Moving and Copying

Now that you can move around, you'll want to be able to move and copy files. Commands to know:

cp
copies the first argument (a file) to the second argument (either a file or a directory). There are many other options; see the man page for more.
mv
moves the first argument (a file) to the second argument (either a file or a directory).
rm
removes (deletes) the file(s). Caution! This is not a reversible operation: there is no "un-rm" command.

Example:

[wwellesl@tempest public_html] pwd
/students/wwellesl/public_html
[wwellesl@tempest public_html] ls
cgi-bin         cs110-assignments  other-page.part  protected   wendy.html~
copy-of-bashrc  dot-bashrc2        pictures.html    wendy.html  wendy.shtml
[wwellesl@tempest public_html] cp pictures.html images.html
[wwellesl@tempest public_html] cp ~cs304/public_html/home.html cs304-home.html
[wwellesl@tempest public_html] ls
cgi-bin            cs304-home.html  other-page.part  wendy.html
copy-of-bashrc     dot-bashrc2      pictures.html    wendy.html~
cs110-assignments  images.html      protected        wendy.shtml
[wwellesl@tempest public_html] mv cs304-home.html webdb-home.html
[wwellesl@tempest public_html] ls
cgi-bin            dot-bashrc2      pictures.html    wendy.html
copy-of-bashrc     images.html      protected        wendy.html~
cs110-assignments  other-page.part  webdb-home.html  wendy.shtml
[wwellesl@tempest public_html] rm images.html webdb-home.html 
[wwellesl@tempest public_html] ls
cgi-bin         cs110-assignments  other-page.part  protected   wendy.html~
copy-of-bashrc  dot-bashrc2        pictures.html    wendy.html  wendy.shtml
[wwellesl@tempest public_html] 

Programs that are modular often comprise several related files. These files are all essential and if any of them is missing, the program doesn't work.

Consider the following two Java files. Here is Harry.java

public class Harry {
    
    public static String describe() {
        return "Harry Potter, 4 Privet Drive, Little Whinging, Surrey";
    }

    public static void main(String args[]) {
        System.out.println("I am "+describe());
        System.out.println("My best mate is "+Ron.describe());
    }
}
Here is the nearly equivalent code for Ron.java
public class Ron {
    
    public static String describe() {
        return "Ron Weasley, the Burrow, Ottery St. Catchpole, Devon";
    }

    public static void main(String args[]) {
        System.out.println("I am "+describe());
        System.out.println("My best mate is "+Harry.describe());
    }
}

They are currently in the directory ~cs304/pub/java/related/. Here's a picture of their relationship:

Harry.java and Ron.java in the same folder

Harry.java and Ron.java are two files in the same folder. The black lines indicate their location in the related folder. The red double-ended arrow indicates their mutual dependence.

Now, think carefully about what happens if you do the following sequence of commands:

% cp ~cs304/pub/java/related/Harry.java Harry.java
% javac Harry.java
    Harry.java:9: error: cannot find symbol
            System.out.println("My best mate is "+Ron.describe());
                                                  ^
      symbol:   variable Ron
      location: class Harry
    1 error
% 

What has gone wrong? How do you fix it?

The answer, of course, is that you didn't copy both files, and Harry depends on Ron. (And vice versa, but that's not important right now.) The java compiler expects Ron.java to be in the same directory as Harry.java. (More precisely, javac expects to find Ron on the CLASSPATH, and the CLASSPATH includes the current directory. More about CLASSPATH some other time.)

When one file depends on another, that may require both to be copied together

Making Files and Directories

To make a file, you would typically use a text editor, such as Emacs or vi. Emacs and vi are very different in usage, philosophy and user base. Emacs is slower to start up, but bloated with many features. vi is quicker to start up but is leaner. There are many other differences, but this is not the place to continue the decades-long cold war between the Emacs and vi factions.

You should know, however, that I'm firmly in the Emacs camp.

There are zillions of Emacs tutorials on the web; avail yourself of one of them. Spend an hour on a good tutorial, and you will save time over the course of the semester. I promise.

To manage directories, use these commands:

mkdir
creates the named directory.
rmdir
removes (deletes) the directory, but only if it's empty.
rm -r
recursively removes (deletes) the directory tree. Caution! This command is even more dangerous than "rm" itself. Not for the faint of heart.

Example:

[wwellesl@tempest public_html] pwd
/students/wwellesl/public_html
[wwellesl@tempest public_html] ls
cgi-bin         cs110-assignments  other-page.part  protected   wendy.html~
copy-of-bashrc  dot-bashrc2        pictures.html    wendy.html  wendy.shtml
[wwellesl@tempest public_html] mkdir cs304-stuff
[wwellesl@tempest public_html] ls
cgi-bin            cs304-stuff      pictures.html  wendy.html~
copy-of-bashrc     dot-bashrc2      protected      wendy.shtml
cs110-assignments  other-page.part  wendy.html
[wwellesl@tempest public_html] cd cs304-stuff/
[wwellesl@tempest cs304-stuff] ls
[wwellesl@tempest cs304-stuff] cp ../wendy.html .
[wwellesl@tempest cs304-stuff] ls
wendy.html
[wwellesl@tempest cs304-stuff] cd ..
[wwellesl@tempest public_html] rmdir cs304-stuff/
rmdir: failed to remove `cs304-stuff/': Directory not empty
[wwellesl@tempest public_html] rm cs304-stuff/wendy.html 
[wwellesl@tempest public_html] rmdir cs304-stuff/
[wwellesl@tempest public_html] 

Permissions

You can allow or deny someone from reading, writing, or executing your files and directories. This is done with permission bits or mode bits.

A bit is sufficient to represent a yes/no question, and in Unix, 3 bits are used for the permissions. An "r" is used for the "read" bit, a "w" for the "write" bit, and an "x" for the "execute bit. If the bit is zero, an "-" (hyphen) is used. Some examples:

rwx
permission to read, write and execute are all allowed.
r--
permission to read is allowed, but not write or execute
r-x
permission to read and execute is allowed, but not write
---
permission to read, write and execute is denied

Actually, there are three sets of these three bits (for 9 in all), granting or denying permissions to different sets of people:

  • User: the user who owns the file or directory
  • Group: the group that the person or program trying to access the file is in. We won't say more about that here.
  • Other: everyone on the system.

Some examples:

rwxr-x---
You (the owner) can do anything with this file, the group can read and execute, but not write, and others can't do anything.
rwx------
Only you can do anything, and you can do everything.
rwxr--r--
You can do anything, but everyone else can read it.

Commands to know:

ls -l
A "long" version of the "ls" command: gives a lot of info about each entry, including permission bits.
ls -ld
A "long" info about a directory itself, as opposed to the contents.
chmod
Change Mode, allows you to change the mode bits (permission bits).

Here are some examples. I will let you infer the usage of the chmod command from these.

[wwellesl@tempest public_html] cd foo
[wwellesl@tempest foo] cp ../wendy.html .
[wwellesl@tempest foo] ls -l wendy.html 
-rw-rw----. 1 wwellesl wwellesl 152 Jan 25 17:55 wendy.html
[wwellesl@tempest foo] chmod u+x wendy.html 
[wwellesl@tempest foo] ls -l wendy.html 
-rwxrw----. 1 wwellesl wwellesl 152 Jan 25 17:55 wendy.html
[wwellesl@tempest foo] chmod g-w wendy.html 
[wwellesl@tempest foo] ls -l
total 4
-rwxr-----. 1 wwellesl wwellesl 152 Jan 25 17:55 wendy.html
[wwellesl@tempest foo] chmod o+r wendy.html 
[wwellesl@tempest foo] ls -l wendy.html 
-rwxr--r--. 1 wwellesl wwellesl 152 Jan 25 17:55 wendy.html
[wwellesl@tempest foo] chmod ugo+rwx wendy.html 
[wwellesl@tempest foo] ls -l wendy.html 
-rwxrwxrwx. 1 wwellesl wwellesl 152 Jan 25 17:55 wendy.html
[wwellesl@tempest foo] chmod ugo-rwx wendy.html 
[wwellesl@tempest foo] ls -l wendy.html 
----------. 1 wwellesl wwellesl 152 Jan 25 17:55 wendy.html
[wwellesl@tempest foo] chmod u+rw wendy.html 
[wwellesl@tempest foo] ls -l wendy.html 
-rw-------. 1 wwellesl wwellesl 152 Jan 25 17:55 wendy.html
[wwellesl@tempest foo] 

Octal notation: If you know a little bit of binary and octal (base 8) notation, you can also use that. See if you can figure this out:

[wwellesl@tempest foo] chmod 754 wendy.html 
[wwellesl@tempest foo] ls -l wendy.html 
-rwxr-xr--. 1 wwellesl wwellesl 152 Jan 25 17:55 wendy.html
[wwellesl@tempest foo] 

Permissions can be complicated sometimes, but with ls -l and chmod, you can figure out and fix a lot of problems.

tar and gzip

If someone wants to give you a bunch of files and directories, they could attach them all to a mail message to you, or put them all on a web server where you could download them, but what if there were hundreds or thousands of files and directories? Handling them all one-at-a-time would be tedious at best.

One solution, used for decades by Unix people, is a "tarfile," which is a single file that contains a directory tree. If you download a significant collection of software from a website, you will almost certainly be offered the option of downloading a tarfile, among other options.

(Historical aside: the name "tar" comes from "tape archive" from back in the olden days when the "tar" was actually used to write a directory tree to a magnetic tape. However, some bright spark added a command line argument allowing "tar" to write the archive to a file on disk — the "tarfile" or "tarball" — instead of to a tape device. )

You're all familiar with compression, such as MP3 compression of a sound file. A compression program used for decades by Unix people is called "gzip." For text files, gzip does an excellent job, often reducing the size of a file by a factor of three or so. So, in fact, when downloading software, the option is typically not an uncompressed tarfile but a gzipped tarfile.

Commands to know:

tar cf tarfile directory
Create a tarfile from the given directory
tar tf tarfile
List the contents of the given tarfile (the "t" is for "table of contents").
tar xf tarfile
Extract the contents of the given tarfile, creating a subdirectory of the current directory.
gzip file
Compress the given file, replacing it with a .gz version.
gunzip file
Un-compress the given file

Adding a "z" to the first argument of "tar" makes it work with gzip compression.

Examples:

[wwellesl@tempest public_html] pwd
/students/wwellesl/public_html
[wwellesl@tempest public_html] ls
cgi-bin            dot-bashrc2      pictures.html  wendy.html~
copy-of-bashrc     foo              protected      wendy.shtml
cs110-assignments  other-page.part  wendy.html
[wwellesl@tempest public_html] ls cgi-bin/
foo.cgi  foo.cgi~
[wwellesl@tempest public_html] cd ..
[wwellesl@tempest ~] tar cf pub.tar public_html/
[wwellesl@tempest ~] ls -l pub.tar 
-rw-rw----. 1 wwellesl wwellesl 30720 Jan 25 18:19 pub.tar
[wwellesl@tempest ~] tar tf pub.tar 
public_html/
public_html/cgi-bin/
public_html/cgi-bin/foo.cgi~
public_html/cgi-bin/foo.cgi
public_html/wendy.html
public_html/pictures.html
public_html/dot-bashrc2
public_html/.htaccess
public_html/other-page.part
public_html/protected/
public_html/protected/.htaccess
public_html/protected/.htpasswd
public_html/cs110-assignments/
public_html/cs110-assignments/.htaccess
public_html/cs110-assignments/.htpasswd
public_html/cs110-assignments/home.html
public_html/wendy.shtml
public_html/wendy.html~
public_html/copy-of-bashrc
public_html/foo/
public_html/foo/wendy.html
[wwellesl@tempest ~] gzip pub.tar 
[wwellesl@tempest ~] ls -l pub.tar.gz 
-rw-rw----. 1 wwellesl wwellesl 2311 Jan 25 18:19 pub.tar.gz
[wwellesl@tempest ~] 

Notice the relative size of the pub.tar and pub.tar.gz files: 30720 (just over 30KB) and 2311 (less than 3KB), for a 10-fold compression!

You'll also notice that the pathnames in the tarfile are all start with public_html, so wherever we "un-tar" this tarfile, a public_html directory will be created.

Since there's already a public_html directory here, we won't be able to un-tar the file here. We could email it to someone, or put on a web server.

Here are a few more examples of working with these things. What we will do is rename the current public_html, then untar the tarfile, compare it with the original to see that they are the same, right down to timestamps and permission bits (tarfiles contain that meta-data as well as file contents), and then delete the original:

[wwellesl@tempest ~] ls
course-info.pdf  cs789        foo.html          Pictures     renamed-file
cs111            dead.letter  fred              pset1        saved.sql
cs230            Desktop      HelloWorld.class  Public       Templates
cs232            Documents    HelloWorld.java   public_html  tmp
cs304            Download     laptop_key.pub    pub.tar.gz   Videos
cs307            firstclass   Music             read-only
[wwellesl@tempest ~] mv public_html public_html.original
[wwellesl@tempest ~] tar xzf pub.tar.gz 
[wwellesl@tempest ~] ls
course-info.pdf  cs789        foo.html          Pictures              read-only
cs111            dead.letter  fred              pset1                 renamed-file
cs230            Desktop      HelloWorld.class  Public                saved.sql
cs232            Documents    HelloWorld.java   public_html           Templates
cs304            Download     laptop_key.pub    public_html.original  tmp
cs307            firstclass   Music             pub.tar.gz            Videos
[wwellesl@tempest ~] ls -l public_html
total 44
drwxr-x---. 2 wwellesl wwellesl 4096 Feb 25  2007 cgi-bin
-rw-r-----. 1 wwellesl wwellesl  204 Jan  4  2008 copy-of-bashrc
drwxr-x---. 2 wwellesl wwellesl 4096 Jan 21  2007 cs110-assignments
-rw-r-----. 1 wwellesl wwellesl  204 Jan  4  2008 dot-bashrc2
drwxrwx---. 2 wwellesl wwellesl 4096 Jan 25 17:55 foo
-rw-rw----. 1 wwellesl wwellesl   89 Jan 15  2010 other-page.part
-rw-rw----. 1 wwellesl wwellesl  593 Feb  6  2007 pictures.html
drwxr-x---. 2 wwellesl wwellesl 4096 Jan 25  2010 protected
-rw-rw----. 1 wwellesl wwellesl  152 Jan 15  2010 wendy.html
-rw-rw----. 1 wwellesl wwellesl  108 Jun  8  2005 wendy.html~
-rw-rw----. 1 wwellesl wwellesl  152 Jan 15  2010 wendy.shtml
[wwellesl@tempest ~] ls -l public_html.original/
total 44
drwxr-xr-x. 2 wwellesl wwellesl 4096 Feb 25  2007 cgi-bin
-rw-r-----. 1 wwellesl wwellesl  204 Jan  4  2008 copy-of-bashrc
drwxr-xr-x. 2 wwellesl wwellesl 4096 Jan 21  2007 cs110-assignments
-rw-r-----. 1 wwellesl wwellesl  204 Jan  4  2008 dot-bashrc2
drwxrwx---. 2 wwellesl wwellesl 4096 Jan 25 17:55 foo
-rw-rw-r--. 1 wwellesl wwellesl   89 Jan 15  2010 other-page.part
-rw-rw----. 1 wwellesl wwellesl  593 Feb  6  2007 pictures.html
drwxr-s---. 2 wwellesl apache   4096 Jan 25  2010 protected
-rw-rw-r--. 1 wwellesl wwellesl  152 Jan 15  2010 wendy.html
-rw-rw-r--. 1 wwellesl wwellesl  108 Jun  8  2005 wendy.html~
-rw-rw-r--. 1 wwellesl wwellesl  152 Jan 15  2010 wendy.shtml
[wwellesl@tempest ~] rm -r public_html.original/
rm: remove write-protected regular file `public_html.original/.htaccess'?
      y

You'll notice in our recursive deletion of the original (using the powerful but dangerous "rm -r") we were asked about whether "rm" should remove a write-protected file. In this case, "write-protected" means that the "w" permission bit was not set. I typed a "y" to give "rm" permission to go ahead and delete the file.

Note, an alternative to the tar/gzip combination is zip and unzip.

drop

Now that you know about permission bits and such, you understand a bit more deeply what prevents you from copying one of your files into a directory that I own: the permission bits on that directory don't allow you ("others") to write to that directory.

But what if I wanted to allow you to write to one of my directories in a controlled way, say as a way of submitting an assignment. An analogy would be like sliding your printout under my door: you can put something of yours into something I own, but it then becomes mine and you can't pull it back out again, though you might be able to look at it.

There is no standard, built-in, Unix command to do what I've described, but I have written one for us at Wellesley:

drop account file
Copy the given file to the "drop" subdirectory of the given account. Actually, copy it to a special sub-directory for all your submissions, named for your account.

Here's the "drop" command in action:

[wwellesl@tempest public_html] ls -l wendy.html
-rw-rw----. 1 wwellesl wwellesl 152 Jan 15  2010 wendy.html
[wwellesl@tempest public_html] drop cs304 wendy.html
Copying wendy.html (from wwellesl) to /home/cs304/drop/ (uid 1942)
/home/cs304/drop/wwellesl doesn't exist, making it.
Successful drop.
[wwellesl@tempest public_html] ls -l /home/cs304/drop/wwellesl/
total 4
-r--r-----. 1 cs304 wwellesl 152 Jan 25 18:50 wendy.html

Notice that the drop created the wwellesl subfolder of the /home/cs304/drop folder, just for us, since this is our first drop.

Let's drop something else:

[wwellesl@tempest public_html] ls
cgi-bin            dot-bashrc2      pictures.html  wendy.html~
copy-of-bashrc     foo              protected      wendy.shtml
cs110-assignments  other-page.part  wendy.html
[wwellesl@tempest public_html] drop cs304 other-page.part 
Copying other-page.part (from wwellesl) to /home/cs304/drop/ (uid 1942)
Successful drop.
[wwellesl@tempest public_html] ls -l wendy.html other-page.part 
-rw-rw----. 1 wwellesl wwellesl  89 Jan 15  2010 other-page.part
-rw-rw----. 1 wwellesl wwellesl 152 Jan 15  2010 wendy.html
[wwellesl@tempest public_html] ls -l /home/cs304/drop/wwellesl/
total 8
-r--r-----. 1 cs304 wwellesl  89 Jan 25 18:51 other-page.part
-r--r-----. 1 cs304 wwellesl 152 Jan 25 18:50 wendy.html
[wwellesl@tempest public_html] 

Notice how the ownership of the dropped files has changed to "cs304", but the files are still readable by group "wwellesl," so Wendy could actually check that they are correct (unlike sliding papers under a professor's door).

By the way, if we needed to drop a whole bunch of files, we could tar them up and drop the tarfile.

ssh/scp

Often, the computer we are physically touching, using its keyboard and mouse, and looking at its screen, is not the one we want to be working with. For example, you login to your own laptop or to Mac #7 in SCI 257, but you really want to be logging into Tempest and modifying your files there. The following commands enable this remote work across the network:

ssh -Y user@host
Remotely login to the given host computer as the given user account. ssh will prompt you for the password for the account and relay it to the host. If the password is accepted, ssh will start a remote shell for you. The -Y sets up a "tunnel" to allow X11 connections, which means your shell on the remote host can open up windows (such as Emacs) on the computer you're looking at.
scp path/to/local/file user@host:path/to/remote/file
This command is a lot like cp except that you can precede the filenames with user@host to have them copied across the network to the destination. You can use this command to copy a file from your local machine, say your laptop, to Tempest.
scp user@host:path/to/remote/file path/to/local/file
scp can also go the other way, copying a file from the remote host to your local machine.

As an example, I logged into a Mac (station #12 in SCI 257 where I logged in as "sanderso") to do the following. Notice the different prompt on the Mac versus Tempest.

sci-257-12:~ sanderso$ cd Desktop/
sci-257-12:Desktop sanderso$ ls -l mypage.html 
-rw-r--r--  1 sanderso  WELLESLEY\Domain Users  0 Jan 26 12:29 mypage.html
sci-257-12:Desktop sanderso$ scp mypage.html wwellesl@tempest:public_html/
The authenticity of host 'tempest (149.130.136.40)' can't be established.
RSA key fingerprint is ae:53:ce:76:03:10:a9:23:ee:89:14:5a:23:3f:fb:32.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'tempest,149.130.136.40' (RSA) to the list of known hosts.
wwellesl@tempest's password: 
mypage.html                                                         100%    0     0.0KB/s   00:00    

Let's take a moment to look at that scary message from scp. The ssh and scp programs are secure, and they protect against eavesdropping by encrypting all traffic to and fro, and they protect against machine "spoofing" by checking the identity of the remote host. If you've never previously connected to that remote host from this local host, scp can't check the identity so it asks whether you a sure. On-campus, you can comfortably always say "yes," since LTS has good control of the hostnames. Across the wilds of the internet, spoofing can arise, so you have to be more thoughtful. We don't have time to get into that here, though, so let's continue with our example.

sci-257-12:Desktop sanderso$ ssh -Y wwellesl@tempest
wwellesl@tempest's password: 
/opt/X11/bin/xauth:  file /Users/sanderso/.Xauthority does not exist
Last login: Wed Feb 20 15:14:31 2013 from 149.130.206.217
[wwellesl@tempest ~] cd public_html/
[wwellesl@tempest public_html] ls -l mypage.html 
-rw-r--r--. 1 wwellesl wwellesl 0 Jan 26 12:33 mypage.html
[wwellesl@tempest public_html] logout
Connection to tempest closed.

Did you notice that we didn't get the scary message from ssh? Did you also notice the different prompt, so that we know where we are? This is important; it's easy to get confused when you have different shells, all on the same screen, but logged into different machines. (It's not uncommon for me to be logged into 3 or 4 machines from my office machine.)

Oh, and there's the logout command. I didn't teach you that; it's pretty easy to guess what it does. You should always logout of a machine when you're done, because connections do use up resources and a host can't support an infinite number of them.

Finally, we can delete our local file and copy the one from tempest to our local Desktop folder.

  
sci-257-12:Desktop sanderso$ rm mypage.html 
sci-257-12:Desktop sanderso$ scp wwellesl@tempest:public_html/mypage.html .
wwellesl@tempest's password: 
mypage.html                                                         100%    0     0.0KB/s   00:00    
sci-257-12:Desktop sanderso$ ls -l mypage.html 
-rw-r--r--  1 sanderso  WELLESLEY\Domain Users  0 Jan 26 12:35 mypage.html
sci-257-12:Desktop sanderso$ 

Tab completion

The Unix shell has many built-in conveniences for power users and poor typists. One you should know about is "tab completion." If you type part of a filename, enough to identify a unique file in the directory, and you hit the "tab" key (above caps lock on the left side of your keyboard), the shell will fill out the rest of the filename. If your prefix is not unique, the shell will fill out as much as it can, and allow you to make a choice of how to continue.

Here's an example:

[wwellesl@tempest public_html] ls
cgi-bin            dot-bashrc2      pictures.html  wendy.html~
copy-of-bashrc     foo              protected      wendy.shtml
cs110-assignments  other-page.part  wendy.html
[wwellesl@tempest public_html] ls -l pictures.html 
-rw-rw----. 1 wwellesl wwellesl 593 Feb  6  2007 pictures.html
[wwellesl@tempest public_html]

In the example above, I only typed the part in bold; then I hit "tab" and because "pi" uniquely specified "pictures.html" in the current directory, the shell was able to fill out the rest of the filename.

You don't have to do this, of course, but it beats typing the whole name, which is slow and error-prone.

Wildcards

If you want a command such as ls or chmod to apply to several or many files, you can list all of them on the command line, but that can be tedios if there are many files. Wildcards are special characters that match any character, allowing you to specify a pattern for the filenames. (Like a wildcard in a card game. When I was a kid, we played poker with deuces and one-eyed jacks wild: six wildcard characters made for more high-value hands.)

*
The asterisk character matches any character and as many as possible.

Here's the asterisk in action:

[wwellesl@tempest public_html] ls
cgi-bin          cs110-assignments  foo              protected    wendy.shtml
copy-of-bashrc   dot-bashrc2        other-page.part  wendy.html
copy-of-bashrc~  dot-bashrc2~       pictures.html    wendy.html~
[wwellesl@tempest public_html] ls -l *.html
-rw-rw----. 1 wwellesl wwellesl 593 Feb  6  2007 pictures.html
-rw-rw----. 1 wwellesl wwellesl 152 Jan 15  2010 wendy.html
[wwellesl@tempest public_html] ls -l copy*
-rw-r-----. 1 wwellesl wwellesl 204 Jan 25 22:10 copy-of-bashrc
-rw-r-----. 1 wwellesl wwellesl 204 Jan  4  2008 copy-of-bashrc~
[wwellesl@tempest public_html] 

In the examples above, the asterisk matched two filenames that ended with ".html" and two that started with "copy". (Had there been 50 such files, it would have matched all 50.)

Emacs keeps "backup" copies of files, with the tilde at the end of the filename indicating a backup copy. If you want to get rid of all of those, you can easily delete them:

[wwellesl@tempest public_html] ls *~
copy-of-bashrc~  dot-bashrc2~  wendy.html~
[wwellesl@tempest public_html] rm *~
[wwellesl@tempest public_html] ls
cgi-bin         cs110-assignments  foo              pictures.html  wendy.html
copy-of-bashrc  dot-bashrc2        other-page.part  protected      wendy.shtml
[wwellesl@tempest public_html]
      

Just be super careful using both rm and the asterisk; it's really easy to delete all your files!

Command Summary

There are, of course, many other useful commands, but these should get you started. Here they all are, with links to man pages, thanks to linuxcommand.org:

  1. man
  2. ls
  3. cd
  4. pwd
  5. cp
  6. mv
  7. rm
  8. mkdir
  9. rmdir
  10. chmod
  11. tar
  12. gzip
  13. ssh
  14. scp