CS304: Frequently Asked Questions¶

Whenever we discover something, solve a knotty bug, or come across anything else that is worth keeping around as a reminder, please let me know so that I can add it to this list.

How can I resubmit a file using drop¶

If you try to submit the same filename a second time, it will fail:

drop cs304flask file.tar
File /home/cs304flask/drop/ww101/file.tar has already been dropped and will not be removed.
Please drop your file under another filename.

Because the drop command transfers ownership of the file to the recipient, you can't delete it, any more than you can delete an email you already sent someone. But you can send a followup email, right? Similarly, with drop you can rename the file and drop the renamed file.

mv file.tar file_revised.tar
drop cs304flask file_revised.tar
successful drop

How do I copy files from my partner's account to my own?¶

You can't copy them from your partner, but your partner can drop the files to you.

See this explanation for more detail, but the quick summary is that your partner should do the following:

tar cf dir.tar dir/
drop youracct dir.tar

The first command creates a tar file, and second command drops the tarfile to your account.

Then, you can go to your drop folder, untar them, and copy them to where you want them.

How do I create a Virtual Environment¶

Conceptually, the steps are these:

1. get to the directory where you want the venv to be
2. delete any old venv, if any
3. create the virtual environment in a folder whose name is of your choosing.

Typically in CS 304, you will call the folder venv, since you'll only have one venv folder and it'll be under your ~/cs304 folder. But as you get more sophisticated with your virtual environments, you'll want to name them by the project or whatever.

Here are the commands:

cd ~/cs304/
rm -r venv
python -m venv venv

That last command might be confusing. The -m venv means to load and run the venv module, which is built-in to Python. The last venv is the name of the folder to create as a virtual environment. If you wanted to give it a different name, you'd do:

python -m venv my-venv

If you have any trouble, talk to Scott or a tutor.

How do I use (activate) my virtual environment?¶

Whenever you login to your Tempest account to work on CS 304, you would do the following:

source ~/cs304/venv/bin/activate

(The preceding assumes that you created the virtual environment as described above.)

You can then work in any directory using the Python modules in your virtual environment.

As a shortcut, your shell will remember past commands that you've executed, so you can repeat the most recent source command by doing the following:

!source

How do I install PyMySQL/Flask/etc in my Virtual Environment?¶

The pip command (which is customized to your virtual environment), is used to install Python packages into a virtual environment. So, you would do the following:

source ~/cs304/venv/bin/activate
pip install flask
pip install pymysql
pip install bcrypt

If you're already activated the venv (see previous question), you can skip the activate command.

How do I install the DBI module?¶

Our homegrown cs304dbi.py module is not hosted in the cloud, so it is not installed using pip, but is installed by directly copying it into your venv. The following command will do the trick:

cp ~cs304flask/public_html/downloads/pymysql/cs304dbi.py ~/cs304/venv/lib/python3.9/site-packages/

If you located or named your virtual environment folder differently, that command will need to be modified. After activating your virtual environment, you can do:

cp ~cs304flask/public_html/downloads/pymysql/cs304dbi.py $VIRTUAL_ENV/lib/python3.9/site-packages/

Talk to Scott or a tutor if you have any trouble.

How Do I Use the DBI (cs304dbi) module?¶

Assuming you have installed the module, you can use the pydoc command to read the documentation:

pydoc cs304dbi

How do I run my Flask program?¶

It can be helpful to have a checklist. This works for me in the demo account, assuming it has the people_app that we discussed in class:

source ~/cs304/venv/bin/activate
cd ~/cs304/people_app/
python app.py

Then clicking on the link usually works. VS Code automatically sets up an SSH tunnel.

If you already have activated your venv, you can skip the source command, though it won't hurt.

How do I set up an SSH Tunnel?¶

Above, I said that VS Code automatically sets up an SSH tunnel for us. That's great, but sometimes it fails for no apparent reason. The behavior is that the browser just "spins" waiting endlessly for the app to load. Not fun.

Explicitly setting up an SSH tunnel seems to work. The basic idea of an SSH tunnel is that you are using SSH to say that data going to port XYZ on some remote host should be transferred to port ABC on the local host. You can then point your browser to that localhost:abc value and it works.

In the incantation below, I will use 8299 (the port for the cs304guest account) as the remote port, and 8080 as the port on localhost. (8080 is a traditional choice, since web browsers historically listened on port 80, but there's nothing special about that number.) So, here's the incantation:

ssh -L 8080:localhost:8299 cs304guest@cs

This is just like a normal SSH login to the cs304guest@cs account, but we add the -L and its argument to add the SSH tunnel. Obviously, you should substitute your own account instead of the cs304guest and your own port instead of the 8299.

Important note: The SSH command needs to come from your laptop to the server, so run that command outside VS Code. That means:

• on Macs, open a "terminal" window
• on Windows, open a "PowerShell" window

Don't run the ssh command on the server. If you do, the login will be successful, but since the SSH is from cs.wellesley.edu to cs.wellesley.edu it won't tunnel to your laptop. So, make sure you run that ssh -L command on your laptop, not from a remote terminal inside VS Code.

Once you've set up the SSH tunnel, go to your browser and visit localhost:8080.

Flask says "Address is in Use"¶

Your Flask process opens a "port" (a numbered "door" on the computer). Only one process at a time can open a particular port. If you try run your Flask process multiple times simultaneously, you will get an error like this:

[...] python app.py 
 * Serving Flask app 'app'
 * Debug mode: on
Address already in use
Port 1942 is in use by another program. Either identify and stop that program, 
    or start the server with a different port.

(Of course, the port it complains about will be different for you, but that's what the error message looks like.)

Usually, you should stop or kill that other Flask process. See the next question.

How Do I stop (kill) my Flask Process?¶

Assuming that you're at the terminal (shell) that is running Flask, you just type ^C (control-c) to stop (exit) the program. But you probably knew that.

So, this question only arises if you aren't at that terminal or have lost it. That means you have to determine the process ID (PID) of your Flask process and kill that process. The PID is an arbitrary 4 or 5 digit number, and will be different every time you run flask.

I'll describe two approaches, one that uses a simple script that I have supplied, but which won't work on any other computer system, and a general approach that will expand your knowlege of Unix.

Simple way¶

~cs304flask/pub/bin/kill-my-flask-process [port]

With no argument it kills the flask process associated with your UID, which is usually the right choice. If, for some reason, you need to kill a process listening on a different port, you can list that port.

General Way¶

The more general way is to use some Unix commands you probably don't know. Let's assume that the port that your flask process is running on is 2345. You can find open network programs using the netstat command. You can use grep to filter that output to look for your port. That should, hopefully, print just the process PID that you need. Then, you can then kill the process with the kill command, giving the PID you learned from netstat and grep.

Here's an example. You may have to scroll horizontally to see the PID information printed by netstat.

$ netstat -ntlp | grep 2345
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 0.0.0.0:2345            0.0.0.0:*               LISTEN      56789/python
$ kill 56789
$ netstat -ntlp | grep 2345
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)

Note that netstat won't tell you about processes you don't own, so it issues that warning every time. Don't worry about it.

Can I Login to my CS account from off-campus?¶

Yes. The CS server is accessible globally. Just be sure to use the full name: cs.wellesley.edu. For example:

$ ssh cs304guest@cs.wellesley.edu

Can I access my Flask app from off-campus?¶

Yes, but there's a trick. The campus firewall does not allow access to our "UID" ports from off-campus, but you can get inside the campus firewall in one of two ways:

• Use the VPN software that LTS supports.
• Use SSH Port Forwarding/Tunneling VS Code will (usually) automatically set up an SSH tunnel, so you can probably click on the nice blue button. See the picture below.

We must use the VPN when we use CAS.

Checklist for Viewing Flask App from Off-Campus¶

In VS Code:

• login to the server
• activate your venv
• navigate to your app
• run app.py (or the equivalent)
• click the blue button

I'm having trouble with SQL queries that involve the release column.¶

As of MySQL 4+, the release keyword is a reserved word. We can still use it in queries, but it has to be surrounded by back-quotes. So the following fails:

select tt,title,release from movie;

but the following works:

select tt,title,`release` from movie;

Make sure they are backquotes not apostophes. On most keyboards, the backquote shares the key with the tilde on it, not the key with the double-quote character. Look to the left of the digit 1.

In the CRUD assignment, movies with multiple words in the title don't work. It only shows the first word.¶

The full title is probably there in the HTML. Do a "view source" and you will probably see them, but parsed as attributes of the input rather than the value. The template needs to surround a value with quotation marks.

My images or CSS aren't working.¶

Make sure you use url_for() throughout your code. The images and the CSS files need to be in the static folder, not in the templates folder as you might think.

Also, browsers will cache copies of your images and CSS files. To get the latest updates, be sure to use shift+reload.

How can I copy the WMDB to my own database?¶

For the CRUD assignment, I want you not to modify the real WMDB, but instead modify a copy of those tables in your own database. To do that, you need to copy those tables.

~cs304/pub/bin/copy-wmdb

If that fails, it's usually due to a table that can't be dropped due to a foreign key constraint. If you're still puzzled, there's a command in the course account bin directory that might help:

~cs304/pub/bin/copy-wmdb-explain

Of course, if that doesn't help, talk to Scott

How can our team share a database?¶

In your Flask app, you need to configure your database connections so that it connects to the team database. So, if your team database is called team_db, your app.py should have the following code:

dbi.conf('team_db')

That will use your personal credentials (in you ~/.my.cnf file) but that should have permission to access your team database.

If that doesn't work, talk to Scott.

How can I use Multiple Search Criteria in MySQL?¶

Is it possible to combine 2, 3, or even more search criteria in a single SQL query without losing your mind?

Yes, it's possible to allow the user to search by different criteria. It's tricky; much harder than the stuff we've done so far, but it's been done in past student projects.

The first strategy is simple but suffers from combinatorial explosion, namely to choose the search based on different criteria. Here's some pseudo-code:

if cond1:    # search by both num beds and user
    select * from listing where city = %s and user = %s and num_beds = %s 
elif cond2:  # search by num beds only
   select * from listing where num_beds = %s
elif cond3:  # search by city and user
    select * from listing where city = %s and user = %s
elif cond4:
    ...

You can see how the first case checks for both criteria and then we check for singletons. So, we get 3 cases with 2 criteria, and 7 cases with 3 criteria and 2ⁿ-1 with n criteria, hence the combinatorial explosion.

Of course, if you disallow combinations, it's a bit easier.

The alternative is more sophisticated, where you use criteria that can be made into wildcards. Let's just discuss num_beds. We'll use 0 (zero) to mean any number of beds. (Or you could use -1 as the wildcard value.)

if cond1:   # search by num beds:
    bed_val = request.args.get('desired-beds')
else:
    bed_val = 0
# finally
curs.execute('''select * from listing 
                 where (%s = 0 or %s = numBeds) 
                    and ...''',
   [bed_val, bed_val, ...]

Notice how we use bed_val twice: once to compare to the wildcard value and once to compare to the number of beds in the listing. If bed_val is 2, it has to match the number of beds in the listing, but if it's zero, it'll match the constant zero and the number of beds is irrelevant. But the resulting SQL is very large.

Yet another approach is to have your Python code add clauses to an initial query and arguments to a list of values. You should decide if you are combining clauses using AND or OR. Here's some pseudo-code:

sql = 'select * from listing where false ';
args = []
if cond1:  # search by num_beds
     sql += ' or num_beds = %s '
     args.append(num_beds)
if cond2:  # search by city
     sql += ' or city = %s '
     args.append(city)
curs.excute(sql, args)