Python 102

The reading is fairly long. It covers Python skills that you may already have, and so hopefully that will be easier to read. We will use all of these skills in our PyMySQL/Flask apps, but these are all generic Python skills.

Python Skills¶

All of you said you know Python, though some of you have said privately that it's been a while, and some parts of it have faded a bit since CS 111 or whenever you last used it. Furthermore, CS 111 doesn't teach the whole language. Here are a few topics that students have wanted more information on. So, if CS 111 (or its equivalent) is Python 101, this is Python 102.

Let me know if there are other aspects of Python you'd like to learn more about.

• keyword arguments
• formatting strings
• Modules
• Import
• Dictionaries
• Dates and Times
• Exceptions

Keyword Arguments¶

Most functions will use positional arguments:

def reciprocal(x):
    return 1/x

def ratio(x,y):
    return x/y

This works well, is compact and simple, and is our basic technique.

It doesn't scale well, though. What if a function takes a dozen arguments and most of them are optional, with default values? A cool feature that Python has (and so do many other languages, but not all) is keyword arguments.

For this example, we have to assume that we are representing a row of a table (think databases) with a Python list. This is for simplicity of the example, not for its realism. The point is the arguments and how we invoke the function, not the function's code.

def make_pet(name, age=None, weight=0.0, kind=None):
    return [name, kind, age, weight]

pet1 = make_pet('scabbers', age=8, weight=1.2, kind='rat')
pet2 = make_pet('hedwig', kind='owl', age=2)
pet3 = make_pet('crookshanks', kind='cat')

print(pet1)
print(pet2)
print(pet3)

The make_pet function has one required, positional argument, and three optional, keyword arguments.

Copy/paste that code into a Python shell to see what it does! You should get something like:

>>> print(pet1)
['scabbers', 'rat', 8, 1.2]
>>> print(pet2)
['hedwig', 'owl', 2, 0.0]
>>> print(pet3)
['crookshanks', 'cat', None, 0.0]

Notice how Scabbers used all four arguments and in their usual order, but Hedwig only used two of the three optional arguments, letting the weight default to 0.0. Hedwig also swapped the order of the two keyword arguments that it used. Finally, Crookshanks left out two keyword arguments, letting them both default.

Keyword arguments allow you to

• omit arguments, in which case they get a default value
• put the keyword arguments in any order

This is very nice! In CS 304, we won't be defining functions with keyword arguments (though you're welcome do if you want), but we will be using many of them. It helps to know about them.

Formatting Strings¶

Python has an interesting history of formatting strings, by which I mean combining fixed text with dynamic data. I'm not going to cover all of it. But here are a few.

Running example: If you have a variable num_students and you want to have a string like there are 30 students in CS 304, but with the number filled in from the variable's value, you're trying to combine some static text with a dynamic data. Let's look at a few ways to do that.

String Concatenation

In Python, you can glue (concatenate) two strings with a + operator. They must be strings. So, you can do:

num_students = 30
msg = 'There are ' + str(num_students) + ' students in CS 304'
print(msg)

Simple and easy, but mixes variables and strings in a way that many people find hard to read. Not my favorite.

C style

Python has a % operator that can combine a format string with a tuple of data:

num_students = 30
msg = 'There are %d students in CS 304' % (num_students,)
print(msg)

This style is considered somewhat obsolete, but you'll see vestiges of it, particularly in prepared queries. Because it's obsolescent, I won't say more now.

The Format Method

This is the technique I advocate:

num_students = 30
msg = 'There are {n} students in CS 304'.format(n=num_students)
print(msg)

This allows you to name the placeholders in your format string, and fill them in with keyword-argument syntax. Very nice.

If the string is long, you can use triple-quoted strings. If you want to put the .format on the next line, be sure to wrap the whole right hand side in parentheses:

msg = ('''
{elven} for the Elven Kings, under the sky
{dwarf} for the Dwarf Lords, in their halls of stone
{mortal} for the Mortal Men, doomed to die,
{sauron} for the Dark Lord on his dark throne...'''
       .format(elven='Three',
               dwarf='Seven',
               mortal='Nine',
               sauron='One'))
print(msg)

The .format method's syntax also bears some resemblance to Jinja2 templating. That's not a coincidence.

There's a variant of the .format method that is positional:

num_students = 30
msg = 'There are {} students in CS 304'.format(num_students)
print(msg)

By "positional" I mean that the {} placeholders are replaced with arguments by their position in the string. So:

msg = 'I like {}, {}, {} and {}'.format('apples','bananas','chocolate','dairy')
print(msg)

The positional form is nice when things are short, but is more awkward when the string gets longer and there are more filler values. Then the keyword style is nicer. Keyword arguments can also be more clear:

msg = 'Dividing {num} by {denom} yields {quo}'.format(denom=4, num=1, quo=0.25)
print(msg)

F-strings

A very new (as of Python 3.6) option for string formatting is f-strings. None of my examples use these (since they pre-date Python 3.6), but you're welcome to use f-strings in your programs if you'd like.

I have a bit more to say about f-strings if you're curious.

Modules¶

Pretty much every civilized programming language has a way to break up big complex programs into separate parts, usually called modules.

A module is a collection of Python functions, classes, variables and such. A package is a collection of modules in a hierarchy. We're not going to worry about the difference, and I'll use the terms pretty interchangeably.

Usually a module is contained in a file, and the name of the file matches the name of the module. For example, the cs304dbi module is in the cs304dbi.py file.

If you'd like to understand more about the concept, you can read more about the concept of modules

To use the code in a module, adding its functionality to your own program, you import the module. We'll turn to that now.

Import¶

Simple uses of import are, well, simple. To find out your UID, we can import the os module and use the getuid method defined by it.

import os

myuid = os.getuid()
print(myuid)

The preceding is a complete, working Python script, though it doesn't do much.

Because the os module is one of the standard modules, we don't have to install it in our virtual environment.

Another standard module is the datetime module. The datetime modules defines several classes, including one called datetime, which combines the features of the date class and the time class.

Here's a simple use of the datetime module to print the current time. We saw this code in the servertime.py example:

import datetime

dt = datetime.datetime.now()
print(dt.strftime('%A %B, %d, %Y at %H:%M %p in %Z'))

That imports the datetime module, and to refer to the datetime name in that module, we have to use the dotted notation: module.name. Then to use the now method, we use yet another dot.

It can be convenient sometimes to import the names so that you can avoid the module.name syntax. So the example above could be re-written as:

from datetime import datetime

dt = datetime.now()
print(dt.strftime('%A %B, %d, %Y at %H:%M %p))

both print

Saturday September, 19, 2020 at 13:33 PM

Note: it looks redundant to say from datetime import datetime but those are different things that happen to have the same name. The first datetime is the name of a module. Inside that module is a class called datetime. This is a little confusing, but don't let it throw you. As you know, a module is a kind of collection or container, and a class can be in that collection/container.

The strftime method formats the datetime object. The cryptic string that is the first argument contains a template with a bunch of formatting codes. There are a ton of these % format codes. (This might remind you of one of Python's string-formatting techniques; no coincidence.) This particular string shows the day of the week, as as string in the local language, the name of the month, the date of the month, the hour and minute with am/pm. There's more about dates and times in that later section.

From module import¶

Returning to the import statement, we have:

from module import (name1, name2, ...)

We'll use this technique a lot with Flask, to avoid having to prefix all of the names with flask. like this:

from flask import (Flask, render_template, make_response, url_for, request,
                   redirect, flash, session, send_from_directory, jsonify)

The parentheses are only necessary if, as above, we want to list more names than can fit on one line.

Warning: importing names like this means that they will conflict with any local identical names. So, we couldn't have a different name in our app called, for example, request, because this would be the same name as the one we are importing from Flask. This is not really a new idea: you already know that you can't have two different global variables named fred. Importing just gives you a new source of names.

Import *¶

Note that if we want to import every name from another module, we can use a wildcard:

from some_module import *

This is generally considered a bad idea. It completely tears down the wall between two modules. The point of modules is to build walls between different parts of our program, thereby giving more freedom of naming and clarifying where things come from. So tearing down the walls is usually wrong.

However, sometimes it can be useful. See advanced import, but only once you know some Flask.

Importing Local Files¶

We've looked at importing os and datetime, both of which are standard modules, so they don't have to be installed. However, what if you want to import a file of your own making. In fact, we saw just that, above: the file all_examples.py imported names from example_form_processing_one_route.py.

For that to work, both files must be in the same directory. Say app.py imports servertime. If you copy app.py to another directory and run it, you'll get an error like this:

$ python app.py 
Traceback (most recent call last):
  File "app.py", line 1, in <module>
    import servertime
ModuleNotFoundError: No module named 'servertime'

If that happens, consider the following possibilities:

• It's not in your local directory
• You misspelled the name of the module
• It's not in your venv

Dictionaries¶

Dictionaries are very useful and powerful. Every civilized language should have them, and, fortunately, you are living in time when most of the languages you will use will have them. When we do queries of the database, we will typically return our values as dictionaries. (But see the next section about tuples.)

If you've forgotten a lot about dictionaries in python that W3Schools link will refresh your memory.

Terminology note: Dictionaries store key/value pairs. The key is how we look up the value. This is similar to how we look up a row in a database table: we look it up with a key, which uniquely specifies the row we want. So the term key is used in a similar way for both Python dictionaries and SQL tables: it's the way you look something up.

This section reviews a couple of useful methods for Python dictionaries. See online references for more information about all python dictionary methods

Dictionaries can be used as a generalization of arrays:

pets = {}
pets['harry'] = 'hedwig'
pets['ron'] = 'scabbers'
pets['hermione'] = 'crookshanks'
for owner,pet in pets.items():
   print('{} owns {}'.format(owner, pet))

The first few lines creates an empty dictionary and then stores values in it using assignment statements similar to those for lists.

The .items() method returns an iterator with a tuple for each key/value pair.

(Also notice the use of the positional format method.)

Running that code yields:

harry owns hedwig
ron owns scabbers
hermione owns crookshanks

We can also read values out using the square bracket notation:

print('Harry owns {}'.format(pets['harry']))

But there's a small pitfall in that. If we mispell the key or otherwise use a key that's not in the dictionary, we get an error, specifically a KeyError:

>>> print('Hermione owns {}'.format(pets['hermoine']))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'hermoine'
>>> 

KeyErrors are annoying at best. Sometimes, we aren't sure whether the value is in the dictionary. An alternative is to use the .get() method, which has the key (as a string) as the first argument and takes an optional second argument that is default value. That is, it's the value to return if the key is not in the dictionary. For example:

>>> print('Malfoy owns {}'.format(pets.get('malfoy','no pet')))
Malfoy owns no pet

If you omit the second argument, None is used:

>>> print('Malfoy owns {}'.format(pets.get('malfoy')))
Malfoy owns None

To remove a value from a dictionary, we can use the .pop() method. So, after The Prisoner of Azkaban, we would do:

pets.pop('ron')

to remove Ron and his pet Scabbers from the pets dictionary.

The Flask system represents requests and other things as "dictionary-like" objects, and consequently we will have occasion to use the .get() and .pop() methods from time to time.

In fact, dictionaries and dictionary-like objects are very common in Computer Science. We'll see lots of key/value collections in this course.

Tuples¶

Tuples and lists are pretty easy to work with, but require numeric indices, and that can lead to opaque code. The following is hard to read:

heros = ('harry','ron','hermione','neville','luna','fred','george',
         'dumbledore','lupin', 'mad-eye','mcgonagall')

print('My favorite hero is {}'.format(heros[7]))  # who?

If the print statement is in a different function from the assignment statement, maybe in a different file, the code becomes downright opaque.

One technique that can help at times is destructuring assignment. Destructuring assignment is a general feature of several languages, including Python and JavaScript. Let's see some examples first:

(x, y, z) = (0, 0, 0)   # initialize all to zero
(a, b) = (b, a)         # swap two variables

The idea is that there's a tuple of variable names on the left hand side and a tuple of values on the right hand side, and each varible is assigned the corresponding value. So, we could compute the next step of the Fibonacci sequence in one step:

(fib_next, fib, fib_prev) = (fib+fib_prev, fib_next, fib)

In Python, the parentheses on the left hand side are optional:

x, y, z = 0, 0, 0   # initialize all to zero
a, b = b, a         # swap two variables
fib_next, fib, fib_prev = fib+fib_prev, fib_next, fib

If we fetch a tuple from the database, we can pull it apart in one easy step:

curs.execute('select min(sales), average(sales), max(sales) from ...')
min, avg, max = curs.fetchone()

Here's another example. Suppose we get four columns in a tuple query and we don't want to be referring to row[3] and the like, we can create four local variables in one quick assignment:

curs = dbi.cursor(conn)
curs.execute('select nm, name, birthdate, addedby from person')
for row in curs.fetchall():
    nm, name, bday, staff = row
    print('{} ({}) was born on {} and added by {}'.format(name, nm, bday, staff)

The next to last line is an assignment of four variables at once, being the four elements of the tuple. (Lists work the same way.)

Tuple Pitfall

There's one pitfall for tuples that is a little tricky, because they use parentheses. Consider the following:

x = ()
y = (1)
z = (2,3)

This looks like three assignments of tuples, of length 0, 1 and 2, but not so:

• x is a tuple of length 0
• y is an integer
• z is a tuple of length 2

This is weird and inconsistent, but arises because we can also use parentheses for non-tuple use in arithmetic expressions, so the right hand side of the assignment to y looks like that. So, how do you get a tuple of length 1? You have to put a comma in the parentheses. The following works as expected:

x = ()
y = (1,)
z = (2,3)

Dates and Times¶

MySQL represents dates and times with the ISO standard of YYYY-MM-DD HH:MM:SS. Our PyMySQL API automatically parses those and represents them as Python datetime objects.

If the user has handed you a date as 'MM/DD/YYYY' and you want to parse it, the datetime module provides you the strptime method to parse a time string:

from datetime import datetime

string = '02/14/2020' # valentine's day
dt = datetime.strptime(string, '%m/%d/%Y')
print(dt.strftime('%A'))        # day of the week
print(dt.strftime('%Y-%m-%d'))  # MySQL format

The output is:

Friday
2020-02-14

Exceptions¶

Sometimes things go wrong, which we call run-time (or runtime) errors. There may be other anomalous situations which aren't really errors, but are similar.

One option is to check first (look before you leap). Here's an example:

def inefficiency(distance, fuel):
    if distance == 0:
        return 'div zero'    # special value for /0
    else:
        return fuel/distance

Then, of course, the caller has to think about the error as well. On my old gas-powered Subaru Outback, the instantaneous mileage display would have "--" in the two-digit dashboard display when I was idling at a stoplight: infinite inefficiency.

def update_display(distance, fuel):
    val = inefficiency(distance, fuel)
    if val == 'div zero':
        display.update('inefficiency,'--')
    else:
        display.update('inefficiency,'{:2d}'.format(val))

(The {:2d} is a code for 2-digit string formatting in Python. See Python String formatting if you'd like to learn more.)

The "check-first" strategy is fine, but we have to return special values. Sometimes a very long way. So, I'm going to switch to a more abstract example.

Suppose function a calls function b which calls function c which calls function d which calls function e. The last function might get an error. The function a might be the best function to display the error message or otherwise handle it.

For example, function e function discover that a filename was in use and therefore can't be created. Function a might be a user-interface function and therefore has the ability to tell the user that they chose a filename that is in use and they should pick a different one. Yuck!

• We don't want a to have to do the filename checking that e does, so we can't check there. Function e should do so (or get the error)
• Function e would have to return a special error code, which would have to be passed through d, c and b to get to a. That junks up those functions for no good reason.

This style is still used in low-level languages like C, which has an errno code for many operating system calls which then has to be passed around.

For situations like these, higher-level languages like Python (and Java and JavaScript) use Exceptions:

• The code gets the error or notices the anomaly, like e, raises or throws the exception.
• The code that handles the error/anomaly, like a, catches the exception.

(If you're curious about how this is implemented, you can talk to me in office hours. Briefly, the exception-raising code crawls back through the call stack to find a handler.)

So, the handler does:

def a():
    filename = input('enter filename: ')
    try:
        b(filename, other, args)
    except Exception:
        print('Sorry, that filename is in use. Please try again')

and the code that discovers the error raises the exception:

def e(filename):
    if os.file_exists(filename):
        raise Exception
    normal code

Ideally, you create or use different exceptions for different kinds of anomalous conditions, which might be handled differently. This example shows some of the built-in exceptions:

def a():
    filename = input('enter filename: ')
    try:
        b(filename, other, args)
        f(more, code, here)
    except FileExistsError:
        print('Sorry, that filename is in use. Please try again')
    except KeyError:
        print('Sorry, we got a key error. Oops.')
    except ZeroDivisionError:
        format('Sorry, I divided by zero. My bad.)

Python can allow us to define our own exceptions, but we won't be doing so in this course. Feel free to search the web if you want to learn how.

We typically will not raise exceptions in this course, but PyMySQL and Flask might do so. Sometimes, we will catch these exceptions.

One cool effect of Exceptions is that sometimes you abandon checking beforehand and just plunge ahead, allowing the exception to occur. So, we could re-write some of our code above:

def inefficiency(distance, fuel):
    return fuel/distance  # might raise an exception

def update_display(distance, fuel):
    try:
        val = inefficiency(distance, fuel)
        display.update('inefficiency,'{:2d}'.format(val))
    except ZeroDivisionError:
        display.update('inefficiency,'--')

This nicely segregates the code that handles the unusual or anomalous situation from the usual or normal situations.

Conclusion¶

If you discover some aspect of Python that would have been helpful to review as part of CS 304, please let me know.

You can stop reading here, unless you want to know more. If so, you can read the appendices