File Upload¶

File upload is an important feature of many web apps. Think about uploading profile pictures or posting pictures of your vacation or breakfast to Facebook.

Plan¶

1. Announcements
2. Basics of file upload, both on front-end and back-end
   • I'm going to zoom through this; people seemed bored on Tuesday
3. Quiz Questions
4. Breakout sessions

Announcements¶

• Grading status

File Upload Story¶

First, a story ....

What are the important points for today?

• File Upload is an important feature: images, documents, etc. Your projects should try to include this in some way.
• File Upload is fraught with peril.
• New kind of form input and form encoding type (ENCTYPE).
• Sometimes need to learn about MIME types, such as "image/jpeg"
• Two major places to store them:
  • The filesystem, meaning regular files; store pathnames in the database. This is usually more convenient and efficient. We'll focus on this.
  • The database (BLOBS). Do this if you either cannot save to the filesystem or are extremely security conscious (maybe people are supposed to be uploading executable scripts, but you need to keep them safe). We'll punt this topic this semester, but let me know if you personally want to know more.
• Images in the filesystem are connected to entities by storing the filename as an attribute (column), or having a pattern like the filename is <nm>.jpg. Images in the database are easily connected, though you do need to be careful in displaying them to your terminal.
• Serving images from the filesystem is easy: just generate a URL pointing to the file. Flask/Apache does the rest. (Flask requires a bit more, but not difficult.) Serving images from BLOBs means additional handlers that pull out the data and return it as the response along with a MIME header.
• File upload is accomplished by
  • ENCTYPE="multipart/form-data"
  • INPUT TYPE=FILE
  • POST method
• Flask applications run as you, but the uploaded files are either in memory or saved to a temporary location. An object similar to Python's built-in file object is returned; it has a useful .save() method to save the contents in a pathname of your specification.
• Responses should supply the correct MIME type for any file. Determining the correct MIME type is best done with the python-magic; see python-magic on pypi

Summary¶

• File upload is an important feature of many web databases, not just FaceBook and Yelp. You should consider its usefulness in your project.
• File upload can be a security risk, so you should be careful.

Specific technology pieces:

• form enctype: <FORM ENCTYPE="MULTIPART/FORM-DATA" ...>
• form inputs: <INPUT TYPE=FILE NAME=name-of-file-input>
• folders: you'll probably create a folder for uploaded files, say, uploads, next to templates and static. You could, of course, have different folders for different kinds of files, etc.
• files: you'll need a naming scheme for your uploaded files.
  • You could use a counter: fileNNN.jpg
  • You could use an ID: fileNM.jpg
  • You could use a timestamp: file-2022-04-01-23-01-12.jpg (your phone's camera does that)
• You'll probably store the filename in an appropriate database table, unless the naming scheme allows the filename to be inferred from other data.
• Flask's request object has a files dictionary, containing all the uploaded files, as Python file objects.
• file objects have a .save() method, allowing you to save the file to your desired pathname
• You'll construct your pathname from the uploads folder and the chosen filename using os.path.join(folder, file) from the Python os module.
• Avoid user data in the filename/pathname, though that is possible.
• Use secure_filename if there's any user data in the filename
• from werkzeug.utils import secure_filename
• To deliver files to the browser, use send_from_directory(uploaddir, filename)
• The browser requests files (e.g. images) in a separate request, so you have to have a route/endpoing to respond to those requests.

Detailed Review¶

Review of File Upload and how to use them in Flask.

Flask File Uploads.

• the request.files dictionary lets you get a file object for each upload.
• each file has properties like .filename
• The .save() method allows you to save the file to a name you specify.
• The secure_filename function strips off "../" and other dangerous stuff
• send_from_directory lets you serve these static files from a directory you specify.

File Upload Example¶

Here's the main Python code (app.py) for today:

'''This example implements file upload to the filesystem'''

from flask import (Flask, render_template, make_response, request, redirect,
                   url_for, session, flash, send_from_directory, Response)
from werkzeug.utils import secure_filename
app = Flask(__name__)

import sys, os
import cs304dbi as dbi

import secrets
app.secret_key = secrets.token_hex()

# This gets us better error messages for certain common request errors
app.config['TRAP_BAD_REQUEST_ERRORS'] = True

# new for file upload
app.config['UPLOADS'] = 'uploads'
app.config['MAX_CONTENT_LENGTH'] = 1*1024*1024 # 1 MB

@app.route('/')
def index():
    return render_template('base.html')

@app.route('/pic/<nm>')
def pic(nm):
    conn = dbi.connect()
    curs = dbi.dict_cursor(conn)
    numrows = curs.execute(
        '''select filename from picfile where nm = %s''',
        [nm])
    if numrows == 0:
        flash('No picture for {}'.format(nm))
        return redirect(url_for('index'))
    row = curs.fetchone()
    return send_from_directory(app.config['UPLOADS'],row['filename'])

    
@app.route('/pics/')
def pics():
    conn = dbi.connect()
    curs = dbi.dict_cursor(conn)
    curs.execute('''select nm,name,filename
                    from picfile inner join person using (nm)''')
    pics = curs.fetchall()
    return render_template('all_pics.html',n=len(pics),pics=pics)

@app.route('/upload/', methods=["GET", "POST"])
def file_upload():
    if request.method == 'GET':
        return render_template('form.html',src='',nm='')
    else:
        try:
            nm = int(request.form['nm']) # may throw error
            f = request.files['pic']
            user_filename = f.filename
            ext = user_filename.split('.')[-1]
            filename = secure_filename('{}.{}'.format(nm,ext))
            pathname = os.path.join(app.config['UPLOADS'],filename)
            f.save(pathname)
            conn = dbi.connect()
            curs = dbi.dict_cursor(conn)
            curs.execute(
                '''insert into picfile(nm,filename) values (%s,%s)
                   on duplicate key update filename = %s''',
                [nm, filename, filename])
            conn.commit()
            flash('Upload successful')
            # decided to just re-render the form (rather than use
            # post-redirect-get) so that we can display the uploaded
            # picture.
            return render_template('form.html',
                                   src=url_for('pic',nm=nm),
                                   nm=nm)
        except Exception as err:
            flash('Upload failed {why}'.format(why=err))
            return render_template('form.html',src='',nm='')
            

if __name__ == '__main__':
    if len(sys.argv) > 1:
        # arg, if any, is the desired port number
        port = int(sys.argv[1])
        assert(port>1024)
    else:
        port = os.getuid()
    dbi.conf()                  # connect to personal database
    app.debug = True
    app.run('0.0.0.0',port)

Here's a link to the upload folder.

Quiz Questions¶

I'll answer your quiz questions

Optional Demo¶

There's a file upload video in the videos collection. But I will do a live demo today.

cd ~/cs304
source ~/cs304/venv/bin/activate
cp -r ~cs304flask/pub/downloads/upload/ upload
cd upload

Here are the steps of the demo:

1. Activate the venv if you haven't already
2. Use the create-table-picfile.sql file to create a table picfile that stores the name of the picture file associated with a person (using nm)
3. Use the insert-picfiles.py file to insert all the files that are already in the uploads folder
4. Run app.py and try the links:
   • /pic/<nm> to see a single picture
   • /pics/ to see all pictures (view source to see the embedded URLs, all generated by url_for()!)
   • /upload/ to upload an nm/file pair. Do view source to see the <form> element.
5. Optionally, do mysql < show-picfiles.sql in the terminal to see the current contents
6. Optionally, use the delete-image.sh command if you want to delete a particular image (say to re-run an upload). Feel free to read that file to see how it works.

Commands to copy/paste:

mysql < create-table-picfile.sql
python insert-picfiles.py
mysql < show-picfiles.sql
python app.py

Breakouts¶

• You can work on CRUD
• You can work on your project draft

Summary¶

• Provide file upload if it's useful and relevant
• Worry about security, but don't panic