File Upload¶

File upload is an important feature of many web apps. Think about uploading profile pictures or posting pictures of your vacation or breakfast to Facebook.

Plan¶

1. Announcements
2. Basics of file upload, both on front-end and back-end
3. Quiz Questions
4. Breakout sessions

Announcements¶

• Grading status

File Upload Story¶

First, a story ....

What are the important points for today?

• File Upload is an important feature: images, documents, etc. Your projects should try to include this in some way.
• File Upload is fraught with peril.
• New kind of form input and form encoding type (ENCTYPE).
• Sometimes need to learn about MIME types, such as "image/jpeg"
• Two major places to store them:
  • The filesystem, meaning regular files; store pathnames in the database. This is usually more convenient and efficient. We'll focus on this.
  • The database (BLOBS). Do this if you either cannot save to the filesystem or are extremely security conscious (maybe people are supposed to be uploading executable scripts, but you need to keep them safe). We'll punt this topic this semester, but let me know if you personally want to know more.
• Images in the filesystem are connected to entities by storing the filename as an attribute (column), or having a pattern like the filename is <nm>.jpg. Images in the database are easily connected, though you do need to be careful in displaying them to your terminal.
• Serving images from the filesystem is easy: just generate a URL pointing to the file. Flask/Apache does the rest. (Flask requires a bit more, but not difficult.) Serving images from BLOBs means additional handlers that pull out the data and return it as the response along with a MIME header.
• File upload is accomplished by
  • ENCTYPE="multipart/form-data"
  • INPUT TYPE=FILE
  • POST method
• Flask applications run as you, but the uploaded files are either in memory or saved to a temporary location. An object similar to Python's built-in file object is returned; it has a useful .save() method to save the contents in a pathname of your specification.
• Responses should supply the correct MIME type for any file. Determining the correct MIME type is best done with the python-magic and python-magic

Summary¶

• File upload is an important feature of many web databases, not just FaceBook and Yelp. You should consider its usefulness in your project.
• File upload can be a security risk, so you should be careful.

Specific technology pieces:

• form enctype: <FORM ENCTYPE="MULTIPART/FORM-DATA" ...>
• form inputs: <INPUT TYPE=FILE NAME=name-of-file-input>
• folders: you'll probably create a folder for uploaded files, say, uploads, next to templates and static. You could, of course, have different folders for different kinds of files, etc.
• files: you'll need a naming scheme for your uploaded files.
  • You could use a counter: fileNNN.jpg
  • You could use an ID: fileNM.jpg
  • You could use a timestamp: file-2022-04-01-23-01-12.jpg (your phone's camera does that)
• You'll probably store the filename in an appropriate database table, unless the naming scheme allows the filename to be inferred from other data.
• Flask's request object has a files dictionary, containing all the uploaded files, as Python file objects.
• file objects have a .save() method, allowing you to save the file to your desired pathname
• You'll construct your pathname from the uploads folder and the chosen filename using os.path.join(folder, file) from the Python os module.
• Avoid user data in the filename/pathname, though that is possible.
• Use secure_filename if there's any user data in the filename
• from werkzeug.utils import secure_filename
• To deliver files to the browser, use send_from_directory(uploaddir, filename)
• The browser requests files (e.g. images) in a separate request, so you have to have a route/endpoing to respond to those requests.

Detailed Review¶

Review of File Upload and how to use them in Flask.

Flask File Uploads.

• the request.files dictionary lets you get a file object for each upload.
• each file has properties like .filename
• The .save() method allows you to save the file to a name you specify.
• The secure_filename function strips off "../" and other dangerous stuff
• send_from_directory lets you serve these static files from a directory you specify.

File Upload Example¶

Here's the main Python code (app.py) for today:

'''This example implements file upload to the filesystem'''

from flask import (Flask, render_template, make_response, request, redirect,
                   url_for, session, flash, send_from_directory, Response)
from werkzeug.utils import secure_filename
app = Flask(__name__)

import sys, os, random
import imghdr
import cs304dbi as dbi

app.secret_key = ''.join([ random.choice(('ABCDEFGHIJKLMNOPQRSTUVXYZ' +
                                          'abcdefghijklmnopqrstuvxyz' +
                                          '0123456789'))
                           for i in range(20) ])

# This gets us better error messages for certain common request errors
app.config['TRAP_BAD_REQUEST_ERRORS'] = True

# new for file upload
app.config['UPLOADS'] = 'uploads'
app.config['MAX_CONTENT_LENGTH'] = 1*1024*1024 # 1 MB

@app.route('/')
def index():
    return render_template('base.html')

@app.route('/pic/<nm>')
def pic(nm):
    conn = dbi.connect()
    curs = dbi.dict_cursor(conn)
    numrows = curs.execute(
        '''select filename from picfile where nm = %s''',
        [nm])
    if numrows == 0:
        flash('No picture for {}'.format(nm))
        return redirect(url_for('index'))
    row = curs.fetchone()
    return send_from_directory(app.config['UPLOADS'],row['filename'])

    
@app.route('/pics/')
def pics():
    conn = dbi.connect()
    curs = dbi.dict_cursor(conn)
    curs.execute('''select nm,name,filename
                    from picfile inner join person using (nm)''')
    pics = curs.fetchall()
    return render_template('all_pics.html',n=len(pics),pics=pics)

@app.route('/upload/', methods=["GET", "POST"])
def file_upload():
    if request.method == 'GET':
        return render_template('form.html',src='',nm='')
    else:
        try:
            nm = int(request.form['nm']) # may throw error
            f = request.files['pic']
            user_filename = f.filename
            ext = user_filename.split('.')[-1]
            filename = secure_filename('{}.{}'.format(nm,ext))
            pathname = os.path.join(app.config['UPLOADS'],filename)
            f.save(pathname)
            conn = dbi.connect()
            curs = dbi.dict_cursor(conn)
            curs.execute(
                '''insert into picfile(nm,filename) values (%s,%s)
                   on duplicate key update filename = %s''',
                [nm, filename, filename])
            conn.commit()
            flash('Upload successful')
            # decided to just re-render the form (rather than use
            # post-redirect-get) so that we can display the uploaded
            # picture.
            return render_template('form.html',
                                   src=url_for('pic',nm=nm),
                                   nm=nm)
        except Exception as err:
            flash('Upload failed {why}'.format(why=err))
            return render_template('form.html',src='',nm='')
            

if __name__ == '__main__':
    if len(sys.argv) > 1:
        # arg, if any, is the desired port number
        port = int(sys.argv[1])
        assert(port>1024)
    else:
        port = os.getuid()
    dbi.conf()                  # connect to personal database
    app.debug = True
    app.run('0.0.0.0',port)

Here's a link to the upload folder.

Quiz Questions¶

More than half are feeling comfortable with this material, which is great, but there's a significant number who aren't.

I'll answer your quiz questions

Optional Demo¶

There's a file upload video in the videos collection. But I will do a live demo today.

cd ~/cs304
source ~/cs304/venv/bin/activate
cp -r ~cs304/pub/downloads/upload/ upload
cd upload

Here are the steps of the demo:

1. Activate the venv if you haven't already
2. Use the create-table-picfile.sql file to create a table picfile that stores the name of the picture file associated with a person (using nm)
3. Use the insert-picfiles.py file to insert all the files that are already in the uploads folder
4. Run app.py and try the links:
   • /pic/<nm> to see a single picture
   • /pics/ to see all pictures (view source to see the embedded URLs, all generated by url_for()!)
   • /upload/ to upload an nm/file pair. Do view source to see the <form> element.
5. Optionally, do mysql < show-picfiles.sql in the terminal to see the current contents
6. Optionally, use the delete-image.sh command if you want to delete a particular image (say to re-run an upload). Feel free to read that file to see how it works.

Commands to copy/paste:

mysql < create-table-picfile.sql
python insert-picfiles.py
mysql < show-picfiles.sql
python app.py

Breakouts¶

• You can work on CRUD
• You can work on your project draft

Anonymous Comment¶

I got this yesterday evening from the Anonymous Feedback form:

This is somewhat of a vent, and I apologize in advance because it’ll likely be rather lengthy. I’m really frustrated by the amount of generative AI being used by my classmates, and I’m not really sure where else to unload this, since I don’t want my partners to face consequences. I’ve noticed in my last 2 pair assignments that my partners turn to AI at the first instance of trouble instead of checking older notes on the site or even just googling it. It’s so immensely frustrating for us both to be confused and then notice my partner trying to be subtle about turning their screen away from me as I see ChatGPT open in their tabs. Or when I have asked classmates before for help with assignments and can immediately tell the code they wrote was not written by them, and then feel uncomfortable with their help. I’m sure this is a concern professors have as well and I have no idea how to remedy it, but I’m just so unbelievably tired of working really hard on assignments and having to explain tons of concepts to my partners who won’t take 5 minutes to reread notes. I’m not the tattle tale type, and I really don’t mind helping my partners out or even explaining concepts they don’t understand well. But it’s entirely different when there’s no effort on their end, and it’s evident they just want to get the work done as fast as possible, even if it means neither of us understand what’s going on.

• Thanks for taking the time to send this.
• I truly sympathize with your frustration and fatigue
• I'm having lots of reactions, some emotional and some practical
• I'm still figuring that out
• I'd like to hear from all of you. Maybe not right now, but send me an email over the weekend or use the anonymous feedback form
• We'll talk again on Tuesday.

Summary¶

• Provide file upload if it's useful and relevant
• Worry about security, but don't panic