Welcome!
Everything is fine.

File Upload

File upload is an important feature of many web apps. Think about uploading profile pictures or posting pictures of your vacation or breakfast to Facebook, Instagram or whatever. Your project should find a place for it.

Plan

  1. Announcements
  2. Basics of file upload, both on front-end and back-end
  3. Quiz Questions
  4. Breakout sessions

Announcements

  • Grading status
  • CRUD partners assigned. Please start ASAP.

CRUD Partners

  • 2 Ada+Maya
  • 4 Alyssa+Lyra
  • 6 Amy+Sarah
  • 8 AnnaLieb+Nya
  • 10 AnnaZhou+Catherine
  • 12 Annie+Thea
  • 14 Annissa+Austen
  • 16 Arya+Lauren
  • 18 Becky+Kathy
  • 20 Bella+Kaitlyn
  • 22 Bellen+Chelsea
  • 24 Dechen+Mary Jo
  • 26 Eleanor+Nico
  • 28 Fridah+Maria
  • 30 Jada+Jelimo
  • 32 Sofia+Soo

File Upload Story

First, a story ....

Summary

What are the important points for today?

  • File Upload is an important feature: images, documents, etc. Your projects should try to include this in some way.
  • File Upload is fraught with peril — be conscious of security.
  • New kind of form input and form encoding type (ENCTYPE).
  • Sometimes need to learn about MIME types, such as image/jpeg
  • Two major places to store them:
    • The filesystem, meaning regular files; store pathnames in the database. This is often more convenient and efficient. We'll focus on this.
    • The database (as encoded strings). Do this if you either cannot save to the filesystem or are extremely security conscious (maybe people are supposed to be uploading executable scripts, but you need to keep them safe). We'll punt this topic this semester, but let me know if you personally want to know more.
  • Files are connected to entities by storing the filename as an property in the database document, or having a pattern like the filename is <nm>.jpg. Images in the database are easily connected.
  • Serving images from the filesystem is easy: either use the staticServer feature (for public files) or create a handler that determines the pathname and then does res.sendFile(pathname)
  • File upload forms have:
    • ENCTYPE="multipart/form-data"
    • <INPUT TYPE=FILE ...>
    • method=POST
  • The back-end parses the request, gets the file, and saves it to the filesystem (or the database).

Detailed Review

Review of File Upload and how to use them in Express.

  • Use milter middleware
  • configure it to specify storage (directory and filename), as well as limits
  • the req.file value gets you a file object for each upload.
  • each file has properties like .filename
  • res.sendFile() lets you serve these files

File Upload Example

Here's the main code (open.js) for today:

// start app with either 'npm run dev' or 'node server.js'
// go to http://149.130.15.5:3000/

// standard modules, loaded from node_modules
const path = require('path');
require("dotenv").config({ path: path.join(process.env.HOME, '.cs304env')});
const express = require('express');
const morgan = require('morgan');
const serveStatic = require('serve-static');
const bodyParser= require('body-parser');
const cookieSession = require('cookie-session');
const flash = require('express-flash');
const multer = require('multer');

// our modules loaded from cwd

const { Connection } = require('./connection');
const cs304 = require('./cs304');

// Create and configure the app

const app = express();

// Morgan reports the final status code of a request's response
app.use(morgan('tiny'));

app.use(cs304.logStartRequest);

// This handles POST data
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());

app.use(cs304.logRequestData);  // tell the user about any request data

app.use(serveStatic('public'));
app.set('view engine', 'ejs');

const mongoUri = cs304.getMongoUri();

app.use(cookieSession({
    name: 'session',
    keys: [cs304.randomString(20)],

    // Cookie Options
    maxAge: 24 * 60 * 60 * 1000 // 24 hours
}));
app.use(flash());

// ================================================================
// configure Multer

app.use('/uploads', express.static('uploads'));

function timeString(dateObj) {
    if( !dateObj) {
        dateObj = new Date();
    }
    d2 = (val) => val < 10 ? '0'+val : ''+val;
    let hh = d2(dateObj.getHours())
    let mm = d2(dateObj.getMinutes())
    let ss = d2(dateObj.getSeconds())
    return hh+mm+ss
}

const ALLOWED_EXTENSIONS = ['.jpg', '.jpeg', '.png', '.gif' ];

var storage = multer.diskStorage({
    destination: function (req, file, cb) {
        cb(null, 'uploads')
    },
    filename: function (req, file, cb) {
        // the path module provides a function that returns the extension
        let ext = path.extname(file.originalname).toLowerCase();
        console.log('extension', ext);
        let hhmmss = timeString();
        cb(null, file.fieldname + '-' + hhmmss + ext);
    }
})
var upload = multer(
    { storage: storage,
      // check whether the file should be allowed
      // should also install and use mime-types
      // https://www.npmjs.com/package/mime-types
      fileFilter: function(req, file, cb) {
          let ext = path.extname(file.originalname).toLowerCase();
          let ok = ALLOWED_EXTENSIONS.includes(ext);
          console.log('file ok', ok);
          if(ok) {
              cb(null, true);
          } else {
              cb(null, false, new Error('not an allowed extension:'+ext));
          }
      },
      // max fileSize in bytes
      limits: {fileSize: 1_000_000 }});

// ================================================================
// custom routes here
// collections in the user's personal database

const DB = process.env.USER;
const FILES = 'files';

app.get('/', async (req, res) => {
    const db = await Connection.open(mongoUri, DB);
    let files = await db.collection(FILES).find({}).toArray();
    return res.render('open.ejs', {uploads: files});
});

app.post('/upload', upload.single('photo'), async (req, res) => {
    console.log('uploaded data', req.body);
    console.log('file', req.file);
    // insert file data into mongodb
    const db = await Connection.open(mongoUri, DB);
    const result = await db.collection(FILES)
          .insertOne({title: req.body.title,
                      path: '/uploads/'+req.file.filename});
    console.log('insertOne result', result);
    // always nice to confirm with the user
    req.flash('info', 'file uploaded');
    return res.redirect('/');
});

app.post('/clear', async (req, res) => {
    const db = await Connection.open(mongoUri, DB);
    const result = await db.collection(FILES).deleteMany({});
    return res.redirect('/');
});
    
// ================================================================
// postlude

const serverPort = cs304.getPort(8080);

// this is last, because it never returns
app.listen(serverPort, function() {
    console.log(`File documents will be stored in user's database: ${DB}`);
    console.log(`open http://localhost:${serverPort}`);
});

Here's a link to the fileUpload folder.

Quiz Questions

  • 0
  • 5
  • 14
  • 10
  • 0

I'll answer your quiz questions

Optional Demo

There's a file upload video in the videos collection. But I will do a live demo today. 

cd ~/cs304/apps
cp -rd ~cs304node/apps/fileUpload fileUpload
cd fileUpload

Here are the steps of the demo:

  1. Find a couple of pictures to use for the demo. Put them someplace handy, such as your Desktop
  2. Run the app: node private.js
  3. Register as one username and upload a picture
  4. Click on the /myphotos link, too.
  5. Logout
  6. Register as another username and upload a picture
  7. Try to view the first person's pictures

Starting CRUD

Let's all do the getting started steps together.

Breakouts

  • You can work on Lookup, CRUD, or your project draft

Summary

  • Provide file upload if it's useful and relevant
  • Worry about security, but don't panic