Reading on Ajax, APIs and JSON¶

This reading covers the concepts behind REST APIs, Ajax and JSON, and why they fit together, and then explores some particular examples, with code. We won't cover all aspects of Ajax and JSON, but you'll be ready for the Ajax assignment.

This reading focuses on the back end, because in this semester we are not learning or assuming you know JavaScript. If you're interested in an alternative reading that covers Ajax both front-end and back-end, follow that link.

This presentation starts with REST APIs and later gets to Ajax, and finally gets to JavaScript and web browsers.

Ajax¶

What is Ajax? Why should we care?

It can be successfully argued that no single technology shift has transformed the landscape of the web more than Ajax. The ability to make asynchronous requests back to the server without the need to reload entire pages has enabled a whole new set of user-interaction paradigms and made DOM-scripted applications possible.

Bear Bibeault & Yehuda Katz
jQuery in Action, 2nd edition

Ajax is a technique where a user's interaction with the page causes some JavaScript to run, which connects (asynchronously) to a server to retrieve some information, and then the page is updated with the results. It's the essential functionality behind such many nice web applications, such as:

• Facebook menus (e.g. specifying the state and place where you went to high school). Posting to someone's timeline; liking a post, commenting on a post.
• Gmail. You can read and delete an email without reloading the whole Gmail window. You can compose and send an email without reloading the whole Gmail window.
• Google Docs. You can edit a document and the browser will automatically save to the cloud without you having to reload the whole document.
• Google maps, etc.

Ajax can also be used by other clients, not necessarily just from JavaScript running in the browser. Let's turn to that; we'll return to web pages running JavaScript at the end of the reading.

APIs, Clients and Servers¶

Throughout the course, we've talked about the client (the web browser) making requests (using HTTP) of a server (our Flask app). The response has always been a web page. We're now going to generalize that notion.

HTTP is a protocol; it just defines the way that a client talks to a server over a network. Each request has an elaborate structure comprising a sequence of header lines, a blank line and a body, with the header including things like the method and the URL. Each response has a similar structure of headers and body, with the header including things like the response code and cookies and the like. You're welcome to learn more about the details of this, but focus on just a few things, namely methods and response codes.

For this reading, the word API, which generally means Application Programming Interface, will be more specifically used to mean a Web API, which is a way to use HTTP to access some database-like application.

The API is typically used to access and manipulate resources, which is another word for the database idea of an entity (or maybe an entity set). In this presentation, I'll give examples of an API for the WMDB Person and Staff entities.

HTTP Methods¶

The HTTP standard defines a lot more methods than GET and POST, which are the only ones we've used so far. (And the only ones supported by the <form> element, so you can't have a <form method='put'> or <form method='delete>, which I find weird, but it's true.) See HTTP request methods for a comprehensive list. These are sometimes called verbs because of their part of speech and active nature.

Let's fill out our list with a few more (this is not all HTTP methods).

• GET The GET method requests a representation of the specified resource. Requests using GET should only retrieve data. We'll use this to return a particular person or staff member from the WMDB.
• POST The POST method is used to submit an entity to the specified resource, often causing a change in state or side effects on the server. We'll use this to insert a new person or staff member to the WMDB. For the person, our API will require that they supply the NM; for the staff member, we'll generate a UID for them and return the UID with the response. The data for the new entity is sent in the body of the request, not in the URL.
• PUT The PUT method replaces all current representations of the target resource with the request payload. So, we'll use this to replace the person. Like POST, the data for the entity is in the body of the request.
• DELETE The DELETE method deletes the specified resource. This one is an easy mapping: We'll delete the specified person.

The HTTP method is only part of the request. The request also includes the URL or endpoint. The combination of the two will map to a particular chunk of Flask code that will implement the operation:

You'll notice that there are many combinations we don't support. We don't support PUT to the /people/ URL. Presumably, this would replace the entire entity set with a new entity set. This is certainly possible, but not required, and we won't.

Response Codes¶

The response from the server can be categorized with various kinds of successes and failures, each captured by a response code (a three-digit number like 200 or 404). There are actually a lot of response codes, and we don't need to learn all of them. Here are a few of the most important ones:

In Flask, we can return one of these codes by supplying a second value returned by the handler function. The following example returns 401 (Unauthorized) if someone who is not logged in makes a POST request to the /people/ endpoint:

@app.request('/people/', methods=['post'])
def post_person():
    if 'userid' not in session:
        return 'must be logged in', 401

Response Data in JSON¶

As mentioned, so far in CS 304, the response data has almost always been a rendered web page. With an API, the client is often interested in data in a standardized form, easily parsed and understood. Nowadays, that format is almost always JSON: the JavaScript Object Notation.

JSON¶

JSON, the JavaScript Object Notation, is a new standard for information interchange between programs, that is, for APIs. So, let's see how JSON works.

JSON is a subset of the JavaScript language, omitting things like function definitions. It's intended to represent data and data structures: compositional data, like lists and dictionaries, and atomic (non-compositional) data like numbers, strings, and booleans. Even though this reading omits JavaScript programming, all back-end developers need to understand the JSON language. Fortunately, it's pretty similar to Python, so it won't be hard to understand.

Here's an example:

{"title": "Dr. Zhivago", 
 "tt": 59113, 
 "year": 1965, 
 "stars": ["Omar Sharif", "Julie Cristie", "Geraldine Chaplin"], 
 "isGreat": true, 
 "director": {"name": "David Lean", 
              "nm": 180, 
              "birth": "1908-03-25"}, 
 "IMDB rating": 8.0} 

Note that values can be scalars (strings, numbers, booleans) or compound (lists and dictionaries); here we have an list of strings for the movie stars and an dictionary for the director.

See JSON.org if you'd like to know more. That site also has a ton of info on packages and modules to process JSON notation in a variety of languages.

JSON notation comprises the following:

• null
• strings
• numbers
• booleans
• lists/arrays (denoted with square brackets around any number of JSON values)
• dictionarys/objects (denoted with curly braces around a set of key/value pairs, where the keys are strings and the value can be any JSON data, the pairs are separated with commas, and a colon appears after a key)

The Python JSON Module¶

Unsurprisingly, Python has a convenient JSON module that can convert Python data structures comprising lists, dictionaries, None, strings, numbers and booleans into JSON and back again. So a Python program can easily parse and use data that is delivered to it in JSON format. It can also produce such data.

The JSON module is also integrated into Flask.

Jsonify the Back End¶

How does the back end create and send a JSON response? The back-end processing is very similar to the normal way; the only difference is the response object. Flask provides a function called jsonify that will turn a Python data structure into a JSON string and wrap it in a response object suitable for sending back to the browser.

Here's an example route where we use a helper function to look up a person by their NM and return the database entry:

@app.route('/v1/people/<nm>')
def person(nm):
    conn = dbi.connect()
    them = mp.get_person(conn, nm)
    if them is None:
        return jsonify({'error': 'No person with that NM'}), 404
    else:
        return jsonify(them)

The return value from this route is the value of jsonify: this is a proper response object that encapsulates the JSON string and makes it ready to send to the browser.

Note that we have to handle errors in a different way; we can't use flashing. Flashing works because we render a template, but the Ajax response isn't rendering a template so flashing is useless. Here, in addition to returning a 404 error code, we put additional information into the response. The client is responsible for looking at it.

Demo¶

Before we dig farther into REST APIs and Ajax, let's see a live example. Visit some of the following links. Note that this data won't necessarily match the WMDB; it reads from a different MySQL database. Note that if you have any trouble from off-campus, use the VPN.

• /people/ returns a list of all the people in the database
• /people/123 returns George Clooney's entry in the database¹
• /people/1234 returns a 404, because there is no such person

Feel free to edit the George Clooney URL to look up someone different.

API Clients¶

This is super cool, but how is it useful? First, we can write programs that interact with a database application (API) that we don't control, such as Twitter, Spotify, Facebook and many others that provide an API.

There's a Unix command-line program called curl. (It's also available on Mac and Windows computers.) Here's an example:

curl https://cs.wellesley.edu/cs304ajax/v2/people/123

Copy/paste that into your shell and see that it gets the correct data (assuming you're inside the firewall, because I've limited the app to on-campus IP address, -- if not, try it after starting the VPN). We could learn a lot more about curl, but let's turn to another client.

Python Requests¶

Using the Python requests module, we can write a Python program that can make HTTP requests. For example:

import requests

resp = requests.get('https://cs.wellesley.edu/cs304ajax/v2/people/123')
data = resp.json()  # a Python dictionary
print("{} with NM {}, born on {}".format(
    data['name'], 
    data['nm'], 
    data['birthdate']))

Try that from a Python shell. If you're off-campus, use the VPN.

Now we can write Python programs that interact with remote databases that we can connect to over HTTP. CS 304 is, after all, named "Databases with Web Interfaces". But the client doesn't have to be a web browser.

REST APIs¶

We could write a Flask app that provides a bunch of useful endpoints that allow clients to interact with the resources (entities) in our database, making up the names and meanings of the endpoints in an ad hoc way, but the world is moving towards REST APIs, which is essentially a bunch of conventions about how endpoints for resources should be named and managed.

Here are a few sites that I found helpful in understanding REST APIs. You're welcome to read them and to suggest better ones. You're not required to read them:

• Introduction to REST APIs
• How to create a RESTful API
• Flask-restful quickstart minimal API
• Designing a RESTful API, step by step
• The 10 REST Commandments

REST Concepts¶

As you know, in databases, we store entities. Our ER diagrams have lots of entity sets. These ideas map pretty well onto the terminology used in REST APIs, where they use the term "resource" for an entity. URLs are Uniform Resource Locators, so each URL will map to a particular entity.

Let's be concrete. Assume we have an entity set for a "to do" list. (This example is drawn from the flask-restful quickstart minimal API example.) The attributes of each "todo" will be just simple strings; in principle, they could be a lot more complex. Each todo will also have a id.

The server (our Flask app) will have a "database" of these todo items. We can implement the database just with a Python global variable holding a dictionary, or we can use a database table; the implementation doesn't matter, only the interaction with the browser.

The key for each todo will be its id, which is a string with an integer appended, like todo17. The value will be a dictionary, with a single key, task. (Representing the value as dictionary makes it easy to imagine adding additional attributes/columns to the entity.)

What operations might we want to support on this database? The following operations seem like a good minimal set:

• LIST all todo entities in the database
• ADD a new todo entity, with the database assigning an ID for it
• for a particular todo entity, identified by its id:
  • GET all information about it
  • REPLACE all information about it
  • DELETE it

So, 5 operations. These are very similar to CRUD:

• ADD is CREATE; it always creates a new entity
• GET is READ
• REPLACE is similar to UPDATE, but the represented entity is entirely replaced.
• DELETE is DELETE

(These are similar enough that people confuse them, resulting in articles like this Stack Overflow post about REST API and POST method and this Medium article about CRUD Mapping to HTTP verbs. You don't have to worry a lot about these subtleties, but just remember that it's not quite as simple as we'd like.)

HTTP Methods and Endpoints¶

Given these ideas, we could design an API for our "to-do entity set" like this:

Again, you'll notice that there are many combinations we don't support. We don't support PUT to the /todos URL. Presumably, this would replace the entire entity set with a new entity set. This is certainly possible, but not required, and we won't.

You'll notice this is very similar to the API we designed for /people/, above. So, we've been using a RESTful API all along.

Well, there are some small differences. The API above generates an ID for a new "todo" item when a new item is POSTed to the API. That's common. In our WMDB API, we require a new person to specify the NM (because we're using the IMDB values), so POST doesn't have to generate and return an NM. So, in our API, POST and PUT are more similar than they might otherwise be.

Handling Methods in Flask¶

These additional HTTP Methods are easily handled in Flask. We just add them to the list of supported methods for a handler function, and we can sort out the different kinds in the body of the function using if statements. Like this:

@app.route('/endpoint/', methods=['GET','POST','PUT','DELETE'])
def handle_endpoint():
    if request.method == 'GET':
        # handle GET
        ...
    elif request.method == 'POST':
        # handle POST
        ...
    elif request.method == 'PUT':
        # handle PUT
        ...
    elif request.method == 'POST':
        # handle DELETE
        ...

Complete Example of People API¶

Let's build a REST API to our personal copy of the WMDB. (Using our personal copy means that we can put in test data without junking up the real WMDB, which has enough junk already.) It will be organized as follows:

• A Flask app.py that provides RESTful endpoints and JSON responses
• A movie_people.py module that provides useful database functions. There won't be anything conceptually new in this file.
• A client.py script that tests our API, using the Python requests module.

We'll play with this code in class.

API¶

Note that in real life applications, the developers sometimes have to change the API. To accommodate that, it's good practice to somehow denote which version of the API you are using. One good way to do that is just to include the API version in the endpoints. I've done that below.

The JSON for a person is:

{nm: 12345, name: "Wendy", birthdate: "2002-05-06", addedby: 1}

The API:

Database Functions¶

Here's the complete code for the database functions. Skim over this, but you shouldn't find anything too surprising.

The functions to insert a new person and to update an existing person both return the database data for that person. This is a useful reply to the API, particularly if action of inserting a person generates an ID (like an NM or a TT). The functions could do a second database lookup, but in this code we already have all the data we need, so we can just return the desired dictionary.

There's a couple dozen lines of testing code below the module; the module code itself is just over 100 lines long.

'''Database interaction functions for the People API, essentially mirroring the endpoints:

get_people(conn)       # returns a list of all people
get_person(conn, nm)   # returns a dictionary of that one person
insert_person(conn, entry) # inserts the person, returning the entry
update_person(conn, entry) # updates the person, returning the entry
delete_person(conn, nm) # deletes the person, returning none

Scott Anderson
March 2022
'''

import cs304dbi as dbi

def get_people(conn):
    '''Returns a list of dictionaries for all people in the WMDB.'''
    curs = dbi.dict_cursor(conn)
    curs.execute('''SELECT nm, name, birthdate, addedby FROM person''')
    return curs.fetchall()

def insert_person(conn, person_entry):
    '''Inserts a person into the database. This version is idempotent,
using the ON DUPLICATE KEY technique, so it's safe to insert George
Clooney and other existing people. Note that this function can raise
KeyError if person_entry is missing any required fields.

    '''
    curs = dbi.cursor(conn)
    nm = person_entry['nm']
    name = person_entry['name']
    birthdate = person_entry['birthdate']
    addedby = person_entry['addedby']
    curs.execute('''INSERT INTO person(nm, name, birthdate, addedby) 
                    VALUES (%s, %s, %s, %s)
                    ON DUPLICATE KEY UPDATE
                    nm = %s, name = %s, birthdate = %s, addedby = %s''',
                 [nm, name, birthdate, addedby,
                  nm, name, birthdate, addedby ])
    conn.commit()
    return_the_inserted_data = False
    if return_the_inserted_data:
        # assuming that worked, we can just return the appropriate
        # dictionary: Note that we could probably return the
        # person_entry, but what if that were a dictionary that
        # included extra stuff? This cleans that up.
        return {'nm': nm, 'name': name, 'birthdate': birthdate, 'addedby': addedby}
    else:
        return get_person(conn, nm)
    
def get_person(conn, nm):
    '''Returns a dictionary for that person, or None.'''
    curs = dbi.dict_cursor(conn)
    curs.execute('''SELECT nm, name, birthdate, addedby FROM person WHERE nm = %s''', [nm])
    return curs.fetchone()

def update_person(conn, person_entry):
    '''Replaces that person; returns the new entry.'''
    # this code is virtually identical to insert_person. In many APIs,
    # the distinction would be that POST is not idempotent, requires
    # the entity to not exist and creates a NEW entity (returning any
    # generated ID), while PUT is idempotent and requires the entity
    # to exist. We could do this with, but it would be a gratuitous
    # burden on the client, requiring them to know whether the entity
    # is new or not. However, this function *does* require the entity
    # to exist.
    curs = dbi.cursor(conn)
    nm = person_entry['nm']
    name = person_entry['name']
    birthdate = person_entry['birthdate']
    addedby = person_entry['addedby']
    curs.execute('''UPDATE person SET
                    nm = %s, name = %s, birthdate = %s, addedby = %s
                    WHERE nm = %s''',
                 [nm, name, birthdate, addedby, nm])
    conn.commit()
    return_the_inserted_data = False
    if return_the_inserted_data:
        # assuming that worked, we can just return the appropriate
        # dictionary: Note that we could probably return the
        # person_entry, but what if that were a dictionary that
        # included extra stuff? This cleans that up.
        return {'nm': nm, 'name': name, 'birthdate': birthdate, 'addedby': addedby}
    else:
        return get_person(conn, nm)

def delete_person(conn, nm):
    '''Deletes that person, returning None'''
    curs = dbi.cursor(conn)
    curs.execute('''DELETE FROM  person WHERE nm = %s''', [nm])
    conn.commit()

# ================================================================
# Staff API

def insert_staff_member(conn, staff_name):
    '''Inserts a new member of the staff; returns the new UID. Assumes the
person does not exist (will raise an exception if they do). NOT
idempotent.
    '''
    curs = dbi.cursor(conn)
    curs.execute('''insert into staff(name) values (%s)''', [staff_name])
    conn.commit()
    curs.execute('''select last_insert_id()''')
    row = curs.fetchone()
    return row[0]

if __name__ == '__main__':
    # testing code
    dbi.conf(None)              # use personal copy of the WMDB
    conn = dbi.connect()
    folk = get_people(conn)
    print('db has {num} people. The first 10 are'.format(num=len(folk)),
          [p['name'] for p in folk[0:10]])
    george = get_person(conn, 123)
    print('one person', george)

    flower = insert_person(conn, {'nm': 1234,
                                  'name': 'Fictitious Flower',
                                  'birthdate': '2001-05-06',
                                  'addedby': 1})
    print('new person', flower)

    flower = get_person(conn, 1234)
    print('get flower', flower)

    flower = update_person(conn, {'nm': 1234,
                                  'name': 'Fantasy Flower',
                                  'birthdate': '2001-05-06',
                                  'addedby': 1})
    print('update flower', flower)

    delete_person(conn, 1234)

    # insert a new staff person
    wendy = insert_staff_member(conn, "Wendy Worker")
    print('Wendy is a new worker with UID: ', wendy)

Idempotent INSERT¶

One interesting in the database code is that the function to insert a new person is idempotent, which means that it has the same result if it's invoked with the same arguments more than once. That's different from how INSERT typically works in MySQL: if we do:

INSERT INTO person values (123, "George Clooney", "1961-05-06", 1);

That will work as expected if there is no person with nm = 123 in the database. But if there is, we'll get a Duplicate Key error, even if we're inserting the same data that is already there. So normal database INSERT is not idempotent.

We can make it idempotent by doing this:

INSERT INTO person values (123, "George Clooney", "1961-05-06", 1)
ON DUPLICATE KEY UPDATE 
nm = 123, name = "George Clooney", birthdate = "1961-05-06", addedby = 1;

We've make the insert_person Python function idempotent, making it more similar to the update_person Python function. Look at them both, above.

Person App Providing the REST API¶

Here's the Flask app providing the REST API:

'''This demo shows a pure, JSON-returning, REST API.  It makes
endpoints similar to People App. Namely:

These are all for API v1, so we put the version in the endpoint

The JSON for a person is:

{nm: 12345, name: "Wendy", birthdate: "2002-05-06", addedby: 1}

HTTP   |  endpoint         | body | response
GET    | /v1/              | none | rendered web page documenting the API
GET    | /v1/people/       | none | JSON for all people 
POST   | /v1/people/       | JSON for new person | JSON for this person
GET    | /v1/people/:nm    | none | JSON for that person
PUT    | /v1/people/:nm    | JSON for that person | JSON for that person
DELETE | /v1/people/:nm    | none  | --

There is no authentication

There's a companion Python script using the requests module that
demonstrates all this.

Scott Anderson
March 2022
'''

from flask import (Flask, render_template, make_response, url_for, request,
                   redirect, flash, session, send_from_directory, jsonify, abort)
from werkzeug.utils import secure_filename
app = Flask(__name__)



# one or the other of these. Defaults to MySQL (PyMySQL)
# change comment characters to switch to SQLite

import cs304dbi as dbi
# import cs304dbi_sqlite3 as dbi

import secrets

app.secret_key = 'your secret here'
# replace that with a secure random key. Works great, but only in Python > 3.6
app.secret_key = secrets.token_hex()

# This gets us better error messages for certain common request errors
app.config['TRAP_BAD_REQUEST_ERRORS'] = True

import movie_people as mp

@app.route('/v1/')
def index():
    return render_template('people-api.html',title='Hello')

REQUIRED_KEYS = set(['nm','name', 'birthdate', 'addedby'])

@app.route('/v1/people/', methods=["GET", "POST"])
def people():
    conn = dbi.connect()
    if request.method == 'GET':
        folk = mp.get_people(conn)
        return jsonify(folk)
    elif request.method == 'POST':
        # actually, it has to be POST if it's not GET
        if REQUIRED_KEYS > set(request.form.keys()):
            return jsonify({'error': 'bad request, missing keys'}), 400
        newbie = mp.insert_person(conn, request.form)
        return jsonify(newbie)

@app.route('/v1/people/<nm>', methods=['GET','PUT','DELETE'])
def person(nm):
    conn = dbi.connect()
    if not nm.isdigit():
        return jsonify({'error': 'bad request, non-numeric NM'}), 400
    nm = int(nm)
    if request.method == 'GET':
        them = mp.get_person(conn, nm)
        if them is None:
            return jsonify({'error': 'No person with that NM'}), 404
        else:
            return jsonify(them)
    elif request.method == 'PUT':
        if REQUIRED_KEYS > set(request.form.keys()):
            return jsonify({'error': 'bad request, missing keys'}), 400
        else:
            them = mp.update_person(conn, request.form)
            return jsonify(them)
    elif request.method == 'DELETE':
        mp.delete_person(conn, nm)
        # we could create a response without a body, but this is simple and easy
        return jsonify('ok')


if __name__ == '__main__':
    import sys, os
    if len(sys.argv) > 1:
        # arg, if any, is the desired port number
        port = int(sys.argv[1])
        assert(port>1024)
    else:
        port = os.getuid()
    # configure and report database connections
    dbi.conf(None)              # use default (personal) db
    curs = dbi.cursor(conn)
    curs.execute('select database()')
    print('database',curs.fetchone())
    # start the app
    app.debug = True
    app.run('0.0.0.0',port)

Request Data¶

Note that the data from the client is presented to our Flask app in the same way it was before we started learning REST and Ajax. It is still in the request object, still in the request.form dictionary for POST and PUT. This is important to remember. Students often ask how an Ajax request is different from a regular request. And the answer is: The request is the same. Ajax differs in the response but not the request. The response is JSON instead of the usual rendered web page. But the requests send the data from the client in the same way.

Code Notes¶

Most of the code in the section above should be familiar to you. One new piece is code like this:

        them = mp.get_person(conn, nm)
        if them is None:
            return jsonify({'error': 'No person with that NM'}), 404
        else:
            return jsonify(them)

The them variable is a Python dictionary as returned by our database function. If it's None, then no person was found, so we return a JSON dictionary with an error message in it. We also return an HTTP Status Code of 404 (Not Found). Other places in the code return 400 (Bad Request).

Another trick is a way to check that all the required data has been sent. When the client creates a new person, we require them to send 4 pieces of data: NM, name, birthdate, and addedby. To check that the form data has all the required keys, we can do the following:

REQUIRED_KEYS = set(['nm','name', 'birthdate', 'addedby'])
...
        if REQUIRED_KEYS > set(request.form.keys()):
            return jsonify({'error': 'bad request, missing keys'}), 400

Python has a set datatype that is like an order-independent list. So if the set of supplied keys is a strict subset (>) of the required keys, there is a key missing and we can complain and generate a 400 response.

Of course, as cool as that is, very often the client is a web browser. Let's talk for a couple minutes about that:

Ajax Concepts in Web Pages¶

Ajax can be used in two main modes, which also correspond to the methods of an HTML form.

• The web browser can GET some data from the back end. This data can be combined with the existing data on the page. You see this in Facebook, when it retrieves posts, comments, and so forth without you having to reload the page.
• The web browser can POST some data to the back-end. The back-end can do whatever it needs to with it. Gmail might POST a message you've sent, sending it to a Gmail server that will then forward it to the recipients. Similarly, when you click a like button on Facebook, that information is sent to the back end, updating counters on Facebook's servers.

Of course, these ideas can be combined, so when you like a Facebook post, it not only increments the counters in the back end, but returns the new value (which might include increments from other people), and your browser can update the number of likes that you see.

Later, we will learn about other methods.

The difference between a normal form submission to the back-end and a submission via Ajax is that the response from a normal interaction has to be the entire web page, while with an Ajax interaction, the response might be only the relevant information, such as "your message was sent" or "the number of likes is now 25." That smaller payload is quicker to load over a network, and (usually) less of the page needs to be re-renderered.

Purpose of Ajax in a Web Page¶

It's worth highlighting the concept at the end of the last section as one of the main purposes of Ajax. Suppose we are on a lengthy, content-rich web page, maybe several megabytes of data. It could the the New York Times, Facebook, Gmail, anything. But suppose it's the Times, and I click on the "like" button of a comment I appreciated. Before Ajax, that click would result in an HTML <form> being POSTed to the server, indicating my appreciation, and the whole page would be re-rendered and sent back, with the only change being the number of likes for that comment. That's several megabytes of unnecessary web traffic, plus I'll have to scroll back to where I was, so it spoils the user experience.

Instead, with Ajax, the form data is POSTed using JavaScript, asynchronously, and the response is the new number of likes, which more JavaScript can use to dynamically update the page. So a handful of bytes are transmitted instead of megabytes, and there's no interruption to the user experience, because the interaction was asynchronous (concurrent with the user's actions).

Graphic of Ajax Interaction¶

This graphic depicts the general idea. The sender() JavaScript function initiates the Ajax request, sending a URL that includes the user's ID to the server. The sender function designates the receiver function as the callback, so the response object should go to the receiver() function. The receiver() function is the ensuing computation. In this case, receiver() inserts the high score for this user onto the page.

The green arrow from browser to server shows the request being sent. The blue arrow from server to browser shows the response being sent.

You can ignore the JavaScript code in the browser (front-end) box, but if you take CS 204, you'll learn how to implement that.

JavaScript and JSON in Flask¶

Here is a page about Flask, Ajax, JavaScript and JSON. It talks a lot more about the front-end.

Summary¶

• APIs are
  • a way for clients to talk to servers,
  • the client can GET data from the server
  • the client can POST, PUT and DELETE data as well
  • these are expanded HTTP request methods
  • responses are usually machine-friendly data, usually JSON
• JSON is the JavaScript Object Notation, both human- and machine-friendly
  • similar to Python data structures and notation
  • scalars: numbers, strings, booleans, null
  • arrays
  • objects (dictionaries)
• REST APIs
  • is a particular understanding of how the HTTP requests should behave
  • data is a collection of entities or resources
  • use GET /collection/ to get the entire collection
  • use GET /collection/<id> to get a particular entity/resource
  • GET purely retrieves data: idempotent, cacheable
  • use POST /collection/ to create a new entity
  • use PUT /collection/<id> to replace an entity
  • use DELETE /collection/<id> to delete an entity
• Flask
  • add extra HTTP methods to the list for the handler function: @app.route('/endpoint/', methods=['GET','POST','PUT','DELETE'])
  • data usually still comes in using request.form
  • data is returned using jsonify(data) rather than render_template()
• Python requests module
  • allows you to make HTTP requests from a python program
  • so you can write an HTTP client
• Ajax
  • is a way for JavaScript in the browser to initiate a request.
  • the browser does not replace the document with the response
  • JavaScript in the browser can use the response data to update the page
  • advantage is that large pages need only be updated, rather than replaced:
    • less burden on network bandwidth
    • less browser rendering time
  • integrates nicely with REST APIs

Conclusion¶

We've come a long way! We learned

• Ajax as a way to respond to a browser request
• JSON, as a data representation to communicate with the browser
• additional HTTP methods
• REST API using those additional methods and a general protocol