Ajax¶

This reading introduces Ajax, which is how we will replace our in-memory DataStore and our persistent LocalStore with a server-based RemoteDataStore. The JavaScript will communicate with that server using asynchronous network communication, which falls under the general umbrella term of Ajax. (That word is actually an acronym, but the acronym isn't helpful to understanding the word, except for the fact that it is Asynchronous JavaScript.)

Before Ajax, the way that a browser communicated with a server (the "back-end") was typically to submit a form (or get a URL), whereupon the web page was replaced with the response. That replacement can sometimes interrupt a user's experience. Imagine if everytime you clicked "like" on a Facebook page, the entire page was replaced with a new one, even if it looked 99% like the old one.

Ajax is a powerful modern technique whereby a browser page can communicate with a server without having to be replaced by the response. Instead, it's in a kind of "side" conversation going on in parallel (asynchronously) with the browser paying attention to you.

It can be successfully argued that no single technology shift has transformed the landscape of the web more than Ajax. The ability to make asynchronous requests back to the server without the need to reload entire pages has enabled a whole new set of user-interaction paradigms and made DOM-scripted applications possible.

Bear Bibeault & Yehuda Katz
jQuery in Action, 2nd edition

Let's begin.

Ajax Concepts¶

The main idea of Ajax is pretty straightforward: our JavaScript can talk to a remote server to either send data to it or get data from it. We're pretty used to "saving" Google Docs or FaceBook posts and lots of other things to cloud servers, so that idea isn't tricky.

The tricky part comes with coordinating the requests with JS code that needs to run after the request. Let's call it the "ensuing" code. (That's not a technical term, but I just wanted a distinctive word for "after".) That coordination is tricky because the request goes to a remote server over an unpredictable network and so it will take an unknown amount of time. So, it's hard to know when we will be able to execute the ensuing code. We have to set it up like an event handler, where the event is the completion of the request. Thus, we have asynchronous requests.

A Metaphor¶

When I was a kid, cereal manufacturers would send you a toy if you cut off box tops from a few boxes of their cereal and send them the box tops. I'd do that, take the envelope out to the mailbox, put it in, and stand by the mailbox waiting for the toy. My mother called me into the house to tell me it would 4-6 weeks for the toy to arrive, and I shouldn't put my life on hold waiting for the response.

Things come faster nowadays, but we still don't wait by the front door when we order from Amazon or GrubHub, because the response will still take a while.

The reason that Ajax is tricky to work with is because we can't have our code wait around for the response. Instead, we have to set up a response handler that will be invoked when the response arrives. Let's see how that works with code.

Asynchronous Requests¶

When we first learned functions and function calling, in a course like CS 111, we call the function and get a return value, which we can then use, maybe passing the value to other functions:

var val = foo(x);
bar(val);
more_work();

For example, we might do:

var root = sqrt(x);
console.log('square root of ',x,' is ',root);
more_work();

That's fine in the context of normal programming, but it doesn't work for lengthy operations. Suppose foo took tens of milliseconds to return and might not ever return. Not only does bar have to wait (it would anyhow), but so does more_work, even if it doesn't depend on foo. More importantly, if the browser has something else to do (such as paying attention to the user), it can't be sitting around waiting for foo to complete. This is the idea of "putting your life on hold while waiting for a response."

(You wouldn't think waiting a few milliseconds is a big deal, but that's because it's hard to understand how incredibly fast computers are. An ordinary function call might take only a few nanoseconds, which is a million times faster. A millisecond is a thousandth of a second; a microsecond is a millionth of a second; a nanosecond is a billionth of a second.)

Instead, we want to use an asynchronous style of programming, where instead of foo returning a value, we pass foo a function that it should invoke on the result. Meanwhile, neither more_work nor the browser will have to wait for foo to complete. Here's the programming style:

foo(x, bar);
more_work();

For example:

sqrt(x, console.log);
more_work();

The more_work function and whatever else the browser is doing can execute immediately. Some tens of milliseconds later, once foo is done, foo will arrange to invoke bar.

We could implement the sqrt function like this:

function sqrt(x, callback) {
    let root = Math.sqrt(x);
    callback(root);
}

This programming style where we pass in a callback function should remind you of setting up event handlers, and, indeed, it's exactly the same: we pass in a function that will be executed later, when some event occurs.

Here's the thing we need to keep reminding ourselves of:

In Ajax, we use callbacks instead of return values

Once we wrap our heads around this style of programming and give up on return values, Ajax becomes relatively straightforward. (Which is not to say you won't make mistakes. I made lots of mistakes adjusting to using Ajax, because after years and years of coding, return values is so instinctive.)

Then, in the Coffeerun app, we can send the data to a database application the original authors have running on Heroku, which is a cloud-based server for web developers. We'll see that below and in class.

Ajax Graphic¶

This graphic depicts the general idea of Ajax. The sender() JavaScript function initiates the Ajax request, sending a URL that includes the user's ID to the server. The sender function designates the receiver function as the callback, so the response object should go to the receiver() function. The receiver() function is the ensuing computation. In this case, receiver() inserts the high score for this user onto the page.

The green arrow from browser to server shows the request being sent. The blue arrow from server to browser shows the response being sent.

You can ignore the Python code in the server (back-end) box, but if you take CS 304, you'll learn how to implement that.

jQuery Ajax API¶

As you now know, Ajax works by sending a request over the network to the server and setting up a response (callback) function that will be invoked when the server's response arrives back at the browser. jQuery has several functions to make Ajax calls relatively easy. Here's one way to send some data to a server at some url:

$.post(server_url, data, callback);

The first argument is the URL of the server to send the data to. The second argument is the data, usually as a JavaScript object literal. Finally, the last argument is a callback function that will get invoked with the response from the server. (The callback function is the ensuing code that I referred to in the introduction.)

For example, if our Facebook ID is 123 and the post that we like has an id of 456, we could like that post by sending that information to Facebook's servers. (The following example is entirely made up; it's just to illustrate the idea. The actual FaceBook API is different and much more complex. I'm also ignoring authentication and such.)

$.post(server_url, 
       {fbid: 123, postid: 456, like: true}, 
       function (resp) => { console.log('like recorded') };

Here, the callback function ignores the response from the server (resp) and just prints a message to the console saying the "like" was recorded. It could also update the page in various ways, which we know how to do, using jQuery methods like .text() and .css().

Alternatively, the following example uses a named callback function:

function gotit (response) {
    console.log('Got '+response);
}

$.post(server_url, data, gotit);

In the example above, the gotit function is the ensuing function that we want to execute after the Ajax request completes. It can do whatever we want, but it won't get executed until the response comes back from the server.

The jQuery .get() method works the same way as .post(), though it doesn't send any data in the second argument. Also, .get() typically requests some data from the server and the callback will want to process that. For example:

function process_likes(resp) {
    let num = resp.num_likes;
    console.log('that post has ',num,' likes');
}

$.get(server_url, process_likes);

In practice, the server_url in the example above would probably specify which post we are asking about, so maybe something like https://facebook.com/get_likes/postid456. (Again, that URL is completely made up, but the idea is what is important.)

In fact, in jQuery both .get() and .post() are shorthands for jQuery's general purpose .ajax() method that needs to know what HTTP type or method of request you want to make:

$.ajax(url, {type: 'get', data: data, success: func});
$.ajax(url, {type: 'post', data: data, success: func});
$.ajax(url, {type: 'put', data: data, success: func});
$.ajax(url, {type: 'delete', data: data, success: func});

GET and POST are the most common HTTP methods, but PUT and DELETE are part of the protocols. We'll talk more about these next time.

jQuery Reference¶

There are essentially three jQuery methods that make Ajax easier: two shortcuts and a general but low-level method:

• $.get(url, callback)
• $.post(url, data, callback)
• $.ajax(url, settings)

Cloud Storage in CoffeeRun¶

Given the Heroku cloud server, we can implement a RemoteDataStore class that is analogous to the DataStore class from the original CoffeeRun app and also analogous to the LocalStore class that we developed last time.

However, because we don't get to have return values with Ajax, we need to change the API. The original API had two methods that returned values:

    get (key) {
        return this.data[key];
    }

    getAll () {
        return this.data;
    }

We can't do that with Ajax, so we'll have to add a callback argument to the method:

    get (key, callback) {
        callback(this.data[key]);
    }

    getAll (callback) {
        callback(this.data);
    }

Remember, there's nothing special about naming the argument callback; we could name it cb or func or response or whatever.

Given the need to eliminate return values, let's first rewrite the DataStore class before we write a class for the RemoteDataStore,

In-Memory DataStore class with Callbacks¶

Here's the rewritten code. With the exception of the get and getAll methods, it's pretty much the same. Still, it's good to remind ourselves of the methods we need to implement.

/* Rewritten by Scott, Summer 2020. This implements a class DataStore
 * that has an API for a key/value database. 
 * 
 * This version of the API uses callbacks instead of return values.
*/

class DataStore {
    // instance variables
    data = {};

    constructor() {
        console.log('running the DataStore function')
        this.data = {};
    }

    add (key, val) {
        this.data[key] = val;
    }

    get (key, callback) {
        callback(this.data[key]);
    }

    getAll (callback) {
        callback(this.data);
    }

    remove (key) {
        delete this.data[key];
    }
}

var ex1 = new DataStore();
ex1.add('harry', {name: 'Harry Potter',
                  hair: 'black',
                  house: 'Gryffindor'})
ex1.add('ron', {name: 'Ron Weasley',
                hair: 'red',
                house: 'Gryffindor'})
ex1.add('hermione', {name: 'Hermione Granger',
                     hair: 'brown',
                     house: 'Gryffindor'})

ex1.get('harry', (val) => console.log('lookup of harry:', val));

function showDictionary(dic) {
    let keys = Object.keys(dic);
    keys.map(k => console.log(k, '=>', dic[k]));
}

console.log('all values before removing ron');
ex1.getAll(showDictionary);

ex1.remove('ron');

console.log('all values after removing ron');
ex1.getAll(showDictionary);

export { DataStore };

We had to re-write the example code at the end, as well. Storing Harry, Ron and Hermione into the database using the add method is the unchanged, but looking up Harry is different. Instead of:

console.log(ex1.get('harry'));

we have to write

ex1.get('harry', console.log);

Where console.log is a function that we supply as the callback.

Similarly, instead of getting a dictionary of all the data as a return value and printing it using the showDictionary function, like this:

showDictionary(ex1.getAll())

we have to pass showDictionary as a callback function, like this:

ex1.getAll(showDictionary);

As you can see, the code isn't much longer or even much more complicated, but it requires a huge change in how we think. This is not easy, but it's incredibly important, because callback-based coding is the way modern web programming works.

Cloud-Based RemoteDataStore with Callback¶

Now we're ready for the cloud-based data store. Because the URL for the cloud-based server might change, we'll supply that information in the constructor. Here's the code just of the constructor:

    constructor (url) {
        if (!url) {
            throw new Error('No remote URL supplied');
        }

        this.serverUrl = url;
    }

Pretty straightforward. The caller supplies the URL for the cloud database server, which the constructor squirrels away in an instance variable called serverUrl for use by the various methods. We don't initialize the database in any way, since there might be data in there from past interactions.

The add Method¶

Now let's look at storing a coffee order in the database, which is what the add method does:

    add (key, val) {
        $.post(this.serverUrl, val, function (serverResponse) {
            console.log(serverResponse);
        });
    }

This works by posting the data (val) to the server, using the server's URL. The add method doesn't take a callback, but the implementation just prints a message to the console with the server's response. That way we won't be entirely clueless if the server goes away or there's some mysterious error. But as far as the client is concerned, they just have to trust that the key and value are successfully stored.

The getAll Method¶

Next, we can look at getting a copy of the entire database. We'll need this when we initialize our page, so that it can display any orders that are already in the database.

    getAll (callback) {
        $.get(this.serverUrl, function (serverResponse) {
            console.log('getAll serverResponse: ',serverResponse);
            callback(serverResponse);
        });
    }

The getAll method takes a callback function that is invoked with the data from the server, which has been automatically parsed by JSON.parse so it's already been turned into a JavaScript data structure --- here, an object literal (dictionary) containing all the coffee orders.

(Note that a feature of all of jQuery's Ajax methods is intelligently guessing the type of response from the server and automatically using JSON.parse if it looks like JSON. That feature works well here.)

The get Method¶

Next, we can implement looking up just a single key. To do that, we append the key to the general URL (separated by a slash). (As we saw, without the appended stuff, the same .get returns all the data in the store. Including coffee orders submitted by other people on other browsers. We truly have a shared database.)

(This is a general rule about the HTTP GET request: the information is all the URL.)

    get (key, callback) {
        $.get(this.serverUrl + '/' + key, function (serverResponse) {
            console.log('get '+key+' serverResponse: ',serverResponse);
            callback(serverResponse);
        });
    }

Why would you get just a single order when you could get the entire database? If the database were large, it would be more efficient to get the smaller amount of data. Think about how Ajax is used by Gmail, Facebook, and other apps.

The remove Method¶

Similarly, to delete a single order from the database, we use the DELETE method and we again append the order's key to the URL.

    remove (key) {
        $.ajax(this.serverUrl + '/' + key, {
            type: 'DELETE'
        });
    }

Here, we can't use the jQuery shortcut methods .get and .post, we have to use the more general .ajax method. The second argument is a JavaScript object literal (dictionary) of settings. Here, we set the type of request to DELETE instead of GET or POST. The type of request is also called the "method", which is a seriously overused word in Computer Science.

RemoteDataStore¶

That's all the code for the RemoteDataStore class. This is just for reference, so you can skim this section; there's nothing new.

#!js
/* Rewritten by Scott, Summer 2020. This defines a class
 RemoteDataStore that implements the same API as the DataStore class,
 although the constructor has a different signature (adding the url).
 
 The methods get() and getAll() differ in having requiring a callback function
 but in Chapter 14 they make that argument optional and return a promise instead, but
 we won't get into promises. */

class RemoteDataStore {
    // instance variables
    serverUrl = null;
    
    constructor (url) {
        if (!url) {
            throw new Error('No remote URL supplied');
        }

        this.serverUrl = url;
    }
    
    add (key, val) {
        $.post(this.serverUrl, val, function (serverResponse) {
            console.log(serverResponse);
        });
    }

    getAll (callback) {
        $.get(this.serverUrl, function (serverResponse) {
            console.log('getAll serverResponse: ',serverResponse);
            callback(serverResponse);
        });
    }

    get (key, callback) {
        $.get(this.serverUrl + '/' + key, function (serverResponse) {
            console.log('get '+key+' serverResponse: ',serverResponse);
            callback(serverResponse);
        });
    }

    remove (key) {
        $.ajax(this.serverUrl + '/' + key, {
            type: 'DELETE'
        });
    }
}

export { RemoteDataStore };

Adjustments in Main¶

As with our class for LocalStore, we have to make some adjustments to our main module. Fortunately, relatively small ones. We have to create an instance of a different class and pass that to the Truck constructor:

In main-module.js, instead of

    ds1 = new DataStore();
    truck1 = new Truck('ncc-1701', ds1);

we do this:

    ds1 = new RemoteDataStore('CoffeeRun truck');
    truck1 = new Truck('ncc-1701', ds1);

(In practice, we'll actually do something more elaborate, to allow the user to dynamically choose the kind of data store, using a popup menu.)

Adjustments to Truck¶

We have some other critically important things to do, since the API (arguments and return value) for get and getAll have changed drastically.

Any code that uses those methods has to be re-written to use the new API. As it happens, those are only in the Truck module, specifically in the printOrders method.

Here's the original printOrders method, using arrow functions and a data store API with return values for getAll() and get(id)

    printOrders_arrow () {
        console.log('Truck #' + this.truckId + ' has pending orders:');
        let allOrders = this.db.getAll();
        let customerIdArray = Object.keys(allOrders);
        console.log(customerIdArray.length+' orders');
        customerIdArray.forEach( (id) => {
            // the following works because in an arrow function,
            // `this` doesn't change, so it still has the correct value
            let order = this.db.get(id);
            console.log(order);
        });
    }

Here's the revised printOrders method, now using a data store API with callbacks for getAll(callback) and get(id, callback):

    printOrders_arrow () {
        console.log('Truck #' + this.truckId + ' has pending orders:');
        this.db.getAll( (allOrders) => {
            let customerIdArray = Object.keys(allOrders);
            console.log(customerIdArray.length+' orders');
            customerIdArray.forEach( (id) => {
                // the following works because in an arrow function,
                // `this` doesn't change, so it still has the correct value
                this.db.get(id, function(order) {
                    console.log(order);
                });
            });
        });
    }

Yikes! That's a bit more complex. But, deep breath, we'll figure it out. The argument to this.db.getAll is an arrow-style callback function whose argument is a JS object literal (a dictionary) with all the orders in it. The argument is called allOrders, and it has exactly the same value and meaning as the same variable in the return-value version.

Next, we get an array of the keys in the dictionary of orders, customerIdArray and print its length; that's the same in both versions. Then we use forEach over that array, supplying an arrow-style callback function whose argument is id. Again, this code is exactly the same between the two versions.

But, then the argument to this.db.get changes. Instead of just an id, it also gets a callback function. Here, the callback is a standard-style function that just does a console.log of the order. (The callback could just as easily have been an arrow function; it makes no difference here.)

So, although the code initially looks dauntingly different, it's not terrible. It's just a different coding style, because with Ajax, we

send callback functions instead of getting return values

Cloud-Based CoffeeRun¶

Here's a link to the current CoffeeRun. It includes the callback-based versions of all three data stores:

• the original in-memory DataStore
• the local storage based LocalDataStore and
• the new Heroku Ajax-based RemoteDataStore

There's also data store similar to the Heroku one, based here at Wellesley. I'll demo that in class. If you take CS 304, you will learn how to implement the back-end for this cloud-based data store.

When you load the app, there's a pop-up window that asks you which data store you want to use. Enter "2" to use the Ajax-based one, but feel free to try them all. Well, we'll try number 3 in class.

Summary¶

Many, if not most, modern web applications talk to a remote (cloud) data store, to retrieve advertisements if nothing else.

• If you have to login, the app probably talks to a remote data store.
• If you can sync your data across devices, it talks to a remote data store.
• If you can share data with others, it talks to a remote data store.

This is all very cool.

The means by which a web browser talks to a remote data store is Ajax. Ajax is a set of rules by which a browser can send an asynchronous request, and receive a response, later. The browser and server talk using HTTP, just as they always do, but one difference is that the browser doesn't replace the current page with the response, as it would when we click a hyperlink or submit a form. Instead, the response is received and handed off to some callback function. The callback function can do whatever the developer chooses.

So,

• Ajax is asynchronous and so you have to program using callbacks
• There are request "types" or "methods" like GET, POST, DELETE and others
• The request can send data to the server
• The server can respond with data

Next time, we'll talk about REST APIs, which is a common Ajax-based protocol for browsers to talk to server-based databases.