Newly revised for Spring 2019!
This reading covers the concepts behind Ajax and JSON, and why they fit together, and then explores a particular example, with code. We won't cover all aspects of Ajax and JSON, but we'll see more later in the course.
What is Ajax? Why should we care?
It can be successfully argued that no single technology shift has transformed the landscape of the web more than Ajax. The ability to make asynchronous requests back to the server without the need to reload entire pages has enabled a whole new set of user-interaction paradigms and made DOM-scripted applications possible.
Bear Bibeault & Yehuda Katz
jQuery in Action, 2nd edition
Ajax is a technique where a user's interaction with the page causes some JavaScript to run, which connects (asynchronously) to a server to retrieve some information, and then the page is updated with the results. It's the essential functionality behind such many nice web applications, such as:
Ajax can be used in two main modes, which also correspond to the methods of an HTML form.
likebutton on Facebook, that information is sent to the back end, updating counters on Facebook's servers.
Of course, these ideas can be combined, so when you like a Facebook post, it not only increments the counters in the back end, but returns the new value (which might include increments from other people), and your browser can update the number of likes that you see.
The difference between a normal form submission to the back-end and a submission via Ajax is that the response from a normal interaction has to be the entire web page, while with an Ajax interaction, the response might be only the relevant information, such as "your message was sent" or "the number of likes is now 25." That smaller payload is quicker to load over a network, and (usually) less of the page needs to be re-renderered.
Consider the following slideshow on Ajax interaction. Here, the HTML part of the page is in blue and the JavaScript is in green. Click on the links below to go to that slide. The context of this slideshow is that the back-end is computing some values for us, based on values from the browser. In this example, we are sending the number 100 and getting the number 50 back. (The example uses PHP in the back-end, but it could just as easily be Flask.)
The harry.php script does the computation. The JS function
in the browser that sends the data (the number 100) is
named fred(), and the JS function that receives the
response (the number 50) is
named george(). The george() function
inserts the number 50 into a list on the page.
An alternative explanation that you're welcome to consult is this gentle introduction to Ajax.
There's an important concept depicted in that interaction, namely the
notion of a callback. When the user clicks the button, the
function fred() runs immediately, sending the data to the
server. However, the response from the server is not immediate. It
could take tens of milliseconds or longer for the response to arrive.
We don't want fred() to wait around for the response; that
would lock up the browser, allowing nothing to proceed until the
response arrives. If we did this, which is possible, the interaction
is called synchronous. Normally, however, the processing
is asynchronous. What that means is that fred()
tells the browser:
When the response arrives, invoke the function
george()with the data
The fred() function can then finish, returning to its
caller, and the browser proceeds.
Once the response arrives, whenever that is, george() runs
and does whatever we want done with the response. In this example, we
wanted it added to a list of results on the page.
The response from the back-end can come in many forms. In our example above, it was just a number. In general, it could be snippets of text or HTML to insert onto the page. It could be XML to be processed (this is the X in the acronym AJAX). Nowadays, it's most likely to be JSON. Let's take a detour to talk about this kind of data.
JSON, the JavaScript Object Notation, is a new standard for information interchange between programs, competing in the same market niche as XML. Ajax will allow us to get data from the back end, typically formatted as JSON. So, let's see how JSON works.
JSON is a subset of the JavaScript language, omitting things like function definitions. It's intended to comprise hierarchical data, like arrays and objects, and non-compositional data like numbers, strings, and booleans. Here's an example that we've seen before:
{"title": "Dr. Zhivago",
"tt": 59113,
"year": 1965,
"stars": ["Omar Sharif", "Julie Cristie", "Geraldine Chaplin"],
"isGreat": true,
"director": {"name": "David Lean",
"nm": 180,
"birth": "1908-03-25"},
"IMDB rating": 8.0}
Note that values can be scalars or compound; here we have an array literal of strings for the stars and an object for the director.
See JSON.org for more. That site also has a ton of info on packages and modules to process JSON notation in a variety of languages.
JSON notation comprises the following:
Modern JavaScript implementations come with two functions that are useful for working with JSON:
JSON.stringify() function turns a data
structure into a string.JSON.parse() function turns a string back
into a data structure.Thus, if the back-end wanted to send us some complex data, instead of a simple number, sending it as JSON makes it easy to work with.
For the rest of this reading, we'll look at an example using Flask and JavaScript/jQuery. In class, we'll download and run this example and I'll answer your questions, but it's best to be prepared. This reading will help you prepare.
Here it is in action: cs304collatz
In the example, the back-end will compute the Collatz function:
def collatz(num):
if num %2 == 0:
return num / 2
else:
return (num * 3) + 1
This is, of course, a trivial function that we don't need a server-based solution for, but imagine that the number it is returning is, say, the number of likes of a FaceBook post.
Let's first get a conceptual overview of the workings of the Ajax behavior:
{error: false, in: 50, out: 25}
{error: false, in: 25, out: 76}
{error: true, err: "Could not convert 'five' to integer"}
We won't necessarily talk about these steps in this order. In fact, we won't start with any of them, we'll start with progressive enhancement.
Ajax is very slick, but you have to have JavaScript in your browser to use it. Most people do, of course, but there are special screen-readers for the blind and other special web browsers for people who need assistive technology, and not all of those unusual web browsers have JavaScript. There may also be people who have chosen to turn JavaScript off for some reason.
You could decide that you just don't care about such people, but you don't have to. The idea of progressive enhancement is to start with a functional but old-fashioned, 1990s-style web application, with no JavaScript, and then use the power of JavaScript to hide the old and reveal the new (or convert the old into the new). Then, people without JavaScript just see the old version, and people with JavaScript see the new version.
So, we're first going to look at the code that handles the non-Ajax usage. First, here's the relevant part of the HTML page:
<form id="number-submit-form"
method='GET'
action="{{url_for('getCollatzNormal')}}">
<p><label for="form-num">Form Number</label>:
<input type="text" name="num" id="form-num"></p>
<p><input type="submit" id="btn-submit" value="submit"></p>
</form>
{% if x %}
<p id="result">The next value after
<span class="x">{{x}}</span>
is
<span class="y">{{y}}</span></p>
{% endif %}
In this case, I decided to use the GET method because it makes it easier to see the data (it's in the URL).
And here is the back-end code that makes the non-Ajax form work:
@app.route('/')
def index():
return render_template('collatz.html',title='Collatz')
@app.route('/getCollatzNormal/', methods=['GET','POST'])
def getCollatzNormal():
if request.method == 'POST':
num = request.form.get('num')
else:
num = request.args.get('num')
try:
x = int(num)
except:
flash('Could not convert {num} to int'.format(num))
return render_template('collatz.html',title='Collatz')
y = collatz(x)
return render_template('collatz.html',
title='Collatz', x=x, y=y)
This is just two simple routes. I've omitted the definition
of collatz, since we saw it earlier. The route handles
both GET and POST, though our form uses GET.
Let's start on the Ajax behavior, but we'll start at the end
of the Ajax behavior, where we add the results to the list on the
page. (Recall our high-level view of the
Ajax behavior.) Why at the end? Because the beginning is going to mention
the addToList function we are about to define, and it's
useful to know what that is referring to.
Let's assume that the back end will send us a response that looks like:
{error: false, in: 50, out: 25}
{error: false, in: 25, out: 76}
{error: true, err: "Could not convert 'five' to integer"}
What do we want to do with it? We want to add it to the end of the list. We'll write a JS/JQ function that does that. But first, let's add some stuff to our page to put error messages and results:
<div id="errors"></div> <ul id="collatzNumbers"></ul>
Now, with our HTML expanded a little, we can write a function to update those elements:
function addToList(obj) {
if(obj.error) {
$("#errors").empty().html('Error: '+obj.err);
} else {
$("<li>")
.attr("data-num",obj.out)
.text("the number "+obj.in+" produces "+obj.out)
.appendTo("#collatzNumbers");
}
}
addToList({error: false, in: 50, out: 25});
addToList({error: false, in: 27, out: 82});
This code implements the function and shows two examples of it. Those are added to the list when the page loads. You'll see that in action in class.
The function adds a data-num attribute to
the LI element, with the value being the output of the
pair. We'll use this to add our behavior where clicking on a list item
submits the output of that pair so that we can see what's next in the
sequence.
Note how in our JQ code, sometimes we select the destination
(#errors) and then operate on it, and sometimes we create
some structure and then add it to some element
(#collatzNumbers). jQuery accommodates many styles.
How does the back end create and send such a response? The back-end processing is very similar to the normal way; the only difference is the response object. Flask provides a function called jsonify that will turn a Python data structure into a JSON string suitable for sending back to the browser. Here's the route:
@app.route('/getCollatzAjax/', methods=['GET','POST'])
def getCollatzAjax():
if request.method == 'POST':
num = request.form.get('num')
else:
num = request.args.get('num')
try:
x = int(num)
y = collatz(x)
print(x,y)
return jsonify( {'error': False, 'in': x, 'out': y} )
except Exception as err:
return jsonify( {'error': True, 'err': str(err) } )
Contrast that with our original, non-Ajax
version; they are very similar! In particular, they process the
request data in the same way. In fact, the back-end can't even tell
whether the request was made using Ajax or the normal method. This route
is called *Ajax because of the response, not the
request.
Note that we have to handle errors in a different way. Flashing works
because we render a template, but the Ajax response isn't rendering a
template so flashing is useless. Here, we put a flag in the response
saying whether there was an error and what it was. The front end is
responsible for showing it somewhere (or ignoring it). We saw that
handling in the addToList() function.
Now the centerpiece of the Ajax technology: making a request.
The jQuery library makes this much easier. The essential step is using one of the following methods:
Notice that there's no selector here; the functions are used like this:
$.get(url,data,success,datatype); $.post(url,data,success,datatype);
All arguments except the first are optional. The second argument allows you to send arbitrary form data, encoded in several different ways. Today, we'll use a JSON literal.
function sendNum(num) {
console.log("Sending "+num+" to the back end");
$.get(URL,{'num': num},addToList);
}
Note that the third argument is the addToList function
from the last section. This function sends the data to the back-end, but
it does not wait for the response. It hands
the addToList function to the browser and says,
when the response arrives, invoke this function with the response as an argument
Now, we need to set up an event handler to get the data from the form
and send it via Ajax. What we'll do is attach an event handler function to
the submit button of the form. Note that clicking on that
button would normally submit the form in the old-fashioned way, but we
don't want that default behavior. Therefore, we'll use a method of
the event object that the browser will invoke our event handler
with. Here's the code:
$("#btn-submit").on("click", function (event) {
if(progressive_enhancement_on) {
// don't do the normal form submission:
event.preventDefault();
var num = $("#form-num").val();
sendNum(num);
}
});
Did you see the use of the progressive_enhancement_on
global? That's how we'll be able to tell if we are in "normal" mode or
progressive-enhanced "ajax" mode. If we are in normal mode, this event
handler does nothing. If we are in "ajax" mode, the event handler turns
off normal form submission (by using event.preventDefault()
and instead makes an Ajax request. It pulls the input value out of the
form to send to the back end.
Looking back at our high-level view, you remember that we also implemented a behavior where the user could click on items in the list. Let's implement that now.
It turns out that there's a phenomenon called event bubbling
where a click on, say, one of our LI elements also counts as a
click on the UL that is its parent, the ancestor of that UL,
and so forth all the way up to the top of the document tree. We say that
the event bubbles up and every element on that ancestry path can
choose to handle the event or not.
Therefore, instead of adding an event handler to every LI in
the UL#collatzNumbers, we'll add one event handler to the
static ancestor and handle everything there.
But how does the single event handler know which of its many
descendants was actually clicked? jQuery has a nice feature
of delegated handlers, where you can specify a selector
for the descendants to be matched, and the magic JS variable
called this is bound to the particular descendant that was
clicked.
Here it is in action:
$("#collatzNumbers").on(
"click", // event to handle
"[data-num]", // which descendants are handled
function(event) {
var num = $(this).attr("data-num");
sendNum(num);
});
The selector is the second argument to the .on
method, and the handler function moves to third place. Here, we
used [data-num] as the selector, meaning every descendant
that has an attribute named data-num. The
variable this is bound to the particular element that was
clicked, and we can then pull the attribute value off the element and send
it to the back end using our Ajax function.
Now, to add the progressive enhancement, we can see that our JS/JQ code just sets a global variable and changes the text of the button, so that we know that it is on or not:
var progressive_enhancement_on = false;
$("#btn-progressive").on('click', function () {
progressive_enhancement_on = !progressive_enhancement_on;
if(progressive_enhancement_on) {
$("#btn-progressive").text('Progressive Enhancement ON');
} else {
$("#btn-progressive").text('Progressive Enhancement OFF');
}
});
In real life, we would just turn progressive enhancement on, under the assumption
that anyone who has JavaScript will want the better interface, but for the
purposes of this class, it's useful to be able to turn it on/off so that we can
see both behaviors. As you read the JS/JQ code below, keep in mind the
global JS variable progressive_enhancement_on.
Here's the whole of the front-end. In real life, we would put the JS/JQ code in a separate file, but here I wanted to make it easy to look back and forth between the JS/JQ and HTML, since the first is operating on the second and there are many connections between them.
<!doctype html>
<html lang='en'>
<head>
<meta charset='utf-8'>
<meta name=author content="Scott D. Anderson">
<title>{{title}}</title>
<style>
#errors { color: red; font-weight: bold; }
#result { font-weight: bold; }
#result .x, #result .y { color: red }
</style>
</head>
<body>
<p>This app computes elements of the Collatz sequence using Ajax.
<p>Use a url like
<a href="{{ url_for('getCollatzNormal', num=50 ) }}">
{{ url_for('getCollatzNormal', num=50 ) }}
</a></p>
<p><button id="btn-progressive">
progressive enhancement OFF
</button></p>
<div id="errors"></div>
<form id="number-submit-form"
method='GET'
action="{{url_for('getCollatzNormal')}}">
<p><label for="form-num">Form Number</label>:
<input type="text" name="num" id="form-num"></p>
<p><input type="submit" id="btn-submit" value="submit"></p>
</form>
{% if x %}
<p id="result">The next value after
<span class="x">{{x}}</span>
is
<span class="y">{{y}}</span></p>
{% endif %}
<p>Click on the list items to append the next value!</p>
<ul id="collatzNumbers"></ul>
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js" ></script>
<script>
/* global $ */
var URL = "{{url_for('getCollatzAjax')}}";
function addToList(obj) {
if(obj.error) {
$("#errors").empty().html('Error: '+obj.err);
} else {
$("<li>")
.attr("data-num",obj.out)
.text("the number "+obj.in+" produces "+obj.out)
.appendTo("#collatzNumbers");
}
}
addToList({error: false, in: 50, out: 25});
addToList({error: false, in: 27, out: 82});
$("#btn-submit").on("click", function (event) {
if(progressive_enhancement_on) {
// don't do the normal form submission
event.preventDefault();
var num = $("#form-num").val();
sendNum(num);
}
});
$("#collatzNumbers").on("click","[data-num]",function(event) {
var num = $(this).attr("data-num");
sendNum(num);
});
function sendNum(num) {
console.log("Sending "+num+" to the back end");
$.get(URL,{'num': num},addToList);
}
var progressive_enhancement_on = false;
$("#btn-progressive").on('click', function () {
progressive_enhancement_on = !progressive_enhancement_on;
if(progressive_enhancement_on) {
$("#btn-progressive").text('Progressive Enhancement ON');
} else {
$("#btn-progressive").text('Progressive Enhancement OFF');
}
});
</script>
</body>
</html>
Several things to note:
We've come a long way! We learned