LocalStorage¶

This reading is about storing data in the browser's local storage, which persists across reloads of the browser tab or even restarting the browser!

Motivation and Context¶

We are used to saving data in global variables. For example, we counted wins, losses, and ties in the Rock-Paper-Scissor (RPS) assignment. We counted the number of matches and tries in the Concentration assignment. We represented the state of the puzzle in the Sliding Tiles assignment using global variables.

Global variables are persistent (compared to local variables, which only exist as long as the function is running), but they only last while that browser tab is open. The moment that the browser tab is closed, they disappear. So much for preserving our high scores.

The browser offers a feature called localStorage, which provides methods to save data to local storage, and to retrieve it later. (This actually saves to files on the device, but we don't have to worry about what files or where they are.)

Where is the Data?¶

It may seem obvious, but let's state it clearly: the data is stored in a particular browser.¹ If you're using Firefox on one of the public lab computers in, say, L180, the data is saved on that computer for that browser.

• if you switch to a different computer, the data is not there.
• if you switch to a different browser on the same computer (Chrome or Safari), the data is not there.
• if you switch to the browser on your laptop or your phone, the data is not there.

This is both good and bad. It means that if you save some personal data to the localstorage on your phone or laptop, it's as secure as anything else on your phone or laptop. No need for (additional) worry about hackers and such, any more than we already worry about them.

The bad, of course, is that we can't share that data across devices, say between laptop and phone, and between users, say between your device and your friend's device. For that ability, we need the Internet and cloud-based storage, which we'll talk about next time.

Partition by Origin¶

If we have two different apps, running in different browser tabs, and both have a global variable called high_score, those variables are separate and can have different values.

(This may remind you of the topic of modularity but modularity is about a program being built out of different parts, but interlocking and cooperating. The separation between browser tabs is total: not just a wall between the two, but an ocean, with no planes or ships.)

Similarly, there's a separation of localStorage values, but this is based on the origin of the page. Essentially, the origin is the server that the page is on. For example, the origin for this page is:

https://cs.wellesley.edu/

If we were to move our app to Heroku, which is a cloud platform that lets you build prototypes of apps, the origin might be related to that. In fact, the book authors for the original CoffeeRun do have a cloud-based data store running on Heroku. We'll look at it next time. They created a custom subdomain, to the app lives at the following origin: at:

https://coffeerun-v2-rest-api.herokuapp.com/

So that would be the origin for that app.

If you're interested in a more technical definition of the origin ², follow that footnote link.

Our browser stores key/value pairs partitioned by the origin of the page. You can picture it like this:

The browser looks for localStorage key/value pairs only within the origin corresponding to the current page.

However, you'll note that all of us are on https://cs.wellesley.edu, so if you visit two different pages that set the same value in localstorage, they will replace each other.

Let's be very concrete. Suppose there are two pages in the accounts of two people, Harry and Ron:

• Page 1: https://cs.wellesley.edu/~hpotter/cs204/game.html
• Page 2: https://cs.wellesley.edu/~rweasley/cs204/game.html

Suppose both of those pages set a value in localstorage (we'll see how in a moment). Those pages could be different games or the same game; doesn't matter. The crucial fact is that they might both set values in localstorage. But let's suppose that they are in fact identical and so they set the same values. (We'll do this in class.)

Both pages have the same origin, namely:

https://cs.wellesley.edu/

So they get stored in the same place in localStorage, in a particular browser.

If Hermione visits Harry's page and plays his game using her own laptop and sets a high score, then Hermione's laptop browser stores

If Hermione next visits Ron's page, plays his game, then Hermione's laptop browser stores

(I guess she got better.)

The point is that Harry's game and Ron's game store data in the same section of Hermione's browser, so she can't have different values for those different pages. We'll see a scheme later to minimize this conflict. But first, let's see how to set and get data.

API Demo¶

To store something in the local storage of the browser, any of the following syntaxes will work:

// put data into localstorage
localStorage.fred = 'Fred Weasley';
localStorage['george'] = 'George Weasley';
localStorage.setItem('harry', 'Harry Potter');

// on the other hand, this puts data into a global variable
var hermione = "Hermione Granger";

(I put the setting of the global variable in there just for contrast in the demo.)

To get the values back out, just use the expressions on the left hand side of the assignment statements above, or the getItem method:

// retrieve data from localstorage
console.log('fred:', localStorage.fred);
console.log('george:', localStorage['george']);
console.log('harry:', localStorage.getItem('harry'));

// try to retrieve data from a global variable
console.log('hermione', hermione);

Try those! Open the JS console and copy/paste the setting statements into the console. Then shift+reload this browser tab, and try the retrieval statements. The localStorage will work, but the global variable will not.

While there are three different syntaxes to set/get values from local storage, I'll use the getItem and setItem methods. See the next section.

API¶

Here's what I recommend:

• localStorage.getItem(key) returns a value from local storage or null if it doesn't exist.
• localStorage.setItem(key, value) stores a key/value pair into local storage
• localStorage.removeItem(key) removes a key/value pair from local storage
• localStorage.clear() removes every key/value pair for this origin.

I avoid .clear() to avoid removing key/value pairs from other apps in this origin.

See the MDN page on the Web Storage API

MDN has a nice web storage demo that is worth trying.

Since we'll avoid using .clear(), all of these methods use a key to look up and store data. In that way, the localStorage feature is very much like the Python/JavaScript dictionaries/objects that we know and love.

Developer Tools¶

How can you know what data is stored in your browser? The developer tools will tell you.

In Firefox:

1. open the developer tools,
2. switch to the "Storage" tab (highlighted in blue in the screenshot below)
3. toggle open the "LocalStorage" part of the page
4. click on the origin for the page you're on or interested in (highlighted in blue)
5. See the key/value pairs, as shown:

You can right-click on an item to delete it or delete all things, etc.

(I don't know the maximum capacity of localStorage; I'm sure it varies from browser to browser and device to device, and is almost certainly more than you'll need.)

Representation¶

Note that values are stored in local storage as strings, which handles a lot but not everything. For example, if you try to save a date object into local storage, it'll be converted to a string. When you read it back out, it'll be a string, not a date object. Of course, you can convert it back to an date object, but that's not automatic.

Similarly, numbers are stored as strings, and need to be converted back to numbers when they are read back out again. We'll see that in the example below.

Objects in general are converted to some kind of string, which might be just "[object Object]" and therefore could not be converted back. You would have to go to the effort of thinking about a printed representation that can be converted back.

JSON¶

Since JavaScript has a nice syntax for representing arrays (square brackets around items separated by commas) and object literals (braces around key:value pairs, separated by commas), the JavaScript Object Notation, or JSON has become a standard way to "serialize" a data structure. (There's support for JSON in both Python and Java, in addition to JavaScript, so if you want to copy a data structure from your browser to your Python program, JSON is your friend.) JSON doesn't handle everything (dates and functions are two important exceptions), but overall it does an excellent job.

Modern browser have two built-in functions:

JSON.stringify() which takes a JavaScript data structure as an argument and returns a string that represents the object.

JSON.parse() takes a string of JSON and (re-)creates the JavaScript data structure.

Obviously, these two functions are inverses, allowing us to convert between data structures and strings in an easy way.

Here's an example. Feel free to copy/paste that into the JS console to see how it works.

var d = {names: ["Tom Riddle", 
                 "Lord Voldemort", 
                 "He who must not be named"],
         age: 77,
         nose: false};
var s = JSON.stringify(d);
console.log(s);
var x = JSON.parse(s);
console.log(x.names[1]);
var s2 = JSON.stringify(x);
console.log('s == s2 ? ', s==s2);

Look at the x and d variables, and you'll see they are identical data structures, where x is a complete copy of d. Of course, s and s2 are identical strings.

Initializing A Global From LocalStorage¶

When we initialize a global variable in a normal program (when the JS file loads), we know that it has no prior value; it's being set for the first time in this execution of the program.

But if we use local storage, there are two possibilities:

• there is a prior value (saved sometime in the past), so we should use that value, or
• there is no prior value, in which case, we should set it to some default value.

Remember that getItem(key) returns null if there is nothing set, so we could do something like this:

var ls_value = localStorage.getItem('high score');
if( ls_value != null ) {
    high_score = parseInt(ls_value);
} else {
    high_score = 0;
}

The two branches of the if correspond to our two cases. We have to use the parseInt because the data is always saved/retrieved as a string, so if we want to make it an integer, we have to do so. Alternatively, JSON.parse would work. Imagine we have an array of the top 5 scores and who did them.

We could store it like this:

var top_five = [{name: 'hermione', score: 92},
                {name: 'fred', score: 88},
                {name: 'george', score: 88},
                {name: 'harry', score: 85},
                {name: 'ron', score: 84}];
const LS_KEY = 'top_five';
localStorage.setItem(LS_KEY, JSON.stringify(top_five));

and retrieve it like this:

const LS_KEY = 'top_five';
var top_five = JSON.parse(localStorage.getItem(LS_KEY));

You can try those as well, by copy/pasting them into the JS console. Try reloading the browser, or checking in the developer tools to see that the data is really saved there.

A LocalStorage Key¶

Given the earlier information about how data from one app might intefere with data from another if both apps save to local storage in a particular browser and device, I suggest using JSON to save all your data under one key in localstorage. We'll call it LS_KEY, as we saw in the previous section. The key we use is kinda like a filename: it can be relatively unique to minimize the chance that data from one app will conflict with data from another app.

For example, Ron's RPS game can save things like this:

const LS_KEY = 'Ron-RPS-game';
var data = {wins: 3, losses: 2, ties: 4};
localStorage.setItem(LS_KEY, JSON.stringify(data));

Meanwhile, Hermione's Concentration game can save things like this:

const LS_KEY = 'Hermione-Concentration-game';
var data = {games: 4, fewest_clicks: 20};
localStorage.setItem(LS_KEY, JSON.stringify(data));

Given a scheme like this, all the different games can coexist. Here's a screenshot from my browser's localstorage after doing both of those and also using the Personal Best example, described in the next section.

How would these different apps load their saved data? Like this:

const LS_KEY = 'Ron-RPS-game';
var wins, losses, ties;
var ls_data = localStorage.getItem(LS_KEY);
if( ls_data == null) {
    wins = 0;
    losses = 0;
    ties = 0;
} else {
    let data = JSON.parse(ls_data);
    wins = data.wins;
    losses = data.losses;
    ties = data.ties;
}
console.log('initialized', wins, losses, ties);

Try it!

Of course, the details of the initialization of the global variables of an app will be unique to that app, but the notion of pulling out the saved values from a JSON dictionary that was saved to localstorage is common.

Personal Best Example¶

Here's a working example that will let you save your personal best to the localstorage in this browser: personal best. Try it, including closing the browser tab and re-opening it. Try different names as well.

Do "view source" on the page to see the HTML, but there's nothing new there. Here's the JS code.

"use strict";

// This example saves all the data in a JSON dictionary stored in
// localStorage under the key 'scott_personal_best_example'. It's a
// little wordy, but less likely to conflict with other data.

const LS_KEY = 'scott_personal_best_example';

// The following all happens when the page loads

// Recall that getItem returns null if nothing is set
// so these evaluate to zero or empty string if no stored value

var your_name;
var personal_best;
var games_played;

if( localStorage.getItem(LS_KEY) == null ) {
    // initialize globals when there was no saved data
    your_name = 'unknown';
    personal_best = 0;
    games_played = 0;
} else {
    let data = JSON.parse(localStorage.getItem(LS_KEY));
    // initialize the globals from the saved data
    your_name = data.your_name;
    personal_best = data.personal_best;
    games_played = data.games_played;
}
    
// be nice and fill in the name if it's known
function fillName() {
    if( your_name != 'unknown' ) {
        $("[name=holder]").val(your_name);
    }
}
fillName();

function updatePage() {
    $("#your-name").one().text(your_name);
    $("#personal-best").one().text(personal_best);
    $("#games-played").one().text(games_played);
}
updatePage();

function storeValues() {
    let data = {'your_name': your_name,
                'personal_best': personal_best,
                'games_played': games_played};
    localStorage.setItem(LS_KEY, JSON.stringify(data));
}

function updateScores() {
    // extract values to variables
    let name = $("[name=holder]").val();
    // use parseInt so that it's a number, not a string,
    // so we can do arithmetic and compare as numbers
    let score = parseInt($("[name=score]").val());
    if( name != your_name ) {
        // new user, so record both
        your_name = name;       // store this user's name
        personal_best = 0;      // don't know personal best
        games_played = 0;
    }
    if( name == your_name && score > personal_best ) {
        // new personal best
        personal_best = score;
    }
    // always increase the number of games
    games_played ++;
    updatePage();
    storeValues();
}

// form submission handler

$("#record-score").on('submit',
                      function (evt) {
                          evt.preventDefault();
                          updateScores();
                          // reset the form
                          this.reset();
                          fillName();
                          // focus the score input
                          $("[name=score]")[0].focus();
                      });
$("#clear").on('click',
               function (evt) {
                   localStorage.removeItem(LS_KEY);
               });

You'll notice that I didn't use the localStorage.clear() method; I just removed the single LS_KEY that this app uses. The reason for that is because I might use localStorage in this same origin for other apps, and I don't want to mess up their data.

DataStore in CoffeeRun¶

Before we look at implementing a data storage class in CoffeeRun that uses local storage, let's remind ourselves of the original in-memory DataStore class:

/* Rewritten by Scott, Summer 2020. This implements a class DataStore
 * that has an API for a key/value database. 
 *
 * This version of the API uses return values.
*/

class DataStore {
    // instance variables
    data = {};

    constructor() {
        console.log('running the DataStore function')
        this.data = {};
    }

    add (key, val) {
        this.data[key] = val;
    }

    get (key) {
        return this.data[key];
    }

    getAll () {
        return this.data;
    }

    remove (key) {
        delete this.data[key];
    }
}

var ex1 = new DataStore();
ex1.add('harry', {name: 'Harry Potter',
                  hair: 'black',
                  house: 'Gryffindor'})
ex1.add('ron', {name: 'Ron Weasley',
                hair: 'red',
                house: 'Gryffindor'})
ex1.add('hermione', {name: 'Hermione Granger',
                     hair: 'brown',
                     house: 'Gryffindor'})

console.log('lookup harry', ex1.get('harry'));

function showDictionary(dic) {
    let keys = Object.keys(dic);
    keys.map(k => console.log(k, '=>', dic[k]));
}

console.log('all values before removing ron');
showDictionary(ex1.getAll());

ex1.remove('ron');

console.log('all values after removing ron');
showDictionary(ex1.getAll());

export { DataStore };

Our local storage equivalent will have to replicate that API as much as possible.

LocalStorage in CoffeeRun¶

We can use local storage in CoffeeRun as well. In CoffeeRun, we saved the orders to memory in a class called DataStore.

Of course, memory is cleared every time we close the browser. If we wanted our coffee orders to persist, we could implement a similar class with an almost identical API. We'll name it LocalStore by analogy with DataStore, but don't confuse the class LocalStore with the built-in localStorage feature. (Maybe we should come up with a less confusing name; I'm open to nominations.)

/* Written by Scott, Fall 2021. This implements a class DataStore
 * that has an API for a key/value database, saving to localStorage.
 *
 * This version of the API uses return values.
*/

class LocalStore {
    // where we save the data in localstorage
    lskey;
    data = {};
    
    constructor(lskey) {
        console.log('running the LocalStore function')
        this.lskey = lskey;
        let stored = localStorage.getItem(this.lskey);
        if( stored == null ) {
            this.data = {};
            localStorage.setItem(this.lskey, JSON.stringify(this.data));
        } else {
            this.data = JSON.parse(stored);
        }
    }

    add (key, val) {
        this.data[key] = val;
        localStorage.setItem(this.lskey, JSON.stringify(this.data));
    }

    get (key) {
        this.data = JSON.parse(localStorage.getItem(this.lskey));
        return this.data[key];
    }

    getAll () {
        this.data = JSON.parse(localStorage.getItem(this.lskey));
        return this.data;
    }

    remove (key) {
        delete this.data[key];
        localStorage.setItem(this.lskey, JSON.stringify(this.data))
    }
}

var ex1 = new LocalStore('ex1');
ex1.add('harry', {name: 'Harry Potter',
                  hair: 'black',
                  house: 'Gryffindor'})
ex1.add('ron', {name: 'Ron Weasley',
                hair: 'red',
                house: 'Gryffindor'})
ex1.add('hermione', {name: 'Hermione Granger',
                     hair: 'brown',
                     house: 'Gryffindor'})

console.log('lookup harry', ex1.get('harry'));

function showDictionary(dic) {
    let keys = Object.keys(dic);
    keys.map(k => console.log(k, '=>', dic[k]));
}

console.log('all values before removing ron');
showDictionary(ex1.getAll());

ex1.remove('ron');

console.log('all values after removing ron');
showDictionary(ex1.getAll());

export { LocalStore };

You can see that basically how this works is that the in-memory dictionary (in the instance variable this.data) is read/written from localStorage as often as necessary³ but otherwise works the same as the original memory datastore.

However, the constructor has changed a lot. First of all, we might want to have several databases in local storage, so we will store each under a key. We'll call that lskey (for Local Storage key) to distinguish it from the keys that we'll use for the coffee orders. The constructor requires an argument that specifies where the database will be stored.

The constructor looks in localstorage to see if there is a stored value. If not, it initializes (and stores) an empty dictionary. If there is a stored value, it's parsed and used.

Adjustments to Main¶

Because the methods all stayed the same (in name, arguments and return values), we don't have to change anything in the truck module and other modules that use the data store. That's great. However, we have to adjust how the truck is created.⁴ In main-module.js, instead of

    ds1 = new DataStore();
    truck1 = new Truck('ncc-1701', ds1);

we do this:

    ds1 = new LocalStore('CoffeeRun truck');
    truck1 = new Truck('ncc-1701', ds1);

(Remember that we store the data collection in localStorage under a key: that's the argument to the constructor.)

Actually, we'll give the user a pop-up choice as to whether to use the in-memory data storage or the local storage, so the new code really does:

    let useLS = confirm('use localstorage?');
    if( useLS ) {
        ds1 = new LocalStore('CoffeeRun truck');
    }  else {
        ds1 = new DataStore();
    }
    truck1 = new Truck('ncc-1701', ds1);

The built-in confirm function just pops up a box (like alert) but with two buttons:

If you click "ok" the function returns true and if you click "cancel", the function returns false. So we can easily ask the user how they want to set up Coffee Run.

Past Orders¶

There's one other change we should make. If there are saved coffee orders from the past, we should create checklist items for them. Fortunately, that's not hard:

    if( useLS ) {
        // print past orders to the console
        truck1.printOrders();
        // create checkboxes for existing orders
        let stored_orders = ds1.getAll();
        let emails = Object.keys(stored_orders);
        emails.forEach((email) => list1.addRow(ds1.get(email)));
    }

Check out the new CoffeeRun with Local Storage. You can use the developer tools to see the data saved in local storage.

Summary¶

Whew! We've come a long way. While there were a lot of details, the overall idea is pretty straightforward: the localStorage feature of a browser will let you store stuff long-term, right in the browser. So, even if you close the tab that your app is running in, or even quit the browser entirely, when you reload the app, it can pick up where it left off. Pretty cool.

• you can use localStorage to save key/value pairs in the current origin (domain)
• save values with localStorage.setItem(key, value);
• retrieve values with localStorage.getItem(key);
• remove keys with with localStorage.removeItem(key);
• clear all keys in the current origin with localStorage.clear();
• values can be any string or something convertible to a string
• keys are unique for an origin
• data persists indefinitely until it's cleared.

Here's a link to the MDN page on localStorage

Warning¶

I once counted on the persistence of localStorage to save some moderately valuable data, and it disappeared. I think this was because of a browser upgrade, so the upgrade may have replaced the files that localStorage uses, but I was still astonished, disappointed, and even a little enraged. So, while localStorage is awesome, if something is really valuable, make sure you save it in multiple locations. In other words, don't put all your eggs in one basket.

Wise words throughout life.