Modern
async JS

A talk by Christophe Porteneuve at dotJS 2015

whoami


var christophe = {
  age:        38.09034907597536,
  family:     { wife: 'Élodie', son: 'Maxence' },
  city:       'Paris, FR',
  company:    'Delicious Insights',
  trainings:  ['360° JS', 'Node.js', '360° Git'],
  webSince:   1995,
  claimsToFame: [
    'Prototype.js',
    'Ruby On Rails',
    'Bien Développer pour le Web 2.0',
    'Prototype and Script.aculo.us',
    'Paris Web'
  ]
};
          

Your daily

Callback Hell

reminder

A simple example


function readPost(req, res) {
  Post.findById(req.params.id, function(err, post) {
    if (err) {
      return handleError(err);
    }

    post.findRecentComments(function(err, comments) {
      if (err) {
        return handleError(err);
      }

      res.render('posts/show', { post: post, comments: comments, title: post.title });
    });
  });
}
            

A more involved example


function normalizeSKUs(skus, cb) {
  var normalizations = {}, count, xhrs = [];

  function handleError(err) {
    _(xhrs).invoke('abort');
    cb(err);
  }

  function handleSKU(payload) {
    normalizations[payload.from] = payload.to;
    if (++count < xhrs.length)
      return;

    cb(null, normalizations);
  }

  function normalizeFromServer(sku) {
    xhrs.push($.ajax({
      url:       '/api/v1/skus/normalize',
      data:      { sku: sku },
      onSuccess: handleSKU,
      onError:   handleError
    }));
  }
}
            

What’s wrong?

Code doesn’t flow well

Mind / Code mapping is terrible.
(mostly because no returned value / entity)

Error management is a friggin’ mess.

Untrustworthy

Will it even run?

Will it run either error or success callbacks, not both?

Will it run only once?

No composability

Staying with callbacks

Async.js

A simple example


async.waterfall([
  Store.getDefaultBundleId,
  Bundle.findById,
  function(bundle, cb) { bundle.getSKUs(cb); }
], function(err, results) {
  if (err) {
    return handleError(res, err);
  }

  res.render('bundle/skus', { bundle: bundle });
});
            

A more involved example


var computationDone = false;
async.whilst(
  function loopCheck() { return !computationDone; },
  function loopBody(cb) {
    $.getJSON('/api/v1/jobs/status/' + job.id), function(status) {
      computationDone = 'done' === status;
      cb();
    }
  },
  function loopDone() {
    async.map(
      job.imageIds,
      function mappingRequester(imageId, cb) {
        $.getJSON('/api/v1/images/' + imageId, cb);
      },
      function mapDone(err, results) {
        if (err) {
          renderError(err);
        } else {
          renderCloudImages(results);
        }
      }
    ); // map
  }
); // whilst
            

Pros

No extra concepts

No abstraction penalty (performance or code cruft)

Wide range of use cases

Well-tested, well-maintained (h/t @caolan)

Solves…

…some of the code flow and composability issues we’ve seen.

But not all. And none of the other issues.

Promises

are cool, I swear.

Old-timers already

Been there a long time, under various forms and names.

Came into JS in 2007 through Dōjō.

Promises/A proposed in CommonJS in 2009 (Kris Zyp)

Promises/A+ in 2012 (Brian Cavalier, Domenic Denicola et at.)


Recent web APIs use promises (e.g. fetch, ServiceWorker…)

Tons of libs (Q, rsvp, bluebird…), polyfills… Now ES6/2015 native.

Basic examples

XHR wrapping:


function get(url) {
  return new Promise(function(resolve, reject) {
    var req = new XMLHttpRequest();
    req.open('GET', url);
    req.onload = function() {
      if (req.status === 200) {
        resolve(req.response);
      } else {
        reject(new Error(req.statusText));
      }
    };
    req.onerror = function() { reject(new Error("Network error")); };
    req.send();
  });
}
            

jQuery’s qXHR objects are almost promises:


$.getJSON('/api/v1/status').then(renderStatus, handleError);
            

A more involved example


function getJSON(url) {
  return get(url).then(JSON.parse); // Sync, throwing steps are cast!
}

getJSON('story.json').then(function(story) {
  return getJSON(story.chapterUrls[0]); // Chain injection!
}).then(function(chapter1) {
  addHtmlToPage(chapter1.html);
}).catch(function() { // Exception catching!
  addTextToPage("Chapter couldn’t display.  FML.");
}).then(hideSpinner);
            

Pro’s and Con’s

Awesome:

Trustworthy: only one callback, only once, guaranteed. Plus, exceptions!

Composable: chainable, with injection and casting.

Not so awesome:

New abstractions

Slight performance penalty (improves, esp. with natives)

Code flow remains non-intuitive / very different from sync code.

Generators

Lazy little critters

Sheer Laziness

Generators let us do lazy eval in JS. Which is beyond awesome!

I CAN HAZ coroutines: ad hoc suspending of the current code path (incl. stack), to be resumed later. At any points in our code.

Builds upon a more general concept: iterators (and iterables)

Building blocks

Generative functions

Calling them spawns a generator, which combines the properties of an iterator and an iterable. Uses function* (note the star).


function* myGenFunction() { … }
            

I yield! I yield!

Suspend current function until caller gives control back. Coroutines!


  function* myGenFunction() {
    // …
    yield someValue;
    // …
  }
              

A conversation: yield can return a value the caller sends back, and the generator API lets the caller control the callee (returns, throws).

Your run-of-the-mill example


function* fibonacci(n) {
  const infinite = !n && n !== 0;
  let current = 0;
  let next = 1;

  while (infinite || n--) {
    yield current;
    [current, next] = [next, current + next];
  }
}

// Old-school:
var fibGen = fibonacci();
fibGen.next().value // => 0
fibGen.next().value // => 1
fibGen.next().value // => 1

// Generators are iterables, so for…of work, as do spreads:
let [...first10] = fibonacci(10);
first10 // => [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
            

Composability / Delegatability

We can delegate to another generator / iterable by using yield*.


function* getComposedGen() {
  yield* fibonacci(3);
  yield* [2, 3, 5];
  yield* "8C";
}

[...getComposedGen()] // => [0, 1, 1, 2, 4, 5, "8", "C"]
            

But… async?

Coroutines ≠ Async

The code remains synchronous, even if we jump around stack frames.


…So, generators don’t really solve async issues, do they?

Not by themselves, that’s true. But…

Promises + Generators

Or: proof that you grokked it all

Underpinnings

Forbes famously called this “Control-Flow Utopia” at JSConf.EU 2013.

The generator API is a conversation, remember? The caller can ask the callee to not just proceed (with an optional value sent back into it), but return or even throw when it resumes.

So it has to be possible to wire a runner function for generators that lets their code look very much like sync code.

This was first demonstrated in 2011 by Dave Herman, in Task.js.

Sweet examples (user side)


var loadStory = runSyncLookingAsync(function*() {
  try {
    let story = yield getJSON('story.json');
    addHtmlToPage(story.heading);

    for (let chapterPromise of story.chapterUrls.map(getJSON)) {
      let chapter = yield chapterPromise;
      addHtmlToPage(chapter.html);
    }

    addTextToPage("All done");
  } catch (err) {
    addTextToPage("Argh, broken: " + err.message);
  }
  hideSpinner();
});
            

var login = runSyncLookingAsync(function*(username, password, session) {
  var user = yield getUser(username);
  var hash = yield crypto.hashAsync(password + user.salt);
  if (user.hash !== hash) {
    throw new Error('Incorrect password');
  }
  session.setUser(user);
});
// …
login('user', 'secret', session);
            

Example runner implementation


function runSyncLookingAsync(genMakingFx){
  return function() {
    var generator = genMakingFx(...arguments);

    function handle(result) {
      if (result.done) {
        return Promise.resolve(result.value);
      }

      return Promise.resolve(result.value).then(
        function(res) { return handle(generator.next(res)); },
        function(err) { return handle(generator.throw(err)); }
      );
    }

    try {
      return handle(generator.next());
    } catch (ex) {
      return Promise.reject(ex);
    }
  }
}
            

Pro’s and Con’s

Awesome:

Code is indeed async (non-blocking).

Code looks very much sync (easier to reason about).

You can have your cake and eat it too.

Not so awesome:

A bit hackish.

Added weight (syntax + performance) due to generators + promises.

You could argue these building blocks were not designed for this, and are therefore not optimized for this use-case.


Which is why…

async / await

ZOMG.

Nailed it.

Sort of the Holy Grail of async JS.

Basically copy-pasted from C# 5.0 / .NET 4.5 (2012).

(Yes, there is much goodness in MS language designs; h/t Anders)

Pretty much find-and-replace


async function loadStory() {
  try {
    let story = await getJSON('story.json');
    addHtmlToPage(story.heading);

    for (let chapterPromise of story.chapterUrls.map(getJSON)) {
      let chapter = await chapterPromise;
      addHtmlToPage(chapter.html);
    }

    addTextToPage("All done");
  } catch (err) {
    addTextToPage("Argh, broken: " + err.message);
  }
  hideSpinner();
});
            

async function login(username, password, session) {
  var user = await getUser(username);
  var hash = await crypto.hashAsync(password + user.salt);
  if (user.hash !== hash) {
    throw new Error('Incorrect password');
  }
  session.setUser(user);
});
            

Pro’s and Con’s

Hugs, Kisses and High Fives!

Like all such new features (e.g. promises and generators), it still requires optimization for performance to really get there. But totally worth it. And it will get there. Just look at the trend of JS performance in general.

DO WANT!

Currently at ES Stage 3 (“Candidate”) stage. Check out the draft.

Almost certainly in ES7/2016.

In the meantime, available behind flags in Edge (ha!), and in Babel/Traceur, obviously.

Reactive Programming

Making sense of ongoing events

Following the events lately?

So far every async op was about getting value(s) just once.

What of multiple values over time? What of streams, especially event streams?


Promises won’t cut it: they’re designed to resolve only once.

This is what Reactive Programming is for.

JS lends itself especially well to Functional Reactive Programming (FRP), which is RP mixed with FP “building blocks” such as map, reduce, filter

A powerful RxJS example


function searchWikipedia(term) {
  return $.ajaxAsObservable( /* … */ )
    .select(function(results) { return results.data; }); // Grab the data property
}

var input = $('#textInput');
var keyup = input.keyupAsObservable()                    // Event stream!
  .select(function() { return input.val(); })            // Grab the field value
  .where(function(text) { return text.length > 2; })     // Only proceed when long enough
  .throttle(500);                                        // Slow down, breathe…

var searcher = keyup
  .select(function(text) { return searchWikipedia(text); }) // Map on an XHR *result*!
  .switchLatest()                                           // Only keep latest response
  .where(function(data) { return data.length === 2; });     // Only keep useful results

searcher.subscribe(renderResults); // React to final stream by rendering UI
            

A word about

Debugging Async

Quick reminders

JS is an event loop

JS is single-threaded

Async callbacks run on a basically empty call stack

Which makes debugging… interesting.

Async call stacks in DevTools

Chrome got async call stacks around version 35 (mid-2014).

“Traverses” async handovers: timers, XHR, promises, rAF, most observers, postMessage, FileSystem, IndexedDB…

An example async call stack

Check out this comprehensive article

Node async stack traces

When in Node, you can enable (in development, preferrably: performance penalty…) so-called long stack traces through specific modules. npm has a few, the most popular apparently being longjohn.

Thank you!

And may JS be with you


Christophe Porteneuve

@porteneuve

Get the slides on bit.ly/dotjs-async