Async iteration and for await…of

Generators produce values on demand, but each value has to be ready immediately. Plenty of data is not: pages from an API, chunks of a file download, messages arriving one by one. Async iteration is the same protocol with promises added, and for await...of lets you loop over it as if it were an ordinary list.

The async iteration protocol

It mirrors the synchronous version:

• An async iterable has a method under Symbol.asyncIterator that returns an async iterator.
• An async iterator has a next() method that returns a promise of { value, done }.

for await...of calls next(), awaits the promise, and runs the loop body with each value until done is true. It can be used inside async functions and at the top level of ES modules.

Async generators

Writing that protocol by hand is tedious. An async generator, declared with async function*, can use both await and yield, and implements the protocol for you:

const wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function* ticks(count, interval) {
  for (let i = 1; i <= count; i++) {
    await wait(interval);
    yield `tick ${i}`;
  }
}

for await (const tick of ticks(3, 500)) {
  console.log(tick);
}
// → tick 1   (after 0.5 s)
// → tick 2   (after 1 s)
// → tick 3   (after 1.5 s)
console.log('done'); // → done

The loop body runs once per value, and the next value is not requested until the body has finished. That built-in back-pressure is the main difference from firing off every request at once with Promise.all.

Paging through an API lazily

APIs usually return large collections in pages. Each response carries some items and a pointer to the next page. A fake API makes the example runnable anywhere:

const allCourses = [
  'HTML', 'CSS', 'Responsive Web Design', 'Web Accessibility',
  'JavaScript', 'TypeScript', 'WordPress',
];

async function fetchCoursePage(page) {
  await wait(200); // Pretend network delay.
  const size = 3;
  const start = (page - 1) * size;
  const items = allCourses.slice(start, start + size);
  const hasMore = start + size < allCourses.length;
  return { items, next: hasMore ? page + 1 : null };
}

An async generator hides the paging. Callers see one stream of courses:

async function* allPages() {
  let page = 1;
  while (page !== null) {
    const response = await fetchCoursePage(page);
    yield* response.items;
    page = response.next;
  }
}

for await (const title of allPages()) {
  console.log(title);
}
// → HTML
// → CSS
// → Responsive Web Design
// → Web Accessibility
// → JavaScript
// → TypeScript
// → WordPress

yield* inside an async generator delegates to any iterable, here the array of items on each page. The generator fetches page 2 only after the loop has consumed every item from page 1. It is lazy: if the caller stops early, later pages are never requested.

async function findCourse(name) {
  for await (const title of allPages()) {
    if (title === name) {
      return title; // Stops iteration; no more pages are fetched.
    }
  }
  return null;
}

console.log(await findCourse('CSS')); // → CSS (only page 1 was fetched)

The same pattern with fetch()

On the course dashboard, imagine the data split into numbered files, data/courses-1.json, data/courses-2.json and so on, each shaped like { "items": [...], "next": 2 }, with "next": null on the last page. The real version has the same shape as the fake one:

async function* fetchAllCourses() {
  let page = 1;
  while (page !== null) {
    const response = await fetch(`data/courses-${page}.json`);
    if (!response.ok) {
      throw new Error(`Page ${page} failed with HTTP ${response.status}`);
    }
    const { items, next } = await response.json();
    yield* items;
    page = next;
  }
}

Many real APIs return a next URL rather than a page number, or a cursor string to send with the next request. Only the line that builds the request changes; the generator and every caller stay the same. As in earlier lessons, serve the folder with a local server so that fetch() can load the files.

Showing progress as pages arrive

Async iteration suits interfaces that fill in as data arrives:

<button type="button" id="load">Load all courses</button>
<p id="status" role="status" aria-live="polite"></p>
<ul id="course-list"></ul>

<script type="module">
  import { fetchAllCourses } from './courses.js';

  const button = document.querySelector('#load');
  const status = document.querySelector('#status');
  const list = document.querySelector('#course-list');

  button.addEventListener('click', async () => {
    button.disabled = true;
    list.replaceChildren();
    status.textContent = 'Loading courses…';
    let count = 0;

    try {
      for await (const course of fetchAllCourses()) {
        const item = document.createElement('li');
        item.textContent = `${course.title} (${course.level})`;
        list.append(item);
        count++;
      }
      status.textContent = `Loaded ${count} courses.`;
    } catch (error) {
      status.textContent = `Loaded ${count} courses, then an error occurred: ${error.message}`;
    } finally {
      button.disabled = false;
    }
  });
</script>

This assumes courses.js exports fetchAllCourses and that each item is a course object with title and level. The status region announces the start and the final count only; announcing every item would flood screen reader users.

Errors and early exit

An error thrown inside an async generator, or a rejected promise it awaits, rejects the promise from next(). for await...of turns that into an exception at the loop, so a normal try and catch around the loop handles it, as above.

Exiting the loop early with break, return or an exception calls the iterator's return() method. For an async generator, that runs its finally blocks, which is where clean-up belongs:

async function* messages() {
  console.log('connect');
  try {
    let n = 1;
    while (true) {
      await wait(100);
      yield `message ${n++}`;
    }
  } finally {
    console.log('disconnect');
  }
}

for await (const message of messages()) {
  console.log(message);
  if (message === 'message 2') break;
}
// → connect
// → message 1
// → message 2
// → disconnect

for await with ordinary iterables

for await...of also accepts synchronous iterables of promises. It awaits each item in turn:

const requests = [wait(300).then(() => 'a'), wait(100).then(() => 'b')];

for await (const value of requests) {
  console.log(value);
}
// → a
// → b

Both promises were already running, so the total time is about 300 ms and the output follows array order, not finishing order. Be careful with this pattern: if a later promise rejects while the loop is still waiting on an earlier one, the rejection is not handled until the loop reaches it, and runtimes may report it as an unhandled rejection in the meantime. For a fixed list of promises, Promise.all or Promise.allSettled is usually clearer.

Built-in async iterables

Some platform objects are async iterable. In Node.js, readable streams and readline interfaces can be consumed with for await...of, which is a convenient way to process a large file line by line (covered properly in a Node.js course). The web ReadableStream, such as response.body from fetch(), is async iterable in some environments, including Node.js, but not yet in every browser, so check MDN before relying on it. Array.fromAsync() collects an async iterable into an array; it is newer, so check support for it too.

Summary

• Async iterables have a Symbol.asyncIterator method; their next() returns a promise of { value, done }.
• async function* combines await and yield to write async iterators easily.
• for await...of requests the next value only after the loop body finishes, which gives natural back-pressure.
• Wrap paginated APIs in an async generator so callers see one lazy stream of items.
• Use try and catch around the loop for errors, and finally in the generator for clean-up.
• Breaking out of a loop does not cancel an in-flight request; use an AbortSignal for that.