← Back to blogs
5 min read

Rebuilding a Site 11 Million People Read, With Antora and Gatsby

tech opensource
Read with ChatGPT Claude

My Google Summer of Code 2023 project had a title that sounds boring until you sit with it: “Building jenkins.io with alternative tools.” Translation: take the documentation site that around 11 million people rely on, and rebuild the machinery underneath it without the readers ever noticing. No new features they’d see. Just a better engine, swapped in mid-flight.

That “mid-flight” part is the whole story. You can’t take the Jenkins docs offline for a month while you re-architect them. So this post is about the two tools we reached for, Antora and Gatsby, why that specific pairing, and the thing nobody warns you about when you migrate a site this big: the content is the hard part, not the code.

The problem with a giant docs site

Documentation for a project as old and sprawling as Jenkins isn’t one book. It’s advisories, guides, plugin docs, pipeline step references, event pages, author contributions, upgrade notes. Written by hundreds of people over many years. Living in different places, different formats, different assumptions.

The old tooling worked, but “works” and “pleasant to maintain and evolve” are different sentences. The goal was a stack where docs are versioned properly, modular, and rendered by something modern enough that contributors in 2023 actually enjoy touching it. That’s where the two-tool split comes in.

Why Antora for the docs

Antora is built for exactly one thing: multi-repo, multi-version technical documentation written in AsciiDoc. The reason it fits Jenkins is versioning. A project like this needs “here are the docs for the version you’re actually running,” and Antora treats versions as a first-class idea instead of a folder-naming convention you pray everyone respects.

The magic is that a whole documentation site is described by one small playbook file that points at content sources in git:

content:
  sources:
    - url: https://github.com/some-org/some-docs
      branches: [main, v2.x, v1.x]
      start_path: docs
ui:
  bundle:
    url: ./ui-bundle.zip

Antora clones those sources, reads the structured content, resolves cross-references between pages across versions and repos, and hands you a coherent model. You stop hand-maintaining a navigation tree. The structure comes from the content itself. For a docs set the size of Jenkins’, that’s the difference between sustainable and not.

Why Gatsby for the site

Gatsby is a React-based static site generator with a GraphQL data layer. That GraphQL part is the reason it earns its place here. In Gatsby, every content source, the Antora-built docs, Markdown pages, JSON, gets pulled into a single unified data graph, and you query the exact slice each page needs:

query {
  allMarkdownRemark(sort: { frontmatter: { date: DESC } }) {
    nodes {
      frontmatter { title date }
      fields { slug }
    }
  }
}

Then Gatsby renders everything to static HTML at build time, so the site is fast and cheap to serve (no server rendering per request), while contributors still get to work in React components instead of hand-written templates. The full rebuilt site lives in Vandit1604/jenkins-io-docs, and the big merge that brought it into the official infra is docs.jenkins.io#106.

The part that actually eats your summer

Here’s what I did not expect. The interesting engineering (wiring Antora into Gatsby, building the data pipeline, the components) was maybe 40% of the work. The other 60% was the content itself. Advisories, guides, author pages, event info, pipeline step docs, upgrade guides. When you move a site with this much history, every category of content has its own quirks, its own edge cases, its own “well, this one page does it differently for a reason nobody remembers.”

You cannot rebuild the engine and quietly drop half the cargo. Every kind of page had to keep working, keep its URLs, keep making sense. That’s the unglamorous truth of any real migration: the code is finite and knowable, but the content is a decade of accumulated decisions, and you have to honor all of them.

What I took from it

Rebuilding infrastructure that’s actively in use, for an audience this size, taught me a kind of engineering humility you don’t get from greenfield projects. Nobody claps when a migration goes well. Success looks like nothing happening from the reader’s side. The site just quietly gets better to maintain, and the 11 million people never know there was surgery.

That’s a good chunk of what real infrastructure work is. Do the hard thing carefully enough that no one notices you did it.

TL;DR

  • GSoC 2023: rebuild the tooling behind jenkins.io docs (11M+ users) without users noticing. New engine, same flight.
  • Antora owns the content: versioned, multi-repo AsciiDoc turned into a structured model from one small playbook. Versioning is first-class.
  • Gatsby owns the site: React + a GraphQL data layer, everything rendered to fast static HTML at build time.
  • The code was the easy 40%. The content (advisories, guides, upgrade docs, a decade of edge cases) and not breaking a single URL was the real work.
  • Good infra work is invisible. Success is the reader noticing nothing.

Go deeper