MDN/Projects/Completed/GitHub Integration

What do we want?

• To keep the documentation source for a product in GitHub, but the published docs on MDN.
• To automate the publication of the docs to MDN when the source is changed.
• To publish the most recent build of the docs as well as the set of docs applicable to each formal release of the product.
• To indicate to users which version of the docs is applicable to the most recent release of the product.
• To enable users to download a zip of a version of the docs, for use offline.

How could we do it?

This requires a service running on MDN to listen for changes in the repo and update the MDN site when relevant changes happen. Some of this code can be generic, and some would have to be product-specific. I'd expect the product-specific code would live in the same repo as the product.

The generic service would need to know the name of each product and the name of the repo containing its documentation.

URL Structure

On MDN, each product has a URL structure like this:

<base_url> + <product_name> + development/
<base_url> + <product_name> + latest/
<base_url> + <product_name> + version_N/
<base_url> + <product_name> + version_N-1/
<base_url> + <product_name> + version_N-2/
<base_url> + <product_name> + version N-.../

<base_url> is defined by MDN, while <product_name> and the version syntax are defined by each product.

For example:

/en-US/docs/Mozilla/addon-sdk/development/index.html
/en-US/docs/Mozilla/addon-sdk/latest/index.html
/en-US/docs/Mozilla/addon-sdk/1.14/index.html
/en-US/docs/Mozilla/addon-sdk/1.13/index.html

• The subtree under "development" is for the latest, bleeding-edge, revision of the docs, and usually isn't part of a formal release.

• Each subtree under a version identifier is the version of the docs for that released version of the product.

• The subtree under "latest" is an alias pointing to the most recent released version: so in the example above it would point to 1.14.

Post-receive hook

The product's GitHub repo includes a post-receive hook that posts to an MDN URL. The service running at that URL looks at the JSON data about the commit to decide if:

1. it changed any docs
2. it represented a new release of the product, and if so, what the version identifier for this release is

The way to answer both these questions is probably product-specific, as it depends on how products organize docs and code, and on how they tag releases. So the generic service would have to ask a product-specific piece of code these questions.

If either is true, the service:

• clones the repo
• calls a function in the repo that generates the docs
• copies the generated docs to "<base_url> + <product_name> + development/", replacing what was there before

If this commit also represents a new release, the service creates a new directory for the release like "<base_url> + <product_name> + <version_identifier>", copies the generated docs there as well, and updates the alias at "<base_url> + <product_name> + latest/" to point to this release.

Obsoleting old releases

It would be great if users visiting old versions of the docs were non-intrusively informed of this fact and offered a link to the latest version. The SDK docs do this at the moment (for example: https://addons.mozilla.org/en-US/developers/docs/sdk/1.10/packages/addon-kit/page-mod.html), but I have to add this when I generate the docs.

Perhaps the server could look at the version component of the URI, and insert a "this page is obsolete" notice of the version != the current value of the "latest" alias?

Supporting downloads

It would be great to enable users to download a zipped version of the docs for a given release. I'm not quite sure how this would be done. Perhaps the product-specific docs code could embed a relative link to the docs in the pages it generates (for example, it could be at <base_url> + <product_name> + downloads/ + <version_identifier>.zip) and the generic code could handle zipping up the docs and copying them under there?

Product-specific versus generic code

I'm not sure where we should draw the line between product-specific and generic code. In this proposal, I think the product-specific code implements 2 interfaces:

• one that takes the JSON payload of the post-receive hook, and tells the generic code whether this commit changed the docs and whether it represents a new release, and if so, what the release identifier is

• one that takes an optional release identifier and a path to a copy of the repo, and builds the docs from that repo

For the second interface, the code could be taken from the repo itself. For the first, it can't, unless the generic service is happy to clone the repo on every commit (which might be OK, depending on how large and active the repo is).

Readthedocs.org?

We've talked in the past about using readthedocs.org to accomplish this. I'd love to do that, but it's unclear to me where in this proposal we're using readthedocs rather than just rolling our own thing.

I can understand how we could do our own thing. On the other side, I can understand how we could use readthedocs.org more or less as-is. But having looked a bit at readthedocs.org, it seems to be very heavily dependent on the docs using Sphinx/reStructuredText - so if it's important to us to support other formats (which it probably is), then it might be a big change.