The main function of library-web, the tool that published the API reference of the various GNOME libraries, was to take release archives and put their contents in a location that would be visible to a web server. In 2006, this was the apex of automation, of course. These days? Not so much.
Since library-web is going the way of the Dodo, and we do have better ways to automate the build and publishing of files with GitLab, how do we replace library-web in 2021? The answer is, unsurprisingly: continuous integration pipelines.
I will assume that you’re already building—and testing—your library using GitLab’s CI; if you aren’t, then you have bigger problems than just publishing your API.
So, let’s start with these preconditions:
- your project is hosted on GNOME’s GitLab instance
- your project is using Meson
- your project is using gtk-doc or gi-docgen
If your project doesn’t satisfy these preconditions you might want to work on doing so; alternatively, you can implement your own CI pipeline.
Let’s start with a simple job template:
# Expected variables:
# PROJECT_DEPS: the dependencies for your own project
# MESON_VERSION: the version of Meson you depend on
# MESON_EXTRA_FLAGS: additional Meson setup options
# you wish to pass to the configuration phase
# DOCS_FLAGS: the Meson setup option for enabling the
# documentation, if any
# DOCS_PATH: the path of the generated reference,
# relative to the build root
.gidocgen-build:
image: fedora:latest
before_script:
- export PATH="$HOME/.local/bin:$PATH"
- >
dnf install -y
python3
python3-pip
python3-wheel
gobject-introspection-devel
graphviz
ninja-build
redhat-rpm-config
- dnf install -y ${PROJECT_DEPS}
- >
pip3 install
meson==${MESON_VERSION}
gi-docgen
jinja2
Markdown
markupsafe
pygments
toml
typogrify
script:
- meson setup ${MESON_EXTRA_FLAGS} ${DOCS_FLAGS} _docs .
- meson compile -C _docs
- |
pushd "_docs/${DOCS_PATH}" > /dev/null
tar cfJ ${CI_PROJECT_NAME}-docs.tar.xz .
popd > /dev/null
- mv _docs/${DOCS_PATH}/${CI_PROJECT_NAME}-docs.tar .
artifacts:
when: always
name: 'Documentation'
expose_as: 'Download the API reference'
paths:
- ${CI_PROJECT_NAME}-docs.tar.xz
This CI template will:
- download all the required dependencies for building the API reference using gi-docgen
- build your project, including the API reference
- create an archive with the API reference
- store the archive as a CI artefact that you can easily download
Incidentally, by adding a meson test -C _build
to the script
section,
you can easily test your build as well; and if you have a test()
target in
your build that runs gi-docgen check
, then you can
verify that your documentation is always complete.
Now, all you have to do is create your own CI job that inherits from the template inside its own stage. I will use JSON-GLib as a reference:
stages:
- docs
api-reference:
stage: docs
extends: .gidocgen-build
needs: []
variables:
MESON_VERSION: 0.55.3
DOCS_FLAGS: -Dgtk_doc=true
PROJECT_DEPS:
gcc
git
gettext
glib2-devel
DOCS_PATH: docs/json-glib-1.0
“What about gtk-doc!”, I hear from the back of the room. Well, fear not, because there’s a similar template you can use if you’re still using gtk-doc in your project:
# Expected variables:
# PROJECT_DEPS: the dependencies for your own project
# MESON_VERSION: the version of Meson you depend on
# MESON_EXTRA_FLAGS: additional Meson setup options you
# wish to pass to the configuration phase
# DOCS_FLAGS: the Meson setup option for enabling the
# documentation, if any
# DOCS_TARGET: the Meson target for building the
# documentation, if any
# DOCS_PATH: the path of the generated reference,
# relative to the build root
.gtkdoc-build:
image: fedora:latest
before_script:
- export PATH="$HOME/.local/bin:$PATH"
- >
dnf install -y
python3
python3-pip
python3-wheel
gtk-doc
ninja-build
redhat-rpm-config
- dnf install -y ${PROJECT_DEPS}
- pip3 install meson==${MESON_VERSION}
script:
- meson setup ${MESON_EXTRA_FLAGS} ${DOCS_FLAGS} _docs .
# This is exceedingly annoying, but sadly its how
# gtk-doc works in Meson
- ninja -C _docs ${DOCS_TARGET}
- |
pushd "_docs/${DOCS_PATH}" > /dev/null
tar cfJ ${CI_PROJECT_NAME}-docs.tar.xz .
popd > /dev/null
- mv _docs/${DOCS_PATH}/${CI_PROJECT_NAME}-docs.tar .
artifacts:
when: always
name: 'Documentation'
expose_as: 'Download the API reference'
paths:
- ${CI_PROJECT_NAME}-docs.tar.xz
And now you can use extends: .gtkdoc-build
in your api-reference
job.
Of course, this is just half of the job: the actual goal is to publish the
documentation using GitLab’s Pages. For that, you will need
another CI job in your pipeline, this time using the deploy
stage:
stages:
- docs
- deploy
# ... the api-reference job goes here...
pages:
stage: deploy
needs: ['api-reference']
script:
- mkdir public && cd public
- tar xfJ ../${CI_PROJECT_NAME}-docs.tar.xz
artifacts:
paths:
- public
only:
- master
- main
Now, once you push to your main development branch, your API reference will be built by your CI pipeline, and the results published in your project’s Pages space—like JSON-GLib.
The CI pipeline and GitLab Pages are also useful for building complex, static websites presenting multiple versions of the documentation; or presenting multiple libraries. An example of the former is libadwaita’s website, while an example of the latter is the GTK documentation website. I’ll write a blog post about them another time.
Given that the CI templates are pretty generic, I’m working on adding them into the GNOME ci-templates repository, so you will be able to use something like:
include: 'https://gitlab.gnome.org/GNOME/citemplates/raw/HEAD/docs/gidocgen.yml'
or:
include: 'https://gitlab.gnome.org/GNOME/citemplates/raw/HEAD/docs/gtkdoc.yml'
without having to copy-paste the template in your own .gitlab-ci.yml
file.
The obvious limitation of this approach is that you will need to depend on the latest version of Fedora to build your project. Sadly, we cannot use Flatpak and the GNOME run time images for this, mainly because we are building libraries, not applications; and because extracting files out of a Flatpak build after it has completed isn’t entirely trivial. Another side effect is that if you bump up the dependencies of your project to something on the bleeding edge and currently not packaged on the latest stable Fedora, you will need to have it included as a Meson sub-project. Of course, you should already be doing that, so it’s a minor downside.
Ideally, if GNOME built actual run time images for the SDK, we could install gtk-doc, gi-docgen, and all their dependencies into the SDK itself, and avoid depending on a real Linux distribution for the libraries in our platform.