Origin script can only accept a file as parameter, this commit enables
it to scan a directory.
Usage example:
./scripts/checktransupdate.py Documentation/translations/zh_CN/dev-tools
And it will output something like:
"""
[1772967203.351603] Documentation/translations/zh_CN/dev-tools/kmemleak.rst
[1772967205.074201] commit 7591c127f3 ("kmemleak: iommu/iova: fix transient kmemleak false positive")
[1772967205.074337] 1 commits needs resolving in total
[1772967205.301705] Documentation/translations/zh_CN/dev-tools/index.rst
[1772967206.912395] commit a592a36e49 ("Documentation: use a source-read extension for the index link boilerplate")
[1772967206.921424] commit 6eac13c876 ("Documentation: dev-tools: add container.rst page")
[1772967206.930220] commit 8f32441d7a ("Documentation: Add documentation for Compiler-Based Context Analysis")
[1772967206.939002] commit 1e9ddbb2cd ("docs: Pull LKMM documentation into dev-tools book")
[1772967206.948636] commit d5af79c05e ("Documentation: move dev-tools debugging files to process/debugging/")
[1772967206.957562] commit d5dc958361 ("kbuild: Add Propeller configuration for kernel build")
[1772967206.966255] commit 315ad8780a ("kbuild: Add AutoFDO support for Clang build")
[1772967206.966410] 7 commits needs resolving in total
"""
Signed-off-by: Haoyang LIU <tttturtleruss@gmail.com>
[jc: tweaked coding style]
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260308111314.27333-1-tttturtleruss@gmail.com>
Documentation builds were using "-q" for a long time, but sometimes
it is nice to see the Sphinx progress, without increasing build
verbosity - which would also turn on kernel-doc verbosity.
Instead of doing that, let's parse the sphinx-build already-existing
-v: each time it is used, it increases the verbosity level.
With that, if the default is to use -q, a single -v will disable
quiet mode. Passing more -v will keep increasing its verbosity.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <38b24e97a3cbd2def418359a8e69b1b087a945ad.1769500383.git.mchehab+huawei@kernel.org>
When the user explicitly specifies SPHINXDIRS to build a specific
subdirectory it is unexpected that the rust docs are also generated.
Especially as their generation may dominate the execution time.
Only generate the rust docs when they are part of the SPHINXDIRS
requested by the user. 'rust/rustdocs' is not considered, as it is
not a valid SPHINXDIRS anyways.
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260118-docs-spurious-rust-v1-4-998e14b9ed9e@weissschuh.net>
Now that the Sphinx build does not use the kerneldoc_bin configuration
variable, we shouldn't try to set it in the build wrapper or we get a nifty
warning:
WARNING: unknown config value 'kerneldoc_bin' in override, ignoring
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The recent move of this script from scripts/ to tools/docs/
did not account for the 'cd' directory usage.
Update "cd .." to "cd ../.." to make the script self-correcting.
This also eliminates a shell warning:
./tools/docs/find-unused-docs.sh: line 33: cd: Documentation/: No such file or directory
Fixes: 184414c6a6 ("docs: move find-unused-docs.sh to tools/docs")
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Fixes: 184414c6a6 (docs: move find-unused-docs.sh to tools/docs)
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260102200657.1040234-1-rdunlap@infradead.org>
The makefile logic to detect if rust is enabled is not working
the way it was expected: instead of using the current setup
for CONFIG_RUST, it uses a cached version from a previous build.
The root cause is that the current logic inside docs/Makefile
uses a cached version of CONFIG_RUST, from the last time a non
documentation target was executed. That's perfectly fine for
Sphinx build, as it doesn't need to read or depend on any
CONFIG_*.
So, instead of relying at the cache, move the logic to the
wrapper script and let it check the current content of .config,
to verify if CONFIG_RUST was selected.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <c06b1834ef02099735c13ee1109fa2a2b9e47795.1763722971.git.mchehab+huawei@kernel.org>
As we want to call Python code directly at the Sphinx extension,
convert get_feat.pl to Python.
The code was made to be (almost) bug-compatible with the Perl
version, with two exceptions:
1. Currently, Perl script outputs a wrong table if arch is set
to a non-existing value;
2. the ReST table output when --feat is used without --arch
has an invalid format, as the number of characters for the
table delimiters are wrong.
Those two bugs were fixed while testing the conversion.
Additionally, another caveat was solved:
the output when --feat is used without arch and the feature
doesn't exist doesn't contain an empty table anymore.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <03c26cee1ec567804735a33047e625ef5ab7bfa8.1763492868.git.mchehab+huawei@kernel.org>
Now that we have tools/lib/python for our Python modules, turn them into
proper packages with a single namespace so that everything can just use
tools/lib/python in sys.path. No functional change.
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251110220430.726665-3-corbet@lwn.net>
"scripts/lib" was always a bit of an awkward place for Python modules. We
already have tools/lib; create a tools/lib/python, move the libraries
there, and update the users accordingly.
While at it, move the contents of tools/docs/lib. Rather than make another
directory, just put these documentation-oriented modules under "kdoc".
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20251110220430.726665-2-corbet@lwn.net>
Our documentation-related tools are spread out over various directories;
several are buried in the scripts/ dumping ground. That makes them harder
to discover and harder to maintain.
Recent work has started accumulating our documentation-related tools in
/tools/docs. This series nearly completes that task, moving most of the
rest of our various utilities there, hopefully fixing up all of the
relevant references in the process.
The one exception is scripts/kernel-doc; that move turned up some other
problems, so I have dropped it until those are ironed out.
At the end, rather than move the old, Perl kernel-doc, I simply removed it.
Move this tool out of scripts/ to join the other documentation tools; fix
up a couple of erroneous references in the process.
It's worth noting that this script will fail badly unless one has a
PYTHONPATH referencing scripts/lib/abi.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
The scripts for managing the features docs are found in three different
directories; unite them all under tools/docs and update references as
needed.
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Acked-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Quoth Mauro:
This series should probably be called:
"Move the trick-or-treat build hacks accumulated over time
into a single place and document them."
as this reflects its main goal. As such:
- it places the jobserver logic on a library;
- it removes sphinx/parallel-wrapper.sh;
- the code now properly implements a jobserver-aware logic
to do the parallelism when called via GNU make, failing back to
"-j" when there's no jobserver;
- converts check-variable-fonts.sh to Python and uses it via
function call;
- drops an extra script to generate man pages, adding a makefile
target for it;
- ensures that return code is 0 when PDF successfully builds;
- about half of the script is comments and documentation.
I tried to do my best to document all tricks that are inside the
script. This way, the docs build steps is now documented.
It should be noticed that it is out of the scope of this series
to change the implementation. Surely the process can be improved,
but first let's consolidate and document everything on a single
place.
Such script was written in a way that it can be called either
directly or via a Makefile. Running outside Makefile is
interesting specially when debug is needed. The command line
interface replaces the need of having lots of env vars before
calling sphinx-build:
$ ./tools/docs/sphinx-build-wrapper --help
usage: sphinx-build-wrapper [-h]
[--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]] [--conf CONF]
[--builddir BUILDDIR] [--theme THEME] [--css CSS] [--paper {,a4,letter}] [-v]
[-j JOBS] [-i] [-V [VENV]]
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
Kernel documentation builder
positional arguments:
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
Documentation target to build
options:
-h, --help show this help message and exit
--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]
Specific directories to build
--conf CONF Sphinx configuration file
--builddir BUILDDIR Sphinx configuration file
--theme THEME Sphinx theme to use
--css CSS Custom CSS file for HTML/EPUB
--paper {,a4,letter} Paper size for LaTeX/PDF output
-v, --verbose place build in verbose mode
-j, --jobs JOBS Sets number of jobs to use with sphinx-build
-i, --interactive Change latex default to run in interactive mode
-V, --venv [VENV] If used, run Sphinx from a venv dir (default dir: sphinx_latest)
the only mandatory argument is the target, which is identical with
"make" targets.
The call inside Makefile doesn't use the last four arguments. They're
there to help identifying problems at the build:
-v makes the output verbose;
-j helps to test parallelism;
-i runs latexmk in interactive mode, allowing to debug PDF
build issues;
-V is useful when testing it with different venvs.
When used with GNU make (or some other make which implements jobserver),
a call like:
make -j <targets> htmldocs
will make the wrapper to automatically use POSIX jobserver to claim
the number of available job slots, calling sphinx-build with a
"-j" parameter reflecting it. ON such case, the default can be
overriden via SPHINXDIRS argument.
Visiable changes when compared with the old behavior:
When V=0, the only visible difference is that:
- pdfdocs target now returns 0 on success, 1 on failures.
This addresses an issue over the current process where we
it always return success even on failures;
- it will now print the name of PDF files that failed to build,
if any.
In verbose mode, sphinx-build-wrapper and sphinx-build command lines
are now displayed.
C domain supports a ".. c:namespace::" tag that allows setting a
symbol namespace. This is used within the kernel to avoid warnings
about duplicated symbols. This is specially important for syscalls,
as each subsystem may have their own documentation for them.
This is specially true for ioctl.
When such tag is used, all C domain symbols have c++ style,
e.g. they'll become "{namespace}.{reference}".
Allow specifying C namespace at the exception files, avoiding
the need of override rules for every symbol.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <cc27ec60ceb3bdac4197fb7266d2df8155edacda.1759329363.git.mchehab+huawei@kernel.org>
When used in practice, one may want to have multiple header
files on a single rst file, like:
***********************
Digital TV uAPI symbols
***********************
.. contents:: Table of Contents
:depth: 2
:local:
Frontend
========
.. kernel-include:: include/uapi/linux/dvb/frontend.h
:generate-cross-refs:
:toc:
Demux
=====
.. kernel-include:: include/uapi/linux/dvb/dmx.h
:generate-cross-refs:
:toc:
...
So, don't add ..contents:: here.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <4bf353e5248133a3b0abd82519a38453402fe7c6.1759329363.git.mchehab+huawei@kernel.org>
On recent versions of openSUSE Tumbleweed, sphinx-buildis is no longer
a Python script, but something else. Such change is due to how
it now handles alternatives:
https://en.opensuse.org/openSUSE:Migrating_to_libalternatives_with_alts
The most common approach that distros use for alternatives is via
symlinks:
lrwxrwxrwx 1 root root 22 out 31 2024 /usr/bin/java -> /etc/alternatives/java
lrwxrwxrwx 1 root root 37 mar 5 2025 /etc/alternatives/java -> /usr/lib/jvm/java-21-openjdk/bin/java
With such approach, one can sun the script with either:
<sphinx>
python3 <script>
However, openSUSE's implementation uses an ELF binary (/usr/bin/alts),
which breaks the latter format.
It is needed to allow users to specify the Python version to be
used while building docs, as some distros like Leap 15.x are
shipped with:
- older, unsupported python3/python3-sphinx packages;
- more modern python3xx/python3xx-sphinx packages that work
properly.
On such distros, building docs require running make with:
make PYTHON3=python3.11 htmldocs
Heh, even on more moderen distros where python3-sphinx
is supported, one may still want to use a newer package,
for instance, due to performance issues, as:
- with Python < 3.11, kernel-doc is 3 times slower;
- while building htmldocs with Python 3.13/Sphinx 8.x
takes about 3 minutes on a modern machine, using
Sphinx < 8.0 can take up to 16 minutes to build docs
(7.x are the worse ones and require lots of RAM).
So, even with not too old distros, one still may want to use
for instance PYTHON3=python3.11.
To acommodate using PYTHON3 without breaking on Tumbleweed,
add a workaround that will only use:
$(PYTHON3) sphinx-build
if PYTHON3 env var is not default.
While here, drop the special check for venv, as, with venv,
we can just call sphinx-build directly without any extra
checks.
Reported-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://lore.kernel.org/all/883df949-0281-4a39-8745-bcdcce3a5594@infradead.org/
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <8917f862e0b8484c68408c274129c9f37a7aefb4.1758539031.git.mchehab+huawei@kernel.org>
The code here was meant to handle 3 functions:
1. allow having a separate conf.py file, per subdir;
2. generate a list of latex documents.
3. set "subproject" tag if SPHINXDIRS points to a subdir.
We don't have (1) anymore, and (3) is now properly handled
entirely inside conf.py.
So, only (3) is still needed, and this is a single-line change
at conf.py.
So, drop it, moving the remaining code to conf.py.
While here, drop a duplicated $(RUSTDOC) command-line argument.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <ec998f9f268a401ca6aa36e3221d39c97efeccaa.1758361087.git.mchehab+huawei@kernel.org>
Before this patch, building htmldocs on opensuseLEAP works
fine:
# make htmldocs
Available Python versions:
/usr/bin/python3.11
Python 3.6.15 not supported. Changing to /usr/bin/python3.11
Python 3.6.15 not supported. Changing to /usr/bin/python3.11
Using alabaster theme
Using Python kernel-doc
...
As the logic detects that Python 3.6 is too old and recommends
intalling python311-Sphinx. If installed, documentation builds
work like a charm.
Yet, some develpers complained that running python3.11 instead
of python3 should not happen. So, let's break the build to make
them happier:
$ make htmldocs
Python 3.6.15 not supported. Bailing out
You could run, instead:
/usr/bin/python3.11 tools/docs/sphinx-build-wrapper htmldocs \
--sphinxdirs=. --conf=conf.py --builddir=Documentation/output --theme= --css= \
--paper=
Python 3.6.15 not supported. Bailing out
make[2]: *** [Documentation/Makefile:76: htmldocs] Error 1
make[1]: *** [Makefile:1806: htmldocs] Error 2
make: *** [Makefile:248: __sub-make] Error 2
It should be noticed that:
1. after this change, sphinx-pre-install needs to be called
by hand:
$ /usr/bin/python3.11 tools/docs/sphinx-pre-install
Detected OS: openSUSE Leap 15.6.
Sphinx version: 7.2.6
All optional dependencies are met.
Needed package dependencies are met.
2. sphinx-build-wrapper will auto-detect python3.11 and
suggest a way to build the docs using the parameters passed
via make variables. In this specific example:
/usr/bin/python3.11 tools/docs/sphinx-build-wrapper htmldocs --sphinxdirs=. --conf=conf.py --theme= --css= --paper=
3. As this needs to be executed outside docs Makefile, it won't run
the validation check scripts nor build Rust documentation if
enabled, as the extra scripts are part of the docs Makefile.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-ID: <0635c311295300e9fb48c0ea607e2408910036e3.1758196090.git.mchehab+huawei@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Mauro Carvalho Chehab
Haoyang LIU
Thomas Weißschuh
Jonathan Corbet
Randy Dunlap
Akira Yokosawa