| Contributor: | Jugal Patel (Jugal59) |
| Organization: | Debian |
| Project: | Provide debuginfod server |
| Mentor: | Colin Watson |
About the project and me
Your program crashes. You open gdb and get ?? instead of a stack trace. So
you go find the right -dbgsym package, for the right version, for the right
architecture, install it, and start again.
Debuginfod removes that
entire detour: gdb asks a server for symbols by the build-ID baked into the
binary. Debusine already built packages, already produced -dbgsym files, and
already hosted the archives; it just couldn’t answer the question.
This summer I made it answer. My project was to add debuginfod server functionality to Debusine so that it not only hosts -dbgsym packages, but also serves their debug symbols over the debuginfod(8) protocol. Debian developers can then debug binaries by setting a single URL that gdb uses to fetch the matching debug symbols. This project took me through design, backend work, an extraction pipeline on the worker, HTTP serving, documentation, and testing from the first blueprint all the way to a live demo on debusine.debian.net.
Initial planning and design changes
A design first, in !3030. The proposal submitted for GSoC 2026 was just an overview of how things will work, but in reality there were a lot of design questions which needed to be answered before starting with contribution. Debusine keeps development blueprints in its docs tree, reviewed like code, it’s basically a blueprint of what feature or new changes are we going to make. I was assigned the work item #957, which was basically about how the idea of implementing a debuginfod server functionality inside Debusine was initially proposed by a fellow member which later became a project idea under GSoC 2026. My developer blueprint pinned down the four decisions everything else depends on: extraction happens on the worker after the build, symbols are stored as artifacts keyed by build-ID, they’re published into suites alongside their binaries, and they’re served from the archive root rather than per-suite. Settling that up front meant the design discussions happened in a document instead of across three merged branches.
One of those arguments became its own fix. My wording implied symbols were unpacked inside the isolated sbuild environment (the consequence was I was handed a bug to be solved in the first week of contribution period), when they’re actually extracted afterwards on the worker, where the build output already sits, a distinction that matters, because doing work inside the unshare environment means extra tooling in the chroot and more ways to affect the build. !3119 corrected it before the wrong model spread into the code.
A new artifact type
Artifacts are a major concept in Debusine overall, so as per the developer blueprint we introduced a new artifact which was debian:debug-symbols. It holds every .debug file from one -dbgsym package. Its data is a validated list of lowercase 40-character build-IDs, and each file is stored under its build-ID as the path, so answering “what are the symbols for this ID?” is a direct lookup, with no path translation in the request handler. One artifact per package rather than per file: a util-linux build would otherwise spray hundreds of artifacts, collection items and relations across the database for no benefit. For implementing debian:debug-symbols artifact, I changed the main models.py file, along with that since it’s a norm to write unit tests, all mentioned under !3088.
Publishing workflow and solving a bug
Extracting symbols is only useful if they reach the archive people actually
install from, so
!3180
taught package_publish to follow the relates-to relation: copying binaries
into a suite now brings their debug symbols along automatically, with
nothing extra for the publisher to configure. Each build-ID becomes its own
collection item, for example debugsym:hello_2.10-5_amd64_fcc9064… each
carrying the package name, version and architecture copied from the binary,
so the item is meaningful on its own without dereferencing anything.
Uniqueness is enforced at both the suite and archive level, because the
serving URLs are archive-wide and two suites must never disagree about what
a build-ID means: republishing an identical file is accepted quietly, while
two different files claiming the same ID is an error worth failing on. A
partial index on the build-ID keeps the eventual HTTP lookup fast.
That looked finished until symbols started arriving in target suites disconnected from their binaries published, but unfindable, because copying items between collections silently dropped their artifact relations, and that relation is the only thing tying the two together. The fix sat one level above my feature, in the generic CopyCollectionItems task that does the copying, and since it was reusable infrastructure rather than anything debuginfod-specific, Colin implemented it himself in !3228. My project needed it to work at all; every other Debusine feature that copies items now gets it for free.
Endpoint and CI tests
With symbols in the archive,
!3212
added the part users actually touch: GET /{scope}/{workspace}/buildid/<build-id>/debuginfo looks the ID up across
every suite in that workspace’s archive, streams the file, and sets the
X-DEBUGINFOD-FILE and X-DEBUGINFOD-SIZE headers the protocol expects. It
also handles the two things gdb actually does: a HEAD probe before
committing to a download, and ranged requests to pull individual ELF
sections instead of the whole file. Scoping it to the archive rather than
the suite is what lets one URL cover a whole workspace, so the developer
never has to know which suite their binary came from.
Every merge request above landed with unit tests, but those only tell you
that the pieces behave correctly. What Colin and I wanted was a real gdb
fetching real symbols from a real instance, so
!3261
adds an autopkgtest that builds a package, publishes it, checks the HTTP
headers, then sets DEBUGINFOD_URLS and makes gdb go and get the symbols,
wired into the CI integration tests so it runs on every change. It took me a
day to learn that skipping the signing worker doesn’t simplify that test, it
just hangs until the 30-minute timeout, because update_suites needs signing
to produce a usable repository.
The last piece,
!3301
covers the new artifact, the suite and archive changes, the new archive URL,
and a how-to for using it. My first how-to draft explained how everything
worked and offered four ways to set DEBUGINFOD_URLS; the version that
shipped gives one recommended setup and gets out of the way. The same pass
trimmed the blueprint down to only what’s still unimplemented, since a
design document describing merged code is just an obstacle for the next
reader.
What’s left
Only one item on my original plan didn’t land: an archive-level
build_debug_symbols switch, modelled on Launchpad’s equivalent, letting an
archive skip building -dbgsym packages entirely by passing
DEB_BUILD_OPTIONS=noautodbgsym to sbuild. It was always the stretch goal
rather than core scope, landing the extract-publish-serve path solidly
mattered more than landing it broadly. The design is written up in the
blueprint, and I intend to implement it myself.
The other gaps were deliberately out of scope from the start, and the blueprint says so. DWZ supplement files aren’t ingested, so packages using compressed debug info may render without the alternate strings table; debugging still works, it’s just less complete. Source-file serving runs into the same Debian packaging limits that constrain debuginfod.debian.net today, making it a design question rather than a coding one. Executable serving, the metrics and metadata endpoints, and federation to upstream debuginfod servers were excluded for similar reasons, none of them are needed for Debusine’s core use case, and each would have crowded out the parts that are.
One open bug is left too. On the last day of the coding period, Stefano Rivera found that publishing ledger and linux was failing, because I had told the database that a build-ID identifies one exact debug file which isn’t true in Debian, since dh_dwz runs once per binary package, so when one object ships in two binary packages their .debug files differ while describing identical code. How to fix it is still an open discussion #1582, though it may not land before the formal end of the project.
None of that is a handoff. GSoC’s timeline is ending, my involvement isn’t, I’m carrying on with Debusine until both the build_debug_symbols switch and DWZ supplement support are merged, and I expect to keep contributing beyond that. This project got me familiar with a codebase I enjoy working in, and the remaining pieces are mine to finish.
Thanks!
The biggest thanks go to my mentor, Colin Watson, whose reviews consistently found the thing I hadn’t thought about. He also gave me room to get things wrong first and understand why, which taught me more than being handed the answer would have.
Thanks as well to Raphaël Hertzog, Enrico Zini, Stefano Rivera, Carles Pina i Estany and Helmut Grohne and everyone else around Debusine and Freexian for reviews, comments and patience with my questions.
Special thanks to Freexian for developing Debusine in the open and for giving me access to test on debusine.debian.net.
Finally, thanks to the wider Debian community, whose build-ID and -dbgsym conventions did most of the hard work before I arrived and to Google Summer of Code for providing a platform and the time to do this properly.