2025-12-30 10:35:30
That was fun!
Gentoo, with all tweaks and stuff (zram, latest kernel etc): 5 hours and 25 minutes 🐮 Fully working Cosmic desktop, flathub enabled.
@… : I'm back 👻
#gentoo
That was fun!
Gentoo, with all tweaks and stuff (zram, latest kernel etc): 5 hours and 25 minutes 🐮 Fully working Cosmic desktop, flathub enabled.
@… : I'm back 👻
#gentoo
New on my #Gentoo blog: One #jobserver to rule them all
"""
A common problem with running Gentoo builds is concurrency. Many packages include extensive build steps that are either fully serial, or cannot fully utilize the available CPU threads throughout. This problem becomes less pronounced when running building multiple packages in parallel, but then we are risking overscheduling for packages that do take advantage of parallel builds.
Fortunately, there are a few tools at our disposal that can improve the situation. Most recently, they were joined by two experimental system-wide jobservers: #guildmaster and #steve. In this post, I’d like to provide the background on them, and discuss the problems they are facing.
"""
https://blogs.gentoo.org/mgorny/2025/11/30/one-jobserver-to-rule-them-all/
Wow, #Gentoo moving from GitHub to Codeberg is cool. Haven't run Gentoo for years now but still have a soft spot for it (I learned so much running it as main driver).
I also use Codeberg für my code (and joined the association) but we can't just "move everything to Codeberg". That's neither sustainable nor a good model. We should have more associations like Codeberg t…
If I were to give a prize for the most disruptive workflow change of my life, it will be probably `less` changing home/end key behavior from scrolling to top/bottom (something which I do *all the time*) to disabling line wrapping and scrolling left/right (something I never do).
It's like XKCD#1172, except it's "we've decided spacebar heating is so cool, we've removed all other spacebar functions".
#Gentoo #Linux
RE: #GentooLinux
Worked on some more #Gentoo global #jobserver goodies today.
Firstly, Portage jobserver support patch: #PyTest jobs will also be counted towards total job count.
Again, it's not a perfect solution, but it works reasonably. The plugin still starts -n jobs as specified by the arguments, but it acquired job tokens prior to executing every test, therefore delaying actual testing until tokens are available. It doesn't seem to cause noticeable overhead either.
I'm building webkit-gtk right now. It's one of these messy packages where a few source files need a lot of memory to compile, and ninja can randomly order jobs so that all of them suddenly start compiling simultaneously. So to keep things going smoothly without OOM-ing, I've been dynamically adjusting the available job count via steve the #jobserver.
While doing that, I've noticed that ninja isn't taking new jobs immediately after I increased the job count. So I've started debugging steve, and couldn't find out anything wrong with it. Finally, I've looked into ninja and realized how lazy their code is.
So, there are two main approaches to acquiring job tokens. Either you do blocking reads, and therefore wait for a token to become available, or you use polling to get noticed when it becomes available. Ninja instead does non-blocking reads, and if there are no more tokens available… it waits till one of its own jobs finish.
This roughly means that as other processes release tokens, ninja won't take them until one of its own jobs finish. And if ninja didn't manage to acquire any job tokens to begin with, it is just running a single process via implicit slot, and that process finishing provides it with the only chance to acquire additional tokens. So realistically speaking, as long as there are other build jobs running in parallel, ninja is going to need to be incredibly lucky to ever get a job token, since all other processes will grab the available tokens immediately.
This isn't something that steve can fix.
#Gentoo #NinjaBuild
Installing ungoogled-chromium-bin on Gentoo turned out to be very straightforward. Just as adding the Librewolf repo is btw. Recently I've been moving all browser installs over to these two. Both projects have been around for a long time and consistently deliver quality.
#gentoo #librewolf
(Troll, but not really)
It's official: #CPython is planning to kill the #Gentoo WD40 profiles.
https://discuss.python.org/t/pre-pep-rust-for-cpython/104906
I hate 2026 already.
The reason:
IncorrectCopyright: version 6.30.2-r2: incorrect copyright year 2025: '# Copyright 2008-2025 Gentoo Authors'
#Gentoo
Did a reverse install of Gentoo today. Installed dwm alongside KDE, made sure all was running well. Then deinstalled all KDE parts, did a deep clean and switched profile to just desktop (from desktop-plasma). Smooth, and keeping all settings, applications and tweaks as they were. Easily beats a 4 hour reinstall 😅
#gentoo
Quick update on #Gentoo stuffs:
1. virtual/zlib and virtual/minizip are in, sys-libs/zlib-ng[compat] and sys-libs/minizip-ng[compat] are unmasked. Unfortunately, due to scale of this you have to do a --changed-deps rebuild to be able to switch. Or package.provided.
2. I've filed bugs for all the packages depending on app-crypt/gnupg. Should get us closer to app-alternatives/gpg being fully supported.
3. Started preparing to replace gentoo-mirror/ scripts with something much simpler. So I guess we're going to be removing most of the mirrors this week after all.
When you turn to dwm (as opposed to KDE, for a shorter compile) and forget Ghostty requires Zig 😜🤪
#gentoo
> As a long-time Arch user all I can say is that (at least in my experience) #Gentoo requires so little maintenance compared to Arch and if I have a problem there is a wiki page for everything (literally).
#ArchLinux is much more demanding. And now I hide…
#Gentoo #jobserver revealed another problem with steve in particular, and (I believe) the jobserver protocol in general: blocking clients are prioritized over polling clients.
The problem is simple: when handling blocking reads, steve can issue a job token immediately. When handling a poll, it merely indicates that a token is available, and the client must issue another read request to get it. So if tokens are scarce and there are both blocking and polling clients running, the former are likely to be taking all the incoming tokens.
My idea of working around this is to implement temporary reservations. If a client polls for a token, we reserve one for it. The reserved token can afterwards be only read by the same client. This way, both blocking and polling clients get a token — the former get it immediately, the latter get it reserved for them. And if there are no tokens available, both get into a single FIFO queue, for a poor man's round-robin (steve also throttles all reads to one token at a time).
However, polls technically don't guarantee that the client will eventually read the token, so we need to handle reservation expirations as well.
Inevitable moment:
f4e4ddaf69337 dev-build/steve: is no longer simple
#Gentoo
Yeah, so the GNU #make jobserver protocol is trivial, which can be a blessing and a curse. It puts the job management entirely on the clients, which means that they must reliably return job tokens, or otherwise the jobserver will be left with no jobs available and everything will hang. The make documentation is clear on this:
> Your tool should be sure to write back the tokens it read, even under error conditions. This includes not only errors in your tool but also outside influences such as interrupts (SIGINT), etc. You may want to install signal handlers to manage this write-back.
#NinjaBuild jobserver implementation may not handle this correctly, but fortunately it does. The irony is, it turns out that GNU make does not…
#Gentoo
#Steve the #Jobserver has undergone a major rewrite over the last week. It's now implemented using CUSE, the #FUSE API for character devices. It is using pidfd to track processes acquiring job tokens, and automatically reclaims them if processes die without returning them, preventing dead processes from effectively locking the system jobserver.
The code's still a bit ugly — it's a C-changed-midway-to-C , with libevent for event loops and (still) FUSE's ugly argument parsing.
If someone wants to play with it, the live ebuild is available in #Gentoo as dev-build/steve.
https://gitweb.gentoo.org/proj/steve.git/
BREAKING: #CPython 3.13.10 and 3.14.1 changed the multiprocessing message format in patch release. As a result, programs using multiprocessing may break randomly if they are running while #Python is upgraded (i.e. need restarting).
But apparently it's not a big deal, since all the cool kids are running Python in containers, and nobody is using Python for system tools anymore. Everything has been RIIR-ed and Python is only omnipresent in some backwaters like #Gentoo.
https://github.com/python/cpython/issues/142206
Yeah, why not neglect all the good recommendations in the #Python ecosystem, and instead fork your own C extension package, force people to build it with #ZigLang (it's still C), add unconditional dependency on that, and on top of that, refuse to publish wheels, "allowing for optimised compilation according to your machine's specific architecture and capabilities, instead of some (low performance) common denominator."
Fortunately, looks like #Gentoo can just ignore all the fancy crap and compile it with GCC.
https://pypi.org/project/ruamel.yaml.clibz/
[UPDATE: didn't last long: https://sourceforge.net/p/ruamel-yaml/code/ci/6846cf136e775ed0052cdef6ff02330269c86011/]
#Sphinx joined the list of packages dropping #Python 3.11 (and therefore #PyPy) support. Of course, we could just go through the effort of dropping it from respective packages in #Gentoo, given it's not technically that common… but honestly, at this point I have zero motivation to put the extra effort for this, just to learn that next month some core package starts requiring Python 3.12.
So, would anyone really mind if I removed Python 3.11 and PyPy support completely from Gentoo packages?
Why not switch your crappy software from crappy #SCons build system to even more crappy #Bazel build system over a patch release, call it "fully backwards compatible", and then effectively leave all the distros stuck on old versions with half a dozen vulnerabilities?
Yeah, just a random reminder that #MongoDB is total crap, and you shouldn't use it. Or expect people to package it for you.
#Gentoo
0 days since it turned out that GNU #make does not respect the #jobserver protocol it designed itself in yet another way. Or to put it otherwise, I've wasted my whole morning implemented something that cannot work because it didn't occur to me that GNU make people only set rules without caring to actually follow them.
#Gentoo #Linux
#TIL that #Gitolite can't handle repositories with different default branch names. As in, if you push a "main" branch into a "master" server, you get no HEAD 🤦. And you can only change that via SSH-ing to the server and modifying the underlying repository.
Apparently, you could also install a hook to automatically fix HEAD for you: #Gentoo #git
#SigStore / #PyPI attestations: #PGP is hard! We must invent a new signing scheme that's so much easier on users.
The tools, after I've spent hours *integrating* them into #Gentoo, and getting them working for everything before:
* Verifying google_auth-2.46.0.tar.gz ...
Provenance signed by a Google Cloud account, but no service account provided; use '--gcp-service-account'
Yeah, I'm sure that's *so much simpler* than PGP.
#security
Let's get this straight: it is entirely normal for a #OpenSource project to accumulate bug reports over time. They're not a thing to be ashamed of.
On the contrary, if you see a nontrivial project with a very small number of bug reports, it usually means one of the following:
a. you've hit a malicious fake,
b. the project is very young and it doesn't have many users (so it's likely buggy),
c. the project is actively shoving issues under the carpet.
None of that is a good sign. You don't want to use that (except for b., if you're ready to be the beta tester).
#FreeSoftware #Gentoo #GitHub #Python
How to absolutely *not* do #OpenSource: require people to commit to work on other issues with your project in order to file bugs. So, sorry, #Typer, I won't be filing bugs. You figure out how you messed up your release yourself.
Also, please don't use #FastAPI. They are clearly bothered by the fact that people dare use their projects and waste their precious time with support requests.
#Python #FreeSoftware #Gentoo
According to #FreePG right now: #ArchLinux, #Debian, #Fedora, #NixOS and #Ubuntu. Now #Gentoo joins that list, except that instead of silently making intrusive patching on top of GnuPG, we provide it as a separate package (app-crypt/freepg), and mark appropriately:
$ gpg --version
gpg (GnuPG) 2.5.13-freepg