jmtd → drafts
The following posts are drafts whilst I work on the precise settings for this blog.
17 drafts.
A few years ago I learned about IndieWeb — an initiative for people to control their online identities and content by self-hosting it on their own domain and syndicating it out to "silos" like Twitter, Facebook etc., retaining ownership and control.
They advocate adding some small amount of markup and attributes to your site in order for some data to be more easily consumed by scripts and programs. Adding this to my site looked like it would be very quick and easy, so I thought I'd give it a go.
h-card
The first suggested metadata to add is an "h-card": effectively a digital equivalent of a business card. I already have the data at about, so it was just a matter of wrapping that data in some additional tags, or adding additional attributes to existing tags.
My main objection to what I've read about h-card is there's a strong lean to put it at the root of your site. I've not done that, and I don't particularly want to do it: I think it would make the front page look cluttered.
h-card - https://indiewebify.me/validate-h-card/
- can I have some kind of forward pointer from / to /about and put the h-card stuff all there?
u-photo on the img... except it's not inside an h-card thing yet
h-entry
The second suggestion is to mark up "entries" on your site with metadata to make them "h-entries". This seems very close to what RSS/Atom syndication do already: my atom feed already has author, entry permalink and publication date metadata.
None-the-less I looked into what is needed to add this metadata to the HTML
version of blog posts. The h-entry spec requires each entry to have a permanent,
canonical URI for itself in the content of the page. Surprisingly, IkiWiki
didn't already do this. I had to write a (trivial) plugin to add a template
variable that can be included into pages.
h-entry wants you to include the publication date for entries, as content,
and stipulates the format that the date must be represented with: ISO8601.
I already have the HTML "date" tag for entries, e.g.
<time datetime="2020-04-03T15:44:01Z" pubdate="pubdate">
Fri Apr 3 15:44:01 2020
</time>
Which happens to use ISO8601 for the datetime attribute already, but is flexible enough for me to include any representation I like for humans.
stop generating pubdate=pubdate (invalid HTML5) and instead apply class dt-published
add to page.tmpl, .e-content, .p-name, u-url (permalink plugin), dt-published (above patch)
https://indiewebify.me/validate-h-entry/
web3
You might have heard about a recent initiative/branding exercise from the crypto-currency world called "web3". What web3 is, or claims to be, is not exactly clear (except being part of an ongoing enormous-scale, environmentally destructive fraud).
- ?Discussion
LANG=C systemctl list-units --failed --plain 2>/dev/null | grep failed
less than ideal but "works"
- ?Discussion
One of the big selling points of Podman is its support for rootless containers: there are a few arguments made for this but the most significant appeal (IMHO) is that a developer can easily start up a container as a non-privileged user without needing an arrangement like Docker's persistent daemon, with a socket the non-root user can write to, opening up risks of privilege escalation.
For my use-case I don't think there are many advantages to rootless containers, and there may be drawbacks:
- user namespacing and isolation between independent rootless containers
- network arrangement
- can they connect to a bridge netwrok?
- slirp4netns: user mode TCP/IP network stack, TAP network device connected to it;
- perforamnce?
- benchmarks: https://github.com/rootless-containers/slirp4netns#iperf3-netns---host 2018,
fuse-overlayfs (usermode )
a non-privileged user can start up a container and the container runtime is never privileged. However for my use-case,
launch containers as root (not using rootless)
adv/dis
need slipr4netns to provide user-mode networking (unprivilegd network namespaces)
not comparing to native, CI apparently runs a benchmark job but I can't see it
- ?Discussion
As part of refreshing my approach to things, I'd been re-reading "getting things done", a book I first read in around 2004.
Since I first read it I've got quite good at some of the front-end suggestions: gathering up TODO items, organising them by context ("stuff I can do at home", "stuff I can only do in Newcastle", etc), and thinking in terms of "next action". However, I can still find that lots of TODO items languish for a while.
I eventually hit on the idea of using a Kanban board - in my case Trello - as a personal system for assigning some tasks to days. I have 1 column per weekday plus a "backlog" column. The top task on each weekday is a coloured/themed card to indicate where I will be working (Home or Town). There are a couple of other cards that live permanently in their columns, with a unicode "pin" in the task name (e.g. "decide on next week's town/home days").
Other than that, I have a smattering of TODO items, one per card, that I assign to days of the week. If a particular day gets a lot of cards on it, it becomes clear I've probably picked too much stuff to try and do on that day. I move cards from days and clear the week's task cards to Backlog at the end of Friday.
This seems to work for me. I feel like I'm getting more things off my lists and fewer TODOs are languishing indefinitely. I'm effectively I'm using Trello like a Calendar.
What was interesting to me on re-reading GTD is that Allen advises against doing exactly this. He argues that the Calendar should accurately represent the "hard" edges of your work commitments. I think he is somewhat optimised towards people who have a lot more real appointments and hard deadlines in their day-to-day calendars than I have.
I'd like to get off Trello, if possible. It's not ideal. the app has some offline support but it's not perfect so sometimes, when I'm away from the Internet, I am am prevented from accessing or updating my plan; Sometimes the App and/or website are unavailable; occasionally I need to do the login dance again which is time consuming and distracting from whatever I opened the App to achieve. On the other hand, it is very finely-tuned to be a pleasure to use with a simple and attractive UI and some reasonable keyboard shortcuts for speedily filing things.
Trello is not the "top copy" of any of my TODO items either, so I've duplicated (a subset of) them into Trello, and so there's the mental or time cost of managing that duplication.
I've looked at NextCloud Deck, but it's too bare bones (or at least was when I looked at it), lacking the ability to colour or theme cards at all. There is a Microsoft Planner, and I have a fully-fledged Microsoft Office 365 account via the University, so that's an option. There are many other such apps.
I did wonder whether, since I was using it as a Calendar anyway, I could use a Calendar app for the purpose. I use Google Calendar (on web/desktop) for work, and increasingly for home; I access it via iOS Calendar on phone (not the Google Calendar App). The UI flow for iOS Calendar, at least, is totally unsuited to mimic what I am doing with Trello.
The Google Calendar Web UI is also too cluttered to realistically do this either.
This lead me to look at a few other Calendar apps.
- ?Discussion
Now it's easy for machines to interact with locally-running containers, I can more easily try out lots of web-apps and see if they're useful for me or my family.
First up, Navidrome.
before streaming music
I was a hold-out on carrying around my music collection on a portable music player for a long time, most recently with a hacked ipod, but before that a series of Sandisk Sansa devices, an early ipod clone, and of course before that minidiscs and walkmans.
I fell out of the habit of using the iPod for two reasons: the move to wireless headphones for my phone (forced by the removal of the phono socket from modern phones) added a bit of friction, but the lack of an FM tuner in the iPod meant switching devices to flick between music and the Radio.
The second reason was during the Pandemic I didn't need to use a portable device at all, as I didn't leave the house. (I at least spent some time with my old vinyl collection)
post-Pandemic
Once we pretended the Pandemic was over and started going out again, I didn't pick up the iPod, and either exclusively listened to the radio, or a limited amount of music locally on my phone, or occasionally used a streaming service like Spotify.
bandcamp...
Navidrome, Subsonic API
Navidrome is...
iOS clients
There are several iOS clients, at least two of which are free and open source: Amperfyand Substreamer.
Whilst my Navidrome instance is available at home only, the apps work fine in that environment, and I can choose to download a set of tracks, which then play fine when I'm out and the app can't talk to the server.
Depending on where I get to with WireGuard, I might move to having the Navidrome server available out and about, too.
- ?Discussion
alesis micron - stereo out, DIN MIDI
SB Extigy: midi in/out spdif in mic in lin in?
soft synths: using the micron asa controller
software to drive the micron Vhttps://ctrlr.org/akai-miniak-alesis-micron/ what about hte little things
Contact cleaner WITH LUBE and your volume pot will be as good as new!
Normal cleaner without lube will work for a week,or so, but with lube will last indefinitely. Tried on 2 Miniaks and both perfect over a year later. Need to take off top of case to access pot,spray liberally inside pot!
Every Miniak has volume problem and the fix is so simple it should have been done by Akai. https://github.com/retroware/micronau
bastl kastl monotron delay KO-33 sampler
iOS! samplers
- ?Discussion
Last time I wrote about Interzone, I was discussing issue 294, the first published under the new editor, in the new, paperback-sized format ("JB6"). The format and presentation of the magazine was fantastic: it fit in a lot of my pockets, and was packed with 15 stories as well as the regular columns, in full colour with fantastic layouts and illustrations. Sadly there was only one more physical issue before Interzone was forced to become a digital-only publication.
I don't want to dwell on the sad necessity to move to digital. Interzone continues on, recently releasing the milestone issue #300. They now seem keen that subscriptions are via Patreon.
Instead I wanted to write a small bit about how I engaged with the paper magazine.
- ?Discussion
When I was growing up, magazines were a significant part of my recreation and education. We didn't have the Internet, or the World Wide Web; we did have libraries, but for up-to-the-minute information, magazines were best.
In the 90s those magazines included Amiga Format, and later film-oriented magazines like SFX. It's where we learned about new software, games, computer hardware developments; as well as TV shows, movies, and novels.
Magazines haven't exactly died out: this is evident if you have a local WH Smiths. Some of them seem to be enjoying a renaissance, largely by riding on a large wave of nostalgia. But they aren't fulfilling the role they used to, that's been completely usurped by the Web.
After writing about Interzone's new publisher and the lovely new perfect-bound, novel-sized format it was printed in, IZ squeezed out one last physical issue before becoming digital-only.
The later TTA press-era Interzones were approximately A5 sized and were posted out with delivery slip of paper, which was blank on the reverse. The slip was slightly smaller than the issue. I began a habit of putting the slip inside the cover and writing on it to mark which stories I had read in that particular issue, and whether I really enjoyed them or not. That physical habit became part of the experience of reading the magazine.
- ?Discussion
Podman (an alternative to Docker) supports "rootless containers" since around 2019.
what are rootless containers?
- containers that created/run/managed by users w/o admin rights
- protect against engine/runtime/orchestrator compromise
drawbacks?
- slirp4netns (not needed?)
$HOME/.local/share/containers/storage vs /var/lib/containers
Questions
what does conmon do
do they use distinct user namespaces?
I'm running two user-namespace ikiwiki containers. what namespace do they use
├─conmon,2031027 --api-version 1 -c 3353fef59670f6f12de3cbb9094d009fe3cc78796376e28c5313013f777806fd -u... │ └─lighttpd,2031042 -Df /etc/lighttpd/lighttpd.conf ├─conmon,2031121 --api-version 1 -c 55abef9846fb0130ef094c6fe4012317816b4b251d1099dc06c3db2d066bdd23 -u... │ └─lighttpd,2031135 -Df /etc/lighttpd/lighttpd.conf
they seem to be mapped to the same uuid outside
100999 2031042 2031027 0 14:26 ? 00:00:00 /usr/sbin/lighttpd -Df /etc/lighttpd/lighttpd.conf 100999 2031135 2031121 0 14:26 ? 00:00:00 /usr/sbin/lighttpd -Df /etc/lighttpd/lighttpd.conf
the conmon instances are running as uid=jon
containers I am running:
container-calibreweb.service all running as root container-freshrss.service runs apache, cron, sensitive-log.sh as root container-graphite.service whole thing running as root container-navidrome.service
container-nextcloud.service container-paperless.service container-privwiki.service container-redis.service
does podman use a uidmap by default? Can I inspect it?
graphite: USER HUSER COMMAND root root entrypoint root root runsvdir root root runsv root root runsv root root runsv root root runsv root root runsv root root runsv root root runsv root root runsv root root runsv root root runsv root root runsv root root nginx root root crond root root gunicorn root root tee root root tee root root tee root root python3 root root statsd /opt/sta root root tee root root python3 nginx systemd-network nginx nginx systemd-network nginx nginx systemd-network nginx nginx systemd-network nginx root root gunicorn root root gunicorn root root gunicorn root root gunicorn navidrome: USER HUSER COMMAND 1000 jon navidrome freshrss: USER HUSER COMMAND root root apache2 root root cron root root sensitive-log.s root root sed www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 redis: USER HUSER COMMAND redis systemd-coredump redis-server privwiki: USER HUSER COMMAND ikiwiki jon lighttpd paperless: USER HUSER COMMAND root root supervisord paperless jon gunicorn: maste paperless jon [celeryd: celer paperless jon [celery beat] - paperless jon python3 paperless jon gunicorn: worke paperless jon [celeryd: celer calibreweb: USER HUSER COMMAND root root s6-svscan root root s6-supervise root root s6-linux-init-s root root s6-supervise root root s6-supervise root root s6-supervise root root s6-supervise root root s6-ipcserverd abc 911 python3 root root bash root root sleep nextcloud: USER HUSER COMMAND root root apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2 www-data www-data apache2
no uidmapping argh
simplifying my investigations down to uid mapping and the like.
sudo podman top freshrss user,pid,huser,hpid USER PID HUSER HPID root 1 root 3632
sudo ls -l /proc/3632/ns
sudo lsns -t user
NS TYPE NPROCS PID USER COMMAND
4026531837 user 272 1 root /sbin/init
4026532989 user 1 223410 uuidd └─/usr/sbin/uuidd --socket-activation
there's only two user namespaces on the system, so podman is not using them yet.
what's uuidd? a local daemon, systemd has PrivateUsers=yes hence the namespace!
which container to experiment on? either privwiki or paperless, neither important
privwiki /var/lib/containers/storage/volumes/privwiki/_data
if I use --userns=auto, the container thinks its running as uid=1000 (can't be sure what is actually chosen) but it cannot read the volume (mapped to nobody)
using a fresh volume, it gets initialised to a big uid (2147484647), but is this consistent? can I rely on it?
second problem: /dev/stderr not writeable without -t, so lighttpd dies
== 2025-05-29 ==
the /dev/stderr thing, that's weird. is it a phobos specific issue? can I recreate it?
there's no root entry in /etc/subuid, but there is (now) a containers entry containers:2147483647:2147483648
podman run -d --replace --name privwiki --userns=auto quay.io/jdowland/opinionated-ikiwiki:latest bash -c 'date > /dev/stderr'
privwiki 9b610a64e032207eed690f18dd4ce59546fe8d810c694ccdd0f74ccdfb1225d0 root@phobos:~# podman logs 9b610a bash: line 1: /dev/stderr: Permission denied
reproduced locally:
- podman run -d --replace --name date --userns=auto date bash -c 'id -u; stat /proc/self/fd/2; date >> /proc/self/fd/2'
- 2e6fc313bb49d19222e45447a561942e79f1288baaecc50b765e562838351a40
- carbyne ▶;podman logs 2e6fc
- 1000
- File: /proc/self/fd/2 -> pipe:[24066627]
- Size: 64 Blocks: 0 IO Block: 1024 symbolic link
- Device: 0,111 Inode: 24070420 Links: 1
- Access: (0300/l-wx------) Uid: ( 1000/ jon) Gid: ( 0/ root)
- Access: 2025-05-29 13:47:31.763178125 +0000
- Modify: 2025-05-29 13:47:31.763178125 +0000
- Change: 2025-05-29 13:47:31.763178125 +0000
- Birth: -
- bash: line 1: /proc/self/fd/2: Permission denied
- carbyne ▶;podman run -d --replace --name date --userns=auto date bash -c 'id -u; stat /proc/self/fd/2; date >&2'
- 617daffb70ec0e4e1f314407530e72723f448d1de9fd7781b4013da545a80599
- carbyne ▶;podman logs 617daff 1000 File: /proc/self/fd/2 -> pipe:[24011709] Size: 64 Blocks: 0 IO Block: 1024 symbolic link Device: 0,110 Inode: 24071393 Links: 1 Access: (0300/l-wx------) Uid: ( 1000/ jon) Gid: ( 0/ root) Access: 2025-05-29 13:47:55.659164875 +0000 Modify: 2025-05-29 13:47:55.659164875 +0000 Change: 2025-05-29 13:47:55.659164875 +0000 Birth: - Thu May 29 13:47:55 UTC 2025
== 2025-05-30 ==
should a httpd container log to stderr/stdout by default?
apache: (SIGWINCH!) ErrorLog /proc/self/fd/2 CustomLog /proc/self/fd/1 common
yet seems to work as user with default usermapping stuff
;podman top apache huser user hpid pid comm HUSER USER HPID PID COMMAND 1000 root 2174688 1 httpd 100032 www-data 2174702 8 httpd 100032 www-data 2174703 9 httpd 100032 www-data 2174705 11 httpd
refuses to start with --userns=auto. with userns=host:
podman top apache huser user hpid pid comm HUSER USER HPID PID COMMAND 1000 root 2176081 1 httpd 100032 www-data 2176094 8 httpd 100032 www-data 2176095 9 httpd 100032 www-data 2176097 11 httpd 100999 1000 2176355 92 bash
how to interpret this: container user 0 is mapped to host user 1000, so the root process has my user privileges; my uid 1000 in container context is mapped to something 'safe'
is the issue mapping the default container-user (e.g. root) to not-host-me, so it hasn't permissions on /proc/dev/fd/2?
lighttpd:
== 2025-05-30 ==
need to take a step back. Given quadlet is around the corner, does it invoke containers as root? does it do user mapping?
I want to invoke as root (unless quadlet advises against) so stick to testing uid stuff invoked as root
podman quadlet doesn't do any --userns stuff by default you can put UserNS=, User= etc. into the .container (service) file
== 2025-05-30 ==
OK so root-invoked podman, userns=auto, lighttpd breaks: perms oin /dev/setderr. let's try and debug that
- ?Discussion
[[! meta description="""
"""]]
Next up, user namespaces.
When you run a container, you are running a process, with an altered execution context. One of the alterations can be the user namespace.
OpenShift advice: random UID, unpredictable, make relevant files group root and g+w.
is uid mapping better?
- ?Discussion
On Monday I flew to Paris for one night to see Nine Inch Nails. I'd missed all their UK tour dates due to a clash with our family holiday. New for this tour, they've structured the show around four separate acts, two of which (a stripped-down, quiet section and an EDM-flavoured remix one, featuring Dutch DJ Boyz Noize) take place on a "B stage" separate from the main one, where the remaining two conventional sets take place.
Last week I received my official PhD corrections list, which is not too long, so I used the travel time to get started on that.
It's the end of the school terms next week. One of my kids is moving to middle school next academic year (we're in a three-tier system) which is a new challenge and adventure for her. She's very anxious in general and specifically about the transition so we've been doing a lot to help her prepare for that.
Last week I officially started at IBM. On-boarding is going OK, a few hiccups but nothing significant (for me at least) and crucially my access to relevant Red Hat systems hasn't broken so I've been able to continue working. It's been a busy couple of weeks for container respins, with a new CVE to address almost every day. I'm taking another look at removing Python from our containers to see if we can reduce the attack surface area and corresponding engineering cost.
This is not how I'd usually write a blog post, but, I haven't been writing them much recently so I thought I'd try something different.
- ?Discussion
FOSDEM 2026 was great! The video of my talk is up, as are my slides with speaker notes and links.
Dangerzone, a tool for viewing PDFs in a sandboxed environment
RaySession, a "session manager" for audio workflows.
PAW, a programmable Digital Audio Workstation (DAW)
- ?Discussion
This Christmas I was gifted a PiStorm, to fit in my Amiga A500. The PiStorm is an adaptor board that plugs into the Amiga mainboard in place of the original M68K CPU. You then plug a Raspberry PI into the other side of the PiStorm board, via the GPIO pins. The Emu68 software runs on the Pi and soft-emulates the M68k, considerably faster than the original chip. I wrote about the PiStorm, Emu68 last August.
Pistorm Xmas, Fosdem 2025
Current status: kickstart 1.3, no hdd, gotek
Gotek wiring problem
Pistorm in practise
Next steps
Denise
- ?Discussion
One of my two external backup drives died, so I had to buy and commission a new one. Here's the steps I had to follow.
https://jmtd.net/log/systemd_ext_backups/
prep the drive
Format the new drive as a LUKS encrypted volume. Set a passphrase as the decryption key for now. I used
luksformatbefore forgetting it defaults to putting a FAT32 filesystem on the device.Format the new device as an
ext4filesystem. My older drive was usingxfs, in common with all my backup devices, because at one point my backup system was consuming files from Macs andxfshad a large enough space for extended attributes. I've never had a problem withxfsbut I don't have that requirement any more, and I'm more experienced managingext4filesystems.Make sure the volume label matches the old drive (
extbackup).Add a second decryption key to the device (a file that lives on the internal drives of my NAS). This means it can be mounted non-interactively when I plug it in.
initial sync and randomizing the drive
Manually mount the filesystem and start an initial sync from my internal backup drive. This is going to take a long time.
Once it's done, fill up the rest of the space on the drive with a file full of zeroes. This causes the whole drive to be written to, ensuring the underlying device doesn't have identifiable regions that are allocated or not. Once the drive is full, the all-zero file can be removed.
Reconfigure systemd
Systemd is used to initiate the backup when the drive is plugged in.
Start re-configuring Systemd. My
backup-exthdd.serviceis configure to beWantedBya device corresponding to the failed drive. Update the device name to have the UUID of the new drive.My backup service depends upon triggering the drive decryption. This is done by having
RequiresandAfterrelationships on a decryption service,systemd-cryptsetup@extbackup.service. As the name implies, this was once created automatically by the systemd generator that reads/etc/crypttab. However it is now managed manually. The main difference is I addStopWhenUnneeded=true. I could probably do that with an override file, instead of having a manually-managed.servicefile.
- ?Discussion
