jmtd → Jonathan Dowland's Weblog
Below are the five most recent posts in my weblog. You can also see a chronological list of all posts, dating back to 1999.
Sometimes I have to pore over long debugging logs which have originally been written out to a terminal and marked up with colour or formatting via ANSI escape codes. The formatting definitely makes reading them easier, but I want to read them in Vim, rather than a terminal, and (out of the box) Vim doesn't render the formatting.
Cue AnsiEsc.vim: an OG Vim script1 that translates some ANSI escape codes — in particular some colour specifying ones — into Vim syntax highlighting.
This makes viewing and navigating around multi-MiB console log files much nicer.
-
AnsiEscis old enough to have been distributed as a script, and then as a "vimball", an invention of the same author to make installing Vim scripts easier. It pre-dates the current fashion for plugins, but someone else has updated it and repackaged it as a plugin. I haven't tried that out.↩
This week I stumbled across a footgun in the Dockerfile/Containerfile ARG
instruction.
ARG is used to define a build-time variable, possibly with a default value
embedded in the Dockerfile, which can be overridden at build-time (by passing
--build-arg). The value of a variable FOO is interpolated into any following
instructions that include the token $FOO.
This behaves a little similar to the existing instruction ENV, which, for
RUN instructions at least, can also be interpolated, but can't (I don't
think) be set at build time, and bleeds through to the resulting image
metadata.
ENV has been around longer, and the documentation indicates that, when both
are present, ENV takes precedence. This fits with my mental model of how
things should work, but, what the documentation does not make clear is, the
ENV doesn't need to have been defined in the same Dockerfile: environment
variables inherited from the base image also override ARGs.
To me this is unexpected and far less sensible: in effect, if you are building
a layered image and want to use ARG, you have to be fairly sure that your
base image doesn't define an ENV of the same name, either now or in the
future, unless you're happy for their value to take precedence.
In our case, we broke a downstream build process by defining a new environment
variable USER in our image.
To defend against the unexpected, I'd recommend using somewhat unique ARG
names: perhaps prefix something unusual and unlikely to be shadowed. Or don't
use ARG at all, and push that kind of logic up the stack to a Dockerfile
pre-processor like CeKit.
Last year I put together a Halloween playlist and tried, where possible, to link to the tracks on Bandcamp. At the time, Bandcamp did not offer a Playlist feature; they've since added one, but it's limited to mobile only (and I've found, not a lot of use there either).
(I also provided a Spotify playlist. Not all of the tracks in the playlist were available on Spotify, or Bandcamp.)
Since then I discovered an independent service bndcmpr which lets you build and share playlists from tracks hosted on Bandcamp.
I'm not sure whether it will have longevity, but for now at least, I've ported my Halloween 2022 playlist over to bndcmpr. You can find it here: https://bndcmpr.co/cae6814a, or with any luck, embedded below (if you are reading this post on an aggregation site, or newsreader, it's less likely that this will appear):
I'm overdue cutting the next playlist, theme TBA. Stay tuned.
I've finally landed a patch/feature for HLedger I've been working on-and-off (mostly off) since around March.
HLedger has a powerful CSV importer which you configure with a set of rules. Rules consist of conditional matchers (does field X in this CSV row match this regular expression?) and field assignments (set the resulting transaction's account to Y).
motivating problem 1
Here's an example of one of my rules for handling credit card repayments. This rule is applied when I import a CSV for my current account, which pays the credit card:
if AMERICAN EXPRESS
account2 liabilities:amex
This results in a ledger entry like the following
2023-10-31 AMERICAN EXPRESS
assets:current £- 6.66
liabilities:amex £ 6.66
My current account statements cover calendar months. My credit card period spans mid-month to mid-month. I pay it off by direct debit, which comes out after the credit card period, towards the very end of the calendar month. That transaction falls roughly halfway through the next credit card period.
On my credit card statements, that repayment is "warped" to the start of the list of transactions, clearing the outstanding balance from the previous period.
When I import my credit card data to HLedger, I want to compare the result against a PDF statement to make sure my ledger matches reality. The repayment "warping" makes this awkward, because it means the balance for roughly half the new transactions (those that fall before the real-date of the repayment) don't match up.
motivating problem 2
I start new ledger files each year. I need to import the closing balances from the previous year to the next, which I do by exporting the final balance from the previous year in CSV and importing that into the new ledgers in the usual way.
Between 2022 and 2023 I changed the scheme I use for account names so I need to translate between the old and the new in the opening balances. I couldn't think of a way of achieving this in the import rules (besides writing a bespoke rule for every possible old account name) so I abused another HLedger feature instead, HLedger aliases. For example I added this alias in my family ledger file for 2023
alias /^family:(.*)/ = \1
These are ugly and I'd prefer to get rid of them.
regex match groups
A common feature of regular expressions is defining match groups which can be referenced elsewhere, such as on the far-side of a substitution. I added match group support to HLedger's field assignments.
addressing date warping
Here's an updated version rule from the first motivating problem:
if AMERICAN EXPRESS
& %date (..)/(..)/(....)
account2 liabilities:amex
comment2 date:\3-\2-16
We now match on on extra date field, and surround the day/month/year components with parentheses to define match groups. We add a second field assignment too, setting the second posting's "comment" field to a string which, once the match groups are interpolated, instructs HLedger to do date warping (I wrote about this in date warping in HLedger)
The new transaction looks like this:
2023-10-31 AMERICAN EXPRESS
assets:current £- 6.66
liabilities:amex £ 6.66 ; date:2023-10-16
getting rid of aliases
In the second problem, I can strip off the unwanted account name prefixes at CSV import time, with rules like this
if %account2 ^family:(.*)$
account2 \1
When!
This stuff landed a week ago in early November, and is not yet in a Hledger release.
I haven't done one of these in a while!
Denver Luna is the latest single from Underworld, here on a pink 12" vinyl. The notable thing about this release was it was preceded by an "acapella" mix, consisting of just Karl Hyde's vocals: albeit treated and layered. Personally I prefer the "main" single mix, which calls back to their biggest hits. The vinyl also features an instrumental take, which is currently unavailable in any other formats.
The previous single (presumably both from a forthcoming album) was and the colour red.
In this crazy world we live in, this was limited to 1,000 copies. Flippers have sold 4 on eBay already, at between £ 55 and £ 75.
Older posts are available on the all posts page.