jmtd → log → hledger → Separate hledgers
In a previous blog post I described the use of virtual postings to track accidental personal/family expenses. I've always been uncomfortable with that, and in hledger 1yr I outlined a potential scheme for finally addressing the virtual posting problem.
My outline built on top of continuing to maintain both personal and family financial data in the same place, but I've decided that this can't work, because the different "directions" (or signs) of accidental transactions originating from either the family or personal side can't be addressed with any kind of alternate view on the same data.
To illustrate with an example.
A negative balance in
family:liabilities:jon means "family owes jon". A
coffee bought by mistake on the family credit card will have a negative
posting on the credit card, and thus a positive one on the liabilities
account. ("jon owes family"). That's fine.
But what about when I buy family stuff on a personal card? The other side of
of the transaction is also going to have a positive sign, so it can't be
family:liabilities:jon: it would have to go to somewhere else,
jon:liabilities:family. Now I have two accounts which track versions
of the same thing, and they cannot be combined with a simple transaction
since they're looking at the same value from opposite directions (and signs).
Back when I first described the problem I was using a single journal file for all my transactions. After moving to lots of separate journal files (in hledger 1yr), it's become clearer to me that I don't need to maintain the Family and Personal data together, at all: they can be entirely separate journals.
getting data between journals
When I moved to a new set of ledger files for 2023, I needed to carry forward the balances from 2022 in the form of "opening balance" transactions. This was achieved by a report on the 2022 data, exported as CSV, and imported into the 2023 data (all following the scheme outlined by fully-fledged hledger.))
Separate Personal and Family journals need some information from each other, and I can achieve that in the same way as for opening balances: with an export of the relevant transactions as CSV, then imported on the other side. HLedger's CSV import system is flexible enough that we can effectively invert the sign of liabilities, addressing the problem above.
We start with an accidental coffee purchased on the family card (and so this belongs to the Family ledger)
2022-08-20 coffee liabilities:creditcard £ -3 liabilities:jon:expenses:coffee £ 3
I've encoded the expense category that the Personal ledger will be interested
in (the last bit,
expenses:coffee) as a sub-account of the liabilities
category that the Family ledger is interested in1 (the first bit,
liabilities:jon). When viewed on the Family side, the expense category is not
interesting, and we can hide it with HLedger's alias feature2:
alias /^liabilities:jon(.*)$/ = liabilities:jon
It then looks like this from the Family side:
2022-08-20 coffee liabilities:creditcard £ -3 liabilities:jon £ 3
This transaction (and others like it) are exported via
hledger reg -f family/2023-back.journal liabilities:jon: -O csv \ jon/import/family/liabilities.csv
(The trailing colon on
liabilities:jon: is important here!)
In the resulting CSV file, the running example transaction looks like
"55","2022-08-20","","coffee","liabilities:jon:expenses:coffee","£ 3.00","£ 3.00"
This is then converted into a journal file by
hledger import. The rules file
for the import is very simple: the fields
amount are taken as-is;
account2 is hard-coded to
The resulting transaction looks like
2022-08-20 coffee liabilities:jon:expenses:coffee £ 3 liabilities:family £ -3
Before this journal is included by the main one, we have to adjust the expense
account, to remove the
liabilities:jon: prefix. The import rules can't do
this3 , so we use another journal file as a go-between with another alias
alias /^liabilities.jon:/ =
This results, finally, in the following transaction in the Personal ledger:
2022-08-20 coffee expenses:coffee £ 3 liabilities:family £ -3
There's one set of transactions that we don't want to export across this divide,
and that's because they're already there: any time I transfer money from myself
to the family accounts (or vice versa) to address the accrued debt, the transaction
is visible from both my family and personal statements. To avoid exporting these
and double-counting them, I make sure those transactions don't post to an account
matching the pattern used in the
hledger reg report. That's what the trailing
colon is for: It ensures I only export transactions which are to a sub-account
liabilities:jon, and not to the root account
which is where I put the repayment transactions. I could instead use a more
explicit sub-account like
liabilities:jon:repayments or similar, since the
trailing colon is quite subtle, but this works for me.
I've been really on the fence as to whether the complexity of this scheme is worth it to avoid the virtual postings. The previous scheme was much simpler. I have definitely made some mistakes with it, which didn't get caught by the double-entry rules that virtual postings ignore, but they're for small sums of money anyway.
On the other hand, a lot of the "machinery" of this already existed for getting
opening balances between calendar years, and the gory details are written down
and hidden inside the
Makefile. I also expect that I will continue to see
advantages in having Family and Personal entirely separate, as they can each
develop and adapt to their own needs without having to consider the other side
of things every time.
It's a running experiment, and time will tell if it's a good idea.
- This scheme was originally suggested to me by Pranesh on Twitter (described in dues), but I discounted it at the time because of the exact arrangement they suggested, not realising the broader idea might work.↩
I've hand-waved one problem with using hledger aliases here. If
we use them as described, to hide the Personal expense details, we need
them to not be applied when performing the CSV-generating report.
Therefore, in practise I have them in a front-most
family/2023.journalfile, which imports the data from another
family/2023-back.journal, and the CSV export is performed on the backing journal with the data and not the alias.↩
- HLedger import rules can't manipulate the fields from the CSV a great deal, but one change I proposed and started hacking on would allow for this: to expose Regexp match-groups as interpolatable tokens: https://github.com/simonmichael/hledger/issues/2009.↩