Below are the five most recent posts in my weblog. You can also see a chronological list of all posts, dating back to 1999.

I don't generally write consumer reviews, here or elsewhere; but I have been so impressed by this one I wanted to mention it.

For Holly's birthday this year, taking place under Lockdown, we decided to buy a year's subscription to "Disney+". Our current TV receiver (A Humax Freesat box) doesn't support it so I needed to find some other way to get it onto the TV.

After a short bit of research, I bought the "Roku Express" streaming media player. This is the most basic streamer that Roku make, bottom of their range. For a little bit more money you can get a model which supports 4K (although my TV obviously doesn't: it, and the basic Roku, top out at 1080p) and a bit more gets you a "stick" form-factor and a Bluetooth remote (rather than line-of-sight IR).

I paid £20 for the most basic model and it Just Works. The receiver is very small but sits comfortably next to my satellite receiver-box. I don't have any issues with line-of-sight for the IR remote (and I rely on a regular IR remote for the TV itself of course). It supports Disney+, but also all the other big name services, some of which we already use (Netflix, YouTube BBC iPlayer) and some of which we didn't, since it was too awkward to access them (Google Play, Amazon Prime Video). It has now largely displaced the FreeSat box for accessing streaming content because it works so well and everything is in one place.

There's a phone App that remote-controls the box and works even better than the physical remote: it can offer a full phone-keyboard at times when you need to input text, and can mute the TV audio and put it out through headphones attached to the phone if you want.

My aging Plasma TV suffers from burn-in from static pictures. If left paused for a duration the Roku goes to a screensaver that keeps the whole frame moving. The FreeSat doesn't do this. My Blu Ray player does, but (I think) it retains some static elements.

Tags:

This is a fairly self-indulgent post, sorry!

Encouraged by Evgeni, Michael and others, given I'm spending a lot more time at my desk in my home office, here's a picture of it:

Fisheye shot of my home office

Fisheye shot of my home office

Near enough everything in the study is a work in progress.

The KALLAX behind my desk is a recent addition (under duress) because we have nowhere else to put it. Sadly I can't see it going away any time soon. On the up-side, since Christmas I've had my record player and collection in and on top of it, so I can listen to records whilst working.

The arm chair is a recent move from another room. It's a nice place to take some work calls, serving as a change of scene, and I hope to add a reading light sometime. The desk chair is some Ikea model I can't recall, which is sufficient, and just fits into the desk cavity. (I'm fairly sure my Aeron, inaccessible elsewhere, would not fit.)

I've had this old mahogany, leather-topped desk since I was a teenager and it's a blessing and a curse. Mostly a blessing: It's a lovely big desk. The main drawback is it's very much not height adjustable. At the back is a custom made, full-width monitor stand/shelf, a recent gift produced to specification by my Dad, a retired carpenter.

On the top: my work Thinkpad T470s laptop, geared more towards portable than powerful (my normal preference), although for the forseeable future it's going to remain in the exact same place; an Ikea desk lamp (I can't recall the model); a 27" 4K LG monitor, the cheapest such I could find when I bought it; an old LCD Analog TV, fantastic for vintage consoles and the like.

Underneath: An Alesis Micron 2½ octave analog-modelling synthesizer; various hubs and similar things; My Amiga 500.

Like Evgeni, my normal keyboard is a ThinkPad Compact USB Keyboard with TrackPoint. I've been using different generations of these styles of keyboards for a long time now, initially because I loved the trackpoint pointer. I'm very curious about trying out mechanical keyboards, as I have very fond memories of my first IBM Model M buckled-spring keyboard, but I haven't dipped my toes into that money-trap just yet. The Thinkpad keyboard is rubber-dome, but it's a good one.

Wedged between the right-hand bookcases are a stack of IT-related things: new printer; deprecated printer; old/spare/play laptops, docks and chargers; managed network switch; NAS.

Tags:

I've written about what Template Haskell is, and given an example of what it can be used for, it's time to explain why I was looking at it in the context of my PhD work.

Encoding stream-processing programs

StrIoT is an experimental distributed stream-processing system that myself and others are building in order to explore our research questions. A user of StrIoT writes a stream-processing program, using a set of 8 functional operators provided for the purpose. A simple example is

streamFn :: Stream Int -> Stream Int
streamFn = streamFilter (<15)
         . streamFilter (>5)
         . streamMap (*2)

Our system is distributed: we take a stream-processing program and partition it into sub-programs, which are distributed to and run on separate nodes (perhaps cloud instances, or embedded devices like Raspberry Pis etc.). In order to do that, we need to be able to manipulate the stream-processing program as data. We've initially opted for a graph data-structure, with the vertices in the graph defined as

data StreamVertex = StreamVertex
    { vertexId   :: Int
    , operator   :: StreamOperator
    , parameters :: [String]
    , intype     :: String
    , outtype    :: String
    } deriving (Eq,Show)

A stream-processing program encoded this way, equivalent to the first example

path [ StreamVertex 0 Map    ["(*2)"]  "Int" "Int"
     , StreamVertex 1 Filter ["(>5)"]  "Int" "Int"
     , StreamVertex 2 Filter ["(<15)"] "Int" "Int"
     ]

We can easily manipulate instances of such types, rewrite them, partition them and generate code from them. Unfortunately, this is quite a departure from the first simple code example from the perspective of a user writing their program.

Template Haskell gives us the ability to manipulate code as a data structure, and also to inspect names to gather information about them (their type, etc.). I started looking at TH to see if we could build something where the user-supplied program was as close to that first case as possible.

TH limitations

There are two reasons that we can't easily manipulate a stream-processing definition written as in the first example. The following expressions are equivalent, in some sense, but are not equal, and so yield completely different expression trees when quasi-quoted:

[| streamFilter (<15) . streamFilter (>5) . streamMap (*2) |]
| \s -> streamFilter (<15) (streamFilter (>5) (streamMap (*2) s)) |]
[| streamMap (*2) >>> streamFilter (>5) >>> streamFilter (<15) |]
[| \s -> s & streamMap (*2) & streamFilter (>5) & streamFilter (<15) |]
[| streamFn |] -- a named expression, defined outside the quasi-quotes

In theory, reify can give you the definition of a function from its name, but in practice it doesn't, because this was never implemented. So at the very least we would need to insist that a user included the entirety of a stream-processing program within quasi-quotes, and not split it up into separate bits, with some bits defined outside the quotes and references within (as in the last case above). We would probably have to insist on a consistent approach for composing operators together, such as always use (.) and never >>>, &, etc. which is limiting.

Incremental approach

After a while ruminating on this, and before moving onto something else, I thought I'd try approaching it from the other side. Could I introduce some TH into the existing approach, and improve it? The first thing I've tried is to change the parameters field to TH's ExpQ, meaning the map instance example above would be

StreamVertex 0 Map [ [| (*2) |] ] "Int" "Int"

I worked this through. It's an incremental improvement ease and clarity for the user writing a stream-processing program. It catches a class of programming bugs that would otherwise slip through: the expressions in the brackets have to be syntactically valid (although they aren't type checked). Some of the StrIoT internals are also much improved, particularly the logical operator. Here's an excerpt from a rewrite rule that involves composing code embedded in strings, dealing with all the escaping rules and hoping we've accounted for all possible incoming expression encodings:

let f' = "(let f = ("++f++"); p = ("++p++"); g = ("++g++") in\
         \ \\ (a,b) v -> (f a v, if p v a then g b v else b))"
    a' = "("++a++","++b++")"
    q' = "(let p = ("++p++"); q = ("++q++") in \\v (y,z) -> p v y && q v z)"

And the same section after, manipulating ExpQ types:

let f' = [| \ (a,b) v -> ($(f) a v, if $(p) v a then $(g) b v else b) |]
    a' = [| ($(a), $(b)) |]
    q' = [| \v (y,z) -> $(p) v y && $(q) v z |]

I think the code-generation part of StrIoT could be radically refactored to take advantage of this change but I have not made huge inroads into that.

Next steps

This is, probably, where I am going to stop. This work is very interesting to me but not the main thrust of my research. But incrementally improving the representation gave me some ideas of what I could try next:

  • intype and outtype could be TH Types instead of Strings. This would catch some simple problems like typos, etc., but we could possibly go further, and
  • remove the explicit in-and-out-types and infer their values from the parameters field, as its an expression with some type that should match
  • parameters is a list, because the different stream operators have different arities. streamFilter has one parameter (the filter predicate), so the list should have one element in that case, but streamExpand has none, so it should be empty. We could collapse this to a single ExpQ, which encoded however many parameters are necessary, either in an internal list, or…
  • the operator field could be merged in too, so that the parameters expression was actually a call to the relevant operator with its parameters supplied.

The type would have collapsed down to

data StreamVertex = StreamVertex
    { vertexId   :: Int
    , opAndParams :: ExpQ
    } deriving (Eq,Show)

Example instances might be

StreamVertex 0 [| streamMap (*2) |]
StreamVertex 1 [| streamExpand |]
StreamVertex 2 [| streamScan (\c _ -> c+1) 0 |]

The vertexId field is a bit of wart, but we require that due to the graph data structure that we are using. A change there could eliminate it, too. By this point we are not that far away from where we started, and certainly much closer to the "pure" function application in the very first example.

Tags:

Here's a practical example of applying Template Haskell to reduce the amount of boilerplate code that is otherwise required. I wrote the below after following this excellent blog post by Matt Parsons. This post will be much higher-level, read Matt's blog for the gorier details.

Liquorice

Liquorice is a toy project of mine from a few years ago that lets you draw 2D geometric structures similar to LOGO. Liquorice offers two interfaces: pure functions that operate on an explicit Context (the pen location: existing lines, etc.), and a second "stateful" interface where the input and output are handled in the background. I prefix the pure ones P. and the stateful ones S. in this blog post for clarity.

The stateful interface can be much nicer to use for larger drawings. Compare example8b.hs, written in terms of the pure functions, and the stateful equivalent example8.hs.

The majority of the stateful functions are "wrapped" versions of the pure functions. For example, the pure function P.step takes two numbers and moves the pen forward and sideways. Its type signature is

P.step :: Int -> Int -> Context -> Context

Here's the signature and implementation of the stateful equivalent:

S.step :: Int -> Int -> State Context ()
S.step x y = modify (P.step x y)

Writing these wrapped functions for the 29 pure functions is boilerplate that can be generated automatically with Template Haskell.

Generating the wrapper functions

Given the Name of a function to wrap, we construct an instance of FunD, the TH data-type representing a function definition. We use the base name of the incoming function as the name for the new one.

mkWrap fn = do
    …
    let name = mkName (nameBase fn)
    return $ FunD name [ Clause (map VarP args) (NormalB rhs) [] ]

To determine how many arguments the wrapper function needs to accept, we need to determine the input function's arity. We use Template Haskell's reify function to get type information about the function, and derive the arity from that. Matt Parson's covers this exactly in his blog.

info    <- reify fn
let ty   = (\(VarI _ t _ ) -> t) info
let n    = arity ty - 1
args    <- replicateM n (newName "arg")

We can use the list "args" directly in the clause part of the function definition, as the data-type expects a list. For the right-hand side, we need to convert from a list of arguments to function application. That's a simple left-fold:

-- mkFnApp f [a,b,c] => ((f a) b) c => f a b c
mkFnApp = foldl (\e -> appE e . varE)
rhs     <- [| modify $(mkFnApp (varE fn) args) :: State Context () |]

We use TH's oxford brackets for the definition of rhs. This permits us to write real Haskell inside the brackets, and get an expression data-type outside them. Within we have a splice (the $(…)), which does the opposite: the code is evaluated at compile time and generates an Exp that is then converted into the equivalent Haskell code and spliced into place.

Finally, we need to apply the above to a list of Names. Sadly, we can't get at the list of exported names from a Module automatically. There is an open request for a TH extension for this. In the meantime, we export a list of the functions to wrap from the Pure module and operate on that

import Liquorice.Pure
wrapPureFunctions = mapM mkWrap pureFns

Finally, we 'call' wrapPureFunctions at the top level in our state module and Template Haskell splices all the function definitions into place.

The final code ended up only around 30 lines of code, and saved about the same number of lines of boilerplate. But in doing this I noticed some missing functions, and it will pay dividends if more pure functions are added.

Limitations

The current implementation has one significant limitation: it cannot handle higher-order functions. An example of a pure higher-order function is place, which moves the pen, performs an operation, and then moves it back:

P.place :: Int -> Int -> (Context -> Context) -> Context -> Context

Wrapping this is not sufficient because the higher-order parameter has the pure function signature Context -> Context. If we wrapped it, the stateful version of the function would accept a pure function as the parameter, but you would expect it to accept another stateful function.

To handle these, at a minimum we would need to detect the function arguments that have type Context -> Context and replace them with State Context (). The right-hand side of the wrapped function would also need to do more work to handle wrapping and unwrapping the parameter. I haven't spent much time thinking about it but I'm not sure that a general purpose wrapper would work for all higher-order functions. For the time being I've just re-implemented the half-dozen of them.

Tags:

tricky

tricky

This one was a bit of a surprise hit. Golf Peaks is a Golf-themed puzzle game. The objective is to get the golf ball into the hole, obviously, but how you do that is by choosing from a set of fixed moves (e.g., move one square; jump one square; etc.) and a direction.

This works well with my eldest daughter. She takes one joy-con and I take the other. I typically am responsible for direction, and she hits 'A' to move the ball. We discuss which move to make, which has been a good test of her numeracy.

Tags:

Older posts are available on the all posts page.


Comments