This page is a draft.

Introduction

Proff said in an email sometime that they were reluctant to accept patches due to a risk of damaging demo compatibility.

When fraggle started the chocolate doom project it occured to us that some kind of regression test suite would be really useful, both to prevent compatibility-regressions during development, and also as a way of checking off the goals when adjusting the chocolate doom code to increase compatibility.

Prototype

The general shape of a solution is to write out a checksum of data for each gametic to a file. Pairs of files can then be compared using standard tools like @diff@.

For a checksum algorithm, I chose MD5. Speed is the real concern rather than security, but it was relatively easy to find a reusable MD5 implementation (there's a public domain one inside dpkg.)

For the data to hash, I decided to re-use the existing savegame code. The savegame data is enough to restore the game without a desync, so it should be adequate. With chocolate-doom's source code up to 0.1.4, the save game code is very similar to the doom original, and as it turns out, very easy to repurpose.

This worked well enough to distinguish a geniune desync. However it threw up false positives for demos which played back fine between chocolate doom 0.1.3, 0.1.4 and a SVN checkout from around June 2006.

Test harness

The focus of my investigation therefore shifted to working out what set of data to include in the checksums to catch famous desyncs but not throw up too many false positives.

Colin Phipps (a developer of prboom) helpfully published a list of demos he used to catch regressions in prboom.

I decided to finish the test harness at this stage, as it would help me with identifying what info to include in checksums and see the result on a host of demos.

Results

...