Fungicide

Fungicide is a benchmark suite for Befunge–98 interpreters. I suppose I wrote it mostly to prove that CCBI version 2 rocks.

The name plays on the fungal theme that has attached itself to Befunge and is some kind of reference to “weeding the good from the bad” or thereabouts.

Most likely you want to just look at the current Fungicide rankings.

Downloads

The latest version, 1.0:

All associated files, no results data:
- .txz, 70.7 Kio

Documentation

Fungicide suffers from a lack of proper documentation, so here’s a little cryptic something.

Benchmarks

Each individual benchmark is identified by a script which produces it (often also called the benchmark, confusingly) and the parameter passed to the script. This parameter is the problem size, and its meaning depends on the benchmark in question.

Benchmarks tend to end with the f.@ sequence to report completion: what’s interesting is what comes before that.

Brief descriptions of each benchmark:

horizontal.b98: The > character repeated the given number of times.
vertical.b98: The v character repeated the given number of times (obviously on separate lines).
diagdown.b98: 11x followed by the given number of z in a diagonal line.
diagup.b98: Like diagdown.b98, but traversing the diagonal line northeast instead of southeast.
hollow-square.b98: The edge of a square whose edge length is the given number, traversed clockwise using >v<^.
filled-square.b98: A filled square whose edge length is the given number: each of the n*n cells in the square are one of >v<^ and they are all visited.

The above all have -p forms as well: this signifies that instead of placing the path to traverse into the file directly, it is first generated in the code using the p instruction. For example, horizontal-p.b98 first loops the given number of times, placing > instructions as it goes: it then executes the first of the ones it placed and proceeds to traverse the path as if it were a horizontal.b98 benchmark.

This not only benchmarks file loading versus runtime space manipulation, it also allows for longer diagdown and diagup benchmarks: the benchmark files generated for the non--p versions grow very fast.

filled-square.b98 has two -p forms, horizontal and vertical: they differ in the order in which the square is built: rowwise or columnwise respectively.

push.b98: The f instruction repeated the given number of times.
pushpop.b98: The :$ instruction sequence repeated the given number of times.
yn-rep.b98: The yn instruction sequence repeated the given number of times.
y-rep-n.b98: The y0 instruction sequence repeated the given number of times: followed by an n prior to exit.
fork.b98: Spawns the given number of threads (must be a power of two) using t, then runs them all into an @ for termination.

Measurements

Time is measured simply from a Perl script using the POSIX gettimeofday() function.

Memory usage is measured using a Python script which repeatedly reads the /proc/<pid>/smaps pseudofile, summing up any “Shared” and “Private” values. This is done as often as possible in a busy loop while the interpreter process is still alive.

First, the interpreter is run on a benchmark once and its time and memory use are measured. Memory usage is assumed to not vary, and thus it is measured only this one time per benchmark. Based on the time it took to run this, there are three possible continuations:

If the time was less than a minute, it is run once more. This result is discarded. (A sort of cache-cleaning thing between the memory and time measurements.) Then, ten time-measured runs are performed.
If the time was instead less than ten minutes, three time-measured runs are performed, with no in-between run.
If the time exceeded ten minutes, it is run only once more for time measuring.

Thus, we may have either two, four, or eleven temporal measurements in total. The representative one, for analysis, is chosen as the mean of all but the maximum of these (which, in almost all cases, simply cuts off the memory-measuring run).

Usage

Unless you want to mess with the scripts, in order to run the benchmarks yourself you’ll need at least:

A Linux system (due to the /proc usage for memory use measurement) with GNU Coreutils
Fungify
Perl
Python
zsh

For sensible memory timing results you should have at least two CPU cores available, due to the busy-looping nature of the memory use measurer.

Write an interpreters.dat containing lines of the form interpreter path-to-binary and run runallruns.sh, which will deposit raw results in the data directory and report its progress as it goes.

Now, if you want to run the scripts in analysis to generate graphs and the like, you’ll need, in addition to the above, gnuplot. First you need to preprocess the raw data into a more easily usable form: run process.sh, which’ll populate the preprocessed-data directory. (I recommend having PRLL in order to speed it up if you have a multicore processor.) Then run make-all-analyses.sh (again, PRLL is supported and recommended) to populate plotstables and you’ll have essentially everything that’s at the Fungicide rankings page.