[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Run benchmark analyses even after failure
- Subject: Run benchmark analyses even after failure
- From: Chris Warburton
- Date: Sun, 26 Nov 2017 19:17:44 +0000
- Resolution: fixed
- State: resolved
In particular, we need to know the number of wanted conjectures for
*all* data points, including those which failed. This is easy enough,
since it only depends on the sample, which we know whether or not the
exploration succeeded. We can supply '[]' to the analyser's stdin in the
case of a failure, which will give us 0 recall, undefined precision and
a list of all the wanted conjectures.
The reason we need to know this is to work out the standard deviation of
our precisions and recalls.