Guile-QuickCheck
This Guile library provides tools for randomized, property-based testing. It follows closely the QuickCheck library written in Haskell, with inspiration from the Racket version. You can use it to define a property (a predicate with specifications for its inputs) and test it by generating many random inputs and seeing if it holds.
Example
Let’s say that we want to write a procedure that ensures we have a
tasty list of pizza toppings. We will call it ensure-tasty
, and it
will save us from having to eat merely mediocre pizza. First, we
need to define what tasty pizza is! It’s a good thing that
absolutely everyone agrees about pizza toppings, so we just use the
universally acknowledged rule for tasty pizza (taken from RFC 91334):
A tasty pizza MUST have pineapples and MUST NOT have tomato slices.
In Scheme, we can define a predicate for testing if a list of toppings is tasty or not:
(define (tasty? toppings) (and (memq 'pineapples toppings) (not (memq 'tomato-slices toppings))))
Now, before we define ensure-tasty
, we will write a property that it
must satisfy using the Guile-QuickCheck tools for specifying inputs.
The property we want says “given any list of toppings as input,
ensure-tasty
returns a tasty list of toppings”. There are two
things here that we need translate into to Scheme code. First we need
to specify what “any list of toppings” means, and then we need to
explain how to check if ensure-tasty
returns tasty toppings.
Specifying inputs
Guile-QuickCheck provides a few combinators for specifying inputs.
These combinators all begin with “$” (which can be read as
“arbitrary”). We will define a topping as any one of a list of
symbols. To define a single constant symbol (like 'olives
), we use
the $const
combinator. Then, we can use the $choose
combinator to
specify that we want to use one of many specifications. With
$choose
, each specification has to be paired with a predicate, so
the specification ends up rather verbose:
(define $topping ($choose ((cut eq? <> 'mushrooms) ($const 'mushrooms)) ((cut eq? <> 'olives) ($const 'olives)) ((cut eq? <> 'peppers) ($const 'peppers)) ((cut eq? <> 'onions) ($const 'onions)) ((cut eq? <> 'tomato-slices) ($const 'tomato-slices)) ((cut eq? <> 'pineapples) ($const 'pineapples))))
With $topping
in place, we can use the $list
combinator provided
by Guile-QuickCheck to construct a list of toppings:
(define $toppings ($list $topping))
Specifying properties
Now that we have $toppings
to stand in for “any list of toppings,”
we can finish writing our property:
(define ensure-tasty-property (property ((toppings $toppings)) (tasty? (ensure-tasty toppings))))
The property
syntax is provided by Guile-QuickCheck, and it
specifies how to generate random inputs and how to test them. In this
case we specify that toppings
should be drawn from $toppings
(“any
list of toppings”). Just like let
, the property
syntax allows
binding multiple names, but we only need one here. We then say that
if we apply ensure-tasty
to toppings
, the result should satisfy
tasty?
.
Testing
At this point, we can write a definition for ensure-tasty
and test
it against ensure-tasty-property
. Let’s start with a bad definition
and see if Guile-QuickCheck can catch it:
(define (ensure-tasty toppings) (cons 'pineapples toppings))
We can test our property using the quickcheck
procedure:
(quickcheck ensure-tasty-property)
What does Guile-QuickCheck say?
Falsifiable after 4 tests. Seed: 2086549941 toppings = (tomato-slices mushrooms onions)
This means that after running four tests, Guile-QuickCheck found a
counter example for our property. It says that if it applies
ensure-tasty
to the toppings (tomato-slices mushrooms onions)
, the
result is not tasty. The other thing it tells us is that the seed
used to generate the inputs is 2086549941. This could be used to
generate the same inputs again.
Let’s see if we can fix our definition of ensure-tasty
:
(define (ensure-tasty toppings) (filter (lambda (topping) (not (eq? topping 'tomato-slices))) (cons 'pineapples toppings)))
Running quickcheck
again, we see that Guile-QuickCheck says:
OK, passed 100 tests.
Hooray! Guile-QuickCheck thought up 100 lists of pizza toppings, and
ensure-tasty
made every single one of them tasty. Now that you’ve
learned Guile-QuickCheck, you deserve a pizza. Just make sure that
it’s tasty!
(Also, here’s the complete example in one file.)
Download
https://files.ngyro.com/guile-quickcheck/guile-quickcheck-0.1.0.tar.gz
You should verify that this file comes from me using this signature.
Development
The main Git repository for this project is at https://git.ngyro.com/guile-quickcheck.