Guile-QuickCheck

This Guile library provides tools for randomized, property-based testing. It follows closely the QuickCheck library written in Haskell, with inspiration from the Racket version. You can use it to define a property (a predicate with specifications for its inputs) and test it by generating many random inputs and seeing if it holds.

Example

Let’s say that we want to write a procedure that ensures we have a tasty list of pizza toppings. We will call it ensure-tasty, and it will save us from having to eat merely mediocre pizza. First, we need to define what tasty pizza is! It’s a good thing that absolutely everyone agrees about pizza toppings, so we just use the universally acknowledged rule for tasty pizza (taken from RFC 91334):

A tasty pizza MUST have pineapples and MUST NOT have tomato slices.

In Scheme, we can define a predicate for testing if a list of toppings is tasty or not:

(define (tasty? toppings)
  (and (memq 'pineapples toppings)
       (not (memq 'tomato-slices toppings))))

Now, before we define ensure-tasty, we will write a property that it must satisfy using the Guile-QuickCheck tools for specifying inputs. The property we want says “given any list of toppings as input, ensure-tasty returns a tasty list of toppings”. There are two things here that we need translate into to Scheme code. First we need to specify what “any list of toppings” means, and then we need to explain how to check if ensure-tasty returns tasty toppings.

Specifying inputs

Guile-QuickCheck provides a few combinators for specifying inputs. These combinators all begin with “$” (which can be read as “arbitrary”). We will define a topping as any one of a list of symbols. To define a single constant symbol (like 'olives), we use the $const combinator. Then, we can use the $choose combinator to specify that we want to use one of many specifications. With $choose, each specification has to be paired with a predicate, so the specification ends up rather verbose:

(define $topping
  ($choose
   ((cut eq? <> 'mushrooms)     ($const 'mushrooms))
   ((cut eq? <> 'olives)        ($const 'olives))
   ((cut eq? <> 'peppers)       ($const 'peppers))
   ((cut eq? <> 'onions)        ($const 'onions))
   ((cut eq? <> 'tomato-slices) ($const 'tomato-slices))
   ((cut eq? <> 'pineapples)    ($const 'pineapples))))

With $topping in place, we can use the $list combinator provided by Guile-QuickCheck to construct a list of toppings:

(define $toppings ($list $topping))

Specifying properties

Now that we have $toppings to stand in for “any list of toppings,” we can finish writing our property:

(define ensure-tasty-property
  (property ((toppings $toppings))
    (tasty? (ensure-tasty toppings))))

The property syntax is provided by Guile-QuickCheck, and it specifies how to generate random inputs and how to test them. In this case we specify that toppings should be drawn from $toppings (“any list of toppings”). Just like let, the property syntax allows binding multiple names, but we only need one here. We then say that if we apply ensure-tasty to toppings, the result should satisfy tasty?.

Testing

At this point, we can write a definition for ensure-tasty and test it against ensure-tasty-property. Let’s start with a bad definition and see if Guile-QuickCheck can catch it:

(define (ensure-tasty toppings)
  (cons 'pineapples toppings))

We can test our property using the quickcheck procedure:

(quickcheck ensure-tasty-property)

What does Guile-QuickCheck say?

Falsifiable after 4 tests.
Seed: 2086549941
toppings = (tomato-slices mushrooms onions)

This means that after running four tests, Guile-QuickCheck found a counter example for our property. It says that if it applies ensure-tasty to the toppings (tomato-slices mushrooms onions), the result is not tasty. The other thing it tells us is that the seed used to generate the inputs is 2086549941. This could be used to generate the same inputs again.

Let’s see if we can fix our definition of ensure-tasty:

(define (ensure-tasty toppings)
  (filter (lambda (topping)
            (not (eq? topping 'tomato-slices)))
          (cons 'pineapples toppings)))

Running quickcheck again, we see that Guile-QuickCheck says:

OK, passed 100 tests.

Hooray! Guile-QuickCheck thought up 100 lists of pizza toppings, and ensure-tasty made every single one of them tasty. Now that you’ve learned Guile-QuickCheck, you deserve a pizza. Just make sure that it’s tasty!

Releases

Development

The main Git repository for this project is at https://git.ngyro.com/guile-quickcheck.