Module Hegel

Introduction

Property-based testing for OCaml, powered by the native Hegel engine based on Hypothesis.

Hegel runs the test function on many generated inputs. You generate data inline, drawing values with draw as the test runs, rather than generating the data then running the property body. Each draw returns an ordinary OCaml value that you bind with let, compute with, and branch on, so a later draw can depend on an earlier generated value or a value from the system under test.

Because Hegel uses integrated shrinking, shrinking comes for free.

Getting started

Install Hegel

To install Hegel for OCaml:

  opam install hegel

The version of Hegel in OPAM sometimes lags behind the version in Github. To pin the version in Github:

  opam pin add hegel "git+https://github.com/hegeldev/hegel-ocaml.git"

Hegel for OCaml supports Linux (amd64/arm64) and macOS (Apple Silicon). macOS amd64 (Intel) has no published libhegel artifact, so on that platform point HEGEL_LIBHEGEL_PATH at a locally built libhegel.dylib.

Hegel works with whatever test framework your project already uses. The examples below use Alcotest.

Add hegel and alcotest to your dune test stanza. The examples in this documentation also use ppx_sexp_conv's [%sexp_of: t] to write value printers.

  (test
   (name my_tests)
   (libraries hegel alcotest)
   (preprocess (pps ppx_hegel_test ppx_sexp_conv)))

Write your first test

Write a property test using let%hegel_test:

  open Hegel

  let%hegel_test commutative_addition tc =
    let a = draw tc (integers ~min_value:(-1000) ~max_value:1000 ()) in
    let b = draw tc (integers ~min_value:(-1000) ~max_value:1000 ()) in
    require_equal tc [%sexp_of: int] (a + b) (b + a)

  let () =
    Alcotest.run
      "my_tests"
      [ "properties", [ Alcotest.test_case "commutative addition" `Quick commutative_addition ] ]

We check the property with require_equal rather than assert (a + b = b + a). It takes a printer for the values and, when they differ, shows a structural diff of the two sides in the failure report instead of a bare "assertion failed". Use require for a boolean check with a custom message. An assert can be used as well, but it does not provide as much information as require_equal and require.

Run dune runtest. You should see Alcotest report the test as passing. Hegel generates up to 100 random input pairs and reports the minimal counterexample if it finds one. When a test fails, Hegel prints each value you drew from the failing case, named after the let binding it was bound to (a = …, b = …).

The rest of the examples in the documentation assume you have open Hegel at the top of the test file like in the example above.

Next, let's try a test that fails.

  let%hegel_test every_int_is_small tc =
    let n = draw tc (integers ()) in
    assert (n < 50)

This test asserts that any integer is less than 50, which is obviously incorrect. Hegel finds a test case that makes the assertion fail, then shrinks it to the smallest counterexample (n = 50). The final replay prints the drawn values, the exception, and a rerun with: line that replays the exact case:

  --- Failure: every_int_is_small (my_tests.ml:3) ------------------

    n = 50

  Exception: File "my_tests.ml", line 5, characters 2-8: Assertion failed
  rerun with: [@@failure_blobs [ "AAEAAAAACgEAAAAy" ]]

To fix this test, you can constrain the integers you generate with min_value and max_value:

  let%hegel_test every_int_is_small tc =
    let n = draw tc (integers ~min_value:0 ~max_value:49 ()) in
    assert (n < 50)

Use generators

Hegel provides a rich library of generators that you can use out of the box. See Generators for the full reference.

For instance, you can use lists to construct a list of integers:

  let%hegel_test append_increases_length tc =
    let xs = draw tc (lists (integers ()) ()) in
    let initial_length = List.length xs in
    let xs = draw tc (integers ()) :: xs in
    require tc ~msg:"prepending an element must grow the list"
      (List.length xs > initial_length)

Custom generators are also supported. Suppose you have a person record that requires generation. Build a generator for it with composite, drawing each field in sequence:

  type person =
    { age : int
    ; name : string
    }

  let person =
    composite (fun tc ->
      let age = draw_silent tc (integers ()) in
      let name = draw_silent tc (text ()) in
      { age; name })

You can chain drawing operations together, so a later draw depends on an earlier one. For instance, extending person with a driving_license field that can only be true once age is at least 18:

  type person =
    { age : int
    ; name : string
    ; driving_license : bool
    }

  let person =
    composite (fun tc ->
      let age = draw_silent tc (integers ()) in
      let name = draw_silent tc (text ()) in
      let driving_license =
        if age >= 18 then draw_silent tc (booleans ()) else false
      in
      { age; name; driving_license })

Derive generators

Annotate the type with [@@deriving hegel_generator] and add ppx_hegel_generator to your preprocess stanza:

  (test
   (name my_tests)
   (libraries hegel alcotest)
   (preprocess (pps ppx_hegel_test ppx_hegel_generator)))

open Hegel is required before the first @@deriving hegel_generator, unless you want to derive generators for Core types (see Hegel_jane.Derive).

  open Hegel

  type point =
    { x : int
    ; y : int
    }
  [@@deriving hegel_generator]

  let%hegel_test point_roundtrip tc =
    let p = draw tc hegel_generator_point in
    assert ({ x = p.x; y = p.y } = p)

The type t derives a value named hegel_generator. Any other type foo derives hegel_generator_foo. Derived generators print drawn values as s-expressions.

Deriving generators also works on types in modules:

  module Temperature = struct
    type t = { celsius : float } [@@deriving hegel_generator]
  end

  type reading =
    { sensor_id : int
    ; temp : Temperature.t
    }
  [@@deriving hegel_generator]

The reading generator draws its temp field through Temperature.hegel_generator.

[@hegel.generator expr] sets the generator for a type instead of deriving it:

  type ranked =
    { name : string
    ; level :
        (int[@hegel.generator integers ~min_value:1 ~max_value:5 ()])
    }
  [@@deriving hegel_generator]

[@hegel.do_not_generate] excludes a variant constructor from being generated.

  type response =
    | Ok_response of int
    | Errored of exn [@hegel.do_not_generate]
  [@@deriving hegel_generator]

If a field's type has no sexp_of_* representation, mark the field [@sexp.opaque]:

  type connection = { send : bytes -> unit }

  type session =
    { id : int
    ; conn : (connection [@sexp.opaque])
    }
  [@@deriving hegel_generator]

The conn field prints as <opaque>.

Changing test settings

To override the default settings, attach a [@@settings ...] attribute:

  let%hegel_test commutative_addition tc =
    let a = draw tc (integers ()) in
    let b = draw tc (integers ()) in
    require_equal tc [%sexp_of: int] (a + b) (b + a)
  [@@settings Settings.create ~test_cases:500 ()]

This increases the number of test cases run from 100 to 500.

Settings can be changed with record update syntax.

  let%hegel_test commutative_addition tc =
    let a = draw tc (integers ()) in
    let b = draw tc (integers ()) in
    require_equal tc [%sexp_of: int] (a + b) (b + a)
  [@@settings { (Settings.create ~test_cases:500 ()) with verbosity = Settings.Verbose }]

Debugging failing test cases

Use note to attach debug information:

  let%hegel_test every_int_is_small tc =
    let n = draw tc (integers ()) in
    note tc (Printf.sprintf "n is %d" n);
    assert (n < 50)

A failing run prints a framed report: the shrunk counterexample's draws and notes, the exception, and a copy-pasteable rerun with: line whose base64 blob encodes the choice sequence that caused the failure (disable it with print_blob = false). On a terminal the report headers (and require_equal diffs) print in color; set HEGEL_COLOR to 1 or 0 to force colors on or off.

For an equality property, prefer require_equal over assert (x = y): it adds a structural diff of the two values to this report, so you see exactly how they differ. require is the message-carrying boolean variant.

  let%hegel_test every_int_is_small tc =
    let n = draw tc (integers ()) in
    assert (n < 50)

  --- Failure: every_int_is_small (my_tests.ml:3) ------------------

    n = 50

  Exception: File "my_tests.ml", line 5, characters 2-8: Assertion failed
  rerun with: [@@failure_blobs [ "AAEAAAAACgEAAAAy" ]]

The blob can then be used to replay the failing test case:

  let%hegel_test every_int_is_small tc =
    let n = draw tc (integers ()) in
    assert (n < 50)
  [@@failure_blobs [ "AAEAAAAACgEAAAAy" ]]

The blob is only meant to reproduce the failure within a specific version of Hegel, since the choice sequence leading to a failure can change from version to version.

Jane Street Core support

Projects that use Jane Street's Core can also use the optional Hegel_jane library to generate Core values. See Hegel_jane.

Learning more

See Generators for the generators and Stateful for state-machine testing.

Hegel module documentation

Submodules

Settings

Running tests

  let my_settings = Settings.create ~test_cases:50 ~seed:5 () in
  let () =
    run_hegel_test ~settings:my_settings (fun tc ->
      let n = draw tc (integers ~min_value:0 ~max_value:9 ()) in
      assert (n >= 0 && n <= 9))

Drawing values

  let%hegel_test draw_example tc =
    let n = draw tc (integers ~min_value:0 ~max_value:100 ()) in
    assert (n >= 0)

  let%hegel_test draw_silent_example tc =
    let n = draw_silent tc (map (fun x -> x * 2) (integers ~min_value:0 ~max_value:9 ())) in
    assert (n >= 0)

Guiding generation

  let%hegel_test head_cons_tail_reconstructs tc =
    let xs = draw tc (lists (integers ()) ()) in
    (* The property is only meaningful for non-empty lists. *)
    assume tc (xs <> []);
    assert (List.hd xs :: List.tl xs = xs)

  let%hegel_test grow_size tc =
    let v = draw tc (integers ~min_value:0 ~max_value:1000 ()) in
    target tc ~label:"size" ~value:(float_of_int v);
    assert (v <= 1000)

Collecting statistics

With show_statistics enabled in Settings.t, every run prints test statistics.

  let%hegel_test list_statistics tc =
    let xs = draw_silent tc (lists (integers ()) ()) in
    (match xs with
     | [] -> event tc ~label:"empty input"
     | _ -> ());
    event_value tc ~label:"length" ~value:(float_of_int (List.length xs))

prints:

  Statistics (over 100 test cases):
    * empty input: 5.0% of test cases
    * length: count 100, min 0, median 4, mean 5.12, p90 9, max 15

  let%hegel_test observe_emptiness tc =
    let xs = draw tc (lists (integers ()) ()) in
    if List.is_empty xs then event tc ~label:"empty input";
    assert (List.length (List.sort compare xs) = List.length xs)

  let%hegel_test observe_length tc =
    let xs = draw tc (lists (integers ()) ()) in
    event_value tc ~label:"length" ~value:(float_of_int (List.length xs));
    assert (List.length (List.sort compare xs) = List.length xs)

Debugging tests

  let%hegel_test note_value tc =
    let n = draw tc (integers ~min_value:0 ~max_value:99 ()) in
    note tc (Printf.sprintf "n is %d" n);
    assert (n < 100)

  let%hegel_test balanced tc =
    let l = draw tc (lists (integers ()) ()) in
    require tc ~msg:"sum must stay non-negative" (running_sum l >= 0)

  let%hegel_test sort_is_stable tc =
    let l = draw tc (lists (integers ()) ()) in
    require_equal
      tc
      [%sexp_of: int list]
      (List.sort compare l)
      (stable_sort l)

  let%hegel_test with_printer_example tc =
    let doubled = map (fun x -> x * 2) (integers ~min_value:0 ~max_value:9 ()) in
    let n = draw tc (with_printer [%sexp_of: int] doubled) in
    assert (n >= 0)

Cloning a test case