Protocol reference

This is an RFC-style reference for the Hegel protocol. It is intended to be detailed enough that you could implement a compatible Hegel server or client only by referencing this document.

Terminology

• Server: The server process, implementing the core functionality of a property-based testing library. Hegel-core is an example of a server.
• Client: The client library, implementing the user interface of a property-based testing library. Hegel-rust is an example of a client.
• Packet: The unit of data sent in the protocol. A packet is always associated with a stream. A packet is comprised of a 31-bit integer ID, and 1 bit indicating whether it is a reply.
• Request packet: A packet with the reply bit set to 0. All packets are either request packets or reply packets. Note that not every request packet expects a reply packet in response.
• Reply packet: A packet with the reply bit set to 1. Each reply packet is associated with a specific request packet. All packets are either request packets or reply packets.
• Stream: A unique identifier used to associate logically-related packets together. All packets are associated with a stream.
• Control stream: The stream used for high-level packets, for example because no other stream has been established yet. The control stream has id 0.
• Test case: A single execution of the test function with the concrete set of values that were generated for it during its execution.
• Test run: The full lifecycle of a test function, including executing multiple test cases and shrinking any failures.
• Connection: A generic connection allowing packets to be sent between a client and a server. The protocol is agnostic to the details of this connection and does not specify what transport layer it uses or how it is established. Connections are intended to be created once per process and shared between test runs, though this is not a requirement. A single test run uses the same connection for all its test cases. Multiple test runs are permitted in parallel on a single connection.

Wire specification

Packet

A packet consists of a 20-byte header, a variable-length payload, and a 1-byte terminator.

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Magic (0x4845474C)                       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Checksum (CRS32)                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Stream id                             |S|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|                    Message id                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Payload length                           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Payload (variable length)                |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  Term. (0x0A) |
+-+-+-+-+-+-+-+-+

The first five fields comprise the header. Each field is an unsigned 32-bit big-endian integer:

• Magic: The constant 0x4845474C. This is ASCII for "HEGL".
• Checksum: Defined as crS32(H), where H is the packet with the terminator removed and the checksum field set to 0.
• Stream ID: The stream this packet is associated with. The S (source) bit is 1 for streams created by the client, and 0 for streams created by the server¹.
• Message ID: The id of the message associated with this packet. The R (reply) bit is set iff this packet is a reply to a request packet. The message id of a reply packet is the same as the message id of the corresponding request packet, but with the R bit set².
• Payload length: The length of the payload, in bytes.

The header is followed by the variable-length payload field, and then a single terminator byte (0x0A).

The encoding of the payload varies, and is documented for individual packets. We encode the payload of many packets in the protocol with CBOR.

Test lifecycle

The protocol is agnostic of the transport layer used to send and receive packets³. We will assume here that a connection has been established between the server and the client and do not specify how that connection is established.

Handshake

After the connection has been established, the client is expected to initiate a handshake over that connection. A handshake must be initiated before any other communication. A handshake must not be initiated more than once per connection.

Client                            Server
    |                                  |
 1  |---[control]-- handshake -------->|
 1R |<--[control]-- handshake_reply ---|
    |                                  |

handshake

reply bit	0
encoding	ASCII
payload	hegel_handshake_start

handshake_reply

reply bit	1
encoding	ASCII
payload	Hegel/{version}, where version is the protocol version.

Test run

After completing the handshake, the client may at any time choose to start a test run.

Client                           Server
    |                                 |
 1  |---[control]-- run_test -------->|
 1R |<--[control]-- run_test_reply ---|
    |                                 |

run_test

reply bit	0
encoding	CBOR
fields	command	"run_test"
		type	text
		required	yes
	stream_id	A new stream id chosen by the client that will be used for this test run. Note that because the client is creating this stream, stream_id must have its |S| bit set to 1, i.e. must be odd.
		type	integer
		required	yes
	test_cases	The maximum number of test cases to execute.
		type	integer
		required	yes
	seed	Random seed.
		type	integer
	derandomize	If true, and seed is not set, derive a deterministic seed from database_key.
		type	boolean
		default	false
	database_key	Stable database key for this test.
		type	bytes
	database	Path to the test case database directory, or null to disable the database.
		type	text | null
	suppress_health_check	Array of health check names to suppress. Valid names are "test_cases_too_large", "filter_too_much", "too_slow", and "large_initial_test_case".
		type	text[]
		default	[]

run_test_reply

reply bit	1
encoding	CBOR
payload	true

Test cases

Once a test run has been started, the server initiates the execution of test cases. The client uses the stream established in the run_test packet for packets related to this test run.

Client                                       Server
    |                                             |
 1  |<--[S1]-- test_case -------------------------|
 1R |---[S1]-- test_case_reply ------------------>|
    |                                             |
 2  |---[S2]-- command -------------------------->|
 2R |<--[S2]-- (command reply, if appropriate) ---|
    |---[S2]-- ... ------------------------------>|
    |                                             |
 3  |---[S2]-- mark_complete -------------------->|
 3R |<--[S2]-- mark_complete_reply ---------------|
    |                                             |
 4  |---[S2]-- stream_close --------------------->|
    |                                             |
    :       ... repeat for each test case         :
    |                                             |

In the test_case packet, it establishes a new stream (S2) which is used for packets for this test case.

test_case

reply bit	0
encoding	CBOR
fields	event	"test_case"
		type	text
		required	yes
	stream_id	The stream id for this test case.
		type	integer
		required	yes
	is_final	false during normal execution, true during final replays.
		type	boolean
		required	yes

test_case_reply

reply bit	1
encoding	CBOR
payload	null

During the test case, the client may send test case commands to the server. These command packets form the main interaction between the client and the server during the test case and are used to request generated values, mark spans of choices, etc.

After some number of test cases, the server will consider the test run to be over. This might be because we hit the test_cases setting (communicated in the initial run_test packet), or because a test case found an error in the test function. Whenever the server decides the test run is finished, it sends a mark_complete command on the test case stream with information describing the status of the test case.

mark_complete

reply bit	0
encoding	CBOR
fields	command	"mark_complete"
		type	text
		required	yes
	status	"VALID" (test case passed), "INVALID" (test case rejected, for example by via assume(false)), or "INTERESTING" (test case failed).
		type	text
		required	yes
	origin	Description of the failure location. Should be set when status is "INTERESTING".
		type	text

mark_complete_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

Test case commands

All commands are request packets sent by the client on the test case stream. The server replies with the corresponding reply packet.

Instead of the documented reply packets below, the server might also return command_error_reply to any command.

command_error_reply

reply bit	1
encoding	CBOR
fields	error	The error message.
		type	text
		required	yes
	type	The error type (for example, "ValueError").
		type	text
		required	yes

generate

Generate a value from a schema.

generate

reply bit	0
encoding	CBOR
fields	command	"generate"
		type	text
		required	yes
	schema	A generator schema.
		type	map
		required	yes

generate_reply

reply bit	1
encoding	CBOR
fields	result	The generated value.
		type	any
		required	yes

start_span

Mark the start of a span of choices.

start_span

reply bit	0
encoding	CBOR
fields	command	"start_span"
		type	text
		required	yes
	label	Identifies the span type.
		type	integer
		default	0

start_span_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

stop_span

Mark the end of a span of choices.

stop_span

reply bit	0
encoding	CBOR
fields	command	"stop_span"
		type	text
		required	yes
	discard	If true, this span is excluded from shrinking.
		type	boolean
		default	false

stop_span_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

target

Record an observation for targeted property-based testing.

target

reply bit	0
encoding	CBOR
fields	command	"target"
		type	text
		required	yes
	value	The observation value.
		type	float
		required	yes
	label	Identifies the observation.
		type	integer
		required	yes

target_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

new_collection

Create a new collection. Collections are used for the sizing of variable-length collections.

new_collection

reply bit	0
encoding	CBOR
fields	command	"new_collection"
		type	text
		required	yes
	min_size	Minimum number of elements.
		type	integer
		default	0
	max_size	Maximum number of elements.
		type	integer | null
		default	unbounded

new_collection_reply

reply bit	1
encoding	CBOR
fields	result	The collection id as an integer.
		type	text
		required	yes

collection_more

Ask whether the collection should produce another element.

collection_more

reply bit	0
encoding	CBOR
fields	command	"collection_more"
		type	text
		required	yes
	collection_id	The collection id.
		type	text
		required	yes

collection_more_reply

reply bit	1
encoding	CBOR
fields	result	true if another element should be produced, false if the collection is done.
		type	boolean
		required	yes

collection_reject

Indicate that the most recently produced element was not added to the collection.

collection_reject

reply bit	0
encoding	CBOR
fields	command	"collection_reject"
		type	text
		required	yes
	collection_id	The collection id.
		type	text
		required	yes

collection_reject_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

new_pool

Create a new variable pool, which is a collection of identifiers intended to point to values that were generated or otherwise produced during testing. This is primarily useful for stateful testing, but can be used in normal tests too.

new_pool

reply bit	0
encoding	CBOR
fields	command	"new_pool"
		type	text
		required	yes

new_pool_reply

reply bit	1
encoding	CBOR
fields	result	The pool id (0-indexed).
		type	integer
		required	yes

pool_add

Add a new variable to a pool.

pool_add

reply bit	0
encoding	CBOR
fields	command	"pool_add"
		type	text
		required	yes
	pool_id	The pool to add to.
		type	integer
		required	yes

pool_add_reply

reply bit	1
encoding	CBOR
fields	result	The variable id.
		type	integer
		required	yes

pool_generate

Draw a variable from a pool. The server selects which variable to return.

pool_generate

reply bit	0
encoding	CBOR
fields	command	"pool_generate"
		type	text
		required	yes
	pool_id	The pool to draw from.
		type	integer
		required	yes
	consume	If true, the variable is removed from the pool after being returned.
		type	boolean
		default	false

pool_generate_reply

reply bit	1
encoding	CBOR
fields	result	The variable id. If the pool is empty, the server marks the test case as invalid.
		type	integer
		required	yes

Completing a test run

After all test cases have been executed and shrinking is complete, the server sends a test_done event on the test stream.

Client                        Server
    |                              |
 1  |<--[S1]-- test_done ----------|
 1R |---[S1]-- test_done_reply --->|
    |                              |

test_done

reply bit	0
encoding	CBOR
fields	event	"test_done"
		type	text
		required	yes
	results.passed	Whether the test passed.
		type	boolean
		required	yes
	results.test_cases	Total number of test cases executed.
		type	integer
		required	yes
	results.valid_test_cases	Number of valid test cases.
		type	integer
		required	yes
	results.invalid_test_cases	Number of invalid test cases.
		type	integer
		required	yes
	results.interesting_test_cases	Number of interesting (failing) test cases.
		type	integer
		required	yes
	results.seed	The random seed that was used, as a string.
		type	text
		required	yes
	results.flaky	Present if the test was detected to be flaky.
		type	text
	results.health_check_failure	Present if a health check failed.
		type	text
	results.error	Present if there was an error. This could be a usage error (e.g. invalid arguments) or indicate a server bug (e.g. unexpected exception raised).
		type	text

test_done_reply

reply bit	1
encoding	CBOR
payload	true

Final replays

After test_done, the server sends a test_case packet with is_final set to true for each interesting test case. The intent is for the client to use this to replay the minimal failing test case. Each final replay follows the standard test case flow.

Stream close

For clarity at runtime, stream can be marked as closed when they are no longer needed.

stream_close

reply bit	0
encoding	raw
payload	The single byte 0xFE. The message id of this packet is always (1 << 31) - 1.

Schemas

Fields in bold are required.

Some fields list their type as schema. This means it accepts any generator schema. For example, {"type": "integer"} is a valid schema, so {"type": "one_of", "generators": [{"type": "integer"}]} is as well.

constant	Generate value.
	value	The value to return.
		type	any
		required	yes
sampled_from	Generate one of values.
	values	Array of concrete values to sample from. Must be non-empty
		type	any[]
		required	yes
one_of	Generate a value drawn from one of generators.
	generators	Array of generators. Must be non-empty.
		type	schema[]
		required	yes
null	Generate null.
boolean	Generate either true or false.
integer	Generate an integer.
	min_value	Minimum value, inclusive.
		type	integer
		default	unbounded
	max_value	Maximum value, inclusive.
		type	integer
		default	unbounded
float	Generate a float.
	min_value	Minimum value.
		type	float
		default	unbounded
	max_value	Maximum value.
		type	float
		default	unbounded
	allow_nan	Whether NaN can be generated.
		type	boolean
		default	true if neither min_value nor max_value is set, false otherwise
	allow_infinity	Whether +Infinity and -Infinity can be generated.
		type	boolean
		default	true if either min_value or max_value is unset, false otherwise
	width	Float width. Either 32 or 64.
		type	integer
		default	64
	exclude_min	Exclude the minimum value.
		type	boolean
		default	false
	exclude_max	Exclude the maximum value.
		type	boolean
		default	false
string	Generate a Unicode string. Returned as a custom CBOR tag 6, with a payload equivalent to the UTF-8 representation of the string except that surrogate code points are allowed.
	min_size	Minimum length, in the number of code points.
		type	integer
		default	0
	max_size	Maximum length, in the number of code points.
		type	integer
		default	unbounded
binary	Generate a byte string.
	min_size	Minimum length, in bytes.
		type	integer
		default	0
	max_size	Maximum length, in bytes.
		type	integer
		default	unbounded
regex	Generate a string that matches the given pattern regular expression.
	pattern	The regular expression to match.
		type	string
		required	yes
	fullmatch	If true, pattern must match the entire string. If false, pattern may match a substring.
		type	boolean
		default	false
list	Generate a list of values from the elements generator.
	elements	The elements generator.
		type	schema
		required	yes
	min_size	Minimum number of elements.
		type	integer
		default	0
	max_size	Maximum number of elements.
		type	integer
		default	unbounded
	unique	If true, all generated list elements will be distinct.
		type	boolean
		default	false
dict	Generate a map with keys drawn from keys and values drawn from values. Returns in the format [[key1, value1], ...].
	keys	The keys generator.
		type	schema
		required	yes
	values	The values generator.
		type	schema
		required	yes
	min_size	Minimum number of map entries.
		type	integer
		default	0
	max_size	Maximum number of map entries.
		type	integer
		default	unbounded
tuple	Generate a fixed-length array, where each element is drawn from the corresponding generator at that position.
	elements	A list of generators of the same length as the desired tuple.
		type	schema[]
		required	yes
email	Generate an email address string, according to RFC 5322 Section 3.4.1.
url	Generate an http/https URL string, according to RFC 3986.
domain	Generate a fully qualified domain name string, according to RFC 1035.
	max_length	Maximum length of the domain name.
		type	integer
		default	255
ipv4	Generate an IPv4 address string.
ipv6	Generate an IPv6 address string.
date	Generate an ISO 8601 date string. For example: "2024-03-15".
time	Generate an ISO 8601 time string. For example: "14:30:00".
datetime	Generate an ISO 8601 datetime string. For example: "2024-03-15T14:30:00".

Appendix

Reading sequence diagrams

We use the following conventions for the sequence diagrams in this document:

Client                                           Server
    |                                               |
 1  |---[S1]-- a ---------------------------------->|
 1R |<--[S1]-- a_reply -----------------------------|
    |                                               |
 2  |<--[S2]-- b -----------------------------------|
 2R |---[S2]-- b_reply ---------------------------->|
    |                                               |

Each named arrow --[C]-- name --> represents a packet, associated with stream C, being sent from either the client to the server (-->) or the server to the client (<--). name is purely illustrative and is not part of the spec.

Each packet is associated with an auto-incrementing number on the left. n indicates a request packet, and nR indicates a reply packet responding to packet n. Not all request packets expect a reply packet. These packet numbers are illustrative and not part of the spec, except for where they associate a specific request packet with a specific reply packet.

Footnotes

1. The benefit of the S bit is to allow both the client and server to create streams without coordinating with each other. ↩

2. The benefit of the message id is to support out-of-order replies that are associated with the same stream. ↩

3. For example, hegel-core uses process stdin and stdout as its transport layer. ↩