Protocol reference

This is an RFC-style reference for the Hegel protocol. It is intended to be detailed enough that you could implement a compatible Hegel server or client only by referencing this document.

Terminology

• Server: The server process, implementing the core functionality of a property-based testing library. Hegel-core is an example of a server.
• Client: The client library, implementing the user interface of a property-based testing library. Hegel-rust is an example of a client.
• Packet: The unit of data sent in the protocol. A packet is always associated with a stream. A packet is comprised of a 31-bit integer ID, and 1 bit indicating whether it is a reply.
• Request packet: A packet with the reply bit set to 0. All packets are either request packets or reply packets. Note that not every request packet expects a reply packet in response.
• Reply packet: A packet with the reply bit set to 1. Each reply packet is associated with a specific request packet. All packets are either request packets or reply packets.
• Stream: A unique identifier used to associate logically-related packets together. All packets are associated with a stream.
• Control stream: The stream used for high-level packets, for example because no other stream has been established yet. The control stream has id 0.
• Test case: A single execution of the test function with the concrete set of values that were generated for it during its execution.
• Test run: The full lifecycle of a test function, including executing multiple test cases and shrinking any failures.
• Connection: A generic connection allowing packets to be sent between a client and a server. The protocol is agnostic to the details of this connection and does not specify what transport layer it uses or how it is established. Connections are intended to be created once per process and shared between test runs, though this is not a requirement. A single test run uses the same connection for all its test cases. Multiple test runs are permitted in parallel on a single connection.

Wire specification

Packet

A packet consists of a 20-byte header, a variable-length payload, and a 1-byte terminator.

 0               1               2               3
 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Magic (0x4845474C)                       |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Checksum (CRS32)                         |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Stream id                              |S|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|                    Message id                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Payload length                           |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                      Payload (variable length)                |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  Term. (0x0A) |
+-+-+-+-+-+-+-+-+

The first five fields comprise the header. Each field is an unsigned 32-bit big-endian integer:

• Magic: The constant 0x4845474C. This is ASCII for "HEGL".
• Checksum: Defined as crS32(H), where H is the packet with the terminator removed and the checksum field set to 0.
• Stream ID: The stream this packet is associated with. The S (source) bit is 1 for streams created by the client, and 0 for streams created by the server¹.
• Message ID: The id of the message associated with this packet. The R (reply) bit is set iff this packet is a reply to a request packet. The message id of a reply packet is the same as the message id of the corresponding request packet, but with the R bit set².
• Payload length: The length of the payload, in bytes.

The header is followed by the variable-length payload field, and then a single terminator byte (0x0A).

The encoding of the payload varies, and is documented for individual packets. We encode the payload of many packets in the protocol with CBOR.

Test lifecycle

The protocol is agnostic of the transport layer used to send and receive packets³. We will assume here that a connection has been established between the server and the client and do not specify how that connection is established.

Handshake

After the connection has been established, the client is expected to initiate a handshake over that connection. A handshake must be initiated before any other communication. A handshake must not be initiated more than once per connection.

Client                            Server
    |                                  |
 1  |---[control]-- handshake -------->|
 1R |<--[control]-- handshake_reply ---|
    |                                  |

handshake

reply bit	0
encoding	ASCII
payload	hegel_handshake_start

handshake_reply

reply bit	1
encoding	ASCII
payload	Hegel/{version}, where version is the protocol version.

Test run

After completing the handshake, the client may at any time choose to start a test run.

Client                           Server
    |                                 |
 1  |---[control]-- run_test -------->|
 1R |<--[control]-- run_test_reply ---|
    |                                 |

run_test

reply bit	0
encoding	CBOR
fields	command	"run_test"
		type	text
		required	yes
	stream_id	A new stream id chosen by the client that will be used for this test run. Note that because the client is creating this stream, stream_id must have its |S| bit set to 1, i.e. must be odd.
		type	integer
		required	yes
	test_cases	The maximum number of test cases to execute.
		type	integer
		required	yes
	seed	Random seed.
		type	integer
	derandomize	If true, and seed is not set, derive a deterministic seed from database_key.
		type	boolean
		default	false
	database_key	Stable database key for this test.
		type	bytes
	database	Path to the test case database directory, or null to disable the database.
		type	text | null
	suppress_health_check	Array of health check names to suppress. Valid names are "test_cases_too_large", "filter_too_much", "too_slow", and "large_initial_test_case".
		type	text[]
		default	[]
	phases	Only run the given phases. Valid phases are "generate", "shrink", "reuse", "target", "explicit", and "explain".
		type	text[]
		default	all phases
	report_multiple_failures	If true, the server will report all the distinct bugs it finds during a test run. Otherwise, the server will report only the bug with the simplest failing test case.
		type	boolean
		default	true
	failure_blob	If set, the server runs exactly one test case, corresponding to the test case encoded in this blob. The standard phases are skipped. The blob here corresponds to the blob returned by test_done.results.failure_blobs.
		type	bytes

run_test_reply

reply bit	1
encoding	CBOR
payload	true

Test cases

Once a test run has been started, the server initiates the execution of test cases. The client uses the stream established in the run_test packet for packets related to this test run.

Client                                       Server
    |                                             |
 1  |<--[S1]-- test_case -------------------------|
 1R |---[S1]-- test_case_reply ------------------>|
    |                                             |
 2  |---[S2]-- command -------------------------->|
 2R |<--[S2]-- (command reply, if appropriate) ---|
    |---[S2]-- ... ------------------------------>|
    |                                             |
 3  |---[S2]-- mark_complete -------------------->|
 3R |<--[S2]-- mark_complete_reply ---------------|
    |                                             |
 4  |---[S2]-- stream_close --------------------->|
    |                                             |
    :       ... repeat for each test case         :
    |                                             |

In the test_case packet, it establishes a new stream (S2) which is used for packets for this test case.

test_case

reply bit	0
encoding	CBOR
fields	event	"test_case"
		type	text
		required	yes
	stream_id	The stream id for this test case.
		type	integer
		required	yes
	is_final	false during normal execution, true during final replays.
		type	boolean
		required	yes

test_case_reply

reply bit	1
encoding	CBOR
payload	null

During the test case, the client may send test case commands to the server. These command packets form the main interaction between the client and the server during the test case and are used to request generated values, mark spans of choices, etc.

When the client decides the test case is complete , it sends a mark_complete command on the test case stream with information describing the final status of the test case. The server replies with a mark_complete_reply packet.

mark_complete

reply bit	0
encoding	CBOR
fields	command	"mark_complete"
		type	text
		required	yes
	status	"VALID" (test case passed), "INVALID" (test case rejected, for example by via assume(false)), or "INTERESTING" (test case failed).
		type	text
		required	yes
	origin	A unique identifier for the failure. Should only be set when status is "INTERESTING". The server treats each unique origin as a separate failure, and will shrink and report each independently. The granularity of origin is up to the client. For example, if the client decides to report failure_line_number as its origin, any failure on that line, including those with a different exception type, will be treated by the server as the same failure. (If the client did want these to be treated as different failures, it might report (failure_line_number, exception_type) as its origin). Be careful not to include a string representation of the input in origin. If origin depends on the specific inputs given by the server, each shrink attempt will be treated as a unique failure, which is almost certainly not what you want.
		type	text

mark_complete_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

Test case commands

All commands are request packets sent by the client on the test case stream. The server replies with the corresponding reply packet.

Instead of the documented reply packets below, the server might also return command_error_reply to any command.

command_error_reply

reply bit	1
encoding	CBOR
fields	error	The error message.
		type	text
		required	yes
	type	The error type (for example, "ValueError").
		type	text
		required	yes

generate

Generate a value from a schema.

generate

reply bit	0
encoding	CBOR
fields	command	"generate"
		type	text
		required	yes
	schema	A generator schema.
		type	map
		required	yes

generate_reply

reply bit	1
encoding	CBOR
fields	result	The generated value.
		type	any
		required	yes

start_span

Mark the start of a span of choices.

start_span

reply bit	0
encoding	CBOR
fields	command	"start_span"
		type	text
		required	yes
	label	Identifies the span type.
		type	integer
		default	0

start_span_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

stop_span

Mark the end of a span of choices.

stop_span

reply bit	0
encoding	CBOR
fields	command	"stop_span"
		type	text
		required	yes
	discard	If true, this span is excluded from shrinking.
		type	boolean
		default	false

stop_span_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

target

Record an observation for targeted property-based testing.

target

reply bit	0
encoding	CBOR
fields	command	"target"
		type	text
		required	yes
	value	The observation value.
		type	float
		required	yes
	label	Identifies the observation.
		type	integer
		required	yes

target_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

new_collection

Create a new collection. Collections are used for the sizing of variable-length collections.

new_collection

reply bit	0
encoding	CBOR
fields	command	"new_collection"
		type	text
		required	yes
	min_size	Minimum number of elements.
		type	integer
		default	0
	max_size	Maximum number of elements.
		type	integer | null
		default	unbounded

new_collection_reply

reply bit	1
encoding	CBOR
fields	result	The collection id as an integer.
		type	text
		required	yes

collection_more

Ask whether the collection should produce another element.

collection_more

reply bit	0
encoding	CBOR
fields	command	"collection_more"
		type	text
		required	yes
	collection_id	The collection id.
		type	text
		required	yes

collection_more_reply

reply bit	1
encoding	CBOR
fields	result	true if another element should be produced, false if the collection is done.
		type	boolean
		required	yes

collection_reject

Indicate that the most recently produced element was not added to the collection.

collection_reject

reply bit	0
encoding	CBOR
fields	command	"collection_reject"
		type	text
		required	yes
	collection_id	The collection id.
		type	text
		required	yes

collection_reject_reply

reply bit	1
encoding	CBOR
fields	result	null
		type	null
		required	yes

new_pool

Create a new variable pool, which is a collection of identifiers intended to point to values that were generated or otherwise produced during testing. This is primarily useful for stateful testing, but can be used in normal tests too.

new_pool

reply bit	0
encoding	CBOR
fields	command	"new_pool"
		type	text
		required	yes

new_pool_reply

reply bit	1
encoding	CBOR
fields	result	The pool id (0-indexed).
		type	integer
		required	yes

pool_add

Add a new variable to a pool.

pool_add

reply bit	0
encoding	CBOR
fields	command	"pool_add"
		type	text
		required	yes
	pool_id	The pool to add to.
		type	integer
		required	yes

pool_add_reply

reply bit	1
encoding	CBOR
fields	result	The variable id.
		type	integer
		required	yes

pool_generate

Draw a variable from a pool. The server selects which variable to return.

pool_generate

reply bit	0
encoding	CBOR
fields	command	"pool_generate"
		type	text
		required	yes
	pool_id	The pool to draw from.
		type	integer
		required	yes
	consume	If true, the variable is removed from the pool after being returned.
		type	boolean
		default	false

pool_generate_reply

reply bit	1
encoding	CBOR
fields	result	The variable id. If the pool is empty, the server marks the test case as invalid.
		type	integer
		required	yes

Completing a test run

After some number of test cases, the server will consider the test run to be over. This might be because we hit the test_cases setting (communicated in the initial run_test packet), or because a test case found an error in the test function.

When the server decides the test run is finished, it sends a test_done packet on the test stream.

Client                        Server
    |                              |
 1  |<--[S1]-- test_done ----------|
 1R |---[S1]-- test_done_reply --->|
    |                              |

test_done

reply bit	0
encoding	CBOR
fields	event	"test_done"
		type	text
		required	yes
	results.passed	Whether the test passed.
		type	boolean
		required	yes
	results.test_cases	Total number of test cases executed.
		type	integer
		required	yes
	results.valid_test_cases	Number of valid test cases.
		type	integer
		required	yes
	results.invalid_test_cases	Number of invalid test cases.
		type	integer
		required	yes
	results.interesting_test_cases	Number of interesting (failing) test cases.
		type	integer
		required	yes
	results.seed	The random seed that was used, as a string.
		type	text
		required	yes
	results.failure_blobs	One entry per interesting test case, in the same order as the failures are reported. Each entry encodes its failing test case as an opaque blob of bytes. A blob can be sent as run_test.failure_blob to replay that specific failure.
		type	bytes[]
		required	yes
	results.flaky	Present if the test was detected to be flaky.
		type	text
	results.health_check_failure	Present if a health check failed.
		type	text
	results.error	Present if there was an error. This could be a usage error (e.g. invalid arguments) or indicate a server bug (e.g. unexpected exception raised).
		type	text

test_done_reply

reply bit	1
encoding	CBOR
payload	true

Final replays

After the test_done packet, the server sends a test_case packet with is_final set to true for each interesting test case. The intent is for the client to use this to replay the minimal failing test case. Each final replay follows the standard test case flow.

Stream close

For clarity at runtime, stream can be marked as closed when they are no longer needed.

stream_close

reply bit	0
encoding	raw
payload	The single byte 0xFE. The message id of this packet is always (1 << 31) - 1.

Schemas

Fields in bold are required.

Some fields list their type as schema. This means it accepts any generator schema. For example, {"type": "integer"} is a valid schema, so {"type": "one_of", "generators": [{"type": "integer"}]} is as well.

constant	Generate value.
	value	The value to return.
		type	any
		required	yes
one_of	Generate a value drawn from one of generators. Returned as a 2-element list [index, value], where index is the 0-based index into generators and value is the value drawn from generators[index].
	generators	Array of generators. Must be non-empty.
		type	schema[]
		required	yes
boolean	Generate either true or false.
	p	Probability of generating true. Must be in the range [0, 1].
		type	float
		default	0.5
integer	Generate an integer.
	min_value	Minimum value, inclusive.
		type	integer
		default	unbounded
	max_value	Maximum value, inclusive.
		type	integer
		default	unbounded
float	Generate a float.
	min_value	Minimum value.
		type	float
		default	unbounded
	max_value	Maximum value.
		type	float
		default	unbounded
	allow_nan	Whether NaN can be generated.
		type	boolean
		default	true if neither min_value nor max_value is set, false otherwise
	allow_infinity	Whether +Infinity and -Infinity can be generated.
		type	boolean
		default	true if either min_value or max_value is unset, false otherwise
	width	Float width. Either 32 or 64.
		type	integer
		default	64
	exclude_min	Exclude the minimum value.
		type	boolean
		default	false
	exclude_max	Exclude the maximum value.
		type	boolean
		default	false
fraction	Generate a rational number. Returned as a 2-element array [numerator, denominator], where both elements are integers and denominator > 0.
	min_value	Minimum value, inclusive.
		type	integer | float
		default	unbounded
	max_value	Maximum value, inclusive.
		type	integer | float
		default	unbounded
	max_denominator	Maximum denominator. Must be positive.
		type	integer
		default	unbounded
complex	Generate a complex number. Returned as a 2-element array [real, imaginary], where both elements are floats.
	min_magnitude	Minimum magnitude (abs(z)) of generated values.
		type	float
		default	0
	max_magnitude	Maximum magnitude (abs(z)) of generated values.
		type	float
		default	unbounded
	allow_nan	Whether NaN can be generated. Applies to both the real and imaginary part.
		type	boolean
		default	true if min_magnitude is 0 and max_magnitude is unset, false otherwise
	allow_infinity	Whether +Infinity and -Infinity can be generated. Applies to both the real and imaginary part.
		type	boolean
		default	true if max_magnitude is unset, false otherwise
	allow_subnormal	Whether subnormal numbers can be generated. Applies to both the real and imaginary part.
		type	boolean
		default	true
	width	Total width of the complex number in bits. One of 32, 64, or 128. For example, width=128 indicates both the real and imaginary parts are 64-bit.
		type	integer
		default	128
string	Generate a Unicode string. Returned as a custom CBOR tag 91, with a payload equivalent to the UTF-8 representation of the string except that surrogate code points are allowed. To exclude surrogates, for example because the client's programming language does not allow UTF-8 strings with surrogate code points, set exclude_categories to ["Cs"]. Characters are generated as follows: When no filtering rules are specified, any character can be generated. If min_codepoint or max_codepoint is specified, then only characters having a code point in that range will be generated. If categories is specified, then only characters from those Unicode categories will be generated. This is a further restriction, characters must also satisfy min_codepoint and max_codepoint. If exclude_categories is specified, then any character from those categories will not be generated. It is invalid to pass both categories and exclude_categories; these arguments are alternative ways to specify the same thing. If include_characters is specified, then any additional characters in that list will also be generated. If exclude_characters is specified, then any characters in that list will not be generated. It is invalid to have any overlap between include_characters and exclude_characters. If codec is specified, only characters encodable in that codec will be generated.
	min_size	Minimum length, in the number of code points.
		type	integer
		default	0
	max_size	Maximum length, in the number of code points.
		type	integer
		default	unbounded
	min_codepoint	Minimum Unicode code point, inclusive.
		type	integer
		default	unbounded
	max_codepoint	Maximum Unicode code point, inclusive.
		type	integer
		default	unbounded
	categories	Restrict generated characters to the given Unicode general categories. Each category can be either a one-letter major category (e.g. "L"), or a two-letter minor category (e.g. "Lu").
		type	text[]
		default	no restriction
	exclude_categories	Exclude characters in the given Unicode general categories.
		type	text[]
		default	no restriction
	include_characters	Characters that should always be allowed, even if they would otherwise be excluded by the other filters.
		type	text
		default	no restriction
	exclude_characters	Characters that should never be generated.
		type	text
		default	no restriction
	codec	If set, only characters encodable in the given codec will be generated. hegel-core accepts any name from Python's codecs module (e.g. "ascii", "utf-8", "latin-1").
		type	text
		default	no restriction
binary	Generate a byte string.
	min_size	Minimum length, in bytes.
		type	integer
		default	0
	max_size	Maximum length, in bytes.
		type	integer
		default	unbounded
regex	Generate a string that matches the given pattern regular expression.
	pattern	The regular expression to match.
		type	string
		required	yes
	fullmatch	If true, pattern must match the entire string. If false, pattern may match a substring.
		type	boolean
		default	false
	alphabet	These fields have the same meaning as the corresponding fields in string. The only difference is the default: if alphabet is not passed, it defaults here to {"codec": "utf-8"}.
		type	object
		default	Defaults to {"codec": "utf-8"}.
		min_codepoint Same meaning as min_codepoint in string. type integer default unbounded max_codepoint Same meaning as max_codepoint in string. type integer default unbounded categories Same meaning as categories in string. type text[] default no restriction exclude_categories Same meaning as exclude_categories in string. type text[] default no restriction include_characters Same meaning as include_characters in string. type text default no restriction exclude_characters Same meaning as exclude_characters in string. type text default no restriction codec Same meaning as codec in string. type text default no restriction
list	Generate a list of values from the elements generator.
	elements	The elements generator.
		type	schema
		required	yes
	min_size	Minimum number of elements.
		type	integer
		default	0
	max_size	Maximum number of elements.
		type	integer
		default	unbounded
	unique	If true, all generated list elements will be distinct.
		type	boolean
		default	false
dict	Generate a map with keys drawn from keys and values drawn from values. Returns in the format [[key1, value1], ...].
	keys	The keys generator.
		type	schema
		required	yes
	values	The values generator.
		type	schema
		required	yes
	min_size	Minimum number of map entries.
		type	integer
		default	0
	max_size	Maximum number of map entries.
		type	integer
		default	unbounded
tuple	Generate a fixed-length array, where each element is drawn from the corresponding generator at that position.
	elements	A list of generators of the same length as the desired tuple.
		type	schema[]
		required	yes
email	Generate an email address string, according to RFC 5322 Section 3.4.1.
url	Generate an http/https URL string, according to RFC 3986.
domain	Generate a fully qualified domain name string, according to RFC 1035.
	max_length	Maximum length of the domain name.
		type	integer
		default	255
ip_address	Generate an IP address string.
	version	The IP version. Must be 4 or 6.
		type	integer
		required	yes
date	Generate an ISO 8601 date string. For example: "2024-03-15".
time	Generate an ISO 8601 time string. For example: "14:30:00".
datetime	Generate an ISO 8601 datetime string. For example: "2024-03-15T14:30:00".
uuid	Generate a UUID string. For example: "550e8400-e29b-41d4-a716-446655440000".
	version	If set, generates a UUID of the corresponding RFC 4122 version. Must be one of 1, 2, 3, 4, 5.
		type	integer
		default	any version

Appendix

Reading sequence diagrams

We use the following conventions for the sequence diagrams in this document:

Client                                           Server
    |                                               |
 1  |---[S1]-- a ---------------------------------->|
 1R |<--[S1]-- a_reply -----------------------------|
    |                                               |
 2  |<--[S2]-- b -----------------------------------|
 2R |---[S2]-- b_reply ---------------------------->|
    |                                               |

Each named arrow --[C]-- name --> represents a packet, associated with stream C, being sent from either the client to the server (-->) or the server to the client (<--). name is purely illustrative and is not part of the spec.

Each packet is associated with an auto-incrementing number on the left. n indicates a request packet, and nR indicates a reply packet responding to packet n. Not all request packets expect a reply packet. These packet numbers are illustrative and not part of the spec, except for where they associate a specific request packet with a specific reply packet.

Footnotes

1. The benefit of the S bit is to allow both the client and server to create streams without coordinating with each other. ↩

2. The benefit of the message id is to support out-of-order replies that are associated with the same stream. ↩

3. For example, hegel-core uses process stdin and stdout as its transport layer. ↩