ppx_sexp_conv is a PPX syntax extension that generates code for
converting OCaml types to and from s-expressions, as defined in the
=sexplib= library. S-expressions are defined by the following type:
type sexp = Atom of string | List of sexp listand are rendered as parenthesized lists of strings, e.g. (This (is
an) (s expression)).
ppx_sexp_conv fits into the =ppx_deriving= framework, so you can
invoke it the same way you invoke any other deriving plug-in. Thus,
we can write
type int_pair = (int * int) [@@deriving sexp]to get two values defined automatically, sexp_of_int_pair and
int_pair_of_sexp. If we only want one direction, we can write one
of the following.
type int_pair = (int * int) [@@deriving sexp_of]
type int_pair = (int * int) [@@deriving of_sexp]These sexp-converters depend on having a set of converters for basic
values (e.g., int_of_sexp) already in scope. This can be done by
writing:
open Sexplib.StdIf you’re using =Core=, you can get the same effect with open Core.
It’s also possible to construct converters based on type expressions, i.e.:
[%sexp_of: (int * string) list] [1,"one"; 2,"two"]
|> Sexp.to_string;;
=> "((1 one) (2 two))"
[%sexp_of: (int * string) list] [1,"one"; 2,"two"]
|> [%of_sexp: (int * string) list];;
=> [1,"one"; 2,"two"]For %sexp_of, we can also omit the conversion of some types by
putting underscores for that type name.
[%sexp_of: (int * _) list] [1,"one"; 2,"two"]
|> Sexp.to_string;;
=> "((1 _)(2 _))"Converters for the implicit unboxed version of a record can be derived with the ~unboxed
flag. For example,
type t = { x : int } [@@deriving sexp ~unboxed]will give us sexp_of_t and t_of_sexp, as expected, along with
t_u_of_sexp : sexp -> t# and sexp_of_t_u : t# -> sexp.
The ~stackify flag generates an additional sexp_of_t__stack function
alongside the usual sexp_of_t. The __stack variant takes a local
input and produces a local sexp, allowing sexp conversion without heap
allocation.
type t = int * int * int [@@deriving sexp ~stackify]This produces:
val sexp_of_t : t -> Sexp.t
val sexp_of_t__stack : local_ t -> local_ Sexp.t
val t_of_sexp : Sexp.t -> tYou can also use it with [%sexp_of] via the [%sexp_of__stack] extension
point. Or with ppx-template ([@sexp_of: ...] [@alloc stack]).
The ~localize flag changes sexp_of_t to accept a local input while still
returning a heap-allocated sexp. This is more general than the default, as you
can still pass a global t to it. However, you may need extra type annotations
to coerce the local_ t -> Sexp.t back into a t -> Sexp.t.
type t = { a : int } [@@deriving sexp ~localize]This produces:
val sexp_of_t : local_ t -> Sexp.t
val t_of_sexp : Sexp.t -> tThe two flags can be combined:
type t = { a : int } [@@deriving sexp ~localize ~stackify]This produces both sexp_of_t : local_ t -> Sexp.t and
sexp_of_t__stack : local_ t -> local_ Sexp.t.
If ppx_sexp_conv can derive of_sexp, it can also generate a description of
the sexps that the resulting t_of_sexp would accept. This is the sexp grammar.
See Sexplib0.Sexp_grammar for details. Use [@@deriving sexp_grammar] to derive
the grammar for a type.
It is possible to construct sexp grammars directly from type expressions, e.g.,
[%sexp_grammar: (int, bool array) Either.t Base.Map.M(String).t]Use [@sexp_grammar.tag key = value], where (key : string) and (value :
Sexp.t), to annotate a grammar with a tag that can be inspected at runtime.
Use [@sexp_grammar.custom grammar] to override a type’s sexp grammar with
grammar.
Annotate a type with [@sexp_grammar.any] to use a stub grammar that accepts
any sexp. Alternately, write [@sexp_grammar.any desc] where (desc : string)
to use desc as a human-readable description for the stub grammar.
In the following, we’ll review the serialization rules for different OCaml types.
Basic types are represented as atoms. For numbers like int,
int32, int64, float, the string in the atom is what is accepted
the standard ocaml functions int_of_string, Int32.of_string, etc.
For the types char or string, the string in the atom is
respectively a one character string or the string itself.
OCaml-lists and arrays are represented as s-expression lists.
OCaml tuples are treated as lists of values in the same order as in
the tuple. The type unit is treated like a 0-tuple. e.g.:
(3.14, "foo", "bar bla", 27) => (3.14 foo "bar bla" 27)With options, None is treated as a zero-element list, and Some is
treated as a singleton list, as shown below.
None => ()
Some value => (value)We also support reading options following the ordinary rules for variants i.e.:
None => None
Some value => (Some value)The rules for variants are described below.
Records are represented as lists of lists, where each inner list is a key-value pair. Each pair consists of the name of the record field (first element), and its value (second element). e.g.:
{ foo = (3,4);
bar = "some string"; }
=> ((foo (3 4)) (bar "some string"))Type specifications of records allow the use of several attributes. The
attribute sexp.option indicates that an option record field should be optional, while
the attribute sexp.or_null indicates that an or_null record field should be optional.
e.g.:
type t =
{ x : int option;
y : int option [@sexp.option];
z : int or_null [@sexp.or_null];
} [@@deriving sexp]The following examples show how this works.
{ x = Some 1; y = Some 2; } => ((x (1)) (y 2))
{ x = None ; y = None; } => ((x ()))Note that, when present, an optional value is represented as the bare value, rather than explicitly as an option.
The attribute sexp.bool indicates that a boolean record field is shown
as either present or absent, but not as containing a value.
type t = { enabled : bool [@sexp.bool] } [@@deriving sexp]
{ enabled = true } => ((enabled))
{ enabled = false } => ()The attributes sexp.list and sexp.array indicate that a list or array record
field, respectively, can be omitted when it is empty.
type t =
{ arr : int array [@sexp.array]
; lst : int list [@sexp.list]
}
[@@deriving sexp]
{ arr = [||]; lst = [] } => ()
{ arr = [|1;2|]; lst = [3;4] } => ((arr (1 2)) (lst (3 4)))More complex default values can be specified explicitly using several constructs, e.g.:
type t =
{ a : int [@default 42];
b : int [@default 3] [@sexp_drop_default (=)];
c : int [@default 3] [@sexp_drop_if fun x -> x = 3];
d : int Queue.t [@sexp.omit_nil]
} [@@deriving sexp]The @default annotation lets one specify a default value to be
selected if the field is not specified, when converting from an
s-expression. The @sexp_drop_default annotation implies that the
field will be dropped when generating the s-expression if the value
being serialized is equal to the default according to the specified equality
function. @sexp_drop_if is like @sexp_drop_default, except that
it lets you specify the condition under which the field is dropped.
Finally, @sexp.omit_nil means to treat a missing field as if it
has value List [] when reading, and drop the field if it has value
List [] when writing.
The equality used by [@sexp_drop_default] is customizable. There are several ways to specify the equality function:
type t =
{ a : u [@default u0] [@sexp_drop_default (=)]; (* explicit user-provided function *)
b : u [@default u0] [@sexp_drop_default.compare]; (* uses [%compare.equal: u] *)
c : u [@default u0] [@sexp_drop_default.equal]; (* uses [%equal: u] *)
d : u [@default u0] [@sexp_drop_default.sexp]; (* compares sexp representations *)
} [@@deriving sexp]The @sexp.allow_extra_fields annotation lets one specify that the
sexp-converters should silently ignore extra fields, instead of
raising. This applies only to the record to which the annotation is
attached, and not to deeper sexp converters that may be called during
conversion of a sexp to the record.
type t = { a: int } [@@deriving sexp]
((a 0)(b b)) => exception
type t = { a: int } [@@deriving sexp] [@@sexp.allow_extra_fields]
((a 0)(b b)) => {a = 0}
type t = A of { a : int } [@sexp.allow_extra_fields] [@@deriving sexp]
(A (a 0)(b b)) => A {a = 0}Constant constructors in variants are represented as strings. Constructors with arguments are represented as lists, the first element being the constructor name, the rest being its arguments. Constructors may also be started in lowercase in S-expressions, but will always be converted to uppercase when converting from OCaml values.
For example:
type t = A | B of int * float * t [@@deriving sexp]
B (42, 3.14, B (-1, 2.72, A)) => (B 42 3.14 (B -1 2.72 A))The above example also demonstrates recursion in data structures.
Variants support the attribute sexp.list when a clause has a single
list as its argument.
type t =
| A of int list
| B of int list [@sexp.list]
A [1; 2; 3] => (A (1 2 3))
B [1; 2; 3] => (B 1 2 3)Constructors with inline records are represented as lists, the first element being the constructor name, the rest being the record fields, represented the same way as in record types, but without being wrapped in an extra layer of parentheses.
type t = A of { x : int }
A { x = 8 } => (A (x 8))Polymorphic variants behave almost the same as ordinary variants. The notable difference is that polymorphic variant constructors must always start with an either lower- or uppercase character, matching the way it was specified in the type definition. This is because OCaml distinguishes between upper and lowercase variant constructors. Note that type specifications containing unions of variant types are also supported by the S-expression converter, for example as in:
type ab = [ `A | `B ] [@@deriving sexp]
type cd = [ `C | `D ] [@@deriving sexp]
type abcd = [ ab | cd ] [@@deriving sexp]However, because `ppx_sexp_conv` needs to generate additional code to support inclusions of polymorphic variants, `ppx_sexp_conv` needs to know when processing a type definition whether it might be included in a polymorphic variant. `ppx_sexp_conv` will only generate the extra code automatically in the common case where the type definition is syntactically a polymorphic variant like in the example above. Otherwise, you will need to indicate it by using `[@@deriving sexp_poly]` (resp `of_sexp_poly`) instead of `[@@deriving sexp]` (resp `of_sexp`):
type ab = [ `A | `B ] [@@deriving sexp]
type alias_of_ab = ab [@@deriving sexp_poly]
type abcd = [ ab | `C | `D ] [@@deriving sexp]There is nothing special about polymorphic values as long as there are conversion functions for the type parameters. e.g.:
type 'a t = A | B of 'a [@@deriving sexp]
type foo = int t [@@deriving sexp]In the above case the conversion functions will behave as if foo had
been defined as a monomorphic version of t with 'a replaced by
int on the right hand side.
If a data structure is indeed polymorphic and you want to convert it,
you will have to supply the conversion functions for the type
parameters at runtime. If you wanted to convert a value of type 'a
t as in the above example, you would have to write something like
this:
sexp_of_t sexp_of_a vwhere sexp_of_a, which may also be named differently in this
particular case, is a function that converts values of type 'a to an
S-expression. Types with more than one parameter require passing
conversion functions for those parameters in the order of their
appearance on the left hand side of the type definition.
Opaque values are ones for which we do not want to perform
conversions. This may be, because we do not have S-expression
converters for them, or because we do not want to apply them in a
particular type context. e.g. to hide large, unimportant parts of
configurations. To prevent the preprocessor from generating calls to
converters, simply apply the attribute sexp.opaque to the type. If the type
is for a record field, it will likely need parentheses to avoid applying the
attribute to the record field itself, e.g.:
type foo = int * (stuff [@sexp.opaque]) [@@deriving sexp]
type bar =
{ a : int
; b : (stuff [@sexp.opaque])
}
[@@deriving sexp]Thus, there is no need to specify converters for type stuff, and if
there are any, they will not be used in this particular context.
Needless to say, it is not possible to convert such an S-expression
back to the original value. Here is an example conversion:
(42, some_stuff) => (42 <opaque>)When using ppx_sexp_conv on types that are functions, the derived
converters have limited behavior. For example:
type t = unit -> unit [@@deriving sexp]The derived sexp_of_t function will generate a dummy s-expression
<fun>, since functions cannot be meaningfully serialized. The
derived t_of_sexp function will raise an exception on all
s-expressions, since there is no way to deserialize a function from an
s-expression.
Phantom type parameters are ones which are marked to not actually appear on the
right-hand side of the type definition, and are thus excluded from the derived
sexp_of and of_sexp combinators. To prevent the preprocessor from including
type parameters when deriving or calling combinators, add the attribute
sexp.phantom to the type parameter itself. Here is an example:
type ('a[@sexp.phantom], 'b) index = 'b
[@@deriving sexp]
type 'a[@sexp.phantom] int_index = ('a[@sexp.phantom], int) index
[@@deriving sexp]The derived functions will have the following types. Note that none of them take a combinator for the phantom parameter as an argument.
val sexp_of_index : ('b -> Sexp.t) -> ('a, 'b) index -> Sexp.t
val index_of_sexp : (Sexp.t -> 'b) -> Sexp.t -> ('a, 'b) index
val sexp_of_int_index : 'a int_index -> Sexp.t
val int_index_of_sexp : Sexp.t -> 'a int_indexS-expression converters for exceptions can be automatically registered.
module M = struct
exception Foo of int [@@deriving sexp]
endSuch exceptions will be translated in a similar way as sum types, but
their constructor will be prefixed with the fully qualified module
path (here: M.Foo) so as to be able to discriminate between them
without problems.
The user can then easily convert an exception matching the above one
to an S-expression using sexp_of_exn. User-defined conversion
functions can be registered, too, by calling add_exn_converter.
This should make it very convenient for users to catch arbitrary
exceptions escaping their program and pretty-printing them, including
all arguments, as S-expressions. The library already contains
mappings for all known exceptions that can escape functions in the
OCaml standard library.
The Stdlib’s Hash tables, which are abstract values in OCaml, are represented as association lists, i.e. lists of key-value pairs, e.g.:
((foo 42) (bar 3))Reading in the above S-expression as hash table mapping strings to
integers ((string, int) Hashtbl.t) will map foo to 42 and bar
to 3.
Note that the order of elements in the list may matter, because the OCaml-implementation of hash tables keeps duplicates. Bindings will be inserted into the hash table in the order of appearance. Therefore, the last binding of a key will be the “visible” one, the others are “hidden”. See the OCaml documentation on hash tables for details.
In signatures, ppx_sexp_conv tries to generate an include of a named
interface, instead of a list of value bindings.
That is:
type 'a t [@@deriving sexp]will generate:
include Sexpable.S1 with type 'a t := 'a tinstead of:
val t_of_sexp : (Sexp.t -> 'a) -> Sexp.t -> 'a t
val sexp_of_t : ('a -> Sexp.t) -> 'a t -> Sexp.tThere are however a number of limitations:
- the type has to be named t
- the type can only have up to 3 parameters
- there shouldn’t be any constraint on the type parameters
If these aren’t met, then ppx_sexp_conv will simply generate a list of value
bindings.
In some cases, a type can meet all the conditions listed above, in which case the rewriting will apply, but lead to a type error. This happens when the type [t] is an alias to a type which does have constraints on the parameters, for instance:
type 'a s constraint 'a = [> `read ]
val sexp_of_s : ...
val s_of_sexp : ...
type 'a t = 'a s [@@deriving_inline sexp]
include Sexpable.S1 with type 'a t := 'a t
[@@@end]will give an error looking like:
Error: In this `with' constraint, the new definition of t
does not match its original definition in the constrained signature:
Type declarations do not match:
type 'a t = 'a t constraint 'a = [> `read ]
is not included in
type 'a t
File "sexpable.mli", line 8, characters 21-58: Expected declaration
Their constraints differ.
To workaround that error, simply copy the constraint on the type which has the
[@@deriving] annotation. This will force generating a list of value bindings.
Originally, ppx_sexp_conv used special types instead of attributes. Those
types have been replaced with attributes. Here are the appropriate conversions
to update from code using the old types to the new attributes.
Convert uses of sexp_opaque to uses of [@sexp.opaque]. The [@sexp.opaque]
attribute usually needs explicit parentheses to clarify what type it annotate.
Before:
type t = int sexp_opaque list
[@@deriving sexp]After:
type t = (int [@sexp.opaque]) list
[@@deriving sexp]Convert uses of sexp_option, sexp_list, sexp_array, and sexp_bool to
uses of [@sexp.option], [@sexp.list], [@sexp.array], and [@sexp.bool] as
appropriate. The attribute only specifies the modification, not the type, so you
will need to use the regular types option, list, array, and/or bool as
well. Unlike [@sexp.opaque], these attributes do not need extra parentheses.
Before:
type t =
{ a : int sexp_option
; b : int sexp_list
; c : int sexp_array
; d : sexp_bool
}
[@@deriving sexp]After:
type t =
{ a : int option [@sexp.option]
; b : int list [@sexp.list]
; c : int array [@sexp.array]
; d : bool [@sexp.bool]
}
[@@deriving sexp]Convert uses of sexp_list in variants and polymorphic variants to uses of
[@sexp.list]. You need to add the regular type list as well. Unlike
[@sexp.opaque], this attribute does not need extra parentheses.
Before:
type t = A of int sexp_list
[@@deriving sexp]
type u = [`B of int sexp_list]
[@@deriving sexp]After:
type t = A of int list [@sexp.list]
[@@deriving sexp]
type u = [`B of int list [@sexp.list]]
[@@deriving sexp]