Strict whitespace adherence option for JSON

[CoffeeVampir3]

An example json grammar, E.G:

          start: json_object
          json_object: %json {
            "type": "object",
            "properties": {
              "health": { 
                "type": "integer", 
                "minimum": 1, 
                "maximum": 100 
              },
              "mana": { 
                "type": "integer", 
                "minimum": 1, 
                "maximum": 100 
              },
              "attack": { 
                "type": "integer", 
                "minimum": 1, 
                "maximum": 100 
              },
              "defense": { 
                "type": "integer", 
                "minimum": 1, 
                "maximum": 100 
              },
              "agility": { 
                "type": "integer", 
                "minimum": 1, 
                "maximum": 100 
              },
              "description": {
                "type": "string"
              },
              "current": {
                "type": "string"
              },
              "current_thoughts": {
                "type": "array",
                "items": {
                  "type": "string"
                }
              }
            },
            "required": ["health", "mana", "attack", "defense", "agility", "description", "current", "current_thoughts"]
          }

This does not strictly enforce white spacing, and it doesn't seem to be possible to use this style of grammar to do so.

Here's an example literally output from an llm:

{"health": 100, "mana": 100, "attack": 20, "defense": 10, "agility": 15, "description": "You stand on the windswept cliff, the ocean crashing against the rugged stone beneath your feet. The sun beats down upon your skin, warming your bones, and illuminating the vast expanse of sea and sky.", "current": "standing", "current_thoughts": ["The beauty of the ocean, the thrill of the wind", "The vastness of the sea, the mystery of the unknown"]}"

Though this is technically correct, I'd like to propose a strict whitespace mode that enforces what you might consider "human readable json"

This is a grammar that produces such json, but it's significantly less readable for humans:

          start: json_object
          json_object: "{" NEWLINE TAB first_prop comma_props NEWLINE "}"

          first_prop: "\"health\"" ": " INTEGER
          comma_props: numeric_props string_props array_prop
          numeric_props: comma_prop comma_prop comma_prop comma_prop
          string_props: comma_prop comma_prop

          comma_prop: "," NEWLINE TAB property
          array_prop: "," NEWLINE TAB "\"current_thoughts\"" ": " array

          property: "\"mana\"" ": " INTEGER
                  | "\"attack\"" ": " INTEGER
                  | "\"defense\"" ": " INTEGER
                  | "\"agility\"" ": " INTEGER
                  | "\"description\"" ": " STRING
                  | "\"current\"" ": " STRING

          array: "[" NEWLINE TAB TAB first_string array_items NEWLINE TAB "]"
          first_string: STRING
          array_items: comma_string comma_string
          comma_string: "," NEWLINE TAB TAB STRING

          INTEGER: /([1-9][0-9]?|100)/
          STRING: "\"" /[^"]*/ "\""

          NEWLINE: "\n"
          TAB: "\t"

Example output:

{
        "health": 100,
        "mana": 100,
        "attack": 20,
        "defense": 10,
        "agility": 15,
        "description": "You stand at the edge of a cliff, gazing out at the endless ocean. The wind carries the cries of the gulls, and the waves crash against the shore. You feel a sense of wonder and excitement, and your heart begins to pound with anticipation. What do you do next?",
        "current": "gazing out at the horizon",
        "current_thoughts": [
                "the vastness of the ocean, the wind, the gulls, the waves",
                "your heart pounding with anticipation, your senses on high alert",
                "the wind carries the cries of the gulls, and the waves crash against the shore"
        ]
}

Having a standard strict-whitespacing I think would be a great feature to have around.

Cheers, really love the library!

[hudson-ai]

Hi @CoffeeVampir3! See the docs here for info on how to specify whitespace constraints on JSON outputs:
https://github.com/guidance-ai/llguidance/blob/main/docs/json_schema.md#whitespace-handling

Note that we do not (yet!) support "pretty-print" style constraints (i.e. setting some fixed indentation strategy like what you'd get from json.dump(..., indent=4) in python).

---

Now for some unsolicited thoughts / opinions:

I'd actually wager that forcing whitespace in a way that disagrees with "what the LM wants to do on its own" is going to degrade your output quality. Hopeful any such effect would be marginal, but any time your constraints disagree with the LM itself, you're risking throwing it off-distribution.

That being said, if you are using llguidance in a context that allows token-forcing, you can get some pretty significant speedups by setting whitespace_flexible: false and choosing some fixed item_separator and key_separator (e.g. ", " and ": "). This specific choice of separators will agree well with the LM for which you showed the output above.

Aside from quality, pretty-printed JSON is going to have much higher token usage than what your LM already wants to do. On the other extreme, you can force an even more "compact" output with no whitespace at all. But again... this will probably degrade quality.

In all though, I just want to point out that llguidance guarantees that your JSON will be well-formed. This means you can do things like "force one output format for the LM (or let it make that decision itself) but parse it as JSON and dump it to a format of your choice before displaying it to a user"!

[mmoskal]

Also, while for non-recursive schemas it's possible to hard-code whitespace in the grammar the way you did, for recursive ones we need special support in the lexer. This has been started in #107 but is far from complete...