Skip to content

Add support for tagless types in binary writer through standard API #992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

nirosys
Copy link
Contributor

@nirosys nirosys commented Aug 1, 2025

Issue #, if available: n/a

Description of changes:

NOTE: This PR includes the changes in #991.

This PR extends the binary writer API by adding a BinaryEExpParameterValueWriter_1_1 which becomes the ValueWriter implementation returned by the BinaryEExpWriter_1_1. The implementation wires up uint8 encoding, and continues supporting tagged encodings for all of the existing types that will ultimately support taggless encodings.

This extends the writer API to provide seamless handling of tagless encoding for code like:

//  (macro foo (uint8::x) (%x))
let mut eexp_writer = writer.eexp_writer("foo")?;
eexp_writer.write_int(&5.into())?;
eexp_writer.close()?;

To generate the expected:

$ ./target/release/ion inspect ~/Code/sandbox/tagless_output/output.10n
┌──────────────┬──────────────┬─────────────────────────┬──────────────────────┐
│    Offset    │    Length    │       Binary Ion        │       Text Ion       │
├──────────────┼──────────────┼─────────────────────────┼──────────────────────┘
│            0 │            4 │ e0 01 01 ea             │ $ion_1_1 // Version marker
├──────────────┼──────────────┼─────────────────────────┤
│            4 │            6 │ e7 f9 24 69 6f 6e       │ $ion:: // <text>
│           10 │           44 │ fc 55                   │ (
│           12 │            2 │ ee 10                   │ · module
│           14 │            2 │ a1 5f                   │ · _
│           16 │            5 │ c4                      │ · (
│           17 │            2 │ ee 0f                   │ · · symbol_table
│           19 │            2 │ a1 5f                   │ · · _
│              │              │                         │ · )
│           21 │           33 │ fc 3f                   │ · (
│           23 │            2 │ ee 0e                   │ · · macro_table
│           25 │            2 │ a1 5f                   │ · · _
│           27 │           27 │ fc 33                   │ · · (
│           29 │            6 │ a5 6d 61 63 72 6f       │ · · · macro
│           35 │            4 │ a3 66 6f 6f             │ · · · foo
│           39 │           10 │ c9                      │ · · · (
│           40 │            7 │ e7 f7 75 69 6e 74 38    │ · · · · uint8:: // <text>
│           47 │            2 │ a1 78                   │ · · · · x
│              │              │                         │ · · · )
│           49 │            5 │ c4                      │ · · · (
│           50 │            2 │ a1 25                   │ · · · · '%'
│           52 │            2 │ a1 78                   │ · · · · x
│              │              │                         │ · · · )
│              │              │                         │ · · )
│              │              │                         │ · )
│              │              │                         │ )
├──────────────┼──────────────┼─────────────────────────┤
│           54 │            2 │ 18                      │ (:foo
│           55 │            1 │ 05                      │ · 5 // x
│              │              │                         │ )
├──────────────┼──────────────┼─────────────────────────┤
│           55 │            1 │ 05                      │ 5
├──────────────┼──────────────┼─────────────────────────┤
│           56 │              │                         │  // End of stream
└──────────────┴──────────────┴─────────────────────────┘

When trying to write a value greater than 255, we get an IonError:

unable to write int: Encoding(EncodingError { description: "error with value provided for 'x': provided unsigned integer value does not fit within 1 byte" })

With this change, type checking and parameter cardinality is checked in a way that broke a couple unit tests. Originally two unit tests existed that wrote invocations for (:none foo bar baz), and (:make_string foo bar baz). The first unit test no longer works due to none not taking arguments. The second unit test fails because the eexp invocation requires building an expression group which currently is not implemented for binary ion. These issues now get caught, rather than writing the equivalent of (:none) foo bar baz. Once the binary e-group writer is implemented and wired up, we can shoot for matching the previous unit tests' desired API.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Copy link

codecov bot commented Aug 1, 2025

Codecov Report

❌ Patch coverage is 48.17708% with 199 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.28%. Comparing base (23e381a) to head (9e69c8a).

Files with missing lines Patch % Lines
src/lazy/encoder/binary/v1_1/value_writer.rs 33.66% 125 Missing and 9 partials ⚠️
src/lazy/encoder/binary/v1_1/container_writers.rs 9.67% 28 Missing ⚠️
src/lazy/encoder/writer.rs 59.25% 4 Missing and 18 partials ⚠️
src/lazy/expanded/macro_evaluator.rs 77.77% 0 Missing and 4 partials ⚠️
src/lazy/encoder/value_writer.rs 0.00% 3 Missing ⚠️
src/lazy/expanded/compiler.rs 88.88% 0 Missing and 2 partials ⚠️
src/lazy/expanded/macro_table.rs 33.33% 0 Missing and 2 partials ⚠️
src/lazy/binary/raw/v1_1/binary_buffer.rs 88.88% 1 Missing ⚠️
src/lazy/binary/raw/v1_1/e_expression.rs 90.90% 0 Missing and 1 partial ⚠️
src/lazy/encoder/binary/v1_1/fixed_uint.rs 88.88% 0 Missing and 1 partial ⚠️
... and 1 more

❌ Your patch check has failed because the patch coverage (48.17%) is below the target coverage (85.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #992      +/-   ##
==========================================
- Coverage   78.57%   78.28%   -0.29%     
==========================================
  Files         138      138              
  Lines       33674    34036     +362     
  Branches    33674    34036     +362     
==========================================
+ Hits        26458    26644     +186     
- Misses       5180     5329     +149     
- Partials     2036     2063      +27     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@nirosys
Copy link
Contributor Author

nirosys commented Aug 2, 2025

Code coverage for patch is pretty low due to adding code in prep for binary e-groups, and adjusting the tests that were previously using write_symbol. Coverage should go back up once more gets filled in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant