Miri detects memory leak in test_anchor_too_long test

[bourumir-wyngs]

    FAIL [  86.763s] (196/503) serde_yaml_bw::test_error test_anchor_too_long

stdout ───

running 1 test
test test_anchor_too_long ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 36 filtered out; finished in 101.28s

--> tests/test_error.rs:11:25
|
11 | use utils::{test_error, deserializer_no_pathology};
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: #[warn(unused_imports)] (part of #[warn(unused)]) on by default

error: memory leaked: alloc1753675 (Rust heap, size: 131080, align: 8), allocated here:
   --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/lib.rs:103:18
    |
103 |         memory = rust::realloc(memory, layout, new_size);
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: stack backtrace:
            0: unsafe_libyaml_norway::externs::realloc
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/lib.rs:103:18: 103:57
            1: unsafe_libyaml_norway::api::yaml_realloc
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/api.rs:31:9: 31:27
            2: unsafe_libyaml_norway::api::yaml_string_extend
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/api.rs:55:39: 58:6
            3: unsafe_libyaml_norway::scanner::READ
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/macros.rs:57:13: 61:14
            4: unsafe_libyaml_norway::scanner::yaml_parser_scan_anchor
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:115:9: 115:45
            5: unsafe_libyaml_norway::scanner::yaml_parser_fetch_anchor
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:861:8: 861:53
            6: unsafe_libyaml_norway::scanner::yaml_parser_fetch_next_token
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:285:16: 285:67
            7: unsafe_libyaml_norway::scanner::yaml_parser_fetch_more_tokens
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:205:12: 205:48
            8: unsafe_libyaml_norway::parser::PEEK_TOKEN
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:38:37: 38:74
            9: unsafe_libyaml_norway::parser::yaml_parser_parse_document_start
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:222:13: 222:31
            10: unsafe_libyaml_norway::parser::yaml_parser_state_machine
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:117:13: 117:66
            11: unsafe_libyaml_norway::parser::yaml_parser_parse
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:80:5: 80:45
            12: serde_yaml_bw::libyaml::parser::Parser::<'_>::next
                at src/libyaml/parser.rs:176:16: 176:53
            13: serde_yaml_bw::loader::Loader::<'_>::next_document
                at src/loader.rs:100:39: 100:52
            14: <serde_yaml_bw::Deserializer<'_> as std::iter::Iterator>::next
                at src/de.rs:313:32: 313:54
            15: <serde_yaml_bw::Deserializer<'_> as std::iter::Iterator>::next
                at src/de.rs:333:17: 333:28
            16: utils::test_error::<'_, serde_yaml_bw::Value>
                at tests/utils/mod.rs:44:35: 44:54
            17: test_anchor_too_long
                at tests/test_error.rs:321:5: 321:41
            18: test_anchor_too_long::{closure#0}
                at tests/test_error.rs:315:26: 315:26

error: memory leaked: alloc1754086 (Rust heap, size: 24, align: 8), allocated here:
  --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/lib.rs:87:22
   |
87 |         let memory = rust::alloc(layout);
   |                      ^^^^^^^^^^^^^^^^^^^
   |
   = note: stack backtrace:
           0: unsafe_libyaml_norway::externs::malloc
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/lib.rs:87:22: 87:41
           1: unsafe_libyaml_norway::api::yaml_malloc
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/api.rs:26:5: 26:17
           2: unsafe_libyaml_norway::scanner::yaml_parser_scan_plain_scalar
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/macros.rs:38:25: 38:40
           3: unsafe_libyaml_norway::scanner::yaml_parser_fetch_plain_scalar
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:917:8: 917:52
           4: unsafe_libyaml_norway::scanner::yaml_parser_fetch_next_token
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:327:16: 327:54
           5: unsafe_libyaml_norway::scanner::yaml_parser_fetch_more_tokens
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:205:12: 205:48
           6: unsafe_libyaml_norway::parser::PEEK_TOKEN
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:38:37: 38:74
           7: unsafe_libyaml_norway::parser::yaml_parser_parse_node
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:440:21: 440:39
           8: unsafe_libyaml_norway::parser::yaml_parser_state_machine
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:122:40: 122:90
           9: unsafe_libyaml_norway::parser::yaml_parser_parse
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:80:5: 80:45
           10: serde_yaml_bw::libyaml::parser::Parser::<'_>::next
               at src/libyaml/parser.rs:176:16: 176:53
           11: serde_yaml_bw::loader::Loader::<'_>::next_document
               at src/loader.rs:100:39: 100:52
           12: <serde_yaml_bw::Deserializer<'_> as std::iter::Iterator>::next
               at src/de.rs:313:32: 313:54
           13: <serde_yaml_bw::Deserializer<'_> as std::iter::Iterator>::next
               at src/de.rs:333:17: 333:28
           14: utils::test_error::<'_, serde_yaml_bw::Value>
               at tests/utils/mod.rs:44:35: 44:54
           15: test_anchor_too_long
               at tests/test_error.rs:321:5: 321:41
           16: test_anchor_too_long::{closure#0}
               at tests/test_error.rs:315:26: 315:26

error: memory leaked: alloc899657 (Rust heap, size: 24, align: 8), allocated here:
  --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/lib.rs:87:22
   |
87 |         let memory = rust::alloc(layout);
   |                      ^^^^^^^^^^^^^^^^^^^
   |
   = note: stack backtrace:
           0: unsafe_libyaml_norway::externs::malloc
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/lib.rs:87:22: 87:41
           1: unsafe_libyaml_norway::api::yaml_malloc
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/api.rs:26:5: 26:17
           2: unsafe_libyaml_norway::scanner::yaml_parser_scan_plain_scalar
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/macros.rs:38:25: 38:40
           3: unsafe_libyaml_norway::scanner::yaml_parser_fetch_plain_scalar
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:917:8: 917:52
           4: unsafe_libyaml_norway::scanner::yaml_parser_fetch_next_token
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:327:16: 327:54
           5: unsafe_libyaml_norway::scanner::yaml_parser_fetch_more_tokens
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:205:12: 205:48
           6: unsafe_libyaml_norway::parser::PEEK_TOKEN
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:38:37: 38:74
           7: unsafe_libyaml_norway::parser::yaml_parser_parse_node
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:440:21: 440:39
           8: unsafe_libyaml_norway::parser::yaml_parser_state_machine
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:122:40: 122:90
           9: unsafe_libyaml_norway::parser::yaml_parser_parse
               at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:80:5: 80:45
           10: serde_yaml_bw::libyaml::parser::Parser::<'_>::next
               at src/libyaml/parser.rs:176:16: 176:53
           11: serde_yaml_bw::loader::Loader::<'_>::next_document
               at src/loader.rs:100:39: 100:52
           12: serde_yaml_bw::Deserializer::<'_>::de::<serde_yaml_bw::Value, {closure@<serde_yaml_bw::Deserializer<'_> as _::_serde::Deserializer<'_>>::deserialize_any<serde_yaml_bw::value::de::<impl _::_serde::Deserialize<'de> for serde_yaml_bw::Value>::deserialize::ValueVisitor>::{closure#0}}>
               at src/de.rs:259:30: 259:52
           13: <serde_yaml_bw::Deserializer<'_> as _::_serde::Deserializer<'_>>::deserialize_any::<serde_yaml_bw::value::de::<impl _::_serde::Deserialize<'de> for serde_yaml_bw::Value>::deserialize::ValueVisitor>
               at src/de.rs:354:9: 354:56
           14: serde_yaml_bw::value::de::<impl _::_serde::Deserialize<'_> for serde_yaml_bw::Value>::deserialize::<serde_yaml_bw::Deserializer<'_>>
               at src/value/de.rs:146:9: 146:51
           15: utils::test_error::<'_, serde_yaml_bw::Value>
               at tests/utils/mod.rs:40:18: 40:82
           16: test_anchor_too_long
               at tests/test_error.rs:321:5: 321:41
           17: test_anchor_too_long::{closure#0}
               at tests/test_error.rs:315:26: 315:26

error: memory leaked: alloc899210 (Rust heap, size: 131080, align: 8), allocated here:
   --> /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/lib.rs:103:18
    |
103 |         memory = rust::realloc(memory, layout, new_size);
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: stack backtrace:
            0: unsafe_libyaml_norway::externs::realloc
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/lib.rs:103:18: 103:57
            1: unsafe_libyaml_norway::api::yaml_realloc
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/api.rs:31:9: 31:27
            2: unsafe_libyaml_norway::api::yaml_string_extend
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/api.rs:55:39: 58:6
            3: unsafe_libyaml_norway::scanner::READ
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/macros.rs:57:13: 61:14
            4: unsafe_libyaml_norway::scanner::yaml_parser_scan_anchor
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:115:9: 115:45
            5: unsafe_libyaml_norway::scanner::yaml_parser_fetch_anchor
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:861:8: 861:53
            6: unsafe_libyaml_norway::scanner::yaml_parser_fetch_next_token
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:285:16: 285:67
            7: unsafe_libyaml_norway::scanner::yaml_parser_fetch_more_tokens
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/scanner.rs:205:12: 205:48
            8: unsafe_libyaml_norway::parser::PEEK_TOKEN
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:38:37: 38:74
            9: unsafe_libyaml_norway::parser::yaml_parser_parse_document_start
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:222:13: 222:31
            10: unsafe_libyaml_norway::parser::yaml_parser_state_machine
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:117:13: 117:66
            11: unsafe_libyaml_norway::parser::yaml_parser_parse
                at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/unsafe-libyaml-norway-0.2.15/src/parser.rs:80:5: 80:45
            12: serde_yaml_bw::libyaml::parser::Parser::<'_>::next
                at src/libyaml/parser.rs:176:16: 176:53
            13: serde_yaml_bw::loader::Loader::<'_>::next_document
                at src/loader.rs:100:39: 100:52
            14: serde_yaml_bw::Deserializer::<'_>::de::<serde_yaml_bw::Value, {closure@<serde_yaml_bw::Deserializer<'_> as _::_serde::Deserializer<'_>>::deserialize_any<serde_yaml_bw::value::de::<impl _::_serde::Deserialize<'de> for serde_yaml_bw::Value>::deserialize::ValueVisitor>::{closure#0}}>
                at src/de.rs:259:30: 259:52
            15: <serde_yaml_bw::Deserializer<'_> as _::_serde::Deserializer<'_>>::deserialize_any::<serde_yaml_bw::value::de::<impl _::_serde::Deserialize<'de> for serde_yaml_bw::Value>::deserialize::ValueVisitor>
                at src/de.rs:354:9: 354:56
            16: serde_yaml_bw::value::de::<impl _::_serde::Deserialize<'_> for serde_yaml_bw::Value>::deserialize::<serde_yaml_bw::Deserializer<'_>>
                at src/value/de.rs:146:9: 146:51
            17: utils::test_error::<'_, serde_yaml_bw::Value>
                at tests/utils/mod.rs:40:18: 40:82
            18: test_anchor_too_long
                at tests/test_error.rs:321:5: 321:41
            19: test_anchor_too_long::{closure#0}
                at tests/test_error.rs:315:26: 315:26

note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace

What is happening

This is not a test assertion failure. The test body succeeds:

test test_anchor_too_long ... ok

but miri reports leaked allocations afterward, so the real failure is:

error: memory leaked

The leaked allocations come from unsafe-libyaml-norway while scanning the oversized anchor:

• yaml_parser_scan_anchor
• yaml_string_extend
• yaml_parser_scan_plain_scalar

That tells you the parser allocates scanner buffers for the huge anchor and following scalar, then your code exits with an error before those temporary allocations are fully reclaimed.

Why this happens

The critical path is in src/libyaml/parser.rs:

let ret = convert_event(&*event, &(*self.pin.ptr).input).map_err(error::new)?;
sys::yaml_event_delete(event);
Ok((ret, mark))

If convert_event(...) returns Err(...) — which it does for an anchor longer than MAX_ANCHOR_LEN in optional_anchor(...) — then the ? returns early and sys::yaml_event_delete(event) is never called.

So the immediate bug is:

• yaml_parser_parse successfully created an event
• convert_event rejected it
• the event is not deleted on that error path

That is the reason for the leak report.

Why the stack trace looks like a libyaml leak

Even though the allocations are reported inside unsafe-libyaml-norway, that does not mean the dependency is necessarily at fault.

yaml_event_t owns memory allocated by libyaml. If your wrapper fails to call yaml_event_delete, miri will attribute the leaked allocations to the original allocator inside the dependency.

So the leak is very plausibly in serde_yaml_bw's wrapper logic, not in unsafe-libyaml-norway itself.

The fix

In Parser::next, make sure yaml_event_delete(event) runs even when convert_event fails.

Conceptually, change from this shape:

let ret = convert_event(&*event, &(*self.pin.ptr).input).map_err(error::new)?;
let mark = Mark { sys: (*event).start_mark };
sys::yaml_event_delete(event);
Ok((ret, mark))

to this shape:

let mark = Mark { sys: (*event).start_mark };
let ret = convert_event(&*event, &(*self.pin.ptr).input).map_err(error::new);
sys::yaml_event_delete(event);
ret.map(|ret| (ret, mark))

That way the event is always deleted after a successful yaml_parser_parse, regardless of whether conversion succeeds.

Why this matches test_anchor_too_long

Your test does this:

const MAX: usize = 65_536;
let long = "a".repeat(MAX + 1);
let yaml = format!("&{long} 1\n");
test_error::<Value>(&yaml, expected);

And optional_anchor(...) contains:

if bytes.len() > MAX_ANCHOR_LEN {
    return Err(ErrorImpl::Message(...));
}

So the event is parsed, then rejected during event conversion, which is exactly the path where the early return skips yaml_event_delete.

Diagnosis

The correct diagnosis is:

• test_anchor_too_long is failing under miri because Parser::next leaks a yaml_event_t when convert_event returns an error.
• The leak is triggered by the oversized anchor check in optional_anchor(...).
• sys::yaml_event_delete(event) must be executed even on the error path after yaml_parser_parse succeeds.

SPatch outline

Update src/libyaml/parser.rs in Parser::next so that:

1. yaml_parser_parse succeeds
2. mark is captured
3. convert_event(...) is evaluated without ?
4. sys::yaml_event_delete(event) is always called
5. the conversion result is returned afterward

In one sentence

The bug is an early return before yaml_event_delete, not the anchor-length check itself.

[bourumir-wyngs]

The fix is in commit 9340ffa6f9f60c64461509fbb901648ef90304b7

[bourumir-wyngs]

The history

The bug was introduced by commit 15d0ef2810939574b7003df84330165ab5947a28
(Increase name length limit (#50).

Before that commit, Parser::next always deleted the libyaml event after
calling convert_event, and convert_event itself was infallible in this
path, so there was no way to return early before yaml_event_delete(event).

That commit changed convert_event to return a Result and updated
Parser::next to use:

let ret = convert_event(&*event, &(*self.pin.ptr).input)
    .map_err(|_| error::new(ErrorImpl::TagError))?;
let mark = Mark {
    sys: (*event).start_mark,
};
sys::yaml_event_delete(event);
Ok((ret, mark))

The newly added ? is the introducing change. Once convert_event could fail
for cases such as an oversized anchor, Parser::next could return early after
yaml_parser_parse had already created an event but before
yaml_event_delete(event) was called. That skipped cleanup of memory owned by
the libyaml event and is exactly the leak reported by Miri.

So this commit is the introducing commit because it is the first revision where:

• convert_event became fallible on this path, and
• Parser::next could exit via ? before yaml_event_delete(event).

The later fix in 9340ffa6f9f60c64461509fbb901648ef90304b7 resolves this by
storing the conversion result first, always deleting the event, and only then
returning the Result.