Skip to content

Harden reflection Verifier against out-of-bounds reads on truncated buffers, vtable offsets, and unions - #9430

Open
professor-moody wants to merge 1 commit into
google:masterfrom
professor-moody:fix-reflection-verifier-bounds
Open

professor-moody wants to merge 1 commit into
google:masterfrom
professor-moody:fix-reflection-verifier-bounds

Conversation

@professor-moody

Copy link
Copy Markdown

Summary

This PR hardens the binary-schema reflection verifier in src/reflection.cpp (flatbuffers::Verify, flatbuffers::VerifySizePrefixed, VerifyObject, VerifyVector, and VerifyUnion) against out-of-bounds reads, null pointer dereferences, and verification bypasses when validating untrusted FlatBuffer payloads against a reflection::Schema:

  1. flatbuffers::Verify and flatbuffers::VerifySizePrefixed root offset and size-prefix validation (src/reflection.cpp):

    • flatbuffers::Verify previously called flatbuffers::GetAnyRoot(buf) before verifying buf != nullptr, length >= FLATBUFFERS_MIN_BUFFER_SIZE, or v.VerifyOffset<uoffset_t>(0U). Passing nullptr or a buffer smaller than 4 bytes performed an out-of-bounds 4-byte read in GetAnyRoot(buf), and a self-referential (0) or negative root offset was not checked by VerifyOffset<uoffset_t>(0U).
    • flatbuffers::VerifySizePrefixed previously called flatbuffers::GetAnySizePrefixedRoot(buf) without verifying buf != nullptr, length >= sizeof(uoffset_t) + FLATBUFFERS_MIN_BUFFER_SIZE, or checking that the 4-byte size prefix ReadScalar<uoffset_t>(buf) lies within [FLATBUFFERS_MIN_BUFFER_SIZE, length - sizeof(uoffset_t)]. It also verified offsets against the outer length rather than clamping the verifier length to sizeof(uoffset_t) + static_cast<size_t>(prefixed_size).
    • Updated Verify to check if (!buf || length < FLATBUFFERS_MIN_BUFFER_SIZE) return false; and if (!v.VerifyOffset<uoffset_t>(0U)) return false; before calling GetAnyRoot(buf). Updated VerifySizePrefixed to validate prefixed_size against [FLATBUFFERS_MIN_BUFFER_SIZE, length - sizeof(uoffset_t)], keep Verifier v(buf, sizeof(uoffset_t) + static_cast<size_t>(prefixed_size), max_depth, max_tables) anchored at buf (so 8-byte field alignment is measured from the true start of a FinishSizePrefixed buffer, matching VerifierTemplate::VerifySizePrefixedBuffer), and check if (!v.VerifyOffset<uoffset_t>(sizeof(uoffset_t))) return false; before calling VerifyObject(v, schema, root, flatbuffers::GetAnySizePrefixedRoot(buf), true).
  2. VerifyObject offset and required() validation on Obj, Union, and String fields (src/reflection.cpp):

    • In VerifyObject, case reflection::Obj: (when !child_obj->is_struct()) called flatbuffers::GetFieldT(*table, *field_def) directly without first calling table->VerifyOffset(v, field_def->offset()) (or VerifyOffsetRequired). A vtable slot pointing outside the buffer caused table->GetPointer<const Table*>(field.offset()) to read a 4-byte uoffset_t out of bounds before recursive VerifyObject ran.
    • case reflection::Union: called table->GetField<uint8_t>(utype_offset, 0) and flatbuffers::GetFieldT(*table, *field_def) without verifying utype_offset or field_def->offset(). Because obj.fields() in .bfbs schemas is sorted alphabetically by field name (key), a union value field "u" is visited before its companion "u_type" field in VerifyObject, so utype_offset has not yet been verified when case reflection::Union: executes.
    • case reflection::String: used table->VerifyField<uoffset_t>(v, field_def->offset(), sizeof(uoffset_t)) instead of VerifyOffset / VerifyOffsetRequired, which did not enforce field_def->required() when a required string field was omitted from the vtable and did not validate the uoffset_t target before GetFieldS resolved the pointer.
    • Updated String, non-struct Obj, and Union fields in VerifyObject to validate field offsets with VerifyOffsetRequired / VerifyOffset (and VerifyField<uint8_t> for utype_offset, after checking field_def->offset() >= sizeof(voffset_t)) prior to dereferencing.
  3. VerifyVector offset validation, required() enforcement, and union type-vector verification (src/reflection.cpp):

    • VerifyVector previously checked table.VerifyField<uoffset_t>(v, vec_field.offset(), sizeof(uoffset_t)) rather than VerifyOffsetRequired / VerifyOffset, ignoring vec_field.required() when a required non-struct vector was omitted from the vtable and not validating the vector offset before GetFieldV / GetFieldAnyV resolved the pointer.
    • In case reflection::Union:, VerifyVector called table.GetPointer<Vector<uint8_t>*>(vec_field.offset() - sizeof(voffset_t)) without verifying the companion _type vector offset (table.VerifyOffsetRequired(v, utype_vec_offset)). When the companion _type vector vtable slot pointed out of bounds, GetPointer read 4 bytes out of bounds; when the _type vector vtable slot was 0 (type_vec == nullptr), v.VerifyVector(type_vec) returned true and type_vec->size() immediately dereferenced a null pointer.
    • In case reflection::String:, case reflection::Obj:, and case reflection::Union:, element offset entries inside the vector (vec->Data() + j * sizeof(uoffset_t)) are now validated with v.VerifyOffset(vec->Data() + j * sizeof(uoffset_t), 0) before vec->Get(j) resolves the element offset.
  4. VerifyUnion consistency, sparse enum lookup, and struct alignment (src/reflection.cpp):

    • VerifyUnion now checks if (!utype) return !elem; and if (!elem) return false;, rejecting both (utype == 0 && elem != nullptr) and (utype != 0 && elem == nullptr). Note that rejecting both inconsistent states is intentionally stricter than flatc-generated union verifiers: existing reflection consumers in src/reflection.cpp (CopyTable and ResizeContext) check table.CheckField(fielddef.offset()) on the union value field and call the existing GetUnionType helper (which indexes schema.objects()->Get(-1) when utype == 0, as in [C++] Object API: Pack() dereferences null when a union has its type set but no value (buffer passes the verifier) #9416 and Fix null dereference in generated C++ union Pack when value is absent #9417), and when utype != 0 && elem == nullptr, v.VerifyString(nullptr) previously returned true for null String union members.
    • VerifyUnion previously indexed fb_enum->values()->Get(utype) by raw position instead of fb_enum->values()->LookupByKey(utype) (consistent with existing GetUnionType in src/reflection.cpp) and did not check enumval->union_type() != nullptr.
    • For struct union members (elem_obj->is_struct()), VerifyUnion now calls v.VerifyFieldStruct(elem, 0, elem_obj->bytesize(), elem_obj->minalign()) so struct alignment is verified in addition to byte size.

Testing

  • Added regression tests in tests/reflection_test.cpp (ReflectionTest) covering null, < 4-byte, and < 8-byte buffers, positive FinishSizePrefixed Monster verification and size-prefixed frame mismatches, out-of-bounds vtable slots on sub-table (Obj) and union (UType / Union) fields, omitted required String fields, and inconsistent union utype / elem pairs.
  • Verified with a standalone C++17 regression suite across Release (-O2 -DNDEBUG) and AddressSanitizer + UndefinedBehaviorSanitizer (-fsanitize=address,undefined -fno-omit-frame-pointer -g -O1) builds.

…uffers, vtable offsets, and unions

## Summary

This PR hardens the binary-schema reflection verifier in `src/reflection.cpp` (`flatbuffers::Verify`, `flatbuffers::VerifySizePrefixed`, `VerifyObject`, `VerifyVector`, and `VerifyUnion`) against out-of-bounds reads, null pointer dereferences, and verification bypasses when validating untrusted FlatBuffer payloads against a `reflection::Schema`:

1. **`flatbuffers::Verify` and `flatbuffers::VerifySizePrefixed` root offset and size-prefix validation (`src/reflection.cpp`)**:
   - `flatbuffers::Verify` previously called `flatbuffers::GetAnyRoot(buf)` before verifying `buf != nullptr`, `length >= FLATBUFFERS_MIN_BUFFER_SIZE`, or `v.VerifyOffset<uoffset_t>(0U)`. Passing `nullptr` or a buffer smaller than 4 bytes performed an out-of-bounds 4-byte read in `GetAnyRoot(buf)`, and a self-referential (`0`) or negative root offset was not checked by `VerifyOffset<uoffset_t>(0U)`.
   - `flatbuffers::VerifySizePrefixed` previously called `flatbuffers::GetAnySizePrefixedRoot(buf)` without verifying `buf != nullptr`, `length >= sizeof(uoffset_t) + FLATBUFFERS_MIN_BUFFER_SIZE`, or checking that the 4-byte size prefix `ReadScalar<uoffset_t>(buf)` lies within `[FLATBUFFERS_MIN_BUFFER_SIZE, length - sizeof(uoffset_t)]`. It also verified offsets against the outer `length` rather than clamping the verifier length to `sizeof(uoffset_t) + static_cast<size_t>(prefixed_size)`.
   - Updated `Verify` to check `if (!buf || length < FLATBUFFERS_MIN_BUFFER_SIZE) return false;` and `if (!v.VerifyOffset<uoffset_t>(0U)) return false;` before calling `GetAnyRoot(buf)`. Updated `VerifySizePrefixed` to validate `prefixed_size` against `[FLATBUFFERS_MIN_BUFFER_SIZE, length - sizeof(uoffset_t)]`, keep `Verifier v(buf, sizeof(uoffset_t) + static_cast<size_t>(prefixed_size), max_depth, max_tables)` anchored at `buf` (so 8-byte field alignment is measured from the true start of a `FinishSizePrefixed` buffer, matching `VerifierTemplate::VerifySizePrefixedBuffer`), and check `if (!v.VerifyOffset<uoffset_t>(sizeof(uoffset_t))) return false;` before calling `VerifyObject(v, schema, root, flatbuffers::GetAnySizePrefixedRoot(buf), true)`.

2. **`VerifyObject` offset and `required()` validation on `Obj`, `Union`, and `String` fields (`src/reflection.cpp`)**:
   - In `VerifyObject`, `case reflection::Obj:` (when `!child_obj->is_struct()`) called `flatbuffers::GetFieldT(*table, *field_def)` directly without first calling `table->VerifyOffset(v, field_def->offset())` (or `VerifyOffsetRequired`). A vtable slot pointing outside the buffer caused `table->GetPointer<const Table*>(field.offset())` to read a 4-byte `uoffset_t` out of bounds before recursive `VerifyObject` ran.
   - `case reflection::Union:` called `table->GetField<uint8_t>(utype_offset, 0)` and `flatbuffers::GetFieldT(*table, *field_def)` without verifying `utype_offset` or `field_def->offset()`. Because `obj.fields()` in `.bfbs` schemas is sorted alphabetically by field name (`key`), a union value field `"u"` is visited before its companion `"u_type"` field in `VerifyObject`, so `utype_offset` has not yet been verified when `case reflection::Union:` executes.
   - `case reflection::String:` used `table->VerifyField<uoffset_t>(v, field_def->offset(), sizeof(uoffset_t))` instead of `VerifyOffset` / `VerifyOffsetRequired`, which did not enforce `field_def->required()` when a required string field was omitted from the vtable and did not validate the `uoffset_t` target before `GetFieldS` resolved the pointer.
   - Updated `String`, non-struct `Obj`, and `Union` fields in `VerifyObject` to validate field offsets with `VerifyOffsetRequired` / `VerifyOffset` (and `VerifyField<uint8_t>` for `utype_offset`, after checking `field_def->offset() >= sizeof(voffset_t)`) prior to dereferencing.

3. **`VerifyVector` offset validation, `required()` enforcement, and union type-vector verification (`src/reflection.cpp`)**:
   - `VerifyVector` previously checked `table.VerifyField<uoffset_t>(v, vec_field.offset(), sizeof(uoffset_t))` rather than `VerifyOffsetRequired` / `VerifyOffset`, ignoring `vec_field.required()` when a required non-struct vector was omitted from the vtable and not validating the vector offset before `GetFieldV` / `GetFieldAnyV` resolved the pointer.
   - In `case reflection::Union:`, `VerifyVector` called `table.GetPointer<Vector<uint8_t>*>(vec_field.offset() - sizeof(voffset_t))` without verifying the companion `_type` vector offset (`table.VerifyOffsetRequired(v, utype_vec_offset)`). When the companion `_type` vector vtable slot pointed out of bounds, `GetPointer` read 4 bytes out of bounds; when the `_type` vector vtable slot was `0` (`type_vec == nullptr`), `v.VerifyVector(type_vec)` returned `true` and `type_vec->size()` immediately dereferenced a null pointer.
   - In `case reflection::String:`, `case reflection::Obj:`, and `case reflection::Union:`, element offset entries inside the vector (`vec->Data() + j * sizeof(uoffset_t)`) are now validated with `v.VerifyOffset(vec->Data() + j * sizeof(uoffset_t), 0)` before `vec->Get(j)` resolves the element offset.

4. **`VerifyUnion` consistency, sparse enum lookup, and struct alignment (`src/reflection.cpp`)**:
   - `VerifyUnion` now checks `if (!utype) return !elem;` and `if (!elem) return false;`, rejecting both `(utype == 0 && elem != nullptr)` and `(utype != 0 && elem == nullptr)`. Note that rejecting both inconsistent states is intentionally stricter than `flatc`-generated union verifiers: existing reflection consumers in `src/reflection.cpp` (`CopyTable` and `ResizeContext`) check `table.CheckField(fielddef.offset())` on the union value field and call the existing `GetUnionType` helper (which indexes `schema.objects()->Get(-1)` when `utype == 0`, as in google#9416 and google#9417), and when `utype != 0 && elem == nullptr`, `v.VerifyString(nullptr)` previously returned `true` for null `String` union members.
   - `VerifyUnion` previously indexed `fb_enum->values()->Get(utype)` by raw position instead of `fb_enum->values()->LookupByKey(utype)` (consistent with existing `GetUnionType` in `src/reflection.cpp`) and did not check `enumval->union_type() != nullptr`.
   - For struct union members (`elem_obj->is_struct()`), `VerifyUnion` now calls `v.VerifyFieldStruct(elem, 0, elem_obj->bytesize(), elem_obj->minalign())` so struct alignment is verified in addition to byte size.

## Testing

- Added regression tests in `tests/reflection_test.cpp` (`ReflectionTest`) covering null, `< 4`-byte, and `< 8`-byte buffers, positive `FinishSizePrefixed` `Monster` verification and size-prefixed frame mismatches, out-of-bounds vtable slots on sub-table (`Obj`) and union (`UType` / `Union`) fields, omitted required `String` fields, and inconsistent union `utype` / `elem` pairs.
- Verified with a standalone C++17 regression suite across Release (`-O2 -DNDEBUG`) and AddressSanitizer + UndefinedBehaviorSanitizer (`-fsanitize=address,undefined -fno-omit-frame-pointer -g -O1`) builds.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c++ codegen Involving generating code from schema

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant