Buffer errors

[troepolik]

Describe the bug

We faced with very rare errors in consuming :

System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'length')
   at System.ThrowHelper.ThrowStartOrEndArgumentValidationException(Int64 start)
   at System.Buffers.ReadOnlySequence`1.Slice(Int64 start, Int64 length)
   at RabbitMQ.Stream.Client.RawConsumer.<>c__DisplayClass19_0.<ParseChunk>g__MessageFromSequence|0(ReadOnlySequence`1& unCompressedData, Int32& compressOffset) in /_/RabbitMQ.Stream.Client/RawConsumer.cs:line 269

Error | | Error while parsing message Consumer id 0 for stream: xxx, identifier: , reference: xxx, OffsetSpec RabbitMQ.Stream.Client.OffsetTypeOffset Client ProvidedName xxx, No SuperStream, IsSingleActiveConsumer: False, Token IsCancellationRequested: False . The message will be skipped. Please report this issue to the RabbitMQ team on GitHub https://github.com/rabbitmq/rabbitmq-stream-dotnet-client/issues

Issue looks like corrupted internal buffer or mb buffer that used by few threads in the same time.
I dont know if it related or not - we have few parallel consumers, each consumes own queue.

Some times we also have some different type of error, but it also look like corrupted or used by few threads buffer:

System.Text.Json.JsonException: 'r' is an invalid end of a number. Expected a delimiter. Path: $ | LineNumber: 0 | BytePositionInLine: 7850.

We control message body, so I can guarantee that we publish proper json.
We deserialize messages like that:

Utf8JsonReader reader = new(message.Data.Contents);
var msg = JsonSerializer.Deserialize<TMessage>(ref reader);

client version : 1.8.14 - latest
server: RabbitMQ 3.12.0, Erlang 26.2.5

Reproduction steps

1. Error happens very rare - once per 2 days , so I can not reproduce it locally

Expected behavior

I expect consuming without errors

Additional context

No response

[Gsantomaggio]

@troepolik thank you for reporting the issue. Do you have the same error if you re-consume the same messages?

I dont know if it related or not - we have few parallel consumers, each consumes own queue.

What do you mean? Can you explain better? ty

used by few threads buffer

What do you mean here?
By the library, or by your code?

[Gsantomaggio]

Can you enable the Crc32? here an example:

rabbitmq-stream-dotnet-client/Tests/Crc32.cs

Line 9 in b5d784f

public class Crc32 : ICrc32

[troepolik]

Do you have the same error if you re-consume the same messages?

@Gsantomaggio As I understand, error happens inside library and message was skiped, so we dont have possibility to make retry.
"The message will be skipped."
But seccond error about json deserialization - I'l try to make retry there.

I dont know if it related or not - we have few parallel consumers, each consumes own queue.
What do you mean? Can you explain better?

We have few queues and they are consumed by few consumers in parallel in one application.

used by few threads buffer
What do you mean here? By the library, or by your code?

By library. error stack trace is inside library

System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'length')
   at System.ThrowHelper.ThrowStartOrEndArgumentValidationException(Int64 start)
   at System.Buffers.ReadOnlySequence`1.Slice(Int64 start, Int64 length)
   at RabbitMQ.Stream.Client.RawConsumer.<>c__DisplayClass19_0.<ParseChunk>g__MessageFromSequence|0(ReadOnlySequence`1& unCompressedData, Int32& compressOffset) in /_/RabbitMQ.Stream.Client/RawConsumer.cs:line 269

[troepolik]

Can you enable the Crc32? here an example:

@Gsantomaggio thank you! I'l try

[Gsantomaggio]

@troepolik, I am unable to reproduce the issue.
Please

• Let me know if the CRC fails
• Provide a code on how you use the library
• Check the server logs if there are errors

[Gsantomaggio]

Looking at the issue:

System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. (Parameter 'length')

The buffer seems to be corrupted here.

Let's see if the CRC passes before going ahead with the tests

[troepolik]

@Gsantomaggio yes, after couple days we've spotted two crc errors, both about the same time:

2025-06-19 02:33:15.530
Error | | CRC32 does not match, server crc: 4116709835, local crc: 3226410350, Consumer id 0 for stream: xxx, identifier: xxx, reference: xxx, OffsetSpec RabbitMQ.Stream.Client.OffsetTypeOffset Client ProvidedName xxx, No SuperStream, IsSingleActiveConsumer: False, Token IsCancellationRequested: False , Chunk Consumed 75

on rmq server side I see only that warning:
2025-06-19 02:33:15.541742+00:00 [warning] <0.136764009.1> Socket #Port<0.441349> closed [<0.136764009.1>]

this warning was in both of crc error cases.

[Gsantomaggio]

Repeated and real CRC issues are likely to be a symptom of a more significant underlying problem.
It is essential to know if the chunk is always corrupted in this case: Chunk Consumed 75.

Can you please create a simple consumer?, like:

 var consumerConfiguration = new ConsumerConfig(streamSystem, "<<your_stream>>")
        {
            OffsetSpec = new OffsetTypeFirst(),
            MessageHandler = async (stream, consumerSource, context, message) =>
            {
                Console.WriteLine($"offset: {context.Offset}, ChunkId:{context.ChunkId}");  

Could you see if it is systematic?

Run it more than once.

[troepolik]

@Gsantomaggio it's not systematic. we have 50rps in this queue and only two errors for 2 days.
one with Chunk Consumed 1
another with Chunk Consumed 75

[acogoluegnes]

Can you confirm that if you read and re-read the exact same stream, with the same program, the CRC error happens just from time to time and you are able to read the messages correctly most of the time?

[troepolik]

btw what driver do when crc error happens? will this message be just skipped?
or it depends on few factors? because as we see Socket was closed after error, so as I understand it was reconnection.

[Gsantomaggio]

The TCP connection is closed if you enable the CRC and it doesn't match.
This behaviour can be changed, of course, but we should understand the root problem: Why the CRC fails.

If you want to skip corrupted chunks, you can disable the CRC check.

Is your client remote or in a local network?