CBOR: decodeFromByteArray<List<ByteArray>> fails with "Expected start of array, but found XX" for indefinite-length array of byte strings on Android

[darrenleeleelee1]

Describe the bug
When attempting to decode a CBOR indefinite-length array (9F ... FF) containing byte strings into a List<ByteArray>, the kotlinx-serialization-cbor library throws a CborDecodingException with the message "Expected start of array, but found XX" (where XX is the major type and length of the first byte string in the array). This occurs even though the input ByteArray correctly starts with the 9F (indefinite-array) marker. The issue seems to be specific to the Android environment or a particular setup therein, as similar code might work in a pure JVM environment.

To Reproduce
The following minimal Kotlin code, when run in an Android Activity's onCreate method, reproduces the issue with a simple indefinite-length array containing one empty byte string (9F40FF):

import android.os.Bundle
import android.util.Log
import androidx.activity.ComponentActivity
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.cbor.Cbor
import kotlinx.serialization.decodeFromByteArray

// Helper to convert hex string to ByteArray
fun String.hexToByteArray(): ByteArray {
    check(length % 2 == 0) { "Must have an even length" }
    return chunked(2)
        .map { it.toInt(16).toByte() }
        .toByteArray()
}

// Helper to convert ByteArray to hex string for logging
fun ByteArray.toHexString() = joinToString("") { "%02X".format(it) }

@OptIn(ExperimentalSerializationApi::class)
class MainActivity : ComponentActivity() {
    private val TAG = "CBOR_BUG_REPORT"

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        // Test case: Simplified data (an indefinite array with one empty byte string)
        // CBOR: 9F (array*) 40 (bytes(0)) FF (break)
        val hexString = "9F40FF"
        val bytes = hexString.hexToByteArray()

        Log.d(TAG, "Bytes to decode: ${bytes.toHexString()}")
        Log.d(TAG, "First byte as hex: ${bytes[0].toUByte().toString(16).uppercase()}")
        Log.d(TAG, "Is first byte 0x9F? ${bytes[0] == 0x9F.toByte()}")


        try {
            // Attempt to decode as List<ByteArray>
            val result = Cbor.Default.decodeFromByteArray<List<ByteArray>>(bytes)
            Log.d(TAG, "Successfully decoded. Result size: ${result.size}")
            if (result.isNotEmpty()) {
                 Log.d(TAG, "First element (hex): ${result[0].toHexString()}")
            }
        } catch (e: Exception) {
            Log.e(TAG, "Error decoding '${hexString}': ${e.message}", e)
            // Example Stacktrace for "9F40FF":
            // kotlinx.serialization.cbor.internal.CborDecodingException: Expected start of array, but found 40
            //  at kotlinx.serialization.cbor.internal.CborDecodingExceptionKt.CborDecodingException(CborDecodingException.kt:13)
            //  at kotlinx.serialization.cbor.internal.CborParser.startSized-kvxxsfM(Decoder.kt:212)
            //  at kotlinx.serialization.cbor.internal.CborParser.startArray-uLth9ew(Decoder.kt:196)
            //  at kotlinx.serialization.cbor.internal.CborListReader.skipBeginToken-uLth9ew(Decoder.kt:543) // Line number might vary
            //  ...
            //  at kotlinx.serialization.cbor.Cbor.decodeFromByteArray(Cbor.kt:89)
            //  at com.your.package.name.MainActivity.onCreate(MainActivity.kt:LINE_NUMBER_OF_DECODE_CALL)
            //  ...
        }
    }
}

Logcat Output for 9F40FF:

D/CBOR_BUG_REPORT: Bytes to decode: 9F40FF
D/CBOR_BUG_REPORT: First byte as hex: 9F
D/CBOR_BUG_REPORT: Is first byte 0x9F? true
E/CBOR_BUG_REPORT: Error decoding '9F40FF': Expected start of array, but found 40
    kotlinx.serialization.cbor.internal.CborDecodingException: Expected start of array, but found 40
        at kotlinx.serialization.cbor.internal.CborDecodingExceptionKt.CborDecodingException(CborDecodingException.kt:13)
        at kotlinx.serialization.cbor.internal.CborParser.startSized-kvxxsfM(Decoder.kt:212)
        at kotlinx.serialization.cbor.internal.CborParser.startArray-uLth9ew(Decoder.kt:196)
        at kotlinx.serialization.cbor.internal.CborListReader.skipBeginToken-uLth9ew(Decoder.kt:543) 
        at kotlinx.serialization.cbor.internal.CborReader.beginStructure(Decoder.kt:49)
        at kotlinx.serialization.internal.AbstractCollectionSerializer.merge(CollectionSerializers.kt:29)
        at kotlinx.serialization.internal.PrimitiveArraySerializer.deserialize(CollectionSerializers.kt:179)
        // ... (rest of your stack trace) ...
        at com.your.package.name.MainActivity.onCreate(MainActivity.kt:LINE_NUMBER_OF_DECODE_CALL)

Expected behavior

Environment

• Kotlin version: 2.0.0
• Library version: 1.8.1
• Kotlin platforms: Android
• Gradle version: 8.10.2
• IDE version: Android Studio Ladybug Feature Drop | 2024.2.2

[fzhinkin]

@darrenleeleelee1, a byte array you're trying to decode is represented as an empty byte string (the one which is CBOR's major type). You can read about byte strings and byte arrays handling here: https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/formats.md#byte-arrays-and-cbor-data-types

Using something like

Cbor { alwaysUseByteString = true }.decodeFromByteArray<List<ByteArray>>(bytes)

will help and 9F40FF will be successfully decoded.

I think, we can improve an error message for this particular type mismatch, so it will be easier to understand the root cause and how to mitigate it.