Panic at internal/encoder/vm_escaped/util.go:33

[silverwind]

While trying to migrate to go-json, we are seeing a crash in the encoder. The same encoding works fine using encoding/json and json-iterator/go.

I don't have the details on what exact struct causes the crash yet, but by the off-chance that you can already see the issue from the stack trace alone, I'm posting it here. Origin of the encoder call here. Struct definition here.

PANIC: runtime error: invalid memory address or nil pointer dereference
  /usr/local/go/src/runtime/panic.go:212 (0x43805a)
    panicmem: panic(memoryError)
  /usr/local/go/src/runtime/signal_unix.go:734 (0x451db2)
    sigpanic: panicmem()
  /drone/src/vendor/github.com/goccy/go-json/internal/encoder/vm_escaped/util.go:33 (0x6c10eb)
    ptrToUint64: func ptrToUint64(p uintptr) uint64              { return **(**uint64)(unsafe.Pointer(&p)) }
  /drone/src/vendor/github.com/goccy/go-json/internal/encoder/vm_escaped/vm.go:688 (0x6c10d0)
    Run: b = appendInt(b, ptrToUint64(p+code.Offset), code)
  /drone/src/vendor/github.com/goccy/go-json/encode.go:281 (0x7475fd)
    encodeRunCode: return vm_escaped.Run(ctx, b, codeSet, encoder.Option(opt))
  /drone/src/vendor/github.com/goccy/go-json/encode.go:211 (0x746e94)
    encode: buf, err := encodeRunCode(ctx, b, codeSet, opt)
  /drone/src/vendor/github.com/goccy/go-json/encode.go:95 (0x7467cb)
    (*Encoder).encodeWithOption: buf, err = encode(ctx, v, opt)
  /drone/src/vendor/github.com/goccy/go-json/encode.go:74 (0x746504)
    (*Encoder).EncodeWithOption: err := e.encodeWithOption(ctx, v, optFuncs...)
  /drone/src/vendor/github.com/goccy/go-json/encode.go:67 (0x1b1ce84)
    (*Encoder).Encode: return e.EncodeWithOption(v)
  /drone/src/modules/context/context.go:373 (0x1b1cdea)
    (*Context).JSON: if err := json.NewEncoder(ctx.Resp).Encode(content); err != nil {
  /drone/src/routers/api/v1/user/gpg_key.go:30 (0x20230d4)
    listGPGKeys: ctx.JSON(http.StatusOK, &apiKeys)

[goccy]

@silverwind
Thank you for the reporting. I would like to resolve this problem as soon as possible, but please share more information to debug. The debug information is provided by json.Debug() option.

if err := json.NewEncoder(ctx.Resp).EncodeWithOption(content, json.Debug()); err != nil {
  ctx.ServerError("Render JSON failed", err)
}

If panic occurred, go-json produces debug information to the console ( related issue: #149 (comment) )

[silverwind]

Okay, we will try that and possibly also to get the data that causes it.

[silverwind]

Some debug output from a build:

=============[DEBUG]===============
* [TYPE]
*[]*structs.GPGKey

* [ALL OPCODE]
[0]Slice ([idx:0][headIdx:0][elemIdx:1][length:2])
[1]-StructPtrHeadInt ([idx:3][key:id][offset:0][headIdx:3])
[2]--StructFieldString ([idx:5][key:primary_key_id][offset:8][headIdx:3])
[3]--StructFieldString ([idx:6][key:key_id][offset:24][headIdx:3])
[4]--StructFieldString ([idx:7][key:public_key][offset:40][headIdx:3])
[5]--StructFieldSlice ([idx:8][key:emails][offset:56][headIdx:3])
[6]--Slice ([idx:9][headIdx:9][elemIdx:10][length:11])
[7]---StructPtrHeadString ([idx:12][key:email][offset:0][headIdx:12])
[8]----StructFieldBool ([idx:14][key:verified][offset:16][headIdx:12])
[9]---StructEnd ([idx:15][key:][offset:0][headIdx:0])
[10]--SliceElem ([idx:16][headIdx:9][elemIdx:10][length:11][size:8])
[11]--SliceEnd ([idx:17])
[12]--StructFieldSlice ([idx:18][key:subkeys][offset:80][headIdx:3])
[13]--Slice ([idx:19][headIdx:19][elemIdx:20][length:21])
[14]---RecursivePtr ([idx:22])
[15]--SliceElem ([idx:23][headIdx:19][elemIdx:20][length:21][size:8])
[16]--SliceEnd ([idx:24])
[17]--StructFieldBool ([idx:25][key:can_sign][offset:104][headIdx:3])
[18]--StructFieldBool ([idx:26][key:can_encrypt_comms][offset:105][headIdx:3])
[19]--StructFieldBool ([idx:27][key:can_encrypt_storage][offset:106][headIdx:3])
[20]--StructFieldBool ([idx:28][key:can_certify][offset:107][headIdx:3])
[21]--StructFieldOmitEmptyMarshalJSON ([idx:29][key:created_at][offset:112][headIdx:3])
[22]--StructFieldOmitEmptyMarshalJSON ([idx:30][key:expires_at][offset:136][headIdx:3])
[23]-StructEnd ([idx:31][key:][offset:0][headIdx:0])
[24]SliceElem ([idx:32][headIdx:0][elemIdx:1][length:2][size:8])
[25]SliceEnd ([idx:33])

* [CURRENT OPCODE]
&{Op:StructPtrHeadInt Type:int64 DisplayIdx:1 Key:[34 105 100 34 58] EscapedKey:[34 105 100 34 58] PtrNum:1 DisplayKey:id IsTaggedKey:true AnonymousKey:false AnonymousHead:false Indirect:true Nilcheck:true AddrForMarshaler:false IsNextOpPtrType:false IsNilableType:false RshiftNum:63 Mask:18446744073709551615 Indent:1 Idx:24 HeadIdx:24 ElemIdx:0 Length:0 MapIter:0 MapPos:0 Offset:0 Size:0 MapKey:<nil> MapValue:<nil> Elem:<nil> End:0x4005e64e00 PrevField:<nil> NextField:0x4003294e00 Next:0x4003294e00 Jmp:<nil>}

* [CONTEXT]
&{Buf:[] Ptrs:[274961801856 275061516544 2 2 0 0 0 0 0 274982893376 0 1 275009499096 0 0 0 0 0 0 274982893368 0 1 274985818848 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] KeepRefs:[0x4007d7efc0] SeenPtr:[274985818848] BaseIndent:0 Prefix:[] IndentStr:[]}
===================================

[goccy]

@silverwind Thank you for the detailed information to debug . I fixed this issue with #172 !