image/png: `fatal error: runtime: out of memory` via `(*decoder).parseIDAT` #53778

secsys-go · 2022-07-11T04:01:13Z

What version of Go are you using (`go version`)?

$ go version
go version go1.17.8 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN="/home/zjx/workspace/gowork/bin"
GOCACHE="/home/zjx/.cache/go-build"
GOENV="/home/zjx/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/zjx/workspace/gowork/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/zjx/workspace/gowork"
GOPRIVATE=""
GOPROXY="https://goproxy.cn,direct"
GOROOT="/home/zjx/.local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/zjx/.local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.17.8"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/zjx/workspace/gowork/src/go-fdg-exmaples/std/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1878510231=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I attempt to decode an png data, and the details can be seen at:

go.dev/play

What did you expect to see?

It decodes the data successfully or just returns an error

What did you see instead?

It triggered a fatal error like:

go env Output

fatal error: runtime: out of memory
runtime stack:

runtime.throw({0x4af45e, 0x31ce0000000})

/home/zjx/.local/go/src/runtime/panic.go:1198 +0x71

runtime.sysMap(0xc000400000, 0x426780, 0x7fff08857758)

/home/zjx/.local/go/src/runtime/mem_linux.go:169 +0x96

runtime.(*mheap).grow(0x5652e0, 0x18e70000)

/home/zjx/.local/go/src/runtime/mheap.go:1393 +0x225

runtime.(*mheap).allocSpan(0x5652e0, 0x18e70000, 0x0, 0x1)

/home/zjx/.local/go/src/runtime/mheap.go:1179 +0x165

runtime.(*mheap).alloc.func1()

/home/zjx/.local/go/src/runtime/mheap.go:913 +0x69

runtime.systemstack()

/home/zjx/.local/go/src/runtime/asm_amd64.s:383 +0x49
goroutine 1 [running]:

runtime.systemstack_switch()

/home/zjx/.local/go/src/runtime/asm_amd64.s:350 fp=0xc00010da00 sp=0xc00010d9f8 pc=0x458e20

runtime.(*mheap).alloc(0x0, 0xc00010daa8, 0x6f, 0x0)

/home/zjx/.local/go/src/runtime/mheap.go:907 +0x73 fp=0xc00010da50 sp=0xc00010da00 pc=0x422ab3

runtime.(*mcache).allocLarge(0x40b3fe, 0x31ce0000000, 0x87, 0x1)

/home/zjx/.local/go/src/runtime/mcache.go:227 +0x89 fp=0xc00010dab0 sp=0xc00010da50 pc=0x413949

runtime.mallocgc(0x31ce0000000, 0x499220, 0x1)

/home/zjx/.local/go/src/runtime/malloc.go:1088 +0x5c5 fp=0xc00010db30 sp=0xc00010dab0 pc=0x40bb45

runtime.makeslice(0xc0001b6000, 0x0, 0xc00010db98)

/home/zjx/.local/go/src/runtime/slice.go:98 +0x52 fp=0xc00010db58 sp=0xc00010db30 pc=0x444892

image.NewNRGBA64({{0x0, 0x0}, {0xc00010dc18, 0x1000000465c25}})

/home/zjx/.local/go/src/image/image.go:601 +0x66 fp=0xc00010dbd0 sp=0xc00010db58 pc=0x489fa6

image/png.(*decoder).readImagePass(0xc0000d7000, {0x7f125c277030, 0xc000180050}, 0x0, 0x0)

/home/zjx/.local/go/src/image/png/reader.go:493 +0x7b6 fp=0xc00010dda0 sp=0xc00010dbd0 pc=0x48c7f6

image/png.(*decoder).decode(0xc0000d7000)

/home/zjx/.local/go/src/image/png/reader.go:372 +0x1af fp=0xc00010de70 sp=0xc00010dda0 pc=0x48bb0f

image/png.(*decoder).parseIDAT(0xc0000d7000, 0xd7078)

/home/zjx/.local/go/src/image/png/reader.go:849 +0x25 fp=0xc00010de88 sp=0xc00010de70 pc=0x48f6e5

image/png.(*decoder).parseChunk(0xc0000d7000)

/home/zjx/.local/go/src/image/png/reader.go:908 +0x128 fp=0xc00010def8 sp=0xc00010de88 pc=0x48f888

image/png.Decode({0x4cb540, 0xc0001a6000})

/home/zjx/.local/go/src/image/png/reader.go:967 +0x11b fp=0xc00010df60 sp=0xc00010def8 pc=0x48fe5b

main.main()

/home/zjx/workspace/gowork/src/go-fdg-exmaples/std/image/png/pocTestIncolplete/poc.go:9 +0x56 fp=0xc00010df80 sp=0xc00010df60 pc=0x4903b6

runtime.main()

/home/zjx/.local/go/src/runtime/proc.go:255 +0x227 fp=0xc00010dfe0 sp=0xc00010df80 pc=0x4324a7

runtime.goexit()

/home/zjx/.local/go/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc00010dfe8 sp=0xc00010dfe0 pc=0x45af01

exit status 2

Found by go-fuzz,

The text was updated successfully, but these errors were encountered:

robpike · 2022-07-11T05:48:17Z

Your program says it exits without error, but it doesn't. My version, which checks the error returned by Decode, reports

% go run x.go
<nil> flate: corrupt input before offset 5
%

https://go.dev/play/p/bAa9vbuJfgT

It may be running out of memory or CPU time on the playground, but the input is invalid and the error is reported on a proper machine.

Did you find this example by fuzzing?

bcmills · 2022-07-11T14:40:35Z

I am able to reproduce the reported behavior on Go at HEAD.

$ go version
go version devel go1.19-59ab6f351a Mon Jul 11 08:23:57 2022 +0000 linux/amd64

$ go mod init example
go: creating new go.mod: module example

$ cat > main.go
package main

import (
        "fmt"
        "image/png"
        "runtime"
        "strings"
)

func main() {
        fmt.Println(runtime.Version())
        _, err := png.Decode(strings.NewReader("\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\xffd\x00\x00\x00\x10\x06\x00\x00\x00\x8bӧ\xed0000IDATx\x9c\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x00\xffd\x00\x00\x00\x10\x06\x00\x00"))
        fmt.Println("decoded:", err)
}

$ go run .
devel go1.19-59ab6f351a Mon Jul 11 08:23:57 2022 +0000
fatal error: runtime: out of memory

runtime stack:
runtime.throw({0x4b3483?, 0x80000?})
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/panic.go:1047 +0x5d fp=0x7fff582f0740 sp=0x7fff582f0710 pc=0x431afd
runtime.sysMapOS(0xc000400000, 0x31ce0000000?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mem_linux.go:187 +0x11b fp=0x7fff582f0788 sp=0x7fff582f0740 pc=0x4152fb
runtime.sysMap(0x5695a0?, 0x7f721d373000?, 0x428520?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mem.go:142 +0x35 fp=0x7fff582f07b8 sp=0x7fff582f0788 pc=0x414cd5
runtime.(*mheap).grow(0x5695a0, 0x18e70000?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mheap.go:1459 +0x23d fp=0x7fff582f0828 sp=0x7fff582f07b8 pc=0x4255dd
runtime.(*mheap).allocSpan(0x5695a0, 0x18e70000, 0x0, 0x1)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mheap.go:1191 +0x1be fp=0x7fff582f08c0 sp=0x7fff582f0828 pc=0x424d3e
runtime.(*mheap).alloc.func1()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mheap.go:910 +0x65 fp=0x7fff582f0908 sp=0x7fff582f08c0 pc=0x4247c5
runtime.systemstack()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:492 +0x49 fp=0x7fff582f0910 sp=0x7fff582f0908 pc=0x45b0a9

goroutine 1 [running]:
runtime.systemstack_switch()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:459 fp=0xc00005f9d0 sp=0xc00005f9c8 pc=0x45b040
runtime.(*mheap).alloc(0x413b2b?, 0x57fd20?, 0x7?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mheap.go:904 +0x65 fp=0xc00005fa18 sp=0xc00005f9d0 pc=0x424705
runtime.(*mcache).allocLarge(0x7?, 0x31ce0000000, 0x1)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mcache.go:233 +0x85 fp=0xc00005fa68 sp=0xc00005fa18 pc=0x413c65
runtime.mallocgc(0x31ce0000000, 0x49c4c0, 0x1)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/malloc.go:1029 +0x57e fp=0xc00005fae0 sp=0xc00005fa68 pc=0x40bd5e
runtime.makeslice(0xc00005fb38?, 0xc00005fb38?, 0x446b72?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/slice.go:103 +0x52 fp=0xc00005fb08 sp=0xc00005fae0 pc=0x446b72
image.NewNRGBA64({{0xc000104bb8?, 0x484d6e?}, {0x100c000121088?, 0x4d1f08?}})
        /usr/local/google/home/bcmills/sdk/gotip/src/image/image.go:603 +0x66 fp=0xc00005fb80 sp=0xc00005fb08 pc=0x48da26
image/png.(*decoder).readImagePass(0x4d1f68?, {0x7f71f5ec6020, 0xc000078050}, 0x0?, 0x0)
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:495 +0x845 fp=0xc00005fd58 sp=0xc00005fb80 pc=0x4902c5
image/png.(*decoder).decode(0xc000052c00)
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:374 +0x1ae fp=0xc00005fe28 sp=0xc00005fd58 pc=0x48f56e
image/png.(*decoder).parseIDAT(0xc000052c00, 0x52c78?)
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:859 +0x25 fp=0xc00005fe40 sp=0xc00005fe28 pc=0x493085
image/png.(*decoder).parseChunk(0xc000052c00)
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:918 +0x129 fp=0xc00005feb0 sp=0xc00005fe40 pc=0x493229
image/png.Decode({0x4d1fe8?, 0xc00006e020})
        /usr/local/google/home/bcmills/sdk/gotip/src/image/png/reader.go:977 +0x11b fp=0xc00005ff18 sp=0xc00005feb0 pc=0x4937fb
main.main()
        /tmp/tmp.oRxzJ95Mq5/main.go:12 +0xa5 fp=0xc00005ff80 sp=0xc00005ff18 pc=0x493d65
runtime.main()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:250 +0x212 fp=0xc00005ffe0 sp=0xc00005ff80 pc=0x434352
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00005ffe8 sp=0xc00005ffe0 pc=0x45d121

goroutine 2 [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:363 +0xd6 fp=0xc00004efb0 sp=0xc00004ef90 pc=0x434716
runtime.goparkunlock(...)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:369
runtime.forcegchelper()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:302 +0xad fp=0xc00004efe0 sp=0xc00004efb0 pc=0x4345ad
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00004efe8 sp=0xc00004efe0 pc=0x45d121
created by runtime.init.6
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:290 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:363 +0xd6 fp=0xc00004f790 sp=0xc00004f770 pc=0x434716
runtime.goparkunlock(...)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:369
runtime.bgsweep(0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgcsweep.go:278 +0x8e fp=0xc00004f7c8 sp=0xc00004f790 pc=0x4217ae
runtime.gcenable.func1()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgc.go:178 +0x26 fp=0xc00004f7e0 sp=0xc00004f7c8 pc=0x416686
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00004f7e8 sp=0xc00004f7e0 pc=0x45d121
created by runtime.gcenable
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgc.go:178 +0x6b

goroutine 4 [GC scavenge wait]:
runtime.gopark(0xc00006c000?, 0x4d18c8?, 0x1?, 0x0?, 0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:363 +0xd6 fp=0xc00004ff70 sp=0xc00004ff50 pc=0x434716
runtime.goparkunlock(...)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:369
runtime.(*scavengerState).park(0x550e60)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgcscavenge.go:389 +0x53 fp=0xc00004ffa0 sp=0xc00004ff70 pc=0x41f853
runtime.bgscavenge(0x0?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgcscavenge.go:617 +0x45 fp=0xc00004ffc8 sp=0xc00004ffa0 pc=0x41fe25
runtime.gcenable.func2()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgc.go:179 +0x26 fp=0xc00004ffe0 sp=0xc00004ffc8 pc=0x416626
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00004ffe8 sp=0xc00004ffe0 pc=0x45d121
created by runtime.gcenable
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mgc.go:179 +0xaa

goroutine 5 [finalizer wait]:
runtime.gopark(0x434a97?, 0x49?, 0xe0?, 0xe7?, 0xc00004e770?)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:363 +0xd6 fp=0xc00004e628 sp=0xc00004e608 pc=0x434716
runtime.goparkunlock(...)
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/proc.go:369
runtime.runfinq()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mfinal.go:180 +0x10f fp=0xc00004e7e0 sp=0xc00004e628 pc=0x41578f
runtime.goexit()
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc00004e7e8 sp=0xc00004e7e0 pc=0x45d121
created by runtime.createfing
        /usr/local/google/home/bcmills/sdk/gotip/src/runtime/mfinal.go:157 +0x45
exit status 2

bcmills · 2022-07-11T14:42:26Z

(CC @nigeltao)

robpike · 2022-07-12T00:21:03Z

Interesting. I am on an M1 with 128GB of memory and see an error, not a crash.

rsc · 2022-07-12T00:21:25Z

I ran Rob's version and Ctrl-'ed it after a few seconds and it looks like it is allocating 3.4 TB. Probably different systems handle that differently.

rsc · 2022-07-12T00:23:23Z

png.DecodeConfig runs fine and says 255x1677721600, so this may be another instance where untrusted inputs should be checked with DecodeConfig before being passed to Decode.

robpike · 2022-07-12T00:28:20Z

I tried it on a smaller M1 and still saw the error, not the crash.

My summary: this is a buggy program that does not protect against bad inputs as instructed, but the variant behavior on different CPU types and/or operating systems is worth investigating.

secsys-go · 2022-07-13T08:06:53Z

Your program says it exits without error, but it doesn't. My version, which checks the error returned by Decode, reports
% go run x.go
<nil> flate: corrupt input before offset 5
% 
https://go.dev/play/p/bAa9vbuJfgT

It may be running out of memory or CPU time on the playground, but the input is invalid and the error is reported on a proper machine.

Did you find this example by fuzzing?

Yeah, I found it by go-fuzz, and the program exceeded the memory limit of the system when applying for a slice according to mutated size of the image.

In fact, what I want to say at the last println in playground is "If it exits without crashes, you should see this sentence", and the link of playground has been updated as go.dev/play

nigeltao · 2022-07-13T11:10:09Z

this may be another instance where untrusted inputs should be checked with DecodeConfig before being passed to Decode.

I'd agree with that.

I'd also say that the bug is a dupe of #5050 (whose title says image/gif but really applies to all the image formats).

ianlancetaylor · 2022-07-13T19:58:07Z

One approach we could use for this kind of problem is a incremental image build, as in https://go.dev/cl/417477. @nigeltao Any opinions about that approach? Thanks.

gopherbot · 2022-07-13T19:58:07Z

Change https://go.dev/cl/417477 mentions this issue: image/png: build large images incrementally

nigeltao · 2022-07-13T23:12:22Z

One approach we could use for this kind of problem is a incremental image build, as in https://go.dev/cl/417477. @nigeltao Any opinions about that approach?

I dropped a code review comment on that page.

ianlancetaylor · 2022-07-14T00:54:03Z

Bringing the discussion back to the issue.

It's a fair point that the CL doesn't protect well against zlib. Although I do think of this kind of change more as aimed at a corrupt file than as a defense against an attacker.

I also think it's mildly unfortunate to expect people to check the DecodeConfig result first, as there is no obvious limit to check for, and there is no natural way to call DecodeConfig, check values, and then call Decode, as you need to rewind the reader. Or so it seems to me, I don't know what much about these packages.

Certainly the CL needs more work, it's just a sketch.

What concerns me about these issues is the unrecoverable crash that occurs on many systems, particularly when running in a container. It's not a friendly response to corrupt data. This is related to the discussion at #20169, which remains unresolved. I've been trying to tackle a subset of these issues in the CLs starting at https://go.dev/cl/408678. Basically I'm trying to work within our existing constraints to return an error rather than crashing. This CL, https://go.dev/cl/417477, is an attempt at taking this approach to image/png, but it may be a poor fit.

bcmills changed the title ~~image/png: fatal error: runtime: out of memory~~ image/png: fatal error: runtime: out of memory in NewNRGBA64 Jul 11, 2022

bcmills changed the title ~~image/png: fatal error: runtime: out of memory in NewNRGBA64~~ image/png: fatal error: runtime: out of memory via (*decoder).parseIDAT Jul 11, 2022

bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jul 11, 2022

seankhliao added this to the Backlog milestone Aug 20, 2022

nigeltao mentioned this issue Sep 1, 2022

image/png: makeslice: len out of range in Decode with insane image size #54586

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

image/png: `fatal error: runtime: out of memory` via `(*decoder).parseIDAT` #53778

image/png: `fatal error: runtime: out of memory` via `(*decoder).parseIDAT` #53778

secsys-go commented Jul 11, 2022 •

edited

Loading

robpike commented Jul 11, 2022 •

edited

Loading

bcmills commented Jul 11, 2022

bcmills commented Jul 11, 2022

robpike commented Jul 12, 2022

rsc commented Jul 12, 2022

rsc commented Jul 12, 2022

robpike commented Jul 12, 2022 •

edited

Loading

secsys-go commented Jul 13, 2022

nigeltao commented Jul 13, 2022

ianlancetaylor commented Jul 13, 2022

gopherbot commented Jul 13, 2022

nigeltao commented Jul 13, 2022

ianlancetaylor commented Jul 14, 2022

image/png: fatal error: runtime: out of memory via (*decoder).parseIDAT #53778

image/png: fatal error: runtime: out of memory via (*decoder).parseIDAT #53778

Comments

secsys-go commented Jul 11, 2022 • edited Loading

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

robpike commented Jul 11, 2022 • edited Loading

bcmills commented Jul 11, 2022

bcmills commented Jul 11, 2022

robpike commented Jul 12, 2022

rsc commented Jul 12, 2022

rsc commented Jul 12, 2022

robpike commented Jul 12, 2022 • edited Loading

secsys-go commented Jul 13, 2022

nigeltao commented Jul 13, 2022

ianlancetaylor commented Jul 13, 2022

gopherbot commented Jul 13, 2022

nigeltao commented Jul 13, 2022

ianlancetaylor commented Jul 14, 2022

image/png: `fatal error: runtime: out of memory` via `(*decoder).parseIDAT` #53778

image/png: `fatal error: runtime: out of memory` via `(*decoder).parseIDAT` #53778

secsys-go commented Jul 11, 2022 •

edited

Loading

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

robpike commented Jul 11, 2022 •

edited

Loading

robpike commented Jul 12, 2022 •

edited

Loading