memory allocation / garbage collection error

[iraytrace]

There seems to be an issue with adafruit_framebuf.py (perhaps in conjunction with adafruit_ssd1306.py) in some circumstances.

This is observed when running

Adafruit CircuitPython 4.0.1 on 2019-05-22; Adafruit Feather M0 Express with samd21g18

and using libraries from the 4.0 bundle downloaded today the following behavior is observed:

In the code below line 4 and 12 are identical. Which one is uncommented determines which behavior is seen.

import board
import busio
import digitalio
import adafruit_ssd1306  # bad if done here

btns = list()
for inp in [board.A3, board.A4, board.A5]:
    da_btn = digitalio.DigitalInOut(inp)
    da_btn.switch_to_input(digitalio.Pull.UP)
    btns.append(da_btn)

# import adafruit_ssd1306  # fine if done here

i2c = busio.I2C(board.SCL, board.SDA)
oled = adafruit_ssd1306.SSD1306_I2C(128, 32, i2c)

oled.fill(0)
oled.text('Hello', 5, 0, 1)
oled.text('World', 5, 10, 1)
oled.show()

old = [x.value for x in btns]

while True:
    for i in range(len(btns)):
        v = btns[i].value
        if v != old[i]:
            print('%d %s becomes %s' % (i, old[i], btns[i].value))
            oled.pixel(1, i*10, not v)
            oled.show()
        old[i] = v

if line 4 is uncommented, circuitpython reports:

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
main.py output:
Traceback (most recent call last):
File "main.py", line 5, in
File "adafruit_ssd1306.py", line 38, in
File "adafruit_framebuf.py", line 342, in
MemoryError: memory allocation failed, allocating 136 bytes

Whereas if line 4 is commented and line 12 is uncommented, the program runs without problem.

This is particularly odd, as adafruit_framebuf.py does not seem to have that many lines. However, I am running with the .mpy version of the file from the bundle download. Perhaps this is as simple as a bad .mpy file in the bundle.

[dhalbert]

I don't think this is a bug per se, but is due to memory fragmentation and when garbage collection is triggered. We see this kind of thing due to the limited RAM available on M0 boards. As mentioned in discord, you can intersperse gc.collect() calls between imports and that may help.

[iraytrace]

If you have enough experience with the memory management system to assert this, then I bow to your expertise.

Question: Give that adafruit_framebuf.py has 148 lines, how is the following happening:

... File "adafruit_framebuf.py", line 342, in ...

Answer: Because I was reading the wrong repository source code.

The things that make me scratch my head include:

1. Calling gc.mem_free() shows at least 2K free once code starts executing. (ok so free might not be contiguous)
2. Calling gc.collect() doesn't change anything. Grated, for some memory management implementations, this is expected as uncollected memory is already considered free. Is this the case for CircuitPython?
3. I can just as easily get the code to work by injecting random nonsensical code like "newvar = 1 + 2 + 3" at arbitrary places.

It is item 3 which really makes me wonder. If I was really out of memory, adding code and variables (especially once I've added 10 or more lines) shouldn't improve things. OK, I admit it can change the fragmentation of memory.

Just for grins, I modified the code to print memory availability after almost every statement.

def stat(lbl):
    print('%s %g' % (lbl, gc.mem_free() / 1024))
import gc
stat('gc')
import board
stat('board')
import busio
stat('busio')
import digitalio
stat('digitalio')
import adafruit_ssd1306  # bad if done here
stat('ssd1306')
btns = list()
stat('btns')
for inp in [board.A3, board.A4, board.A5]:
    da_btn = digitalio.DigitalInOut(inp)
    da_btn.switch_to_input(digitalio.Pull.UP)
    btns.append(da_btn)
stat('for')

i2c = busio.I2C(board.SCL, board.SDA)
stat('i2c')
oled = adafruit_ssd1306.SSD1306_I2C(128, 32, i2c)
stat('oled')

oled.fill(0)
stat('fill')
oled.text('Hello', 5, 0, 1)
oled.text('World', 5, 10, 1)
oled.show()
stat('show')

old = [x.value for x in btns]
stat('old')

while True:
    for i in range(len(btns)):
        v = btns[i].value
        if v != old[i]:
            print('%d %s becomes %s' % (i, old[i], btns[i].value))
            oled.pixel(1, i*10, not v)
            oled.show()
        old[i] = v

At no time am I below 2K memory free.

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
main.py output:
gc 19.2188
board 19.1563
busio 19.1094
digitalio 19.0469
ssd1306 5.23438
btns 5.14063
for 5
i2c 4.90625
oled 2.32813
fill 2.28125
show 3.54688
old 3.45313

And note that in this instance there was no memory allocation error. So the question remains: What is happening when we are running out of memory? Is it possible to start on soft boot with fragmented memory?

Looking at the source for adafruit_ssd1306 I would not have expected it to consume 14K just on import.

Unfortunately, I don't see any way of asking the interpreter for things like memory maps. I would gladly allocate variables in an appropriate order to reduce fragmentation if I could get some metric to judge this by. Can circuitpython be run as an 'emulator' where the status of the interpreter could be inspected?

[dhalbert]

I agree that it's more problematic than I'd normally expect. I'll try to reproduce this with a debugger running and see exactly where it's failing internally in terms of memory. It may be that it's on the hairy edge of running out of memory, but that should force a gc, so perhaps it's some kind of edge case which we should check on.

https://github.com/adafruit/Adafruit_CircuitPython_framebuf/blob/master/adafruit_framebuf.py is 409 lines. Where did you see only 148?

[iraytrace]

OK, my total mistake there. I landed on https://github.com/adafruit/micropython-adafruit-framebuf/blob/master/framebuf.py which of course is not the right repository. I need to read what google gives me more carefully.

[iraytrace]

I edited my earlier comment with the second example code to make it easier to understand what code was just run in the output that shows memory usage.

[tannewt]

Someone care to summarize where we're at on this?

[loganwedwards]

I have a similar experience documented here. Essentially, on my Feather M0 Lora, I am running into MemoryAllocation errors and if it was just due to the number of imports I have, then I would immediately suck it up and aquire a Feather M4 (192k vs 32k RAM), but when I use the verbatim example code (last post by lecreate), I am still seeing these memory errors, which leads me to believe something else is amuck. Is there anything I can try or data that I can provide that would help?

[dhalbert]

I replied to @loganwedwards in the forum. I was able to get the .py file to load by tweaking some bytearray initializations.

[iraytrace]

So what is the status / expectation?

[dhalbert]

Hmm. I tried the example in your first post with CircuitPython 4.0.1 and 4.1.0-beta.1 and with .mpy's from the 0626 bundle, and it worked fine. I saw "Hello World" on the SSD1306.

Make sure that .py versions of library files are not present on CIRCUITPY. They'll take precedence over the .mpy's when you do an import.

[dhalbert]

Closing for now due to no further information. Please re-open if you can reproduce with 4.1.0. Note that framebuf has been replaced by displayio in 5.0.0