Huge memory usage by canvas.loadFromJSON on NodeJS

[DanieleSassoli]

Hy
I'm rendering some canvas on node 0.10.25, the json arrives from a client, I change all the sources of the images just to load the hres images and the I load the canvas, then I render it and finally writeit to a file. The problem is that creating the canvas occupies 200M average, and loadFromJSON in some cases reaches 1GB PER CANVAS, and this memory never get released, or at least not all of it.
I have to render many canvas and this juge memory usage often leads to a ENOMEM error, and if I try to load them in parallel ( as I would like to) it always throw an ENOMEM. This is my code:

     var canvas = fabric.createCanvasForNode(2500, 2000);
    //change the src of the images, load the fonts. 
    canvas.loadFromJSON(page['canvasData'], function () {
         canvas.renderAll();
         var out = fs.createWriteStream(finalOutputFileName);
         var stream = canvas.createJPEGStream({quality: 100});
         out.on('finish', sCb);
         stream.pipe(out);
     });

I tried calling canvas.destroy(); and then canvas = null; on the out.on('finish') event, but nothing changes.
I'm I getting something wrong or is this an issue?
thank's.

[alanmastro]

I'm expriencing something similar to.

[asturur]

Can we see an hires image also?
And how many of them are there?

[DanieleSassoli]

[DanieleSassoli]

just an example image... In the test case I'm using I never hae more than 6 images.

[asturur]

So 1600x1200 is more or less 8MB ( how many of them? )

2000x2500 canvas is 20MB + upperCanvas is other 20MB

Fonts i have no idea sincely how much they take.

Using filters also? cause this could easily make bigger memory usages.

Any tool on nodejs to find out on wich operation the memory grows? can you do something like
console.log(memusage) to find out where is growing so much?

[kangax]

Possibly related node-canvas issues:

Automattic/node-canvas#140
Automattic/node-canvas#93
Automattic/node-canvas#73

[DanieleSassoli]

Hy,
I'll check if some of the suggestions made in the related issues work for me, although the operation in wich the memory grows in enlivenObjects, when it gets to load the image it starts occupying the memory and never frees it up, I don't think the fonts are a problem because in many pages I have no fonts, and rendering them uses as much memory as the others...

[alanmastro]

Under further investigation seems to me like the problem is on line 23639 img.src = new Buffer(data, 'binary'); seems like he never deallocates the buffer.

[DanieleSassoli]

Just finished trying with node 0.12.0, nothing changed.

[DanieleSassoli]

launching node with node --expose-gc --always-compact options, and calling global.gc() at the end of every rendering and again at the end of every rendering fixes the issue... I don't like it like solutions but having no other alternative... Also declaring one canvas and changing width and height at every iteration helped

[kangax]

@DanieleSassoli I did the same thing in our app (couple years ago and it's still the same). I'm not sure if there's anything we can do in Fabric about this. Perhaps it should just be taken care of on an application level.

[DPflasterer]

I know this is closed but I just wanted to give my two cents because I still had problems after reading the suggestions in this thread.

I had a class with a method that was creating a new canvas object every time it was called. For some reason they weren't being garbage collected so the memory was never being released. What fixed it for me is the last thing @DanieleSassoli suggested. I made the canvas a property of a class (so there was ever only one instance of canvas created), and within the method I cleared and resized it, then did what I needed to do.

I did not have to use --expose-gc --always-compact or call glocal.gc() to prevent the leak. Hope this helps someone.

[vjames19]

@DPflasterer can you send me a gist of what you did? Currently having this same problem in production and Fabric.js seems like the best API out there to generate this images.

@DanieleSassoli does the solution you provided keep working for you?

[DanieleSassoli]

@vjames19 yes, still working for me.

[vjames19]

@DanieleSassoli It works but it definitely brings downs performance.

[DanieleSassoli]

@vjames19 haven't had accurate tests on this, at the moment I haven't got the time to do this.

[joshbeckman]

We also ran into this problem when doing any kind of concurrent or repetitive manipulation of multiple canvas instances.
Short answer, when done using a canvas, you need to call:

var canvas = fabric.createCanvasForNode();
// blah blah canvas manipulation
canvas.clear();
canvas.dispoose();
// garbage collection should be good now

Long answer is written up here.

[ozooner]

Just a side-note - canvas.dispose() already calls clear()

/**
     * Clears a canvas element and removes all event listeners
     * @return {fabric.Canvas} thisArg
     * @chainable
     */
    dispose: function () {
      this.clear();
      this.interactive && this.removeListeners();
      return this;
    },

[terrancesnyder]

FYI guys we run on embedded PCs which are really tight in terms of memory. We need to run under 80mb of ram consistently. We did some memory analysis... our issues were memory leaks within fabricjs in the addListener(fabric.window, 'resize', this._onResize);. After removing those within fabricjs core, this avoided the largest of our issues.

[asturur]

what are you doing with your app?