WebP decoding support #1258
Comments
|
Would absolutely love to see a PR that adds this using a small WASM module 😍 |
|
Here's a working example using @kenchris's repo as a starting point: And here's one with the wasm file inlined: https://gist.github.com/josephrocca/3d26325f1b76b3b10cb5e7c402c6dfd8 I'm new to all this wasm stuff so it may not be the best code, but it's a start! 👍 |
|
I might be wrong, but from what I can tell, that code uses the asm.js version instead of the WASM one. It should probably be one binary const fs = require('fs')
const path = require('path')
const code = fs.readFileSync(path.join(__dirname, 'webp.wasm'))
const module = new WebAssembly.Module(code)
const instance = new WebAssembly.Instance(module, {})
exports.decode = function (input) {
// Allocate memory to hand over the input data to WASM
const inputPointer = instance.exports.allocate(input.byteLength)
const targetView = new Uint8Array(instance.exports.memory, inputPointer, input.byteLength)
// Copy input data into WASM readable memory
targetView.set(input)
// Decode input data into
const metadataPointer = instance.exports.decode(inputPointer, input.byteLength)
// Free the input data in WASM land
instance.exports.free(inputPointer)
// Read returned metadata
const metadata = new Uint32Array(instance.exports.memory, metadataPointer, 3)
const [width, height, outputPointer] = metadata
// Create an empty buffer for the resulting data
const outputSize = (width * height * 4)
const output = new Uint8Array(outputSize)
// Copy decoded data from WASM memory to JS
output.set(new Uint8Array(instance.exports.memory, outputPointer + 4, outputSize))
// Free WASM copy of decoded data
instance.exports.free(outputPointer)
// Return decoded image as raw data
return { width, height, data: output }
}edit: added support for returning width & height |
|
Yeah, I was referring to the Gist made by @josephrocca I'm going to try and plug your compiled wasm into my glue code and see if I can get it working 🙌 |
|
The gists I posted are just rearranged versions of kenchris's code so it works in node. I don't know if the In any case, looking forward to testing out webp stuff with |
|
Hmm, the Emscripten shim is 30 KiB 😅 would be nice to not use that. I didn't know that you had to provide your own Going to try and write some glue using wasmception and see how it goes 😄 edit: super much work in progress: https://github.com/LinusU/js-webp |
|
but why not the official C lib? wouldn't be more straight forward? |
|
The advantage would be that we could add a small It is the official C lib that I'm compiling btw. |
|
can you share some compilation detail? i need the webp muxer and encoder! |
|
Once the WASM version is working, I'd love to benchmark it against the C version. I'm happy to make the binding for that. |
|
Remember, wasm will be faster with time. Also future versions will support SIMD and threads, which emscripten then need to build for |
|
Also that would make node canvas browserifiable? i mean transitioning all C to wasm. ( cairo included ) |
|
@asturur I suppose, but it seems redundant given that node-canvas exists to emulate the Canvas API that's already in browsers. |
|
well i would never do that. But for people really caring about the same output OR just because till now to people that asked me how to browserify canvas i said that it could not be done. I was just exploring. |
|
Okay, a status update, I'm soooo close to getting it working. It can decode some webp files currently, but I'm running into problems with YUV-images. To be honest I'm not exactly sure where the problem lies, but I've filed a bug report on libwebp here: https://bugs.chromium.org/p/webp/issues/detail?id=403 The current code can be found here: https://github.com/LinusU/cwasm-webp I have published the first version of it on npm 🎉 Currently, only Node.js is supported, but browser support should be easy to add. Just want to research how to best do the loading. Also, only decoding is supported, but it should be trivial to add encoding as well. I would absolutely love some help in tracking down the memory out of bounds issue ❤️ |
|
Update! During the weekend I wrote a Node.js compatible It uses WebAssembly to decode all the image formats 🎉
Should be trivial to start using that |
|
Amazing work!! Looks like you were able to get different C libs compiled with the latest LLVM's WASM support? I've actually been experimenting with this too. In an unrelated project, I'm working on a JS implementation of (parts of) Pango. It also relies on some WebAssembly-compiled C libraries. If we were to use it in node-canvas that would be another native dependency gone, plus it would get us perfect font matching. It would be interesting to experiment with a fork that only uses Cairo/pixman. I'm going to bet though that performance will suffer enough that we might still need to maintain the native version somehow 😑 |
Yeah 🙌 the WASI SDK made it really easy to get going! I was thinking of trying out to do the same with Cairo and see if I could get a nice I don't know if there is a way to link a bunch of different wasm files together into one file, it might be expensive to go thru JavaScript every time the different part needs to talk to each other (on the other hand, maybe it doesn't matter that much, since it will probably only be when loading in new images 🤔) It's going to be very interesting to see how the performance will be. Even if it's not stellar, it would be cool if we could have the natively compiled parts as an Also, potentially it's faster to call between JavaScript and WASM then JavaScript and the C++ functions. I don't have any data to back this up 😁, but I think that JS and WASM can be JIT compiled into the same VM, and then the calls are very cheap. This could potentially be a win when calling into Cairo which is mostly calling functions with a few integers and it does a lot of pixel manipulation internally. Anyhow, the only way to find out is to try 😄 |
|
Is this issue fixed, how do I import remote webp images from urls to node canvas to use the ctx.drawImage(webp_image) function |
|
The |
|
@Xetera if they are doing |
|
After playing around with it for a bit I realized if (Nan::New(Image::constructor)->HasInstance(obj)) { ... }Where node-canvas internally relies on its own definition of const imageResponse = await axios.get(image.rawUrl, {
responseType: "arraybuffer",
});
const ca = canvas.createCanvas(500, 500);
const ctx = ca.getContext("2d");
const imageElem = await imageFromBuffer(imageResponse.data);
ctx.drawImage(imageElem, 0, 0, image.width, image.height);I wanted to keep the conversation related to the solution here but I'm happy with moving over to the EDIT: I tried patching the node-canvas instance check to see if that's the issue but I didn't have much luck. I either get a black canvas output or runtime errors. EDIT2: I was able to solve my issue by combining |
The same issue, I can convert webp to png before I use it on node-canvas, It's an extraordinary and inefficient way... |
|
For those who want to decode WebP images so they can use them in their canvas. Here you go: const webp = require("@cwasm/webp");
const source = fs.readFileSync("./image.webp");
// Decoding the WebP file and putting it on the canvas.
const image = webp.decode(source);
const imageData = ctx.createImageData(image.width, image.height);
imageData.data.set(image.data);
ctx.putImageData(imageData, 0, 0);
// Converting the canvas to buffer and saving it.
writeFileSync("./result.png", canvas.toBuffer());This is a synchronous solution (Bundlephobia: @cwasm/webp). |
Issue or Feature
I'd like to be able to draw webp images onto the canvas like I can in the browser. Note that I'm not talking about encoding (i.e.
toDataURL("image/webp")), since that already has an open issue, and an extension.Steps to Reproduce
Below is a minimal example that should work, but doesn't (It predictably throws
Error: Unsupported image type). You can comment out the webp dataURL and uncomment the PNG url to test that it's working fine with PNGs (no surprise).Your Environment
The text was updated successfully, but these errors were encountered: