-
Notifications
You must be signed in to change notification settings - Fork 143
Alpha support #207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Chrome's implementation has RGBA, BGRA, RGBX, and BGRX. Would this be trimmed down to RGBA and BGRA by your proposal? |
I don't immediately see the value in having the 'X' formats externally. IIUC, they have an alpha channel that conveys no meaningful info (all 255)? If so, I'm inclined to always call RGBX as RGB and so on, and hide the all-255 alpha channel from JS. |
I think we still want the X variants since we aren't reducing 4 bytes to 3 bytes during readInto. Alternatively having just RGBA, BGRA would also be okay. |
Ah, good point. I don't have a strong intuition about whether sites could implement impact optimizations for RGBX (ignoring the alpha plane). If not, I'd vote to just simplify to RGBA. |
If we go with RGBA, BGRA only we need the hasAlpha flag. |
Triage note: marking 'extension' as the proposals are primarily about adding attributes for keeping/discarding/having alpha. |
Editors call:
|
Uh oh!
There was an error while loading. Please reload this page.
I've been meaning to send a PR for the latest thinking from the Chrome explorations... I'll try to have it out this week.
Generally, my proposal will be:
attribute bool hasAlpha
toVideoFrame
. The semantics for planar formats would be that the last plane (highest index) is the alpha plane. More on this in Alpha support #200 (comment)enum AlphaOption { keep, discard }
, and add attributes/members using this enum toVideoFrameInit
andVideoEncoderConfig
. Choose defaults based on expected use cases.VideoEncoderConfig.alpha
, default to=discard
. Alpha encoding is a thing, but relatively rare. In the event that theVideoFrame
is backed by a texture, we can use=discard
to optimize readback to a alpha-less format that is supported by theVideoEncoder
(e.g. NV12)VideoFrameInit.alpha
, default to=keep
. Seems least disruptive in the event you weren't planning to encode the frame (which is already covered above).VideoEncoder
, if configured withalpha=keep
(and if supported), we emit the alpha data as side data in the chunk output callback. This follows the path of existing underlying encoders (e.g. ffmpeg'sAV_PKT_DATA_MATROSKA_BLOCKADDITIONAL
).VideoDecoderConfig
needed to decode the chunk. PR Add SVC modes and metadata #187 is expanding that into a metadata bucket for homing chunk SVC metadata. I propose to just add alpha side data there as well... TBD some naming bikeshed.Note: we haven't yet thought about how to do decoding of alpha. Something to consider.
The text was updated successfully, but these errors were encountered: