Skip to content

Commit d39fda0

Browse files
chcunninghamchcunningham
authored
Merge pull request #187 from w3c/svc_config
Add SVC modes and metadata
2 parents 7a0a448 + 636611f commit d39fda0

File tree

1 file changed

+143
-50
lines changed

1 file changed

+143
-50
lines changed

‎index.src.html

Lines changed: 143 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,9 @@
9090
type: dfn; text: natural dimensions; url: #natural-dimensions
9191
type: dfn; text: natural width; url: #natural-width
9292
type: dfn; text: natural height; url: #natural-height
93+
94+
spec: webrtc-svc; urlPrefix: https://w3c.github.io/webrtc-svc/
95+
type: dfn; text: scalability mode identifier; url:#scalabilitymodes*
9396
</pre>
9497

9598
<style>
@@ -123,6 +126,10 @@
123126
objects. Such resources may be quickly exhuasted and should be released
124127
immediately when no longer in use.
125128

129+
: <dfn>Temporal Layer</dfn>
130+
:: A grouping of {{EncodedVideoChunk}}s who's timestamp cadence produces a
131+
particular framerate. See {{VideoEncoderConfig/scalabilityMode}}.
132+
126133
: <dfn>Progressive Image</dfn>
127134
:: An image that supports decoding to multiple levels of detail, with lower
128135
levels becoming available while the encoded data is not yet fully buffered.
@@ -732,7 +739,9 @@
732739
required WebCodecsErrorCallback error;
733740
};
734741

735-
callback EncodedAudioChunkOutputCallback = undefined(EncodedAudioChunk output);
742+
callback EncodedAudioChunkOutputCallback =
743+
undefined (EncodedAudioChunk output,
744+
optional EncodedAudioChunkMetadata metadata = {});
736745
</xmp>
737746

738747
Internal Slots {#audioencoder-internal-slots}
@@ -744,6 +753,13 @@
744753
<dd>Callback given at construction for encoded outputs.</dd>
745754
<dt><dfn attribute for=AudioEncoder>[[error callback]]</dfn></dt>
746755
<dd>Callback given at construction for encode errors.</dd>
756+
<dt><dfn attribute for=AudioEncoder>[[active encoder config]]</dfn></dt>
757+
<dd>The {{AudioEncoderConfig}} that is actively applied.</dd>
758+
<dt><dfn attribute for=AudioEncoder>[[active output config]]</dfn></dt>
759+
<dd>
760+
The {{AudioDecoderConfig}} that describes how to decode the most recently
761+
emitted {{EncodedAudioChunk}}.
762+
</dd>
747763
</dl>
748764

749765
Constructors {#audioencoder-constructors}
@@ -755,7 +771,9 @@
755771
2. Assign `init.output` to the {{AudioEncoder/[[output callback]]}} internal slot.
756772
3. Assign `init.error` to the {{AudioEncoder/[[error callback]]}} internal slot.
757773
4. Assign "unconfigured" to `e.state`.
758-
5. Return e.
774+
5. Assign `null` to {{AudioEncoder/[[active encoder config]]}}.
775+
6. Assign `null` to {{AudioEncoder/[[active output config]]}}.
776+
7. Return e.
759777

760778
Attributes {#audioencoder-attributes}
761779
-------------------------------------
@@ -801,7 +819,8 @@
801819
{{AudioEncoder/[[codec implementation]]}} with an implementation
802820
supporting |config|.
803821
3. Otherwise, run the <a>Close AudioEncoder</a> algorithm with
804-
{{NotSupportedError}}.
822+
{{NotSupportedError}} and abort these steps.
823+
4. Assign |config| to {{AudioEncoder/[[active encoder config]]}}
805824
</dd>
806825

807826
<dt><dfn method for=AudioEncoder>encode(data)</dfn></dt>
@@ -920,17 +939,48 @@
920939
|output|.
921940
2. Let |chunk| be a new {{EncodedAudioChunk}} constructed with
922941
|chunkInit|.
923-
3. Invoke {{AudioEncoder/[[output callback]]}} with |chunk|.
942+
3. Let |chunkMetadata| be a new {{EncodedAudioChunkMetadata}}.
943+
4. Let |encoderConfig| be the
944+
{{AudioEncoder/[[active encoder config]]}}.
945+
5. Let |outputConfig| be a new {{AudioDecoderConfig}} that describes
946+
|output|. Intialize |outputConfig| as follows:
947+
1. Assign |encoderConfig|.{{AudioEncoderConfig/codec}} to
948+
|outputConfig|.{{AudioDecoderConfig/codec}}.
949+
2. Assign |encoderConfig|.{{AudioEncoderConfig/sampleRate}} to
950+
|outputConfig|.{{AudioDecoderConfig/sampleRate}}.
951+
3. Assign to
952+
|encoderConfig|.{{AudioEncoderConfig/numberOfChannels}} to
953+
|outputConfig|.{{AudioDecoderConfig/numberOfChannels}}.
954+
4. Assign |outputConfig|.{{AudioDecoderConfig/description}} with a
955+
sequence of codec specific bytes as determined by the
956+
{{AudioEncoder/[[codec implementation]]}}. The user agent must
957+
ensure that the provided description could be used to
958+
correctly decode output.
959+
960+
NOTE: The codec specific requirements for populating the
961+
{{AudioDecoderConfig/description}} are described in the
962+
[[WEBCODECS-CODEC-REGISTRY]].
963+
964+
6. If |outputConfig| and {{AudioEncoder/[[active output config]]}} are
965+
not [=equal dictionaries=]:
966+
1. Assign |outputConfig| to
967+
|chunkMetadata|.{{EncodedAudioChunkMetadata/decoderConfig}}.
968+
2. Assign |outputConfig| to
969+
{{AudioEncoder/[[active output config]]}}.
970+
7. Invoke {{AudioEncoder/[[output callback]]}} with |chunk| and
971+
|chunkMetadata|.
924972
</dd>
925973
<dt><dfn>Reset AudioEncoder</dfn></dt>
926974
<dd>
927975
Run these steps:
928976
1. If {{AudioEncoder/state}} is `"closed"`, throw an {{InvalidStateError}}.
929977
2. Set {{AudioEncoder/state}} to `"unconfigured"`.
930-
3. Signal {{AudioEncoder/[[codec implementation]]}} to cease producing
978+
3. Set {{AudioEncoder/[[active encoder config]]}} to `null`.
979+
4. Set {{AudioEncoder/[[active output config]]}} to `null`.
980+
5. Signal {{AudioEncoder/[[codec implementation]]}} to cease producing
931981
output for the previous configuration.
932-
4. [=Reset the control message queue=].
933-
5. Set {{AudioEncoder/encodeQueueSize}} to zero.
982+
6. [=Reset the control message queue=].
983+
7. Set {{AudioEncoder/encodeQueueSize}} to zero.
934984
</dd>
935985
<dt><dfn>Close AudioEncoder</dfn> (with |error|)</dt>
936986
<dd>
@@ -944,6 +994,23 @@
944994
</dd>
945995
</dl>
946996

997+
EncodedAudioChunkMetadata {#encoded-audio-chunk-metadata}
998+
---------------------------------------------------------
999+
The following metadata dictionary is emitted by the
1000+
{{EncodedVideoChunkOutputCallback}} alongside an associated
1001+
{{EncodedVideoChunk}}.
1002+
1003+
<xmp class='idl'>
1004+
dictionary EncodedAudioChunkMetadata {
1005+
AudioDecoderConfig decoderConfig;
1006+
};
1007+
</xmp>
1008+
1009+
: <dfn dict-member for=EncodedAudioChunkMetadata>decoderConfig</dfn>
1010+
:: A {{AudioDecoderConfig}} that authors may use to decode the associated
1011+
{{EncodedAudioChunk}}.
1012+
1013+
9471014
VideoEncoder Interface {#videoencoder-interface}
9481015
================================================
9491016

@@ -969,7 +1036,9 @@
9691036
required WebCodecsErrorCallback error;
9701037
};
9711038

972-
callback EncodedVideoChunkOutputCallback = undefined(EncodedVideoChunk output, VideoDecoderConfig? output_config);
1039+
callback EncodedVideoChunkOutputCallback =
1040+
undefined (EncodedVideoChunk chunk,
1041+
optional EncodedVideoChunkMetadata metadata = {});
9731042
</xmp>
9741043

9751044
Internal Slots {#videoencoder-internal-slots}
@@ -1046,7 +1115,7 @@
10461115
supporting |config|.
10471116
3. Otherwise, run the <a>Close VideoEncoder</a> algorithm with
10481117
{{NotSupportedError}} and abort these steps.
1049-
2. Set {{VideoEncoder/[[active encoder config]]}} to `config`.
1118+
4. Assign |config| to {{VideoEncoder/[[active encoder config]]}}.
10501119
</dd>
10511120

10521121
<dt><dfn method for=VideoEncoder>encode(|frame|, |options|)</dfn></dt>
@@ -1155,45 +1224,7 @@
11551224
<dd>
11561225
Run these steps:
11571226
1. For each |output| in |outputs|:
1158-
1. Let |encoder_config| be the
1159-
{{VideoEncoder/[[active encoder config]]}}.
1160-
1161-
ISSUE: The intent is for |encoder_config| to be the
1162-
{{VideoEncoder/[[active encoder config]]}} that was used to
1163-
encode |output|. But, as written, it may occur that |output| was
1164-
encoded using a previous {{VideoEncoderConfig}} that has since
1165-
been replaced by a later call to {{VideoEncoder/configure()}}.
1166-
See [#138](https://github.com/w3c/webcodecs/issues/138).
1167-
1168-
2. Let |output_config| be a {{VideoDecoderConfig}} that describes
1169-
|output|. Initialize |output_config| as follows:
1170-
1. Assign `encoder_config.codec` to `output_config.codec`.
1171-
2. Assign `encoder_config.width` to `output_config.cropWidth`.
1172-
3. Assign `encoder_config.height` to `output_config.cropHeight`.
1173-
4. Assign `encoder_config.displayWidth` to
1174-
`output_config.displayWidth`.
1175-
5. Assign `encoder_config.displayHeight` to
1176-
`output_config.displayHeight`.
1177-
6. Assign the remaining keys of `output_config` as determined by
1178-
{{VideoEncoder/[[codec implementation]]}}. The user agent
1179-
must ensure that the configuration is completely described
1180-
such that |output_config| could be used to correctly decode
1181-
|output|.
1182-
1183-
NOTE: This includes supplying the
1184-
{{VideoDecoderConfig/description}} to describe codec
1185-
specific "extradata", the use of which may be further
1186-
described in codec registrations listed in the
1187-
[[WEBCODECS-CODEC-REGISTRY]].
1188-
3. If |output_config| and {{VideoEncoder/[[active output config]]}} are
1189-
<a>equal dictionaries</a>, set |output_config| to null. Otherwise,
1190-
set {{VideoEncoder/[[active output config]]}} to |output_config|.
1191-
1192-
NOTE: The {{VideoDecoderConfig}} |output_config| will be `null`
1193-
if the configuration hasn't changed from previous outputs. The
1194-
first output will always include a non-null |output_config|.
1195-
1196-
4. Let |chunkInit| be an {{EncodedVideoChunkInit}} with the following
1227+
1. Let |chunkInit| be an {{EncodedVideoChunkInit}} with the following
11971228
keys:
11981229
1. Let {{EncodedVideoChunkInit/data}} contain the encoded video data
11991230
from |output|.
@@ -1205,9 +1236,44 @@
12051236
4. Let {{EncodedVideoChunkInit/duration}} be the
12061237
{{VideoFrame/[[duration]]}} from the {{VideoFrame}} associated
12071238
with |output|.
1208-
5. Let |chunk| be a new {{EncodedVideoChunk}} constructed with
1239+
2. Let |chunk| be a new {{EncodedVideoChunk}} constructed with
12091240
|chunkInit|.
1210-
6. Invoke {{VideoEncoder/[[output callback]]}} with |chunk|.
1241+
3. Let |chunkMetadata| be a new {{EncodedVideoChunkMetadata}}.
1242+
4. Let |encoderConfig| be the
1243+
{{VideoEncoder/[[active encoder config]]}}.
1244+
5. Let |outputConfig| be a {{VideoDecoderConfig}} that describes
1245+
|output|. Initialize |outputConfig| as follows:
1246+
1. Assign `encoderConfig.codec` to `outputConfig.codec`.
1247+
2. Assign `encoderConfig.width` to `outputConfig.cropWidth`.
1248+
3. Assign `encoderConfig.height` to `outputConfig.cropHeight`.
1249+
4. Assign `encoderConfig.displayWidth` to
1250+
`outputConfig.displayWidth`.
1251+
5. Assign `encoderConfig.displayHeight` to
1252+
`outputConfig.displayHeight`.
1253+
6. Assign the remaining keys of `outputConfig` as determined by
1254+
{{VideoEncoder/[[codec implementation]]}}. The user agent
1255+
must ensure that the configuration is completely described
1256+
such that |outputConfig| could be used to correctly decode
1257+
|output|.
1258+
1259+
NOTE: The codec specific requirements for populating the
1260+
{{VideoDecoderConfig/description}} are described in the
1261+
[[WEBCODECS-CODEC-REGISTRY]].
1262+
1263+
6. If |outputConfig| and {{VideoEncoder/[[active output config]]}} are
1264+
not <a>equal dictionaries</a>:
1265+
1. Assign |outputConfig| to
1266+
|chunkMetadata|.{{EncodedVideoChunkMetadata/decoderConfig}}.
1267+
2. Assign |outputConfig| to
1268+
{{VideoEncoder/[[active output config]]}}.
1269+
7. If |encoderConfig|.{{VideoEncoderConfig/scalabilityMode}}
1270+
describes multiple [=temporal layers=]:
1271+
1. Let |temporal_layer_id| be the zero-based index describing the
1272+
temporal layer for |output|.
1273+
2. Assign |temporal_layer_id| to
1274+
|chunkMetadata|.{{EncodedVideoChunkMetadata/temporalLayerId}}.
1275+
8. Invoke {{VideoEncoder/[[output callback]]}} with |chunk| and
1276+
|chunkMetadata|.
12111277
</dd>
12121278
<dt><dfn>Reset VideoEncoder</dfn></dt>
12131279
<dd>
@@ -1233,6 +1299,27 @@
12331299
</dd>
12341300
</dl>
12351301

1302+
EncodedVideoChunkMetadata {#encoded-video-chunk-metadata}
1303+
---------------------------------------------------------
1304+
The following metadata dictionary is emitted by the
1305+
{{EncodedVideoChunkOutputCallback}} alongside an associated
1306+
{{EncodedVideoChunk}}.
1307+
1308+
<xmp class='idl'>
1309+
dictionary EncodedVideoChunkMetadata {
1310+
VideoDecoderConfig decoderConfig;
1311+
unsigned long temporalLayerId;
1312+
};
1313+
</xmp>
1314+
1315+
: <dfn dict-member for=EncodedVideoChunkMetadata>decoderConfig</dfn>
1316+
:: A {{VideoDecoderConfig}} that authors may use to decode the associated
1317+
{{EncodedVideoChunk}}.
1318+
1319+
: <dfn dict-member for=EncodedVideoChunkMetadata>temporalLayerId</dfn>
1320+
:: A number that identifying the [=temporal layer=] for the associated
1321+
{{EncodedVideoChunk}}.
1322+
12361323

12371324
Configurations{#configurations}
12381325
===============================
@@ -1569,6 +1656,7 @@
15691656
unsigned long displayWidth;
15701657
unsigned long displayHeight;
15711658
HardwareAcceleration hardwareAcceleration = "allow";
1659+
DOMString scalabilityMode;
15721660
};
15731661
</xmp>
15741662

@@ -1642,6 +1730,11 @@
16421730
Configures hardware acceleration for this codec. See
16431731
{{HardwareAcceleration}}.
16441732
</dd>
1733+
1734+
<dt><dfn dict-member for=VideoEncoderConfig>scalabilityMode</dfn></dt>
1735+
<dd>
1736+
An encoding [=scalability mode identifier=] as defined by [[WebRTC-SVC]].
1737+
</dd>
16451738
</dl>
16461739

16471740
Hardware Acceleration{#hardware-acceleration}

0 commit comments

Comments
 (0)