Skip to content

Commit 1812fde

Browse files
ianelliottusnvlduc
authored andcommitted
WIP: Add the VK_EXT_present_timing extension
This extension allows an application that uses the VK_KHR_swapchain extension to obtain information about the presentation engine's display, to obtain timing information about each present operation, and to schedule a present to happen at a specific time. Applications can use this to minimize various visual anomalies (e.g., stuttering).
1 parent ce03761 commit 1812fde

File tree

9 files changed

+1062
-94
lines changed

9 files changed

+1062
-94
lines changed

appendices/VK_EXT_present_timing.adoc

Lines changed: 255 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,255 @@
1+
// Copyright (c) 2017-2022 Khronos Group.
2+
//
3+
// SPDX-License-Identifier: CC-BY-4.0
4+
5+
include::{generated}/meta/{refprefix}VK_EXT_present_timing.adoc[]
6+
7+
*Last Modified Date*::
8+
2020-07-06
9+
*IP Status*::
10+
No known IP claims.
11+
*Contributors*::
12+
- Ian Elliott, Google
13+
- James Jones, NVIDIA
14+
- Jeff Juliano, NVIDIA
15+
- Daniel Rakos, AMD
16+
- Daniel Stone, Collabora
17+
- Daniel Vetter, Intel
18+
- Aric Cyr, AMD
19+
- Faith Ekstrand, Intel
20+
- Nicolai Hähnle, AMD
21+
- Alon Or-Bach, Samsung
22+
- Niklas Smedberg, Unity Technologies
23+
- Tobias Hector, AMD
24+
25+
This device extension allows an application that uses the
26+
`<<VK_KHR_swapchain>>` extension to obtain information about the
27+
presentation engine's display, to obtain timing information about each
28+
present, and to schedule a present to happen no earlier than a desired time.
29+
An application can use this to minimize various visual anomalies (e.g.
30+
stuttering).
31+
32+
Traditional game and real-time animation applications need to correctly
33+
position their geometry for when the presentable image will be presented to
34+
the user.
35+
To accomplish this, applications need various timing information about the
36+
presentation engine's display.
37+
They need to know when presentable images were actually presented, and when
38+
they could have been presented.
39+
Applications also need to tell the presentation engine to display an image
40+
no sooner than a given time.
41+
This allows the application to avoid stuttering, so the animation looks
42+
smooth to the user.
43+
44+
45+
include::{generated}/interfaces/VK_EXT_present_timing.adoc[]
46+
47+
=== Issues
48+
49+
1) How does the application determine refresh duration, quanta for change,
50+
whether FRR vs. VRR, etc.
51+
52+
*PROPOSED*: the query returns two values: 1) a refresh-cycle duration
53+
(pname:refreshDuration), and 2) an indication whether the timing is
54+
currently fixed (FRR) or variable (VRR).
55+
If pname:refreshDuration is zero, the platform cannot supply these
56+
values until after at least one flink:vkQueuePresentKHR has been done,
57+
from this time (e.g. if flink:vkQueuePresentKHR has been previously
58+
called for this swapchain, at least one additional call must be made).
59+
After calling flink:vkQueuePresentKHR, the query can be repeated until
60+
pname:refreshDuration is non-zero, at which point the FRR vs. VRR
61+
indication will also be valid.
62+
63+
If the presentation engine's pname:refreshDuration is a fixed value,
64+
the application's image present duration (IPD) must be a multiple of
65+
pname:refreshDuration.
66+
That is, the quanta for changing the IPD is pname:refreshDuration.
67+
For example, if pname:refreshDuration is 16.67ms, the IPD can be
68+
16.67ms, 33.33ms, 50.0ms, etc.
69+
70+
If the presentation engine's pname:refreshDuration is variable,
71+
pname:refreshDuration is the minimum value of the application's IPD, and
72+
the IPD can be larger by any quanta that is meaningful to the application.
73+
For example, if the pname:refreshDuration is 10ms (i.e. the maximum
74+
refresh rate is 100Hz), the application can choose an IPD of 11ms,
75+
13.33ms, 13.5ms, or 66.0ms; any value greater than or equal to 10ms is
76+
valid.
77+
There may be negative consequences for choosing an IPD that is too
78+
high, as the presentation engine may actually have a practical maximum
79+
pname:refreshDuration, where it needs to display the previous image
80+
again, and during this time the presentation engine might delay
81+
displaying a newly-presented image.
82+
83+
FRR displays on at least one platform (Wayland) are not necessarily
84+
fixed; but can change over time.
85+
For example, if a full-screen video player application is visible, the display
86+
may operate at a 24Hz refresh cycle; and then later switch to 60Hz when
87+
multiple windows are visible.
88+
The special value of zero for pname:refreshDuration is used to
89+
indicate the condition where the platform cannot yet answer the query.
90+
91+
VRR displays on some platforms can also be seen as having different
92+
characteristics over time.
93+
For example, if an application's window is full-screen-exclusive (i.e. no other
94+
window or window system component is visible), the display can look like a VRR
95+
display (however that is defined).
96+
If the application's window is not full-screen-exclusive (e.g. a normal
97+
multi-window case), the display can look like an FRR display (i.e. because the
98+
compositor is trying to treat all windows in a consistent manner).
99+
A different issue will deal with these how the timing characteristics
100+
can change over time.
101+
102+
103+
2) Do we return min/max Values for Refresh Duration for VRR?
104+
105+
*PROPOSED*: return only the minimum value of refreshDuration for a VRR.
106+
107+
VRR displays have a minimum and maximum refresh rate, and therefore a minimum
108+
and maximum refreshDuration.
109+
It has been asserted that the display effectively does not have a minimum
110+
refresh rate.
111+
That is because if an application does not present soon enough, the display
112+
hardware will automatically re-display the previous image.
113+
However, when the display does that, an application cannot present a new image
114+
for a certain period of time.
115+
It is unclear about whether that period is large enough to cause visual
116+
artifacts.
117+
118+
119+
3) How to deal with changes in timing properties?
120+
121+
*RESOLVED*: The slink:VkPastPresentationTimingEXT structure that is
122+
returned by flink:vkGetPastPresentationTimingEXT will contain
123+
pname:timeDomainChanged, which will be ename:VK_TRUE if the time
124+
domain enabled for the swapchain is not currently available.
125+
126+
An example of why display timing properties can change is if a surface
127+
changes from being a window that’s a subset of the display size, to
128+
becoming full-screen-exclusive.
129+
While the surface was a subset of the display, a compositor might
130+
enforce fixed timings on the surface (e.g. FRR of 60Hz), where the
131+
presentation engine might be free to allow VRR behavior of a
132+
full-screen-exclusive surface.
133+
134+
It is possible that a full-screen-exclusive window can become
135+
temporarily obscured (e.g. when a short-term dialog pops up).
136+
In this case, the surface might use FRR timings while the dialog is
137+
visible and VRR otherwise.
138+
139+
140+
4) One Query for all Timing info vs. an initial query to determine FRR vs. VRR,
141+
and then FRR-specific vs VRR-specific queries?
142+
143+
*PROPOSED*: Have one query, as described in issue 1, that can be
144+
called whenever the application needs to obtain the timing properties
145+
of the surface.
146+
147+
148+
5) Query to Determine Time Domain?
149+
150+
*PROPOSED*: Have a query to determine the time domain.
151+
This extension will define some return values, including some that are
152+
platform-specific.
153+
Other extensions can add other time domains.
154+
155+
156+
6) What Time to use for targetPresentTime for Early Images?
157+
158+
*PROPOSED*: Have no query for determining the current time in the PE’s time
159+
domain; and do allow the special value of zero for targetPresentTime and
160+
idealPresentTime, meaning that there is no target nor ideal time.
161+
162+
On some platforms, there is no way to determine the current time, nor
163+
to determine surface timing properties until after at least one image
164+
has been presented.
165+
166+
In such cases, the special value of zero allows the application to
167+
indicate that timing feedback is desired, but that no
168+
targetPresentTime nor idealPresentTime is requested.
169+
Later, once the application has obtained feedback, it can specify
170+
targetPresentTime and idealPresentTime.
171+
172+
173+
7) How long before an application’s request for new image duration is honored?
174+
175+
*UNRESOLVED*: Apparently, changes to some vendors' display hardware settings do
176+
not take effect immediately.
177+
It is not clear what settings, and therefore, it is not clear how to
178+
address this issue.
179+
180+
181+
8) Do we have a query for the anticipated latency from present to feedback?
182+
183+
*UNRESOLVED*: There is some amount of latency from when an application calls
184+
vkQueuePresentKHR to when the image is displayed to the user, to when feedback
185+
is available to the application on when the image was actually displayed to the
186+
user.
187+
The first time (from the call till the image is presented) generally doesn’t
188+
matter, because the application will likely be providing a targetPresentTime
189+
(i.e. the application may have some indication for how long this will be).
190+
However, the latency between targetPresentTime until feedback is available may
191+
be much long.
192+
For example, on Android on the 1st-generation Pixel phone (60Hz FRR display),
193+
the latency was approximately 5 refresh cycles (83.33ms).
194+
For higher-frequency displays, the latency may have a larger number of refresh
195+
cycles.
196+
197+
Is there value in having a query for the application to know how long it may
198+
have to wait for feedback?
199+
Can such a query be reasonably answered by the driver?
200+
201+
Is there other interesting information in this space that we may wish to
202+
capture?
203+
204+
205+
9) Do we have a query(s) about the number of VkPastPresentationTimingEXT
206+
structs to keep?
207+
208+
*UNRESOLVED*: At the Montreal F2F, there was discussion about how much
209+
feedback that the driver needs to keep and/or how much feedback that
210+
the application needs to be able to query.
211+
212+
The way that the LunarG cube demo (official WSI example code) used
213+
VK_GOOGLE_display_timing, and what is proposed for the new extension is that
214+
the application query what feedback is available during every render-present
215+
loop.
216+
If the application never skips querying for feedback, and always obtains
217+
whatever feedback is available, there doesn’t seem much need for such a
218+
query(s).
219+
What I saw with the cube demo on a Pixel phone was that most of the time, the
220+
application obtained feedback for 1 previous present.
221+
Occasionally, it would get 2 VkPastPresentationTimingEXT structs on time and
222+
then 0 the next (or vice versa).
223+
224+
Perhaps, a video player application might present several images at once, and
225+
then later get feedback for several images at the same time.
226+
That would be the most-likely use-case that I can come up with for why a query
227+
might be useful.
228+
Is that compelling enough?
229+
230+
What might the model for the query(s) be?
231+
Potentially the application can tell the driver how many presents it might do
232+
at a time, and the driver can use that to size its internal buffer.
233+
Is there value in having a query that would influence the driver’s behavior
234+
(beyond what’s provided for in the currently-proposed API)?
235+
236+
10) How is the SWAPCHAIN_LOCAL time domain used with the calibrated
237+
timestamps extension?
238+
239+
PROPOSED: Define a struct to chain into VkCalibratedTimestampInfoEXT::pNext
240+
that specifies a swapchain.
241+
Is anything additional needed for
242+
vkGetPhysicalDeviceCalibrateableTimeDomainsEXT, or are swapchain-local
243+
timestamps always calibrateable or always not calibrateable for a given
244+
device?
245+
246+
11) Should VK_PRESENT_MODE_FIFO_LATEST_READY_EXT be part of this extension,
247+
or split out into its own extension?
248+
249+
PROPOSED: It is only tangentially related.
250+
Split it out into its own extension and define the interaction here.
251+
252+
=== Version History
253+
254+
* Revision 1, 2018-05-11 (Ian Elliott)
255+
** Internal revisions

appendices/glossary.adoc

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -730,6 +730,11 @@ Framebuffer Region::
730730
A framebuffer region is a set of sample (x, y, layer, sample)
731731
coordinates that is a subset of the entire framebuffer.
732732

733+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
734+
Frame Rate::
735+
A non-Vulkan term for Image Present Rate (IPR).
736+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
737+
733738
Front-Facing::
734739
See Facingness.
735740

@@ -822,6 +827,18 @@ Image::
822827
of device memory.
823828
Represented by a slink:VkImage object.
824829

830+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
831+
Image Present Duration::
832+
The amount of time the application intends for each
833+
newly-presented image to be visible to the user.
834+
This value should: be a multiple of the refresh cycle duration.
835+
836+
Image Present Rate::
837+
The number of newly-presented images the application intends to present
838+
each second (a.k.a. frame rate).
839+
This value should: be a multiple of the refresh rate.
840+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
841+
825842
Image Subresource::
826843
A specific mipmap level, layer, and set of aspects of an image.
827844

@@ -1011,6 +1028,11 @@ Invocation Repack Instruction::
10111028
implementation may: change the set of invocations that are executing.
10121029
endif::VK_KHR_ray_tracing_pipeline,VK_NV_ray_tracing[]
10131030

1031+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1032+
IPD::
1033+
Image Present Duration.
1034+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1035+
10141036
ifdef::VK_KHR_deferred_host_operations[]
10151037
Join (Deferred Host Operations)::
10161038
The act of instructing a thread to participate in the execution of a
@@ -1530,6 +1552,11 @@ ifdef::VK_KHR_ray_tracing_pipeline[flink:vkCmdTraceRaysKHR, and flink:vkCmdTrace
15301552
.
15311553
endif::VK_KHR_ray_tracing_pipeline,VK_NV_ray_tracing[]
15321554

1555+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1556+
RC::
1557+
Refresh Cycle.
1558+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1559+
15331560
ifdef::VK_KHR_video_queue[]
15341561
Reconstructed Picture::
15351562
A video picture resource reconstructed from a compressed bitstream using
@@ -1552,6 +1579,17 @@ Reference Picture Metadata::
15521579
Opaque state associated with a DPB slot, maintained by a video session.
15531580
endif::VK_KHR_video_queue[]
15541581

1582+
ifdef::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1583+
Refresh Cycle::
1584+
The periodic process for updating the contents of the Presentation Engine's display.
1585+
1586+
Refresh Cycle Duration::
1587+
The amount of time from the start of one refresh cycle to the next.
1588+
1589+
Refresh Rate::
1590+
The number of refresh cycles per second.
1591+
endif::VK_EXT_present_timing,VK_GOOGLE_display_timing[]
1592+
15551593
Release Operation (Resource)::
15561594
An operation that releases ownership of an image subresource or buffer
15571595
range.

0 commit comments

Comments
 (0)