|
| 1 | +// Copyright (c) 2017-2022 Khronos Group. |
| 2 | +// |
| 3 | +// SPDX-License-Identifier: CC-BY-4.0 |
| 4 | + |
| 5 | +include::{generated}/meta/{refprefix}VK_EXT_present_timing.adoc[] |
| 6 | + |
| 7 | +*Last Modified Date*:: |
| 8 | + 2020-07-06 |
| 9 | +*IP Status*:: |
| 10 | + No known IP claims. |
| 11 | +*Contributors*:: |
| 12 | + - Ian Elliott, Google |
| 13 | + - James Jones, NVIDIA |
| 14 | + - Jeff Juliano, NVIDIA |
| 15 | + - Daniel Rakos, AMD |
| 16 | + - Daniel Stone, Collabora |
| 17 | + - Daniel Vetter, Intel |
| 18 | + - Aric Cyr, AMD |
| 19 | + - Faith Ekstrand, Intel |
| 20 | + - Nicolai Hähnle, AMD |
| 21 | + - Alon Or-Bach, Samsung |
| 22 | + - Niklas Smedberg, Unity Technologies |
| 23 | + - Tobias Hector, AMD |
| 24 | + |
| 25 | +This device extension allows an application that uses the |
| 26 | +`<<VK_KHR_swapchain>>` extension to obtain information about the |
| 27 | +presentation engine's display, to obtain timing information about each |
| 28 | +present, and to schedule a present to happen no earlier than a desired time. |
| 29 | +An application can use this to minimize various visual anomalies (e.g. |
| 30 | +stuttering). |
| 31 | + |
| 32 | +Traditional game and real-time animation applications need to correctly |
| 33 | +position their geometry for when the presentable image will be presented to |
| 34 | +the user. |
| 35 | +To accomplish this, applications need various timing information about the |
| 36 | +presentation engine's display. |
| 37 | +They need to know when presentable images were actually presented, and when |
| 38 | +they could have been presented. |
| 39 | +Applications also need to tell the presentation engine to display an image |
| 40 | +no sooner than a given time. |
| 41 | +This allows the application to avoid stuttering, so the animation looks |
| 42 | +smooth to the user. |
| 43 | + |
| 44 | + |
| 45 | +include::{generated}/interfaces/VK_EXT_present_timing.adoc[] |
| 46 | + |
| 47 | +=== Issues |
| 48 | + |
| 49 | +1) How does the application determine refresh duration, quanta for change, |
| 50 | +whether FRR vs. VRR, etc. |
| 51 | + |
| 52 | +*PROPOSED*: the query returns two values: 1) a refresh-cycle duration |
| 53 | +(pname:refreshDuration), and 2) an indication whether the timing is |
| 54 | +currently fixed (FRR) or variable (VRR). |
| 55 | +If pname:refreshDuration is zero, the platform cannot supply these |
| 56 | +values until after at least one flink:vkQueuePresentKHR has been done, |
| 57 | +from this time (e.g. if flink:vkQueuePresentKHR has been previously |
| 58 | +called for this swapchain, at least one additional call must be made). |
| 59 | +After calling flink:vkQueuePresentKHR, the query can be repeated until |
| 60 | +pname:refreshDuration is non-zero, at which point the FRR vs. VRR |
| 61 | +indication will also be valid. |
| 62 | + |
| 63 | +If the presentation engine's pname:refreshDuration is a fixed value, |
| 64 | +the application's image present duration (IPD) must be a multiple of |
| 65 | +pname:refreshDuration. |
| 66 | +That is, the quanta for changing the IPD is pname:refreshDuration. |
| 67 | +For example, if pname:refreshDuration is 16.67ms, the IPD can be |
| 68 | +16.67ms, 33.33ms, 50.0ms, etc. |
| 69 | + |
| 70 | +If the presentation engine's pname:refreshDuration is variable, |
| 71 | +pname:refreshDuration is the minimum value of the application's IPD, and |
| 72 | +the IPD can be larger by any quanta that is meaningful to the application. |
| 73 | +For example, if the pname:refreshDuration is 10ms (i.e. the maximum |
| 74 | +refresh rate is 100Hz), the application can choose an IPD of 11ms, |
| 75 | +13.33ms, 13.5ms, or 66.0ms; any value greater than or equal to 10ms is |
| 76 | +valid. |
| 77 | +There may be negative consequences for choosing an IPD that is too |
| 78 | +high, as the presentation engine may actually have a practical maximum |
| 79 | +pname:refreshDuration, where it needs to display the previous image |
| 80 | +again, and during this time the presentation engine might delay |
| 81 | +displaying a newly-presented image. |
| 82 | + |
| 83 | +FRR displays on at least one platform (Wayland) are not necessarily |
| 84 | +fixed; but can change over time. |
| 85 | +For example, if a full-screen video player application is visible, the display |
| 86 | +may operate at a 24Hz refresh cycle; and then later switch to 60Hz when |
| 87 | +multiple windows are visible. |
| 88 | +The special value of zero for pname:refreshDuration is used to |
| 89 | +indicate the condition where the platform cannot yet answer the query. |
| 90 | + |
| 91 | +VRR displays on some platforms can also be seen as having different |
| 92 | +characteristics over time. |
| 93 | +For example, if an application's window is full-screen-exclusive (i.e. no other |
| 94 | +window or window system component is visible), the display can look like a VRR |
| 95 | +display (however that is defined). |
| 96 | +If the application's window is not full-screen-exclusive (e.g. a normal |
| 97 | +multi-window case), the display can look like an FRR display (i.e. because the |
| 98 | +compositor is trying to treat all windows in a consistent manner). |
| 99 | +A different issue will deal with these how the timing characteristics |
| 100 | +can change over time. |
| 101 | + |
| 102 | + |
| 103 | +2) Do we return min/max Values for Refresh Duration for VRR? |
| 104 | + |
| 105 | +*PROPOSED*: return only the minimum value of refreshDuration for a VRR. |
| 106 | + |
| 107 | +VRR displays have a minimum and maximum refresh rate, and therefore a minimum |
| 108 | +and maximum refreshDuration. |
| 109 | +It has been asserted that the display effectively does not have a minimum |
| 110 | +refresh rate. |
| 111 | +That is because if an application does not present soon enough, the display |
| 112 | +hardware will automatically re-display the previous image. |
| 113 | +However, when the display does that, an application cannot present a new image |
| 114 | +for a certain period of time. |
| 115 | +It is unclear about whether that period is large enough to cause visual |
| 116 | +artifacts. |
| 117 | + |
| 118 | + |
| 119 | +3) How to deal with changes in timing properties? |
| 120 | + |
| 121 | +*RESOLVED*: The slink:VkPastPresentationTimingEXT structure that is |
| 122 | +returned by flink:vkGetPastPresentationTimingEXT will contain |
| 123 | +pname:timeDomainChanged, which will be ename:VK_TRUE if the time |
| 124 | +domain enabled for the swapchain is not currently available. |
| 125 | + |
| 126 | +An example of why display timing properties can change is if a surface |
| 127 | +changes from being a window that’s a subset of the display size, to |
| 128 | +becoming full-screen-exclusive. |
| 129 | +While the surface was a subset of the display, a compositor might |
| 130 | +enforce fixed timings on the surface (e.g. FRR of 60Hz), where the |
| 131 | +presentation engine might be free to allow VRR behavior of a |
| 132 | +full-screen-exclusive surface. |
| 133 | + |
| 134 | +It is possible that a full-screen-exclusive window can become |
| 135 | +temporarily obscured (e.g. when a short-term dialog pops up). |
| 136 | +In this case, the surface might use FRR timings while the dialog is |
| 137 | +visible and VRR otherwise. |
| 138 | + |
| 139 | + |
| 140 | +4) One Query for all Timing info vs. an initial query to determine FRR vs. VRR, |
| 141 | +and then FRR-specific vs VRR-specific queries? |
| 142 | + |
| 143 | +*PROPOSED*: Have one query, as described in issue 1, that can be |
| 144 | +called whenever the application needs to obtain the timing properties |
| 145 | +of the surface. |
| 146 | + |
| 147 | + |
| 148 | +5) Query to Determine Time Domain? |
| 149 | + |
| 150 | +*PROPOSED*: Have a query to determine the time domain. |
| 151 | +This extension will define some return values, including some that are |
| 152 | +platform-specific. |
| 153 | +Other extensions can add other time domains. |
| 154 | + |
| 155 | + |
| 156 | +6) What Time to use for targetPresentTime for Early Images? |
| 157 | + |
| 158 | +*PROPOSED*: Have no query for determining the current time in the PE’s time |
| 159 | +domain; and do allow the special value of zero for targetPresentTime and |
| 160 | +idealPresentTime, meaning that there is no target nor ideal time. |
| 161 | + |
| 162 | +On some platforms, there is no way to determine the current time, nor |
| 163 | +to determine surface timing properties until after at least one image |
| 164 | +has been presented. |
| 165 | + |
| 166 | +In such cases, the special value of zero allows the application to |
| 167 | +indicate that timing feedback is desired, but that no |
| 168 | +targetPresentTime nor idealPresentTime is requested. |
| 169 | +Later, once the application has obtained feedback, it can specify |
| 170 | +targetPresentTime and idealPresentTime. |
| 171 | + |
| 172 | + |
| 173 | +7) How long before an application’s request for new image duration is honored? |
| 174 | + |
| 175 | +*UNRESOLVED*: Apparently, changes to some vendors' display hardware settings do |
| 176 | +not take effect immediately. |
| 177 | +It is not clear what settings, and therefore, it is not clear how to |
| 178 | +address this issue. |
| 179 | + |
| 180 | + |
| 181 | +8) Do we have a query for the anticipated latency from present to feedback? |
| 182 | + |
| 183 | +*UNRESOLVED*: There is some amount of latency from when an application calls |
| 184 | +vkQueuePresentKHR to when the image is displayed to the user, to when feedback |
| 185 | +is available to the application on when the image was actually displayed to the |
| 186 | +user. |
| 187 | +The first time (from the call till the image is presented) generally doesn’t |
| 188 | +matter, because the application will likely be providing a targetPresentTime |
| 189 | +(i.e. the application may have some indication for how long this will be). |
| 190 | +However, the latency between targetPresentTime until feedback is available may |
| 191 | +be much long. |
| 192 | +For example, on Android on the 1st-generation Pixel phone (60Hz FRR display), |
| 193 | +the latency was approximately 5 refresh cycles (83.33ms). |
| 194 | +For higher-frequency displays, the latency may have a larger number of refresh |
| 195 | +cycles. |
| 196 | + |
| 197 | +Is there value in having a query for the application to know how long it may |
| 198 | +have to wait for feedback? |
| 199 | +Can such a query be reasonably answered by the driver? |
| 200 | + |
| 201 | +Is there other interesting information in this space that we may wish to |
| 202 | +capture? |
| 203 | + |
| 204 | + |
| 205 | +9) Do we have a query(s) about the number of VkPastPresentationTimingEXT |
| 206 | +structs to keep? |
| 207 | + |
| 208 | +*UNRESOLVED*: At the Montreal F2F, there was discussion about how much |
| 209 | +feedback that the driver needs to keep and/or how much feedback that |
| 210 | +the application needs to be able to query. |
| 211 | + |
| 212 | +The way that the LunarG cube demo (official WSI example code) used |
| 213 | +VK_GOOGLE_display_timing, and what is proposed for the new extension is that |
| 214 | +the application query what feedback is available during every render-present |
| 215 | +loop. |
| 216 | +If the application never skips querying for feedback, and always obtains |
| 217 | +whatever feedback is available, there doesn’t seem much need for such a |
| 218 | +query(s). |
| 219 | +What I saw with the cube demo on a Pixel phone was that most of the time, the |
| 220 | +application obtained feedback for 1 previous present. |
| 221 | +Occasionally, it would get 2 VkPastPresentationTimingEXT structs on time and |
| 222 | +then 0 the next (or vice versa). |
| 223 | + |
| 224 | +Perhaps, a video player application might present several images at once, and |
| 225 | +then later get feedback for several images at the same time. |
| 226 | +That would be the most-likely use-case that I can come up with for why a query |
| 227 | +might be useful. |
| 228 | +Is that compelling enough? |
| 229 | + |
| 230 | +What might the model for the query(s) be? |
| 231 | +Potentially the application can tell the driver how many presents it might do |
| 232 | +at a time, and the driver can use that to size its internal buffer. |
| 233 | +Is there value in having a query that would influence the driver’s behavior |
| 234 | +(beyond what’s provided for in the currently-proposed API)? |
| 235 | + |
| 236 | +10) How is the SWAPCHAIN_LOCAL time domain used with the calibrated |
| 237 | +timestamps extension? |
| 238 | + |
| 239 | +PROPOSED: Define a struct to chain into VkCalibratedTimestampInfoEXT::pNext |
| 240 | +that specifies a swapchain. |
| 241 | +Is anything additional needed for |
| 242 | +vkGetPhysicalDeviceCalibrateableTimeDomainsEXT, or are swapchain-local |
| 243 | +timestamps always calibrateable or always not calibrateable for a given |
| 244 | +device? |
| 245 | + |
| 246 | +11) Should VK_PRESENT_MODE_FIFO_LATEST_READY_EXT be part of this extension, |
| 247 | +or split out into its own extension? |
| 248 | + |
| 249 | +PROPOSED: It is only tangentially related. |
| 250 | +Split it out into its own extension and define the interaction here. |
| 251 | + |
| 252 | +=== Version History |
| 253 | + |
| 254 | + * Revision 1, 2018-05-11 (Ian Elliott) |
| 255 | + ** Internal revisions |
0 commit comments