feat: Add custom placeholder pages for scale-from-zero scenarios #1303

malpou · 2025-05-30T10:03:31Z

Allows HTTPScaledObjects to serve configurable HTML pages while workloads scale up from zero, with support for templates,
custom headers, and automatic refresh.

Description

This PR implements customizable placeholder pages that are displayed to users while applications are scaling up from zero.
This significantly improves the user experience during cold starts by providing immediate feedback instead of connection
timeouts or errors.

Key features:

Adds placeholderConfig section to HTTPScaledObject CRD for configuring placeholder pages
Supports both inline HTML content and ConfigMap-based templates
Includes Go template support with variables like {{.ServiceName}} and {{.RefreshInterval}}
Automatic page refresh capability to check when the service becomes available
Custom HTTP headers and status codes
Configurable timeout for how long to show placeholder pages

Implementation details:

New placeholder handler in the interceptor that serves configured HTML when backends are unavailable
Graceful fallback to upstream requests once the service is ready
Full backward compatibility - disabled by default

Checklist

Commits are signed with Developer Certificate of Origin (DCO)
Changelog has been updated and is aligned with our changelog requirements
Any necessary documentation is added, such as:

Fixes #874

JorTurFer

This is a great feature! Thanks for the contribution 🙇

JorTurFer · 2025-06-01T18:53:59Z

interceptor/handler/placeholder.go

+<html>
+<head>
+    <title>Service Starting</title>
+    <meta http-equiv="refresh" content="{{.RefreshInterval}}">


Should we inject this value instead of expect that users provide it? I mean, this looks as something that we want to enforce. @wozniakjan ?

I was thinking about injecting it but wanted some feedback first.

I think it also would be possible to add some JavaScript instead which calls the URL the browser is on and checks if the X-KEDA-HTTP-Placeholder-Served: true header is there. If it's not we should do a full refresh.
This avoids having the browser do an actual refresh every X seconds. This would mostly be an UX improvement.

Both approaches could be injected into whatever template the user provides, if that's the approach you would prefer.

Example:

<script> (function() { const checkInterval = {{.RefreshInterval}} * 1000; // Convert seconds to milliseconds async function checkServiceStatus() { try { // Make a HEAD request to the current URL to check headers const response = await fetch(window.location.href, { method: 'HEAD', cache: 'no-cache' }); // Check if the placeholder header is present const placeholderHeader = response.headers.get('X-KEDA-HTTP-Placeholder-Served'); if (placeholderHeader !== 'true') { // Service is ready! Do a full page refresh window.location.reload(); } else { // Still showing placeholder, check again later setTimeout(checkServiceStatus, checkInterval); } } catch (error) { // On error, continue checking console.error('Error checking service status:', error); setTimeout(checkServiceStatus, checkInterval); } } // Start checking after the interval setTimeout(checkServiceStatus, checkInterval); })(); </script>

I like this approach! it's quite elegant IMHO, the only thing is that JS needs to be injected as well, doesn't

@wozniakjan @zroubalik WDYT?

@JorTurFer I've tried to make a take on the injection with a <script> tag and some JS to figure out when to refresh the browser.

interceptor/handler/placeholder.go

interceptor/proxy_handlers.go

tests/checks/placeholder_pages/placeholder_pages_test.go

Allows HTTPScaledObjects to serve configurable HTML pages while workloads scale up from zero, with support for templates, custom headers, and automatic refresh. Signed-off-by: malpou <[email protected]>

Signed-off-by: malpou <[email protected]>

…itten to Signed-off-by: malpou <[email protected]>

…should be comptatible in the GHA environment Signed-off-by: malpou <[email protected]>

Signed-off-by: Malthe Poulsen <[email protected]>

Signed-off-by: malpou <[email protected]>

Signed-off-by: Malthe Poulsen <[email protected]>

interceptor/handler/placeholder.go

tests/checks/placeholder_pages/placeholder_pages_test.go

interceptor/handler/placeholder_test.go

Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: Malthe Poulsen <[email protected]>

Signed-off-by: malpou <[email protected]>

Copilot

Pull Request Overview

This PR introduces configurable placeholder pages for scale-from-zero scenarios, enhancing the user experience during cold starts. Key changes include:

Adding a new placeholder handler and supporting API changes in the HTTPScaledObject CRD.
Adjusting proxy handler logic and test cases to incorporate placeholder page responses.
Updating documentation, examples, and deepcopy functions to reflect the new placeholderConfig functionality.

Reviewed Changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/checks/placeholder_pages/placeholder_pages_test.go	Added E2E tests for placeholder page responses and script injection.
operator/apis/http/v1alpha1/zz_generated.deepcopy.go	Added DeepCopy functions for the new PlaceholderConfig type.
operator/apis/http/v1alpha1/httpscaledobject_types.go	Introduced the PlaceholderConfig type in the CRD schema.
interceptor/proxy_handlers_test.go, proxy_handlers_integration_test.go	Updated tests to pass new parameters for placeholder handling.
interceptor/proxy_handlers.go	Integrated the placeholder handler into the forwarding logic.
interceptor/main.go, main_test.go	Updated server initialization to include the placeholder handler.
interceptor/handler/placeholder.go	Implemented the placeholder page rendering and caching logic.
examples/vX.X.X/httpscaledobject.yaml	Provided example configuration for placeholder pages.
docs/ref/vX.X.X/http_scaled_object.md	Documented the placeholderConfig section details.
config/crd/bases/http.keda.sh_httpscaledobjects.yaml	Updated the CRD with placeholderConfig properties.
CHANGELOG.md	Added an entry for the custom placeholder pages feature.

Comments suppressed due to low confidence (1)

interceptor/handler/placeholder.go:206

[nitpick] Consider logging the error returned from getTemplate before falling back to the inline response to improve debuggability of template parsing issues.

tmpl, err := h.getTemplate(r.Context(), hso)

interceptor/handler/placeholder.go

JorTurFer

awesome job! Only one small nit inline and it's okey to merge ❤️

interceptor/handler/placeholder.go

Nico385412 · 2025-07-09T01:00:24Z

Hello any news on this feature ? 🙏

JorTurFer · 2025-07-14T20:07:18Z

Sorry, @wozniakjan told me that he wanted to review the feature and I just kept this to him.
Did you take a look @wozniakjan ?

wozniakjan

this looks really great, thank you for contributing!

I do have a couple of discussion points before it gets merged

usage of ConfigMaps for storing content for placeholder pages - imho defaults would be better configured globally as a statically mounted ConfigMap or env variable on Deployment
simplifying content processing - the default as a nicely formatted HTML page makes great sense but then we should allow arbitrary payloads with arbitrary headers, not all usage of this tool is aimed at web browsers, sometimes it's microservices talking between each other

wozniakjan · 2025-07-25T08:33:15Z

interceptor/handler/placeholder.go

+const placeholderScript = `<script>
+(function() {
+	const checkInterval = {{.RefreshInterval}} * 1000;
+
+	async function checkServiceStatus() {
+		try {
+			const response = await fetch(window.location.href, {
+				method: 'HEAD',
+				cache: 'no-cache'
+			});
+
+			const placeholderHeader = response.headers.get('X-KEDA-HTTP-Placeholder-Served');
+
+			if (placeholderHeader !== 'true') {
+				window.location.reload();
+			} else {
+				setTimeout(checkServiceStatus, checkInterval);
+			}
+		} catch (error) {
+			console.error('Error checking service status:', error);
+			setTimeout(checkServiceStatus, checkInterval);
+		}
+	}
+
+	setTimeout(checkServiceStatus, checkInterval);
+})();
+</script>`
+
+const defaultPlaceholderTemplateWithoutScript = `<!DOCTYPE html>
+<html>
+<head>
+    <title>Service Starting</title>
+    <meta charset="utf-8">
+    <style>
+        body {
+            font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
+            display: flex;
+            align-items: center;
+            justify-content: center;
+            min-height: 100vh;
+            margin: 0;
+            background: #f5f5f5;
+        }
+        .container {
+            text-align: center;
+            padding: 2rem;
+            background: white;
+            border-radius: 8px;
+            box-shadow: 0 2px 10px rgba(0,0,0,0.1);
+            max-width: 400px;
+        }
+        h1 {
+            color: #333;
+            margin-bottom: 1rem;
+            font-size: 1.5rem;
+        }
+        .spinner {
+            width: 40px;
+            height: 40px;
+            margin: 1.5rem auto;
+            border: 4px solid #f3f3f3;
+            border-top: 4px solid #3498db;
+            border-radius: 50%;
+            animation: spin 1s linear infinite;
+        }
+        @keyframes spin {
+            0% { transform: rotate(0deg); }
+            100% { transform: rotate(360deg); }
+        }
+        p {
+            color: #666;
+            margin: 0.5rem 0;
+        }
+    </style>
+</head>
+<body>
+    <div class="container">
+        <h1>{{.ServiceName}} is starting up...</h1>
+        <div class="spinner"></div>
+        <p>Please wait while we prepare your service.</p>
+    </div>
+</body>
+</html>`


wdyt about moving these default templates to configuration instead of having them hardcoded in the binary? I can imagine setting it through environment variables in the Deployment or alternatively, which might be even better, mounted as a ConfigMap.

http-add-on/config/interceptor/deployment.yaml

Line 30 in bf35564

env:

These default pages are pretty good examples, but I'd imagine people may want to tweak them a little bit or centrally replace with different defaults. As implemented, iiuc, users would have to copy the same ConfigMap into each namespace and then reference it in the HTTPScaledObject or inline the same string in each HTTPScaledObject.

wozniakjan · 2025-07-25T09:08:44Z

interceptor/handler/placeholder.go

+	}
+
+	// For non-HTML content, wrap it in a minimal HTML structure with the script
+	return fmt.Sprintf(`<!DOCTYPE html>


what if users don't want HTML content? The client doesn't have to be web browser, it could be two microservices talking to each other and they may have other L7 protocols (json, protobuf, etc) with their own mechanisms on how to handle retries.

wozniakjan · 2025-07-25T09:34:53Z

operator/apis/http/v1alpha1/httpscaledobject_types.go

+	// Path to ConfigMap containing placeholder HTML content (in same namespace)
+	// +optional
+	ContentConfigMap string `json:"contentConfigMap,omitempty" description:"ConfigMap name containing placeholder HTML content"`
+	// Key in ConfigMap containing the HTML template
+	// +kubebuilder:default="template.html"
+	// +optional
+	ContentConfigMapKey string `json:"contentConfigMapKey,omitempty" description:"Key in ConfigMap containing the HTML template"`


mind the fact interceptor doesn't have RBAC for ConfigMaps. Is it worth adding clusterwide read access to ConfigMaps so users can configure content of a placeholder page per namespace? Imho it's a bit too much and I would rather just add defaults as mounted ConfigMaps to interceptor or env variables on the Deployment as mentioned above.

wozniakjan · 2025-07-25T09:44:48Z

interceptor/handler/placeholder.go

+	if config.ContentConfigMap != "" {
+		cacheKey := fmt.Sprintf("%s/%s/cm/%s", hso.Namespace, hso.Name, config.ContentConfigMap)
+
+		cm, err := h.k8sClient.CoreV1().ConfigMaps(hso.Namespace).Get(ctx, config.ContentConfigMap, metav1.GetOptions{})


I would rather not use ConfigMaps at the HTTPScaledObject level but if we decide to go with them, please use cached client instead of the direct client. Fetching ConfigMap for each request on zero-scaled apps might be tricky with kube-client QPS and could easily cause a lot of load for kube-apiserver if users increase the burst config to "deal with errors".

wozniakjan · 2025-07-25T09:52:10Z

interceptor/handler/placeholder.go

+// PlaceholderHandler handles serving placeholder pages during scale-from-zero
+type PlaceholderHandler struct {
+	k8sClient     kubernetes.Interface
+	routingTable  routing.Table


Suggested change

routingTable routing.Table

the routing table seems to be unused, there are only three references and they don't seem to carry much value?

1 interceptor/handler/placeholder.go|114 col 2| routingTable routing.Table 2 interceptor/handler/placeholder.go|140 col 3| routingTable: routingTable, 3 interceptor/handler/placeholder_test.go|32 col 27| assert.NotNil(t, handler.routingTable)

wozniakjan · 2025-07-25T10:10:00Z

interceptor/proxy_handlers.go

+			endpoints, err := endpointsCache.Get(httpso.GetNamespace(), httpso.Spec.ScaleTargetRef.Service)
+			if err == nil && workloadActiveEndpoints(endpoints) == 0 {
+				if placeholderErr := placeholderHandler.ServePlaceholder(w, r, httpso); placeholderErr != nil {
+					lggr.Error(placeholderErr, "failed to serve placeholder page")
+					w.WriteHeader(http.StatusBadGateway)
+					if _, err := w.Write([]byte("error serving placeholder page")); err != nil {
+						lggr.Error(err, "could not write error response to client")
+					}
+				}
+				return
+			}


situation when placeholder pages are configured and the endpoints cache returns error is not handled, which means the code will fall through to waiting on cold-start. Is that desired, or should the interceptor instead return a 503 error indicating an internal problem?

malpou requested a review from a team as a code owner May 30, 2025 10:03

malpou closed this May 30, 2025

malpou reopened this May 30, 2025

malpou force-pushed the placeholder-pages branch from 92f8f69 to 02451ee Compare May 30, 2025 10:33

JorTurFer reviewed Jun 1, 2025

View reviewed changes

JorTurFer reviewed Jun 2, 2025

View reviewed changes

tests/checks/placeholder_pages/placeholder_pages_test.go Outdated Show resolved Hide resolved

malpou added 7 commits June 2, 2025 21:19

feat: Add custom placeholder pages for scale-from-zero scenarios

97a04e6

Allows HTTPScaledObjects to serve configurable HTML pages while workloads scale up from zero, with support for templates, custom headers, and automatic refresh. Signed-off-by: malpou <[email protected]>

remove ClearCache funtion

c7192d3

Signed-off-by: malpou <[email protected]>

remove info log

b6c68aa

Signed-off-by: malpou <[email protected]>

use same image for e2e tests

ce3bf5e

Signed-off-by: malpou <[email protected]>

adjust tests according to comments

906d006

Signed-off-by: malpou <[email protected]>

add RWMutex to ensure that the template cache is locked when being wr…

85fe5f1

…itten to Signed-off-by: malpou <[email protected]>

adjust e2e tests away from my local setup and over to something that …

a3d021f

…should be comptatible in the GHA environment Signed-off-by: malpou <[email protected]>

malpou force-pushed the placeholder-pages branch from 2e3f5f5 to a3d021f Compare June 2, 2025 19:19

malpou and others added 7 commits June 3, 2025 13:43

Update placeholder_pages_test.go

d723af0

Signed-off-by: Malthe Poulsen <[email protected]>

take on injecting reloading watching js using HEAD requests

7fb8adc

Signed-off-by: malpou <[email protected]>

remove outdated text

1cb5317

Signed-off-by: malpou <[email protected]>

fix unit tests

937a69c

Signed-off-by: Malthe Poulsen <[email protected]>

fix e2e tests

003c048

Signed-off-by: Malthe Poulsen <[email protected]>

simplify e2e tests

379eb09

Signed-off-by: Malthe Poulsen <[email protected]>

remove sleep

582c8fa

Signed-off-by: Malthe Poulsen <[email protected]>

malpou requested a review from JorTurFer June 3, 2025 16:53

refinement

413a6d6

Signed-off-by: Malthe Poulsen <[email protected]>

JorTurFer reviewed Jun 3, 2025

View reviewed changes

interceptor/handler/placeholder.go Outdated Show resolved Hide resolved

tests/checks/placeholder_pages/placeholder_pages_test.go Outdated Show resolved Hide resolved

interceptor/handler/placeholder_test.go Outdated Show resolved Hide resolved

malpou and others added 3 commits June 4, 2025 08:54

Update interceptor/handler/placeholder.go

73f739d

Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: Malthe Poulsen <[email protected]>

Update tests/checks/placeholder_pages/placeholder_pages_test.go

2f5c4df

Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: Malthe Poulsen <[email protected]>

linting issues

8f8ffab

Signed-off-by: malpou <[email protected]>

malpou requested a review from JorTurFer June 4, 2025 07:08

wozniakjan requested a review from Copilot June 10, 2025 12:03

Copilot AI reviewed Jun 10, 2025

View reviewed changes

interceptor/handler/placeholder.go Show resolved Hide resolved

JorTurFer reviewed Jun 17, 2025

View reviewed changes

interceptor/handler/placeholder.go Outdated Show resolved Hide resolved

interceptor/handler/placeholder.go Outdated Show resolved Hide resolved

Merge upstream/main and address PR review comments

fb1acfb

malpou requested a review from JorTurFer June 21, 2025 10:17

wozniakjan reviewed Jul 25, 2025

View reviewed changes

feat: Add custom placeholder pages for scale-from-zero scenarios #1303

Are you sure you want to change the base?

feat: Add custom placeholder pages for scale-from-zero scenarios #1303

Uh oh!

Conversation

malpou commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Key features:

Implementation details:

Checklist

Uh oh!

JorTurFer left a comment

Choose a reason for hiding this comment

Uh oh!

JorTurFer Jun 1, 2025

Choose a reason for hiding this comment

Uh oh!

malpou Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

JorTurFer Jun 2, 2025

Choose a reason for hiding this comment

Uh oh!

JorTurFer Jun 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

malpou Jun 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

JorTurFer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Nico385412 commented Jul 9, 2025

Uh oh!

JorTurFer commented Jul 14, 2025

Uh oh!

wozniakjan left a comment

Choose a reason for hiding this comment

Uh oh!

wozniakjan Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

wozniakjan Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wozniakjan Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wozniakjan Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wozniakjan Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

wozniakjan Jul 25, 2025

Choose a reason for hiding this comment

malpou commented May 30, 2025 •

edited

Loading

JorTurFer Jun 2, 2025 •

edited

Loading

wozniakjan Jul 25, 2025 •

edited

Loading

wozniakjan Jul 25, 2025 •

edited

Loading

wozniakjan Jul 25, 2025 •

edited

Loading