Skip to content

tilt: improve Tilt development workflow (#12956) #13255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Wolf-06 wants to merge 5 commits into
base: main
Choose a base branch
from
Open

tilt: improve Tilt development workflow (#12956) #13255

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
d6f0847
tilt: improve Tilt development workflow (#12956)
Wolf-06 Jan 9, 2026
1a4ef53
chore: revert local test values in tilt-settings
Wolf-06 Jan 12, 2026
c958521
Add helm_sets support and Envoy binary for strict validation mode
Wolf-06 Jan 13, 2026
ead187c
feat: enhance Tilt setup with strict mode debugging and shell quoting…
Wolf-06 Jan 13, 2026
65bcaee
feat: refactor tilt helm flags and revert common recommendations config
Wolf-06 Jan 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 21 additions & 7 deletions Tiltfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ if str(local("command -v " + helm_cmd + " || true", quiet = True)) == "":
settings = {
"helm_installation_name": "kgateway",
"helm_installation_namespace": "kgateway-system",
"helm_values_files": ["./test/kubernetes/e2e/tests/manifests/common-recommendations.yaml"],
"helm_values_files": [],
Copy link
Contributor

@davidjumani davidjumani Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not in favour of this change

}

tilt_file = "./tilt-settings.yaml" if os.path.exists("./tilt-settings.yaml") else "./tilt-settings.json"
Expand Down Expand Up @@ -57,9 +57,18 @@ RUN chmod 777 ./$binary_name
standard_entrypoint = "ENTRYPOINT /app/start.sh /app/$binary_name"
debug_entrypoint = "ENTRYPOINT /app/start.sh /go/bin/dlv --listen=0.0.0.0:$debug_port --api-version=2 --headless=true --only-same-user=false --accept-multiclient --check-go-version=false exec --continue /app/$binary_name"

get_resources_cmd = "{0} -n {1} template {2} --include-crds install/helm/kgateway/ --set image.pullPolicy='Never' --set image.registry=ghcr.io/kgateway-dev --set image.tag='{3}'".format(helm_cmd, settings.get("helm_installation_namespace"), settings.get("helm_installation_name"), image_tag)
def _shell_escape_single_quotes(value):
"""Escape a value for safe inclusion in a single-quoted shell string.
In POSIX shells, a single quote inside a single-quoted string is represented as: '"'"'
"""
return str(value).replace("'", "'\"'\"'")

get_resources_cmd = "{0} -n {1} template {2} --include-crds install/helm/kgateway/ --set image.pullPolicy='Never' --set image.registry=ghcr.io/kgateway-dev --set image.tag='{3}' --set controller.extraEnv.KGW_DISABLE_LEADER_ELECTION='true'".format(helm_cmd, settings.get("helm_installation_namespace"), settings.get("helm_installation_name"), image_tag)
Copy link
Contributor

@davidjumani davidjumani Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets move this over helm_sets. This should be as clean as possible as it only gets the resources and not installs kgateway

Copy link
Contributor Author

@Wolf-06 Wolf-06 Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working on it.

for f in settings.get("helm_values_files") :
get_resources_cmd = get_resources_cmd + " --values=" + f
for key, value in settings.get("helm_sets", {}).items() :
escaped_value = _shell_escape_single_quotes(value)
get_resources_cmd = get_resources_cmd + " --set {0}='{1}'".format(key, escaped_value)

arch = str(local("make print-GOARCH", quiet = True)).strip()

Expand Down Expand Up @@ -104,10 +113,12 @@ def build_docker_image(provider):
tilt_helper_dockerfile,
tilt_dockerfile,
])
if provider.get("debug_port") :
dockerfile_contents = dockerfile_contents + debug_entrypoint
else :
dockerfile_contents = dockerfile_contents + standard_entrypoint

# Append the appropriate entrypoint based on whether debug_port is set
if provider.get("debug_port") :
dockerfile_contents = dockerfile_contents + debug_entrypoint
else :
dockerfile_contents = dockerfile_contents + standard_entrypoint

dockerfile_contents = dockerfile_contents.replace("$binary_name", provider.get("binary_name"))
dockerfile_contents = dockerfile_contents.replace("$debug_port", str(provider.get("debug_port")))
Expand Down Expand Up @@ -205,9 +216,12 @@ def install_kgateway():
install_helm_cmd = """
kubectl get crd gateways.gateway.networking.k8s.io &> /dev/null || {{ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml; }} ;
{0} upgrade --install -n {1} --create-namespace kgateway-crds install/helm/kgateway-crds ;
{0} upgrade --install -n {1} --create-namespace {2} install/helm/kgateway/ --set controller.image.pullPolicy='Never' --set image.registry=ghcr.io/kgateway-dev --set image.tag='{3}'""".format(helm_cmd, settings.get("helm_installation_namespace"), settings.get("helm_installation_name"), image_tag)
{0} upgrade --install -n {1} --create-namespace {2} install/helm/kgateway/ --set controller.image.pullPolicy='Never' --set image.registry=ghcr.io/kgateway-dev --set image.tag='{3}' --set controller.extraEnv.KGW_DISABLE_LEADER_ELECTION='true'""".format(helm_cmd, settings.get("helm_installation_namespace"), settings.get("helm_installation_name"), image_tag)
Copy link
Contributor

@davidjumani davidjumani Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as above, lets move this over to helm_sets

Copy link
Contributor Author

@Wolf-06 Wolf-06 Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working on it.

for f in settings.get("helm_values_files") :
install_helm_cmd = install_helm_cmd + " --values=" + f
for key, value in settings.get("helm_sets", {}).items() :
escaped_value = _shell_escape_single_quotes(value)
install_helm_cmd = install_helm_cmd + " --set {0}='{1}'".format(key, escaped_value)
local_resource(
name = settings.get("helm_installation_name") + "_helm",
cmd = ["bash", "-c", install_helm_cmd],
Expand Down
31 changes: 24 additions & 7 deletions tilt-settings.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@

helm_installation_name: kgateway
helm_values_files:
- ./test/e2e/tests/manifests/common-recommendations.yaml
Copy link
Contributor

@davidjumani davidjumani Jan 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm personally not in favour of this as the common-recommendations.yaml contain recommended values, especially in prod. They are used in tests and tilt is used for validating behaviour when writing tests, not just local dev
Instead, devs can just comment out this line when bringing up tilt

Copy link
Contributor

@puertomontt puertomontt Jan 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file is kind of a holdover from v1, doesn't really have much in it. Tilt needs some specific values, esp. KGW_DISABLE_LEADER_ELECTION.

Copy link
Contributor

@davidjumani davidjumani Jan 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the file again, I see now that it doesn't have a lot and is leftover from v1, but afaict most tests use this as the ProfileValuesManifestFile. The other specific values can be added to a tilt-profile.yaml or passed as helm flags in tilt-setting.yaml

helm_values_files: []
# - ./my-custom-values.yaml
helm_installation_namespace: kgateway-system

# Inline Helm --set flags (key: value format)
helm_sets: {}
Copy link
Contributor

@davidjumani davidjumani Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this be renamed to helm_flags

Copy link
Contributor Author

@Wolf-06 Wolf-06 Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

# Example: controller.extraEnv.KGW_VALIDATION_MODE: "strict"

enabled_providers:
- kgateway

Expand All @@ -23,13 +26,27 @@ providers:
# The service name
label: kgateway
# Command to build the binary
#build_binary: GCFLAGS='all="-N -l"' make -B kgateway
#build_binary: make -B kgateway GCFLAGS="all=-N -l" LDFLAGS=""
Copy link
Contributor

@davidjumani davidjumani Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this change be reverted? It only updates a comment

Copy link
Contributor Author

@Wolf-06 Wolf-06 Jan 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @davidjumani
It is not just a comment. It is an option of which type of binary to build.
make -B kgateway GCFLAGS="all=-N -l" LDFLAGS="" is used to build the kgateway binary specifically for debugging.
I will add a comment to make it more clear, does that work ?
Thank you

Copy link
Contributor

@davidjumani davidjumani Jan 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I mean is that the line is commented out, so any changes made don't have an effect. The command to build the binary is the subsequent line build_binary: make -B kgateway

Copy link
Contributor Author

@Wolf-06 Wolf-06 Jan 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make -B kgateway and make -B kgateway GCFLAGS="all=-N -l" LDFLAGS="" are basically two options for building the binary with different motives.

  • With make -B kgateway GCFLAGS="all=-N -l" LDFLAGS="" we build binary with motive of debugging
  • With make -B kgateway we build the normal binary with optimisation and Inlining.
Use case:
  • use the standard one for normal coding and testing.
  • switch to the debugger one ONLY if we are actively trying to attach a debugger

We toggle between the options by commenting the other and uncommenting the one we wan to use.
i will add comments for the instruction on using them to make it clear.

Copy link
Contributor

@davidjumani davidjumani Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has debugging been verified with this change ?

Copy link
Contributor Author

@Wolf-06 Wolf-06 Jan 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes
I have verified the debugging with the help of breakpoints in reconcile func (in pkg/kgateway/controller and it hit the breakpoints.
I tried the same with normal build (keeping the debug_port as it is), it didn't hit the breakpoint.

build_binary: make -B kgateway
# Name of the binary file when built
binary_name: kgateway-linux-$ARCH
# A custom docker file. This might be required when the base image is different or managing an external project such as envoy
#dockerfile_contents: ...
# To enable debugging, just add the `debug_port` param to a provider (provided it supports debugging)
# Custom Dockerfile with Envoy binary for strict validation mode.
# NOTE: Do NOT include ENTRYPOINT here - the Tiltfile will append the correct
# entrypoint (standard or debug) based on whether debug_port is set.
dockerfile_contents: |
FROM quay.io/solo-io/envoy-gloo:1.36.3-patch1 as envoy-bin

FROM golang:latest as tilt
WORKDIR /app
COPY --from=tilt-helper /go/bin/dlv /go/bin/dlv
COPY --from=tilt-helper /process.txt .
COPY --from=tilt-helper /start.sh .
COPY --from=tilt-helper /restart.sh .
# Copy envoy binary for strict validation
COPY --from=envoy-bin /usr/local/bin/envoy /usr/local/bin/envoy
COPY $binary_name .
RUN chmod 777 ./$binary_name
# To enable debugging with strict validation, uncomment both debug_port and port 50100
#debug_port: 50100
links:
- http://localhost:9092/metrics
Expand Down
Loading