Skip to content

Commit b22fc52

Browse files
zhxiepre-commit-ci[bot]Xin Huang
authored
Add gateway guardrails (#363)
* proxy: initial commit Signed-off-by: Xie Zhihao <[email protected]> * proxy: support Docker build Signed-off-by: Xie Zhihao <[email protected]> * proxy: add Hyprescan matcher and regex engine Signed-off-by: Xie Zhihao <[email protected]> * proxy: remove unused Bazel target Signed-off-by: Xie Zhihao <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * proxy: rename proxy to istio Signed-off-by: Xie Zhihao <[email protected]> * proxy: add guardrails filter Signed-off-by: Xie Zhihao <[email protected]> * proxy: add .clang-format Signed-off-by: Xie Zhihao <[email protected]> * guardrails: meet ChatQnA API Signed-off-by: Xie Zhihao <[email protected]> * proxy: build image Signed-off-by: Xie Zhihao <[email protected]> * proxy: add README Signed-off-by: Xie Zhihao <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * guardrails: add deployment docs Signed-off-by: Xie Zhihao <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * proxy: update README Signed-off-by: Xie Zhihao <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * guardrails: add architecture Signed-off-by: Xie Zhihao <[email protected]> * proxy: update README Signed-off-by: Xie Zhihao <[email protected]> * proxy: remove Hyperscan temporarily Signed-off-by: Xie Zhihao <[email protected]> * guardrails: refactor Signed-off-by: Xie Zhihao <[email protected]> * guardrails: add tests Signed-off-by: Xie Zhihao <[email protected]> * proxy: add license header on source files Signed-off-by: Xie Zhihao <[email protected]> * proxy: add Makefile Signed-off-by: Xie Zhihao <[email protected]> * proxy: minor fix Signed-off-by: Xie Zhihao <[email protected]> * Update proxy/Makefile Co-authored-by: Xin Huang <[email protected]> Signed-off-by: Xie Zhihao <[email protected]> * Update proxy/Makefile Co-authored-by: Xin Huang <[email protected]> Signed-off-by: Xie Zhihao <[email protected]> * guardrails: minor deployment fix Signed-off-by: Xie Zhihao <[email protected]> --------- Signed-off-by: Xie Zhihao <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xin Huang <[email protected]>
1 parent 9107af9 commit b22fc52

File tree

33 files changed

+6848
-0
lines changed

33 files changed

+6848
-0
lines changed

‎.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,6 @@
22
*.out
33
**/Chart.lock
44
**/charts/*.tgz
5+
6+
bazel-*
7+
compile_commands.json

‎proxy/.bazelrc

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
# Keep envoy.bazelrc up-to-date by run:
5+
# curl -sSL https://raw.githubusercontent.com/istio/proxy/master/.bazelrc > envoy.bazelrc
6+
import %workspace%/envoy.bazelrc
7+
8+
# Build with Clang by default.
9+
build --config=clang
10+
11+
# Build with embedded V8-based WebAssembly runtime.
12+
build --define wasm=v8
13+
14+
# Build Proxy-WASM plugins as native extensions.
15+
build --copt -DNULL_PLUGIN
16+
17+
# Build with Docker.
18+
build:docker --action_env='PATH=/opt/llvm/bin:/opt/llvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin' --host_action_env='PATH=/opt/llvm/bin:/opt/llvm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin'
19+
build:docker --action_env='LLVM_CONFIG=/opt/llvm/bin/llvm-config' --host_action_env='LLVM_CONFIG=/opt/llvm/bin/llvm-config'
20+
build:docker --repo_env='LLVM_CONFIG=/opt/llvm/bin/llvm-config'
21+
build:docker --linkopt='-L/opt/llvm/lib'
22+
build:docker --linkopt='-Wl,-rpath,/opt/llvm/lib'

‎proxy/.bazelversion

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
6.5.0

‎proxy/.clang-format

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
Language: Cpp
3+
AccessModifierOffset: -2
4+
ColumnLimit: 100
5+
DerivePointerAlignment: false
6+
PointerAlignment: Left
7+
SortIncludes: false
8+
TypenameMacros: ['STACK_OF']
9+
...
10+
11+
---
12+
Language: Proto
13+
ColumnLimit: 100
14+
SpacesInContainerLiterals: false
15+
AllowShortFunctionsOnASingleLine: false
16+
ReflowComments: false
17+
...

‎proxy/BUILD

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
load(
5+
"@envoy//bazel:envoy_build_system.bzl",
6+
"envoy_cc_binary",
7+
)
8+
9+
exports_files(["LICENSE"])
10+
11+
ISTIO_EXTENSIONS = [
12+
"@istio//source/extensions/common/workload_discovery:api_lib", # Experimental: WIP
13+
"@istio//source/extensions/filters/http/alpn:config_lib",
14+
"@istio//source/extensions/filters/http/istio_stats",
15+
"@istio//source/extensions/filters/http/peer_metadata:filter_lib",
16+
"@istio//source/extensions/filters/network/metadata_exchange:config_lib",
17+
]
18+
19+
OPEA_EXTENSIONS = [
20+
"//source/extensions/filters/http/guardrails:config",
21+
]
22+
23+
envoy_cc_binary(
24+
name = "envoy",
25+
repository = "@envoy",
26+
deps = ISTIO_EXTENSIONS + OPEA_EXTENSIONS + [
27+
"@envoy//source/exe:envoy_main_entry_lib",
28+
],
29+
)

‎proxy/Makefile

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
BUILD_IMG ?= envoy-build-ubuntu
5+
BUILD_TAG ?= 75238004b0fcfd8a7f71d380d7a774dda5c39622
6+
ISTIO_IMG ?= proxyv2
7+
ISTIO_TAG ?= 1.23.0
8+
DOCKER_REGISTRY ?= docker.io/opea
9+
10+
BUILD_WITH_CONTAINER ?= 0
11+
BAZEL_ARG ?=
12+
13+
ifeq ($(BUILD_WITH_CONTAINER),0)
14+
15+
.PHONY: build
16+
build:
17+
bazel build -c opt $(BAZEL_ARG) envoy
18+
mkdir -p bin
19+
cp -f bazel-bin/envoy bin/envoy
20+
21+
.PHONY: test
22+
test:
23+
bazel test $(BAZEL_ARG) //test/...
24+
25+
else
26+
27+
.PHONY: build
28+
build:
29+
docker run -it \
30+
--rm \
31+
-v /tmp/proxy-docker-build:/root/.cache \
32+
-v $(shell pwd):/source \
33+
$(DOCKER_REGISTRY)/$(BUILD_IMG):$(BUILD_TAG) \
34+
"/bin/bash" "-c" "cd /source && export PATH=/opt/llvm/bin:$$PATH && BAZEL_ARG=\"--config=docker\" make"
35+
36+
.PHONY: test
37+
test:
38+
docker run -it \
39+
--rm \
40+
-v /tmp/proxy-docker-build:/root/.cache \
41+
-v $(shell pwd):/source \
42+
$(DOCKER_REGISTRY)/$(BUILD_IMG):$(BUILD_TAG) \
43+
"/bin/bash" "-c" "cd /source && export PATH=/opt/llvm/bin:$$PATH && BAZEL_ARG=\"--config=docker\" make test"
44+
45+
endif
46+
47+
.PHONY: image
48+
image:
49+
docker build -f tools/Dockerfile -t $(DOCKER_REGISTRY)/$(ISTIO_IMG):$(ISTIO_TAG)
50+
51+
.PHONY: build-image
52+
build-image:
53+
docker build -f tools/Dockerfile-build -t $(DOCKER_REGISTRY)/$(BUILD_IMG):$(BUILD_TAG) .
54+
55+
.PHONY: compilation-database
56+
compilation-database:
57+
$(shell bazel info output_base)/external/envoy/tools/gen_compilation_database.py --vscode //source/... //test/...

‎proxy/README.md

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
# OPEA Pipeline Proxy
2+
3+
OPEA Pipeline Proxy is an enhancement of the default Istio proxy with additional features designed specifically for OPEA RAG pipelines.
4+
5+
## Features
6+
7+
- [Guardrails](./deployments/guardrails/README.md)
8+
9+
## Build
10+
11+
OPEA Pipeline Proxy is based on Istio proxy and Envoy, you can refer to [Building Envoy with Bazel](https://github.com/envoyproxy/envoy/blob/main/bazel/README.md) for build locally. In addition, Clang and OpenVINO is required to build OPEA Pipeline Proxy.
12+
13+
```sh
14+
# Build OPEA Pipeline Proxy. The binary will be generated to `bin/envoy`.
15+
make
16+
17+
# Build the image. The image will be tagged as `opea/proxyv2:<ISTIO_TAG>` by default.
18+
make image
19+
```
20+
21+
You can also build OPEA Pipeline Proxy in the build container.
22+
23+
```sh
24+
# Build the build image.
25+
make build-image
26+
27+
# Build OPEA Pipeline Proxy with the build container. The binary will be generated to `bin/envoy`.
28+
BUILD_WITH_CONTAINER=1 make
29+
30+
# Build the image. The image will be tagged as `opea/proxyv2:<ISTIO_TAG>` by default.
31+
make image
32+
```
33+
34+
## Deployment
35+
36+
Before deploying OPEA Pipeline Proxy, you have to install Istio. Please follow the steps [here](https://istio.io/latest/docs/setup/install/istioctl/) for Istio installation.
37+
38+
During the installation, you have to assign the OPEA Pipeline Proxy to deploy instead of the default one delivered by Istio.
39+
40+
```sh
41+
# Use the default Istio pilot and the proxyv2 delivered by OPEA.
42+
istioctl install --set hub=docker.io/opea --set components.pilot.hub=docker.io/istio
43+
```
44+
45+
You can also use the annotation [here](https://istio.io/latest/docs/reference/config/annotations/#SidecarProxyImage) to inject OPEA Pipeline Proxy as sidecars.
46+
47+
## Development
48+
49+
You can generate the [JSON Compilation Database](https://clang.llvm.org/docs/JSONCompilationDatabase.html) for Visual Studio Code with [clangd](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd) extension and other compatible tools.
50+
51+
```sh
52+
make compilation-database
53+
```
54+
55+
You can test OPEA Pipeline Proxy with the following command.
56+
57+
```sh
58+
make test
59+
```

‎proxy/WORKSPACE

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
workspace(name = "dev_opea_proxy")
5+
6+
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
7+
8+
# Update Istio and Envoy by:
9+
# 1. Determine SHA256 `wget https://github.com/istio/proxy/archive/$COMMIT.tar.gz && sha256sum $COMMIT.tar.gz`.
10+
# 2. Update .bazelversion, envoy.bazelrc, BUILD, Makefile and tools/Dockerfile* if needed.
11+
#
12+
# Istio 1.23.0.
13+
ISTIO_SHA = "165f7597596f5b4de7025bb635a7b59d1c3fe27e"
14+
ISTIO_SHA256 = "ac44db76cd00442c0485f5ce99d69ae1044f33ea62c2dccb8e72c26bcf03b4e3"
15+
ISTIO_ORG = "istio"
16+
ISTIO_REPO = "proxy"
17+
18+
http_archive(
19+
name = "istio",
20+
sha256 = ISTIO_SHA256,
21+
strip_prefix = ISTIO_REPO + "-" + ISTIO_SHA,
22+
url = "https://github.com/" + ISTIO_ORG + "/" + ISTIO_REPO + "/archive/" + ISTIO_SHA + ".tar.gz",
23+
)
24+
25+
# TODO(zhxie): remove external repository fetch since it is the same repository as Istio.
26+
http_archive(
27+
name = "envoy_build_config",
28+
sha256 = ISTIO_SHA256,
29+
strip_prefix = ISTIO_REPO + "-" + ISTIO_SHA + "/bazel/extension_config",
30+
url = "https://github.com/" + ISTIO_ORG + "/" + ISTIO_REPO + "/archive/" + ISTIO_SHA + ".tar.gz",
31+
)
32+
33+
ENVOY_SHA = "97d97011f522006a816f8fe340c8a6f23d0dd712"
34+
ENVOY_SHA256 = "0ed516f812f477b23da723aa379fbe96fcd84a04112d8a5e27bfcfb34c986eea"
35+
ENVOY_ORG = "envoyproxy"
36+
ENVOY_REPO = "envoy"
37+
38+
http_archive(
39+
name = "envoy",
40+
sha256 = ENVOY_SHA256,
41+
strip_prefix = ENVOY_REPO + "-" + ENVOY_SHA,
42+
url = "https://github.com/" + ENVOY_ORG + "/" + ENVOY_REPO + "/archive/" + ENVOY_SHA + ".tar.gz",
43+
)
44+
45+
load("@envoy//bazel:api_binding.bzl", "envoy_api_binding")
46+
47+
envoy_api_binding()
48+
49+
load("@envoy//bazel:api_repositories.bzl", "envoy_api_dependencies")
50+
51+
envoy_api_dependencies()
52+
53+
load("@envoy//bazel:repositories.bzl", "envoy_dependencies")
54+
55+
envoy_dependencies()
56+
57+
load("@envoy//bazel:repositories_extra.bzl", "envoy_dependencies_extra")
58+
59+
envoy_dependencies_extra(ignore_root_user_error = True)
60+
61+
load("@envoy//bazel:python_dependencies.bzl", "envoy_python_dependencies")
62+
63+
envoy_python_dependencies()
64+
65+
load("@base_pip3//:requirements.bzl", "install_deps")
66+
67+
install_deps()
68+
69+
load("@envoy//bazel:dependency_imports.bzl", "envoy_dependency_imports")
70+
71+
envoy_dependency_imports()

‎proxy/bazel/BUILD

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Copyright (C) 2024 Intel Corporation
2+
# SPDX-License-Identifier: Apache-2.0

‎proxy/bazel/get_workspace_status

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
#! /bin/bash
2+
3+
# Copyright (C) 2024 Intel Corporation
4+
# SPDX-License-Identifier: Apache-2.0
5+
6+
echo "BUILD_CONFIG ${BUILD_CONFIG:-default}"

0 commit comments

Comments
 (0)