Skip to content

Releases: alibaba/higress

v2.1.9

13 Nov 11:52
ec5031c

Choose a tag to compare

Higress

πŸ“‹ Overview of This Release

This release includes 44 updates, covering various aspects such as feature enhancements, bug fixes, and performance optimizations.

Update Distribution

  • New Features: 23
  • Bug Fixes: 14
  • Refactoring Optimizations: 2
  • Documentation Updates: 1
  • Testing Improvements: 4

⭐ Key Highlights

This release contains 3 significant updates, which are recommended for special attention:

  • feat(mcp-server): add server-level default authentication and MCP proxy server support (#3096): This feature enhances Higress's security management capabilities for MCP traffic, allowing users to set up authentication through a unified interface, simplifying the deployment process of security policies, and enhancing system security and flexibility.
  • feat: add higress api mcp server (#2923): By adding the higress-ops MCP Server, users can use the hgctl agent command to manage Higress configurations and troubleshoot issues, improving operational efficiency and user experience.
  • feat: implement hgctl agent & mcp add subcommand (#3051): This enhancement improves Higress's operational capabilities, especially through interactive management and debugging via the Agent, making it easier for users to configure and debug MCP traffic governance. It is a significant step towards AI-native operations for Higress.

For more details, please refer to the detailed descriptions of key features below.


🌟 Detailed Description of Key Features

Below are the detailed explanations of the important features and improvements in this release:

1. feat(mcp-server): add server-level default authentication and MCP proxy server support

Related PR: #3096 | Contributor: @johnlanni

Usage Background

As the AI-native API gateway Higress develops, users' demands for API security, flexibility, and ease of use are increasing. In practical applications, the MCP (Model Context Protocol) is widely used for managing and invoking AI models. However, existing MCP servers lack a unified security authentication mechanism, leading to the need for configuring different authentication mechanisms in various scenarios (such as direct proxying by MCP Server, or REST API conversion to MCP Server). This update addresses these issues, targeting user groups including, but not limited to, developers, operations personnel, and system administrators, who require a more secure, flexible, and easy-to-manage API gateway.

Feature Details

This update primarily implements two core features: 1. Adding default authentication at the MCP server level, including client-to-gateway and gateway-to-backend authentication; 2. Introducing a new type of MCP proxy server that can proxy MCP requests from clients to backend MCP servers, supporting timeout configuration and full authentication support. Technically, this is achieved by updating dependency library versions (such as wasm-go and proxy-wasm-go-sdk) to support the new features, while also refactoring existing code to accommodate the new authentication and proxy logic.

Usage

To enable this feature, you need to set the corresponding parameters in the Higress configuration file. For example, to configure default downstream security, specify the authentication policy in the defaultDownstreamSecurity field; similarly, upstream authentication is configured through the defaultUpstreamSecurity field. To use the MCP proxy server, define a new mcp-proxy type server and specify the backend MCP server address via the mcpServerURL field. Additionally, you can control the request timeout time using the timeout field. Best practices recommend utilizing the priority configuration mechanism to ensure that tool-level settings can override server-level defaults, thereby achieving finer-grained control.

Feature Value

This feature significantly enhances the security and flexibility of Higress, making API management more efficient. By introducing server-level default authentication, it reduces the workload of repetitive configurations and lowers the security risks caused by configuration errors. The newly added MCP proxy server capability not only simplifies the complexity of MCP service governance but also effectively alleviates the pressure on backend MCP servers by offloading state-keeping tasks to the Higress side. These improvements collectively contribute to enhancing the stability and user experience of the entire ecosystem, laying a solid foundation for Higress to become an indispensable API gateway in the AI era.


2. feat: add higress api mcp server

Related PR: #2923 | Contributor: @Tsukilc

Usage Background

As AI technology advances, API gateways need to better support AI-related functionalities. Higress, as an AI-native API gateway, needs to provide more powerful management tools to unify the management of core API assets such as LLM APIs, MCP APIs, and Agent APIs. This PR integrates the Higress API MCP Server, providing comprehensive management capabilities for AI routing, AI providers, and MCP servers. These new features help users more efficiently configure and maintain Higress's AI features, meeting the needs of modern applications. The target user groups include Higress operators and developers, especially those with deep needs in the AI domain.

Feature Details

This PR mainly implements the following features:

  1. AI Routing Management: Added tools such as list-ai-routes, get-ai-route, add-ai-route, update-ai-route, and delete-ai-route to allow users to manage AI routes.
  2. AI Provider Management: Added tools such as list-ai-providers, get-ai-provider, add-ai-provider, update-ai-provider, and delete-ai-provider to allow users to manage AI providers.
  3. MCP Server Management: Added tools such as list-mcp-servers, get-mcp-server, add-or-update-mcp-server, and delete-mcp-server to allow users to manage MCP servers and their consumers.
  4. Authentication Configuration: Uses HTTP Basic Authentication for authorization, carrying the Authorization header in the client request.
  5. Code Changes: Removed hard-coded usernames and passwords, instead providing them at runtime via the MCP Client, enhancing security. Additionally, added the higress-ops module for hgctl agent command integration, enabling Agent-based management of Higress configurations.

Usage

To enable and configure this feature, follow these steps:

  1. Configure Higress API MCP Server: Add the Higress API MCP Server configuration in the Higress configuration file, specifying the URL of the Higress Console.
  2. Use hgctl agent: Start the interactive Agent using the hgctl agent command, allowing you to manage Higress using natural language. For example, use the mcp add subcommand to add a remote MCP Server to the Higress MCP management directory.
  3. Manage AI Routes: Use tools like list-ai-routes, get-ai-route, add-ai-route, update-ai-route, and delete-ai-route to manage AI routes.
  4. Manage AI Providers: Use tools like list-ai-providers, get-ai-provider, add-ai-provider, update-ai-provider, and delete-ai-provider to manage AI providers.
  5. Manage MCP Servers: Use tools like list-mcp-servers, get-mcp-server, add-or-update-mcp-server, and delete-mcp-server to manage MCP servers and their consumers.
    Note: Ensure that you correctly configure the authentication information and carry the Authorization header in the request.

Feature Value

This feature brings the following specific benefits to users:

  1. Enhanced Management Capabilities: Users can more easily manage and debug Higress's AI routing, AI provider, and MCP server configurations using the new MCP tools, improving management efficiency.
  2. Higher Security: By providing usernames and passwords at runtime via the MCP Client rather than hard-coding them in the configuration file, the system's security is enhanced.
  3. Better User Experience: The interactive management method via hgctl agent allows users to manage Higress using natural language, reducing the learning curve and difficulty of use.
  4. Improved System Performance and Stability: The new MCP tools provide more management and debugging options, helping to promptly identify and resolve issues, thereby improving system stability and performance.
  5. Ecosystem Importance: As the first step for Higress to transition from traditional operations to Agent-based operations, this feature is significant for the development of the Higress ecosystem, laying the groundwork for future innovations.

3. feat: implement hgctl agent & mcp add subcommand

Related PR: #3051 | Contributor: @erasernoob

Usage Background

Higress is an AI-native API gateway used to unify the management of LLM APIs, MCP APIs, and Agent APIs. As Higress evolves, traditional command-line tools no longer meet user needs, especially in the management and debugging of MCP services. This PR introduces an interactive Agent similar to Claude Code, allowing users to manage Higress using natural language. Additionally, the new mcp add subcommand makes it easy to add remote MCP services to Higress's MCP management directory, enabling MCP traffic governance. These features not only simplify the configuration process for MCP services but also enhance the system's maintainability and usability.

Feature Details

Thi...

Read more

v2.1.9-rc.1

09 Oct 09:33
13261bd

Choose a tag to compare

Higress

πŸ“‹ Overview of This Release

This release includes 11 updates, covering areas such as feature enhancements, bug fixes, performance optimizations, and more.

Distribution of Updates

  • New Features: 3
  • Bug Fixes: 5
  • Refactoring and Optimization: 1
  • Documentation Updates: 2

πŸ“ Full Changelog

πŸš€ New Features (Features)

  • Related PR: #2978
    Contributor: @rinfx
    Change Log: In the key-auth plugin, regardless of whether authentication is successful, the consumer name will be recorded after it is determined. This is achieved by adding the X-Mse-Consumer field to the HTTP request header.
    Feature Value: This feature allows the system to obtain and record the consumer's name earlier, which is very important for logging and subsequent processing, improving the traceability and transparency of the system.

  • Related PR: #2968
    Contributor: @2456868764
    Change Log: This PR introduces the core functionality of vector database mapping, including a field mapping system and index configuration management, supporting various index types.
    Feature Value: By providing flexible field mapping and index configuration capabilities, users can more easily integrate with different database architectures, enhancing the system's compatibility and flexibility.

  • Related PR: #2943
    Contributor: @Guo-Chenxu
    Change Log: Added a feature for customizing system prompts, allowing users to add personalized notes when generating release notes. This is implemented by modifying the GitHub Actions workflow file.
    Feature Value: This feature allows users to include customized system prompts when generating release notes, enhancing the flexibility and richness of information in the release notes, thereby improving the user experience.

πŸ› Bug Fixes (Bug Fixes)

  • Related PR: #2973
    Contributor: @CH3CHO
    Change Log: This PR fixes an issue in Higress version 2.1.8 where the mcp-session filter did not support setting match_rule_domain to an empty string, using wildcards to match all domains and eliminate compatibility risks.
    Feature Value: This resolves a compatibility issue caused by specific configurations, ensuring that users do not encounter errors due to empty string settings during upgrades or configuration, thus improving the stability and user experience of the system.

  • Related PR: #2952
    Contributor: @Erica177
    Change Log: Corrected the JSON tag for the Id field in the ToolSecurity struct from type to id, ensuring correct mapping during data serialization.
    Feature Value: This fix addresses data inconsistency issues caused by incorrect field mapping, enhancing the stability and data accuracy of the system.

  • Related PR: #2948
    Contributor: @johnlanni
    Change Log: Fixed the Azure service URL type detection logic to support custom full paths. Enhanced the handling of Azure OpenAI response APIs and improved edge case parsing in streaming events.
    Feature Value: This ensures better compatibility with Azure OpenAI services, improves error handling and user experience, especially when using non-standard paths or streaming responses.

  • Related PR: #2942
    Contributor: @2456868764
    Change Log: Fixed the issue of LLM provider being empty and optimized documentation and prompt messages. Specifically, updated README.md for better explanations and adjusted the default LLM model.
    Feature Value: By enhancing the robustness of LLM provider initialization and optimizing related documentation, this improves the stability and user experience of the system, making it clearer for users to understand system configuration and usage.

  • Related PR: #2941
    Contributor: @rinfx
    Change Log: This PR fixes compatibility issues with old configurations, ensuring the system can correctly handle outdated configuration parameters, avoiding potential errors due to configuration changes.
    Feature Value: By supporting older version configurations, this enhances the system's backward compatibility, reducing inconvenience to users during upgrades or configuration adjustments, and improving the user experience.

♻️ Refactoring and Optimization (Refactoring)

  • Related PR: #2945
    Contributor: @rinfx
    Change Log: Optimized the logic for selecting pods based on the minimum number of requests globally, updated the Lua script code related to ai-load-balancer, reducing unnecessary checks and improving performance.
    Feature Value: By improving the minimum request count algorithm in load balancing strategies, this enhances the system's response speed and resource allocation efficiency, allowing users to utilize cluster resources more efficiently.

πŸ“š Documentation Updates (Documentation)

  • Related PR: #2965
    Contributor: @CH3CHO
    Change Log: Updated the description of the azureServiceUrl field in the ai-proxy plugin README file to provide clearer and more accurate information.
    Feature Value: By improving the description in the documentation, users can better understand how to configure the Azure OpenAI service URL, thus enhancing the user experience and configuration accuracy.

  • Related PR: #2940
    Contributor: @github-actions[bot]
    Change Log: This PR adds English and Chinese release notes for version 2.1.8, detailing 30 updates in this version.
    Feature Value: By providing detailed release notes, users can more easily understand the new features, bug fixes, and other information included in the new version, allowing them to make better use of the new features.


πŸ“Š Release Statistics

  • πŸš€ New Features: 3
  • πŸ› Bug Fixes: 5
  • ♻️ Refactoring and Optimization: 1
  • πŸ“š Documentation Updates: 2

Total: 11 changes

Thank you to all contributors for their hard work! πŸŽ‰

Higress Console

πŸ“‹ Overview of This Release

This release includes 4 updates, covering multiple aspects such as feature enhancements, bug fixes, and performance optimizations.

Update Distribution

  • New Features: 1
  • Bug Fixes: 2
  • Documentation Updates: 1

⭐ Key Highlights

This release contains 1 significant update, which is recommended for special attention:

  • feat: Support using a known service in OpenAI LLM provider (#589): This feature allows users to use predefined services within the OpenAI LLM, thereby enhancing development efficiency and flexibility, and meeting the needs of a wider range of application scenarios.

For more details, please refer to the Important Feature Details section below.


🌟 Important Feature Details

Below are detailed explanations of key features and improvements in this release:

1. feat: Support using a known service in OpenAI LLM provider

Related PR: #589 | Contributor: @CH3CHO

Usage Background

As more organizations and services adopt large language models (LLMs), access and management of these models have become increasingly important. Especially when integration with specific known services, such as an on-premises OpenAI API server or a custom API endpoint, is required. This feature addresses the need for direct support of custom OpenAI services within the Higress system, allowing users to more flexibly configure and use their services. The target user groups include, but are not limited to, developers, operations personnel, and enterprises requiring highly customized solutions.

Feature Details

The update primarily focuses on the OpenaiLlmProviderHandler class, introducing support for custom service sources. By adding new configuration options like openaiCustomServiceName and openaiCustomServicePort, users can now directly specify the details of their custom OpenAI service. Additionally, the code has been improved so that if a custom upstream service is specified, a service source will not be created for the default service. This design not only simplifies the configuration process but also enhances the system's scalability. Technically, this is achieved by overriding the buildServiceSource and buildUpstreamService methods, which include checks for user-defined settings.

Usage Instructions

To enable and configure this new feature, users first need to provide the necessary custom service information in their OpenAI LLM provider settings. This typically involves filling in fields such as the custom service name, host address, and port number. The general steps are: 1. Locate the relevant LLM provider settings section in the Higress console or corresponding configuration file; 2. Enter the appropriate custom service details as prompted; 3. Save the changes. A typical use case might be a company wishing to use its own internally hosted OpenAI interface instead of the publicly available one. It is important to ensure that the provided custom service address is accurate and network-accessible.

Feature Value

This feature greatly enhances the adaptability of the Higress platform to different environments, especially for scenarios requiring high levels of customization. It not only improves the user experienceβ€”making the configuration process more intuitive and simpleβ€”but als...

Read more

v2.1.8

21 Sep 07:34

Choose a tag to compare

Higress

πŸ“‹ Overview of This Release

This release includes 30 updates, covering various aspects such as feature enhancements, bug fixes, performance optimizations, and more.

Update Distribution

  • New Features: 13
  • Bug Fixes: 7
  • Refactoring and Optimization: 5
  • Documentation Updates: 4
  • Testing Improvements: 1

⭐ Key Highlights

This release includes 2 major updates, which are highly recommended for your attention:

  • feat: add rag mcp server (#2930): By introducing the RAG MCP server, this update provides a new way for users to manage and retrieve knowledge, enhancing the functionality and practicality of the system.
  • refactor(mcp): use ECDS for golang filter configuration to avoid connection drain (#2931): Using ECDS for filter configuration avoids instability caused by directly embedding golang filter configurations, improving the system's stability and maintainability, and reducing unnecessary service interruptions for users.

For more details, please refer to the important features section below.


🌟 Detailed Description of Important Features

Below is a detailed description of the key features and improvements in this release:

1. feat: add rag mcp server

Related PR: #2930 | Contributor: @2456868764

Use Case

In modern applications, knowledge management and retrieval have become increasingly important. Many systems require fast and accurate extraction and retrieval of information from large volumes of text data. RAG (Retrieval-Augmented Generation) technology combines retrieval and generation models to effectively enhance the efficiency and accuracy of knowledge management. This PR introduces a Model Context Protocol (MCP) server specifically for knowledge management and retrieval, meeting the needs of users for efficient information processing. The target user group includes enterprises and developers who need to handle large amounts of text data, especially in the fields of natural language processing (NLP) and machine learning.

Feature Details

This PR implements the RAG MCP server, adding multiple functional modules, including knowledge management, chunk management, search, and chat functions. The core features include:

  1. Knowledge Management: Supports creating knowledge blocks from text.
  2. Chunk Management: Provides functionalities for listing and deleting knowledge chunks.
  3. Search: Supports keyword-based search.
  4. Chat Function: Allows users to send chat messages and receive responses.
    Technically, the server uses several external libraries, such as github.com/dlclark/regexp2, github.com/milvus-io/milvus-sdk-go/v2, and github.com/pkoukk/tiktoken-go, which provide regular expression handling, vector database management, and text encoding functionalities. Key code changes include adding an HTTP client, configuration files, and multiple processing functions to ensure the flexibility and configurability of the system.

Usage Instructions

To enable and configure the RAG MCP server, follow these steps:

  1. Enable the MCP server in the higress-config configuration file and set the corresponding path and configuration items.
  2. Configure the basic parameters of the RAG system, such as splitter type, chunk size, and overlap.
  3. Configure the LLM (Large Language Model) provider and its API key, model name, etc.
  4. Configure the embedding model provider and its API key, model name, etc.
  5. Configure the vector database provider and its connection information.
    Example configuration:
rag:
  splitter:
    type: "recursive"
    chunk_size: 500
    chunk_overlap: 50
  top_k: 5
  threshold: 0.5
llm:
  provider: "openai"
  api_key: "your-llm-api-key"
  model: "gpt-3.5-turbo"
embedding:
  provider: "openai"
  api_key: "your-embedding-api-key"
  model: "text-embedding-ada-002"
vectordb:
  provider: "milvus"
  host: "localhost"
  port: 19530
  collection: "test_collection"

Notes:

  • Ensure all configuration items are correct, especially API keys and model names.
  • In production environments, it is recommended to adjust parameters such as timeout appropriately to adapt to different network conditions.

Feature Value

The RAG MCP server provides a complete solution for knowledge management and retrieval, enhancing the intelligence and automation of the system. Specific benefits include:

  1. Improved Efficiency: Through integrated knowledge management and retrieval functions, users can quickly process and retrieve large volumes of text data, saving time and resources.
  2. Enhanced Accuracy: Combining RAG technology, the system can more accurately extract and retrieve information, reducing error rates.
  3. Flexible Configuration: Provides rich configuration options, allowing users to flexibly adjust according to actual needs, meeting the requirements of different scenarios.
  4. High Scalability: Supports multiple providers and models, making it easy for users to choose suitable components and technology stacks based on business needs.
  5. Stability Improvement: Through detailed configuration validation and error handling mechanisms, the stability and robustness of the system are ensured.

2. refactor(mcp): use ECDS for golang filter configuration to avoid connection drain

Related PR: #2931 | Contributor: @johnlanni

Use Case

In the current implementation, Golang filter configurations are directly embedded in the HTTP_FILTER patch, which can lead to connection drain when configurations change. The main reason is the inconsistent sorting of Go maps in the map[string]any field, and the listener configuration changes triggered by HTTP_FILTER updates. This issue affects the stability and user experience of the system. The target user group is developers and operations personnel using Higress for service mesh management.

Feature Details

This PR splits the configuration into two parts: HTTP_FILTER only contains filter references with config_discovery, while EXTENSION_CONFIG contains the actual Golang filter configuration. This way, configuration changes do not directly cause connection drain. The specific implementation includes updating the constructMcpSessionStruct and constructMcpServerStruct methods to return formats compatible with EXTENSION_CONFIG and updating unit tests to match the new configuration structure. The core innovation lies in using the ECDS mechanism to separate configurations, making configuration changes smoother.

Usage Instructions

Enabling and configuring this feature does not require any additional operations as it is automatically handled in the background. A typical use case is when configuring Golang filters in Higress; the system will automatically split them into HTTP_FILTER and EXTENSION_CONFIG. Users only need to configure Golang filters as usual. Note that when upgrading to the new version, ensure all related configuration files are updated and thoroughly tested in the production environment to ensure that configuration changes do not introduce other issues.

Feature Value

By separating configurations and using ECDS, this feature eliminates the connection drain problem during configuration changes, significantly improving the system's stability and user experience. Additionally, this design makes configurations easier to manage and maintain, reducing potential issues caused by configuration changes. For large-scale service mesh deployments, this improvement is particularly important as it reduces service interruptions caused by configuration changes, thereby enhancing the overall reliability and availability of the system.


πŸ“ Full Changelog

πŸš€ New Features (Features)

  • Related PR: #2926
    Contributor: @rinfx
    Change Log: This PR adds support for multimodal, function calls, and thinking in vertex-ai, involving the introduction of a regular expression library and improvements to the processing logic.
    Feature Value: By adding new features, vertex-ai can better support application needs in complex scenarios, such as multimodal data processing and more flexible function call methods, enhancing the system's flexibility and practicality.

  • Related PR: #2917
    Contributor: @Aias00
    Change Log: This PR adds support for Fireworks AI, expanding the functionality of the AI agent plugin, including the addition of necessary configuration files and test code.
    Feature Value: Adding support for Fireworks AI allows users to leverage the AI features provided by the platform, broadening the range of AI services that applications can integrate with, and enhancing the user experience.

  • Related PR: #2907
    Contributor: @Aias00
    Change Log: This PR upgrades wasm-go to support outputSchema, involving dependency updates for jsonrpc-converter and oidc plugins.
    Feature Value: By supporting outputSchema, the functionality and flexibility of the wasm-go plugin are enhanced, making it easier for users to handle and define output data structures.

  • Related PR: #2897
    Contributor: @rinfx
    Change Log: This PR adds multimodal support and thinking functionality to the ai-proxy bedrock, achieved by extending the relevant code in bedrock.go.
    Feature Value: The added multimodal and thinking support enriches the ai-proxy's feature set, enab...

Read more

v2.1.7

01 Sep 07:59
020b5f3

Choose a tag to compare

Higress

πŸ“‹ Overview of This Release

This release includes 42 updates, covering various aspects such as feature enhancements, bug fixes, and performance optimizations.

Update Distribution

  • New Features: 21 items
  • Bug Fixes: 14 items
  • Refactoring and Optimization: 4 items
  • Documentation Updates: 2 items
  • Testing Improvements: 1 item

⭐ Key Highlights

This release includes 3 significant updates, which are recommended for your attention:

  • feat: add MCP SSE stateful session load balancer support (#2818): This feature enables applications based on SSE to better maintain persistent connections between clients and servers, enhancing user experience and application performance, especially in scenarios requiring long-term connections for data pushing.
  • feat: Support adding a proxy server in between when forwarding requests to upstream (#2710): This feature allows users to use a proxy server when forwarding requests to upstream services, enhancing the system's flexibility and security, suitable for scenarios where communication through specific proxies is required.
  • feat(ai-proxy): add auto protocol compatibility for OpenAI and Claude APIs (#2810): By automatically detecting and converting protocols, users can seamlessly use APIs from different AI service providers without manual configuration, improving user experience and flexibility.

For more details, please refer to the Important Features section below.


🌟 Detailed Description of Important Features

Here are the detailed descriptions of important features and improvements in this release:

1. feat: add MCP SSE stateful session load balancer support

Related PR: #2818 | Contributor: @johnlanni

Usage Background

As the demand for real-time communication grows, Server-Sent Events (SSE) have become a key technology for many applications. However, in distributed systems, ensuring that requests from the same client are always routed to the same backend service to maintain session state has been a challenge. Traditional load balancing strategies cannot meet this need. This feature addresses this issue by introducing MCP SSE stateful session load balancing support. By specifying the mcp-sse type in the higress.io/load-balance annotation, users can easily manage SSE connection state sessions. The target user group mainly consists of application developers and service providers who need to perform real-time data pushing in distributed environments.

Feature Details

This PR mainly implements the following features:

  1. Extend load-balance annotation: In the loadbalance.go file, support for the mcp-sse value is added, and the McpSseStateful field is added to the LoadBalanceConfig struct.
  2. Simplified Configuration: Users only need to set mcp-sse in the higress.io/load-balance annotation to enable this feature, with no additional configuration required.
  3. Backend Address Encoding: When MCP SSE stateful session load balancing is enabled, the backend address will be Base64 encoded and embedded in the session ID of the SSE message. This ensures that the client can correctly identify and maintain the session. The core innovation lies in dynamically generating SSE session-related configurations through EnvoyFilter, thereby achieving stateful session management.

Usage Instructions

To use this feature, users need to follow these steps:

  1. Enable the Feature: Add the higress.io/load-balance: mcp-sse annotation to the Ingress resource.
  2. Configuration Example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sse-ingress
  annotations:
    higress.io/load-balance: mcp-sse
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /mcp-servers/test/sse
        pathType: Prefix
        backend:
          service:
            name: sse-service
            port:
              number: 80
  1. Testing: Access the SSE endpoint using the curl command and check if the returned messages contain the correct session ID.
    Notes:
  • Ensure that the backend service can handle Base64 encoded session IDs.
  • Avoid frequent changes to the backend service deployment to prevent session consistency issues.

Feature Value

This feature brings the following specific benefits to users:

  1. Session Consistency: Ensures that requests from the same client are always routed to the same backend service, maintaining session state consistency.
  2. Simplified Configuration: Enables the feature with simple annotation configuration, reducing the complexity of user configuration.
  3. Enhanced User Experience: For applications that rely on SSE, such as real-time notifications and stock market data, it provides a more stable and consistent service experience.
  4. Reduced Operations Costs: Reduces errors and failures caused by inconsistent sessions, lowering the workload of the operations team.

2. feat: Support adding a proxy server in between when forwarding requests to upstream

Related PR: #2710 | Contributor: @CH3CHO

Usage Background

In modern microservice architectures, especially in complex network environments, directly forwarding requests from the client to the backend service may encounter various issues, such as network security and performance bottlenecks. Introducing an intermediate proxy server can effectively solve these problems, for example, by performing traffic control, load balancing, and SSL offloading through the proxy server. Additionally, in some cases, enterprises may need to use specific proxy servers to meet compliance and security requirements. The target user group for this feature mainly consists of enterprises and developers who need to optimize request forwarding paths in complex network environments.

Feature Details

This PR mainly implements the ability to configure one or more proxy servers in the McpBridge resource and allows specifying proxy servers for each registry. The specific implementation includes:

  1. Adding the proxies field in the McpBridge resource definition to configure the list of proxy servers, and adding the proxyName field in the registries item to associate the proxy server with the registry.
  2. When creating or updating the McpBridge resource, the system automatically generates the corresponding EnvoyFilter resources, which define how to forward requests to the specified proxy server.
  3. Additionally, EnvoyFilters are generated for each service bound to a proxy, ensuring they correctly point to the local listener on the corresponding proxy server. The entire technical implementation is based on Envoy's advanced routing capabilities, demonstrating the project's powerful functionality in handling complex network topologies.

Usage Instructions

To enable this feature, at least one proxy server must first be configured in the McpBridge resource. This can be done by adding new ProxyConfig objects to the spec.proxies array, each containing necessary information such as name, serverAddress, and serverPort. Next, for the registry entries that need to use a proxy server, simply reference the defined proxy name in their proxyName field. Once configured, the system will automatically handle all related EnvoyFilter generation work. It is worth noting that before actual deployment, the correctness of the configuration files should be carefully checked to avoid service unavailability due to misconfiguration.

Feature Value

The newly added proxy server support feature greatly enhances the system's network flexibility, allowing users to flexibly adjust request forwarding paths according to their needs. For example, by setting up different proxy servers, it is easy to achieve data transmission optimization across multiple regions; at the same time, with the additional security features provided by the proxy layer (such as SSL encryption), the overall system security is significantly improved. In addition, this feature also helps simplify operations management, especially in situations where frequent adjustments to the network architecture are needed. Through simple configuration changes, rapid responses to changes can be achieved without major modifications to the underlying infrastructure. In summary, this improvement not only expands the project's scope but also provides users with more powerful tools to tackle increasingly complex network challenges.


3. feat(ai-proxy): add auto protocol compatibility for OpenAI and Claude APIs

Related PR: #2810 | Contributor: @johnlanni

Usage Background

In the AI proxy plugin, users may need to interact with multiple AI service providers (such as OpenAI and Anthropic Claude) simultaneously. These providers typically use different API protocols, leading to the need for manual configuration of protocol types when switching services, which increases complexity and the likelihood of errors. This feature solves this problem, allowing users to seamlessly use different providers' services without worrying about the differences in underlying protocols. The target user group consists of developers and enterprises who want to simplify the AI service integration process.

Feature Details

This PR implements the automatic protocol compatibility feature. The core technological innovation lies in automatically detecting the request path and intelligently converting the protocol based on the target provider's capabilities. Specifically, when the request path is /v1/chat/completions, i...

Read more

v2.1.6

30 Jul 13:53
d98f8b8

Choose a tag to compare

Higress

πŸ“‹ Overview of This Release

This release includes 31 updates, covering enhancements, bug fixes, performance optimizations, and more.

Update Distribution

  • New Features: 13 items
  • Bug Fixes: 5 items
  • Refactoring and Optimization: 7 items
  • Documentation Updates: 5 items
  • Testing Improvements: 1 item

⭐ Key Highlights

This release contains 2 major updates, which are highly recommended to focus on:

  • feat: Add Higress API MCP server (#2517): The newly added Higress API MCP server functionality enhances AI Agent's management capabilities over Higress resources, supporting the creation, deletion, modification, and querying of routes and services through MCP, thereby improving the system's flexibility and maintainability.
  • Migrate WASM Go Plugins to New SDK and Go 1.24 (#2532): The underlying compilation dependency for developing Wasm Go plugins has been switched from TinyGo to native Go 1.24, improving plugin compatibility and performance, ensuring alignment with the latest technology stack, and providing users with more stable and efficient plugin support.

For more details, please refer to the detailed description of key features below.


🌟 Detailed Description of Key Features

Below are the detailed descriptions of the important features and improvements in this release:

1. feat: Add Higress API MCP server

Related PR: #2517 | Contributor: @cr7258

Usage Background

In modern microservice architectures, the API gateway, as the entry point, requires flexible and powerful configuration management capabilities. Higress, as a high-performance API gateway, provides rich features for managing routes, service origins, and plugins. However, the existing configuration management methods may not be flexible enough to meet complex operational needs. To address this issue, PR #2517 introduces the Higress API MCP Server, providing a new way to manage configurations through the Higress Console API. This feature is primarily aimed at operations personnel and developers who need advanced and dynamic management of Higress.

Feature Details

This change implements the Higress API MCP Server, re-implementing an MCP server using golang-filter that can call the Higress Console API to manage routes, service origins, and plugins. The specific implementation includes:

  1. Added the HigressClient class to handle interactions with the Higress Console API.
  2. Implemented various management tools such as route management (list-routes, get-route, add-route, update-route), service origin management (list-service-sources, get-service-source, add-service-source, update-service-source), and plugin management (get-plugin, delete-plugin, update-request-block-plugin).
  3. Modified relevant configuration files and README documentation, providing detailed configuration examples and usage instructions.
  4. Code changes involve multiple files, including config.go, client.go, server.go, etc., ensuring the completeness and extensibility of the feature.

Usage Instructions

To enable and configure the Higress API MCP Server, follow these steps:

  1. Add the MCP Server configuration in the Higress ConfigMap, specifying the URL, username, and password of the Higress Console.
  2. When starting the Higress Gateway, ensure that mcpServer.enable is set to true.
  3. Use the provided tool commands (e.g., list-routes, add-route) to manage routes, service origins, and plugins.
  4. Configuration example:
apiVersion: v1
kind: ConfigMap
metadata:
  name: higress-config
  namespace: higress-system
data:
  higress: |-
    mcpServer:
      sse_path_suffix: /sse
      enable: true
      servers:
        - name: higress-api-mcp-server
          path: /higress-api
          type: higress-api
          config:
            higressURL: http://higress-console.higress-system.svc.cluster.local
            username: admin
            password: <password>

Notes:

  • Ensure that the Higress Console URL, username, and password are correct.
  • It is recommended to use environment variables or encrypted storage for the password to enhance security.

Feature Value

The Higress API MCP Server brings the following specific benefits to users:

  1. Improved Operational Efficiency: Through a unified MCP interface, users can more conveniently manage and configure Higress resources via AI Agent, reducing the complexity and error rate of manual operations.
  2. Enhanced System Flexibility: Support for dynamic management and updating of routes, service origins, and plugins makes the system more flexible and able to quickly respond to changes in business requirements.
  3. Increased System Stability: Automated configuration management reduces the possibility of human errors, thereby enhancing the stability and reliability of the system.
  4. Easy Integration: The design of the Higress API MCP Server makes it easy to integrate with other AI agents and tools, facilitating the construction of a complete automated operations system.

2. Migrate WASM Go Plugins to New SDK and Go 1.24

Related PR: #2532 | Contributor: @erasernoob

Usage Background

With the development of the Go language, new versions provide many performance optimizations and security improvements. This PR aims to migrate WASM Go plugins from the old SDK to the new SDK and upgrade the Go version to 1.24. This not only resolves some known issues in the old version but also paves the way for future feature expansion and performance optimization. The target user group includes developers and operations personnel using Higress for microservice management and traffic control.

Feature Details

This PR mainly implements the following features: 1) Updated the workflow files for building and testing plugins to support the new Go version; 2) Modified the Dockerfile and Makefile, removing support for TinyGo and switching to the standard Go compiler for generating WASM files; 3) Updated the go.mod file, referencing new package paths and versions; 4) Adjusted the import path of the logging library, unifying the use of the new logging library. These changes allow the plugins to better utilize the new features of Go 1.24, such as improved garbage collection and more efficient compiler optimizations. Additionally, removing support for TinyGo simplifies the build process and reduces potential compatibility issues.

Usage Instructions

To enable and configure this feature, first ensure that your development environment has Go 1.24 installed. Then, you can specify the new build parameters by modifying the project's Makefile and Dockerfile. For example, set GO_VERSION ?= 1.24.4 in the Makefile and use ARG BUILDER=higress-registry.cn-hangzhou.cr.aliyuncs.com/plugins/wasm-go-builder:go1.24.4-oras1.0.0 in the Dockerfile. A typical use case is when you need to deploy new WASM plugins in Higress. Best practices include regularly updating dependencies to the latest versions and ensuring that all related code is adapted to the new version.

Feature Value

This refactoring brings multiple benefits to users: 1) Improved plugin runtime efficiency and stability, thanks to the new features and optimizations in Go 1.24; 2) Simplified build process, reducing dependency on third-party tools (such as TinyGo) and lowering maintenance costs; 3) Unified code style and dependency management, improving the readability and maintainability of the project; 4) Enhanced system security by adopting the latest Go version to fix known security vulnerabilities. These improvements make the Higress ecosystem more robust, providing a more powerful and reliable microservice management platform for users.


πŸ“ Full Changelog

πŸš€ New Features (Features)

  • Related PR: #2679
    Contributor: @erasernoob
    Change Log: This PR adds support for external service FQDN in image annotations and includes corresponding test cases to ensure the correctness and stability of the new feature.
    Feature Value: Allows users to specify external FQDN as the image target, enhancing the system's flexibility and applicability, and facilitating the integration of more external resources.

  • Related PR: #2667
    Contributor: @hanxiantao
    Change Log: This PR adds support for setting a global route rate limit threshold for the AI Token rate-limiting plugin, while optimizing the underlying logic related to the cluster-key-rate-limit plugin and improving log messages.
    Feature Value: By adding support for global rate limit thresholds, users can more flexibly manage traffic, avoiding the impact of a single route's excessive traffic on the entire system's stability.

  • Related PR: #2652
    Contributor: @OxalisCu
    Change Log: This PR adds support for the first-byte timeout for LLM streaming requests in the ai-proxy plugin by modifying the provider.go file.
    Feature Value: This feature allows users to set a first-byte timeout for LLM streaming requests, improving system stability and user experience.

  • Related PR: #2650
    Contributor: @zhangjingcn
    Change Log: This PR implements the functionality to fetch ErrorResponseTemplate configuration from the Nacos MCP registry by modifying the mcp_model.go and watcher.go files to support new metadata handling.
    Feature Value: This feature enhances t...

Read more

v2.1.5

02 Jul 10:04
36bcb59

Choose a tag to compare

Higress

πŸ“‹ Overview of This Release

This release includes 41 updates, covering various aspects such as feature enhancements, bug fixes, and performance optimizations.

Update Content Distribution

  • New Features: 19
  • Bug Fixes: 14
  • Refactoring Optimizations: 2
  • Documentation Updates: 6

⭐ Key Focus

This release includes 2 significant updates, which are recommended for your attention:

  • feat: add DB MCP Server execute, list tables, describe table tools (#2506): By adding these tools, users can more conveniently manage and operate databases, enhancing the system's flexibility and usability, making database operations more intuitive and efficient.
  • feat: advanced load balance policies for LLM service through wasm plugin (#2531): By introducing advanced load balancing strategies, the performance and resource utilization of LLM services have been improved, allowing users to choose the most suitable strategy to optimize their services based on their needs.

For more details, please refer to the key features section below.


🌟 Detailed Description of Key Features

Here is a detailed description of the important features and improvements in this release:

1. feat: add DB MCP Server execute, list tables, describe table tools

Related PR: #2506 | Contributor: hongzhouzi

Usage Background

In many application development scenarios, developers need to frequently interact with databases, such as executing SQL statements and viewing table structures. While the existing MCP server supports basic database query functions, it lacks more advanced operation tools. This update adds three tools: execute (execute SQL), list tables (list tables), and describe table (describe table), aiming to meet higher user demands for database management. The target user groups include, but are not limited to, database administrators, backend developers, and application developers who need to frequently interact with databases.

Feature Details

Specifically, by modifying the db.go file, new database type constants were introduced, and the new tools were registered in the server.go file. The newly added tools implement the functionality of executing arbitrary SQL statements, listing all table names, and obtaining detailed information about specific tables. The core technical points lie in using the GORM framework to handle different types of database connections and providing customized SQL query logic for each type of database. Additionally, the code changes also involved optimizing the error handling mechanism, such as unifying the error handling function handleSQLError, improving the maintainability of the code. These improvements not only enriched the MCP server's feature set but also enhanced its applicability in various database environments.

Usage Instructions

Enabling these new features is straightforward; just ensure that your MCP server configuration includes the correct database DSN and type. For the execute tool, users can send requests containing the sql parameter to perform INSERT, UPDATE, or DELETE operations; the list tables tool requires no additional parameters and can be called directly to return all table names in the current database; the describe table tool requires a table parameter to specify the table name to view. Typical use cases include, but are not limited to, periodically checking the consistency of database table structures, generating automated scripts, and verifying data before and after migration. It is important to note that when using the execute tool, caution should be exercised to avoid executing commands that may compromise data integrity.

Feature Value

This feature significantly expands the application scope of the MCP server in database management, enabling users to complete daily tasks more efficiently. It not only simplifies complex manual operations and reduces the likelihood of errors but also provides a solid foundation for building automated O&M processes. Especially for projects that need to work across multiple database platforms, this unified and flexible interface design is undoubtedly a boon. Additionally, by improving error handling logic and adding security measures (such as preventing SQL injection), this PR further ensures the stability and security of the system.


2. feat: advanced load balance policies for LLM service through wasm plugin

Related PR: #2531 | Contributor: rinfx

Usage Background

With the widespread application of large language models (LLMs), the demand for high performance and high availability is growing. Traditional load balancing strategies may not meet these requirements, especially when handling a large number of concurrent requests. The new load balancing strategies aim to address these issues by providing smarter request distribution. The target user group includes enterprises and developers who require high-performance and high-availability LLM services.

Feature Details

This PR implements three new load balancing strategies: 1. Minimum Load Strategy, implemented using WASM, suitable for gateway-api-inference-extension; 2. Global Least Request Strategy based on Redis, which tracks and manages the number of requests for each host via Redis, ensuring that requests are allocated to the host with the least current load; 3. Prompt Prefix Matching Strategy, which selects backend nodes based on prompt prefixes, and if no match is found, uses the Global Least Request Strategy. These strategies are implemented using WASM plugins, providing high scalability and flexibility.

Usage Instructions

To enable these load balancing strategies, you need to specify the corresponding strategy type and configuration parameters in the Higress gateway configuration. For example, to enable the Global Least Request Strategy based on Redis, set lb_policy to global_least_request in the configuration file and provide the FQDN, port, username, and password of the Redis service. For the Prompt Prefix Matching Strategy, set lb_policy to prefix_cache and make the corresponding configuration. Best practice is to choose the appropriate strategy based on the actual application scenario and regularly monitor and adjust the configuration to optimize performance.

Feature Value

These new load balancing strategies bring significant performance improvements to LLM services. The Minimum Load Strategy ensures that requests are allocated to the host with the least current load, thereby improving response speed and resource utilization. The Global Least Request Strategy based on Redis further optimizes resource allocation by tracking the number of requests for each host in real time. The Prompt Prefix Matching Strategy improves processing efficiency by caching and reusing KV Cache. These features not only enhance system performance and stability but also improve user experience, especially in high-concurrency scenarios.


πŸ“ Complete Changelog

πŸš€ New Features (Features)

  • Related PR: #2533
    Contributor: @johnlanni
    Change Log: Added support for the subPath field, allowing users to configure rules for removing request path prefixes, and updated the Chinese and English documentation to include usage instructions for the new feature.
    Feature Value: By introducing the subPath configuration option, the flexibility and customizability of the AI proxy plugin have been enhanced, enabling developers to more finely control the request path processing logic and improve the user experience.

  • Related PR: #2514
    Contributor: @daixijun
    Change Log: This PR commented out the default tracing.skywalking configuration in values.yaml, resolving the issue where skywalking configurations were automatically added when users chose other tracing types.
    Feature Value: By removing unnecessary skywalking configurations, conflicts with user-defined tracing settings are avoided, enhancing the system's flexibility and user experience.

  • Related PR: #2509
    Contributor: @daixijun
    Change Log: This PR implemented handling of the OpenAI responses interface Body and added support for the Volcano Ark large model responses interface, achieved by extending the logic in the provider/doubao.go file.
    Feature Value: The new feature enables the system to support more types of AI response processing, particularly for users using the Volcano Ark large model, significantly enhancing the system's compatibility and flexibility.

  • Related PR: #2488
    Contributor: @rinfx
    Change Log: Added trace_span_key and as_separate_log_field configuration options, allowing the keys for logging and span attribute recording to be different and enabling log content to exist as separate fields.
    Feature Value: By providing more flexible logging and tracing data recording methods, the system's monitoring capabilities have been enhanced, helping developers better understand and optimize application performance.

  • Related PR: #2485
    Contributor: @johnlanni
    Change Log: This PR introduced the errorResponseTemplate feature, allowing the mcp server plugin to customize response content when the backend HTTP status code is greater than 300.
    ...

Read more

v2.1.5-rc.1

18 Jun 01:17

Choose a tag to compare

What's Changed

New Contributors

Full Changelog: v2.1.4...v2.1.5-rc.1

v2.1.4

10 Jun 14:24
9f5067d

Choose a tag to compare

Detailed release notes

https://github.com/alibaba/higress/tree/main/release-notes/2.1.4

What's Changed

  • feat(ai-proxy): support Amazon Bedrock Image Generation by @daixijun in #2212
  • fix: Fix the incorrect rewrite config generated for Nacos 3 MCP Servers by @CH3CHO in #2211
  • feat: update translate-readme action by @littlejiancc in #2208
  • fix: Remove the Authorization request header when using AI-proxy to proxy Gemini by @hanxiantao in #2220
  • fix : fix issue #2222 by @Erica177 in #2231
  • Add test translation workflow by @MAVRICK-1 in #2228
  • mcp server support API auth through OAS3 security schemes by @johnlanni in #2241
  • Fix : add fail strategy for wasmplugin generated by mcp server by @Erica177 in #2237
  • fix: modify log level WARN -> DEBUG in key-auth plugin by @lexburner in #2275
  • fix proxy-wasm-cpp-sdk by @johnlanni in #2281
  • feat: Supports recording request header, request body, response header and response body information in the access log by @forgottener in #2265
  • feat(mcp-server): add HackMD mcp server by @Whitea029 in #2260
  • add mcp service shebao tools by @hourmoneys in #2303
  • feat: Add a github action to copy CRD definitions from api folder to helm folder by @CH3CHO in #2268
  • feat: allow skipping higress dev image build during wasmplugin e2e tests by @cr7258 in #2264
  • feat: Refactor mcpServer.matchList config generation logic by @CH3CHO in #2207
  • add info log of ai-search plugin by @johnlanni in #2323
  • fix(ai-proxy): URL encode model name in Bedrock requests by @HecarimV in #2321
  • feat(ai-proxy): add doubao Image Generation support by @daixijun in #2331
  • feat: cluster-key-rate-limit support setting global rate limit thresholds for routes​ by @hanxiantao in #2262
  • feat(ai-proxy): support OpenAI-compatible image and audio model Mapping by @daixijun in #2341
  • fix: set "EnableSemanticCachefalse" to false when no vector configured in ai-cache by @mirror58229 in #2351
  • feat(ai-proxy): add batches & files support by @daixijun in #2355
  • feat: support dify ai-proxy e2e test || feat: support diify ai-proxy e2e test by @VinciWu557 in #2319
  • fix content-length header not remove in ai-search plugin by @johnlanni in #2363
  • feat(ai-proxy): add modelMapping regexp support by @daixijun in #2358
  • feat(ai-proxy): Fixed the issue that the API pass-through path error does not support openaiCustomUrl after openai is configured. by @daixijun in #2364
  • feat(frontend-gray): Add uniqueGrayTag configuration detection by @heimanba in #2371
  • feat(ai-proxy): add models & image generation support for gemini by @daixijun in #2380
  • feat(ai-proxy): support Google Cloud Vertex by @HecarimV in #2119
  • add upstream override wasm abi by @johnlanni in #2387
  • feat(ai-proxy): Add Claude image understanding and Tools calling capabilities by @daixijun in #2385
  • fix: refactored mcp server auto discovery logic and fix some issue by @Erica177 in #2382
  • fix : fix credential process logic for nacos mcp util and add ut for it by @Erica177 in #2394
  • fix: Support mixing line breaks in a single SSE response by @CH3CHO in #2344
  • Update CRD file in the helm folder by @github-actions in #2392

New Contributors

Full Changelog: v2.1.3...v2.1.4

v2.1.3

09 May 07:31

Choose a tag to compare

What's Changed

  • Update helm translated README.zh.md by @github-actions in #2152
  • fix: update golang filter README by @Jing-ze in #2147
  • fix : Add nacos username and password login option by @Erica177 in #2170
  • feat: Support /v1/models API in ai-proxy by @CH3CHO in #2164
  • fix(ai-statistics): adjust requestBodyBufferLimit by @HecarimV in #2192
  • fix : fix vs rewrite when mcp protocol is http by @Erica177 in #2203
  • feat: Add SSE direct proxy support to mcp-session filter by @CH3CHO in #2157
  • fix param type error by @Erica177 in #2204

Full Changelog: v2.1.2...v2.1.3

v2.1.2

29 Apr 12:55

Choose a tag to compare

What's Changed

  • feat:Getting MatchLabels dynamically via gatewaySelectorKey/Value #1857 by @waTErMo0n in #1883
  • fix: update module replacements by @Similarityoung in #2090
  • feat: optimize elasticsearch ai-search plugin and update related docs" by @cr7258 in #2100
  • feat: Support extracting model argument from body in multipart/form-data format by @CH3CHO in #1940
  • update mcp descriptions by @mirror58229 in #2105
  • feat(ai-proxy): support Amazon Bedrock by @HecarimV in #2039
  • update github & e2bdev mcp descriptions by @mirror58229 in #2107
  • test: add test for /pkg/ingress/kube/common by @Tsukilc in #2123
  • add mcp yuque descriptions by @mirror58229 in #2125
  • fix: ai_data_masking add compatibility handling for non-compliant API response structures by @007gzs in #2130
  • fix: Refactor MCP Server into MCP Session and MCP Server by @Jing-ze in #2120
  • feat : support mcp server auto discovery for nacos registry by @Erica177 in #2122
  • Update helm translated README.zh.md by @github-actions in #2141
  • fix : when nacos push empty service instance list, should skip generate by @Erica177 in #2144
  • fix: make mcp server redis client config based by @Jing-ze in #2145
  • When the service source type is nacos3, if mcpserver is turned off, then the discovery mechanism of nacos2 will be enabled by @johnlanni in #2150
  • add: add mcp-context7 descriptions by @mirror58229 in #2149

New Contributors

Full Changelog: v2.1.1...v2.1.2