Feature: OpenAI-compatible endpoint for text generation

[edlee123]

Description

This pull request introduces code and tests using OpeaTextGenService that can connect to any OpenAI style endpoint.
This allows devs a large choice of endpoints using openapi-like endpoints such as OpenRouter.ai. , and use OPEA with low resources

This PR updates OpeaTextGenService so it can connect with OpenAI-like endpoints from OpenRouter.ai (in addition to vLLM and TGI):

Key Changes:

• OpenAI-like endpoints such as OpenRouter.ai disallows some inputs from being null e.g. top_p (omitting it is ok). The input in OpeaTextGenService.align_input previously passed None/null defaults from ChatCompletionRequest, LLMParamsDoc, SearchDoc defaults. This PR adds a method _filter_api_params to take care of null + not omitted inputs.
• Added docker service , textgen-service-endpoint-openai, in the comps/llms/deployment/docker_compose/compose_text-generation.yaml file to make explicit use the use of OpeaTextGenService for openai-like endpoints.
• Added unit test to test OpeaTextGenService can connect with API key to OpenAI-like API.

Issues

n/a

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)
• Others (enhancement, documentation, validation, etc.)

Dependencies

n/a

Tests

tests/llms/test_llms_textgen_endpoint_openai.sh

This spins up a vllm endpoint, and then tests the connection with a token to the openAI style endpoint using OpeaTextGenService .

[edlee123]

Related to #1399 , so the opea microservice can have the api key.

[xiguiw]

@edlee123

I am afraid CI cannot verify the OPENAI API without api Key.

@chensuyue
How do we process such case?

[chensuyue]

@edlee123

I am afraid CI cannot verify the OPENAI API without api Key.

@chensuyue How do we process such case?

Yes, we don't have this key for CI test.

[chensuyue]

Know issues:
textgen-native-phi4-gaudi #1760
llms_text-generation_service_vllm_on_intel_hpu #1769

[edlee123]

Know issues: textgen-native-phi4-gaudi #1760 llms_text-generation_service_vllm_on_intel_hpu #1769

Thank you!

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Files with missing lines	Coverage Δ
comps/cores/proto/api_protocol.py	92.11% <100.00%> (+0.03%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[edlee123]

HI @ftian1 and @lkk12014402 thank you for the review. I think there would be three tests that fail because of these known issues:

textgen-native-phi4-gaudi #1760
llms_text-generation_service_vllm_on_intel_hpu #1769

Would it be ok to merge this PR (unrelated to above issues)?

[edlee123]

Thank you all!