Skip to content

Refactor Retrievers #1138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 55 commits into from
Jan 16, 2025
Merged

Refactor Retrievers #1138

merged 55 commits into from
Jan 16, 2025

Conversation

letonghan
Copy link
Collaborator

@letonghan letonghan commented Jan 14, 2025

Description

Refactor retrievers into E-RAG style.
The folder structure after refactored is as below:

  • retrievers
    • deployment
      • docker_compose
      • kubernetes
    • src
      • integrations
        • redis.py
        • milvus.py
        • ...
      • Dockerfile
      • opea_retrievers_microservice.py
      • README.md
      • requirements.txt
      • set_env.sh

Issues

#1004

Type of change

List the type of change like below. Please delete options that are not relevant.

  • Breaking change (fix or feature that would break existing design and interface)

Dependencies

In requirements.txt

Tests

Local tested

Signed-off-by: letonghan <[email protected]>
@letonghan
Copy link
Collaborator Author

Related PR to fix GenAIExamples issues: opea-project/GenAIExamples#1387

letonghan and others added 16 commits January 15, 2025 15:05
Signed-off-by: letonghan <[email protected]>
@rbrugaro
Copy link
Collaborator

@letonghan neo4j yaml is still missing in docker_compose folder

@rbrugaro
Copy link
Collaborator

@letonghan the retriever_neo4j test passed but seems the telemetry traces endpoint 4318 didn't connect: https://github.com/opea-project/GenAIComps/actions/runs/12788252024/job/35649181305?pr=1138#step:4:4037
I also see some neo4j warnings that will be resolved with my upcoming PR that is on hold until this refactor is complete (is related to a package version)

@joshuayao joshuayao linked an issue Jan 16, 2025 that may be closed by this pull request
@letonghan
Copy link
Collaborator Author

@letonghan neo4j yaml is still missing in docker_compose folder

Thanks @rbrugaro, the missing docker compose yaml files are added now.

@chensuyue
Copy link
Collaborator

comps/retrievers/vdms/langchain/Dockerfile will be implement in another PR.

@chensuyue chensuyue merged commit c49db2b into main Jan 16, 2025
24 of 25 checks passed
@chensuyue chensuyue deleted the refactor_retrievers branch January 16, 2025 06:12
chensuyue pushed a commit to opea-project/GenAIExamples that referenced this pull request Jan 16, 2025
Delete redundant retrievers docker image in docker_images_list.md.
Refactor Retrievers related Examples READMEs.
Change all of the comps/retrievers/xxx/xxx/Dockerfile path into comps/retrievers/src/Dockerfile.

Fix the Examples CI issues of PR opea-project/GenAIComps#1138.
Signed-off-by: letonghan <[email protected]>
smguggen pushed a commit to opea-aws-proserve/GenAIComps that referenced this pull request Jan 23, 2025
Refactor retrievers into E-RAG style.
The folder structure after refactored is as below:

- retrievers
    - deployment
        - docker_compose
        - kubernetes
    - src
        - integrations
            - redis.py
            - milvus.py
            - ...
        - Dockerfile
        - opea_retrievers_microservice.py
        - README.md
        - requirements.txt
        - set_env.sh

opea-project#1004

Signed-off-by: letonghan <[email protected]>
chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request Mar 4, 2025
Delete redundant retrievers docker image in docker_images_list.md.
Refactor Retrievers related Examples READMEs.
Change all of the comps/retrievers/xxx/xxx/Dockerfile path into comps/retrievers/src/Dockerfile.

Fix the Examples CI issues of PR opea-project/GenAIComps#1138.
Signed-off-by: letonghan <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>
madison-evans pushed a commit to SAPD-Intel/GenAIComps that referenced this pull request May 12, 2025
Refactor retrievers into E-RAG style.
The folder structure after refactored is as below:

- retrievers
    - deployment
        - docker_compose
        - kubernetes
    - src
        - integrations
            - redis.py
            - milvus.py
            - ...
        - Dockerfile
        - opea_retrievers_microservice.py
        - README.md
        - requirements.txt
        - set_env.sh

opea-project#1004

Signed-off-by: letonghan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] ERAG refactor - retrievers
4 participants