Refactor dataprep microservice #1153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

chensuyue merged 90 commits into main from lvl/dataprep_refactor

Jan 20, 2025

Collaborator

lvliang-intel commented Jan 15, 2025

Description

Refactor dataprep microservice, including opensearch, elasticsearch, pinecone, pgvector, neo4j, qdrant, vdms.

Issues

n/a.

Type of change

List the type of change like below. Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds new functionality)
Breaking change (fix or feature that would break existing design and interface)
Others (enhancement, documentation, validation, etc.)

Dependencies

None, just code refactor

Tests

Local test and CI test.

lvliang-intel added 24 commits

January 13, 2025 17:49


          refactor dataprep code

eafa1b2

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

ec817d7

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Dataprep code refactor

7c545a5

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

ff6cdfc

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

b0558dd

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

ce3504c

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

29cc5c6

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

218da1b

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

1c8d870

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

3c0c6c3

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

b1635c0

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

2768c00

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

cab0a5d

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

7f66f78

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

f439f1c

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

6d931bc

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

8a4559c

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

4d92e05

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

4d6f1f9

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

632cbaa

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

c95c7b2

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

18dc889

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

0c82e24

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

31672f9

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>

lvliang-intel requested review from XinyuYe-Intel, letonghan, chensuyue, ZePan110 and ftian1 as code owners

January 15, 2025 15:11


          Merge branch 'main' into lvl/dataprep_refactor

c7fc298

lvliang-intel and others added 14 commits

January 19, 2025 02:49


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

ce1c62b

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

6d848f2

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

744541d

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

40b3aac

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

07e49f2

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

35fc7be

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

b256387

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          [pre-commit.ci] auto fixes from pre-commit.com hooks

9f9b2b0

for more information, see https://pre-commit.ci


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

0ad3e95

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

9ec463f

…ct/GenAIComps into lvl/dataprep_refactor


          [pre-commit.ci] auto fixes from pre-commit.com hooks

52cc446

for more information, see https://pre-commit.ci


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

9d91d73

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

31a0dfb

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          [pre-commit.ci] auto fixes from pre-commit.com hooks

efb0f22

for more information, see https://pre-commit.ci

XinyuYe-Intel approved these changes

View reviewed changes

lvliang-intel added 6 commits

January 19, 2025 21:44


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

6f874eb

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

dc709da

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

a4813ff

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

6ca467c

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'main' of https://github.com/opea-project/GenAIComps int…

4d91a12

…o lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>


          Merge branch 'lvl/dataprep_refactor' of https://github.com/opea-proje…

9d12f72

…ct/GenAIComps into lvl/dataprep_refactor

Signed-off-by: lvliang-intel <[email protected]>

chensuyue mentioned this pull request

For test only #1178

Closed

chensuyue merged commit c223a9d into main

26 of 27 checks passed

chensuyue deleted the lvl/dataprep_refactor branch

January 20, 2025 12:30

chensuyue mentioned this pull request

Update GraphRAG to be compatible with latest component changes opea-project/GenAIExamples#1427

Merged

4 tasks

chensuyue pushed a commit to opea-project/GenAIExamples that referenced this pull request


          Adapt code for dataprep microservice refactor (#1408)

0f7e5a3

opea-project/GenAIComps#1153

Signed-off-by: lvliang-intel <[email protected]>

smguggen pushed a commit to opea-aws-proserve/GenAIComps that referenced this pull request


          Refactor dataprep microservice (opea-project#1153)

8f4d887

Refactor dataprep microservice, including opensearch, elasticsearch, pinecone, pgvector, neo4j, qdrant, vdms.

Signed-off-by: lvliang-intel <[email protected]>
Co-authored-by: chen, suyue <[email protected]>

chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request


          Adapt code for dataprep microservice refactor (opea-project#1408)

fd5c9fe

opea-project/GenAIComps#1153

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>

chyundunovDatamonsters pushed a commit to chyundunovDatamonsters/OPEA-GenAIExamples that referenced this pull request


          Adapt code for dataprep microservice refactor (opea-project#1408)

fcfc586

opea-project/GenAIComps#1153

Signed-off-by: lvliang-intel <[email protected]>
Signed-off-by: Chingis Yundunov <[email protected]>

madison-evans pushed a commit to SAPD-Intel/GenAIComps that referenced this pull request


          Refactor dataprep microservice (opea-project#1153)

c708cc3

Refactor dataprep microservice, including opensearch, elasticsearch, pinecone, pgvector, neo4j, qdrant, vdms.

Signed-off-by: lvliang-intel <[email protected]>
Co-authored-by: chen, suyue <[email protected]>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

chensuyue chensuyue left review comments

yao531441 yao531441 approved these changes

lkk12014402 lkk12014402 approved these changes

XinyuYe-Intel XinyuYe-Intel approved these changes

lianhao lianhao left review comments

letonghan Awaiting requested review from letonghan letonghan is a code owner

ZePan110 Awaiting requested review from ZePan110 ZePan110 is a code owner

ftian1 Awaiting requested review from ftian1 ftian1 is a code owner

XinyaoWa Awaiting requested review from XinyaoWa

Labels

None yet