Skip to content

Support federated query #280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wangxiaoying merged 36 commits into from
May 10, 2022
Merged

Support federated query #280

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
03ec8ab
add end-to-end example code
wangxiaoying Mar 21, 2022
e01e64f
update
wangxiaoying Mar 22, 2022
1a37e07
update
wangxiaoying Mar 30, 2022
6a82a42
end-to-end fed
wangxiaoying Apr 2, 2022
a70cdee
fix hash join error on i32 and i64
wangxiaoying Apr 4, 2022
e20f04e
update
wangxiaoying Apr 6, 2022
57772ec
update
wangxiaoying Apr 8, 2022
f9ce786
arrow largeutf8 to utf8 in order to support comparison on string locally
wangxiaoying Apr 14, 2022
8a2556f
update
wangxiaoying Apr 14, 2022
ff8314f
update
wangxiaoying Apr 15, 2022
e8061ff
python - rust - java
wangxiaoying Apr 19, 2022
3db2441
update benchmark
wangxiaoying Apr 19, 2022
7915b7b
add rewriter submodule
wangxiaoying Apr 19, 2022
b3a4f35
update
wangxiaoying Apr 19, 2022
75953b5
update
wangxiaoying Apr 20, 2022
a1b3c99
update
wangxiaoying Apr 20, 2022
dc339d9
merge
wangxiaoying Apr 20, 2022
04c93d4
udpate queries
wangxiaoying Apr 20, 2022
2754c92
add q19
wangxiaoying Apr 20, 2022
458951b
update
wangxiaoying Apr 20, 2022
48fd7cd
update
wangxiaoying Apr 20, 2022
ca1ed19
parallel query executions
wangxiaoying Apr 20, 2022
0583b1c
update
wangxiaoying Apr 20, 2022
76572ee
support limit
wangxiaoying Apr 21, 2022
c842116
add correlated subquery to benchmark
wangxiaoying Apr 22, 2022
7b06c36
update
wangxiaoying Apr 22, 2022
fe13f27
update
wangxiaoying Apr 22, 2022
8732fb4
add q21
wangxiaoying Apr 22, 2022
ee6e6ff
update rust nightly to 2022-04-17
wangxiaoying Apr 24, 2022
cc3e8b5
upgrade datafusion, arrow and rust-nightly version to compile with la…
wangxiaoying Apr 24, 2022
206caf0
add jars to wheel
wangxiaoying May 6, 2022
aada1a8
maintain precompiled jar file to repo
wangxiaoying May 6, 2022
f53d88b
able to run in both dev and deployed env
wangxiaoying May 9, 2022
fa49b83
update
wangxiaoying May 9, 2022
7e003a6
merge main
wangxiaoying May 9, 2022
ca2d42b
fix ci
wangxiaoying May 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2021-11-18
toolchain: nightly-2022-04-17
components: rustfmt
default: true

Expand Down
11 changes: 9 additions & 2 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ jobs:
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2021-11-18
toolchain: nightly-2022-04-17
components: rustfmt
default: true

Expand Down Expand Up @@ -194,10 +194,15 @@ jobs:
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2021-11-18
toolchain: nightly-2022-04-17
components: rustfmt
default: true

- uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: '8'

- uses: actions/cache@v2
with:
path: |
Expand Down Expand Up @@ -251,4 +256,6 @@ jobs:
SQLITE_URL: "sqlite:///tmp/test.db"
MYSQL_URL: "mysql://root:mysql@mysql:3306/mysql"
MSSQL_URL: "mssql://sa:mssql!Password@mssql:1433/tempdb"
DB1: "postgresql://postgres:postgres@postgres:5432/postgres"
DB2: "postgresql://postgres:postgres@postgres:5432/postgres"
SQLITE3_STATIC: 1
2 changes: 1 addition & 1 deletion .github/workflows/docs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
- name: Install Rust
uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2021-11-18
toolchain: nightly-2022-04-17
components: rustfmt
default: true

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:

- uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2021-11-18
toolchain: nightly-2022-04-17
components: rustfmt
target: aarch64-unknown-linux-gnu
default: true
Expand Down Expand Up @@ -88,7 +88,7 @@ jobs:

- uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2021-11-18
toolchain: nightly-2022-04-17
components: rustfmt
default: true

Expand Down Expand Up @@ -135,7 +135,7 @@ jobs:

- uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2021-11-18
toolchain: nightly-2022-04-17
components: rustfmt
target: aarch64-apple-darwin
default: true
Expand Down
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,4 @@ connectorx-python/LICENSE
*.db
benchmark.json
docs/_build
connectorx/examples/test.rs
3 changes: 3 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "federated-query"]
path = federated-query
url = [email protected]:sfu-db/federated-query.git
Loading