@@ -11,11 +11,56 @@ About
11
11
12
12
Example programs demonstrating CrateDB's SQLAlchemy adapter and dialect.
13
13
14
+ This section and examples are mostly about `DataFrame operations with SQLAlchemy `_,
15
+ specifically about how to insert data into `CrateDB `_ efficiently using `pandas `_ and
16
+ `Dask `_.
17
+
18
+
14
19
15
20
*****
16
21
Usage
17
22
*****
18
23
24
+ The CrateDB Python driver provides a convenience function ``insert_bulk ``. It
25
+ can be used like this::
26
+
27
+ # CrateDB Cloud
28
+ # DBURI = "crate://admin:<PASSWORD>@example.aks1.westeurope.azure.cratedb.net:4200?ssl=true"
29
+
30
+ # CrateDB Self-Managed
31
+ # DBURI = "crate://crate@localhost:4200/"
32
+
33
+ import sqlalchemy as sa
34
+ from crate.client.sqlalchemy.support import insert_bulk
35
+
36
+ # pandas
37
+ engine = sa.create_engine(DBURI, **kwargs)
38
+ df.to_sql(
39
+ name="testdrive",
40
+ con=engine,
41
+ if_exists="append",
42
+ index=False,
43
+ chunksize=5_000,
44
+ method=insert_bulk,
45
+ )
46
+
47
+ # Dask
48
+ ddf.to_sql(
49
+ "testdrive",
50
+ uri=DBURI,
51
+ index=False,
52
+ if_exists="replace",
53
+ chunksize=10_000,
54
+ parallel=True,
55
+ method=insert_bulk,
56
+ )
57
+
58
+
59
+
60
+ *****
61
+ Setup
62
+ *****
63
+
19
64
To start a CrateDB instance on your machine for evaluation purposes, invoke::
20
65
21
66
docker run -it --rm --publish=4200:4200 --publish=5432:5432 crate
@@ -29,6 +74,11 @@ Navigate to example program directory, and install prerequisites::
29
74
source .venv/bin/activate
30
75
pip install --upgrade --requirement requirements.txt
31
76
77
+
78
+ ********
79
+ Examples
80
+ ********
81
+
32
82
Run example programs::
33
83
34
84
# Connect to CrateDB on localhost.
@@ -42,9 +92,27 @@ Run example programs::
42
92
43
93
time python insert_dask.py
44
94
45
- # Connect to CrateDB Cloud.
46
- time python insert_pandas.py --dburi='crate://admin:<PASSWORD>@example.aks1.westeurope.azure.cratedb.net:4200?ssl=true'
95
+ Use ``insert_pandas.py `` to connect to any other database instance::
96
+
97
+ export DBURI="crate://crate@localhost:4200/"
98
+ export DBURI="crate://admin:<PASSWORD>@example.aks1.westeurope.azure.cratedb.net:4200?ssl=true"
99
+ time python insert_pandas.py --dburi="${DBURI}"
47
100
48
101
.. TIP ::
49
102
50
- For more information, please refer to the header section of each of the provided example programs.
103
+ For more information, please refer to the header sections of each of the provided example programs.
104
+
105
+
106
+ *****
107
+ Tests
108
+ *****
109
+
110
+ To test the accompanied example programs all at once, invoke the software tests::
111
+
112
+ pytest
113
+
114
+
115
+ .. _CrateDB : https://github.com/crate/crate
116
+ .. _Dask : https://www.dask.org/
117
+ .. _DataFrame operations with SQLAlchemy : https://cratedb.com/docs/python/en/latest/by-example/sqlalchemy/dataframe.html
118
+ .. _pandas : https://pandas.pydata.org/
0 commit comments