Skip to content

Commit fa26a9d

Browse files
committed
Update IMPORT_EXISTING_DOCUMENTS.md
1 parent 028f926 commit fa26a9d

File tree

1 file changed

+16
-0
lines changed

1 file changed

+16
-0
lines changed

firestore-bigquery-export/guides/IMPORT_EXISTING_DOCUMENTS.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,22 @@ To retry the failed imports, you can use the output file to manually inspect or
140140
141141
> **Note:** If the specified file already exists, it will be **cleared** before writing new failed batch paths.
142142
143+
### Using the "Generate Schema Views" After Import
144+
145+
After using fs-bq-import-collection to import your Firestore data to BigQuery, your data will be available in two forms: a 'raw changelog' table that streams all Firestore events chronologically, and a 'raw latest' view showing the current state of each document. However, the raw data doesn't have proper typing; all fields are stored as strings inside a JSON structure. To make this data more useful for querying, you should generate schema views.
146+
147+
#### Why Use Schema Views
148+
149+
**Proper Data Types**: Convert string-based JSON to properly typed BigQuery columns.
150+
**Easier Querying**: Query your data using column names rather than JSON functions.
151+
**Preserve Complex Types**: Handle Firestore-specific types like arrays, maps, and geopoints.
152+
153+
#### Guide For Generate Schema Views
154+
155+
To generate a schema view, you may use the official fs-bq-schema-views CLI tool. You can find a guide for using this tool [here](./GENERATE_SCHEMA_VIEWS.md).
156+
157+
This Generate Schema Views tool has an optional AI schema generation tool, powered by Gemini, where it can sample from your original Cloud Firestore collection and generate an appropriate schema for your BigQuery Views as a first step. You can review and customize this schema before applying it to BigQuery.
158+
143159
### Using a Transform Function
144160
145161
You can optionally provide a transform function URL (`--transform-function-url` or `-f`) that will transform document data before it's written to BigQuery. The transform function should should recieve document data and return transformed data. The payload will contain the following:

0 commit comments

Comments
 (0)