Skip to content

Consolidate configuration sections in docs #16544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 25, 2025
+40 −128
Merged

Consolidate configuration sections in docs #16544

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dcb1d34
Consolidate configuration sectionds in docs
alamb Jun 24, 2025
7bb8acb
remove another referece
alamb Jun 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .github/workflows/rust.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -707,11 +707,6 @@ jobs:
# If you encounter an error, run './dev/update_function_docs.sh' and commit
./dev/update_function_docs.sh
git diff --exit-code
- name: Check if runtime_configs.md has been modified
run: |
# If you encounter an error, run './dev/update_runtime_config_docs.sh' and commit
./dev/update_runtime_config_docs.sh
git diff --exit-code

# Verify MSRV for the crates which are directly used by other projects:
# - datafusion
Expand Down
6 changes: 0 additions & 6 deletions datafusion/core/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -901,12 +901,6 @@ doc_comment::doctest!(
user_guide_configs
);

#[cfg(doctest)]
doc_comment::doctest!(
"../../../docs/source/user-guide/runtime_configs.md",
user_guide_runtime_configs
);

#[cfg(doctest)]
doc_comment::doctest!(
"../../../docs/source/user-guide/crate-configuration.md",
Expand Down
23 changes: 23 additions & 0 deletions dev/update_config_docs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ cd "${SOURCE_DIR}/../" && pwd

TARGET_FILE="docs/source/user-guide/configs.md"
PRINT_CONFIG_DOCS_COMMAND="cargo run --manifest-path datafusion/core/Cargo.toml --bin print_config_docs"
PRINT_RUNTIME_CONFIG_DOCS_COMMAND="cargo run --manifest-path datafusion/core/Cargo.toml --bin print_runtime_config_docs"


echo "Inserting header"
cat <<'EOF' > "$TARGET_FILE"
Expand Down Expand Up @@ -70,6 +72,27 @@ EOF
echo "Running CLI and inserting config docs table"
$PRINT_CONFIG_DOCS_COMMAND >> "$TARGET_FILE"

echo "Inserting runtime config header"
cat <<'EOF' >> "$TARGET_FILE"

# Runtime Configuration Settings

DataFusion runtime configurations can be set via SQL using the `SET` command.

For example, to configure `datafusion.runtime.memory_limit`:

```sql
SET datafusion.runtime.memory_limit = '2G';
```

The following runtime configuration settings are available:

EOF

echo "Running CLI and inserting runtime config docs table"
$PRINT_RUNTIME_CONFIG_DOCS_COMMAND >> "$TARGET_FILE"


echo "Running prettier"
npx [email protected] --write "$TARGET_FILE"

Expand Down
76 changes: 0 additions & 76 deletions dev/update_runtime_config_docs.sh
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions docs/source/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,4 +125,5 @@

redirects = {
"library-user-guide/adding-udfs": "functions/index.html",
"user-guide/runtime_configs": "configs.html",
}
1 change: 0 additions & 1 deletion docs/source/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,6 @@ To get started, see
user-guide/expressions
user-guide/sql/index
user-guide/configs
user-guide/runtime_configs
user-guide/explain-usage
user-guide/faq

Expand Down
16 changes: 16 additions & 0 deletions docs/source/user-guide/configs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -144,3 +144,19 @@ Environment variables are read during `SessionConfig` initialisation so they mus
| datafusion.format.time_format | %H:%M:%S%.f | Time format for time arrays |
| datafusion.format.duration_format | pretty | Duration format. Can be either `"pretty"` or `"ISO8601"` |
| datafusion.format.types_info | false | Show types in visual representation batches |

# Runtime Configuration Settings

DataFusion runtime configurations can be set via SQL using the `SET` command.

For example, to configure `datafusion.runtime.memory_limit`:

```sql
SET datafusion.runtime.memory_limit = '2G';
```

The following runtime configuration settings are available:

| key | default | description |
| ------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| datafusion.runtime.memory_limit | NULL | Maximum memory limit for query execution. Supports suffixes K (kilobytes), M (megabytes), and G (gigabytes). Example: '2G' for 2 gigabytes. |
40 changes: 0 additions & 40 deletions docs/source/user-guide/runtime_configs.md
View file
Open in desktop

This file was deleted.