Skip to content

Commit 6fdc091

Browse files
authored
feat: local fix mode (#5978)
Certain linters and formatters support fixing linting and formatting issues (fix mode). Before this change, Super-linter runs linters and formatters in a mode that doesn't modify the source code in any way (check only mode). With this change, Super-linter supports running linters and formatters in fix mode if explicitly requested by the configuration. If the configuration includes a variable named FIX_<language_name>, Super-linters modifies the command to run the linter or formatter for <language_name> to enable fix mode. The modifications to the linter or formatter command that Super-linter applies depend on what is the default for a particular linter: it either removes or adds options to the command to run the linter or formatter.
1 parent 633b8af commit 6fdc091

17 files changed

+787
-71
lines changed

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -89,7 +89,7 @@ test/reports
8989
# Test leftovers
9090
.lintr
9191
test/linters/rust_clippy/**/Cargo.lock
92-
test/linters/rust_clippy/**/target/**
92+
test/linters/rust_clippy/**/target
9393

9494
# Super-linter ouputs
9595
super-linter-output/

Makefile

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
all: info docker test ## Run all targets.
55

66
.PHONY: test
7-
test: info validate-container-image-labels docker-build-check docker-dev-container-build-check test-lib inspec lint-codebase test-default-config-files test-actions-runner-debug test-actions-steps-debug test-runner-debug test-find lint-subset-files test-custom-ssl-cert test-non-default-workdir test-git-flags test-non-default-home-directory test-git-initial-commit test-git-merge-commit-push test-log-level test-use-find-and-ignore-gitignored-files test-linters-expect-failure-log-level-notice test-bash-exec-library-expect-success test-bash-exec-library-expect-failure test-save-super-linter-output test-save-super-linter-output-custom-path test-save-super-linter-custom-summary test-linters ## Run the test suite
7+
test: info validate-container-image-labels docker-build-check docker-dev-container-build-check test-lib inspec lint-codebase fix-codebase test-default-config-files test-actions-runner-debug test-actions-steps-debug test-runner-debug test-find lint-subset-files test-custom-ssl-cert test-non-default-workdir test-git-flags test-non-default-home-directory test-git-initial-commit test-git-merge-commit-push test-log-level test-use-find-and-ignore-gitignored-files test-linters-expect-failure-log-level-notice test-bash-exec-library-expect-success test-bash-exec-library-expect-failure test-save-super-linter-output test-save-super-linter-output-custom-path test-save-super-linter-custom-summary test-linters test-linters-fix-mode-expect-success ## Run the test suite
88

99
# if this session isn't interactive, then we don't want to allocate a
1010
# TTY, which would fail, but if it is interactive, we do want to attach
@@ -231,18 +231,45 @@ test-git-flags: ## Run super-linter with different git-related flags
231231
.PHONY: lint-codebase
232232
lint-codebase: ## Lint the entire codebase
233233
docker run \
234+
-e CREATE_LOG_FILE=true \
234235
-e RUN_LOCAL=true \
235236
-e LOG_LEVEL=DEBUG \
236237
-e DEFAULT_BRANCH=main \
237238
-e ENABLE_GITHUB_ACTIONS_GROUP_TITLE=true \
238239
-e FILTER_REGEX_EXCLUDE=".*(/test/linters/|CHANGELOG.md).*" \
239240
-e GITLEAKS_CONFIG_FILE=".gitleaks-ignore-tests.toml" \
240241
-e RENOVATE_SHAREABLE_CONFIG_PRESET_FILE_NAMES="default.json,hoge.json" \
242+
-e SAVE_SUPER_LINTER_OUTPUT=true \
241243
-e SAVE_SUPER_LINTER_SUMMARY=true \
242244
-e VALIDATE_ALL_CODEBASE=true \
243245
-v "$(CURDIR):/tmp/lint" \
244246
$(SUPER_LINTER_TEST_CONTAINER_URL)
245247

248+
# Return an error if there are changes to commit
249+
.PHONY: fix-codebase
250+
fix-codebase: ## Fix and format the entire codebase
251+
docker run \
252+
-e CREATE_LOG_FILE=true \
253+
-e DEFAULT_BRANCH=main \
254+
-e ENABLE_GITHUB_ACTIONS_GROUP_TITLE=true \
255+
-e FILTER_REGEX_EXCLUDE=".*(/test/linters/|CHANGELOG.md).*" \
256+
-e FIX_ENV=true \
257+
-e FIX_JAVASCRIPT_ES=true \
258+
-e FIX_JAVASCRIPT_PRETTIER=true \
259+
-e FIX_JSON=true \
260+
-e FIX_MARKDOWN=true \
261+
-e FIX_SHELL_SHFMT=true \
262+
-e GITLEAKS_CONFIG_FILE=".gitleaks-ignore-tests.toml" \
263+
-e LOG_LEVEL=DEBUG \
264+
-e RUN_LOCAL=true \
265+
-e SAVE_SUPER_LINTER_OUTPUT=true \
266+
-e SAVE_SUPER_LINTER_SUMMARY=true \
267+
-e VALIDATE_ALL_CODEBASE=true \
268+
-v "$(CURDIR):/tmp/lint" \
269+
$(SUPER_LINTER_TEST_CONTAINER_URL) \
270+
&& /bin/bash -c "source test/testUtils.sh; if ! CheckUnexpectedGitChanges ${CURDIR}; then exit 1; fi"
271+
272+
246273
# This is a smoke test to check how much time it takes to lint only a small
247274
# subset of files, compared to linting the whole codebase.
248275
.PHONY: lint-subset-files
@@ -389,6 +416,14 @@ test-non-default-home-directory: ## Test a non-default HOME directory
389416
"run_test_cases_non_default_home" \
390417
"$(IMAGE)"
391418

419+
.PHONY: test-linters-fix-mode-expect-success
420+
test-linters-fix-mode-expect-success: ## Run the linters test suite (fix mode) expecting successes
421+
$(CURDIR)/test/run-super-linter-tests.sh \
422+
$(SUPER_LINTER_TEST_CONTAINER_URL) \
423+
"run_test_case_fix_mode" \
424+
"$(IMAGE)"
425+
426+
392427
.PHONY: test-linters
393428
test-linters: test-linters-expect-success test-linters-expect-failure ## Run the linters test suite
394429

README.md

Lines changed: 70 additions & 4 deletions
Large diffs are not rendered by default.

docs/add-new-linter.md

Lines changed: 71 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,10 @@ new tool, it should include:
88
- Provide test cases:
99

1010
1. Create the `test/linters/<LANGUGAGE>` directory.
11-
2. Provide at least one test case with a file that is supposed to pass validation: `test/linters/<LANGUAGE>/<name-of-tool>-good`
12-
3. Provide at least one test case with a file that is supposed to fail validation: `test/linters/<LANGUAGE>/<name-of-tool>-bad`
11+
2. Provide at least one test case with a file that is supposed to pass validation,
12+
with the right file extension if needed: `test/linters/<LANGUAGE>/<name-of-tool>-good`
13+
3. Provide at least one test case with a file that is supposed to fail validation,
14+
with the right file extension if needed: `test/linters/<LANGUAGE>/<name-of-tool>-bad`
1315

1416
- Update the test suite to check for installed packages, the commands that your new tool needs in the `PATH`, and the expected version command:
1517

@@ -89,28 +91,73 @@ new tool, it should include:
8991
9092
- Update the orchestration scripts to run the new tool:
9193
92-
- `globals/languages.sh`: add a new item to `LANGUAGES_ARRAY` array. Use the
93-
"name" of the language, then a `_`, and finally the name of the linter. Example: `PYTHON_RUFF`
94+
- `lib/globals/languages.sh`: add a new item to `LANGUAGES_ARRAY` array. Use the
95+
"name" of the language, then a `_`, and finally the name of the linter. Example: `PYTHON_RUFF`.
96+
In the context of this document, to avoid repetitions we reference this new
97+
item as `<LANGUAGE_NAME>`.
98+
9499
- Linter configuration:
95-
- `globals/linterRules.sh`:
96-
- If the new linter accepts a configuration files from the command line, add a new variable
97-
with a default filename using the item that you added to the `LANGUAGES_ARRAY` as a prefix,
98-
followed by the `CONFIG_FILE` suffix. Example:
99-
`PYTHON_RUFF_FILE_NAME="${PYTHON_RUFF_CONFIG_FILE:-.ruff.toml}"`.
100-
- If there are arguments that you can only pass using the command line, and you think users
101-
might want to customize them, define a new variable using the item that
102-
you added to the `LANGUAGES_ARRAY` as a prefix, followed by the
103-
`COMMAND_ARGS` suffix. Example:
104-
`GITHUB_ACTIONS_COMMAND_ARGS="${GITHUB_ACTIONS_COMMAND_ARGS:-""}"`
105100
- Create a new minimal configuration file in the `TEMPLATES` directory with the same name as the
106101
default configuration filename. Example: `TEMPLATES/.ruff.toml`.
107-
- `lib/linter.sh`
108-
- `lib/functions/linterCommands.sh`: define a new array to invoke the new linter.
109-
- Provide the logic to populate the list of files or directories to examine: `lib/buildFileList.sh`
110-
- If necessary, provide elaborate logic to detect if the tool should examine a file or a directory: `lib/detectFiles.sh`
111-
- If the tool needs to take into account special cases:
112-
113-
- Provide new runtime validation checks in `lib/validation.sh`.
114-
- Customize the logic to get the installed version of the tool: `scripts/linterVersions.sh`
115-
- Provide custom logic to load configuration files: `lib/linterRules.sh`
116-
- Provide custom logic for test cases and to run the tool: `lib/worker.sh`
102+
- `lib/globals/linterRules.sh`:
103+
- If the new linter accepts a configuration files from the command line,
104+
define a new variable:
105+
`<LANGUAGE_NAME>_FILE_NAME="${<LANGUAGE_NAME>_CONFIG_FILE:-"default-config-file-name.conf"}"`
106+
where `default-config-file-name.conf` is the name of the new,
107+
minimal configuration for the linter. Example:
108+
`PYTHON_RUFF_FILE_NAME="${PYTHON_RUFF_CONFIG_FILE:-.ruff.toml}"`.
109+
- If there are arguments that you can only pass using the command line, and you think users
110+
might want to customize them, define a new variable using
111+
`<LANGUAGE_NAME>_COMMAND_ARGS` and add it to the command if the
112+
configuration provides it. Example:
113+
114+
```bash
115+
<LANGUAGE_NAME>_COMMAND_ARGS="${<LANGUAGE_NAME>_COMMAND_ARGS:-""}"
116+
if [ -n "${<LANGUAGE_NAME>_COMMAND_ARGS:-}" ]; then
117+
export <LANGUAGE_NAME>_COMMAND_ARGS
118+
LINTER_COMMANDS_ARRAY_<LANGUAGE_NAME>+=("${<LANGUAGE_NAME>_COMMAND_ARGS}")
119+
fi
120+
```
121+
122+
- Define the command to invoke the new linter:
123+
124+
- `lib/functions/linterCommands.sh`: add the command to invoke the linter.
125+
Define a new variable: `LINTER_COMMANDS_ARRAY_<LANGUAGE_NAME>`.
126+
Example:
127+
`LINTER_COMMANDS_ARRAY_GO_MODULES=(golangci-lint run --allow-parallel-runners -c "${GO_LINTER_RULES}")`
128+
129+
If the linter needs to load a configuration file, add the relevant options
130+
and paths to the command you just defined. The path to the configuration
131+
file is automatically initialized by Super-linter using in the
132+
`<LANGUAGE_NAME>_LINTER_RULES` variable, as in the `GO_LINTER_RULES`
133+
example above for the `GO` language.
134+
135+
- `lib/globals/linterCommandsOptions.sh`: add "check only mode" and "fix
136+
linting and formatting issues mode" options if the linter supports it.
137+
Super-linter will automatically add them to the command to run the linter.
138+
139+
- If the linter runs in "fix linting and formatting issues mode" by
140+
default, define a new variable with the options to add to the linter
141+
command to enable "check only mode":
142+
`<LANGUAGE_NAME>_CHECK_ONLY_MODE_OPTIONS=(....)`.
143+
Example: `PYTHON_BLACK_CHECK_ONLY_MODE_OPTIONS=(--diff --check)`
144+
145+
- If the linter runs in "check only mode" by
146+
default, define a new variable with the options to add to the linter
147+
command to enable "fix linting and formatting issues mode":
148+
`<LANGUAGE_NAME>_FIX_MODE_OPTIONS=(...)`.
149+
Example: `ANSIBLE_FIX_MODE_OPTIONS=(--fix)`
150+
151+
- Provide the logic to populate the list of files or directories to examine: `lib/functions/buildFileList.sh`
152+
- If necessary, provide elaborate logic to detect if the tool should examine a file or a directory: `lib/functions/detectFiles.sh`
153+
- If the tool needs to take into account special cases, reach out to the
154+
maintainers by creating a draft pull request and ask relevant questions
155+
there. For example, you might need to provide new logic or customize
156+
the existing one to:
157+
158+
- Validate the runtime environment: `lib/functions/validation.sh`.
159+
- Get the installed version of the linter: `scripts/linterVersions.sh`
160+
- Load configuration files: `lib/functions/linterRules.sh`
161+
- Run the linter: `lib/functions/worker.sh`
162+
- Compose the linter command: `lib/functions/linterCommands.sh`
163+
- Modify the core Super-linter logic: `lib/linter.sh`

docs/run-linter-locally.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,3 +139,12 @@ To get the list of the available `Make` targets, run the following command:
139139
```shell
140140
make help
141141
```
142+
143+
### Automatically fix formatting and linting issues
144+
145+
To automatically fix linting and formatting issues when supported, run the
146+
following command:
147+
148+
```shell
149+
make fix-codebase
150+
```

docs/upgrade-guide.md

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -39,17 +39,14 @@ This section helps you upgrade from super-linter `v6.7.0` to `v6.8.0`.
3939

4040
- If you set `JAVASCRIPT_DEFAULT_STYLE=standard`, set
4141
`VALIDATE_JAVASCRIPT_PRETTIER=false`
42-
4342
- If you set `TYPESCRIPT_DEFAULT_STYLE=standard`, set
4443
`VALIDATE_TYPESCRIPT_PRETTIER=false`
45-
4644
- If you set `JAVASCRIPT_DEFAULT_STYLE=prettier`, set
4745
`VALIDATE_JAVASCRIPT_STANDARD=false`
48-
4946
- If you set `TYPESCRIPT_DEFAULT_STYLE=prettier`, set
5047
`VALIDATE_TYPESCRIPT_STANDARD=false`
5148

52-
Finally, you remove both `JAVASCRIPT_DEFAULT_STYLE` and
49+
Finally, remove both `JAVASCRIPT_DEFAULT_STYLE` and
5350
`TYPESCRIPT_DEFAULT_STYLE` from your Super-linter configuration.
5451

5552
## Upgrade from v5 to v6

0 commit comments

Comments
 (0)