Merge pull request #344 from Workiva/add-fast-format

CPLAT-11875: Add Fast Format Command

Author: rmconsole7-wk

.travis.yml

@@ -11,7 +11,7 @@ jobs:
       name: "SDK: stable"
       script:
         - pub run dart_dev analyze
-        - pub run dependency_validator -i pedantic
+        - pub run dependency_validator -i pedantic over_react_format
         - pub run dart_dev format --check
         - pub run dart_dev test
         - pub publish --dry-run

CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## [3.6.0](https://github.com/Workiva/dart_dev/compare/3.5.0...3.6.0)
+
+- Support a faster format command for better integration with JetBrains file
+  watching for format-on-save functionality.
+
 ## [3.5.0](https://github.com/Workiva/dart_dev/compare/3.4.0...3.5.0)
 
 - Added an optional `collapseDirectories` param to `FormatTool.getInputs()`.

README.md

@@ -13,6 +13,7 @@ Easily configurable.
 - [Project-Level Configuration](#project-level-configuration)
 - [Extending/Composing Functionality](#extendingcomposing-functionality)
 - [Shared Configuration](#shared-configuration)
+- [Format on save](#format-on-save)
 - [Additional Docs][docs]
 
 ## Quick Start
@@ -231,8 +232,94 @@ final config = {
 };
 ```
 
+## Format on save
+dart_dev can be used to facilitate formatting on save inside of JetBrains IDEs. For setup instructions, see below.
+
+### A Note on VS Code
+A VS code extension exists to run either `dartfmt` or `over_react_format` on save. For information on it, see [its project](vs-code-formatter). However, that VS Code extension does not run `dart_dev`, but rather has its own logic to run a formatting command.
+
+### JetBrains IDEs (WebStorm, IntelliJ, etc.)
+Webstorm exposes a File Watcher utility that can be used to run commands when a file saves. For this approach, all you need to do is set up the file watcher. Shoutout to @patkujawa-wf for creating the original inspiration of this solution!
+
+__NOTE:__ Before setting up the watcher, there are three basic limitations when using it:
+1. dart_dev's minimum must be at least version `3.6.0` in the projects that uses the watcher.
+1. Only dart_dev's `FormatTool` and OverReact Format's `OverReactFormatTool` are supported.
+1. Literals need to be used when possible when configuring the formatter. This primarily pertains to the formatter tool itself and setting the property that is responsible for line-length. For example:
+    ```dart
+    // Good
+    final Map<String, DevTool> config = {
+      // ... other config options
+      'format': FormatTool()
+        ..formatter = Formatter.dartStyle
+        ..formatterArgs = ['-l', '120'],
+    };
+
+    // Bad
+
+    // Example 1: Line-length as a variable
+    const lineLength = 120;
+
+    final Map<String, DevTool> config = {
+      // ... other config options
+      'format': FormatTool()
+        ..formatter = Formatter.dartStyle
+        ..formatterArgs = ['-l', lineLength],
+    };
+
+    // Example 2: Args as a variable
+    const formatterArgs = ['-l', '120'];
+
+    final Map<String, DevTool> config = {
+      // ... other config options
+      'format': FormatTool()
+        ..formatter = Formatter.dartStyle
+        ..formatterArgs = formatterArgs,
+    };
+
+    // Example 3: Formatter as a variable
+    final formatter = FormatTool()
+        ..formatter = Formatter.dartStyle
+        ..formatterArgs = ['-l', '120'];
+
+    final Map<String, DevTool> config = {
+      // ... other config options
+      'format': formatter,
+    };
+    ```
+
+#### Setting Up the File Watcher
+1. Go into Webstorm's preferences. It doesn't matter what project you do this in, as you'll ultimately want to make the watcher global. More on that later, though!
+1. Navigate to the "File Watchers" settings. This is under "Preferences > Tools > File Watchers". The File Watcher pane should look something like: 
+
+    <img src='./images/file_watcher_pane.png' width="75%" alt='File Watcher Pane'>
+
+1. Clicking on the import icon on the bottom toolbar.
+1. Import the `format_on_save.xml` file found in this project, at "[dart_dev/tool/file_watchers/format_on_save.xml](tool/file_watchers/ddev_format_on_save.xml)".
+1. After importing, change the watcher scoping (AKA "level") to `Global` on the right hand side under the "level" column, which makes the watcher available for use in all projects.
+1. In each project, you will also have to enable the watcher by checking the box on the file watcher's row.
+
+For additional reference on how the watcher is set up, see [JetBrains File Watcher Configuration](#jetbrains-file-watcher-configuration).
+
+
+#### JetBrains File Watcher Configuration
+<img src='./images/file_watcher_config.png' width="75%" alt='Final File Watcher Configuration'>
+  
+  1. __The Name:__ Webstorm treats this like the process name, so it's the identifier that will be used to display any output that the process is running. It can be whatever you like!
+  1. __File Type:__ `Dart`, since that's what the formatter was built for.
+  1. __Scope:__ `Project Files` will produce the desired effect, but if a different option works better for you then feel free! For more information on scoping, see [the docs](file-watcher-docs).
+  1. __Program:__ The exutable to run. In this case, it can just be `pub`. If there are any issues, providing a full path to the executable may have the desired outcome. For pub, this is most likely `/usr/local/bin/pub`.
+  1. __Arguments:__ The rest of the command, and by default should be `run dart_dev hackFastFormat "$FilePathRelativeToProjectRoot$"`. Here's the breakdown:
+      - `run dart_dev hackFastFormat`: Simply the process to run. 
+      - `"$FilePathRelativeToProjectRoot$"`: The environment variable that will target only the changed file.
+  1. __Output Paths to Refresh:__ `"$FilePathRelativeToProjectRoot$"`.
+  1. __Working Directory:__ `$ContentRoot$`.
+  1. __Advanced Options:__ Uncheck all the boxes. Again, if you experiment and find having some of them checked is better then feel free! However, the expected behavior occurs when none of them are checked.
+
+
 [api-docs]: https://pub.dev/documentation/dart_dev/latest/dart_dev/dart_dev-library.html
 [build-filter]: https://github.com/dart-lang/build/blob/master/build_runner/CHANGELOG.md#new-feature-build-filters
 [core-config]: https://github.com/Workiva/dart_dev/blob/master/lib/src/core_config.dart
 [docs]: https://github.com/Workiva/dart_dev/blob/master/doc/
+[file-watcher-docs]: https://www.jetbrains.com/help/webstorm/using-file-watchers.html#
 [upgrade-guide]: https://github.com/Workiva/dart_dev/blob/master/doc/v3-upgrade-guide.md
+[vs-code-formatter]: https://github.com/Workiva/vs-code-format-on-save/blob/master/README.md

images/file_watcher_config.png

@@ -1,15 +1,21 @@
 import 'dart:async';
 import 'dart:io';
+import 'dart:math';
 
+import 'package:analyzer/dart/analysis/utilities.dart';
 import 'package:args/command_runner.dart';
+import 'package:dart_dev/dart_dev.dart';
 import 'package:dart_dev/src/dart_dev_tool.dart';
+import 'package:dart_dev/src/utils/format_tool_builder.dart';
 import 'package:dart_dev/src/utils/parse_flag_from_args.dart';
 import 'package:io/ansi.dart';
 import 'package:io/io.dart' show ExitCode;
 import 'package:logging/logging.dart';
 import 'package:path/path.dart' as p;
 
+import '../utils.dart';
 import 'dart_dev_runner.dart';
+import 'tools/over_react_format_tool.dart';
 import 'utils/assert_dir_is_dart_package.dart';
 import 'utils/dart_tool_cache.dart';
 import 'utils/ensure_process_exit.dart';
@@ -30,25 +36,28 @@ final _relativeDevDartPath = p.relative(
   from: p.absolute(p.dirname(_runScriptPath)),
 );
 
-final _log = Logger('DartDev');
-
 Future<void> run(List<String> args) async {
   attachLoggerToStdio(args);
-
   final configExists = File(_configPath).existsSync();
   final oldDevDartExists = File(_oldDevDartPath).existsSync();
 
   if (!configExists) {
-    _log.fine('No custom `tool/dart_dev/config.dart` file found; '
+    log.fine('No custom `tool/dart_dev/config.dart` file found; '
         'using default config.');
   }
   if (oldDevDartExists) {
-    _log.warning(yellow.wrap(
+    log.warning(yellow.wrap(
         'dart_dev v3 now expects configuration to be at `$_configPath`,\n'
         'but `$_oldDevDartPath` still exists. View the guide to see how to upgrade:\n'
         'https://github.com/Workiva/dart_dev/blob/master/doc/v3-upgrade-guide.md'));
   }
 
+  if (args.contains('hackFastFormat') && !oldDevDartExists) {
+    await handleFastFormat(args);
+
+    return;
+  }
+
   generateRunScript();
   final process = await Process.start(
       Platform.executable, [_runScriptPath, ...args],
@@ -57,9 +66,45 @@ Future<void> run(List<String> args) async {
   exitCode = await process.exitCode;
 }
 
+Future<void> handleFastFormat(List<String> args) async {
+  assertDirIsDartPackage();
+
+  DevTool formatTool;
+  final configFile = File(_configPath);
+  if (configFile.existsSync()) {
+    final toolBuilder = FormatToolBuilder();
+    parseString(content: configFile.readAsStringSync())
+        .unit
+        .accept(toolBuilder);
+    formatTool = toolBuilder
+        .formatDevTool; // could be null if no custom `format` entry found
+
+    if (formatTool == null && toolBuilder.failedToDetectAKnownFormatter) {
+      exitCode = ExitCode.config.code;
+      log.severe('Failed to reconstruct the format tool\'s configuration.\n\n'
+          'This is likely because dart_dev expects either the FormatTool class or the\n'
+          'OverReactFormatTool class.');
+      return;
+    }
+  }
+
+  formatTool ??= chooseDefaultFormatTool();
+
+  try {
+    exitCode = await DartDevRunner({'hackFastFormat': formatTool}).run(args);
+  } catch (error, stack) {
+    log.severe('Uncaught Exception:', error, stack);
+    if (!parseFlagFromArgs(args, 'verbose', abbr: 'v')) {
+      // Always print the stack trace for an uncaught exception.
+      stderr.writeln(stack);
+    }
+    exitCode = ExitCode.unavailable.code;
+  }
+}
+
 void generateRunScript() {
   if (shouldWriteRunScript) {
-    logTimedSync(_log, 'Generating run script', () {
+    logTimedSync(log, 'Generating run script', () {
       createCacheDir();
       _runScript.writeAsStringSync(buildDartDevRunScriptContents());
     }, level: Level.INFO);
@@ -93,7 +138,7 @@ Future<void> runWithConfig(
   try {
     assertDirIsDartPackage();
   } on DirectoryIsNotPubPackage catch (error) {
-    _log.severe(error);
+    log.severe(error);
     return ExitCode.usage.code;
   }
 
@@ -119,11 +164,25 @@ Future<void> runWithConfig(
     stderr.writeln(error);
     exitCode = ExitCode.usage.code;
   } catch (error, stack) {
-    _log.severe('Uncaught Exception:', error, stack);
+    log.severe('Uncaught Exception:', error, stack);
     if (!parseFlagFromArgs(args, 'verbose', abbr: 'v')) {
       // Always print the stack trace for an uncaught exception.
       stderr.writeln(stack);
     }
     exitCode = ExitCode.unavailable.code;
   }
 }
+
+/// Returns [OverReactFormatTool] if `over_react_format` is a direct dependency,
+/// and the default [FormatTool] otherwise.
+DevTool chooseDefaultFormatTool({String path}) {
+  final pubspec = cachedPubspec(path: path);
+  const orf = 'over_react_format';
+  final hasOverReactFormat = pubspec.dependencies.containsKey(orf) ||
+      pubspec.devDependencies.containsKey(orf) ||
+      pubspec.dependencyOverrides.containsKey(orf);
+
+  return hasOverReactFormat
+      ? OverReactFormatTool()
+      : (FormatTool()..formatter = Formatter.dartStyle);
+}

images/file_watcher_pane.png

@@ -354,10 +354,8 @@ Iterable<String> buildArgs(
       '-n',
       '--set-exit-if-changed',
     ],
-    if (mode == FormatMode.overwrite)
-      '-w',
-    if (mode == FormatMode.dryRun)
-      '-n',
+    if (mode == FormatMode.overwrite) '-w',
+    if (mode == FormatMode.dryRun) '-n',
 
     // 2. Statically configured args from [FormatTool.formatterArgs]
     ...?configuredFormatterArgs,
@@ -402,9 +400,14 @@ FormatExecution buildExecution(
   String path,
 }) {
   FormatMode mode;
+
+  final useRestForInputs = (context?.argResults?.rest?.isNotEmpty ?? false) &&
+      context.commandName == 'hackFastFormat';
+
   if (context.argResults != null) {
     assertNoPositionalArgsNorArgsAfterSeparator(
         context.argResults, context.usageException,
+        allowRest: useRestForInputs,
         commandName: context.commandName,
         usageFooter: 'Arguments can be passed to the "dartfmt" process via the '
             '--formatter-args option.');
@@ -422,11 +425,21 @@ FormatExecution buildExecution(
             'format tool to use "dartfmt" instead.'));
     return FormatExecution.exitEarly(ExitCode.config.code);
   }
-  final inputs = FormatTool.getInputs(
-    exclude: exclude,
-    root: path,
-    collapseDirectories: true,
-  );
+
+  if (context.commandName == 'hackFastFormat' && !useRestForInputs) {
+    context.usageException('"hackFastFormat" must specify targets to format.\n'
+        'hackFastFormat should only be used to format specific files. '
+        'Running the command over an entire project may format files that '
+        'would be excluded using the standard "format" command.');
+  }
+
+  final inputs = useRestForInputs
+      ? FormatterInputs({...context.argResults.rest})
+      : FormatTool.getInputs(
+          exclude: exclude,
+          root: path,
+          collapseDirectories: true,
+        );
 
   if (inputs.includedFiles.isEmpty) {
     _log.severe('The formatter cannot run because no inputs could be found '

lib/src/executable.dart

@@ -0,0 +1,39 @@
+import 'dart:async';
+import 'dart:io';
+
+import 'package:dart_dev/dart_dev.dart';
+import 'package:dart_dev/utils.dart';
+
+import '../tools/format_tool.dart';
+
+class OverReactFormatTool extends DevTool {
+  /// Wrap lines longer than this.
+  ///
+  /// Default is 80.
+  int lineLength;
+
+  @override
+  String description =
+      'Format dart files in this package with over_react_format.';
+
+  @override
+  FutureOr<int> run([DevToolExecutionContext context]) async {
+    Iterable<String> paths = context?.argResults?.rest;
+    if (paths?.isEmpty ?? true) {
+      context?.usageException(
+          '"hackFastFormat" must specify targets to format.\n'
+          'hackFastFormat should only be used to format specific files. '
+          'Running the command over an entire project may format files that '
+          'would be excluded using the standard "format" command.');
+    }
+    final args = [
+      'run',
+      'over_react_format',
+      if (lineLength != null) '--line-length=$lineLength'
+    ];
+    final process = ProcessDeclaration('pub', [...args, ...paths],
+        mode: ProcessStartMode.inheritStdio);
+    logCommand('pub', paths, args, verbose: context?.verbose);
+    return runProcessAndEnsureExit(process);
+  }
+}

lib/src/tools/format_tool.dart

@@ -198,8 +198,7 @@ List<String> buildArgs({
     // 1. Statically configured args from [WebdevServeTool.buildArgs]
     ...?configuredBuildArgs,
     // 2. Pass through the --release flag if provided.
-    if (flagValue(argResults, 'release') ?? false)
-      '--release',
+    if (flagValue(argResults, 'release') ?? false) '--release',
     // 3. Build filters to narrow the build to only the target tests.
     //    (If no test dirs/files are passed in as args, then no build filters
     //     will be created.)
@@ -236,15 +235,12 @@ List<String> buildArgs({
   return [
     // `pub run test` or `pub run build_runner test`
     'run',
-    if (useBuildTest)
-      'build_runner',
+    if (useBuildTest) 'build_runner',
     'test',
 
     // Add the args targeting the build_runner command.
-    if (useBuildTest)
-      ...buildArgs,
-    if (useBuildTest && testArgs.isNotEmpty)
-      '--',
+    if (useBuildTest) ...buildArgs,
+    if (useBuildTest && testArgs.isNotEmpty) '--',
 
     // Add the args targeting the test command.
     ...testArgs,

lib/src/tools/over_react_format_tool.dart

@@ -142,8 +142,7 @@ List<String> buildArgs(
     // 1. Statically configured args from [WebdevServeTool.webdevArgs]
     ...?configuredWebdevArgs,
     // 2. The -r|--release flag
-    if (argResults != null && argResults['release'] ?? false)
-      '--release',
+    if (argResults != null && argResults['release'] ?? false) '--release',
     // 3. Args passed to --webdev-args
     ...?splitSingleOptionValue(argResults, 'webdev-args'),
   ];