Standardizing on a directory in `$HOME` for Dart (and Flutter) tooling

[dcharkes]

@bkonyi suggested we should standardize on a directory in $HOME we use for Dart tooling.

We currently have:

• ~/.dart-tool/ (containing telemetry data)
• ~/.dart/ (analytics)
• ~/.dartServer/ (analysis server)
• ~/.pub-cache (introduced in the days when pub was a standalone tool rather than dart pub)
• ~/.flutter-devtools/.devtools

For a the workspace-specific directory we have standardized on <workspace>/.dart_tool/<package-name>/ (doc). (Note the underscore instead of dash.)

Should we standardize?

And if so, should we standardize to ~/.dart/?

• ~/.dart/telemetry/
• ~/.dart/analytics/
• ~/.dart/analysis_server/
• ~/.dart/pub/ (Can we migrate, or is that too painful? @sigurdm)
• ~/.dart/.devtools/
• ~/.dart/installed/ ([cli] Introduce dart install (replacing dart pub global activate to use) #60889)

[sigurdm]

~/.dart_tool/pub/ (Can we migrate, or is that too painful? @sigurdm)

We could, but it would have to be planned well. The pub-cache can be very big, so we would need a system in place for either moving it, or reminding people they can delete it after they stop using older sdks.

If we do indeed decide to move it I have a number of smaller changes I'd like to introduce at the same time.

[mraleph]

If we do any efforts on standardizing then it certainly should not be in the $HOME. On Linux we should follow XDG: #41560. On Mac we should probably start putting stuff into appropriate subdirectories of $HOME/Library/.

[sigurdm]

Pub stores its login credentials in
https://github.com/dart-lang/pub/blob/be6868ba132c782d9835b1638a634ecb73428579/doc/cache_layout.md#L49

• On Linux $XDG_CONFIG_HOME/dart/pub-credentials.json if $XDG_CONFIG_HOME is
  defined, otherwise $HOME/.config/dart/pub-credentials.json
• On macOS: $HOME/Library/Application Support/dart/pub-credentials.json
• On Windows: %APPDATA%/dart/pub-credentials.json

That is probably good for configuration, but I'm not sure it is good for more ephemeral data.

[lrhn]

(On Windows, %LOCALAPPDATA% would probably be better for data that should not be synced to other machines, like big caches. The %APPDATA% is <user_profile>\AppData\Roaming\ which is intended to be migratable to other machines on the same domain.)

[sigurdm]

(On Windows, %LOCALAPPDATA% would probably be better for data that should not be synced to other machines, like big caches. The %APPDATA% is <user_profile>\AppData\Roaming\ which is intended to be migratable to other machines on the same domain.)

Good point. We migrated the pub cache from %APPDATA% to %LOCALAPPDATA% at some point for the same reason.

(back in dart-lang/pub#2297)

[bkonyi]

cc @bwilkerson @kenzieschmoll

[bwilkerson]

~/.dartServer/ (analysis server)

I'll mirror @sigurdm's comments. This directory is also typically quite large and it would be best if we had a way to move the content rather than just abandoning it.

I don't actually have any preference for where this data lives, but the process of changing the location should be as transparent to users as possible.

We'll also need to consider how and when to update any documentation that mentions the locations of these directories.

[sigurdm]

(what does ~/.dartServer contain?)

[bwilkerson]

I contains the (extended) summary files related to the Dart files that have recently been analyzed in the IDE. (The summary files are extended with information about diagnostics that are reported.) It's used to improve startup time and to make the analyzer more incremental during normal use. It's an LRU cache, so files are removed from it as the server is running, but it can still be fairly large. (The directory on my disk is 1.4G, which is close to the 1.8G used by .pub-cache.)

[kenzieschmoll]

Another directory we could consider standardizing is the location of the DevTools storage file:

• ~/.flutter-devtools/.devtools --> ~/.dart_tool/.devtools

[goderbauer]

I don't have a strong opinion on where this should live, but I do like the current cross-platform consistency when jumping between e.g. Linux and macOS and finding these things in the same place.

[dcharkes]

If we do any efforts on standardizing then it certainly should not be in the $HOME. On Linux we should follow XDG: #41560. On Mac we should probably start putting stuff into appropriate subdirectories of $HOME/Library/.

So this means we'll have multiple dart/<dart_tool>/ directories spread out over the file system:

• MacOS
  • Config: ~/Library/Preferences/dart/<dart_tool>/ (should only contain .plist files)
  • Cache: ~/Library/Caches/dart/<dart_tool>/
  • Data: ~/Library/Application Support/dart/<dart_tool>/
  • Executables: ~/.local/bin/ (needs to be manually added to the PATH by the user)
• Linux
  • Config: $XDG_CONFIG_HOME/dart/<dart_tool>/ defaults to ~/.config/dart/<dart_tool>/
  • Cache: $XDG_CACHE_HOME/dart/<dart_tool>/ defaults to ~/.cache/dart/<dart_tool>/
  • Data: $XDG_DATA_HOME/dart/dart/<dart_tool>/ defaults to ~/.local/share/dart/<dart_tool>/
  • Executables: ~/.local/bin/
• Windows
  • Config: %APPDATA%\dart\<dart_tool>\ defaults to C:\Users\<YourUser>\AppData\Roaming\dart\<dart_tool>\.
  • Cache & Data: %LOCALAPPDATA%\dart\<dart_tool>\ defaults to C:\Users\<YourUser>\AppData\Local\dart\<dart_tool>\.
  • Executables: %LOCALAPPDATA%\dart\<dart_tool>\bin\ (needs to be manually added to the PATH by the user)

This will lead to multiple directories for a single Dart tool:

• pub: has a cache, but the dart pub global activated should be in data, and the binstub in executables, and the config in config. (And then the activated apps should not be pointing to the cache most likely, because at least MacOS proactively prunes caches. Or pub should put its whole cache in data.)
• dart install, has data and executable.

Some other uses seem to fit more in a single directory:

• dart analysis server: only data
• analytics: only contains a config file

If we were to stick to a home directory, we'd not split Config, Cache, Data, and Executables:

• MacOS
  • Config & Cache & Data: ~/.dart/<dart_tool>/
    • Executables: ./bin/ (needs to be manually added to the PATH by the user)
• Linux
  • Config & Cache & Data: ~/.dart/<dart_tool>/
  • Executables: ~/.local/bin/
• Windows
  • Config & Cache & Data: ~\dart\<dart_tool>\
    • Executables: .\bin\ (needs to be manually added to the PATH by the user)

The befits of using a single home directory for all Dart tools:

• Only a single home directory instead of multiple (current situation).
• Consistency across OSes.
• Single directory (easy to find, easy to delete).
• No need to make a distinction what kind of data.
• Every tool can have <TOOL_NAME>_HOME environment variable to override the single directory.

Benefits of using XDG, MacOS defaults, and Windows env vars:

• No home directory used.
• OS can prune caches. (MacOS actively prunes, Linux users might rm -rf their cache dir, Windows no support.)
• Configuration can travel with users to different machines. (Windows support, MacOS supported not used, Linux no support.)

(Edit: gist for a potential API for tool writers.)

[kenzieschmoll]

+1 to using a single home directory. This will greatly reduce complexity and opportunity for error for all Dart / Flutter tools that need to read from these files as opposed to having to read from a specific directory for a specific file for the current platform.