Handle configurations of hooks

[jgsogo]

As soon as hooks start to proliferate there will be more and more environment variables to configure them, I think we should consider that risk and think about alternatives, there could be a better approach. It is time for brainstorming:

• Do nothing, hooks will start to introduce env variables (there are already some of them), I think it could be a mess in the long term

• Use a json file to handle the configuration of all the hooks (shareable with a conan config install), the HookManager could pass those variables in every function call:

  hooks_config.json

  {
      "hook_name_1": {
          "key1": "value",
          "key2": "value"
      },
      "hook_name_2": {
          "...": "..."
      }
  }

  hooks/hook_name_1.py

  ...
  def pre_export(..., config):
      assert config == {"key1": "value", "key2": "value"}
      ...
  
  def post_package(..., config):
      ...

• add them to conan.conf file inside the cache

• ...more ideas?

[SSE4]

do hooks have an access to the conan config via some API?

[jgsogo]

Do you mean the conan.conf file? No, they don't, that file is read at the beginning of every command and all those variables are applied as environment variables. It will require some work, but it is an option to consider (I'm adding it to the points in the first comment)

[SSE4]

yeah, I mean conan.conf, I use occam's razor here, as we already have an entity to handle configuration, when may be it's not necessary to invent a new wheel?

[danimtb]

What about having a json file next to the hook entrypoint and let the hook functions load it with a helper or somehting like that? I think it could be very convenient without any further modification to the conan codebase than just adding a "load_from_file_dirname" tool. WDYT?

My orther alternative would be conan.conf but I don't like it in first instace because its content is already packed (many options already in it) and syntax is evolving wildly

[SSE4]

as a side note, we're currently very inconsistent in usage of configuration files:

• YAML: settings.yml, conanws.yml
• INI: conan.conf, conanfile.txt, conan profiles, layouts
• JSON: registry.json

may be there are others, I am not sure.
@danimtb why do you propose JSON in this particular case? it has some known limitations (e.g. it doesn't allow comments). are there any advantages for JSON usage here? why not INI or YAML or TOML? IMO if we're not going to pass this directly to some rest API, then there is no much sense to use JSON here...

[danimtb]

I don't care too much about the configuration syntax as long as it is a standard format

[uilianries]

The old discussion about configuration file formats, we can sit and drink a cold beer 🍺 🗣️

I would prefer follow the same standard from Conan project, so we can re-use all well known code base. But if we want to change the file format, I vote for TOML. Both JSON and XML are easier to be parsed, but since we are talking about configuration files they should be easy for human, not for machines. Thus, I prefer simple syntax like .INI, but .TOML I see like the evolution of .INI. Yaml is cool, but sometimes sucks when it complains about wrong spaces number or tabs.

Also, I would like to set the logging level of the Hooks since we could run hooks on the CI and most of the services offer a limited line number.

Regards!

[jgsogo]

I don't care about the format itself, it is something we will address once we know where to put the configurations variables and how to pass them to each hook. For me, that is the key.

[SSE4]

I would prefer to avoid re-inventing the wheel and use some common practices about config files in general:

1. use a global file (doesn't matter now json/xml/toml/ini/yaml/etc) within XDG_CONFIG_HOME directory. something like conan-plugins.conf
2. allow to override global settings for the particular project/directory via .rc file (something like .bashrc, .vimrc, .screenrc) - might be conan-plugins.rc
3. allow to override per command line invocation - environment variables might be good choice, probably auto-generated, like CONAN_PLUGIN_<plugin_name>_<section_name>_<setting_name>, e.g. CONAN_PLUGIN_BINTRAY_GENERAL_TOKEN. or simply something like CONAN_PLUGIN_SETTINGS=bintray.general.token:something;bintray.general.user:anonymous. or alternatively, via command line argument (like -s)
4. each plug-in should have an access to config object or at least python dictionary. plug-in shouldn't try to directly locate and read config files - it should be abstracted away from it. this way it allows to easily change storage location or format.