Skip to content

(BOLT-1589) Support installing any git-based module#3115

Merged
beechtom merged 4 commits into
puppetlabs:mainfrom
beechtom:1589/git-modules
Jul 27, 2022
Merged

(BOLT-1589) Support installing any git-based module#3115
beechtom merged 4 commits into
puppetlabs:mainfrom
beechtom:1589/git-modules

Conversation

@beechtom

@beechtom beechtom commented Jun 23, 2022

Copy link
Copy Markdown
Contributor

This updates Bolt to support installing git-based modules from any
source. Now, Bolt will attempt to download metadata directly from GitHub
or GitLab for public repositories and fall back to cloning the git
repo's metadata if that fails.

This also updates the spec resolver to no longer calculate a SHA based
on the ref provided in project configuration. Previously, Bolt would
calculate a SHA and write it to the Puppetfile to pin the module to a
specific commit when modules are installed. However, doing this is
difficult and time-consuming with the move to supporting any git-based
module.

!feature

  • Support installing any git-based module
    (#3109)

    Bolt now supports installing any git-based module. Previously, Bolt
    only supported installing and resolving dependencies for modules
    hosted in a public GitHub repository. Now, Bolt will check both GitHub
    and GitLab before falling back to cloning a module's metadata using
    the git executable. This allows users to install modules from
    private repositories, or from locations other than GitHub.


This fixes a bug where the proxy configured in module-install.proxy
was not being set when retrieving metadata for git-based modules.
Previously, the proxy was only configured when resolving and
installing modules.

!bug

  • Configure proxy when retrieving metadata for git-based modules

    Bolt now configures a proxy when module-install.proxy is set and it
    retrieves metadata for git-based modules. Previously, Bolt was only
    configuring a proxy when it resolved and installed modules.

@beechtom beechtom linked an issue Jun 23, 2022 that may be closed by this pull request
@beechtom beechtom force-pushed the 1589/git-modules branch 2 times, most recently from e690e74 to 269235d Compare June 23, 2022 17:18
@beechtom beechtom marked this pull request as ready for review June 23, 2022 19:07
@beechtom beechtom requested a review from a team as a code owner June 23, 2022 19:07
@nicklewis

Copy link
Copy Markdown
Contributor

Not calculating a SHA means the Puppetfile no longer refers to a specific version of the module, correct? In the case where the ref is a branch, that is.

@beechtom

Copy link
Copy Markdown
Contributor Author

@nicklewis Correct. I can add that back in, just a bit more work to get it working for GitLab & cloned sources.

@beechtom beechtom marked this pull request as draft June 27, 2022 22:34
@beechtom beechtom force-pushed the 1589/git-modules branch 2 times, most recently from f8fd164 to 8b5c9b6 Compare June 27, 2022 23:19
@beechtom beechtom added the Do Not Merge Work that should not be merged yet. label Jun 27, 2022
@beechtom beechtom force-pushed the 1589/git-modules branch 2 times, most recently from e3e95cb to efdd65b Compare June 27, 2022 23:35
Comment thread lib/bolt/module_installer/specs/id/gitclone.rb
@beechtom beechtom marked this pull request as ready for review June 27, 2022 23:49
@beechtom

Copy link
Copy Markdown
Contributor Author

Blocked by glennsarti/puppetfile-resolver#38

@donoghuc donoghuc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the "in person" walkthrough on this. Looks good, and we have some thoughts for improving this moving forward.

beechtom added 3 commits July 21, 2022 09:44
This updates Bolt to support installing git-based modules from any
source. Now, Bolt will attempt to download metadata directly from GitHub
or GitLab for public repositories and fall back to cloning the git
repo's metadata if that fails.

This also updates the spec resolver to no longer calculate a SHA based
on the ref provided in project configuration. Previously, Bolt would
calculate a SHA and write it to the Puppetfile to pin the module to a
specific commit when modules are installed. However, doing this is
difficult and time-consuming with the move to supporting any git-based
module.

!feature

* **Support installing any git-based module**
  ([puppetlabs#3109](puppetlabs#3109))

  Bolt now supports installing any git-based module. Previously, Bolt
  only supported installing and resolving dependencies for modules
  hosted in a public GitHub repository. Now, Bolt will check both GitHub
  and GitLab before falling back to cloning a module's metadata using
  the `git` executable. This allows users to install modules from
  private repositories, or from locations other than GitHub.
This updates documentation for installing git modules to no longer say
only public GitHub repositories are supported.
This fixes a bug where the proxy configured in `module-install.proxy`
was not being set when retrieving metadata for git-based modules.
Previously, the proxy was only configured when resolving and
installing modules.

!bug

* **Configure proxy when retrieving metadata for git-based modules**

  Bolt now configures a proxy when `module-install.proxy` is set and it
  retrieves metadata for git-based modules. Previously, Bolt was only
  configuring a proxy when it resolved and installed modules.
Comment thread lib/bolt/module_installer/specs/id/base.rb
Comment thread lib/bolt/module_installer/specs/git_spec.rb

@donoghuc donoghuc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of minor suggestions, but overall this looks good!

This updates the `GitSpec` class to resolve the SHA for a git module's
ref.
@beechtom beechtom requested a review from donoghuc July 26, 2022 16:14
@beechtom

Copy link
Copy Markdown
Contributor Author

@donoghuc Updated with suggestions.

@beechtom beechtom removed the Do Not Merge Work that should not be merged yet. label Jul 26, 2022

@donoghuc donoghuc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! How do we want to handle bumping the puppetfile resolver dep in the runtime? https://github.com/puppetlabs/puppet-runtime/blob/master/configs/components/rubygem-puppetfile-resolver.rb

I think it may be simplest to bump it in the runtime and then once that promotes merge this.

@beechtom

Copy link
Copy Markdown
Contributor Author

@donoghuc Here's the PR to bump puppetfile-resolver (and other gem components): puppetlabs/puppet-runtime#561

@beechtom beechtom merged commit ada6800 into puppetlabs:main Jul 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bump puppetfile-resolver to 0.6.0

3 participants