Unable to bundle install latest psych 5.0.0 on github actions

[danielwestendorf]

Ensure the following before filing this issue

• I verified it reproduces with the latest version with - uses: ruby/setup-ruby@v1 (see Versioning policy)

• I tried to reproduce the issue locally by following the workflow steps (including all commands done by ruby/setup-ruby, except for Downloading Ruby & Extracting Ruby),
  and it did not reproduce locally (if it does reproduce locally, it's not a ruby/setup-ruby issue)

Are you running on a GitHub-hosted runner or a self-hosted runner?

GitHub-hosted runner

The workflow code or a link to the workflow file

Generated via GH web UI Actions > Ruby > Configure and commenting out the test line.

https://github.com/danielwestendorf/gh-actions-ruby-psych-5/blob/main/.github/workflows/ruby.yml

Link to the log of a failed workflow job, or to a gist with the output

https://github.com/danielwestendorf/gh-actions-ruby-psych-5/actions/runs/3622724366/jobs/6107778167#step:3:44

The command and output of the failing step

bundle install
  /opt/hostedtoolcache/Ruby/2.7.6/x64/bin/bundle config --local path /home/runner/work/gh-actions-ruby-psych-5/gh-actions-ruby-psych-5/vendor/bundle
  /opt/hostedtoolcache/Ruby/2.7.6/x64/bin/bundle lock
  Fetching gem metadata from https://rubygems.org/........
  Resolving dependencies...
  Writing lockfile to /home/runner/work/gh-actions-ruby-psych-5/gh-actions-ruby-psych-5/Gemfile.lock
  Cache key: setup-ruby-bundler-cache-v4-ubuntu-20.04-ruby-2.7.6-Gemfile.lock-32b0ae944f0f9b2d8f3c61f68f6d3cb975db47fe2ae2efbf5bf6fa7b2f6a4dbe
  /opt/hostedtoolcache/Ruby/2.7.6/x64/bin/bundle install --jobs 4
  Fetching gem metadata from https://rubygems.org/........
  Using bundler 2.3.26
  Fetching stringio 3.0.2
  Installing stringio 3.0.2 with native extensions
  Fetching psych 5.0.0
  Installing psych 5.0.0 with native extensions
  Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
  
  current directory:
  /home/runner/work/gh-actions-ruby-psych-5/gh-actions-ruby-psych-5/vendor/bundle/ruby/2.7.0/gems/psych-5.0.0/ext/psych
  /opt/hostedtoolcache/Ruby/2.7.6/x64/bin/ruby -I
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0 -r
  ./siteconf20221205-1835-1muibuj.rb extconf.rb
  checking for yaml.h... no
  yaml.h not found
  *** extconf.rb failed ***
  Could not create Makefile due to some reason, probably lack of necessary
  libraries and/or headers.  Check the mkmf.log file for more details.  You may
  need configuration options.
  
  Provided configuration options:
  	--with-opt-dir
  	--without-opt-dir
  	--with-opt-include
  	--without-opt-include=${opt-dir}/include
  	--with-opt-lib
  	--without-opt-lib=${opt-dir}/lib
  	--with-make-prog
  	--without-make-prog
  	--srcdir=.
  	--curdir
  	--ruby=/opt/hostedtoolcache/Ruby/2.7.6/x64/bin/$(RUBY_BASE_NAME)
  	--with-libyaml-source-dir
  	--without-libyaml-source-dir
  	--with-yaml-0.1-config
  	--without-yaml-0.1-config
  	--with-pkg-config
  	--without-pkg-config
  	--with-libyaml-dir
  	--without-libyaml-dir
  	--with-libyaml-include
  	--without-libyaml-include=${libyaml-dir}/include
  	--with-libyaml-lib
  	--without-libyaml-lib=${libyaml-dir}/lib
  
  To see why this extension failed to compile, please check the mkmf.log which can
  be found here:
  
  /home/runner/work/gh-actions-ruby-psych-5/gh-actions-ruby-psych-5/vendor/bundle/ruby/2.7.0/extensions/x86_64-linux/2.7.0/psych-5.0.0/mkmf.log
  
  extconf failed, exit code 1
  
  Gem files will remain installed in
  /home/runner/work/gh-actions-ruby-psych-5/gh-actions-ruby-psych-5/vendor/bundle/ruby/2.7.0/gems/psych-5.0.0
  for inspection.
  Results logged to
  /home/runner/work/gh-actions-ruby-psych-5/gh-actions-ruby-psych-5/vendor/bundle/ruby/2.7.0/extensions/x86_64-linux/2.7.0/psych-5.0.0/gem_make.out
  
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/ext/builder.rb:99:in
  `run'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/ext/ext_conf_builder.rb:48:in
  `block in build'
    /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/tempfile.rb:291:in `open'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/ext/ext_conf_builder.rb:30:in
  `build'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/ext/builder.rb:169:in
  `block in build_extension'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/ext/builder.rb:165:in
  `synchronize'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/ext/builder.rb:165:in
  `build_extension'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/ext/builder.rb:210:in
  `block in build_extensions'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/ext/builder.rb:207:in
  `each'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/ext/builder.rb:207:in
  `build_extensions'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/2.7.0/rubygems/installer.rb:844:in
  `build_extensions'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/rubygems_gem_installer.rb:72:in
  `build_extensions'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/rubygems_gem_installer.rb:28:in
  `install'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/source/rubygems.rb:207:in
  `install'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/installer/gem_installer.rb:54:in
  `install'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/installer/gem_installer.rb:16:in
  `install_from_spec'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/installer/parallel_installer.rb:186:in
  `do_install'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/installer/parallel_installer.rb:177:in
  `block in worker_pool'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/worker.rb:62:in
  `apply_func'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/worker.rb:57:in
  `block in process_queue'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/worker.rb:54:in
  `loop'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/worker.rb:54:in
  `process_queue'
  /opt/hostedtoolcache/Ruby/2.7.6/x64/lib/ruby/gems/2.7.0/gems/bundler-2.3.26/lib/bundler/worker.rb:91:in
  `block (2 levels) in create_threads'
  
  An error occurred while installing psych (5.0.0), and Bundler cannot continue.
  
  In Gemfile:
    psych
  Took   5.42 seconds
Error: Error: The process '/opt/hostedtoolcache/Ruby/2.7.6/x64/bin/bundle' failed with exit code 5
    at ExecState._setResult (/home/runner/work/_actions/ruby/setup-ruby/0a29871fe2b0200a17a4497bae54fe5df0d973aa/dist/index.js:6023:25)
    at ExecState.CheckComplete (/home/runner/work/_actions/ruby/setup-ruby/0a29871fe2b0200a17a4497bae54fe5df0d973aa/dist/index.js:6006:18)
    at ChildProcess.<anonymous> (/home/runner/work/_actions/ruby/setup-ruby/0a29871fe2b0200a17a4497bae54fe5df0d973aa/dist/index.js:5900:27)
    at ChildProcess.emit (node:events:390:28)
    at maybeClose (node:internal/child_process:1064:16)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)

Any other notes?

No response

[MSP-Greg]

A good place to check are the Actions workflows for the repo, in this case, ruby/psych. It looks like libyaml-dev is not installed?

In general, when building Ruby or any extension gems (like psych), there are often build dependencies. They may not be included with the 'base' Actions image.

[eregon]

As it says:

  checking for yaml.h... no
  yaml.h not found
  *** extconf.rb failed ***
  Could not create Makefile due to some reason, probably lack of necessary
  libraries and/or headers.  Check the mkmf.log file for more details.  You may
  need configuration options.

So you'll need to install libyaml-dev or so, or use --libyaml-source-dir.
cc @hsbt --libyaml-source-dir should be documented in https://github.com/ruby/psych/blob/master/README.md probably. But even then it's not really convenient to get libyaml.

[dentarg]

Shouldn't installing Pysch 5.0.0 on Ruby 2.7.6 just work? Thanks to ruby/psych#550 or am I missing something?

[MSP-Greg]

Psych version 4 has yaml bundled. Psych version 5 does not?

See https://github.com/ruby/psych/tree/4-0-stable/ext/psych/yaml vs https://github.com/ruby/psych/tree/v5.0.0/ext/psych/yaml

[dentarg]

Oh I thought Ruby itself bundled libyaml

[MSP-Greg]

3.1 does, master does not

[dentarg]

but Pysch 5.0.0 can't use it?

[MSP-Greg]

I suppose it could, never tried. Vendoring c source for dependencies has pros and cons, and other default extension gems do not vendor, like Ruby's OpenSSL.

[eregon]

Oh I thought Ruby itself bundled libyaml

It was just psych itself vendoring libyaml.
So this is an expected breaking change for Psych >= 5, that it needs a system libyaml-dev installed (like openssl indeed).

[hsbt]

👋

Can we install libyaml-dev into https://github.com/actions/runner-images by ruby/setup-ruby layer?

I'm not sure how handle the additional packages like libffi, libgmp and libyaml in ruby/setup-ruby.

[MSP-Greg]

Yes. Similar code is in setup-ruby-pkgs, but the packages are user defined. Windows builds already have all packages installed by setup-ruby. By packages, I mean Ruby dependencies, but you knew that.

[MSP-Greg]

Briefly looked at ruby/ruby CI, libncurses5-dev & libyaml-dev are needed with Ubuntu, but not macOS?