Tool cache path on self-hosted runners can't be hardcoded

[Vyazovoy]

Hi! I use self-hosted runners and I have several instances on the same physical machine. I see that this action uses hardcoded path to tool cache but there is no guarantee that even when people have only one self-hosted runner that it will be /Users/runner.... Would it be possible to avoid all path hardcoding?

[MSP-Greg]

I believe the code used is below. The hardcoded values match ENV['RUNNER_TOOL_CACHE']. Would that work in your situation?

See most recent 'Image Info' run in https://github.com/MSP-Greg/actions-image-testing/actions. Currently is https://github.com/MSP-Greg/actions-image-testing/actions/runs/1492346296, see 'Show ENV' step.

setup-ruby/common.js

Lines 126 to 137 in 471466b

	function getPlatformToolCache(platform) {
	// Hardcode paths rather than using $RUNNER_TOOL_CACHE because the prebuilt Rubies cannot be moved anyway
	if (platform.startsWith('ubuntu-')) {
	return '/opt/hostedtoolcache'
	} else if (platform.startsWith('macos-')) {
	return '/Users/runner/hostedtoolcache'
	} else if (platform.startsWith('windows-')) {
	return 'C:/hostedtoolcache/windows'
	} else {
	throw new Error('Unknown platform')
	}
	}

[MSP-Greg]

@Vyazovoy

This is problematic with self-hosted runners. Two cases:

1. You're using a Ruby that is contained in your runner's toolcache. I believe using ENV['RUNNER_TOOL_CACHE'] would work.

2. You want to download and install one of the pre-built Rubies (ie, not one in your toolcache). I believe the path is hardcoded for all Rubies except the Windows builds. Not sure about Window non-MRI builds.

So, I suppose the code could check for a match in the toolcache at ENV['RUNNER_TOOL_CACHE'], and if a match is not found, install to the hardcoded path...

[Vyazovoy]

Thanks for the explanation, @MSP-Greg 🙏 Here the code checks for a match in the toolcache so I should probably populate the cache manually to make this action working but it would be good to have a configurable parameter that would allow to force installation with ruby-build instead of downloading the prebuilt Rubies. By using that parameter and non-hardcoded toolcache path the action would install a Ruby version with ruby-build when called first time on a self-hosted runner and all subsequent runs would use the cached version. Also, if it is possible to detect that a runner is self-hosted, that would be good to apply the behavior automatically.

[MSP-Greg]

several instances on the same physical machine.

That certainly seems possible. Question - do they run concurrently/at the same time? If not, could the default toolcache path be a symlink?

Not.enough.coffee.yet...

[Vyazovoy]

It is possible (and desirable) that several instances of self-hosted runner work at the same time. As I can see, each instance has its toolcache at /path_to_an_instance/_work/_tool. I could symlink that _tool to some shared dir but even if the action will build every (needed) Ruby version for every runner instance, it would be totally acceptable as it would happen only several times in a year.

[eregon]

Short answer is it's unsupported, almost all built Rubies only work if extracted in those paths, because they are built in those paths and can't be moved.

If you already built a populated toolcache, then there is probably little point to use this action, except for the bundler cache.

but it would be good to have a configurable parameter that would allow to force installation with ruby-build instead of downloading the prebuilt Rubies.

I don't want to add such complexity in this action, but you are free to make your own action / script / look at https://github.com/ruby/ruby-builder

By using that parameter and non-hardcoded toolcache path the action would install a Ruby version with ruby-build when called first time on a self-hosted runner and all subsequent runs would use the cached version.

If two jobs run on same machine it would need inter-process locking and that's typically tricky to use, i.e., significant additional complexity that's not for this action.

---

I think being able to use an already-built ruby prefix by specifying an absolute path, or to use the ruby currently in PATH could be an addition which still allows to use bundler caching for such cases with custom toolcaches.
The general solution to split this action into two "setup Ruby" and "bundler cache" seems like it would also address this.

If you have a custom toolcache, it's on your own to build ruby in it.

[Vyazovoy]

Hi, @eregon! Thanks for your detailed response 🙏 I understand your position but would like to clarify several things in hope that maybe it could help us find a good solution instead of just closing the issue 😊

If you already built a populated toolcache, then there is probably little point to use this action, except for the bundler cache.

The main point for me to use this action is to be able to use both self-hosted and GitHub-hosted runners without having separate workflows/steps (consistency) + the bundler caching.

If two jobs run on same machine it would need inter-process locking and that's typically tricky to use, i.e., significant additional complexity that's not for this action.

As I mentioned, there are separate toolcaches for each runner instance so I don't see where would we need inter-process locking as each runner instance able to run one job at a time.

The general solution to split this action into two "setup Ruby" and "bundler cache" seems like it would also address this.

Sure, that is what I have now but I would avoid that for the reasons I mentioned earlier (consistency and simplicity).

If you have a custom toolcache, it's on your own to build ruby in it.

I don't have anything custom in my config. I use self-hosted runners that provided by GitHub and I configured them with default values. And GitHub never mentioned that I can't have several instances of it or some other restrictions.

[eregon]

As I mentioned, there are separate toolcaches for each runner instance so I don't see where would we need inter-process locking as each runner instance able to run one job at a time.

I missed that, then that locking does not apply, but I still wouldn't want to call ruby-build from setup-ruby (e.g., it makes a job's time unpredictable, what if the job is cancelled in the middle, etc).

If you use a single toolcache for all runners then you could place them in the same location as GH-hosted runners and it'd work.
But you would need to make sure software installed there is done beforehand (seems best anyway when it comes to populating a toolcache), or at least when no concurrent jobs are running.

The general solution to split this action into two "setup Ruby" and "bundler cache" seems like it would also address this.

Sure, that is what I have now but I would avoid that for the reasons I mentioned earlier (consistency and simplicity).

I meant a new action for "bundler cache" not just actions/cache which is difficult to use correctly.

I don't have anything custom in my config. I use self-hosted runners that provided by GitHub and I configured them with default values. And GitHub never mentioned that I can't have several instances of it or some other restrictions.

The GitHub docs don't mention it, but as we see here and I guess in other actions there is the expectation that the toolcache is in the same path, otherwise things might break.

[t0rr3sp3dr0]

@eregon, isn't it possible to replicate https://github.com/Homebrew/homebrew-portable-ruby and build Ruby in a way it can be moved around the filesystem? Homebrew successfully does it for Linux (amd64) and macOS (amd64 and arm64). It doesn't seem to be too complicated.

[eregon]

I tried a while ago and it's badly supported and breaks several things, even more on older Rubies: #98 (comment)
And AFAIK CRuby doesn't test --enable-load-relative at all so I consider it untested and unsupported (it does too much hacks which breaks too many things including gem install).
That's probably one reason why "portable rubies" projects appear and disappear every few years, I'd think it's too much work and hacks to maintain it long-term.

I'll close this issue.

[eregon]

With #473 this action now gives instructions for how to make it work for self-hosted runners not matching a GitHub-hosted runner image.

So it's kind of close to #242 (comment) (reading it again that's a well-thought comment) except you would need to run ruby-build manually, because it's not that trivial (e.g., need to find which OS packages are needed per OS and since it's self-hosted there is no idea of what is already available, etc) and also if the action did it automatically people would likely be tempted to file an issue here if ruby-build fails, while it would be a problem of their self-hosted runner, not of this action.

it would be good to have a configurable parameter ... Also, if it is possible to detect that a runner is self-hosted, that would be good to apply the behavior automatically.

There doesn't seem to be a reliable way to detect it, and for self-hosted runners that happen to match a GH-hosted runner image it seems better to reuse precompiled binaries. So what I did is if it's not one of the well-known GH-hosted runner images OS & OS version then it's considered self-hosted.

Probably we should also consider self-hosted if the RUNNER_TOOL_CACHE is not the default one, I would consider that the proper solution for this issue.

And the configurable parameter is a good idea since there is no way to reliably detect it.

[eregon]

Done now.

[eregon]

I'll need to revert the detection of different RUNNER_TOOL_CACHE as self-hosted, to fix #475.
So for those cases you would need to set self-hosted: true explicitly.