Add conf-mbedtls.4.0.0

[dbuenzli]

This takes the mbedtls.2 opam file and adds a new package that checks that the mbedtls >= 4.0.0 depext is installed (modelled after the the conf-zstd.1.3.8 package). Note that this is expected to fail mostly everywhere as the release is quite new.

[dbuenzli]

Should I do something special about this conf package ? E.g. for the failures ?

[raphael-proust]

Thanks for the contribution. Your PR highlights an issue we've been having in the opam-repo, one we've discussed before but never reached a satisfying decision about. The issue is:

What should the version number of conf packages mean?

Your PR uses the conf package version as a bound on the underlying package. That is: conf-mbedtls.4.0.0 means mbedtls >= 4.0.0.

The two existing conf-mbedtls packages use the version number as version of the packaging. That is: they indicate that the metadata about the installation process and such has been changed/improved.

An issue we would face merging your package is that for any package p that depend on conf-mbedtls, the action opam install p would trigger an installation attempt of the highest version of conf-mbedtls which might fail for the reason that it's not packaged yet in many distros.

Ideas and suggestions to improve the versioning of conf packages?

Maybe we should have conf-mbedtls which has the list of depexts and just a check the lib is installed. And some separate conf-mbedtls-version which imposes version constraints on the installed library. This is similar to the ocaml-option packages where some constraints on the ocaml installation is delegated to a separate package, allowing greater expressivity in terms of what a package might depend on.

[dbuenzli]

I don't know I just followed what was done for zstd :-)

Ideas and suggestions to improve the versioning of conf packages?

I think that if we want to follow the notion of open bounds in the opam-repo then we could have:

1. conf-$(LIB).$VERSION package is versioned according the version of the software it requires exactly (--exact-version $VERSION).
2. The highest conf-$(LIB).$VERSION package requires that version or higher (--atleast $VERSION).

I think that this lets users specify bounds on the version they need and the depext mecanism could translate this into queries for the external solvers ?

[raphael-proust]

then we could have:

this seems reasonable in terms of semantics for packages, it leaves the following unanswered:

• you might get failures if you try to opam install some package because it depends on conf-foobar without upper-bounds because it's really fine with whatever has been released, but the latest conf-foobar has a version constraint on the underlying foobar which is not packaged for the user's machine (e.g., some old debian). What do we recommend for the users in that case? What's their workflow? Maybe pin a conf-foobar that works?

• what's the plan to transition from the previous model (e.g., the .1 and .2 version of the package you propose a version 4.0.0 for) to this new one?

(I'm puting those questions out but, just to be clear, I am not trying to put the responsability of figuring out the entire solution that opam-repo should adopt onto you @dbuenzli . I put those questions out to be discussed in general, not just by you+me.)

[dbuenzli]

Maybe some of the authors of this paper know better. Hello @avsm, @dra27 etc. :-)

FWIW the current state of affairs is not blocking me but it may result in unpleasant experiences for some users of an upcoming package release (it's a depopts so it may not be too annoying for the opam-repository maintainers)

[dbuenzli]

• what's the plan to transition from the previous model (e.g., the .1 and .2 version of the package you propose a version 4.0.0 for) to this new one?

Ah in this case I suggest to constrain the (single) package, namely haxe, to <= .2. It likely can't work with an mbedtls 4.0.0 because it breaks the API.

[chris-armstrong]

Some things come to mind:

• We may want to try and model the version constraints for an external package using the opam version, but this may get annoying when there is a large number of incompatible versions of an external package, because we'd need several opam package versions to match
• Another way of modelling it is to have a separate set of "version constraint" fields inside a conf package that specifies different actions / different work depending on the external package version, but this could leave opam packages that depend on the conf-<> package struggling to establish what they're compatible with
• External packages all have their own "versioning compatibility systems" i.e. some may be ABI/API compatible through an entire major version, some may break things on minor versions, etc. - it seems like we'd need to enforce upper version ranges on depexts if their versions are modelled with opam versions (directly or indirectly)

[raphael-proust]

So that we can discuss specifics, I have made #28770 as a demonstration of what a separate package for constraints would look like.

With such a system, users who need >4.0.0 should depend both on conf-mbedtls and conf-mbedtls-options-version-at-least >= 4.0.0. Users who don't need specific version constraints can depend on just conf-mbedtls.

I'm worried this could lead to a large number of packages. Although we only ever need to introduce new packages when a specific constraint is needed which doesn't seem to be too often.

[dbuenzli]

@raphael-proust in general I don't think it is a good idea to encode configuration problems in package names and I don't see what problem it solves in this particular case.

In practice people always do depend on a specific range of versions if only implicitly (e.g. the haxe package is unlikely to compile with the new mbedtls you would like to be able to say this to avoid CI failures). Why do all my packages depend on precise software in OCaml but not in its non-OCaml dependencies?

The root problem seems to be that we don't want to have precise versioning of depexts. Which in fact may not be a very good idea and perhaps worth pondering. If people do not want to change that then perhaps this:

1. conf-$(lib).$(TOP) with $(TOP) something that sorts above everything like dev. This just checks for $(lib)'s existence.
2. conf-$(lib).$VERSION. This checks for --at-least $VERSION.

Now if you only care about the package being present (but then as mentioned above, it's likely not the case), you can depend on version $(TOP). If you need above a specific version you can depend on >= $VERSION if you need a version range you can depend >= $MIN & < $MAX.

I still find it odd that we reintepreted the notion of package version to denote ranges in a system that is supposed to excel at dealing with version constraints but that's what it is :-)

[raphael-proust]

if you need a version range you can depend >= $MIN & < $MAX.

I don't think that'd work. Say we merge in conf-mbedtls.5.0.0 with --at-least 5.0.0 and a package really wants to depend on the underlying library being >= 4 & < 5. Well opam will install conf-mbedtls.4.0.0 which will check --at-least 4.0.0 which is true if you have the underlying library at 5+.

With the additional constraint package you could depend on conf-mbedtls-options-at-least >= 4 & conf-options-less-than <= 5.

I don't know that it's a good idea either to introduce this level of complexity (it's a lot of new packages and a whole new semantic for users to learn).

---

Maybe a better alternative is to shift the responsibility for checking version constraints to the non-conf packages? It's not ideal in that it doesn't compose well: you might need to packages that express to incompatible sets of constraints and you can't discover that easily/automatically.

[dbuenzli]

Maybe a better alternative is to shift the responsibility for checking version constraints to the non-conf packages?

But that doesn't solve the problem. What you want is that the opam repository CI does not try to install a package if its conf dependency version cannot be satisfied otherwise you are constantly dealing with failure noise.

In the end I'm not sure that just having what has been done in the zstd (and what is proposed here) is "good enough". That is conf-$(lib).$VERSION always means at least $VERSION. If you don't care about the version then you don't specify one.

[dra27]

My opinion on these is unchanged since #25441 - I think that without actual solver support for the underlying depext version, this doesn’t scale (or even work)

[dbuenzli]

this doesn’t scale (or even work)

Can you perhaps be more specific ?

[dra27]

The issue is in the future when, say, a 5.0.0 package is wanted - because there’s no connection at the solver level for which package to select, it ends up being confusing for any system which only has a 4.x version of the library. Without lots of manual pinning, opam will keep trying to upgrade the conf package, and it then looks like that’s what’s at fault, rather than needing to hold packages using that conf package back to older versions (I think this started to happen with labltk?). Having real versions on the conf packages makes it look like one can put constraints on the dependency, but that’s not the reality, and I think that’s more confusing than just having to abort during a specific package’s configuration (but I don’t mean to imply that’s ideal either!)

I think it’s something which needs solving on the opam side first rather than the opam side (that’s part of what the hyperres paper was about)

[dbuenzli]

Thanks for the details. I see now. So basically the tradeoff is between being annoying to end-users and being annoying to the opam repo maintainers/submitters.

I think it’s something which needs solving on the opam side first rather than the opam side (that’s part of what the hyperres paper was about)

rather than opam repo side I guess.

Ok so let's close this.