[Funding]: Nix team funding for RFC134 and C-API

[tomberek]

(this is currently still in-progress and will be updated/edited)

Team funding

Whereas, there is not a process for funding of teams,
and the Nix team has an identifiable and approved need;
in order to help establish a formal team budgeting process,
this is a request for funding from the NixOS Foundation.

Purpose

• The Nix team is requesting funding to for the specific task of isolating and polishing a store-only Nix (RFC 134).
• Help create a process for formal teams to request funding, both a regular and purpose-driven budget.

Details

isolating and polishing a store-only Nix (RFC 134)

• Split out the Nix store as a stand-alone tool, complete with

  • A separate command-line executable which doesn’t depend on the Nix language implementation proving it is story-only.

  • “functional tests” of that CLI
    These can be in our existing style of simple shell scripts exercising the CLI.

  • documentation
    CLI documentation is already auto-generated, and so rending just a subset of commands/features is rote. But we should improve the prose documentation of store layer concepts, which would be part of the store-layer manual and full Nix manual alike.

• Add a C API to the Nix store
  The goal is to provide a simpler ABI for FFI for bindings in other languages, easing the barrier for high-performance projects built atop the store layer.

(note: The Nix store layer isn’t just for interacting with local-file-system /nix/store, but also includes interacting with the daemon, binary caches, and many associated tasks. Interfaces in the code already exist reflecting the fact that these are the same conceptional tasks implemented many different ways.)

Deliverables

• Split out the Nix store as a stand-alone tool:

  • have a separate derivation for the store (~4w)
    • potentially rewrite the build in Meson (~1w)
  • tests
    • daemon tests (~4days)
    • taking inventory and documenting intent (~2w)
  • command-line interface
    • WIP PR: Allow building a "store-only" nix nix#6225
    • isolate the relevant components for the store CLI (~1w)
    • Need to rework --help (at least in store-only Nix) to not use evaluator (~week)
    • Overlaps with plans to improve doc debug cycle
  • documentation (~2d)

• Add a C API to the Nix store (~2weeks)

  • https://github.com/tweag/nix/tree/nix-c-bindings
  • https://github.com/tweag/nix/blob/nix-c-bindings/src/libstore/c/nix_api_store.h
  • https://github.com/tweag/python-nix

• Common tasks
  Additionally, there are some code quality / “state of good repair” tasks that are not specific to either initiative but heavily relied upon by both:

  • code reviews, testing, refactoring, and extending in-code documentation
  • finish NixOS test setup and port relevant integration tests to that
  • use the MemorySourceAccessor to run in-memory unit tests for the store (~1week)
  • Find some way of writing “interface testing” — to re-use the same tests for multiple store types. GTest has something for this. (~1week)
  • roll out state-of-the-art C++ patterns throughout the codebase with the goal to improve clarity and maintainability

total: 47d

Expected benefits

• Improved codebase quality directly translates to reduced barriers for new contributors - our primary objective as maintainers!
• A better code structure will make solving long-standing user-facing issues and their underlying technical problems tractable, something that we currently don’t have the bandwidth to handle.
• free the maintainers attention to deal more substantial issues such as
  • stabilising the new CLI
  • stabilising functionality currently subsumed under the Flakes experimental feature
  • RFCs ACLs in the store
• Enable developers to conveniently build independent applications that are compatible with the existing Nix ecosystem, and
• Get the powerful idea behind Nix into the hands of even more computer users.

Budget amount

30,000 EUR for 300 hours of work

Conflicts of interest and Notices

• this proposal would be open to anyone to apply (contact the Nix team) with the appropriate C++ and Nix domain expertise and availability.
• https://github.com/tfc has expressed interest and helped develop this statement of work

Process and expectations

• be able to meet with the Nix team at least once a week
• issues and PRs tracked in GitHub
• monthly summary to track progress

References

Proposal repository

[zimbatm]

👍 for getting that kind of process sorted out.

Some points I have in my head:
a. The foundation knows how much it has in its bank account but not how much it can spend and stay sustainable. We need to clarify this before we can start allocating more budget I think.
b. Assuming a fixed budget, how do we put this effort in relationship with other initiatives? If the foundation starts deciding, then it starts making technical choices. Given that this is the first proposal, we might just decide to go ahead, but it would need to be sorted out quickly.

[fricklerhandwerk]

With #103 addressed the team could take care of funding itself, at least to some degree. We'd still need help from e.g. marketing with outreach. Ideally we'd have a combination of base funding by the foundation with direct corporate and community donations.

[fricklerhandwerk]

Related: NixOS/teams-collaboration#2

If we agree on a funding scheme for teams, we should probably have a dedicated budget in the foundation's Open Collective, as well as teams having access to their own projects.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/is-summer-of-nix-worth-the-money/43856/1

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-06-05-nix-team-meeting-minutes-150/46583/1