Skip to content

Conversation

@YorikSar
Copy link
Contributor

@YorikSar YorikSar commented Aug 4, 2025

As in #430132, it saves a lot of set allocations and merge operations, but makes code harder to read.

I've tried introducing a function like this to make code cleaner, but it increases number of envs and space taken by them significantly:

  optionalAttr = cond: name: if cond then name else null;

I've also tried applying this logic to the section with isDarwin, but it makes things measurably worse for x86_64-linux eval.

Here are significant metrics gathered for x86_64-linux eval on my machines:

metric mean_before mean_after mean_diff mean_%_change p_value t_stat
cpuTime 38.1802 37.8518 -0.3284 -1.2641 0.5332 -0.6454
envs.bytes 893432387.6364 866216152.7273 -27216234.9091 -3.2010 0.0001 -6.4520
envs.elements 65051632.1818 63350762.0909 -1700870.0909 -2.7357 0.0001 -6.4520
envs.number 46627416.2727 44926257.0000 -1701159.2727 -3.8583 0.0001 -6.4519
gc.cycles 10.9091 10.7273 -0.1818 -1.2554 0.1669 -1.4907
gc.heapSize 3793414330.1818 3808570647.2727 15156317.0909 0.3018 0.7473 0.3313
gc.totalBytes 6934161339.6364 6792827748.3636 -141333591.2727 -2.1639 0.0001 -6.5707
nrFunctionCalls 42352816.8182 40651657.5455 -1701159.2727 -4.2193 0.0001 -6.4519
nrOpUpdateValuesCopied 112653866.8182 108331752.7273 -4322114.0909 -4.0958 0.0001 -6.5699
nrOpUpdates 5010432.6364 4033244.9091 -977187.7273 -19.3867 0.0001 -6.4689
nrThunks 56828714.1818 55127844.0909 -1700870.0909 -3.1665 0.0001 -6.4520
sets.bytes 2586801697.4545 2535201988.3636 -51599709.0909 -2.1743 0.0001 -6.5502
sets.elements 151548136.5455 148559516.3636 -2988620.1818 -2.1445 0.0001 -6.5267
sets.number 10126969.5455 9890607.9091 -236361.6364 -2.6579 0.0000 -6.8169
time.cpu 38.1802 37.8518 -0.3284 -1.2641 0.5332 -0.6454
time.gc 8.1961 7.7776 -0.4185 -2.8841 0.4146 -0.8511
time.gcFraction 0.2630 0.2505 -0.0125 -2.2664 0.2787 -1.1456
values.bytes 1919952840.0000 1879131957.8182 -40820882.1818 -2.3468 0.0001 -6.4520
values.number 79998035.0000 78297164.9091 -1700870.0909 -2.3468 0.0001 -6.4520

Things done


Add a 👍 reaction to pull requests you find important.

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. 6.topic: stdenv Standard environment labels Aug 4, 2025
@YorikSar YorikSar force-pushed the remove-attrs-merges-from-mkderivation branch from 1aa44ad to 571ec7c Compare August 4, 2025 15:44
Copy link
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damn, those are some really impressive savings without any loss of functionality! 🚀

@nixpkgs-ci nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Aug 6, 2025
Copy link
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future: This function itself needs to be on the chopping block -- spot its use down below, where it can be inlined into the two uses.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think inlining it would yield much benefit - it’s called from three separate places with different arguments and there is no repetition in them. But doesn’t hurt to try :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition would definitely be more clear if it were extracted to a variable in the let block, ideally near the definition of hardeningDisable', defaultHardeningFlags, etc.

Unrelated to this PR, but hardeningDisable' has a unique which looks performance intensive and would be great to remove if it can be done without incident. I suspect the order of this list doesn't matter.

@philiptaron
Copy link
Contributor

@YorikSar, I'm very OK merging this as-is then doing updates in some future PR. There's a good deal more micro-optimizations to be done, I think.

@nixpkgs-ci nixpkgs-ci bot added 12.approvals: 2 This PR was reviewed and approved by two persons. and removed 12.approvals: 1 This PR was reviewed and approved by one person. labels Aug 6, 2025
Copy link
Member

@infinisil infinisil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving definitions to a let block is better for a separate PR, so we can independently evaluate the performance difference

As in NixOS#430132, it saves a lot of
set allocations and merge operations, but makes code harder to read.

I've tried introducing a function like this to make code cleaner, but it
increases number of envs and space taken by them significantly:

  optionalAttr = cond: name: if cond then name else null;

I've also tried applying this logic to the section with isDarwin, but
it makes things measurably worse for x86_64-linux eval.
@YorikSar YorikSar force-pushed the remove-attrs-merges-from-mkderivation branch from 571ec7c to 1d4489d Compare August 7, 2025 11:40
Copy link
Contributor Author

@YorikSar YorikSar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've addressed the comment about the contend addressed comment. I want to address the rest of them in a separate PR.

@infinisil infinisil enabled auto-merge August 7, 2025 11:41
@infinisil infinisil merged commit d3256e5 into NixOS:master Aug 7, 2025
24 of 27 checks passed
@infinisil infinisil deleted the remove-attrs-merges-from-mkderivation branch August 7, 2025 11:45
@github-project-automation github-project-automation bot moved this to Done in Stdenv Aug 7, 2025
Copy link
Contributor Author

@YorikSar YorikSar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ve opened another PR optimising this part at #431756, please take a look.
@philiptaron I addressed a couple of your comments there, although I hoped to cover more of them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think inlining it would yield much benefit - it’s called from three separate places with different arguments and there is no repetition in them. But doesn’t hurt to try :)

@philiptaron
Copy link
Contributor

Thanks! I'm really excited by your work here. I deeply appreciate it, and its ability to make the experience for literally every single package in Nixpkgs better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: stdenv Standard environment 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. 12.approvals: 2 This PR was reviewed and approved by two persons.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants