Skip to content

Update dependency: deps/pyk_release #2272

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 54 commits into from
Jan 31, 2024

Conversation

rv-jenkins
Copy link
Contributor

@rv-jenkins rv-jenkins commented Jan 24, 2024

Blocked on: #2276
Blocked on: runtimeverification/pyk#842

@tothtamas28
Copy link
Contributor

tothtamas28 commented Jan 25, 2024

This is a bug: https://github.com/runtimeverification/evm-semantics/actions/runs/7652685768/job/20853079912?pr=2272#step:7:2111

ValueError: Expected identifier, got: 3A0B47780D60687968FB8E6D11D8B1F76D24E73F861EC35B43D2DD8DE0B81935-DEPENDS-MODULE

(Identifiers should start with ['A'..'Z', 'a'..'z'].)

Probably introduced in runtimeverification/pyk#832. I will investigate and prepare the fix.

Edit: Fix: runtimeverification/pyk#840.

@goodlyrottenapple
Copy link
Contributor

Not sure if the failing proof is just flakiness, as it failed for me locally on this branch

FAILED src/tests/integration/test_prove.py::test_pyk_prove[mcd/vat-slip-pass-rough-spec.k]

@ehildenb
Copy link
Member

ehildenb commented Jan 25, 2024

It appears that this test (mcd/vat-slip-pass-rough-spec.k) is timing out, and it is written in such a way that runtimeverification/pyk#832 could affect it. Investigating.

@goodlyrottenapple
Copy link
Contributor

I have finally identified the regression which is causing (most) of the slowdown: runtimeverification/hs-backend-booster#470. The regression is caused by the huge amount of tracing data that the backend starts emitting after this change. The best course of action right now is probably to revert this PR and re-design the tracing to return less data.

A second, smaller, but not insignificant regression was indeed caused by the switch to a threaded runtime in the booster. Whilst not totally unexpected, the regression is still noticeable and I will try to run further tests to see if this slowdown can be somewhat minimised. If tweaks to the runtime options don't produce a good enough result, we could start exporting two binaries in the booster. One threaded and one non-threaded and put the threaded server behind a flag for now.

@goodlyrottenapple
Copy link
Contributor

goodlyrottenapple commented Jan 30, 2024

Update: Looks like the slowdown from the concurrent backend was just caused by running too many threads. Will set the default to 1 OS thread (currently set to the number of CPU cores). Here are the numbers with the above PR revert + 1 thread per rpc-booster:

Test 1-booster-f295a6a99b9cf6e8ebc33f553373fa41c1270f90 time 2-booster-revert-470-georgy-dont-lose-traces time (1-booster-f295a6a99b9cf6e8ebc33f553373fa41c1270f90/2-booster-revert-470-georgy-dont-lose-traces) time
mcd/flipper-addu48u48-fail-rough-spec.k 63.97 68.2 0.9379765395894427
erc20/hkg/balanceOf-spec.k 51.89 50.13 1.0351087173349292
benchmarks/bytes00-spec.k 60.21 58.15 1.0354256233877903
erc20/ds/transfer-failure-2-a-spec.k 81.46 78.67 1.0354645989576712
erc20/hkg/transfer-failure-2-spec.k 68.21 65.86 1.035681749164895
erc20/ds/transfer-failure-2-b-spec.k 57.81 55.75 1.03695067264574
benchmarks/storagevar02-nooverflow-spec.k 63.91 61.51 1.039018045846204
erc20/ds/transfer-failure-1-b-spec.k 114.87 110.55 1.0390773405698779
bihu/forwardToHotWallet-success-1-spec.k 67.89 65.32 1.0393447642375997
mcd/dsvalue-read-pass-summarize-spec.k 66.22 63.59 1.0413587041987733
erc20/ds/balanceOf-spec.k 56.47 54.03 1.045160096242828
erc20/hkg/totalSupply-spec.k 48.95 46.79 1.046163710194486
benchmarks/storagevar00-spec.k 54.01 51.56 1.0475174553917765
benchmarks/keccak00-spec.k 58.05 55.16 1.0523930384336475
mcd/vat-subui-fail-rough-spec.k 98.07 87.12 1.1256887052341595
mcd/vat-move-diff-rough-spec.k 177.18 130.88 1.3537591687041566
mcd/vat-slip-pass-rough-spec.k 268.9 119.4 2.2520938023450583
TOTAL 1458.0700000000002 1222.67 1.1925294642053865

rv-jenkins pushed a commit to runtimeverification/pyk that referenced this pull request Jan 31, 2024
…ut (#842)

We were having trouble in
runtimeverification/evm-semantics#2272 selecting
the correct claims because they can be unqualified by the user. This PR:

- Fully qualifies all claim labels and dependency labels returned by
`KProve.get_claim_modules`.
- Makes sure that we return not only the user-selected claims, but their
dependencies (unless the user turns that off), from `KProve.get_claims`.
- Adds a test-suite for the claim selection functionality.

There is a bug in this code, and a test added (and skipped using
`pytest.skip()`) which demonstrates a (probably minor) issue where we
are not checking that claim dependencies are in a module imported by the
current module.

---------

Co-authored-by: devops <[email protected]>
@rv-jenkins rv-jenkins merged commit 6d6ec8f into master Jan 31, 2024
@rv-jenkins rv-jenkins deleted the _update-deps/runtimeverification/pyk branch January 31, 2024 01:55
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 9, 2024
…ut (#842)

We were having trouble in
runtimeverification/evm-semantics#2272 selecting
the correct claims because they can be unqualified by the user. This PR:

- Fully qualifies all claim labels and dependency labels returned by
`KProve.get_claim_modules`.
- Makes sure that we return not only the user-selected claims, but their
dependencies (unless the user turns that off), from `KProve.get_claims`.
- Adds a test-suite for the claim selection functionality.

There is a bug in this code, and a test added (and skipped using
`pytest.skip()`) which demonstrates a (probably minor) issue where we
are not checking that claim dependencies are in a module imported by the
current module.

---------

Co-authored-by: devops <[email protected]>
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 9, 2024
…ut (runtimeverification/pyk#842)

We were having trouble in
runtimeverification/evm-semantics#2272 selecting
the correct claims because they can be unqualified by the user. This PR:

- Fully qualifies all claim labels and dependency labels returned by
`KProve.get_claim_modules`.
- Makes sure that we return not only the user-selected claims, but their
dependencies (unless the user turns that off), from `KProve.get_claims`.
- Adds a test-suite for the claim selection functionality.

There is a bug in this code, and a test added (and skipped using
`pytest.skip()`) which demonstrates a (probably minor) issue where we
are not checking that claim dependencies are in a module imported by the
current module.

---------

Co-authored-by: devops <[email protected]>
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 9, 2024
…ut (runtimeverification/pyk#842)

We were having trouble in
runtimeverification/evm-semantics#2272 selecting
the correct claims because they can be unqualified by the user. This PR:

- Fully qualifies all claim labels and dependency labels returned by
`KProve.get_claim_modules`.
- Makes sure that we return not only the user-selected claims, but their
dependencies (unless the user turns that off), from `KProve.get_claims`.
- Adds a test-suite for the claim selection functionality.

There is a bug in this code, and a test added (and skipped using
`pytest.skip()`) which demonstrates a (probably minor) issue where we
are not checking that claim dependencies are in a module imported by the
current module.

---------

Co-authored-by: devops <[email protected]>
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 10, 2024
…ut (runtimeverification/pyk#842)

We were having trouble in
runtimeverification/evm-semantics#2272 selecting
the correct claims because they can be unqualified by the user. This PR:

- Fully qualifies all claim labels and dependency labels returned by
`KProve.get_claim_modules`.
- Makes sure that we return not only the user-selected claims, but their
dependencies (unless the user turns that off), from `KProve.get_claims`.
- Adds a test-suite for the claim selection functionality.

There is a bug in this code, and a test added (and skipped using
`pytest.skip()`) which demonstrates a (probably minor) issue where we
are not checking that claim dependencies are in a module imported by the
current module.

---------

Co-authored-by: devops <[email protected]>
Baltoli pushed a commit to runtimeverification/k that referenced this pull request Apr 10, 2024
…ut (runtimeverification/pyk#842)

We were having trouble in
runtimeverification/evm-semantics#2272 selecting
the correct claims because they can be unqualified by the user. This PR:

- Fully qualifies all claim labels and dependency labels returned by
`KProve.get_claim_modules`.
- Makes sure that we return not only the user-selected claims, but their
dependencies (unless the user turns that off), from `KProve.get_claims`.
- Adds a test-suite for the claim selection functionality.

There is a bug in this code, and a test added (and skipped using
`pytest.skip()`) which demonstrates a (probably minor) issue where we
are not checking that claim dependencies are in a module imported by the
current module.

---------

Co-authored-by: devops <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants