Skip to content

Improve tuple/list index-by-long speed #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
heeres opened this issue Jun 5, 2021 · 3 comments
Closed

Improve tuple/list index-by-long speed #57

heeres opened this issue Jun 5, 2021 · 3 comments

Comments

@heeres
Copy link

heeres commented Jun 5, 2021

This patch special-cases indexing of tuples and lists by a PyLong: heeres/cpython@ade7269

Benchmark using:

import timeit
def x():
    a = list(range(16))
    for j in range(10000):
        a[j%16]
print('%.01f ms' % (timeit.timeit(x, number=1000)*1000))

As baseline, the loop with a pass instruction is timed at 120ms.
Without the patch, time is 446ms (456ms) with a defined as a tuple (list)
With the suggested patch, time is about 325ms for both tuple/list. Subtracting the time for the loop with a pass, this means the indexing is performed about 35% faster. Since we're not accounting for the modulo operation in this calculation, my guess would be somewhere between 40-45%.

The culprit is the rather complex PyNumber_AsSsize_t function, which performs a few calls and some reference juggling even in case of a simple PyLong.

If the digit size would be increased, the fastest path could apply to even more cases than it does now.

@gvanrossum
Copy link
Collaborator

Cool, that's probably something we could approach more systematically once we've got Quickening working (#27).

@markshannon
Copy link
Member

Quickening is now working 🙂

If the digit size would be increased, the fastest path could apply to even more cases than it does now.

I think digits are large enough at 30 bits (on 64 bits machines). There aren't many lists or tuples with more than a billion elements.

@markshannon
Copy link
Member

This is a subset of #59

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants