MultiIndex.is_monotonic_increasing

[xin-jin]

This does not seem to be documented anywhere. For MultiIndex, how is .is_monotonic_increasing different from .is_lexsorted?

[charlesdong1991]

.is_lexsorted is used for checking sortedness of MI, while is_monotonic_increasing/decreasing for a regular Index

The DOC could be improved and better clarified

[xin-jin]

.is_lexsorted is used for checking sortedness of MI, while is_monotonic_increasing/decreasing for a regular Index

The DOC could be improved and better clarified

Thanks for the reply.

MultiIndex also has the attribute .is_monotonic_increasing. Does it mean anything or just not accurate?

[MarcoGorelli]

Does it mean anything or just not accurate?

Given the bug you uncovered in #32259, at the moment I'd say it's more accurate :)

[MarcoGorelli]

Does it mean anything or just not accurate?

Given the bug you uncovered in #32259, at the moment I'd say it's more accurate :)

Sorry, I think I was wrong. By looking at the docs, it says

Return True if the codes are lexicographically sorted.

Notice how it explicitly says codes rather than values.

So, I think #32259 is actually fine as it is (not a bug), because there the values are sorted. This clarifies the difference here - is_monotonic_increasing/decreasing check that the values are increasing / decreasing, while is_lexsorted just checks the codes.

The DOC could be improved and better clarified

Yes, IMO a note to clarify this difference would be good

[xin-jin]

Does it mean anything or just not accurate?

Given the bug you uncovered in #32259, at the moment I'd say it's more accurate :)

Sorry, I think I was wrong. By looking at the docs, it says

Return True if the codes are lexicographically sorted.

Notice how it explicitly says codes rather than values.

So, I think #32259 is actually fine as it is (not a bug), because there the values are sorted. This clarifies the difference here - is_monotonic_increasing/decreasing check that the values are increasing / decreasing, while is_lexsorted just checks the codes.

The DOC could be improved and better clarified

Yes, IMO a note to clarify this difference would be good

Does that mean MultiIndex.sortlevel will also only be sorting the codes, rather than values? This would be rather confusing and will not be useful for users ...

I am wondering in what cases codes have different order from values? sortlevel is critical to a software I am writing (alternatives like MultiIndex.sort_values() are extremely slow for my cases).

[MarcoGorelli]

Does that mean MultiIndex.sortlevel will also only be sorting the codes, rather than values?

By looking at the source code, that seems to be the case.

This would be rather confusing and will not be useful for users ...

If I've understood is_lexsorted and sortlevel correctly, I agree.

Might want to keep an eye on the thread in #32312

[xin-jin]

I guess potentially in MultiIndex.sortlevel, it can check the MultiIndex.levels[i].is_monotonic_increasing on each of its level, and resort those MultiIndex.levels[i] that are not sorted (as well as resetting its corresponding codes).

[MarcoGorelli]

sortlevel is critical to a software I am writing (alternatives like MultiIndex.sort_values() are extremely slow for my cases).

Would resetting the index (.reset_index) and then sorting the levels with .sort_values and then setting the index back again (.set_index) work as a temporary solution until this is sorted out?

[xin-jin]

sortlevel is critical to a software I am writing (alternatives like MultiIndex.sort_values() are extremely slow for my cases).

Would resetting the index (.reset_index) and then sorting the levels with .sort_values and then setting the index back again (.set_index) work as a temporary solution until this is sorted out?

Thanks. That would be a bit slow for our usage (our typical usage has 4 ~ 5 levels and 10m ~ 100m rows). What I end up doing now is to always check that all MultiIndex.levels[i].is_monotonic_increasing is True before doing sortlevel. It seems that unless the MultiIndex is intentionally set with unsorted levels (like what happens with groupby with sort=False), in all other cases the levels are indeed sorted.