Skip to content

Conversation

@ueshin
Copy link
Collaborator

@ueshin ueshin commented May 8, 2020

Fixing GroupBy.filter to follow complex group keys.

>>> kdf = ks.DataFrame({"a": [1, 2, 3, 4, 5, 6], "b": [1, 1, 2, 3, 5, 8], "c": [1, 4, 9, 16, 25, 36]})
>>> kdf.groupby(kdf["b"] // 5).filter(lambda x: any(x.a == 2))
   a  b  c
0  1  1  1
1  2  1  4

This should be:

>>> pdf.groupby(pdf["b"] // 5).filter(lambda x: any(x.a == 2))
   a  b   c
0  1  1   1
1  2  1   4
2  3  2   9
3  4  3  16

@ueshin ueshin requested a review from HyukjinKwon May 8, 2020 20:07
@codecov-io
Copy link

codecov-io commented May 8, 2020

Codecov Report

Merging #1471 into master will decrease coverage by 0.04%.
The diff coverage is 66.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1471      +/-   ##
==========================================
- Coverage   93.74%   93.70%   -0.05%     
==========================================
  Files          36       36              
  Lines        8409     8415       +6     
==========================================
+ Hits         7883     7885       +2     
- Misses        526      530       +4     
Impacted Files Coverage Δ
databricks/koalas/groupby.py 88.40% <66.66%> (-0.23%) ⬇️
databricks/koalas/frame.py 95.38% <0.00%> (-0.15%) ⬇️
databricks/koalas/generic.py 97.02% <0.00%> (+0.37%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e68e03...92a3c5a. Read the comment docs.

Copy link
Member

@HyukjinKwon HyukjinKwon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ueshin
Copy link
Collaborator Author

ueshin commented May 11, 2020

Thanks! I'd merge this now. I'll address the comment in the following PRs.

@ueshin ueshin merged commit af121d0 into databricks:master May 11, 2020
@ueshin ueshin deleted the groupby_filter branch May 11, 2020 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants