Skip to content

Implement Series.factorize() #1972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Jan 13, 2021
Merged

Implement Series.factorize() #1972

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
a86af14
Prototype
xinrong-meng Dec 16, 2020
4adae2f
Merge branch 'master' into seriesFactorize
xinrong-meng Jan 4, 2021
a4799a7
int32
xinrong-meng Jan 4, 2021
0e2915e
Typo
xinrong-meng Jan 7, 2021
414f86e
Deal with None
xinrong-meng Jan 8, 2021
017b01d
non-na unique_to_code
xinrong-meng Jan 8, 2021
ba9c375
Deal with None and np.nan
xinrong-meng Jan 8, 2021
a011ff4
Codes as pd Series; return type
xinrong-meng Jan 8, 2021
d33d1d3
Alias for new col
xinrong-meng Jan 8, 2021
e1ddc78
Fix type
xinrong-meng Jan 8, 2021
6c89cb9
NaN for FloatType or DoubleType
xinrong-meng Jan 9, 2021
f1d4c3e
Type in doc
xinrong-meng Jan 9, 2021
1e6a727
toPandas w/o limit
xinrong-meng Jan 9, 2021
be2e4df
Map<null, null> adjust
xinrong-meng Jan 11, 2021
fc94110
Branch earlier
xinrong-meng Jan 11, 2021
8f733e9
pandas takes NaN and null to np.nan
xinrong-meng Jan 12, 2021
3e3fac4
Adjust for pd version
xinrong-meng Jan 12, 2021
7c092c8
Comment
xinrong-meng Jan 12, 2021
2e538a7
Compare w respective pd version
xinrong-meng Jan 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions databricks/koalas/tests/test_series.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2364,13 +2364,16 @@ def test_factorize(self):

pcodes, puniques = pser.factorize(sort=True, na_sentinel=-2)
kcodes, kuniques = kser.factorize(na_sentinel=-2)

self.assert_eq(pcodes.tolist(), kcodes.to_list())
self.assert_eq(
[0, 1, 0, -2, -2] if is_lower_pandas_version else pcodes.tolist(), kcodes.to_list()
)
self.assert_eq(puniques, kuniques)

pcodes, puniques = pser.factorize(sort=True, na_sentinel=2)
kcodes, kuniques = kser.factorize(na_sentinel=2)
self.assert_eq(pcodes.tolist(), kcodes.to_list())
self.assert_eq(
[0, 1, 0, 2, 2] if is_lower_pandas_version else pcodes.tolist(), kcodes.to_list()
)
self.assert_eq(puniques, kuniques)

if not is_lower_pandas_version:
Expand Down