Skip to content

Tweak date/datetime inference, remove dependency on non-core Numpy/Pandas API#802

Merged
texodus merged 10 commits intomasterfrom
numpy-fix
Nov 10, 2019
Merged

Tweak date/datetime inference, remove dependency on non-core Numpy/Pandas API#802
texodus merged 10 commits intomasterfrom
numpy-fix

Conversation

@sc1f
Copy link
Contributor

@sc1f sc1f commented Nov 8, 2019

This PR fixes the inference of dates and datetimes for pandas DataFrames, specifically for DatetimeIndex of various frequencies. Instead of inferring the data type based on the datetime64 unit, we use the index's datetime frequency to decide whether to type the column as a date or a datetime.

Other fixes:

  • remove fillna from our dataframe processing flow
  • remove dependencies on masked_invalid and non top-level numpy/pandas API
  • enforce all column names as strings, and throw an error otherwise
  • add new tests + util.make_dataframe and util.make_series as test utilities

@timkpaine timkpaine added enhancement Feature requests or improvements Python labels Nov 9, 2019
@texodus texodus dismissed timkpaine’s stale review November 10, 2019 03:05

Reviewed offline

@texodus texodus merged commit 3ec9b22 into master Nov 10, 2019
@texodus texodus deleted the numpy-fix branch November 10, 2019 03:05
@texodus texodus added internal Internal refactoring and code quality improvement and removed enhancement Feature requests or improvements labels Jan 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Internal refactoring and code quality improvement Python

Development

Successfully merging this pull request may close these issues.

4 participants