Implement first_valid_index for DataFrame & Integrate with Series

[itholic]

Implement DataFrame.first_valid_index

>>> kdf = ks.DataFrame({'a': [None, 2, 3, 2],
...                     'b': [None, 2.0, 3.0, 1.0],
...                     'c': [None, 200, 400, 200]},
...                     index=['Q', 'W', 'E', 'R'])
>>> kdf
     a    b      c
Q  NaN  NaN    NaN
W  2.0  2.0  200.0
E  3.0  3.0  400.0
R  2.0  1.0  200.0

>>> kdf.first_valid_index()
'W'

Support for MultiIndex columns

>>> kdf.columns = pd.MultiIndex.from_tuples([('a', 'x'), ('b', 'y'), ('c', 'z')])
>>> kdf
     a    b      c
     x    y      z
Q  NaN  NaN    NaN
W  2.0  2.0  200.0
E  3.0  3.0  400.0
R  2.0  1.0  200.0

>>> kdf.first_valid_index()
'W'

and integrate with Series.first_valid_index() into generic.py since they can share same implementation.

[softagram-bot]

Softagram Impact Report for pull/1092 (head commit: 1cfae9f)

⚠️ Copy paste found

ℹ️ series.py: Copy paste fragment on line 1250 shared with ../frame.py:

    def to_latex(self, buf=None, columns=None, col_space=None, header=True, index=True,
                 na_rep='NaN',...(truncated 256 chars)

ℹ️ series.py: Copy paste fragment on line 2017 shared with ../frame.py:

                   level: Optional[Union[int, List[int]]] = None, ascending: bool = True,
                   inplace: bool = False, kind: str = None, na_positio...(truncated 56 chars)

ℹ️ series.py: Copy paste fragment inside the same file on lines 3171, 3263:

        results = sdf.select([scol] + index_scols).take(1)
        if len(results) == 0:
            raise ValueError(\"attempt to get...(truncated 376 chars)

ℹ️ series.py: Copy paste fragment inside the same file on lines 4153, 4307:

        sdf = self._internal.sdf \
            .select(cols) \
            .where(reduce(lambda x, y: x & y, rows))

        if len(self._inter...(truncated 255 chars)

ℹ️ series.py: Copy paste fragment inside the same file on lines 3430, 4320:

            internal = _InternalFrame(sdf=sdf, index_map=[(SPARK_INDEX_NAME_FORMAT(0), None)])
            return ...(truncated 148 chars)

ℹ️ generic.py: Copy paste fragment on line 301 shared with ../groupby.py:

            def negative_check(s):
                assert len(s) == 0 or ((s > 0) | (s.isnull())).all(), \
           ...(truncated 188 chars)

ℹ️ frame.py: Copy paste fragment on line 112 shared with ../series.py:

    get_values = unsupported_function('get_values', deprecated=True)
    to_dense = unsupported_function('to_dense', deprecated=True)
 ...(truncated 199 chars)

Now that you are on the file, it would be easier to pay back some tech. debt.

⭐ Change Overview

(Open in Softagram Desktop for full details)

💡 Insights

• Co-change Alert: You modified series.py. Often frame.py (databricks/koalas) is modified at the same time.

📄 Full report

• Permalink: Full report for pull/1092

Impact Report explained. Give feedback on this report to [email protected]

[codecov-io]

Codecov Report

Merging #1092 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master   #1092      +/-   ##
=========================================
+ Coverage    95.2%   95.2%   +<.01%     
=========================================
  Files          34      34              
  Lines        6883    6884       +1     
=========================================
+ Hits         6553    6554       +1     
  Misses        330     330

Impacted Files	Coverage Δ
databricks/koalas/series.py	96.5% <ø> (-0.04%)	⬇️
databricks/koalas/missing/frame.py	100% <ø> (ø)	⬆️
databricks/koalas/generic.py	95.92% <100%> (+0.19%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e7170cb...1cfae9f. Read the comment docs.