CLN: Preliminary formatter refactor #20051

shangyian · 2018-03-08T10:27:19Z

based on discussion here: Latex bugs #20032 (review)
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

This PR splits the various formatters into different files so that it's easier to refactor a specific formatter.

jreback

looks good. just some very minor comments. want to do this refactor w/o really changing much code (as its more transparent that way). next pass will clean / fix more things.

jreback · 2018-03-08T11:30:00Z

pandas/io/formats/common.py

+
+def buffer_put_lines(buf, lines):
+    """
+    Appends lines to a buffer


can you use numpy doc string format here

jreback · 2018-03-08T11:31:16Z

pandas/io/formats/latex.py

+    longtable : boolean, default False
+        Use a longtable environment instead of tabular.
+
+    See also


jreback · 2018-03-08T11:32:13Z

pandas/io/formats/html.py

+                           OrderedDict, unichr)
+from pandas.core.config import get_option
+from pandas.io.formats.printing import pprint_thing
+from pandas.io.formats.common import get_level_lengths, \


use parens rather than line continuation \

jreback · 2018-03-08T11:32:48Z

pandas/io/formats/csv.py

+from pandas.core.indexes.datetimes import DatetimeIndex
+from pandas.core.indexes.period import PeriodIndex
+import numpy as np
+


put these import first (csv then numpy)

Cool, just fixed these

codecov · 2018-03-08T12:49:47Z

Codecov Report

Merging #20051 into master will increase coverage by <.01%.
The diff coverage is 93.75%.

@@            Coverage Diff             @@
##           master   #20051      +/-   ##
==========================================
+ Coverage   91.72%   91.73%   +<.01%     
==========================================
  Files         150      152       +2     
  Lines       49122    49180      +58     
==========================================
+ Hits        45057    45114      +57     
- Misses       4065     4066       +1

Flag	Coverage Δ
#multiple	`90.11% <93.75%> (ø)`	⬆️
#single	`41.79% <18.59%> (-0.07%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/formats/latex.py	`100% <100%> (ø)`
pandas/io/formats/excel.py	`97.36% <100%> (ø)`	⬆️
pandas/core/frame.py	`97.18% <100%> (ø)`	⬆️
pandas/io/formats/html.py	`88.85% <88.85%> (ø)`
pandas/io/formats/format.py	`98.24% <96.15%> (+1.98%)`	⬆️
pandas/io/formats/csvs.py	`98.05% <98.05%> (ø)`
pandas/core/base.py	`96.78% <0%> (-0.02%)`	⬇️
pandas/core/indexes/datetimes.py	`95.64% <0%> (-0.01%)`	⬇️
pandas/core/series.py	`93.85% <0%> (-0.01%)`	⬇️
pandas/core/groupby.py	`92.14% <0%> (-0.01%)`	⬇️
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c14e4f...c9b4504. Read the comment docs.

shangyian · 2018-03-09T12:04:09Z

Hmm. Not sure why Travis failed and AppVeyor succeeded.

jreback · 2018-03-10T01:25:42Z

you have a lint error

inside ci/lint.sh
Linting *.py
pandas/io/formats/format.py:33:1: F401 'pandas.io.formats.csv_format.CSVFormatter' imported but unused
Linting *.py DONE

jreback · 2018-03-10T01:37:57Z

ok I fixed up the imports a bit to make them more idiomatic. I think its possible to completely remove pandas.io.formats.common and just move these 2 functions back to .formats. No real reason to have an extra file here. see if you can do that.

jreback · 2018-03-10T12:38:17Z

thanks @shangyian

will merge a bit later (after doc sprint)

shangyian · 2018-03-10T14:44:54Z

@jreback - cool. I also moved the two functions in io.formats.common to formats.format

jreback

great. there is also a _get_level_lengths in style.py, you might be able to combine these (can do here or next PR)

jreback · 2018-03-10T15:16:06Z

pandas/core/frame.py

@@ -1841,7 +1842,7 @@ def info(self, verbose=None, buf=None, max_cols=None, memory_usage=None,
            - If False, never show counts.

        """
-        from pandas.io.formats.format import _put_lines
+        from pandas.io.formats.format import buffer_put_lines


you can reference this directly as fmt.buffer_put_lines as fmt is already imported

jreback · 2018-03-10T15:16:14Z

pandas/core/frame.py

@@ -1940,7 +1941,7 @@ def _sizeof_fmt(num, size_qualifier):
            mem_usage = self.memory_usage(index=True, deep=deep).sum()
            lines.append("memory usage: %s\n" %
                         _sizeof_fmt(mem_usage, size_qualifier))
-        _put_lines(buf, lines)
+        buffer_put_lines(buf, lines)


Good point, changed those. I'll take a look at the two get_level_lengths, but may not get around to fixing that today, so it's probably better as a new PR

jreback · 2018-03-13T23:24:33Z

can you rebase on master. ping on green.

shangyian mentioned this pull request Mar 8, 2018

Latex bugs #20032

Closed

4 tasks

jreback added Output-Formatting __repr__ of pandas objects, to_string Clean labels Mar 8, 2018

jreback requested changes Mar 8, 2018

View reviewed changes

jreback added IO CSV read_csv, to_csv IO HTML read_html, to_html, Styler.apply, Styler.applymap IO LaTeX to_latex labels Mar 8, 2018

shangyian closed this Mar 8, 2018

shangyian reopened this Mar 8, 2018

shangyian closed this Mar 9, 2018

shangyian reopened this Mar 9, 2018

jreback changed the title ~~Preliminary formatter refactor~~ CLN: Preliminary formatter refactor Mar 10, 2018

jreback added this to the 0.23.0 milestone Mar 10, 2018

jreback approved these changes Mar 10, 2018

View reviewed changes

jreback requested changes Mar 10, 2018

View reviewed changes

shangyian closed this Mar 14, 2018

shangyian force-pushed the prelim_format_refactor branch from c9b4504 to 3783ccc Compare March 14, 2018 08:32

jorisvandenbossche merged commit 3783ccc into pandas-dev:master Mar 14, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLN: Preliminary formatter refactor #20051

CLN: Preliminary formatter refactor #20051

shangyian commented Mar 8, 2018

jreback left a comment

jreback Mar 8, 2018

jreback Mar 8, 2018

jreback Mar 8, 2018

jreback Mar 8, 2018

shangyian Mar 8, 2018

codecov bot commented Mar 8, 2018 •

edited

Loading

shangyian commented Mar 9, 2018

jreback commented Mar 10, 2018

jreback commented Mar 10, 2018

jreback commented Mar 10, 2018

shangyian commented Mar 10, 2018

jreback left a comment

jreback Mar 10, 2018

jreback Mar 10, 2018

shangyian Mar 10, 2018

jreback commented Mar 13, 2018

CLN: Preliminary formatter refactor #20051

CLN: Preliminary formatter refactor #20051

Conversation

shangyian commented Mar 8, 2018

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Mar 8, 2018 • edited Loading

Codecov Report

shangyian commented Mar 9, 2018

jreback commented Mar 10, 2018

jreback commented Mar 10, 2018

jreback commented Mar 10, 2018

shangyian commented Mar 10, 2018

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Mar 13, 2018

codecov bot commented Mar 8, 2018 •

edited

Loading