Merge changes from master

Author: wesm

RELEASE.rst

@@ -22,6 +22,71 @@ Where to get it
 * Binary installers on PyPI: http://pypi.python.org/pypi/pandas
 * Documentation: http://pandas.pydata.org
 
+pandas 0.7.3
+============
+
+**Release date:** April 12, 2012
+
+**New features / modules**
+
+  - Added fixed-width file reader, read_fwf (PR #952)
+  - Add group_keys argument to groupby to not add group names to MultiIndex in
+    result of apply (GH #938)
+  - DataFrame can now accept non-integer label slicing (GH #946). Previously
+    only DataFrame.ix was able to do so.
+  - DataFrame.apply now retains name attributes on Series objects (GH #983)
+  - Numeric DataFrame comparisons with non-numeric values now raises proper
+    TypeError (GH #943). Previously raise "PandasError: DataFrame constructor
+    not properly called!"
+  - Add ``kurt`` methods to Series and DataFrame (PR #964)
+  - Can pass dict of column -> list/set NA values for text parsers (GH #754)
+  - Allows users specified NA values in text parsers (GH #754)
+  - Parsers checks for openpyxl dependency and raises ImportError if not found
+    (PR #1007)
+  - New factory function to create HDFStore objects that can be used in a with
+    statement so users do not have to explicitly call HDFStore.close (PR #1005)
+  - pivot_table is now more flexible with same parameters as groupby (GH #941)
+  - Added stacked bar plots (GH #987)
+  - scatter_matrix method in pandas/tools/plotting.py (PR #935)
+  - DataFrame.boxplot returns plot results for ex-post styling (GH #985)
+  - Short version number accessible as pandas.version.short_version (GH #930)
+  - Additional documentation in panel.to_frame (GH #942)
+  - More informative Series.apply docstring regarding element-wise apply
+    (GH #977)
+  - Notes on rpy2 installation (GH #1006)
+  - Add rotation and font size options to hist method (#1012)
+  - Use exogenous / X variable index in result of OLS.y_predict. Add
+    OLS.predict method (PR #1027, #1008)
+
+**API Changes**
+
+  - Calling apply on grouped Series, e.g. describe(), will no longer yield
+    DataFrame by default. Will have to call unstack() to get prior behavior
+  - NA handling in non-numeric comparisons has been tightened up (#933, #953)
+
+**Bug fixes**
+
+  - Fix logic error when selecting part of a row in a DataFrame with a
+    MultiIndex index (GH #1013)
+  - Series comparison with Series of differing length causes crash (GH #1016).
+  - Fix bug in indexing when selecting section of hierarchically-indexed row
+    (GH #1013)
+  - DataFrame.plot(logy=True) has no effect (GH #1011).
+  - Broken arithmetic operations between SparsePanel-Panel (GH #1015)
+  - Unicode repr issues in MultiIndex with non-ascii characters (GH #1010)
+  - DataFrame.lookup() returns inconsistent results if exact match not present
+    (GH #1001)
+  - DataFrame arithmetic operations not treating None as NA (GH #992)
+  - DataFrameGroupBy.apply returns incorrect result (GH #991)
+  - Series.reshape returns incorrect result for multiple dimensions (GH #989)
+  - Series.std and Series.var ignores ddof parameter (GH #934)
+  - DataFrame.append loses index names (GH #980)
+  - DataFrame.plot(kind='bar') ignores color argument (GH #958)
+  - Inconsistent Index comparison results (GH #948)
+  - Improper int dtype DataFrame construction from data with NaN (GH #846)
+  - Removes default 'result' name in grouby results (GH #995)
+  - DataFrame.from_records no longer mutate input columns (PR #975)
+
 pandas 0.7.2
 ============
 

doc/source/dsintro.rst

@@ -687,7 +687,20 @@ For example, compare to the construction above:
 
    Panel.from_dict(data, orient='minor')
 
-Orient is especially useful for mixed-type DataFrames.
+Orient is especially useful for mixed-type DataFrames. If you pass a dict of
+DataFrame objects with mixed-type columns, all of the data will get upcasted to
+``dtype=object`` unless you pass ``orient='minor'``:
+
+.. ipython:: python
+
+   df = DataFrame({'a': ['foo', 'bar', 'baz'],
+                   'b': np.random.randn(3)})
+   df
+   data = {'item1': df, 'item2': df}
+   panel = Panel.from_dict(data, orient='minor')
+   panel['a']
+   panel['b']
+   panel['b'].dtypes
 
 .. note::
 
@@ -747,3 +760,18 @@ For example, using the earlier example data, we could do:
     wp.major_xs(wp.major_axis[2])
     wp.minor_axis
     wp.minor_xs('C')
+
+Conversion to DataFrame
+~~~~~~~~~~~~~~~~~~~~~~~
+
+A Panel can be represented in 2D form as a hierarchically indexed
+DataFrame. See the section :ref:`hierarchical indexing <indexing.hierarchical>`
+for more on this. To convert a Panel to a DataFrame, use the ``to_frame``
+method:
+
+.. ipython:: python
+
+   panel = Panel(np.random.randn(3, 5, 4), items=['one', 'two', 'three'],
+                 major_axis=DateRange('1/1/2000', periods=5),
+                 minor_axis=['a', 'b', 'c', 'd'])
+   panel.to_frame()

doc/source/io.rst

@@ -94,7 +94,7 @@ data into a DataFrame object. They can take a number of arguments:
   - ``converters``: a dictionary of functions for converting values in certain
     columns, where keys are either integers or column labels
   - ``encoding``: a string representing the encoding to use if the contents are
-    non-ascii, for python versions prior to 3 
+    non-ascii
   - ``verbose`` : show number of NA values inserted in non-numeric columns
 
 .. ipython:: python
@@ -139,6 +139,67 @@ fragile. Type inference is a pretty big deal. So if a column can be coerced to
 integer dtype without altering the contents, it will do so. Any non-numeric
 columns will come through as object dtype as with the rest of pandas objects.
 
+.. _io.fwf:
+
+Files with Fixed Width Columns
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+While `read_csv` reads delimited data, the :func:`~pandas.io.parsers.read_fwf`
+function works with data files that have known and fixed column widths.
+The function parameters to `read_fwf` are largely the same as `read_csv` with
+two extra parameters:
+
+  - ``colspecs``: a list of pairs (tuples), giving the extents of the
+    fixed-width fields of each line as half-open intervals [from, to[
+  - ``widths``: a list of field widths, which can be used instead of
+    ``colspecs`` if the intervals are contiguous
+
+.. ipython:: python
+   :suppress:
+
+   f = open('bar.csv', 'w')
+   data1 = ("id8141    360.242940   149.910199   11950.7\n"
+            "id1594    444.953632   166.985655   11788.4\n"
+            "id1849    364.136849   183.628767   11806.2\n"
+            "id1230    413.836124   184.375703   11916.8\n"
+            "id1948    502.953953   173.237159   12468.3")
+   f.write(data1)
+   f.close()
+
+Consider a typical fixed-width data file:
+
+.. ipython:: python
+
+   print open('bar.csv').read()
+
+In order to parse this file into a DataFrame, we simply need to supply the
+column specifications to the `read_fwf` function along with the file name:
+
+.. ipython:: python
+
+   #Column specifications are a list of half-intervals
+   colspecs = [(0, 6), (8, 20), (21, 33), (34, 43)]
+   df = read_fwf('bar.csv', colspecs=colspecs, header=None, index_col=0)
+   df
+
+Note how the parser automatically picks column names X.<column number> when
+``header=None`` argument is specified. Alternatively, you can supply just the
+column widths for contiguous columns:
+
+.. ipython:: python
+
+   #Widths are a list of integers
+   widths = [6, 14, 13, 10]
+   df = read_fwf('bar.csv', widths=widths, header=None)
+   df
+
+The parser will take care of extra white spaces around the columns
+so it's ok to have extra separation between the columns in the file.
+
+.. ipython:: python
+   :suppress:
+
+   os.remove('bar.csv')
+
 Files with an "implicit" index column
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -281,7 +342,7 @@ function takes a number of arguments. Only the first is required.
   - ``mode`` : Python write mode, default 'w'
   - ``sep`` : Field delimiter for the output file (default "'")
   - ``encoding``: a string representing the encoding to use if the contents are
-    non-ascii, for python versions prior to 3 
+    non-ascii, for python versions prior to 3
 
 Writing a formatted string
 ~~~~~~~~~~~~~~~~~~~~~~~~~~

doc/source/r_interface.rst

@@ -15,8 +15,10 @@ rpy2 / R interface
 If your computer has R and rpy2 (> 2.2) installed (which will be left to the
 reader), you will be able to leverage the below functionality. On Windows,
 doing this is quite an ordeal at the moment, but users on Unix-like systems
-should find it quite easy. As a general rule, I would recommend using the
-latest revision of rpy2 from bitbucket:
+should find it quite easy. rpy2 evolves in time and the current interface is
+designed for the 2.2.x series, and we recommend to use over other series 
+unless you are prepared to fix parts of the code. Released packages are available
+in PyPi, but should the latest code in the 2.2.x series be wanted it can be obtained with:
 
 ::
 
@@ -25,7 +27,7 @@ latest revision of rpy2 from bitbucket:
 
     cd rpy2
     hg pull
-    hg update
+    hg update version_2.2.x
     sudo python setup.py install
 
 .. note::

doc/source/visualization.rst

@@ -112,8 +112,8 @@ Other plotting features
 
 .. _visualization.barplot:
 
-Plotting non-time series data
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Bar plots
+~~~~~~~~~
 
 For labeled, non-time series data, you may wish to produce a bar plot:
 
@@ -124,8 +124,47 @@ For labeled, non-time series data, you may wish to produce a bar plot:
    @savefig bar_plot_ex.png width=4.5in
    df.ix[5].plot(kind='bar'); plt.axhline(0, color='k')
 
-Histogramming
-~~~~~~~~~~~~~
+Calling a DataFrame's ``plot`` method with ``kind='bar'`` produces a multiple
+bar plot:
+
+.. ipython:: python
+   :suppress:
+
+   plt.figure();
+
+.. ipython:: python
+
+   df2 = DataFrame(np.random.rand(10, 4), columns=['a', 'b', 'c', 'd'])
+
+   @savefig bar_plot_multi_ex.png width=5in
+   df2.plot(kind='bar');
+
+To produce a stacked bar plot, pass ``stacked=True``:
+
+.. ipython:: python
+   :suppress:
+
+   plt.figure();
+
+.. ipython:: python
+
+   @savefig bar_plot_stacked_ex.png width=5in
+   df2.plot(kind='bar', stacked=True);
+
+To get horizontal bar plots, pass ``kind='barh'``:
+
+.. ipython:: python
+   :suppress:
+
+   plt.figure();
+
+.. ipython:: python
+
+   @savefig barh_plot_stacked_ex.png width=5in
+   df2.plot(kind='barh', stacked=True);
+
+Histograms
+~~~~~~~~~~
 .. ipython:: python
 
    plt.figure();
@@ -160,7 +199,7 @@ a uniform random variable on [0,1).
    plt.figure();
 
    @savefig box_plot_ex.png width=4.5in
-   df.boxplot()
+   bp = df.boxplot()
 
 You can create a stratified boxplot using the ``by`` keyword argument to create
 groupings.  For instance,
@@ -173,7 +212,7 @@ groupings.  For instance,
    plt.figure();
 
    @savefig box_plot_ex2.png width=4.5in
-   df.boxplot(by='X')
+   bp = df.boxplot(by='X')
 
 You can also pass a subset of columns to plot, as well as group by multiple
 columns:
@@ -187,4 +226,20 @@ columns:
    plt.figure();
 
    @savefig box_plot_ex3.png width=4.5in
-   df.boxplot(column=['Col1','Col2'], by=['X','Y'])
+   bp = df.boxplot(column=['Col1','Col2'], by=['X','Y'])
+
+.. _visualization.scatter_matrix:
+
+Scatter plot matrix
+~~~~~~~~~~~~~~~~~~~
+
+*New in 0.7.3.* You can create a scatter plot matrix using the
+ ``scatter_matrix`` method in ``pandas.tools.plotting``:
+
+.. ipython:: python
+
+   from pandas.tools.plotting import scatter_matrix
+   df = DataFrame(np.random.randn(1000, 4), columns=['a', 'b', 'c', 'd'])
+
+   @savefig scatter_matrix_ex.png width=6in
+   scatter_matrix(df, alpha=0.2, figsize=(8, 8))

doc/source/whatsnew.rst

@@ -16,6 +16,8 @@ What's New
 
 These are new features and improvements of note in each release.
 
+.. include:: whatsnew/v0.7.3.txt
+
 .. include:: whatsnew/v0.7.2.txt
 
 .. include:: whatsnew/v0.7.1.txt