Importing pandas in python changes how matplotlib handles datetime objects?

Question

On my debian squeeze system, I ran into a python problem that can be distilled to the following:

import numpy
import datetime
from matplotlib import pyplot
x = [datetime.datetime.utcfromtimestamp(i) for i in numpy.arange(100000,200000,3600)]
y = range(len(x))

# See matplotlib handle a series of datetimes just fine..
pyplot.plot(x, y)
# [<matplotlib.lines.Line2D object at 0xad10f4c>]

import pandas

# Now we try exactly what we did before..
pyplot.plot(x, y)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/pymodules/python2.6/matplotlib/pyplot.py", line 2141, in plot
    ret = ax.plot(*args, **kwargs)
  File "/usr/lib/pymodules/python2.6/matplotlib/axes.py", line 3432, in plot
    for line in self._get_lines(*args, **kwargs):
  File "/usr/lib/pymodules/python2.6/matplotlib/axes.py", line 311, in _grab_next_args
    for seg in self._plot_args(remaining, kwargs):
  File "/usr/lib/pymodules/python2.6/matplotlib/axes.py", line 288, in _plot_args
    x, y = self._xy_from_xy(x, y)
  File "/usr/lib/pymodules/python2.6/matplotlib/axes.py", line 204, in _xy_from_xy
    bx = self.axes.xaxis.update_units(x)
  File "/usr/lib/pymodules/python2.6/matplotlib/axis.py", line 982, in update_units
    self._update_axisinfo()
  File "/usr/lib/pymodules/python2.6/matplotlib/axis.py", line 994, in _update_axisinfo
    info = self.converter.axisinfo(self.units, self)
  File "/usr/local/lib/python2.6/dist-packages/pandas/tseries/converter.py", line 184, in axisinfo
    majfmt = PandasAutoDateFormatter(majloc, tz=tz)
  File "/usr/local/lib/python2.6/dist-packages/pandas/tseries/converter.py", line 195, in __init__
    dates.AutoDateFormatter.__init__(self, locator, tz, defaultfmt)
TypeError: __init__() takes at most 3 arguments (4 given)

I'm not interested in the cause of the particular error shown, it's obvious enough that pandas expected a different version of matplotlib -- that's a fair risk of getting one package from the standard debian repository and the other through pip, and I already 'solved' that part of the problem by allowing pip to upgrade matplotlib.

The real issue is -- and now comes a threefold question: how can it be that just importing pandas broke matplotlib's ability to handle datetime objects, when just two lines earlier pandas was clearly not even involved in that same operation? Does pandas upon import silently alter other modules in the top level namespace to force them to make use of pandas methods? And is this acceptable behavour for a python module? Because I need to be able to rely on it that importing, say, a random number module, won't silently change, say, the pickle module to apply a random salt to everything it writes..

Update with further information

python is 2.6.6 (current debian stable from package 2.6.6-3+squeeze7)

matplotlib version was debian's 0.99.3-1 (current debian stable from package python-matplotlib)

pandas version was 0.9.0 (installed with 'pip install pandas', a while ago -- not today)

Platform is an i386 running debian Squeeze

Steps to replicate

(obvious) Bootstrap a clean debian squeeze i386 installation and chroot into it.
apt-get update
apt-get install python python-matplotlib
apt-get install python-pip build-essential python-dev
pip install --upgrade numpy
pip install pandas

Now start an interactive python session

import numpy
import datetime
# Next two lines added to original example to avoid hassle with DISPLAY in chroot
import matplotlib
matplotlib.use('agg')
from matplotlib import pyplot

x = [datetime.datetime.utcfromtimestamp(i) for i in numpy.arange(100000,200000,3600)]
y = range(len(x))

pyplot.plot(x, y)

import pandas

pyplot.plot(x, y)

It really shouldn't do! Could you replicate this? What pandas/matplotlib versions were you using? — Andy Hayden
– Andy Hayden, Commented Dec 21, 2012 at 10:31
I cannot replicate it on a Windows 7 with pandas 0.9.1 and matplotlib 1.1.0 — joris
– joris, Commented Dec 21, 2012 at 10:39
@joris he was using an old version of matplotlib (Note: <1 is not supported with pandas). Saying that you don't expect pyplots behaviour to change after an import. Unless perhaps pandas was installed during this time (and python was kept open)! ... — Andy Hayden
– Andy Hayden, Commented Dec 21, 2012 at 10:57
python is 2.6.6 (current debian stable from package 2.6.6-3+squeeze7) matplotlib version was debian's 0.99.3-1 (current debian stable from package python-matplotlib) pandas version was 0.9.0 Platform is an i386 running debian Squeeze — user1921146
– user1921146, Commented Dec 21, 2012 at 11:04
I have succesful replication on other machines. Steps to replicate have been added to the main article. Note that in the replication, the version of pandas retrieved by pip was 0.10.0, but the result is exactly same as observed in 0.9.0. — user1921146
– user1921146, Commented Dec 21, 2012 at 12:44

tacaswell · Accepted Answer · 2012-12-21 16:19:59Z

3

When you import pandas it registers a bunch of unit converters with matplotlib. This is from more updated versions of both libraries, but I assume that the overall behavior is the same.

In [4]: import matplotlib.units as muints

In [5]: muints.registry
Out[5]: 
  {datetime.date: <matplotlib.dates.DateConverter instance at 0x2ab8908>,
   datetime.datetime: <matplotlib.dates.DateConverter instance at 0x2ab8ab8>}


In [6]: import pandas

In [7]: muints.registry
Out[7]: 
{pandas.tseries.period.Period: <pandas.tseries.converter.PeriodConverter instance at 0x2627e60>,
 pandas.tslib.Timestamp: <pandas.tseries.converter.DatetimeConverter instance at 0x264ea28>,
 datetime.date: <pandas.tseries.converter.DatetimeConverter instance at 0x2532fc8>,
 datetime.datetime: <pandas.tseries.converter.DatetimeConverter instance at 0x2627ab8>,
 datetime.time: <pandas.tseries.converter.TimeConverter instance at 0x2532f38>}

This registry is used by axis (with a few layers of re-direction) to determine how to format information that is not numbers and it matches on the class of the thing it is trying to label with (hence, the entries in the dictionary keyed to datetime.*).

I suspect you can fix this by replacing the offending entries in dict

edited Dec 21, 2012 at 16:19

answered Dec 21, 2012 at 16:12

tacaswell

88.2k23 gold badges222 silver badges203 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Chang She Over a year ago

currently pandas uses its own converters for nicer looking tick labeling as you zoom. It should register those on first plot rather than import though. I made an issue here: github.com/pydata/pandas/issues/2579 Hopefully we'll be able to get to it in the next version

blais Over a year ago

Hi Chang,I disagree that this should be automatic; this has caused some bugs while using a recent version of Pandas with an older version of matplotlib. These kinds of side-effects should be called explicitly by the user, even if that means that by default the user gets the older / uglier tick labels.

tacaswell Over a year ago

@blais You should leave that comment on the github issue

tacaswell Over a year ago

@ChangShe see above comment.

Dave X Over a year ago

pandas-0.18.1 and pandas-0.20.3 have this problem, but pandas-0.22.0 does not do the automatic registering.

Collectives™ on Stack Overflow

Importing pandas in python changes how matplotlib handles datetime objects?

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related