New axes.set_prop_cycle method cannot handle any generic iterable #5368

Closed
u55 opened this Issue Oct 31, 2015 · 19 comments

Comments

Projects
None yet
4 participants
Contributor

u55 commented Oct 31, 2015

Dear matplotlib developers,

The docstring for the new method axes.set_prop_cycle, says that it can take any iterable for the property value. However, I found that it fails with a ValueError if given a numpy.ndarray. Here is a minimal working example:

from __future__ import print_function
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm

cmap = cm.Set1(np.linspace(0,1,9))
print(type(cmap),'\n')

fig, ax = plt.subplots()
ax.set_prop_cycle('color', cmap.tolist())    # works
ax.set_prop_cycle('color', cmap)    # raises ValueError
plt.show()

which produces this traceback:

<type 'numpy.ndarray'> 

Traceback (most recent call last):
  File "/tmp/untitled.py", line 12, in <module>
    ax.set_prop_cycle('color', cmap)    # raises ValueError
  File "/usr/lib/python2.7/site-packages/matplotlib/axes/_base.py", line 1112, in set_prop_cycle
    prop_cycle = cycler(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/matplotlib/rcsetup.py", line 720, in cycler
    vals = validator(vals)
  File "/usr/lib/python2.7/site-packages/matplotlib/rcsetup.py", line 90, in f
    raise ValueError(msg)
ValueError: 's' must be of type [ string | list | tuple ]

Tested with matplotlib 1.5.0, python 2.7 and python 3.5, running in Arch Linux.
This failure in matplotlib 1.5.0 also affects the method axes.set_color_cycle, which is supposed to be deprecated but raises a ValueError instead. The following code worked in matplotlib 1.4.3 but fails in 1.5.0:

import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm

cmap = cm.Set1(np.linspace(0,1,9))

fig, ax = plt.subplots()
ax.set_color_cycle(cmap)    # raises ValueError
plt.show()

I think that, for convenience and for backwards compatibility with prior versions of axes.set_color_cycle, the new method axes.set_prop_cycle should be able to take any iterable, as the documentation suggests, and not be limited to only a string, list, or tuple.

Thank you,

u55 changed the title from New axes.set_prop_cycle method cannot handle any iterable to New axes.set_prop_cycle method cannot handle any generic iterable Oct 31, 2015

Owner

tacaswell commented Oct 31, 2015

Can you take a look at how this validation is done? The intention is definitely to take any iterable.

Attn @WeatherGod

Contributor

u55 commented Oct 31, 2015

Yes, the validation comes about because axes/_base.py has the line:

from matplotlib.rcsetup import cycler

instead of using something such as:

from cycler import cycler

and the version of cycler in rcsetup.py does the validating, including this test:

elif type(s) in (list, tuple):

Doing this sort of validating for the input parameters of axes.set_prop_cycle is unpythonic, in my opinion, because it goes against the principle of duck typing.

However, fixing the problem is not a simple matter of replacing from matplotlib.rcsetup import cycler with from cycler import cycler, because axes.set_prop_cycle has three different input forms:

Form 1 simply sets given `Cycler` object.
Form 2 creates and sets  a `Cycler` from a label and an iterable.
Form 3 composes and sets  a `Cycler` as an inner product of the
       pairs of keyword arguments. In other words, all of the
       iterables are cycled simultaneously, as if through zip().

and the logic to handle this is currently in rcsetup.cycler rather than in axes.set_prop_cycle.

Owner

tacaswell commented Oct 31, 2015

That test should probably be replaced with a try/except block once the
strings are excluded.

Can you take a crack at that?

On Sat, Oct 31, 2015, 16:51 u55 notifications@github.com wrote:

Yes, the validation comes about because axes/_base.py has the line
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/axes/_base.py#L36
:

from matplotlib.rcsetup import cycler

instead of using something such as:

from cycler import cycler

and the version of cycler in rcsetup.py does the validating, including this
test
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/rcsetup.py#L81
:

elif type(s) in (list, tuple):

Doing this sort of validating for the input parameters of
axes.set_prop_cycle is unpythonic, in my opinion, because it goes against
the principle of duck typing.

However, fixing the problem is not a simple matter of replacing from
matplotlib.rcsetup import cycler with from cycler import cycler, because
axes.set_prop_cycle has three different input forms
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/axes/_base.py#L1076-L1082
:

Form 1 simply sets given Cycler object.
Form 2 creates and sets a Cycler from a label and an iterable.
Form 3 composes and sets a Cycler as an inner product of the
pairs of keyword arguments. In other words, all of the
iterables are cycled simultaneously, as if through zip().

and the logic
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/rcsetup.py#L688-L712
to handle this is currently in rcsetup.cycler rather than in
axes.set_prop_cycle.

—
Reply to this email directly or view it on GitHub
#5368 (comment)
.

Member

WeatherGod commented Nov 1, 2015

The fix is easy. Use cbook.isiterable instead of checking the types. That
check has always been there, but previously it was only applied to values
going through the rcparams route. When I rejiggered everything, I purposely
wanted validation everywhere and just used the available mechanism. I
didn't think about users supplying a ndarray of rgb values because I
focused on the rcparams codepath.
On Oct 31, 2015 4:57 PM, "Thomas A Caswell" notifications@github.com
wrote:

That test should probably be replaced with a try/except block once the
strings are excluded.

Can you take a crack at that?

On Sat, Oct 31, 2015, 16:51 u55 notifications@github.com wrote:

Yes, the validation comes about because axes/_base.py has the line
<
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/axes/_base.py#L36

:

from matplotlib.rcsetup import cycler

instead of using something such as:

from cycler import cycler

and the version of cycler in rcsetup.py does the validating, including
this
test
<
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/rcsetup.py#L81

:

elif type(s) in (list, tuple):

Doing this sort of validating for the input parameters of
axes.set_prop_cycle is unpythonic, in my opinion, because it goes against
the principle of duck typing.

However, fixing the problem is not a simple matter of replacing from
matplotlib.rcsetup import cycler with from cycler import cycler, because
axes.set_prop_cycle has three different input forms
<
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/axes/_base.py#L1076-L1082

:

Form 1 simply sets given Cycler object.
Form 2 creates and sets a Cycler from a label and an iterable.
Form 3 composes and sets a Cycler as an inner product of the
pairs of keyword arguments. In other words, all of the
iterables are cycled simultaneously, as if through zip().

and the logic
<
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/rcsetup.py#L688-L712

to handle this is currently in rcsetup.cycler rather than in
axes.set_prop_cycle.

—
Reply to this email directly or view it on GitHub
<
#5368 (comment)

.

—
Reply to this email directly or view it on GitHub
#5368 (comment)
.

Contributor

u55 commented Nov 2, 2015

@tacaswell @WeatherGod
Question, how should rcsetup._listify_validator behave when passed an empty dictionary?

I tried replacing this line:

elif type(s) in (list, tuple):
    ...

with this line:

import matplotlib.cbook as cbook
elif cbook.iterable(s):
    ...

which fixes the bug, but now this test fails:

...
        {'validator': validate_stringlist,
         'success': (('', []),
                     ('a,b', ['a', 'b']),
                     ('aardvark', ['aardvark']),
                     ('aardvark, ', ['aardvark']),
                     ('aardvark, ,', ['aardvark']),
                     (['a', 'b'], ['a', 'b']),
                     (('a', 'b'), ['a', 'b']),
                     ((1, 2), ['1', '2'])),
            'fail': ((dict(), ValueError),  # <------- this test fails
                     (1, ValueError),)
            },
...

Previously, an empty list would have been validated but an empty dictionary would have raised a ValueError. An empty list must be valid since certain rcParams, such as 'animation.ffmpeg_args', default to an empty list. I see no reason that an empty dictionary should raise a ValueError, so should I remove this failing test or should I add an extra test on this line for an empty dictionary?

Member

WeatherGod commented Nov 2, 2015

That test has been in there for a long time, And I think an unordered
iterable is a bit silly. However, that probably doesn't mean that it is
correct to error out on it. The problem is creating a test that would pass
reliably on it. So, perhaps the code should be a bit tighter and test for
ordered iterables? Perhaps ABC's would help?

On Sun, Nov 1, 2015 at 7:33 PM, u55 notifications@github.com wrote:

@tacaswell https://github.com/tacaswell @WeatherGod
https://github.com/WeatherGod
Question, how should rcsetup._listify_validator behave when passed an
empty dictionary?

I tried replacing this line
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/rcsetup.py#L81
:

elif type(s) in (list, tuple):
...

with this line:

import matplotlib.cbook as cbookelif cbook.iterable(s):
...

which fixes the bug, but now this test fails
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/tests/test_rcparams.py#L284
:

...
{'validator': validate_stringlist,
'success': (('', []),
('a,b', ['a', 'b']),
('aardvark', ['aardvark']),
('aardvark, ', ['aardvark']),
('aardvark, ,', ['aardvark']),
(['a', 'b'], ['a', 'b']),
(('a', 'b'), ['a', 'b']),
((1, 2), ['1', '2'])),
'fail': ((dict(), ValueError), # <------- this test fails
(1, ValueError),)
},
...

Previously, an empty list would have been validated but an empty
dictionary would have raised a ValueError. An empty list must be valid
since certain rcParams, such as
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/rcsetup.py#L1165
'animation.ffmpeg_args', default to an empty list. I see no reason that
an empty dictionary should raise a ValueError, so should I remove this
failing test or should I add an extra test on this line
https://github.com/matplotlib/matplotlib/blob/38782e31579a10faca98b348767b0231f041a7e9/lib/matplotlib/rcsetup.py#L81
for an empty dictionary?

—
Reply to this email directly or view it on GitHub
#5368 (comment)
.

Owner

tacaswell commented Nov 2, 2015

The dict should fail because this is expecting a sequence of values, not a mapping.

Contributor

u55 commented Nov 2, 2015

Then I'm not sure how to do the test. A numpy.ndarray reports that it is not an instance of collections.Sequence:

>>> import collections
>>> import numpy as np
>>> isinstance(np.arange(9), collections.Sequence)
False
Contributor

u55 commented Nov 4, 2015

@tacaswell @WeatherGod
I see no reason to reject unordered iterables, such as sets, however I can see why dictionaries are not really appropriate.
How about replacing the validation test with this?:

try:
    import collections.abc as abc
except ImportError:
    # python 2
    import collections as abc
...

...
        elif isinstance(s, abc.Iterable) and not isinstance(s, abc.Mapping):
            ...

This will correctly pass a numpy.ndarray and a generic generator.

Member

WeatherGod commented Nov 4, 2015

I am ok with that. I still don't like the idea of allowing sets and bags,
but I see no reason to prevent them, either.

On Wed, Nov 4, 2015 at 11:23 AM, u55 notifications@github.com wrote:

@tacaswell https://github.com/tacaswell @WeatherGod
https://github.com/WeatherGod
I see no reason to reject unordered iterables, such as sets, however I can
see why dictionaries are not really appropriate.
How about replacing the validation test
https://github.com/matplotlib/matplotlib/blob/cb3c6f361376f14b10dae7027ba1e30b52f858a2/lib/matplotlib/rcsetup.py#L81
with this?:

try:
import collections.abc as abcexcept ImportError:
# python 2
import collections as abc
...

...
elif isinstance(s, abc.Iterable) and not isinstance(s, abc.Mapping):
...

This will correctly pass a numpy.ndarray and a generic generator.

—
Reply to this email directly or view it on GitHub
#5368 (comment)
.

Owner

tacaswell commented Nov 4, 2015

In the case of color_cycle an un-ordered iterable does not make sense as there being an order is important to the function.

I think it is a problem that from process-to-process these validators will give different results.

Member

WeatherGod commented Nov 4, 2015

@tacaswell, note that the validator will convert whatever iterable into a
list, so it becomes ordered for the purpose of the cycler. Yes, which order
that will be is arbitrary, but that is for the user to decide if that is an
issue or not for their code.

On Wed, Nov 4, 2015 at 11:29 AM, Thomas A Caswell notifications@github.com
wrote:

In the case of color_cycle an un-ordered iterable does not make sense as
there being an order is important to the function.

I think it is a problem that from process-to-process these validators will
give different results.

—
Reply to this email directly or view it on GitHub
#5368 (comment)
.

Owner

tacaswell commented Nov 4, 2015

I am just forseeing lots of very confused bug reports about randomized color cycles.

Member

WeatherGod commented Nov 4, 2015

Do you really think there are a lot of people keeping configuration values
around as sets?

Besides, I think this is a good compromise given that numpy's NDArray is
not a Sequence, so any check for orderability then falls to whitelisting
object types, which we know is doomed as soon as someone tries to pass a
pandas object through.

On Wed, Nov 4, 2015 at 11:47 AM, Thomas A Caswell notifications@github.com
wrote:

I am just forseeing lots of very confused bug reports about randomized
color cycles.

—
Reply to this email directly or view it on GitHub
#5368 (comment)
.

Owner

tacaswell commented Nov 4, 2015

Do not underestimate users 😜

It is probably worth letting that go through to keep the code simple, can we at least add a big developer warning in code comments summarizing this conversation?

Member

WeatherGod commented Nov 4, 2015

Sure:

# @WeatherGod: meh, let the users mess up themselves
# @tacaswell: SETS BAD!

;-)

On Wed, Nov 4, 2015 at 11:57 AM, Thomas A Caswell notifications@github.com
wrote:

Do not underestimate users [image: 😜]

It is probably worth letting that go through to keep the code simple, can
we at least add a big developer warning in code comments summarizing this
conversation?

—
Reply to this email directly or view it on GitHub
#5368 (comment)
.

Member

WeatherGod commented Nov 6, 2015

@u55, I hope the banter between @tacaswell and I didn't discourage you from putting together a pull request with your suggested fix.

Contributor

u55 commented Nov 6, 2015

Yeah, no problem. I'm just busy during the week. I'll submit a pull request soon.

@u55 u55 added a commit to u55/matplotlib that referenced this issue Nov 9, 2015

@u55 u55 Fix axes.set_prop_cycle to handle any generic iterable sequence.
Closes issue #5368.
8ac2ba0
Member

WeatherGod commented Nov 10, 2015

Fixed by #5452 and #5459

WeatherGod closed this Nov 10, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment