Mpld3: "TypeError: *** is not JSON serializable" when using xticks

Ah - it looks like mplexporter incorrectly assumes that xticks are a list. That's a bug.

In the meantime, if you manually convert your xticks to a list, it should work.

Actually, that's not the case. That is, the following works just fine:

fig = plt.figure()
ax = fig.add_subplot(111, xticks=np.arange(11))
ax.plot([0, 10], [0, 10], '-k')
mpld3.show()

Pandas is doing something strange here with its ticks: the error says that 2000 is not serializable, which is crazy. Does Pandas have some sort of built-in non-serializable floating point type?

It’s not the number 2000, it’s the year. Welcome to datetime hell.
(edit 6/19/2014: it seems I was wrong about datetime being the issue...)

Ugh... I'm not sure how to best handle that. Any thoughts?

Does it help to use the to_json in pandas?

http://pandas.pydata.org/pandas-docs/dev/io.html#date-handling

Does it help to use the to_json in pandas?

Perhaps, but then we'll have to do a try...except statement for a lot of things within the code (i.e. do it one way if the value is a Pandas variable, do it another way otherwise) and then make sure the axis labels on the javascript side can handle any JSON structure that Pandas decides to output. I'd prefer not to add that level of complexity, because it sounds like a very fragile solution and a maintenance nightmare.

I see. This boils down to Python's inability to expose json serialization method for user defined class then. You can close this issue.

We should leave it open... there may be a better way to handle this. I'll think about it.

Hold on a sec, this seems to work fine for me in Python 2.7. Could it be a Python3 issue only?

In [56]: matplotlib.__version__
Out[56]: '1.3.1'

In [57]: pandas.__version__
Out[57]: '0.13.1'

Hello all,
Not sure if related, but I get a similar issue when trying to plot this seaborn graph : http://www.stanford.edu/~mwaskom/software/seaborn/examples/many_facets.html

mpld3.fig_to_html(grid.fig, 'test.html')
/home/vagrant/graph/local/lib/python2.7/site-packages/mpld3/mplexporter/exporter.py:82: UserWarni ng: Blended transforms not yet supported. Zoom behavior may not work as expected.
  warnings.warn("Blended transforms not yet supported. "
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vagrant/graph/local/lib/python2.7/site-packages/mpld3/_display.py", line 236, in fig_to_html
    figure_json=json.dumps(figure_json),
  File "/usr/lib/python2.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 178, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: 0 is not JSON serializable

The issue is that numpy.int64 numbers are used internally, and that these are not json serializable by default.
There should be a hook in the library to provide a json.JSONEncoder to handle this.

Just to provide room for ideas:

I’ve got the same problem, but connected to Django, which is what I use as backend for the project I’m using in the project where I use mpld3.

I tried to serialize a numpy array and I got this error, when using django 1.6. I downgraded to 1.5 and it works. I found the reason to be that from 1.6 and up, the serialization is done with json, while from 1.5 and below, via Pickle.

I hope this can help you guys find the problem could be.

Med vennlig hilsen,
Gerardo de La Riva Espinosa
Høgskolelektor
Høgskolen i Gjøvik
e-post: gerardo.[email protected]
Mobil: +47 950 13 322

On 19 Jun 2014, at 16:56, lvasseur [email protected] wrote:

Hello all,
Not sure if related, but I get a similar issue when trying to plot this seaborn graph : http://www.stanford.edu/~mwaskom/software/seaborn/examples/many_facets.html

mpld3.fig_to_html(grid.fig, 'test.html')
/home/vagrant/graph/local/lib/python2.7/site-packages/mpld3/mplexporter/exporter.py:82: UserWarni ng: Blended transforms not yet supported. Zoom behavior may not work as expected.
warnings.warn("Blended transforms not yet supported. "
Traceback (most recent call last):
File "", line 1, in
File "/home/vagrant/graph/local/lib/python2.7/site-packages/mpld3/_display.py", line 236, in fig_to_html
figure_json=json.dumps(figure_json),
File "/usr/lib/python2.7/json/__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
return _iterencode(o, 0)
File "/usr/lib/python2.7/json/encoder.py", line 178, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: 0 is not JSON serializable
The issue is that numpy.int64 numbers are used internally, and that these are not json serializable by default.
There should be a hook in the library to provide a json.JSONEncoder to handle this.

—
Reply to this email directly or view it on GitHub.

FYI,

    from json import JSONEncoder
    class MyEncoder(JSONEncoder):
        def default(self, obj):
            if isinstance(obj, numpy.int64):
                return int(obj)
            return JSONEncoder.default(self, obj)

    return template.render(figid=json.dumps(figid),
                           d3_url=d3_url,
                           mpld3_url=mpld3_url,
                           figure_json=json.dumps(figure_json, cls=MyEncoder),
                           extra_css=extra_css,
                           extra_js=extra_js)

solves the issue in /mpld3/_display.py : fig_to_html

I don't have time to try now, but it looks like this is issue and the right direction for a solution. The json.dump docs make me think that there is a slightly lighter-weight solution with the default option, although it might be just one line shorter in the end. Are you up for doing a pull request?

I expected this to be fixed by PR #213, but the ipython notebook from the original post is still not working for me. Can some else try it?

I'll give this a try at some point

Still confused by this... I'm not sure whether #213 actually fixed anything here. I'm able to serialize int64 objects without it...

was this solved?
I cannot seem to fig_to_html a pandas generated figure ( I get TypeError: 0 is not JSON serializable). Figure is generated fine.
When I retried by using plt.bar I also get the error if I use yticks with custom strings.

How to solve?

Not sure. It seems to be some issue with axis labels being non-serializable versions of integers. If you figure anything out, let us know.

Just wanted to point out that I get this error when using

dates = df.Date.apply(lambda dt: dt.strftime('%B')[:3] + dt.strftime(' %d')).values
plt.xticks(x, dates, rotation='vertical')

but not with

ax.set_xticklabels(dates, rotation='vertical')

Hope this helps.

Edit: On further inspection, while the latter version doesn't produce an error, the resulting figure's xticks don't actually get set.

Edit 2: My bad, just found out ticks are not supported (#22).

FWIW, using boxplots can repro the problem:

import numpy as np, mpld3
np.random.seed(937)
data = np.random.lognormal(size=(37, 4), mean=1.5, sigma=1.75)
labels = list('ABCD')
fs = 10  # fontsize

fig, axes = plt.subplots(1, 1, figsize=(6, 6)) 
axes.boxplot(data, labels=labels)
axes.set_title('Default', fontsize=fs)

html_fig = mppld3.fig_to_html(fig)

Specifically, here are the versions from pip freeze

cycler==0.10.0
Jinja2==2.8
MarkupSafe==0.23
matplotlib==1.5.1
mpld3==0.2
numpy==1.10.4
pandas==0.17.1
ptyprocess==0.5.1
pyparsing==2.1.0
python-dateutil==2.5.0
pytz==2015.7
six==1.10.0
wheel==0.29.0

for Python 3.4.3.

The patch suggested by @lvasseur seems to fix the problem, although I don't know if it's _the_ solution.

To make the original example work, use [float(x) for x in range(2000,2002)] instead of np.arange(2000,2002,1).

Not sure if related, but I get a similar issue when trying to plot this seaborn graph : http://www.stanford.edu/~mwaskom/software/seaborn/examples/many_facets.html

I get same error while trying mtld3.display(fig) for seaborn barplot figure:

TypeError: 0 is not JSON serializable

Can anyone render seaborn plots with mpld3?

This seem to happen because there is nan in data values. If I break at _display.py:243
and replace nan with None in figure_json dict, I get expected output.

Edit: above does not seem to solve the problem (I was testing in debug interpreter, where it worked but not when using mpld3).
So, rendering sns.barplot() does not work, while rendering plt.bar() works fine. I couldn't see obvious difference in figures from both plots, but provided patch (https://github.com/mpld3/mpld3/pull/213) solved the problem.

I fixed this by adding the line

elif isinstance(obj, (numpy.ndarray,)):
        return obj.tolist()

to /mpld3/_display.py: NumpyEncoder

Thankyou @jaklinger this worked!!!

@jaklinger , Hi there, thanks for your suggestion, it works perfectly! I just wanted to clarify one thing - i converted the numpy array to a dictionary and tried to return it but it was throwing error "float 32 is not json serializable", although there was no issue with the dictionary, printed and checked it. Just by using your tolist() method it converted the array to python list.
previous :[100. 0. 0. 0. 0.]
with tolist: [100.0, 0.0, 0.0, 0.0, 0.0] and it returned perfectly.
So what was the issue exactly? Many thanks in advance.

@padmaksha18 I'm not 100% sure what you mean by

converted the numpy array to a dictionary

but (in short) numpy and python types aren't the same (float32 is not the same as a float; an array is not the same as a list, etc) and the JSONEncoder class only 'allows' native python types. So... the point of this function (which contains the new fix) is to convert the input obj into native python equivalents.

@jaklinger :Hi Joel, thanks a lot again. By converting to a dictionary i just meant that i took two such arrays and formed a key value pair, sorry for not explaining it clearly. Thanks a lot for making this thing clear to me, i must convert it to native python type first and that was the issue. Just wanted to make one more thing clear, like if don't jsonify a dictionary and try to return it as it is, will the jsonencoder class come into picture then? Many thanks!

pip install "git+https://github.com/javadba/mpld3@display_fix"