Pandas: ValueError: cannot set WRITEABLE flag to True of this array

@macd2 : Thanks for reporting this! A couple of things:

• In the issue, could you provide your environment information from pandas.show_versions ?
• Assuming you can, do you mind sharing the file that triggers this error?

Im getting all of a sudden this Error, any idea?

• It sounds like this was working for you on a previous versions of pandas . When did this code last work for you? What version are you using now (related to the first question)?

cc @jreback

When using numpy=1.16.0 I get this error, when I downgrade numpy=1.15.4 problem is gone

@gfyoung Sure here is the Versions:
```pandas.show_versions
'dependencies':
{'pandas': '0.23.4', 'pytest': '3.4.0', 'pip': '18.1', 'setuptools': '40.6.3', 'Cython': '0.29.3', 'numpy': '1.16.0', 'scipy': '1.2.0', 'pyarrow': None, 'xarray': None, 'IPython': '6.5.0', 'sphinx': None, 'patsy': '0.5.0', 'dateutil': '2.7.5', 'pytz': '2018.7', 'blosc': None, 'bottleneck': '1.2.1', 'tables': '3.4.4', 'numexpr': '2.6.8', 'feather': None, 'matplotlib': '3.0.2', 'openpyxl': '2.5.12', 'xlrd': '1.1.0', 'xlwt': '1.3.0', 'xlsxwriter': '0.7.3', 'lxml': '4.1.1', 'bs4': '4.4.1', 'html5lib': '1.0b8', 'sqlalchemy': '1.2.15', 'pymysql': '0.9.2', 'psycopg2': '2.7.6.1 (dt dec pq3 ext lo64)', 'jinja2': '2.10', 's3fs': None, 'fastparquet': None, 'pandas_gbq': None, 'pandas_datareader': '0.7.0'}

```

unfortunately i can not share the file but i think the issues indeed comes from numpy as @vvvlc said here is an other issue on their git:

https://github.com/nipy/nibabel/issues/697

PS: just downgraded to numpy=1.15.4 and indeed it resolves the issue

Is there anything pandas can do in the meantime? Or just wait for the next
pytables release?

On Mon, Jan 21, 2019 at 5:34 AM macd2 notifications@github.com wrote:

@gfyoung https://github.com/gfyoung Sure here is the Versions:

'dependencies':
{'pandas': '0.23.4', 'pytest': '3.4.0', 'pip': '18.1', 'setuptools': '40.6.3', 'Cython': '0.29.3', 'numpy': '1.16.0', 'scipy': '1.2.0', 'pyarrow': None, 'xarray': None, 'IPython': '6.5.0', 'sphinx': None, 'patsy': '0.5.0', 'dateutil': '2.7.5', 'pytz': '2018.7', 'blosc': None, 'bottleneck': '1.2.1', 'tables': '3.4.4', 'numexpr': '2.6.8', 'feather': None, 'matplotlib': '3.0.2', 'openpyxl': '2.5.12', 'xlrd': '1.1.0', 'xlwt': '1.3.0', 'xlsxwriter': '0.7.3', 'lxml': '4.1.1', 'bs4': '4.4.1', 'html5lib': '1.0b8', 'sqlalchemy': '1.2.15', 'pymysql': '0.9.2', 'psycopg2': '2.7.6.1 (dt dec pq3 ext lo64)', 'jinja2': '2.10', 's3fs': None, 'fastparquet': None, 'pandas_gbq': None, 'pandas_datareader': '0.7.0'}

unfortunately i can not share the file but i think the issues indeed comes
from numpy as @vvvlc https://github.com/vvvlc said here is an other
issue on their git:

nipy/nibabel#697 https://github.com/nipy/nibabel/issues/697

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pandas-dev/pandas/issues/24839#issuecomment-456043488,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABQHIqiIyuE-lmMlN2Ep-4htyMrTBdefks5vFaWqgaJpZM4aJQqs
.

For me pip install numpy==1.15.4 also resolves this issue.

@TomAugspurger this issue comes up on Google on top (at least for me it did) and it seems that the remedy is simple, so maybe it's enough to just wait for an upstream fix. Really great that you would ask :)

pip3 install numpy==1.15.4 also solved for me..

However, when I downgraded numpy i kept getting this error ImportError: No module named 'numpy.core._multiarray_umath'. Finally figured out it was happening because I stored the .h5 file with numpy 1.16 installed and it wouldn't reopen with the downgraded numpy...

you can avoid the error:
ValueError: cannot set WRITEABLE flag to True of this array
passing format='table' to HDFStore.append or HDFStore.put when you save data with pandas.

This will likely solve your problem, tested with pandas 0.24 and numpy 1.16+

@dev72
ok but how about old HDF files that already exist?

I think the best way to read old hdf files is to downgrade your pandas+numpy version, read all data and write it in a new hdf store with format='table'.
Then it should work with newer numpy and pandas versions.

@dev72 yes right im at the same point, but than i rather stick with the downgrade until this is properly fixed

yes same came from google, thanks for filing this issue @macd2

So after chasing around a few issues submitted at numpy and PyTables this post by @avalentino would suggest this issue is fixed in PyTables master but not in a release yet.

https://github.com/PyTables/PyTables/issues/719#issuecomment-455612656

Has anyone tried using PyTables master w/ numpy >= 0.16?

I got it to work using this

HDF5_DIR={HDF5_PATH} pip install -e git+https://github.com/PyTables/PyTables@492ee2f#egg=tables
pip install numpy==1.16.0

Make sure cython and hdf5 are installed.

What versions of pytables and numpy reproduce this? Is it specific to the data?

with

pandas: 0.24.1
numpy: 1.16.2
tables: 3.4.4

this doesn't raise,

In [8]: df = pd.DataFrame({"A": [1, 2]})

In [9]: df.to_hdf('x.hdf5', key='x')

In [10]: pd.read_hdf('x.hdf5', 'x', mode='r')
Out[10]:
   A
0  1
1  2

FYI, pytables 3.5.0 and 3.5.1 are on PyPI with the fix from the pytables side.

Upgrading to pytables 3.5.1 fixes the problem for me also with numpy 1.16.2

dealing with tf 2.0.0 need numpy (at least) 1.16.0. downgrading numpy to the previous version won't work on tf 2.0.0