numpy.vdot
may segfault if passed an object that implements __array__
in a non-standard way.
The segfault happens in this line of code:
https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/multiarraymodule.c#L2245
since type
may be NULL
if PyArray_DescrFromType
fails.
import numpy as np
class Foo(object):
def __array__(self, a):
return self
np.vdot(Foo(), Foo())
Backtrace:
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
* frame #0: 0x0000000100cef88e _multiarray_umath.cpython-37m-darwin.so`array_vdot + 126
frame #1: 0x00000001005e0994 python`PyCFunction_Call + 148
frame #2: 0x0000000100c48dc1 _multiarray_umath.cpython-37m-darwin.so`array_implement_array_function + 305
frame #3: 0x00000001005e11b3 python`_PyMethodDef_RawFastCallKeywords + 227
frame #4: 0x00000001005e071c python`_PyCFunction_FastCallKeywords + 44
frame #5: 0x00000001006b6048 python`call_function + 664
frame #6: 0x00000001006b2a18 python`_PyEval_EvalFrameDefault + 27080
frame #7: 0x00000001006b6e45 python`_PyEval_EvalCodeWithName + 2997
frame #8: 0x00000001005e06d6 python`_PyFunction_FastCallKeywords + 230
frame #9: 0x00000001006b60d9 python`call_function + 809
frame #10: 0x00000001006b2987 python`_PyEval_EvalFrameDefault + 26935
frame #11: 0x00000001006b6e45 python`_PyEval_EvalCodeWithName + 2997
frame #12: 0x00000001006abfb0 python`PyEval_EvalCode + 48
frame #13: 0x00000001006ee677 python`PyRun_InteractiveOneObjectEx + 615
frame #14: 0x00000001006ede4e python`PyRun_InteractiveLoopFlags + 190
frame #15: 0x00000001006edd5c python`PyRun_AnyFileExFlags + 60
frame #16: 0x0000000100710c51 python`pymain_main + 7873
frame #17: 0x000000010071127f python`_Py_UnixMain + 111
frame #18: 0x00007fff5f6e63d5 libdyld.dylib`start + 1
I've reproduced this both on Mac OS X with:
1.17.0 3.7.2 (default, Jan 16 2019, 11:36:28)
[Clang 10.0.0 (clang-1000.11.45.2)]
and on an internal Linux build.
This issue was originally reported as https://github.com/google/jax/issues/1162
Hmm, vdot
checks what the output dtype will be ahead of time (which I suppose is fine). However, that function can fail and return NPY_NOTYPE
, so it should be returning an error when NPY_NOTYPE
is returned. This is in https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/multiarraymodule.c#L2241 (and the following line), the error you see is an error return later on, but I think that function really should not return an error (though it would be fine to add an error check there as well).
Do you want to make a PR?
(Might be nice to check if there are more similar usages where the error check is missing.)
Hi,
I am a newcomer and would like to work on this issue. Any leads on where to begin?
Thanks in advance.
Read about our developement workflow. Then you should get to a point where you can run python runtests.py
off the master
branch (git checkout master; git clean -xfd; python runtests.py
).
When you are set up, you can start hacking: create a branch, add a failing test (numpy/core/tests/teste_multiarray.py
in TestVDot
would be a good place), try to fix the C code from the hint above. If you get stuck feel free to ask for help, the more you can show that you made a true effort to follow the documentation and hints the better.
Welcome @Soniyanayak51!
To add to what @mattip said: one bit of info that's missing from our docs (I think) is how we work on issues. We don't use the GitHub "assignees" feature, so unless it's clear from recent comments/PR that someone is actively working on an issue, you can just dive in.
@rgommers @mattip I have added checks for types obtained from PyArray_DescrFromType(). I am not sure if I should add them for PyArray_DESCR() as well.
Also, about adding tests in test_multiarray.py, should I add a test_PyArrayType and add tests where the type is NULL? I am not sure how to move forward with adding tests.
@rgommers @mattip I have added checks for types obtained from PyArray_DescrFromType(). I am not sure if I should add them for PyArray_DESCR() as well.
Also, about adding tests in test_multiarray.py, should I add a test_PyArrayType and add tests where the type is NULL? I am not sure how to move forward with adding tests.
Looks like @mattip answered this partly on the PR (I think). Probably better to keep the conversation there.
should I add a test_PyArrayType and add tests where the type is NULL?
To add a little: the principle for adding tests normally is to only test the public API. So you'd take
...
np.vdot(Foo(), Foo())
from the issue description above, and add that as a test. It doesn't look like it's necessary to test the C API separately.