Numpy: Crash in `np.vdot` for array-like object

Created on 10 Aug 2019  ·  6Comments  ·  Source: numpy/numpy

numpy.vdot may segfault if passed an object that implements __array__ in a non-standard way.

The segfault happens in this line of code:
https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/multiarraymodule.c#L2245
since type may be NULL if PyArray_DescrFromType fails.

Reproducing code example:

import numpy as np

class Foo(object):
     def __array__(self, a):
         return self

np.vdot(Foo(), Foo())

Error message:

Backtrace:

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000100cef88e _multiarray_umath.cpython-37m-darwin.so`array_vdot + 126
    frame #1: 0x00000001005e0994 python`PyCFunction_Call + 148
    frame #2: 0x0000000100c48dc1 _multiarray_umath.cpython-37m-darwin.so`array_implement_array_function + 305
    frame #3: 0x00000001005e11b3 python`_PyMethodDef_RawFastCallKeywords + 227
    frame #4: 0x00000001005e071c python`_PyCFunction_FastCallKeywords + 44
    frame #5: 0x00000001006b6048 python`call_function + 664
    frame #6: 0x00000001006b2a18 python`_PyEval_EvalFrameDefault + 27080
    frame #7: 0x00000001006b6e45 python`_PyEval_EvalCodeWithName + 2997
    frame #8: 0x00000001005e06d6 python`_PyFunction_FastCallKeywords + 230
    frame #9: 0x00000001006b60d9 python`call_function + 809
    frame #10: 0x00000001006b2987 python`_PyEval_EvalFrameDefault + 26935
    frame #11: 0x00000001006b6e45 python`_PyEval_EvalCodeWithName + 2997
    frame #12: 0x00000001006abfb0 python`PyEval_EvalCode + 48
    frame #13: 0x00000001006ee677 python`PyRun_InteractiveOneObjectEx + 615
    frame #14: 0x00000001006ede4e python`PyRun_InteractiveLoopFlags + 190
    frame #15: 0x00000001006edd5c python`PyRun_AnyFileExFlags + 60
    frame #16: 0x0000000100710c51 python`pymain_main + 7873
    frame #17: 0x000000010071127f python`_Py_UnixMain + 111
    frame #18: 0x00007fff5f6e63d5 libdyld.dylib`start + 1

Numpy/Python version information:

I've reproduced this both on Mac OS X with:

1.17.0 3.7.2 (default, Jan 16 2019, 11:36:28)
[Clang 10.0.0 (clang-1000.11.45.2)]

and on an internal Linux build.

This issue was originally reported as https://github.com/google/jax/issues/1162

00 - Bug

All 6 comments

Hmm, vdot checks what the output dtype will be ahead of time (which I suppose is fine). However, that function can fail and return NPY_NOTYPE, so it should be returning an error when NPY_NOTYPE is returned. This is in https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/multiarraymodule.c#L2241 (and the following line), the error you see is an error return later on, but I think that function really should not return an error (though it would be fine to add an error check there as well).

Do you want to make a PR?

(Might be nice to check if there are more similar usages where the error check is missing.)

Hi,
I am a newcomer and would like to work on this issue. Any leads on where to begin?
Thanks in advance.

Read about our developement workflow. Then you should get to a point where you can run python runtests.py off the master branch (git checkout master; git clean -xfd; python runtests.py).

When you are set up, you can start hacking: create a branch, add a failing test (numpy/core/tests/teste_multiarray.py in TestVDot would be a good place), try to fix the C code from the hint above. If you get stuck feel free to ask for help, the more you can show that you made a true effort to follow the documentation and hints the better.

Welcome @Soniyanayak51!

To add to what @mattip said: one bit of info that's missing from our docs (I think) is how we work on issues. We don't use the GitHub "assignees" feature, so unless it's clear from recent comments/PR that someone is actively working on an issue, you can just dive in.

@rgommers @mattip I have added checks for types obtained from PyArray_DescrFromType(). I am not sure if I should add them for PyArray_DESCR() as well.
Also, about adding tests in test_multiarray.py, should I add a test_PyArrayType and add tests where the type is NULL? I am not sure how to move forward with adding tests.

@rgommers @mattip I have added checks for types obtained from PyArray_DescrFromType(). I am not sure if I should add them for PyArray_DESCR() as well.
Also, about adding tests in test_multiarray.py, should I add a test_PyArrayType and add tests where the type is NULL? I am not sure how to move forward with adding tests.

Looks like @mattip answered this partly on the PR (I think). Probably better to keep the conversation there.

should I add a test_PyArrayType and add tests where the type is NULL?

To add a little: the principle for adding tests normally is to only test the public API. So you'd take

...
np.vdot(Foo(), Foo())

from the issue description above, and add that as a test. It doesn't look like it's necessary to test the C API separately.

Was this page helpful?
0 / 5 - 0 ratings