Shapeworks: rendering volumes of some resolutions crashes when using itkwidgets.view after using shapeworks in a notebook

Created on 23 Mar 2021  ·  22Comments  ·  Source: SCIInstitute/ShapeWorks

This used to be "getting-started-with-grooming-segmentations notebook crashes on MacOS," but lots of investigation has uncovered the actual issue, possibly due to incompatible shared libraries.

Clean system, clean conda.

This is the cell that crashes:

shapeSeg = shapeSegList[10]
itkw.view( image          = sw2vtkImage(shapeSeg),
           slicing_planes = True,
           axes           = True,
           rotate         = True,
           interpolation  = True)
Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x000000014b76fff3
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Segmentation fault: 11
Termination Reason:    Namespace SIGNAL, Code 0xb
Terminating Process:   exc handler [0]

VM Regions Near 0x14b76fff3:
    MALLOC_LARGE           000000014b50a000-000000014b769000 [ 2428K] rw-/rwx SM=PRV  
--> 
    MALLOC_LARGE           000000014b770000-000000014d863000 [ 32.9M] rw-/rwx SM=PRV  

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_platform.dylib        0x00007fff564c85e6 _platform_memmove$VARIANT$Nehalem + 486
1   libvnl.dylib                    0x000000010ae40e6b vnl_vector<double>::operator=(vnl_vector<double> const&) + 139
2   _ITKCommonPython.so             0x000000014ec0d18b vnl_vector<double>::operator=(vnl_vector<double>&&) + 65
3   _ITKCommonPython.so             0x000000014eadec9d 0x14de03000 + 13483165
4   _ITKCommonPython.so             0x000000014e869165 0x14de03000 + 10903909
5   _ITKCommonPython.so             0x000000014ec611a7 itk::ProcessObject::UpdateOutputInformation() + 339
6   _ITKCommonPython.so             0x000000014e47d8fe 0x14de03000 + 6793470
7   _ITKImageGridPython.so          0x00000001538b5619 itk::ProcessObject::UpdateOutputInformation() + 117
8   _ITKImageGridPython.so          0x00000001538b5ebc itk::ProcessObject::UpdateLargestPossibleRegion() + 18
9   _ITKCommonPython.so             0x000000014e10726b 0x14de03000 + 3162731
10  python                          0x00000001044c8843 _PyMethodDef_RawFastCallKeywords + 131
11  python                          0x00000001044c81d6 _PyObject_FastCallKeywords + 598
12  python                          0x0000000104600bb7 call_function + 455
...
bug

All 22 comments

Yup, I'm trying to run it now and hitting the same crash, right here:
```shapeSeg = shapeSegList[10]
itkw.view( image = sw2vtkImage(shapeSeg),
slicing_planes = True,
axes = True,
rotate = True,
interpolation = True)

Simplified the issue and it looks like Image::resample is the culprit here. After resampling, ellipsoid_05 loads the viewer while ellipsoid_00 causes the kernel to crash when trying to use itk.view()

Screen Shot 2021-03-24 at 11 02 20 PM

Screen Shot 2021-03-24 at 11 01 06 PM

After testing with this, something _may_ be wrong with Image::resample wrt itkwidgets.view.
Images tried: ellipsoid_1mode/segmentations/ellipsoid_00.nrrd and ellipsoid_1mode/segmentations/ellipsoid_05.nrrd
-> 00 fails to be plotted when wrapped by pyvista in a vtk image after resampling
-> 05 works fine
=> _both_ work fine when plotted by pyvista
I threw some debug spew into Image::resample and saw nothing out of the ordinary so far).

Here's a simple notebook if you want to experiment.
wishy_washy_resample.ipynb.zip

...and what it looks like empty when you load it:
Screen Shot 2021-03-24 at 7 59 23 PM

...and what it looks like when you run it. Change the 05 to 00 and run it again to crash when itkw tries to render.
Screen Shot 2021-03-24 at 7 55 29 PM

Transpose or... The Array That Could Be Rendered
If you recall, itkwidgets can render a numpy array directly. But it interprets that array as the transpose of what is passed, something it does _not_ do when a vtk array is used. Therefore we typically transpose what we get from the image before wrapping it in a vtk array, and simply use the vtk array with itkwidgets to keep things simple.

I tried both this morning and unfortunately it doesn't make a difference. Transposed or not, numpy or vtk arrays, it all crashes when trying to rendering using itkwidgets after Image.resample.

Copy or... Am I Repeating Myself?
The array coming from the Image might be the problem. Maybe it's pointing to invalid memory. So let's create a copy!
After copying the array--in both order='C' and order='F', transposing, modifying, standing on my head while composing the functions, and wishfully thinking about it all, _it still crashes every time_.

This suggested the size of the array itself might be a problem. One way to confirm this was to call resample with the same spacing ([1,1,2]), _which worked fine_. Another way was to create a bogus numpy array of said questionable size. And...
Drum roll please...
presenting...
for the first time ever... (in GitHub)
A _shapeworks-free_ repro!

array = np.ndarray([109, 77, 204])
itkw.view(image = array)

So go put that in your notebook and smoke* it.

*smoke implying crash, not advocating for actual inhalation of anything, note this is a non-smoking facility, not valid with any other offer, ...anyway, I hope someone is smiling, 'cuz this ain't our problem! Except that it is. 😞

array = np.ndarray([109, 77, 204])
itkw.view(image = array)
array = np.ndarray([109, 109, 204])
itkw.view(image = array)



md5-047f51858ff0a12b3bec85f300fc2efa



array = np.ndarray([109, 109, 109])
itkw.view(image = array)

All these examples render the itk viewer when shapeworks isn't imported.
It crashes when shapeworks is imported.

Hmm, do we have some conda itk/vtk vs shapeworks itk/vtk library interaction going on?

I guess itkwidgets is using another version of itk

Order of import is not important, but order of usage is. This means if you import everything, but use itkwidgets before shapeworks, it crashes when you use shapeworks, and vice versa.

Since the solution to #1179 was simply "don't use itk," I propose we resolve this issue the same. Unless there is a better option at this time, let's remove itkwidgets from our notebooks.

That sounds like a good plan for now. Is it reasonable to remove itkwidgets from the notebooks? What are we using it for?

We are using it to view the segmentations. I think we can use pyvista instead

Note that pyvista (as far as I know) does not respect image meta-information (origin, voxel spacing, axes direction), yet itkwidget does.

Note that pyvista (as far as I know) does not respect image meta-information (origin, voxel spacing, axes direction), yet itkwidget does.

We'll dig into this, but there must be some respect for origin since multiple volumes can be simultaneously plotted with different locations, and this image shows that spacing is used for plotting. Our sw2vtkImage function(s) simply didn't keep it (I know I changed this in some, but maybe those are in the new pybind branch).

Screen Shot 2021-03-26 at 1 34 32 PM

I want to suggest we release and push this to the next release since that one has many other python and notebook improvements, and essentially we'll be forced to rewrite them. Maybe we just let them be broken. It's a known issue in a public repository and we have a workaround in case any users run into it.

(We will still need to modify the getting-started-with-segmentations notebook so it completes without crashing, which simply means picking a different arbitrary image to demonstrate rendering.)

I agree. Let's push this to 6.1 release and I will update the release notes and the graphical abstract to de-emphasize python stuff.

After using an arbitrary image that doesn't crash the viewer, it still crashes at 3 more places that use the mean shape image.
@cchriste

I propose we exclude this notebook from the release except in the documentation.

Works for me. @sheryjoe ?

pyvista showing logical coordinates
versus itkwidgets showing physical coordinate

Given issue #900, it's hard to imagine itk having ever showed physical coordinates.

For this image (with z-spacing = 2):

{
    dims: [93, 87, 94],
    origin: [-24, -19, -21],
    size: [93, 87, 188],
    spacing: [1, 1, 2]
}

The itk viewer can only show the unscaled version since otherwise it crashes.
Screen Shot 2021-03-30 at 3 57 34 PM

Here's a single pyvista volume showing both the unscaled ellipsoid and the scaled version.
Screen Shot 2021-03-30 at 3 56 20 PM

I don't really like the way the pyvista viewer shows bounds compared to the itk viewer.
But they're correct, and they're showing physical coordinates.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

jadie1 picture jadie1  ·  8Comments

akenmorris picture akenmorris  ·  32Comments

akenmorris picture akenmorris  ·  16Comments

sheryjoe picture sheryjoe  ·  13Comments

iyerkrithika21 picture iyerkrithika21  ·  12Comments