Libvips: segfault in python or libvips during approximate convolution of a short or ushort image

Hi John,

I was experimenting with the approximate convolution you mentioned in #1160 and I came across a persistent segfault but only when convolving ushort images. I can make the segfault happen either in python or libvips depending on the size of the image, so I suppose this is might be a double free. Here is a test program with some experiments to see where the problem happens. Below that is a stack trace of the segfault location inside libvips.so.42.9.0 (without using a debugging build at the moment, so maybe not so helpful).

I'm running vips vips-8.7.0-Sun Jan 28 17:23:05 UTC 2018 using the pyvips interface under python Python 2.7.15 :: Anaconda custom (64-bit)

Thanks in advance.

Note: actually a .py file:
segfault.txt

# /usr/bin/env python
import pyvips

slab = pyvips.Image.new_from_array([[1]*10]*10, scale=100)  # 10x10 moving average

n = 30000 # also segfaults when n=30000.  See note below.

# The following are all fine
uchar_im = pyvips.Image.black(n, n).cast('uchar')  # just to be sure it's uchar
uchar_im.conv(slab, precision='approximate', layers=1).write_to_file('uchar_conv.v')

float_im = pyvips.Image.black(n, n).cast('float')
float_im.conv(slab, precision='approximate', layers=1).write_to_file('float_conv.v')


# EVERY CONVOLUTION BELOW THIS LINE SEGFAULTS

# if n=1, completes by writing result to file, then segfaults in python:
# *** Error in `python': free(): invalid pointer: 0x00005651d79154c0 ***

# if n=30000, does not complete and segfaults in libvips
# [1049087.739802] worker[599]: segfault at 7f4cd631a000 ip 00007f4ce7f93318 sp 00007f4cd2caea70 error 4 in libvips.so.42.9.0[7f4ce7eb6000+1f5000]

ushort_im = pyvips.Image.black(n, n).cast('ushort')
ushort_im.conv(slab.cast('ushort'), precision='approximate', layers=1).write_to_file('ushort_conv.v')  # segfault happens here

ushort_im = pyvips.Image.black(n, n).cast('ushort')
ushort_im.conv(slab, precision='approximate', layers=1).write_to_file('ushort_conv.v')  # segfault would also happen here

short_im = pyvips.Image.black(n, n).cast('short')
short_im.conv(slab, precision='approximate', layers=1).write_to_file('short_conv.v')  # segfault would also happen here

Possibly relevant stack trace when n=30000:

Thread 52 "worker" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe64d5700 (LWP 4069)]
0x00007ffff4ab4c7d in ?? () from /usr/lib/libvips.so.42
(gdb) up
#1  0x00007ffff4b4e411 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#2  0x00007ffff4b4f0c9 in vips_region_fill () from /usr/lib/libvips.so.42
(gdb) up
#3  0x00007ffff4b50d53 in vips_region_prepare () from /usr/lib/libvips.so.42
(gdb) up
#4  0x00007ffff4ab65b3 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#5  0x00007ffff4b4e411 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#6  0x00007ffff4b4f0c9 in vips_region_fill () from /usr/lib/libvips.so.42
(gdb) up
#7  0x00007ffff4b50d53 in vips_region_prepare () from /usr/lib/libvips.so.42
(gdb) up
#8  0x00007ffff4b40d59 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#9  0x00007ffff4b4e411 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#10 0x00007ffff4b4f0c9 in vips_region_fill () from /usr/lib/libvips.so.42
(gdb) up
#11 0x00007ffff4b50d53 in vips_region_prepare () from /usr/lib/libvips.so.42
(gdb) up
#12 0x00007ffff4b40d59 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#13 0x00007ffff4b4e411 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#14 0x00007ffff4b4f0c9 in vips_region_fill () from /usr/lib/libvips.so.42
(gdb) up
#15 0x00007ffff4b50d53 in vips_region_prepare () from /usr/lib/libvips.so.42
(gdb) up
#16 0x00007ffff4b40d59 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#17 0x00007ffff4b4e411 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#18 0x00007ffff4b4f615 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#19 0x00007ffff4b50f5f in vips_region_prepare_to () from /usr/lib/libvips.so.42
(gdb) up
#20 0x00007ffff4b48228 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#21 0x00007ffff4b5171b in ?? () from /usr/lib/libvips.so.42
(gdb) up
#22 0x00007ffff4b51504 in ?? () from /usr/lib/libvips.so.42
(gdb) up
#23 0x00007ffff44e0a8a in g_thread_proxy (data=0x5555559df720) from /opt/anaconda/lib/libglib-2.0.so.0
(gdb) up
#24 0x00007ffff77e36ba in start_thread (arg=0x7fffe64d5700) at pthread_create.c:333
333 pthread_create.c: No such file or directory.