Sorl-thumbnail: Thumbnail not linked to source image in key value store
I observed following behavior:
My page has two calls to the {% thumbnail %} template tag each requesting a thumbnail of the same source image in another size. Like this:
{% thumbnail user.get_profile.avatar_image "46x46" crop="center" as im %} .. {% endthumbnail %}
{% thumbnail user_profile.avatar_image "187x187" crop="center" as im %} .. {% endthumbnail %}
(the calls are made from different Django templates)
If sorl-thumbnail can not find the thumbnails it will create them as expected. But one of the thumbnails is not referenced in the sorl-thumbnail||thumbnails||xxx entry in the key value store.
After the page is rendered the key value store looks like this:
# select * from thumbnail_kvstore;
key | value
--------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------
sorl-thumbnail||image||3314722c993608c1dab30949d2504172 | {"storage": "outdoorish.storage.OverwriteStorage", "name": "profiles/avatars/2205.jpg", "size": [300, 423]}
sorl-thumbnail||image||4d545ece8fb09c9176996000cddc0fac | {"storage": "django.core.files.storage.FileSystemStorage", "name": "cache/7c/ab/7cab89bcadad6122d441e6d5443fccc2.jpg", "size": [187, 187]}
sorl-thumbnail||thumbnails||3314722c993608c1dab30949d2504172 | ["4d545ece8fb09c9176996000cddc0fac"]
sorl-thumbnail||image||256cd182b71debc11e8aae0ed8ee1a9d | {"storage": "django.core.files.storage.FileSystemStorage", "name": "cache/b2/71/b27103adff0bdbba7d05fc98a51be131.jpg", "size": [46, 46]}
(4 rows)
Why is one of the images not connected to the source file? Is this behavior intentional? (Because I think this is causing thumbnails not being deleted when calling sorl.thumbnail.delete.)
I am using sorl-thumbnail==11.12.1b
This is the Django log of creation of the thumbnails:
Log getting/creating 1st thumbnail:
2014-10-14 14:58:35,131 DEBUG base Getting thumbnail for file [profiles/avatars/2205.jpg] at [187x187] | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/sorl/thumbnail/base.py:67 (get_thumbnail), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,182 DEBUG util (0.001) SELECT "thumbnail_kvstore"."key", "thumbnail_kvstore"."value" FROM "thumbnail_kvstore" WHERE "thumbnail_kvstore"."key" = 'sorl-thumbnail||image||4d545ece8fb09c9176996000cddc0fac' ; args=(u'sorl-thumbnail||image||4d545ece8fb09c9176996000cddc0fac',) | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,210 DEBUG base Creating thumbnail file [cache/7c/ab/7cab89bcadad6122d441e6d5443fccc2.jpg] at [187x187] with [{'rounded': None, 'padding_color': '#ffffff', 'format': 'JPEG', 'colorspace': 'RGB', 'cropbox': None, 'padding': False, 'upscale': True, 'crop': u'center', 'image_info': {'jfif_version': (1, 1), 'jfif': 257, 'jfif_unit': 1, 'jfif_density': (72, 72), 'dpi': (72, 72)}, 'quality': 95}] | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/sorl/thumbnail/base.py:148 (_create_thumbnail), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,248 DEBUG util (0.001) SELECT "thumbnail_kvstore"."key", "thumbnail_kvstore"."value" FROM "thumbnail_kvstore" WHERE "thumbnail_kvstore"."key" = 'sorl-thumbnail||image||3314722c993608c1dab30949d2504172' ; args=(u'sorl-thumbnail||image||3314722c993608c1dab30949d2504172',) | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,273 DEBUG util (0.001) SELECT "thumbnail_kvstore"."key", "thumbnail_kvstore"."value" FROM "thumbnail_kvstore" WHERE "thumbnail_kvstore"."key" = 'sorl-thumbnail||image||3314722c993608c1dab30949d2504172' ; args=(u'sorl-thumbnail||image||3314722c993608c1dab30949d2504172',) | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,308 DEBUG util (0.001) INSERT INTO "thumbnail_kvstore" ("key", "value") VALUES ('sorl-thumbnail||image||3314722c993608c1dab30949d2504172', '{"storage": "outdoorish.storage.OverwriteStorage", "name": "profiles/avatars/2205.jpg", "size": [300, 423]}'); args=(u'sorl-thumbnail||image||3314722c993608c1dab30949d2504172', '{"storage": "outdoorish.storage.OverwriteStorage", "name": "profiles/avatars/2205.jpg", "size": [300, 423]}') | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,345 DEBUG util (0.001) SELECT "thumbnail_kvstore"."key", "thumbnail_kvstore"."value" FROM "thumbnail_kvstore" WHERE "thumbnail_kvstore"."key" = 'sorl-thumbnail||image||4d545ece8fb09c9176996000cddc0fac' ; args=(u'sorl-thumbnail||image||4d545ece8fb09c9176996000cddc0fac',) | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,371 DEBUG util (0.000) INSERT INTO "thumbnail_kvstore" ("key", "value") VALUES ('sorl-thumbnail||image||4d545ece8fb09c9176996000cddc0fac', '{"storage": "django.core.files.storage.FileSystemStorage", "name": "cache/7c/ab/7cab89bcadad6122d441e6d5443fccc2.jpg", "size": [187, 187]}'); args=(u'sorl-thumbnail||image||4d545ece8fb09c9176996000cddc0fac', '{"storage": "django.core.files.storage.FileSystemStorage", "name": "cache/7c/ab/7cab89bcadad6122d441e6d5443fccc2.jpg", "size": [187, 187]}') | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,426 DEBUG util (0.001) SELECT "thumbnail_kvstore"."key", "thumbnail_kvstore"."value" FROM "thumbnail_kvstore" WHERE "thumbnail_kvstore"."key" = 'sorl-thumbnail||thumbnails||3314722c993608c1dab30949d2504172' ; args=(u'sorl-thumbnail||thumbnails||3314722c993608c1dab30949d2504172',) | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,449 DEBUG util (0.001) SELECT "thumbnail_kvstore"."key", "thumbnail_kvstore"."value" FROM "thumbnail_kvstore" WHERE "thumbnail_kvstore"."key" = 'sorl-thumbnail||thumbnails||3314722c993608c1dab30949d2504172' ; args=(u'sorl-thumbnail||thumbnails||3314722c993608c1dab30949d2504172',) | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412452124416
2014-10-14 14:58:35,472 DEBUG util (0.000) INSERT INTO "thumbnail_kvstore" ("key", "value") VALUES ('sorl-thumbnail||thumbnails||3314722c993608c1dab30949d2504172', '["4d545ece8fb09c9176996000cddc0fac"]'); args=(u'sorl-thumbnail||thumbnails||3314722c993608c1dab30949d2504172', '["4d545ece8fb09c9176996000cddc0fac"]') | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412452124416
Log getting/creating 2nd thumbnail:
2014-10-14 14:58:37,314 DEBUG base Getting thumbnail for file [profiles/avatars/2205.jpg] at [46x46] | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/sorl/thumbnail/base.py:67 (get_thumbnail), PID:8096 Thread:140412426946304
2014-10-14 14:58:37,333 DEBUG util (0.001) SELECT "thumbnail_kvstore"."key", "thumbnail_kvstore"."value" FROM "thumbnail_kvstore" WHERE "thumbnail_kvstore"."key" = 'sorl-thumbnail||image||256cd182b71debc11e8aae0ed8ee1a9d' ; args=(u'sorl-thumbnail||image||256cd182b71debc11e8aae0ed8ee1a9d',) | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412426946304
2014-10-14 14:58:37,345 DEBUG base Creating thumbnail file [cache/b2/71/b27103adff0bdbba7d05fc98a51be131.jpg] at [46x46] with [{'rounded': None, 'padding_color': '#ffffff', 'format': 'JPEG', 'colorspace': 'RGB', 'cropbox': None, 'padding': False, 'upscale': True, 'crop': u'center', 'image_info': {'jfif_version': (1, 1), 'jfif': 257, 'jfif_unit': 1, 'jfif_density': (72, 72), 'dpi': (72, 72)}, 'quality': 95}] | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/sorl/thumbnail/base.py:148 (_create_thumbnail), PID:8096 Thread:140412426946304
2014-10-14 14:58:37,386 DEBUG util (0.001) SELECT "thumbnail_kvstore"."key", "thumbnail_kvstore"."value" FROM "thumbnail_kvstore" WHERE "thumbnail_kvstore"."key" = 'sorl-thumbnail||image||256cd182b71debc11e8aae0ed8ee1a9d' ; args=(u'sorl-thumbnail||image||256cd182b71debc11e8aae0ed8ee1a9d',) | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412426946304
2014-10-14 14:58:37,390 DEBUG util (0.001) INSERT INTO "thumbnail_kvstore" ("key", "value") VALUES ('sorl-thumbnail||image||256cd182b71debc11e8aae0ed8ee1a9d', '{"storage": "django.core.files.storage.FileSystemStorage", "name": "cache/b2/71/b27103adff0bdbba7d05fc98a51be131.jpg", "size": [46, 46]}'); args=(u'sorl-thumbnail||image||256cd182b71debc11e8aae0ed8ee1a9d', '{"storage": "django.core.files.storage.FileSystemStorage", "name": "cache/b2/71/b27103adff0bdbba7d05fc98a51be131.jpg", "size": [46, 46]}') | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412426946304
2014-10-14 14:58:37,429 DEBUG util (0.001) SELECT "thumbnail_kvstore"."key", "thumbnail_kvstore"."value" FROM "thumbnail_kvstore" WHERE "thumbnail_kvstore"."key" = 'sorl-thumbnail||thumbnails||3314722c993608c1dab30949d2504172' ; args=(u'sorl-thumbnail||thumbnails||3314722c993608c1dab30949d2504172',) | File:/home/anton/.virtualenvs/outdoorish/lib/python2.7/site-packages/django/db/backends/util.py:50 (execute), PID:8096 Thread:140412426946304
antonpirker
All 3 comments
Are there any updates on this? Can anyone explain why one of the images is not connected to the source file in the key value store?
I would love to help fix this, but I need a pointer on where to start!
antonpirker
on 5 Nov 2014
I digged a little in the code and it seems there is a problem with the function KVStore._set_raw() (in cached_db_kvstore.py) (Link to code line: https://github.com/mariocesar/sorl-thumbnail/blob/master/sorl/thumbnail/kvstores/cached_db_kvstore.py#L43 )
This is the situation:
The _set_raw() function is called with these parameters (A):
key: thumbnail||thumbnails||3314722c993608c1dab30949d2504172
value: ["4d545ece8fb09c9176996000cddc0fac", "256cd182b71debc11e8aae0ed8ee1a9d"]
There is already an entry in the KV store (Postgres database) with these parameters (B):
key: thumbnail||thumbnails||3314722c993608c1dab30949d2504172
value: ["4d545ece8fb09c9176996000cddc0fac"]
This is the problem:
Now the _set_raw() makes a call to KVStoreModel.objects.get_or_create() and fetches the entry from the database ((A), the one where one thumbnail is referenced). After this it saves the entry with the references to two thumbnails (B) to the cache (which is a django.core.cache.backends.locmem.LocMemCache).
The new version of the entry (B) is never saved to the database. If I change the image only one of the thumbnails is updated, the other one stays the same, because it is not referenced by the thumbnail||thumbnails||3314722c993608c1dab30949d2504172 entry.
I hope I have described the problem so one of the maintainers can reproduce the problem. Would be great if you could help me fixing this! Thank's!
antonpirker
on 5 Nov 2014
👍4
I`m not alone :) Please, merge this fix, it is very necessary!
TyVik
on 14 Oct 2016
Was this page helpful?
0 / 5 - 0 ratings
Related issues
philgyford
·
16Comments
stopdesign
·
7Comments
jlariza
·
8Comments
nuschk
·
7Comments
ghost
·
3Comments
Most helpful comment
I digged a little in the code and it seems there is a problem with the function
KVStore._set_raw()(incached_db_kvstore.py) (Link to code line: https://github.com/mariocesar/sorl-thumbnail/blob/master/sorl/thumbnail/kvstores/cached_db_kvstore.py#L43 )This is the situation:
The
_set_raw()function is called with these parameters (A):There is already an entry in the KV store (Postgres database) with these parameters (B):
This is the problem:
Now the
_set_raw()makes a call toKVStoreModel.objects.get_or_create()and fetches the entry from the database ((A), the one where one thumbnail is referenced). After this it saves the entry with the references to two thumbnails (B) to the cache (which is adjango.core.cache.backends.locmem.LocMemCache).The new version of the entry (B) is never saved to the database. If I change the image only one of the thumbnails is updated, the other one stays the same, because it is not referenced by the
thumbnail||thumbnails||3314722c993608c1dab30949d2504172entry.I hope I have described the problem so one of the maintainers can reproduce the problem. Would be great if you could help me fixing this! Thank's!