Xapian-haystack: ๊ธฐ๊ฐ์ด ๋๋ฌด ๊ธด ๋ฌธ์
When I run manage.py rebuild_index I get following error: manage.py rebuild_index ๋ฅผ ์คํํ๋ฉด ๋ค์ ์ค๋ฅ๊ฐ ๋ฐ์ํฉ๋๋ค.
```Traceback (most recent call last): ```์ถ์ (๊ฐ์ฅ ์ต๊ทผ ํธ์ถ ๋ง์ง๋ง):
File "./manage.py", line 11, in ํ์ผ "./manage.py", 11ํ,
execute_manager(settings) execute_manager(์ค์ )
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py", line 438, in execute_manager "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py" ํ์ผ, 438ํ, execute_manager
utility.execute() ์ ํธ๋ฆฌํฐ.execute()
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py", line 379, in execute "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py" ํ์ผ, 379ํ, ์คํ ์ค
self.fetch_command(subcommand).run_from_argv(self.argv) self.fetch_command(ํ์ ๋ช
๋ น).run_from_argv(self.argv)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv run_from_argv์์ ํ์ผ "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", 191ํ
self.execute(_args, *_options. dict ) self.execute(_args, *_options. dict )
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute ํ์ผ "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", ๋ผ์ธ 220, ์คํ ์ค
output = self.handle(_args, *_options) ์ถ๋ ฅ = self.handle(_args, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/rebuild_index.py", line 14, in handle ํ์ผ "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/rebuild_index.py", ์ค 14, ํธ๋ค
call_command('update_index', *_options) call_command('์
๋ฐ์ดํธ_์ธ๋ฑ์ค', *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py", line 166, in call_command call_command์์ ํ์ผ " /home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/init .py", 166ํ
return klass.execute(_args, *_defaults) ๋ฐํ klass.execute(_args, *_defaults)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute ํ์ผ "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", ๋ผ์ธ 220, ์คํ ์ค
output = self.handle(_args, *_options) ์ถ๋ ฅ = self.handle(_args, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", line 184, in handle ํ์ผ "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", ์ค 184, ํธ๋ค
return super(Command, self).handle(_apps, *_options) return super(๋ช
๋ น, ์๊ธฐ).handle(_apps, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 286, in handle ํ์ผ "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", 286ํ, ํธ๋ค
app_output = self.handle_app(app, *_options) app_output = self.handle_app(์ฑ, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", line 218, in handle_app ํ์ผ "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", 218ํ, handle_app
do_update(index, qs, start, end, total, self.verbosity) do_update(์ธ๋ฑ์ค, qs, ์์, ์ข
๋ฃ, ํฉ๊ณ, self.verbosity)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", line 100, in do_update ํ์ผ "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", do_update์ 100ํ
index.backend.update(index, current_qs) index.backend.update(์ธ๋ฑ์ค, current_qs)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/xapian_backend.py", line 257, in update ํ์ผ "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/xapian_backend.py", 257ํ, ์
๋ฐ์ดํธ ์ค
database.replace_document(document_id, document) database.replace_document(document_id, ๋ฌธ์)
xapian.InvalidArgumentError: Term too long (> 245): 4f6d6e6961206d6561206d6563756d20706f72746f202d20e2f1e520f1e2eee520edeef8f320f120f1eee1eefe0d0a566974612073696e65206c69626572746174652c206e6968696c202d20e6e8e7edfc20e1e5e720f1e2eee1eee4fb202d20ede8f7f2ee0d0a417273206c6f6e67612c207669746120627265766973 xapian.InvalidArgumentError: Term too long (> 245): 4f6d6e6961206d6561206d6563756d20706f72746f202d20e2f1e520f1e2eee520edeef8f320f120f1eee1eefe0d0a566974612073696e65206c69626572746174652c206e6968696c202d20e6e8e7edfc20e1e5e720f1e2eee1eee4fb202d20ede8f7f2ee0d0a417273206c6f6e67612c207669746120627265766973
Do you have any idea how to handle this problem?
Thanks.
P.S.
python 2.6
python-xapian (debian) 1.2.4-1
libxapian (debian) 1.2.5
recent versions of django, haystack-xapian, django-xapian
gpetukhov
๋ชจ๋ 3 ๋๊ธ
I've created quick fix, it just ignores such error and I think documents with invalid terms are not added to index completely. ๋น ๋ฅธ ์์ ์ ๋ง๋ค์์ต๋๋ค. ๊ทธ๋ฐ ์ค๋ฅ๋ฅผ ๋ฌด์ํ๊ณ ์๋ชป๋ ์ฉ์ด๊ฐ ์๋ ๋ฌธ์๊ฐ ์์ธ์ ์์ ํ ์ถ๊ฐ๋์ง ์์ ๊ฒ ๊ฐ์ต๋๋ค.
``` lorien@big :/tmp/xapian-haystack$ git diff ``` lorien@big :/tmp/xapian-haystack$ git diff
diff --git a/xapian_backend.py b/xapian_backend.py diff --git a/xapian_backend.py b/xapian_backend.py
index fbbe221..1884613 100755 ์ธ๋ฑ์ค fbbe221..1884613 100755
--- a/xapian_backend.py --- a/xapian_backend.py
+++ b/xapian_backend.py +++ b/xapian_backend.py
@@ -259,7 +259,10 @@ class SearchBackend(BaseSearchBackend): @@ -259,7 +259,10 @@ ํด๋์ค SearchBackend(BaseSearchBackend):
DOCUMENT_CT_TERM_PREFIX + u'%s.%s' % DOCUMENT_CT_TERM_PREFIX + u'%s.%s' %
(obj._meta.app_label, obj._meta.module_name) (obj._meta.app_label, obj._meta.module_name)
) )
- database.replace_document(document_id, document) database.replace_document(document_id, ๋ฌธ์)
- try: ๋ ธ๋ ฅํ๋ค:
- database.replace_document(document_id, document) database.replace_document(document_id, ๋ฌธ์)
- except xapian.InvalidArgumentError, ex: xapian.InvalidArgumentError ์ ์ธ, ์:
- ``` ```
sys.stderr.write('xapian.InvalidArgumentErrorn') sys.stderr.write('xapian.InvalidArgumentErrorn')
except UnicodeDecodeError: UnicodeDecodeError ์ ์ธ:
sys.stderr.write('Chunk failed.n') sys.stderr.write('์ฒญํฌ ์คํจ.n')
``` ```
This is actually raised by Xapian itself. ์ด๊ฒ์ ์ค์ ๋ก Xapian ์์ฒด์ ์ํด ์ ๊ธฐ๋ฉ๋๋ค. I've left the exception to bubble up so it's possible for a developer to see the issue. ๊ฐ๋ฐ์๊ฐ ๋ฌธ์ ๋ฅผ ๋ณผ ์ ์๋๋ก ์์ธ๋ฅผ ๊ฑฐํ์ผ๋ก ๋จ๊ฒจ๋์์ต๋๋ค. The solution is to ensure your terms are no longer than 245 characters, unfortunately. ํด๊ฒฐ์ฑ ์ ๋ถํํ๋ ์ฉ์ด๊ฐ 245์๋ฅผ ๋์ง ์๋๋ก ํ๋ ๊ฒ์ ๋๋ค.
en
notanumber
์ 2011๋
05์ 24์ผ
@notanumber @jorgecarleitao What do you think about this https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ? @notanumber @jorgecarleitao https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ์ ๋ํด ์ด๋ป๊ฒ ์๊ฐํ์ธ์?
the text document is usually more than 245, I know it's raised from xapian itself but this shouldn't break the [re]building the index. ํ ์คํธ ๋ฌธ์๋ ์ผ๋ฐ์ ์ผ๋ก 245๋ณด๋ค ํฌ๋ฉฐ xapian ์์ฒด์์ ๊ฐ์ ธ์จ ๊ฒ์ผ๋ก ์๊ณ ์์ง๋ง ์ธ๋ฑ์ค [์ฌ]๋น๋๋ฅผ ์ค๋จํด์๋ ์๋ฉ๋๋ค.
en
Alir3z4
์ 2015๋
11์ 10์ผ
๊ด๋ จ ๋ฌธ์
AndreaCrotti
ยท
3์ฝ๋ฉํธ
kodeshpa
ยท
3์ฝ๋ฉํธ
ipython
ยท
3์ฝ๋ฉํธ
ispmarin
ยท
3์ฝ๋ฉํธ
chuma
ยท
3์ฝ๋ฉํธ
๊ฐ์ฅ ์ ์ฉํ ๋๊ธ
@notanumber @jorgecarleitao What do you think about this https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ? @notanumber @jorgecarleitao https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ์ ๋ํด ์ด๋ป๊ฒ ์๊ฐํ์ธ์?
the text document is usually more than 245, I know it's raised from xapian itself but this shouldn't break the [re]building the index. ํ ์คํธ ๋ฌธ์๋ ์ผ๋ฐ์ ์ผ๋ก 245๋ณด๋ค ํฌ๋ฉฐ xapian ์์ฒด์์ ๊ฐ์ ธ์จ ๊ฒ์ผ๋ก ์๊ณ ์์ง๋ง ์ธ๋ฑ์ค [์ฌ]๋น๋๋ฅผ ์ค๋จํด์๋ ์๋ฉ๋๋ค.
en