Xapian-haystack: Masalah jangka terlalu panjang

Dibuat pada 22 Mei 2011 · 3Komentar · Sumber: notanumber/xapian-haystack

When I run manage.py rebuild_index I get following error: Ketika saya menjalankan manage.py rebuild_index saya mendapatkan kesalahan berikut:

```Traceback (most recent call last): ```Traceback (panggilan terakhir terakhir):
File "./manage.py", line 11, in File "./manage.py", baris 11, di
execute_manager(settings) eksekusi_manager(pengaturan)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py", line 438, in execute_manager File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/Django/core/management/ init .py", baris 438, di execute_manager
utility.execute() utilitas.jalankan()
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py", line 379, in execute File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/Django/core/management/ init .py", baris 379, dalam eksekusi
self.fetch_command(subcommand).run_from_argv(self.argv) self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/Django/core/management/base.py", baris 191, di run_from_argv
self.execute(_args, *_options. dict ) self.execute(_args, *_options. dict )
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/Django/core/management/base.py", baris 220, dalam eksekusi
output = self.handle(_args, *_options) output = self.handle(_args, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/rebuild_index.py", line 14, in handle File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/rebuild_index.py", baris 14, di pegangan
call_command('update_index', *_options) call_command('update_index', *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py", line 166, in call_command File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/Django/core/management/ init .py", baris 166, di call_command
return klass.execute(_args, *_defaults) kembalikan klass.execute(_args, *_defaults)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/Django/core/management/base.py", baris 220, dalam eksekusi
output = self.handle(_args, *_options) output = self.handle(_args, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", line 184, in handle File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", baris 184, di pegangan
return super(Command, self).handle(_apps, *_options) kembalikan super(Command, self).handle(_apps, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 286, in handle File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/Django/core/management/base.py", baris 286, dalam pegangan
app_output = self.handle_app(app, *_options) app_output = self.handle_app(aplikasi, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", line 218, in handle_app File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", baris 218, di handle_app
do_update(index, qs, start, end, total, self.verbosity) do_update(indeks, qs, awal, akhir, total, self.verbosity)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", line 100, in do_update File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", baris 100, di do_update
index.backend.update(index, current_qs) index.backend.update(indeks, current_qs)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/xapian_backend.py", line 257, in update File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/xapian_backend.py", baris 257, dalam pembaruan
database.replace_document(document_id, document) database.replace_document(id_dokumen, dokumen)
xapian.InvalidArgumentError: Term too long (> 245): 4f6d6e6961206d6561206d6563756d20706f72746f202d20e2f1e520f1e2eee520edeef8f320f120f1eee1eefe0d0a566974612073696e65206c69626572746174652c206e6968696c202d20e6e8e7edfc20e1e5e720f1e2eee1eee4fb202d20ede8f7f2ee0d0a417273206c6f6e67612c207669746120627265766973 xapian.InvalidArgumentError: Term too long (> 245): 4f6d6e6961206d6561206d6563756d20706f72746f202d20e2f1e520f1e2eee520edeef8f320f120f1eee1eefe0d0a566974612073696e65206c69626572746174652c206e6968696c202d20e6e8e7edfc20e1e5e720f1e2eee1eee4fb202d20ede8f7f2ee0d0a417273206c6f6e67612c207669746120627265766973

Do you have any idea how to handle this problem?

Thanks.

P.S.
python 2.6
python-xapian (debian) 1.2.4-1
libxapian (debian) 1.2.5
recent versions of django, haystack-xapian, django-xapian

Sumber

gpetukhov

Komentar yang paling membantu

@notanumber @jorgecarleitao What do you think about this https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ? @notanumber @jorgecarleitao Apa pendapat Anda tentang ini https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ?

the text document is usually more than 245, I know it's raised from xapian itself but this shouldn't break the [re]building the index. dokumen teks biasanya lebih dari 245, saya tahu itu dibangkitkan dari xapian itu sendiri tetapi ini seharusnya tidak merusak [kembali] membangun indeks.

Alir3z4 pada 10 Nov 2015

👍3

Semua 3 komentar

I've created quick fix, it just ignores such error and I think documents with invalid terms are not added to index completely. Saya telah membuat perbaikan cepat, itu hanya mengabaikan kesalahan seperti itu dan saya pikir dokumen dengan istilah yang tidak valid tidak ditambahkan ke indeks sepenuhnya.

``` lorien@big :/tmp/xapian-haystack$ git diff ``` lorien@big :/tmp/xapian-haystack$ git diff
diff --git a/xapian_backend.py b/xapian_backend.py diff --git a/xapian_backend.py b/xapian_backend.py
index fbbe221..1884613 100755 indeks fbbe221..1884613 100755
--- a/xapian_backend.py --- a/xapian_backend.py
+++ b/xapian_backend.py +++ b/xapian_backend.py
@@ -259,7 +259,10 @@ class SearchBackend(BaseSearchBackend): @@ -259,7 +259,10 @@ kelas SearchBackend(BaseSearchBackend):
DOCUMENT_CT_TERM_PREFIX + u'%s.%s' % DOCUMENT_CT_TERM_PREFIX + u'%s.%s' %
(obj._meta.app_label, obj._meta.module_name) (obj._meta.app_label, obj._meta.module_name)
) )

database.replace_document(document_id, document) database.replace_document(id_dokumen, dokumen)
try: mencoba:
database.replace_document(document_id, document) database.replace_document(id_dokumen, dokumen)
except xapian.InvalidArgumentError, ex: kecuali xapian.InvalidArgumentError, mis:
``` ```
sys.stderr.write('xapian.InvalidArgumentErrorn') sys.stderr.write('xapian.InvalidArgumentErrorn')

except UnicodeDecodeError: kecuali UnicodeDecodeError:
sys.stderr.write('Chunk failed.n') sys.stderr.write('Potongan gagal.n')
``` ```

gpetukhov pada 22 Mei 2011

This is actually raised by Xapian itself. Hal inilah yang sebenarnya dimunculkan oleh Xapian sendiri. I've left the exception to bubble up so it's possible for a developer to see the issue. Saya membiarkan pengecualian muncul sehingga pengembang dapat melihat masalahnya. The solution is to ensure your terms are no longer than 245 characters, unfortunately. Solusinya adalah untuk memastikan istilah Anda tidak lebih dari 245 karakter, sayangnya.