Xapian-haystack: Begriff zu langes Problem
When I run manage.py rebuild_index I get following error: Wenn ich manage.py rebuild_index starte, bekomme ich folgende Fehlermeldung:
```Traceback (most recent call last): ```Traceback (letzter Aufruf zuletzt):
File "./manage.py", line 11, in Datei "./manage.py", Zeile 11, in
execute_manager(settings) execute_manager(einstellungen)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py", line 438, in execute_manager Datei „ /home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/init.py “, Zeile 438, in execute_manager
utility.execute() Utility.execute()
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py", line 379, in execute Datei „/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py“, Zeile 379, in Ausführung
self.fetch_command(subcommand).run_from_argv(self.argv) self.fetch_command(Unterbefehl).run_from_argv(self.argv)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 191, in run_from_argv Datei „/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py“, Zeile 191, in run_from_argv
self.execute(_args, *_options. dict ) self.execute(_args, * _options.dict )
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute Datei „/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py“, Zeile 220, in Ausführung
output = self.handle(_args, *_options) Ausgabe = self.handle(_args, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/rebuild_index.py", line 14, in handle Datei "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/rebuild_index.py", Zeile 14, im handle
call_command('update_index', *_options) call_command('update_index', *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py", line 166, in call_command Datei „/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/ init .py“, Zeile 166, in call_command
return klass.execute(_args, *_defaults) return klass.execute(_args, *_defaults)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 220, in execute Datei „/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py“, Zeile 220, in Ausführung
output = self.handle(_args, *_options) Ausgabe = self.handle(_args, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", line 184, in handle Datei „/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py“, Zeile 184, im Handle
return super(Command, self).handle(_apps, *_options) return super(Command, self).handle(_apps, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py", line 286, in handle Datei „/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/django/core/management/base.py“, Zeile 286, im Handle
app_output = self.handle_app(app, *_options) app_output = self.handle_app(app, *_options)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", line 218, in handle_app Datei „/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py“, Zeile 218, in handle_app
do_update(index, qs, start, end, total, self.verbosity) do_update(index, qs, start, end, total, self.ausführlichkeit)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py", line 100, in do_update Datei „/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/haystack/management/commands/update_index.py“, Zeile 100, in do_update
index.backend.update(index, current_qs) index.backend.update(index, aktuelle_qs)
File "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/xapian_backend.py", line 257, in update Datei "/home/lorien/web/dumpz/.env/lib/python2.6/site-packages/xapian_backend.py", Zeile 257, im Update
database.replace_document(document_id, document) database.replace_document (Dokument-ID, Dokument)
xapian.InvalidArgumentError: Term too long (> 245): 4f6d6e6961206d6561206d6563756d20706f72746f202d20e2f1e520f1e2eee520edeef8f320f120f1eee1eefe0d0a566974612073696e65206c69626572746174652c206e6968696c202d20e6e8e7edfc20e1e5e720f1e2eee1eee4fb202d20ede8f7f2ee0d0a417273206c6f6e67612c207669746120627265766973 xapian.InvalidArgumentError: Term too long (> 245): 4f6d6e6961206d6561206d6563756d20706f72746f202d20e2f1e520f1e2eee520edeef8f320f120f1eee1eefe0d0a566974612073696e65206c69626572746174652c206e6968696c202d20e6e8e7edfc20e1e5e720f1e2eee1eee4fb202d20ede8f7f2ee0d0a417273206c6f6e67612c207669746120627265766973
Do you have any idea how to handle this problem?
Thanks.
P.S.
python 2.6
python-xapian (debian) 1.2.4-1
libxapian (debian) 1.2.5
recent versions of django, haystack-xapian, django-xapian
gpetukhov
Alle 3 Kommentare
I've created quick fix, it just ignores such error and I think documents with invalid terms are not added to index completely. Ich habe eine schnelle Lösung erstellt, sie ignoriert solche Fehler einfach und ich denke, dass Dokumente mit ungültigen Begriffen nicht vollständig zum Index hinzugefügt werden.
``` lorien@big :/tmp/xapian-haystack$ git diff ``` lorien@big :/tmp/xapian-haystack$ git diff
diff --git a/xapian_backend.py b/xapian_backend.py diff --git a/xapian_backend.py b/xapian_backend.py
index fbbe221..1884613 100755 Index fbbe221..1884613 100755
--- a/xapian_backend.py --- a/xapian_backend.py
+++ b/xapian_backend.py +++ b/xapian_backend.py
@@ -259,7 +259,10 @@ class SearchBackend(BaseSearchBackend): @@ -259,7 +259,10 @@ Klasse SearchBackend(BaseSearchBackend):
DOCUMENT_CT_TERM_PREFIX + u'%s.%s' % DOCUMENT_CT_TERM_PREFIX + u'%s.%s' %
(obj._meta.app_label, obj._meta.module_name) (obj._meta.app_label, obj._meta.modul_name)
) )
- database.replace_document(document_id, document) database.replace_document (Dokument-ID, Dokument)
- try: Versuchen:
- database.replace_document(document_id, document) database.replace_document (Dokument-ID, Dokument)
- except xapian.InvalidArgumentError, ex: außer xapian.InvalidArgumentError, z. B.:
- ``` ```
sys.stderr.write('xapian.InvalidArgumentErrorn') sys.stderr.write('xapian.InvalidArgumentErrorn')
except UnicodeDecodeError: außer UnicodeDecodeError:
sys.stderr.write('Chunk failed.n') sys.stderr.write('Chunk fehlgeschlagen.n')
``` ```
This is actually raised by Xapian itself. Dies wird tatsächlich von Xapian selbst ausgelöst. I've left the exception to bubble up so it's possible for a developer to see the issue. Ich habe die Ausnahme sprudeln lassen, damit ein Entwickler das Problem sehen kann. The solution is to ensure your terms are no longer than 245 characters, unfortunately. Die Lösung besteht leider darin, sicherzustellen, dass Ihre Begriffe nicht länger als 245 Zeichen sind.
en
notanumber
am 24. Mai 2011
@notanumber @jorgecarleitao What do you think about this https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ? @notanumber @jorgecarleitao Was denkst du darüber https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ?
the text document is usually more than 245, I know it's raised from xapian itself but this shouldn't break the [re]building the index. Das Textdokument ist normalerweise größer als 245, ich weiß, dass es von xapian selbst ausgelöst wurde, aber dies sollte den [Neu-] Aufbau des Index nicht unterbrechen.
en
Alir3z4
am 10. Nov. 2015
Verwandte Themen
nvie
·
3Kommentare
mw44118
·
3Kommentare
DarwinSurvivor
·
3Kommentare
maanbsat
·
3Kommentare
ispmarin
·
3Kommentare
Hilfreichster Kommentar
@notanumber @jorgecarleitao What do you think about this https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ? @notanumber @jorgecarleitao Was denkst du darüber https://github.com/Alir3z4/xapian-haystack/commit/a249b46c48957f4d8a776ef41b0ce12490ad52dd ?
the text document is usually more than 245, I know it's raised from xapian itself but this shouldn't break the [re]building the index. Das Textdokument ist normalerweise größer als 245, ich weiß, dass es von xapian selbst ausgelöst wurde, aber dies sollte den [Neu-] Aufbau des Index nicht unterbrechen.
en