if offsets:
for i in offsets:
query_word = self._tokens[i]
# Find the context of query word.
left_context = self._tokens[i-context:i]
Когда первое вхождение поискового запроса находится в начале текста (например, со смещением 7), предположим, что параметр ширины установлен на 20, тогда [i- context: i ] будет оцениваться как [-13: 7] .
В этом случае, если текст состоит более чем из 20 слов, переменная left_context будет пустым списком, а не списком, содержащим первые 7 слов текста.
Подойдет простое исправление:
if offsets:
for i in offsets:
query_word = self._tokens[i]
# Find the context of query word.
if i - context < 0:
left_context = self._tokens[:i]
else:
left_context = self._tokens[i-context:i]
Не могли бы вы предоставить образец входных данных и желаемый результат, чтобы мы могли добавить их к регрессионному тесту?
Вход:
jane_eyre = 'Chapter 1\nTHERE was no possibility of taking a walk that day. We had been wandering, indeed, in the leafless shrubbery an hour in the morning; but since dinner (Mrs. Reed, when there was no company, dined early) the cold winter wind had brought with it clouds so sombre, and a rain so penetrating, that further outdoor exercise was now out of the question.'
text = nltk.Text(nltk.word_tokenize(jane_eyre))
text.concordance('taking')
text.concordance_list('taking')[0]
Выход (NLTK 3.3):
Displaying 1 of 1 matches:
taking a walk that day . We had been wander
ConcordanceLine(left=[],
query='taking',
right=['a', 'walk', 'that', 'day', '.', 'We', 'had', 'been', 'wandering', ',', 'indeed', ',', 'in', 'the', 'leafless', 'shrubbery', 'an', 'hour'],
offset=7,
left_print='',
right_print='a walk that day . We had been wande',
line=' taking a walk that day . We had been wande')
Желаемый результат:
Displaying 1 of 1 matches:
Chapter 1 THERE was no possibility of taking a walk that day . We had been wander
ConcordanceLine(left=['Chapter', '1', 'THERE', 'was', 'no', 'possibility', 'of'],
query='taking',
right=['a', 'walk', 'that', 'day', '.', 'We', 'had', 'been', 'wandering', ',', 'indeed', ',', 'in', 'the', 'leafless', 'shrubbery', 'an', 'hour'],
offset=7,
left_print='Chapter 1 THERE was no possibility of',
right_print='a walk that day . We had been wande',
line='Chapter 1 THERE was no possibility of taking a walk that day . We had been wande')
Спасибо @BLKSerene за сообщение об ошибке!
Ах, здесь есть отличное решение. Вместо if-else. Мы можем обрезать минимальную границу до max()
, например
left_context = self._tokens[max(0, i-context):i]
Было бы очень полезно добавить doctest в https://github.com/nltk/nltk/blob/develop/nltk/test/concordance.doctest для теста непрерывной интеграции / регрессии =)
Patching https://github.com/nltk/nltk/issues/2088
The left slice of the left context should be clip to 0 if the `i-context` < 0.
>>> from nltk import Text, word_tokenize
>>> jane_eyre = 'Chapter 1\nTHERE was no possibility of taking a walk that day. We had been wandering, indeed, in the leafless shrubbery an hour in the morning; but since dinner (Mrs. Reed, when there was no company, dined early) the cold winter wind had brought with it clouds so sombre, and a rain so penetrating, that further outdoor exercise was now out of the question.'
>>> text = Text(word_tokenize(jane_eyre))
>>> text.concordance_list('taking')[0].left
['Chapter', '1', 'THERE', 'was', 'no', 'possibility', 'of']
Исправлена # 2103. Спасибо @BLKSerene и @ dnc1994!