Nltk: "IndexError: string index out of range" on trying to stem the word "oing"

Created on 8 Feb 2017  ·  5Comments  ·  Source: nltk/nltk

Easy to reproduce:

>>> from nltk import PorterStemmer
>>> stemmer = PorterStemmer()
>>> stemmer.stem('oing')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/peterbe/virtualenvs/songsearch/lib/python3.5/site-packages/nltk/stem/porter.py", line 665, in stem
    stem = self._step1b(stem)
  File "/Users/peterbe/virtualenvs/songsearch/lib/python3.5/site-packages/nltk/stem/porter.py", line 376, in _step1b
    lambda stem: (self._measure(stem) == 1 and
  File "/Users/peterbe/virtualenvs/songsearch/lib/python3.5/site-packages/nltk/stem/porter.py", line 258, in _apply_rule_list
    if suffix == '*d' and self._ends_double_consonant(word):
  File "/Users/peterbe/virtualenvs/songsearch/lib/python3.5/site-packages/nltk/stem/porter.py", line 214, in _ends_double_consonant
    word[-1] == word[-2] and
IndexError: string index out of range
>>> import nltk
>>> nltk.__version__
'3.2.2'
bug pleaseverify stelemma

Most helpful comment

So, has the issue been resolved?

All 5 comments

I got this error for the word "aed":

from nltk.stem.porter import PorterStemmer
from nltk.corpus import stopwords
stemmer = PorterStemmer()
stemmer.stem('aed')

Error is:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/peter.hurford/.virtualenvs/rex/lib/python2.7/site-packages/nltk/stem/porter.py", line 665, in stem
    stem = self._step1b(stem)
  File "/Users/peter.hurford/.virtualenvs/rex/lib/python2.7/site-packages/nltk/stem/porter.py", line 376, in _step1b
    lambda stem: (self._measure(stem) == 1 and
  File "/Users/peter.hurford/.virtualenvs/rex/lib/python2.7/site-packages/nltk/stem/porter.py", line 258, in _apply_rule_list
    if suffix == '*d' and self._ends_double_consonant(word):
  File "/Users/peter.hurford/.virtualenvs/rex/lib/python2.7/site-packages/nltk/stem/porter.py", line 214, in _ends_double_consonant
    word[-1] == word[-2] and
IndexError: string index out of range

Installed with:

pip install nltk
python -m nltk.downloader -d

Version:

import nltk
nltk.__version__ # '3.2.2'

Duplicate of https://github.com/nltk/nltk/issues/1581. My fault; sorry. :(

The bug was introduced in version 3.2.2 and is fixed on master; you can either use develop or version 3.2.1 to get rid of the bug.

Close as fixed?

So, has the issue been resolved?

This issue should have been resolved by #1582 😉

>>> import nltk
>>> nltk.__version__
'3.2.5'

>>> from nltk import PorterStemmer
>>> porter = PorterStemmer()
>>> porter.stem('oing')
u'o'

Was this page helpful?
0 / 5 - 0 ratings