์ฌ์ฉ์๊ฐ์ด ๋ฌธ์ฅ์ ๋ํด NLTK์์ Stanford์ DependencyParser
API๋ฅผ ์ฌ์ฉํ ๋์ด ๋ฌธ์ฅ์ด ๋์ง๊ณ AssertionError
๊ฐ ๋ฐ์ํ๋ค๊ณ ๋ณด๊ณ ํ์ต๋๋ค.
์ญ๋ ์ถ์ ๊ฟ์ ์ธ๊ณ์ ๋ํ ๋ชจ๋ ํต์ฐฐ๋ ฅ๊ณผ ์ฌ์ด๋ฒ ๋ฌธํ๋ฅผ ํตํ ์ ์์ ํํ์ผ๋ก ์ธํด ์ํ๋ 2 ์๊ฐ 30 ๋ถ์ ์์ ์๊ฐ์์ ์ผ๊ด๋ ์ด์ผ๊ธฐ๋ฅผ ์ด๋์ด ๋ด๊ณ ์ํ๋ ์ฌ๋์๊ฒ ๋ถ๊ธฐ๋ฅผ์ฃผ์ง ์์ต๋๋ค.
์ํธ:
>>> from nltk.parse.stanford import StanfordDependencyParser >>> dep_parser=StanfordDependencyParser(model_path="edu/stanford/nlp/models/lexparser/englishPCFG.ser.gz")
>>> sent = 'for all of its insights into the dream world of teen life , and its electronic expression through cyber culture , the film gives no quarter to anyone seeking to pull a cohesive story out of its 2 1/2-hour running time . '
>>> dep_parser.raw_parse(sent)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Python/2.7/site-packages/nltk/parse/stanford.py", line 132, in raw_parse
return next(self.raw_parse_sents([sentence], verbose))
File "/Library/Python/2.7/site-packages/nltk/parse/stanford.py", line 150, in raw_parse_sents
return self._parse_trees_output(self._execute(cmd, '\n'.join(sentences), verbose))
File "/Library/Python/2.7/site-packages/nltk/parse/stanford.py", line 91, in _parse_trees_output
res.append(iter([self._make_tree('\n'.join(cur_lines))]))
File "/Library/Python/2.7/site-packages/nltk/parse/stanford.py", line 339, in _make_tree
return DependencyGraph(result, top_relation_label='root')
File "/Library/Python/2.7/site-packages/nltk/parse/dependencygraph.py", line 84, in __init__
top_relation_label=top_relation_label,
File "/Library/Python/2.7/site-packages/nltk/parse/dependencygraph.py", line 328, in _parse
assert cell_number == len(cells)
AssertionError
์๋ง๋ DependencyGraph
๊ฐ ์ถ๋ ฅ์ ์ฝ๋ ๋ฐฉ๋ฒ์ด๊ฑฐ๋ Stanford ์ถ๋ ฅ์ด ์ผ์นํ์ง ์์ ์ ์์ต๋๋ค.
NLTK + Stanford ๋๊ตฌ ์ค์ ์ ๋ํ ์์ธํ ๋ด์ฉ์ https://gist.github.com/alvations/e1df0ba227e542955a8a#stanford -parser์ ์์ต๋๋ค.
์๋
ํ์ธ์ @alvations ์ด๊ฒ์ ๋ํ ์
๋ฐ์ดํธ๊ฐ ์์ต๋๊น?
๊ฐ์ฌ
@ hoavt-54 # 1249์ ์ ์ธํฐํ์ด์ค๋ฅผ ์ฌ์ฉํ์ฌ ๋ฌธ์ ๋ฅผ ์ผ์ผํค๋ ์คํ ํฌ๋ ์ธก์ธ์ง DependencyGraph
์ฝ๋์ธ์ง ๋น ๋ฅด๊ฒ ํ์ธํ ์์๋ ๋ฐฉ๋ฒ์ด ์๋ค๊ณ ์๊ฐํฉ๋๋ค. ์ค๋์ ์กฐ๊ธ ๋ฐ์ ๊ฒ ๊ฐ์ง๋ง ๋ค๋ฅธ ์ฌ๋์ด ํ์ธํ๊ณ ๋ค์ ํ ์์์ ๊ฒ์
๋๋ค.
์ด ๋ฌธ์ ๋ฅผ ๋์ณค์ต๋๋ค.
@dimazest ์๋ ํ์ธ์, ๋ฐฉ๊ธ์ด ์ค๋ฅ๊ฐ ๋ฐ์ํ์ต๋๋ค. ์ด ๋ฌธ์ ๋ฅผ ์ด๋ป๊ฒ ํด๊ฒฐํด์ผํฉ๋๊น?
@tesslocl ๋น์ ์ ๋ฌธ์ฅ์ ๋ฌด์์ ๋๊น? ๋์ CoreNLP (nltk / parse / corenlp.py)๋ฅผ ์ฌ์ฉํ๋ ค๊ณ ํ์ต๋๊น?
@dimazest ๋ฐฉ๊ธํ๊ณ ๋ค๋ฅธ ์ค๋ฅ๊ฐ ๋ฐ์ํ์ต๋๋ค.
Traceback (most recent call last):
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 386, in _make_request
six.raise_from(e, None)
File "<string>", line 2, in raise_from
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 382, in _make_request
httplib_response = conn.getresponse()
File "C:\Users\Admin\Anaconda3\lib\http\client.py", line 1198, in getresponse
response.begin()
File "C:\Users\Admin\Anaconda3\lib\http\client.py", line 297, in begin
version, status, reason = self._read_status()
File "C:\Users\Admin\Anaconda3\lib\http\client.py", line 258, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "C:\Users\Admin\Anaconda3\lib\socket.py", line 576, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\adapters.py", line 423, in send
timeout=timeout
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 649, in urlopen
_stacktrace=sys.exc_info()[2])
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\packages\urllib3\util\retry.py", line 347, in increment
raise six.reraise(type(error), error, _stacktrace)
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\packages\urllib3\packages\six.py", line 686, in reraise
raise value
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 600, in urlopen
chunked=chunked)
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 388, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 308, in _raise_timeout
raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
requests.packages.urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9000): Read timed out. (read timeout=60)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\classifier\feature_extraction.py", line 473, in <module>
print(feature_extraction(test_file_id))
File "E:\classifier\feature_extraction.py", line 146, in feature_extraction
for line in dep_parse:
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 279, in raw_parse_sents
parsed_data = self.api_call(sentence, properties=default_properties)
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 247, in api_call
timeout=60,
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\sessions.py", line 535, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Admin\Anaconda3\lib\site-packages\requests\adapters.py", line 499, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='localhost', port=9000): Read timed out. (read timeout=60)
StanfordDependencyParser ๋ถ๋ถ ๋ง ๋ณ๊ฒฝํ๊ณ ๋๋จธ์ง ์ฝ๋๋ ๋ณ๊ฒฝํ์ง ์์์ต๋๋ค. ๋ฌธ์๋ฅผ ํ์ธํ๊ณ CoreNLP์ ๋ฐฉ๋ฒ์ด ๋์ผํ๋ค๊ณ ์๊ฐํฉ๋๋ค. ๋ง์ต๋๊น? ๊ตฌ๋ฌธ ๋ถ์ ๊ฒฐ๊ณผ๋ฅผ ๋ฐ๋ณตํ๋ ๋ค์ ์ค์ ์ค๋ฅ๊ฐ ์๊ธฐ ๋๋ฌธ์ ๊ตฌ๋ฌธ ๋ถ์์ด ์ฑ๊ณตํ ๊ฒ ๊ฐ์ต๋๋ค.
cornlp ์๋ฒ๋ฅผ ์์ํด์ผํฉ๋๋ค.
with CoreNLPServer(port=9000) as server:
parser = CoreNLPParser(url=server.url)
parser.parse(...)
ํด๋ ์ ํ์ ์ ๋ ฅํ๋ ๋์ ๋๋ฝ ๋ ๋ฌธ์์ ์ ์ธ ๋ต์ฅ์ ๋ํด ์ฃ์กํฉ๋๋ค.
@dimazest ๊ทํ์ ๋์๊ณผ ๋น ๋ฅธ ๋ต๋ณ์ ์ ๋ง ๊ฐ์ฌ๋๋ฆฝ๋๋ค. ๊ทธ๋ฌ๋ ์ค๋ฅ๋ ๊ณ์๋ฉ๋๋ค :(
Traceback (most recent call last):
File "E:\classifier\feature_extraction.py", line 474, in <module>
print(feature_extraction(test_file_id))
File "E:\classifier\feature_extraction.py", line 135, in feature_extraction
with CoreNLPServer(port=9000) as server:
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 81, in __init__
try_port(port)
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 35, in try_port
sock.bind(('', port))
OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted
์ธํฐ๋ท ๊ฒ์์ ์๋ํ์ง๋ง ์์ผ์ด ์ด๋ป๊ฒ ์๋ํ๋์ง ๋ชจ๋ฅด๊ฒ ์ต๋๋ค ...
๋ค๋ฅธ ํฌํธ (์ : CoreNLPServer (port = 9001))๋ฅผ ์๋ํ๊ฑฐ๋ CoreNLPServer () ๋ง ์๋ ํ ์ ์์ต๋๋ค. ๊ทธ๋ฌ๋ฉด ์ฌ์ฉ ๊ฐ๋ฅํ ํฌํธ๋ฅผ ์ ํํด์ผํฉ๋๋ค.
@dimazest ๋๋ 9010๊น์ง 9001์ ์๋ํ๊ณ ๋น ๊ดํธ๋ ์๋ํ์ผ๋ฉฐ ์ด๊ฒ์ ๋งค๋ฒ ์ป๋ ๊ฒ์ ๋๋ค.
Traceback (most recent call last):
File "E:\classifier\feature_extraction.py", line 509, in <module>
print(feature_extraction(test_file_id))
File "E:\classifier\feature_extraction.py", line 136, in feature_extraction
with CoreNLPServer() as server:
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 170, in __enter__
self.start()
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 149, in start
'Could not connect to the server.'
nltk.parse.corenlp.CoreNLPServerError: Could not connect to the server.
๋ด๊ฐ ์ค๊ตญ์ ์๋ค๋ ๊ฒ์ ๊ณ ๋ คํ ๋ VPN์ ์คํํ๋ ๋์ VPN์ ๊ณ์ ์ฌ์ฉํ์ง๋ง ์ฌ์ ํ ์ด์ด ์์ต๋๋ค. ์ฌ๊ธฐ์ ๋ด ๋ฌธ์ ๋ ๋ฌด์์ผ๊น์?
corenlp .jars๊ฐ ์์ต๋๊น? corenlp ์๋ฒ๊ฐ ๋ก์ปฌ๋ก ์คํ๋์ด์ผํฉ๋๋ค.
์ด ์์ ๋ฅผ ์คํํ ์ ์์ต๋๊น https://github.com/nltk/nltk/pull/1249#pullrequestreview -18096061
E:\classifier\stanford\stanford-corenlp-full-2016-10-31
๋๋ ํ ๋ฆฌ ์๋์ ํญ์๋ฆฌ๊ฐ ์์ผ๋ฉฐ ์ด๊ฒ์ด ์ฐธ์กฐํ๋ ๊ฒ์ผ๋ก ๊ฐ์ ํฉ๋๋ค.
stanford-corenlp-3.7.0.jar
stanford-corenlp-3.7.0-javadoc.jar
stanford-corenlp-3.7.0-models.jar
stanford-corenlp-3.7.0-sources.jar
๊ทธ๋ฆฌ๊ณ ๋๋ ํ ๋ฆฌ๋ CLASSPATH ํ๊ฒฝ ๋ณ์๋ก ์ค์ ๋์์ต๋๋ค.
Windows ๋ช ๋ น ํ๋กฌํํธ์์ ์์ ๋ฅผ ์คํํ ์ ์์ผ๋ฉฐ ์ด๊ฒ์ด ์ถ๋ ฅ์ ๋๋ค.
Python 3.5.3 |Anaconda custom (64-bit)| (default, Feb 22 2017, 21:28:42) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from nltk.parse.corenlp import *
>>> global server
>>> server = CoreNLPServer()
>>> server.start()
>>> parser = CoreNLPParser(url='http://localhost:9000')
>>> sent = 'the quick brown fox jumps over the lazy dog'
>>> parser.raw_parse(sent)
<list_iterator object at 0x000001F0EFED69E8>
>>> fox_parsed = next(parser.raw_parse(sent))
>>> fox_parsed.pretty_print()
ROOT
|
NP
_______________|_________
| NP
| _________|___
| | PP
| | ________|___
NP NP | NP
____|__________ | | _______|____
DT JJ JJ NN NNS IN DT JJ NN
| | | | | | | | |
the quick brown fox jumps over the lazy dog
๊ทธ๋ฆฌ๊ณ ์ด ์ค์ server.start()
์คํํ์ ๋ Windows ๋ณด์ ๊ฒฝ๊ณ ๊ฐ ๋ํ๋๊ณ ์ด๊ฒ์ด ๋ฐฉํ๋ฒฝ์ ์๋ชป์ด๋ผ๊ณ ์๊ฐํ๊ธฐ ๋๋ฌธ์ ๋ฐฉํ๋ฒฝ ์ค์ ์ผ๋ก ๊ฐ์ ๋ฐฉํ๋ฒฝ์ ํตํด Java (TM) Platform SE ๋ฐ์ด๋๋ฆฌ๋ฅผ ํ์ฉํ์ต๋๋ค. ์ด๊ฒ์ด ๋ฌธ์ ๋ฅผ ํด๊ฒฐํ ๊ฒ์ด๋ผ๊ณ ์๊ฐํ์ง๋ง ํธ์ง๊ธฐ๋ฅผ ๋ค์ ์ด๊ณ ์ฝ๋๋ฅผ ์คํํ๋ฉด ์ฌ์ ํ ๋์ผํ ์ค๋ฅ๊ฐ ๋ฐ์ํฉ๋๋ค.
Traceback (most recent call last):
File "E:\classifier\feature_extraction.py", line 503, in <module>
print(feature_extraction(test_file_id))
File "E:\classifier\feature_extraction.py", line 130, in feature_extraction
with CoreNLPServer() as server:
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 170, in __enter__
self.start()
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 149, in start
'Could not connect to the server.'
nltk.parse.corenlp.CoreNLPServerError: Could not connect to the server.
์๋ฒ๋ฅผ ์์ํ๋ฉด ๋ธ๋ผ์ฐ์ ์์ http : // localhost : 9000์ ์ก์ธ์ค ํ ์ ์์ต๋๊น?
์ง์ ์๋ฒ๋ฅผ ์์ํ ์๋ ์์ต๋๋ค. https://stanfordnlp.github.io/CoreNLP/corenlp-server.html ์
ํ๋๋ ์คํ ์ค์ด๊ณ ๋ธ๋ผ์ฐ์ ๋ฅผ ํตํด ์ก์ธ์ค ํ ์ ์์ผ๋ฉฐ ํ์๋ฅผ ์ฌ์ฉํ ์ ์์ด์ผํฉ๋๋ค.
parser = CoreNLPParser(url='http://localhost:9000')
# and so on
์ด๋ฅผ ํ์ธํ๊ธฐ ์ํด ๋ช ๋ น ํ๋กฌํํธ์์ ์์ ๋ฅผ ๋ค์ ์คํํ์ง๋ง ์ด๋ฒ์๋ ์ต์ํ ์ค๋ฅ๊ฐ ๋ฐ์ํฉ๋๋ค.
Python 3.5.3 |Anaconda custom (64-bit)| (default, Feb 22 2017, 21:28:42) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from nltk.parse.corenlp import *
>>> global server
>>> server = CoreNLPServer()
>>> server.start()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 149, in start
'Could not connect to the server.'
nltk.parse.corenlp.CoreNLPServerError: Could not connect to the server.
>>> server.start()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Admin\Anaconda3\lib\site-packages\nltk\parse\corenlp.py", line 149, in start
'Could not connect to the server.'
nltk.parse.corenlp.CoreNLPServerError: Could not connect to the server.
... ๊ทธ๋ฆฌ๊ณ ๊ทธ ์ฌ์ด์ ๋ฌด์จ ์ผ์ด ์ผ์ด ๋ฌ๋์ง ์ ํ ๋ชจ๋ฆ ๋๋ค. ๊ตฌ์ฑ์ ๋ณ๊ฒฝํ์ง ์์์ต๋๋ค.
์ค๋ ์ ์์ด ํ๋ก์ ํธ๋ฅผ Linux๋ก ์ฎ๊ธฐ๊ณ ์ถ์์ง๋ง Linux์์ NLTK was unable to find ***.jar! Set the CLASSPATH environment variable
์ค๋ฅ๊ฐ ๊ณ์ ๋ฐ์ํฉ๋๋ค. /etc/environment
, /etc/profile
๋ฐ ~/.bash_profile
์์ CLASSPATH๋ฅผ ์ค์ ํ์ผ๋ฉฐ ํด๋น ํญ์๋ฆฌ๋ฅผ $JAVA_HOME/lib/
๋ณต์ฌํ๋ ค๊ณ ์๋ํ์ง๋ง ๋ฌธ์ ๊ฐ ๊ณ์ ๋ฐ์ํฉ๋๋ค. ๋ค๋ฅธ ๋ฌธ์ ๋ฅผ ์ด์ด์ผํฉ๋๊น?
Python์ด ์๋ ํฐ๋ฏธ๋์์ corenlp ์๋ฒ๋ฅผ ์์ํ ์ ์์ต๋๊น? ์์ธํ ๋ด์ฉ์ https://stanfordnlp.github.io/CoreNLP/corenlp-server.html ์ ํ์ธํ์ญ์์ค.
# Run the server using all jars in the current directory (e.g., the CoreNLP home directory)
java -mx4g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port 9000 -timeout 15000
E:\classifier\stanford\stanford-corenlp-full-2016-10-31>java -mx4g -cp "E:\classifier\stanford\stanford-corenlp-full-2016-10-31" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port 9000 -timeout 15000
Error: Could not find or load main class edu.stanford.nlp.pipeline.StanfordCoreNLPServer
E:\classifier\stanford\stanford-corenlp-full-2016-10-31>java -Xmx4g -cp "E:\classifier\stanford\stanford-corenlp-full-2016-10-31" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -serverProperties StanfordCoreNLP-chinese.properties -port 9000 -timeout 15000
Error: Could not find or load main class edu.stanford.nlp.pipeline.StanfordCoreNLPServer
E:\classifier\stanford\stanford-corenlp-full-2016-10-31>java -mx4g -cp "E:\classifier\stanford\stanford-corenlp-full-2016-10-31" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -timeout 15000
Error: Could not find or load main class edu.stanford.nlp.pipeline.StanfordCoreNLPServer
๊ทธ๋ ์ง ์์ ๊ฒ ๊ฐ์ต๋๋ค. ๋ด๊ฐ ์ ๋๋กํ๊ณ ์๋๊ฐ?
๋ง์นจ๋ด Linux์์ ์๋ํ๋ Stanford ๋ชจ๋์ ์ป์์ต๋๋ค. ๋์ผํ ์ฝ๋ ์ค๋ก CoreNLP ์๋ฒ๊ฐ ๋ฌธ์ ์์ด ์์๋๋ ๊ฒ์ฒ๋ผ ๋ณด์ด์ง๋ง ๊ตฌ๋ฌธ ๋ถ์ ๊ฒฐ๊ณผ๋ฅผ ๋ฐ๋ณตํ๋ ์ค์์ ๋ค๋ฅธ ์ค๋ฅ๊ฐ ๋ฐ์ํฉ๋๋ค.
Traceback (most recent call last):
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/packages/urllib3/util/connection.py", line 83, in create_connection
raise err
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 356, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/home/tesslo/anaconda3/lib/python3.5/http/client.py", line 1107, in request
self._send_request(method, url, body, headers)
File "/home/tesslo/anaconda3/lib/python3.5/http/client.py", line 1152, in _send_request
self.endheaders(body)
File "/home/tesslo/anaconda3/lib/python3.5/http/client.py", line 1103, in endheaders
self._send_output(message_body)
File "/home/tesslo/anaconda3/lib/python3.5/http/client.py", line 934, in _send_output
self.send(msg)
File "/home/tesslo/anaconda3/lib/python3.5/http/client.py", line 877, in send
self.connect()
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 166, in connect
conn = self._new_conn()
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
requests.packages.urllib3.exceptions.NewConnectionError: <requests.packages.urllib3.connection.HTTPConnection object at 0x7f110a9c4940>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/adapters.py", line 438, in send
timeout=timeout
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 649, in urlopen
_stacktrace=sys.exc_info()[2])
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
requests.packages.urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=9000): Max retries exceeded with url: /?properties=%7B%22annotators%22%3A+%22tokenize%2Cpos%2Clemma%2Cssplit%2Cdepparse%22%2C+%22outputFormat%22%3A+%22json%22%2C+%22ssplit.isOneSentence%22%3A+%22true%22%7D (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f110a9c4940>: Failed to establish a new connection: [Errno 111] Connection refused',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/media/tesslo/classifier/feature_extraction.py", line 503, in <module>
print(feature_extraction(test_file_id))
File "/media/tesslo/classifier/feature_extraction.py", line 142, in feature_extraction
for line in dep_parse:
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/nltk/parse/corenlp.py", line 279, in raw_parse_sents
parsed_data = self.api_call(sentence, properties=default_properties)
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/nltk/parse/corenlp.py", line 247, in api_call
timeout=60,
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/sessions.py", line 565, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/sessions.py", line 518, in request
resp = self.send(prep, **send_kwargs)
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/sessions.py", line 639, in send
r = adapter.send(request, **kwargs)
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/requests/adapters.py", line 502, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=9000): Max retries exceeded with url: /?properties=%7B%22annotators%22%3A+%22tokenize%2Cpos%2Clemma%2Cssplit%2Cdepparse%22%2C+%22outputFormat%22%3A+%22json%22%2C+%22ssplit.isOneSentence%22%3A+%22true%22%7D (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f110a9c4940>: Failed to establish a new connection: [Errno 111] Connection refused',))
์ฐธ๊ณ ๋ก Linux์ Windows๋ ๋์ผํ ํ๋์จ์ด๋ฅผ ๊ณต์ ํฉ๋๋ค.
๋ค, ๋ ๋จ๊ณ๊ฐ ๊ด๋ จ๋ฉ๋๋ค.
1) CoreNLP Java ํ๋ก์ธ์ค๋ฅผ ์์ํฉ๋๋ค. java -Xmx4g -cp ...
๋ช
๋ น์ ์ฌ์ฉํ๋ ๋ ๊ฐ์ง ๋ฐฉ๋ฒ์ด ์์ต๋๋ค. ์๋์ผ๋ก ์์ํ๋ ๊ฒ์ด ์ข์ต๋๋ค. ๋น์ ์ ๊ทธ๊ฒ์ ์ฑ๊ณต ํ์ต๋๊น? http : // localhost : 9000์ ๋ฐฉ๋ฌธํ์ฌ ๋ธ๋ผ์ฐ์ ๋ฅผ ํตํด ์๋ฒ์ ์ก์ธ์ค ํ ์ ์์ด์ผํฉ๋๋ค. ์ฝ์ ์ถ๋ ฅ์์ โโ์ฌ์ฉ์ค์ธ ํฌํธ๋ฅผ ํ์ธํด์ผํฉ๋๋ค.
2) ์๋ฒ๊ฐ ์คํ๋๋ฉด CoreNLP Python ํด๋ผ์ด์ธํธ parser = CoreNLPParser(url='http://localhost:9000')
๋ง๋ค ์ ์์ต๋๋ค. CoreNLP Java ์๋ฒ๋ฅผ ์ง์ ์์ ํ์ผ๋ฏ๋ก Python ์ธ์
๋ด์์ ์์ํ ํ์ ๊ฐ ์์ต๋๋ค ( server = CoreNLPServer()
์คํํ์ง ๋ง์ญ์์ค).
๊ฒ์ ํ ์ค๋ฅ ๋ฉ์์ง๋ CoreNLP Java ์๋ฒ๊ฐ ์คํ๋๊ณ ์์ง ์์์ ๋ํ๋ ๋๋ค.
์คํจํ์ต๋๋ค.
tesslo<strong i="6">@TLU</strong>:/media/tesslo/classifier/stanford/stanford-corenlp-full-2016-10-31$ java -mx4g -cp "/media/tesslo/classifier/stanford/stanford-corenlp-full-2016-10-31" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port 9000 -timeout 15000
Error: Could not find or load main class edu.stanford.nlp.pipeline.StanfordCoreNLPServer
tesslo<strong i="7">@TLU</strong>:/media/tesslo/classifier/stanford/stanford-corenlp-full-2016-10-31$ java -Xmx4g -cp "/media/tesslo/classifier/stanford/stanford-corenlp-full-2016-10-31" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -serverProperties StanfordCoreNLP-chinese.properties -port 9000 -timeout 15000
Error: Could not find or load main class edu.stanford.nlp.pipeline.StanfordCoreNLPServer
tesslo<strong i="8">@TLU</strong>:/media/tesslo/classifier/stanford/stanford-corenlp-full-2016-10-31$ java -mx4g -cp "/media/tesslo/classifier/stanford/stanford-corenlp-full-2016-10-31" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -timeout 15000
Error: Could not find or load main class edu.stanford.nlp.pipeline.StanfordCoreNLPServer
๊ทธ๋ฆฌ๊ณ http : // localhost : 9000์ ERR_CONNECTION_REFUSED ์ค๋ฅ๋ฅผ ํ์ํฉ๋๋ค.
"*"
๋ฅผ ํด๋์ค ๊ฒฝ๋ก๋ก ์ฌ์ฉํ์ฌ ์๋ ํ์ต๋๊น : j ava -mx4g -cp "*" ...
?
์๋
ํ์ธ์, ๋๋์ด ๋ฌธ์ ์ ์ง๋ฉด ํ ๊ฒ ๊ฐ์ต๋๋ค. ๋ด ๋ฌธ์ฅ์ :
'์ด์ฉ๋ฉด 2 21/2 ํผํธ ์ฝ๋ ์ผ๊น?', u '๋ง์ดํฌ๋ก USB๋ฅผ ์ฐ๊ฒฐํ๋ ์ถฉ์ ๊ธฐ ๋ถ๋ถ๋ณด๋ค ํ์ง์ด ์ ๋ ดํฉ๋๋ค ...'
๊ทธ๋ฆฌ๊ณ ๋๋ '/'๊ฐ์ด ์ค๋ฅ๋ฅผ ์ผ์ผํค๋ ๊ฒ์ผ๋ก ๋ณด์
๋๋ค.
@caisinong ์๋ก์ด CoreNLP ์ธํฐํ์ด์ค๋ฅผ ์ฌ์ฉํด ๋ณด์ จ์ต๋๊น? ์์ ๋ด ์๊ฒฌ์ ์ฐธ์กฐํ์ญ์์ค.
@dimazest ์ง์ฐ ์ฃ์กํฉ๋๋ค. ๋๋ ๋ฐฉ๊ธํ๋ค :
tesslo<strong i="7">@TLU</strong>:/media/tesslo/classifier/stanford/stanford-corenlp-full-2016-10-31$ java -mx4g -cp "*" edu.stanford.nlp.pipeline.StanfordCoreNLPServer -port 9000 -timeout 15000
[main] INFO CoreNLP - --- StanfordCoreNLPServer#main() called ---
[main] INFO CoreNLP - setting default constituency parser
[main] INFO CoreNLP - warning: cannot find edu/stanford/nlp/models/srparser/englishSR.ser.gz
[main] INFO CoreNLP - using: edu/stanford/nlp/models/lexparser/englishPCFG.ser.gz instead
[main] INFO CoreNLP - to use shift reduce parser download English models jar from:
[main] INFO CoreNLP - http://stanfordnlp.github.io/CoreNLP/download.html
[main] INFO CoreNLP - Threads: 2
[main] INFO CoreNLP - Starting server...
[main] INFO CoreNLP - StanfordCoreNLPServer listening at /0:0:0:0:0:0:0:0:9000
์ด์ http : // localhost : 9000์ ๋ฐฉ๋ฌธ ํ ์ ์์ง๋ง ํธ์ง๊ธฐ๋ก ๋์ ๊ฐ๋ฉด ์๋ฒ๋ฅผ ์์ํ๋ ์ค์์ ์ฌ์ ํ์ด ์ค๋ฅ๊ฐ ๋ฐ์ํฉ๋๋ค.
Traceback (most recent call last):
File "/media/tesslo/classifier/feature_extraction.py", line 503, in <module>
print(feature_extraction(test_file_id))
File "/media/tesslo/classifier/feature_extraction.py", line 130, in feature_extraction
with CoreNLPServer() as server:
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/nltk/parse/corenlp.py", line 170, in __enter__
self.start()
File "/home/tesslo/anaconda3/lib/python3.5/site-packages/nltk/parse/corenlp.py", line 149, in start
'Could not connect to the server.'
nltk.parse.corenlp.CoreNLPServerError: Could not connect to the server.
์๋ฒ๋ฅผ ์๋์ผ๋ก ์์ํ ํ์๋ ์ฝ๋์์ ์๋ฒ๋ฅผ ์์ํ ํ์ ๊ฐ ์์ต๋๋ค .
์๋ฒ๋ฅผ ๊ณ์ ์คํํ๊ณ ํ์๋ฅผ ์ธ์คํด์คํํฉ๋๋ค.
parser = CoreNLPParser(url='http://localhost:9000')
@dimazest ๋ฌธ์ ๊ฐ ํด๊ฒฐ๋์์ต๋๋ค. ํญ์ ๋์๊ณผ ์ธ๋ด์ฌ์ ๊ฐ์ฌ๋๋ฆฝ๋๋ค!
๋น์ทํ ๊ฒฝํ์ด ์์ต๋๋ค. ์ฝ๋์์ Stanford CorenNLP ์๋ฒ๋ฅผ ์์ํ๋ ๊ฒ์ ๋ณต์กํ๋ฉฐ ํ ์คํธ ๋ชฉ์ ์ผ๋ก ๋ง ์ฌ์ฉํด์ผํฉ๋๋ค. ์ด์ฉ๋ฉด ์ฐ๋ฆฌ๋ ๊ทธ๊ฒ์ ์ฌ์ฉ์์๊ฒ ๋ ธ์ถ์ํค์ง ์์์ผ ํ ๊ฒ์ ๋๋ค.
์ผ์ด ์๋์ด์ ๊ธฐ์ฉ๋๋ค. ์ค์ ๋ก ์๋ฒ๋ Python ์ฝ๋ ์ธ๋ถ์์ ์์ํด์ผํฉ๋๋ค.
์๋ก์ด CoreNLP API์ ์ํด ํจ์น ๋ฐ ํด๊ฒฐ๋จ =)
@dimazest Hi ... ํ
์คํธ์ \
๋๋ /
๊ฒฝ์ฐ Assertion Error
๋ํ ์๋ฃจ์
์ Core NLP ๋ง ์ฌ์ฉํ๋ ๊ฒ์
๋๊น? ๋๋ stanford-parser-full-2017-06-09
๊ตฌ๋ฌธ ๋ถ์์ ์ฌ์ฉ ๋ ๋ฌธ์ฅ์ Iraqi security forces drove Islamic State fighters from the centre of a town just south of the militants\' main stronghold of Mosul on Saturday and reached within a few km (miles) of an airport on the edge of the city, a senior commander said.
@ kavin26 ์, nltk.parse.corenlp.CoreNLPParser
๋ฅผ ์ฌ์ฉํ์ญ์์ค.
@alvations ์ ๋ง ๊ฐ์ฌํฉ๋๋ค : +1 :
๊ฐ์ฅ ์ ์ฉํ ๋๊ธ
@dimazest ๋ฌธ์ ๊ฐ ํด๊ฒฐ๋์์ต๋๋ค. ํญ์ ๋์๊ณผ ์ธ๋ด์ฌ์ ๊ฐ์ฌ๋๋ฆฝ๋๋ค!