<p>Detectron ops ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ์ฐพ์„ ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.</p>

์— ๋งŒ๋“  2018๋…„ 02์›” 07์ผ  ยท  27์ฝ”๋ฉ˜ํŠธ  ยท  ์ถœ์ฒ˜: facebookresearch/Detectron

Ubuntu 16.04์˜ ์†Œ์Šค์—์„œ caffe2๋ฅผ ์„ค์น˜ํ•˜๊ณ  python2 detectron/tests/test_spatial_narrow_as_op.py ๋กœ ํ…Œ์ŠคํŠธํ•˜๋ ค๊ณ  ์‹œ๋„ํ•œ ํ›„ ๋‹ค์Œ์„ ์–ป์Šต๋‹ˆ๋‹ค.

No handlers could be found for logger "caffe2.python.net_drawer"
net_drawer will not run correctly. Please install the correct dependencies.
E0207 16:36:41.320443  4125 init_intrinsics_check.cc:59] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
Traceback (most recent call last):
  File "detectron/tests/test_spatial_narrow_as_op.py", line 88, in <module>
    utils.c2.import_detectron_ops()
  File "/home/gene/detectron/lib/utils/c2.py", line 41, in import_detectron_ops
    detectron_ops_lib = envu.get_detectron_ops_lib()
  File "/home/gene/detectron/lib/utils/env.py", line 73, in get_detectron_ops_lib
    'version includes Detectron module').format(detectron_ops_lib)
AssertionError: Detectron ops lib not found at '/usr/local/lib/python2.7/dist-packages/lib/libcaffe2_detectron_ops_gpu.so'; make sure that your Caffe2 version includes Detectron module

๊ทธ๋Ÿฌ๋‚˜ ๊ฐ์ง€๊ธฐ ๋ชจ๋“ˆ์€ modules ํด๋”์— ์žˆ์Šต๋‹ˆ๋‹ค. caffe2๊ฐ€ ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ํฌํ•จ๋˜์—ˆ๋Š”์ง€ ํ™•์ธํ•˜๊ธฐ ์œ„ํ•ด caffe2๋ฅผ ์„ค์น˜ํ•˜๊ธฐ ์ „์— CMakeList๋ฅผ ์–ด๋–ป๊ฒŒ๋“  ์ˆ˜์ •ํ•ด์•ผ ํ•ฉ๋‹ˆ๊นŒ?

์‹œ์Šคํ…œ ์ •๋ณด

  • ์šด์˜ ์ฒด์ œ: ์šฐ๋ถ„ํˆฌ 16.04
  • ์ปดํŒŒ์ผ๋Ÿฌ ๋ฒ„์ „: gcc ๋ฒ„์ „ 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5)
  • ์ฟ ๋‹ค ๋ฒ„์ „: 8.0
  • cuDNN ๋ฒ„์ „: 6.0.21
  • NVIDIA ๋“œ๋ผ์ด๋ฒ„ ๋ฒ„์ „:
  • GPU ๋ชจ๋ธ(๋ชจ๋“  ์žฅ์น˜๊ฐ€ ๋™์ผํ•˜์ง€ ์•Š์€ ๊ฒฝ์šฐ): 4x Tesla k80
  • PYTHONPATH ํ™˜๊ฒฝ ๋ณ€์ˆ˜: /usr/local:/home/ubuntu/caffe2/build
  • python --version ์ถœ๋ ฅ: 2.7.12

๊ฐ€์žฅ ์œ ์šฉํ•œ ๋Œ“๊ธ€

์„ค์น˜ํ•˜๋Š” ๋™์•ˆ ๋ชจ๋“  ๋ช…๋ น์„ ์ ์–ด ๋‘์—ˆ์œผ๋ฏ€๋กœ ์ด๊ฒƒ์ด ๋ฒ„๊ทธ ์—ฐ๊ตฌ์— ๋„์›€์ด ๋  ์ˆ˜ ์žˆ๋‹ค๊ณ  ์ƒ๊ฐํ–ˆ์Šต๋‹ˆ๋‹ค. ํ•ด๊ฒฐํ•˜์‹ ๊ฑฐ ๋ดค์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์ด๊ฒƒ์€ ์ง€์นจ์œผ๋กœ ๋‹ค๋ฅธ ์‚ฌ๋žŒ๋“ค์—๊ฒŒ ๋„์›€์ด ๋  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋ชจ๋‘์—๊ฒŒ ํ–‰์šด์„ ๋น•๋‹ˆ๋‹ค!

์‹œ์Šคํ…œ ์ •๋ณด

  • Amazon AWS ์ธ์Šคํ„ด์Šค g3.4xlarge(NVIDIA Tesla M60)
  • ์šฐ๋ถ„ํˆฌ 16.04

์„ค์น˜

Caffe2 ๋ฌธ์„œ ์™€

NVIDIA ๋“œ๋ผ์ด๋ฒ„ ์—…๋ฐ์ดํŠธ

๋“œ๋ผ์ด๋ฒ„ ๋‹ค์šด๋กœ๋“œ

sudo dpkg -i nvidia-diag-driver-local-repo-ubuntu1604_375.66-1_amd64.deb
sudo apt-get update && sudo apt-get install wget -y --no-install-recommends

์ข…์†์„ฑ ์„ค์น˜

sudo apt-get update
sudo apt-get install -y --no-install-recommends build-essential cmake git libgoogle-glog-dev libgtest-dev libiomp-dev libleveldb-dev liblmdb-dev libopencv-dev libopenmpi-dev libsnappy-dev libprotobuf-dev openmpi-bin openmpi-doc protobuf-compiler python-dev python-pip
sudo pip install setuptools future numpy protobuf enum networkx
sudo apt-get install -y --no-install-recommends libgflags-dev

์ฟ ๋‹ค ์„ค์น˜

๋งํฌ: http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/
์ตœ์‹  ๋ฒ„์ „์„ ๊ฒ€์ƒ‰ํ•ฉ๋‹ˆ๋‹ค. ๊ฐ€๋Šฅํ•œ ๊ฒฝ์šฐ cuda 8์„ ์‚ฌ์šฉํ•˜์‹ญ์‹œ์˜ค. cuda-repo-ubuntu1604_8.0.61-1_amd64.deb๋ฅผ ์‚ฌ์šฉํ–ˆ์Šต๋‹ˆ๋‹ค.

wget "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/..." //add selected file name
sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda-8-0

cuDNN ์„ค์น˜

์ตœ์‹  ๋ฒ„์ „ ๊ฒ€์ƒ‰: https://developer.nvidia.com/cudnn

wget http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-x64-v5.1.tgz
sudo tar -xzf cudnn-8.0-linux-x64-v5.1.tgz -C /usr/local
rm cudnn-8.0-linux-x64-v5.1.tgz && sudo ldconfig

์นดํŽ˜2 ์„ค์น˜

Caffe2์˜ ๋น ๋ฅธ ๊ฐœ๋ฐœ๋กœ ์ธํ•ด ์ €์žฅ์†Œ๋ฅผ ๋งˆ์ง€๋ง‰ ๋นŒ๋“œ ์ „๋‹ฌ ๋ฒ„์ „์œผ๋กœ ์žฌ์„ค์ •ํ•ด์•ผ ํ•  ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค. Caffe2 GitHub ์ €์žฅ์†Œ ์—์„œ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. README.md ์ƒ๋‹จ์— ์žˆ๋Š” ๋นŒ๋“œ ํ†ต๊ณผ/์‹คํŒจ ์•„์ด์ฝ˜์„ ํด๋ฆญํ•ฉ๋‹ˆ๋‹ค. ์™ผ์ชฝ์—์„œ ๋งˆ์ง€๋ง‰ ๋นŒ๋“œ๋ฅผ ํ™•์ธํ•˜๊ณ  ๋งˆ์ง€๋ง‰ ํ†ต๊ณผ ๋นŒ๋“œ๋ฅผ ์„ ํƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋นŒ๋“œ๋ฅผ ํด๋ฆญํ•˜๊ณ  ์ปค๋ฐ‹ ๋ฒˆํ˜ธ๋ฅผ ๋ณต์‚ฌํ•ฉ๋‹ˆ๋‹ค.

Go to your selected installation directory: cd directory
git clone --recursive https://github.com/caffe2/caffe2.git && cd caffe2
git reset --hard $COMMIT_NUMBER //unnecessary when builds passing
sudo make -j16 //exchange 16 by your number of cores
cd build && sudo make install

ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •

๋‹ค์Œ ํ–‰์„ .bashrc์— ์ถ”๊ฐ€ํ•˜์‹ญ์‹œ์˜ค.

export PYTHONPATH=/usr/local
export PYTHONPATH=$PYTHONPATH:/home/ubuntu/caffe2/build
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

Caffe2 ๋ฐ GPU ํ…Œ์ŠคํŠธ

Caffe2 ์„ค์น˜ ํ…Œ์ŠคํŠธ:

python2 -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

GPU ์„ค์น˜๋ฅผ ํ…Œ์ŠคํŠธํ•ฉ๋‹ˆ๋‹ค. ๋ฐ˜ํ™˜ ๊ฐ’์€ 0๋ณด๋‹ค ์ปค์•ผ ํ•ฉ๋‹ˆ๋‹ค.

python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())'

Detectron ์ข…์†์„ฑ ์„ค์น˜

sudo pip install numpy>=1.13 pyyaml>=3.12 matplotlib opencv-python>=3.2 setuptools Cython mock scipy
sudo git clone https://github.com/cocodataset/cocoapi.git $WANTED-DIRECTORY/cocoapi
cd $WANTED-DIRECTORY/cocoapi/PythonApi
sudo -H make install
sudo python setup.py install

.bashrc์— ๋ผ์ธ ์ถ”๊ฐ€:

export COCOAPI=$WANTED-DIRECTORY/cocoapi

์ด ๋””๋ ‰ํ† ๋ฆฌ์˜ coco ์›น์‚ฌ์ดํŠธ์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ๋‹ค์šด๋กœ๋“œํ•˜๋Š” ๊ฒƒ์„ ์žŠ์ง€ ๋งˆ์‹ญ์‹œ์˜ค. ๋ฐ์ดํ„ฐ README.md์— ์„ค๋ช…๋œ ๊ตฌ์กฐ

๋””ํ…ํŠธ๋ก  ์„ค์น˜

git clone https://github.com/facebookresearch/detectron detectron
cd detectron/lib && sudo make -j16

์„ค์น˜ ํ™•์ธ

python2 detectron/tests/test_spatial_narrow_as_op.py

๋‚˜๋Š” ๋‚ด๊ฐ€ ์–ด๋–ค ์‚ฌ๋žŒ๋“ค์„ ๋„์šธ ์ˆ˜ ์žˆ๊ธฐ๋ฅผ ๋ฐ”๋ž๋‹ˆ๋‹ค :)

๋ชจ๋“  27 ๋Œ“๊ธ€

๋‚˜๋Š” ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ–ˆ๋‹ค. ๋ฌธ์ œ๋Š” ์—†์—ˆ์Šต๋‹ˆ๋‹ค. ๊ฐ„๋‹จํ•˜๊ณ  ์–ด๋ฆฌ์„๊ฒŒ๋„ ๋‚ด PYTHONPATH์— ์˜คํƒ€๊ฐ€ ์žˆ์—ˆ์Šต๋‹ˆ๋‹ค. ๋‚˜๋Š” ๊ทธ๊ฒƒ์„ ๊ณ ์ณค๊ณ  ์ด์ œ ์ œ๋Œ€๋กœ ์ž‘๋™ํ•ฉ๋‹ˆ๋‹ค. ์ด ํ›Œ๋ฅญํ•œ ์ž๋ฃŒ์— ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค!

์„ค์น˜ํ•˜๋Š” ๋™์•ˆ ๋ชจ๋“  ๋ช…๋ น์„ ์ ์–ด ๋‘์—ˆ์œผ๋ฏ€๋กœ ์ด๊ฒƒ์ด ๋ฒ„๊ทธ ์—ฐ๊ตฌ์— ๋„์›€์ด ๋  ์ˆ˜ ์žˆ๋‹ค๊ณ  ์ƒ๊ฐํ–ˆ์Šต๋‹ˆ๋‹ค. ํ•ด๊ฒฐํ•˜์‹ ๊ฑฐ ๋ดค์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์ด๊ฒƒ์€ ์ง€์นจ์œผ๋กœ ๋‹ค๋ฅธ ์‚ฌ๋žŒ๋“ค์—๊ฒŒ ๋„์›€์ด ๋  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋ชจ๋‘์—๊ฒŒ ํ–‰์šด์„ ๋น•๋‹ˆ๋‹ค!

์‹œ์Šคํ…œ ์ •๋ณด

  • Amazon AWS ์ธ์Šคํ„ด์Šค g3.4xlarge(NVIDIA Tesla M60)
  • ์šฐ๋ถ„ํˆฌ 16.04

์„ค์น˜

Caffe2 ๋ฌธ์„œ ์™€

NVIDIA ๋“œ๋ผ์ด๋ฒ„ ์—…๋ฐ์ดํŠธ

๋“œ๋ผ์ด๋ฒ„ ๋‹ค์šด๋กœ๋“œ

sudo dpkg -i nvidia-diag-driver-local-repo-ubuntu1604_375.66-1_amd64.deb
sudo apt-get update && sudo apt-get install wget -y --no-install-recommends

์ข…์†์„ฑ ์„ค์น˜

sudo apt-get update
sudo apt-get install -y --no-install-recommends build-essential cmake git libgoogle-glog-dev libgtest-dev libiomp-dev libleveldb-dev liblmdb-dev libopencv-dev libopenmpi-dev libsnappy-dev libprotobuf-dev openmpi-bin openmpi-doc protobuf-compiler python-dev python-pip
sudo pip install setuptools future numpy protobuf enum networkx
sudo apt-get install -y --no-install-recommends libgflags-dev

์ฟ ๋‹ค ์„ค์น˜

๋งํฌ: http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/
์ตœ์‹  ๋ฒ„์ „์„ ๊ฒ€์ƒ‰ํ•ฉ๋‹ˆ๋‹ค. ๊ฐ€๋Šฅํ•œ ๊ฒฝ์šฐ cuda 8์„ ์‚ฌ์šฉํ•˜์‹ญ์‹œ์˜ค. cuda-repo-ubuntu1604_8.0.61-1_amd64.deb๋ฅผ ์‚ฌ์šฉํ–ˆ์Šต๋‹ˆ๋‹ค.

wget "http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64/..." //add selected file name
sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda-8-0

cuDNN ์„ค์น˜

์ตœ์‹  ๋ฒ„์ „ ๊ฒ€์ƒ‰: https://developer.nvidia.com/cudnn

wget http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-x64-v5.1.tgz
sudo tar -xzf cudnn-8.0-linux-x64-v5.1.tgz -C /usr/local
rm cudnn-8.0-linux-x64-v5.1.tgz && sudo ldconfig

์นดํŽ˜2 ์„ค์น˜

Caffe2์˜ ๋น ๋ฅธ ๊ฐœ๋ฐœ๋กœ ์ธํ•ด ์ €์žฅ์†Œ๋ฅผ ๋งˆ์ง€๋ง‰ ๋นŒ๋“œ ์ „๋‹ฌ ๋ฒ„์ „์œผ๋กœ ์žฌ์„ค์ •ํ•ด์•ผ ํ•  ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค. Caffe2 GitHub ์ €์žฅ์†Œ ์—์„œ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. README.md ์ƒ๋‹จ์— ์žˆ๋Š” ๋นŒ๋“œ ํ†ต๊ณผ/์‹คํŒจ ์•„์ด์ฝ˜์„ ํด๋ฆญํ•ฉ๋‹ˆ๋‹ค. ์™ผ์ชฝ์—์„œ ๋งˆ์ง€๋ง‰ ๋นŒ๋“œ๋ฅผ ํ™•์ธํ•˜๊ณ  ๋งˆ์ง€๋ง‰ ํ†ต๊ณผ ๋นŒ๋“œ๋ฅผ ์„ ํƒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋นŒ๋“œ๋ฅผ ํด๋ฆญํ•˜๊ณ  ์ปค๋ฐ‹ ๋ฒˆํ˜ธ๋ฅผ ๋ณต์‚ฌํ•ฉ๋‹ˆ๋‹ค.

Go to your selected installation directory: cd directory
git clone --recursive https://github.com/caffe2/caffe2.git && cd caffe2
git reset --hard $COMMIT_NUMBER //unnecessary when builds passing
sudo make -j16 //exchange 16 by your number of cores
cd build && sudo make install

ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •

๋‹ค์Œ ํ–‰์„ .bashrc์— ์ถ”๊ฐ€ํ•˜์‹ญ์‹œ์˜ค.

export PYTHONPATH=/usr/local
export PYTHONPATH=$PYTHONPATH:/home/ubuntu/caffe2/build
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

Caffe2 ๋ฐ GPU ํ…Œ์ŠคํŠธ

Caffe2 ์„ค์น˜ ํ…Œ์ŠคํŠธ:

python2 -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

GPU ์„ค์น˜๋ฅผ ํ…Œ์ŠคํŠธํ•ฉ๋‹ˆ๋‹ค. ๋ฐ˜ํ™˜ ๊ฐ’์€ 0๋ณด๋‹ค ์ปค์•ผ ํ•ฉ๋‹ˆ๋‹ค.

python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())'

Detectron ์ข…์†์„ฑ ์„ค์น˜

sudo pip install numpy>=1.13 pyyaml>=3.12 matplotlib opencv-python>=3.2 setuptools Cython mock scipy
sudo git clone https://github.com/cocodataset/cocoapi.git $WANTED-DIRECTORY/cocoapi
cd $WANTED-DIRECTORY/cocoapi/PythonApi
sudo -H make install
sudo python setup.py install

.bashrc์— ๋ผ์ธ ์ถ”๊ฐ€:

export COCOAPI=$WANTED-DIRECTORY/cocoapi

์ด ๋””๋ ‰ํ† ๋ฆฌ์˜ coco ์›น์‚ฌ์ดํŠธ์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ๋‹ค์šด๋กœ๋“œํ•˜๋Š” ๊ฒƒ์„ ์žŠ์ง€ ๋งˆ์‹ญ์‹œ์˜ค. ๋ฐ์ดํ„ฐ README.md์— ์„ค๋ช…๋œ ๊ตฌ์กฐ

๋””ํ…ํŠธ๋ก  ์„ค์น˜

git clone https://github.com/facebookresearch/detectron detectron
cd detectron/lib && sudo make -j16

์„ค์น˜ ํ™•์ธ

python2 detectron/tests/test_spatial_narrow_as_op.py

๋‚˜๋Š” ๋‚ด๊ฐ€ ์–ด๋–ค ์‚ฌ๋žŒ๋“ค์„ ๋„์šธ ์ˆ˜ ์žˆ๊ธฐ๋ฅผ ๋ฐ”๋ž๋‹ˆ๋‹ค :)

@genekogan

AWS์˜ AMI ํ”Œ๋žซํผ์—์„œ ํƒ์ง€๊ธฐ๋ฅผ ๊ตฌ์ถ•ํ•˜๋ ค๊ณ  ํ•  ๋•Œ๋„ ๋™์ผํ•œ ๋ฌธ์ œ๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค.
๋‚ด PYTHONPATH๊ฐ€ ์˜ฌ๋ฐ”๋ฅธ์ง€ ํ™•์‹คํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.
์—ฌ๊ธฐ ๋‚ด PYTHONPATH๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค:/home/ec2-user/src/caffe2/build

"build" ๋””๋ ‰ํ† ๋ฆฌ์— ๋ฌด์—‡์ด ํฌํ•จ๋˜์–ด์•ผ ํ•˜๋Š”์ง€ ๊ถ๊ธˆํ•ฉ๋‹ˆ๋‹ค.
๋‚ด "๋นŒ๋“œ" ๋””๋ ‰ํ† ๋ฆฌ์— ํฌํ•จ๋œ ํŒŒ์ผ์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
screen shot 2018-03-14 at 12 29 28 pm

๊ทธ๋“ค์€ ๋‹น์‹ ๊ณผ ๋™์ผํ•ฉ๋‹ˆ๊นŒ?

๊ฐ์‚ฌ ํ•ด์š”!

${CAFFE2_ROOT}/build๋ฅผ ํฌํ•จํ•˜๋„๋ก PYTHONPATH ๋˜๋Š” pth ํŒŒ์ผ ์„ค์ •

์‚ฌ์šฉ์ž ์ง€์ • Caffe2 CPU ๋˜๋Š” GPU ์—ฐ์‚ฐ์ž ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ์ฐพ์„ ์ˆ˜ ์—†์Œ: AssertionError: '/path/to/to/detectron/build/libcaffe2_detectron_custom_ops_gpu.so'์—์„œ ์‚ฌ์šฉ์ž ์ง€์ • ์ž‘์—… ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ์ฐพ์„ ์ˆ˜ ์—†์ง€๋งŒ test_zero_even_op.py๋ฅผ ์‹คํ–‰ํ•  ๋•Œ ๋ฐœ๊ฒฌ๋˜์ง€๋งŒ ๋‹ค๋ฅธ ํ…Œ์ŠคํŠธ๋Š” ํ†ต๊ณผํ•  ์ˆ˜ ์žˆ์Œ, Detectron ์ž‘์—… ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ : /usr/local/lib/libcaffe2_detectron_ops_gpu.so๋ฅผ ์ฐพ์•˜์Šต๋‹ˆ๋‹ค. ์•„์ด๋””์–ด ์ข€ ์ฃผ์‹œ๊ฒ ์–ด์š”?

@genekogan ๋„์™€์ฃผ์‹œ๊ฒ ์–ด์š”? ์ผ๋ฐ˜ ํ…Œ์ŠคํŠธ๋Š” ๋ชจ๋‘ "/usr/local/lib/libcaffe2_detectron_ops_gpu.so found" ์ •๋ณด์™€ ํ•จ๊ป˜ ํ†ต๊ณผํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์‚ฌ์šฉ์ž ์ง€์ • ์ž‘์—…์„ ํ…Œ์ŠคํŠธํ•˜๊ณ  test_zero_even_op.py๋ฅผ ์‹คํ–‰ํ•˜๋ฉด "Custom ops lib not found at '/path/to/to/detectron/build/libcaffe2_detectron_custom_ops_gpu" ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.

@ ym547559398 ๋‚˜๋Š” ๋‹น์‹ ์ด ์–ป๋Š” ํŠน์ • ์˜ค๋ฅ˜๊ฐ€ ํ™•์‹คํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. @huhai463127310์ด ๋งํ•œ ๋Œ€๋กœ ๋‚ด ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ–ˆ์Šต๋‹ˆ๋‹ค. ${CAFFE2_ROOT}/build๋ฅผ ํฌํ•จํ•˜๋„๋ก PYTHONPATH๋ฅผ ์„ค์ •ํ–ˆ๋Š”์ง€ ํ™•์ธํ•˜์„ธ์š”.

@genekogan ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค! ๋‚˜๋Š” ๊ทธ๊ฒƒ์„ ํ•ด๊ฒฐํ–ˆ๋‹ค. ์•„์ฃผ ๊ฐ„๋‹จํ•œ ๊ฐˆ๋“ฑ~

caffe2๊ฐ€ pytorch์— ํ†ตํ•ฉ๋˜์—ˆ์œผ๋ฏ€๋กœ ์—ฌ์ „ํžˆ ๋™์ผํ•œ ๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค. ์ตœ์‹  ์†Œ์Šค์—์„œ pytorch๋ฅผ ์ปดํŒŒ์ผํ–ˆ์ง€๋งŒ detetron simple_infer๋ฅผ ์‹คํ–‰ํ•˜๋ฉด ์—ฌ์ „ํžˆ detectron_ops๋ฅผ ์ฐพ์ง€ ๋ชปํ•˜๋Š” ๊ฒƒ์— ๋Œ€ํ•ด ๋ถˆํ‰ํ•ฉ๋‹ˆ๋‹ค(๋‚ด ๊ฒฝ์šฐ์—๋Š” GPU ์—†์Œ)

@mehditlili PYTHONPATH๋ฅผ ์„ค์ •ํ•˜์…จ์Šต๋‹ˆ๊นŒ? ์•„๋‹ˆ๋ฉด ํ™•์žฅ๋œ ์ž‘์—…์ด ๋ถ€์กฑํ•˜๋‹ค๊ณ  ์ƒ๊ฐํ•˜๊ณ  ์ž‘์—… ํด๋”๋ฅผ ์‚ฌ์šฉํ•˜๊ณ  ํ™•์žฅ๋œ ์ž‘์—…์„ ๋นŒ๋“œํ•˜๋Š” ๋ฐฉ๋ฒ•์— ๋„์›€์ด ๋˜๋Š” ๊ฐ€์ด๋“œ๊ฐ€ ์žˆ์Šต๋‹ˆ๊นŒ?

@mattifrind

conda๋ฅผ ํ†ตํ•œ pytorch/caffe2 ๋ฐฐํฌ๋Š” ๊ณต์‹์ ์ด๋ฏ€๋กœ ํ•ด๋‹น ops ๋ชจ๋“ˆ์„ ์ฐพ๊ธฐ ์œ„ํ•œ ์ถ”๊ฐ€ ์„ค์ •์ด ์—†์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. libcaffe2_detectron_ops_gpu.so ๋Š” ํ‘œ์ค€ PATH ํ™˜๊ฒฝ์„ ํ†ตํ•ด ์ฐพ์„ ์ˆ˜ ์žˆ์ง€๋งŒ pytorch ์„ค์น˜ ๊ฒฝ๋กœ๋Š” ๊ธฐ๋ณธ์ ์œผ๋กœ ์‚ฌ์šฉ์ž ๊ฐœ์ž… ์—†์ด ๊ฒ€์ƒ‰๋˜์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๋Ÿฐํƒ€์ž„์— pytorch sys.path.insert(0, torch.__path__[0] ์˜ ์„ค์น˜ ๊ฒฝ๋กœ๋ฅผ ์‚ฝ์ž…ํ•˜๊ธฐ

๋‚ด ๋ฌธ์ œ๋Š” Detectron์ด ๋ถ„๋ช…ํžˆ Caffe2/core์˜ ํ—ค๋” ํŒŒ์ผ์„ ์›ํ•˜์ง€๋งŒ Anacdona์˜ Pytorch ๊ณต์‹ ๋นŒ๋“œ์—๋Š” Caffe2/core๊ฐ€ ํฌํ•จ๋˜์–ด ์žˆ์ง€ ์•Š๋‹ค๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์ถ”๊ฐ€ pytorch ์„ค์น˜๋ฅผ ์ˆ˜ํ–‰ํ•˜๊ณ  ํ•ด๋‹น ๊ฒฝ๋กœ๋ฅผ Caffe2_Dir์— ์ถ”๊ฐ€ํ•ด์•ผ ํ•ฉ๋‹ˆ๊นŒ?

~/.conda/envs/dl/lib/python2.7/site-packages" ๋””๋ ‰ํ† ๋ฆฌ์— "torch.pth"๋ฅผ ์ถ”๊ฐ€ํ•˜์—ฌ ์ด ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ–ˆ์Šต๋‹ˆ๋‹ค. "libcaffe2_detectron_ops_gpu.so"๊ฐ€ "site-packages/torch" ๋””๋ ‰ํ† ๋ฆฌ์— ์žˆ๊ธฐ ๋•Œ๋ฌธ์ž…๋‹ˆ๋‹ค.

torch.pth์—๋Š” ํ•˜๋‚˜์˜ ๊ฒฝ๋กœ๋งŒ ํฌํ•จ๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค.
/home/ruixin/.conda/envs/dl/lib/python2.7/site-packages/torch

์ถ”์‹ : ์ด ์˜ค๋ฅ˜๋Š” ์•„๋ž˜ ๊ธฐ๋Šฅ์„ ์‹คํ–‰ํ•ฉ๋‹ˆ๋‹ค. "ops_path = os.path.join(prefix, 'lib/libcaffe2_detectron_ops_gpu.so')"๊ฐ€ .so ํŒŒ์ผ์„ ๊ฐ€์ ธ์˜ค๋Š” ๊ฒƒ์„ ๋ณผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

def get_detectron_ops_lib():
    """Retrieve Detectron ops library."""
    # Candidate prefixes for the detectron ops lib path
    # print("=> path list = :\n", _CMAKE_INSTALL_PREFIX, "\n", sys.prefix, '\n', sys.exec_prefix, '\n', sys.path)
    prefixes = [_CMAKE_INSTALL_PREFIX, sys.prefix, sys.exec_prefix] + sys.path
    # Search for detectron ops lib
    # print("=> prefixes = {}".format(prefixes))
    for prefix in prefixes:
        ops_path = os.path.join(prefix, 'lib/libcaffe2_detectron_ops_gpu.so')
        # print("=> ops_path = {}".format(ops_path))
        if os.path.exists(ops_path):
            # TODO(ilijar): Switch to using a logger
            print('Found Detectron ops lib: {}'.format(ops_path))
            break
    assert os.path.exists(ops_path), \
        ('Detectron ops lib not found; make sure that your Caffe2 '
         'version includes Detectron module')
    return ops_path

@farleylai

๊ท€ํ•˜์˜ ์ œ์•ˆ์„ ์‹œ๋„ํ–ˆ์ง€๋งŒ ์ž‘๋™ํ•˜์ง€ ์•Š์•˜๊ณ  ๋™์ผํ•œ ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.
๋ˆ„๊ตฌ๋“ ์ง€ ์ œ์•ˆํ•  ๋‹ค๋ฅธ ํ•ด๊ฒฐ ๋ฐฉ๋ฒ•์ด ์žˆ์Šต๋‹ˆ๊นŒ?

@zabatani

libcaffe2_detectron_ops_gpu.so ์ด lib/ ์•„๋ž˜์˜ torch.__path__[0] lib/ ๊ฒฝ๋กœ์— ์žˆ๋Š”์ง€ ํ™•์ธํ•ฉ๋‹ˆ๊นŒ?
์„ค์น˜์— ๋”ฐ๋ผ ๋‹ค๋ฅธ ์œ„์น˜์— ์žˆ๊ฑฐ๋‚˜ torch.__path__ ์—ฌ๋Ÿฌ ํ•ญ๋ชฉ์ด ์žˆ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
๋‚ด์šฉ์„ ์ธ์‡„ํ•˜๊ณ  ๋‚˜์—ดํ•˜์—ฌ ์œ„์น˜๋ฅผ ์ฐพ์Šต๋‹ˆ๋‹ค.
๋˜๋Š” ํ™˜๊ฒฝ ๋ณ€์ˆ˜ PATH์— ์ถ”๊ฐ€ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

pytorch-nightly๋ฅผ ์„ค์น˜ํ•˜์—ฌ caffe2๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒฝ์šฐ libcaffe2_detectron_ops_gpu.so ( /home/ruixin/.conda/envs/dl/lib/python2.7/site-packages/torch ์™€ ๊ฐ™์€ ์œ„์น˜์— ์žˆ์Œ)๋ฅผ /usr/lib ๋˜๋Š” ๊ฐ€์ƒ ํ™˜๊ฒฝ์˜ lib ๋ณต์‚ฌํ•˜๊ฑฐ๋‚˜ ์—ฐ๊ฒฐํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค
๊ทธ๊ฒƒ์€ ๋‚˜๋ฅผ ์œ„ํ•ด ์ž‘๋™ํ•ฉ๋‹ˆ๋‹ค.

PYTHONPATH ํ™˜๊ฒฝ ๋ณ€์ˆ˜๋ฅผ ์ถ”๊ฐ€ํ•œ ํ›„: /path/to/pytorch/build, ํ…Œ์ŠคํŠธ ํ†ต๊ณผ

@suica ๊ทธ๊ฒƒ์€ ๋‚˜๋ฅผ ์œ„ํ•ด ์ผํ–ˆ์Šต๋‹ˆ๋‹ค. /ur/lib ๋งํฌ๋ฅผ ๋„ฃ์„ ์ˆ˜ ์—†์œผ๋ฉฐ virtualenv ๋Œ€์‹  conda๋ฅผ ์‚ฌ์šฉํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๋‚˜๋Š” ๊ทธ๊ฒƒ์„ /path/to/conda/env/lib ๋„ฃ๊ณ  ์ž‘๋™ํ•ฉ๋‹ˆ๋‹ค. ๊ฐ์‚ฌ ํ•ด์š”!

@genekogan ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค! ์•„์ฃผ ๊ฐ„๋‹จํ•œ ๊ฐˆ๋“ฑ์„ ํ•ด๊ฒฐํ–ˆ์Šต๋‹ˆ๋‹ค~

๋ฐœ์ƒํ•œ ๋ฌธ์ œ๋ฅผ ์–ด๋–ป๊ฒŒ ํ•ด๊ฒฐํ–ˆ์Šต๋‹ˆ๊นŒ?

๋‚˜๋Š” ๊ฐ™์€ ๋ฌธ์ œ๊ฐ€ ์žˆ์ง€๋งŒ colab์—์„œ ์–ด๋–ค ์•„์ด๋””์–ด๊ฐ€ ์žˆ์Šต๋‹ˆ๊นŒ?

@genekogan ๋„์™€์ฃผ์‹œ๊ฒ ์–ด์š”? ์ผ๋ฐ˜ ํ…Œ์ŠคํŠธ๋Š” ๋ชจ๋‘ "/usr/local/lib/libcaffe2_detectron_ops_gpu.so found" ์ •๋ณด๋ฅผ ์ „๋‹ฌํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์‚ฌ์šฉ์ž ์ง€์ • ์ž‘์—…์„ ํ…Œ์ŠคํŠธํ•˜๊ณ  test_zero_even_op.py๋ฅผ ์‹คํ–‰ํ•˜๋ฉด "์‚ฌ์šฉ์ž ์ง€์ • ์ž‘์—… ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ '/path/to/to/detectron/build/libcaffe2_detectron_custom_ops_gpu์—์„œ ์ฐพ์„ ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค"๋ผ๋Š” ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.
์ด ๋ฌธ์ œ๋ฅผ ์–ด๋–ป๊ฒŒ ํ•ด๊ฒฐํ•˜์…จ๋Š”์ง€ ์•Œ๋ ค์ฃผ์‹ค ์ˆ˜ ์žˆ๋‚˜์š”? ์ €๋„ ๊ฐ™์€ ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ–ˆ์Šต๋‹ˆ๋‹ค

@genekogan ๋„์™€์ฃผ์‹œ๊ฒ ์–ด์š”? ์ผ๋ฐ˜ ํ…Œ์ŠคํŠธ๋Š” ๋ชจ๋‘ "/usr/local/lib/libcaffe2_detectron_ops_gpu.so found" ์ •๋ณด๋ฅผ ์ „๋‹ฌํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์‚ฌ์šฉ์ž ์ง€์ • ์ž‘์—…์„ ํ…Œ์ŠคํŠธํ•˜๊ณ  test_zero_even_op.py๋ฅผ ์‹คํ–‰ํ•˜๋ฉด "์‚ฌ์šฉ์ž ์ง€์ • ์ž‘์—… ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ '/path/to/to/detectron/build/libcaffe2_detectron_custom_ops_gpu์—์„œ ์ฐพ์„ ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค"๋ผ๋Š” ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.
์ด ๋ฌธ์ œ๋ฅผ ์–ด๋–ป๊ฒŒ ํ•ด๊ฒฐํ•˜์…จ๋Š”์ง€ ์•Œ๋ ค์ฃผ์‹ค ์ˆ˜ ์žˆ๋‚˜์š”? ์ €๋„ ๊ฐ™์€ ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ–ˆ์Šต๋‹ˆ๋‹ค

ํ•ด๊ฒฐํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค, ์ฃ„์†กํ•ฉ๋‹ˆ๋‹ค.

๋‹ค์Œ์„ ์‚ฌ์šฉํ•˜์—ฌ ๋„คํŠธ์›Œํฌ๋ฅผ ํ›ˆ๋ จํ•˜๋ ค๊ณ  ํ•  ๋•Œ๋„ ๋™์ผํ•œ ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.
python2 ๋„๊ตฌ/train_net.py \
--cfg ๊ตฌ์„ฑ/DensePose_ResNet50_FPN_single_GPU.yaml \
OUTPUT_DIR /tmp/๊ฒ€์ถœ๊ธฐ ์ถœ๋ ฅ
Detectron ๋ฐ caffe2 ์„ค์น˜ ํ…Œ์ŠคํŠธ ๋ชจ๋‘ ์„ฑ๊ณต์ ์œผ๋กœ ํ†ต๊ณผํ–ˆ์Šต๋‹ˆ๋‹ค.
์˜ค๋ฅ˜ ์ถœ๋ ฅ:
๋ฐœ๊ฒฌ๋œ Detectron ops ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ: /usr/local/lib/python2.7/site-packages/torch/lib/libcaffe2_detectron_ops_gpu.so
์—ญ์ถ”์ (๊ฐ€์žฅ ์ตœ๊ทผ ํ˜ธ์ถœ ๋งˆ์ง€๋ง‰):
ํŒŒ์ผ "/usr/local/densepose/tools/train_net.py", 35ํ–‰,
c2_utils.import_custom_ops()
import_custom_ops์˜ ํŒŒ์ผ "/usr/local/densepose/detectron/utils/c2.py", 39ํ–‰
custom_ops_lib = envu.get_custom_ops_lib()
get_custom_ops_lib์˜ ํŒŒ์ผ "/usr/local/densepose/detectron/utils/env.py", 85ํ–‰
'\'{}\''.format(custom_ops_lib)์—์„œ ์‚ฌ์šฉ์ž ์ง€์ • ์ž‘์—… ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ์ฐพ์„ ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.
AssertionError: '/usr/local/densepose/build/libcaffe2_detectron_custom_ops_gpu.so'์—์„œ ์‚ฌ์šฉ์ž ์ง€์ • ์ž‘์—… ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋ฅผ ์ฐพ์„ ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.

@genekogan ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค! ๋‚˜๋Š” ๊ทธ๊ฒƒ์„ ํ•ด๊ฒฐํ–ˆ๋‹ค. ์•„์ฃผ ๊ฐ„๋‹จํ•œ ๊ฐˆ๋“ฑ~

@genekoganไฝ ่ƒฝๅธฎๅธฎๆˆ‘ๅ—๏ผŸไธ€่ˆฌๆต‹่ฏ•ๅ…จ้ƒจ้€š่ฟ‡ไฟกๆฏ"/usr/local/lib/libcaffe2_detectron_ops_gpu.so ๋ฐœ๊ฒฌ.ๅœจ'/ path / to / to / detectron / build / libcaffe2_detectron_custom_ops_gpuไธญๆ‰พๅˆฐ่‡ชๅฎšไน‰ๆ“ไฝœๅบ“โ€ใ€‚
์ด ๋ฌธ์ œ๋ฅผ ์–ด๋–ป๊ฒŒ ํ•ด๊ฒฐํ•˜์…จ๋Š”์ง€ ๋ง์”€ํ•ด ์ฃผ์‹œ๊ฒ ์Šต๋‹ˆ๊นŒ? ๋‚˜๋Š” ๊ฐ™์€ ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ–ˆ์Šต๋‹ˆ๋‹ค

๊ทธ๋ฆฌ๊ณ  'libcaffe2_detectron_custom_ops_gpu.so' ํŒŒ์ผ์„ ์ฐพ์„ ์ˆ˜ ์—†๋Š” ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค. caffe2/pytorch๊ฐ€ PYTHONPATH ํ™˜๊ฒฝ ๋ณ€์ˆ˜์— ์žˆ๋Š”์ง€ ํ™•์ธํ–ˆ์Šต๋‹ˆ๋‹ค.

ํ•ด๊ฒฐ์ฑ…์„ ์ฐพ์•˜์Šต๋‹ˆ๋‹ค

/๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ/densepose/detectron/utils/env.py์— ๋Œ€ํ•œ ๊ฒฝ๋กœ
19ํ–‰ ํŽธ์ง‘
_CMAKE_INSTALL_PREFIX = '/usr/lib'
์—๊ฒŒ
_CMAKE_INSTALL_PREFIX = 'cmake ๋˜๋Š” setup.py ํŒŒ์ผ์„ ์‚ฌ์šฉํ•˜์—ฌ ๋นŒ๋“œํ•œ pytorch์˜ lib ํด๋” ๊ฒฝ๋กœ/'
๊ทธ๋ฆฌ๊ณ  ๋‹ค์‹œ ์‹คํ–‰:)

caffe2์— detectron ๋ชจ๋“ˆ์ด ์—†์Šต๋‹ˆ๋‹ค. pytorch๋ฅผ ํ†ตํ•ด caffe2๋ฅผ ์„ค์น˜ํ–ˆ์Šต๋‹ˆ๋‹ค. ์ œ๋Œ€๋กœ ์„ค์น˜๋˜๊ณ  ์žˆ์ง€๋งŒ ๊ฐ์ง€๊ธฐ ๋ชจ๋“ˆ์ด ์—†์Šต๋‹ˆ๋‹ค. ๋„์›€์„ ๋ฐ›์„ ์ˆ˜ ์žˆ๋‚˜์š”?

๋ฏธ๋ฆฌ ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค

์—ฌ๋Ÿฌ๋ถ„, ์ด ์ฝ”๋“œ ๋ธ”๋ก์˜ ๋ชฉํ‘œ๋Š” "libcaffe2_detectron_ops_gpu.so" ํŒŒ์ผ์„ ์ฐพ๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค.
", ๊ทธ๋ž˜์„œ ๊ทธ๋ƒฅ ์ด๋ ‡๊ฒŒ ํ•˜์„ธ์š”:

  1. ์Šค์Šค๋กœ ํŒŒ์ผ ์ฐพ๊ธฐ: find / -name libcaffe2_detectron_ops_gpu.so
  2. ๊ฒ€์ƒ‰ ๋ชฉ๋ก์— ๊ฒ€์ƒ‰ ๊ฒฐ๊ณผ ์ถ”๊ฐ€

@genekogan ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค! ๋‚˜๋Š” ๊ทธ๊ฒƒ์„ ํ•ด๊ฒฐํ–ˆ๋‹ค. ์•„์ฃผ ๊ฐ„๋‹จํ•œ ๊ฐˆ๋“ฑ~

๊ทธ๊ฒƒ์„ ํ•ด๊ฒฐํ•˜๋Š” ๋ฐฉ๋ฒ•, ๋‚˜๋Š” ๊ฐ™์€ ๋ฌธ์ œ๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค, ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค

์ด ํŽ˜์ด์ง€๊ฐ€ ๋„์›€์ด ๋˜์—ˆ๋‚˜์š”?
0 / 5 - 0 ๋“ฑ๊ธ‰