Scikit-learn: ๊ด€๋ จ์„ฑ ๋ฒกํ„ฐ ๋จธ์‹ (RVM)

์— ๋งŒ๋“  2013๋…„ 01์›” 03์ผ  ยท  68์ฝ”๋ฉ˜ํŠธ  ยท  ์ถœ์ฒ˜: scikit-learn/scikit-learn

RVM์€ ํšŒ๊ท€ ๋ฐ ๋ถ„๋ฅ˜ ์ž‘์—…์— ๋Œ€ํ•œ ํฌ์†Œ ์†”๋ฃจ์…˜์„ ์–ป๊ธฐ ์œ„ํ•œ ๋ฒ ์ด์ง€์•ˆ ํ”„๋ ˆ์ž„์›Œํฌ์ž…๋‹ˆ๋‹ค. SVM(Support Vector Machine)๊ณผ ๋™์ผํ•œ ํ˜•ํƒœ์˜ ๋ชจ๋ธ์„ ์‚ฌ์šฉํ•˜์˜€๋‹ค. SVM์˜ ๋‹ค์Œ๊ณผ ๊ฐ™์€ ๋‹จ์ ์„ ํ•ด๊ฒฐํ•ฉ๋‹ˆ๋‹ค.
-- SVM์˜ ๊ธฐ์ € ํ•จ์ˆ˜์˜ ์ˆ˜๋Š” ํ›ˆ๋ จ ์„ธํŠธ์˜ ํฌ๊ธฐ์— ๋”ฐ๋ผ ์„ ํ˜•์ ์œผ๋กœ ์ฆ๊ฐ€ํ•ฉ๋‹ˆ๋‹ค.
RVM์—์„œ ์šฐ๋ฆฌ๋Š” 0 ๊ธฐ๋ฐ˜์œผ๋กœ ์‹œ์ž‘ํ•˜์—ฌ ์ˆ˜๋ ด๋  ๋•Œ๊นŒ์ง€ ๊ธฐ์ € ๊ธฐ๋Šฅ ์„ธํŠธ๋ฅผ ์ ์ง„์ ์œผ๋กœ ์—…๋ฐ์ดํŠธ(์ถ”๊ฐ€/์‚ญ์ œ)ํ•ฉ๋‹ˆ๋‹ค.

-- SVM ์˜ˆ์ธก์€ ํ™•๋ฅ ์ ์ด์ง€ ์•Š์ง€๋งŒ RVM์€ ํ™•๋ฅ ์ ์ž…๋‹ˆ๋‹ค.

-- SVM์—์„œ๋Š” RVM์˜ ๊ฒฝ์šฐ๊ฐ€ ์•„๋‹Œ ๋งˆ์ง„ ์ ˆ์ถฉ ๋งค๊ฐœ๋ณ€์ˆ˜ 'C'๋ฅผ ์ถ”์ •ํ•˜๋Š” ๊ฒƒ์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.

-- SVM ์ปค๋„์€ ์–‘์˜ ์ •๋ถ€ํ˜ธ์—ฌ์•ผ ํ•ฉ๋‹ˆ๋‹ค. RVM์—์„œ๋Š” ๋ชจ๋“  ์ปค๋„์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

dlib http://dlib.net/dlib/svm/rvm_abstract.h.html์— ์ด๋ฏธ ๊ตฌํ˜„๋˜์–ด ์žˆ์œผ๋ฉฐ http://www.vectoranomaly.com/downloads/downloads.htm์— matlab ๊ตฌํ˜„๋„ ์žˆ์Šต๋‹ˆ๋‹ค
scikit-learn์— ์ถ”๊ฐ€ํ•˜๋ฉด ์ข‹์„ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.

์ฐธ์กฐ:
1- Tipping, M. E. and A. C. Faul (2003). Fast marginal likelihood maximisation for sparse Bayesian models. In C. M. Bishop and B. J. Frey (Eds.), Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, Key West, FL, Jan 3-6.

2- Tipping, M. E. (2001). Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research 1, 211โ€“244.

New Feature

๊ฐ€์žฅ ์œ ์šฉํ•œ ๋Œ“๊ธ€

์•ˆ๋…•ํ•˜์„ธ์š” @amueller ๊ทธ๋ฆฌ๊ณ  ์—ฌ๋Ÿฌ๋ถ„! ์šฐ๋ฆฌ๋Š” ์ด ์Šค๋ ˆ๋“œ๋ฅผ ๋ณด๊ณ  RVM์˜ sklearn ํ˜ธํ™˜ ๋ฒ„์ „(https://github.com/Mind-the-Pineapple/sklearn-rvm)์„ ๊ตฌํ˜„ํ•˜๊ธฐ๋กœ ๊ฒฐ์ •ํ–ˆ์Šต๋‹ˆ๋‹ค. ์šฐ๋ฆฌ๋Š” JamesRitchie ๊ตฌํ˜„์„ ๊ธฐ๋ฐ˜์œผ๋กœ ํ–ˆ์Šต๋‹ˆ๋‹ค. ๋ˆ„๊ตฐ๊ฐ€ ๊ธฐ๊บผ์ด ๊ทธ๊ฒƒ์„ ์‚ดํŽด๋ณด๊ณ  ํ”ผ๋“œ๋ฐฑ๊ณผ ๊ธฐ์—ฌ(@themrzmaster)๋ฅผ ํ™˜์˜ํ•œ๋‹ค๋ฉด ์ข‹์„ ๊ฒƒ์ž…๋‹ˆ๋‹ค. :)
์šฐ๋ฆฌ๋Š” ์ด ๋ฆฌํฌ์ง€ํ† ๋ฆฌ๋ฅผ ์œ ์ง€ ๊ด€๋ฆฌํ•˜๊ณ  ๋ฒ„์ „ 0.2์—์„œ ๋น ๋ฅธ ๋ฒ„์ „์„ ๊ตฌํ˜„ํ•  ๊ณ„ํš์ด ์žˆ์œผ๋ฉฐ ์–ธ์  ๊ฐ€ ์ด ์ฝ”๋“œ๊ฐ€ scikit-learn ๋ฉ”์ธ ๋ฆฌํฌ์ง€ํ† ๋ฆฌ์— ๋„์›€์ด ๋  ์ˆ˜ ์žˆ๊ธฐ๋ฅผ ๋ฐ”๋ž๋‹ˆ๋‹ค. ๐Ÿ˜Š ๐Ÿ

๋ชจ๋“  68 ๋Œ“๊ธ€

๋‹ค์‹œ ์ฝ์–ด๋ด์•ผ ํ•˜๊ฒ ์ง€๋งŒ ์ผ๋ฐ˜์ ์œผ๋กœ RVM์ด ์ข‹์€ ์ถ”๊ฐ€ ๊ธฐ๋Šฅ์ด ๋  ๊ฒƒ์ด๋ผ๊ณ  ์ƒ๊ฐํ•ฉ๋‹ˆ๋‹ค.
dlib๋Š” ํ˜ธํ™˜๋˜์–ด์•ผ ํ•˜๋Š” ๋ถ€์ŠคํŠธ ๋ผ์ด์„ ์Šค์ž…๋‹ˆ๋‹ค. ํ•˜์ง€๋งŒ ๋ถ€์ŠคํŠธ๊ฐ€ ๋งŽ์€ ์ฝ”๋”ฉ ์Šคํƒ€์ผ ๋•Œ๋ฌธ์— ๋ž˜ํ•‘ํ•˜๊ธฐ๊ฐ€ ์‰ฝ์ง€ ์•Š์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
๋ฌธ์ œ๊ฐ€ SMO๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ตœ์ ํ™”๋˜์—ˆ์Šต๋‹ˆ๊นŒ? SMO๋ฅผ ๊ตฌํ˜„ํ•˜๋Š” ๊ฒƒ์ด ํ•ฉ๋ฆฌ์ ์ž…๋‹ˆ๊นŒ?

๋‚ด ๋น„์ˆ์„ ์žก์•„์•ผ ํ•ด์š”.

ARD์™€ RVM์˜ ๊ด€๊ณ„๋Š” ๋ฌด์—‡์ž…๋‹ˆ๊นŒ? RVM์€ ARD์˜ "๊ธฐ๋ณธ ๊ธฐ๋Šฅ" ๋ฒ„์ „์ž…๋‹ˆ๊นŒ?

Btw Generalized Linear Models ์„น์…˜์— ์ผ๋ฐ˜ํ™”๋œ ๋ชจ๋ธ์ด ํฌํ•จ๋˜์–ด ์žˆ์ง€ ์•Š๋‹ค๋Š” ์‚ฌ์‹ค์— ๊ดด๋กœ์›Œํ•œ ์‚ฌ๋žŒ์ด ์žˆ์Šต๋‹ˆ๊นŒ?

์ž, ๊ทธ๋Ÿผ sequential sparse learning algorithm Bishop p๋ฅผ ์‚ฌ์šฉํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. 352 ํŒ”๋กœ์ž‰?
์ž์‹ ์„ ์•Œ์•„๋ณด์„ธ์š” ;)

ARD์—๋„ ๋น„์Šทํ•œ ๋ฐฉ๋ฒ•์ด ์žˆ๋Š”์ง€ ๊ถ๊ธˆํ•ฉ๋‹ˆ๋‹ค. ํ˜„์žฌ ARD ๊ตฌํ˜„์ด ๋งค์šฐ ๋Š๋ฆฌ๊ธฐ ๋•Œ๋ฌธ์— ๋ฉ‹์งˆ ๊ฒƒ์ž…๋‹ˆ๋‹ค :-/

์•„๋‹ˆ์š”, RVM์˜ ๊ตฌํ˜„์€ ํ™•์‹คํžˆ SMO๋ฅผ ์‚ฌ์šฉํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. SMO๋Š” SVM ์ตœ์ ํ™”์—๋งŒ ์‚ฌ์šฉ๋œ๋‹ค๊ณ  ์ƒ๊ฐํ•ฉ๋‹ˆ๋‹ค.
์˜ˆ, ์ฐธ์กฐ 1 ํŽ˜์ด์ง€ 7์—์„œ sequential sparse learning algorithm ๋ฅผ ์‚ฌ์šฉํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. (bishop p 352์— ์žˆ์Šต๋‹ˆ๊นŒ? ์–ด๋Š ๊ฒƒ์ž…๋‹ˆ๊นŒ?) ์ด ์•Œ๊ณ ๋ฆฌ์ฆ˜์€ ์ถฉ๋ถ„ํžˆ "๊ฐ„๋‹จํ•˜๊ณ " dlib๋ฅผ ์‚ฌ์šฉํ•˜์ง€ ์•Š๊ณ  ์ž‘์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ํŒŒ์ด์ฌ์œผ๋กœ ์ž‘์„ฑํ•œ ๋‹ค์Œ ์ตœ์ ํ™”๋ฅผ ์œ„ํ•ด cython์„ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์— ๋Œ€ํ•ด ์ƒ๊ฐํ•˜๊ณ  ์žˆ์—ˆ์Šต๋‹ˆ๋‹ค. ์ด ๊ฒฝ์šฐ ์šฐ๋ฆฌ๋Š” matlab ๊ตฌํ˜„์„ ์ตœ๋Œ€ํ•œ ํ™œ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์–ด๋–ป๊ฒŒ ์ƒ๊ฐํ•ด?
์–ด์จŒ๋“  C++๋กœ ์ž‘์„ฑํ•˜๋Š” ๊ฒƒ์ด ๊ฐ€๋Šฅํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์ด๋ฅผ ์œ„ํ•ด์„œ๋Š” C++๋กœ ๋œ ์ข‹์€ ์„ ํ˜• ๋Œ€์ˆ˜ํ•™ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๊ฐ€ ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค. ๊ธฐ๋ณธ์ ์œผ๋กœ scikit-learn์ด ํ•จ๊ป˜ ์ œ๊ณต๋˜๋Š”์ง€ ํ™•์‹คํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

Bishop์€ "๊ธฐ๊ณ„ ํ•™์Šต ๋ฐ ํŒจํ„ด ์ธ์‹"์ž…๋‹ˆ๋‹ค.

์•Œ๊ณ ๋ฆฌ์ฆ˜์„ ์ œ๋Œ€๋กœ ์ดํ•ดํ•˜๋Š” ๊ฒƒ์€ ์•„๋งˆ๋„ ์™„์ „ํžˆ ์‰ฝ์ง€ ์•Š์„ ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์ด๊ฒƒ์„ ์‹œ์ž‘ํ•˜๊ณ  ์‹ถ๋‹ค๋ฉด ๋ถ„๋ช…ํžˆ ๋” ํฐ ํ”„๋กœ์ ํŠธ๊ฐ€ ๋  ๊ฒƒ์ž…๋‹ˆ๋‹ค. ๊ด€์‹ฌ์ด ์žˆ๊ณ  ์–ด๋–ค ์‹์œผ๋กœ๋“  ๊ตฌํ˜„ํ•˜๋ ค๋Š” ๊ฒฝ์šฐ ์ง„ํ–‰ํ•˜์‹ญ์‹œ์˜ค.

๊ทธ๊ฒƒ์„ ๊ตฌํ˜„ํ•˜๋Š” ๊ฒƒ๋„ scikit-learn์— ๋„ฃ๋Š” ๊ฒƒ๊ณผ๋Š” ์ƒ๋‹นํžˆ ๋‹ค๋ฆ…๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์—๋Š” ํ…Œ์ŠคํŠธ, ๋ฌธ์„œ, ์‚ฌ์šฉ์ž ๊ฐ€์ด๋“œ ๋ฐ ์˜ˆ์œ ์ฝ”๋“œ ์ž‘์„ฑ๋„ ํฌํ•จ๋ฉ๋‹ˆ๋‹ค.

์ฒซ ๋ฒˆ์งธ ์‹œ๋„์—์„œ๋Š” ํ™•์‹คํžˆ numpy๋ฅผ ์‚ฌ์šฉํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ๋‚ด๋ถ€์ ์œผ๋กœ blas๋ฅผ ์‚ฌ์šฉํ•˜๋ฏ€๋กœ ๋งค์šฐ ๋น ๋ฆ…๋‹ˆ๋‹ค.
Cython์„ ์‚ฌ์šฉํ•˜์—ฌ ์†๋„๋ฅผ ๋†’์ด๋Š” ๊ฒƒ์€ ํŒŒ์ด์ฌ ์˜ค๋ฒ„ํ—ค๋“œ๊ฐ€ ๋งŽ์€ ๊ฒฝ์šฐ์—๋งŒ ์˜๋ฏธ๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค. BLAS ํ˜ธ์ถœ์— ๋ชจ๋“  ์‹œ๊ฐ„์„ ํ• ์• ํ•œ๋‹ค๋ฉด Cython์„ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์€ ์˜๋ฏธ๊ฐ€ ์—†์Šต๋‹ˆ๋‹ค.

Cython๊ณผ numpy์—์„œ๋Š” ๊ดœ์ฐฎ์Šต๋‹ˆ๋‹ค. ์ €๋Š” ๋น„์ˆ์ด RVM์— ๋Œ€ํ•ด ์ด์•ผ๊ธฐํ•˜๋Š” ๊ฒƒ์„ ๋ชฐ๋ž์Šต๋‹ˆ๋‹ค.
ARD ๋ฐ RVM ๊ด€๊ณ„์šฉ. ์ €๋Š” ARD์— ๋Œ€ํ•ด ์ž˜ ๋ชจ๋ฆ…๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์ฐธ์กฐ 2์—์„œ ์ €์ž๋Š” RVM์ด ARD๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ ํ•œ๋‹ค๊ณ  ๋งํ–ˆ์Šต๋‹ˆ๋‹ค. (213) 8ํ–‰.
์–ด์จŒ๋“  ARD๋Š” ์–ด๋–ป๊ฒŒ ์ž‘๋™ํ•ฉ๋‹ˆ๊นŒ?

ARD๋Š” Bishops ์ฑ…๊ณผ ์‚ฌ์šฉ ์„ค๋ช…์„œ ์—๋„ ์„ค๋ช…๋˜์–ด ์žˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๊ฒƒ์€ ๊ฐ€์ค‘์น˜์— ๋Œ€๊ฐ์„  ๊ฐ€์šฐ์‹œ์•ˆ์„ ์šฐ์„ ์ ์œผ๋กœ ๋†“๊ณ  RVM์ด ์ˆ˜ํ–‰ํ•˜๋Š” ๊ฒƒ๊ณผ ๋™์ผํ•œ ๋ถ„์‚ฐ์„ ์ถ”์ •ํ•˜๋ ค๊ณ  ์‹œ๋„ํ•ฉ๋‹ˆ๋‹ค. ๊ทธ ๋งž์Šต๋‹ˆ๊นŒ?

๋‚˜๋Š” ์‹ฌํŒ์ด ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์–ธ๊ธ‰ํ–ˆ๋‹ค๋Š” ๊ฒƒ์„ ๊นจ๋‹ฌ์•˜์Šต๋‹ˆ๋‹ค.

http://books.nips.cc/papers/files/nips20/NIPS2007_0976.pdf

์šฐ๋ฆฌ๊ฐ€ ์‚ฌ์šฉํ•˜๋Š” ๊ตฌํ˜„์ด ์•„๋‹™๋‹ˆ๋‹ค. @vmichel์ด ๊ตฌํ˜„ํ–ˆ๋‹ค๊ณ  ์ƒ๊ฐํ•ฉ๋‹ˆ๋‹ค.
๋น„์ˆ ์ ‘๊ทผ ๋ฐฉ์‹์€ ์ด ๋ฌธ์„œ๊ฐ€ ๋‹ค์Œ๊ณผ ์œ ์‚ฌํ•œ ๊ณ ์ • ์†Œ์ˆ˜์  ์ ‘๊ทผ ๋ฐฉ์‹์„ ๋ชฉ์ ์œผ๋กœ ํ•ฉ๋‹ˆ๋‹ค.
์ขŒํ‘œ ํ•˜๊ฐ• ์ ‘๊ทผ. ์ด ์ฝ”๋“œ๋Š” ํ™•์‹คํžˆ ์‚ฌ๋ž‘์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค...

@agramfort ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค. ๊ถ๊ธˆํ–ˆ์Šต๋‹ˆ๋‹ค. ์ž์„ธํžˆ๋Š” ์•ˆ ๊ฐ€๋ดค์ง€๋งŒ ๋…ผ๋ฌธ๋งŒ ์ฐธ๊ณ ๊ฐ€ ๋˜๋Š” ์ค„ ์•Œ์•˜๋Š”๋ฐ...

์‚ฌ์šฉ๋œ bishop์˜ ์žฅ์„ ์ธ์šฉํ•˜์—ฌ ๋‹ค๋ฅธ ๋ฌธ์„œ๋ฅผ ๋Œ€์‹  ๊ตฌํ˜„ํ•ด์•ผ ํ•œ๋‹ค๋Š” ์„ค๋ช…์„ ์ถ”๊ฐ€ํ•ด ์ฃผ์‹œ๋ฉด ๋Œ€๋‹จํžˆ ๊ฐ์‚ฌํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

(ํ˜„์žฌ ํ…Œ์ŠคํŠธ ์Šค์œ„ํŠธ์—์„œ ๊ฐ€์žฅ ๋Š๋ฆฐ ๊ฒƒ ์ค‘ ํ•˜๋‚˜๋Š” ๊ณตํ†ต ํ…Œ์ŠคํŠธ์˜ ๋ณด์Šคํ„ด ๋ฐ์ดํ„ฐ ์„ธํŠธ์— ARD๋ฅผ ๋งž์ถ”๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค)

@agramfort ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค. ๊ถ๊ธˆํ–ˆ์Šต๋‹ˆ๋‹ค. ์ž์„ธํžˆ๋Š” ์•ˆ ๊ฐ€๋ดค์ง€๋งŒ ๋…ผ๋ฌธ๋งŒ ์ฐธ๊ณ ๊ฐ€ ๋˜๋Š” ์ค„ ์•Œ์•˜๋Š”๋ฐ...

์‚ฌ์šฉ๋œ bishop์˜ ์žฅ์„ ์ธ์šฉํ•˜์—ฌ ๋‹ค๋ฅธ ๋ฌธ์„œ๋ฅผ ๋Œ€์‹  ๊ตฌํ˜„ํ•ด์•ผ ํ•œ๋‹ค๋Š” ์„ค๋ช…์„ ์ถ”๊ฐ€ํ•ด ์ฃผ์‹œ๋ฉด ๋Œ€๋‹จํžˆ ๊ฐ์‚ฌํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

์ฐธ์กฐ : https://github.com/scikit-learn/scikit-learn/pull/1530

Btw๋Š” ์ผ๋ฐ˜ํ™” ์„ ํ˜• ์„น์…˜์ด
๋ชจ๋ธ์— ์ผ๋ฐ˜ํ™”๋œ ๋ชจ๋ธ์ด ํฌํ•จ๋˜์–ด ์žˆ์ง€ ์•Š์Šต๋‹ˆ๊นŒ?

์ด๊ฒƒ์€ ๋กœ์ง€์Šคํ‹ฑ ํšŒ๊ท€๋ฅผ ์ˆ˜ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

ARD์—๋„ ๋น„์Šทํ•œ ๋ฐฉ๋ฒ•์ด ์žˆ๋Š”์ง€ ๊ถ๊ธˆํ•ฉ๋‹ˆ๋‹ค. ๊ทธ๊ฒƒ์€ ๋ฉ‹์ง„ ๊ฒƒ์ž…๋‹ˆ๋‹ค
ํ˜„์žฌ ARD ๊ตฌํ˜„์€ ์ƒ๋‹นํžˆ ๋Š๋ฆฝ๋‹ˆ๋‹ค :-/

ARD์˜ ๊ฐ€์žฅ ์œ ๋งํ•œ Rapid solver๋Š” ๋‹ค์Œ์„ ๊ตฌํ˜„ํ•˜๋Š” ๊ฒƒ์ด๋ผ๊ณ  ์ƒ๊ฐํ•ฉ๋‹ˆ๋‹ค.
๋‹ค์Œ์—์„œ ๋…ธ์ถœ๋œ ์ „๋žต:
http://books.nips.cc/papers/files/nips20/NIPS2007_0976.pdf

์–ผ๋งˆ ์ „์— ์ž‘์„ฑํ•œ ์ฝ”๋“œ๋ฅผ ์š”์ ์œผ๋กœ ์˜ฎ๊น๋‹ˆ๋‹ค.

๋ˆ„๊ตฐ๊ฐ€ ARD ์ž‘์—…์„ ํ•˜๊ณ  ์‹ถ๋‹ค๋ฉด ์œ ์šฉํ•  ์ˆ˜ ์žˆ๋‹ค๊ณ  ์ƒ๊ฐํ–ˆ์Šต๋‹ˆ๋‹ค.

https://gist.github.com/4494613

๊ฒฝ๊ณ : ๋งŽ์ด ํ…Œ์ŠคํŠธ๋˜์ง€ ์•Š์•˜์œผ๋ฉฐ ์ •ํ™•์„ฑ์„ ๋ณด์žฅํ•˜์ง€๋Š” ์•Š์ง€๋งŒ
์ž‘๋™ํ•˜๋Š” ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.

@amueller
๋‚˜๋Š” http://books.nips.cc/papers/files/nips20/NIPS2007_0976.pdf ์—์„œ ARD๋ฅผ ๋ถ„์„ํ–ˆ๊ณ  RVM๊ณผ ARD๊ฐ€ ๋™์ผํ•œ ๋ชฉ์  ํ•จ์ˆ˜๋ฅผ ์ตœ์ ํ™”ํ•˜๊ธฐ๋ฅผ ์›ํ•œ๋‹ค๊ณ  ์ƒ๊ฐํ•ฉ๋‹ˆ๋‹ค. ์ฐจ์ด์ ์€ ์ด ๊ธฐ๋Šฅ์„ ์ตœ์ ํ™”ํ•˜๋Š” ๋ฐ ์‚ฌ์šฉ๋œ ๋ฐฉ๋ฒ•์—์„œ ๋‚˜ํƒ€๋‚ฉ๋‹ˆ๋‹ค. RVM์—์„œ ์ €์ž๋“ค์€ ๋Œ€๋ถ€๋ถ„์˜ ๊ฐ€์ค‘์น˜๊ฐ€ 0์— ๊ฐ€๊น๋‹ค๋Š” ๊ฒƒ์„ ์•Œ์•„์ฐจ๋ ธ๊ณ  ์ด๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ "๋น ๋ฅธ" ์•Œ๊ณ ๋ฆฌ์ฆ˜์„ ๋„์ถœํ–ˆ์Šต๋‹ˆ๋‹ค.

์ด์ƒํ•˜๊ฒŒ ๋“ค๋ฆฐ๋‹ค. ๋ชฉ์ ์ด ๊ฐ™์œผ๋ฉด ๊ฐ™์€ ๋ฐฉ๋ฒ•์œผ๋กœ ์ตœ์ ํ™”๋ฅผ ํ•  ์ˆ˜ ์žˆ์–ด์•ผ ํ•˜์ง€ ์•Š๊ฒ ์Šต๋‹ˆ๊นŒ?

๋„ค, ๋‹น์—ฐํžˆ ๊ทธ๋ž˜์•ผ ํ•ฉ๋‹ˆ๋‹ค. ํ•˜์ง€๋งŒ RVM์˜ ์ž‘์„ฑ์ž๋Š” ๋” ๋น ๋ฅด๊ณ  ํฌ์†Œํ•œ ์•Œ๊ณ ๋ฆฌ์ฆ˜์„ ์‚ฌ์šฉํ•˜๊ธฐ ์œ„ํ•ด ๋‹ค๋ฅธ ์ตœ์ ํ™” ์ „๋žต์„ ์‚ฌ์šฉํ–ˆ๋‹ค๊ณ  ์ƒ๊ฐํ•ฉ๋‹ˆ๋‹ค.

@yedtoss ๋‹ค๋ฅธ ์ฐจ์ด์ ์ด ์žˆ๋‹ค๊ณ  ํ™•์‹ ํ•ฉ๋‹ˆ๋‹ค. ์ „์— ๋งํ–ˆ๋“ฏ์ด ์ด๊ฒƒ์€ RVM์ด ๊ธฐ๋Šฅ ๊ณต๊ฐ„์—์„œ ๋˜๋Š” ์ปค๋„๊ณผ ํ•จ๊ป˜ ์ž‘๋™ํ•˜๋Š” ๊ฒƒ์ผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋ ‡์ง€ ์•Š์œผ๋ฉด ARD ๊ตฌํ˜„์„ ๋Œ€์ฒดํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๊นŒ? ํ•˜์ง€๋งŒ ๊ทธ๊ฒƒ์€ ํšŒ๊ท€์ด๊ณ  ๋ถ„๋ฅ˜๋ฅผ ์›ํ•ฉ๋‹ˆ๋‹ค. ๋งž์Šต๋‹ˆ๊นŒ?

@agramfort ARD์™€ RVM์˜ ์ฐจ์ด์ ์— ๋Œ€ํ•ด ์•Œ๊ณ  ์žˆ์Šต๋‹ˆ๊นŒ?

@amueller
์ฒ˜์Œ์— RVM์€ ํšŒ๊ท€ ๊ธฐ์ˆ ์ž…๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ์ €์ž๋Š” ๋ถ„๋ฅ˜์— ์‚ฌ์šฉํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์ œ์‹œํ–ˆ์Šต๋‹ˆ๋‹ค. RVM์€ ๋ชจ๋“  ์ข…๋ฅ˜์˜ ์ปค๋„์„ ์‚ฌ์šฉํ–ˆ์Šต๋‹ˆ๋‹ค.
๋‚ด ๋ง์€ ARD(A New View of Automatic Relevance Determination์˜ ๋ฐฉ์ •์‹ 2)์™€ RVM(ํฌ์†Œ ๋ฒ ์ด์ง€์•ˆ ๋ชจ๋ธ์— ๋Œ€ํ•œ ๋น ๋ฅธ ํ•œ๊ณ„ ๊ฐ€๋Šฅ์„ฑ ์ตœ๋Œ€ํ™”์˜ ๋ฐฉ์ •์‹ 7)์˜ ๋กœ๊ทธ ๊ฐ€๋Šฅ์„ฑ์ด ๋™์ผํ•˜๋‹ค๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค.

๋ฌด์Šจ ์ผ์ด ์ผ์–ด๋‚˜๊ณ  ์žˆ๋Š”์ง€ ์•Œ์•„๋ณด๋ ค๋ฉด ์‹ ๋ฌธ์„ ์ฝ์–ด๋ด์•ผ ํ•  ๊ฒƒ ๊ฐ™์•„์š”....

๋ฏธ์•ˆํ•ด ์–˜๋“ค์•„ ๋‚˜๋Š” ๋ฒ ์ด์ง€์•ˆ ์‚ฌ๋žŒ์ด ์•„๋‹ˆ๋ผ... ์ž˜ ๋ชจ๋ฅด๊ฒ ์–ด
์ž๋ง‰...

RVM์€ Microsoft์—์„œ ํŠนํ—ˆ๋ฅผ ๋ฐ›์•˜์Šต๋‹ˆ๋‹ค.

๋ฏธ์นœ.

@larsmans @amueller ๋ฏธ๊ตญ์—๋Š” RVM์— ๋Œ€ํ•œ ํŠนํ—ˆ๊ฐ€ ์žˆ์ง€๋งŒ ์ €์ž๋Š” ์›น ํŽ˜์ด์ง€์—์„œ GPLv2 Matlab ๊ตฌํ˜„์„ ๊ถŒ์žฅํ•˜๋ฏ€๋กœ ๊ตฌํ˜„ํ•ด๋„ ๊ดœ์ฐฎ์„ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค...
http://www.miketipping.com/sparsebayes.htm

์ตœ์ƒ์˜,
์•ˆ์ ค๋กœ์Šค

@kalxas ๋ผ์ด์„ ์Šค์™€ ํŠนํ—ˆ๋Š” ์ƒ๋‹นํžˆ ์ง๊ตํ•˜๋ฉฐ ํŠนํžˆ GPLv2๋Š” ์†Œํ”„ํŠธ์›จ์–ด ํŠนํ—ˆ๋ฅผ ๋‹ค๋ฃจ์ง€ ์•Š์•˜์Šต๋‹ˆ๋‹ค. ์ด๋Ÿฌํ•œ ๊ตฌํ˜„์— ๋Œ€ํ•œ ๊ท€ํ•˜์˜ ๊ถŒ๋ฆฌ๋Š” GPL์ด ๋ถ€์—ฌํ•œ ๊ถŒ๋ฆฌ์™€ ํŠนํ—ˆ ๋ณด์œ ์ž๊ฐ€ ๋ถ€์—ฌํ•œ ๊ถŒ๋ฆฌ์˜ ๊ต์ฐจ์ ์ž…๋‹ˆ๋‹ค.

์ฆ‰, ์ง€์› ๋ฒกํ„ฐ ๋จธ์‹ ์ด AT&T์— ์˜ํ•ด ํŠนํ—ˆ๋ฅผ ๋ฐ›์•˜์ง€๋งŒ ๊ทธ ํŠนํ—ˆ๋Š” ๋ถ„๋ช…ํžˆ ์ง‘ํ–‰๋˜์ง€ ์•Š์•˜๋‹ค๋Š” ๊ฒƒ์„ ์•Œ๊ฒŒ ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ๋น„์Šทํ•œ ๊ฒƒ์ด RVM์— ๋Œ€ํ•ด ์ž…์ฆ๋  ์ˆ˜ ์žˆ๋‹ค๋ฉด ๊ทธ๊ฒƒ์— ๋Œ€ํ•ด ์ƒ๊ฐ์„ ๋ฐ”๊ฟ€ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

@larsmans ๋‚˜๋Š” dlib ๊ตฌํ˜„์˜ ์ˆœ์ˆ˜ํ•œ numpy/python ํฌํŠธ๋ฅผ ์ž‘์„ฑํ–ˆ์Šต๋‹ˆ๋‹ค. ํ—ค๋”์— ๋”ฐ๋ฅด๋ฉด dlib์˜ implem์€ 2008๋…„๋ถ€ํ„ฐ ์‚ฌ์šฉ๋˜์—ˆ์œผ๋ฉฐ ๊ดœ์ฐฎ์Šต๋‹ˆ๋‹ค. sklearn์— RVM์„ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์— ๋Œ€ํ•ด ์ƒ๊ฐ์„ ๋ฐ”๊พธ์‹œ๊ฒ ์Šต๋‹ˆ๊นŒ?

์ด์— ๋Œ€ํ•œ @GaelVaroquaux ์˜ ์˜๊ฒฌ์„ ๋“ค์–ด๋ณด์ž. dlib implem์€ ํŠนํ—ˆ ๋ผ์ด์„ ์Šค ์—†์ด ๋„๋ฆฌ ์‚ฌ์šฉ๋œ๋‹ค๋Š” ๊ฒƒ์„ ์ฆ๋ช…ํ•  ์ˆ˜ ์—†๋Š” ํ•œ ์•„๋ฌด ๊ฒƒ๋„ ํ‘œ์‹œํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

์ด ์ฃผ์ œ์— ๋Œ€ํ•œ ์—…๋ฐ์ดํŠธ๊ฐ€ ์žˆ์Šต๋‹ˆ๊นŒ? ๋‚˜๋Š” ์ตœ๊ทผ์— RVM์„ ์กฐ์‚ฌํ•˜๊ณ  ์žˆ์—ˆ๊ณ  ์ฃผ๋ณ€์— ์ฝ”๋“œ๊ฐ€ ์žˆ๋Š”์ง€ ๊ถ๊ธˆํ–ˆ์Šต๋‹ˆ๋‹ค ...

๋น ๋ฅธ ๊ตฌํ˜„์„ ์œ„ํ•ด ์‹œ๋„ํ•œ ์‚ฌ๋žŒ์€ ์•„๋ฌด๋„ ์—†๋‹ค๊ณ  ์ƒ๊ฐํ•˜๋ฉฐ ์•„์ง ๋ฒ•์  ์ƒํƒœ๊ฐ€ ํ™•์‹คํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

@jlopezpena dlib๋ฅผ ์‚ดํŽด๋ณด๋ฉด ์ฝ”๋“œ๊ฐ€ ๋งค์šฐ ๋ช…ํ™•ํ•˜๊ณ  ํ—ค๋”(ํ…œํ”Œ๋ฆฟ)๋งŒ ์žˆ์Šต๋‹ˆ๋‹ค. numpy์—์„œ ์‚ฌ์šฉํ•  C ํ™•์žฅ์„ ๋นŒ๋“œํ•˜๋Š” ๊ฒƒ์€ ์ƒ๋‹นํžˆ ์‰ฌ์›Œ์•ผ ํ•ฉ๋‹ˆ๋‹ค.

์•ˆ๋…•ํ•˜์„ธ์š” ์—ฌ๋Ÿฌ๋ถ„,

๋‚˜๋Š” ์ตœ๊ทผ์— RVM์„ ์ฃผ๋กœ ๊ตฌํ˜„ํ•˜๋Š” Mike Tipping์˜ ๋ฌด๋ฃŒ SparseBayes MATLAB ํ”„๋กœ๊ทธ๋žจ์„ MATLAB์—์„œ Python์œผ๋กœ ๋ฒˆ์—ญํ–ˆ์Šต๋‹ˆ๋‹ค. https://github.com/jhallock7/SparseBayes-Python ์—์„œ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. Mike Tipping์—๊ฒŒ ์—ฐ๋ฝ์„ ์ทจํ–ˆ๋Š”๋ฐ ๊ทธ๋Š” Microsoft์˜ ํŠนํ—ˆ๊ฐ€ ์›๋ž˜์˜ ๋Š๋ฆฐ ์•Œ๊ณ ๋ฆฌ์ฆ˜์—๋งŒ ํ•ด๋‹นํ•˜๋Š” ๋ฐ˜๋ฉด SparseBayes ํ”„๋กœ๊ทธ๋žจ์€ http://www.miketipping.com/papers/met-fastsbl.pdf์— ์žˆ๋Š” ๋” ๋น ๋ฅธ ์•Œ๊ณ ๋ฆฌ์ฆ˜์„ ์‚ฌ์šฉํ•œ๋‹ค๊ณ  ๋งํ–ˆ์Šต๋‹ˆ๋‹ค. ๋”ฐ๋ผ์„œ ๊ทธ์˜ ํ”„๋กœ๊ทธ๋žจ์˜ ์ผ๋ถ€ ํ˜•ํƒœ๊ฐ€ scikit-learn์œผ๋กœ ์ ‘ํ˜€๋„ ๊ดœ์ฐฎ์„ ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์ €๋Š” ๋น„๊ต์  Python์„ ์ฒ˜์Œ ์‚ฌ์šฉํ•˜๋ฏ€๋กœ ๋ฒˆ์—ญ์ด ์˜์‹ฌ์˜ ์—ฌ์ง€ ์—†์ด ๊ฐœ์„ ๋˜๊ฑฐ๋‚˜ ์ˆ˜์ •๋  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

๊ธฐ์—ฌํ•ด ์ฃผ์…”์„œ ๊ฐ์‚ฌํ•˜๊ณ  ํŠนํ—ˆ ํ˜„ํ™ฉ์„ ํ™•์ธํ•ด์ฃผ์…”์„œ ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค.
๊ทธ๋Ÿฌ๋‚˜ ์ด ์•Œ๊ณ ๋ฆฌ์ฆ˜์ด ๋„๋ฆฌ ์œ ์šฉํ•œ์ง€ ์—ฌ๋ถ€์— ๋Œ€ํ•œ ๋˜ ๋‹ค๋ฅธ ์งˆ๋ฌธ์ด ์žˆ์Šต๋‹ˆ๋‹ค.

๋‚˜๋Š” ์–ผ๋งˆ ์ „์— Bishop์˜ ์ฑ…์„ ๋งˆ์ง€๋ง‰์œผ๋กœ ์ฝ์€ ์ดํ›„๋กœ ์•Œ๊ณ ๋ฆฌ์ฆ˜๊ณผ ๊ทธ ์‚ฌ์šฉ๋ฒ•์„ ๋ณธ ์ ์ด ์—†์Šต๋‹ˆ๋‹ค.
ํฅ๋ฏธ๋กœ์šด ์˜ˆ๋Š” ๊ต์ •๋ณด๋‹ค ๋” ๋‚˜์€ ๋ถˆํ™•์‹ค์„ฑ์„ ์ œ๊ณตํ•œ๋‹ค๋Š” ๊ฒƒ์„ ๋ณด์—ฌ์ฃผ๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค.
๋˜๋Š” SVM์ด ๋” ๋น ๋ฆ…๋‹ˆ๋‹ค(SVM์„ ๋ณด์ •ํ•˜๊ณ  C๋ฅผ ๊ฒ€์ƒ‰ํ•˜๋ ค๋ฉด ๋งŽ์€ ๊ต์ฐจ ๊ฒ€์ฆ์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค).

@amueller RVM(์ฃผ๋กœ Revelance Vector Regression(RVR))์€ ์‹ ๊ฒฝ ์˜์ƒ ๋ฐ์ดํ„ฐ ๋ถ„์„์— ๋งค์šฐ ์œ ์šฉํ•ฉ๋‹ˆ๋‹ค. ๋งŽ์€ ๋…ผ๋ฌธ์—์„œ ์˜ˆ์ธก์„ ์œ„ํ•ด SVR๋ณด๋‹ค ์ด ๋ฐฉ๋ฒ•์„ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค. ์ด ๋ฐฉ๋ฒ•์ด scikit Learn ๋„๊ตฌ ์ƒ์ž์— ์ถ”๊ฐ€๋  ์ˆ˜ ์žˆ๋‹ค๋ฉด ์™„๋ฒฝํ•  ๊ฒƒ์ž…๋‹ˆ๋‹ค.

@amueller ๋‚˜๋Š” ๋ชจ๋ธ(์ฃผ๋กœ ํ•™์Šต/ํ•™๋ฌธ ๋ชฉ์ )์— ๋งž์ถ”๊ธฐ ์œ„ํ•ด EM ๋˜๋Š” ๊ณ ์ • ์†Œ์ˆ˜์  ์•Œ๊ณ ๋ฆฌ์ฆ˜์„ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ๋Š๋ฆฐ ๋ฒ„์ „์˜ RVM์„ ๊ตฌํ˜„ํ–ˆ์œผ๋ฉฐ ๋ช‡ ๊ฐ€์ง€ ์˜ˆ์—์„œ ์–ธ๊ธ‰ํ•œ RVM๊ณผ SVR์˜ ์ฃผ์š” ์ฐจ์ด์ ์€ ํฌ์†Œ์„ฑ, ์ฆ‰ '์ง€์›์˜ ์ˆ˜'์ž…๋‹ˆ๋‹ค. ' ์˜ˆ์ธก์— ์‚ฌ์šฉ๋œ ๋ฒกํ„ฐ. ๋งŽ์€ ๊ฒฝ์šฐ์— RVM์€ SVR์ด ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์˜ ์ผ๋ถ€์— ๋ถˆ๊ณผํ•œ ์ง€์› ๋ฒกํ„ฐ์˜ ์ˆ˜์™€ ํ•จ๊ป˜ SVR๊ณผ ์œ ์‚ฌํ•œ ๊ฒฐ๊ณผ๋ฅผ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.
( ์—ฌ๊ธฐ๋Š” Tipping 2001์—์„œ๋„ ์‚ฌ์šฉ๋œ ๊ฐ„๋‹จํ•œ ์˜ˆ์ž…๋‹ˆ๋‹ค.)

@amueller (์ด์ „ ์˜๊ฒฌ์— ์ถ”๊ฐ€) ๊ทธ๋ฆฌ๊ณ  ๋ถ„๋ช…ํžˆ ์ ์€ ์ˆ˜์˜ ์ง€์› ๋ฒกํ„ฐ๋Š” ๋งค์šฐ ๋น ๋ฅธ ์˜ˆ์ธก์„ ์˜๋ฏธํ•ฉ๋‹ˆ๋‹ค.
RVM์˜ ๋˜ ๋‹ค๋ฅธ ์žฅ์ ์€ ํ™•๋ฅ ์  ์น˜๋ฃŒ์ž…๋‹ˆ๋‹ค. ํ…Œ์ŠคํŠธ ์„ธํŠธ์˜ ๋ชจ๋“  ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ์— ๋Œ€ํ•œ RVM์„ ์‚ฌ์šฉํ•˜๋ฉด ํฌ์ธํŠธ ์ถ”์ •๋ฟ๋งŒ ์•„๋‹ˆ๋ผ ์˜ˆ์ธก ๋ถ„ํฌ๋„ ์ฐพ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

RVR์€ ํ™•๋ฅ  ๋ถ„ํฌ๋„ ์ œ๊ณตํ•ฉ๋‹ˆ๊นŒ?

RVR ๋ฐ RVM์ด ํฌํ•จํ•˜๊ธฐ์— ํ•ฉ๋‹นํ•œ ํ›„๋ณด์ธ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค. ํ•˜์ง€๋งŒ ์ตœ์‹  ์•Œ๊ณ ๋ฆฌ์ฆ˜์— ๋Œ€ํ•ด์„œ๋Š” ์ž˜ ๋ชจ๋ฅด๊ฒ ์Šต๋‹ˆ๋‹ค. http://www.miketipping.com/papers/met-fastsbl.pdf ์ž…๋‹ˆ๊นŒ? ๊ฝค ์˜ค๋ž˜๋œ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค. SparseBayes ์ฝ”๋”ฉ ์Šคํƒ€์ผ์€.... ํฅ๋ฏธ๋กญ๊ณ , sklearn ๊ตฌํ˜„์˜ ๊ธฐ์ดˆ๊ฐ€ ๋˜๊ธฐ๋ณด๋‹ค๋Š” ์ฐธ์กฐ๋กœ ๋” ์ž˜ ์‚ฌ์šฉ๋  ๊ฒƒ์ด๋ผ๊ณ  ์ƒ๊ฐํ•ฉ๋‹ˆ๋‹ค.

์˜ˆ RVR์€ ํ™•๋ฅ  ๋ถ„ํฌ๋ฅผ ์ œ๊ณตํ•˜์ง€๋งŒ ์–ด๋–ค ๊ฒฝ์šฐ์—๋Š” ์˜ˆ์ธก ๋ถ„ํฌ์˜ ๋ถ„์‚ฐ์ด ํ›ˆ๋ จ ์„ธํŠธ ์˜ˆ์ œ ์˜ ๋„๋ฉ”์ธ ์™ธ๋ถ€์— ์žˆ๋Š” ๋ฐ์ดํ„ฐ ํฌ์ธํŠธ์— ๋Œ€ํ•ด ๋” ์ž‘์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
๋‚ด๊ฐ€ ์•„๋Š” ํ•œ ๋‹น์‹ ์ด ์–ธ๊ธ‰ํ•œ RVM์˜ ๋งˆ์ง€๋ง‰ ๋ฒ„์ „์€ Tipping์˜ ์›น์‚ฌ์ดํŠธ(๋ฒ„์ „ 2)์˜ Matlab ๊ตฌํ˜„์—๋„ ํ•ด๋‹นํ•ฉ๋‹ˆ๋‹ค.

๋˜ํ•œ Kevin Murphy์˜ ์ฑ…์—์„œ RVM๊ณผ SVM ์†๋„์— ๋Œ€ํ•œ ํฅ๋ฏธ๋กœ์šด ๋น„๊ต๋ฅผ ์ฐพ์•˜์Šต๋‹ˆ๋‹ค.
"RVM์€ ๋˜ํ•œ ํ›ˆ๋ จํ•˜๊ธฐ์— ๊ฐ€์žฅ ๋น ๋ฆ…๋‹ˆ๋‹ค. ์ด๊ฒƒ์€ RVM ์ฝ”๋“œ๊ฐ€ Matlab์— ์žˆ๊ณ  SVM ์ฝ”๋“œ๊ฐ€ C์— ์žˆ๋‹ค๋Š” ์‚ฌ์‹ค์—๋„ ๋ถˆ๊ตฌํ•˜๊ณ "(14์žฅ, p.490). ๊ทธ๋Ÿฌ๋‚˜ ๊ทธ๋“ค์€ ์ž‘์€ ๋ฐ์ดํ„ฐ ์„ธํŠธ์— ๋Œ€ํ•ด์„œ๋งŒ ๋น„๊ต๋ฅผ ํ•œ ๊ฒƒ์œผ๋กœ ๋ณด์ž…๋‹ˆ๋‹ค.

@amueller RVM(์ฃผ๋กœ revelance ๋ฒกํ„ฐ ํšŒ๊ท€(RVR))์€ ๋‹ค์Œ์—์„œ ๋งค์šฐ ์œ ์šฉํ•ฉ๋‹ˆ๋‹ค.
์‹ ๊ฒฝ ์˜์ƒ ๋ฐ์ดํ„ฐ ๋ถ„์„.

๋‚˜๋Š” ํ™•์‹ ์ด ์„œ์ง€ ์•Š๋Š”๋‹ค(๊ทธ๋ฆฌ๊ณ  ๋‚˜๋Š” ์‹ ๊ฒฝ์˜์ƒ์ดฌ์˜์„ ํ•œ๋‹ค). ์ข‹์€๊ฑด ๋ชป๋ดค์–ด
๊ฒฝํ—˜์  ๋น„๊ต.

๊ฐˆ ๋ฐฉ๋ฒ•์€ RVM์„ ๋ณ„๋„์˜ ํŒจํ‚ค์ง€์— ํฌํ•จํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค.
scikit-learn API๋ฅผ ์‚ฌ์šฉํ•˜๊ณ  ์ข‹์€ ๊ฒฝํ—˜์  ์ž‘์—…์„ ๊ถŒ์žฅํ•˜์—ฌ
์œ ์šฉ์„ฑ. ์œ ์šฉํ•˜๋‹ค๋ฉด scikit-learn์— ๋ณ‘ํ•ฉํ•˜์‹ญ์‹œ์˜ค.

์ผ๋ถ€ ์‹ ๊ฒฝ ์˜์ƒ ์—ฐ๊ตฌ์—์„œ๋Š” ๊ด€๋ จ์„ฑ ๋ฒกํ„ฐ ํšŒ๊ท€(RVR) ๋ฐ
RVR๊ณผ SVR์„ ๋น„๊ตํ–ˆ์Šต๋‹ˆ๋‹ค.
๋ช‡ ๊ฐ€์ง€๋ฅผ ๋‚˜์—ดํ•˜๋ ค๋ฉด:
http://www.sciencedirect.com/science/article/pii/S1053811910000108
http://www.sciencedirect.com/science/article/pii/S1053811910012644
http://www.sciencedirect.com/science/article/pii/S1053811910003459
http://www.nature.com/npp/journal/v39/n3/abs/npp2013251a.html

๊ทธ๋ฆฌ๊ณ  RVR์€ ํŒจํ„ด ์ธ์‹ ๋„๊ตฌ ์ƒ์ž์—์„œ ๊ตฌํ˜„๋ฉ๋‹ˆ๋‹ค.
์‹ ๊ฒฝ ์˜์ƒ ๋ฐ์ดํ„ฐ:
http://www.mlnl.cs.ucl.ac.uk/pronto/

RVR์ด scikit-learn์— ํฌํ•จ๋  ์ˆ˜ ์žˆ๊ธฐ๋ฅผ ๋ฐ”๋ž๋‹ˆ๋‹ค.

์ตœ๊ณ ์˜ ์†Œ์›

์ž์ด์ˆ˜

2015๋…„ 10์›” 15์ผ ๋ชฉ์š”์ผ ์˜คํ›„ 12์‹œ 57๋ถ„, Gael Varoquaux [email protected]
์ผ๋‹ค:

@amueller RVM(์ฃผ๋กœ Revelance Vector Regression(RVR))์€ ๊ฝค
์œ ์šฉํ•œ
์‹ ๊ฒฝ ์˜์ƒ ๋ฐ์ดํ„ฐ ๋ถ„์„.

๋‚˜๋Š” ํ™•์‹ ์ด ์„œ์ง€ ์•Š๋Š”๋‹ค(๊ทธ๋ฆฌ๊ณ  ๋‚˜๋Š” ์‹ ๊ฒฝ์˜์ƒ์ดฌ์˜์„ ํ•œ๋‹ค). ์ข‹์€๊ฑด ๋ชป๋ดค์–ด
๊ฒฝํ—˜์  ๋น„๊ต.

๊ฐˆ ๋ฐฉ๋ฒ•์€ RVM์„ ๋ณ„๋„์˜ ํŒจํ‚ค์ง€์— ํฌํ•จํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค.
scikit-learn API๋ฅผ ์‚ฌ์šฉํ•˜๊ณ  ์ข‹์€ ๊ฒฝํ—˜์  ์ž‘์—…์„ ๊ถŒ์žฅํ•˜์—ฌ
์œ ์šฉ์„ฑ. ์œ ์šฉํ•˜๋‹ค๋ฉด scikit-learn์— ๋ณ‘ํ•ฉํ•˜์‹ญ์‹œ์˜ค.

โ€”
์ด ์ด๋ฉ”์ผ์— ์ง์ ‘ ๋‹ต์žฅํ•˜๊ฑฐ๋‚˜ GitHub์—์„œ ํ™•์ธํ•˜์„ธ์š”.
https://github.com/scikit-learn/scikit-learn/issues/1513#issuecomment -148281784
.

๊ฐˆ ๊ธธ์€ scikit-learn API๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ณ„๋„์˜ ํŒจํ‚ค์ง€์— RVM์„ ๊ฐ–๊ณ  ์œ ์šฉ์„ฑ์„ ๋ณด์—ฌ์ฃผ๊ธฐ ์œ„ํ•ด ์ข‹์€ ๊ฒฝํ—˜์  ์ž‘์—…์„ ๊ถŒ์žฅํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์œ ์šฉํ•˜๋‹ค๋ฉด scikit-learn์— ๋ณ‘ํ•ฉํ•˜์‹ญ์‹œ์˜ค.

+1

์ƒ๋Œ€์ ์œผ๋กœ ํ˜ธํ™˜๋˜๋Š” https://github.com/AmazaspShumik/Bayesian-Regression-Methods/blob/master/Relevance%20Vector%20Machine%20%26%20ARD/rvm.py ๊ฐ€ set_params ๋ฐ get_params ๋˜๋Š” BaseEstimator ์—์„œ ์ƒ์†์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค.

๊ทธ๋ฆฌ๊ณ  ๋ž˜ํ•‘๋  ์ˆ˜ ์žˆ๋Š” https://github.com/jhallock7/SparseBayes-Python ์ด ์žˆ์Šต๋‹ˆ๋‹ค.

@ZaixuCui ๊ตฌํ˜„์„ ์‚ฌ์šฉํ•  ์ค€๋น„๊ฐ€ ๋˜์—ˆ์„ ๋•Œ scikit-learn์— ์žˆ๊ธฐ๋ฅผ ์›ํ•˜๋Š” ์ด์œ ๋Š” ๋ฌด์—‡์ž…๋‹ˆ๊นŒ?

@GaelVaroquaux ๋ฐ @mblondel ์— ๋™์˜ํ•˜๋Š” ๊ฒฝํ–ฅ์ด ์žˆ์Šต๋‹ˆ๋‹ค. ๊ฑฐ์˜ 10๋…„ ๋™์•ˆ ์•„๋ฌด๋„ ์•Œ๊ณ ๋ฆฌ์ฆ˜์— ๋Œ€ํ•ด ๋ฐœํ‘œํ•˜์ง€ ์•Š์•˜๋‹ค๋ฉด ์‚ฌ๋žŒ๋“ค์€ ๋ณ„๋กœ ๊ด€์‹ฌ์„ ๊ฐ–์ง€ ์•Š๋Š” ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค. [์˜ค, ํ‘œ์ค€ ์•Œ๊ณ ๋ฆฌ์ฆ˜์€ ์‹ฌ์ง€์–ด 2003๋…„๋ถ€ํ„ฐ์ž…๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ๋‹ค์‹œ libsvm์€ 2005์ž…๋‹ˆ๋‹ค.]

scikit์„ ์‚ฌ์šฉํ•˜๊ธฐ ๋•Œ๋ฌธ์— SVR, elastic-net์„ ์ˆ˜ํ–‰ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ๋ฐฐ์›๋‹ˆ๋‹ค.
๋”ฐ๋ผ์„œ RVR ๊ตฌํ˜„์ด ์žˆ๋Š” ๊ฒฝ์šฐ matlab์„ ์‚ฌ์šฉํ•  ํ•„์š”๊ฐ€ ์—†์Šต๋‹ˆ๋‹ค.
๊ธฐ๊ณ„ ํ•™์Šต ๋ถ„์„์„ ์ˆ˜ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

๋งค์šฐ ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค.
๋‹น์‹ ์—๊ฒŒ ์ตœ์„ ์„ ๊ธฐ์›ํ•ฉ๋‹ˆ๋‹ค

์ž์ด์ˆ˜

2015๋…„ 10์›” 15์ผ ๋ชฉ์š”์ผ ์˜ค์ „ 11์‹œ 13๋ถ„, Andreas Mueller [email protected]
์ผ๋‹ค:

๋‚ด ๋ง์€
https://github.com/AmazaspShumik/Bayesian-Regression-Methods/blob/master/Relevance%20Vector%20Machine%20%26%20ARD/rvm.py
์ƒ๋Œ€์ ์œผ๋กœ ํ˜ธํ™˜๋˜๋Š” ๊ฒƒ์ฒ˜๋Ÿผ ๋ณด์ž…๋‹ˆ๋‹ค. set_params ๋ฐ get_params๊ฐ€ ํ•„์š”ํ•˜๊ฑฐ๋‚˜
BaseEstimator์—์„œ ์ƒ์†.

๊ทธ๋ฆฌ๊ณ  https://github.com/jhallock7/SparseBayes-Python ์ด ์žˆ์Šต๋‹ˆ๋‹ค.
ํฌ์žฅ๋˜๋‹ค.

@ZaixuCui https://github.com/ZaixuCui ์™œ ๋“ค์–ด๊ฐ€๊ณ  ์‹ถ์œผ์„ธ์š” ?
๊ตฌํ˜„์„ ์‚ฌ์šฉํ•  ์ค€๋น„๊ฐ€ ๋˜๋ฉด scikit-learn?

@GaelVaroquaux https://github.com/GaelVaroquaux ์— ๋™์˜ํ•˜๋Š” ๊ฒฝํ–ฅ์ด
๋ฐ @mblondel https://github.com/mblondel . ์•„๋ฌด๋„ ๊ฒŒ์‹œํ•˜์ง€ ์•Š์€ ๊ฒฝ์šฐ
๊ฑฐ์˜ 10๋…„ ๋™์•ˆ ์•Œ๊ณ ๋ฆฌ์ฆ˜์„ ๊ฐœ๋ฐœํ–ˆ์ง€๋งŒ ์‚ฌ๋žŒ๋“ค์€ ๋ณ„๋กœ ๊ด€์‹ฌ์„ ๊ฐ–์ง€ ์•Š๋Š” ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค.

โ€”
์ด ์ด๋ฉ”์ผ์— ์ง์ ‘ ๋‹ต์žฅํ•˜๊ฑฐ๋‚˜ GitHub์—์„œ ํ™•์ธํ•˜์„ธ์š”.
https://github.com/scikit-learn/scikit-learn/issues/1513#issuecomment -148440665
.

scikit์„ ์‚ฌ์šฉํ•˜๊ธฐ ๋•Œ๋ฌธ์— SVR, elastic-net์„ ์ˆ˜ํ–‰ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ๋ฐฐ์›๋‹ˆ๋‹ค.
๋”ฐ๋ผ์„œ RVR ๊ตฌํ˜„์ด ์žˆ๋Š” ๊ฒฝ์šฐ matlab์„ ์‚ฌ์šฉํ•  ํ•„์š”๊ฐ€ ์—†์Šต๋‹ˆ๋‹ค.
๊ธฐ๊ณ„ ํ•™์Šต ๋ถ„์„์„ ์ˆ˜ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

์—์„œ ์ง€์ ํ•œ RVR์„ ์ˆ˜ํ–‰ํ•˜๋Š” Python ์ฝ”๋“œ๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
๋…ผ์˜.

์•Œ์•˜์–ด ๊ณ ๋งˆ์›Œ

2015๋…„ 10์›” 19์ผ ์›”์š”์ผ ์˜ค์ „ 8:29 Gael Varoquaux [email protected]
์ผ๋‹ค:

scikit์„ ์‚ฌ์šฉํ•˜๊ธฐ ๋•Œ๋ฌธ์— SVR, elastic-net์„ ์ˆ˜ํ–‰ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ๋ฐฐ์›๋‹ˆ๋‹ค.
๋”ฐ๋ผ์„œ RVR ๊ตฌํ˜„์ด ์žˆ๋Š” ๊ฒฝ์šฐ matlab์„ ์‚ฌ์šฉํ•  ํ•„์š”๊ฐ€ ์—†์Šต๋‹ˆ๋‹ค.
์–ธ์ œ
๊ธฐ๊ณ„ ํ•™์Šต ๋ถ„์„์„ ์ˆ˜ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

์—์„œ ์ง€์ ํ•œ RVR์„ ์ˆ˜ํ–‰ํ•˜๋Š” Python ์ฝ”๋“œ๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
๋…ผ์˜.

โ€”
์ด ์ด๋ฉ”์ผ์— ์ง์ ‘ ๋‹ต์žฅํ•˜๊ฑฐ๋‚˜ GitHub์—์„œ ํ™•์ธํ•˜์„ธ์š”.
https://github.com/scikit-learn/scikit-learn/issues/1513#issuecomment -149213151
.

์šฐ๋ฆฌ์˜ ์ƒˆ๋กœ์šด Gaussian Process ๊ตฌํ˜„์„ ๊ธฐ๋ฐ˜์œผ๋กœ ์ด๊ฒƒ์„ ๋งค์šฐ ๊ฐ€๋ฒผ์šด ํด๋ž˜์Šค๋กœ ๊ตฌํ˜„ํ•  ์ˆ˜ ์—†์—ˆ์Šต๋‹ˆ๊นŒ? ๋‚ด๊ฐ€ ์ดํ•ดํ•˜๋Š” ํ•œ RVR์€ ํŠน๋ณ„ํ•œ ์ข…๋ฅ˜์˜ ์ปค๋„์ด ์žˆ๋Š” GP์— ๋ถ€์—ฌ๋œ ์ด๋ฆ„์ผ ๋ฟ์ž…๋‹ˆ๋‹ค.

์ด๊ฒƒ์€ ์ตœ์†Œํ•œ์˜ ๋…ธ๋ ฅ๋งŒ ํ•„์š”๋กœ ํ•˜์ง€๋งŒ GP ์ค‘ ํ•˜๋‚˜๋ฅผ ๊ธฐ๋ฐ˜์œผ๋กœ RVR์„ ๊ตฌํ˜„ํ•˜๋Š” ๊ฒƒ์ด ๊ฐ€์žฅ ์ ์ ˆํ•˜์ง€ ์•Š์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๊นŒ? ์ฐธ์กฐ: @jmetzen

@amueller @GaelVaroquaux @ZaixuCui @yedtoss @jlopezpena

์•ˆ๋…•ํ•˜์„ธ์š”, scikit-learn API๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ Relevance Vector Machine์˜ ๋น ๋ฅธ ๋ฒ„์ „์„ ๊ตฌํ˜„ํ–ˆ์Šต๋‹ˆ๋‹ค.
๋”ฐ๋ผ์„œ ๋ˆ„๊ตฌ๋“ ์ง€ ์‚ฌ์šฉํ•˜๋ ค๋Š” ๊ฒฝ์šฐ ์ž์œ ๋กญ๊ฒŒ ์‚ฌ์šฉํ•˜์‹ญ์‹œ์˜ค.

์ฝ”๋“œ: https://github.com/AmazaspShumik/sklearn_bayes/blob/master/sklearn_bayes/rvm/fast_rvm.py

์˜ˆ: https://github.com/AmazaspShumik/sklearn_bayes/blob/master/ipython_notebooks_tutorials/rvm_ard/rvm_demo.ipynb

์ฝ”๋“œ์—๋Š” 4๊ฐœ์˜ ๊ตฌํ˜„๋œ ํด๋ž˜์Šค๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค.
-ํšŒ๊ท€ARD
-๋ถ„๋ฅ˜ARD
-RVC
-RVR

๋”ฐ๋ผ์„œ RegressionARD ๋ฐ ClassificationARD๋„ ์œ ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

@AmazaspShumik ๊ตฌํ˜„์— ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค. ํ›Œ๋ฅญํ•œ ์ž‘ํ’ˆ:+1:

@AmazaspShumik

๋‹น์‹ ์˜ ๋…ธ๋ ฅ์— ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค.
๋‚˜๋Š” ํ™•์‹คํžˆ์ด ํŒจํ‚ค์ง€๋ฅผ ์‹œ๋„ ํ•  ๊ฒƒ์ž…๋‹ˆ๋‹ค.

์ตœ์„ ์„ ๋‹คํ•˜์‹œ๊ธธ ๋ฐ”๋ž๋‹ˆ๋‹ค.

์ž์ด์ˆ˜

@AmazaspShumik predict_proba ๋ฉ”์„œ๋“œ๋ฅผ ๊ตฌํ˜„ํ•˜๋Š” ๋ฐ ๋ฌธ์ œ๊ฐ€ ์žˆ๋Š” ์‚ฌ๋žŒ์ด ์žˆ์Šต๋‹ˆ๊นŒ?

์—ฌ๊ธฐ PHP์— RVM์šฉ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๊ฐ€ ์žˆ๋Š” ์‚ฌ๋žŒ์ด ์žˆ์Šต๋‹ˆ๊นŒ? ๋‚˜๋Š” RVm์ด ๋‚˜๋ฅผ ์œ„ํ•ด ์„ค๋ช…ํ•  ์ˆ˜ ์žˆ๋‹ค๋Š” ๊ฒƒ์„ ์ดํ•ดํ•˜์ง€ ๋ชปํ•ฉ๋‹ˆ๊นŒ?

PHP์šฉ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ RVM์ด ์žˆ๋Š” ์‚ฌ๋žŒ์ด ์žˆ์Šต๋‹ˆ๊นŒ?

RVM์€ Microsoft์—์„œ ํŠนํ—ˆ๋ฅผ ๋ฐ›์•˜์Šต๋‹ˆ๋‹ค.

ํŠนํ—ˆ๊ฐ€ ๊ณง ๋งŒ๋ฃŒ๋ฉ๋‹ˆ๋‹ค

2019-09-04
์˜ˆ์ƒ ๋งŒ๋ฃŒ

@AmazaspShumik ๊ตฌํ˜„์— ๋Œ€ํ•œ ํ† ๋ก  ๋งํฌ๊ฐ€ ์†์ƒ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ๊ด€์‹ฌ์ด ์žˆ๋Š” ์‚ฌ๋žŒ๋“ค์„ ์œ„ํ•ด ์—ฌ๊ธฐ์— ๋‘์‹ญ์‹œ์˜ค(๋ฐ ์ผ๋ถ€ ๋‹ค๋ฅธ ๊ตฌํ˜„).

https://github.com/AmazaspShumik/sklearn_bayes - RVM + ์ผ๋ถ€ ๋‹ค๋ฅธ algs ๊ตฌํ˜„
https://github.com/JamesRitchie/scikit-rvm - scipy๋ฅผ ์‚ฌ์šฉํ•œ ๊ฐ„๋‹จํ•œ ๊ตฌํ˜„
https://github.com/siavashserver/neonrvm - Python ๋ฐ”์ธ๋”ฉ์„ ์‚ฌ์šฉํ•œ C ๊ตฌํ˜„

๋˜ํ•œ ๊ด€๋ จ ๋…ผ๋ฌธ ๋ชจ์Œ์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
http://www.miketipping.com/sparsebayes.htm

๊ฑฐ๊ธฐ์— RVM์˜ C++ ๊ตฌํ˜„์ด ์žˆ์Šต๋‹ˆ๋‹ค(๋ฌธ์„œ์˜ ๋ณด์ถฉ ์ž๋ฃŒ).
https://pubs.acs.org/doi/abs/10.1021/ci200128w
์ •ํ™•ํžˆ ์—ฌ๊ธฐ:
https://pubs.acs.org/doi/suppl/10.1021/ci200128w/suppl_file/ci200128w_si_001.zip

Microsoft ํŠนํ—ˆ๊ฐ€ ๋งŒ๋ฃŒ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. sklearn์— ์ถ”๊ฐ€ํ•  ์ˆ˜ ์žˆ์„๊นŒ์š”?

ํ‘œ์ค€ ์š”๊ตฌ ์‚ฌํ•ญ์„ ์‰ฝ๊ฒŒ ์ง€์šธ ์ˆ˜ ์žˆ์œผ๋ฏ€๋กœ ์™œ ์•ˆ๋˜๋Š”์ง€ ๋ชจ๋ฅด๊ฒ ์Šต๋‹ˆ๋‹ค. ์•„๋งˆ๋„ ์ข‹์€/ํ™•์‹คํ•œ ์˜ˆ๋ฅผ ๊ฐ–๋Š” ๊ฒƒ์ด ํฅ๋ฏธ๋กœ์šธ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. scikit-rvm ๋ฐ sklearn_bayes ๋Š” ์œ ์ง€ ๊ด€๋ฆฌ๋˜์ง€ ์•Š์€ ๊ฒƒ์ฒ˜๋Ÿผ ๋ณด์ด์ง€๋งŒ ์—ฌ์ „ํžˆ ์œ ์šฉํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
์•„๋งˆ๋„ ์ด์ œ ์‹ค์ œ๋กœ ์ž‘์—…์„ ์ˆ˜ํ–‰ํ•˜๋ ค๋Š” ์ฑ”ํ”ผ์–ธ์ด ๊ฐ€์žฅ ํ•„์š”ํ•  ๊ฒƒ์ž…๋‹ˆ๋‹ค.

Murphy์˜ ์ฑ…์—์„œ ๊ทธ๋Š” RVM ์„ฑ๋Šฅ์ด SVM๊ณผ ์‹ค์ œ๋กœ ์œ ์‚ฌํ•˜์ง€๋งŒ ์ง„์ •ํ•œ ํ™•๋ฅ ๋ก ์  ๋ฐฉ๋ฒ•์ด๋ผ๋Š” ์ด์ ์ด ์žˆ์œผ๋ฏ€๋กœ ๋ณด์ •๋œ ํ™•๋ฅ ์„ ๋‹ต๋ณ€์œผ๋กœ ์ œ๊ณตํ•œ๋‹ค๊ณ  ์ฃผ์žฅํ•ฉ๋‹ˆ๋‹ค. ์—ฌ๊ธฐ https://github.com/probml/pmtk3/blob/master/docs/tutorial/html/tutKernelClassif.html ๊ทธ๋Š” ์ž‘์€ ๋ฐ์ดํ„ฐ ์„ธํŠธ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ฐฉ๋ฒ•์„ ๋น„๊ตํ–ˆ์Šต๋‹ˆ๋‹ค

๋ Œ๋”๋ง๋œ ๋ฒ„์ „์— ๋Œ€ํ•œ ๋งํฌ๋ฅผ ์ œ๊ณตํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๊นŒ?
๋˜ํ•œ ํ™•์žฅ์„ฑ ๋ฌธ์ œ๊ฐ€ ์žˆ์„ ์ˆ˜ ์žˆ๋‹ค๋Š” ์ ์„ ๊ฐ์•ˆํ•  ๋•Œ ์ž‘์€ ๋ฐ์ดํ„ฐ ์„ธํŠธ์— ์žˆ๋‹ค๋Š” ๊ฒƒ๋„ ๋†€๋ผ์šด ์ผ์ด ์•„๋‹™๋‹ˆ๋‹ค.

@amueller http://htmlpreview.github.io/?https : //github.com/probml/pmtk3/blob/master/docs/tutorial/html/tutKernelClassif.html

๊ตฌํ˜„์„ ์œ„ํ•ด ๋…ธ๋ ฅํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ๋„์›€์„ ์ฃผ์‹œ๋ฉด ๊ฐ์‚ฌํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

SVM๋ณด๋‹ค RVM์˜ ์žฅ์  ์ค‘ ํ•˜๋‚˜์ธ IIRC๋Š” ์ตœ์ ํ™” ๋‹จ๊ณ„๋ฅผ ์ˆ˜ํ–‰ํ•˜์ง€ ์•Š๊ณ ๋„ ์ตœ์ ์˜ C ๋งค๊ฐœ๋ณ€์ˆ˜๋ฅผ ์ฐพ์„ ์ˆ˜ ์žˆ๋‹ค๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค.
์›์ €์ž๊ฐ€ ์ฐธ์กฐ ๊ตฌํ˜„์— ๊ธฐ๊บผ์ด ๊ธฐ์—ฌํ•  ๊ฒƒ์ธ์ง€ ๊ถ๊ธˆํ•ฉ๋‹ˆ๋‹ค.
๊ธ€์Ž„, ๊ทธ๊ฒƒ์€ Matlab์— ์žˆ๊ณ  Mike Tipping์€ github์—๋„ ์—†์Šต๋‹ˆ๋‹ค ...

์•ˆ๋…•ํ•˜์„ธ์š” @amueller ๊ทธ๋ฆฌ๊ณ  ์—ฌ๋Ÿฌ๋ถ„! ์šฐ๋ฆฌ๋Š” ์ด ์Šค๋ ˆ๋“œ๋ฅผ ๋ณด๊ณ  RVM์˜ sklearn ํ˜ธํ™˜ ๋ฒ„์ „(https://github.com/Mind-the-Pineapple/sklearn-rvm)์„ ๊ตฌํ˜„ํ•˜๊ธฐ๋กœ ๊ฒฐ์ •ํ–ˆ์Šต๋‹ˆ๋‹ค. ์šฐ๋ฆฌ๋Š” JamesRitchie ๊ตฌํ˜„์„ ๊ธฐ๋ฐ˜์œผ๋กœ ํ–ˆ์Šต๋‹ˆ๋‹ค. ๋ˆ„๊ตฐ๊ฐ€ ๊ธฐ๊บผ์ด ๊ทธ๊ฒƒ์„ ์‚ดํŽด๋ณด๊ณ  ํ”ผ๋“œ๋ฐฑ๊ณผ ๊ธฐ์—ฌ(@themrzmaster)๋ฅผ ํ™˜์˜ํ•œ๋‹ค๋ฉด ์ข‹์„ ๊ฒƒ์ž…๋‹ˆ๋‹ค. :)
์šฐ๋ฆฌ๋Š” ์ด ๋ฆฌํฌ์ง€ํ† ๋ฆฌ๋ฅผ ์œ ์ง€ ๊ด€๋ฆฌํ•˜๊ณ  ๋ฒ„์ „ 0.2์—์„œ ๋น ๋ฅธ ๋ฒ„์ „์„ ๊ตฌํ˜„ํ•  ๊ณ„ํš์ด ์žˆ์œผ๋ฉฐ ์–ธ์  ๊ฐ€ ์ด ์ฝ”๋“œ๊ฐ€ scikit-learn ๋ฉ”์ธ ๋ฆฌํฌ์ง€ํ† ๋ฆฌ์— ๋„์›€์ด ๋  ์ˆ˜ ์žˆ๊ธฐ๋ฅผ ๋ฐ”๋ž๋‹ˆ๋‹ค. ๐Ÿ˜Š ๐Ÿ

์•ˆ๋…•ํ•˜์„ธ์š” @amueller ๊ทธ๋ฆฌ๊ณ  ์—ฌ๋Ÿฌ๋ถ„! ์šฐ๋ฆฌ๋Š” ์ด ์Šค๋ ˆ๋“œ๋ฅผ ๋ณด๊ณ  RVM์˜ sklearn ํ˜ธํ™˜ ๋ฒ„์ „(https://github.com/Mind-the-Pineapple/sklearn-rvm)์„ ๊ตฌํ˜„ํ•˜๊ธฐ๋กœ ๊ฒฐ์ •ํ–ˆ์Šต๋‹ˆ๋‹ค. ์šฐ๋ฆฌ๋Š” JamesRitchie ๊ตฌํ˜„์„ ๊ธฐ๋ฐ˜์œผ๋กœ ํ–ˆ์Šต๋‹ˆ๋‹ค. ๋ˆ„๊ตฐ๊ฐ€ ๊ธฐ๊บผ์ด ๊ทธ๊ฒƒ์„ ์‚ดํŽด๋ณด๊ณ  ํ”ผ๋“œ๋ฐฑ๊ณผ ๊ธฐ์—ฌ(@themrzmaster)๋ฅผ ํ™˜์˜ํ•œ๋‹ค๋ฉด ์ข‹์„ ๊ฒƒ์ž…๋‹ˆ๋‹ค. :)
์šฐ๋ฆฌ๋Š” ์ด ๋ฆฌํฌ์ง€ํ† ๋ฆฌ๋ฅผ ์œ ์ง€ ๊ด€๋ฆฌํ•˜๊ณ  ๋ฒ„์ „ 0.2์—์„œ ๋น ๋ฅธ ๋ฒ„์ „์„ ๊ตฌํ˜„ํ•  ๊ณ„ํš์ด ์žˆ์œผ๋ฉฐ ์–ธ์  ๊ฐ€ ์ด ์ฝ”๋“œ๊ฐ€ scikit-learn ๋ฉ”์ธ ๋ฆฌํฌ์ง€ํ† ๋ฆฌ์— ๋„์›€์ด ๋  ์ˆ˜ ์žˆ๊ธฐ๋ฅผ ๋ฐ”๋ž๋‹ˆ๋‹ค. ๐Ÿ˜Š ๐Ÿ

์•ˆ๋…•ํ•˜์„ธ์š” @PedroFerreiradaCosta
์ด scikit-learn API๋ฅผ ํ…Œ์ŠคํŠธํ–ˆ๋Š”๋ฐ ์—ฌ์ „ํžˆ ๋„ˆ๋ฌด ๋Š๋ฆฐ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค(์•„์ง ์‘๋‹ตํ•˜์ง€ ์•Š๋Š” ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค). ๊ทธ ์ด์œ ๋Š” Windows์—์„œ ๊ตฌํ˜„๋  ์ˆ˜ ์žˆ๋‹ค๊ณ  ์ƒ๊ฐํ•˜์‹ญ๋‹ˆ๊นŒ? ์•„๋ž˜๋Š” ๋‚ด๊ฐ€ ์‚ฌ์šฉํ•œ ๊ฒƒ์ž…๋‹ˆ๋‹ค.
EMRVC(kernel='rbf', gamma='scale', n_iter_posterior=10, max_iter=500, compute_score=True , verbose=True ) ๋‹ต๋ณ€ ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค @themrzmaster @PedroFerreiradaCosta

์•ˆ๋…•ํ•˜์„ธ์š” @mustuner์ž…๋‹ˆ๋‹ค ! API๋ฅผ ์‚ฌ์šฉํ•ด ์ฃผ์…”์„œ ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค!
RVM์€ ์˜ˆ๋ฅผ ๋“ค์–ด SVM(O(M^3))๋ณด๋‹ค ๊ณ„์‚ฐ ๋ณต์žก๋„๊ฐ€ ๋†’๊ธฐ ๋•Œ๋ฌธ์— ๋งŽ์€ ์ˆ˜์˜ ๊ธฐ๋ณธ ํ•จ์ˆ˜๊ฐ€ ์žˆ๋Š” ๊ฒฝ์šฐ ์†๋„๊ฐ€ ๋Š๋ ค์งˆ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์–ด๋Š ์ชฝ์ด๋“ , ํ”„๋กœ์„ธ์Šค ์†๋„๋ฅผ ๋†’์ด๊ธฐ ์œ„ํ•ด ํ•  ์ˆ˜ ์žˆ๋Š” ์ผ์ด ๋งŽ์ด ์žˆ์Šต๋‹ˆ๋‹ค. ์ปค๋„ ํ–‰๋ ฌ์„ ๋ฏธ๋ฆฌ ๊ณ„์‚ฐํ•˜์—ฌ X ๋Œ€์‹  ์•Œ๊ณ ๋ฆฌ์ฆ˜์— ๊ณต๊ธ‰ํ•˜๊ฑฐ๋‚˜(kernel='precomputed'๋กœ ์„ค์ •) alpha_threshold์˜ ํฌ๊ธฐ๋ฅผ ์ค„์ผ ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค(๊ธฐ๋ณธ์ ์œผ๋กœ 1e5๋กœ ์„ค์ •). ์ด ๋‘ ๋ฒˆ์งธ ์˜ต์…˜์€ ๋ชจ๋ธ์˜ ์ •ํ™•๋„๋ฅผ ๋‚ฎ์ถœ ์ˆ˜ ์žˆ์Œ์„ ์—ผ๋‘์— ๋‘์‹ญ์‹œ์˜ค.
๋„์›€์ด ๋˜์—ˆ๊ธฐ๋ฅผ ๋ฐ”๋ž๋‹ˆ๋‹ค! ์ถ”๊ฐ€๋กœ ๋„์›€์„ ๋“œ๋ฆด ์ˆ˜ ์žˆ๋„๋ก ๋ฌธ์ œ๋ฅผ ์ž์œ ๋กญ๊ฒŒ ์—ฌ์‹ญ์‹œ์˜ค.

์ตœ์ƒ์˜,
ํŽ˜๋“œ๋กœ

์•ˆ๋…•ํ•˜์„ธ์š” @mustuner์ž…๋‹ˆ๋‹ค ! API๋ฅผ ์‚ฌ์šฉํ•ด ์ฃผ์…”์„œ ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค!
RVM์€ ์˜ˆ๋ฅผ ๋“ค์–ด SVM(O(M^3))๋ณด๋‹ค ๊ณ„์‚ฐ ๋ณต์žก๋„๊ฐ€ ๋†’๊ธฐ ๋•Œ๋ฌธ์— ๋งŽ์€ ์ˆ˜์˜ ๊ธฐ๋ณธ ํ•จ์ˆ˜๊ฐ€ ์žˆ๋Š” ๊ฒฝ์šฐ ์†๋„๊ฐ€ ๋Š๋ ค์งˆ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์–ด๋Š ์ชฝ์ด๋“ , ํ”„๋กœ์„ธ์Šค ์†๋„๋ฅผ ๋†’์ด๊ธฐ ์œ„ํ•ด ํ•  ์ˆ˜ ์žˆ๋Š” ์ผ์ด ๋งŽ์ด ์žˆ์Šต๋‹ˆ๋‹ค. ์ปค๋„ ํ–‰๋ ฌ์„ ๋ฏธ๋ฆฌ ๊ณ„์‚ฐํ•˜์—ฌ X ๋Œ€์‹  ์•Œ๊ณ ๋ฆฌ์ฆ˜์— ๊ณต๊ธ‰ํ•˜๊ฑฐ๋‚˜(kernel='precomputed'๋กœ ์„ค์ •) alpha_threshold์˜ ํฌ๊ธฐ๋ฅผ ์ค„์ผ ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค(๊ธฐ๋ณธ์ ์œผ๋กœ 1e5๋กœ ์„ค์ •). ์ด ๋‘ ๋ฒˆ์งธ ์˜ต์…˜์€ ๋ชจ๋ธ์˜ ์ •ํ™•๋„๋ฅผ ๋‚ฎ์ถœ ์ˆ˜ ์žˆ์Œ์„ ์—ผ๋‘์— ๋‘์‹ญ์‹œ์˜ค.
๋„์›€์ด ๋˜์—ˆ๊ธฐ๋ฅผ ๋ฐ”๋ž๋‹ˆ๋‹ค! ์ถ”๊ฐ€๋กœ ๋„์›€์„ ๋“œ๋ฆด ์ˆ˜ ์žˆ๋„๋ก ๋ฌธ์ œ๋ฅผ ์ž์œ ๋กญ๊ฒŒ ์—ฌ์‹ญ์‹œ์˜ค.

์ตœ์ƒ์˜,
ํŽ˜๋“œ๋กœ

@PedroFerreiradaCosta ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค

์ด ํŽ˜์ด์ง€๊ฐ€ ๋„์›€์ด ๋˜์—ˆ๋‚˜์š”?
0 / 5 - 0 ๋“ฑ๊ธ‰