Pytorch: ๊ฒ€๋ฒจ ๋…ธ์ด์ฆˆ

์— ๋งŒ๋“  2017๋…„ 01์›” 30์ผ  ยท  3์ฝ”๋ฉ˜ํŠธ  ยท  ์ถœ์ฒ˜: pytorch/pytorch

์•ˆ๋…•,

์—ฌ๊ธฐ ์—์„œ์™€ ๊ฐ™์ด Gumbel ๋…ธ์ด์ฆˆ๋ฅผ ์ถ”๊ฐ€ํ•˜๋ ค๊ณ  ์‹œ๋„ํ–ˆ์ง€๋งŒ ์„ฑ๊ณตํ•˜์ง€ ๋ชปํ–ˆ์Šต๋‹ˆ๋‹ค.
์—ฌ๋Ÿฌ ๋ชจ๋“ˆ์ด ์—ฌ์ „ํžˆ ๋ˆ„๋ฝ๋œ ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๊นŒ(์˜ˆ: nn.Uniform()๊ณผ ๊ฐ™์€ ๊ธฐ๋ณธ ๋ณ€์ˆ˜ ํ•จ์ˆ˜), ์•„๋‹ˆ๋ฉด ์ œ๊ฐ€ ์ž˜๋ชป ์•Œ๊ณ  ์žˆ์Šต๋‹ˆ๊นŒ? ์˜ˆ๋ฅผ ๋“ค์–ด ๋‹ค์Œ ํ–‰๊ณผ ๊ฐ™์ด pytorch์—์„œ ๊ตฌํ˜„ํ•˜๋Š” ๋ฐฉ๋ฒ•์€ ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.

``` -- ๋…ธ์ด์ฆˆ ฮต ์ƒ˜ํ”Œ ๋ชจ๋“ˆ ์ƒ์„ฑ
๋กœ์ปฌ ๋…ธ์ด์ฆˆ ๋ชจ๋“ˆ = nn.Sequential()
no iseModule:add (nn.Uniform(0, 1)) -- U(0, 1)์˜ ์ƒ˜ํ”Œ
-- ๊ท ์ผํ•œ ์ƒ˜ํ”Œ์„ Gumbel ์ƒ˜ํ”Œ๋กœ ๋ณ€ํ™˜
no iseModule:add (nn.AddConstant(1e-9, true)) -- ์ˆ˜์น˜์  ์•ˆ์ •์„ฑ ํ–ฅ์ƒ
iseModule: ์ถ”๊ฐ€ (nn.Log()) ์—†์Œ
iseModule: ์ถ”๊ฐ€ (nn.MulConstant(-1, true)) ์—†์Œ
no iseModule:add (nn.AddConstant(1e-9, true)) -- ์ˆ˜์น˜์  ์•ˆ์ •์„ฑ ํ–ฅ์ƒ
iseModule: ์ถ”๊ฐ€ (nn.Log()) ์—†์Œ
iseModule: ์ถ”๊ฐ€ (nn.MulConstant(-1, true)) ์—†์Œ

-- ์ƒ˜ํ”Œ๋Ÿฌ ์ƒ์„ฑ q(z) = G(z) = softmax((log(ฯ€) + ฮต)/ฯ„) (์žฌ๋งค๊ฐœ๋ณ€์ˆ˜ํ™” ํŠธ๋ฆญ)
๋กœ์ปฌ ์ƒ˜ํ”Œ๋Ÿฌ = nn.Sequential()
๋กœ์ปฌ ์ƒ˜ํ”Œ๋Ÿฌ ๋‚ด๋ถ€ = nn.ConcatTable()
sample rInternal:add (nn.Identity()) -- ์ •๊ทœํ™”๋˜์ง€ ์•Š์€ ๋กœ๊ทธ ํ™•๋ฅ  log(ฯ€)
sample rInternal:add (noiseModule) -- ๋…ธ์ด์ฆˆ ฮต ์ƒ์„ฑ
์ƒ˜ํ”Œ๋Ÿฌ:์ถ”๊ฐ€ (์ƒ˜ํ”Œ๋Ÿฌ ๋‚ด๋ถ€)
์ƒ˜ํ”Œ๋Ÿฌ:์ถ”๊ฐ€ (nn.CAddTable())
self.temperature = nn.MulConstant(1 / self.tau, true) -- softmax์˜ ์˜จ๋„ ฯ„
์ƒ˜ํ”Œ๋Ÿฌ:์ถ”๊ฐ€ (์ž์ฒด ์˜จ๋„)
sampler:add (nn.View(-1, self.k)) -- k์—์„œ ์ž‘๋™ํ•˜๋„๋ก ํฌ๊ธฐ ์กฐ์ •
์ƒ˜ํ”Œ๋Ÿฌ:์ถ”๊ฐ€ (nn.SoftMax())
sampler:add (nn.View(-1, self.N * self.k)) -- ๋‹ค์‹œ ํฌ๊ธฐ ์กฐ์ •
```

๊ฐ€์žฅ ์œ ์šฉํ•œ ๋Œ“๊ธ€

์—ฌ๊ธฐ ์žˆ์Šต๋‹ˆ๋‹ค. ํ›จ์”ฌ ๋” ์ฝ๊ธฐ ์‰ฝ๊ณ  ๋ชจ๋“ˆ์ด ํ•„์š”ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

import torch.nn.functional as F
from torch.autograd import Variable

def sampler(input, tau, temperature):
    noise = torch.rand(input.size())
    noise.add_(1e-9).log_().neg_()
    noise.add_(1e-9).log_().neg_()
    noise = Variable(noise)
    x = (input + noise) / tau + temperature
    x = F.softmax(x.view(input.size(0), -1))
    return x.view_as(input)

์šฐ๋ฆฌ๋Š” ๋ฒ„๊ทธ ๋ณด๊ณ ์šฉ์œผ๋กœ๋งŒ GitHub๋ฅผ ์‚ฌ์šฉํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ์งˆ๋ฌธ์ด ์žˆ๋Š” ๊ฒฝ์šฐ ํฌ๋Ÿผ ์— ๊ฒŒ์‹œํ•ด ์ฃผ์„ธ์š”.

๋ชจ๋“  3 ๋Œ“๊ธ€

์—ฌ๊ธฐ ์žˆ์Šต๋‹ˆ๋‹ค. ํ›จ์”ฌ ๋” ์ฝ๊ธฐ ์‰ฝ๊ณ  ๋ชจ๋“ˆ์ด ํ•„์š”ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค.

import torch.nn.functional as F
from torch.autograd import Variable

def sampler(input, tau, temperature):
    noise = torch.rand(input.size())
    noise.add_(1e-9).log_().neg_()
    noise.add_(1e-9).log_().neg_()
    noise = Variable(noise)
    x = (input + noise) / tau + temperature
    x = F.softmax(x.view(input.size(0), -1))
    return x.view_as(input)

์šฐ๋ฆฌ๋Š” ๋ฒ„๊ทธ ๋ณด๊ณ ์šฉ์œผ๋กœ๋งŒ GitHub๋ฅผ ์‚ฌ์šฉํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ์งˆ๋ฌธ์ด ์žˆ๋Š” ๊ฒฝ์šฐ ํฌ๋Ÿผ ์— ๊ฒŒ์‹œํ•ด ์ฃผ์„ธ์š”.

์‹ ์†ํ•œ ๋‹ต๋ณ€์— ๊ฐ์‚ฌ๋“œ๋ฆฝ๋‹ˆ๋‹ค! ๋‹ค์Œ ๋ฒˆ์— ํฌ๋Ÿผ์— ๊ธ€์„ ์“ธ ๊ฒƒ์ž…๋‹ˆ๋‹ค.

๋‹ต๋ณ€ ํ•ด์ฃผ์…”์„œ ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค!

์ด ํŽ˜์ด์ง€๊ฐ€ ๋„์›€์ด ๋˜์—ˆ๋‚˜์š”?
0 / 5 - 0 ๋“ฑ๊ธ‰