Zenodo: Commend Line์„ ์‚ฌ์šฉํ•˜์—ฌ Zenode์—์„œ ํŒŒ์ผ ๋‹ค์šด๋กœ๋“œ

์— ๋งŒ๋“  2019๋…„ 10์›” 16์ผ  ยท  8์ฝ”๋ฉ˜ํŠธ  ยท  ์ถœ์ฒ˜: zenodo/zenodo

๋ฒˆ๊ฑฐ๋กญ๊ฒŒ ํ•ด์„œ ์ฃ„์†กํ•ฉ๋‹ˆ๋‹ค๋งŒ, ์ €๋Š” ์˜ค๋žซ๋™์•ˆ ๋ฌธ์ œ์— ์ง๋ฉดํ•ด ์žˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ ๋ธŒ๋ผ์šฐ์ €๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ Zenode ์—์„œ ๋Œ€์šฉ๋Ÿ‰ ๋ฐ์ดํ„ฐ ์„ธํŠธ๋ฅผ ๋‹ค์šด๋กœ๋“œํ•˜๋Š” ๊ฒƒ์€ ์•ˆ์ •์ ์ด์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ๊ทธ๋ž˜์„œ 'wget'์„ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์ด ์ƒ๊ฐ๋‚˜์ง€๋งŒ Issues ์— ์ œ๊ณต๋œ ์ ‘๊ทผ ๋ฐฉ์‹์„ ๋”ฐ๋ž์„ ๋•Œ ์‹คํŒจํ–ˆ์Šต๋‹ˆ๋‹ค.

์ด์ „ ๋ฌธ์ œ์—์„œ ๋‹ค์Œ์„ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค.

curl  https://zenodo.org/api/records/3407840

REST API๊ฐ€ ์žˆ์Šต๋‹ˆ๋‹ค.

https://zenodo.org/api/files/f0d7eb3a-9575-4297-8a9f-39564e383750/Center%28Left_Down%29.zip.001

ํ•˜์ง€๋งŒ REST API๋ฅผ ์‚ฌ์šฉํ•˜์ง€ ์•Š๋Š” ๊ฒƒ์ฒ˜๋Ÿผ 404๊ฐ€ ๋ฐœ์ƒํ–ˆ์Šต๋‹ˆ๋‹ค .

wget -r https://zenodo.org/api/files/f0d7eb3a-9575-4297-8a9f-39564e383750/Center%28Left_Down%29.zip.001

์ € ์ข€ ๋„์™€์ฃผ์‹œ๊ฒ ์–ด์š”? ๊ณ ๋งˆ์›Œ!

๊ฐ€์žฅ ์œ ์šฉํ•œ ๋Œ“๊ธ€

๋‚˜๋Š” ์ด๊ฒƒ์„ ์‹œ๋„ํ–ˆ๊ณ  ๊ทธ๊ฒƒ์€ ๋‚˜๋ฅผ ์œ„ํ•ด ์ผํ–ˆ์Šต๋‹ˆ๋‹ค.
https://gist.github.com/slint/d47fe5628916d14b8d0b987ac45aeb66#file -zenodo-restricted-curl-sh-L26

๋ชจ๋“  8 ๋Œ“๊ธ€

์ด ๋ ˆ์ฝ”๋“œ์˜ ํŒŒ์ผ์€ ๊ณต๊ฐœ์ ์œผ๋กœ ์‚ฌ์šฉํ•  ์ˆ˜ ์—†๋Š” ์ œํ•œ๋œ ์•ก์„ธ์Šค๋กœ ์„ค์ •๋˜์–ด ์žˆ์œผ๋ฏ€๋กœ ์•ก์„ธ์Šคํ•˜๋ ค๋ฉด ID๋ฅผ ํ™•์ธํ•˜๊ธฐ ์œ„ํ•œ ์•ก์„ธ์Šค ํ† ํฐ์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค. ์—ฌ๊ธฐ์„œ ์ƒ์„ฑํ•˜๊ฑฐ๋‚˜ ํ‘œ์‹œ๋œ ์ ˆ์ฐจ๋ฅผ ๋”ฐ๋ผ์•ผ ํ•ฉ๋‹ˆ๋‹ค.

์˜ค๋ฅ˜๊ฐ€ ์•„๋‹ˆ๋ฏ€๋กœ ๋‹ซ์Šต๋‹ˆ๋‹ค.

๋น„๊ณต๊ฐœ ํ† ํฐ์„ ์‚ฌ์šฉํ•˜๋Š” ๋ฐฉ๋ฒ•์€ ๋ฌด์—‡์ž…๋‹ˆ๊นŒ? ์–ด๋–ค ์•„์ด๋””์–ด๋ผ๋„? ์ •๋ง ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค

๋‚˜๋„ ํ•„์š”ํ•ด, ์•„๋Š” ์‚ฌ๋žŒ?

๋‚˜๋„ ๊ถ๊ธˆํ•˜๋‹ค

@thesby @keunwoochoi

ํŒŒ์ผ์„ ๋‹ค์šด๋กœ๋“œํ•˜๋Š” ๋‹จ๊ณ„๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
๋ช…๋ น์ค„ ์ž์ฒด์—์„œ

1- ๊ฐœ์ธ ํ† ํฐ ํš๋“(์ด๋ฏธ ๊ฐ€์ง€๊ณ  ์žˆ์„ ์ˆ˜ ์žˆ์Œ)
2- ๋‹ค์Œ ๋ช…๋ น์„ ์‹คํ–‰ํ•˜์‹ญ์‹œ์˜ค

curl --cookie zenodo-cookies.txt "https://zenodo.org/record/1117372?token=<your token here>"
ํ•ด๋‹น ๋ ˆ์ฝ”๋“œ ๋ฒˆํ˜ธ 1117372๋ฅผ ์ ์ ˆํ•œ ๊ฒƒ์œผ๋กœ ๊ต์ฒดํ•˜์‹ญ์‹œ์˜ค.

์ฟ ํ‚ค ํŒŒ์ผ์„ ์ƒ์„ฑํ•œ ํ›„
์ด์ œ ๋ธŒ๋ผ์šฐ์ €์—์„œ ๋‹ค์šด๋กœ๋“œ ๋ฒ„ํŠผ์„ ๋งˆ์šฐ์Šค ์˜ค๋ฅธ์ชฝ ๋ฒ„ํŠผ์œผ๋กœ ํด๋ฆญํ•˜๊ณ  URL์„ ๋ณต์‚ฌํ•˜์‹ญ์‹œ์˜ค.

https://zenodo.org/record/1117372/files/musdb18.zip?download=1 ๊ณผ ๊ฐ™์•„์•ผ ํ•ฉ๋‹ˆ๋‹ค.

์ง€๊ธˆ ,

ํ„ฐ๋ฏธ๋„์—์„œ ์ด ๋ช…๋ น์„ ์‹คํ–‰ํ•˜๊ธฐ๋งŒ ํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค.

curl --cookie zenodo-cookies.txt "https://zenodo.org/record/1117372/files/musdb18.zip?download=1" --output musdb18.zip

๋‹ค์šด๋กœ๋“œ ๋งํฌ๋ฅผ ํŠน์ • ๋‹ค์šด๋กœ๋“œ URL๋กœ ๊ต์ฒด
๋‹ค์šด๋กœ๋“œ๋ฅผ ์‹œ์ž‘ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

--output musdb18.zip์˜ ํŒŒ์ผ ์ด๋ฆ„์„ ๋ฐ”๊พธ์‹ญ์‹œ์˜ค.

์›ํ•˜๋Š” ์ ์ ˆํ•œ ํŒŒ์ผ ์ด๋ฆ„์œผ๋กœ

์ด๊ฒƒ์€ ๋‚˜๋ฅผ ์œ„ํ•ด ์ž‘๋™ํ•˜๋Š” ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค.

๋ฌธ์ œ๊ฐ€ ๋ฐœ์ƒํ•˜๋ฉด ์•Œ๋ ค์ฃผ์‹ญ์‹œ์˜ค.
์ข‹์€ ํ•˜๋ฃจ ๋˜์„ธ์š”!

์ฟ ํ‚ค ์ ‘๊ทผ ๋ฐฉ์‹์— ๋ฌธ์ œ๊ฐ€ ์žˆ์—ˆ๊ธฐ ๋•Œ๋ฌธ์— ๋Œ€์‹  zenodo API ์ ‘๊ทผ ๋ฐฉ์‹(Python์—์„œ)์„ ์‚ฌ์šฉํ–ˆ์Šต๋‹ˆ๋‹ค.

๋จผ์ € ๊ฐœ์ธ ์•ก์„ธ์Šค ํ† ํฐ (๊ฐœ๋ฐœ์ž ์‘์šฉ ํ”„๋กœ๊ทธ๋žจ ์•„๋‹˜)์„ ๋งŒ๋“ค๊ณ  ๋ณ€์ˆ˜ ACCESS_TOKEN ํ• ๋‹นํ–ˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋Ÿฐ ๋‹ค์Œ ๋‚ด ๊ธฐ๋ก์—์„œ ๋‹ค์šด๋กœ๋“œํ•  ์ˆ˜ ์žˆ๋Š” ๋ชจ๋“  ํŒŒ์ผ์„ ์ฟผ๋ฆฌํ–ˆ์Šต๋‹ˆ๋‹ค.

import requests

ACCESS_TOKEN = "replace this with your access token"
record_id = "replace this with your record"

r = requests.get(f"https://zenodo.org/api/records/{record_id}", params={'access_token': ACCESS_TOKEN})
download_urls = [f['links']['self'] for f in r.json()['files']]
filenames = [f['key'] for f in r.json()['files']]

print(r.status_code)
print(download_urls)

์ด์ œ ๋‹ค์šด๋กœ๋“œํ•˜๋ ค๋Š” ํŒŒ์ผ์˜ URL๊ณผ ์ด๋ฆ„(์˜ˆ: ํ‚ค)์ด ์žˆ์œผ๋ฏ€๋กœ requests.get ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ํŒŒ์ผ์„ ๋‹ค์šด๋กœ๋“œํ•˜๊ณ  ํŒŒ์ผ์— ์ €์žฅํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

for filename, url in zip(filenames, download_urls):
    print("Downloading:", filename)
    r = requests.get(url, params={'access_token': ACCESS_TOKEN})
    with open(filename, 'wb') as f:
        f.write(r.content)

๋‚˜๋Š” ์ด๊ฒƒ์„ ์‹œ๋„ํ–ˆ๊ณ  ๊ทธ๊ฒƒ์€ ๋‚˜๋ฅผ ์œ„ํ•ด ์ผํ–ˆ์Šต๋‹ˆ๋‹ค.
https://gist.github.com/slint/d47fe5628916d14b8d0b987ac45aeb66#file -zenodo-restricted-curl-sh-L26

์ด ํŽ˜์ด์ง€๊ฐ€ ๋„์›€์ด ๋˜์—ˆ๋‚˜์š”?
0 / 5 - 0 ๋“ฑ๊ธ‰