<p>restic prune : λ©λͺ¨λ¦¬ λΆμ‘±</p>
restic version
restic 0.8.3 (v0.8.3-0-g272ccec7)
linux / amd64μμ go1.10μΌλ‘ μ»΄νμΌ
μ νν μ΄λ»κ² ν΄μμ μ·¨νμ΅λκΉ?
RESTIC_PASSWORD = foobar
RESTIC_REPOSITORY = / data / joe / restic
restic prune
μ μ₯μλ₯Ό μ μ₯νλ λ° μ¬μ©ν λ°±μλ / μλ² / μλΉμ€λ 무μμ λκΉ?
μ§μ νμΌ μ‘μΈμ€.
μμλλ νλ
Resticμ repoλ₯Ό μ 리ν©λλ€.
μ€μ νλ
2018-04-16 18:33:51: Starting restic prune
counting files in repo
building new index for repo
[11:59] 100.00% 291017 / 291017 packs
incomplete pack file (will be removed): 02b612398378d4b6d7b6a77b5f51cf3c541baa78f542886810c27488fc52f8a2
incomplete pack file (will be removed): 08ce4b79b2adacd1d85373c59ad28777e215cec52b16bf9963cb03106e1d547f
incomplete pack file (will be removed): 08d76e1953f8b06d936f46679986f0af9bfb0260ca1b81e641d30262f4e2fb69
incomplete pack file (will be removed): 0ada7a6774140391104de82da908a1027eaedc97be83a2c30ae26db85093aba1
incomplete pack file (will be removed): 0f960d42718f0e0b29ea10a47fef530638811fd1620843e13e4c4c3c139c9477
incomplete pack file (will be removed): 11aa5b87e3a44ff1678de573e21280ff56e8cdf93755c363c7ba1a03537b84b0
incomplete pack file (will be removed): 13b211c7fc26e7de47e8a5387749bb93f7737217809143e01caf7f826deeea13
incomplete pack file (will be removed): 13f00bb9960f95b0d71abc9ff5e5b86f53d370cd602a4c45e03505cefab328ab
incomplete pack file (will be removed): 1b0be1e761e0c369aaed7b2d03f07163854946dfe7b6bb749d4ff5d5c62cfd61
incomplete pack file (will be removed): 1ee341e8778661a76bc0f4d2a285457851cb5984f6ac6ee51d42bbdf705025ab
incomplete pack file (will be removed): 26de4ab07458e880217accac26098edcc46622d11d20dcbb57996cfebedbc859
incomplete pack file (will be removed): 274ede1a69f1f2634abeb6f7f7a9af2a4027a7f224d7a61a6e209ef8ee4c152f
incomplete pack file (will be removed): 383bad67ef592da43d0cc9793bc6e1afc4a4327fc2c504b753111888b6c189f3
incomplete pack file (will be removed): 39d74d322db6b80f613efabe4c1b7242b3ff31ab936b60a0a4ee903713f9b299
incomplete pack file (will be removed): 419b5a6e3ebbab53150b5106316e8f3183ef7dd6d3b0c10de0a611dedb7ea846
incomplete pack file (will be removed): 44ba8b9892a33391201339cbea4c963707ee16d30c6ca96d38d96bbeadfe23ca
incomplete pack file (will be removed): 45b75bb12def8334143681bf89811c68baace1376311e3ca79478021faf6b062
incomplete pack file (will be removed): 47947ddb8dd54856e22ec44b0de268668b89ccac77e6145a61f05870ab23c57d
incomplete pack file (will be removed): 4ba9ef53b1579d10d0682153382a6a30d12e444bf3ac0b55dd3e5bbe1f151d9f
incomplete pack file (will be removed): 4d36946b6c368baf824590d0846b1c28cb34943c8fd16734b83847b42ded8dab
incomplete pack file (will be removed): 61b294bea904324ef980d75c5b5da2f0e8b6d9be1aa16e55b7606afe5623df69
incomplete pack file (will be removed): 65555bcda953ddb00af9e3d141a4c274614d5dfd3a6345bf396872028852797c
incomplete pack file (will be removed): 716d9a5a48c2d364069fc9c9576c1683bd99cecc7b0232c53034a70bca9aed2b
incomplete pack file (will be removed): 71766e7583ba835480aa87e02566b23c26ba940ecc0f697beb2aaa0621422f4c
incomplete pack file (will be removed): 74f82d71b0f881c95f5eb54668355ce2d9f32a5a6087ec61e52618f909af377c
incomplete pack file (will be removed): 8476543253237a28264c87c70f41eb87883488f8d84314045cc07c3e7deee656
incomplete pack file (will be removed): 8d9d77e05a32a84da1a62159c245a004d55386923137f406c74b1b01281d07d3
incomplete pack file (will be removed): 8fa209c571b2cace97f5725c12c3a110deef8b8f7cfe6705f28175ac6182e9b8
incomplete pack file (will be removed): 9146ec8c10a77c248aa0db6aa291c25b83498848afddfbb4bb33b9928b003fdf
incomplete pack file (will be removed): 9f5e6c422683e05fe9cb7212d44c6492c55d24cb55230407cc59ebd06c96dde7
incomplete pack file (will be removed): a4ac5898b871d3a4343e2fd4c88aaae2e1a31879cce0d90d4e93527334a9deeb
incomplete pack file (will be removed): aa0d2b035ebcab381cd816ba2000d61c43f824621fafc74b81e5cfa1524ae94c
incomplete pack file (will be removed): aa2f5ba3a21a7b12ed23ab198ee457c5fde4921344cd350f2cb9ea675d05a1d6
incomplete pack file (will be removed): aa8f152200a3657728e68eddc1a9ad4c3cecf20af568d4fa01727ffb86005856
incomplete pack file (will be removed): af13847a8a95013afc34b59dfb40daa1c74f0067100a49adf5b1279559d3afd5
incomplete pack file (will be removed): af287ee5aa3c436cba495f4a73f30bf43d23e995d94913c9e64e12dd670b0a36
incomplete pack file (will be removed): ba38890edd8e2c84feb3af506427b6fd5cb4bd4150d7a5784f15f82e01598ca1
incomplete pack file (will be removed): bca933d271cd67feb9b42c55eb168d9f44bb568a121d3947fa443975e47e1ba1
incomplete pack file (will be removed): bd6cdb11204f445cc413cd04fbf56038eee417bc6312dd26624c037eba153417
incomplete pack file (will be removed): c436f5f332c6dbefa57456c8b0532cb3a89beb99564ab67b3ecc433bb0654df3
incomplete pack file (will be removed): cffa984794599843ec0b673c906a84f7813a4ee2346af3bacb6b7c2a217e172f
incomplete pack file (will be removed): d1aeb4847d269c3e6c469e2ac7a82e846f7891f357b0c3b25ef7fa8cc1bd9388
incomplete pack file (will be removed): d2269e9659755adb4d0c8944cb2abe1d6b2e75fa48e9c2ac749ec07c1222e196
incomplete pack file (will be removed): d29e8e66df6380d8f6bf65e18f35bc4597539e73c9a1932bca3dc067fa219ad6
incomplete pack file (will be removed): eb9e4049591e77ad69bc52d9bf830872798bc5c0814ad7cf65eb03befc38bed8
incomplete pack file (will be removed): ec458035798f3e4159314a50e0b7e3f5b8766564aef307c7f10fa7ac3f6dfa5d
incomplete pack file (will be removed): ecf1542bf4a15bc44d447973fc6312fe1a4e2528b5fd07aa9264f5b4bae6820c
incomplete pack file (will be removed): ee2ce3bd4a13ee395ad3a4a95dd0aa017e0b17ecf2cbd1faa66124e510717762
incomplete pack file (will be removed): ee637a1a84fc5819dee56c3192f38bf3895a00b42f6d3ddec5e2ed43c208e18a
incomplete pack file (will be removed): ef4e136efdd73d6e667c8afc8deccaf1d8e6c425d66eb377d5fa1fa85c484394
incomplete pack file (will be removed): f1d0b28ff0c5bdb8d77f9a7381324a54f574a7e9eb0f7e7320f315a26a71cb7a
repository contains 290966 packs (4042496 blobs) with 1.350 TiB
processed 4042496 blobs: 14417 duplicate blobs, 947.112 MiB duplicate
load all snapshots
find data that is still in use for 30 snapshots
[14:50] 100.00% 30 / 30 snapshots
found 4020862 of 4042496 data blobs still in use, removing 21634 blobs
will remove 51 invalid files
will delete 110 packs and rewrite 34143 packs, this frees 1.797 GiB
fatal error: runtime: out of memory
runtime stack:
runtime.throw(0xae011d, 0x16)
/usr/local/go/src/runtime/panic.go:619 +0x81
runtime.sysMap(0xc4f7c60000, 0x490000, 0x43fd00, 0xe91e58)
/usr/local/go/src/runtime/mem_linux.go:216 +0x20a
runtime.(*mheap).sysAlloc(0xe78660, 0x490000, 0x7fd3f522c098)
/usr/local/go/src/runtime/malloc.go:470 +0xd4
runtime.(*mheap).grow(0xe78660, 0x248, 0x0)
/usr/local/go/src/runtime/mheap.go:907 +0x60
runtime.(*mheap).allocSpanLocked(0xe78660, 0x248, 0xe91e68, 0xc45cd39ee0)
/usr/local/go/src/runtime/mheap.go:820 +0x301
runtime.(*mheap).alloc_m(0xe78660, 0x248, 0x410101, 0xc41e8c7fff)
/usr/local/go/src/runtime/mheap.go:686 +0x118
runtime.(*mheap).alloc.func1()
/usr/local/go/src/runtime/mheap.go:753 +0x4d
runtime.(*mheap).alloc(0xe78660, 0x248, 0xc45c010101, 0x4142ac)
/usr/local/go/src/runtime/mheap.go:752 +0x8a
runtime.largeAlloc(0x48f87c, 0x450101, 0x7fd3f5ca56c8)
/usr/local/go/src/runtime/malloc.go:826 +0x94
runtime.mallocgc.func1()
/usr/local/go/src/runtime/malloc.go:721 +0x46
runtime.systemstack(0x0)
/usr/local/go/src/runtime/asm_amd64.s:409 +0x79
runtime.mstart()
/usr/local/go/src/runtime/proc.go:1170
goroutine 1 [running]:
runtime.systemstack_switch()
/usr/local/go/src/runtime/asm_amd64.s:363 fp=0xc480ad81c8 sp=0xc480ad81c0 pc=0x454010
runtime.mallocgc(0x48f87c, 0x9cfd20, 0x979301, 0xc4d6371720)
/usr/local/go/src/runtime/malloc.go:720 +0x8a2 fp=0xc480ad8268 sp=0xc480ad81c8 pc=0x410882
runtime.makeslice(0x9cfd20, 0x48f87c, 0x48f87c, 0x1, 0x1, 0x58f)
/usr/local/go/src/runtime/slice.go:61 +0x77 fp=0xc480ad8298 sp=0xc480ad8268 pc=0x43fdc7
github.com/restic/restic/internal/repository.Repack(0xb5a640, 0xc420068e40, 0xb62260, 0xc4202510a0, 0xc424eb4120, 0xc475dfecf0, 0xc4200c38c0, 0x9c5582208afec32f, 0xc475dfecf0, 0xc475dfed20)
/tmp/restic-build-762914258/src/github.com/restic/restic/internal/repository/repack.go:60 +0xec3 fp=0xc480ad8a60 sp=0xc480ad8298 pc=0x752d93
main.pruneRepository(0xc420024012, 0x10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
src/github.com/restic/restic/cmd/restic/cmd_prune.go:278 +0x1e99 fp=0xc480ad9990 sp=0xc480ad8a60 pc=0x94d959
main.runPrune(0xc420024012, 0x10, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
src/github.com/restic/restic/cmd/restic/cmd_prune.go:85 +0x153 fp=0xc480ad9ab8 sp=0xc480ad9990 pc=0x94b943
main.glob..func17(0xe6b180, 0xe8ffd8, 0x0, 0x0, 0x0, 0x0)
src/github.com/restic/restic/cmd/restic/cmd_prune.go:25 +0x4d fp=0xc480ad9bb8 sp=0xc480ad9ab8 pc=0x95e07d
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).execute(0xe6b180, 0xe8ffd8, 0x0, 0x0, 0xe6b180, 0xe8ffd8)
/tmp/restic-build-762914258/src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:698 +0x46d fp=0xc480ad9c60 sp=0xc480ad9bb8 pc=0x6b34ed
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xe69e60, 0xffffffffffffffff, 0x0, 0x19)
/tmp/restic-build-762914258/src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:783 +0x2e4 fp=0xc480ad9d90 sp=0xc480ad9c60 pc=0x6b3c64
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).Execute(0xe69e60, 0x24, 0xc420217eb8)
/tmp/restic-build-762914258/src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:736 +0x2b fp=0xc480ad9dc0 sp=0xc480ad9d90 pc=0x6b395b
main.main()
src/github.com/restic/restic/cmd/restic/main.go:69 +0x1d6 fp=0xc480ad9f88 sp=0xc480ad9dc0 pc=0x95c796
runtime.main()
/usr/local/go/src/runtime/proc.go:198 +0x212 fp=0xc480ad9fe0 sp=0xc480ad9f88 pc=0x42b472
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc480ad9fe8 sp=0xc480ad9fe0 pc=0x456b31
goroutine 5 [syscall, 28 minutes]:
os/signal.signal_recv(0x0)
/usr/local/go/src/runtime/sigqueue.go:139 +0xa6
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.0
/usr/local/go/src/os/signal/signal_unix.go:28 +0x41
goroutine 6 [chan receive, 28 minutes]:
github.com/restic/restic/internal/restic.init.0.func1.1()
/tmp/restic-build-762914258/src/github.com/restic/restic/internal/restic/lock.go:266 +0xef
created by github.com/restic/restic/internal/restic.init.0.func1
/tmp/restic-build-762914258/src/github.com/restic/restic/internal/restic/lock.go:263 +0x35
goroutine 7 [select, 28 minutes, locked to thread]:
runtime.gopark(0xb01d80, 0x0, 0xad384f, 0x6, 0x18, 0x1)
/usr/local/go/src/runtime/proc.go:291 +0x11a
runtime.selectgo(0xc420042f50, 0xc42002c2a0)
/usr/local/go/src/runtime/select.go:392 +0xe50
runtime.ensureSigM.func1()
/usr/local/go/src/runtime/signal_unix.go:549 +0x1f4
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2361 +0x1
goroutine 17 [chan receive, 28 minutes]:
github.com/restic/restic/internal/restic.init.2.func1(0xc42002c1e0)
/tmp/restic-build-762914258/src/github.com/restic/restic/internal/restic/progress_unix.go:17 +0xa3
created by github.com/restic/restic/internal/restic.init.2
/tmp/restic-build-762914258/src/github.com/restic/restic/internal/restic/progress_unix.go:16 +0xab
goroutine 18 [chan receive, 28 minutes]:
main.CleanupHandler(0xc42021c1e0)
src/github.com/restic/restic/cmd/restic/cleanup.go:62 +0x5d
created by main.init.0
src/github.com/restic/restic/cmd/restic/cleanup.go:24 +0x6d
goroutine 33 [select, 3 minutes]:
main.refreshLocks(0xe71be0, 0xc4201322a0)
src/github.com/restic/restic/cmd/restic/lock.go:72 +0x221
created by main.lockRepository
src/github.com/restic/restic/cmd/restic/lock.go:49 +0x373
goroutine 40 [select]:
github.com/restic/restic/internal/restic.(*Progress).reporter(0xc4200c38c0)
/tmp/restic-build-762914258/src/github.com/restic/restic/internal/restic/progress.go:169 +0x130
created by github.com/restic/restic/internal/restic.(*Progress).Start
/tmp/restic-build-762914258/src/github.com/restic/restic/internal/restic/progress.go:96 +0x132
2018-04-16 19:02:18: Terminated: restic prune (return code 2)
νλμ μ¬ννλ λ¨κ³
4GiBμ λ©λͺ¨λ¦¬μ Intel (R) Core ( TM) i3-3225 CPU @ 3.30GHz. κ·Έλ° λ€μ ~ 1.5TB ν¬κΈ°μ μ μ₯μλ₯Ό λ§λ€κ³ μλ² μΈ‘μμ restic pruneμ μ€ννμμμ€.
μ΄ λ¬Έμ μ μμΈμ΄ 무μμΈμ§ μκ³ μμ΅λκΉ?
λ΄κ° resticμ μ€ννλ μμ€ν μλ 4GiBμ λ©λͺ¨λ¦¬κ° μμ΅λλ€. λ¬Όλ‘ , λ λ§μμλ‘ μ’μ΅λλ€. κ·Έλ¬λ μ΄λ¬ν μ νμ μ€λ₯μ λν΄ resticμ΄ μ€ν¨νμ§ μλλ€λ©΄ λ μ’μ κ²μ λλ€.
λ¬Έμ λ₯Ό ν΄κ²°νλ λ°©λ²μ μκ³ μμ΅λκΉ?
νμ€ν. μμ€ν μ λ λ§μ λ©λͺ¨λ¦¬λ₯Ό μΆκ°νλ κ²μ΄ νλμ μ루μ μ λλ€. λλλ³΄λ€ ν¨μ¨μ μΌλ‘ ν΄μμ μ·¨νμμμ€ (리μμ€ λμΆμ΄ μλμ§ νμΈ).
resticμ΄ λΉμ μ λμκ±°λ μ΄λ€ μ μΌλ‘λ λΉμ μ ν볡νκ² λ§λ€μμ΅λκΉ?
μΉμ°¬μ μν΄ λμνλ κ±°μ£ ? λ¬Όλ‘ μ
λλ€. Resticμ μμ¬ ν μ¬μ§μμ΄ κ½€ λ©μ§ μ νμ
λλ€. μ λ ν΄λΌμ΄μΈνΈ μΈ‘ μνΈν, KDFμ μ μ ν μ¬μ©, ACL μ§μ, FUSEλ₯Ό ν΅ν λ°±μ
λ§μ΄νΈλ₯Ό μ’μν©λλ€. κ·Έκ²μ λ¬Όμμ μ£½μ λμ μ΄μ rsync μ κ·Ό λ°©μμ λ₯κ°ν©λλ€. νμ§λ§ μμμ λν λ°°κ³ νμ μ‘°κΈ λμ΄ μ€ μ μμκΉμ? :-)
johndoe31415
λͺ¨λ 31 λκΈ
μ μΆμ μ 보면 ν΅μ¬ λ¬Έμ λ κ³ λ£¨ν΄ 1μ μλ€κ³ μκ°ν©λλ€.
μ΄κ²μ μ§λ¬Έμ μ κΈ°ν©λλ€ : μ°λ¦¬κ° λ ν¨μ¨μ μΌλ‘ μ¬ ν¬μ₯ ν μ μμ΅λκΉ? μλ₯Ό λ€μ΄, μμ°¨μ λλ μμ μμ blobμλ‘λνκ±°λ κΈ°λ‘ μ€ λ³΅μ¬λ₯Ό μ¬μ©νμ¬ blobμ λ°μ΄ν°λ₯Ό μ μ§νλ €λ©΄ λλ³΄λ€ λ λ§μ μν€ν μ² μ§μμ κ°μ§ μ¬λμ΄ νμν©λλ€.
meisterluk
μ 2018λ
04μ 16μΌ
μ, κ°μ§ μΉκΈ°μλ λ무 λ§μ λ©λͺ¨λ¦¬κ° νμνλ©° ν¨μ¬ λ ν¨μ¨μ μΌ μ μμ΅λλ€. μ΅μ ν ν μκ°μ΄ νμν©λλ€. μ κ³ ν΄ μ£Όμ μ κ°μ¬ν©λλ€!
fd0
μ 2018λ
05μ 05μΌ
μ΄κ²μ 곡μ νκΈ°μ κ°μ₯ μ’μ μ₯μλ νμ€νμ§ μμ§λ§ μ¬κΈ°μ μΆκ° ν λ°μ΄ν° ν¬μΈνΈκ° νλ λ μμ΅λλ€. μ΄μ λ°€ OOM μ€λ₯κ° λ°μν ν resticμ λ€μ λΉνμ±νν©λλ€. μλ²μμ λ°±μ νλ λμ μ½ 25GBμ RAMμ μ¬μ©νλ μμΉμ λλ¬νμ΅λλ€.
Aug 21 03:24:35 courtlistener kernel: [3131661.383314] Out of memory: Kill process 28903 (restic) score 359 or sacrifice child
Aug 21 03:24:35 courtlistener kernel: [3131661.384564] Killed process 28903 (restic) total-vm:25856956kB, anon-rss:24415312kB, file-rss:0kB
Aug 21 03:24:35 courtlistener kernel: [3131661.400102] postgres invoked oom-killer: gfp_mask=0x2000d0, order=2, oom_score_adj=0
μ½ 20GBμ RAMμ μ¬μ©νλ λ°μ΄ν°λ² μ΄μ€κ°μλ μλ²μμλ ν΄μκ³Ό κ°μ μμ μ μν΄ μ μ§νλ μ€λ² ν€λκ° μμ§λ§ λ°±μ μ μννκΈ° μν΄ 30GB μ΄μμ RAM (λλ κ·Έ μ΄μ?)μ 무λ£λ‘ μ μ§ν μλ μμ΅λλ€. μ°λ¦¬λ μ¬μ ν DBμ μ€νΈλ¦¬λ° λ°±μ μ μν΄ resticμ μ¬μ©νκ³ μμ§λ§, μ΄κ²μ λ무 λ§μ λ©λͺ¨λ¦¬μ λλ€.
mlissner
μ 2018λ
08μ 21μΌ
μ, λ°±μ μ€μ΄ μλμ? μλλ©΄ κ°μ§ μΉκΈ°?
fd0
μ 2018λ
08μ 21μΌ
λ°±μ μ€. μλκ° λΉνμ±νλμμ΅λλ€. pruneμ΄ μλ€λ κ²μ Backblazeκ° λ λ§μ λΉμ©μ μ²κ΅¬νλ€λ κ²μ μλ―Ένλ―λ‘ μ±λ₯ / λ©λͺ¨λ¦¬ λ¬Έμ κ° ν΄κ²° λ λκΉμ§ λΉνμ±ννλ κ²μ΄ μ’μ΅λλ€. νμ§λ§ μ΄μ λ°±μ λ λΉνμ±νν΄μΌνμ΅λλ€.
mlissner
μ 2018λ
08μ 21μΌ
νμ¨, μ£μ‘ν©λλ€.
fd0
μ 2018λ
08μ 21μΌ
κΈμ, κ·Έκ²μ΄ 무μμ λκΉ! μ°λ¦¬λ μ€μΌμΌλ§μ΄ μ½μ§ μκ³ λλλ‘ μΌμ μΆμ§νλ€λ κ²μ λꡬλ μκ³ μμ΅λλ€. λλ μΌλ°μ μΌλ‘ κ·Έλλ μ¬μ ν resticμ μ’μν©λλ€. μ€νΈλ¦¬λ° λ°±μ ? μ. μκ°λ½μ ν€μ€ . λ§μλ.
mlissner
μ 2018λ
08μ 21μΌ
@mlissner μ μ₯μ ν¬κΈ°λ μΌλ§μ λκΉ?
dionorgua
μ 2018λ
08μ 22μΌ
μ΄μ λ κΈ°λΆμ΄ μ’μ§ μμμ§λ§ μ€λμ ν¨μ¬ λμμ‘μ΅λλ€ :)
fd0
μ 2018λ
08μ 22μΌ
μ΄λ΄ μλ μ°λ, μ¬κΈ°μ μΆκ° ν κΈ°μ μ΄ μμ§λ§ λ§μ§λ§ μ½λ©νΈ νμλ λ€μκ³Ό κ°μ΄ λ§ν νμκ° μμ΅λλ€. μλ΅μ΄ λ§€μ° λΉ λ¦ λλ€. μ΄μ μ μ¬κΈ°μμ κ°μ§ λ²κ·Έλ₯Όλ³΄κ³ νλλ° (μλ₯Ό λ€μ΄ HW λ¬Έμ μ λΏλ¦¬λ₯Όλκ³ μλ κ²μΌλ‘ νλͺ λ¨) λΉμ μ νμ λ§€μ° μ§μν΄ μμ΅λλ€. λͺ¨λ μ€ν μμ€ νλ‘μ νΈλ λΉμ μ νμμλ‘ μΌλ κ²μ΄ νμ΄μ΄λΌκ³ μκ°ν μ μμΌλ©°, λΉμ μ΄ ν΄μμ μν΄ κ·Έ μ΄μμΌλ‘ λμκ°κ³ μμμ λΆλͺ ν 보μ¬μ€λλ€. κ·Έκ²μ λλΌμ΄ λꡬμ΄λ©° μ°λ¦¬ λͺ¨λλ κ·Έκ²μ μκ³ μμ΅λλ€. μ΄μ κΈ°λΆμ΄ μ’μ§ μλ€κ³ λ§νμ λ λ―Ώκ³ μμ§λ§ λ°μΈμλ νμ€ν λνλμ§ μμμΌλ μμ¬νμΈμ. κ³μν΄μ λ©μ§ μμ μνκ³ Resticμ λμμμ΄ κ°μ ν΄μ£Όμ μ λ€μ ν λ² κ°μ¬λ립λλ€. 건배, μνλ€μ€
johndoe31415
μ 2018λ
08μ 22μΌ
restic checkμ ν¨κ» # 1830μμμ λμΌν λ¬Έμ κ° μμ΅λλ€.
GOGC = 20μ μ€μ νλ €κ³ νμ§λ§ μ΄μ΄ μμ΅λλ€. restic checkλ μ μ μ€ν ν νμλ μ¬μ ν oom-killedμ
λλ€ (8GB ram, ~ 1,5TB repo).
/ tmp / restic-check-cache-961467159μμ μμ μΊμ μ¬μ©
02a14a0b μ μ₯μκ° μ±κ³΅μ μΌλ‘ μ΄λ Έμ΅λλ€. λΉλ°λ²νΈκ° μ νν©λλ€.
μ μ₯μμ λν λ
μ μ κΈ μμ±
λ‘λ μΈλ±μ€
λͺ¨λ ν© νμΈ
μ€λ
μ·, νΈλ¦¬ λ° Blob νμΈ
μ΄ν΄
κ³Ό
μ²μΉ κ³Όμ 16,350 (restic) μ΄-VM : 2428260kB, anon- RSS : 2375472kB νμΌ - RSS : 0KB , shmem- RSS : 0KB
Olen
μ 2018λ
08μ 22μΌ
@mlissner μ μ₯μ ν¬κΈ°λ μΌλ§μ λκΉ?
μ£μ‘ν©λλ€.μ΄ λ¬Έλ§₯μμ "μ μ₯μ"κ° λ¬΄μμ μλ―Ένλμ§ μ λͺ¨λ₯΄κ² μ΅λλ€.νμ§λ§ μ°λ¦¬κ° λ°± λΈλ μ΄μ¦μμλ 물건μ μμ μλ―Ένλ κ² κ°λμ? λ§λ€λ©΄ λ²ν· ν¬κΈ°λ μ½ 5.2TBμ λλ€.
λ°±μ νλ μμ΄ μΌλ§λλλμ§ μλ―Ένλ€λ©΄ νμΌμ΄ 15M μ λ μΈ 2TB μ λμ λκΉ? μ΄κ²μ λλΆλΆμ κ²°μ½ λ³νμ§ μμ§λ§ μ°λ¦¬λ λ§€μΌ λ λ§μ΄ μ»μ΅λλ€. λ²λ₯ λ¬Έμ μμΉ΄μ΄λΈ μΈ CourtListener.comμ μ΄μν©λλ€.
mlissner
μ 2018λ
08μ 22μΌ
@ johndoe31415 μΉμ ν λ§ κ°μ¬ν©λλ€! λλ‘λ μ¬μ νν΄μΌ ν μΌμ μμ΄ μλμ μΌ μ μμ΅λλ€ ...
fd0
μ 2018λ
08μ 22μΌ
κ°μ λ¬Έμ κ° μμ΅λλ€ (8GB RAM, repoλ 267GB)
$ restic version
restic 0.9.3 compiled with go1.10.4 on linux/amd64
λͺ
λ Ή : restic -r sftp:[email protected]:/mybackup --password-file mypassword --cache-dir /var/cache/restic forget --prune --keep-daily=7 --keep-weekly=4 --keep-monthly=6 --keep-yearly=1
1 snapshots have been removed, running prune
counting files in repo
building new index for repo
[1:14:56] 100.00% 56025 / 56025 packs
repository contains 56025 packs (4844700 blobs) with 257.876 GiB
processed 4844700 blobs: 0 duplicate blobs, 0B duplicate
load all snapshots
find data that is still in use for 44 snapshots
fatal error: runtime: out of memory
runtime stack:
runtime.throw(0xb92259, 0x16)
/usr/lib/go-1.10/src/runtime/panic.go:616 +0x81
runtime.sysMap(0xc50f780000, 0x12980000, 0xf6ce00, 0xf85fd8)
/usr/lib/go-1.10/src/runtime/mem_linux.go:216 +0x20a
runtime.(*mheap).sysAlloc(0xf6c6e0, 0x12980000, 0x7f7d482609c8)
/usr/lib/go-1.10/src/runtime/malloc.go:470 +0xd4
runtime.(*mheap).grow(0xf6c6e0, 0x94c0, 0x0)
/usr/lib/go-1.10/src/runtime/mheap.go:907 +0x60
runtime.(*mheap).allocSpanLocked(0xf6c6e0, 0x94c0, 0xf85fe8, 0x7f7d6bffed88)
/usr/lib/go-1.10/src/runtime/mheap.go:820 +0x301
runtime.(*mheap).alloc_m(0xf6c6e0, 0x94c0, 0x410101, 0xc41a96e1ff)
/usr/lib/go-1.10/src/runtime/mheap.go:686 +0x118
runtime.(*mheap).alloc.func1()
/usr/lib/go-1.10/src/runtime/mheap.go:753 +0x4d
runtime.(*mheap).alloc(0xf6c6e0, 0x94c0, 0x7f7d6b010101, 0x415afc)
/usr/lib/go-1.10/src/runtime/mheap.go:752 +0x8a
runtime.largeAlloc(0x12980000, 0x450101, 0x7f7d79aded90)
/usr/lib/go-1.10/src/runtime/malloc.go:826 +0x94
runtime.mallocgc.func1()
/usr/lib/go-1.10/src/runtime/malloc.go:721 +0x46
runtime.systemstack(0xc400000000)
/usr/lib/go-1.10/src/runtime/asm_amd64.s:409 +0x79
runtime.mstart()
/usr/lib/go-1.10/src/runtime/proc.go:1175
goroutine 1 [running]:
runtime.systemstack_switch()
/usr/lib/go-1.10/src/runtime/asm_amd64.s:363 fp=0xc4d2cc6b88 sp=0xc4d2cc6b80 pc=0x4552d0
runtime.mallocgc(0x12980000, 0xaffe40, 0xbb6b01, 0xaed4a0)
/usr/lib/go-1.10/src/runtime/malloc.go:720 +0x8a2 fp=0xc4d2cc6c28 sp=0xc4d2cc6b88 pc=0x4120d2
runtime.newarray(0xaffe40, 0x110000, 0xc42013c000)
/usr/lib/go-1.10/src/runtime/malloc.go:855 +0x6a fp=0xc4d2cc6c58 sp=0xc4d2cc6c28 pc=0x41245a
runtime.makeBucketArray(0xb32640, 0x42f5cad737fd4814, 0xc4d2cc6cd0, 0x401e32)
/usr/lib/go-1.10/src/runtime/hashmap.go:881 +0xe2 fp=0xc4d2cc6c90 sp=0xc4d2cc6c58 pc=0x40a6b2
runtime.hashGrow(0xb32640, 0xc44ee1c960)
/usr/lib/go-1.10/src/runtime/hashmap.go:905 +0x80 fp=0xc4d2cc6ce0 sp=0xc4d2cc6c90 pc=0x40a830
runtime.mapassign(0xb32640, 0xc44ee1c960, 0xc4d2cc6f9f, 0xc4cd1f7d40)
/usr/lib/go-1.10/src/runtime/hashmap.go:579 +0x2b7 fp=0xc4d2cc6d70 sp=0xc4d2cc6ce0 pc=0x409777
github.com/restic/restic/internal/restic.BlobSet.Insert(...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/blob_set.go:26
github.com/restic/restic/internal/restic.FindUsedBlobs(0xc15640, 0xc42020e9c0, 0xc1d760, 0xc42021fa40, 0x467d86ab22e4ce70, 0x21a44382a1438885, 0x829c686ff3f4eb07, 0x2bbf9f1c26cb983a,
0xc44ee1c960, 0xc44ee1c990, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/find.go:9 +0x10c fp=0xc4d2cc6fe8 sp=0xc4d2cc6d70 pc=0x6f9ecc
github.com/restic/restic/internal/restic.FindUsedBlobs(0xc15640, 0xc42020e9c0, 0xc1d760, 0xc42021fa40, 0xad0966088f423528, 0xc0855b2604177a2c, 0x416e998bdcc27c99, 0xd45bd561fbc152b3,
0xc44ee1c960, 0xc44ee1c990, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/find.go:31 +0x5a6 fp=0xc4d2cc7260 sp=0xc4d2cc6fe8 pc=0x6fa366
github.com/restic/restic/internal/restic.FindUsedBlobs(0xc15640, 0xc42020e9c0, 0xc1d760, 0xc42021fa40, 0x7ed337032395c12e, 0x697a1ab6f4be111b, 0x77f9fa7adfc1bd09, 0x1821c35434fcefb9,
0xc44ee1c960, 0xc44ee1c990, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/find.go:31 +0x5a6 fp=0xc4d2cc74d8 sp=0xc4d2cc7260 pc=0x6fa366
github.com/restic/restic/internal/restic.FindUsedBlobs(0xc15640, 0xc42020e9c0, 0xc1d760, 0xc42021fa40, 0x4edfb5f894134ac9, 0xf0bba160fd7f8f19, 0xe83800bc1f59b772, 0xc1b5cb4b8c32a833,
0xc44ee1c960, 0xc44ee1c990, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/find.go:31 +0x5a6 fp=0xc4d2cc7750 sp=0xc4d2cc74d8 pc=0x6fa366
github.com/restic/restic/internal/restic.FindUsedBlobs(0xc15640, 0xc42020e9c0, 0xc1d760, 0xc42021fa40, 0x5efbd8585a9898d2, 0x448917a1ace19706, 0x8ee237ec0622eb45, 0xc9d306cd39e8d05d,
0xc44ee1c960, 0xc44ee1c990, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/find.go:31 +0x5a6 fp=0xc4d2cc79c8 sp=0xc4d2cc7750 pc=0x6fa366
github.com/restic/restic/internal/restic.FindUsedBlobs(0xc15640, 0xc42020e9c0, 0xc1d760, 0xc42021fa40, 0x260fdd5acba993c3, 0x3ed04f4eeab05c9f, 0xe5ef8da60b35f98, 0xd090ad85e968ef5f,
0xc44ee1c960, 0xc44ee1c990, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/find.go:31 +0x5a6 fp=0xc4d2cc7c40 sp=0xc4d2cc79c8 pc=0x6fa366
github.com/restic/restic/internal/restic.FindUsedBlobs(0xc15640, 0xc42020e9c0, 0xc1d760, 0xc42021fa40, 0x60f7906c006bf615, 0x1bd2d33fc9768849, 0xee593a9983a1f5bf, 0x9ab809619e8cc152,
0xc44ee1c960, 0xc44ee1c990, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/find.go:31 +0x5a6 fp=0xc4d2cc7eb8 sp=0xc4d2cc7c40 pc=0x6fa366
github.com/restic/restic/internal/restic.FindUsedBlobs(0xc15640, 0xc42020e9c0, 0xc1d760, 0xc42021fa40, 0x8f00b9047475f470, 0x2094345d9e4dbb28, 0x3905971b8623f05c, 0x14b3bcdc097489ee,
0xc44ee1c960, 0xc44ee1c990, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/find.go:31 +0x5a6 fp=0xc4d2cc8130 sp=0xc4d2cc7eb8 pc=0x6fa366
main.pruneRepository(0x7ffc3e67bcea, 0x39, 0x7ffc3e67bd34, 0x58, 0x0, 0x0, 0x0, 0x7ffc3e67bd99, 0x11, 0x0, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/cmd/restic/cmd_prune.go:191 +0xea3 fp=0xc4d2cc9060 sp=0xc4d2cc8130 pc=0x9d23e3
main.runForget(0x0, 0x0, 0x7, 0x4, 0x6, 0x1, 0x0, 0x0, 0x0, 0x0, ...)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/cmd/restic/cmd_forget.go:241 +0xece fp=0xc4d2cc9970 sp=0xc4d2cc9060 pc=0x9ca82e
main.glob..func10(0xf59720, 0xc420276160, 0x0, 0xb, 0x0, 0x0)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/cmd/restic/cmd_forget.go:24 +0xb5 fp=0xc4d2cc9b60 sp=0xc4d2cc9970 pc=0x9e4935
github.com/spf13/cobra.(*Command).execute(0xf59720, 0xc420282180, 0xb, 0xc, 0xf59720, 0xc420282180)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/spf13/cobra/command.go:762 +0x468 fp=0xc4d2cc9c50 sp=0xc4d2cc9b60 pc=0x6bae28
github.com/spf13/cobra.(*Command).ExecuteC(0xf59980, 0xffffffffffffffff, 0x0, 0x19)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/spf13/cobra/command.go:852 +0x30a fp=0xc4d2cc9d90 sp=0xc4d2cc9c50 pc=0x6bb83a
github.com/spf13/cobra.(*Command).Execute(0xf59980, 0x23, 0xc4201cfeb8)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/spf13/cobra/command.go:800 +0x2b fp=0xc4d2cc9dc0 sp=0xc4d2cc9d90 pc=0x6bb50b
main.main()
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/cmd/restic/main.go:86 +0x1d6 fp=0xc4d2cc9f88 sp=0xc4d2cc9dc0 pc=0x9e3ac6
runtime.main()
/usr/lib/go-1.10/src/runtime/proc.go:198 +0x212 fp=0xc4d2cc9fe0 sp=0xc4d2cc9f88 pc=0x42ccc2
runtime.goexit()
/usr/lib/go-1.10/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4d2cc9fe8 sp=0xc4d2cc9fe0 pc=0x457e61
goroutine 5 [syscall, 92 minutes]:
os/signal.signal_recv(0x0)
/usr/lib/go-1.10/src/runtime/sigqueue.go:139 +0xa6
os/signal.loop()
/usr/lib/go-1.10/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.0
/usr/lib/go-1.10/src/os/signal/signal_unix.go:28 +0x41
goroutine 6 [chan receive, 92 minutes]:
github.com/restic/restic/internal/restic.init.0.func1.1()
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/lock.go:257 +0xef
created by github.com/restic/restic/internal/restic.init.0.func1
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/lock.go:254 +0x35
goroutine 7 [select, 92 minutes, locked to thread]:
runtime.gopark(0xbb6a48, 0x0, 0xb84e9f, 0x6, 0x18, 0x1)
/usr/lib/go-1.10/src/runtime/proc.go:291 +0x11a
runtime.selectgo(0xc42003ef50, 0xc420086240)
/usr/lib/go-1.10/src/runtime/select.go:392 +0xe50
runtime.ensureSigM.func1()
/usr/lib/go-1.10/src/runtime/signal_unix.go:549 +0x1f4
runtime.goexit()
/usr/lib/go-1.10/src/runtime/asm_amd64.s:2361 +0x1
goroutine 8 [chan receive, 92 minutes]:
github.com/restic/restic/internal/restic.init.2.func1(0xc420086180)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/progress_unix.go:17 +0xa3
created by github.com/restic/restic/internal/restic.init.2
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/progress_unix.go:16 +0xab
goroutine 10 [IO wait, 2 minutes]:
internal/poll.runtime_pollWait(0x7f7d7829fbc0, 0x72, 0xc42004abe0)
/usr/lib/go-1.10/src/runtime/netpoll.go:173 +0x57
internal/poll.(*pollDesc).wait(0xc4202146a8, 0x72, 0xffffffffffffff01, 0xc0ef60, 0xf14738)
/usr/lib/go-1.10/src/internal/poll/fd_poll_runtime.go:85 +0x9b
internal/poll.(*pollDesc).waitRead(0xc4202146a8, 0xc4b4f12001, 0x4, 0x4)
/usr/lib/go-1.10/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc420214690, 0xc4b4f120c4, 0x4, 0x4, 0x0, 0x0, 0x0)
/usr/lib/go-1.10/src/internal/poll/fd_unix.go:157 +0x17d
os.(*File).read(0xc42020a140, 0xc4b4f120c4, 0x4, 0x4, 0x18, 0x0, 0x0)
/usr/lib/go-1.10/src/os/file_unix.go:226 +0x4e
os.(*File).Read(0xc42020a140, 0xc4b4f120c4, 0x4, 0x4, 0x18, 0x0, 0x0)
/usr/lib/go-1.10/src/os/file.go:107 +0x6a
io.ReadAtLeast(0xc0d480, 0xc420144ae0, 0xc4b4f120c4, 0x4, 0x4, 0x4, 0xa6e780, 0xc42013a301, 0xc4b4f120c4)
/usr/lib/go-1.10/src/io/io.go:309 +0x86
io.ReadFull(0xc0d480, 0xc420144ae0, 0xc4b4f120c4, 0x4, 0x4, 0xc42004aea8, 0x4056e7, 0xc42d7cc420)
/usr/lib/go-1.10/src/io/io.go:327 +0x58
github.com/pkg/sftp.recvPacket(0xc0d480, 0xc420144ae0, 0x4051c5, 0xc42d7cc420, 0xc42004af70, 0x1, 0x985f32, 0xc445504001)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/pkg/sftp/packet.go:143 +0x86
github.com/pkg/sftp.(*conn).recvPacket(0xc420144ae0, 0xc42004af70, 0xc4000378eb, 0xc445504108, 0x1, 0x0, 0x0)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/pkg/sftp/conn.go:22 +0x37
github.com/pkg/sftp.(*clientConn).recv(0xc420144ae0, 0x0, 0x0)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/pkg/sftp/conn.go:64 +0xe5
github.com/pkg/sftp.(*clientConn).loop(0xc420144ae0)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/pkg/sftp/conn.go:49 +0x5b
created by github.com/pkg/sftp.NewClientPipe
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/pkg/sftp/client.go:128 +0x1c3
goroutine 18 [chan receive, 92 minutes]:
main.CleanupHandler(0xc4201fa060)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/cmd/restic/cleanup.go:62 +0x5d
created by main.init.0
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/cmd/restic/cleanup.go:24 +0x6d
goroutine 19 [IO wait, 92 minutes]:
internal/poll.runtime_pollWait(0x7f7d7829ff00, 0x72, 0xc420038c70)
/usr/lib/go-1.10/src/runtime/netpoll.go:173 +0x57
internal/poll.(*pollDesc).wait(0xc420214568, 0x72, 0xffffffffffffff01, 0xc0ef60, 0xf14738)
/usr/lib/go-1.10/src/internal/poll/fd_poll_runtime.go:85 +0x9b
internal/poll.(*pollDesc).waitRead(0xc420214568, 0xc4202b8001, 0x1000, 0x1000)
/usr/lib/go-1.10/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc420214550, 0xc4202b8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/lib/go-1.10/src/internal/poll/fd_unix.go:157 +0x17d
os.(*File).read(0xc42020a120, 0xc4202b8000, 0x1000, 0x1000, 0x0, 0x0, 0xc420038d78)
/usr/lib/go-1.10/src/os/file_unix.go:226 +0x4e
os.(*File).Read(0xc42020a120, 0xc4202b8000, 0x1000, 0x1000, 0x1000, 0x1000, 0x0)
/usr/lib/go-1.10/src/os/file.go:107 +0x6a
bufio.(*Scanner).Scan(0xc420038f30, 0xc115a0)
/usr/lib/go-1.10/src/bufio/scan.go:208 +0xbf
github.com/restic/restic/internal/backend/sftp.startClient.func1(0xc115a0, 0xc42020a120, 0xb827c8, 0x3)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/backend/sftp/sftp.go:53 +0x1e6
created by github.com/restic/restic/internal/backend/sftp.startClient
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/backend/sftp/sftp.go:51 +0x1b9
goroutine 20 [syscall, 92 minutes]:
syscall.Syscall6(0xf7, 0x1, 0x2035, 0xc420039598, 0x1000004, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/lib/go-1.10/src/syscall/asm_linux_amd64.s:44 +0x5
os.(*Process).blockUntilWaitable(0xc4202326c0, 0x0, 0x0, 0x0)
/usr/lib/go-1.10/src/os/wait_waitid.go:31 +0x98
os.(*Process).wait(0xc4202326c0, 0x0, 0x0, 0x0)
/usr/lib/go-1.10/src/os/exec_unix.go:22 +0x3c
os.(*Process).Wait(0xc4202326c0, 0x0, 0x0, 0x0)
/usr/lib/go-1.10/src/os/exec.go:123 +0x2b
os/exec.(*Cmd).Wait(0xc420238580, 0x0, 0x0)
/usr/lib/go-1.10/src/os/exec/exec.go:461 +0x5c
github.com/restic/restic/internal/backend/sftp.startClient.func2(0xc420238580, 0xc420144a80)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/backend/sftp/sftp.go:76 +0x2f
created by github.com/restic/restic/internal/backend/sftp.startClient
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/backend/sftp/sftp.go:75 +0x29e
goroutine 34 [select, 2 minutes]:
main.refreshLocks(0xf65c00, 0xc420136060)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/cmd/restic/lock.go:72 +0x221
created by main.lockRepository
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/cmd/restic/lock.go:49 +0x373
goroutine 98 [select, 15 minutes]:
github.com/restic/restic/internal/restic.(*Progress).reporter(0xc44ee282c0)
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/progress.go:169 +0x130
created by github.com/restic/restic/internal/restic.(*Progress).Start
/build/restic-XDCNVl/restic-0.9.3+ds/obj-x86_64-linux-gnu/src/github.com/restic/restic/internal/restic/progress.go:96 +0x132
legrostdg
μ 2018λ
11μ 07μΌ
μꡬμ μΈ μ루μ μ μλμ§λ§, resticμ΄ "ν΄λ μ±"μ΄μ΄μ λ΄κ° ν μΌμ λ¨μν νΌλΈλ¦ ν΄λΌμ°λμμ VMμ μ€ν μ νλ κ²μ΄ μμ΅λλ€. vCPUμ RAMμ λ§μ΄ μ 곡νκ³ (16 μ½μ΄μ 64GB RAM λ±μ μ¬μ©νλ€κ³ μκ°ν©λλ€) κ±°κΈ°μμ ν΄μμ μ·¨νμ΅λλ€.
κ·Έκ²μ λͺ μκ° λμ μ€νλμκ³ λͺ λ¬λ¬μ λΉμ©μ΄ λ€μμ§λ§ μ μ΄λ λλ¬μ΅λλ€.
μ€λλ κ²μ μ 리 ν ν μ΄μ μΌλ° μλ²μμ forget --pruneμ μ€νν μ μμ΅λλ€.
λ¬Όλ‘ νΌλΈλ¦ ν΄λΌμ°λμμ λ°±μ μλ²λ₯Ό μ¬μ©ν μ μμ΄μΌν©λλ€.
Olen
μ 2018λ
11μ 07μΌ
λ¬Όλ‘ νΌλΈλ¦ ν΄λΌμ°λμμ λ°±μ μλ²λ₯Ό μ¬μ©ν μ μμ΄μΌν©λλ€.
(Un) μνκΉκ²λ .. μ΄μ¨λ κ³ λ§μμ!
legrostdg
μ 2018λ
11μ 07μΌ
μλ νμΈμ
μλ²μ λ©λͺ¨λ¦¬κ° μΆ©λΆνλλΌλ λμΌν λ¬Έμ κ° λ°μν©λλ€. μ½ 150Tλ₯Ό λ°±μ νκ³ λͺ κ°μ μ€λ μ·μ μ κ±°νλ €κ³ ν©λλ€. μ€λ₯ λ° λ©λͺ¨λ¦¬ μ¬μ©λ₯ μ μ²¨λΆ λ μ€ν¬λ¦° μ·μ νμλ©λλ€.
μμ λ μ€ν 0.9.3.
prune_outofmemory.txt
ifelsefi
μ 2019λ
01μ 20μΌ
λλ μ΄κ²μ μ€ννκ³ μμ΅λλ€.
restic 0.9.4 (1107eef2) compiled with go1.10.4 on linux/arm
512MB RAM λ§μλ ReadyNAS Duo v1 B2μ μ μ₯λ μ΄ 130190 κ°μ νμΌμ΄ 689.646GiB (Restic μΆλ ₯μ λ°λΌ)μ μμ΅λλ€.
κ°μ§ μΉκΈ°λ μκ° (~ 32 μκ°)μ΄ κ±Έλ¦¬μ§ λ§ λ¬Έμ μμ΄ μλ£λ©λλ€.
λ΄κ° ν ν κ°μ§λ λ€μκ³Ό κ°μ΄ μ€μ νλ κ²μ
λλ€.
$ export GOGC=33
$ restic --option b2.connections=3 <other options here are omitted>
λ΄ Repoκ° ν΄λΌμ°λμ μμΌλ―λ‘ λͺ¨λ μ€μ μ λ¬Έμ κ°μλ κ²μ μλκΉμ?
richard-scott
μ 2019λ
01μ 22μΌ
λκ° μμ£Ό μ΄μν μΌμ΄ λ²μ΄μ§κ³ μμ΅λλ€ ... κ·Έλλ μ§μ ν μλ μμ΅λλ€. prune κ° λ무 λ§μ λ©λͺ¨λ¦¬λ₯Ό μ¬μ©νκ³ μλ€κ³ μκ°νμ§λ§, μ΄κ²μ΄ μ΄λμμ μΌμ΄λλμ§ μμ ν μ΄λ‘μ΅λλ€ ...
fd0
μ 2019λ
01μ 22μΌ
λκ΅°κ°κ° resticμ νλ‘νμΌ λ§νκ³ μ μ₯μμμ λμ λ©λͺ¨λ¦¬ μ¬μ©μ μμΈμ μλ³νλ λ° μκ°μ 보λ΄κ³ μΆλ€λ©΄ κ°μ¬ ν κ²μ΄λ©° μμ μλλ₯Ό λμ΄λ λ° λμμ΄ λ κ²μ λλ€.
mholt
μ 2019λ
01μ 22μΌ
μ¬λΆν νμλ μ¬μ ν λλ¦¬κ³ μ½κ°μ λ©λͺ¨λ¦¬λ₯Ό μ¬μ©νμ¬ 0.93μΌλ‘ μ 리ν μ μμ΅λλ€.
ifelsefi
μ 2019λ
01μ 22μΌ
λκ΅°κ° λ΄κ°ν΄μΌ ν μΌμ μ νν λ§ν΄ μ€ μ μλ€λ©΄ νλ‘νμΌ λ§κ³Ό ν
μ€νΈλ₯Ό ν μν₯μ΄ μμ΅λλ€.
μ½ 500GBμ λ°μ΄ν°, 8GB RAMμ΄μλ VMμμ λ°±μλλ Amazon S3μ
λλ€.
Olen
μ 2019λ
01μ 23μΌ
@Olen λμ μ£Όμ μ κ°μ¬ν©λλ€. μ μ©ν κ²°κ³Όλ₯Ό μ»μΌλ €λ©΄ Goμ λν κ²½νμ΄ νμνλ©° νλ‘μΈμ€λ λ°λ³΅μ μ΄κ³ λ€μ 볡μ‘νλ©° ν μ€νΈλ‘ μ€λͺ νκΈ°κ° μ΄λ ΅μ΅λλ€. μνκΉκ²λ μ°λ¦¬λ ν° μλμλ λΆκ΅¬νκ³ μ¬μ©μλ‘μ λ§μ λμμ΄ λ μμλ λ¨κ³μ μμ§ μμ΅λλ€. :)
fd0
μ 2019λ
01μ 23μΌ
μμλλ©΄ ν₯λ―Έλ‘μ΄ μ μ λ€μκ³Ό κ°μ΅λλ€. μ 리 νλ‘μΈμ€μ μ΄λ λ¨κ³μμ λ©λͺ¨λ¦¬ μ¬μ©λμ΄ μμ λ€μ΄μ€μ§ μλκ°?
fd0
μ 2019λ
01μ 23μΌ
μ μκ²λ restic check μ΄ prune. κ³Ό λμμ μ€νλ λ λ°μνμ΅λλ€.
μ§κΈκΉμ§ prune μ²λΌ 보μ΄μ§λ§ λ©λͺ¨λ¦¬ λ¬Έμ μμ΄ μ¬μ ν μ€ν μ€μ
λλ€.
restic check λ λͺ¨λ prune μμ
μ μ μ€νλ©λκΉ?
ifelsefi
μ 2019λ
01μ 24μΌ
κ·Έλ λ€λ©΄ λͺ¨λ μ 리 μμ μ μ restic κ²μ¬κ° μ€νλ©λκΉ?
μ λλ‘ μ΄ν΄νμ§ λͺ»νμ΅λλ€. restic check λ€μμ restic prune μ
λκΉ?
prune λ check μ μ μ¬ν μμ
μ μννμ§λ§ prune μλ μ¬λ¬ λ¨κ³ (μΈλ±μ€ λΉλ, μ°Έμ‘° λ λ°μ΄ν° μ°ΎκΈ°, νμΌ λ€μ μ°κΈ°, μμ , μΈλ±μ€ λ€μ λΉλ)κ° μμ§λ§ λ κ°μ§ λ€λ₯Έ μμ
μ΄ μμ΅λλ€
fd0
μ 2019λ
01μ 24μΌ
λͺ κ°μ§ λ€λ₯Έ μλ²μμ νμΈνκ³ νμΈμ μ€ννλ κ²μ΄ κ°μΉκ° μμ΅λλ€.
restic check --no-lock
using temporary cache in /tmp/restic-check-cache-300914548
repository a63c6698 opened successfully, password is correct
created new cache in /tmp/restic-check-cache-300914548
load indexes
check all packs
check snapshots, trees and blobs
fatal error: runtime: out of memory
runtime stack:
runtime.throw(0xc0d786, 0x16)
/usr/local/go/src/runtime/panic.go:608 +0x72
runtime.sysMap(0xc0f0000000, 0x4000000, 0x11e41d8)
/usr/local/go/src/runtime/mem_linux.go:156 +0xc7
runtime.(*mheap).sysAlloc(0x11ca7a0, 0x4000000, 0x11ca7b8, 0x7ff95adeb1e0)
/usr/local/go/src/runtime/malloc.go:619 +0x1c7
runtime.(*mheap).grow(0x11ca7a0, 0x1, 0x0)
/usr/local/go/src/runtime/mheap.go:920 +0x42
runtime.(*mheap).allocSpanLocked(0x11ca7a0, 0x1, 0x11e41e8, 0x400)
etc etc
μ΄κ²μ 200GB μ μ₯μ, 4GB RAM λ° νμ¬ 2.3GB μ¬μ 곡κ°μ΄μλ μλ²μ μμ΅λλ€.
benlilley
μ 2019λ
02μ 01μΌ
λλ λν λ©λͺ¨λ¦¬ λ¬Έμ κ° λ°μνμ§λ§ check μμ΅λλ€. μ²μμλ λ¬Έμ μμ΄ prune λ₯Ό μ€ννμ§λ§ (130 κ° μ€λ
μ·, 17 κ° μ€λ
μ·κ³Ό 100GBλ‘ μΆμ λ 200GB 리ν¬μ§ν 리μμ) νμ κ²μ¬μμ λ©λͺ¨λ¦¬κ° λΆμ‘±νμ΅λλ€ (λ λ² μλ, λμΌν κ²°κ³Ό).
using temporary cache in /data/backup/restic/restic-check-cache-567709739
created new cache in /data/backup/restic/restic-check-cache-567709739
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
fatal error: runtime: out of memory
κ·Έλ° λ€μ κ° κ³ λ£¨ν΄μ λν μ μΆμ μ μΈμν©λλ€ (μμ¬ ν μ λλ‘ λ§κ² 보μ΄μ§λ§ grepμ 90 κ°λ₯Ό κ³μ°νμ§λ§ μΌλ°μ μΌλ‘ κ° μ€μ μμ μ μ€λ λμ κ΄λ ¨λ κ³ λ£¨ν΄μ΄ λͺ κ° μλ€κ³ μκ°ν©λλ€).
μ΄κ²μ λ€μ μ€λλ Atom μμ€ν μ μμ΅λλ€ (4GB RAM, μ½ 2.5G 무λ£, μ€μμ λλΆλΆ κ°λ μ°¬ κ² κ°μ΅λλ€). Debian ν μ€νΈμμ restic λ²μ 0.9.4 + ds-1μ μ€ννκ³ μμμ΅λλ€.
μ΄κ²μ΄ μ€μ λ‘ OPμ λμΌν λ¬Έμ μΈμ§ νμ€νμ§ μμ§λ§ μΌλΆ μ 보λ₯Ό μΆκ° ν λ§νΌ μΆ©λΆν κ΄λ ¨λμ΄μλ κ² κ°μ΅λλ€.
matthijskooijman
μ 2019λ
02μ 01μΌ
μ΄κ²μ μ€μ λ‘ κ·μ€ν λ°μ΄ν° ν¬μΈνΈμ
λλ€. μ¦,μ΄ λ¬Έμ λ₯Ό λλ²κ·ΈνκΈ° λ μ¬μ΄ μν©μ΄ μμ΅λλ€ ( check λ 리ν¬μ§ν 리λ₯Ό μμ νμ§ μκΈ° λλ¬Έμ
λλ€). κ°μ¬!
fd0
μ 2019λ
02μ 01μΌ
νμ¬ λ§μ€ν°μμ μΌλ° λ©λͺ¨λ¦¬ μ¬μ©λμ΄ μ΄λ―Έ κ°μνμΌλ©° λ λ§μ μ΅μ νκ° μ§ν μ€μ λλ€.
μ£Όμ μ£Όμ : @ johndoe31415 νμ¬ λ§μ€ν°μμ λ¬Έμ κ° μμ§ μ§ν μ€μΈμ§ νμΈν΄ μ£Όμκ² μ΅λκΉ?
κ·Έλ λ€λ©΄ μν©μ΄ κ°μ λλ©΄ # 2718μ μλν΄ λ³Ό μ μμ΅λκΉ?
κ·Έλ μ§ μλ€λ©΄ IMOμ΄ λ¬Έμ λ μ’
κ²°λμ΄μΌν©λλ€.
aawsome
μ 2020λ
07μ 09μΌ
μλ νμΈμ @aawsome- λΆννλ λλ ν μ μμ΅λλ€. μ΄ μ€λ₯λ₯Ό μ¬ννλ λ° μλ μ¬μ©νλ μ»΄ν¨ν°λ νλμ ν΄μ²΄λμ΄ μ€λͺ λ λ¬Έμ λ₯Ό μ ν λνλ΄μ§ μλ κ΅¬μ‘°κ° λ€λ₯Έ μ μ₯μλ₯Ό μ¬μ©νλ λ€λ₯Έ μ»΄ν¨ν°λ‘ λ§μ΄κ·Έλ μ΄μ λμμ΅λλ€. μ£μ‘ν©λλ€.
johndoe31415
μ 2020λ
07μ 19μΌ
κ΄λ ¨ λ¬Έμ
reallinfo
Β·
4μ½λ©νΈ
RafaelAybar
Β·
3μ½λ©νΈ
rakor
Β·
5μ½λ©νΈ
fbartels
Β·
3μ½λ©νΈ
fd0
Β·
4μ½λ©νΈ
κ°μ₯ μ μ©ν λκΈ
μ΄λ΄ μλ μ°λ, μ¬κΈ°μ μΆκ° ν κΈ°μ μ΄ μμ§λ§ λ§μ§λ§ μ½λ©νΈ νμλ λ€μκ³Ό κ°μ΄ λ§ν νμκ° μμ΅λλ€. μλ΅μ΄ λ§€μ° λΉ λ¦ λλ€. μ΄μ μ μ¬κΈ°μμ κ°μ§ λ²κ·Έλ₯Όλ³΄κ³ νλλ° (μλ₯Ό λ€μ΄ HW λ¬Έμ μ λΏλ¦¬λ₯Όλκ³ μλ κ²μΌλ‘ νλͺ λ¨) λΉμ μ νμ λ§€μ° μ§μν΄ μμ΅λλ€. λͺ¨λ μ€ν μμ€ νλ‘μ νΈλ λΉμ μ νμμλ‘ μΌλ κ²μ΄ νμ΄μ΄λΌκ³ μκ°ν μ μμΌλ©°, λΉμ μ΄ ν΄μμ μν΄ κ·Έ μ΄μμΌλ‘ λμκ°κ³ μμμ λΆλͺ ν 보μ¬μ€λλ€. κ·Έκ²μ λλΌμ΄ λꡬμ΄λ©° μ°λ¦¬ λͺ¨λλ κ·Έκ²μ μκ³ μμ΅λλ€. μ΄μ κΈ°λΆμ΄ μ’μ§ μλ€κ³ λ§νμ λ λ―Ώκ³ μμ§λ§ λ°μΈμλ νμ€ν λνλμ§ μμμΌλ μμ¬νμΈμ. κ³μν΄μ λ©μ§ μμ μνκ³ Resticμ λμμμ΄ κ°μ ν΄μ£Όμ μ λ€μ ν λ² κ°μ¬λ립λλ€. 건배, μνλ€μ€