According to this, you can use the environment variable RESTIC_PASSWORD to specify the password.

As @pvgoran already said, you can use the environment variable RESTIC_PASSWORD. This is documented in the manual here http://restic.readthedocs.io/en/latest/Manual/#initialize-a-repository. If you have an idea on how to document this better, please create a pull-request.

There is also issue #278, which is about reading the password from a file.

I'm going to close this issue as there is nothing we need to do right now. If you disagree, please leave a comment.

Ouch, and I thought I had read that section through....

Maybe give that paragraph it's own heading or subheading? The first four paragraphs in that section cover everything about initializing a repository. Automating backups doesn't really fit under that.

Thanks @pvgoran for the fast answer, btw. :smile:

Just a quick follow-up question on this. If a backup doesn't finish by the time cron executes restic again, what will happen?

In this case, restic will start a second (parallel) backup, which uses data already uploaded by the first (still running) backup. I guess that both backups will finish almost at the same time. There will be some data duplication, which is cleaned up when the prune command is run the next time. It won't cause any data corruption or data loss, though. The repository format is designed in a way which allows parallel upload of data.

This is an interesting case I haven't thought about yet. Do you think we need code to check whether the same backup is already running on the same host, and exit in this case?

@fd0 Possibly, yes. For the same backup only. Or, a clean way to script automated backups: some instructions will do, maybe "restic backup" then "restic unlock" then "restic prune" for example. But not having to worry about those extra commands would be nice.

restic backup followed by restic forget and restic prune is the usual workflow, and that cleans it up. I'll add another issue anyway, so we can track this idea.

Cool, thank you! For now I'll use that flow.

Another option here that could help is a timeout parameter. If you know your cron job schedules a backup every X hours, you could pass a --timeout to restic so it will end before the next one starts. It would also be handy for other things too. (this may already exist, I'm new to restic)

Detecting that restic is already running sounds tricky and I'm not how that would be done accurately and cleanly from within restic itself. Especially since you could have different restic backups running at the same time which shouldn't count.

Maybe a start up script that is similar to many of the other linux start up scripts that takes the PID of restic when it starts and saves it to a tmp file then deletes it when restic ends. Each time the script is run it would check for the file. You'd need a different tmp file for each unique scheduled backup.

Unfortunately, any means of detecting a running restic instance would fail on ALL backends (SFTP, REST, S3...) except the local one.

@zcalusic we could use the information stored in the lock files to do that: https://github.com/restic/restic/blob/master/src/restic/lock.go#L27-L33, maybe add the list of files to backup or the exact command line arguments or something similar.

I still don't see how would restic on machine A, finding a lock on backup server machine B, distinguish between a) currently running restic session on machine C and b) stale lock left after the restic on machine C crashed?

Or are we starting talking about RPC mechanisms here? Or even better, distributed lock managers? 😄

@zcalusic I'm talking about #711: detecting when a second backup with the same directories on the same machine is started. That should be possible.

@bwmarrin I don't understand what you're suggesting:

If you know your cron job schedules a backup every X hours, you could pass a --timeout to restic so it will end before the next one starts.

Either a backup is run until the end, or it is cancelled/terminated. I don't think that it's possible to somehow time a backup that it at most takes whatever duration is given in a hypothetical --timeout parameter. How could that work?

Could you please describe the semantics of such a parameter? Thanks!

@fd0 I'm saying if the backup doesn't finish within the --timeout value, it is terminated. I realize this means it would be an incomplete or unfinished backup. However, since restic's incremental design it's not a big deal really. The next time the backup is called it will start off where it left off.

This means if you have a cron job scheduled to run hourly, and you pass a --timeout 50m parameter to restic. It will abort/terminate the backup if it takes more than 50 minutes. In that case, 10 minutes later the cron job will start it off again and it would resume where it left off. This would prevent multiple instances of the same backup from running concurrently.

Thanks for the explanation. I think that this is not a good idea. What happens if the backup location is slow without you noticing it (somebody in your network forgot a bittorrent client, which maxes out you upstream bandwidth), so the backup doesn't finish, is aborted, restarts again, is aborted again and so on. Then you never end up with a full working backup.

Or consider a directory tree for which data is added constantly. A single run of restic will finish eventually (it is designed that way), but restarting restic may never finish when too much new data is added between the runs.

Besides, you can easily implement this behavior by using the standard utility timeout (from coreutils), by running timeout 40m restic backup [...]. So I don't think adding this option to restic is a good idea.

I realize I'm late to the party but couldn't there be a lock file that makes the second instance wait for the first to finish before it starts?

@Karl-Gustav , may be you can achieve that with a bit of scripting. https://stackoverflow.com/a/1985512/244009

That was a bit more advanced than my locks:-) I just use if file {wait 5sec and check again}

I may be a bit late to the discussion but I created some systemd units you can find here.
These are my restic config files, they may be useful for someone.

Hi there, I'm reading on how to use restic to schedule my backups. For now my idea is to use anacron to schedule for example a 2 days per week backup to Backblaze. The point is, if my backup is scheduled with anacron e.g. Tuesday and Friday at 12PM and my laptop is turned off Tuesday and don't start it again until Friday at say 11:59AM, what will happen? AFAIK (and if I'm not wrong) anacron should start the missed job of Tuesday; and a minute later (while first restic instance is running), will run the second backup, producing 2 concurrent backups for the same directory?

Or is it any kind of /tmp lockfile to prevent the second instance from running?
How should I manage it to correctly schedule my backup? Thx :)

@gerardbosch Hi! This question is a better fit for the forum, please consider that next time :)

One way to handle it is to write a script that does the running of restic for you, and as part of doing that creates a runfile (e.g. /var/run/restic.pid containing the PID of the restic process`), which it can then check to determine whether or not restic is already working.

It will not harm in any way if you were to run two concurrent backups, but it's of course rather pointless if they cover more or less the same files and point in time.

Whether anacron tries to catch up on missed backup runs I don't know, it should say in its documentation I suppose. If you are on macOS and use launchd for scheduling, you have the option to do so or not, it's up to you.

FYI for more people landing here from Google searches:

Here is how I do backup on a schedule using systemd services and times, instead of cron jobs. Also featuring email notifications when backup fails.

https://github.com/erikw/restic-systemd-automatic-backup

@erikw Fantastic schedule script, congrats!
Some questions:

• What is the main advantage of using systemd timers over cron or anacron?
• Can systemd scripts/setup be installed in homedir instead of /etc?
• Can anacron tab reside somewhere in $HOME?

I am planning to backup the homedir of my laptop, so backing up the same scheduler scripts will provide in case of disaster recovery, an out-of-the-box scheduled backup system (i.e. restoring the whole backup after the disaster, will provide a new user account already configured to do backup on the same previous schedule).

@gerardbosch

• If you have a systemd system, it's quite nice to be able to use the default tools, no need to install a cron daemon. You get great control over the status of failed jobs, and you can see when a job will be executed the next time for example. See the arch wiki for a short intro. Just play around with it yourself is the most fun way to learn I would say though.

• Yes, I run several systemd timers for my local user. Check my dotfiles for the timers and corresponding services I currently use. The key is to use --user to control the user's timers instead of system timers:

$ systemctl --user list-timers

• It seems like they are in /var/spool usually. If the can be moved to ~/ I do't know. https://askubuntu.com/questions/216692/where-is-the-user-crontab-stored

As it hasn't been mentioned here yet, Backupninja is a great way to handle scheduling. Restic support is being added in a merge request; it just hasn't been committed yet. All of the basic functionality should be there though.

@colans Backupninja is awesome, can't wait to be able to use it with restic! Thank you for this work.

restic backup followed by restic forget and restic prune is the usual workflow, and that cleans it up. I'll add another issue anyway, so we can track this idea.

If I'm setting up a daily scheduled task/cron job without the expectation of possible parallel jobs should the script still do restic backup -> restic forget -> restic prune ? Seems like it's adding more overhead if we're only executing one instance at a time

Can we reopen this as a documentation task? I think we should add a "Scheduling" section to the manual to increase clarity around this topic. We can say something like:

Scheduling is outside the scope of Restic. However, there are external tools that can be used for this purpose.

• Backupninja along with the Restic support branch
• Automatic restic backups using systemd services and timers
• Restic backup configs

Thoughts?

Another suggestions to re-open. If restic doesn't support backup scheduling/monitoring, the docs can explain this and link to something that does, since that's the primary way backup tools are used.

@SigmaX how to schedule depends entirely on what OS and software you have at your disposal. I think suggestions like that are outside of the documentation for restic concern, but it could indeed be a helpful article in someone's blog or even in the forum's Recipes section. It could also be a candidate for the Examples section in the restic doc website at https://restic.readthedocs.io/en/latest/080_examples.html , but it would have to be written in such a way that it requires a minimum of maintenance, since we don't want to maintain a set of detailed instructions for how to schedule this on several different platforms (as it's a pretty complex topic). All that said, there's already several articles and examples about this (e.g. with cron and systemd) on the net, if one does search for it. I'm not entirely sure there's a need to have that in the docs website.

I'm not entirely sure there's a need to have that in the docs website.

I can see how it would be a matter of opinion, especially since restic is cross-platform. For me, it was surprising. Having a backup tool without prominent docs on how to schedule it sounds seems to me like selling a car with no wheels. I spent quite a while searching the docs, assuming I was missing something. I would never launch a backup manually, but that's all the docs describe.

At the very least, a docs section could save users time: by indicating they need to search someplace else/find a different tool to setup routine backups with restic.

Having a backup tool without prominent docs on how to schedule it sounds seems to me like selling a car with no wheels

Not really. What we "sell" you is a program that you tell what to back up, and it backs it up. How often you want to do that is a separate concern :) But I digress.

I'd expect most users if not finding a certain topic in the docs to simply DDG/Google for it, and then find the answers within a minute or two. But that doesn't mean we shouldn't add a pointer of some sort to the documentation, even if we don't detail specifics on how to set up various scheduling software.

What we "sell" you is a program that you tell what to back up, and it backs it up. How often you want to do that is a separate concern :)

And it's perfectly legitimate to say "this is a DIY kit---go find your own wheels!"

I'd expect most users if not finding a certain topic in the docs to simply DDG/Google for it, and then find the answers within a minute or two.

Actually, Googling around for "restic backup schedule" brought me here first match :smile:

2122 is related as it provides some sample systemd timers and other discussion regarding scheduling.

I think the "least maintenance" approach here seems to make sense...maybe include hints about logging output and preventing multiple runs at the same time?

maybe include hints about logging output and preventing multiple runs at the same time?

I like that idea. I'll make a draft later this week, most likely for a little "pointer" section to place under the backing up section in the documentation.