Certbot: Create an official PPA for Ubuntu letsencrypt packages

:+1:

:+1:

:thumbsup:

:+1:

pretty please?

Yes please!

Yes, please.

+1

+1

+1

There has been a ton of effort spent on letsencrypt-auto and none here. What's going on? I find this situation absolutely awkward, as letsencrypt-auto was meant to be temporary solution until we get packaged. @hlieberman and @fmarier invested a lot of time into the debs and we're now wasting time on some home-cooked "auto-magical" script that tries to basically replace all the packaging solutions (and, despite great efforts, fails miserably at it). There has been numerous complaints about how ridiculously complicated the client is and I thought it's obvious that things should be cut down rather than infinitely extended.

@pde - please clarify above and provide ETA on the PPA.

For recent versions of Ubuntu, the package in Debian unstable should work fine. Once we have backported the package to Debian jessie, making it work in Ubuntu 14.04 should be easier.

This will of course require packaging a number of dependent libraries as well.

+1 for a PPA to provide a more traditional/standardized way of installing unpackaged software.

The update we got last week from Harlan was that various debian maintainers had done a great job of backporting all of our python dependencies, but the outstanding blocker was sphinx-doc for the doc packages, and that it was going to be a non-trivial undertaking. Not sure if PPA's are sigificantly different or easier.

Can't the Sphinx doc building be temporarily disabled for PPA packages? As a user, I'm perfectly fine with having to go online to check documentation (in fact I prefer that instead of reading local HTML files in /usr/share/doc/).

if it's only the sphinx-doc building of the doc then where is the problem ?

• provide 2 packages letsencrypt and letsencrypt-doc
• provide a man letsencrypt.1 for letsencrypt_x.y.deb
• pre-build the doc as HTML
  and package into letsencrypt-doc_x.y.deb

reason:
automatically build the doc is nice but it is optional
removing the sphinx-doc to build the doc does not prevent letsencrypt
to be build, installed and to work which is what everyone want

The proposal by @zwetan to separate the debs for the application and the documentation is already well proven by many other applications, from GIMP to LibreOffice. It benefits the developers, who don't have to rebuild the documentation every time there is a security upgrade, and it gives them have the freedom to update the documentation during application quiet-times. I commend the idea.

There is already letsencrypt in xenial (16.04 upcoming): http://packages.ubuntu.com/xenial/letsencrypt

The docs seem to be optional deps there.

my point exactly, some servers in prod only rolls with LTS -> security patches -> next LTS
having to wait for 16.04 LTS, when 14.04 LTS could already have the deb package
only because the doc build is problematic? com'on :)

Most recent update is that a sphinx backport has unblocked packaging for Debian 8, and a PPA will be on the way shortly afterwards.

@zwetan :+1: I can't use letsencrypt since I'm using 14.04 and there is no official repo for 14.04

:+1:

:+1:

+1 I will never ever in my life run letsencrypt-auto again....

I've created a PPA for Ubuntu Wily. Use at your own risk; it's largely untested at the moment.

https://launchpad.net/~letsencrypt/+archive/ubuntu/letsencrypt

@hlieberman could that package run on 14.0.4 alias TrustyThar or can that be denied?

just a headsup, the PPA doesnt create the dirs for the configs in /etc and /opt

And now letsencrypt-auto is failing with dependency issues (urllib3, acme). This should be a simple breezy process, given the problem it solves. You don't want to spend hours wrangling with dependencies et-al, makes it feel brittle.

(PS: PPA doesn't work with 14.04)

from my latest tests on Ubuntu 14.04.5 LTS
it is simpler to ignore the PPA

just do a "manual" install

git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt
/opt/letsencrypt/letsencrypt-auto --help

(the second line is to force the dependencies to install)
this have worked well for me on a dozen servers

One additional suggestion: If you're uncomfortable with the relatively large number of dependencies, just create a small lxc instance (approximately 350MB in size at the moment) and follow the aforementioned manual installation:
lxc-create -n letsencrypt -t ubuntu

In order to use the certificates on the host and allow for the automated tests, just add two bind mounts to your config as follows (the idea is that an external web server--either on the host or living in another container--will take care of this):
lxc.mount.entry = /var/www/letsencrypt var/www/letsencrypt none bind 0 0
lxc.mount.entry = /etc/letsencrypt etc/letsencrypt none bind 0 0

(Of course, you need to trigger synchronised renewals/web server service restarts from the outside of the _letsencrypt_ container, but it's pretty convenient...)

personally I don't mind the dependencies

what I really need is automation so I can add a provisioning script
to know if I want to add Let's Encrypt support on a server
it will work, been tested and take about 10mn to setup

A large number dependencies is a problem for me as I'm limited on resources.

If there is interest in containers, I can throw together a Docker container that would do the job very very quickly. There seems to be some work toward this on the Docker hub, but it appears abandoned/old.

This would mean it would be as simple as docker run -v/etc/letsencrypt:/etc/letsencrypt -p80:80 -p443:443 letsencrypt <FQDN> to generate certs.

EDIT: Here you are everyone. https://hub.docker.com/r/samyaple/certbot/
Its going to pull about 250MB down including the base image. Im working on reducing that footprint now. This was just a rough implementation in a few minutes. The builds are automated so you can see exactly what is building, no funny business.

I am more than happy to support this under the letsencrypt banner if thats desired.

:+1:

For those interested, I've got a little Ansible role hooked up to automate cert installation for my Ubuntu sites. It's very limited at the moment, just for the functionality I need, PR's welcome if someone wants to use it too.

The role uses the official client from git and does cert installation/renewal in certonly mode.

https://galaxy.ansible.com/jaywink/letsencrypt/

Re-assigning harlan after the repo move removed him. This is still important, but will now be a lower priority than renaming letsencrypt -> certbot in the places where we are already packaged.

👍

+1

+1

+1

+1

Would be nice to have the PPA updated with the latest version. It's lagging behind - still on 0.4.1 it seems: https://launchpad.net/ubuntu/xenial/+package/letsencrypt

@larssn That's not a PPA - that's the package in the Ubuntu repositories. Two considerations: First, there's an SRU process [1] to go through to get package updates into an Ubuntu release. Second, the package is in 'universe' so isn't officially supported by Ubuntu/Canonical.

[1] https://wiki.ubuntu.com/StableReleaseUpdates

(edit: add URL)

Got it.

In any case; still wishing for a update. :)

Folks, this is not an "official" repository, but I do home I have some credibility as a Debian Developer and Ubuntu Member (and my PHP PPA is used by thousands), so I created PPA for certbot 0.8.x. It has been built from Xenial only at the moment, but I might look at backporting more of the stack to Trusty if I find some time.

The PPA is here: https://launchpad.net/~ondrej/+archive/ubuntu/letsencrypt

just tested it on ubuntu 16.04 works great, already using your stuff for php, apache, and mysql

@oerdnj You just saved my day once again. Thank you so much for your work!

@oerdnj do you think your PPA is reliable enough that we should consider pointing at it in our instruction generator?

wrt Trusty: I believe harlan became a little afraid of the dependency set that needed to be built for Trusty to get a reliable PPA going. But @jonathonf seems to have made some progress on that front, so it may be a tractable problem.

@pde It's stable as it gets. I get busy from time to time and somebody might ping me or fill an issue at my gh tracker here: https://github.com/oerdnj/deb.sury.org/issues

But I tend to release and fix critical stuff within hours-days and less critical things within weeks.

I'll be attending some conferences during October and I usually fill the idle time with packaging, so I might be able to finish the Trusty packaging. Of course, if I there's already existing work on that front I would be glad to take hints/patches/help.

As an alternative I would be happy to create a "official" packaging group at launchpad and invite more people into and add somebody from LE as an admin.

The Debian packaging team did a great job, so for Xenial it was mostly just backporting/repacking existing package. For Trusty it might require some more modifications, but I haven't looked at it yet.

For the record, I'm using @jonathonf 's PPA on trusty and everything works nicely, just needed a couple of updated python-* packages.

@pde There's https://launchpad.net/~letsencrypt and it seems abandoned. Maybe LE could ask Canonical to grant somebody from LE administrator access to this launchpad team and then add me and @jonathonf, and we'll create an official repository for certbot under this namespace? Seems like best approach to combine efforts.

@oerdnj A certbot repo under letsencrypt would be nice. Personally I always go looking for a letsencrypt package, and would get confused if I couldn't find ti by searching for letsencrypt.

I have granted @oerdnj ownership of the ~letsencrypt organization.

If someone from Letsencrypt wants to be added to the org and made owner, please ping me.

I have now copied my certbot packages to ppa:letsencrypt/letsencrypt? I'll look at @jonathonf trusty repository to check what packages needs to be backported for Ubuntu Trusty.

@floe Is this https://launchpad.net/~jonathonf/+archive/ubuntu/letsencrypt the correct Trusty repository you are using?

Yes, that's the right one. From my sources.list:

deb http://ppa.launchpad.net/jonathonf/letsencrypt/ubuntu trusty main

If we want to make an official Certbot PPA, I think we should set it up under https://launchpad.net/~certbot instead of ~letsencrypt due to Certbot no longer being a Let's Encrypt project (source1, source2). ~certbot is owned by @hlieberman who is our Debian package maintainer.

Would ppa:letsencrypt/certbot be a better match? I am fine with either or moving to @hlieberman ~certbot. I am even fine with not doing any work at all if not needed. Let's just agree on something and move forward, so there's an official repository.

I agree having an official Certbot PPA would be a huge benefit for users. How exactly we want to do this is something the core certbot devs should talk about though. We have a meeting every Wednesday. I'll bring this up then and get back to you guys after the meeting.

@bmw @oerdnj @ArchimedesPi I think the correct thing to do is probably to wind down ppa:letsencrypt, ideally by having launchpad redirect it to certbot, but by other means if necessary?

@oerdnj Nice to see you here again :) Actually i just stumbled over this thread in the course of backporting to trusty. I already backported the Debian packages for xenial in July but only those two packages we run now in production (https://launchpad.net/~trio-interactive/+archive/ubuntu/stable/+sourcepub/6825687/+listing-archive-extra & https://launchpad.net/~trio-interactive/+archive/ubuntu/stable/+sourcepub/6825689/+listing-archive-extra).

@oerdnj @pde @bmw Beside the PPA naming issue who did a clean backport from Debian to trusty yet? And where is the best place to synchronize our efforts on that as our PHP5 apps really want LE as well? ;)

Debian have their 0.8.1 packages backported to jessie-backports so there's a very clean backport from there to xenial. Lowering deps in the Debian packages for pyopenssl and python-cryptography will mean both python-acme and python-certbot will backport without anything else needed. If you're doing this, though, you might as well just backport the yakkety package.

Trusty still needs a patch for the python-acme package - everything else builds OK against (mostly) the jessie-backport versions. There is still a large number of updated dependency library packages which could have who-knows what knock-on effects.

@-all using ppa:jonathonf/letsencrypt: this is going to disappear soon in favour of the cleaner backport in ppa:jonathonf/certbot based on the jessie-backport Debian packages. I also might be re-jigging that newer PPA slightly to clean the xenial backport as above.

Edit: my certbot PPA brings in a few extra Python libraries from Jessie - this makes backporting direct from Sid much easier at the cost of deviating from the Trusty Python stack. This isn't a great concern if the libraries are in universe anyway.

@jonathonf Good to hear :) Can you give more hints on the patch for "python-acme" on trusty? For the knock-on effects i will have a stabilization phase while bringing it to production and of course we always need to break the development servers ;) How can i help you with the trusty packages or who is working on it?

@jonathonf When testing your trusty packages from ppa:jonathonf/certbot i get:
pkg_resources.DistributionNotFound: The 'python2-pythondialog>=3.2.2rc1' distribution was not found and is required by certbot
But with a quick no source change backport for that the dependency the client worked now: https://launchpad.net/~trio-interactive/+archive/ubuntu/unstable/+sourcepub/7045266/+listing-archive-extra

@all: For clean backporting the python dependencies are a big hassle, as we need to upgrade several Python packages that come with the distro release. That can break other Python tools with the same dependencies and also makes the maintenance burden a little security nightmare, as the PPA would need to ship every security updates for all the backported dependencies. This is nontrivial problem for shipping trusty packages! Even if you construct the backporting pipeline from xenial or jessie we always would need to stand in sync with the security teams ... while we mess with the Python stack in the release ;) What do you think?

Hrm. Could you visited the required versions of the dependencies locally within the certbot package? 

@bhat3: if you look in the debian.tar.gz for python-acme you'll find the quilt patch. I'm pretty sure I got it from earlier in this thread, but I didn't see it after a quick scan just now.

@jonathonf I try to have a look at the packaging today, but can't promise having time yet. Beside your patch, what would you see as minimal dependencies for trusty, so we don't need the mess with trusty's Python stack so much?
@oerdnj Knowing that you're quite in sync with the security people, do you got interest in providing trusty packages through the PPA?

It depends how much packaging you want to do. The deps in my original letsencrypt PPA are about as minimal as I could get but make further backports from Debian more difficult. The backported deps in my certbot PPA are more numerous/wide-ranging but make direct backports much easier.

If Python libraries are in universe we don't need to worry as much - nothing in the core Ubuntu stack relies on anything in universe (and packages in universe aren't updated as a matter of course anyway, even for security issues).

@jonathonf Good thought about the "universe" corner case in Ubuntu ;) Actually i couldn't continue my efforts yesterday and no chance for this week anymore :( But i hope to do so on monday or after the 7th of November. Till then :)

We're almost a year from when this issue was first created, and still haven't agreed on anything?

Can we get some momentum going here? One way or the other.

Is https://launchpad.net/~jonathonf/+archive/ubuntu/certbot as official as we're gonna get?

@jonathonf and all - I teamed up with @hlieberman in ppa:certbot/certbot and I think I have most of the build deps for trusty, xenial and yakkety. The only thing missing is recent sphinx and I hope there will be some workaround for that.

@oerdnj Is it ready for use?
I get a "Error: signing key fingerprint does not exist" when trying to add it.

Sorry not yet, but it will be finished in ~week. I am leaving for IETF tomorrow and I will have some free time there to finish it.

If it's just as a builddep I have a Sphinx backport PPA that can be added
as a PPA dependency. You could use that, copy the packages to a "support"
PPA, or backport the same set of packages.

J

On 9 Nov 2016 12:39, "Ondřej Surý" [email protected] wrote:

@jonathonf https://github.com/jonathonf and all - I teamed up with
@hlieberman https://github.com/hlieberman in ppa:certbot/certbot and I
think I have most of the build deps for trusty, xenial and yakkety. The
only thing missing is recent sphinx and I hope there will be some
workaround for that.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/certbot/certbot/issues/1706#issuecomment-259405442,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAiJCY1dmSHhspzOgNcT8tQu4XigyNJdks5q8b7ygaJpZM4Guho5
.

@oerdnj @jonathonf @hlieberman
Thank you for your work on this.

I think we can all wait another week.

@oerdnj @jonathonf Is there something i can help you on the trusty backports within my working hours?

@bhat3 It would be awesome, if you can do an extensive testing of what is now in ppa:certbot/certbot. I had to tighten some (Build-)Depends before it stopped throwing runtime errors, but all looks good now.

JFTR the ppa:certbot/certbot now contains packages that could be used with some caution. Please report any breakages here for the time being.

@oerdnj Okay will do that and replace my own packaging on our development servers with ppa:certbot/certbot. But beside standard Debian/Ubuntu tools we don't use Python but your PHP there, so i won't be able to stress out those breakages ;)

On Ubuntu-16.04-xeinal, I was stuck on 0.4.1 which was failing to renew with following error:

WARNING:letsencrypt.cli:Attempting to renew cert from /etc/letsencrypt/renewal/gableroux.com.conf produced an unexpected error: 'server'. Skipping.

After upgrading, renews are now working again, thanks! :D

python -mplatform

Linux-3.11.0-12-generic-x86_64-with-Ubuntu-16.04-xenial

letsencrypt --version

letsencrypt 0.4.1

sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install --upgrade letsencrypt
letsencrypt --version

letsencrypt 0.9.3

letsencrypt renew --agree-tos 

Congratulations, all renewals succeeded.

:heart:

@oerdnj So far it looks good on the dev server i used @jonathonf's trusty packages plus one own backport:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  certbot letsencrypt python-acme python-certbot python-cffi-backend
  python-configargparse python-cryptography python-dialog python-dnspython
  python-idna python-ipaddress python-ndg-httpsclient python-openssl
  python-parsedatetime python-pkg-resources python-pyasn1 python-requests
  python-rfc3339 python-setuptools python-six python-urllib3
21 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,410 kB of archives.
After this operation, 398 kB of additional disk space will be used.
Do you want to continue? [Y/n]

I successfully renewed a precreated LE cert with 99 subdomains and that works well. More testing will follow tomorrow ...

As said i am a bit skeptical with the full blown dependencies backporting, as LE was already working on the same system without upgrading the whole dependency chain ;) Otherwise i am happy to finally "mainline" our systems :)

BTW if you rename the certbot/certbot PPA to the default name (certbot/ppa), users could then use the shortened version of the name, i.e.:

sudo add-apt-repository ppa:certbot

@oerdnj Next test for trusty was an automated renewal via cron and shell, but actually i found some problems i didn't encounter before:

/usr/bin/letsencrypt certonly --force-renewal --webroot -w /var/www/letsencrypt -d DOMAIN.TLD
An unexpected error occurred:
Bug in pythondialog: expected an empty output from u'infobox', but got: u'Error opening terminal: unknown.\n'Please see the logfile 'certbot.log' for more details.

While i log every output of that renewal script and got mail notification in it i don't find the certbot.log and can't really debug it. Do you have any hints on that?

@bhat3 you should be using --noninteractive or --quiet (which implies --noninteractive) if you're invoking certbot from a cron script.

(I would look for certbot.log in /var/log/, maybe in /var/log/letsencrypt/? TBH I only have letsencrypt.log in there; certbot.log looks like something new.)

doesnt certbot renew attempt to renew all? i run that and it does that

@bhat3 is that a default cronjob as supplied by the package?

@oerdnj Nope that's an own one and the --noninteractive switch was not needed before with the package from @jonathonf . The reason for this own cronjob is that we need to restart Nginx if the renewal is sucessful and such things must happen in a fixed time frame. The last requirement can be done in /etc/cron.d/certbot but not sure about the interaction with nginx -t && systemctl restart nginx.

@chrismccoy And how do you restart your webserver after renewal?

@mgedmin Thx, it worked with the switch. But i am still curious if the certbot.log is just a branding issue and /var/log/letsencrypt/letsencrypt.log is meant by that, which seemed to be not created without the switch.

/usr/bin/letsencrypt certonly --noninteractive --force-renewal --webroot -w /var/www/letsencrypt -d DOMAIN.TLD
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Look at the new crop job, it has hooks directory

@oerdnj You mean that i can do a hook after successful renewals only that then takes care about Nginx?

The certbot renew call now includes: /usr/bin/certbot -q renew --pre-hook '/bin/run-parts /etc/letsencrypt/pre-hook.d/' --post-hook '/bin/run-parts /etc/letsencrypt/post-hook.d/' --renew-hook '/bin/run-parts /etc/letsencrypt/renew-hook.d/'

And I've just sent @hlieberman the patch for Debian bug#838548

# cat /etc/letsencrypt/post-renewal.d/nginx 
#!/bin/sh
set -e
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
nginx -t -q && nginx -s reload
exit 0

@oerdnj, we asked @hlieberman specifically not to add directories like that in Debian in favor of us developing a solution upstream from which all distros could benefit. What do you think about removing these directories and waiting for a solution from us?

Does the packaged version of certbot in the PPA have the apache configuration plugin? I notice it doesn't have an equivalent package to python-letsencrypt-apache as Universe does.

I've got Certbot/Letsencrypt working with the version from Universe, but when I upgrade using the PPA I get errors surrounding the apache plugin. e.g.

$ sudo certbot renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

-------------------------------------------------------------------------------
Processing /etc/letsencrypt/renewal/XXXXXX.conf
-------------------------------------------------------------------------------
Renewal configuration file /etc/letsencrypt/renewal/XXXXXX.conf produced an unexpected error: 'Namespace' object has no attribute 'apache_enmod'. Skipping.
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

No renewals were attempted.

Additionally, the following renewal configuration files were invalid: 
  /etc/letsencrypt/renewal/XXXXXXXXX.conf (parsefail)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
0 renew failure(s), 1 parse failure(s)

and

$ sudo certbot --apache
The requested apache plugin does not appear to be installed

I've tried deleting the configuration file (renamed /etc/letscrypt) but no improvement. Fortunately it's simple to downgrade to 0.4.1.

@bmw Too be honest i actually welcome the Debian integration here and want to consolidate to it from our own cron'ed BASH script. What do you want to do as an upstream and distro agnostic solution ... and how long does it take to deliver?

There are so horrible solutions in the open regarding the renewal and the needed interaction with the service using the certs, that i really favor that @oerdnj et al delivers a sane solution on which we can standardize on Debian distros now :+1:

@bmw I wholeheartedly disagree. The Debian stretch is entering a freeze period and if you don't want everybody to cook-up their own script for next 2+ years, the Debian integrations is the best thing you can get right now, as it is under control and when (and if) you provide an official run-parts equivalent, it would be fairly easy to migrate.

Perhaps I am in this field for too long, but waiting for a perfect solution is usually only way to a disaster, because people will start to be creative.

I would really appreciate if @hlieberman would integrated run-parts solution into main Debian package as it delivers a working solution for Debian stretch right now and well integrates in a way how Debian internals works.

Hi @oerdnj, two points:

1. Since we control upstream and have a prettty good pipeline from our releases into unstable and testing, we feel there's time to land a patch in upstream Certbot that works both for Debian and most/all other distros, and get it into Debian. That can very easily happen before the February freeze if you like; just send us a PR or encourage @hlieberman to do so :)
2. We've been having a lot of conversations about how to cope with the stretch freeze. We don't think Certbot is mature enough to be pinned at a given version for the 5+ years that people seem to try to enforce for Debian stable releases. We're still trying to work out what to do about that. On Ubuntu we're going through the SRU process to get newer Certbot versions into Xenial LTS, but we're not so optimistic that Debian will take them for stretch. Our current plan is to ask the debian-devel list to choose between (1) committing to accept a well-tested Certbot release into Stretch updates every ~6 months; (2) asking Debian not to ship Certbot in stable releases, and rely instead on giving our users backport instructions which is what we're doing with Jessie right now, and we feel it's fairly sane.
3. You're welcome to join our weekly Certbot dev calls to talk these issues over with us if you'd like to! I think Brad sent you an invite, but if not, lmk and we'll fix that.

@pde ok, make sense. I'll try to reserve some free time to cook up something similar to run-parts, but in native python code.

Yep, I got the invite (I'll check with you on Freenode for exact time). I just can't promise anything beyond this particular issue, as there's only a finite amount of time in day/week/month/year :)

Edited: Maybe it's the IETF experience within me, but I actually prefer a running code to talking :). If I only have more time to produce more running code.

Would be nice if the apache plugin could be added as well :)

Hi there, just wanted to check in as to when we might be able to see a PPA? I'm on the verge of installing certbot-auto in the near term to get us through our first upcoming renewal since first going with Lets Encrypt. Cheers and thanks for the work you guys are doing on getting this together.

@jesseiqmi You can already give it a ride:

sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install --upgrade letsencrypt
letsencrypt --version

I also use it on production, but still with no proper renewal integration, but my production tested home brewn script. But i would suggest at least that you always test new versions on development and staging systems before you touch production. You can stay lean with KISS by just using apt-mark hold PKGNAME, disable the PPA on production until next testing is passed on staging or use own PPAs to forward the packages between your systems in the QA chain.

The PPA doesn't seem to include any of the plugins such as certbot-apache. What is the correct procedure to install these?

@oerdnj any updates, or things you'd like help with?

@pde If you have any idea how to fix documentation with sphinx-build 1.2.2, it might help: https://launchpadlibrarian.net/301768953/buildlog_ubuntu-trusty-i386.python-certbot-nginx_0.9.3-1+certbot~trusty+2_BUILDING.txt.gz

That seems to be a code bug in certbot_nginx/nginxparser.py, nothing to do with Sphinx (other than that Sphinx tries to import the module for autodoc purposes, which fails because of the bug).

AFAICS pyparsing wants you to use ... + restOfLine instead of ... + restOfLine().

@mgedmin Thanks, I'll try to patch that out in the package and submitted as https://github.com/certbot/certbot/pull/3989

Ok, so the PPA now contains both apache and nginx plugins. Thanks for the poke.

Thanks for all your work here @oerdnj! Do you think the PPA is at a point we can start suggesting it for the average Certbot user?

@bmw I believe so. Well, apart from the issue that I still have the rundirs hack in the repository, I don't want to let go - but I would be happy to prepare a stable migration path once the feature hits upstream certbot.

@oerdnj Yeah! Would love to finally consolidated on the rundirs "hack" without need to fork the PPA later :+1:
@bmw I can understand your need for upstream Python approach, but it's about Debian packages here right?! ;)

Sorry for posting my question here as there are so many subscribing to this issue, but I couldn't find a good answer anywhere else so here it goes:

Ubuntu 16.04.2
Installing packages with apt install letsencrypt:
letsencrypt --version gives me 0.4.1 but when installing from git I get 0.12.0 - so when will Ubuntu update the package, or is the 0.4.1 just the Ubuntu version for 0.12.0?

Ubuntu never updates packages in released versions (except when they need to fix serious bugs -- see their Stable Release Updates policy).

Ubuntu will update the package for 16.04 as and when it goes through the SRU process. The Ubuntu package is not linked to the git repo.

You're better off using the PPA, then an apt-get install letsencrypt will install the latest packaged version.

Unless you want the latest git version, in which case do what you did.

@jonathonf I installed the PPA in this issue, but no difference. :/

I don't think you did. ;)

Try this one: https://launchpad.net/~certbot/+archive/ubuntu/certbot

@jonathonf Awesome! Missed that.

So, will this be maintained? I noticed it's "semi-official". Also, will this install all the needed dependencies? It seems to me that the Git version installs more packages...

Hi all, can I use the cited PPA https://launchpad.net/~certbot/+archive/ubuntu/certbot
?

It is secure, reliable, and updated?

As for all PPAs, this is your call.

However, @oerdnj is a Debian packager and already does an awful lot of work in other PPAs for Ubuntu (e.g. his incredibly useful PHP PPA).

If you trust him, you can trust his PPAs.

It seems to me that the Git version installs more packages...

The git version sets up a Python virtualenv to build the software and installs all of the associated dependencies locally. A PPA does this behind the scenes so there are fewer end-result packages.

This PPA is ready for use! We'll be updating certbot.eff.org to tell people to use it over certbot-auto or the current Ubuntu packages (see certbot/website#198). We can finally close this issue.

@hlieberman, @oerdnj: can you remove "(semi-official)" from the title of the PPA?

Any chance of getting the PPA renamed to ppa so that people can run sudo add-apt-repository ppa:certbot instead of sudo add-apt-repository ppa:certbot/certbot? This was mentioned in this comment: https://github.com/certbot/certbot/issues/1706#issuecomment-260585028

When will the PPA be updated? Version 12 has been out for 13 days.

im pretty sure you need a prefix to a ppa so it knows which user and repo it belongs too, or else it wont know which repo in that username to pick, I may be wrong, someone else can chime in, of all the ppa i use none just have a prefix.

Sent from my iPhone

On Mar 15, 2017, at 5:18 PM, Geoffrey Fairchild notifications@github.com wrote:

I like @elyscape's suggestion.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@elyscape That would force all the people that are already using the ppa to change their configurations just for aesthetics, so sorry, no, that's not an option at this stage.

@chrismccoy , ppa is the assumed repo if you only specify a user for a PPA.

user@ubuntu-device:~$ sudo add-apt-repository ppa:certbot
Cannot add PPA: 'ppa:~certbot/ubuntu/ppa'.
The team named '~certbot' has no PPA named 'ubuntu/ppa'
Please choose from the following available PPAs:
 * 'certbot':  Certbot PPA (semi-official)
 * 'certbot-build':  Certbot Build PPA (don't use this, use ppa:certbot/certbot)

As I was writing this, @oerdnj replied, so the point is now moot anyways :)

A PPA named just 'ppa' can be added, as @elyscape says, with a shortened apt-add-repository command, but this is really just bike-shedding at this point. It's the certbot PPA for the certbot project and the extra nine characters is hardly onerous.

In terms of updates, 0.12 has not yet landed in Debian (not even sid or experimental). Once it has, and has been suitably wrangled, it should make its way into the PPA:

This is the PPA for packages prepared by Debian Let's Encrypt Team and backported for Ubuntu(s).

I assume you want a tested, working package to deploy? :)

--edit
Two other updates while typing... meh, I'm posting anyway. :P

@enoch85 Is there anything broken for you? #4233 is troubling, but that's it.

I usually follow @hlieberman workflow and he apparently hasn't had time to update Debian packages yet, and I don't want to step on his toes.

I can pull a patch from #4243 on top of 0.11.1 if this gives problems for more people using this PPA, but I don't really follow this "new version just for a sake of new version" worldview.

@bmw "(semi-official)" removed. If you have any other suggestions for the title or description, I would be happy to change it.

@oerdnj No, not yet but maybe soon, as I converted to this PPA from the git version which is 12 and I get warnings about the certs generated with 12. Would be awsome with an update. And as always, you rock!

how long does it usally take to get an update from the launchpad maintainers, i see certbot/certbot on launchpad is at 0.11.1 and 0.12 is out on github.com

@chrismccoy Read the comment by the maintainer of the PPA three comments back from yours, i.e. https://github.com/certbot/certbot/issues/1706#issuecomment-286890691.

@oerdnj I ended up here specifically because of #4233. I'm just one person and this was a simple personal project, but I look forward to the 0.12 release.

Getting this when trying to install a new cert with Certbot 12 from the PPA in this issue. Don't know if it's a Let's Encrypt bug or something with the PPA. Also reported it here: https://community.letsencrypt.org/t/ubuntu-14-04-with-certbot-auto-failing-tls-sni-challenge-with-apache/33439/2

tls-sni-01 challenge for cloud.domin.com
/usr/lib/python2.7/dist-packages/OpenSSL/rand.py:58: UserWarning: implicit cast from 'char *' to a different pointer type: will be forbidden in the future (check that the types are as you expect; use an explicit ffi.cast() if they are correct)
  result_code = _lib.RAND_bytes(result_buffer, num_bytes)
Waiting for verification...

My certs are generated correctly, but I think someone should look into the python warning message.

@oerdnj Sorry to bother you, but is there any way I can follow the packaging of the new versions? i.e. when a new package will/could be ready.

I'm waiting for new packages (v0.14.x) so I can auto-renew certs using Cloudflare and DNS auth.

@shaneog There's no plan as it's on "free time right now" basis (unless there would be a critical bug). However update to 0.14.1 seems as a simple update, so I am uploading the builds right now.

Thank you!

Since the crontab created by installing from the PPA doesn't include the /bin/run-parts bit anymore, what's the recommended way of specifying renew hooks?

For now, I'm just changing my /etc/cron.d/certbot script back to what @oerdnj had in place a few months ago (I'm on Ubuntu 16.04):

0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && certbot -q renew --pre-hook '/bin/run-parts /etc/letsencrypt/pre-hook.d/' --post-hook '/bin/run-parts /etc/letsencrypt/post-hook.d/' --renew-hook '/bin/run-parts /etc/letsencrypt/renew-hook.d/'

...and putting my hook scripts in those directories. Is there a better way to go about this?

It depends on what you'd like to accomplish. There are two main options, but make sure you read the warnings I placed at the bottom of this post.

Global configuration file

You can get behavior similar to the run-parts solution by including hooks in Certbot's configuration file. Your file at /etc/letsencrypt/cli.ini could look like:

pre-hook = /bin/run-parts /etc/letsencrypt/pre-hook.d/
post-hook = /bin/run-parts /etc/letsencrypt/post-hook.d/
renew-hook = /bin/run-parts /etc/letsencrypt/renew-hook.d/

The only behavioral difference between this and the previous crontab is these hooks sometimes also run when obtaining a certificate with other Certbot subcommands such as certbot certonly, certbot run, certbot (with no subcommand), etc. Pre and post hooks will always run if a certificate is obtained, while renew hook is only run if renewing a certificate at an existing path.

Certificate configuration file

Another option is to define hooks per certificate. This allows you to run different hooks depending on the certificate. When you obtain a certificate, the hooks you used to obtain that certificate are stored in /etc/letsencrypt/renewal/domain.conf. When you run certbot renew, these hooks will automatically be run again when the certificate is renewed. You can also edit these files manually by placing hooks under the [renewalparams] section header using the same syntax as I included above.

Warnings

1. These two options are not compatible. Hooks placed in a global configuration file are treated the same as if they were included on the command line which overrides the values found in the configuration file for the certificate.
2. Whatever method you use, make sure you run certbot renew --dry-run after editing the files to make sure everything still works properly.

Perfect, the global config file is exactly what I need, thanks!

So what's the actual status of renewal hooks? Strange how this has become so messy. First it was a cronjob calling parts in different dirs, then a cli file, and now all I see with a clean install are the dirs, since the cli doesn't hold any info on it anymore.
Could anyone at certbot dev lead please just place clear comments with directions in its config files? How hard can that be? For an example, please check how they've done that for CSF/LFD: https://gist.github.com/skt-bford/4987434

I agree it's super confusing and there is a ton of conflicting or outdated info about this. My understanding is that the current best practice is to use --deploy-hook (which is better than --renew-hook) using the certbot command line, and that causes it to save the hook info to the domain in /etc/letsencrypt/renewal, so the hook will get run each time automatically. I think that's considered to be a preferred solution rather than editing /etc/letsencrypt/cli.ini directly, and it also has the advantage that it lets you have different hooks for different certificates.

This is the best info I've found on it: https://community.letsencrypt.org/t/certbot-dovecot-postfix-certificate-renewal-issue/72226/11

I agree it's super confusing and there is a ton of conflicting or outdated info about this.

yes, confusing (!)... But your link is also confuse and its content not reliable....

Hi all, can some of the experts to clean and summarize? Express a "reliable summary for 2020".

Well, about title of this issue, "PPA for UBUNTU", the summary is there, seems perfect!
https://certbot.eff.org/lets-encrypt/ubuntubionic-nginx

@oerdnj I was curious what it takes to get an updated certbot + dns plugins in the repo nowadays? Specifically because certbot 1.2 is adding support for limited-scope Cloudflare DNS-01 tokens, which is a big security boost over the old way which required global API keys

https://github.com/certbot/certbot/milestone/81?closed=1

So what's the actual status of renewal hooks?

I don't think this is the right issue to be discussing that. This _closed_ issue is about an apt repository for installing the certbot tool - which appears to be completely unrelated to your query.

So what's the actual status of renewal hooks?

I don't think this is the right issue to be discussing that. This _closed_ issue is about an apt repository for installing the certbot tool - which appears to be completely unrelated to your query.

Well, I was merely continuing on what drewcking asked, since the answer to that actually did not work in my case. Which has to do with the ubuntu LE packages (that aren't up to date)..