Mopidy: Move Mopidy infrastructure away from Rackspace

apt.mopidy.com

I have a new (reduced, more manageable) Pi Musicbox v1.0 which only has a couple of bits packaged. I want to get this out there at some point. Would I be able to get sufficient access if it's a VM on your private machine? And I assume you guys pay for this yourselves, which isn't fair.

I think we should apply to Discourse for free hosting. I can do this.

I agree, I think we fit the bill.

Pi Musicbox release image hosting.

I think we should upload all of these to GitHub Releases ASAP and drop the vhost at the end of the year. I can do this if @kingosticks agrees.

I agree and I will do this, probably at the weekend.

Sorry, I missed that @jodal said he'd upload the old releases. If you want to do that then go right ahead!

The only expense for the VM machine is buying it initially and repairs if something goes wrong. We've not used it before as we don't want to rely on "private" infrastructure and we can't give out as much access to it.

For the OAuth bridge I was looking at if AppEngine or something like that could have a tier that would work. It really doesn't need much to function, and could easily be ported to an other storage system than sqlite if needed.

Netlify do a free tier that's suitable for OSS. Could we use them to host apt.mopidy.com?

Some combination of https://cloud.google.com/free/docs/always-free-usage-limits might work for the OAuth bridge.

I wonder if you could get a company that doesn't normally do free hosting to sponsor the project? OVH, Hetzner, online.net... there are plenty of options which are extremely high volume hosting companies that own their own DCs and would hardly notice giving out a physical machine with 128GB of RAM and 2 x Xeon E5 to an OSS project :) Then you could use a secure container solution like LXD, OpenShift or SmartOS to create areas with different levels of sensitivity/access (e.g. to give certain people ability to update / publish packages in one area without giving them dangerous user-level SSH to the whole infrastructure).

I've now applied to Discourse at https://blog.discourse.org/2016/03/free-discourse-forum-hosting-for-community-friendly-github-projects/ for a hosted Discourse instance, with the following motivational letter:

Greetings!

The Mopidy project already have a Discourse site at https://discuss.mopidy.com which serves the community around the Mopidy project, the 50+ Mopidy extensions (https://pypi.org/search/?q=mopidy), and the Pi Musicbox (http://www.pimusicbox.com) distribution for Raspberry Pi.

We've been self-hosted on a sponsored Rackspace server for the last few years, but Rackspace are now ending their Open Source Discount program (see https://github.com/mopidy/mopidy/issues/1643 for details), and we're looking for a new host for our Discourse site.

Can you help us out?

All Pi Musicbox releases from https://dl.mopidy.com/pimusicbox have now been uploaded to https://github.com/pimusicbox/pimusicbox/releases.

Disclaimers:

• To be able to create GitHub Releases, I needed a git tag for each release. As old Pi Musicbox releases was quite handmade there is probably no commit that matches perfectly. As a workaround I created an orphan commit that I pointed all the new tags to.
• I tried setting the commit and tag date to 1970-01-01T00:00:00Z to get them sorted after the currently latest release, 0.7.0rc4, but that didn't work.

Thanks @jodal, sounds like that would have taken me a long time to figure out and do.

There are some suggestions for Rackspace alternatives at https://news.ycombinator.com/item?id=15502356, Linode sounds interesting.

Three updates...

dl.mopidy.com

Thanks to @trygveaa we figured out that only the author date and not the commiter date for the release placeholder commit was set to 1970. With that fixed, the old Pi Musicbox releases are now sorted after the latest releases.

The dl.mopidy.com vhost has been updated with the following redirect rules to keep all URLs working:

rewrite ^/pimusicbox/$ https://github.com/pimusicbox/pimusicbox/releases permanent;
rewrite ^/pimusicbox/pimusicbox-(.*).zip https://github.com/pimusicbox/pimusicbox/releases/download/v$1/pimusicbox-$1.zip permanent;

discuss.mopidy.com

Discourse has approved our application for free hosting with the only requirement being that we change the hostname from discuss.mopidy.com to discourse.mopidy.com. I don't have an issue with that and would love to get rid of the regular task of upgrading Discourse and the irregular task of rebuilding the Docker image.

I'll move ahead with this migration unless someone has strong arguments against. I'll of course make discuss.mopidy.com redirect to discourse.mopidy.com.

Rackspace turns around

I received the following email from Rackspace today:

We apologize for any confusion yesterday’s communication may have caused. While we’re no longer accepting new applicants to the Open Source Discount (OSS) program, all existing OSS program customers will keep their credits. Rackspace is committed to open source technology, and we look forward to continuing to support you and your Open Source project.

The use of the word "credits" makes this a bit vague, but that's probably just their internal solution to giving us free hosting. I interpret this as: we're free to stay on Rackspace for as long as we like.

With that message in mind I suggest that we:

• continue the deprecation of dl.mopidy.com,
• continue the move to Discourse for hosting of the forum, and
• leave apt.mopidy.com and auth.mopidy.com running on the Rackspace server for now.

Two updates...

dl.mopidy.com

All references I know of in the Pi MusicBox docs and web site have been updated to refer to GitHub Releases instead of dl.mopidy.com.

I think the only thing that remains now is to let a couple of years pass before dl.mopidy.com silently disappears.

discuss.mopidy.com est mort, vive discourse.mopidy.com

I've just completed the migration of the Discourse forum from our own Rackspace server to sponsored hosting by Discourse over at https://discourse.mopidy.com. Thanks to Discourse for great support so far! :-)

Migration process

1. Put discuss.m.c in read only mode.
2. Take a full site backup.
3. Share the backup with Discourse support.
4. Discourse restored the backup on discourse.m.c and adjusted upload paths, etc, to match their setup.
5. Update the settings for GitHub/Google/Facebook/Twitter OAuth login. Yahoo seems to not require any settings. Verification of Yahoo login still working would be nice.
6. Test, test, test.
7. Redirect discuss.m.c to discourse.m.c.
8. Tear down old email reply setup which included a Mailgun forwarding route, a GMail account, and POP3 polling.
9. Update all references to discuss.mopidy.com across the repos in https://github.com/mopidy and https://github.com/pimusicbox.

Known regressions

Replying by email to emails sent out from before the migration, with reply address on the form "[email protected]", does not work as Discourse has their own email reply setup. I've removed the Mailgun route to make these replies bounce instead of disappearing in silence.

@kingosticks will probably be affected by this, so a mention by name here to make him aware of this.

Brilliant thanks @jodal. I tend not to reply by email much the days since it was quite slow.

Two years later, I've now removed all traces of dl.mopidy.com and discuss.mopidy.com.