Fabric: Add Tunneling Context to Fab

Created on 19 Aug 2011 · 13Comments · Source: fabric/fabric

Description

This would be very helpful for me because it would make it very easy
to for example connect to a remote MySQL server using my command line
MySQL client.

We could use a "with" context statement, something like:

with tunnel(local=3307, remote=3306):
    local('mysql --port=3007 --host=localhost' mydb < db/dbdump.sql')

This would eliminate the need to upload mysql dump file to the server
just to be able to run import.

Another application could be for administering Cherokee web servers.

Cherokee web admin by default is only accessible from the server that
it's running on. So you want to access the admin you have to tunnel
into the server and access the admin interface using a local port.
This could also be simplified with this functionality.

with tunnel(local=9090, remote=9090):
   sudo('cherokee-admin')
   prompt('Stop cherokee admin?')

That last line would keep the tunnel open until it's closed by providing input.

Originally submitted by Taras Mankovski (tarasm) on 2009-11-02 at 09:30am EST

Relations

Related to #38: Implement tunnelling

Feature Network

Source

bitprophet

Most helpful comment

939 is still in a release bucket, I just punted it from the upcoming 1.11 because I needed to cut _some_ stuff, but it'll get priority for the next feature cycle. (And it looks like #1218 supersedes #939 so I'll probably end up merging that & crediting #939 in the changelog.)

bitprophet on 9 Apr 2016

👍3

All 13 comments

Jeff Forcier (bitprophet) posted:

(modified description so code blocks were indented :))

on 2009-11-02 at 09:35am EST

bitprophet on 19 Aug 2011

It would be awesome if tunnel would also support the other way around. Meaning listening on remote and forwarding it into localhost/other locally available host:port.

munhitsu on 9 Jan 2012

@munhitsu I'm not sure how that's a use case for Fabric specifically, though. Can you elaborate?

bitprophet on 11 Jan 2012

Imagine a DMZ setup which does not have an outbound http/proxy access.
Whole deployment goes through fabric.

In such case we could use fabric to tunnel local proxy through ssh, to be temporarily accessible for the host being just provisioned in DMZ. Right now I'm opening a separate ssh tunnel on second console, so that fabric works within "context" of this tunnel.

example usage:

with rev_tunnel(local=8080, remote=8080):
    sudo("http_proxy='http://localhost:8080' apt-get install -y puppet")

munhitsu on 11 Jan 2012

OK, so it is a fairly standard reverse tunnel setup after all I guess, and your desire re: Fabric is for it to handle doing the tunnel vs you having to run eg local(ssh -R ...).

I went back and forth on whether this is truly worth going into core, but it really would make a decent amount of sense for it to be supported in Fabric proper; the other solutions are hacky (eg some thread or subprocess running ssh -- how to do this well, make sure it shuts down when Fab does, etc) and I do see the validity of the use case (sharing local resources with the remote end during execution.)

Main roadblock is that I am not sure the SSH lib supports this yet; we'll have to figure that out and somebody would have to implement it if it doesn't. (I think that would be a good addition to said library, however, vs solidifying aforementioned ssh -R workaround.)

EDIT: #38 discusses implementation and patching and such. Might be best to actually close this and simply note there that when implemented, it should be possible to trigger with a context manager if possible.

bitprophet on 11 Jan 2012

I fully agree that having it implemented so that we enter, leave contexts, or even worse multiple levels of contexts is tricky.

Regarding EDIT: Anything that keeps us closer to having this functionality is a good idea.

munhitsu on 17 Jan 2012

For now I'll leave this open, just to keep things granular. Assigned it to 1.4 so I don't forget, though. Chances are I will try to knock this out right as I am doing #38. Will update here when it's in master, thanks.

bitprophet on 18 Jan 2012

This is actually not as related to #38 as I thought, since its changes are solely about gatewaying the SSH traffic itself, not tunneling additional ports through the SSH connection. That will require a different (or at least, additional) solution. Punting for now, sorry :) (Meaning: still open, just not going to arrive in 1.5.)

bitprophet on 5 Nov 2012

I recently needed this functionality to sync to a remote rsync server whose port is not directly reachable and found that the paramiko's forward.py demo code has sample code that I could use, so I came up with a solution that worked well for me and I submitted it as a patch for forward.py here: https://github.com/paramiko/paramiko/pull/504/

We could add ForwardServer from that patch and have a local_tunnel() that simply returns an instance of ForwardServer. As per recommendation from @bitprophet on the pull request, I will work on a patch for Fabric.

haridsv on 7 Apr 2015

I actually didn't realize, but there is already a patch for local_tunnel(), though I am not fully sure about its status. What should I do?

haridsv on 7 Apr 2015

@haridsv If you're able, testing it out & mentioning that you successfully used it, on that patch (#939), would help, otherwise just have patience :) thanks!

bitprophet on 4 May 2015

Is there any plan to resolve this issue? There are two outstanding pull requests that together resolve this, but they haven't been reviewed? Any way we could speed this along?