Fabric: Documentation for runtime configuration

Created on 17 May 2018  ·  4Comments  ·  Source: fabric/fabric

I have a bunch of Fabric 1 fabfiles which set env. It's currently non-obvious to migrate that code to Fabric 2. The documentation references e.g. connect_kwargs but there's not much about how or where to set that. After reading through the Invoke docs, it doesn't seem like I can use any of the YAML/JSON configuration options since I need to execute code (in the example below, using keyring to retrieve passwords).

It seemed like something like moving all of it into a fabric.py file might work:

import subprocess
import sys
from getpass import getuser
import os

domain = os.environ.get("DOMAIN", "loctest")

def get_config():
    user = getuser()

    try:
        password = subprocess.check_output(["keyring", "get", domain, user]).strip()
        print("Loaded %s password for %s from keyring" % (domain, user))
    except subprocess.CalledProcessError:
        print(
            "Unable to set password using `keyring` (is it in your PATH?) — expect to be nagged",
            file=sys.stderr,
        )

    return {'user': user, 'password': password}

connect_kwargs = get_config()

Unfortunately, that gets TypeError: can't pickle module objects. Setting __all__ isn't enough to avoid that but moving everything into the function appears to be working:

def get_config():
    import subprocess
    import sys
    from getpass import getuser
    import os

    domain = os.environ.get("DOMAIN", "loctest")
    user = os.environ.get("USER", getuser())

    try:
        password = subprocess.check_output(["keyring", "get", domain, user]).strip()
        print("Loaded %s password for %s from keyring" % (domain, user))
    except subprocess.CalledProcessError:
        print(
            "Unable to set password using `keyring` (is it in your PATH?) — expect to be nagged",
            file=sys.stderr,
        )

    return {"user": user, "password": password}

connect_kwargs = get_config()

Is there / planned to be a cleaner way to do this? Among other things, this code isn't quite equivalent to the Fabric 1 version because it doesn't pull the username in from the config which isn't something I need but it seems like there should be a way to say “Get the processed config and update this one value”.

Bug Feature Needs investigation

Most helpful comment

Is there a simple hello world example using a fabfile.py file? I have been reading all the docs and scouring the internet for a couple of days and have not been able to run a single command remotely yet on v2. As far as I get is stuck with an SSHException "No authentication methods available"

All 4 comments

I was intending that "Python format" config files would roughly fill the hole of "populate config values via runtime code execution", so you were arguably on the right track with the .py file. That said – I did not follow that thought all the way to conclusion and it sounds like you're paying for that, apologies. (I will note that part of the reason is I assumed most advanced users would be doing things in their tasks file/fabfile, but either way the bridge from config-esque data to runtime task bodies needs work.)

Don't remember offhand what would be pickling things but it's probably some quirk of the config system, I'll need to dig into that. I don't like pickle (these days, who does) so it was a little surprising to find it pop up in this context.

Regardless, the proximate cause is that we assume the module contents to "be" the configuration, so we need more stringent filtering when we go from module object attributes to configuration nested dict. Stripping out modules might be all that's needed honestly, since any other variable assignment would arguably be bad behavior on the part of the config file author.

Re: the higher level design question of "mutate just one nested config value", that _ought_ to work intuitively in most cases (the config merging process tends to do a deep merge, so your .py file just setting connect_kwargs = {'user': foo', 'password': 'bar'} ought to end up merged with any other config source's connect_kwargs contents) but I think there might be some bugs around connect_kwargs specifically, e.g. #1762.

I should retest this since that was my initial Fabric 2 upgrade attempt and I've seen a lot of commits flying by since then.

Is there a simple hello world example using a fabfile.py file? I have been reading all the docs and scouring the internet for a couple of days and have not been able to run a single command remotely yet on v2. As far as I get is stuck with an SSHException "No authentication methods available"

Was this page helpful?
0 / 5 - 0 ratings