Ansible: TRANSFORM_INVALID_GROUP_CHARS doesn't document valid group patterns

Created on 24 May 2019  ·  104Comments  ·  Source: ansible/ansible



SUMMARY

With the addition of the TRANSFORM_INVALID_GROUP_CHARS. Other than reading the source, it was not clear which characters must be avoided going forward, only that the warning (with -vvvv) points out which characters you currently use that are invalid.

Please clarify that you are pushing the names to be valid python vars. This is missing from the documentation for the cfg option, warning and online documentation

(https://github.com/ansible/ansible/commit/d241794daa6d413e6447890e2a4f11e0d818cf0e#diff-b77962b6b54a830ec373de0602918318R122)

There appears to be no mention of this on https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.8.html either.

ISSUE TYPE
  • Documentation Report
COMPONENT NAME


group

ANSIBLE VERSION

ansible 2.8.0
  config file = /home/awoodward/ansible-skynet/ansible.cfg
  configured module search path = [u'/home/awoodward/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.5 (default, Apr  9 2019, 14:30:50) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
CONFIGURATION

n/a

OS / ENVIRONMENT

n/a

ADDITIONAL INFORMATION

n/a

affects_2.8 docs has_pr module core system

Most helpful comment

What was the reasoning behind dropping the dash from group names? I really struggle to find good reason for that, especially as this will require lots of code refactoring.

All 104 comments

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

i started to get this warning but found no references in the porting guide and no references how or what to fix.

most of my warnings are from the ec2.py, where the instance_id used the - (eg: i-033f62b586143dff7) and the regions (eg: eu-central-1c) , so we have no real fix for this ones

Finally, this broke some of my playbooks, where i used when: ansible_hostname in groups['varnish'] and the ansible_hostname is varnish-eu-central-1c-001 .
In the past this worked fine, now i need to use inventory_hostname to get varnish_eu_central_1c_001 and get a match vs the groups['varnish']

So this needs at least and urgently a warning in the porting guide that inventory_hostname and groups[] may be returning different data

What was the reasoning behind dropping the dash from group names? I really struggle to find good reason for that, especially as this will require lots of code refactoring.

@ssbarnea For one thing, we are making a push to only allow variable names, and other similar keys, that are valid python identifiers. To explain a little further about group names, it causes a problem for users trying to use "dot syntax" such as groups.foo-group, which doesn't do what the user expects. The number of issues and support requests caused by small problems like these has caused us to go down a path to safe guard names to ensure that problems like this do not occur.

For those wanting to keep what we consider invalid characters, can opt out of this functionality.

What must we do to opt out of this functionality? Our local Ansible deployment scripts are littered with hyphen-containing group names. We don't use them with dot notation, of course. But changing all of them would be a truly monumental task. I would prefer to opt out and at the same time encourage my team to avoid using hyphens in the future and where possible to convert hyphens to underscores, tho' that last part is not always as straightforward as it might seem.

So, does one simply set force_valid_group_names = false in ansible.cfg ? That seems right based on https://github.com/ansible/ansible/commit/d241794daa6d413e6447890e2a4f11e0d818cf0e#diff-fd24ad93fbc32f454761746c1ac908f2

What must we do to opt out of this functionality? Our local Ansible deployment scripts are littered with hyphen-containing group names. We don't use them with dot notation, of course. But changing all of them would be a truly monumental task. I would prefer to opt out and at the same time encourage my team to avoid using hyphens in the future and where possible to convert hyphens to underscores, tho' that last part is not always as straightforward as it might seem.

So, does one simply set force_valid_group_names = false in ansible.cfg ? That seems right based on d241794#diff-fd24ad93fbc32f454761746c1ac908f2

export ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=never or export ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore -- the latter is not in the docs yet: https://github.com/ansible/ansible/pull/57318

Thanks, James. Since people are coming to this issue to follow-up on the warning message, I'm including information I think may be useful:

To disable the ≥2.10 group name auto-transformation more portably/permanently until such time as you may be ready to clear out invalid groups from your inventory, add force_valid_group_names = never to the [defaults] INI-section of ansible.cfg.

To see all groups and invalid characters which trigger the warning (perhaps so you can target them for phase-out), you can do something like this ansible CLI no-op:

ansible-inventory -vvvv --host=localhost 2>&1 | grep replacing

These invalid characters are (as of 2019-06-04) defined as a constant, INVALID_VARIABLE_NAMES, in:
https://github.com/ansible/ansible/blob/devel/lib/ansible/constants.py#L119
as '^[\d\W]|[^\w]',
that is: any leading non-alpha character OR any character other than alpha-numeric and underscore.
(I hope I got that right)

If you find deprecation warnings annoying, you may also disable permanently them for any ansible- command or the ansible ad-hoc command by adding deprecation_warnings = False to the same [defaults] section of ansible.cfg, but I would recommend against that (since you may miss important news), and instead use inline shell environment variables like this:
ANSIBLE_DEPRECATION_WARNINGS=False ansible-inventory --host=localhost

The inventory parsing [WARNING]s won't go away, however. There's no specific config or env var to turn off all warnings (yet?), but if it really bugs you, you can just send all stderr to /dev/null (insert "best practice" caveats here):

2>/dev/null ansible-inventory --host=localhost

Hope this helps someone, somewhere.

I only find deprecation warning messages annoying when they do not provide a migration path. Considering that the space is limited and that the remediation is likely to need update I would find very useful to provide links to tickets which can document solutions, workarounds,...

An approach like this could save extra work needed for improving incomplete warnings messages as we would not have to update the message, backport it to few versions back.

PS. Disabling deprecations warnings is something I would not recommend to anyone, maybe only if a project is already facing its ultimate fate ;)

i started to get this warning but found no references in the porting guide and no references how or what to fix.

most of my warnings are from the ec2.py, where the instance_id used the - (eg: i-033f62b586143dff7) and the regions (eg: eu-central-1c) , so we have no real fix for this ones

Finally, this broke some of my playbooks, where i used when: ansible_hostname in groups['varnish'] and the ansible_hostname is varnish-eu-central-1c-001 .
In the past this worked fine, now i need to use inventory_hostname to get varnish_eu_central_1c_001 and get a match vs the groups['varnish']

So this needs at least and urgently a warning in the porting guide that inventory_hostname and groups[] may be returning different data

I'd like to echo statement about the warning being generated by the EC2 dynamic inventory script. I noticed that there is an ec2.ini configuration setting to disable grouping hosts by instance_id (group_by_instance_id = False), but setting that didn't resolve the warning for me like I anticipated it would - I made sure I cleared the local inventory cache.

Any workarounds for EC2 dynamic inventory specifically?

These invalid characters are (as of 2019-06-04) defined as a constant, INVALID_VARIABLE_NAMES, in:
https://github.com/ansible/ansible/blob/devel/lib/ansible/constants.py#L119
as '^[\d\W]|[^\w]',
that is: any leading non-alpha character OR any character other than alpha-numeric and underscore.
(I hope I got that right)

Sounds accurate to me. You should submit a docs PR with that info.

If you find deprecation warnings annoying, you may also disable permanently them for any ansible- command or the ansible ad-hoc command by adding deprecation_warnings = False to the same [defaults] section of ansible.cfg, but I would recommend against that (since you may miss important news), and instead use inline shell environment variables like this:
ANSIBLE_DEPRECATION_WARNINGS=False ansible-inventory --host=localhost

The inventory parsing [WARNING]s won't go away, however. There's no specific config or env var to turn off all warnings (yet?), but if it really bugs you, you can just send all stderr to /dev/null (insert "best practice" caveats here):

The undocumented ignore option provides this functionality. Docs PR here: https://github.com/ansible/ansible/pull/57318

Starting in 2.8.2, this deprecation warning will be squashed if you explicitly set any of choices.

Where does the ansible dev team discuss this type of decisions? It is very hard for us users to understand the reasoning to this. If it is pure "python style" reasoning, rather than a practical reasoning, it might be worth reconsidering? If a dash in group names break things in future releases of ansible, it might rather be a problem with the implementation, more than the naming of a group?

To me, this sounds more like a cosmetic change, than something that has been thought through properly.

A group name is not a variable name, it is the content of such. A hyphen/dash is just a character, that also happens to be an extremely popular way of grouping information in a naming convention. Compared to the exclamation point or the star, it does not have a special meaning in a limit clause.

The cost to mitigate this issue is enorm, given that thousands of sites has to not only change the group names in the inventories, but also go through all their playbooks and home grown roles, and test them all again.

If there is any way at all for "peasants" to make their voice heard, I would love to chip in my opinion and try and understand how this idea came to be.

I've come to understand that the change was made to ansible because users made mistakes such as trying to use groups.group-name rather than groups['group-name']. AIUI, it is a change purely for the purpose of reducing support issues. (I'm personally opposed to the change.)

The old behavior will not go away; it will just become unavailable without explicitly choosing the old behavior.

Sad to hear.

My use case, is that I am embedding the command "ansible-inventory" in a Vagrant file, in a way where it is impolite to put things in ansible.cfg, and that it would help to be able to be able to override the behavior as a command line option (not environment variable).

Usually changes like this are due to good intentions, but might not always lead to the outcome one has in mind.

My issue with this change is that group names now become somewhat "special" - dashes are allowed in host names, but not in group names which makes it a bit weird considering at the start of a playbook in the hosts: section I could write host and/or group names.

Is the explanation given by @sivel really the only reason behind this change? What about hosvars['foo-host'] then? I hope nobody is considering to make dashes invalid characters in inventory host names as well...
Besides hostvars there are a ton of other examples where the "dot-notation" cannot be used so the need to know when to use which form will remain. I find it rather arbitrary to single out group names.

While the dot-notation argument is a somewhat valid excuse, I don't see this fixing your support issues without instead improving documentation. If your users are doing something stupid, your documentation isn't adequate. All the devs have succeeded in doing is alienating a lot of users. Group names I see as arbitrary string values. Restricting to alphanumeric and underscore honestly is somewhat of a pain, especially when hostname RFC's allow dashes, periods, etc... If underscores were the de-facto standard for naming conventions I don't think this would be an issue. Hyphens are used widely for descriptor strings. If you want to reduce your support volume try addressing the dot notation issue from another direction; build a validation script your support teams can provide that checks for best-practice issues and provides warnings or guidance as an example. Update your documentation about the dot-notation caveat to be big, bold, red, flashing, whatever... Such support cases end up being 1 minute calls if your documentation already covers the issue. Answer the phone, see the issue, provide the doc link, done.

Dashes in group names are both valid INI and valid YAML, I don't see why I as a user should have to rename all my groups just because the names can't be used as Python variable names?

Also questioning the rationale behind this decision to deprecate - in group names. Not being able to use dashes in dynamic inventory keyed_groups was already annoying enough, but having to rename all of our groups in our inventory files and ansible-playbook -l ... commands just to avoid hypothetical syntax-related support issues is going to be painful.

FWIW we have a convention for naming role-groups like foo_server, and host-groups like foo-dev or foo-test. Almost 100% of our Ansible usage is commands like ansible-playbook -l foo-dev, so this change will take a lot of effort to fight muscle-memory.

Not sure if adding another me 2 here will encourage a reversal of this specific decision, but I tend to agree with the detractors of the requirement that group names be valid python identifiers.

Please support hyphens together with letters, numbers and underscores in group names (but I don't have anything against dots either)!

We use hyphens heavily in group names. Both for grouping names like this:

[server-3x]
server-31.example.com
server-32.example.com
server-33.example.com

and to _abuse_ inventory to keep host list in one place (instead of maintaining host names in different var files) like this:

[prometheus_node-exporter_cluster1:children]
server-3x
server-5x
````
We use such groups in templates like this:

{% set _hostgroup = [_service, _job, _cluster]|join('_') %}
{% set _hostlist = groups[_hostgroup]|d([])|sort %}
{% if _hostlist %}
{% for host in _hostlist %}
...
```

We don't use dots just to make visible differences between group and host names.

The word INVALID in TRANSFORM_INVALID_GROUP_CHARS doesn't give any confidence that it is possible to continue using them in long run.

If the intent is to avoid using these characters then better call them _UNSAFE_ characters, show warning and let users decide if they see this warning or not. But never disallow or replace these characters!

Users should a) mute this warning (using keyword like ALLOW_UNSAFE_GROUP_CHARS), b) change their group names (when possible) or c) just live with that warning. Most will choose between first two options anyway.

I also feel this is pointless, as a dash "-" is a standard delimiter character used in almost every type of computer related tool, and trying to conform to one "religion" seems constricting!!!

The old behavior will not go away; it will just become unavailable without explicitly choosing the old behavior.

I wouldn't be as concerned about this deprecation if were actually possible to opt-in to dashes in group names. Then it might be understandable from a new-user perspective.

However, the deprecation warning implies that the TRANSFORM_INVALID_GROUP_CHARS=never option is going away in Ansible 2.10, and so we would need to start renaming all of our groups before Ansible 2.10 is released?

[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be user configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

Also, using dynamic inventory plugin keyed_groups forces the transformation of group names, even if TRANSFORM_INVALID_GROUP_CHARS=never is set: https://github.com/ansible/ansible/blob/db0fe4b1884e6bb9c25e970c7585abb7edd9d664/lib/ansible/plugins/inventory/__init__.py#L45 https://github.com/ansible/ansible/blob/db0fe4b1884e6bb9c25e970c7585abb7edd9d664/lib/ansible/inventory/group.py#L39

Desired behavior

  • Using TRANSFORM_INVALID_GROUP_CHARS=never needs to keep being supported in the future

    EDIT: reading the code, it sounds like the intention is to keep TRANSFORM_INVALID_GROUP_CHARS but change the default to always in 2.10 - in that case, the deprecation warning is not very well worded: https://github.com/ansible/ansible/blob/db0fe4b1884e6bb9c25e970c7585abb7edd9d664/lib/ansible/inventory/group.py#L50

  • Using TRANSFORM_INVALID_GROUP_CHARS=never should silence the deprecation warning

    This seems to already be possible with the undocumented ignore option: https://github.com/ansible/ansible/pull/57318

  • Using TRANSFORM_INVALID_GROUP_CHARS=never should also allow the use of dashes in dynamic inventory keyed_groups

    EDIT: this is clearly for backwards-compatibility for Ansible 2.7, which unconditionally transformed the generated group names. It would be great to have an explicit opt-out for this.

Regarding variable names, I don't understand why should format of dictionary key be equalized with syntax of variable name? AFAIK no programming language has such limitation. In Python you can use pretty any string as dictionary key.

Isn't "groups" a dictionary type variable and both host and group names are just plain dictionary keys in Ansible. They are not properties nor variables themselves or are they?

I'd rather disallow groups.foo-group syntax than groups["foo-group"]. If g = "foo-group", then do you use groups.g or groups[g]?

Using ansible.cfg [default] force_valid_group_names = ignore or export ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore doesn't seem to work on Ansible 2.8.1. It still gives the deprecation warning.

$ ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore ANSIBLE_VAULT_PASSWORD_FILE=vault-password.secret ansible-playbook --diff -i xyz-dev.ini xyz-infra-install.yml -l xyz-dev --check
[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be user configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

Is this because it's not yet listed in the valid choices? https://github.com/ansible/ansible/blob/v2.8.1/lib/ansible/config/base.yml#L1501

Using ansible.cfg [default] force_valid_group_names = ignore or export ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore doesn't seem to work on Ansible 2.8.1. It still gives the deprecation warning.

$ ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore ANSIBLE_VAULT_PASSWORD_FILE=vault-password.secret ansible-playbook --diff -i xyz-dev.ini xyz-infra-install.yml -l xyz-dev --check
[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be user configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

Is this because it's not yet listed in the valid choices? https://github.com/ansible/ansible/blob/v2.8.1/lib/ansible/config/base.yml#L1501

This is a bug that's fixed in the upcoming version 2.8.2. You'll be able to export ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore and it will squash all warnings.

(The ignore option is still not documented: https://github.com/ansible/ansible/pull/57318 )

This is going to break everyone. Bad decision.

Would there be a way to reason with the maintainers about this?

Perhaps one of the maintainers could elaborate a bit here, if it is just a support issue or if they are using python constructs that really breaks?

I just want to add this quite annoying and the inablilty to really figure out the problem was also annoying, I litterally had to do ansible-playbook "insert yaml file here" > output.txt to figure out the problem.

Concur with most of the posters here. Removal of dashes from group names seems like a poorly thought-through decision, or one that is implementation, not semantically, driven.

This change makes absolutely no sense to me. Ansible devs want to force thousands and thousands of users to change their group naming just because they want an additional syntax(not a missing one) for accessing groups? Is that a joke?

We use dashes and dots in big setup.
Our pattern is product-name.environment.datacenter and it makes things very clear.
I cannot imagine dropping - and . as that would make inventory totally unreadable.

We're using ansible inventory plugins that query the local CMDB for group names which contain (and will continue to contain) dashes. It would break a lot of stuff if this would be invalid in the future.

We use dashes and dots in big setup.
Our pattern is product-name.environment.datacenter and it makes things very clear.
I cannot imagine dropping - and . as that would make inventory totally unreadable.

We are using a similar approach with a hierarchical naming scheme (inspired by java, e.g. org.company.product-name.component).
It would be an absolute horror having to revert to underscores.

heh. we faced that issue as well. we're using dash in our groups names intensively.
If one could explain what problem are due to the dash usage in a dict I would be happy to know

I am mainly reiterating what others have said, but I wanted to add some input. I think if this change is implemented, that the flag to allow dashes should be kept and maintained. Although I understand that Python expects underscores, dashes are commonly used for host names and host group names. In our environment we generate the inventory dynamically from hosts and host groups in our LDAP/Kerberos directory. I mention this because although it would be possible for us to change the host and group names, it is not preferable.

What must we do to opt out of this functionality? Our local Ansible deployment scripts are littered with hyphen-containing group names. We don't use them with dot notation, of course. But changing all of them would be a truly monumental task. I would prefer to opt out and at the same time encourage my team to avoid using hyphens in the future and where possible to convert hyphens to underscores, tho' that last part is not always as straightforward as it might seem.

So, does one simply set force_valid_group_names = false in ansible.cfg ? That seems right based on d241794#diff-fd24ad93fbc32f454761746c1ac908f2

Test on Ansible 2.8.2, this INI setting doesn't work as expected I believe, this will only remove the DEPRECATION WARNING, while, what I want is for Ansible to use my groups with dashes without complaining.

Here are the results without:

[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be user configurable on deprecation. This feature will be removed in version 2.10. Deprecation
warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

And with the setting set to "false" in ansible.cfg:

[WARNING]: Invalid characters were found in group names and automatically replaced, use -vvvv to see details

Using ansible.cfg [default] force_valid_group_names = ignore or export ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore doesn't seem to work on Ansible 2.8.1. It still gives the deprecation warning.

$ ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore ANSIBLE_VAULT_PASSWORD_FILE=vault-password.secret ansible-playbook --diff -i xyz-dev.ini xyz-infra-install.yml -l xyz-dev --check
[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be user configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

Is this because it's not yet listed in the valid choices? https://github.com/ansible/ansible/blob/v2.8.1/lib/ansible/config/base.yml#L1501

This is a bug that's fixed in the upcoming version 2.8.2. You'll be able to export ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS=ignore and it will squash all warnings.

(The ignore option is still not documented: #57318 )

But is it just squashing warnings, or will that allow us to keep using dashes in groups?
This is not very clear.

I agree with all the detractors here.

In addition to breaking playbooks, this contributes to what I call the convention mess in ansible. Now hostnames and groups names have a different convention, just because some isolated beginners stumble upon the hyphen issue in dot notation ? Guess what ? They will stumble on it still, and the feature will have succeeded in angrying people and not solving any problem. Bravo.

Ansible group names should be capable of respecting the naming of the real world groups they represent.

If all other tooling calls a set of hosts my-backend-service why should ansible force operators to translate that to my_backend_service to satisfy python's naming rules.

It's is a really sad day today.. When a JR co-worker raised this deprecation to me I was like no way would the Ansible team be so out of touch with reality to make such a selfish choice. I absolutely love Ansible for what it can accomplish (from a user perspective that has ZERO to do with it being written in Python) The direction here to push PEP standards on end users makes me completely loose faith in the core Ansible development team's ability to make rational decisions. I hope IBM straightens it out..
OR
Maybe there will be a new-shiny GO equivalent we can move to.

As this behavior is obviously very controversial, I am asking myself if this is a done deal and this is going to be implemented in any case?

I would very much appreciate a response from the people behind this decision and hope for some elaboration beyond "this is a python standard thing".

As this behavior is obviously very controversial, I am asking myself if this is a done deal and this is going to be implemented in any case?

I would very much appreciate a response from the people behind this decision and hope for some elaboration beyond "this is a python standard thing".

I agree with you. Just recently the "go" project backed out of an impopular proposal (see https://github.com/golang/go/issues/32437#issuecomment-512035919), so things like this can (and sometimes should) be revisited and eventually also backed out.

It is also an interesting topic and discussions, perhaps not only for this feature change. It is hard to figure out how the governance of Ansible as a product works. Perhaps something _someone_ should bring with them to https://www.ansible.com/ansiblefest ?

As many of us are scratching our heads, not understanding how a string/variable content/group name in any way can pose any issues with python coding styles, it would be nice to get a reply here from the maintainers, arguing why this would pose an issue.

I can understand if they want to keep a coding style for variable names and structures, but the content of an array or variable?

Here is a quick discussion about dot notation of dicts. It is possible, but ugly. https://stackoverflow.com/questions/16279212/how-to-use-dot-notation-for-dict-in-python

The fact that there are support issues surrounding this is in my opinion a documentation issue not a functionality issue. If anything I would argue that group names should NOT be variables.

As is the fact that this change was implemented before any documentation was/is available.

What is the impact of this change? Do I have to carefully edit all my ansible inventories to make sure I don't have dashes in group names?

@CMoH IMO the best workaround for now is to add force_valid_group_names = ignore to your configuration and run ansible 2.8.2 or later.

@skyscooby, even this is a PITA. The thing that it is not possible to put this line as a default in /etc/ansible.cfg and use local ansible.cfg in playbook directories for other config. This means that all exitsing ansible.cfg files needs to be changed.

Or is there any way to set up global defaults (without adding another variable to the user environment)?

@Cougar agreed this choice from Ansible is a PITA..

Your problem however is not unique to this setting.. we experienced similar pain and now we discourage the use of per-project ansible.cfg files as most settings can be set with environment variables which override cfg file settings.. so if a projects for some obscure reason needs a specific setting we ask they use the ENV method which leaves the rest of the settings they don't need to change set to corporate standards. We build a base docker container with this standard configuration and individual projects simply add ENV entries into their own Dockerfile while basing their image of the base ansible container. All ansible is executed within the container so we are certain all pip modules, ansible versions and runtime tools are identical end to end.

EDIT: this also gives us the ability to stage in new versions of ansible and control issues like this before everyone in the company gets hit with them :)

I did some digging.

this functionality was originally added in PR https://github.com/ansible/ansible/pull/52748 allegedly to support the feature request https://github.com/ansible/ansible/issues/40581

one description of the goal: https://github.com/ansible/ansible/pull/52748#issuecomment-467976473

first version of THIS symptom (though different cause): https://github.com/ansible/ansible/issues/51844

Man, I have been reading #52748 so many times now.

As I understand it, group names were previously sanitised in the plugins and the core, and someone (for whatever reason, as it is still not at all clear to me why) decided that group names should follow python variable naming conventions.

So #52748 pushed the sanitation into the inventory instead, which breaks things for so many people. Especially people who use clever naming conventions, for example in AWS, Azure, etc, to map hosts to groups.

If we were to use the same standard/naming convention for host names, we would for sure lose momentum and lose users.

A group name is a name, not a variable. Using dashes in group names (just as in host names) makes sense. The translation (sanitation) should not need to be done on an inventory level (by us, the users), and in the best of all worlds actually never.

I really don't see the benefit of enforcing this. The discussion seems to cover also "." and ":", which some people like using in group names. Personally I don't use them, but I also don't see the harm in doing so.

As long as cloud providers re using dashes in their meta information, we should be able to use them for grouping. Actually, that should not even be the driver. If I feel like naming a group a-b-c-d-e, it should not really be an issue. It is a very useful delimiter.

This thread does not seem to attract any attention from devs or maintainers, though. I think we are speaking to deaf ears.

Devs/Maintainers: Please, pretty please allow dashes in group names!

To clarify some misconceptions, part of them were to due to my mistakes and making initial messages unclear, the latest versions have fixes to some of the issues people keep bringing up here, other fixes are still incomming:

Just to say this once, clearly , YOU WILL ALWAYS BE ABLE TO USE DASHES IN GROUP NAMES also dots and other characters that are now considered 'invalid', just not by default. This 'default' is what is being deprecated, the default will be 'safe' in 2.11, but you will always have an option to 'opt in' to the old behaviour.

And to explain how and why we got here:

First, group names were ALWAYS sanitized, they just had different inconsistent rules, depending on the inventory type you were using, scripts were all over the place, YAML and INI formats did each their own thing. The major change was 'centralizing and normalizing sanitation', this was decided back in 2.4 but was not totally implemented until 2.8. The intent was to provide a norm or baseline all could use safely in Ansilbe, that said we recognize there are many people using 'unsafe' or 'invalid' characters for variables so we made this configurable, not only globally but also by some of the inventory plugins.

The initial implementation had some issues and a lot of discussion (no we don't decide this in hiding, we have public meetings in irc to which you are all welcome, https://github.com/ansible/community/blob/master/meetings/README.md) and a lot of the feedback was incorporated (These are also logged so you can go back to see the discussions and reasoning, but to avoid 'log dumpster diving', I'll explain most of issues below). After going out in 2.8 we got another round of feedback and we have been fixing the some bugs, like always getting a deprecation, not just when using the default and specially with the wording of the documentation and warnings.

  • 'Why Python names?'
    Mostly because Ansible uses Python and JInja (which also uses Python) and some uses of groups (mostly in our early examples, but also plenty of 3rd party ones) can create errors in playbooks, i.e stuff: '{{ groups.gropup-name-with-dash ... }}' or worse a group.name.with.dots. This creates confusion for many users that want to use the Jinja feature of 'dot notation for variable access' and this is why the default should be safe for all users. Most people on this post might disagree with this, but this is a real issue for a lot of people and should not be a 'trap' waiting for new or old Ansible users. Those that 'opt out' are then responsible for avoiding the breaking usage in other parts of Ansible.

  • 'What if I liked that each inventory had different sanitation?'
    Well, you can still turn off the 'central' sanitation and enable the one for your specific inventory source, most popular new inventory plugins that substitute the old scripts have had options added to emulate the script behaviour, worst case scenario, you can still use the inventory scripts .

  • 'Why not hostnames/are hostnames next?'
    Like for groups, sanatization of hostnames has always existed, but has not changed, hostnames have a different requirements, like being DNS resolvable
    for network connections or a valid path for chroot connections. Also, fortunately, there are little to no examples of using hostnames in dot notation, this has not been a common practice and would become a problem if people suddenly started using them, but unlike group names this is something we have avoided until now. If it becomes a problem going forward ... I don't see a good solution either.

Note that this specific ticket (not good enough description/information) is something I'm addressing already, hope to have fix out soon. As for the rest of the discussion, the devs don't use Github as a forum, some tickets devolve into that, the previous ticket that is closed and also has a long thread was ignored until recently, mostly because devs filter out closed issues and expect discussions in IRC the mailing list or new issues.

I hope this addresses all the major issues, as always we are open for discussion, feel free to drop by ML or IRC, we just avoid using github since it is not a good place for such things.

Thanks a lot for the clarification.

Appreciate you taking the time to explain even though it would have been far simpler to stop supporting dot notation and deprecate that support over a few releases. Fewer people use that vs the amount of people who have invalid chars in their group names. Se la vie

@skyscooby the problem is that it is not Ansible doing that, it is Jinja.

Just to say this once, clearly , YOU WILL ALWAYS BE ABLE TO USE DASHES IN GROUP NAMES also dots and other characters that are now considered 'invalid', just not by default.

OK, this is good to know, thanks for the clarification. However, the user experience really needs to be improved. You have the "curse of knowledge". Just try to imagine yourselves in the shoes of a user who sees this:

[DEPRECATION WARNING]: The TRANSFORM_INVALID_GROUP_CHARS settings is set to allow bad characters in group names by default, this will change, but still be
user configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
[WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

That is a long, long way from

[DEPRECATION WARNING] Group name 'my-servers' contains '-', which will become invalid by default from Ansible 2.11. Set force_valid_group_names in ansible.cfg or the ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS environment variable to "ignore" to suppress this. See https://docs.ansible.com/something for more information.

... which is what I, as a user, would have really wanted to see. It would have saved me over an hour. Now multiply by that by the number of people who have or will run into this issue.

Having dashes and dots be invalid in group-names is not a sensible default. People will always have them in their group names. To require them to set yet another variable in a config file to enable sensible behaviour is IMHO indefensible.

Thanks @bcoca for your comment above. It's much appreciated.

Although I'm not happy with the decision I understand that a discussion took place and a decision was made. If the decision is still open for debate it should be continued at the mailling list or on irc but may not capture this issue.

For the topic of this issue I would like to find the following information in the official documentation and the porting guides, to be aware of this change.

  • What are group names that are valid by default?
  • What to do to keep using group names including dashes, hyphens, dots and colons?

Because we use dashes in all our group and host names and we won't change that. So I have to opt-in and change my ansible.cfg every time I setup a new installation/enviroment. That's unfortunate to me but I have to deal with it somehow. The least I would expect is that this is documented in accordingly.

To continue the discussion whether this change is wise or not, I opened a thread in the Ansible Development group.

Best regards,
Tronde

I want to thanks to all contributors on this issue. Based on what I read here I decided to write a blog post https://docs.sbarnea.com/ansible/naming-hosts-and-groups -- Hopefully it would summarize what users need to do.

@loop-evgeny I agree that we as the core team do have the "curse of knowledge" and it inhibits us from creating docs and errors that are useful to everyone. We also completely rely on the community to help us shape ansible and keep it simple for as many users as possible, so when folks have recommendations on improving our docs and our error/warning messages, I always encourage them to help us out by sending a pullrequest. The message you point out is held in the following file and we would greatly appreciate it if you could send us a PR with the suggested change ...

https://github.com/ansible/ansible/blob/4ef2545eb5d661566e06629015967c2d1b8924e3/lib/ansible/inventory/group.py#L54-L55

@jctanner Ordinarily I'd be happy to submit a PR to improve a free and useful program that I use. However, the general attitude of the Ansible developers towards usability, their eagerness to close as "working as intended" issues that I consider to be self-evident bugs (even if design bugs) and the fact that Ansible currently has 2025 (that's two thousand!) open PRs gives me very little confidence that my work will not go to waste. If you really want to "rely on the community", as you say, then a substantial culture change is needed IMHO.

Hmm.. That chance hit me too.

Unfortunately we use network names as group names and that is not easily changable. I would love to opt out for the dot syntax sugar for group names as that is nothing I ever used (Although I use it with other variables).

It would be preferable to use ansible-playbook whatever.yaml -l some.network.to.use in future. Using any other than the network as group name would reduce the usecase massively.

Hi,
I'm somewhat confused at the moment. Could somebody tell me what I have to set in ansible.cfg to allow invalid chars in group names in the future, please?

force_valid_group_names = ignore

What is the future version of ansible regrading to this issues? Will at some time ansible will reject all dashes in group name without a work around using force_valid_group_names ? (without hearing feedback from users who has suffered by the change, and who has never suffered the issues by using hyphen in group names)

Sorry Just read the comments from @bcoca and happy to see that I would be able to use hyphen if I want to in the future - that is enough for me.

Hi,
I see the same warning, but, I don't understand what I should change, and if we should change it.
Is it something related to python?
Howto resolve?

If I ignore with force_valid_group_names = ignore, it would be with that necessary, and when I upgrade to Ansible >= 2.10?

Regards,
Cesar Jorge

If i understand this correctly. The only thing that is deprecated is the automatic transformation of group names. This means it should be totally fine to set force_valid_group_names = ignore after 2.10 and beyond.

It should also be totally fine to continue to use dashes and whatever you want in group names. What Ansible won't do in the future is to sanatize them so you can use the dotted notation even for "invalid" group names. For example:

Your inventory contains a group named foo-bar.xyz. Now you want to write a template that creates a list of hosts that belong to that group:

{% for host in groups['foo-bar.xyz'] %}
{{ host }}
{% endfor %}

Mind that the following version of the template would not work:

{% for host in groups.foo-bar.xyz %}
{{ host }}
{% endfor %}

This is because the - and the . have special meaning in this case. The dotted notation however would be totally fine if your group would have the name foo_bar_xyz because the template then becomes:

{% for host in groups.foo_bar_xyz %}
{{ host }}
{% endfor %}

Which is of course totally fine.

In an attempt to make things easier for users, Ansible apparently always did some sanitation for group names. This means it was (and until 2.10 still is) possible to use foo_bar_xyz in the above examples even though the group actually is called foo-bar.xyz. Personally i don't think this makes things easier at all and it seems that the core team now also agrees with that.
So their next attempt to tackle the issue is to make it impossible to have "invalid" group names in the first place. However as far as i understand it will always be possible to opt-out of this limitation by setting force_valid_group_names = ignore.

Long story short it is actually two different changes that have been intertwined[1] with each other. Whence the confusing name and wording of the warning.

Again this is only how i understand the issue. Please correct me if i'm wrong!

[1] for further details see RFC1925 Paragraph 2, Point (5)

I just wanted to leave my 2¢ since I'm firmly on the side of dashes > underscores. Just doing a quick Google search on the matter, almost universally underscores are labeled as inferior to dashes for any kind of pathing and labeling purposes. They are also more difficult to work with in many text editors and filesystems.

Even if that were not the case, the fact that I see dashes used for things like server labels and groups in the wild a lot more often than underscores means this will be yet another thing I'll have to make sure to add to all my and my clients' ansible.cfg files (I tend to have one per playbook).

I have no problem with Ansible trying to force a stricter default where it improves the experience, but first you came for the dashes in my role names (and sometimes allowed singular exceptions for older roles that were grandfathered in), then you came for dashes in my collections (they are not allowed in any way, shape or form), and now you've come for dashes in my inventory!

It's a war against dashes out there... and I want to draw a line somewhere—in this case, it's the one place where it's actually impossible for me to prevent people from using dashes, as many of the dynamic inventory providers create groups based on server names and labels, and many (if not most) organizations seem to label things using dashes (e.g. us-east-1a, not us_east_1a).

It's not fun having a default that almost always must be overridden to make software work, but it sounds like as of Ansible 2.11, that's going to be the case.

If it's all because some users unfamiliar with Jinja2 and Python don't realize something.with-some-dashes is invalid, I would argue it's better to teach them "if there are dashes, you should use the bracket notation for dict access, e.g. something['with-some-dashes']. You can even intermingle the two if needed. It's not super pure and holistic, but we're not all Rust developers here...

Very well put, Jeff. I strongly agree with you here - this change will be so disruptive, and rather than just requiring a one-off migration, will change the workflow of a huge number of users. Ansible will no longer work out of the box.

Hostnames cannot include an underscore, so in a sane world, inventory_hostname would not be forced to. This means that our inventories will now look quite inconsistent, with hostnames that cannot contain underscores, and groups that cannot contain hyphens.

Please, just don't switch the default.

https://en.m.wikipedia.org/wiki/Hostname

Hi,
I totally agree with Jeff here.

But as @bcoca stated above most of the developers don't look at these discussions here on a regular basis and this issue may not be the right place to discuss the change because it is about the correct documentation.

For discussion please join the thread Is changing the default setting of TRANSFORM_INVALID_GROUP_CHARS a good idea? in Google Groups.

Great points Jeff.

It's not fun having a default that almost always must be overridden to make software work, but it sounds like as of Ansible 2.11, that's going to be the case.

This is the big takeaway for me from all these discussions. I understand the problem that needs to be solved, but the solution seems to be the opposite of what is needed. It makes things easier for support but hard for the users - that is a backwards solution.

If it's all because some users unfamiliar with Jinja2 and Python don't realize something.with-some-dashes is invalid, I would argue it's better to teach them "if there are dashes, you should use the bracket notation for dict access, e.g. something['with-some-dashes']. You can even intermingle the two if needed.

This here is the best solution, not breaking things that have been used for years.

Great comment from @geerlingguy - spot on!

I'd like to add that as a user of Ansible, why should I need to know what valid Python syntax is? Having used Ansible for a long time now, I understand that Ansible (and its modules) is written in Python, but why should I need to care about that? Exposing that fact to the end user is just bad design.

Similar to only allowing valid JavaScript/Ruby/.NET/whatever notation for things like usernames in a web application. Why would the end user care which language the application is written in?

In addition to that, introducing breaking changes is a difficult topic, I try to avoid that if possible. If I have to make a change, I typically leave in the old, existing behavior as the default, and let people opt-in for the new behavior. Why wasn't this done here? Why do I have to either change my configuration, or worse my whole inventory? Why not the other way around?

If a system requires strictly compliant tokens internally, then the system should internally generate the tokens and create a lookup table that associates the internal tokens with the user data. This way Ansible can change its token rules as needed and limit the impact on the users. Users ought to be able to name their inventory, roles, etc as they or their customers see fit.

It seems to me like this change may have the opposite of its intended effect (to decrease support queries):

Now there is no delimiter supported (by default) both by hostnames (which should be DNS resolvable, i.e., not contain underscores) and group names (which should not contain dashes).

Definitely should be free to name any hosts

El mié., 14 ago. 2019 16:16, Christian Pointner notifications@github.com
escribió:

If i understand this
https://github.com/ansible/ansible/issues/56930#issuecomment-516863432
correctly. The only thing that is deprecated is the automatic
transformation of group names. This means it should be totally fine to set force_valid_group_names
= ignore after 2.10 and beyond.

It should also be totally fine to continue to use dashes and whatever you
wan't in group names. What Ansible won't do in the future is to sanatize
them so you can use the dotted notation even for "invalid" group names. For
example:

Your inventory contains a group named foo-bar.xyz. Now you want to write
a template that creates a list of hosts that belong to that group:

{% for host in groups['foo-bar.xyz'] %}
{{ host }}
{% endfor %}

Mind that this version of the template would not work:

{% for host in groups.foo-bar.xyz %}
{{ host }}
{% endfor %}

This is because the - and the . have special meaning in this case. The
dotted notation however would be totally fine if your group would have the
name foo_bar_xyz because the template then becomes:

{% for host in groups.foo_bar_xyz %}
{{ host }}
{% endfor %}

Which is of course totally fine.

In an attempt to make things easier for users, Ansible apparently always
did some sanitation for group names. This means it was (and until 2.10
still is) possible to use foo_bar_xyz in the above examples even though
the group actually is called foo-bar.xyz. Personally i don't think this
makes things easier at all and it seems that the core team now also agrees
with that.
So their next attempt to tackle the issue is to make it impossible to have
"invalid" group names in the first place. However as far as i understand it
will always be possible to opt-out of this limitation by setting force_valid_group_names
= ignore.

Long story short it is actually two different changes that have been
intertwined[1] with each other. Whence the confusing name and wording of
the warning.

Again this is only how i understand the issue. Please correct me if i'm
wrong!

[1] for further details see RFC1925
http://www.faqs.org/rfcs/rfc1925.html Paragraph 2, Point (5)


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ansible/ansible/issues/56930?email_source=notifications&email_token=AA5N2CJCIELW7JWHC6OJ35DQEQHSPA5CNFSM4HPRGLKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4I5U3Y#issuecomment-521263727,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA5N2CIVT5PLD2QCAGGBK6LQEQHSPANCNFSM4HPRGLKA
.

This is truly a wrong decision in my honest opinion. And for the wrong reason. Reducing the number of support requests, really?

Ansible, as a tool, should not impose language specific details to the end user. I'm already so upset with Terraform imposing me all that Golang down the throat, now the same is happening here with Ansible and the "pythonic" style. Don't get me wrong, I work quite well with both Go and Python, but when it comes to infrastructure as code, why should I care? And what happened with the promise of having YAML dictating the shape of the code base to manage? "Infrastructure as data, that you can read and run", I heard that so many times... As far I know, YAML doesn't care at all with both dashes and underscores.

BTW, there are quite a lot of things out there not supporting underscores. Hostnames, AWS regions and IDs for literally everything, just to mention some really important ones. Good luck with maintaining all the exceptions where the transformation is not supposed to happen...

To people coming here just looking for a quick solution on how to make this go away, just add the line force_valid_group_names = ignore to your ansible.cfg and be happy.

My understanding is you can't use the dot notation for variables with spaces anyways, and while I never create variables with spaces there are unfortunately lots of vendors that return dictionary keys with spaces through json API responses. The sensible option to me seems to switch to the square bracket notation. Hopefully setting force_valid_group_names to ignore doesn't cause ill effects further down the road, who knows what else is planned in the future with this change.

This is a pretty dreadful decision particularly when it comes to working with dynamic inventories as with Openstack (and AWS).
Instance names and metadata keys containing "forbidden characters" are often returned as inventory items and/or group variables from the underlying cloud. This is going to make life hell for a lot of Openstack (and AWS) admins trying to manage their fleets using meta tags and or instance ID's like:
instance-8ca09c33-f255-440f-9544-b0ab318c79d9
meta-os_ubuntu

Ansible devs should be taking @geerlingguy opinion seriously. He is one of the biggest contributors to Ansible Galaxy and his Roles are consumed by tons of people. I think this change is really bad for people who have thousands of hosts named like: $env-$role-[0..99]. Are we supposed to go rename everything to appease our Ansible overlords?

@ssbarnea For one thing, we are making a push to only allow variable names, and other similar keys, that are valid python identifiers. To explain a little further about group names, it causes a problem for users trying to use "dot syntax" such as groups.foo-group, which doesn't do what the user expects. The number of issues and support requests caused by small problems like these has caused us to go down a path to safe guard names to ensure that problems like this do not occur.

For those wanting to keep what we consider invalid characters, can opt out of this functionality.

How long will users be allowed to opt out of this behaviour? Will there be a permanent configuration option to disable this behaviour in all ansible versions moving forward, or will it only be supported until 2.11? I'd be happy for the option to be turned on by default, as long as I always have the option to turn it off.

If this becomes a hard restriction in 2.11+, then you're likely going to find yourself losing customers that are bound by the constraints of their organisation (not all ansible users have the power to dictate the naming conventions used by their company). It seems this change also would introduce a significant challenge for those using ansible to manage cloud infrastructure, where dashes tend to be heavily used.

Just to remind those who haven't read the whole thread here. There is also a thread on the devel mailling list: https://groups.google.com/forum/#!topic/ansible-devel/bjAcM9ferIw

IMHO this change was a really poor choice. Codebreaking syntax changes in minor release versions holds us back from extending the use of Ansible in our envirionment. Because I won't be able to update Ansible when it breaks the playbooks of my users.

But as @bcoca stated above most of the developers don't look at these discussions here on a regular basis and this issue may not be the right place to discuss the change because it is about the correct documentation.

@Tronde : one would argue that contributors AND customers are consulted before stories are written to understand the impact and gather feedback well before someone codes a solution. As several here have mentioned, this is a product mgmt failing we've seen more than once.

As an example of the situation @andyfeller describes about this change:

We have a problem with this on our site.

We use Red Hat Identity Manager as an external inventory, we do not control it, it contains many host groups with dashes instead of underscores. This will not be changed (because of all the other things that exist using those names).

So, we need:

  • To configure Ansible to maintain the current behaviour
  • Silence the deprecation warning
  • Do this for command line Ansible and Ansible Tower

FYI PR https://github.com/ansible/ansible/pull/66650 (please no pitchforks there) is slated for 2.10 (as of the current moment), meaning anyone who currently sees this warning would (once they upgrade to 2.10, again assuming that PR is merged) start having playbook failures instead (until they set force_valid_group_names = ignore in an ansible.cfg).

Just posting for visibility. I'm still firmly behind my earlier assertion that this is a user-hostile default, as there are still many dynamic inventory scripts (some part of Ansible itself or now moved into 'official-ish' collections) that generate group names with dashes or other valid DNS characters.

Practically anyone who uses Ansible with AWS is going to have to override the default.

@geerlingguy Is that the right PR #? Looks like that points to this issue.

FYI this was discussed at a Core Meeting here, starting at 19:06:55

@apple4ever oops, updated the link, it's https://github.com/ansible/ansible/pull/66650

So i've seen above many comments of things that were already answered/debunked/etc, so just going to link here my previous post .

https://github.com/ansible/ansible/issues/56930#issuecomment-516863432

Please don't add new posts that don't add NEW items for discussion as they hide the previous ones that were already answered.

By the way, where would be a good place to link to in the Python documentation about how valid variable names look like? There's https://docs.python.org/3/reference/lexical_analysis.html#grammar-token-identifier, but that's not really user friendly or readable for people without Computer Science backgrounds.

The reason for asking is that I'm not sure if the actual initial complaint has actually been dealt with. There's just a warning that there's something wrong, but it takes a lot of digging to find out what exactly and how one - if willing or able - could actually choose a valid group name. I would expect at least a clear "The group name foo-bar contains invalid characters (-). Valid group names should be valid Python identifiers (see https://docs.python.org/??? for more information)" message, not just "there are bad characters, check again with -vvvv to actually find out which ones!". Ideally this would also mention that this can be disabled, but could lead to other unexpected problems (like making it really hard for Ansible to distinguish the groups foo-bar, foo.bar and foo_bar).

Currently it's more of an "You did something wrong, fix it" message with no clear way laid out how to proceed which might also have contributed to the strong response here.

@geerlingguy wrote in comment 56930:

(until they set force_valid_group_names = false in an ansible.cfg)

The documentation does not mention 'false' as a valid value for this key. I've set the value to 'ignore' which should do the trick. But is 'false' an invalid keyword or is it correct and the documentation is not complete here?

@bcoca in a previous comment:

Just to say this once, clearly , YOU WILL ALWAYS BE ABLE TO USE DASHES IN GROUP NAMES also dots and other characters that are now considered 'invalid', just not by default. This 'default' is what is being deprecated, the default will be 'safe' in 2.11, but you will always have an option to 'opt in' to the old behaviour.

You've stated repeatedly that the current behaviour can be preserved, but what are the exact ansible.cfg settings required to do this _now_ and squash the deprecation warning.

I've tried as @geerlingguy wrote in comment 56930:

(until they set force_valid_group_names = false in an ansible.cfg)

Which causes my playbooks to fail when it can't find hosts or groups with hyphens (they're coming in from an inventory plugin we wrote BTW, do theses have to do the transform too, or is that done when Ansible ingests the inventory from the plugin?)

I've tried as @geerlingguy wrote in comment 56930:

(until they set force_valid_group_names = false in an ansible.cfg)

Which causes my playbooks to fail when it can't find hosts or groups with hyphens (they're coming in from an inventory plugin we wrote BTW, do theses have to do the transform too, or is that done when Ansible ingests the inventory from the plugin?)

This was mentioned in several comments and is in the documentation. You should use never or ignore.

So are we just not supposed to use the EC2 dynamic inventory script anymore since it groups everything by 'us-east-1', 'us-east-2', etc? Or is there a plan to update it? I just went to Ansible's documentation for the EC2 dynamic inventory script and the link to download it on Github doesn't work anymore, so that's interesting.

I just went to Ansible's documentation for the EC2 dynamic inventory script and the link to download it on Github doesn't work anymore, so that's interesting.

See https://github.com/ansible/ansible/issues/68419

for those of you who don't bother to read the IRC log, here is the decision, i.e. no decision:

19:15:40 <sivel> I've got to say, that brining this topic up all the time isn't a good use of time
19:15:52 <cyberpear> bcoca nominated it
19:16:07 <felixfontein> I think the aim was to solve this once and for all (like, again :) )
19:16:29 <cyberpear> since bcoca is not here, move on to next topic?
19:16:34 <sivel> honestly, I don't think this is going to be the right forum to make a decision on this
19:16:45 <jillr> +2 moving on
19:16:47 <cyberpear> sivel: what's the correct forum?
19:16:55 <felixfontein> sivel: what is the right forum for making that decision?
19:17:02 <cyberpear> "declaration from Red Hat On High"?
19:17:15 <sivel> I'm going to abstain on that, but this project is not a democracy
19:17:16 <cyberpear> -1 to "declaration from Red Hat On High"
19:17:24 <sivel> too many cooks in the kitchen distract
19:17:45 <sivel> We know the arguments at this point
19:17:59 <sivel> anywho, next topic

yes, someone wrote "feel free to drop by ML or IRC". no, "this project is not a democracy".

yes, someone wrote "feel free to drop by ML or IRC". no, "this project is not a democracy".

To be honest this is wrong with opensource - If it leads to a not popular way - people can fork it, can it be forked?

I can see accepting PR in ansible is terribly slow. Patches looks obviously needed and simple change but it never gets in. Luckily ansible itself is flexible to allow people to use custom plugins however it seems stale will make myself lesser in contributions - or even more hassle to do so.

Feeling a bit sad, really ...

@sunshine69 I feel your pain. But that is a discussion that should take place on IRC or the Google Group for Ansible Development.

This issue is not the right place for it. Because to few people read here.

@sunshine69 I feel your pain. But that is a discussion that should take place on IRC or the Google Group for Ansible Development.

This issue is not the right place for it. Because to few people read here.

While the discussion might be more productive in those other channels, the transparency is appreciated for people following this issue specifically. IRC isn’t everyone’s preference after all.

FYI: Remove deprecation for TRANSFORM_INVALID_GROUP_CHARS was just merged yesterday. There are backport PRs for 2.9 (https://github.com/ansible/ansible/pull/69487) and 2.8 (https://github.com/ansible/ansible/pull/69488) to remove the deprecation warnings there.

Files identified in the description:

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

When I set force_valid_group_names = ignore the WARNING went away, but the DEPRECATION NOTICE did not go away.

Finally I found in the docs: force_valid_group_names = silently which will do the replacement and won't clog up the output - if that's what you're looking to do.

Nonetheless, this whole issue could have been avoided in the first place if pointless changes like this weren't made in the first place.

@emmm-dee - For that particular problem, I opened https://github.com/ansible/ansible/issues/70908 — note that this issue still persists, as there is still no official documentation for what _are_ 'valid' group chars.

thanks to @geerlingguy for your actions! You are the one who is making ansible better.

I am working for our application for bounce (start/stop) but i am not connected with application host.

I tried ping command, which you sent and it works...

[webadmin@vlodjumpts00 ~]$ ping 8.8.8.8

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

64 bytes from 8.8.8.8: icmp_seq=1 ttl=112 time=10.6 ms

[webadmin@vlodjumpts00 ~]$ mirrorlist.centos.org

-bash: mirrorlist.centos.org: command not found

I want to use this for our organization.. if i ran "ansible all -m ping" command. facing error, below is details:

[aa63457@vlodjumpts00 bin]$ ansible all -m ping

change, but still be user configurable on deprecation. This feature will be removed in version 2.10. Deprecation warnings can be

disabled by setting deprecation_warnings=False in ansible.cfg.

RTE3EPAdmin | UNREACHABLE! => {

"changed": false,

"msg": "Failed to connect to the host via ssh: ###############################################################################\n# CenturyLink computers and the CenturyLink computer network are CenturyLink  #\n# property. Only authorized persons may use them and only for legal and proper#\n# purposes as determined solely by CenturyLink. You consent to the monitoring #\n# of their use. You must use CenturyLink computers and the network in         #\n# accordance with the CenturyLink Code of Conduct, subject to discipline for  #\n# misuse. Customer use is governed by the CenturyLink Acceptable Use Policy.  #\n###############################################################################\nUse CTL credentials (login/password) on this server.\nAUTH-NOTICE:\nAUTH-NOTICE: Use your cuid as your username\nAUTH-NOTICE:\nPermission denied (publickey,password).",

"unreachable": true

}

localhost | SUCCESS => {

"ansible_facts": {

    "discovered_interpreter_python": "/usr/bin/python"

},

"changed": false,

"ping": "pong"

}

Please help me... what i need to do this. Actually, we don't have UN/PWD for hosts file for connecting host machine..

localhost ansible_connection=local

[RTE3VFO]

RTE3VFOAdmin ansible_host=vlddwblasts001.test.intranet

RTE3VFOManaged ansible_host=vlddwblasts002.test.intranet

[RTE3EP]

RTE3EPAdmin ansible_host=vlddwblasts002.test.intranet

RTE3EPManaged ansible_host=vlddwblasts003.test.intranet

[RTE3RES]

RTE3RESAdmin ansible_host=vlddwblasts003.test.intranet

RTE3RESAManaged ansible_host=vlddwblasts004.test.intranet

[RTE3ORCH]

RTE3ORCHAdmin ansible_host=vlddwblasts004.test.intranet

RTE3ORCHManaged ansible_host=vlddwblasts005.test.intranet

[RTE3EASE]

RTE3EASEAdmin ansible_host=vlddwblasts005.test.intranet

RTE3EASEManaged ansible_host=vlddwblasts006.test.intranet

[RTE3RTS]

RTE3RTSAdmin ansibke_host=vlddwblasts006.test.intranet

[EASE-ASR-Test2:children]

RTE3VFO

RTE3EP

RTE3RES

RTE3ORCH

RTE3EASE

RTE3RTS

and the directory structure is:

[webadmin@vlodjumpts00 ansible]$ pwd

/etc/ansible

[webadmin@vlodjumpts00 ansible]$ ll

total 84

-rw------- 1 webadmin webadmin 607 Jul 12 2017 1

-rw-r--r-- 1 webadmin webadmin 17910 Sep 19 09:55 ansible.cfg

-rw-r--r-- 1 root root 19985 Dec 8 2019 ansible.cfg.rpmnew

-rw------- 1 webadmin webadmin 213 Jul 3 2017 easeasr-rte2-ease.yml

-rwxr-xr-x 1 webadmin webadmin 1034 Sep 19 09:16 ease-hosts

-rwxr-xr-x 1 webadmin webadmin 1647 Sep 19 10:50 hosts

-rw------- 1 webadmin webadmin 2679 Jul 3 2017 hosts.bkp

-rw------- 1 webadmin webadmin 273 Jul 6 2017 lineinsfile_tst.yml

drwx------ 4 webadmin webadmin 4096 Nov 2 2017 playbooks

drwxr-xr-x 3 root root 19 Dec 8 2019 roles

-rwxr-xr-x 1 webadmin webadmin 7321 Nov 2 2017 servmix_hosts

-rw------- 1 webadmin webadmin 208 Sep 19 10:55 test.yml

-rw------- 1 webadmin webadmin 122 Sep 19 10:54 vars.yaml


We are not connected directly to host... fist login our jump server and than ssh host...

jump server is "vmdcltctws217" port using =22, connection type=ssh

and then enter with our UN/PWD

after that we did sudo for connection to host server..

sudo su - easesqa

and then ssh host server like..

vlddwblasts001.test.intranet

then we run start/stop command from here..

Please help me, what can i do it for?

Was this page helpful?
0 / 5 - 0 ratings

Related issues

tspivey picture tspivey  ·  3Comments

hryamzik picture hryamzik  ·  3Comments

rokka-n picture rokka-n  ·  3Comments

renaudguerin picture renaudguerin  ·  3Comments

eparis picture eparis  ·  3Comments