Ninja: Document how to set environment variables in subprocesses

Reading through that discussion, it seems that environment variables aren't supported in Ninja because CreateProcess doesn't support setting PATH to find the executable to run when using lpCommandLine instead of lpApplicationName? The thing is, execvpe and execle have the same restriction, so I don't understand what the issue is here.

It's really simple to set the environment inherited by a process on all platforms, so I'd like to understand what the blocker is here. Cheers!

FWIW, we would really make good use of this feature in Meson: https://github.com/mesonbuild/meson/issues/266, https://github.com/mesonbuild/meson/issues/384, etc.

Would the project be interested in a PR that implements this? It's a troublesome blocker downstream for https://github.com/mesonbuild/meson/ since we have to add wrapper scripts for any command that needs environment variables set, which is slowly turning out to be quite a few.

Even Makefiles and Visual Studio project files support this, so Ninja stands out in not supporting this. :)

We'd be happy to implement this though if there is interest in having this feature in Ninja.

On POSIXy platforms, you can do

rule foo
  command = ENV1=env1 ENV2=env2 my_command

or

rule foo
  command = $env my_command
foo out: in
  env = ENV1=env1 ENV2=env2

Is that not enough for you? Most commands shouldn't need env vars, and generally supporting less stuff keeps ninja simple.

Yes, that part is documented, but Meson is a cross-platform build system (meant to have feature-parity across all supported platforms) and we currently have to use Python script wrappers whenever we need to set environment variables or have arguments with special characters (especially newlines).

At first, we also thought that setting env vars is a niche use-case and pushed back when users asked for it, but a surprisingly large number of people need it. For instance, gobject-introspection in GNOME requires you to set CC/CXX/etc while invoking its tools, otherwise it uses an auto-detection mechanism that ends up using the wrong compiler. "Custom targets" in Meson can run arbitrary commands, and people often show up asking how they can set environment variables while running some tool that they can't change the behaviour of.

The Python script workaround is ok, but invoking the python interpreter for each command really slows things down sometimes.

Maybe gobject-introspection could accept those via command line args instead? (In my experience, builds that rely on env vars instead of flags tend to be much more vulnerable to devs screwing up their local env, etc.)

MSVC's cl.exe requires a few env vars to find includes and whatnot; for that ninja has ninja -t msvc which has a -e flag that can load an environment from a file. So meson could theoretically use that on Windows, and the posix stuff elsewhere.

But using python wrappers for the rare, "needs to be flexible" case and making the common commands work without reliance on the env seems like the best approach to me.

I think I've run into this issue. I'm trying to build a FIPS executable on window, and need to follow these directions:

https://www.openssl.org/docs/fips/UserGuide-2.0.pdf, section 5.3.2:

For the Windows®
 environment a perl script fipslink.pl is provided which performs a
function similar to fipsld for Unix®
/Linux®
. Several environment variables need to be set:
FIPS_LINK is the linker name, normally “link”
FIPS_CC is the C compiler name, normally “cl”
FIPS_CC_ARGS is a string of C compiler arguments for compiling fips_premain.c
PREMAIN_DSO_EXE should be set to the path to fips_premain_dso.exe if a DLL is
being linked (can be omitted otherwise)
PREMAIN_SHA1_EXE is the full path to fips_standalone_sha1.exe
FIPS_TARGET is the path of the target executable or DLL file

Even though FIPS_CC is defined in my environment, it looks like ninja is not passing the env to the child process (perl fipslink.pl ...). I'll see if -e works for me.

Another use-case for this is setting LD_LIBRARY_PATH before running executables built in the project. This is often necessary with executables linked to shared libraries built in the project to ensure that already-existing versions in a custom prefix are not used (-Wl,-rpath doesn't work on distros that set DT_RUNPATH instead of DT_RPATH such as Debian and Ubuntu). Autotools (libtool) uses shell scripts for this. It would be a significant slowdown of the build if we had to spawn a Python interpreter or run a shell script for such cases.

As another datapoint, cmake came across this use-case so much that they have a C wrapper for it which sets the environment and then calls the process. They need it anyway for their make backend, but ninja-only build systems would greatly benefit from this feature.

I'm not really sure what this bug is discussing anymore, since it looks like it was originally about updating the docs. :)

For LD_LIBRARY_PATH, setting it via the command line already works as Nico mentioned above. It doesn't involve another process; we must use a shell to parse the command line anyway. (Subprocesses on Linux are oh-so-cheap anyway; you can't even measure /bin/sh -c "echo hello" because it is so fast.)

On Windows we have ninja -t msvc. You might try writing your meson feature using that and then once we have some experience with that we could figure out how to improve it.

Kconfiglib has a hard $srctree requirement for out of source builds. It uses a few other variables:
https://github.com/ulfalizer/Kconfiglib/blob/424d0d38e7/kconfiglib.py#L108

we currently have to use Python script wrappers whenever we need to set environment variables

From: https://github.com/ulfalizer/Kconfiglib/tree/424d0d38e7be15c5#overview

The entire library is contained in kconfiglib.py. The bundled scripts are implemented on top of it. Implementing your own scripts should be relatively easy, if needed.