I would attempt a PR for this, but I'm not entirely sure this is a bug or if it's intended.
When including a subninja, I get
ninja: error: 'mod.c', needed by 'mod.o', missing and no known rule to make it
although running ninja directly in that directory works just fine. Under the suspicion that relative paths may be to blame, I copied the project structure to a new directory, prepended each input/output with the absolute path, and ran ninja from the parent directory (containing the build.ninja
that subninja
'd the module). It built just fine.
Is this suggesting we should be generating our ninja configurations with absolute paths? This has a lot of problems, along with the fact not every generator I've seen does this. Further, the documentation doesn't make a distinction, and Ninja runs just fine otherwise.
I want to err on the side of _it's a bug_, but I feel like a PR that does runtime path canonicalization might introduce a performance hit (though I'm waving my hands here without any benchmarks to back up that suspicion).
All paths are relative to the build directory, not to the file containing the subninja line.
That makes no sense, though. That means generators are going to have to _all_ implement some way to prepend a prefix to the files, which means changing the working directory commands run in (which may change the original intent of the build configuration depending on how the generator/commands run).
What, then, are contrasting use cases for subninja
vs include
?
A subninja with paths relative to the build file would be useful in that depdencies that are configured to run within their own directory can do so without having to modify their paths, but can still contribute to the dependency graph of the parent ninja configuration.
include
functionality would be unmodified, and would act exactly how subninjas act other than the fact rule names would now be in a combined namespace (as per #921). Currently, subninja
and include
achieve essentially the same thing other than scoping variables and rule names...
The idea is that the generator generates all .ninja files, so they can write paths relative to the build directory. It's an interesting idea to combine ninja files generated by different generators (it sounds like that's what you want to do?), but that's not something that's supported at the moment.
Correct, the difference between subninja
and include
is that the former adds a scope and the latter doesn't. The use case for this is that the toplevel ninja can define common build rules such as cc that reference variables such as cflags
, and each subninja can set cflags
to what's appropriate for that target, and there can be one-off actions in there. To see an example, you can run python misc/write_fake_manifests.py /tmp/foo` to write a bunch of .ninja files to /tmp/foo that use this pattern.
That makes sense, though I still don't see much of a benefit (other than scope).
It's an interesting idea to combine ninja files generated by different generators (it sounds like that's what you want to do?)
Exactly. Being able to include Ninja itself as a submodule for my generator, and then another CMake project (which is configured to output Ninja build files), and then generating the Ninja config files for both and including them as subninja
s in _my_ project's build.ninja
file in order to be able to build them as if they were on their own, but still allow my project to use their outputs (and thus their dependency graphs) in order to build the whole project at once.
If that makes sense. Immediate use-case that I see with my generator in particular is that it's borrowing a few concepts from Tup (which IIRC influenced some design decisions within Ninja itself) in that I can include N subprojects, all with their own build.ninja
files, and then tap into their graphs to allow me to automatically construct a much larger dependency graph.
I personally think that'd make subninja
a whole lot more useful, though I could see it being a potentially breaking change. However, I don't see a way around this unless 1) I modify the dependencies' build.ninja
files with a patch or 2) sacrifice the ability to harness the dependencies' dependency graphs.
Thoughts?
How about:
subninja path/to/build.ninja relative path/to
and an absent relative
defaults to ./
. That would make it non-breaking but still give the functionality if a generator so desires.
Or, following in the steps of other Ninja constructs, maybe
subninja path/to/build.ninja
relative = path/to
Suppose you have a project at .../foo and it has a subdirectory bar, and that Ninja had the relative-path logic you suggest.
If your build system wants to write all build outputs to /foo/obj, a subninja in /foo/bar that used directory-relative paths would need to know to write its output into ../obj/bar, as that's the path to the file from that subdirectory. So whatever is generating your build.ninja files must already be aware of the global path hierarchy, in which case making all paths relative is effectively the same problem as prepending a bar/ to the paths in the bar/ directory.
Maybe there are enough people who write build output in their source directories that the above doesn't matter, though. I mostly hear from people who want even stronger separation -- like the people who sent patches to Ninja to make it so they can build Ninja with the build output in a totally unrelated directory.
I'm not sure I understand. Subninjas, in the use-case I've described in particular, don't normally know about the parent ninja's structure (at least, don't need to know). I'm sure someone out there could find a case where they do, though.
The problem I'm facing right now is dependencies (third party libraries, etc.) that don't use my generator (which is 100% of them) currently need to be built using a bootstrapper, and have to be bootstrapped every time they're updated or changed, etc. Most generators I work with have the options to output to a ninja configuration, but they're all configured for that directory alone (i.e. relative to that directory).
Being able to perform selective rebuilds using their configurations and graphs would be huge. Currently, I cannot do this due to the fact subninja
assumes paths relative to the build directory.
Evan: The way I understood this is that you'd create a tree like this:
subbuild1
subbuild2
and since generators usually support putting the build dir in arbitrary places, building project 1 in subbuild1 and project 2 in subbuild 2 should work. Then there's a toplevel ninja file in builddir that Qix- wants to drive building the subprojects.
Unrelated: this relative
feature also has to change the cwd to its argument if any rules depend on the current directory being equal to their build dir (say, if a rule strips built artifacts or something).
Most helpful comment
I'm not sure I understand. Subninjas, in the use-case I've described in particular, don't normally know about the parent ninja's structure (at least, don't need to know). I'm sure someone out there could find a case where they do, though.
The problem I'm facing right now is dependencies (third party libraries, etc.) that don't use my generator (which is 100% of them) currently need to be built using a bootstrapper, and have to be bootstrapped every time they're updated or changed, etc. Most generators I work with have the options to output to a ninja configuration, but they're all configured for that directory alone (i.e. relative to that directory).
Being able to perform selective rebuilds using their configurations and graphs would be huge. Currently, I cannot do this due to the fact
subninja
assumes paths relative to the build directory.