Less.js: data-uri function uses path of data-uri call, not the string with the path to the file in.

I welcome feedback on whether this is a breaking change (warranting a major up) or a bug fix.

FYI: You can elegantly implement this without breaking backwards-compatibility in a very simple way.

Currently the data-uri() function accepts a Quoted tree node holding a string as the file path and resolves it internally as a URL against the location of the file holding the function call. You can overload the data-uri function to also accept a Url tree node holding an actual URL. This way the URL should normalize against the location where the url() CSS function is called and should no longer be normalized internally in the data-uri() Less function.

E.g.

// app/content/button.less
button {
  .background(url("images/btn.jpg"));
}

@rjgotten

For me it looks more like a workaround rather than a fix. At further iteration they _will_ try to avoid verbosity by moving url to the mixin and the problem comes again. The other problem is that taking the original use-case where this issue came from (#2541), it's often to be used like this:

// mixins.less
.background(@image) {
    background-image: data-uri("@{image}.jpg");
}
// app/content/button.less
button {
  .background("images/btn");
}

(E.g. for a font-face mixin as an example it's usually multiple woff, ttf, eot, eot?#iefix etc. extensions appended to the same file name). And this way the earlier url is also impossible.

@seven-phases-max

Then I don't really think there's a suitable solution period. You need some way of determining the context against which a relative URL should resolve. Either you take that context from the file info of the file that defined the string value going into the data-uri() function, or you make the cut-off point more explicit via the url() function and the Url tree node.

If you want to support path variable substitution, then things quickly become tricky because you need to figure out how the various parts of the path need to be normalized.

E.g. , how do you normalize something like:

// mixins.less
.background(@image) {
    background-image: data-uri("../../@{image}.jpg");
}
// app/content/button.less
button {
  .background("../images/btn");
}

How would you combine the relative url resolution and the token substitution-driven combination of those two paths?

I suppose one option is to only resolve against the file defining the string that is being substituted for a replacer token, when the replacer token is at the head of the final URL value going into a url() or data-uri() function and ignore cases such as the above. That seems most logical.

Another solution could be to introduce some path handling functions to join paths or add/remove/edit file extensions (similar to how the unit() function works for dimensions, maybe?) and make things more explicit.

E.g.

// mixins.less
.background(@image) {
    background-image: data-uri(extension(@image, "jpg"));
}
// app/content/button.less
button {
  .background(url("./images/btn"));
}

For your first example, it does not need any special normalization, "../../@{image}.jpg" expands to "../../../images/btn.jpg" just as it's written (and then it's up to data-uri to handle the path).
And the second example is just... stacking a function to workaround a function to workaround a backward-compatibility with... what exactly? _Wrong_ file paths when calling a mixin defined in other file?

After all if it will be supposed to work "as expected" _only_ with .background(url("images/btn.jpg"));, how it's different from simply writing .background(data-uri("images/btn.jpg")); directly with no changes at all? :)

In other words what I mean is that if this is to be fixed then it should be fixed strait with data-uri no matter how breaking it could be. (Honestly I don't know what would be a better strategy: a. wait for more reports/requests for this and then (if there's enough of them) change or b. modify it earlier to minimize possible breaking impact).

_{More over, since url actually suffers from the same issue itself with --relative-urls: on, technically I'd expect the same to be changed same way for url then too (unfortunately this will be even more breaking change). In other words, assuming url and data-uri were initially designed as interchangeable (i.e. data-uri is just a special version of url (and compiles to CSS url in the end)), it would be quite painful to describe why exactly ulr behaves "like this" (with options like that) while data-uri behaves "like that" (with options like this), and to get certain behavior you'll need data-uri(url()) combo, limited to "data-uri inside a mixin and url _out_ of it with --relative-urls: on" <- Bhrrrr... :)}

For me it looks more like a workaround rather than a fix. At further iteration they will try to avoid verbosity by moving url to the mixin and the problem comes again.

Yeah I agree.

I'm slowly trying to build up to putting some more effort into less for a v3 release and just fixing this.

I was thinking of working out all the possible file paths and trying them one by one - though thats a bit nasty if there are multiple files at different locations... I think I agree its not possible to completely fix this forever, but at least we can fix the normal case.

Possibly a resolve() function might help to pass resolved url's to functions (but then if its not a complete path that wouldn't work and a complete path would work when this was fixed...)

Sorry just quickly jotting down thoughts.

This is just a quicknote, but we will have very same problem with import using variable interpolation.

import using variable interpolation.

That is both Interesting and unsettling.
Is it an actually supported use-case or a happy coincidence?

@rjgotten It is supported, however support is somewhat limited. It was tracked in #410

This works:

@variable: "path.less";
@import "@{variable}";

This does not:

.mixin(@variable) {
  @import "@{variable}";
}

Edit: modified to make link work.

I'm not sure I want to support variable @import's in mixins.
The whole variable import thing is a little bit magic and can create some counter-intuitive code.. so I'd rather discourage that.

A lot of discussion about workaround, maybe just fix the actual problem? Isn't it clear that data-uri should be relative to file being processed.

Right now I have file which imports _a reference_ from another path, and it still complains about data-uri. At least that must be a bug? I mean if I import by reference it should not try to rewrite the paths to relative to current file.

@Ciantic

I mean if I import by reference it should not try to rewrite the paths to relative to current file.

Based on what exactly? What does it have to do with reference?

Right now I have file which imports a reference from another path, and it still complains about data-uri.

It sounds like the opposite to the issue above. Could you provide more details? (e.g. paths of the importing, imported and data files etc.).

styles.less

.something {
    background: data-uri("some.svg");
}

sub/test.less

@import (reference) "../style.less";
.test {
    color: green;
}

It compiles style.less, but not the test.less because it tries to use data-uri relative to test.less.

Why would reference import try to do rewriting the paths, it's not necessary when using references in my opinion.

I would expect data-uri to follow the url rewriting rules in the options. Of course.... hard to say what that actually means with data-uri.

In general, data-uri should resolve relative to the "calling .less file". So in the case of a mixin, it should resolve relative to where the mixin is called, not relative to the mixin location. The mixin "mixes in" those statements and then resolves them. So @lukeapage I think your interpretation is correct:

// app/content/button.less
button {
  .background("images/btn.jpg");
}

It should look for btn.jpg at app/content/images/. If it's not, that's a bug because it's not how mixins should work. I don't think it's a "breaking change" but a bug fix.

That said... I don't think there would be an issue with resolving like:

1. Attempting to resolve relative to the caller.
2. Attempting to resolve relative to the mixin.

Node.js attempts multiple paths for resolution. As long as the resolution order is clearly documented, you can manage expectations. And doing it like that would mean that if someone had based their .less to do behavior #2, it would still work in almost all, if not all cases.

@matthew-dean
In general, data-uri should resolve relative to the "calling .less file". So in the case of a mixin, it should resolve relative to where the mixin is called, not relative to the mixin location. The mixin "mixes in" those statements and then resolves them.

You couldn't ever make that work in a way that is generally correct for all use-cases.

E.g. how would you support parametrized urls, i.e., urls with replacement tokens, that should be resolved against some known base folder with just the replacement tokens filled? That case requires resovling against the callee's file, not the caller's file.

I had a bit of an epiphany on how to fix this transparently without explicit use of url() at the call site, which @seven-phases-max rightfully called out as being a bad idea (because someone eventually _will_ try to refactor it into the mixin call and break things):

When literal Quoted nodes are created, retain their file info. Propagate that info through variable assignments, mixin calls, etc. Any Quoted value originating from a caller file would, when treated by url() or data-uri(), be resolved against that caller file. But a Quoted value that's part of some internal logic of a mixin still gets resolved against the mixin's local file.

That keeps everything working as expected, save for scenario's with substitution strings such as in:

// mixins.less
.background(@image) {
    background-image: data-uri("@{image}.jpg");
}
// app/content/button.less
button {
  .background("images/btn");
}

There's a trick you can pull to fix those as well: when filling out the subsitution tokens, if a token is at the very start of the substitution string, then the resulting string should inherit the file info from the filled in token, rather than that of the subsitution string.

If the intent of the mixin's own author is to have such paths resolve against the mixin file, they can still make that work by using, e.g., "./@{image}.jpg" as a pattern. That effectively shifts burden of responsibility away from the caller, which is what you'd want.

// _mixins.less
.sprite(@image) {
    background: data-uri("../images/sprites/@{image}.png") no-repeat;
}

// main.less
div {
   .sprite('logo');
}

output:

div {
   background: url(data:image/png;base64,...) no-repeat;
}

Wow, this is an very old one.... my two cents as this issue is affecting me as well.

What about adding the option to url function, or creating a new function, that will solve the url as absolute. That way no matter where the mixin is set, it will be working with absolute paths and no room for mistakes.

What about adding the option to url function, or creating a new function, that will solve the url as absolute. That way no matter where the mixin is set, it will be working with absolute paths and no room for mistakes.

The issue here was not about absolute vs relative paths. It was about relative against call site vs relative against declaration site. Totally different problem.

Perhaps the issue has evolved, but the initial comment and title is about data-uri resolving the path in relation to the file where it was being called, which when combined with a mixin placed somewhere else may resolve the path incorrectly. Well, that's the issue I am experiencing.

So where do absolute paths factor into that as a solution?

Assuming data-uri accepts absolute paths and there is an hipotethical absolute-url function, the following code would work no matter where the mixin is placed.

// mixins.less
.background(@image) {
    background-image: data-uri(@image);
}

// app/content/button.less
button {
  .background(absolute-url("images/btn.jpg"));
}

@miljan-aleksic By "absolute" do you mean relative to the file at the location of data-uri? If so "absolute" is probably the wrong term.

It seems like a functional wrapper for a url to make any url file-relative is a good approach though. Or an additional argument to url().

@matthew-dean, by absolute I mean full path to the file, eg: /users/myuser/projects/lessproject/icon.svg.

I don't get your approach as I don't see how url() could make a relative path to data-uri location file without knowing it ubication.

It seems like a functional wrapper for a url to make any url file-relative is a good approach though. Or an additional argument to url().

Funnily enough; that's almost what I suggested a few years ago. ;-)

by absolute I mean full path to the file, eg: /users/myuser/projects/lessproject/icon.svg.

It looks like with absolute you mean this absolute-url function _pre-resolves_ the relative path passed to it to a full output path, based on the location of the file in which the absolute-url function is called, relative to the location where the compiled output CSS file will go.

I.e. you both mean the same thing. As did I, at the time.

it to a full output path, based on the location of the file in which the absolute-url function is called, relative to the location where the compiled output CSS file will go.

As in, rewrite URLs or rootpath would still apply, but the based on the definition location? That seems a little different than @lukeapage's original example, which was not talking about URLs relative to output, but URLs _during compile_; e.g. the data-uri() location.

So this issue is a little hard to track because people have posted about similar, but not exactly identical issues. That is, changing a relative _source_ would probably require a much different solution than changing relative _output_. Or perhaps not; it depends on the pathing logic; but we should be clear that data-uri doesn't produce any path as output.

Maybe we need something like a resolve()? I dunno, just spitballing, but url(resolve(file(), "my/path"))? I guess that's what @miljan-aleksic meant by absolute(), in that it would resolve to an absolute URL. But it should still take an input (like file(), to resolve against). Otherwise you could do something like file-resolve() to designate that logic in one function, but having resolve() and file() as two functions might be separately useful.

The tricky part about all of this is all the rewrite URL options, of which there are now more as of the PR merge that added module support. (https://github.com/less/less.js/pull/3248). So if it returns a file-relative URL, can it still be rewritten? I would assume yes, but we'd need to be clear.

Yes, resolve() and file() defines exactly what I was trying to explain. I hope we could see this implemented in some near future.

@miljan-aleksic Okay, that makes sense then.

Actually, file() isn't quite right, since that would return the filename I would presume, it would be more like dir(). And it should probably be a variable constant per file.

What about:

data-uri(resolve(@DIR, "my/path"))

So, two things added: 1) a resolve() function to combine paths, 2) a @DIR (and @FILE?) constant injected into each file while evaluating. The only tricky thing about that would be testing that those injected vars don't override or get merged with other vars in the root, but that should be fairly simple to test against. Or should they be lowercase, like @arguments? In which case, I would suggest @directory and @filename to avoid conflicts. What's the most Less-y option?

Maybe we need something like a resolve()?

^Bingo. Exactly that.

What's the most Less-y option?

I'd go for the Node.js-y option: __dirname
It has been around for a long time and is well-known. Using the same name for the same concept in Less might be a good idea.

I'd go for the Node.js-y option: __dirname

Err... that wouldn't match Less / CSS keyword nor Less variable nor function semantics. We'd have to do better than that. @__dirname maybe, but the underscores are still a little weird for the language. It doesn't fit at all.

the underscores are still a little weird for the language. It doesn't fit at all.

A double leading underscore is used to indicate 'something the system provides' in a lot of languages, especially when it comes to things like intrinsic variables. So the out-of-placeness is kind of the point there.

Ofcourse; if you don't like it, you could always go with a derivative like @dirname or @dir-name.

However, having thought about this some more, why do we even _need_ the current file path exposed as a variable? Can't it be woven into the conceptual resolve() function itself?

Can't it be woven into the conceptual resolve() function itself?

And we got back to my proposal, just with a different function name. I still like it the most, even better as resolve().

And we got back to my proposal, just with a different function name.

Sort of.

What I'm getting at is that afaik there isn't a need to pass in the URL / path of the file holding the call to a resolve() function, as that location should be _known_ to the function.

this inside a function refers to a FunctionCaller instance, which has a currentFileInfo property.
That property is initialized to the file info of the Call AST Node corresponding to the function call.
I.e.

https://github.com/less/less.js/blob/4e903e8254cc20fec80fccd35794fb797949e653/lib/less/tree/call.js#L47

If I'm reading the code correctly, the file info there corresponds to the file info of the place where the function call is _declared_ - not the file where the function call is evaluated.

That means it should be OK to use this file info for an 'eager' URL resolver. It would act as expected, even if a function call is placed inside a mixin that is imported and evaluated within the context of another file. Namely: with the resolution pinned to the file where the mixin is defined.

However, having thought about this some more, why do we even need the current file path exposed as a variable? Can't it be woven into the conceptual resolve() function itself?

If we're sure no one needs the current file or a general resolver function... maybe... the thing is an explicit local var makes it clear that it's not a generic function, and that the function won't be evaluated like any other function. That's my real concern, is the semantics. No matter what you name it, if you don't have a special "marker" for "current file", then it's not like any other function that's resolved the same based on inputs. In other words, it's a function that resolves according to invisible inputs, and that concerns me. However, if the function is something extremely explicit like current-file-resolve(), maybe that's clear enough. Otherwise you're just going to confuse people why a mixin call didn't resolve specialfunction() according to the file it was called in, instead of the file the mixin was defined in.

So, no, a local var is technically not _needed_, but the meaning / output / behavior needs to be clear from the semantics.

hmm...

Ok. Then how about we have a resolve-url(url, [base]) function - with an optional base; defaulting to the directory of the file in which the function call is written/declared/defined. Then have a declared-dir() function which just pulls this.fileInfo and grabs the path, should authors want to pull the path explicitly; want to take a path from a different file; or need to use this as part of some other feature not related to URL resolution.

I.e. a full form call (without implicit base) would be something like:

resolve-url("../foo", declared-dir())

... and would be equivalent to just doing

resolve-url("../foo")

... which is the eager equivalent to the lazy

url("../foo")

No 'automagically' injected variables needed that way. This could be implemented purely as a set of plugin functions, I think. And the only core mechanic we need is a way to mark URLs as 'already resolved', so that the default resolver logic can be blocked from running on URLs that come out of the resolve-url function.

The semantics should be made clear in documentation ofcourse. And that documentation can explicitly compare url against resolve-url to illustrate the eager and lazy evaluation/resolution.

_{Just in case the "injected variables" idea won't work (w/o additional hacks) anyway because imported files have the same scope. I.e.:}

@__dir: "whatever";
// *everywhere* it's the only @__dir value = the path of "c"
@import "a";
@import "b";
@import "c";

Speaking of the function-based implementation, I think (but can't be sure) that it's still possible to obtain the path of the file where the function is invoked somewhere in its this.context.? or this.context.frames[?] or so.

emmmm, so we don't have a better way to resolve it?

@heynext
emmmm, so we don't have a better way to resolve it?

Lazy evaluation makes this very hard to solve properly, I'm afraid.

@seven-phases-max
Speaking of the function-based implementation, I think (but can't be sure) that it's still possible to obtain the path of the file where the function is invoked somewhere in its this.context.? or this.context.frames[?] or so.

You should be able to find the Call node in its hierarchy, yes.