Three.js: Material: Added onBeforeCompile()

It does seem a bit hacky to require custom string manipulation when WebGLProgram.js already does a lot of processing. There is a related use case, which is to add new #includes rather than overriding existing ones. You can do this today by modifying THREE.ShaderLib at runtime, but it feels equally dirty.

Perhaps a nicer solution for both is to allow you to provide a custom ShaderLib for a material, that would override/augment the built-in chunks without manual string manipulation and without permanently clobbering them. That doesn't preclude an onBeforeCompile hook for other uses, but it seems more in line with what your example is trying to do.

That said, shader composition based on raw includes is always going to be very fragile from one version to the next. If you don't mind including all the skeleton code (like meshphong_vert.glsl), you can already extend the built-in materials with something like this:

export class MyPhongMaterial extends ShaderMaterial {
  constructor({ color, radius, map, normalMap, emissiveMap }) {
    super();
    this.vertexShader = "...."
    this.fragmentShader = "....";
    this.uniforms = UniformsUtils.clone(ShaderLib.phong.uniforms);

    this.isMeshPhongMaterial = true;
    this.lights = true;

    this.uniforms.time = { value: 1 };
    //...
  }
}

In all cases, you have to rely on the internal organization of built-in shaders not to change too much from one release to the next. Each include is already a function in disguise, with an ill-specified signature. I suspect it is going to be much cleaner and maintainable on both sides to provide function-level rather than include-level overrides.

Today I thought..

Nice! I thought so too on february 10th when i made this pull request:

https://github.com/mrdoob/three.js/pull/10791

It seems the only difference is that you're doing it through a callback? I tapped into the unused argument in the WebGLRenderer.

I like using this for example to get threejs to work with normal maps, but I've linked some issues that seem like they would be trivial to solve with this.

You can look at my branch and take this for a spin or check out this super simple example (just tests a custom deformation with lighting/shadows etc, and adds a few uniforms).

http://dusanbosnjak.com/test/webGL/three-material-shader-override/webgl_materials_shader_override.html

Perhaps a nicer solution for both is to allow you to provide a custom ShaderLib for a material, that would override/augment the built-in chunks without manual string manipulation and without permanently clobbering them.

Exactly sums up what happens in: https://github.com/mrdoob/three.js/pull/10791 I wish i could write better descriptions :)

I must admit that i don't know wth is going on here. I think i know someone who gave up contributing to three, didn't understand it then, but now i find it kinda frustrating.

Frustrating mostly because i'm having a deja vu when i read the first paragraph:

...similar way to Object3D's onBeforeRender().

This exact same thing happened with the onBeforeRender() https://github.com/mrdoob/three.js/pull/9738, when it took a year of convincing to get the feature in.

Jeez, twice, with the same dev? I'm obviously doing something wrong here, but what? The first time around was the build files being checked in, i didn't know what to do, and it just got forgotten. But this time around i was on top of this pull request, there's a conflict right now but it's easily solved, were no conflicts for months.

Don't get me wrong, i don't think it's vanity in the works here, i think i just want to run ideas by people and get feedback. @unconed it's obvious that you're familiar with the problem and you've probably tried different things. What may have caused an user like you to miss the other PR?

I felt pretty good about the per material shaderlib, but i did find one function in the renderer that looked/compared entire shaders as strings to figure out caching, wasn't feeling all that great about that. Wish there was some feedback given...

Was the example flawed? The head may make it much more obvious what is going on than my abstract spike ball, but the whole example works with shadows which covers more ground. It's heavy for some reason, but i think it's because sin/cos on many verts.

I guess if you're starting over... Even though scene kit is absolutely horrid, this api is really nice:

https://developer.apple.com/documentation/scenekit/scnshadable

Albeit horribly documented, i can't find the part that's of more interest, but basically they have abstracted hooks at many different stages of the shader.

Earlier this year i made a pull request that seems to do something similar if not the exact same thing:

https://github.com/mrdoob/three.js/pull/10791

It seems the only difference is that you're doing it through a callback? I tapped into the unused argument in the WebGLRenderer.

Sorry for not being able to take care of that PR yet @pailhead. I guess the main advantage of my approach is that it only required 3 new lines.

Having said that, I'll now spend some time studying yours and @unconed suggestions.

This one definitely feels more elegant, only three lines. I was following a different pattern in the other one. Was going to say that it may be a bit more verbose, but not so sure. I guess the only advantage of the other one is that it was good to go a few months ago :)

Jeez, twice, with the same dev? I'm obviously doing something wrong here, but what?

Sorry about that. The only thing I can think of is that probably I got overwhelmed. Maybe a long discussion or a complicated PR that would consume too much of my energies, so I decide to leave it for later and move to simpler PRs.

It's not only you @pailhead. @bhouston has had a lot of PRs like this, even @WestLangley and @Mugen87.

Again, is not that I don't like the PR, is that the PR requires some attention I can't offer at the time. A good example is the Instancing PR. I managed to find time to read through it, did my own experimentation and suggested simplifications. Hopefully I'll be able to revisit it soon.

It's difficult to manage all this and give every PR the attention they deserve.

Was the example flawed? The head may make it much more obvious what is going on than my abstract spike ball, but the whole example works with shadows which covers more ground. It's heavy for some reason, but i think it's because sin/cos on many verts.

Now you mention it... Yeah, runs at 10fps on a new MacBook Pro when zooming in 😮

I think it's the shadows and the rand, sin cos and stuff, but yeah it looks like it takes too much of a hit.

Either way, i'm surprised you pulled this off in three lines, i was focused too much on how material params get turned into uniforms and followed that pattern. The difference is that I provide an alternative ShaderLib like dictionary, so #include <> brings in different code. You remove the include before this happens, and replace it with glsl. I think if you return some kind of a wrapper around the shader, perhaps this syntax could be cleaner (just provide the chunk name + glsl, instead of replace(). But it may end up looking much more like the other one.

It would be really nice to have this, i haven't worked with that many 3d engines, but unity and scene kit seem to have something like this.

@unconed

I guess one benefit from the onBeforeCompile() is that the material properties remain unchanged. So it feels more like extending the built-in materials.

export class MyMeshPhongMaterial extends MeshPhongMaterial {
  constructor( parameters ) {
    super( parameters );
    this.onBeforeCompile = function ( shader ) {
      shader.vertexShader = shader.vertexShader.replace(
        '#include <begin_vertex>',
        'vec3 transformed = vec3( position.x + sin( position.y ) / 2.0, position.y, position.z );'
      );
    };
  }
}

var material = new MyMeshPhongMaterial();
material.color.setRGB( 1, 0, 0 ); // this still works

Messing with ShaderLib is tricky indeed. Adding forever-unchanged hooks should be doable though:

    #include <begin_vertex>
    % vertex %
    #include <morphtarget_vertex>
    #include <skinning_vertex>
    % transformed_vertex %
    #include <project_vertex>
    % projected_vertex %

The replace code will become this:

this.onBeforeCompile = function ( shader ) {
  shader.vertexShader = shader.vertexShader.replace(
    '% vertex %',
    'transformed.x += sin( position.y ) / 2.0;'
  );
);

We'll then remove any hook that hasn't been used before compiling, of course.

here's what it looks like compared to the per instance ShaderChunks

//given some material
var material = new THREE.MeshNormalMaterial();

//and some shader snippet
var myShader = [
    'float theta = sin( time + position.y ) / 2.0;',
    'float c = cos( theta );',
    'float s = sin( theta );',
    'mat3 m = mat3( c, 0, s, 0, 1, 0, -s, 0, c );',
    'vec3 transformed = vec3( position ) * m;', //and making assumptions about THREE's shader framework
    'vNormal = vNormal * m;'
].join( '\n' );

https://github.com/mrdoob/three.js/pull/10791 same as Material.defines approach:

material.shaderIncludes = {

        begin_vertex: myShader,

    //uv_pars_vertex: [
    //  THREE.ShaderChunk['uv_pars_vertex'], //this doesn't have to be
    //  "uniform float time;",
    //].join('\n')
};

material.shaderUniforms = { time: { value: 0, type: 'f' || 'float' } }; //because this could just inject it in the right place (but needs type)

_It's only a dictionary of chunk names, and the uniforms object. The string manipulation here is more along the lines of "i want to reuse this chunk, and do something on top of it"_

this PR:

material.onBeforeCompile = function ( shader ) {

    shader.uniforms.time = { value: 0 };

    shader.vertexShader = 'uniform float time;\n' + shader.vertexShader; //this feels hacky

    shader.vertexShader = shader.vertexShader.replace( //this is more verbose
        '#include <begin_vertex>',
        myShader
    );

};

I guess one benefit from the onBeforeCompile() is that the material properties remain unchanged.

It works the same in the other PR, so I think it doesn't matter how it's done.

One probably irrelevant benefit that i can think of is that you can keep the logic for manipulating the material in the same block and on the same level. Even if the code doesn't look much different, conceptually, you're manipulating .color on Material while manipulating a .uniforms object on a "shader". If you had a piece of code somewhere changing color, and you later add custom GLSL to that material, you can add the code to change the uniform with the color. (skip wall of text)

If the uv_pars_vertex is confusing, i think a good question to ask is:

"where do I inject my custom GLSL function, like float rand( vec2)?"

Then this one might make sense https://github.com/mrdoob/three.js/pull/11050

Another argument could be - assuming you know GLSL, assuming you know THREE's shader framework, you still have to be creative and think about where you inject what. I had to think a bit and look through most of the shaders to figure out that uv_pars_vertex is a convenient place to do my material extension.

It would be better to go from something like this towards an abstraction, have a lightingPhase, objectTransformation , perspectiveTransformation etc hooks. Or at least a guaranteedToBeAboveMainButBelowCommonsAndExtensionCalls dummy chunk that's included in every shader program. It looks like you're going for that with % vertex %? But i'm unfamiliar with that syntax.

This PR, as is, seems like it's going in the opposite direction. On top of knowing where you need to tap in, you also need to be explicit and manipulate strings, repeating some of the work that the renderer already does for you. Also, if you ask another question in addition to the first one

How do i use a function, declared in the commons chunk, inside the function that i'm injecting?

you might find yourself doing more string manipulation than just prepending the uniforms to the entire shader.

I had to think a bit and look through most of the shaders to figure out that uv_pars_vertex is a convenient place to do my material extension.

I don't think there is a way around that unless we over-engineer. The idea with % vertex % is to try helping with that a bit by naming the hooks and more or less tell you at what state you're injecting code, but it would be hard to document what variables are available at what point.

We're opening a door for allowing built-in materials hacking, but I don't think we can provide proper "support". The user should be aware that chances are things may break.

I tried to tackle it in the other PR. I added a dummy hook at least for the functions, varyings, attributes and uniforms. Lots of the GLSL logic already happens on structs, scenekit documented each variable (cant find the same documentation anymore though).

It would probably need to be refactored as we go along but something along these lines:

#ifdef PHASE_FOO

#include <shader_foo> 
//someGlobalStruct.mvPosition = modelViewMatrix * myLogic( transformed );
//if there is glsl provided for phase "foo" document that it should operate on "transformed" 
//and that it should return "mvPosition" 
#end including <shader_foo>

#else 

  //default
  #ifdef USE_SKINNING

    vec4 mvPosition = modelViewMatrix * skinned;

  #else

    vec4 mvPosition = modelViewMatrix * vec4( transformed, 1.0 );

  #endif

#ifdef PHASE_BAR

#include <something_like_this>
//mvPosition = myPostDefaultTransformationLogic( mvPosition );

#endif

gl_Position = projectionMatrix * mvPosition;

#endif

Of course ifdefs and includes would proably not work like this. But i can see it being relatively straight forward with the other approach. We could add #include <some_phase> always, with empty strings. Something could help manage what gets triggered by using #ifdefs. So for example, if you provide logic for "vertex_transformation" phase, the shader will define whatever you provide. Documenting what each chunk does with GLSL would be overall helpful. Without going further into abstraction, i think it would be great to have this map to how the chunks are structured now.

We're opening a door for allowing built-in materials hacking, but I don't think we can provide proper "support". The user should be aware that chances are things may break.

We already have the defines stuff available on every material. https://github.com/mrdoob/three.js/issues/10764 suggested to modify THREE.ShaderLib and then define a .defines property on an instance of a Material. Granted, the property was undocumented until i wrote the entry (but you approved it :)), and is still undefined.

But the way this currently works, and is inherited by every material, it is supported - if you put stuff in it, it will affect the shader for that material. Furthermore, if you happen to define something that is already part of the shader library, it may break things.

Not trying to be a smart-ass but trying to come up with an example. I mostly used defines with ShaderMaterial but it does affect all materials, just because how WebGLRenderer works. There is no logic that says:

if you're a much smarter, surface like abstraction of a material like Phong then don't take defines into account. Because three is not designed to have it's shader chunks overriden, and the Phong template is the final authority on what GLSL needs to do, and serves to hide this from the use, it makes sense to not have GLSL defines interfere with this.

I'm not entirely sure if it can be broken though, i'd have to try.

With that being said, i'd like the option to break things, if it provides a lot of gain. With a pretty encapsulated and tiny GLSL snippet, i could easily change my version of three to render normal maps differently. Taking a normal, and transforming it to get another normal is super straight forward, unless we start doing computer graphics in a completely different way, i can't see this ever breaking :)

Instancing was another example:

oh i'd like to use ANGLE_INSTANCED_ARRAYS, three doesn't seem to support it with it's shadows or PBR materials, let me just add these two lines, and make it work for me"

I'm happy to see that this features is finally coming to three.js :). Yes I also created a yet another material modifier PR (https://github.com/mrdoob/three.js/pull/7581) It was so old that the code is not relevant anymore, but is basically injecting hooks like @mrdoob proposes and then just replace them by your own code.
I like the idea of predefined hooks as it's easy to understand what you're modifying as I believe most of the people that want to use this feature want to modify "slightly" the default material instead of completely rewrite it.

I like the simplicity of this PR but if we're going for a more structured/clean way to do things I'd prefer to have already a dictionary of hooks to replace with your code and a simpler way to add uniforms or custom code than just replacing strings.

@fernandojsg any chance you could check out #10791, give feedback, it seems super similar to what you did except i've put the additional hook just outside of main (something like your "pre_vertex") and there's slightly more management.

@fernandojsg I had forgotten about your PR 😚. I think your PR looks quite a lot of how I was starting to see this working. Subconscious?

Not trying to be a smart-ass but trying to come up with an example. I mostly used defines with ShaderMaterial but it does affect all materials, just because how WebGLRenderer works.

I'm glad the library is hackeable in that way, but we shouldn't use abuse features inernally for things that were not intended for. It's easy to end up with fragile code that way.

Reading #7581 again... On top of https://github.com/mrdoob/three.js/commit/e55898c27a843f69a47e602761c60d9bbe91ee35 we can add the % HOOKS % and then create a class like this:

THREE.ExtendedMaterial = function ( material, hooks ) {

    material.onBeforeCompile = function ( shader ) {
        var vertexShader = shader.vertexShader;
        var fragmentShader = parameters.fragmentShader;
        for ( var name in hooks ) {
           vertexShader = vertexShader.replace( '%' + name + '%', hooks[ name ] );
           fragmentShader = fragmentShader.replace( '%' + name + '%', hooks[ name ] );
        }
        shader.vertexShader = vertexShader;
        shader.fragmentShader = fragmentShader;
    };

    return material;

};

Then we can do this:

```js
var material = new THREE.ExtendedMaterial(
new THREE.MeshBasicMaterial(),
{ vertex: 'transformed.x += sin( position.y ) / 2.0;' }
);

So the question is, what hooks should we have?

VERTEX
TRANSFORMED_VERTEX
PROJECTED_VERTEX

NORMAL
TRANSFORMED_NORMAL

FRAGMENT_UNIFORMS
INPUT_FRAGMENT
OUTPUT_FRAGMENT

@mrdoob As an example, you can see in this gist where code needs to be injected into the MeshBasicMaterial vertex shader to support instancing.

And here for the rest of the materials :)

https://github.com/mrdoob/three.js/pull/10750

VERTEX_UNIFORMS sounds too opinionated, where should you add a function, an attribute or a varying? PRE_VERTEX or something?

NORMAL_MAP is definitely needed

vert:

PRE_VERTEX
VERTEX
TRANSFORMED_VERTEX
PROJECTED_VERTEX

NORMAL
TRANSFORMED_NORMAL

UV

fragment:

PRE_FRAGMENT
INPUT_FRAGMENT
LIGHT
NORMAL

OUTPUT_FRAGMENT

VERTEX_UNIFORMS sounds too opinionated, where should you add a function, an attribute or a varying? PRE_VERTEX or something?

Hmm, opinionated? How so?

attribute vec4 aMyAttribute; is not a uniform :)

float myRand( vec4 foo ) { /*logic*/ return vec4(bar,baz)} is not a uniform

varying vec3 vMyVarying; is also not a uniform

either not plural, or not uniforms at all

What's the use case for "injecting" attributes?
Shouldn't you be using geometry.addAttribute( 'aMyAttribute', ... ) instead?

I didn't know you don't have to declare it. Still leaves varyings and functions?

Still leaves varyings and functions?

Good point.

you really don't have to declare attributes at all anymore in GLSL? I thought it worked more like uniforms , you declare in GLSL but don't have to type it in JS.

I think this list can grow pretty big pretty quick, when it comes to certain things the code gets pretty scattered. One way of doing normal maps involves two additional attributes, varyings and transformation logic in both vert and frag.

I like the latest changes it's still simple and elegant. I agree with @pailhead that we need a place to put code in the global scope. In https://github.com/mrdoob/three.js/pull/7581 I had preMainVertex preMainFragment to be injected globally so you could use them for varying and functions. Probably a name like globalVertex/Fragment or similar could be better.

GLOBAL_VERTEX and GLOBAL_FRAGMENT sound good 👌

exactly how i named them in the other PR 😆 👍

@pailhead But now you know the secret to get your code merged: Just open a PR, leave it there some time, then wait for @mrdoob to eventually think about doing something similar, wait for his PR and then just comment there referencing the exact same way as you did on your PR so he could think it was his idea, he'll add it, merge... profit!

Now seriously I like too that names and with that I believe we're ready to go, or am I missing something? I'll try to convert the example from my PR to this new code ;)

The part about the code getting in is fine, the timeline is kinda off. #7581 was suggested almost two years ago. Wasn't referenced in https://github.com/mrdoob/three.js/issues/10789, so probably went off the radar. I would have bumped it or forked it.

Even when importing modules and cherry picking your own three, people are not too happy about using non-official core modules. Much easier to say, "hey, there's an npm package that handles foo". Thats why i feel PRs like this are important, three is not flexible enough.

profit!

What profit? 😆

The part about the code getting in is fine, the timeline is kinda off. #7581 was suggested almost two years ago. Wasn't referenced in https://github.com/mrdoob/three.js/issues/10789, so probably went off the radar. I would have bumped it or forked it.

What? You didn't check all the open PRs before creating a new one? #trololol

true :)

ok dial in the list and lets have this asap :D

Will this shader (for example):

#include <shadowmap_pars_vertex>

void main() {

    #include <begin_vertex>
    #include <project_vertex>
    #include <worldpos_vertex>
    #include <shadowmap_vertex>

}

look like this:

#include <shadowmap_pars_vertex>

void main() {

    % begin_vertex %
    % project_vertex %
    % worldpos_vertex %
    % shadowmap_vertex %

}

Umm I'm not sure if replace all of them or just inject the code around leaving some of the current behaviour:

#include <shadowmap_pars_vertex>

%GLOBAL_VERTEX% 
void main() {
       %PRE_VERTEX% 
    #include <begin_vertex>
...
}

My initial approach was to inject the extra code leaving the rest of the include untouched.

So % hook % is just for the actual hooks, if one wants to replace <begin_vertex> they still have to look for it in a string, ie. there wont be a begin_vertex hook?

In my PR I just wanted to modify slightly the behaviour of the material, typically do some color corrections after the fs was fully executed or vs transformations similar to the one @mrdoob showed in his example.
But yep if we want to fully replace each one of the includes we should use the hooks instead of the includes and add the logic to inject the include itself in case that you didn't bind the hook.

I'd like to do that for normals (not use derivatives, and use actual TBN information from 3ds max or maya or anywhere where the normal map is being generated). Adapting MeshPhongMaterial to work with instances also required replacing some chunks altogether, but this feels like such an edge case.

Those are the two i can think of, i wonder if there are more.

So % hook % is just for the actual hooks, if one wants to replace <begin_vertex> they still have to look for it in a string, ie. there wont be a begin_vertex hook?

You can replace both. The current code already lets you replace whatever. %HOOK% ones are for convenience.

As I mentioned in #11562 , how about concept of replaceable shader chunks?

Example:

const material = new THREE.MeshPhongMaterial({
  color: 0xffffff,
  map: texture,

  parts: {
    frag: {
      map_pars_fragment: `
        uniform sampler2D map;
        uniform sampler2D map2;
     `,

      map_fragment: `
        vec4 texelColor = texture2D( map, vUv );
        vec4 texelColor2 = texture2D( map2, vUv );

    texelColor = mapTexelToLinear( 
          mix( texelColor, texelColor2, progress)
        );

    diffuseColor *= texelColor;
      `
    }
  }
});

material.uniforms.progress = {value: 1};
material.uniforms.map2 = {value: texture2};

This is how simple can be code of transition between multiple textures on 1 material.

It can be easily implemented and avoid making unnecessary code.

@sasha240100

https://github.com/mrdoob/three.js/pull/10791 looks exactly like the one you are suggesting, except parts are called shaderChunks and it doesn't take it as an argument.
:roll_eyes: 😆

You also don't need the frag{} dictionary, since the chunks are already marked. All of them have _fragment and _vertex.

This one does the same except you have to search through the string to find what you need to replace.

@pailhead Yes, let's drop the frag: {}. I used it as argument because It should be passed before material shader is concatenated. In this case we can simply avoid replacing existing parts after concatenation.

We can actually add both: as argument and as property. Like color does

Where is this at right now? I see the example is doing the .replace() and i see no hooks / replaceable chunks? @mrdoob

Haven't been able to get back to this yet.

Would you like to do a PR for the % hook % thing?

Do you want the extended material, from the comment above?

THREE.ExtendedMaterial = function ( material, hooks ) {

    material.onBeforeCompile = function ( shader ) {
        var vertexShader = shader.vertexShader;
        var fragmentShader = parameters.fragmentShader;
        for ( var name in hooks ) {
           vertexShader = vertexShader.replace( '%' + name + '%', hooks[ name ] );
           fragmentShader = fragmentShader.replace( '%' + name + '%', hooks[ name ] );
        }
        shader.vertexShader = vertexShader;
        shader.fragmentShader = fragmentShader;
    };

    return material;

};

Choosing the spot for the hooks and documenting them might actually not be that trivial? I would maybe replace hooks passed in with chunks and look for the includes, with addition of global_vert and global_frag for now? Which now that I wrote it, sounds like it shouldn't even be three's responsibility, but on the other hand it would stub the THREE.ExtendedMaterial for % hook %s in the future?

I don't know if you are aware of the THREE.BAS extension. It uses placeholders in the original materials that can be accessed from the customised material as arrays, like:

  var material = new BAS.PhongAnimationMaterial({
    flatShading: true,
    vertexColors: THREE.VertexColors,
    side: THREE.DoubleSide,
    uniforms: {
      uTime: {type: 'f', value: 0}
    },
    vertexFunctions: [
      // cubic_bezier defines the cubicBezier function used in the vertexPosition chunk
      BAS.ShaderChunk['cubic_bezier'],
      BAS.ShaderChunk['quaternion_rotation']
    ],
    vertexParameters: [
      'uniform float uTime;',
      'attribute vec2 aDelayDuration;',
      'attribute vec3 aStartPosition;',
      'attribute vec3 aEndPosition;',
      'attribute vec3 aControl0;',
      'attribute vec3 aControl1;',
      'attribute vec4 aAxisAngle;'
    ],
    vertexInit: [
      'float tProgress = mod((uTime + aDelayDuration.x), aDelayDuration.y) / aDelayDuration.y;',
      'vec4 tQuat = quatFromAxisAngle(aAxisAngle.xyz, aAxisAngle.w * tProgress);'
    ],
    vertexPosition: [
      'transformed = rotateVector(tQuat, transformed);',
      'transformed += cubicBezier(aStartPosition, aControl0, aControl1, aEndPosition, tProgress);'
    ]
  });

I like the fact that it is based on simple conventions: vertexInit and fragmentInit will be at the top of the main() function for example. vertexParameters and fragmentParameters are at the top of the shader, to define uniforms or attributes. vertexFunctions and fragmentFunctions are before the main() function. And so on for normals, position etc...

As you can see in the position, you change the variable transformed to change the final position. Similar thing for normals (objectNormal) or colours (diffuseColor) in the fragment shader, for example.

This is how the Phong material looks like, only adds the placeholders: https://github.com/zadvorsky/three.bas/blob/master/src/materials/PhongAnimationMaterial.js

Hi guys, I was trying to follow the convo but honestly Im lost since I come from Front end and not game dev.

Im trying to load a MTL and im getting:

material.onBeforeCompile.toString() is undefined.

My MTL file was pointing originally to a .psd file and I changed it to one of the few png that contain the same folder (Normal) I guess is not possible to load the psd, but anyway even with psd is failing.

# Blender MTL File: 'goblin.blend'
# Material Count: 1

newmtl Material
Ns 96.078431
Ka 1.000000 1.000000 1.000000
Kd 0.640000 0.640000 0.640000
Ks 0.001617 0.005682 0.002517
Ke 0.000000 0.000000 0.000000
Ni 1.000000
d 1.000000
illum 2
map_Kd Normal.png

The console output is:

THREE.WebGLRenderer 87
bundle.js:54570 OBJLoader: 29.447998046875ms
bundle.js:28429 Uncaught TypeError: Cannot read property 'toString' of undefined
    at WebGLPrograms.getProgramCode (bundle.js:28429)
    at initMaterial (bundle.js:32350)
    at setProgram (bundle.js:32542)
    at WebGLRenderer.renderBufferDirect (bundle.js:31605)
    at renderObject (bundle.js:32335)
    at renderObjects (bundle.js:32308)
    at WebGLRenderer.render (bundle.js:32072)
    at bundle.js:9658
    at bundle.js:53976
    at XMLHttpRequest.<anonymous> (bundle.js:40153)

and the code im using is mainly:

OBJLoader(THREE);

const modelLoader = new THREE.OBJLoader();
const textureLoader = new MTLLoader();
textureLoader.setTexturePath( 'models/' );
    textureLoader.load('models/goblin.mtl', function( materials ) {

            materials.preload();

            modelLoader.setMaterials( materials );
            modelLoader.load('models/goblin.obj', function ( obj ) {
                 scene.add( obj );
                renderer.render( scene, camera );


            });

        }); 


    renderer.render( scene, camera );

}

Is this a problem in the model? in Three? in the mtl loader? it have solution?

Any help would be appreciated.
Thanks.

Do we have to do something like material.shader = shader in material.onBeforeCompile(shader=>...)?

I think it feels somewhat clunky, might be worth revisiting the other suggestion at some point :)

@jjalonso

This seems to be a bug:

https://stackoverflow.com/questions/46787166/react-and-three-js-mtlloader-material-onbeforecompile-is-undefined

It's being caused by this, but whats causing the Material to not have this func?
https://github.com/mrdoob/three.js/blob/35a26f178c523514673d992da1aece23c1cfca6e/src/renderers/webgl/WebGLPrograms.js#L244

@mrdoob

Is it possible that these extended shaders are not being cached correctly? I am trying to use two different materials, one has dithering_fragment changed, the other doesnt. When i add both to the scene, it appears as only the one without the dithering_fragment chunk is being used.

I've been unable to recreate it with this fiddle
https://codepen.io/anon/pen/KQPBjd

But with this code in my example

const dithering_fragment = 
`
gl_FragColor.xyz *= foobar
`

// in onBeforeCompile
console.log(shader.fragmentShader)



md5-0f1a3ac67268b230968df20abdbd03d1



/**
#if defined( DITHERING )
  gl_FragColor.rgb = dithering( gl_FragColor.rgb );
#endif

gl_FragColor.xyz *= foobar

}
*/

But it doesn't error out if i add another material to the scene that shares other includes and defines.

It is due to this:

https://github.com/mrdoob/three.js/blob/35a26f178c523514673d992da1aece23c1cfca6e/src/renderers/webgl/WebGLPrograms.js#L244

I have the same function with some logic provided to both materials, the variables for the logic are out of scope of the callback, thus both my materials get the same hash?

array.push( material.onBeforeCompile( obtainShaderSomehow ) )

https://github.com/mrdoob/three.js/pull/10791 solved it like this:

https://github.com/pailhead/three.js/blob/879d52349bd5ef72c64fc962d8ca26bacaf489bf/src/renderers/webgl/WebGLPrograms.js#L242

Would you reconsider the other pull request if i remove the global_vertex and global_fragment placeholders/hooks? With those it touched on a lot of files, without them, it's less code. Not three lines, but it hashes correctly :)

Comment #41 in this fiddle https://codepen.io/anon/pen/KQPBjd
and the material of the other mesh in scene will change.

https://github.com/mrdoob/three.js/issues/13192

Has this been included in a recent release, or is it all still in development?

@cdhanna http://threejs.org/examples/#webgl_materials_modified

@mrdoob I think it's difficult (or impossible) to serialize Materials hacked with .onBeforeCompile(). Do you think users should use ShaderMaterial if they want full functional(render, copy, clone, serialize, and so on) modified built-in materials?

_Please read the following in the nicest, constructive, non-confrontational way possible_ :)

@takahirox why does this keep getting claimed?

I think it's difficult (or impossible) to serialize Materials hacked

When you "hack" materials with onBeforeCompile you just add uniforms which are the same as with any other material. There is no difference. If you can serialize material.color why can't you serialize material.userData.myColor?

I'm not trying to start a fight, walls of text or anything. In the simplest shortest possible terms, can you please explain, or at least point me in the right direction as to why it's impossible or difficult? I am open to the possibility that i'm missing something obvious, but if i am i'd like to understand what it is :)

From a very small sample, generated by articles, this experiment it does not seem like people want to use the ShaderMaterial approach.

This just came to mind? Does there exist some kind of a test that proves that this is hard or impossible? Like:

runSerializationTest( someMaterialWithOnBeforeCompile ).expect( something )

As onBeforeCompile method could contain very arbitrary code, it's certainly impossible to robustly serialize that callback. For example, the method could make a synchronous XMLHttpRequest and do something with the result. 😅

However... for the way it's used practically (to patch the shaders) I agree it's not impossible. But the easy APIs (e.g. fn.toString()) are hacky, and the robust APIs are more complex.

I still don't understand the problem. How about this point of view:

Take rN that had MeshPhongMaterial and didn't have MeshStandardMaterial.

You can serialize MeshPhongMaterial ie. write a JSON like this:

{
  color: 'red',
  specular: 'very',
  glossy: 'not much'
}

Take rN+1 that has both materials:

You can still serialize both the same way:

//phong
{
  color: 'red',
  specular: 'very',
  glossy: 'not much'
}

//standard
{
  color:'red',
  shininess: 'shiny',
  metalness: 'not much'
}

Nowhere did you serialize the GLSL from MeshStandardMaterial.

The same way, serializing any extended material.

//extended PhongMaterial
{
   color: 'red',
   specular: 'very',
   glossy: 'not much',
   hamburger: 'medium rare'
}

Deserializing:

if ( data.userData.hamburger && HamburgerPhongMaterial )
   mat = new HamburgerPhongMaterial( data ) 
else{ 
   console.warn(`I'm terribly sorry, but you don't have the HamburgerPhongMaterial extension, using default fallback`)
   mat = new PhongMaterial( data ) 
}

What am i missing here?

As onBeforeCompile ... serialize that callback.

Why, why would you even consider it :) I think this is the crux of my confusion, why is any thought given to this hook, it's not part of the material definition, description, anything. It has something to do with the engine, plugins, apps etc, not the data.

The example you gave above looks fine to me, it just requires that the user code deserializing the material knows about the existence of HamburgerPhongMaterial? I have no problem with that, it doesn't necessarily even require API changes if you take an approach similar to #14099 and override toJSON and fromJSON.

Maybe we've interpreted @takahirox's question differently here?

...it's difficult (or impossible) to serialize Materials hacked with .onBeforeCompile(). Do you think users should use ShaderMaterial if they want full functional(render, copy, clone, serialize, and so on) modified built-in materials?

I'll try to split that apart:

• if the user calls toJSON() on a plain MeshStandardMaterial, where the GLSL that has been modified by its onBeforeCompile, the changes to the GLSL will not automatically be serialized.
• if the user serializes a MeshStandardMaterial with some extra properties indicating a change to the GLSL, and loads it with custom code that knows what to do with those extra properties, then certainly that's fine.
• various other options (ShaderMaterial, NodeMaterial, glTF's KHR_techniques_webgl, ...) are available for serializing arbitrary custom materials.

If the case we're talking about isn't one of those, then I guess I'm misunderstanding here. If this is about #14099, I'm (still) in favor of merging that.

Hmmm, i wasn't actually distinguishing between the first two. I'm trying to think of something that wouldn't require uniforms, it could be a gl_FragColor.xyz /= gl_FragColor.a; for example. But i can't imagine a scenario where you would want to serialize something like this (along with the material?). It seems like it would fall in the scope of some effect renderer or something, not material data.

materials = loadSerializedMaterials()

effect = new Effect()

effect.load( 'alphaDivide').then(()=>materials.forEach(mat.onBeforeCompile = effect.getOnBeforeCompile('alphaDivide')))

The second example is what i had in mind all along. Along with various inputs, serialize a hint as to what to do with them userData.extension === 'hamburger'.

I think I wrote poorly. What I wanted to mention is

• Built-in Material hacked with .onBeforeCompile() isn't copied, cloned, serialized correctly in the same API as built-in materials'. It's handled as non-hacked material.
• If users want the correct result, they need some extra work in user side.
• If users want the hacked built-in material with Three.js core API without any user-side extra work, they should think of other options, like ShaderMaterial.

My question is just for the policy of .onBeforeCompile() rather than for the specific case. I wanted to clarify the limitation and add a note in the doc.

If i forget about three.js and the whole rendering system, i think i see the point. As a generic object, if it has .clone() you want to be able to clone and have consistent expectations. The problem is both solutions (clone this not clone it) seem to have a valid argument. You don't clone listeners, but you also somewhat expect something that influences drawing to be consistent.

I still think that my proposal from above is a valid way to do this, as i've encountered the same problem in other areas (like sharing depth buffers between targets can also be solved with ownThing || outsideThing.

Instead of caching the function, which doesn't work, you would basically cache userData but a subset of it. Makes no sense to cache userData.hamburger: 'rare' but it does userData.effectOn: true". TheownInput | ownChunks | ownWhatever` would solve this problem.

It also solves the problem of:

As onBeforeCompile method could contain very arbitrary code, it's certainly impossible to robustly serialize that callback.

It has been a long time, people have used onBeforeCompile we should know by now what kind of arbitrary code can be found in there. I think it only operates on shader object that is passed in. You mutate that object, but only mutations that make sense with ShaderMaterial would have an effect anyway. Setting some gl stuff yourself would probably be overridden, and that's about the only thing that pops to my mind.

It's essentially a pre three parse step, where you get a chance to parse the shader yourself along with the mandatory uniforms:{} interface, since you don't have it available on Material (defines:{} on the other hand you do, and both are siblings in ShaderMaterial).

How you do it, i.e whatever the arbitrary code is, doesn't matter. It does not return anything, but it mutates shader:{} and renderer then uses it.

Approaches that i suggest:

1. Add ownThing to various classes. THREE.WebGLRenderTarget could have .ownStencilDepthBuffer. Here, as i've suggested above, it would be some some version of ownGLSL:{} ownInput | ownUniforms . You remove the need for arbitrary code to mutate the shader:{} object:

Given input like this that originates from inside the WebGLRenderer (super private):

const shader = {
  uniforms: buildFromMaterial(mat),
  vs: getVSTemplate(key),
  fs: getFSTemplate(key)
}

nuke this:

shader = onBeforeCompile( shader )

shader.vs //invalid GLSL
shader.uniforms //you got some extra ones that onBeforeCompile may have mutated, maybe removed some others

shader = ___parse(shader) //private step

shader.vs //valid glsl, (not a template)

compile(shader)

Use this:

function parseShader(shader){
   return {
     uniforms: shader.uniforms,
     vs: parseChunks(shader.vs)
     fs: parseChunks(shader.fs)
   }
}
//FIX FOR HASHING BUG
function parseChunk( material, chunkName ){
  return material.ownChunks[chunkName] || THREE.ShaderChunks[chunkName]
}

//do whatever you want with the templates, maybe remove an `#include <>`
shader = onBeforeParse(shader)

shader.vs //template || valid glsl

shader = parseShader(shader) //sample OWN || PRIVATE 

shader.vs //valid GLSL 

//if you want to apply a regex on the entire valid glsl, or compile node material or something else
shader = onBeforeCompile( shader )

compile(shader)

1. Make WebGLRenderer not know about anything other than ShaderMaterial.

Wherever you would have

const mat = new THREE.MeshBasicMaterial()
const sm = new THREE.ShaderMaterial()

Have

const mat = nodeMaterial.compile() //returns ShaderMaterial, with a friendly interface (same as StandardMaterial for example)

const mat = chunkMaterial.compile()

The tl:dr of the stuff above is user does not care about the particular point in time when "compile" (parse) occurs. I think they care more about what happens, and what happens can be isolated to a handful if not a single use case in: shaderString, out: shaderString. The reason why it has to be done at that particular point in the pipeline i think is worth investigating and understanding. It seems to be getting in the way of things more than its helping.

@mrdoob I think it's difficult (or impossible) to serialize Materials hacked with .onBeforeCompile(). Do you think users should use ShaderMaterial if they want full functional(render, copy, clone, serialize, and so on) modified built-in materials?

I didn't expect (considered) the the use of onBeforeCompile() to be serialised...

I just got bitten hard by using onBeforeCompile. I have a shader helper class that allows me to make modifications to built-in shaders in a defined way. I use the onBeforeCompile method to do this. Apparently, like stated in the first comment, WebGLProgram uses onBeforeCompile.toString() as a hash. But since my function is generic (it substitutes parts of the vertex and fragment shaders with variables) onBeforeCompile.toString() doesn't look different for different shaders. This meant that all my different shaders got cached as the same program. An eval call and a uuid and now my functions all look different. Took forever to figure this out.

@donaldr See https://github.com/mrdoob/three.js/issues/13192. I think it would be good to document these limitations.

could you share how you're using the eval? I've been thinking about that but it just seemed too dirty. You could basically add const fdscxnmdrek435rkjl to the top of the body and then eval?

This meant that all my different shaders got cached as the same program. An eval call and a uuid and now my functions all look different. Took forever to figure this out.

That sounds very frustrating, sorry. 😞 If the modifications you needed to make seem like they'd make sense in the library itself please feel welcome to open issues, too.

please feel welcome to open issues, too.

Would you consider re-opening existing ones? #13192 seems to be the same but it was closed :crying_cat_face: It would be really nice if a work around was posted in there if it was deemed a feature.

Hmmm...

const _obc = shader => {...}
const obc = `
function (shader){
  const _${ hash() }
  ${ _obc.toString() }
  _obc(shader)
}
`
material.onBeforeCompile = eval(obc)

^Would this work? I never used eval.

The introduction of Material.customProgramCacheKey() via #17567 ensures that developers now have the possibility that shader programs are not shared for materials modified via onBeforeCompile() with conditional statements.

Please discuss other existing issues or enhancements in context of onBeforeCompile() in new threads (like #13446).