Three.js: Selective lighting

What I am saying is basically having a channel field in THREE.Light and lightChannel field in THREE.Mesh or something like that. If the latter is null then it is affected by all light sources. If the latter is not null then it is affected only the the light channels with the same value.

Or maybe it could be added not to the Mesh itself, but to separate faces of the mesh geometry.

Sounds like you want to use shadows?

Well, I could achieve something like that with shadows (and I've already tried), but it gives different side effects and I feel that it is a hack, because I don't need to light some objects and then cast shadow on them. I must not light them in the first place.

What I need is that a specific light source doesn't affect one set of meshes, but affects the other set.

Usually I do this using a technique like that: Example discussion thread:

glEnable(GL_LIGHT0);
//...
glEnable(GL_LIGHTn);

// Draw the walls to room 1
DrawWalls(room[0]);

// Draw the contents of room 1
DrawContents(room[0]);

glDisable(GL_LIGHT0);
//...
glDisable(GL_LIGHTn);

// Draw the walls to room 2
DrawWalls(room[1]);

// Draw the contents of room 2
DrawContents(room[1]);

So it looks like a different feature than just shadows. Or am I missing something?

Yes... I think this can indeed be handy. Not sure how the API for it should look like though.

Maybe the most simple solution could be the most effective here:

1. Add a channel (or group) to THREE.Light
2. Add a affectedByLightChannel (or affectedByLightGroup) to THREE.Mesh (Or maybe even to a face in the geometry)

What do you think?

Maybe the 'affectedByLightChannel' is too long and something like 'lightChannel' would do, but I think it would be convenient: just channel numbers on light source and receivers.

Like that:

    light = new THREE.PointLight(0xFFF7D6, 1.0, 15)
    light.channel = 123
    testScene.add(light)

    testScene = new THREE.Scene
    geometry = new THREE.BoxGeometry(2,2,2)
    material = new THREE.MeshLambertMaterial 
        color: 0xffffff

    cube = new THREE.Mesh(geometry, material)
    cube.lightChannel = 123
    testScene.add(cube)

If lightChannel is equal to 0, then it is affected by all channels. If channel is equal to 0, then it affects all meshes.

So it would be fully backward compatible with current behavior.

That may be a bit too hard to understand... Maybe it's better something like this:

cube.lightInfluences = [ light1, light2 ];

Seems absolutely fine to me.

Maybe it is a bit more difficult to use in some cases, but it is easier to understand, obviously.

What about a simple integer mask property for meshes and lights?

light = new THREE.PointLight(0xFFF7D6, 1.0, 15)
light.mask = 0xffffffff; // default mask
testScene.add(light);

cube = new THREE.Mesh(geometry, material)
cube.mask = 0xffffffff; // default
testScene.add(cube);

Objects are then only lit by lights if the logical AND of their masks is non- zero.
This would allow lights to be influenced by more than one channel, with no extra methods on the objects.

A mask default of 0xffffffff would not affect existing code.

What @satori99 said.

Although, I think mask should be a property of Light and Mesh*Material, instead. (Only those materials affected by lights.)

Also, the property could alternatively be named lightMask, lightChannel, or channel.

The problem of the channel/mask approach is that the user will need to understand bitwise operations. A bit too intense if you compare it to the rest of the API.

What's something you can do with masks that you can't do with the array approach?

I could give an example from the task with two rooms above.

The main point of using the channel approach over the array approach is that simple operations like 'move light1 from room 1 to room 2' become more complicated if you use arrays.

Instead of just setting

light1.channel = 2

(previously it was set to 1)

you would have to find all objects in room 1 that had light1 in lightInfluences array previously, then remove the light from their arrays, then add it to all objects in the room 2.

Same story with the simple operation like 'move object 1 from room 1 to room 2'. Instead of setting its influenceChannel from 1 to 2, you would need to find all the lights in that room, then remove them from its influence array, then find all lights in room two and add them.

It's not that it can't be done, that's why I said that the lightInfluences approach is absolutely fine to me. But the channel stuff would be the first thing that I would implement above it for myself just to make common operations as simple as 1 assignment.

I think it should be implemented as a mask. (Whether is it implemented on the CPU or GPU is an issue for later discussion.)

We can show via examples how to set it, and users can follow the pattern.

If you think that is still too complicated, then we can create a THREE.Channels API for it.

light.channels = new THREE.Channels();
...
light.channels.clear();
light.channels.add( channel );
light.channels.remove( channel );
light.channels.all();

Same methods for Mesh*Material.

I like that API :)
I can see this working for objects and lights, but how do you see it working for materials?

Only materials respond to lights. This would have to be a property of the material, I would think.

I agree on this with @westlangley. Lights are dependent on materials.

Same story with the simple operation like 'move object 1 from room 1 to room 2'.

Well, that's a problem. Channels would not be object-based.

Well, that's a problem. Channels would not be object-based.

But why? Is it a technical limitation?

Because it kind of invalidates the whole idea of all that. Because there can be different objects, which reuse the same material, but one of them should be lit and the other -- shouldn't.

But why? Is it a technical limitation?

No. It is because objects do not respond to lights. Only materials do.

Because there can be different objects, which reuse the same material, but one of them should be lit and the other -- shouldn't.

You can use the same material for all objects in the scene -- just clone the material for objects whose material requires different uniform values. There should still be only one shader program shared by all.

I think it should be implemented as a mask. (Whether is it implemented on the CPU or GPU is an issue for later discussion.)

Can this be handled somewhat easily in the GPU directly?

Can this be handled somewhat easily in the GPU directly?

Yes, you would need to pass in the additional channels uniforms for the lights and materials.

How about a layer management system? I'd group the meshes into layers and apply the masks from there (could affect lights, shadows, visibility, etc), unity would be a good example?

Shadows is also a related topic. I think there should also be something like selective shadow casting. Like 'receiveShadowFrom = ...' (and list of sources) instead of just 'receiveShadow = true'.

Because when you set lights that only affect a specific room (in my example), you would also immediately want those lights to cast shadows only on this room objects.

Shadows attributes should really be on the materials and not the objects for the same reasons as above in this thread.

Shadows attributes should really be on the materials and not the objects for the same reasons as above in this thread.

Yes, it makes sense!

Are there any plans for including this feature (like a planned target release for the first draft)?

As a work-around, can you achieve most of your requirements by having a separate scene for each room (and its lights)?

renderer.autoClear = false;
...
renderer.render( scene1, camera );
renderer.render( scene2, camera );

Hmm, I'll try this approach!

Selective lighting / shadows is a must for future versions IMO.

BIG +9999 for this one, I would love to be able to select if a material casts a shadow from light source A or from light source B. Anyone got a solution currently other than two scenes? That's gonna make things pretty painful for me...

holy crap how is this still not implemented? after +9999 from rohan )

Haha my guess is that it's kind of hard to implement @tsone can you provide some info on that commit? Browsing on a phone client right now

how is it hard to implement if these are just uniforms they pass to materials. all they have to do is substitude global list of lights with the one defined in material, if it exists, somewhere deep in webgl renderer.

Here is a simple JSFiddle that tests the Dev branch's Layers implementation:
https://jsfiddle.net/Angrypickle/t8f7q3vo/4/
Unfortunately, as of this moment, it doesn't look as though it's working properly. Or am I doing something wrong?

Yes. Layers don't work with lights yet. They do work with camera/object though! 😊

Roger that sir! Here is an updated JSFiddle that uses overlapping cameras to achieve selective lighting with layers:
https://jsfiddle.net/Angrypickle/t4a1eusL/
It seems to work properly on desktop and mobile. Anyone see anything profoundly bad with this approach? At least until lights are tied into the layers functionality?

Anyone see anything profoundly bad with this approach?

of course it is bad. rather than having single camera navigating the scene, people will now have to do things camera1.add( camera2 ); I mean this is WTF when I look at it. Like... what? camera in my camera? what if I have dozens of rooms that need to be individually lit? how many cameras do I need? and layers... there were no layers yesterday, right, and now I have to l̲e̲a̲r̲n̲ about them.

rant over, not to say I have the perfect solution for this. e g alternativa3d people used to put lights on stuff that was in light's bounding boxes. this had an advantage of next-to-0 setup for end users, but it was falling apart when the border between lights had to be at the angle. but still, if I had to solve this problem in actual project right now, I would most probably resolve to scraping used standard materials into ShaderMaterial-s and passing the lights I want to by hand there.

@makc layers are actually super simple (and powerful)!
@Zob1's approach is definitely not the right approach. Hopefully it won't take long until layers also work with lights.

Guys just to make sure I am following correctly; let's say I have a single scene and a "main" light source. Is it possible with this technique to have certain materials ignore this main light source, while also casting shadows from a different light source?

Well, for that we'll also have to add layers support to materials.

What we can do at the moment is show/hide objects from the camera:

Say that you are doing a game and, in the editor, you use spheres to display colliders. Those spheres could be set to layer 1, the camera's editor could have layers 0 and 1 enabled, but the game camera could have be set to layer 0. That way, in your editor you see all the dummies, but in the game they're gone.

I think this will make more sense whenever I add the functionality to the editor itself. We should probably do some examples too.

Right! So basically what I am wanting to do is, I have 2D characters in my Three JS world. These characters need "spot" shadows to make them look like they are part of the environment. I am currently achieving this using transparent black circle geometries placed at their feet and some fairly hacky stuff to get it working at angles. Even then it doesn't work well at all on complex surfaces.

My original idea was to have invisible circle geometries places as "halos" above each character and create a light that is pointing directly down, covering the whole scene. These halos would only cast shadows from this light source while everything else in the scene would cast shadows from the "main" light source.

I guess for this idea to work, we need layers right?

These halos would only cast shadows from this light source while everything else in the scene would cast shadows from the "main" light source.

We used to have a shadowOnly option, but it was removed.

FWIW, there is this approach to creating shadows...

Ah but shadowOnly wouldn't really work because then the halos would cast shadows from both light sources, I only want them to cast shadows from one.

I will check out that example, on the phone right now but looks promising.

Edit Hmm but if that sphere changed its position in the scene and the ground mesh had varying heights, would its shadow match the surface normals of the ground correctly?

Nope...

Yeah thought so. Need layers! Hehe

Hi @rohan-deshpande ,

Yes for your use case, you're going to need layers of some sort. I was the one who implemented the shadowmesh feature a while back. They were meant to be used for a scene containing a flat floor or ground, because they are single-planar shadows. However, if you ever need fast, cheap (but correct) single plane shadows, these are hard to beat performance-wise. The reason I added these was because even for simple demo scenes, shadowmaps dropped my framerate considerably. Shadowmeshes on the other hand run fast even on my phone.

You can check out my game that uses them here:
https://github.com/erichlof/3dLightCycles

I'm rendering the scene twice through 2 different viewports and all game objects (cycles and trail walls) have correct shadows. It runs silky-smooth on my smartphone, which would be impossible with shadowmaps for 2 scene renderings every frame.

+1 here for layers in the future :)

@erichlof game looks and runs great on iPhone6 man.

Okay I guess I'll wait for layers. Until then my hacky solution will have to suffice.

Hey guys
Is this thing implemented?
http://stackoverflow.com/questions/33689781/casting-shadows-based-on-a-specific-light?noredirect=1#comment55153706_33689781

No it hasn't been implemented yet. You'll have to wait for layers or try one of the solutions listed in ITT (separate scenes for example).

+99999 again

This would definitely be very helpful. And we now have layers working, which it sounds like is what's been holding this up. So, here's the friendly bump. ;-)

@manthrax check this!

+99999 again

I'd love for this. I have a situation where each avatar is lit by only the nearest light, and I can have thousands of lights into the scene, it'd be nice to specify a single light for each avatar.

Hi!
I'm going to need this feature, and I thought of implementing it, since I think I've located where this could be done (initMaterial in WebGLRenderer) and it should be pretty straightforward using existing layers.

I'm thinking of creating a light-specific bit mask (as a Layers object) to select which layers are affected by a specific light object (independant of the light's own object layer), then filtering lights/shadowmaps that are set as uniforms for each object in the above-cited function.

Would that be relevant?

I'll try this and submit a pull request as soon as I've acheived this unless you redirect me to some other method before that.

UPDATE:

https://github.com/tiesselune/three.js/tree/selective-lighting

I'm getting somewhere. Not sure how this might affect performance or how to optimize my code for perf, but it's working on the few tests I've made. Per-layer lighting! Yay!

I still have to test some more though, and make a relevant example before submitting a pull request.

UPDATE:

I've made a few more tests, and having a same material on two objects that are on different layers (and have different lighting settings) causes the renderer to update the material constantly (via needsUpdate) since the lighting settings on the material changes between two objects, which was impossible before. This means a huge perf drop and visual oddities (since the last material upload wins at render time)

Of course, this can be avoided by creating a new material instance instead of using the exact same one for two objects with different lighting settings. But I doubt cloning it for the user is a good idea, since it would mean watching the original material for changes to mirror them onto the clone.

Here are two solutions I could use:

1. I could just display a warning when two objects on different layers with different lighting settings share a same material, so the user can create and manage a new clone material by himself.

2. I could add layers at a material level instead of the object's level (and create a parrallel layers system for lights) so that it's mandatory to use different materials to achieve different lighting settings. That would mean there would be the current layer system for visibility (on objects and cameras) and a different one for lighting (on materials and lights).

What do you guys think?

I was just thinking about this as I saw your comment come in. I think displaying a warning is not helpful, as many people probably won't see it. Of the two, I definitely prefer setting light layers on the material rather than on the object. It's a little counter-intuitive, but it still makes sense.

Just to clarify, when needsUpdate is set, in this case, it just recalculates the uniform values, right? It shouldn't have to re-compile a shader because the different shaders should all be cached. What kind of performance hit are you seeing? Did you use a profiler to see exactly where the extra computation is happening?

By the way, filterAmbiantLights should be filterAmbientLights.

You are right about the misspelling. I actually had written everything with "ambiant" before noticing it was wrong and I forgot an occurence. 😇

I think I'm gonna move the layer info on the material then. It seems more relevant and consistent with the way it works.

The performance drop came (and still comes) from initMaterial (and then acquireProgram and getShaderParameter which is tremendously slow for some reason) being called every frame several times because the lightHash would be different everytime, setting material.needsUpdate to true.

BTW I'm using Chrome's profiling tools. (Timelines and profiler).

That's fine by me, but I'm not clear why acquireProgram should be running all the time. If the number of each type of light gets added to the program parameters, then shouldn't recompiling the shader each time be unnecessary?

I don't get it either;

Here's the call stack (at every frame) and the terrible memory leak that follows, if you're familiar with these, that could help me understand what goes wrong. It looks like it's creating new WebGLPrograms every frame...

Maybe it could be replaceLightNums that updates the vertex/fragment shader code during the WebGLProgram's creation, causing it to recompile, or considering it's a different shader... ?

EDIT : Anyway, using layers at material level solves the problem as expected. I just have a memory leak somewhere that I should investigate but otherwise it looks quite good.

It's probably a good idea to understand why this is happening just to make sure you fully understand the code and are not introducing some new bug. Can you step through it to see why the shader code changes every frame?

I don't know the WebGLPrograms well enough to say what's going on, but it seems like it should be caching those shaders for you.

It's quite simple, actually:

• to know wether or not to call acquireProgram, initMaterial gets the shader "code" from the cache (it's more of a hash, actually, constructed in in getProgramCode) and compares it to the"code" of the current material's program.
• Since this function creates the "code" from parameters including number of each type of light, which is filtered and though different between objects, the code can be different for a same material ( so to say, the same program) from an object to the other.
• Since this process is iterated through for each object, the "code" changes several times per frame, causing it to recompile as many times in a single frame: which causes the perf drop.
• That said, since several objects have the same material, and so, the same program, the last object iterated through imposed its parameters to the program, which caused both objects to be lit with the same light setup but not necessarily the same shadowMaps (that can be calculated for objects in a different order, but suffers the same "last object wins" problem), causing my visual oddities.

Now everything works as expected (since each object with different light setups have a different material and a different program), except for a memory optimization I could do with my own light hashes, which I'm going to tackle soon.

There could be some improvement if light numbers were variable, so that each material are just instances of the same program, but since those numbers are inserted in the shader code (in replaceLightNums) it seems incompatible with the way things currently are (and their optimization).

Hope the explanation was understandable enough

I wonder if we can pass the layer mask to the shader (one uniform per object and one more uniform per light). Then we could add something like this in here.

Something like...

for ( int i = 0; i < NUM_POINT_LIGHTS; i ++ ) {

    pointLight = pointLights[ i ];
    pointLight.distance *= float( object.mask & pointlight.mask > 0.0 );

    ...

}

Meh, I don't think we can do bitwise operations in glsl...

Yep, it looks like bitwise operations begin in GLSL 1.30 and that standard WebGL uses 1.00. 😕

Hi @mrdoob and @tiesselune ,

Chrome 56 and Firefox 51 have just landed, which means WebGL 2.0 is enabled by default (WebGL2 follows the OpenGL ES 3.0 spec). That means that Bit operations (and other cool stuff like 3d textures) are now available. I don't know if Three has been slowly preparing for the shift to WebGL 2.0, but I made a few mandatory changes to my copy of three84.js and enabled WebGL 2.0, and did a bit operation in a shader just to make sure it worked, and it did!
https://developers.google.com/web/updates/2017/01/nic56
https://developer.mozilla.org/en-US/Firefox/Releases/51

Just wanted to let you know. +1 for the masking idea :-)

Hi @erichlof ! That's great news for WebGL2; it means that WebGL2 renderer will probably be able to approach the masking problems in a more efficient way.

Anyway, given the very limited use of browsers supporting WebGL2 this far, I don't think we can discard efforts to make it work in WebGL 1 : it's taken a long time until nearly all browsers can run WebGL apps, so it will probably a bit longer until WebGL 2 is really usable at a large scale... 😕 But thanks for the hint!

Hi again!

New update: I've done what I thought necessary for memory management, but JS heap garbage collection seems to have a lot to do with my software configuration (running programs, open tabs, extensions) at the time I run profiles. Can someone with a bit more experience on that confirm that my code is not that memory hungry compared to the current version of threejs?

Anyway, I've made an example in examples/webgl_lights_selective_lighting.html. The thing is, I got shadowmap-related visual artifacts when using two spotlights (on the same layers) simultaneously on the skinned objects of the scene. The static objects behave as expected. I've run tests and it happens on the dev branch also, without considering selective light layers at all (actually, I just added a spotlight in the shadowmap example, on the dev branch). Does anyone know where that comes from?

Here's a screenshot:

It should be possible to view the example live here:

https://rawgit.com/tiesselune/three.js/selective-lighting/examples/webgl_lights_selective_lights.html

I'm getting an error though:

Uncaught TypeError: Cannot read property 'set' of undefined
    at init (webgl_lights_selective_lights.html:117)
    at webgl_lights_selective_lights.html:67

@looeee : you'll have to run npm run build-uglify on a local host. I voluntarily did not include builds in my commits for mergeability purposes...

Or should I?

EDIT : here's a working link on a different branch for test purposes : https://rawgit.com/tiesselune/three.js/selective-lights-test/examples/webgl_lights_selective_lights.html

Hey, +999999 from me. This should be implemented (at least with THREE.Layers)

$$('.comment-body').reduce((acc, el) => {
  let mat = el.textContent.match(/\+(\d+)/)
  let num = +(mat && mat[1] || 0)
  return acc + num
}, 0)
>> 1219997

🤔

@mrdoob he calculated +N-s in this thread

btw, +14570 for selective lights

This will be incredibly great for fine tuning how lighting works in a higher abstraction.

For example in the following pen, I'd like to have lighting work one way for the DOM elements vs another way for the Sphere in order for lighting to be more realistic:

https://codepen.io/trusktr/pen/RjzKJx

Certain effects that look like a single-point-light can be achieved by combining two or more lights that are selectively affecting only certain elements.

For example, in my previous example, I can increase the light intensity to get a good looking shadow on the "DOM element", but then the Sphere will look a bit too shiny and bright. If I were able to have a dimmer light for the Sphere, and a brighter light for the "DOM element", then I could achieve something more realistic this way, where it would seem to the viewer that there is still only one light. Then this sort of thing can be abstracted away in a higher-level API that makes it seem like there is only one light being manipulated when behind the scene two Three.js lights are actually in play.

@WestLangley

As a work-around, can you achieve most of your requirements by having a separate scene for each room (and its lights)?

Do you have a link to an example of something like this? Can it have other unexpected rendering effects on the outcome?

other unexpected rendering effects on the outcome?

rendering order of transparent objects, for example - transparent objects in scene 1 will be rendered before opaque objects in scene 2.

Do you have a link to an example of something like this?

here, I made one just for you: https://jsfiddle.net/f2Lommf5/524/

rendering order of transparent objects, for example - transparent objects in scene 1 will be rendered before opaque objects in scene 2.

That's what I was thinking; it makes the workaround only a workaround for very limited cases. Looking forward to the real solution!

well, you could argue that the problem would only happen if you could see opaque objects of scene2 from scene1 trough said transparent objects. but if that's the case, the lights should pass through as well, and there would be no reason to separate the scenes to begin with. but I agree this kind of argument is not really convincing.

@makc Here's an example of the problem I'm trying to solve:

https://discourse.threejs.org/t/how-to-make-shadows-darker-on-transparent-objects/1389

I think selective lighting would really help with this.

@trusktr I think you can solve that one by altering mrdoob's shadow material. it just shows black-white shadow texture, and you can modify it to be transparent as necessary

Masks in PlayCanvas seem really easy to use for selective lighting: https://forum.playcanvas.com/t/set-certain-object-to-not-receive-light/785.

can we get this merged in ? I think the shadow problems were due to Lambert shader nothing to do with the change to the code. @tiesselune

@ErikBehar I was actually looking for something to do this afternoon, I'm going to try to update the code to latest threejs and submit a pull request maybe? I'd actually enjoy knowing I made a real contribution to such a big project. (And good to have confirmation that this bug had nothing to do with my changes)

EDIT: just discovered that this part of the code changed a lot. Gonna need a little time I guess since the states that I was previously using moved from WebGLRenderer to WebGLLights.

@tiesselune I kinda needed to get this into a project so I went ahead and ported your code over to v93 see: https://github.com/ErikBehar/three.js/tree/selectiveLights

I can do a PR if you guys wish ? @mrdoob

@ErikBehar Well it does look like what I was aiming for but I did not have the time to do it entirely. I think I would have moved the filtering/hash functions out of WebGLRenderer and added them to the WebGLLights state object though. I feel like they belong there now, the layer system being a part of the state and not a part of the renderer.

@ErikBehar a PR would be great!

@tiesselune I'll look into moving it as you suggest, or feel free to push a PR on my fork lol = ] and I'll post the PR soon @mrdoob

What's the status on this? I've merged the r94dev code from @ErikBehar into r94, then merged r97 into it, only a couple very straightforward conflicts (version change, a few variables for selective lighting, and the hash creation in renderers/WebGLRenderer.js; I'd be happy to put in a PR. @tiesselune if you can give me some idea where you intended the selective lighting state to go, I'd be happy to move it, test, and put in a PR.

EDIT:
A little later: I see that this needs some work to function, now, with the new lighthash.

@VaelynPhi I'd think that, regardless of whether they are willing to accept @ErikBehar 's code or not, it is good idea to submit the PR, just to replace the outdated one in case other people will want it, like you yourself did

Sorry to drop the ball on this guys = / ... @VaelynPhi can you post your fork / branch ?

No worries; I understand being busy. Alas, I couldn't get even the branch you had working, @ErikBehar; I settled on reading through the code to try and break it down so I could move the state to the appropriate location and hopefully fix any bugs. I have yet to get to a working port state even on v94. Perhaps I can clean it up and put in the PR just to update it as @makc suggests. Give me a bit; I'm super busy. :) At the very least maybe that'll help highlight the changes that need to be made to pull selective lighting into the recent version.

added a PR based on:
https://github.com/ErikBehar/three.js/commit/ac0499b70b82bc7bb780100a8372fcdf318d1424#diff-5e43a0b5002eb2c419def3baf67d4e67
by @ErikBehar
can anyone give a hand with some review and the example ?

https://github.com/mrdoob/three.js/pull/15223

Hey guys, what's the status on this one? @tiesselune @ErikBehar Might I help you with something? Would be nice to implement it finally after 4 years 😄💯

@flyrell I think we could maybe close this issue since it seems to be closer to done over in #15223 maybe ?