Three.js: SkinnedMesh bug when far from scene root (0,0,0)

Um, your demo looks good on my Pixel.

Confirmed issues on my iPhone SE —

WebGL precision issues in iOS perhaps

But the iOS devices actually use the float texture based code path of the skinning implementation, right? Can you verify this with the following conformance test?

https://www.khronos.org/registry/webgl/conformance-suites/1.0.3/conformance/extensions/oes-texture-float.html

iPhone SE has five failures on that page. 😕

I made a small screencast to show you in my project:
http://cornflex.org/webgl/skin-bug.mp4

If the avatar goes too far from 0,0,0, the skin starts to get really buggy as you can see.
I made a deep profiling of the skeleton and bones. All seems fine except the geometry is "pulled back" in some way... I though about bad animation root but again, all seems fine in Threejs with that...

The prob is clearly linked to the Skeleton and the bonesTexture of the Skinning shader.
I also though about precision but as Mugen said, iOS device should support that with ease.

... why only on iphone then :/

I'm quite stuck... I'm reorienting to png sequences as backup plan ... :(

This is most likely due to the iphone 6 not supporting enough bones.

I had the exact same problem.

Check your bone limit here: https://virtulo.us/2/vr/test

I don't think so.
If the skinnedmesh stays at 0,0,0, there is no prob at all :/
I'm even thinking about moving all the assets instead of the avatar... (this is my workaround #2) but I'd surely prefer to have the skeleton working as good as desktop of course.

A friend just reported the same bug on iPhone 8

What results do you get with the following conformance test?

https://www.khronos.org/registry/webgl/conformance-suites/1.0.1/conformance/misc/shader-precision-format.html

After looking at the ThreeJS sources for 2 days... I think there is a prob when the bonesTexture is updated.
It looks like this texture doesn't udpate correctly and then push back the vertices to original positions... but I could be wrong of course.

Hmmm... here I have some FAILED, look:

Maybe the problem is related to these fails. My Pixel for instance passes all tests.

Let's try something out: Go to WebGLCapabilities and set the value of floatVertexTextures to false (this will prevent the usage of the float texture). Make a build and use this three.js version in your app. I'm curious what happens 😊.

https://github.com/mrdoob/three.js/blob/099e4364541502fdf22fc7b1c0f54239d2ba1708/src/renderers/webgl/WebGLCapabilities.js#L105

Already tried floatVertexTextures = false on the renderer.
The skin stops going crazy but there is no animation anymore :/
I'll push a sample.

Here you are:
http://cornflex.org/webgl/skinning-bug2.html

renderer.capabilities.floatVertexTextures = false;

=> No animation running anymore

Yeah, same on my Pixel (Desktop works). I guess we are hitting a uniform limit.

I made a last sample, with bug appearing progressively:
http://cornflex.org/webgl/skinning-bug3.html

Starts from 0,0,0 and goes forward... wait a few seconds... skin starts getting crazy.

It's quite weird all is OK at 0,0,0. isn't it?

z: 1255 > still OK
z: 18870 > crazy skin

As I said, I deep profiled all the skeleton and the bones positions on CPU side. All values seem to be quite OK. The only clue I have is the bonesTextures and bonesMatrixes not working correctly on iOS when the mesh is getting far from 0,0,0.

I checked the bones positions because it's a typical behavior of steady bones of the root node or things like that.... but no...

I checked about float precision on GPU side in your skin shaders but again... nothing seems to have an impact.

Very strange nobody moved a SkinnedMesh inside a scene and never reported this :/

One more question: Do you have the same problem with Chrome?

Same prob yeah.

Sounds like a precision issue. Have you tried scaling the scene down? (11195 is 11 kilometers).

Another option would be to move the scene instead of the character. Tends to be a common solution for big scenes.

Also, I guess the skinning code is in world space. We could investigate this.

I see your bone count is 67.

My (wifes) iphone 6 has a Max Hardware Bones limit of 27.

The deformations I see are identical to those I saw on her phone in my game (and the same problem I had before I had my animator fix the meshes armitures). I'd be surprised if this wasn't at least part of your issue.

Why it gets worse over time? No idea. Someone else mentioned something similar, not sure if it's related: https://discourse.threejs.org/t/updated-with-fiddle-as-animations-go-faster-there-is-increasing-error-in-accuracy/1707/6

Another annoying thing I found on iphones (iphone 6 at least): I MUST use highp shader precision, mediump and lowp don't distort the mesh in that same scary looking fashion (as my bone issue on the phone), but the textures look weird and garbled/pixelated while animating.

What is your hardware bone limit on the device you are testing?

@mrdoob I already tried to scale down but the skinning goes crazy much faster in fact :/
If you scale down at 0.1, then the bug is appearing 10 times faster.

The demo I used here to explain comes from the threejs examples.
In my game, the scale is not that big but the bug is the same, only appearing faster.

Indeed, The skinning seems to be done in worldspace, or localspace but getting stretch when moved far from center. That's why it's running quite well at 0,0,0

Moving all the world instead of the avatar is my workaroung #2.
You must admit it's a bad choice when you developed all the routine of a runner game in worldspace already with physics and all. And z=11000 is not that far for a runner game :/ ... and it's just a running man, not a ship.

I made plenty of games with ThreeJS but it's the first time I use a skinnedmesh. This iOS bug is really annoying.

Workaroung #1: I'll do 6 png sequences as spritesheet animations. Then I can keep a worldspace logic. Deadline is not far :(

@titansoftime The prob is appearing on the iPhone 8 of my wife too. This is not related to iPhone 6 only.
Also, if you have a max bones problem, then it will be buggy from start. Here, you can see it's not buggy at 0,0,0. Then the bones limit is not the prob. Worldspace calculation of the bones and the bonesMatrix in skinning shaders is, IMO.

What is the Max Hardware Bones of the iPhone 8?

I do not know. And I'm not sure it's device dependant.

But again... IMO, it's not related to maxbones value at all.

The sample is working correctly on iphone 6,8,X ... even 5... only when the SkinnedMesh stays at 0,0,0
https://threejs.org/examples/webgl_loader_fbx.html

If maxbones was a problem, we would not have a good skinning at 0,0,0 neither.

Related:

https://www.opengl.org/discussion_boards/archive/index.php/t-159613.html
https://www.opengl.org/discussion_boards/archive/index.php/t-159364.html

I'd like to highlight this section of the user zeoverlord:

The jittering thing is only natural for floats, variables will loose precision the higher the number is, so precision is the best the closer to 1.0 it is. Now this is not really a problem but it does mean that if you are doing a lot of matrix calculations with huge numbers you are going to have a lot of precision degradation(aka jittering). So you need to do all of your math around the origin with reasonable numbers and then translate it away

Mmmh, very interesting @Mugen87! It sounds you spotted the leak.

z: 1255 > still OK
z: 18870 > crazy skin

The error might be on your side.
A common mistake: the skin modifier was not properly applied.
Error 1: vertices controlled by too many bones
Error 2: vertices without weight (or with a very small weight).

When the SkinnedMesh is close to origin (0,0,0) you won't notice these errors.
But when is moved far away ... you will blame the mobile device precision.
So the first thing to do is to redesign (properly this time) the skin modifier.

@RemusMar Did you look at the source of the page ?
The example I exposed came right from the ThreeJS sources.

Quite well mentioned in my description ;)

I ran many tests before posting this... looking for a clue in my animated exports but I found the bug is happening with any kind of formats (GLTF, FBX, Collada, JSON, ...)

I made a clear example with clean source to expose the problem. It's quite easy to verify it on a smart phone:

http://cornflex.org/webgl/skinning-bug.html

The example I exposed came right from the ThreeJS sources.

So what?

How the error can be on my side when this happening with all the skinnedmesh examples from ThreeJS :)

You said:

The error might be on your side.
A common mistake: the skin modifier was not properly applied.
Error 1: vertices controlled by too many bones
Error 2: vertices without weight (or with a very small weight).

FBX is from ThreeJS... and you can try with GLTF or any other format. The prob is the same.

var loader = new THREE.FBXLoader();
loader.load( 'https://threejs.org/examples/models/fbx/xsi_man_skinning.fbx', OnFBXLoaded);

@Mugen87 has found a clue. Let's hope the guy who made the gpu skinned mesh can fix it.

How the error can be on my side when this happening with all the skinnedmesh examples from ThreeJS

I said "The error might be on your side.".
The bottom line:
Use a simple SkinnedMesh with properly applied skin modifier:

• max 4 bones per vertex
• all the weights = 1.0000

See if you can reproduce the bug.

Are you mad? :)
This bug cannot be on my side... the models (fbx, gltf, dae, whatever) and code source come from ThreeJS.

Please read the @Mugen87 posts.

Use a simple SkinnedMesh with properly applied skin modifier:

• max 4 bones per vertex
• all the weights = 1.0000

See if you can reproduce the bug.

I have to go now.
cheers

Hmmm. You know what guys? This is happening on desktop too :(

As the ThreeJS example mesh is huge (100m tall), I placed it very very far: z=10000000
And look, same prob:
http://cornflex.org/webgl/skinning-bug4.html

It's definitively something related to worldspace bones positions and bonesMatrix calculations, as exposed by @Mugen87 :

https://www.opengl.org/discussion_boards/archive/index.php/t-159613.html
https://www.opengl.org/discussion_boards/archive/index.php/t-159364.html

As explained before to @mrdoob , I already tried to downscale the mesh but the prob is even worse... even on desktop and you encounter the bug not that far from 0,0,0 then.

In my own game (with realistic scales 1:1), the bug is already appearing at z:100.

You continue to use bad designed skinned meshes ...
Anyway, to close this subject, here is the 1 to 1,000,000 (one million !!!) case study:
http://necromanthus.com/Test/html5/Lara_1000000.html
Click on the stage to switch between Y=0 and Y=1000000
A minor global lighting change is available per switch.
So if the skinned mesh is properly designed, the result is (close to) perfect.
cheers

We'll look into it after this month's release.

@mrdoob Thank you.

@RemusMar You continue to ignore what it was said here. Please read the entire thread again :)

I just gave you a working example but you still don't understand.
http://necromanthus.com/Test/html5/Lara_1000000.html

And you continue to talk about bad practices.
Please keep in mind the following:
The single-precision floating point number of digits is 7.
That covers 1234,567 and 123456.7
Most of the weight maps come with 4 decimals, but even a single one provides decent results.
For this reason all the 3D engines recommed a max value of 10,000 for world size.
From 0000.001 to 9999.999
We all want "infinite" or huge 3D worlds, but that's not possible.
What are you doing is completely wrong from many poins of view:
game design, performances, animations and collisions.
I hope you got the main picture now.

Maybe related: http://davenewson.com/posts/2013/unity-coordinates-and-scales.html

@RemusMar
I understood quite well what you said but unfortunately your model has same problem... in lesser degree but still there:
http://cornflex.org/webgl/skin-bug2.mp4

Source:
http://cornflex.org/webgl/skinning-bug5.html

And if I give a position higher than z=60000, it's getting worse... and up to 100000 it's completely invisible.

We all want "infinite" or huge 3D worlds, but that's not possible.

LOL, I don't want infinite huge world... I just want to make a runner game, like I did many times before with ThreeJS... but this time I need a skinnedmesh as avatar.

My scene: http://cornflex.org/webgl/skin-bug.mp4

If we are unable to animate an avatar running over several meters ... then what's the point?? (in realistic scale 1m = 1 threejs unit, the bug already appears from 100).

BTW, your model is like 150m tall... this is not realistic scale.
EDIT:
Here the same scene with scale(0.01, 0.01, 0.01) on your model, to get it realistic sized (around 1.5m then). As you can see, it's already bugging at z=300 on mobile:
http://cornflex.org/webgl/skin-bug3.mp4

What are you doing is completely wrong from many poins of view:
game design, performances, animations and collisions.

LOL LOL... I suggest to visit my websites and blog about making games (http://quentinlengele.com, http://cornflex.org, http://www.ddrsa.com, http://br9732.com)

EDIT: your scene is not visible on iOS, empty screen (http://necromanthus.com/Test/html5/Lara_1000000.html).

The skinning seems to be done in worldspace, or localspace but getting stretch when moved far from center. That's why it's running quite well at 0,0,0

Let's see:

vec3 transformed = vec3( position );
...
#ifdef USE_SKINNING
    vec4 skinVertex = bindMatrix * vec4( transformed, 1.0 );
    vec4 skinned = vec4( 0.0 );
    skinned += boneMatX * skinVertex * skinWeight.x;
    skinned += boneMatY * skinVertex * skinWeight.y;
    skinned += boneMatZ * skinVertex * skinWeight.z;
    skinned += boneMatW * skinVertex * skinWeight.w;
    transformed = ( bindMatrixInverse * skinned ).xyz;
#endif

does not look like world space to me? in regular mesh, transformed is set to position attribute - that is, local coords.

so whatever happens does happen on js side, where bind and bone matrices are calculated. perhaps bind matrix does a lot of stretch, and bone matrices could be doctored to operate in other coordinates that do not require that much stretch in bind matrix... or something like that

(edit: nvm, I think I've got this one wrong)

@makc Unfortunately, I don't see enough revalant data in your copy/paste of the skinning shader to tell if the calculation are done in local or world space coords.

As I said, there is surely something wrong with the bones matrices calculation or maybe the order of these matrices calculation is not right. As you can see on the OpenGL forum and as @Mugen87 mentioned, this is the most revalant clue, at least to me:

The jittering thing is only natural for floats, variables will loose precision the higher the number is, so precision is the best the closer to 1.0 it is. Now this is not really a problem but it does mean that if you are doing a lot of matrix calculations with huge numbers you are going to have a lot of precision degradation(aka jittering). So you need to do all of your math around the origin with reasonable numbers and then translate it away

Again, as explained, the scale of the object is affecting the bug.
The example with ThreeJS sources presents that bug only at z=100000 but it's because the size of the mesh (coming also from ThreeJS source) is huge. The running boy is 150m tall.

When you are doing a game, you don't use that kind of scale of course. You always try to keep your world with 1m = 1unit. This is even mandatory to get good physics and behavior.

The prob here is when you use a SkinnedMesh with realistic size: the bug is already appearing at z=300.

In any engine, you can drop a SkinnedMesh at put it at x:30000, y:60000, z:140000000 and there is no prob with bones and skinning. No jittering.

Surely I understand quite well we are working on a Javascript core here... but still.

Trolls here pretending "we all want infinite worlds but it's impossible" need to get back to school and learn matrix transformations. Or better looking at open world games and ask them self how their avatar can be "skinnedmeshed" sooo far from scene root 0,0,0 ......

I have to say, this bug is fun. Here, I could reproduce it on the desktop, but 60K were not enough to run out of precision:

ok, so at z = 1e8, here is what we have with current shader:

and here is what we have if we change the shader to this (slight variation of my off-the-top-of-my-head test above that I edited out):

#ifdef USE_SKINNING
    vec4 skinVertex = bindMatrix * vec4( transformed, 1.0 );
    vec4 skinned = vec4( 0.0 );
    skinned += ( bindMatrixInverse * boneMatX ) * skinVertex * skinWeight.x;
    skinned += ( bindMatrixInverse * boneMatY ) * skinVertex * skinWeight.y;
    skinned += ( bindMatrixInverse * boneMatZ ) * skinVertex * skinWeight.z;
    skinned += ( bindMatrixInverse * boneMatW ) * skinVertex * skinWeight.w;
    transformed = skinned.xyz;
#endif

still shaky, but looks way better :) now, what does this mean in the context of this issue?

1st, you can't just copy-paste this hack in the shader. I mean, you can, but it replaces 1 matrix multiplication with 4. what I propose instead is either a) merge bone matrix with inverse of bind matrix on js side and pass as a single uniform, or b) create another uniform just for this piece of shader. It would then both solve this issue (somewhat) and decrease the number of matrix multiplications in the shader (by 1).

oh, and

Trolls here pretending "we all want infinite worlds but it's impossible"

I think what they were trying to say is that you could work around the problem without fixing the library on your side. Not necessary should, but could.

@makc Thank you very much for your time on this.
Unfortunately it's quite too late. I have to deliver Friday. I used PNG sequences as workaround.

If you want to debug this deeply enough. Could you try to use a realistic mesh size?
Again, this Lara model is also something like 60m tall if I remember well.

I'm quite curious to see if your patch is working on realistic scale too of course :)

Oh and... are you doing your test under iOS?
As explained in this thread, this is mainly happening on iOS devices (6, 8).
On Desktop, only very very huge values bring the problem.

Thanks again

@OrtOfOn you can try @makc patch by adding this at the top of your code:

THREE.ShaderChunk.skinning_vertex = `
#ifdef USE_SKINNING
    vec4 skinVertex = bindMatrix * vec4( transformed, 1.0 );
    vec4 skinned = vec4( 0.0 );
    skinned += ( bindMatrixInverse * boneMatX ) * skinVertex * skinWeight.x;
    skinned += ( bindMatrixInverse * boneMatY ) * skinVertex * skinWeight.y;
    skinned += ( bindMatrixInverse * boneMatZ ) * skinVertex * skinWeight.z;
    skinned += ( bindMatrixInverse * boneMatW ) * skinVertex * skinWeight.w;
    transformed = skinned.xyz;
#endif
`;

THREE.WebGLShader: Shader couldn't compile. :/

oh man. you ended up with wrong quotes and one-line formatting. how do you copy-paste like that?

also, I kind of did it to see these wacky images, that satisfies my curiosity. I trust that someone else will take it from here :)

@makc CTRL+C/CTRL+V ... just from here ... :/

Unfortunately, the bug is less stronger than before but still there :(

Tested on iOS (patch applied):

Mesh at scale 1.0 (avatar around 20m tall exactly), position z=10000
http://cornflex.org/webgl/skin-patch1.mp4

Mesh at scale 0.1 (avatar around 2m tall), position z=3000
http://cornflex.org/webgl/skin-patch2.mp4

@mrdoob I'll try to reproduce :) It appeared in one single line. I was writting on the page when I saw your reply popping and it did not appear like it appear normally, I promise :p ... I'll get these bonus points, you'll see :)

EDIT: BTW you edited your post ? :)

I believe the residual effect may be unrelated to skinning and actually affect everything but you do not notice because there is no animation.

@OrtOfOn in your place I would try to wrap your whole scene (with a camera) in extra Object3D and set this object coords to -1 * avatar coords every frame. This would keep it in 0,0,0 world coords forever.

@markc I really don't like to cheat like this. The gameplay and the scene are not designed to be tortured like that and as a mobile first game, perfs has to be optimal of course.

Also...not my case but imagine this with several of enemies in SkinnedMesh, coming from everywhere...

I had no choice because the deadline and I replaced the SkinnedMesh with spritesheet animations (client is OK). Otherwise I didn't except to encounter something like this. I use ThreeJS in almost every web project and I'm very very happy with it.

We don't want "infinite worlds" but we want to be able to use these nice SkinnedMesh and animations features in world space coordinates, like we do in famous engines :)

I'm a bit disappointed in a way... because I made a very cool Animator script based on crossfade features and inspired from mecanim... For next project surely :/

@markc I really don't like to cheat like this.

I do not think that's cheating at all. That's how all the space simulations work. In fact, realtime 3D rendering is basically offseting the whole scene so the center is where the camera is.

@mrdoob yeah I know a camera doesn’t even exists. Power of matrices :) (I do some native WebGL/OpenGL dev myself).

Anyway, putting all the scene into a game object seems a bit barbar to me... especially to get a skinned animation working.

@OrtOfOn actually you can use the scene itself. e g in lara example, do this:

scene.add(camera);
scene.position.z = -60000;

and it's ok (edit: however, there's something wrong with the light when you do this.... @mrdoob ?)

@makc Thanks again to propose patches.

I don't want to be pessimist at all but something tells me lighting will not be the only issue by doing this kind of manipulations. That's why I call these tricks and cheats because of the uncertainty of the result and the consequences to other parts of the pipeline.

Unfortunately, people making games have physics, raycasters, particles, ... lots of things that are designed and has to be designed to be in world space with realistic scales.

Also, I'm quite curious to see these patches in action with more than one SkinnedMesh, like enemies hiding or spawning at some places in the scene. Because what's the point to be restricted to one single skinnedmesh avatar at 0,0,0 then?

I'll dig deeply in the source as soon as I can but I have to deliver something in 2 days... so my spritesheet animations are quite fine for now hopefully.

Because what's the point to be restricted to one single skinnedmesh avatar at 0,0,0 then?

The point is that it's highly unlikely that you will simultaneously render the skin at 0,0,0 and another skin at z=60000. Only few skins around the 0,0,0 will make it to the screen, and they will not be far enough to trigger the bug.

@makc Again, your patch is valid under desktop but you are not testing the case on iOS devices, where the bug is appearing already at several meters from 0,0,0 with realistic scaled meshes. I made enough videos show to it. This one, for instance, presents the bug only at z=100 :/
http://cornflex.org/webgl/skin-bug.mp4

Are you always using avatar that are 20m tall in your games? :)
Because the number you focused on (60000) is only used to expose the bug on desktop (with that very huge lara or huge threejs fbx sample)... not on mobile. And my avatar doesn't go that far of course...

I would confirm this to be an iOS problem. Ever since at least the generation 6S/6S+, iOS has had a problem with floatVertexTextures. The fix we've been using is to disable that feature on iOS. This limits the bones we can use, but we have to limit them anyway to support crappier phones.

As iOS devices fail the conformance test posted by @Mugen87, it would probably be a good idea to disable floatVertexTextures, or probably even floatFragmentTextures, on iOS by default, as it clearly does not correctly implement a feature it advertises to support.

@daniel-nth I guess it makes sense. However... @OrtOfOn was saying z=60K reproduces the problem on desktop... I don't have any problems at 60K on desktop, only 1e8 was sufficient for me to "reproduce". So, maybe this issue is really at least 3 separate problems:

• float textures on ios,
• something at 60K on @OrtOfOn 's desktop,
• running out of precision at 1e8 due to bad matrix multiplication order in bones shader bit (the one I messed with above),
• maybe something else in the vertex shader, since the girl was still a bit shaky after the patch.

Hi Guys,
First, thank your for getting back to this bug. I still didn't find the time to dig in it :(

@daniel-nth The scale of the object is affecting the bug:

On iOS, with this very huge "Lara" object (height=120, scale=1), the bug appears at z=60000 indeed. But, if you take a realistic sized mesh (height=1.8, scale=1), it already appears only at z=100.

bigger the skinnedmesh, further the translation must be to see the bug.

As already tested and reported in this thread, disabling floatTextures on iOS results with no skinning at all.

I also suspect a bad matrix multiplication or bar order of multiplication of the bones matrix.

On iOS, with this very huge "Lara" object (height=120, scale=1), the bug appears at z=60000

@OrtOfOn wait, so on ios? but you also said:

the number you focused on (60000) is only used to expose the bug on desktop (with that very huge lara or huge threejs fbx sample)... not on mobile.

On iOS, with this very huge "Lara" object (height=120, scale=1), the bug appears at z=60000 indeed.

It's not huge at all.
And you still don't understand the relationship between model size and world scale.
For the last time:
http://necromanthus.com/Test/html5/Lara_1000000.html
Modeil height approx = 75 and world scale = 9,999,999.
From Z=0 to Z=1,000,000 (one million !!!)
Because the single-precision floating point number of digits is 7, there are no decimals left at all !!!
It runs perfect on all the laptops and desktops I have.
Even more:
it runs perfect on Android 6 & 7 and various Samsung mobile phones.

@RemusMar in your example, enter this into console:

camera.position.z += 1e8;
scene.children[2].position.z += 1e8;

you shall see something like this:

it's trippy

I'll give you that z = 1e6 does not look bad either. around 1e7 it is already noticeable, and 1e8 is just crazy

@OrtOfOn Setting capabilities.floatVertexTextures to false works for us on iOS. I suspect your model uses to many bones to fit into the uniforms.

I'll give you that z = 1e6 does not look bad either. around 1e7 it is already noticeable, and 1e8 is just crazy

I know, but for those values you are out of precision.
For the skinned mesh vertex weight precision, the max world size is 9.999.999
That translates in max distances of a few millions, no more.
cheers

@makc Indeed, I got the wrong numbers for desktop. It has to be very huge, you're right and I mentioned in the thread earlier in fact :/ https://github.com/mrdoob/three.js/issues/13288#issuecomment-364650679

@daniel-nth Good to know but are you sure? Here is the test I did here in this thread, with 3d model from threejs sources. You can still have a look at the code. @Mugen87 made the same feedback too, no animation with this model. https://github.com/mrdoob/three.js/issues/13288#issuecomment-364572653

Oh, I see the model is not present anymore. 404 on https://threejs.org/examples/models/fbx/xsi_man_skinning.fbx

@daniel-nth If you are using a new model with less bones, then I believe you ;)

@RemusMar Quite agreed with you about distances evoked in the latest posts. The numbers (1e6, 1e8, 1e9,...) will certainly result with bad precision. No doubt about that. But please stop trolling me. Stop pretending I don't understand anything and ask yourself if an avatar mesh with height of 120 is not huge. For me, it's very huge and not very usable in a gamedev with other realistic scaled objects made by artists, raycasts, physics, lighting... and to be delivered in 3 weeks.

I'm insisting: keeping assets at realistic scale with metric units is something mandatory to build a physic-based-game correctly :)

Too bad that, on iOS, I cannot run my avatar over 100m in these realistic scale conditions :/

UPDATE: Anyway, @daniel-nth seems to tell us it's now working with floatTextures off and a very limited bones count.

But please stop trolling me
I'm insisting: keeping assets at realistic scale with metric units is something mandatory to build a physic-based-game correctly :)

I'm not trolling you.
But 1.5 units does not mean 1.5 meters
cheers

@OrtOfOn

I'm insisting: keeping assets at realistic scale with metric units is something mandatory to build a physic-based-game correctly :)

but this scale is some arbitrary self-imposed value. you could just as well decide to use centimeters, and then "an avatar mesh with height of 120" is actually smaller than average.

@makc Sure, but physics engine/libs expect you to use metric values. Did you already tried to do some physics on scaled objects or with non-metric values? It becomes quickly awful to develop and you get unexpected and unrealistic results. All your forces must be reviewed and scaled too. It will work, but not accurately.

Of course, it depends the level of precision you need to achieve and values like 1e6, 1e7,... are way too big and will present precision issues. And you'll never need to move a skinnedmesh that far. We agreed about that. But, on iOS, we should be able to move a gpu skinnedmesh (1.5 units tall) in a range of several hundred units, at least.

but physics engine/libs expect you to use metric values.

That's not true.
Meters, centimeters, inches, etc are irrelevant.
As I said (several times) before, all it matters is the world size (scale) and the accuracy.
Keep in mind the following:
if you have a skinned mesh with height = 1.5 units, you are forced by the vertex weights map to use 2-3 decimals at least.
For the physics engine (if any) you might need 3-4 decimals for acceptable results.
If the global accuracy requires 4 decimals, your max world size is 999 (because the single-precision floating point number of digits is 7).
In other words, you can use distances of a few hundreds if you want decent animations and decent physics.
I hope you got the whole picture this time.
cheers

@RemusMar I'll stop here. It seems you didn't or don't want to test your case on an iOS device. You can't use "distance of a few hundreds" on these devices with gpu skinnedmeshes. The mesh goes crazy very quickly. That's the point of this thread and I just wanted to report that at start. So that's it.

I hope you got the fact that all your propositions and samples doesn't work on iOS devices, as reported by other people here and by my screen captures :/

You can't use "distance of a few hundreds" on these devices with gpu skinnedmeshes.

You are wrong again.
See the above screenshot (Android 7 - Samsung mobile phone).
Model height = 75 (bounding sphere radius approx. 38) and distance 1,000,000 (one million !!!)
Or model height = 1.5 and distance 20,000.

p.s.
Is not my problem that iOS was always a collection of bugs.

@RemusMar I'm not wrong, you troll. I'm talking about iOS and you're talking about another device... are you mad??

Not my prob if you hate Apple so much. When one of my client want a 3D web based game done with ThreeJS, I don't tell him "fuck you with your iphones"... If you do it, I'm sorry for you.

End of transmission here... I cannot follow this anymore.

@OrtOfOn Yes, the model I used has 19 bones, the 404 fbx has 67.

https://github.com/mrdoob/three.js/issues/13288#issuecomment-364550036 had a link to a page that shows the uniform bone limit. The formula that page uses is Math.floor((data.webgl.params.MAX_VERTEX_UNIFORM_VECTORS - 20) / 4); Not sure if that is 100% accurate for three.js, but it reports 27 even on iPhone X, which unfortunately is less than half of even a Galaxy S3, but still plenty for a mobile WebGL game. I quickly grepped through my work folder and in the 26 WebGL projects, only one has a model with more bones, and that model stays at 0,0,0.

@OrtOfOn : I am having similar problems with a fixed scene and moving a skinned character around in the scene. the character jitters terribly on ios. bone count is 22. dimensions of the scene are reasonable. i believe the problem is that using vertex textures limits you to 32-bit floats, coupled with the assertion above that Threejs computes the skinning in world space!

I am looking into the world space assertion now, and hopefully will implement the proper way to calculate vert positions in local space and then translate the object - as it should be. i've never heard of anyone using skinning to place the object in world space, so i'm not sure why the implementer would have done such a thing.

i'll post back when i have a solution.

just fyi, float textures on ios is not only broken with vertex shaders. I was recently changing float textures to integer textures to fix color banding on ios, so it's universal ios problem. you could probably use integer textures to fix the bones on ios, too, if you really wanted to - here's relevant snippet from somebody else's shader:

vec2 encodeFloatRG( float v )
{
    vec2 kEncodeMul = vec2(1.0, 255.0);
    float kEncodeBit = 1.0/255.0;
    vec2 enc = kEncodeMul * v;
    enc = fract (enc);
    enc.x -= enc.y * kEncodeBit;
    return enc;
}

float decodeRGFloat( vec2 enc )
{
    vec2 kDecodeDot = vec2(1.0, 1.0/255.0);
    return dot( enc, kDecodeDot );
}

...in fact, you could probably (I am assuming here, did not really try something like that) make float bone textures work, too, if you accept ios precision as global precision limit and store two floats in the texture instead of one

I've solved the issue entirely by having threejs do the skinning computation in local space - where it belongs - and translates the model to real world coords the standard way via the world space matrix.

The problems above are completely fixed for skinning on iOS. I should also point out that this solution is universal - works flawlessly on all desktop browsers as well as the Android devices I've tested. IMHO local space skinning is the correct way to go in nearly all cases, and this method should be the default for ThreeJS. @mrdoob : what is your take on this?

mesh = new THREE.SkinnedMesh(geo.geometry, mat);
mesh.bindMode = 'detached';
var skel = new THREE.Skeleton(bones, boneInvs);
var bsm = new THREE.Matrix4().fromArray(geoPrims.bsm);
mesh.bind(skel, bsm);

// make bsmInv identity for models that have multiple skins
mesh.bindMatrixInverse = new THREE.Matrix4();

// patch SkinnedMesh's updateMtxWorld:
// we're using local skinning, so don't keep resetting binMatrixInverse (leave identity)
// the reason we're using .bind here is for models that have more than one skin
mesh.updateMatrixWorld = PatchSkinnedMeshUpdateMW.bind(mesh);

// re-normalize since input data has been compressed
mesh.normalizeSkinWeights();

// remove bone roots and add to bone root list
for (var bi = 0, bl = bones.length ; bi < bl ; bi++) {
    if (bones[bi].parent && !bones[bi].parent.isBone) {
        bones[bi].parent.remove(bones[bi]);
        boneRoots.push(bones[bi]);
    }
}

Where 'boneRoots' should be added to the root scene object (so they get matrix updates but don't inherit all the matrices in the hierarchy leading up to the actual skinnedmesh). Add these boneRoots to your scene root (code not shown).

And here is the patch to replace threeJS' SkinnedMesh implementation (note this is only necessary since threejs' updatematrixworld clobbers the bindMatrixInverse that we want to keep as identity). Alternatively, you can bind to THREE.Mesh..prototype.UpdateMatrixWorld directly instead of using a patch function.

function PatchSkinnedMeshUpdateMW( force ) {
    THREE.Mesh.prototype.updateMatrixWorld.call( this, force );
}

@denbo-ft I think that sounds good to me. Would you like to create a PR for it?

@denbo-ft Thank you very much for your efforts. Sorry for late reply, I wasn't following this thread anymore.

we have this issue in our product as well - any browser (if you get far enough from 0,0,0) but more noticeable on iOS as it happens as soon as you're about 2-3 units away. Critical for us but at least we have a good laugh when we see this

Ok it looks like there's more people who need this. I have never contributed to a github open source project but i am familiar with git and will try to create a PR. :)

Same here. My SkinnedMesh's look like an episode of Dr. Katz, Professional Therapist on iPhones.

FYI, my post from May 21 has the entire solution. Not much fixes for the ThreeJS sources - most of the fix belongs in your loader and/or scene setup code for your individual projects.

The basic fix is to use mode 'detached' (which is undocumented, btw), and place the bone hierarchy into your scene at the root scene level - not in place with the actual renderable character hierarchy. Then patch up one simple thing from the engine (SkinnedMesh.update), and you should be good to go.

Edit:
We have a completely custom file format and loader for TJS, so unfortunately in order to create a complete PR with an example solution, i would have to fix one of the supplied canned loaders (such as Collada). I will look into doing this now

thanks, I was going to allocate some time to try to understand and also apply your solution in a more generic way (yours seemed a bit specific to your project). Could you share a bit more context for your code?

Dr. Katz, Professional Therapist

I'm not hooked or anything, but damn.

EDIT:

I've successfully created a pull request - let me know if there's anything additional I can do.

Original post:

I have a local branch with the changes, but I can't seem to push to the origin. Even providing my github creds, I get this error:

Pushing to https://github.com/mrdoob/three.js.git
remote: Permission to mrdoob/three.js.git denied to denbo-ft.
fatal: unable to access 'https://github.com/mrdoob/three.js.git/': The requested URL returned error: 403

Would love some advice since this is my first contribution to github. thanks!

FYI if the PR works for mrdoob and is integrated, there is a new bind mode "local" that supports this change. I didn't want to patch or change the undocumented "detached" bind mode since there is no indication who or what is using it and I didn't want to break something I wasn't aware of. But it does seem to me that it may have been intended for this purpose.

All binMode="local" does is not change the bindMatrixInverse during update, unlike "detached", which resets it to the inverse of bindMatrix, instead of leaving at identity.

See the webgl_loader_collada_skinning.html example in this PR to see the few changes necessary at the scene level in order to get proper local skinning on characters that move around the scene.

The stormtrooper DAE loader demo was used to test this fix. The character was moved 2000 units away from the origin for these tests. Screenshots below.

The shots below are from iPhone X / Safari 11

Before this change (at 2000,2000,2000):

After this change:

Do you think there's a way to achieve this without having the bones in the root scene level and (0,0,0)? I'm using the bones to attach stuff to the avatar, make it look at points, etc etc. And I guess i'm not the only one

You have two options:

1) Keep bones in world space at root to benefit from local skinning as is implemented in threejs. Then do extra transforms if you absolutely need to attach a weapon to a Bone, for example, to get it into the avatar space.

2) Leave Bones in avatar space as-is and get warping on ios.

I would definitely do #1. You can always do the math to figure out where the attachable object should be at any given frame when you already have the Bone position and the character's position. But, there is no workaround for #2 because the warping is caused by packing at the HAL level of the iOS devices and there's nothing you can do about it. Sure, you can add additional hacks by trying to use the various HalfWord packing etc above but they are all reducing the problem, not making it go away. The skinning issue on ios will go away completely if you do the skinning in local space (more specifically, keep the bones centered about the world origin to keep the numbers as small as possible). Then xform into character space for attachments.

More info:

The problem with the way it is implemented in TJS is not TJS specifically, but to solve the problem of bone limit due to capped number of uniforms per shader program, BoneTextures are being used. This is pretty standard solution. However, on iOS, floats in textures are being quantized - packed/compressed - into something smaller than 32-bits. This causes the "praying mantis" or other wacky effects of characters when the size/magnitude of the number is "large", i.e. over a few hundred.

Ok great, what do we do now? Well, every big feature of your game (game engine actually), usually requires a Standards of Practice. Example: tell the artists to center the character about the origin. Place the root node of the character a the feet/ground, author the bones and skin a particular way. Then write an exporter (or importer in TJS case) and an importer that are aware of your process rules. Then support the final feature in the engine when loading the assets that has this configuration. This is how we do things in the game industry.

It is unmanageable and undeterministic (both terrible things for game development) to expect ANY authoring method, ANY file format, and ANY generic loader to handle all features of your game perfectly. There must be some standards and processes along the way.

If your character uses skinning and also needs to attached geo for weapons or whatever, think about what is required for the engine to skin at all, then work with that. I have offered one such solution as an example. Unfortunately, the PR wasn't accepted because they for some reason don't want to change the loaders. anyway, I hope I've explained the problem, and the solution in the thread above, as well as the PR i submitted.

I can't really contribute to this much more than I already have. I hope I've helped some people...

Good luck!

The skinning issue on ios will go away completely if you do the skinning in local space

As I said from the very beginning, that's a workaround for a iOS limitation (bug) and you have to waste processing power for that.
That's not a solution!
The real solution is to raise the floating point accuracy on iOS (to the normal value of 7 digit).
http://davenewson.com/posts/2013/unity-coordinates-and-scales.html

@denbo-ft I actually implemented your PR (we have our own loader) so thank you for your contribution, it was appreciated. I was hoping for another approach since we have multiple avatars in our scenes, they move around, grab items, interact with each other, etc. and having the skeleton in 0,0,0 would force us to change a lot of stuff in our code, risking having new issues, etc.

I spent time thinking about the issue but didn't come up with a better solution, though. (Still trying)

@RemusMar

The real solution is to raise the floating point accuracy on iOS

maybe someone should 1st ask @grorg and co what's the deal with floating textures there, and how come it is like that? because I have a feeling that it's not just a bug but apple's "solution" to some other problem

I have a feeling that it's not just a bug but apple's "solution" to some other problem

Probably true.
Similar with the issue on Firefox Android: bug reported, verified by dev them, but not fixed in the past 5 months (2 releases) !?!

This is how the software development works these days:
instead of fixes in the right place, there are workarounds in the 3rd parties.
Final result: collections of bugs, wasted resources and bloatware !!!

@jfcampos
I would recommend keeping the bone hierarchy in the root and simply adding a function to grab the final WS position of any bone by multiplying it by the martixWorld of the avatar's actual location matrix. It's super simple and will give you back what you had before the fix.

Alternatively, if you have the chops, you can return to the original method (with the bug), include the local matrices in the bone texture, doubling its texture usage, Then modify the shader fragments to do the extra computation on the gpu during skinning. Keep in mind this solution, although "cleaner" from a code perspective, is doubling (possibly quadrupling) the amount of texture that is uploaded to the gpu every frame, and therefore a lot less efficient than the suggestion above. But for whatever reasons you may have that I'm not aware of, it may be more beneficial to you to do it the latter way.

From what was no solution previously, there are now several options here to be compatible with ALL devices, which is an obvious requirement for most projects.

Seems like I'll have to go with bones at 0,0,0 and do a few extra matrix calculations for the other stuff. Thanks!

@denbo-ft is correct. This is a three.js problem caused by skinning in world space.

For reference, this is where the change to world-space-skinning was made: https://github.com/mrdoob/three.js/pull/4812.