At the risk of stating the obvious, but have you tried the version of threeocttree that's already in the repo(examples/js/Octree.js)?

I don't vote for a hard-wired usage of spatial search trees in three.js. The overhead/complexity of such algorithms overbalances the performance gain in many applications.

checking bounding box and sphere first before doing actual raycast

This is reasonable and should be sufficient. If users need more performance in context of more advanced use cases, they can use the mentioned example as a starting point. Octrees might be a good choice. But i've never seen a R-Tree solution in interactive 3D applications because its approach is quite sophisticated (the algorithm performs data instead of space partitioning).

Hello and thanks for your replies,
@Mugen87 I agree that many use cases do not require such fast searches. However, having a mechanism for interacting with 3D objects (and I guess that is the case what it is mostly used for), THREE.Raycaster's step from checking bounding box/sphere first (which I totally agree to be reasonable) in let's say constant time to a linear time effort when doing the actual raycast is quite big. I could imagine a kind of "per-geometry-search tree" instead of one global search tree. As long as the geometry does not change (oftentimes transformations are much more likely than geometry changes) the search tree does not need to be updated.

That's why I thought, such kind of optimization using spatial search trees could be a more integral part of three.js. But I also understand that this causes additional complexity and overhead.

@moraxy I will have a look at the example version. I just wanted to know if such a feature would make sense.

Thanks again and kind regards

Maybe a simpler search tree could be integrated into THREE.BufferGeometry. For example a call to computeBoundingBox could also build a kind of read-only search tree referencing the corresponding vertices. Since THREE.BufferGeometry is

best-suited for static objects where you don't need to manipulate the geometry much after instantiating it

this search tree does not need to be changed after initializing it. This would reduce some overhead for updating/deleting. An Octree would be a good starting point (similar concept in BabylonJS).

best-suited for static objects where you don't need to manipulate the geometry much after instantiating it

FWIW, I do not believe that is a true statement.

@WestLangley Quotation from THREE.BufferGeometry documentation

THREE.BufferGeometry documentation

It should probably be removed - in any case, I don't think that's meant as any kind of technical statement, but rather means it's harder for the user to manipulate the geometry after it has been created.

It should probably be removed

Most definitely.

I just implemented a better method for raycasting on PlaneBufferGeometry. I use the far parameter and find the first and last index positions. This works in planebuffer because the index array is in x/y order. Once outside the x/y bounding box all collision must be outside the far range. Is there any desire to commit this somewhere? Current implementation is specific to my use case but I would be willing to try and make more generic if desired. I was able to reduce my raycasting performace significantly on a large planeBuffer(2.5 seconds to 10ms)

@kpetrow I think the three.js octree examples are adequate. You are of course free to share your code on GitHub if you feel it would be useful to others.

I have suggested spatial index to be included into threejs in the past. There have been some work in this direction, and existing oct-tree example is one of those. Ultimately, i think it's lack of interest from the core community with respect to having this functionality as first-class citizen in three.

I would be happy to donate my BVH code to the project, if there is sufficient interest and intent to include it into the main distribution (not examples).
My code focuses on 2 aspects:

• dynamic scenes
  
  
  
  • fast insert
  
  
  • fast deletion
  
  
  • refitting (ability for nodes to change shape without having to be re-inserted)
  
  
• query speed

I have 2 implementations:

• BinaryBVH
  
  
  
  • very compact, using ByteBuffer
  
  
  • mix of UintX and FloatX types via DataView
  
  
  • great for serialization
  
  
  • can be compressed using standard tools like lzma library
  
  
  • immutable
  
  
  • build speed is highly optimized
  
  
  • only works for BufferGeometry
  
  
  • SAH optimization
  
  
  • ray queries
  
  
• BVH
  
  
  
  • Object tree, as a consequence is a lot larger than binary implementation
  
  
  • mutable
  
  
  • fast bulk insert
  
  
  • fast single item insert
  
  
  • refitting
  
  
  • stack, recursive and stack-less traversal implementations (different performance characteristics)
  
  
  • SAH optimization
  
  
  • Frustum queries
  
  
  • Ray queries
  
  

Personally, i can't live without a spatial index, it is a difference between n^2 and log(n) performance. For example, in my current project following parts rely on spatial index:

• foliage system (trees, flowers, bushes etc)
• all object placement (characters and houses)
• picking (used in interactions, such as mouse clicks)

http://server1.lazy-kitty.com/komrade

terrain has about 1m polygons, and placing thousands of trees and running real-time ray-casts on that is simply a no-go, especially for lower-end devices.

I would like to see anything that makes raycast optimized. Is there any room for optimization without adding complexity? Octree requires all new library, with all new adds and updates, etc..

One thing i notice is that once a geometry is converted to a indexed buffer array, It no longer uses the properties of the original geometry smartly. As mentioned above plane buffer geometries can be super optimized by knowing that the arrayBuffer came from a PlaneGeometry. Maybe overwriting the MESH.raycast ( raycaster, intersects ) method to be based on the geometry type?

Another approach is to have Raycaster as a separate plug-in -- as are the interactive controls and the loaders. Then, adding a performant raycaster similar to what @Usnul proposes would be great.

I like the idea of having pluggable Raycasters, but I think there is a bit of confusion. What I think to be useful is a spatial index, not a Raycaster per-se. Spatial index allows things like:

• spatial culling (think frustum culling)
• doing visibility queries
• building a path tracing renderer
• fast sorting (exploiting data locality)

currently three.js does 2 of these explicitly (sorting and culling) and 1 via examples (Ray-tracing renderer)

maintaining a spatial index internally would accelerate sorting and culling, giving progressively more benefit for larger scenes at the cost of extra RAM required to store the index.

Why don't you just use GPU picking? There are some examples around and it is fairly easy to implement. The performance is night and day. In our use cases, with models of over a million polygons, the picking went from nearly a second to running in realtime at 60fps while dragging things over the model.

https://github.com/brianxu/GPUPicker

@hccampos
GPU picking is not necessarily faster than doing it on CPU, for GPU you have to render at resolution proportional to precision you desire, so if you want pixel accuracy - you have to render at 1:1 screen resolution. Bear in mind that this is a separate render pass, you're rendering objects as separate distinct colors. An extra render pass means graphics memory usage (typical for deferred pipeline).
Using a binary space partitioning index gives you log(n) timing profile on your picking, so for 1,000,000 polygons you'd be looking at about 14 operations to resolve a ray query. Depending on your data structure - it's likely to take a few microseconds, allowing you thousands of queries before you start making a dent in your frame budget.

Lets make a comparison, lets say you have a 1m poly model, and you wish to cast a ray from screen space directly along camera direction into the scene (picking use-case). Lets assume you are going with the bottom of the barrel resolution, "full hd", or 1920 × 1080. Lets assume you render RGB only (24 bit per pixel), you will need 6220800byte (about 6 Mb) or graphics RAM to render that. If you use CPU solution, lets say you go with AABB BVH with 10 polygons per leaf, that mean you need about 200,000 nodes, lets say each node is about 6 * 8bytes for coordinates, intermediate nodes have extra 2 * 4 bytes for child pointers and leaf nodes have 10 * 4 bytes for polygon pointer, that's 14,400,000 bytes (about 14Mb). Main difference comes into play when you conder the fact that your BVH queries have very little demand on RAM bandwidth, compared to GPU case, and there's only a handful of operations involved per query, compared to rendering full 1m poly geometry.
If you take a resolution that's more typical for desktops, like 2560x1440, you end up with 11059200byte render target (11Mb).

If you have a large budget of graphics ram bandwidth and a fairly decent shader core count - sure, that's a simple and straight-forward way of doing picking.

@Usnul absolutely, but it is probably the simplest solution to implement. Implementing and maintaining spatial index data structures will probably be a non-insignificant burden for the threejs maintainers.

That being said, I would absolutely love to have a good, well-tested and maintained spatial index data structure as part of three or as a separate npm package.

I was curious about implementing such a spatial index for my used geometries so I implemented a quite simple Octree (create and search only) that splits up my BufferGeomtry. With this simple solution I achieved quite promising results: applied on a geometry with about 500K vertices the raycast time reduced from ~120ms to ~2.3ms. Construction of the tree currently takes ~500ms but since geometry and tree creation is done inside a web worker only once on application start, this is not a such a problem.

I guess the algorithm could be quite easily integrated into THREE.BufferGeometry and maybe switched on or off using a flag such as THREE.BufferAttribute's dynamic. This way, it could be used only when BufferGeometry is not supposed to change. Unfortunately, I had to overwrite THREE.Mesh's raycast method though I only needed to change two lines in it (Octree search call and position array iteration).

Anyway, in my VR application I have to detect object-controller collisions during render loop and not only once on mouse click. Thus, I have to rely on my current solution. I will try if I can further improve it.

UPDATE I made a mistake in measuring the raycast time. The correct value is ~2.3ms (instead of ~0.3ms). I changed the value above accordingly.

I think having an opt-in way of doing fast spacial lookups is necessary in most more complex applications built on top of three.js. For example, a 3D editor. I've toyed around with some implementations out there, but they either don't integrate well, or wrap into a specific version of three.js too tightly. So if anyone has an opt-in way of doing it without breaking the upgrade path to newer versions of three.js it would be great.

@matthias-w I tried the raycaster for an obj model but the controller isn't detecting anything. The rays are passing through the model. Can you tell me how you fixed that problem

On the subject of having optional spatial index. I believe that it is sufficiently useful even for the renderer itself (e.g.: for culling) to be an integral part of the engine. Here's a related discussion:

13909

Hey! I was a bit interested in this, as well (though I have since found other solutions for our raycast needs), but I figured I'd contribute some of my experiments, too. It's a bit rough, but I put this together a few months ago:

https://github.com/gkjohnson/threejs-fast-raycast

It adds a computeBoundsTree function to the THREE geometry objects and overrides the raycast function to use that (and only returns the first hit as an added optimization). It's only really beneficial for static, highly complex meshes and doesn't do anything to spatially segment the scene. Here's the demo where you can see the difference in raycast performance on an 80,000 triangle mesh. Computing the bounds tree is a little slow, but with a little work it could probably be smoother.

Regarding my opinion on this, I feel a bit conflicted. This feels like something that can be built pretty well as an extension to THREE. And ultimately it doesn't seem like doing per-triangle checks / collisions / casts for complex or animated meshes is optimal, anyway. In simple cases it might be fine, but it seems like using the typical plane / cube / sphere / capsule representations or simplified meshes would be better suited for enabling super-fast raycasting (or occlusion-culling, collisions, etc) in complex cases. Of course, if you're looking for pixel-perfect casts this doesn't work quite as well, but the right solution really depends on your use case.

@Usnul @matthias-w Are either of your octree / BVH implementations open source or available online? I'd definitely be interested in taking a look!

@gkjohnson

Are either of your octree / BVH implementations open source or available online? I'd definitely be interested in taking a look!

The code is not open source currently. I could provide you the sources if you contact me privately.

There's an example I did for instanced meshes:
http://server1.lazy-kitty.com/tests/instanced-foliage-1mil/
The above example includes following:

• BVH is updated dynamically as new instances are inserted into the tree
• BVH is optimized incrementally using depth 1-2 rotations on the fly
• BVH is sampled with a frustum query to which instances to render

I have a version of it running in the game i'm working on:
http://server1.lazy-kitty.com/komrade/

With respect to your implementation. I like it, mine is different in 2 major ways:

• I focus on performance
• I focus on memory footprint
• I avoid garbage collection in many places

Some more specific points:

• I also use a mixture of splitting strategies transparently, based on what you want to do with the BVH
• My BVH is AABB based
• I support updates to node bounds via refitting

@Usnul I cannot see your email address or anything, but I am quite interested in studying your spatial indexing solution.

Currently I just loop through all relevant scene objects and calculate distance from camera. Not the most optimal approach.

@titansoftime
it's
travnick at gmail com
Didn't realize it's not publicly available. My bad.

@Usnul I am interested in the 1 million project as well. Perhaps even 10 or 20 million if possible. Please email me : kaori.nakamoto.[email protected]

Thank you VERY kindly!

http://server1.lazy-kitty.com/tests/instanced-foliage-10mil/
this is a version with 10 million

Sorry for the late response.
@gkjohnson The code is not open source. Additionally, your solution sounds much more sophisticated than mine. Thus, I guess there is not so much to learn from my solution.

@sid3007 I am not sure if I understand your problem. I can explain my approach. Maybe it is helpful.

My use case is quite simple. When my application starts, the geometries for different models are loaded. These models can be transformed by the user. The geometries do not change. There are no geometry deformations. Hence, I implemented a very simple Octree that is built once for every geometry when the application starts. The octree is not dynamic. It is built based on the bounding box of the given geometry and the array of vertices. During its construction it checks whether an octant contains a vertex or the octant's Box3 bounding box intersects with a triangle (three consecutive vertices in non-indexed geometry) and stores the vertex array references with the octree's node.
The per-mesh octree is then stored along with the mesh. I also overwrite my mesh instances' THREE.Mesh's raycast method (just one line) in order to actually call the octree intersection test method. This returns the geometries face vertex indices that can be used by the mesh's default intersection logic.
I did one optimization: the octree creation is done once on app start inside a WebWorker (actually a pool of workers). The reason is that the tree creation for large geometries takes quite some time (some seconds). This would block the browser UI so I moved it to another thread.
I hope my approach becomes clear.

@matthias-w Sounds like great work! I independently did almost the same , but don't think I achieved nearly as big a performance improvement. https://discourse.threejs.org/t/octree-injection-for-faster-raytracing/8291/2 (object-oriented octree implementation, and less clean modifications to Mesh.raycast)

Would it be possible for you to open-source/contribute the Octree implementation, allowing more experiments by others?

@EliasHasle Thank you. Actually, the performance gain is not that good, since I made a mistake in my first measures. I corrected the value (see my post above). The timings make more sense now with regard to Octree's logarithmic search complexity. Unfortunately, I cannot make the code available at the moment, but maybe later this year. Anyway, my implementation is quite straightforward (and not that clean BTW ;)). So I guess there would not be any special insights.

@matthias-w I think 2.3 ms vs 120 ms for a raycast on a 500k vertex mesh is still a significant improvement, one that, for instance, enables real-time resolution of fired bullets in games (at a fairly large part of the computation budget per frame).

Did you try raytracing too?

Is your implementation based on a tree of JS objects? Literal objects or instances of a prototype? Mine is fully self-similar, so that every node is an octree, with methods and all.

@EliasHasle

If you're interested in raycasting against static geometry I and others have put significant effort into three-mesh-bvh, which uses a BVH to index triangles and allows high-performance raycasting as well as intersection detection against static complex geometry. The tree building process is more complex so it takes a bit longer but takes raycasting down to under a millisecond and often under 0.1ms when raycasting against geometry with hundreds of thousands of triangles.

The build time could be improved in a few different ways but it's been good enough for what I wanted to do with it -- improving that is on the list, though.

I would like to build a dynamic scene-based octree to enable better raycasting and collision detection for scenes with lots of meshes but I haven't had a good use case myself. Maybe some day!

@EliasHasle My implementation is a tree of JS class objects (I use ES6 classes, i.e. prototype instances). Basically, a node is a tree in my implementation. However, I have an additional Octree class that holds the root node and provides methods for building and searching (and also some debugging and visualization) the tree.

I did not experiment much with the values for maximum tree node level and maximum number of vertices per leaf node. Maybe there is a better combination.
I also use a bounding volume hierarchy to accelerate intersection tests. So I test the mesh's bounding sphere and bounding box before actually checking on the mesh geometry's octree.

I can recommend a book that is a very nice collection of collision detection approaches: C. Ericson, Real-Time Collision Detection, CRC Press, 2004

@gkjohnson That looks great! The performance is impressive. What kind of BVH are you using?

@matthias-w Thanks!

There are a couple split strategy options but splitting the BVH nodes on the center of the longest side builds the tree the fastest. The triangles are partitioned into sides based on the center of their bounds and the tree nodes are expanded to fully contain the child triangles (so there's a small amount of overlap of tree nodes).

@gkjohnson So then the BVH implementation is a kind of unbalanced Kd-tree with axis selection dependent on box shape, right? If so, I was thinking about doing something like that for cases where the root BB was too far from a cube, and then using octrees after the conditional splitting. This would presumably apply well to cases such as @Usnul's map world, where initial splitting would be in the two "map" dimensions and subsequent splitting would proceed in 3D. I think it is a much better solution than mine, which is to expand the root BB to a bounding cube with the same center, and then use octree splitting all the way.

I just implemented a better method for raycasting on PlaneBufferGeometry.

@kpetrow I think optimizing raycasts on PlaneBufferGeometry, the assumption that vertex positions will not be altered will very likely not be valid for the high-resolution geometries where a linear search is insufficient. As far as I can tell, the main usage of high-resolution PlaneBufferGeometry is to reshape it by moving the vertices, while keeping the topology, e.g. to build terrain.

Closing in favor of #13909.