Pixi.js: Combining a Pixi.js canvas and a three.js canvas

Created on 21 Jan 2015 · 12Comments · Source: pixijs/pixi.js

Hi Guys !

First of all Thanks for Pixi.js, it's a great tool , I Love it ! the perfect counter part of three.js for the 2D world.

I am working on a project where i would like to have this configuration :

A 2D UI written in full WebGL with Pixi.js
This 2D UI would be encapsulate at the origin of a 3D world written in Three.js
This 3D world would be then render with a VR stereoscopic camera in Three.js

How would you proceed to encapsulate a Pixi.js Canvas inside a Three.js Canvas ?

Thanks for your time !

Cheers

Emmanuel

🤔 Question

Source

ghost

Most helpful comment

I EDIT the post as a i have solve my issue, i post the working code just in case someone hit the same problem and need an exemple.

<body>
    <script src="js/pixi.js"></script>
    <script src="js/three.min.js"></script>
    <script>
        width = window.innerWidth;
        height = window.innerHeight;

        //-------------------------------------------------------------------------------------
        // 3D Scene canvas
        //-------------------------------------------------------------------------------------
        var scene_3D = new THREE.Scene();
        scene_3D.fog = new THREE.Fog( "#a1a1a1", 2000, 4000 );

        var camera = new THREE.PerspectiveCamera( 75, width / height, 1, 10000 );
        camera.position.set( 0, 0, 700);
        camera.updateProjectionMatrix();

        var canvas_3D = new THREE.WebGLRenderer( { antialias: true } );
        canvas_3D.setSize( width, height );
        canvas_3D.setClearColor( scene_3D.fog.color, 1 );
        document.body.appendChild( canvas_3D.domElement );

        var geometry = new THREE.BoxGeometry( 500, 500, 500 );
        var material = new THREE.MeshNormalMaterial();
        var cube = new THREE.Mesh( geometry, material );
        cube.position.z = -500;
        cube.rotation.z = -45;
        scene_3D.add( cube );

        //-------------------------------------------------------------------------------------
        // 2D UI canvas
        //-------------------------------------------------------------------------------------
        var scene_UI = new PIXI.Stage( 0x66FF99 );

        var canvas_UI = PIXI.autoDetectRenderer(width, height, {transparent:true});
        canvas_UI.view.style.position = "absolute";
        canvas_UI.view.style.top = "0px";
        canvas_UI.view.style.left = "0px";

        var graphics = new PIXI.Graphics();
        graphics.beginFill( 0xe60630 );
        graphics.moveTo( width/2-200, height/2+100 );
        graphics.lineTo( width/2-200, height/2-100 );
        graphics.lineTo( width/2+200, height/2-100 );
        graphics.lineTo( width/2+200, height/2+100 );
        graphics.endFill();

        scene_UI.addChild( graphics );

        //-------------------------------------------------------------------------------------
        // Map 2D UI canvas on 3D Plane
        //-------------------------------------------------------------------------------------
        var texture_UI = new THREE.Texture( canvas_UI.view );
        texture_UI.needsUpdate = true;

        var material_UI = new THREE.MeshBasicMaterial( {map: texture_UI, side:THREE.DoubleSide } );
        material_UI.transparent = true;

        var mesh_UI = new THREE.Mesh( new THREE.PlaneGeometry(width, height), material_UI );
        mesh_UI.position.set(0,0,0);
        scene_3D.add( mesh_UI );

        //-------------------------------------------------------------------------------------
        // Render Animation
        //-------------------------------------------------------------------------------------
        function animate() {
            requestAnimationFrame( animate );
            canvas_UI.render( scene_UI );

            cube.rotation.y += 0.01;
            canvas_3D.render( scene_3D, camera );
        }
        animate();
    </script>
</body>

ghost on 23 Jan 2015

👍16

All 12 comments

If three.js supports a canvas as a texture then you could render pixi to an offscreen canvas and use that as a texture in your three.js scene.

englercj on 21 Jan 2015

👍6

Spot on! thats exactly how we do it!

On Wed, Jan 21, 2015 at 4:45 PM, Chad Engler [email protected]
wrote:

If three.js supports a canvas as a texture then you could render pixi to
an offscreen canvas and use that as a texture in your three.js scene.

—
Reply to this email directly or view it on GitHub
https://github.com/GoodBoyDigital/pixi.js/issues/1366#issuecomment-70872648
.

Mat Groves

_Technical Partner_

Telephone: 07708 114496 :: www.goodboydigital.com
First Floor, Unit 9B, Queens Yard, White Post Lane, London, E9 5EN
goodboy©. All Rights Reserved.

GoodBoyDigital on 21 Jan 2015

Wahou ... Thanks a Lot guys ! :)

This is what i call as Fast Feedback !

ghost on 21 Jan 2015

I am just thinking about a small detail.

I need to have the ability to draw 2D UI panel (Pixi.js) that will be rendered on Top of a 3D World (Three.js).
But it is important that those 2D UI panel have the following properties.

Transparency with an opacity factor from 0-100%
Blur effect of the 3D World in the background

If i use the tricks of rendering my canvas and apply it as a texture. I am afraid that the Blur of the background will not be possible ?

I have attach an image to illustrate this.
transparency

ghost on 21 Jan 2015

Do the blur on the Three.js side? Just make the pixi object transparent and then blend the texture into your 3D scene however you want.

englercj on 21 Jan 2015

Thanks again for your Help ! Now that you said it ... it's definitely the most logical path ! :)

Cheers

ghost on 21 Jan 2015

I EDIT the post as a i have solve my issue, i post the working code just in case someone hit the same problem and need an exemple.

<body>
    <script src="js/pixi.js"></script>
    <script src="js/three.min.js"></script>
    <script>
        width = window.innerWidth;
        height = window.innerHeight;

        //-------------------------------------------------------------------------------------
        // 3D Scene canvas
        //-------------------------------------------------------------------------------------
        var scene_3D = new THREE.Scene();
        scene_3D.fog = new THREE.Fog( "#a1a1a1", 2000, 4000 );

        var camera = new THREE.PerspectiveCamera( 75, width / height, 1, 10000 );
        camera.position.set( 0, 0, 700);
        camera.updateProjectionMatrix();

        var canvas_3D = new THREE.WebGLRenderer( { antialias: true } );
        canvas_3D.setSize( width, height );
        canvas_3D.setClearColor( scene_3D.fog.color, 1 );
        document.body.appendChild( canvas_3D.domElement );

        var geometry = new THREE.BoxGeometry( 500, 500, 500 );
        var material = new THREE.MeshNormalMaterial();
        var cube = new THREE.Mesh( geometry, material );
        cube.position.z = -500;
        cube.rotation.z = -45;
        scene_3D.add( cube );

        //-------------------------------------------------------------------------------------
        // 2D UI canvas
        //-------------------------------------------------------------------------------------
        var scene_UI = new PIXI.Stage( 0x66FF99 );

        var canvas_UI = PIXI.autoDetectRenderer(width, height, {transparent:true});
        canvas_UI.view.style.position = "absolute";
        canvas_UI.view.style.top = "0px";
        canvas_UI.view.style.left = "0px";

        var graphics = new PIXI.Graphics();
        graphics.beginFill( 0xe60630 );
        graphics.moveTo( width/2-200, height/2+100 );
        graphics.lineTo( width/2-200, height/2-100 );
        graphics.lineTo( width/2+200, height/2-100 );
        graphics.lineTo( width/2+200, height/2+100 );
        graphics.endFill();

        scene_UI.addChild( graphics );

        //-------------------------------------------------------------------------------------
        // Map 2D UI canvas on 3D Plane
        //-------------------------------------------------------------------------------------
        var texture_UI = new THREE.Texture( canvas_UI.view );
        texture_UI.needsUpdate = true;

        var material_UI = new THREE.MeshBasicMaterial( {map: texture_UI, side:THREE.DoubleSide } );
        material_UI.transparent = true;

        var mesh_UI = new THREE.Mesh( new THREE.PlaneGeometry(width, height), material_UI );
        mesh_UI.position.set(0,0,0);
        scene_3D.add( mesh_UI );

        //-------------------------------------------------------------------------------------
        // Render Animation
        //-------------------------------------------------------------------------------------
        function animate() {
            requestAnimationFrame( animate );
            canvas_UI.render( scene_UI );

            cube.rotation.y += 0.01;
            canvas_3D.render( scene_3D, camera );
        }
        animate();
    </script>
</body>

ghost on 23 Jan 2015

👍16

We are doing something like this in a project. It gets a little tricky since both engines (pixi and ThreeJS) keep track of GL state to minimize redundant calls. This means that they don't play nicely with each other, so you need to reset their flags before starting the renderers.

With new versions of ThreeJS you can use renderer.resetGLState().

In PIXI, it's a bit more complicated. Things like this would be useful to have as a method in WebGLRenderer. These are needed before

    gl.disable(gl.DEPTH_TEST)
    gl.enable(gl.BLEND)
    gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA)
    gl.disable(gl.STENCIL_TEST)
    gl.disable(gl.CULL_FACE)

    gl.frontFace( gl.CCW )    
    gl.cullFace( gl.FRONT )
    gl.colorMask(true, true, true, true)
    gl.activeTexture(gl.TEXTURE0)

    var shaderManager = session.shaderManager

    shaderManager._currentId = undefined
    for (var i=0; i<shaderManager.attribState.length; i++)
        shaderManager.attribState[i] = false
    shaderManager.setShader(shaderManager.currentShader)

    var blend = session.blendModeManager.currentBlendMode
    if (blend in PIXI.blendModesWebGL) {
        session.blendModeManager.currentBlendMode = undefined
        session.blendModeManager.setBlendMode(blend)
    }

This allows you to render the UI on top of the 3D scene without any frame buffer switching.

Achieving the blur underneath your UI is significantly more complicated. You would need to:

render the 3D scene to a frame buffer
enable scissor test and render the blur region(s) with a multi pass blur shader (ping-ponging FBOs etc)
then render 2D UI on top

mattdesl on 27 Jan 2015

👍3

Thank you for this very precise and documented feedback Matt ! :)

As i'm pretty new to WebGL programming this is extremely helpful.
I was hesitant first to Mix 2 library, especially that a hacked Three.js can in some extend replace pixi for what i need.

But the excellent exemples , the very focus point of the dev on 2D, really convinced me to use pixi for gl2d calls.

As you look to have chosen a Mix approach too, i guess it confirm that Pixi is definitely a more efficient choice for drawing 2D elements with WebGL.

Cheers

ghost on 27 Jan 2015

As you look to have chosen a Mix approach too, i guess it confirm that Pixi is definitely a more efficient choice for drawing 2D elements with WebGL.

Well, maybe. In most cases the extra performance is moot (how often do you need 3000 buttons animating at once?), and things like robust text rendering, text input, and scroll events can be incredibly challenging in WebGL.

The easiest route is just to use DOM/CSS for UI. If you absolutely need to choose WebGL (i.e. for certain effects, depth testing, etc) then Pixi will probably be easier than ThreeJS for 2D UI.

mattdesl on 27 Jan 2015

Going to close this up since it seems answered.

englercj on 27 Jan 2015

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.