Home / Posts tagged "Particles"

WebGL GPU Particle stream

I’ve once blogged about a project which I build an interactive particle stream in Cinder but I lost it when I move to new webspace. Now I rebuild it with WebGL and want to post again and with some tips that I learn while building it. First thing first, the live demo is here :

and also the source code is available here :



Saving data in the texture

This is a quite common technique when dealing a large particle system : Save the information of the particle on a texture ( such as the particle position and particle speed ) and perform the movement calculation in GPU. Then when you want to move the particles, you just need to modify this texture. The basic concept is that a pixel contains these 3 color channels : Red, Green and Blue, so we can use these 3 channels to save the x, y and z coordinates. It could be the x,y,z of a particle’s position or the x,y,z of a particle’s velocity. This idea is simple, but need some works to make it work. The first thing is how to map a position to a color, the range of the position could be anything from negative to positive, but the range of a color channel is only from 0 to 1. In order to make it work we need to set a range for the positions, and the zero point will be (0.5, 0.5, 0.5) anything smaller than .5 will be negative and positive if greater than .5. A simple example that convert a pixel color to a position  range from -100 to 100.

var range = 100;
position.x = ( color.r - .5 ) * range * 2.0;
position.y = ( color.g - .5 ) * range * 2.0;
position.z = ( color.b - .5 ) * range * 2.0;

And vice versa you can save a position to a color like this :

color.r = (position.x/range + 1.0 ) * .5;
color.g = (position.y/range + 1.0 ) * .5;
color.b = (position.z/range + 1.0 ) * .5;

So each pixel on the texture represent a set of x,y,z coordinate, that’s how we save the positions of all particles.



But how exactly we can write our data to a texture ? We need to use a framebuffer. Framebuffer allows your program to render things on a texture instead of render directly to your screen. It’s a very useful tool especially when dealing with post effects. For learning more about framebuffer you can check this post. With framebuffer now we can save the data to a texture, but here I meet the biggest problem in this experiment : Precision. Because we are working in the color space that all the numbers are really small, for example the speed of a particle could be only .01 and the acceleration of the particle will be even smaller. So when you multiply things together sometimes it gets too small and the pixel cannot hold the precision. This happens both to this experiment and the project that I mentioned about with Cinder. In WebGL by default(gl.UNSIGNED_BYTE) each color channel have 8 bits to store the data. In our case this is not enough, luckily there’s a solution for it : Using gl.FLOAT instead of gl.UNSIGNED_BYTE, gl.FLOAT will allow each color channel to have 32 bits to save the data. In order to use gl.FLOAT we need to do one extra step :

gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, this.frameBuffer.width, this.frameBuffer.height, 0, gl.RGBA, gl.FLOAT, null);

This will enable WebGL to use gl.FLOAT and solve our problem with precision. Here is a screenshot of how the framebuffer look like in this experiment, I save the position of the particles on the left side of the framebuffer, and the velocity of the particle on the right.



Particle movements

The next step is to calculate the movement of the particle. It all base on this rule :

new velocity = old velocity + acceleration
new position = old position + velocity

So with our texture, on the left side which is the position of the particle, we just need to get its velocity and add it to the current position, don’t forget that the range of velocity is from 0-1 so need to subtract vec3(.5) from it

if(vTextureCoord.x < .5) {      //  POSITION
    vec2 coordVel       = vec2(vTextureCoord.x + .5, vTextureCoord.y);   // get the coordinate of the velocity pixel
    vec3 position       = texture2D(texture, vTextureCoord).rgb;         
    vec3 velocity       = texture2D(texture, coordVel).rgb;              
    position            += (velocity - vec3(.5) ) * velOffset;       

For right side (which is the velocity), I want to add a random force to the particle based on where the particle is. I found a very useful GLSL noise function here. So the shader code look like this now :

else { // vTextureCoord.x > .5
    vec2 coordPos       = vec2(vTextureCoord.x - .5, vTextureCoord.y);   // get the coordinate of the position pixel
    vec3 position       = texture2D(texture, coordPos).rgb;
    vec3 velocity       = texture2D(texture, vTextureCoord).rgb;

    float xAcc          = snoise(position.x, position.y, time);
    float yAcc          = snoise(position.y, position.z, time);
    float zAcc          = snoise(position.z, position.x, time);

    velocity            += vec3(xAcc, yAcc, zAcc);

Where snoise is the noise function and I passed in time as well so it will keep changing constantly. But this is just roughly how it looks like, in the real life you need to tweak the value in order to get the natural movement feeling. The last thing is that you need to prepare 2 framebuffers and swap them every frame, so you can always get the result of last frame and update it to the other framebuffer.

this._vCal.render( this.fboCurrent.getTexture(), this.fboForce.getTexture() ); // Perform the calculation


var tmp = this.fboTarget;
this.fboTarget = this.fboCurrent;
this.fboCurrent = tmp;


Adding interaction

The final step is to add interaction to it. With Leap motion we can easily get the position and velocity of the hands, so we can easily determine a force with position of the hand, and its strength will be determined by the length of the hand velocity. As for the direction there are couple of options : the first one is to take the direction of  the velocity, which is the most common one. However it can be improved with using the direction of your palm, which leap motion is able to give us (hand.palmNormal). This will make it feel better when you do several movements in a roll, trying to push the particles to same place. And one final touch to this is to check the dot product of the hand velocity and this palmNormal, if the dot result is smaller than zero which means they move in different direction, we should set the strength to zero to avoid the weird movements.

To apply this force to our particles, first we need to create a force texture like this :


Again we use color to represent the force. Back to the shader, when we calculate the velocity of the particle we need to add this force as well. So the shader will look like this now :

else { // vTextureCoord.x > .5
    vec2 coordPos       = vec2(vTextureCoord.x - .5, vTextureCoord.y);   // get the coordinate of the position pixel
    vec3 position       = texture2D(texture, coordPos).rgb;
    vec3 velocity       = texture2D(texture, vTextureCoord).rgb;

    float xAcc          = snoise(position.x, position.y, time);
    float yAcc          = snoise(position.y, position.z, time);
    float zAcc          = snoise(position.z, position.x, time);
    velocity            += vec3(xAcc, yAcc, zAcc);

    // get the force pixel by the position of the particle
    vec3 forceGesture   = texture2D(textureForce, position.xy).rgb;   

    // map the force value to -.5 to .5 and add it to velocity   
    velocity            += forceGesture - vec3(.5);                      


So that’s how I build this. The concept is not complicated, but there are a lot of small steps to take care. Also because everything happens in texture and shader which makes it hard to debug. Sometimes you just get a white or black texture and hard to tell which step went wrong. But once you got it all working and you can push for a huge amount of particles, that feeling is incredible. It’s a really good practice for learning framebuffer, shader and particle movements, I learn a lot and had a lot of fun when building it.

Here is a short video of the Samsung project I build if you are curious how it looks in motion : https://vimeo.com/92043935


Adidas all for this case study – part 2


Continue from the part 1, as I mentioned that we can’t use webgl in this project,  we still want to create a 3D feel like particles visualisation, so I start to run some test on the canvas, which is a whole new world to me, surprisingly, the performance is really good, even on iOS devices. So the basic idea is to create a huge canvas contains everything,  clean it and redraw all the particles back on every frame. It sounds quite scary with “clean everything” and “redraw everything” but actually it’s working quite well. So we solve the technology we are going to use, the next question is how to build a fake 3D illusion.

The power of matrix

At beginning I just trying to create an imaginary camera, so all i need to move is the camera instead of all the particle positions, which will saves a lot of work for me, so i create a camera matrix for that. Then i need to create the perspective projection so it will feel really close to what you will get in 3D. At this point i feel what i’m trying to do is really similar to what i did with WebGL : prepare the model, view and perspective matrix then pass in to the shader. So I took the matrices i used in my webgl project and it works ! Ok, not 100% but for the camera matrix it works perfectly, however i still don’t get the perspective, and then i discover this is the only extra work you need to do, you need to scale by yourself. It’s actually not that complicate because you already have the correct calculated position from those 2 matices, then all you need to do is to decide how much you want to scale by the depth, then call context.scale() and voila !

There are still a lot of work to do to make this one a complete 3D like framework, but in that case it’s probably easier just use Three.js. In our case this is already enough so we just stop here.

Tips for optimise performance

The first one it’s really depends on different project, in our case we are using add blend mode, so we don’t need depth sorting, this really saves us a lot of spaces. The second one is don’t render the thing you can’t see, like the particles with 0 opacity, or the particle with too small size ( <2px) just skip them and you will find you get a huge performance boost. And the last one is a tricky one : be careful with the size of your particle, during the development there’s once we had a huge drop on the performance and I don’t know why, after spending hours trying to find the problem at the end actually it’s because we shrink the size of the particle from 96 to 64, when I scale it back you 96 the fps goes up to it was before right away. It might be that i was trying to scale it too much in code, so when i put the image size back it works.

The performance of the canvas is really surprising, and especially when you see this fake 3D stuff working on the ipad and you can interact with your finger. I’m not a big fan of iPad because you can’t run webgl on it but  i have to say it feels quite nice to interact the 3D with touches, feels more intuitive. Really hoping the coming of the day that we can run webgl on the mobile / tablet devices.

Bubble Man

Just a little toy, i capture the motion using kinect and export them to javascript. Then recreate the motion using particles based on the data obtained from kinect. Rotate it to hear different musics.

Another thing I tried to do is the sound, i want the sound to change when user view it from different angles and transition must be smooth. To be honest it’s just to change the volume of each sound ( in this case i put 3 sounds ) with the position of the camera, the code look like this :

if(angle > (210 - range) && angle < 210 ) { 	 	
theta = (angle - ( 210 - range)) / range * PI2; 	 	this.gains[0].gain.value = Math.cos(theta); 	 	this.gains[1].gain.value = Math.cos(PI2 - theta); 	 	this.gains[2].gain.value = 0;  } else if(angle > (330 - range) && angle < 330 ) { 	 	theta = (angle - ( 330 - range)) / range * PI2; 	 	this.gains[0].gain.value = 0; 	 	this.gains[1].gain.value = Math.cos(theta); 	 	this.gains[2].gain.value = Math.cos(PI2 - theta);  } else if(angle > (90 - range) && angle < 90 ) {
	theta = (angle - ( 90 - range)) / range * PI2;
	this.gains[0].gain.value = Math.cos(PI2 - theta);
	this.gains[1].gain.value = 0;
	this.gains[2].gain.value = Math.cos(theta);

Really no big deal but it’s fun to play.

For the using of web audio, here is a very good article :