WebGL GPU Particle stream

I’ve once blogged about a project which I build an interactive particle stream in Cinder but I lost it when I move to new webspace. Now I rebuild it with WebGL and want to post again and with some tips that I learn while building it. First thing first, the live demo is here :
http://www.bongiovi.tw/projects/particleStream

and also the source code is available here :

https://github.com/yiwenl/WebGL_Particle_Stream

 

Saving data in the texture

This is a quite common technique when dealing a large particle system : Save the information of the particle on a texture ( such as the particle position and particle speed ) and perform the movement calculation in GPU. Then when you want to move the particles, you just need to modify this texture. The basic concept is that a pixel contains these 3 color channels : Red, Green and Blue, so we can use these 3 channels to save the x, y and z coordinates. It could be the x,y,z of a particle’s position or the x,y,z of a particle’s velocity. This idea is simple, but need some works to make it work. The first thing is how to map a position to a color, the range of the position could be anything from negative to positive, but the range of a color channel is only from 0 to 1. In order to make it work we need to set a range for the positions, and the zero point will be (0.5, 0.5, 0.5) anything smaller than .5 will be negative and positive if greater than .5. A simple example that convert a pixel color to a position  range from -100 to 100.

var range = 100;
position.x = ( color.r - .5 ) * range * 2.0;
position.y = ( color.g - .5 ) * range * 2.0;
position.z = ( color.b - .5 ) * range * 2.0;

And vice versa you can save a position to a color like this :

color.r = (position.x/range + 1.0 ) * .5;
color.g = (position.y/range + 1.0 ) * .5;
color.b = (position.z/range + 1.0 ) * .5;

So each pixel on the texture represent a set of x,y,z coordinate, that’s how we save the positions of all particles.

 

Framebuffer

But how exactly we can write our data to a texture ? We need to use a framebuffer. Framebuffer allows your program to render things on a texture instead of render directly to your screen. It’s a very useful tool especially when dealing with post effects. For learning more about framebuffer you can check this post. With framebuffer now we can save the data to a texture, but here I meet the biggest problem in this experiment : Precision. Because we are working in the color space that all the numbers are really small, for example the speed of a particle could be only .01 and the acceleration of the particle will be even smaller. So when you multiply things together sometimes it gets too small and the pixel cannot hold the precision. This happens both to this experiment and the project that I mentioned about with Cinder. In WebGL by default(gl.UNSIGNED_BYTE) each color channel have 8 bits to store the data. In our case this is not enough, luckily there’s a solution for it : Using gl.FLOAT instead of gl.UNSIGNED_BYTE, gl.FLOAT will allow each color channel to have 32 bits to save the data. In order to use gl.FLOAT we need to do one extra step :

gl.getExtension("OES_texture_float");
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, this.frameBuffer.width, this.frameBuffer.height, 0, gl.RGBA, gl.FLOAT, null);

This will enable WebGL to use gl.FLOAT and solve our problem with precision. Here is a screenshot of how the framebuffer look like in this experiment, I save the position of the particles on the left side of the framebuffer, and the velocity of the particle on the right.

textureMap

 

Particle movements

The next step is to calculate the movement of the particle. It all base on this rule :

new velocity = old velocity + acceleration
new position = old position + velocity

So with our texture, on the left side which is the position of the particle, we just need to get its velocity and add it to the current position, don’t forget that the range of velocity is from 0-1 so need to subtract vec3(.5) from it

if(vTextureCoord.x < .5) {      //  POSITION
    vec2 coordVel       = vec2(vTextureCoord.x + .5, vTextureCoord.y);   // get the coordinate of the velocity pixel
    vec3 position       = texture2D(texture, vTextureCoord).rgb;         
    vec3 velocity       = texture2D(texture, coordVel).rgb;              
    position            += (velocity - vec3(.5) ) * velOffset;       
}

For right side (which is the velocity), I want to add a random force to the particle based on where the particle is. I found a very useful GLSL noise function here. So the shader code look like this now :

else { // vTextureCoord.x > .5
    vec2 coordPos       = vec2(vTextureCoord.x - .5, vTextureCoord.y);   // get the coordinate of the position pixel
    vec3 position       = texture2D(texture, coordPos).rgb;
    vec3 velocity       = texture2D(texture, vTextureCoord).rgb;

    float xAcc          = snoise(position.x, position.y, time);
    float yAcc          = snoise(position.y, position.z, time);
    float zAcc          = snoise(position.z, position.x, time);

    velocity            += vec3(xAcc, yAcc, zAcc);
}

Where snoise is the noise function and I passed in time as well so it will keep changing constantly. But this is just roughly how it looks like, in the real life you need to tweak the value in order to get the natural movement feeling. The last thing is that you need to prepare 2 framebuffers and swap them every frame, so you can always get the result of last frame and update it to the other framebuffer.

his.fboTarget.bind();
this._vCal.render( this.fboCurrent.getTexture(), this.fboForce.getTexture() ); // Perform the calculation
this.fboTarget.unbind();

...

var tmp = this.fboTarget;
this.fboTarget = this.fboCurrent;
this.fboCurrent = tmp;

 

Adding interaction

The final step is to add interaction to it. With Leap motion we can easily get the position and velocity of the hands, so we can easily determine a force with position of the hand, and its strength will be determined by the length of the hand velocity. As for the direction there are couple of options : the first one is to take the direction of  the velocity, which is the most common one. However it can be improved with using the direction of your palm, which leap motion is able to give us (hand.palmNormal). This will make it feel better when you do several movements in a roll, trying to push the particles to same place. And one final touch to this is to check the dot product of the hand velocity and this palmNormal, if the dot result is smaller than zero which means they move in different direction, we should set the strength to zero to avoid the weird movements.

To apply this force to our particles, first we need to create a force texture like this :

gestureForce2

Again we use color to represent the force. Back to the shader, when we calculate the velocity of the particle we need to add this force as well. So the shader will look like this now :

else { // vTextureCoord.x > .5
    vec2 coordPos       = vec2(vTextureCoord.x - .5, vTextureCoord.y);   // get the coordinate of the position pixel
    vec3 position       = texture2D(texture, coordPos).rgb;
    vec3 velocity       = texture2D(texture, vTextureCoord).rgb;

    float xAcc          = snoise(position.x, position.y, time);
    float yAcc          = snoise(position.y, position.z, time);
    float zAcc          = snoise(position.z, position.x, time);
    
    velocity            += vec3(xAcc, yAcc, zAcc);

    // get the force pixel by the position of the particle
    vec3 forceGesture   = texture2D(textureForce, position.xy).rgb;   

    // map the force value to -.5 to .5 and add it to velocity   
    velocity            += forceGesture - vec3(.5);                      
}

Summary

So that’s how I build this. The concept is not complicated, but there are a lot of small steps to take care. Also because everything happens in texture and shader which makes it hard to debug. Sometimes you just get a white or black texture and hard to tell which step went wrong. But once you got it all working and you can push for a huge amount of particles, that feeling is incredible. It’s a really good practice for learning framebuffer, shader and particle movements, I learn a lot and had a lot of fun when building it.

Here is a short video of the Samsung project I build if you are curious how it looks in motion : https://vimeo.com/92043935

 

Substrate Cube

I still amazed by Jared Tarbell’s work every time I go back to his site even if it’s created already 10 years ago.  I’ve try to recreate his substrate years ago in flash and it was so much fun to build it.

Last week I went back to his substrate again and wanted to recreate it in javascript. I haven’t done any generative coding for a while, it feels so good when I pick it up. I really like the feeling that you set up some rules and just let the code run. Every time you get an unexpected result and amazed by them. For this substrate experiment, the rules are simple :

1. Start a line and moving forward.
2. When hit the edge of the canvas or another line, stop.
3. If this line is longer than minimum length required, generate 2 more lines from this line.

It’s just this simple and it creates such an amazing result, of course there are few bits to make it looks better but this is the basic idea. So here is the javascript version I created :

substrate_0014_Screen Shot 2014-09-13 at 16.16.39

http://www.bongiovi.tw/experiments/substrate/

Few things about this experiment : First thing is the size of the canvas, I doubled the size of the canvas and resize back to normal. In this way you can get a lot details and feel less pixelate, especially those shadows. The second thing is I draw by directly modifying the image data of the canvas. The process of it is to get the position of the pixel that you want to modify in the big array of the image data. Change it and call context.putImageData. A tip for the performance is that doing putImageData for every time you change a pixel is super heavy. In every frame I need to update a lot of pixels, so the better way to do it is not to call putImageData until you’ve updated all the pixels you want to change, then just call it once every frame.

 

Put it on a cube

After I done this i get an idea to put it on a cube. I image it will be quite interesting to watch the line march over the edge from one side to another. So I started to create the texture like this :

substrate02_texture

It looks simple but actually quite a challenge to find out when the line hit the edge where it needs to appear again and what’s it’s new direction on the texture. I’m really glad I sort it all out eventually. Here is the result :

substrate_0004_Screen Shot 2014-09-15 at 11.06.10

http://www.bongiovi.tw/experiments/substrate3d/

I’ve also put some screenshots here.

 

Summary

I had so much fun building this. And it reminds me the talk of Mario Klingemann ( Quasimondo ) ‘s talk on the RTBC this year. I really like one thing he said : when you create something, most of the time you will find somebody already done it before, and sometimes it’s really really long time ago. But it doesn’t matter , the important thing is the process, you will always have new inspirations from building it or solving the problem. And for me that’s where the new idea begins. I feel so sad when I work with people trying to find new ideas and when there’s an idea being brought to the table someone just said : “It has been done before.” I’m not a thinker, it’s hard for me to just “think” something new. I need to build something and start from there, and after trying add new stuff to it or improve it for number times then I might be able to find a new idea. This is how it works for me, if you just ask me to think a new concept I will never able to  find it by just thinking.

That’s why I like to go back to these sites, they are old indeed. But they are timeless to me and are amazing. Almost every time it gives me some new ideas. So if you haven’t try it, I encourage you to do it on your own. You will find a lot of fun during the process, and you will enjoy every unexpected result it brings to you.

DIY Steampunk Keyboard

It is such a stupid idea but in the same time it is so much fun to build it.

Few month ago I saw this ( Qwerkywriter ) on the internet, it caught my eyes right away, it looks amazing and beautiful. However there’s one problem with it : It’s too expensive. Don’t get me wrong, I believe the quality of the final product will be amazing, and I believe he spend a lot of time and effort to build this and it looks great. But for me it’s just hard to spend 300 dollars on a keyboard. So I looked around and found actually there’s a lot of people doing their own customize vintage or steampunk style keyboard. At the end I found this one, it looks great and seems to be possible for me to build a similar one. So I decided to build one on my own.

 

Getting the parts

The first thing to do is to get the parts. I choose to buy a mechanical keyboard because it has a better type feeling and also the sound of hitting the key is closer to vintage typewriters. This is not too hard to find. And then it comes the challenge : the keys. It took me some time to finally settle with the metal buttons. I was searching for typewriter keys which already comes with the letters but they are quite expensive too, and also for a modern keyboard you have about 105 keys in total, for the vintage typewriter you only get 35-50 keys i think. Which means you need to buy 2 or 3 full sets of them and also need some customize jobs too. Therefore I switch to search for metal buttons, which you can find a lot on amazon or ebay. There are couple things you need to be careful : the first one is the size of the button, you don’t want it to be small but you don’t want it to be too big either, from my point of view i think between 14 ~ 16mm is the best. The second thing is that you want it to be flat, some buttons comes with a small ring in the back. I don’t have proper tool to remove it and bear in mind we are looking at over 100 keys. To remove it for everyone of them is going to be a huge amount of work. At the end I found these :

SteamPunk_02

These buttons are perfect to me, they do have that small thing on the back but it’s really flat so it doesn’t matter. And I really like edge in the front, make it look like one of those vintage typewriter keys.

 

Building it

So finally we get all the things we need and can start building it. What I did is really simple : I remove the key from the keyboard, then cut the 4 sides of it and leave only the top. And then just use super glue to glue these buttons on to the key. There are things you can do to improve this such as minimum the surface the key and also make it thinner as well. As for me this is already good enough.

SteamPunk_03

But these are only the small keys. For the bigger keys such as space bar, shift, backspace and enter, I don’t want to put just a button on it. It will looks empty and hard to type. So I decide just to remove the 4 sides and leave it like that, which looks quite ok to me to be honest. However there some extra work needs to be done for these big keys : I need to polish the edges. Because when I cut them it leaves a very ugly and uneven edge, I want to polish it and make it smoother.

Again I don’t have the right tool to do it, but I don’t want to spend some money on a tool that I won’t be using that often. So I asked myself : why not just build it myself, and I can have some fun with my LegoNxt ! And here it is, my DIY lego nxt polisher :D

SteamPunk_04

The button on the left is for turning on and off, the ultrasonic sensor is for detecting the distance from my hand to the wheels. The original idea is that the wheel will start itself automatically when my hands is close to the machine, and stop itself when I move away. It does work however I get some noise from the ultrasonic sensor ( return a lot of zeros ) and also I find actually easier just let it run. So at the end I just disable it but it’s still very fun to play with these sensors. It’s a simple thing which I spent about 2 hours to build it and make it work, but it’s perfect for polish my keys. Here is a short video of how it works  :

 

So that’s it, that’s my DIY steampunk keyboard, I’ve never feel so nerdy in my life :D There are still things can be done to make it better. But I kind of enjoy the look of it now so i’ll just leave it like this for now. To be honest it’s not very difficult to make one, I spend the most time on cutting the keys, but if you have proper tools it could save you a lot of time. Also I really enjoy building this mini robot. I’ve always been working with codes and haven’t explored hardwares that much. My next goal will be learning Arduino and build some awesome robot !

 

 

Chinese calligraphy in 3D and Reasons to be creative

I’ve been playing with Chinese brushes for a while now, besides the ink drops i created and use it to create mountains, I’ve created these strokes too. 03_strokeMy favorite part of these strokes is the gradient, I created them actually by accident. I find some amazing work of 張大千 and there is a lot of beautiful gradient in his work. I was wondering how to do it but it turns out not as complicated as I thought, of course there’s a lot of trials and errors, I throw out a lot fail tests before i get this. The trick is simple : put only color ink on your brush, when you are about to draw, just dip some black ink on the tip of your brush and there you go, really easy and you get a very nice gradient. I had a lot of fun trying with different portion of color ink, black ink and water, at the end it’s very hard for me to stop and to pick the one I’m going to use in my code, because each one of them is unique and have its own character. I think this is one of the reason why I like to create these ink textures so much. It’s a similar process as doing generative art : you have few controls, and you just let it run and enjoy the result, and then you go back to tweak these controls again and try to discover new controls or new settings. I’ve done this a lot in the code but it’s the first time for me to do it outside computer, really enjoy it.

 

Bringing it to code

After I created these strokes, actually I don’t really know what am I going to do with it, they look amazing but I don’t have any ideas how I can use it. Until one day I was building some prototypes for a project and doing some explorations, and one of the idea is to create ribbons, then suddenly this idea strike me : how about put the textures on the ribbon, and that’s how this started. The moment I put it on the ribbons, it feels totally fit with it. And really gives the feeling of the real brushes. I show it to my colleagues and they all love it. So I start to make some decorations for it : adding a texture background, some ink drops and lastly : a video layer to overlay on the textures, it makes the texture moves constantly, it just a small touch but make it feel different.

Calligraphy_01

A online demo could be found here.

 

Drawing a smooth ribbon in 3D space

One problem I had while building these ribbons is that it sometime twisted in 3D space.  Calligraphy_05

The one on the left is twisted, you can see the normal (  the purple lines ) are flipped to the other side. Luckily there is a solutions for this called Parallel Transport Frames , I was building this in cinder and it’s already part of the framework so it’s very straight forward to use. You can check the Tubular sample of cinder. Using this could generate a smooth ribbon ( the image on the right), you can see the normal is now all in the same side.

 

Another theme, another world

Two years ago when I went back to Taiwan and visited the national museum. I found something really fascinating that haunt my mind for years. Which is this old book, these golden characters are just so beautiful to me and also the dark blue background. And since after I’ve been always want to find a place to try out these colours.

Calligraphy_02

After I build the prototype this idea come to my mind again, therefore I put the golden colour on the strokes and put a dark blue background to give it a test. Surprisingly it works ! I did try to keep the gradient of the stroke but turning them into grey scales then overlay this golden colour on top of it. Also the video layer helps a lot as well.

Calligraphy_03

And it looks a little bit flat so I add some random shadows around it. I put this on my backyard with projector for my kids and they love it :

I actually did couple more prototypes based on this however they are build in C++ with cinder. I’ve put my source code on the github which you can find it here. It includes 2 versions in cinder, one is with leap motion, the other one is for the projection table i made, with kinect. And a version in web which is build with WebGL. I didn’t have too much time to go into details of these codes so if you have any questions please send me an email. And also the video texture that i use is too big to upload, so you might need to find one yourself or remove it from code.

 

Reasons to be creative

One of my resolutions this year is to give a speech on stage, so when I saw that Reasons to be creative is looking for elevator pitchers I didn’t hesitate too much then send them my proposal. I was thinking about to give a talk about my DevArt project with my friend together. But John (Davey) of RTBC replied it’s impossible to do that, the elevator pitch need to be solo, but he is very kind and said that he like both of our works and offer us one pitch each.  Therefore we decide that my friend Bertrand will still be presenting our Kuafu project, and I will be talking about these ink experiments i made ( strokes and ink drops )

It’s first time for me to step on the stage, I was scared to death. I am really glad that i didn’t do this alone, myself and bert we practiced a lot in our hotel room and time ourselves so we had a good sense of our timing. Also the crew from RTBC ( Chris and Andy ) helped a lot as well, they let us know all the details we need to take care of and always cheering for us. We rehearsed couple of times and at the end it all went well on the stage. It is such a great experience that I won’t ever forget. I encourage you to do the same if you haven’t done it before, it’s scary but also it’s a lot of fun ! Also I am really glad to meet all the other elevator pitchers as well, they are all very talented and amazing.

Calligraphy_04

I was at RTBC last year but i was working for the first day and get called back the last day. So this year is the first time for me doing the whole event. I’ve never been to other events like this and this is totally mind blowing. I enjoy almost every talk I went. Pretty much every talk implied one thing : work on your stuff, don’t wait things to happen. You might get a chance to use one of your old stuff someday. I think this is the biggest thing i get from these 3 days, keep doing the stuff you like and enjoy the process. I really feel full of energy and motivations to work on my stuff after this event. Now I’m just trying to find a way to keep the motivations not to be destroyed by the daily life.

 

Summary

I really glad I made this 3 minutes talk, and I really enjoy these 3 days. I’ve been working by myself for a long time and it’s really great to see that there are some people like my work. That gives me a lot of motivations to keep going. I hope i’ll get other chances to give speeches in the future, i’ll keep working my way to it. But the most important of all is to enjoy and have fun. That i’ll keep in my mind and create more stuff.

 

Photoshop curve and gradient map in shader and texture

Recently I’ve been working on a project which I need to recreate some photoshop effects in code. Most of them are quite easy to do or to find sample shader codes, such as hue/saturation/brightness and contrast. However there are still 2 effects need a bit more work : Curve and Gradient map.

Screen Shot 2014-08-15 at 11.25.02  Screen Shot 2014-08-15 at 11.31.43

The way curve and gradient map works is to map your current color value to another one, so the question for us is how to save this mapping information and then pass it into our code. It seems quite complicated at first glance, we need to save a lot data such as all 3 RGB channels plus an overall one for curve. However there is an easy way to do it : using texture. What we need is to create a base gradient texture from black to white like this :

baseTextureand apply your curve or gradient map on this texture to get the mapping texture you need.

export02

The next step is to do the mapping in the fragment shader, for the curve :

uniform sampler2D   texture;
uniform sampler2D   textureCurve;

void main(void) {
    
    vec2 texCoord   = gl_TexCoord[0].st;
    vec4 color      = texture2D(texture, texCoord);
    
    color.r         = texture2D(textureCurve, vec2(gl_FragColor.r, 0)).r;
    color.g         = texture2D(textureCurve, vec2(gl_FragColor.g, 0)).g;
    color.b         = texture2D(textureCurve, vec2(gl_FragColor.b, 0)).b;
    
    gl_FragColor    = color;
}

and for the gradient map :

uniform sampler2D   texture;
uniform sampler2D   textureMap;

#define sq3         1.73205080757

void main(void) {
    
    vec2 texCoord   = gl_TexCoord[0].st;
    vec4 color      = texture2D(texture, texCoord);
    float lenColor  = length(color.rgb) / sq3;
    color.rgb       = texture2D(textureMap, vec2(lenColor, 0)).rgb;
    
    gl_FragColor    = color;
}

This is really nice and easy, we don’t need to find a way to save mapping data with lots of numbers, a simple texture will do the job , and it’s much easier to export the texture too.

Chalkboard wall

About a month ago, my wife have this idea of turning one of our walls into a chalkboard wall, so the kids can draw on that ( instead of on our furnitures …. ) We did it and it looks amazing,  now we even spend more time ourselves to draw than the kids. We use projector to help us to draw as we are not good enough yet to draw without any help, it’s cheating I know but it looks way better :D  Now more and more idea come to our head after we build this chalkboard wall, all kinds of different style we want to try, also we want to try projection mapping too with our drawings, lots of fun stuff to do.

Now I really recommend to all the people who want to decorate their house, it’s really easy to make a chalkboard wall, you just need to clean the wall and put the paint on, that’s it, and you have your own creative corner starting up. And also because it’s black ( or dark ) it actually makes your space feel larger. Search for chalkboard wall on the internet, you will find a lot of interesting stuff, and it’s a easy thing to do to make your space look different and fun !

Lego NXT / Processing

When I left my last job, my friends gave me a big farewell gift : LegoNxt. It’s an amazing gift, i’ve always loved lego. And when I was a boy the toy I love the most is always lego. And LegoNxt is even more awesome for me now as I become a programmer, combining program and lego is the sweetest dream for me.

So in order to return my thanks to my friends, I try to come up with a project using LegoNxt, I did have an idea a year ago and had a working prototype, but the result is not satisfying enough, however I really enjoy the process of building robots and control them through codes.

This time i had a simple idea : I want to record myself building another lego set, but i don’t want the camera stay still. I want it to move slowly from one side to another. So i begin to build this little lego car and control it by processing.

 

NXTComm

I found this NXTComm library and it’s very useful. It allows you to control your legoNxt via bluetooth, and it’s very easy to use. To setup you just need to do this :

_nxt = new LegoNXT(this, Serial.list()[4]);

As I recalled the only tricky thing is to find the right bluetooth port you are using which is the [4] means, you might need to test a little bit to find out which one is exactly the one you need.

After this you are ready to go, the library is very compact, you have all the api you need including both the sensors and motors. I didn’t need any sensor on this project but i did a quick test with ultrasonic sensor to get the distance from the sensor to the object, the result is very responsive, makes me want to do more with it. As for the motor it’s really simple to control as well :

_nxt.motorForwardLimit(LegoNXT.MOTOR_A, maxForce, 150); // moving with limit
_nxt.motorForward(LegoNXT.MOTOR_A, maxForce); // just keep moving, need to call _nxt.motorStop manually.

So this library covers pretty much all the controls you need, the only thing left is to build the actual robot. It’s so much fun to build it e.g. how to place the gears to control the speed so it won’t go too fast, some old knowledge from high school/university i thought i’ll never need them become very handy now :)

I’ve always worked on the coding part and didn’t get involved with the hardware part too much. But now i’m really interested in building robots, hope I’ll have more time to dig into this field and I’d like to learn arduino as well !

 

And last, a short video of the recording result using this little robot:

夸父 Kuafu

It begins sometime about half year ago, i was trying to recreate a chinese style 3D world. By the time the idea is really simple, just grabbing some chinese ink style mountain textures and trying to put them together. Then a good friend of mine saw this, start chatting with me about it. He said it would be interesting to use the real map data to construct the world, and also along with a good story. At that moment this ancient chinese myth “Kuafu” ( 夸父 ) come to my mind, it suits this idea perfectly, a giant chasing the sun across the land. We both like this story and get really excited about this project, we start right away playing the some experiences, not long after my friend came out with this amazing design :

Styleframe_2_50

It was so beautiful and went behind my imagination about this story. Later on I start to work on the first working prototype :LINK


We really like this prototype and get more ideas while play with this, so we start to make it a project. The first step is to create the storyboard :

After this prototype and storyboard, we both caught up by works and kept us busy for some while until one weekend. It was a friday evening and I am on my way home. Suddenly I have this idea to record a ink drop and use it as a texture for mountain. Our first prototype looks good but the mountains are flat, I do want to make a real 3D mountain but I have no idea how to create the texture for it. And then this idea struck me, so i do a quick test on that night.

The result is better than I thought it will be, they make a really good looking chinese style mountain.

mountainsWithTextures

So I went back and create more textures, trying and playing with different colors and also test on different papers. I really enjoy this process of creating textures, it takes time to create them but sit there watching the ink flows create all kinds of interesting and beautiful shapes is really exciting. As a dev we all know the more randomness we throw into the code the more alive it will become and have more variety. But there’s nothing can compare with the actual thing. Every ink drop will create different shapes based on how thick is the ink, how high you drop them , how much water is on the paper, the flow of the water, the tiny different on the paper itself, it’s all these things that you cannot control which make it more beautiful. And also make it feel really different when you put it in 3D render.

After these testings, the next step is to get the real elevation data and try to recreate the terrain. In the beginning I was using the google elevation service to get the data, which works perfectly, and the most amazing thing is that it even return the elevation underneath water. However I was afraid that we will hit the api call limit quite quickly as we are going to generate a good amount of mountains, so I switched to use an elevation map :

earth_height

The idea and the way to do it is really simple : We translate the latitude and longitude to x and y on this map, read the pixel value of this coordinate of this point, the brighter the pixel is, the higher the mountain will be. And each time to create a mountain , I set a minimum height of the mountain and ignore the those lower than this, and also check if there is already a mountain nearby, if not, then I’ll create a new mountain.

heightCombine

And in the same time my friend created these amazing style frames :
Styleframe_4

Styleframe_3

I was really excited with these stunning images he created, and we really like this landscape layout, so we start thinking : Why don’t we make it an installation ? We think it will looks better on a long panoramic format and also this will make it feel more immersive. So we tweak our direction from an online experience to an installation.

Just at this moment, google launch a new project call DevArt, we think it’s a really good chance for us to show our project to the world. So now we’ve put it on here. And also because now we are making it an installation, we think it will be great to have some sound effect in, and would be even better if it’s interactive. So we invited our other friend on board to work on the sound design part. He joins with lots of amazing ideas with sounds and make the sound design a big and interesting part of this project right away.

In order to make an installation, I switched to cinder now, combined with Node.js as our server and an HTML5/JS page as controller. This week I started to work on the communication part. Sending the data from the controller to Node.js server and then to the frontend using different technologies.

So now we are working to finishing the project, it’s really interesting to look back all these testings and prototypes. It seems that we have already done a lot. But the truth is there is still more ahead. I want to thank my friends who give us so many positive feedbacks after we announce this project. It’s definitely a good motivation for us to keep working on it. We will keep update our dev art project page, and I will keep updating my blog on this project too, even if the dev art is over.

WebGL – Depth texture

It took me a while but I finally get the depth texture working. Basically I just follow the instruction from this article and it worked. However I encounter some problems when trying to make this work :

After trying the technique in the article I mentioned, the first result I get is a total white screen, I thought I made an error so nothing got rendered on the screen but after a further research I found somebody said that he had the same situation but it turns out actually the depth texture is working, it’s just the value he got is very close to the max value therefore it looks all white. So i make a simple test to change the actual white (1.0, 1.0, 1.0 in shader) to red and this is the result i got (on the left is the all white screen, on the right is that i dye the actual white to red, you can see the shape of the mountains) : So it turns out the code is actually working, I just need to find a way to display it correctly. So I went back to do more search and i find this way to show the depth texture :

depthTexutre0

void main(void) {
	float n = 1.0;
	float f = 2000.0;
	float z = texture2D(uSampler0, vTextureCoord.st).x;
	float grey = (2.0 * n) / (f + n - z*(f-n));
	vec4 color = vec4(grey, grey, grey, 1.0);
	gl_FragColor = color;
}

n is the nearest depth and   is farthest depth, after applying this method the depth texture seems much more normal :

depthtexure0

But when I switch back to my render there is something wrong :
depthtexure1
You can see the depth is rendered weirdly, looks more like layers, and another thing I discovered is that the depth texture value i got is extremely close to 1 ( > .995). And the depth texture i get looks like layer too, but not like the depth texture inside the article that smooth. Therefor I have to go back to search again then I found out somebody says it’s because I didn’t have the right setup for the zNear and zFar. A too little zNear or a too big zFar will result a very wide range of depth results,and this will limit the precision of depth buffer. By this time the zNear and zFar I was using is .1/10000. So I scale everything down and also bring closer the camera so it looks the same in size, but the zNear and zFar become 1/2000. And this time everything works, the depth texture looks perfect, and the layer effect above is gone as well.

depthtexure2

So finally I got it working, I put up a little depth of field effect using the depth texture here, there is a checkbox for you to see how the depth texture looks like. It’s really a long way for me to finally get it. The main reason is there is not too much resource on this topic, however you will find more articles if you search for opengl depth texture instead of webgl texture. Here is a very good link tell you about the things you want to know about the depth texture. Again, all the code you need is in this article, enjoy the depth texture and now you can create some more advanced effects.