Home / 2016

Case study – Night Eye

Here comes my annual blog post 😛   I should make more posts but i got too busy/lazy.

Recently I was invited to take part of the Christmas experiment this year. Along with my colleague Clement and my friend Bertrand we got an idea of using abstract lines to recreate the shape of the animals in the forest.

You can check out the experiment here :
http://christmasexperiments.com/2016/01/night-eye/

Also if you happen to have HTC VIVE, give it a try in the latest chromium build. It’s also a WebVR project.
In this project Clement has taken care of the line animation while I was focusing on the environment and the VR part. The following section is the case study of my part.

 

Initial Designs

Here are some pictures of initial designs  with different colour theme:

nighteye1

nighteye2

 

Reflection Matrix

The idea started with my experiments with reflections. I’ve always wanted to understand how to create a proper reflection and failed so many times. But then I found a really good tutorial on youtube, walk through the process step by step. I highly recommend to have a look if you are interested in implement the reflection youself. The tutorial is in Java but it covers all the concept and explained it clearly, plus the shader doesn’t change ( too much ) .

https://www.youtube.com/playlist?list=PLRIWtICgwaX23jiqVByUs0bqhnalNTNZh
The only problem I got to follow this tutorial is the clipping plane, which I think webgl doesn’t support ( Please correct me if I am wrong ). So end up just using discard to do the simplest clipping. I’ve also find another really good presentation about rendering reflection in Webgl, in it it mentioned other ways to clip, you could have a look :
https://29a.ch/slides/2012/webglwater/

 

Editor


In order to get the best position and the right angle for the animals, we created a simple editor for us to place the animals and tweak the camera angles. It took a little bit extra time to build it, but it saves us a lot of time tweaking. It’s always easier when you can visualise your settings in live. After we have selected the positions and camera angles in the editor we just export a big json to the project and it’s done.

 

WebVR

In this project we want to try the latest WebVR API. Which is really amazing ! They make it really simple to implement. The first step is to get the VRDisplay and setup the frame data holder :  

vrDisplay = navigator.getVRDisplays();
frameData = new VRFrameData();

Then in the loop you can get the data by :

vrDisplay.getFrameData(frameData);

Rendering

For the rendering it’s become really simple. The WebVR now returns the view matrix and the projection matrix of both eyes to you.

setEye(mDir) {
    this._projection = this._frameData[`${mDir}ProjectionMatrix`];
    this._matrix = this._frameData[`${mDir}ViewMatrix`];
}

You can just pass it into your shader and you are ready to go. No need to setup the eye separation, no need to calculate the projection matrix. It’s just simple like that. And the code become really clean too : Set the scissoring, set the camera, render then it’s done.

GL.enable(GL.SCISSOR_TEST);
const w2 = GL.width/2;

//	get VR data
this.cameraVive.updateCamera(frameData);

//	left eye
this.cameraVive.setEye('left');
scissor(0, 0, w2, GL.height);
GL.setMatrices(this.cameraVive);
this._renderScene();


//	right eye
this.cameraVive.setEye('right');
scissor(w2, 0, w2, GL.height);
GL.setMatrices(this.cameraVive);
this._renderScene();


GL.disable(GL.SCISSOR_TEST);

The next is to present in the VR headset. Which they make it really simple too :

vrDisplay.requestPresent([{ source: canvas }])

Then at the end of your render call, add:

vrDisplay.submitFrame();

Then it’s on.

However one more thing need to do but a simple one : You’ll need to use vrDisplay.requestAnimationFrame instead of window.requestAnimationFrame in order to get the right frame rate.

The WebVR api is really awesome and easy to use. There are couple things to check but I’m pretty sure you can just group them into 1 tool class. Here is a simple checklist for you: 

  • Matrices : View matrix / Projection Matrix
  • Scissor for Stereo Rendering
  • VR frame rate
  • Present mode for VR

And don’t forget to check out the examples from https://webvr.info/ you got everything you need to start in there.

Controls


After rendering, the next step for us is to implement the control. The interaction of our project is simple : press a button to go to next step and press another button to drag the snow particle with your hand. We are using the gamepad API with WebVR. It’s really straightforward. Start with : 

navigator.getGamepads();

To get your gamepads. You might get multiple gamepads so do a check get the one you want. After this for the position and orientation are in the gamepad.pose. The button states are in the gamepad.buttons. And these are everything you need to create the interactions.

 

Summary

It has been a lot of fun to work on this project with friends. And a good challenge too for learning and using the latest WebVR API. Again like I mentioned they’ve made the API so easy to use and recommend everyone to give it a try. I am really surprised by it and also how little time it took me to convert my old projects into WebVR. If you are interested in the code, it’s here : https://github.com/yiwenl/Christmas_Experiment_2016/

So let’s it, hope you enjoy the read and I wish you a merry xmas and happy new year !

nighteye4

 

P.S. Some behind the scenes for the commits 😀

commits

A talk of my projects.

Recently I have been invited by an old friend to give a talk about my projects. Mostly I go through the talk I gave last year at Reasons to be creative and add a bit more. Here is the video.

I got people asked me this question constantly : how could you make so many experiments ? especially you are father of 2 kids, where and how do you find the time to make this ? My answer is always the same : because of the fear, The fear of that if I don’t make this projects now, I won’t have any time in the future to make it. I hate to see these ideas dies in my head. Having children completely changed my life. And without questions it consumes me a lot of time. However I find myself more productive then before I had the kids. Why ? I think I used to think I’ll always have time to build things in the future. So I always get distracted by films, games … etc. But now whenever I had some time and I have an idea, I’ll grab whatever I can to make it. I’m not saying that you should have kids ( or perhaps you should ?! 😛 ) But more about trying to grab as much time as possible. Don’t expect to have a complete and long quiet time to build your idea. Most of the time you won’t have it. Do small parts one by one, divide big tasks into small tasks.

Then the next question I got asked a lot is why I am making these projects ? Because I want to have fun ! How many times you see all the amazing projects from creativeapplications.net and you dreams about making your own ? How many times you see all the crazy WebGL experiments on chromeexperiments.com and wish you had a project like that ? and how many time you actually got the project ? I think we all know the answer. I am not a person of patience, I want to have fun building , experimenting and learning things. I don’t count on the client project to do these. Yes it might be easier with client projects because you’ll have a proper budget and time to do so. But you never know when it will come. I don’t want to wait, I want to have fun right now. I think everyone understands the importance of the R&D, and everyone wants to do it because it is fun. Since you know it’s fun, then there’s nothing stoping you to have these fun yourself.

There’s also one other reason I’m making these : for my kids. The are old enough now to understand what I am doing. They live in a whole new generation and going to use computers a lot in everyday life. Personally I want to show them that computer is just a tool helping us to create or form or ideas. Besides I really enjoy making things for kids because they’ll give you the most straightforward feedbacks. If they love your stuff they’ll just tell you and let you know you’ve done a good job plus a bit smile. I was really lucky to have a chance to setup an installation for the kids at Geneva international school earlier this year. One of the best moments is the a kid (of age 6) approach me and talk to me how much he enjoy the installation and thank me. During these 4 days in the school my friend and I become the super star ( A.K.A. the STARGAZERS ) The smiles and feedback from these kids are truly amazing. They thank me for bringing the installation to them, but the true is I thank them for giving me motivation to build more.

 

DSCF3087

DSCF3124
These are the main points I want to share. Everyone has her/his own way to do R&D. But the most important thing of all is as once my friend told me : have fun ! You gotta to find the joy in building these experiments/projects. Fun is the only way to make you keep doing it, find the stuff that inspires you no matter where they are coming from. It could be a picture, a game, a book or anything. Then you could start from there building your project and having fun !

 

P.S. You can find the presentation of my talk here : http://slides.com/wen/deck-3