Render of the week - Blender + luxrender (arlux)
Background Plate + Lighting - captured environment
Against all the odds the sofas are real (so far it seems that most people tend to think they are the 3d elements). The carpet is pure 3D though (well, based of a 2D image actually). The spheres are also 3d, but we have a lot of the scene elements modeled as “support meshes”. That’s how the spheres can get the right reflection and lighting.
Also can you see what the carpet is covering ?
Ah, and this image has no post-processing or compositing on it. It comes blended with the real elements straight from the render.
Cheers,
Dalai
It’s still in its Beta version and I’m evaluating it in production only now, so things will likely change.
quick render test
However if you want to take a first glance at it you will need:
- Recent Blender (after 2.63a)
- Readme
- Download (import as addon)
Some screens:
blender, top right cycles rendering background, top left addon to add the background with glsl shader
ARLuxrender at work
Note: my goal is to use arluxrender as the final renderer, but all the modelling and editing is to be done inside Blender.
The original teaser with old screenshots can be found here: http://www.dalaifelinto.com/?p=377
For further discussions you can visit the Blender Artists forum thread as well.
If you do some real testing with it, please let me know your thoughts.
Dalai
Project developed with Aldo Zang at the Visgraf lab
]]>After a productive holiday we now have something new to play with
model by Jar-Artist
model by Jar-Artist
Models/scene gently provided by Jar-Artist
http://www.blendswap.com/blends/author/jay-artist/
The proof is in the pudin
It’s “easy” to put up a nice artistic effect together and simply assume everything is working as it should. However, I wanted to make a system that could match the effect produced by real lens. For that I had to build a fail proof experiment. People not into coding may not know, but building a reliable testing and debugging setup is one of the keys for efficient coding. In my opinion it’s always worthy to put time into this. Note: this is also why Blender users can help a lot with bug fixing by simply building proper test files for the (also carefully/methodologically) reported bugs).
1 – Photoshooting
Take some pictures with a tripod rotating the camera on its center (the focal centre actually). We have been doing this for the past two weeks so it was smooth. Those pictures were taken by Aldo Zang in the Visgraf Lab at IMPA.
2 – Stiching
I don’t get tired of recommending Hugin for stitching and panorama making – hugin.sourceforge.net. This open source project sometimes works better even than autopano pro (a pretty good commercial alternative).
3 – Rendering
I put the panorama as a background plate, calibrated the aligment, added a simple floor + spheres. This was done with the (yet to be released) IBL Toolkit. Apart from that my Blender camera needs to match the settings of the real camera+lens I’m aiming at.
In this case all the pictures for the stitching were taken with a Nikon DX2S and a fisheye 10.5mm lens. I created new presets for the sensor dimensions and the render output.
4 – Results
I was quite pleased when I compared the rendered output with the original image. The aspects we should be looking at are only field of view and line distortion across the lens:
Also note the bottom left corner of the photo. This subtle shadowing is due to the vignetting of the lens. This is not present in the cycles render because I’m not implementing a real camera model (as shown here and here).
Algorithm
The complete patch can be found here. The core is the following function (simplified here). I elaborated this from the ‘classic’ fisheye equisolid formula: radius = 2 * focallens * sin ( angle / 2):
float u, float v, float lens, float width, float height)
{
u = (u - 0.5) * width;
v = (v - 0.5) * height;
float r = sqrt(u*u + v*v);
float phi = acos(u/r);
float theta = 2 * asin(r/(2 * lens));
if (v < 0) phi = -phi;
return make_float3(
cos(theta),
-cos(phi)*sin(theta),
sin(phi)*sin(theta)
)
I hope you like it. If you can share any work you did with this feature I would love to see it.
Dalai
This is only possible because Cycles is a full ray tracer render engine. Every pixel in the image is generated from a ray coming from the camera to anywhere in the scene. Enough talking. A quick hack in the code and tcharan:
IBL background plate + ibl toolkit (alignment addon) + cycles 'use panorama' + fisheye patch
And the nice thing is, it previews in 3D just as well:
What comes next? I will talk with Brecht to see if there is any pending design to have this implemented as another camera option. I would like to have an option to set the angle (so we don’t need to do only 180 degrees fisheyes). And to toggle between hemispherical and angular fisheye modes.
I you compile your own Blender and want to try the patch, get it here or:
===================================================================
--- intern/cycles/kernel/kernel_montecarlo.h (revision 45899)
+++ intern/cycles/kernel/kernel_montecarlo.h (working copy)
@@ -215,13 +215,29 @@
__device float3 equirectangular_to_direction(float u, float v)
{
+ u = (u - 0.5f) * 2.f;
+ v = (v - 0.5f) * 2.f;
+
+ float r = sqrt(u*u + v*v);
+ float theta = acosf(u/r);
+
+ if (v < 0.f) theta = -theta;
+
+ return make_float3(
+ sqrtf((1.f - r*r)),
+ -cosf(theta)*r,
+ sinf(theta)*r
+ );
+
+
+/*
float phi = M_PI_F*(1.0f - 2.0f*u);
float theta = M_PI_F*(1.0f - v);
-
return make_float3(
sin(theta)*cos(phi),
sin(theta)*sin(phi),
cos(theta));
+*/
}
/* Mirror Ball <-> Cartesion direction */
]]>
IBL autosetup for Cycles and Luxrender
IBL Re-Aligment
Home made panoramas also work
Point projection for calibration tweaking
Ideas, questions, comments, feel free to drop a line
The course is ministred by the professors:
Luiz Henrique de Figueiredo
Luiz Velho
What’s up doc?
The problem proposed was to create an algorithm to subdivide a triangular mesh inscripted in a sphere (e.g. an icosahedron) multiple times in order to approximate the mesh to a complete sphere. We need to solve two different mesh structures:
- STL-like files – where triangles are listed by the coordinates of their vertices.
- OFF-like files – where triangles are listed by the indices of their vertices, and the vertices are listed by their coordinates.
I’m using only .ply files for both approaches. If you choice -m stl and the faces are indexed, they are internally converted to a storage format that treat them as OFF files.
The delivery of the project is the following program/script. It requires python to be installed in the computer.
How to run it
1.1) Download the final script here – subdivider.zip
1.2) Download a sample ply file: icosahedron.ply.
2) Unzip and run the command:
./subdivider.py icosahedron.ply -l 5 -m ply -o icosahedron_5.ply
* in MS Windows you need to call it with the python console or set it to automatically run using the Python installation available.
* if you want to force the program to use the stl method use -m stl in the arguments.
3) The output file is ready. You can check the result in Blender or MeshLab
STL output file | PLY output file
For the rest of this document I’m explaining only the ‘off’ (ply actually) approach. The ‘stl’ one is more a warm up, given that it’s no challenge compared to what comes next. Both algorithms can be found in the complete script above.
Result and performance
Result rendered in Blender, levels 0, 1, 2, 3 and 4
Performance Comparison
| Subdivision | Python | PyPy |
| level 1 | 0.0001 | 0.0002 |
| level 2 | 0.0003 | 0.0005 |
| level 3 | 0.0011 | 0.0019 |
| level 4 | 0.0046 | 0.0074 |
| level 5 | 0.0186 | 0.0881 |
| level 6 | 0.0746 | 0.2395 |
| level 7 | 0.3336 | 0.2790 |
| level 8 | 1.5101 | 0.8054 |
| level 9 | 6.6167 | 3.0026 |
| level 10 | |
11.3281 |
Without any changes in the code, the execution with PyPy showed an improvement of 100%. It still doesn’t compare to native/compiled code, but those are satisfactory results given the simplicity of the implementation and the flexibility of using a script language.
Rational
The core of the code is the subdivision algorithm. I decided to look for a solution where a face could be seen as as much independent as possible. That means no pointer to neighbour triangles and no need to navigate in the triangles in a fancy particular order. The key I pursued is to create unique indices in a ‘universal’ way. A system that could be used by neighbour triangles unaware of each other and that could result in the same values.
This may not be the most efficient algorithm, but it’s very simple to implement (the code in Python ended very compact).
Code overview
# 1) parse args
input, output, method,level = parse_args()
# 2) import ply
raw_verts, raw_faces = import_ply(input)
# 3) convert to internal structure
verts, faces = convert(raw_verts, raw_faces, method)
# 4) subvidide
verts, faces = subdivide(verts, faces, level, method)
# 5) export ply
export_ply(output, verts, faces)
Apart from the subdivision, they are other small steps needed: (1) parse the arguments (level, method, file, …); (2) import the ply file; (3) convert the raw data into an internal structure (more on that soon); (4) subdivide; (5) dump the new data into another ply file.
The conversion (3) rearrange the faces to store not only the vertices indices but also the indices of the edges. It’s a simple code that creates a key for a pair of vertex indices that will always return the same regardless of their order:
key = "%020d%020d" % (min(v1,v2), max(v1,v2)).
If the key was already created we get the id from a dictionary. Otherwise creates a new id and assign it to this key.
Subdivision code explanation
* take a look at the code in the end of this page, or in the download link in the top.
First I allocate new vertices in the verts array. We are going to create one vertex per edge, thus we know from beginning the size of the new array. With the array pre-created it’s easy to see if a specific vertex has been created (by the other triangle that shares this edge) or if it needs to be initialized in the list.
Next I create a unique vertex index that can be guessed/retrieved from both triangles that share the edge where the vertex is generated. Taking advantage of the fact that the number of new vertices is the same as the number of edges in the previous interaction, I use the edge index to directly assign a value for the vertex. old_verts works as an offset to the new vertices indices.
# new vertex indices
v4 = old_verts + ea
v5 = old_verts + eb
v6 = old_verts + ec
Now that we know the exactly id of the vertex, we can see if it was created already. There is no risk of array overflow since the new verts array was already expanded to accommodate all the vertices (filled with None objects).
if verts[v4] == None:
verts[v4] = (verts[v2]+verts[v3]).normalize()
For the new edges, 6 are external and come from the existent edges. It’s crucial to also find a unique index for them. My first attempt was to use an orientation system, always naming the edges clock-wise. This is not reliable though, I found it impossible to implement correctly. The final solution was to name an edge based on its opposite vertex. For example, in the initial triangle the vertex 1 faces the edge 1, the vertex 2 faces the edge 2 and the vertex 3 faces the edge 3.
Also part of the challenge is to split the edge in two new edges with universal indices. I used arbitrary comparison (vertex a > vertex b) to determine which edge will inherit the original index of the edge, and which one will be allocated passed the initial edge offset (old_edges):
e1 = ea + (old_edges if v2 > v3 else 0)
e4 = ea + (old_edges if v2 < v3 else 0)
The internal edges (edge 7, 8 and 9) are brand new and all they need is to get an index from the edge pile and increment the counter. This is the equivalent in C for e7 = edge_inc ++. edge_inc is initialized as an offset to avoid overriding of edge indices.
e7 = edge_inc; edge_inc += 1
Finally it’s time to create the faces following the original schema.
new vertices and edges
new_faces.append(( v6, v2, v4, e1, e8, e6))
new_faces.append(( v5, v4, v3, e4, e2, e9))
new_faces.append(( v6, v4, v5, e9, e7, e8))
Subdivision code-snippet
The complete code of the subdivision can be seen here:
for i in range(level):
old_verts = len(verts)
old_edges = int(len(faces) * 1.5)
edge_inc = old_edges * 2
verts.extend(list(None for i in range(old_edges)))
new_faces = []
for v1, v2, v3, ea, eb, ec in faces:
# new vertex indices
v4 = old_verts + ea
v5 = old_verts + eb
v6 = old_verts + ec
# add new vertex only if non-existent
if verts[v4] == None:
verts[v4] = (verts[v2]+verts[v3]).normalize()
if verts[v5] == None:
verts[v5] = (verts[v3]+verts[v1]).normalize()
if verts[v6] == None:
verts[v6] = (verts[v1]+verts[v2]).normalize()
# note, the following could be 'optmized' to run the
# if only once . the gain is of 3%
# new external edges
e1 = ea + (old_edges if v2 > v3 else 0)
e4 = ea + (old_edges if v2 < v3 else 0)
e2 = eb + (old_edges if v3 > v1 else 0)
e5 = eb + (old_edges if v3 < v1 else 0)
e3 = ec + (old_edges if v1 > v2 else 0)
e6 = ec + (old_edges if v1 < v2 else 0)
# new internal edges
e7 = edge_inc; edge_inc += 1
e8 = edge_inc; edge_inc += 1
e9 = edge_inc; edge_inc += 1
new_faces.append(( v1, v6, v5, e7, e5, e3))
new_faces.append(( v6, v2, v4, e1, e8, e6))
new_faces.append(( v5, v4, v3, e4, e2, e9))
new_faces.append(( v6, v4, v5, e9, e7, e8))
faces=new_faces
return verts, faces
I hope the explanation was clear.
It’s always fun to reinvent the wheel
Dalai
]]>
Teaser - click to see the fullsize - bge work from Martinsh Upitis
Can you spot the difference between those two images?
Blender 2.49, 250, 2.51, 2.52
Blender 2.63 and beyond
Of course you can. But it took me 3 years to realize that the Spherical Panorama I coded in the Blender Game Engine was wrong. Holly Coding Christ.
This is one of the BGE “dome” modes, but unlike the others, this one is used for debugging more than for deployment. Those past two days I’ve been investigating with Aldo Zang how to optimize the workflow for scene reconstruction from spherical panoramics (aka equirectangular images). The idea is to avoid the current workflow of setting up an IBL image as background, rendering a shot and using a calibration software to get the camera origin and coordinates. Given that we are talking about a very controlled scenario (where we know the FOV – field of view – of the camera, and can assume tons of other information) we believe we can improve the precision of the calibration process by addressing this particular scenario.
IBL in Cycles? Yup, if you use an HDR image in the background the light of the scene is almost solved. It takes only a few nodes:
Click to see the nodes
Click for the full size - HDR from HDRLabs
Our final goal is to use this with LuxRender (ARLuxrender actually), but all the reconstruction modelling will be done in Blender. Thus the need for a smooth workflow.
So what Cycles has to do with the BGE? Not much. But I was using the BGE spherical panorama to test the calibration software. As it turned out the code was wrong from the source 
that’s life I guess. Thanks Aldo for spotting that.
And why to have Spherical Panorama in the BGE again? Well if anything artists can do effects like this:
Spherical Panorama, ready to be transformed in the 1st image of this post - bge work by Martins Upitis
As a bonus you can see an image from the same scene, but warped with the old code:
It may be subtle, but this one can't be planeted like the one before - bge work by Martins Upitis
Using the color coded images from the top of this post you can see why they are different. The ceiling and floor lines are way higher/lower in the fixed warping.
Now for the code savy out there, this was the change made (in a simplified version of the code). For the actual code go to the Blender svn (look for FlattenPanorama).
Original Code:
{
φ = atan2(vert.y, vert.x)
x = φ / π
y = 0
z = atan2(vert.z, 1)
z /= π / 2
return x,y,z
}
New Code:
{
φ = atan2(vert.y, vert.x)
x = φ / π
y = 0
θ = asin(vert.z)
z /= θ / π
return x,y,z
}
Happy Easters,
Dalai
]]>
Two short throw projectors in one seamless desktop
I’ve been following the work of the Vision3D since 2009. This lab in Montreal is specialized in computer vision (recherche fondamentale et appliquée sur les aspects tridimensionnels de la vision par ordinateur). Lead by Sébastien Roy they have been producing (and sharing!) on calibration of projection surface (e.g. domes \o/), multiple projector systems, and content toolsets.
lt-align manual calibration process
The Vision3D lab main tool in that area is Light Twist. This tool was presented in the LGM2009 with a live showcase of the system in a cylinder. In the last week I tried to have light twist going with a multi projector system (aiming to use this for a dome later on) but so far I’m stuck in the playback of content (and I suspect the calibration stage is wrong). Anyways, light twist will be a topic of another post, once I get it up and running.
Plugin enabled – video in the middle of the screens, desktop working normally
Since 2009 the light twist project shifted its focus from labs to end users. In 2011 they finally presented a new project called lt-align and lt-compiz-plugin. The lt-align is a software to quickly calibrate the screens alignment, very easy to use.
The Compiz plugin requires some fooling around with ubuntu settings, but once things are in place it works like a charm. I’m yet to make it work with Unity, so I can have real fullscreen across the desktops.
Recording of the alignment process and video playback
Elephants Dream – Stitched Edition
Note: there is an extra package you need to compile the lt-compiz-plugin:`sudo apt-get install compiz-plugins-main-dev. And I didn’t have to restart compiz with ccp to make it work. Also I changed the shortcuts to start the plugin because Alt+F* were taken by other OS commands.
Time to make it real and project in a large wall
In this picture you can see Djalma Lucio, sys admin that oversees all the computer installations at Visgraf on IMPA. A great professional and a very funny guy to work with. Think about someone that actually enjoys opening a xorg.conf file. And you can also see in the right Aldo Zang. Check it out his ARLuxRender project – a plugin system for LuxRender “which allows to render scenes with mixtures of real and virtual objects directly, without post-processing”.
I hope to post more in the coming months in domes, projections, a special video project … I went on a 3-month leave of my work at UBC to join the research lab at Visgraf/IMPA, under the coordination of prof. Luiz Velho. This is the second week only, but it’s been already a great experience. And above all, it’s nice to be back home (Rio de Janeiro, Brazil).
Happy Twisting,
Dalai
It happened to me today. An image downloaded from the internet was failing to load in my project (a virtual art gallery for domes, more on that once it’s out). The internal framework involves to copy the image to the project folder and open it with Video Texture. For those unfamiliar with the Blender Game Engine, this is a python module to dynamically load and swap in-game textures.
In my tests all the images I tried were working. No exception. But of course it takes only a test-run with the client to get a crash One single image was enough to make me pull my hair.
Our beloved open image editor GIMP opens the image with no problems. In fact if I open it and save it I can open it in my project with no problems. So what’s wrong? Why can’t GIMP warn me about this problematic file?
Looking for ‘file inspectors’ I ran into this Binary File Inspector from Microsoft. It didn’t take more than a glance to spot the problem:
CMYK … Bingo! Opening the image in a station with Photoshop proved this was the issue.
For the adventure seekers out there, remember: open source tools are great. Yet you should be not afraid of getting out of your comfort zone once in a while.
Who could guess that Microsoft would be the cavalry to make it up for the lack of CMYK (and feedback) support on GIMP (or the lack of CMYK support in Video Texture, or me being short in tools for image forensics, …).
Have a great day,
Dalai
credits: Dalai Felinto, Mike Pan (Blender) and Sherman Lai (post processing)
It’s available on Ted Talk the presentation from Dr. Pauly on the ocean’s shifting baseline. The key idea is that we need to stick to a baseline in order to develop a more reliable feeling on the changes that are happening.
But what happens when we can’t see the baseline? In this case the use of simulations – films and images – can be of great help. In the final slide of his presentation, Dr. Pauly showed an image to suggest a simulated ocean in 2010. You can see this at ~ 8:12.
This is not one my favourite works, but it’s an important one. This image was made based on a still from the first animation made in Blender I worked on, back from early 2009: The Life in The Chesapeake Bay. It’s nice to look back and admire how many chances to improve my work I got.
To work with science communication is a thrill, and to have this work recognized really makes my day. Note that this image is not being used only to illustrate a particular ocean scenario. The image is there to make a point. To reenforce the role of art in the understanding of our lives.
. . .
And yes, it’s always great to spread Blender around the world, even when people are unaware of it (I was going to do a screenshot from the Blender file but I can’t find it – it took TedTalk way too long to make the stream online available
A belated thank you for Villy Christensen, Sherman Lai and Mike Pan for the opportunity of doing the original project together. And for York University and the unexpected strike in late 2008 God and his crooked lines, go figure.