Dear visitor, welcome!
This week I visited the Blender Institute and decided to wrap up the multiview project. But since I had an Oculus DK2 with me I decided to patch multiview to support Virtual Reality gadgets. Cool, right?
Gooseberry Benchmark viewed with an Oculus DK2
There is something tricky about them. You can’t just render a pair of panoramas and expect them to work. The image would work great for the virtual objects in front of you, but it would have the stereo eyes swapped when you look at behind you.
How to solve that? Do you remember the 3D Fulldome Teaser? Well, the technique is the exactly same one. We start by determining an interocular distance and a convergence distance based on the stereo depth we want to convey. From there the software (Cycles) will rotate a ‘virtual’ stereo camera pair for each pixel to be rendered, so that both cameras’ rays converge at the specified distance.
Oculus barrel correction screen shader applied to a view inside the panorama
This may sound complicated, but it’s all done under the hood. If you want to read more about this technique I recommend this paper from Paul Bourke on Synthetic stereoscopic panoramic images. The paper is from 2006 so there is nothing new under the Sun.
If you have an Oculus DK2 or similar device, you can grab the final image below to play with. I used Whirligig to visualize the stereo panorama, but there are other alternatives out there.
Top-Bottom Spherical Stereo Equiretangular Panorama – click to save the original image
This image was generated with a spin off branch of multiview named Multiview Spherical Stereo. I’m still looking for a industry standard name for this method. But in the meanwhile that name is growing on me.
I would also like to remark the relevance of Open projects such as Gooseberry. The always warm-welcoming Gooseberry team just released their benchmark file, which I ended up using for those tests. To be able to get a production quality shot and run whatever multi-vr-pano-full-thing you may think of is priceless.
Builds
If you want to try to render your own Spherical Stereo Panoramas, I built the patch for the three main platforms.
* Don’t get frustrated if the links are dead. As soon as this feature is officially supported by Blender I will remove them. So if that’s the case, get a new Blender.
How to render in three steps
- Enable ‘Views’ in the Render Layer panel
- Change camera to panorama
- Panorama type to Equirectangular
And leave ‘Spherical Stereo’ marked (it’s on by default at the moment). Remember to post in the comments the work you did with it!
Last and perhaps least is the small demo video above. The experience of seeing a 3D set doesn’t translate well for the video. But I can guarantee you that the overall impression from the Gooseberry team was super positive.
Also, this particular feature was the exact reason I was moved towards implementing multiview in Blender. All I wanted was to be able to render stereo content for fulldomes with Blender. In order to do that, I had to design a proper 3D stereoscopic pipeline for it.
What started as a personal project in 2013 ended up being embraced by the Blender Foundation in 2014, which supported me for a 2-month work period at the Blender Institute via the Development Fund. And now in 2015, so close to the Multiview completion, we finally get the icing on the cake.
No, wait … the cake is a lie!
Links
- Multiview Spherical Stereo branch [link] *
- Multiview: Cycles Spherical Stereo Support Official Patch [link] *
- Gooseberry Production Benchmark File [link]
- Support the Gooseberry project by signing up in the Blender Cloud [link]
- Support further Blender Development by joining the Development Fund [link]
* Time traveller from the future, hi! If the branch doesn’t exist anymore, it means that the work was merged into master.
Nice Oculus
Thanks! This is not mine though 🙂 Oculus is one of the supported platforms of the Blender-VR project, to be presented at the IEEEVR 2015 next week.
If you are interesting in interactive virtual reality and need an open source solution for your CAVE, multiple Oculus or video wall, give Blender-VR a visit. I’m participating in the development of a framework built on top of the Blender Game Engine.
Also if Oculus feels like sending me my own Oculus, I wouldn’t mind. If you do, though, consider sending one to the Blender Foundation as well. I will feel bad when I take the device away from them next week.
Have a good one,
Dalai
Update:
Due to the long review process the patch is not yet in Blender. That said, since there were enough people interested on this feature, I just updated the links above with a more recent build (on top of current Blender 2.76 RC3).
Update:
The build now also supports regular perspective cameras. This is required for cube map vr renders. For this I also recommend an addon that I was commissioned to build, to render or to simply setup cubemap renders [link].
Note: remember to change your camera pivot to center.
* Last build update: October 2nd 2015
Hello. Thanx for this method, but i cannot render stereopair like you show. It always render only one equirectangular image and i dont see ‘Spherical Stereo’ anywhere.
Hi, where did you find the “Spherical Stereo” option? I am as lost as you were once. 😛
I finally found how to render it! My first Blender render so far ) But here another question, Right now this is only available for Cycles Render, what about Blender Internal renderer? I’m asking because i want render some volumetric stuff and particles, but looks like Cycles dont support this yet.
This method (not multiview, but this spherical stereo method) is Cycles only since it requires a per-pixel algorithm.
The only way of having the equivalent (so you could perhaps mix with Blender Internal) is to render strips of render, and combine them separately. You can find about that in this thread [link]. Maybe you can ask there for people to share their script.
Thanx a lot for info. Do you have any links where i can read about this perpixel algorithm? I wish try implement this into Unity3d. Maybe point me where in Blender sources to explore? Or something else.
For Unity you can just render in realtime the current point of view. Why bother with producing a whole panorama at once?
That said you could do that with vertical strips (see the link above). But that means rendering the image multiple times.
As for the per pixel algorithm it’s in the kernel_projection.h changes present in the patch [link].
Hi Dennis
I seem to be having the same problem you had, I am only getting one image. Could you give me some tips on how you did it please?
Thanks a lot.
Neil
Pingback: Viewing the Gooseberry Benchmark scene on an Oculus DK2
You write “Enable ‘Views’ in the scene panel”
but it is on Render Layers panel
Right, fixed, thanks 😉
Good gravy I hadn’t heard of this or the Blender VR project until now, but I’ve been dreaming of VR headset integration with Blender ever since I heard of the Oculus Rift. This is so exciting!
If you have Janus VR someone converted your example into a 3D skybox you can view with Janus. Here’s the link: http://janusvr.thevirtualarts.com/3D/3Dskybox.html and the discussion on Reddit here: https://www.reddit.com/r/janusVR/comments/2zw8kb/experimental_blender_build_that_can_output/
really excited about this new features.
Here is a quick test scene I came up with
https://drive.google.com/file/d/0B8xQW_KQ6Ph-b3AxNXBFNTR4UVU/view?usp=sharing
A strange thing is that I export the image as LR. But when I view this image with LiveViewRift, I feel more comfortable when view this as RL.
I think it could be that the convergence distance being too close?
I will need to do more testing and get familiar with the setting.
Definitely looking forward to rendering an animation with this.
Cheer~
I ran into this issue as well (LR -> RL), I didn’t run further tests though. It should be easy to fix, but it may as well just be the expected result.
“Panorama type to Equirectangular”
Where is this option under? I can’t find it anywhere.
After you set your camera to be ‘Panoramic’ the next option is the panorama type (the default value being ‘Fisheye Equisolid’)
This is great. I see a few things missing in the stereo side of things, but so much is there, it’s great!
A near / far clipping plane that shows up to aid you in visualizing when things get too far or too near (and get farther than a specified max disparity, like 1/30th or 1/100th of picture size)
The ability to turn OFF stereo.
Rendering both stereo cameras in cycles preview
Turning a 2 frame motion tracked stereo pair -> stereo camera feature. (probably a lot of work, but would save trillions of dollars every year for matching stereo cameras)
Anyways, thanks so much for working on this! It’s super fun.
> The ability to turn OFF stereo.
Well you can disable the ‘Views’ option in your scene, but I guess you want to stop previewing everything in stereo. In the Viewport properties you can choice to see only one of the views.
> A near / far clipping plane that shows up to aid you in visualizing when things get too far or too near (and get farther than a specified max disparity, like 1/30th or 1/100th of picture size)
The ‘Stereoscopy’ panel in the Viewport properties has the ‘Plane’ and ‘Volume’ options. They should help you (though none of them work on a pixel separation base).
My idea is that people can expand the UI to drive the values based on pixel separation, and maybe in the future we integrate that in the official interface.
> Rendering both stereo cameras in cycles preview
This is in the far-fetched todo list, but not as simple as I would like it to be 😉
> Turning a 2 frame motion tracked stereo pair -> stereo camera feature. (probably a lot of work, but would save trillions of dollars every year for matching stereo cameras)
Mind clarifying that?
> Anyways, thanks so much for working on this! It’s super fun.
You’re welcome, thanks for testing it 🙂
> Turning a 2 frame motion tracked stereo pair -> stereo camera feature.
You’d take your stereo pair, and motion track (match move) the camera. Even if it’s two separate lenses, it’ll work. In a scene with enough tracking data, you would be able to get an exact interaxial, and what model the cameras are using (toe-in / parallel). It would aid in matching a scene with a real world camera.
This is the blender 3D plugin I used to use, and it had full near / far plane support. You might be able to re-use some of it’s code.
http://www.noeol.de/s3d/
> The ability to turn OFF stereo. <
It would be really nice to have a stereo-off option in the D menu. (also, how do I activate grease pencil now?)
Hi guys – can I download Multiview branch build for windows somewhere? I’ve checked Graphicall and couldn’t find any. I’m not that keen on building it myself but would love to test stereoscopic spherical panoramas.
Cheers,
Maciej
Thanks for your interest. Any recent build of Blender already has multiview, so grab a fresh copy on http://builder.blender.org/download
Hello Dalai Felinto,
I work for a company using 3D Vision stereo for aerial images, using them to measure 3D data with photogrammetric software.
I like your effort in Blender tryining to integrate different ways of stereo.
We run everything on quad-buffered stereo, but I like your ideas of Oculus :-).
I try to follow you and to see what is coming up.
All the best to you,
Tom
Hi Tom,
I love your company’s work by the way. It’s flattering to know that you’re considering to use part of my work in your pipeline.
> We run everything on quad-buffered stereo
Have you tried your system with Blender 2.75 yet? Quadbuffer is fully supported at the moment.
Also did you see my experimental VR Viewport addon?
https://github.com/dfelinto/virtual_reality_viewport
It was developed for fun initially, but nothing stops it from being used for real work.
And by the way, I’m currently using Blender Game Engine for visualizing aerial captured geometry, and it looks beautiful.
Take a look at my latest post (on Planovision) to see the device. This is Blender Game Engine related, more than Blender itself, but I think you may like it.
hey dalai,
first of all: thank you, that you work on the implementation of such great feature… most people still have no idea how absolutly amazing a stereo 360 image / animation looks like.
you wrote that “Any recent build of Blender already has multiview” – this is correct but not “Multiview Spherical Stereo branch”, right? I downloaded the last builds (ghooseberry branch and experimental) but there is no ‘Spherical Stereo’ option. Do you know when this feature will be in the official builds?
with best regards
Simeon
Hi Simeon,
You are correct.
Multiview is in the upcoming 2.75, but I still have to finish the “Spherical Stereo” implementation.
I should do it shortly, but as far as official builds go, you will have to wait until 2.76. That said, once the feature is committed, it should be easy for you to get a semi-official build with this.
What I can do is to provide an updated build for all platforms as soon as 2.75 is officially out.
And thanks for your kind words. It makes a world of difference to me knowing that people are benefiting from my code 🙂
Hey Dalai,
thank you for the information. Now I know I wasn’t wrong and the feature’s on the blender roadmap soon. Currently I work with the experimental version from this post (win64). An updated build to 2.75 would be great as well, but only if it’s not too time consuming.
Hi Dalai,
I have a gear VR, would it be possible to get a ‘stereo cubemap’ from this stereo equirectanguar using the compositor… found a blender setup that did this for mono pano’s…
http://blenderartists.org/forum/showthread.php?267759-Cubemap-Generator
or even better a camera type that does this directly…
more info here : http://riftdev.com/real-world-stereoscopic-panoramas-with-gear-vr-formats/
so i’d have to create two cubemaps from the equirectangular and then put them together in the carmack format…
i almost have the noodle setup to create a stereo cubemap from the stereo equirectangular render for orbx viewer.
i want to test it , but the current stereo build freezes… any update to 2.75, dalai ?
I’ll post a link here to the noodle when it is ready…
thank you, Frederick
I made a blend file to generate stereo cubemaps for gearVR from the stereo equirectangular panorama’s.
Maybe it is of use to someone on this thread…
https://www.dropbox.com/sh/asmo1kqn78t8w5u/AAAzDkQniwzQ87_Bh_miIUP1a?dl=0
If something’s not working leave a message, i’ll look into it.
greetings, FrederickD
Awesome… thanks Frederick Desimpel… exactly what I needed!
Hello all, once again thanks for being interested. I just updated the above links with fresh builds (on top of Blender 2.75a-ish). Enjoy!
@Frederick, may I suggest something like a pre-rendered EXR to be used with the UV Node?
Something like the method I used for fisheye rendering from Blender Internal:
http://www.dalaifelinto.com/?p=149
Hey! I grabbed the sweet new build, and the panorama camera options have vanished!
I’m really excited to use this for stereo 3D 360s on youtube!
Which platform did you try? Remember that the panorama camera is only available for Cycles.
good catch! I didn’t turn cycles on! Let the fun begin!
Builds updated again with a fix for bumpmaps. Be aware that I also fixed an issue where top and bottom where flipped, so you will experience a regression.
Thank you so much for this feature! I can’t wait to try it. I downloaded today’s build (Aug 19), but when I try to render I get a error “Camera ‘Camera’ is not a multi-view camera.” This happens only when I have RenderLayer/Views enabled, and set to Multi-View. Doesn’t matter if I tick left (eye) or right (eye) or both. What am I doing wrong?
Never mind. I found the solution. I needed to create 2 cameras, position them correctly to account for inter-pupilary distance (IPD), then give them names like “MyCamera_L” and “MyCamera_R” (with suffixes matching the “Camera Suffix” that’s configured in RenderLayer/Views.
You can use the Stereo 3D option for most cases. The Multi-View is for more advanced setups.
Hey! Been trying out this feature for the last 24 hours, and I’m very happy with it! I just have two small issues that I thought I might get some insight on:
1. We have one scene that constantly renders from the same POV, regardless of camera placement. Haven’t been able to figure out why.
2. The panoramas seem to have an odd vertical “boarder” on the left and right edges of the final images, which creates an odd vertical line from zenith to nadir when viewed in an HMD. Has anyone else had this effect, and do they know how to correct it?
Hi Darren,
1. No idea, can you share the file? Are you using the equirectangular panorama render or the cube map addon method?
2. Are you using any compositing effect? Anything similar to blur or vignetting effect will stand out badly in the panorama.
Thank you so much for the fantastic work, I’ve had a fantastic experience with the Multiview Spherical Stereo.
I’ve been having some difficulty with the cube map rendering add-on, unfortunately. It seems like the zenith frame isn’t lining up to the others. Do you have any suggestions for what I might be doing wrong?
If the add-on temporarily creates Left/Right cameras in every direction and directly renders from those cameras, there will inevitably be stitching errors (unlike the bourke slice method you linked to above). I checked out the source, but my Python is rusty, so I couldn’t tell if this is the case
Hi James,
Thanks for your interest on my work.
> It seems like the zenith frame isn’t lining up to the others. Do you have any suggestions for what I might be doing wrong?
Can you reproduce this in a simple file and send it my way?
> If the add-on temporarily creates Left/Right cameras in every direction and directly renders from those cameras, there will inevitably be stitching errors (…)
The addon is to be used on top of the spherical-stereo branch with the “Use Spherical Stereo” option on, the perspective lens, Cycles and pivot set to middle. It shouldn’t cause any stitching error (it’s what the spherical-stereo branch is for).
Frederick Desimpel, your Cube Map blend file is awesome! I came across the same issue that James mentioned above. Fortunately the fix is simple: In the “Render Stereo Cube” node editor (top center pane), the top-right-most “Render Layers” input is set incorrectly to “L_right”. It just needs to be changed to “R_right” (by using the Browse Scene button — it’s already contained in the list.) After that change, upon F12, the scene renders correctly.
Thank you for a clever solution!
Quick update: When running the CubeMap blend file in Blender 2.77, the final render appears nearly completely white (as if it’s been blown out). Any thoughts? (For now I’ll continue to use 2.76b.) Thanks!
I installed Whirligig, downloaded your Top-Bottom Spherical Stereo Rectangular render and it doesn’t work.
The image loads but it’s distorted. What format should I choose in Whirligig to view the render correctly? None of the standard ones display this properly.
If you modified Whirligig to display your format could we get a copy of your modified version??
Hi. I used Whirligig with the “Barrel OU” (over-under) mode. Try swapping the left and right eyes (O key)
Hey! Outstanding work! Really well done (and super problem solving). Looking forward to seeing it integrated for real and also whatever great new tricks you have up your sleeve.
Hi dfelinto,
I’ve been using your code in 2.76b to output Stereo 3D in equirectangular format. When viewed through GearVR and DK2, I’ve noticed that my eyes have a hard time focusing on details at certain angles. For example, in a render of a car interior, when attempting to focus on the stick shift down to my right, it becomes uncomfortable. I actually have to cock (ie tilt) my head to the right a bit as well, and find the sweetspot… otherwise the left image is not parallel to the right image and my eyes cannot converge.
I don’t know if I’m making any sense, but I wondered if anyone else has noticed similar?
If I look at stereoscopic renders from OTOY, etc, I don’t see the same problem; I don’t need to tilt my head side to side. I wonder if it’s particular to Blender?
Is there any chance the algos are rotating the cameras other axes when they shouldn’t (or aren’t when they should?), or somehow over/undercompensating the effect at particular angles? (I’m not even sure I’m asking the right question, hah!)
Just thought I’d pop the question. Thanks!
I just spent all evening researching why my latest stereo render from Blender has broken perspective, when the previous render of nearly the same scene looked ok!
Now I see Justin describing my exact issue. It doesn’t even seem consistent; looking to my left and down I have to tilt my head sideways 90 degs to get the viewpoint to merge well. Looking to my right and down however requires the same a tilt but only half as much angle! That is probably due to a non-symmetric scene; these panorama images aren’t easy to mentally unwrap.
I need to create a ‘ruler’ scene to help quantify what is going on here.
(seen on my Gear VR).
Have you tried rendering with parallel convergence (either set the convergence distance to really high, or use parallel – the parallel will only work with a recent Blender or the upcoming Blender 2.78).
I’m not sure about GearVR but I’ve heard that Youtube requires this.
Sooo, if anyone stumbles across this blog, dfelinto just tweeted that Sergey Sharybin (sergey) has developed a Pole Merge function. That’s great news; I’m assuming it will fix this issue.
So this didn’t get rolled into 2.77? How come?
I’m still seeing it in 2.77. Did you set your renderer to Cycles?
Yeah, Cycles is my default setup but I don’t see it. I see that its on the list for 2.78 but no 2.77.
Justin, yes, that alignment issue is very ‘distracting’ IRL as well– and generally due to a torquing of eye-views upon rotation. Your real eyes have compensatory muscles to resolve this — a conflict of verticality out of vestibular plane– with matching neural alignment compensation, as do most animals. (Vision science anatomy is my specialty)
Check to ensure that cameras parented to center can rotate together in 6D. They need to have a vertical “spring” (like the old gimbaling problem) to avoid conflict, and probably do invisor. There may be a conflict that the eyes are trying to solve that the cameras try to anticipate. In effect, convergence plane may not be rotating: though we know that sounds silly, getting one’s head around this is difficult.
I’m away from my computer, typing on a phone in airport, but desperate to understand this problem.
Dalai, you are amazing. Thanks for all your work.
Hey Dalai, I have a few questions and I hope you have time to answer them. As stated in your article: “The image would work great for the virtual objects in front of you, but it would have the stereo eyes swapped when you look at behind you”. Now the problem is that when I render the panoramic image in cycles, I then use a panoramic photo viewer that I downloaded from the android market to view the picture in VR. Im not sure what is going on, but the problem where the stereo eyes swap is still happening whenever I look 180 degrees behind the front.
Any help is greatly appreciated, thanks for your hard work!
Hi Jake, in order to benefit from the technique presented here you need a coy of Blender 2.78 (yet to be released), as well as turning Spherical Stereo on in the camera panel. You can get it a snapshot of it on builder.blender.org, while waiting for the upcoming release.