Last summer, I got into an interesting course entitled “Emerging topics in information technologies”. During the first part of the course, we focused more on 3D computer vision techniques and how to manipulate 3D images.Issues such as depth-image-based rendering were obscure and captivating enough to get me motivated throughout the summer semester.
In order to go even deeper in our discussions concerning 3D images, we were given Kinect 2.0 for half the semester to do a small project in WPF. The goal of the project was to produce a 3D image in real-time on the 3D glasses free TV Dimenco screen using the Kinect 2.0. Achieving this goal would let to have an hands-on experience with analyzing 3D images and their rendering using a given technology stack.
So this was the workflow.
- Connect on the Kinect
- Connect on both the depth and color sensors from the Kinect
- Acquire a frame from each of the sensors
- Process the information coming from the color frame
- Process the information coming from the depth frame
- Map the depth of the environment with the color to ensure to render 3D environment properly
- Correct physical differences between the two cameras in the Kinect 2.0
- Insert an image header on the top left corner of the Dimenco screen
- Render mapped result of the cameras on the 4K screen
As easy as all this can sound, this was not a trivial student project. Having some experience with C# and .NET made the work a bit easier for me than for my classmates who didn’t have the chance to get some experience with the MSFT stack. Although, it still took me the whole four weeks and 25 hours of development, mostly doing some trials and errors to find the optimal solution that would show the perfect 3D image in real-time on the 4K screen. My last problem by the time I hand in the project was that I thought, personally that the rendering could had been sped up and optimize. All things consider, the project was a success !
You can learn more of my project and use the source code here