Hybrid ASIC/FPGA system for fully automatic stereo-to-multiview conversion using IDW

Recently, multiview autostereoscopic displays (MADs) have become commercially available which enable a limited glasses-free 3D experience. The main problem of MADs is that they require several (typically 8 or 9) views, while most of the 3D video content is in stereoscopic 3D (S3D) today. In order to bridge this gap, the research community started to devise automatic multiview synthesis (MVS) methods. These algorithms require real-time processing and should be portable to end-user devices to develop their full potential. To this end, we revisit an algorithmic solution based on image domain warping (IDW) and devise a hardware architecture of a complete synthesis pipeline, provide insights on where the computationally challenging parts are, and present implementation results of a hybrid FPGA/ASIC prototype-which is the first hardware implementation of a complete, IDW-based MVS system. Based on these results, we also estimate the complexity and energy efficiency of a fully integrated solutionin 65nm and 28nm CMOS technology and show that a full-HD real-time solution on a single chip is within reach. The proposed architecture could be used as a co-processor in a system-on-chip (SoC) targeting 3D TV sets, thereby enabling efficient content generation with limited user interaction (e.g. depth range adjustment) in real-time.