Color Acquisition, Management, Rendering, and Assessment in 3D Reality-Based Models Construction

Emerging Digital Tools for Architectural Surveying, Modeling, and Representation Chapter proposal: Color acquisition, management, rendering, visualization and assessment in 3D models construction from reality-based data Author: Marco Gaiani The aim of chromatic and tone color definition is to identify the fidelity color and tone level of a digital image compared to the original or intermediate document used. The color reproduction coherence depends on a number of factors, such as: illumination level during acquisition, sensor characteristics, mathematical representation of color information along the whole digital pipeline. The principal reasons of the mismatching between colors are manifold and well described in literature. In the field of Architectural Heritage (AH) analysis, conservation and management color definition and reproduction is a key step. In this case, the fundamental problem during acquisition is looking for relationships between incident and reflected light in a surface point. In general, the solution of this problem requires understanding and controlling environmental and artificial light sources over the measurement set. For this reason the color capture of masonry faces, historical architectural handmade and monumental-historical buildings is a very complex problem, because it’s deeply connected to subjective aspects of visual perception, objective light source characteristics and the visualization modality. An accurate capture of the visible spectrum is quite difficult, especially if it’s not well distributed. For these reasons, if it’s not possible to ensure the measured color fidelity, a basic requirement is to ensure the fidelity of perceived color. To give solution to these problems, in the restoration field, was developed many techniques and we could state that the color recording, usually, today refers to three methods: − sample transcription; − visual comparison with color atlas (i.e. the ‘Munsell book of color’); − diffuse reflectance measurement with instruments like colorimeters, spectrophotometers or telephotometers. These techniques, besides to present problems beyond the ability of an actual sample of existing matter, they aren’t able to ensure the correct perception of color on a RGB display (i.e. LCD displays or DLP videoprojectors) or its faithful reproduction on a print support. No one of these methods, in fact, is able to ensure a right color checking on a wide surface, and with a non-uniform color, that is the typical condition of AH. The problem is much more sensitive when the goal is to create reality-based 3D digital models, for which we need to identify color, texture, reflectance property and normal surface directions and some other issues are in play, mainly the need of a rendering engine and some kind of virtual illumination sources to visualize our digitized artifact. A key step of 3D modelling pipeline from reality-based data is then to ensure accurate fidelity of colour reproduction and visualization using web-based real-time rendering techniques of 3D models. This not easy task is, besides, becaming more and more important with the growth of Structure from Motion (SFM) 3D capture techniques, that integrate shape and color capture in an unique solution. It is therefore considered adequate to ensure the fidelity of colour, texture, and surface reflectance perceived properties rather than guaranteeing metric accuracy of rendered data. In this chapter we present main elements of the color acquisition, management, rendering, visualization and assessment in AH 3D models construction from reality-based data. Our aim is to illustrate easy, low-cost and rapid procedures that produce high visual accuracy of the image/model while being accessible to non-specialised users and unskilled operators, typically Heritage architects. The presented processing is developed in order to render reflectance properties with perceptual fidelity on many type of display (from consumer to VR room) and presents two main features: a. is based on an accurate color management system from acquisition to visualization and more accurate reflectance modeling; b. the color pipeline could be used inside well established 3D acquisition pipeline from laser scanner and/or photogrammetry. Besides it could be completely integrated in a SFM pipeline allowing simultaneous processing of color/shape data. Specific attention is given in the description the measurement procedures used in image acquisition, colour management and colour information mapping to a 3D model. Methods and techniques illustrated are based on the use of digital cameras and software post-processing with color temperature control. This allows identification of color and diffuses reflectance of an entire building with just few shots and chromatic reproduction of a sample zone with very high approximation. It’s well known that it’s possibile to acquire surface reclectance using polychrom laser scanner. Howeaver we need to use white light laser, hardly manageable especially outdoors. Today 3D scanner are then equipped with a digital camera, but, generally, the quality (both geometric and radiometric) of images is limited. For this reason, reflectance properties are usually acquired through digital cameras and shoot separated from the scans. In the described pipeline workflow, operational standard and techniques are completely device independent, and for this reason instruments choice, within certain limits, it is not essential. From an operational point of view , in order to produce reality-based 3D models that meet geometrical radiometric properties of the original object, operations related to the acquisition and visualization of reflectance properties of objects are known as color processing, a set of steps that include : - Acquisition of color as texture (image files separate from the 3D model) or as a color assigned to each vertex of the geometry of the model; - Re- collimation of texture or color -per- vertex on the geometric model; - Compensation of the ambient light; - Removal (or radiometric improvement) of shadows and/or specular reflections; - Seamless attributes creation; - Texture Level of Detail processing to ensure consistency during the rendering runtime (pre- calculated or in real-time). The input of this process is a set of color images, the parameters of related camera (i.e. orientation and calibration) and the 3D geometry of the scene. Required output, result of the procedure generally known as texture mapping is a texture map or set of maps seamless or a set of per- vertex colors able to reproduce the reflectance properties of real surface modeled. This output need to be accompanied by a correct and reliable procedure to ensure faithful display of mapped color, in the visualization phase at runtime of 3D textured model. In fact the goals someone might want to achieve when computing synthetic 3D imagery of Architectural buildings is that the rendered image enables to predict accurately what a virtual scene should look like. In order to meet the quality requirement for imagery using predictive Real-Time Rendering (RTR), assessement of acquisition, processing and visualisation techniques is need. In the last part of chapter an image quality assessment using image quality assessment algorithms to evaluate the consistency of each simplification step, is illustrated. The chapter is organized following the colour pipeline from acquisition to visualization highliting main problems of each step and illustrating related solutions. 1. Introduction 2. Technological equipment and workflow 3. Colour fundamentals, color models and color spaces 4. Colour profiles 5. Target and color bar 6. Colour acquisition by BRDF 7. Colour acquisition by High Dynamic Range (HDR) imaging 8. Photometric calibration 9. Colour processing 10. Texture mapping techniques 11. Colour rendering 12. Colour visualization on RGB displays