About Project / Digital model

Difficulties in digitalising the Langweil model


Ing., Bc. Jan Buriánek during photographing the model

Prague City Hall together with the City of Prague Museum announced a public tender, on the basis of which the company Visual Connection, a.s. carried out the digitalisation of the model. This company which has experience in the field of modern picture processing technologies and complex solutions in the field of professional audio and video approached the problem with typical confidence. To handle such a complex problem as the digitalisation of the Langweil model it invited specialists in the fields of intelligent robotics, computer vision and computer graphics to work with it. The resulting team managed this exceptionally difficult task which was full of unique restrictions unlike any other project in the world.


The textures on the buildings
contain thin structures of drawing
in Indian ink which in their detail
exceed hundreds of lines per millimetre

There were many factors that made the digitalisation difficult. The first of these was the fact that the model is made of paper and wood and consequently is very fragile. Digitalisation had to be carried out without contact, purely on optical principles. The model could not be touched in any way as there was a risk of irretrievable damage. Another obstacle arising from the materials used was the limitation in the lighting conditions under which the photography could be carried out. The exhibit is kept permanently under the tinted glass of the protective display case partly because all infrared or ultraviolet light damages the paper model and its drawings. During the optical measuring of the model it was therefore necessary to use specially adjusted flash lighting which limits the light in the infrared and ultraviolet spectrum. A special circular coaxial flash was used that had a filter which was controlled by a special flash aggregate. Also the model could not be exposed to lighting for a long time, i.e. there was a limit on the total length of time that the model could be exhibited, expressed in lux-hours. Last but not least, the actual level of detail and the proportions of the model made the project incredibly complicated. The textures on the buildings contain thin structures of drawing in Indian ink which in their detail exceed hundreds of lines per millimetre (the measured details corresponded to roughly 1200 DPI). Luckily the customer limited the accuracy required in the digitalisation to bearable levels. The model contains more than 2000 “houses”, most of which are 3 - 6 cm tall but of course contain various dominant features (towers, columns, etc.) which are around 20 cm tall.


Example of grand plan – one of the model`s
parts (Malostranské square)

Partition of Langweil`s model into 52 parts
The actual model was created in parts. The whole model is divided into 52 separate parts. So during the digitalisation each part could be worked on separately.

Boroskop
Boroscop
Bc. Tomáš Petrů scanning
Bc. Tomáš Petrů scanning

Exit of boroscop
The level of detail and the geographical accuracy of the model of Prague as it then looked inspired the author to preserve even the smallest alleyways and secluded corners. Some parts of the model contain two to four-millimetre alleyways where it was practically impossible to get either light or photographic equipment. If the proportions of the model allowed it an industrial boroscope (alternative to a medical laparoscope) was used to photograph selected details of the model.


Example of DOF image-fuse exit from left to right:
Source scans wit different level of focusing

The proportions of the model made it necessary to use macro-photography equipment, i.e. the depth of focus of the area photographed was very small (usually a couple of centimetres) and it was necessary to take lots of photographs of one scene so that it was possible to subsequently make the pictures sharper for digital processing. An example of the result of the process of fusing pictures with various depths of focus (DOF image-fuse) can be seen in the pictures.

 

Photographing the model


Robot

The actual digitalisation of the model was carried out in several phases. In the first phase the field of vision was measured. This involves creating a redundant structure of calibrated photographs similar to the Lumigraph concept. In principle this process is the same as the photographic process but the result and the aim is fundamentally different from “ordinary” photography. A special photography robot was practically created for the project. This robot was “tailor-made” for the project of digitalising the model by top professionals in this field. The robot was responsible for semi-automatically photographing each of the 52 parts of the model using purely optical means. The robot contained special and very precise feedback controls which had a total of five levels of freedom and in its “hand” it had a specially adapted “camera” containing macro-bellows, macro-optics, adapted macro-flash equipment and, in particular, a highly precise photographic wall (CCD element with 16Mpix precision and dynamic scope of 13-bits per channel).

The robot took photographs of the model part by part according to carefully prepared controls. The aim was to photograph all the geometry and texture of the model so that a 3D reconstruction of the surface of the model could be made from the photographs. For this purpose the robot took photographs at a speed of one photograph every 4 seconds and during the whole process it took almost a quarter of a million photographs with the highest possible quality of the medium-format CCD wall. The redundancy of the photographs was due to the fact that each point on the surface of the model needed to be seen from as many angles as possible (the controls made it possible to see one place up to seventeen times) and the texture of the model was created by special apparatus fusing each of the photographs.

The fact that it was not possible to use sufficiently strong flashlight caused a lot of problems. The macro-flash equipment had to use special filters and the flash had to be limited considerably (including the length of time that it was used).
The number of photographs necessary for processing the model unfortunately also showed up the limits of the technology that was used. The shutter on the special “camera” only had a finite durability and under the parameters that were used the shutter was damaged after 100,000 photographs had been taken. So for the project the photographic equipment had to be exchanged and replaced with new equipment a total of three times. That too is the price paid for the digitalisation of a historic model in the 21st century…

The result of the photography was approximately 250,000 photographs with 16 Mpix resolution and 13-bit accuracy per channel which is a considerable amount of data to be stored. And also the storage had to be safe because it will not be possible to repeat the photography in the next fifteen years. The total capacity of the stored rough data exceeded 6 TB. With this volume of data and this number of files it is not only complicated “storing” the data but also making backup copies of it. Fortunately Visual Connection, a.s. has a lot of experience in this area and a number of its own solutions for systematically storing such volumes of data. That the model is divided into 52 parts helped considerably in organising the data.

 

Dokumentace modelu


“GIS” Application for the Langweil model

The documentation of the model, i.e. creating a kind of “GIS” for the model, also became part of the project. However the basis for the process was purely a map of the model drawn over by “hand”, which only approximately captures all 2000 houses and their description. The ground plan processing required the digitalisation of the “hand-drawn” base and its “mapping” on to the photographs of the model that were actually taken. The result in the form of a kind of GIS application became very interesting not only to historians but also to current surveyors because a number of the houses in the model no longer exist and the details of the foundations also show significant differences. Langweil made the original model according to the Juttner plan of 1816 but since then Prague has developed massively and so it is only Langweil’s model that records the geographic layout at that time correctly, i.e. the resulting “digital” map will be a kind of “GIS” for Prague in the years 1826 - 1834. The current GIS3D version of Prague was also included in the application for specialists so that it is possible to compare how much the current version differs from the one that Langweil captured in his model.

 

3D reconstruction of the model

In the first stage of photographing the model detailed photographs were created which show the model from many angles and levels. For each photograph a record was made, amongst other things, of where the photograph had been taken by the robot and from what angle, under what parameters, etc. From this information it was possible to proceed to the most difficult phase of the digitalisation, namely the 3D reconstruction of the model from the field of vision. In addition to the process of “developing” and comparing the dynamic properties of the photographs the positions, from which the photographs had been taken, needed to be specified exactly mathematically at this stage. Then it was necessary to gradually “fuse” the layers with the limited depth of focus (DOF). The whole process was procedural, i.e. on the basis of exact mathematical processes which were derived from the RAW data of the photography and at any time any of the steps taken was reversible or could be changed completely.
The actual 3D reconstruction of the model was carried out on the basis of specially adapted 3D computer vision algorithms. The whole process can be demonstrated using a test paper model.


Sample paper model for experiments
(part of Prague Castle)

More precise calibration of the angles
of the photographs on the test model

The physical model was photographed from many angles (systematically, with the help of a robot) using a previously known calibrated optical system and in a previously known light calibrated environment. The source photographs were subsequently adjusted so that the intensity of the colours could be compared and parasitic phenomena in the optic system (labelling, barrel shape, movement of the centre, disproportionate points, etc.) could be removed. The result of this process was a whole set of camera parameters for all the required angles for the reconstructed work.

To a certain extent this part of the process was done by a special calibration device which was developed in collaboration with Autodesk RealViz and which was derived from the MatchMover application (see www.visual.cz). Another step was to extract important points (corners, edges, etc.) of the model, from which the polygonal model was created. This part was divided into obtaining a point “cloud” and key points for the polygonal reconstruction. The point cloud was based on all the main points on the surface of the model and the process was to a certain degree algorithmic and semi-automatic. But for the reconstruction of the key points of the polygonal model manual marking was required and these points had to be selected by operators.

Own software was created for the selection and the polygonal reconstruction which marked points from views from above and from side views separately. This software was developed in collaboration with Autodesk RealViz and the basis was the ImageModeler product. The process of manually marking the polygons of the model was a very long one and required more than thirty thousand hours’ work by the operators.
Implementation of the computer vision algorithms to extract the POI points of the 3D reconstruction was part of the solution of the whole object.


Example of the key points
for the test model

Semi-automatic fusing of the surface geometry
of the reconstructed buildings

As soon as the points of interest and the calibration of the photographs was available it was possible to proceed with the 3D reconstruction and the creation of the surface model. The computed structures were used for the semi-automatic fusing of the surface geometry of the reconstructed buildings, see picture SAMPLE_3DREC_model-structure.jpg.There then followed the process of creating the textures.

Extracting the textures of the surface of the model was an incredibly difficult computational task which required using all the available information from the 3D reconstruction and the original photography. The process of creating surface textures was done with a special tailor-made solution. The extraction had to resolve not only distortion of the optical system but also partially concealed textures, varying depth of focus and intensity. The result was a complex texture created from a fusion of a lot of information and pictorial sources.
The final step was to map the extracted texture, which was the result of a combination of all the relevant photographs of a particular building, on to the resulting geometric structure.


Example of extracted texture
of the 3D test model

Example of extracted texture
of the 3D test model

Example of extracted texture
of the 3D test model

The result of the whole process was a complete polygonal 3D textured model.

Fully reconstructed 3D test model

Fully reconstructed 3D test model

 

 

Presentation to the public and visualisation

The result of the photography and 3D reconstruction stages of the digitalisation of the model was not only detailed documentation (a kind of “GIS” for the Langweil model) but also complete data for creating a highly accurate 3D architectural model. The data contains almost everything that it was possible through systematic optical photography to “catch sight of” in view of the restrictions imposed by the museum. Of course even despite using the most sophisticated technology there remained a number of really detailed and narrow places in the model that could not be photographed because places could not be “seen”. It is true that not even an ordinary visitor can see such a part of the model but it still provides a challenge for further possible digitalisation a few decades from now.

One of the results of the project is the creation of special 3D workplaces to examine the digitalised model, intended primarily for researchers.  Specialists (historians, surveyors, etc.) no longer have to go to the museum in order to see every detail of the model. Now they can examine it in electronic form at selected workplaces. Displaying such detailed geometric and pictorial data was also a very complex problem for current hardware. In the end the resulting workplace used the very best workstations offered by HP and the problem with display was overcome using the latest GPU and CPU. Carrying out such a complicated project would not have been possible without the use of effective brand-name workstations.

As well as the application and model for professionals several applications were also created for the public. One of them is a CD with an interactive fully 3D virtual guide around the Langweil model which allows the user to move freely around the model in the same way as in a computer game. Something else that has come out of the project is a CD with an adventure game for children which is intended for younger users and presents the digitalised model in an entertaining and interactive way.

 

Presentation to the public and visualisation

The result of the photography and 3D reconstruction stages of the digitalisation of the model was not only detailed documentation (a kind of “GIS” for the Langweil model) but also complete data for creating a highly accurate 3D architectural model. The data contains almost everything that it was possible through systematic optical photography to “catch sight of” in view of the restrictions imposed by the museum. Of course even despite using the most sophisticated technology there remained a number of really detailed and narrow places in the model that could not be photographed because places could not be “seen”. It is true that not even an ordinary visitor can see such a part of the model but it still provides a challenge for further possible digitalisation a few decades from now.

One of the results of the project is the creation of special 3D workplaces to examine the digitalised model, intended primarily for researchers.  Specialists (historians, surveyors, etc.) no longer have to go to the museum in order to see every detail of the model. Now they can examine it in electronic form at selected workplaces. Displaying such detailed geometric and pictorial data was also a very complex problem for current hardware. In the end the resulting workplace used the very best workstations offered by HP and the problem with display was overcome using the latest GPU and CPU. Carrying out such a complicated project would not have been possible without the use of effective brand-name workstations.

As well as the application and model for professionals several applications were also created for the public. One of them is a CD with an interactive fully 3D virtual guide around the Langweil model which allows the user to move freely around the model in the same way as in a computer game. Something else that has come out of the project is a CD with an adventure game for children which is intended for younger users and presents the digitalised model in an entertaining and interactive way.

 

Conclusion

The digitalisation of the Langweil model was a commendable act worthy of praise. This project was exceptional not only because of what was being digitalised but also the unique way in which it was carried out and the need to advance the field of digitalisation a step further so that this interesting piece of work could be created. It is worth mentioning that the algorithms that are “tailor-made” for the digitalisation of the model are unique and globally new to the field. Up until now nobody has faced such a complex task as the contact-free, optical 3D digitalisation of the extensive paper model of old Prague. The work can now be presented abroad thus drawing attention to this unique exhibit.

The project is also worthy of note because it was a joint effort involving the latest technology, intelligent robotics knowledge, computer vision, the historical and cultural communities, the “city” and IT professionals…

We thank the hundreds of people who have contributed to this unique project.

 

Jan Buriánek