Why use photogrammetry to build 3D models of indoor spaces?

[ June 28th, 2019] Discover the different scenarios where drone users envision using indoor photogrammetry to improve their business efficiency.

If you read this blog post, you are probably familiar with drone-based photogrammetry and the numerous use cases for outdoor mapping. This is certainly less true if we speak about indoor environments. With the recent release of Elios 2, which enables drone-based photogrammetry indoor and in inaccessible places, we wanted to explore the reasons to use photogrammetry for indoor 3D modeling. This is the subject of this blog post.

Elios 2 in Action 0023 - In the dark entrails of the earth

Why photogrammetry indoors?

There are certainly many reasons to use photogrammetry indoors. Here are recurrent themes that we came across while interviewing our customer about their asset management and surveying activities.

Organizing your data

Considering the complete workflow for drone-based visual inspection, data acquisition is only a first step. In a second step, the data must be analyzed to extract useful information. With Elios 1 and Elios 2, the pilot can record points of interest (POIs) during the flight and easily find them in the video stream using Inspector, Flyability’s desktop software. However, the pilot may have overlooked some defects or objects of interest, and it is therefore common to carefully review the videos after a series of flights. Also, it is often crucial to understand where the data was acquired, and localize the POIs on blueprints or CAD models. Usually, this is achieved by watching the video again and building a mental map of the asset (keep in mind that indoor datasets do not contain geolocalization information). To determine the position of POIs, it is sometimes required to count repeating elements such as beams, welds, intersections, or other obvious visual features. This method works, but is cumbersome and not really scaleable.

In some situations, 3D modeling can drastically simplify this tedious and time-consuming task. With a 3D model and the trajectory of the drone, it is much easier to navigate in the dataset, to review only the relevant images and to understand the location of the POIs. The risk of errors - such as confusing repeating elements - decreases and the overall understanding of the asset geometry becomes clearer.

Photogrammetry image 4

Making Measurements

Visual information is often the starting point of all type of inspection, but being able to put a number on visual features is often crucial to help the decision-making process. By focusing data capture efforts on a small part of an asset, it is possible to build a local 3D model that offers great details and accuracy. For example, Elios 2 takes images with a Ground Sampling Distance (GSD) down to 0.18 mm/px (0.007 in/px) when capturing data at a 30 cm (11.8 in) distance. This resolution offers great perspectives for photogrammetry software to build very precise 3D models.

As Alex Buczkowski reminds us in his blog post “How accurate is your drone survey?”, it is very important to include objects with known dimensions in the model. This allows to scale the model accurately, even without ground control points. It is then possible to measure objects or defects on the original images, and to visualize them in the 3D model.

Photogrammetry image 3

Checking the shape and condition of structures - The case for mining

This application is very specific and it deserves its place here as it will certainly inspire other industries with similar problems. Underground mining consists mainly in excavating large volumes of ore from the entrails of the earth to then extract various minerals through chemical refining processes, for instance. The excavation leaves large openings called stopes. These stopes are extremely dangerous areas, where no humans are allowed.

After blasting the ore body, the ore is extracted from the bottom, in a similar fashion to a cereal dispenser. Geotechnical engineers are in charge of ensuring the underground workings - such as stopes, galleries, shafts, and raises - are stable and they don’t fall down on the miners. To complete their tasks, geotechnical engineers need to understand the shape (3D model) and condition (visual information) of these workings. Photogrammetry successfully combines 3D model reconstruction with visual information and solves many of their problems. 3D models of stopes give access to a whole new level of information about these areas where the risk of being smashed by rocks the size of a car is not negligible.

stope-480-rotating

Download an Underground Mining Dataset

Tracking changes

The classic example of asset management is of course tracking how the conditions of an asset, and more particularly defects, evolve over time. This allows the planning of preventive maintenance and repairs before any defect becomes an actual threat to the asset operation. This is something that doesn’t necessarily require to make a full new digital twin of the asset in outdoor use cases, since GPS and orientation information can be sufficient to know where to retake a picture. For the indoor use case, or more generally GPS-denied environments, making a 3D model is the only option as the drone doesn’t have localization information while it’s flying.

Planning maintenance and design changes

Digital twins are not only useful to track defects over time and decide when it’s time to tackle them. They are also helpful in the planning phase of maintenance and design changes. The digital twin allows to plan where to put scaffolding, and how the area should be accessed. When a large new piece of equipment needs to be moved through a plant, a traversability analysis can be performed to find the optimal route. For design changes as well, it is indispensable to have an up-to-date 3D model to design and visualize the new changes. 

Photogrammetry image 1

Training and safety

Complex and dangerous assets often require extensive training of new personnel on the layout of the factory, before they can take part in operations. A classic example here is nuclear industry. To plan escape routes in case of emergency situations they need up to date floor plans in all parts of the asset. Here again a digital twin of the asset proves to be helpful. In such application the photogrammetry model is often annotated with loads of additional information about the asset.

 

Where does it work? What can I map?

Let’s dive in with a small refresher on the conditions that we face when capturing a photogrammetry dataset indoors.

The case of indoor: what about GPS, light, and cluttered spaces?

First, the absence of GPS signal means that the images cannot be geolocated - the reconstruction relies only on the visual content of the images. Second, the environments are often pitch dark. Artificial lighting is needed, but most often it cannot be easily installed in the area of interest. Therefore we rely solely on the on-board lighting system of the drone. The last important difference with outdoor photogrammetry is the spatial constraint: indoors, the spaces are often narrow and cluttered. This puts a constraint on the possible trajectories to cover the area of interest.

Being built for the indoor, Elios 2 can address these challenges. Thanks to its unobstructed view and powerful onboard lighting system, it captures high-quality images that are suited for photogrammetry even in the darkest environments. Recording 4K videos, it collects enough data to allow for 3D reconstruction even without a GPS Finally, the protective cage of Elios 2 allows it to fly in the most cluttered spaces without risking to crash. This is of tremendous importance when you try to map a narrow area. With Elios 2, you can concentrate on the data you capture, without fearing to collide onto objects, even during backwards or sidewards motions.

What are the easy places to map? Where is it more challenging?

There are two key aspects that determine the difficulty to map an asset: the image content and the flight trajectory.

Photogrammetry works in environments that have a sufficient quantity of visual features. A visual feature is any point that is distinctive enough to be recognized by a photogrammetry algorithm. Note that visual features depend on the scale. Some surfaces appear homogeneous from a given distance, but many features become visible as you come closer.

Favorable Environments

Challenging Environments

Natural rocks

Concrete wall with high uniformity

Brick walls

Homogenous and/or shiny painting

Old painting, cracks, corrosion

Shiny metallic surface, stainless steel

Dirty surfaces, algae

Glass, windows

 

The geometry of the asset is also an important factor for the success of a 3D reconstruction. While it is rather easy to scan a flat wall in a regular pattern, finding an appropriate trajectory becomes much more difficult in assets with complex geometries. Here are a few points to consider:

Favorable Asset Geometries

Challenging Asset Geometries

Large assets that allow flying separate lines to cover the same areas from different angles.

Linear assets such as ducts, sewers or narrow chimneys with less than 2m diameter. There, the images are essentially gathered along a single line. As a consequence, objects are observed from a reduced number of angles, providing a weaker geometry for the reconstruction.

Assets comprising a single volume (such as cylindrical or spherical tanks, large chimneys, boilers, etc.) where all parts of the assets can be more or less seen from everywhere in the asset.

Assets split into several volumes, connected by manholes, corridors, or narrow passages. It is very difficult to keep a proper connection between the images taken on both sides of a manhole, a door, or at the opening of a corridor into a larger room.

Assets containing long linear features (such as welds, pipes, etc) that can guide the pilot and help him follow the desired flight line.

Assets that lack visual features to guide the pilot.

 

A glimpse into the future

With the emergence of new technologies, the future of asset management looks quite bright! “Building 3D models of assets is a first step,” says Péter Kövessi, General Manager at FLIND a SUEZ solution. Based out of Barcelona, Péter and his team provide drone-powered industrial and maintenance inspections with a focus on sewer management. “The greatest value for my customer lies in the ability to map their assets and create as-built plans. Blueprint are often inexact or even missing.”

But Péter dreams even further: “Running AI-powered classification algorithms on 3D dataset to identify, characterize, and localize elements in the model is, ultimately, what makes the most sense in the industry we serve.” 

Are we there yet? Péter has his opinion: “When GIS software first came on the market, it was probably 10 years ahead of the game compared to the means available to produce the required data. This paradigm is also true for asset management software. You can manage pieces of equipment from large pipeworks down to every single nut and bolt. However, the technology available today does not allow you to feed asset management software with this information automatically. At least, not indoors.”

Serving the dreams of our customers and especially those trying to create a safer and more efficient workplace drives our research and development. We are working day in, day out to materialize the dreams of people like Péter. To get there, we need to go step by step and we are keen to learn along the way with our customers and the innovators in this world. If you are interested in photogrammetry and wish to start mapping the great indoors, we are happy to support you in your venture. Reach out to us and we’ll start a discussion.

 

Fly Elios 2

 

This article belongs to a serie about photogrammetry: 

Why use photogrammetry to build 3D models of indoor spaces? 

Mining drones: Elios 2 creates photogrammetric models.

Building 3D models with Elios 2: how to acquire appropriate data for photogrammetry

SEE ALSO

Photogrammetry

Learn how to capture proper dataset to create photogrammetric 3D models with Elios 2. This blog post will guide you through the key points to be […]

Drones Power Generation

Nuclear power plants can be hazardous places to work. There are often situations where workers are called to enter high-dose areas to ensure the safety of […]

Podcast Steel Industry

Elios is having a significant impact on the way Chris and his team are performing visual inspections at AM/NS Calvert, one of the most advanced steel […]

Article

The question of whether drones are intrinsically safe is one of the first that comes to the mind of Oil & Gas and Chemical professionals. So far the […]

Oil & Gas Podcast

Through this podcast, Richard Schutte, Chief Pilot at Viper Drones, shares his experience of drone operations in the Oil & Gas industry and covers the […]

Article

When asset owners try to figure out how to proceed with the integration of drones in their processes and workflows, a critical question comes in; should […]

Chemicals Podcast

Listen to the story of Lieve Van Gijsel who started a drone inspection business and moved to indoor inspection to workaround the difficult Belgium […]

Article

We have compiled for you the 5 most impactful indoor drone stories gathered, throughout 2017, from professionals who are using Elios in the field. Read […]

Mining Podcast

Eric Romersa explains his activity in Mining where he uses different drones to cover the needs of the industry. With his recent acquisition of an Elios he […]

ROI Podcast

Junio Palomba shares with us the experience of his customers about Return On Investment (ROI) with Elios and delivers some tips and tricks for those who […]

Article

The need to inspect confined spaces is present in virtually all industries. Patrick Thévoz explores the modern ways to perform confined space inspection […]