From Photographs to Buildings: The 3D Reconstruction Project

Overview — what we’re trying to achieve

Walk around a building with a phone. Twenty-two minutes later, have a photorealistic 3D model of it that anyone can view in a browser, spin around, and walk through — no plugins, no 3D expertise, no £40,000 laser scanner.

That’s the whole idea. The project builds a complete pipeline for turning ordinary photographs into 3D models: a capture app that guides a phone round a building, a processing rig that reconstructs the mesh, a set of quality checks that tell us exactly how good the result is, and a publishing step that puts the finished model on this website with its own offline viewer.

It didn’t start there. The first attempt was as low-tech as it gets: Open Camera — a free, open-source Android camera app — set to timed-picture mode, firing a photo every few seconds while someone walked slowly round the subject and trusted the clock. That was enough to prove the concept, and enough to expose the problem: a timer takes a photo when the clock says so, not when the position is right. Everything since has grown out of that first walk.

Along the way we’re answering a practical question: how much of the expensive survey-grade workflow (the kind that used to need a specialist contractor with a terrestrial scanner, or a cloud survey subscription) can be replaced by a phone in the right hands, free open-source software, and one well-tuned workstation? The early answer is: most of it. Our first full building captured on the phone app reconstructed 2,920 of 2,921 photographs (99.97%) into a textured model — and registered a higher proportion of its images than an earlier commercial survey app’s capture of the same building did when we processed that through the identical engine.

The end goal goes one step further: an autonomous drone that maps a building by itself, decides where the missing photos are, and takes them without a pilot walking anything. More on that in the last section.

Capture — the building grows in your hands

The earliest captures were made with Open Camera, a free and open-source Android camera app, set to timed-picture mode: a photo every few seconds, phone in hand, walking the subject and trusting the clock. That produced the project’s first reconstructable datasets — and its first lesson, that a timer takes photos when the clock says so, not when the position is right. Every improvement since has been about putting that decision in the phone’s hands instead.

Today, capture is done with a purpose-built Android scanning app. The phone is held normally and walked around the subject; the app triggers a photo automatically every 10 cm of movement or 20° of turning — the same default cadence professional survey apps use — so the operator thinks about covering the building, not about pressing buttons. Position and motion sensing let the app space frames evenly even when you slow down or speed up.

The phone knows where it is and what it’s looking at. The camera works together with the phone’s motion sensors — accelerometer, gyroscope, magnetometer — through Google’s ARCore framework, which fuses them into a live estimate of position, orientation, and distance to the surfaces in view. That’s what spaces the frames; it’s also what makes the next part possible.

The part nobody has done before: the model builds itself while you walk. As the camera covers each surface, the app assembles a coarse 3D mesh of the building live on screen — walls, doorways, and rooflines rising out of the view as solid, photo-textured geometry. This is not a gimmick; it is the capture’s quality guarantee. Photogrammetry’s classic failure is silent: you walk the site, go home, wait hours for processing, and receive an archipelago — islands of mesh where photos happened to overlap, stitched around holes and ghost artifacts where they didn’t. With the live mesh, a hole on screen is a hole in the dataset — visible while you’re still standing next to it, so you photograph it before you leave.

The completeness of the dataset is checked continuously by the cheapest possible inspector: the operator’s own eyes, looking at the actual building growing in their hands.

Keeping a whole building’s mesh alive on a phone took the hardest engineering in the app. A phone cannot hold a building’s worth of 3D detail in working memory, so the app works like a sculptor who finishes one section before starting the next: sealed sections are frozen as finished display geometry and the working memory restarts fresh, so the session’s length never outruns the phone’s resources.

Under real memory pressure the preview degrades gracefully — coarser, then still — but the photo stream is never throttled by the display: every frame is written to storage, complete and final, the instant it’s taken. A crash, a flat battery, even the app being killed mid-session costs at most the last step; everything before it is already on disk and already valid.

Every photo is stamped with the essentials the reconstruction will need: the exact time it was taken, the camera identity, and GPS position when satellites are available. Sessions export as a single archive — the image set, a per-frame record of where the phone was, and a manifest describing the camera’s geometry in plain numbers, so the processing side never has to guess what the lens did. (Getting those numbers to describe the saved pixels exactly — square-pixel lens geometry, portrait frames, orientation tags that match reality — was its own quiet battle, fought frame by frame until the pipeline’s checks came back clean. Today the app writes its proven calibration into every frame as it’s taken - the two halves of the system now agree by construction, not by debugging.)

The app carries three capture modes: the full mesh view described above, a lighter AR overlay mode that paints the captured surfaces directly onto the live camera view, and a wide-lens mode for phones whose motion tracking isn’t supported — plain camera capture with the same automatic framing discipline.

Field discipline still matters and the app encourages it: one session, one lens, good light. Early morning light before the sun gets harsh is the sweet spot for buildings; shadows that move between shots are one of the few things the maths genuinely can’t forgive.

Processing — photographs to mesh

Once a capture lands, it’s booked into a small job ledger and joins a queue on the processing rig: a Lenovo P330 workstation with an RTX 4000 GPU running Meshroom — the open-source photogrammetry platform built on the AliceVision framework — completely headless, driven by our own job manager rather than by someone clicking buttons.

The reconstruction itself is the classic photogrammetry chain, and it’s worth saying plainly what happens to the photos:

1. Find features. Every photograph is scanned for distinctive corner-like details — thousands per image.
2. Match them. The same real-world detail appearing in two different photos is recognised, building a web of correspondences across the whole set.
3. Solve the cameras. From that web, the engine works out where every photo was taken from and what the camera’s lens actually does — a process called structure-from-motion. This is the stage that tells us how many photos were genuinely usable.
4. Depth and surface. Pairs and groups of photos are compared pixel-by-pixel to estimate depth, and those depth estimates are fused into a continuous surface — the mesh.
5. Texture. The sharpest views of every patch of surface are selected and blended into photographs wrapped back over the geometry.

Each stage is cached, so re-running with a change late in the chain doesn’t pay for the early stages twice. Finished models are published atomically — built in a staging area, then swung into place in one step, so a website visitor never catches a half-written model.

Two variants are built per dataset: a full-quality web model (4K textures, ~28 MB) and a lighter mobile model (2K textures) — same mesh, sized for phones on mobile data. Both ship as a single .glb file plus a self-contained offline viewer page, so the model on this site is one file you could equally keep on a USB stick and open anywhere.

Checking the result — coverage, quality, honesty

A model that looks fine can still be quietly wrong, so every dataset gets a structured health check before and after processing:

• Registration analysis. The reconstruction reports exactly how many photographs were placed in 3D. This is our headline honesty number — 99.97% on the first building — and it’s published with the model, not hidden. When photos fail to place, the report says which ones and groups the failures into patterns (isolated clusters that never bridged, images rejected at the geometry check, metadata problems).
• Lens-model verification. The hardest-won check of the project: a reconstruction can place every single photograph — 99.97% of them — and still be quietly, badly wrong, because the solver invented a slightly different camera than the one that took the pictures. One early building came back as a melted blob with every photo “successfully” placed. The fix was teaching the pipeline to seed the solver with the camera the app knows it has, then verify the reconstructed lens against that truth on every build — the difference between a model that says it worked and a model that looks like the building.
• Capture diagnostics. Before a set is even processed, the images are surveyed: camera consistency, timestamps and gaps, GPS sanity. Problem images are quarantined into a separate archive rather than silently corrupting the run.
• Mesh coverage checks. The finished mesh is inspected for holes and thin patches — the places where nobody actually took a photo, which is the honest signature of “the photographer never walked there”. That feeds back into the next capture session: cover the gaps, not the whole building again.
• Test images. Sample frames from through the set are checked for the two silent killers of photogrammetry — motion blur and exposure shifts — because a sharp-looking photo on a phone screen can still be mush to a matching algorithm.
• Orientation check. Every model is measured to confirm it stands up correctly (walls vertical, floor down) and then given a final human pass — checked, and adjusted in Blender where needed — before it goes anywhere near the website.

The numbers for the first building capture tell the whole story:

The toolchain around the engine

Meshroom does the reconstruction; everything around it is ours:

• A job manager service that owns the queue, the retry logic, and the publishing contract — including automatic recovery strategies like re-trying with gentler matching when a dataset is fragile.
• A web interface for the whole system: book datasets in, watch live progress through the twelve reconstruction stages, tune build settings per dataset, run named experiments side-by-side, and manage published outputs.
• Resumable uploads sized for real mobile networks — data moves in chunks that survive signal drops mid-field (the chunk size itself was tuned by timing tests against the actual carrier network).
• A diagnostics engine producing the reports described above, with evidence packs exportable for debugging from the field.
• Content-addressed caching and backups — every intermediate stage is reused where possible, and the entire system state is backed up nightly.

The guiding principle through all of it: the pipeline should never be the reason a good capture fails, and it should always be able to say why something failed when it does.

What’s next — the autonomous drone

Walking a building with a phone works — but it needs a person, twenty minutes, and a spare battery. The next phase takes the operator out of the loop: an autonomous drone that maps a building’s interior with on-board SLAM, plans its own photo positions from that map, and hovers to capture each frame with the same discipline the phone app enforces.

And it solves a limit no software can: a person walking at ground level cannot see over full-height interior walls — no setting, gate, or megapixel count photographs what stands taller than the sightline. A camera that flies can. Where the walked captures end at the eyeline, the drone’s flight plan begins.

The groundwork is already shared: the drone’s data will arrive in the same session format, through the same diagnostics, into the same engine. Its lidar map gives the pipeline something the phone never could — a metric scale reference and a coverage plan that knows where the missing pixels are before landing. The first models from the air will appear on this site alongside the walked ones, same viewer, same honesty numbers.

What’s actually on the drone

The airframe is a rebuilt second-hand quadcopter — four motors, a carbon frame, and landing gear tall enough for derelict floors. What makes it a surveyor is the equipment riding on it, and the division of labour between two brains:

• The reflex brain: a flight controller. A palm-sized avionics board whose only job is staying alive — keeping the aircraft level, holding position against drift, and landing on its own if the battery runs low or the radio goes quiet. It runs open-source autopilot firmware, the same lineage research drones use, and it answers to nobody on board: everything else on the aircraft can fail completely and this box will still bring the drone home.
• The thinking brain: a Raspberry Pi 4. A credit-card-sized computer tucked inside the frame’s double deck, protected by the drone’s own structure. It runs the mapping, the navigation and the camera — all the clever half.
• The eye: a spinning laser scanner. A 360° lidar that sweeps the walls thousands of times per second, out to twelve metres. This is how the drone sees indoors, where satellite positioning is a rumour. From its scans the thinking brain builds the floor plan it navigates by — and because each scan is a true distance measurement, the map comes out at real-world scale, which is exactly what the reconstruction pipeline wants for reference.
• The photographer: a 12-megapixel camera with a fixed lens. No zoom, no autofocus, deliberately: a lens that never changes means every photograph has identical geometry, the same discipline the phone app enforces. It looks slightly downward and forward, and it only fires when the drone is standing still enough for a sharp frame.
• The tape measure: a downward laser rangefinder. It knows the exact height above the floor at every moment — the number that keeps the camera at the right altitude and, later, an independent cross-check on the map’s scale.
• The voice home: two independent links. A long-range radio carries the aircraft’s vitals to the operator’s laptop; a WiFi link carries the heavy data — maps and photographs — after landing. They are deliberately separate: the radio works when the WiFi doesn’t, and the mission doesn’t depend on either.

How the parts talk is the quiet heart of the design. The lidar and camera speak directly to the Pi. The Pi speaks to the flight controller down a single cable, and it speaks politely: it never steers motors, it requests gentle velocities — “drift forward at walking pace” — which the flight controller may decline. The rangefinder talks straight to the flight controller, so height truth survives even if the thinking half is having a bad day.

And above everything sits the human: one switch on the pilot’s radio instantly demotes the entire smart half to a passenger while the aircraft stops and holds still. Autonomy proposes, the reflex brain disposes, and the person in the doorway vetoes.

How the dataset gets built

The mission runs in two phases, because mapping and photographing want different things from a battery.

The mapping flight sends the drone exploring: it picks the nearest unknown frontier of its floor plan, flies there, scans, and repeats — growing the map wall by wall until the building is drawn or the battery says otherwise. Then it comes home and saves the map.

The ten minutes of human judgement happens on the ground: the operator looks at the finished floor plan on the laptop and draws two things — the boundary of what to capture, and any keep-outs (a hole in the floor, a shaft of sunlight through the roof that would poison the camera’s exposure). This is the only creative decision left, and it’s the one a human should own.

The capture flights then work the plan: a ladder of hover-and-shoot stations covering every wall, with slow sweeps between them. Every photograph is stamped with the exact position and attitude of the drone at the instant of the shutter — a per-frame record equivalent to the phone app’s, gathered from the live map instead of a phone’s motion sensors. When the battery runs low the drone lands, gets a fresh pack, reloads its map from exactly where it left off, and carries on. After the final landing, the whole session — images, pose records, height log, map, manifest — is handed over in precisely the format the walked captures use, and joins the same queue.

The missing pixels, solved twice

The phone app’s answer to photogrammetry’s silent holes is the live mesh — the operator sees a gap and photographs it before leaving. The drone’s answer is structural, and it runs both directions in time. Before capture: the lidar floor plan is the coverage plan — every metre of wall gets stations assigned to it, so “the photographer never walked there” becomes geometrically impossible rather than merely unlikely; the planner can name the uncovered metres before the first photo is taken. After processing: the pipeline’s registration report — the same honesty numbers this site publishes — names any regions that came back sparse, and the next mission visits only those gaps, not the whole building again. Coverage is planned on the map and audited by the engine.

Where this stands today

The hardware is built and benched: airframe, both brains, the laser eye, the camera, the height sensor — everything on the list except one power converter still in the post. The full autonomy stack has been compiled and runs on the actual flight computer; the ground-control laptop flies the aircraft every evening in simulation, rehearsing the same emergency drills a pilot would. Next milestones, in order: carry the flight computer and its laser eye around a room by hand and watch the map draw itself; then the first autonomous hover; then the first room captured end-to-end — and the first model on this site that nobody walked.

The deckchair

Where it ends up: an operator draws a boundary on a laptop, then sits in a deckchair by the door with a tea, watching the screen — the floor plan filling in, the photograph count climbing, the battery countdown ticking — while the drone builds the dataset on its own. The deckchair is aspirational. The hand resting on the override switch is mandatory, and it always will be: the whole system is engineered so that the dullest possible failure leaves an aircraft that stops, holds still, and waits to be told what to do.

Built with open-source software (Meshroom / AliceVision), one workstation, and a stubborn refusal to accept “good enough” without measuring it.