Jaysen Marais » photoshop

Spherical panoramas: photographs without edges

jaysen — Fri, 10 Apr 2009 19:52:00 +0000

Most lay-people see photography as the cut-and-dried act of capturing the world “as it is”. Ironically, the simple and necessary action of framing a photograph instantly renders the result subjective (‘un-photogenic’ subjects are typically pushed out-of-frame). This creative choice built into the camera’s design is part of the appeal of photography. However, we occasionally encounter entirely photogenic scenes that we wish to convey in their entirety. Luckily, there is a type of photograph that has no edges at all; the spherical panorama.

Moreton island sand-blow (summit)29th Dec 08 map HD (2.6MB)

This spherical panorama captures the complete scene; from the zenith (above my head) to the complete horizon to the the nadir (under my feet). No photographic paraphernalia (camera, tripod, photographer) distracts the viewer from the scene. What kind of magical camera does this? Well, with the right know-how, software and technique any camera will do. This is because spherical panoramas are simply an amalgamation of many regular photographs. Here’s the kit I used to make my example panorama.

Tripod fitted with a Nodal Ninja 5 panoramic tripod head supporting a Canon 5D mark II with Canon 16 – 35mm f2.8L lens

To get started we need a sequence of regular photographs which cover the entire scene. Typically this involves taking several ‘rows’ of images, each containing a constant number of images (columns). The number of rows and columns required depends on the lens used but nevertheless the angle between rows should be consistent, as should that between each column (made easy by using a panoramic tripod head, e.g. Nodal Ninja 5). In addition, a zenith shot (directly up) and two (complimentary) ‘nadir’ shots (directly down) are required. Here’s the sequence of images I used to produce my sample panorama.

zenithsky rowhorizon rowground rownadir (taking two at opposites enables later removal of feet & shadows)

The rest of the spherical panorama creation process is all in software. I plan to cover the details in a future screen-cast, but for now here’s a quick overview:

Ensure ‘consistent colour’ across shots by correcting white-balance discrepancies (tip: remember to lock white-balance, exposure, shutter-speed, focal length and focus while shooting). Also correct any vignetting (dark edges) and chromatic aberration (coloured fringing on detail). For these tasks I use Canon’s bundled DPP software (if Lightroom‘s vignetting correction tools were better, I’d use it instead).
Import the corrected images into a stitching tool (after trying a few apps I finally ponied up for a copy of PTGui). By creating control-point pairs (i.e. locating similar features in different photographs) I give PTGui the necessary information to correctly align, warp, composite and blend the all images (except for the zenith and nadir) into a single ‘stitched’ equirectilinear result.
In photoshop I work a little magic to combine the two original nadir shots (which both contain my legs and feet) into a single feet-free nadir image
Using PTGui again I produce another equirectilinear image, this time containing only the warped ground (from the composited nadir) and sky (from the zenith shot)
Back in photoshop I combine both equirectilinear images into a single image (‘patching’) and apply any necessary retouching (e.g. making horizons dead-level and apply colour-corrections) to produce the pièce de résistance: the final (70+ megapixel) image.

patching the warped nadir and zenith onto the stitched panorama in photoshop

At first glance this appears to be a regular photo, but in fact it covers 360° horizontally and 180° vertically (i.e. the entire scene). So how does it justify the ‘spherical’ and ‘edge-less’ tags? Well, the most effective way to present such an image is to wrap the rectangular image around an imaginary sphere and give the viewer control of a virtual camera placed at the orb’s centre. Rotating the camera simulates ‘looking’ in the original scene. There are no limits on where the camera may look and no visible edges or seams.

the distortion in the equirectilinear image disappea
rs when the image is wrapped around a sphere (inside which the viewer’s virtual ‘camera’ rotates)

Spherical panoramas really take on a new dimension when combined with software like the PangeaVR iPhone app I’ve mentioned before. Wherever I am I can show friends exactly what my favourite places are like by handing them the phone and saying “drag the image with your finger”. No other explanation, instruction or intervention is required for me to show friends overseas what the beach I learned to surf at is “really like”. Of course, the panoramas can also be viewed in a web browser. Below are links to a few panoramas I’ve created in the past few months from images taken on my Christmas holiday back home.

Hopefully I’ll have more to add to my panoramas page soon.

Making nodal panoramas that don’t suck

jaysen — Tue, 01 Apr 2008 22:57:00 +0000

Update: 6th Feb 2010, changed ‘linear’ in title to ‘nodal’ due to comment feedback

Most modern digital cameras (even phones) have a ‘panorama photo’ mode, allowing you to take multiple photos which are then magically combined into a single large photograph. Sounds great, but there’s a problem. These composite images (whether produced in-camera or using ‘automatic stitching’ programs) look OK at first glance, but are typically full of smudges, ghosts, morphing people and objects sprouting from thin air.

So where exactly is is all going wrong? In theory, the basic steps required to create convincing composite images (including panoramas) are:

Align the images — Ordering and placing images by examining their overlaps.

Correct for angle of view — Think ‘fisheye’, but subtler.

Remap images to compensate for varying distance — Closer subjects ‘move’ more between shots than distant ones. This is fixed by identifying common ‘control points’ then remapping the images so they overlap perfectly.

Blend images to hide image seams — Obscuring (typically softening) the boundaries between images to create the illusion of a single image.

Photostitching software typically gets steps 1 and 2 spot on, but can’t hold a candle to humans when it comes to pattern recognition (the secret sauce behind steps 3 and 4). If we want good composite images we have to hijack the identification of control points and eventually produce a layered photoshop file so we can blend manually (to perfection).

Pitfalls during shooting

First we need to keep an eye out for things that often go pear-shaped during the shoot.

Creeping Horizon — Caused by gradually rotating the camera up/down between shots (when doing a horizontal pano).

Wavy horizons — If the camera’s not perfectly level, the horizon will see-saw up & down.

Moving foreground subjects — People/cars/trees/etc moving between shots.

Moving photographer — Wind, terrain and rampaging wild-life may interrupt your plans.

Rubbish/damaged lenses — Cheap cameras have cheap lenses whose variations become glaringly obvious during stitching.

Varying ‘brightness’ — Not locking exposure results in adjacent shots with lighter or darker skies.

Varying ‘colour’ — Not locking white-balance (when multiple light sources present) can result in dramatically different colour between shots.

Changing light — If clouds arrive halfway through shooting your panorama, give up or start again.

Tips for shooting

Use a tripod

Level your tripod (camera’s rotation axis should typically be perpendicular to the horizon).

Be aware of moving objects near the camera (remember to check 360°). Fast moving objects are generally OK. It’s the slow moving object that cause headaches later on.

Take more shots than the camera recommends (only if using manual stitching). Using 2/3 overlap will give you a spare photoshop layer during blending.

Be quick. Light and subjects will change during your shoot. Plan your panorama, do a dry run, then fly like the wind (but don’t forget the other tips).

Lock any camera settings you can, leave the rest alone. Some cameras lock exposure and white balance in ‘panorama’ mode (check the manual), but it’s up to you to keep focal length (zoom), aperture, focus and ISO constant.

Playing with pixels — stitching an ‘unstitchable’ panorama

All the apps mentioned in this section are available for Windows and OS X (linux too, though using gimp instead of photoshop). Also it’s worth pointing out that even though this post is about horizontal panoramas, these techniques can be be used to create any composite image (blended tiled images, unblended tiled images, spherical panoramas, gigapixel images etc).

Shoot the images & collect on PC — I’m going to use images taken atop Gaisberg mountain near Salzburg, Austria. The images were shot on my Canon Powershot G7 in it’s Stitch Assist mode (exposure & white balance locked). To make things interesting I’ve chosen a set of images with a few of the common issues you may come across (no tripod, inconsistent position).

Download and extract Hugin — Hugin is an ambitious open source (donation-ware) panorama creation GUI (cross-platform, written in c# under mono). It’s still beta so you may find the UI a little cryptic and glitchy, but you can’t complain given the price-tag (free).

Download and install autopano-sift (optional) — Autopano-sift is a helper program for Hugin which identifies control points in your sub-panorama images using some pretty hefty maths. We will use autopano-sift to kick-start the control point detection process, but you can bypass autopano-sift and create all control points manually if you wish.

Open Hugin and load images — When Hugin loads click the ‘load images’ button and locate the images which make up your panorama (some camera naming schemes make this difficult so it’s best to collect them in a folder beforehand). Hugin may ask you to specify your cameras stated focal length (usually silk-screened around the lens itself) and 35mm equivalent (try dpreview or the manual).

Once Hugin has loaded your images, autopano-sift will appear (on first-run, Hugin may ask you to locate the autopano.exe).

Use autopano-sift to identify control points (optional) — The autopano-sift UI has a few tweak-able parameters for the adventurous but for now just click ‘compute’.

Autopano-sift’s ‘PTO generation’ process will kick off. This process may take a few minutes (depending on the size & number of your images). Once completed, click ‘OK’ and close autopano-sift.

The rough draft — Hugin will use the control points generated by autopano-sift to align and distort your images. It will present its results in the ‘preview’ window.

As you can see, there are some misaligments. These are to be expected (especially with dodgy source images like the ones I’m using) but unlike the automatic stitching software which would just blend over these cracks, we have the power to fix them.

Get to know the ‘control points’ tab — Close the preview window and navigate to the ‘control points’ tab.

The ‘control points’ tab enables you to review, edit, delete and add control points to improve the stitching & blending processes. Additionally, it gives a unique perspective on exactly why automated control point detection is so underwhelming. Main points:

Existing control-points on both images (automatically generated by autopano-sift).

We’re only really interested in the area of overlap. Ideally there would be control points evenly distributed througout this entire overlap region.

The list of control points (automatic and manual).

Manually tweaking control points — The ‘control points’ tab in the current build of Hugin (0.7 beta 4) has some frustrating UI glitches, but the following workarounds helped me:

When editing existing control points, set the zoom factor to ’200%’ (be sure to click them dead centre or Hugin will try to create a new control point, frustrating).

When adding new control points, set the zoom factor to ‘fit to window’ then click on both images in turn, position the control points, click the ‘fine-tune’ button and then the ‘add’ button. repeat.

Add a few control points for each troublesome image pair (evenly throughout the overlap area) then go to the ‘optimize’ tab, choose ‘everything’ and click the ‘optimize’ button. Once the optimization process is complete, check the preview (ctrl-p on windows, cmd-P on mac) to see whether it’s made an improvement to the panorama. If there are still major breaks, tweak the control points as necessary.

Here you can see the panorama preview is less ‘wavy’. Don’t worry too much about the rotation, we’ll fix this in photoshop.

Get PTStitcher — Although Hugin comes with a stitching engine (nona), we’ll use PTStitcher as it has better blending plus .psd output. Download the PanoTools zip and extract it. Copy the pano12.dll from the extracted folder into your Hugin folder. Copy the PTStitcher.exe app (from the Helpers folder in the extracted PanoTools folder) to the Hugin folder.

Creating a stitched (layered) psd file — In Hugin, go to the ‘Stitcher’ tab, change the ‘stitch the images’ option to ‘into a layered photoshop file’ and click ‘stitch now’ (beware the psd file may be several hundred megabytes, hope you’ve got the RAM). Hugin may ask you to locate PTStitcher. If so point it to the copy you put in the Hugin folder.

Final blending & cropping in photoshop — Upon opening the psd file in photoshop you’ll notice that the blend lines are pretty conspicuous. However, as each shot is a separate (masked) layer you can tweak each blend independently by editing the mask.

At this point you also have the opportunity to mask in/out moving foreground subjects and make other creative edits (i.e. duplicate people). If you took my advice and shot with 2/3 overlap you’ll have a spare layer to run wild with too. A few blending tips:

Use broad diameter ‘soft’ brushes to feather the masks in areas of sky.

Avoid straight lines in your masks (especially horizontal & vertical lines). Human eyes will spot those straight away.

In areas of high detail use small diameter ‘harder’ edged brushes (this avoid ghosting in the final image).

Hard edged subjects in the foreground make great (hard) mask boundaries

After tweaking each of the seams the whole panorama becomes pretty convincing. This is the most rewarding part of the process, and you run the risk of wasting a lot of time . In this example I decided to merge foreground characters from two different shots, but I left some easter eggs in there for anyone who’s really looking.

The final cut — To get rid of the ‘rising horizon’ problem in this image set (a constant issue when shooting panoramas sans tripod) I’ve selectively cloned and distorted the middle foreground grass and left sky. The final image captures all the things I remember from that instant in one single image (the ultimate aide mémoire).

The length of this post might make the process of manually assembling composite/panoramic images appear complex, but trust me, the second time’s a snap. Oh, and you’ll be the only kid on your street who’s panoramas don’t suck.