Deploy a rails app to bluehost without SSH

Published on April 10, 2010 in web development. 1 Comment Tags: , , , , , , , , .

rails on bluehost

I’ve been doing a little free-lance web development again lately. It’s been a while and the landscape’s changed in the 2 or 3 years since I was last freelancing. Specifically the ‘minimum requirements’ for a public facing site and the quality of the ‘cheap’ generic hosts have both increased (for the better). In light of these two factors I’ve been having a go at making some basic ‘web-presence’ sites using ruby on rails.

Learning enough rails to make a simple site was easy enough, but the deployment turned out to be a bit of a puzzle. Particularly as bluehost, whose control panel is otherwise excellent, just doesn’t seem to have its rails story sorted. To make matters worse, every “rails bluehost” tutorial out there starts with “you need SSH access” and bluehost now requires a faxed or emailed copy of the account-holder’s drivers license to grant SSH access. Seeing as my clients are generally the hosting account holders, this wasn’t an option for me.

After a bit of trial and error I’ve discovered that you can indeed deploy a basic Ruby on Rails site to bluehost without SSH access (assuming you have ftp access). Here’s how:

  1. Create rails directory.Use your ftp client to create a directory alongside the {app} directory to which the domain/sub-domain points.I’ve called mine {app}_rails, but it can be called anything.

  2. Upload rails app.Upload your rails app into the newly created {app}_rails directory.

  3. Symlink to rails public directory using temporary PHP.We need to symlink the {app} directory (to which the domain/sub-domain points) to the {app}_rails/public directory. Ordinarily we’d use the trusty ln -s command, but we can also do it without SSH access by using PHP’s symlink function.

    Create a php script on your local machine (I’ve called mine symlink.php) with the following contents (replacing the {account} and {app} tokens with your own). Make sure the {app} directory doesn’t already exist (rename/move/delete it if it does).

    <?php symlink('/home/atfraser/public_html/maketimelapse_rails/public', '/home/atfraser/public_html/maketimelapse'); ?>

    Upload the symlink.php to an accessable directory and then browse to it in your browser. If the script succeeds you’ll be greeted with a blank page (otherwise you’ll get errors). Once run successfully, you can delete the symlink php script (it’s no longer needed).

  4. Ensure dispatch.fcgi exists.To run under apache, a rails app needs fcgi dispatchers. My understanding is that rails 2.3 no longer fcgi dispatchers by default so if you didn’t use the --with-dispatches option when you created your app (I didn’t) you’ll need to create them on your local machine using the following command:

    rake rails:update:generate_dispatchers

    Then upload the created dispatch.fcgi, dispatch.cgi and dispatch.rb files from your localpublic directory to {app}_rails/public directory on bluehost.

  5. Enable dispatch.fcgi in .htaccess.Create a .htaccess file inside your {app}_rails/public directory with the following contents:

    AddHandler fcgid-script .fcgi
    AddHandler cgi-script .cgi

    RewriteEngine On

    RewriteBase /
    RewriteRule ^$ index.html [QSA]
    RewriteRule ^([^.]+)$ $1.html [QSA]
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^(.*)$ dispatch.fcgi [QSA,L]

    ErrorDocument 500 “<h2>Application error</h2>Rails application failed to start properly“

  6. Check permissions using FTP client.We need to check/set some permissions. Again, we can’t use the standard chmod command without SSH access but luckily most modern ftp clients can also view/set permissions (select the file/directory then ⌘-i in Transmit or ‘right-click > File permissions’ in FileZilla). Set the appopriate permissions for the following files/directories (list courtesy Jack Danger Canty):

    File/Directory Permission
    {app}_rails/public 755
    {app}_rails/public/.htaccess 755
    {app}_rails/public/dispatch.fcgi 755
    {app}_rails/public/dispatch.cgi 755
    {app}_rails/log 775
    {app}_rails/tmp 775

  7. Handling rails gems version issues.If you're app still isn't working, check the error logs in cpanel. It may be that the version of the rails version you built against isn't available on the shared host. In this case,you'll need to either freeze the gems into your app locally (then upload the vendor/gems directory) or comment out the following line in environment.rb which refers to their specific version (risky).

    rake rails:freeze:gems

That should be it!

Synchronize bookmarks with Delicious & Firefox

Published on May 2, 2009 in web development. 0 Comments Tags: , , , , , .

The web is awesome, but managing URLs is a pain. That pain is magnified if you use multiple machines and made worse if you use multiple operating systems (not to mention home vs. work, desktop vs. phone etc). There are roughly a million ways to handle this problem. Here’s mine.

synchronizing bookamrks between multiple computers and OSes using delicious

Step 1. Move bookmarks to delicious.com

About 2 years ago I moved all my bookmarks to delicious.com, a ‘social bookmarking’ service which stores your bookmarks and enables you to access, share and organize (via tagging) these bookmarks over the web. Over time it’s become second nature to save anything interesting I find to my delicious account.

delicious.com screenshot

Step 2. Install delicious.com browser extension

Whilst delicious (and its ilk) can be used via the web interface, things really get interesting when you install the relevant browser extensions. These give you an experience much like regular browser-based bookmarking, but with synchronization and superior bookmark search.

Step 3. Specify “everyday” bookmarks

specifying favourite tagsWe all have bookmarks we use everyday. Delicious allows me to group mine together by applying a common tag (I use the tag “me”). I then tell the Delicious Bookmarks extension for firefox that the “me” tag denotes my favourites and viola! All my everyday bookmarks appear in Firefox’s toolbar. Better still, any additions or edits I make to this list of favourite bookmarks are propagated across all my browsers and computers.

Step 4. Bring on the favicons!

Although the Delicious extension for Firefox is great, it pays too much head to Firefox conventions for my liking. Specifically it doesn’t display favicons on mac (firefox doesn’t do toolbar favicons on mac by default) and doesn’t offer the ability to hide toolbar labels (which take up space). Luckily Firefox is highly customizable so I’ve managed to put together a simple Firefox extension which overrides these design decisions (and fix an intermittent “huge favicon” issue on mac Firefox). Behold: my Delicious Bookmarks Toolbar extension.

the 'Delicious Bookmarks Toolbar' extension replaces the Delicious Bookmark labels with favicons (mac & pc)

By the way, if you thought this blog post was written simply as an excuse to link to that simple extension, you’d be right.

Spherical panoramas: photographs without edges

Published on April 10, 2009 in panorama and photography. 3 Comments Tags: , , , , .

Most lay-people see photography as the cut-and-dried act of capturing the world “as it is”. Ironically, the simple and necessary action of framing a photograph instantly renders the result subjective (‘un-photogenic’ subjects are typically pushed out-of-frame). This creative choice built into the camera’s design is part of the appeal of photography. However, we occasionally encounter entirely photogenic scenes that we wish to convey in their entirety. Luckily, there is a type of photograph that has no edges at all; the spherical panorama.

Moreton island sand-blow (summit)29th Dec 08 map HD (2.6MB)

This spherical panorama captures the complete scene; from the zenith (above my head) to the complete horizon to the the nadir (under my feet). No photographic paraphernalia (camera, tripod, photographer) distracts the viewer from the scene. What kind of magical camera does this? Well, with the right know-how, software and technique any camera will do. This is because spherical panoramas are simply an amalgamation of many regular photographs. Here’s the kit I used to make my example panorama.

tripod, nodal ninja panoramic head and Canon 5D mark II
Tripod fitted with a Nodal Ninja 5 panoramic tripod head supporting a Canon 5D mark II with Canon 16 – 35mm f2.8L lens

To get started we need a sequence of regular photographs which cover the entire scene. Typically this involves taking several ‘rows’ of images, each containing a constant number of images (columns). The number of rows and columns required depends on the lens used but nevertheless the angle between rows should be consistent, as should that between each column (made easy by using a panoramic tripod head, e.g. Nodal Ninja 5). In addition, a zenith shot (directly up) and two (complimentary) ‘nadir’ shots (directly down) are required. Here’s the sequence of images I used to produce my sample panorama.

zenith rowzenithsky rowsky rowzenith rowhorizon rowzenith rowground rowzenith rownadir (taking two at opposites enables later removal of feet & shadows)

The rest of the spherical panorama creation process is all in software. I plan to cover the details in a future screen-cast, but for now here’s a quick overview:

  1. Canon Digitial Photo Professional (needs a better logo)Ensure ‘consistent colour’ across shots by correcting white-balance discrepancies (tip: remember to lock white-balance, exposure, shutter-speed, focal length and focus while shooting). Also correct any vignetting (dark edges) and chromatic aberration (coloured fringing on detail). For these tasks I use Canon’s bundled DPP software (if Lightroom‘s vignetting correction tools were better, I’d use it instead).
  2. PTGuiImport the corrected images into a stitching tool (after trying a few apps I finally ponied up for a copy of PTGui). By creating control-point pairs (i.e. locating similar features in different photographs) I give PTGui the necessary information to correctly align, warp, composite and blend the all images (except for the zenith and nadir) into a single ‘stitched’ equirectilinear result.
  3. In photoshop I work a little magic to combine the two original nadir shots (which both contain my legs and feet) into a single feet-free nadir image
  4. Using PTGui again I produce another equirectilinear image, this time containing only the warped ground (from the composited nadir) and sky (from the zenith shot)
  5. Photoshop logoBack in photoshop I combine both equirectilinear images into a single image (‘patching’) and apply any necessary retouching (e.g. making horizons dead-level and apply colour-corrections) to produce the pièce de résistance: the final (70+ megapixel) image.

patching the warped nadir and zenith onto the stitched panorama in photoshop
patching the warped nadir and zenith onto the stitched panorama in photoshop

At first glance this appears to be a regular photo, but in fact it covers 360° horizontally and 180° vertically (i.e. the entire scene). So how does it justify the ‘spherical’ and ‘edge-less’ tags? Well, the most effective way to present such an image is to wrap the rectangular image around an imaginary sphere and give the viewer control of a virtual camera placed at the orb’s centre. Rotating the camera simulates ‘looking’ in the original scene. There are no limits on where the camera may look and no visible edges or seams.

the distortion in the equirectilinear image disappears when the image is wrapped around a sphere
the distortion in the equirectilinear image disappea
rs when the image is wrapped around a sphere (inside which the viewer’s virtual ‘camera’ rotates)

Spherical panoramas really take on a new dimension when combined with software like the PangeaVR iPhone app I’ve mentioned before. Wherever I am I can show friends exactly what my favourite places are like by handing them the phone and saying “drag the image with your finger”. No other explanation, instruction or intervention is required for me to show friends overseas what the beach I learned to surf at is “really like”. Of course, the panoramas can also be viewed in a web browser. Below are links to a few panoramas I’ve created in the past few months from images taken on my Christmas holiday back home.

view 'moreton island sand blow (summit)' panorama view 'moreton island sand blow (saddle)' panorama view 'Moreton island sand blow (crest, north)' panorama
view 'Moreton island sand blow (crest, middle)' panorama view 'Moreton island sand blow (crest, south) view 'Moreton island eastern beach (dune)' panorama
view 'Moreton island eastern beach (shore)' panorama view 'Palm beach avenue beach (1)' panorama view 'Palm beach avenue beach (2)' panorama

Hopefully I’ll have more to add to my panoramas page soon.

Ragged CSS: makes life easier

Published on January 26, 2009 in web development. 2 Comments Tags: , , , , , .

The web community is generally coming (or has come) to accept that ‘semantic’ class naming conventions (‘old school’ elements with classes describing content not appearance) are preferable to the 'boldGreyLink' type conventions popular around the millennium.

Millenium era HTML & CSS ‘Semantic’ era HTML & CSS 
<html>
<head>
<title>'Millenium' era</title>
<style type="text/css">
div.redSubHead {
  color: red;
}
p.greyText {
  color: #666;
}
td.columnHeaderCell {
  font-weight: bold;
}
td.oddRowCell {
  background: #eee;
}
td.evenRowCell {
  background: #ddd;
}
</style>
</head>
<body>

  <div class="redSub">sub 1</div>
  <p class="greyText">blah</p>
  <table>

    <tr>
      <td class="colHeader">A</td>
      <td class="colHeader">B</td>
    </tr>

    <tr>
      <td class="oddRow">1</td>
      <td class="oddRow">2</td>
    </tr>
    <tr>
      <td class="evenRow">3</td>
      <td class="evenRow">4</td>
    </tr>
  </table>

</body>
</html>
 
<html>
<head>
<title>'Semantic' era</title>
<style type="text/css">
.section h2 {
  color: red;
}
.section p {
  color: #666;
}
.section thead th {
  font-weight: bold;
}
.section td {
  background: #ddd;
}
.section tr.odd td  {
  background: #eee;
}
</style>
</head>
<body>
  <div class="section">
    <h2>sub 1</h2>
    <p>blah</p>
    <table>
      <thead>
        <tr>
          <th>A</th>
          <th>B</th>
        </tr>
      </thead>
      <tr class="odd">
        <td>1</td>
        <td>2</td>
      </tr>
      <tr>
        <td>3</td>
        <td>4</td>
      </tr>
    </table>
  </div>
</body>
</html>
  1. document structure inferred
  2. classes describe appearance (volatile)
  3. direct class application bloats html
  4. changing style properties causes headaches (change all instance names or they’ll be out of sync with appearance).
  1. document structure inherent
  2. classes describe content (stable)
  3. ‘container’ classes give context to descendants
  4. appearance (volatile) is abstracted from structure (stable). Redesign is anticipated & therefore easier

One pillar of the ‘semantic’ technique is the sparing application of class attributes, primarily to container elements. Elements are ‘caught’ using CSS ‘descendant’ selectors (multi-part) which target hierarchies rather than using simple class selectors which require a directly applied class attribute. Using descendant selectors respects the hierarchical nature of the DOM but makes writing bulk CSS quite tricky. Why? Mainly because CSS stylesheets (and style blocks) are fundamentally rule lists, but the DOM elements which our descendent selectors target are obviously trees. How do we bridge the gap between these fundamentally different structures?

Enter ‘ragged’ CSS

In 2006 I (then a contracting developer) found myself filling the shoes of a fantastic design contractor whose contract was up but sadly not extended (project managers eternally underfund and misunderstand UI). I was the new ‘CSS guy’ for a project with scores of developers, hundreds of pages and crippling compatibility/accessibility requirements. I wasn’t really qualified for the task, but my predecessor gave me a CSS tip that got me through: “don’t sweat the properties, focus on rule hierarchies”.

This rule-centric approach is most clearly evident in the CSS formatting style I picked up on that project and used ever since; a style I call ‘Ragged CSS‘.

‘tall’ CSS ‘ragged’ CSS 
.section h2 {
  color: red;
}
.section p {
  color: #666;
}
.section thead th {
  font-weight: bold;
}
.section td {
  background: #ddd;
}
.section tr.odd td  {
  background: #eee;
}
 
.section { }
  .section h2 { color: red; }
  .section p { color: #666; }
  .section table { }
    .section thead th { font-weight: bold; }
    .section td { background: #ddd; }
      .section tr.odd td { background: #eee; }

It differs from the common ‘tall’ CSS formatting style in only a few (primarily cosmetic) ways:

  1. One line per rule, one rule per lineThere may be hundreds of rules, but each will only have a handful of properties.
  2. Indent CSS rules to represent target element hierarchyA block of markup (e.g. a calendar) will typically involve several rules (perhaps dozens). Indenting rules more clearly indicates how their properties will cascade.

  3. Make rules for ‘landmark’ elements (empty rules if necessary)Adding an empty rule for element’s whose classes are used in subsequent selectors makes indentation hierarchy far clearer (improves readability).
  4. ‘namespace’ your rules to avoid conflictClass names may be repeated on large projects unintentionally. Target ‘abc’ class elements inside ‘xyz’ class elements using a ‘.xyz .abc’ selector rather than a global ‘.abc’ selector to avoid tainting someone else’s unrelated ‘abc’ class.
  5. Avoid element prefixes sparingly in your selectorsDon’t use a 'span.content' selector where a plain '.content' selector will do. Dogmatic prefixing (i.e. increasing specificity unnecessarily) makes your rules unnecessarily vulnerable to otherwise inconsequential markup changes (changing the span to a div for instance).

I’ve been using this formatting style for a few years now and it’s made complex CSS development a snap (browser quirks aside). If you’re writing ‘tall’ CSS (or using a tool which generates it), give ragged CSS a go on your next project. Your future self (and/or your successor) will thank you for it!

Scrolling beyond the black stump in Google Earth

Published on November 15, 2008 in Mapping. 0 Comments Tags: , , , , , , .

Recently an American friend of mine was asking about the basics of Australia’s geography over lunch. When I couldn’t make my points clearly using condiments and cutlery I turned to the Google Earth iPhone app (also worth a look is Earthscape). I find myself reaching for the iPhone mid-conversation quite frequently these days. Though awkward (or rude) in a 2 person chat, I find judicious interrogation of the inter-tubes dramatically elevates casual debates, planning sessions and meandering conversations between 3+ people (which this was).

Australia in Google Earth iPhone app
Australia in Google Earth iPhone app

Anyway, the conversation inevitably turned to questions about ‘the outback’, the facet of Australian life which seems to intrigue other nationalities most. Questions typically follow the “Did you live in the outback?” “Have you been there?” “What is it?” “Pet kangaroo?” line (No, not sure, umm, dog person). My difficulty in answering ‘outback’ related questions is that, like most Aussies I know (other South-East Queensland types), I have no firm definition of ‘the outback’ to go on. I’ve camped in some arid and desolate spots (by European and American standards) in Australia (technically not ‘outback’ due to their proximity to the coast), but I’ve never been close to or ‘beyond the black stump‘, let alone into the ‘Never Never‘.

Living in a a disconcertingly rainy country has made me think about the Australia’s arid interior more and more. To keep my hands busy during a skype chat this morning, I fired up the Google Earth desktop client to check out Google’s satellite imagery resolution for the Australian interior. Not bad as it turns out, particularly in areas in the many stripes of 60cm DigitalGlobe imagery (red-tinged for some reason). Even in areas covered by the lower resolution CNES Spot satellite images, the novelty of being able to make out individual trees soon gives way to wonder at the strange patterns visible from space.


Lake Carnegie in Google Earth (larger version)
26°18’16.99″ S, 123°02’09.02″ E
Lake Carnegie, an ‘ephemeral lake’ in WA (image scale: 10km across)
 Nullarbor plain in Google Earth (larger version)
29°55’33.45″ S, 127°01’55.77″ E
The endless Nullarbor plain of SA/WA (image scale: 112km across)
Fire scars (Simpson Desert, NT) in Google Earth (larger version)
24°00’47.77″ S, 137°00’42.19″ E
Fire-ravaged Simpson desert, NT (image scale: 38km across)
 Lake Mackay in Google Earth
22°21’04.26″ S, 128°58’56.72″ E
Massive Lake Mackay, Great Sandy Desert, WA/NT (image scale: 111km across, rotated)

Wind, water, sand, salt, erosion and fire have all shaped the interior with scant evidence of human existence. In particular, note that the Nullarbor shot above covers an area roughly 112km x 63km with no buildings, rivers, lakes or hills. Now that’s desolation!

Whilst assembling the above screen-grabs I noticed that Google Maps seems to use a different database of satellite imagery than Google Earth at zoom levels below 13. This leads to a slightly odd disconnect when viewing Uluru in google maps at zoom level 12 and then zooming once to view Uluru at zoom level 13.

Uluru seen in Google Maps at zoom level 12 Uluru seen in Google Maps at zoom level 13 amatuer aerial photo of uluru (credit rplzzz, flickr)
Photo of Uluru (credit rplzzz, flickr)

True to form, the wordpress team had managed to release several point releases since my last post, making this the 3rd post in a row preceded by a wordpress upgrade. My goal is to have the next post up before WordPress 2.7 hits. I’m doomed.

Out with the old, in with the new (drive)

Published on September 25, 2008 in Uncategorized. 0 Comments Tags: , , , , , .

After my fourth month of constant hard-drive space scavenging (in both OS X and boot-camped Vista) I finally bit the bullet and initiated the oft-delayed drive upgrade. It also seemed like the least irritating time to do the similarly deferred Tiger-to-Leopard upgrade too (hitherto postponed due to space and enthusiasm scarcity).

macbook pro hard drive upgradeAt this point you really hope the rest of the upgrade guide is up to scratch

Everything went pretty smoothly (except for a last minute scramble to find a size 00 philips screwdriver). ExtremeTech’s Upgrade your macbook pro’s hard-drive came in very handy, with only a few deviations necessary to effect the switcheroo.

Here’s a quick list of the necessary bits and pieces I ended up using:

Having the guide on the iPhone proved very handy, and luckily I had a size 0 phillips screwdriver to hand (the recommended size 00 didn’t seem right for the job). Now I’m cruising along nicely with fresh Tiger/Vista installs and throwing RAW files around without a care (hard-drive rationing is like solitary confinement, it warps the mind).

Immersive photographs on the iPhone with PangeaVR

Published on August 25, 2008 in photography. 0 Comments Tags: , , , , , .

It was all I could do to stay true to my anti-iPhone stance so long as my 3G demands remained unmet, so when apple finally relented and released the iPhone 3G I too was forced to concede (it would have been unreasonable of me not too!).

Most curious onlookers are sold after a whistle-stop tour of SMS, Web and Google Maps. However some stubborn folks haughtily demand what makes ‘it so special anyway?’, so I do my solemn best to give them the personalised excuse they need to justify their inevitable purchase. Twitterific seems to be the killer app for folks on twitter (twits?) and the common craft intro to twitter video quickly converts the non-twits. Productivity obsessives (guilty) have trouble restraining themselves when they see remember the milk and reqall. Some nuts are tougher to crack.

The final trick I have up my sleeve (besides games, but that’s cheap) is PangeaVR (app-store link).

PangeaVR iPhone ap displaying 'boat house' panorama from panedia.com
PangeaVR iPhone app displaying ‘boat house‘ panorama from panedia.com

It’s not the most snappily-named app, nor the most seductively dressed but it’s the app that’s come closest to eliciting an audible ‘wow’ from me. In short PangeaVR allows you to view immersive photographs by rubbing your finger on the touch-screen. Drag left to look left, up is up etc. Spin around, zoom in, stare at a foreign sky. See the world not as the photographer frames it, but as they saw it (there’s a big difference). Revisit the places of your youth. Take a peek at that fantasy vacation spot. Show your friends the home town you never shut up about.

example list of panoramas within a PangeaVR portfolio PangeaVR is (amazingly) free, though I reckon that with a less imposing UI metaphor (PangeaVR’s current UI is based around an intimidatingly long list of ‘portfolios’) and a Koi Pond-like price tag, PangeaVR could be a major money spinner for Pangea software (should they choose to go that route). The coolest part about PangeaVR as it stands though is that publishing your own panoramas through the system is also free and (theoretically) within reach of anyone with a digital camera (preferably with a wide lens and a bracket head), a PC and time! Brian Greenstone and the Pangea software guys responsible for this app/system deserve major kudos for their open-handed approach.

Now I should point out that this isn’t the first time I’ve encountered panoramic photography, so my praise isn’t just that of the VR ingénue. I’ve toiled to stitch my own panoramas in the past only to have users unable to load the Quicktime/flash/java plug-in necessary to view them or confused that they must ‘click & drag’ the photo (to which they are unaccustomed) to ‘pan’ the image (some very unhelpful lingo). In the end I gave up and started outputting stitched images as static JPEGs. What makes PangeaVR worth mentioning is that it harnesses people’s natural tendency to touch screens (even when they really shouldn’t) and the ability of even the most timid digital immigrant to comprehend the ‘move the picture with your finger’ concept.

My next move is to devise a workflow for producing my own spherical panoramas without splashing out on any specialist equipment.

Photos — trip to Salzburg & Munich

Published on April 7, 2008 in photography. 0 Comments Tags: , , , , , , .

flickr set: Salzburg and Munich

Just a quick post (to offset my last ridiculously long post on stitching panoramas with Hugin). I’ve finally posted the photos from my recent trip to Salzburg & Munich. It took a few weeks (spare-time) to cull (from ~400 to 85), caption, tag, geocode, post-process and upload the images.

I’m still using the workflow described in Photography ‘08 (ditching iphoto) with a few minor modifications:

  • Re-reading the metadata into Lightroom after geocoding in PictureSync. Otherwise Lightroom is unaware of the geotags and may subsequently overwrite them.
  • I’ve also started exporting temp images from Lightroom (which have the ‘develop’ settings applied) as PictureSync seems unable to accurately apply the crops/tone adjustments defined in Lightroom. The temp images are uploaded and deleted.

Enjoy the pics. Also check out Robert’s pictures of the hike.

Making nodal panoramas that don’t suck

Published on April 2, 2008 in photography. 6 Comments Tags: , , , , , , , , , , , , .

Update: 6th Feb 2010, changed ‘linear’ in title to ‘nodal’ due to comment feedback

Most modern digital cameras (even phones) have a ‘panorama photo’ mode, allowing you to take multiple photos which are then magically combined into a single large photograph. Sounds great, but there’s a problem. These composite images (whether produced in-camera or using ‘automatic stitching’ programs) look OK at first glance, but are typically full of smudges, ghosts, morphing people and objects sprouting from thin air.

Issues with automatic panorama stitching software

So where exactly is is all going wrong? In theory, the basic steps required to create convincing composite images (including panoramas) are:

  1. Align the images — Ordering and placing images by examining their overlaps.
  2. Correct for angle of view — Think ‘fisheye’, but subtler.
  3. Remap images to compensate for varying distance — Closer subjects ‘move’ more between shots than distant ones. This is fixed by identifying common ‘control points’ then remapping the images so they overlap perfectly.
  4. Blend images to hide image seams — Obscuring (typically softening) the boundaries between images to create the illusion of a single image.

Photostitching software typically gets steps 1 and 2 spot on, but can’t hold a candle to humans when it comes to pattern recognition (the secret sauce behind steps 3 and 4). If we want good composite images we have to hijack the identification of control points and eventually produce a layered photoshop file so we can blend manually (to perfection).

Pitfalls during shooting

First we need to keep an eye out for things that often go pear-shaped during the shoot.

  • Creeping Horizon — Caused by gradually rotating the camera up/down between shots (when doing a horizontal pano).
  • Wavy horizons — If the camera’s not perfectly level, the horizon will see-saw up & down.
  • Moving foreground subjects — People/cars/trees/etc moving between shots.
  • Moving photographer — Wind, terrain and rampaging wild-life may interrupt your plans.
  • Rubbish/damaged lenses — Cheap cameras have cheap lenses whose variations become glaringly obvious during stitching.
  • Varying ‘brightness’ — Not locking exposure results in adjacent shots with lighter or darker skies.
  • Varying ‘colour’ — Not locking white-balance (when multiple light sources present) can result in dramatically different colour between shots.
  • Changing light — If clouds arrive halfway through shooting your panorama, give up or start again.

Tips for shooting

  • Use a tripod
  • Level your tripod (camera’s rotation axis should typically be perpendicular to the horizon).
  • Be aware of moving objects near the camera (remember to check 360°). Fast moving objects are generally OK. It’s the slow moving object that cause headaches later on.
  • Take more shots than the camera recommends (only if using manual stitching). Using 2/3 overlap will give you a spare photoshop layer during blending.
  • Be quick. Light and subjects will change during your shoot. Plan your panorama, do a dry run, then fly like the wind (but don’t forget the other tips).
  • Lock any camera settings you can, leave the rest alone. Some cameras lock exposure and white balance in ‘panorama’ mode (check the manual), but it’s up to you to keep focal length (zoom), aperture, focus and ISO constant.

Playing with pixels — stitching an ‘unstitchable’ panorama

All the apps mentioned in this section are available for Windows and OS X (linux too, though using gimp instead of photoshop). Also it’s worth pointing out that even though this post is about horizontal panoramas, these techniques can be be used to create any composite image (blended tiled images, unblended tiled images, spherical panoramas, gigapixel images etc).

  1. Shoot the images & collect on PC — I’m going to use images taken atop Gaisberg mountain near Salzburg, Austria. The images were shot on my Canon Powershot G7 in it’s Stitch Assist mode (exposure & white balance locked). To make things interesting I’ve chosen a set of images with a few of the common issues you may come across (no tripod, inconsistent position).

  2. hugin logoDownload and extract Hugin — Hugin is an ambitious open source (donation-ware) panorama creation GUI (cross-platform, written in c# under mono). It’s still beta so you may find the UI a little cryptic and glitchy, but you can’t complain given the price-tag (free).

  3. Download and install autopano-sift (optional) — Autopano-sift is a helper program for Hugin which identifies control points in your sub-panorama images using some pretty hefty maths. We will use autopano-sift to kick-start the control point detection process, but you can bypass autopano-sift and create all control points manually if you wish.

  4. Open Hugin and load images — When Hugin loads click the ‘load images’ button and locate the images which make up your panorama (some camera naming schemes make this difficult so it’s best to collect them in a folder beforehand). Hugin may ask you to specify your cameras stated focal length (usually silk-screened around the lens itself) and 35mm equivalent (try dpreview or the manual).

    Hugin startup screen

    Once Hugin has loaded your images, autopano-sift will appear (on first-run, Hugin may ask you to locate the autopano.exe).

  5. Use autopano-sift to identify control points (optional) — The autopano-sift UI has a few tweak-able parameters for the adventurous but for now just click ‘compute’.

    autopano-sift UI

    Autopano-sift’s ‘PTO generation’ process will kick off. This process may take a few minutes (depending on the size & number of your images). Once completed, click ‘OK’ and close autopano-sift.

  6. The rough draft — Hugin will use the control points generated by autopano-sift to align and distort your images. It will present its results in the ‘preview’ window.

    initial panorama results

    As you can see, there are some misaligments. These are to be expected (especially with dodgy source images like the ones I’m using) but unlike the automatic stitching software which would just blend over these cracks, we have the power to fix them.

  7. Get to know the ‘control points’ tab — Close the preview window and navigate to the ‘control points’ tab.

    hugin control points tab

    The ‘control points’ tab enables you to review, edit, delete and add control points to improve the stitching & blending processes. Additionally, it gives a unique perspective on exactly why automated control point detection is so underwhelming. Main points:

    1. Existing control-points on both images (automatically generated by autopano-sift).
    2. We’re only really interested in the area of overlap. Ideally there would be control points evenly distributed througout this entire overlap region.
    3. The list of control points (automatic and manual).

  8. Manually tweaking control points — The ‘control points’ tab in the current build of Hugin (0.7 beta 4) has some frustrating UI glitches, but the following workarounds helped me:

    • When editing existing control points, set the zoom factor to ’200%’ (be sure to click them dead centre or Hugin will try to create a new control point, frustrating).
    • When adding new control points, set the zoom factor to ‘fit to window’ then click on both images in turn, position the control points, click the ‘fine-tune’ button and then the ‘add’ button. repeat.

    Add a few control points for each troublesome image pair (evenly throughout the overlap area) then go to the ‘optimize’ tab, choose ‘everything’ and click the ‘optimize’ button. Once the optimization process is complete, check the preview (ctrl-p on windows, cmd-P on mac) to see whether it’s made an improvement to the panorama. If there are still major breaks, tweak the control points as necessary.

    improved stitch

    Here you can see the panorama preview is less ‘wavy’. Don’t worry too much about the rotation, we’ll fix this in photoshop.

  9. Get PTStitcher — Although Hugin comes with a stitching engine (nona), we’ll use PTStitcher as it has better blending plus .psd output. Download the PanoTools zip and extract it. Copy the pano12.dll from the extracted folder into your Hugin folder. Copy the PTStitcher.exe app (from the Helpers folder in the extracted PanoTools folder) to the Hugin folder.

  10. Creating a stitched (layered) psd file — In Hugin, go to the ‘Stitcher’ tab, change the ‘stitch the images’ option to ‘into a layered photoshop file’ and click ‘stitch now’ (beware the psd file may be several hundred megabytes, hope you’ve got the RAM). Hugin may ask you to locate PTStitcher. If so point it to the copy you put in the Hugin folder.

    Hugin stitcher options

  11. Final blending & cropping in photoshop — Upon opening the psd file in photoshop you’ll notice that the blend lines are pretty conspicuous. However, as each shot is a separate (masked) layer you can tweak each blend independently by editing the mask.

    blend issues

    At this point you also have the opportunity to mask in/out moving foreground subjects and make other creative edits (i.e. duplicate people). If you took my advice and shot with 2/3 overlap you’ll have a spare layer to run wild with too. A few blending tips:

    • Use broad diameter ‘soft’ brushes to feather the masks in areas of sky.
    • Avoid straight lines in your masks (especially horizontal & vertical lines). Human eyes will spot those straight away.
    • In areas of high detail use small diameter ‘harder’ edged brushes (this avoid ghosting in the final image).
    • Hard edged subjects in the foreground make great (hard) mask boundaries

    After tweaking each of the seams the whole panorama becomes pretty convincing. This is the most rewarding part of the process, and you run the risk of wasting a lot of time ;) . In this example I decided to merge foreground characters from two different shots, but I left some easter eggs in there for anyone who’s really looking.

    resolved blend issues

  12. The final cut — To get rid of the ‘rising horizon’ problem in this image set (a constant issue when shooting panoramas sans tripod) I’ve selectively cloned and distorted the middle foreground grass and left sky. The final image captures all the things I remember from that instant in one single image (the ultimate aide mémoire).

    final image

The length of this post might make the process of manually assembling composite/panoramic images appear complex, but trust me, the second time’s a snap. Oh, and you’ll be the only kid on your street who’s panoramas don’t suck.

dpreview.com lens review widget sallies forth

Published on January 30, 2008 in dpreview and photography. 2 Comments Tags: , , , , , , , , .

Today’s a big day at dpreview.com as it sees the launch of the first dpreview.com lens reviews. I’m excited as it also sees the launch of my latest and greatest project, the humbly-titled lens review widget (or see it embedded in a lens review).

Screen shot of the lens review widget in sharpness mode

The lens review widget is a flash component embedded into our lens reviews responsible for presenting the results of our studio tests. The challenge of the project is that the lens review studio test output is intricate, multi-dimensional and in acute danger of misinterpretation.

The widget’s goal is to provide users with a constructive mental model of this dataset and an intuitive/responsive/encouraging interface with which to navigate it.

The broad data categories initially displayed are:

  • sharpness demonstrationSharpness is the top of the list for most lens geeks. Not all lenses are created equal, especially considering the difference in the ‘sharpness’ of the image they produce. A single lens can vary greatly between different focal length/aperture combinations and even varies (non-linearly) from centre to corner within a single shot!
  • chromatic aberration demonstrationChromatic aberration is the colour ‘fringing’ you see around light-on-dark or dark-on-light objects in your photographs. Generally purple or green (but occasionally blue/yellow) this effect varies with focal length/aperture and distance from centre.
  • distortion demonstrationDistortion is what causes real world straight lines to appear curved in your photographs. Zoom lenses in particular are prone to distorting ‘inwards’ (pincushion) at one extreme and ‘outwards’ (barrel) at the other. Distortion varies by focal length.
  • falloff demonstrationFalloff is the phenomenon whereby the corners of the frame appear ‘darker’ than the centre (i.e. the light ‘falls off’ from centre to edge). Though often used as an intentional effect, falloff is generally avoided if possible and varies with focal length, aperture and obviously, distance from centre.

In practice these 4 effects are all present (to some extent) in every shot you take, so representing their individual effects is more complicated than simply displaying the raw images (which we also do).

Designing the user interface

With so much data to navigate and several dimensions to deal with (angle from centre, distance from centre, focal length and aperture) the process of designing a useful and intuitive UI was difficult. Eventually a standard hierarchy emerged:

  1. Lens review. It may seem the obvious choice, but I’ve included it on this list as (if allowed in configuration) the user is able to change the review data being viewed within the widget.
  2. Visualization. Sharpness & Chromatic Aberration, Distortion or Fall-off.
  3. Focal Length. Or ‘zoom’ to non-photographers, is generally marked on lenses in arbitrary integer values (generally 4 – 7 per lens).
  4. Aperture. The variable size of the ‘hole’ which regulates the amount of light entering the camera body. Stated in f numbers, an approximately logarithmic semi-standardized decimal value. The aperture values available for a lens depend on the focal length chosen.
  5. Comparison lens review. Users reading reviews are constantly making comparative decisions, so the widget allows users to compare two lens reviews (or two data-points within the same review) cheek-by-jowl, hopefully improving comprehension. Of course, this requires user interface chrome to allow the user to choose.

picker picker controlThere was much internal debate (which still rages) about this UI element which allows users to change/compare reviews, but the eventual decision was to filter reviews by system, lens and finally camera body. Note: Our lens tests are performed on a camera body (instead of an optical test bench) which has relevance upon the results.

The ‘slider’

The real challenge from a UI perspective was creating a UI element which allowed users to change focal length / aperture in an intuitive way, was brand-neutral and could cope with the intricacies of the focal length /aperture systems and their inter-relationship. After initially experimenting with flash’s slider component, I decided to abandon it and start from scratch (before you shout Not Invented Here, be aware that flash’s slider is a major usability dud).

GrabSlider demo animation

Trim

Some neat little features that I pushed into the widget (usually late at night):

  • Full screen mode: At any time a user can click the full screen button at the bottom of the widget to view in a new window/tab (selected reviews, visualization, focal length & aperture are retained).
  • Permalinking: At any point, users can obtain a permalink to the current state of the widget to bookmark and/or link to (again the selected reviews, visualization, focal length & aperture are retained).
  • Keyboard controls: A bit of a power-user feature requested by the guys in the office, the keyboard arrow keys can be used to control focal length and aperture. If multiple reviews are visible, they will all receive the inputs, allowing users to change the visualizations in unison (not currently possible via the sliders).

Though a few minor issues persist, I’m very happy with the current implementation. I feel it strikes the right balance between aesthetics, usability, accuracy, brand-neutrality and ‘fun’.

Time (and the forums) will ultimately tell.