The range of creative and practical apps that use the camera in unusual ways is astonishing. Who would have thought that you could use it as a heart rate monitor or a business card reader? There's even an app for those who are colorblind, to show them the colors present in any object you photograph.
Apple made a brilliant decision when it let developers create apps that access the camera input and use it in non-traditional ways. In this month's column, I offer a roundup of some of these apps. Unless otherwise noted, these apps work with all generations of the iPhone and the 4th generation iPod touch (with camera).
Barcode scanners for shopping
The most popular non-photography use of the camera may be as a barcode scanner. The concept is simple—these apps use your iPhone or iPod camera to take a photo of a barcode, and then the app does an online search to find the lowest price of the product from among thousands of retailers. So, for example, if you're in a mall and scan an item, you can find out whether a different store in the same mall has it for a lower price.
iPad 2 to have a camera?
As this issue goes to press, there's a considerable amount of speculation that a new iPad will be released this quarter and that it will likely have two cameras, FaceTime support, and it may even have dual-network capability, allowing it to work with AT&T's GSM and Verizon's CDMA network. DigiTimes reported that they heard from component manufacturers that it would begin shipping in February. Because of its larger screen, an iPad with a camera would greatly enhance some of the apps featured in this article.
RedLaser is one of the most popular barcode apps. Features include the ability to limit your search to lower prices in nearby stores, e-mail a results list, and e-mail a barcode. The app does its price search via Google Product Search, TheFind Product Search, eBay, and Half.com.
Shop Savvy Barcode Scanner
Shop Savvy is also one of the most popular barcode scanners. In addition to finding the best prices on the Internet and nearby stores, the app also has a "deals finder" that locates promos, coupons, rebates, sales, and more. When you shop locally, Shop Savvy puts a blue dot next to the store price to show you that the product is in stock. The app also determines if the store will match the price if another store offers it for less. The app even includes a map feature that will help you find nearby stores.
TheFind.com claims to be the largest price-comparison search engine online. Last fall they introduced their own app, which combines their search engine with RedLaser's barcode scanner technology and throws in a coupon finder, driving directions, and click-to-call feature. They guarantee that their app will show you the most matching products and find the best prices.
Search via photo in addition to barcode
Not to be outdone, the giants Google and Amazon have come out with their own barcode scanners with some amazing additional features.
Price Check by Amazon
Free (iPhone 4 and 3GS only), app2.me/3244
Amazon's Price Check app lets you search for the product you have in mind in four different ways: you can speak the name of the product, use the iPhone's camera to read the barcode, take a photo of the item, or enter the name of the product. The app uses the same voice recognition technology that powers the popular Dragon NaturallySpeaking.
Google Mobile App
Free (Google Goggles feature works on iPhone 4 and 3GS only), app2.me/3245
Google Mobile App has long been an iPhone staple, letting you search the Web via voice or by entering keywords. Late last year, Google added the Google Goggles feature, which lets you take a photo of an object and then search the Web based on what's in the photo. You can take a photo of a barcode, landmark, product, text, artwork, book, etc., and the app will return information. The help screens say that the app doesn't work very well for items like animals, apparel, or furniture. In my experience, the barcode feature worked well, but the app didn't find a match for many of the objects I photographed.
Productivity: Business card readers, fax, scanner, OCR
A host of apps use the camera to improve productivity.
Shoeboxed Business Card Reader
Free (app and service), app2.me/3246
This app and service may be the most amazing freebie in this article. You use the app to take a photo of the card. It then automatically uploads it to the Shoeboxed service, which uses optical character recognition to translate the image into text for a contact. To make sure the image has been interpreted correctly, a human employee of the service checks it out. After it's approved, the contact information is automatically added to your iPhone contacts over the air. The developer promises that you'll get your accurate contact info within 24 hours, but more typically it's a matter of minutes. If you have a stack of business cards gathering dust, this free app/service is ideal for you.
Free (receiving faxes is free, sending faxes costs money), app2.me/3247
This free app can be used to send and receive faxes via your iPhone. To send a fax, the app uses your iPhone's camera to capture an image. However, to send faxes you need to sign up for an eFax account: eFax Plus ($16.95/mo) or eFax Pro ($18.34/mo). A 30-day free trial of this service is available. In addition, you can select the developer's free service (efax.com/efax-free), which lets you receive faxes only. Note that the free service limits you to 10 faxes per 30 day period.
If you need to convert a printed document to electronic text so that you can edit it, check out Prizmo. The app uses optical character recognition to convert print to e-text in 10 different languages. As with other OCR apps, you'll need to review the electronic text and correct any OCR mistakes. However, reviewers say that Prizmo is quite accurate.
Note that OCR apps need a well-lit and in-focus image to work with. Prizmo helps to make sure the image is correctly oriented and helps to correct geometric distortion. You can add text-to-speech capability and language translation to Prizmo via an in-app purchase. You'll need an Internet connection for the language translation since it uses Google Translate.
JotNot is one of the most popular of the free scanning apps. It allows you to scan multipage documents, but does not include optical character recognition. The app includes camera stabilization and other features that enhance image quality. The Pro version allows you to e-mail scanned docs or upload them to MobileMe iDisk, Dropbox, Google Docs, and Evernote, or transfer them to a desktop computer via Wi-Fi. The Pro version can also save the scanned documents as PDFs, JPGs, or PNGs.
Augmented reality (AR) is one of the most exciting uses for the camera, with considerable potential. AR apps display a live image of whatever the camera is pointed at and then superimposes information or other content on that image. For example, an AR app developed for a museum might let you point your camera at an exhibit and read more about it.
Augmented reality superimposes information based on where you are and where your camera is pointing. Apps determine what's being viewed in one of two ways: First and most commonly, they use the hardware features of the iPhone—such as GPS, accelerometer, compass, and gyroscope—to determine what your camera is pointed at. Second, they use some sort of image recognition to determine what's on the screen.
Star Walk is one of the most popular AR apps out there. It puts a personal planetarium in your pocket, showing the night sky via a live image on screen and identifying over 9,000 stars, planets, constellations, and other celestial objects. Tilt your iPhone towards the sky and the app uses the GPS and digital compass to determine the location and orientation of the phone. The image on your screen will correspond with the area of the sky you're pointed at. The app includes links to Wikipedia for additional information on the objects you're viewing. On the iPod touch, which doesn't have GPS, you'll need to manually enter your location if Wi-Fi isn't available.
$3.99 (for iPhone 3GS, 4, and iPad 3G), app2.me/3251
Spyglass can superimpose a wide range of augmented reality tools on a live camera image, including a mil-spec compass, tracker and finder for GPS locations, bearings, inclinometer, sextant, rangefinder, and angular calculator. The app includes maps, a sun and moon position tracker, a star positions tracker, and more. You'll need an Internet connection to use the maps features.
Theodolite is somewhat similar to Spyglass, in that it overlays information on the live camera image, including position, altitude, bearing, and horizontal/vertical inclination. The Basic version adds the ability to take photos and screenshots within the app, with 2X and 4X zoom options, and the ability to stamp geographical data. It also offers maps. The Pro version adds a zero reference mode for angles and a calculator to compute the height of landmarks, distance to landmarks, triangulate position, and much more.
London Bus, Metro Paris Subway
These two apps from Presselite were among the first apps to bring augmented reality to the iPhone, and they work in a similar fashion. For example, let's say you're using the Metro Paris Subway app and want to find the nearest subway station. Simply launch the app and hold up your phone. Superimposed over the live image are little information boxes describing what you're looking at. Pan your iPhone around until you see one identifying the metro station and walk towards it. Along the way, the app will identify restaurants and other locations of interest. Point your camera at the ground, and you'll see an image of the sidewalk with arrows on the sidewalk guiding you to your destination. Presselite makes similar apps for Barcelona, New York, and Rome.
Layar Reality Browser
Free (iPhone 4 & 3GS only; most augmented reality "layers" free, but some are in-app purchases), app2.me/345
This app is a multi-purpose augmented reality tool with a lot to offer. The digital information is organized into "layers." For example, there are separate layers for tourist information, architecture, nature & parks, nightlife, stores & shopping, etc. To view a particular type of information, you download that layer from within the app. There are hundreds to choose from and most are available for free.
I played with the YouTube layer. As I moved my camera around, I saw icons on the screen indicating videos that had been posted on YouTube by someone living nearby. I simply tapped on the icon to get more information about the video. This app lets you access an amazing amount of content.
According to its developer, Junaio is the first AR browser to offer image-based recognition. Digital information is organized into "channels" (similar to "layers" in the previous review). So, for example, if you wanted to find places to eat, you'd install the "Restaurants" channel. Most of these channels use the iPhone's GPS and digital compass to determine the location and orientation of the phone. However, a few use a special image recognition feature called "Glue." There aren't very many of these channels yet, but tools are available to developers to create Glue-based channels. I tried the eBay channel (which uses the hardware features, not Glue), and it was impressive. As you view the live image on your iPhone, it shows you the eBay classifieds that are nearby.
Pandemica: AR Shooter
Augmented reality has also come to games, with Pandemic winning the award for best augmented reality app in the 2009 Best App Ever Awards. It's an arcade-style game, but the alien organisms that you shoot are superimposed over a live view of your environment.
One handy use of the camera is to take pictures of text in a foreign language and have it translated — very useful, for example, if you're in an ethnic restaurant.
Google Mobile App
Free (iPhone 4 and 3GS only), app2.me/3245
Already mentioned above, the Google Mobile App is also included here because it has a translation feature. Using its new Google Goggles feature, you can take a picture of foreign language text, select the portion of the image you want to have translated, and the app will translate it.
Babelshot (photo translator)
This app works in a similar fashion as Google Goggles but has been around much longer. It supports automatic translation between 32 languages. Note that this and similar apps only work for small amounts of text because you need to take a close-up picture.
$4.99 (includes one language, in-app purchase of additional languages), app2.me/3270
This app lets you select only one of the 16 supported languages to translate into English. Via in-app purchase, you can get additional languages for $1.99 each or buy all 16 languages for $3.99. Five of the languages (French, German, Italian, Portuguese, and Spanish) include audio translations to aid in pronunciation.
Free (languages must be purchased in-app for $4.99), app2.me/3389
Word Lens is a remarkable combination of language translation and augmented reality. For example, with the Spanish-to-English version installed, you can point the camera at a sign or menu written in Spanish. The app shows the sign or menu onscreen and the Spanish words morph into their English translation. The app is self-contained: no Internet connection required. At this point, only Spanish-to-English and English-to-Spanish are available.
An interesting selection of apps use the camera to enhance your vision in some way. They range from one that will identify colors if you're color blind, to eyeglasses apps, to an app that helps you identify counterfeit money.
HueVue: Colorblind tools
This app helps colorblind people identify colors. Simply open the app and take a picture of the desired object or select one from your photo album. The app displays the selected photo on screen. Drag your finger over the image and it will tell you the color of the object you are touching. HueVue will also give you the name of the color, RGB and HSV values, and other detailed information. It can also help you find colors that harmonize with a given color.
This app lets you enlarge the camera image so you can read small text and view hard-to-see details. Simply point the camera at something and you'll see it magnified on the iPhone's screen. You can enlarge images up to 8 times their original size. The app works with any iPhones and the newest iPod touch. However, it works best with the autofocus camera found on the iPhone 3GS or iPhone 4.
Money Scan helps you detect counterfeit money by using your device to magnify and illuminate the currency.
Some of the uses of the camera are simply amazing. How would it even occur to anyone that the camera could be used to measure heart rate?
Heart Rate - Free
This app measures your heart rate using the iPhone 4 camera. (It only works with iPhone 4.) You simply put your finger over the camera lens, and the app measures the minute changes in skin color caused by the ebb and flow of blood in your finger. The developer says that the app isn't intended for medical use (standard disclaimer) but that it provides an accurate measure of your pulse.
This app displays over a dozen interactive and customizable 3D scenes. It's long been an interesting novelty in the App Store, but the latest update adds an astonishing feature: real-time head tracking. The app accesses the front-facing camera of the iPhone 4 to track the motion of your head and alters the perspective on screen based on it. For example, you can turn your head to look around the 3D scene. It'll be interesting to see if motion-based control catches on in the App Store.
If you're addicted to texting, this app's for you. iType2Go lets you text while you walk. The screen shows a live image of what's in front of you and superimposes the text message you're entering over the image. The free version lets you send your text via e-mail, while the Pro version lets you post updates to Facebook and Twitter, as well as send SMS messages.
Inspiring range of possibilities
The built-in camera opens up an inspiring range of possibilities. A big "thanks" goes to Apple for allowing developers to integrate the camera function into their apps. An even bigger "thanks" goes to all the third-party developers who have taken advantage of this feature and used it in some very creative ways.