Embedded Vision: Assisting Those With Patience Limitations

Back in mid-December, I discussed the barcode as a pioneering computer-now-embedded vision application, in the context of reporting the death of its co-inventor, N. Joseph Woodland. As such, a related (and more advanced) OCR (optical character recognition) innovation from Apple that came out around the same time also caught my eye. Apple's latest iTunes v11 supports the ability to automatically read the code off a newer iTunes Store gift card containing a border around the multi-character string, using a computer's webcam, thereby precluding the need to tediously enter the obscure sequence character-by-character via the keyboard.

According to Apple enthusiast site TUAW (the Unofficial Apple Weblog), the new iTunes capability is specific to Apple's latest "Mountain Lion" operating system and therefore likely leverages the Core Recognition framework that's specific to Mac OS 10.8. One of Apple's partner developer companies, iNVASIVECODE, implemented the feature, which Apple brands as "Code Redeemer". Employee Geppy Parziale, a long-time Apple consultant, described its development in detail in a company blog post, which was originally published here but has seemingly been subsequently removed (for unknown reasons). However, the 9to5Mac and Cult of Mac sites captured some choice quotes while the writeup was still live, which I've combined below:

"The last 12 months have been really exciting and intense for me. This project represented a wonderful opportunity for me to learn and touch new things and really push the limits of technologies and devices. Many people and teams were involved in this project. You cannot imagine the quality and quantity of engineering, design, development, integration, testing, manufacturing, marketing, management and coordination behind just this single feature. This is Apple."

"A very sophisticated set of image processing and computer vision algorithms extracts the 16-digit code from the iTunes Card and converts it into a string in few milliseconds. Core Animation, Accelerate, Core Graphics, GCD, AVFoundation and other frameworks were combined together to meet some very strict performance and usability requirements."

Parziale also notes that the feature is optimized for the visually impaired thanks to VoiceOver in OS X. “VoiceOver helps positioning the card in front of the camera and the very fast image processing algorithm generates very quickly the result,” according to Parziale. “The user experience is amazing.”