Last week the Amazon Fire phone started to sale, while reviews has not been kind to it, the phone does brings a host of new technologies that will influence user expectations and experience as a whole, not only for Amazon Fire Phone users.

Specifically (in our point of view), one of the technologies that are changing all this, is the FireFly feature of the phone, which in short, it’s an aid for Amazon to sell you more stuff. Regardless, FireFly is a quite interesting mashup of technologies.

We have wrote previously:

Augmented Reality is a new medium, as such, it offers new opportunities to display and interact with information, creating possibilities that were not previously available. We at Mollejuo think, this is one of the of the many new forms humans are interacting with machines from now on.

FireFly

FireFly allows you to scan anything and the phone we’ll try to recognize it, here is the whole demonstration:

In short (in case you don’t have time to watch the video), it will scan bar and QR codes (standard scanning), it will do OCR to anything in front of it, it can recognize shapes and “images” (AR), it uses sound and video recognition to link to music and movies. So, as you may be inferring, FireFly brings together several technologies (apps) that were previously separated, like bar & QR code scanning, OCR, AR and sound/video recognition (SoundHound, Shazam).

Ancestors

In the beginning (2009) there was Google Goggles, a mobile app that focused on visual search, so instead of a person typing the search on the phone, they would simply point the phone (using the camera) towards the object in question, and Goggles returned a search result. Awesome idea by the time, which if it would have been combined with Google Shopping, it would be even more awesome, but they didn’t. Of course, Google could do such a thing a Goggles, because it has the humongous DB available from their search algorithms, so they married the two of them.

Amazon for its part, while it has offered search for some time, it hasn’t caught up with most people, but Amazon (contrary to Google) does have an incredible precise DB of products, products that people want to buy. And that’s what they did, they’re taking advantage of this huge DB and putting it into a shopping phone.

Of course, FireFly was not the first attempt of doing this from Amazon. They previously launched the app called Flow (late 2011), which does most of the things FireFly does today. With Flow, they had the time to fine tune it and gathered user behaviour information. So when this app turned into a feature for a phone device (i.e. FireFly), it worked as best as it could.

Change in user behaviour

The important thing here is that Amazon is calling it FireFly, is not calling it, QR code reader, nor OCR scanner nor AR enabled (which it is). With it, people will get even more used to pointing the phone at things, waiting for an answer from the device (regardless of the tech behind it). This behaviour wasn’t natural to do from a person 5 years ago, however as more and more people start doing it, it will become second nature, like googling is today.

That can only help the AR world as a whole, as the great barrier for people swiping around their phones or pointing it to something doesn’t look weird anymore, now is natural!