It's not perfect, but it's really cool
The newly updated Pinterest Lens feature is probably best described as "Shazam for food." To use this food recognition technology, which launched in the Pinterest app earlier this week, simply take a picture of an ingredient, and Pinterest will tell you what it is and give you some recommendations for how to use it. If this sounds like something straight out of an episode of Silicon Valley, that's because it kind of is. Earlier this week, an eerily similar, yet entirely fictional, app called SeeFood was featured on an episode of the HBO show. (A Pinterest representative told The Verge that this mimicry of Pinterest's newest feature on TV was "separate and completely coincidental.)
The gimmick sounds great, but I had one question: Does Pinterest's new "Shazam for Food" actually work? Or is it as goofy as it sounds?
Armed with my iPhone and the newly updated app, I took to Extra Crispy's pantry to find out if the Pinterest Lens feature actually works. The first thing I learned is that it's very easy to use, albeit a little tricky to find. To access the food recognition feature, called Pinterest Lens, click on the little round camera button to the right of the search bar on the home feed. Then, point it at the food, click to search, and let the technology happen.
I will say that when it works, it works well. Take, for example, my first attempt at using the Lens feature on these cinnamon sticks.
All you have to do is point your camera at the item in question, click, and search results will pop up. And these results were spot on. Yes, these are cinnamon sticks, and yes, I am looking for cinnamon recipes. So perfect!
Plus, the recipe options that were offered up were totally reasonable. Like I would definitely be interested in making blood orange cinnamon rolls or baked cinnamon sugar doughnuts (and I may have actually pinned those recipes so I could make them later).
But when the feature didn't work, it was frustrating. One of the most frustrating errors was when it took a couple of tries for the camera to recognize the ingredient properly. These coconut flakes were fairly troublesome.
When I tried using Pinterest Lens to take a photo of coconut inside the metal tin, it didn't really work all too well. The first time, the app said I was looking white rice, which I can understand even though that's wrong. The second try, the coconut came up as dried basil, which is totally wrong.
Trying to get a clearer shot of the ingredient, I spilled a little of the coconut onto the table, and only then was Pinterest Lens able to recognize the coconut for what it was. And once again, the recipe recommendations it gave were pretty solid, like coconut cream pie or coconut-flavored cakes. (Though it was hard not to wonder if it would've just been easier for me to search "coconut flakes" in the search bar rather than go through the rigamarole of taking several photos of coconut flakes and spilling some on my table.)
My other issue with Pinterest Lens was that, since the feature searches through all of Pinterest, sometimes the results for food items were not recipes. Pinterest Lens recognized these as eggs, which was great.
But the results I got for these bad boys weren't recipes for frittatas or tips on how to boil eggs. No, I got home decor recommendations, and there wasn't even an option to find egg recipes among the bubbles that popped up.
The feature also didn't really work when there were multiple ingredients in the frame. I tried taking a shot of the eggs with a mini bagel (that happened to be Funfetti-flavored because that's what we had in our office, which is why the bagel below is colored with rainbow flecks, but stay with me here), hoping to get some kind of recipe for an egg sandwich.
The results I got were all over the place, including one for "Secret Garden Bridal Boudoir." There were also a bunch of different doughnut-shaped crafts, which weren't bad—but they weren't what I was looking for.
If you are consistently dissatisfied with the responses you get from the app, you can "teach" it. That's part of how Pinterest is able to keep on updating the feature and make it increasingly accurate. Take the "fruit salad," which was really just chopped up watermelon and pineapple, that I took to work for a snack.
Pinterest recognized that it was healthy, that it was food, even that it was a fruit salad. But it said the bowl was filled with strawberries, not pineapple or watermelon.
To help the feature learn what it's looking at, click on the little bubble with the plus sign, and you can search for what the food actually is.
And ultimately, typing what you have into the search bar is still way more accurate than using the Pinterest Lens feature. But the ability to learn is why I have some hope for this new Pinterest feature, and having played around with it, I totally understand the appeal. Imagine sitting down at a restaurant and taking a picture of your meal, only to have the recipe immediately pop up so you can make it at home later. That's really cool! And even if it's not perfect right now, the visual search technology is only going to get more refined, which means the recipes are going to get better—and it'll definitely be able to tell you more than whether what you're looking at is a hot dog or not.