10 Ways Smart Visual Commerce is Disrupting Retail

It’s no secret that images have dominated the web over the last few years. The rise of Pinterest, Instagram, Snapchat and other image-centric social networks have shown how visual mobile, and the web, really is (just check out page 90 of Mary Meeker’s Internet Trends Report if you’re in any doubt).

Between those three networks, and with Flickr thrown in for good measure, they have 1 billion users. Benedict Evans, a venture capitalist at Andreessen Horowitz (‘a16z’), estimates that taking into account just a few of the most popular apps, there were probably 1.5-2 trillion photos shared in 2015 alone.

These kinds of numbers would suggest a huge opportunity for eCommerce, but they’re not the only one.

Advances in AI (specifically around image recognition), the prevalence and increased quality of mobile cameras, and now the emergence of VR and AR are all converging towards making ‘visual commerce’ a potent and immediate force in eCommerce, with far reaching applications.

Of course this means there is plenty of long-term potential for disrupting the industry, posing huge opportunities and challenges for everyone.

Let’s start by looking at the potential impact of ‘visual social commerce.’

1. Visual Social Commerce

Courtesy: Pinterest

Pinterest was the first major visual social network to get started on its path towards eCommerce (often referred to as social commerce), in June 2015 when it launched ‘Buyable Pins’, and expanded it later that year; while Instagram announced their own shopping features in late 2016.

There have been successes. According to Pinterest, San Francisco retailer Modern Citizen increased sales by 73%, while Gardener’s Supply Company saw a 2.7x increase. However they’re just 2 data points, both direct from Pinterest.

Beyond that, there are limited stats available on how successful Pinterest’s ‘Buyable Pins’ have been, and Instagram’s programme is still being tested with just a select few partners, so it’s tempting to question if this really will revolutionise online retail.

Early signs aren’t too encouraging. In a GlobalWebIndex study (cited by eMarketer), they found that just 14% of Instagram users aged 16-64 were interested in buy buttons, and 13% for Pinterest.

And the reception from major retailers also seems lukewarm, with Sucharita Mulpuru, principal analyst for Forrester telling Digiday that “They generate very low sales volume to retailers and are negligible for most large brands.” In the same post, Digiday found “that “none of the five brands…interviewed for this article has invested heavily in buy buttons on those two platforms.”

Why? Well one of the reasons cited is that “Users can only shop for a limited number of products on social media, and those platforms cannot store inventory for brands the same way Amazon does for retailers.”

So maybe these social networks won’t end up becoming eCommerce giants themselves, but there’s more than one way to skin a cat, and even if direct buy buttons don’t work out, they should be well positioned to continue playing a major role in the development of eCommerce; from becoming recommendation engines to highly valuable sources of referral traffic.

In fact, Shopify found that Instagram had an average order value of $65 per referral, while Pinterest was only just behind with an average referral value of $58.95.

Some stats from 2016 begin to bear this out, with Mary Meeker’s respected annual report saying that 55% of people see Pinterest as a place for shopping. They just don’t seem super keen to buy from them directly, but then if there’s a gulf between interest and availability, that would explain it. For people to take action, according to BJ Fogg’s Behaviour Model, there needs to be a trigger (buy button), motivation (I want that thing) and ability (i.e. available inventory to buy and fulfilment behind it).

I think it’s too early to say their efforts are failing, but it is safe to say they’re not setting the world on fire either.

However given the massive scale, and engagement, achieved by the major visual social networks, they’re in it for the long game, and I’m sure they will figure out a working model that will see them become an increasingly important eCommerce force in 2017 and beyond.

2. Shoppable video = shoppable stories 

Now if you thought images were in vogue, wait until you catch a load of these insane video stats.

But here’s the real kicker.

In January 2016, Facebook reported that it was seeing “100 million hours of daily video watch time,” and that was over 12 months ago, and before it had launched Facebook Live video. And recently it also announced it’s willing to pay for original content, to keep the growth curve steep.

However that still pales in significance to YouTube, with Business Insider calculating that it sees up to 650 million hours of videos viewed everyday.

But what’s this got to do with eCommerce?

Well, a lot actually.

Thanks to advances in various technologies, including AI (more on that below), it’s now not only possible to shop via images, but video too.

And the applications for that could be far more wide-ranging, particularly when it gets good enough to work in real time (i.e. for live video).

First, the ‘straightforward’ stuff. Even without AI, it’s perfectly possible for brands and retailers to create videos and let users shop for items seen in them. Vendors like WireWax and Cinematique offer these kinds of service and have done for year.

There’re some nice examples from Very.com and their collaboration with Rizzle Kicks, or Kate Spade and their #missadventure series, both of which allow users to click a toolip that takes them to a landing page where they can buy items seen in the video; and more recently Ted Baker in collaboration with Guy Ritchie.


Courtesy: Fashion & Mash

However, there are also surprisingly few examples around, which begs the question of how widely adopted this is; begging the further question of why? Is it too much effort for too little reward? They will have to manually configure and label all the content, then sync it with their landing pages, and given eCommerce is seasonal and trends move so quickly, these videos can’t be seen as ‘evergreen’ content, meaning the effort has to be repeated over and over to keep it on-trend.

Which is where AI comes in.

As it gets more advanced, the shoppable labels will be able to start being populated by algorithms trained by machine learning, and with less manual upkeep from retailers (something being offered by ShopMotion.tv).

This would allow for more examples of instant shopping during catwalk shows (as pioneered by Burberry)

Looking even further ahead, this means that any video could become a vehicle for selling goods and wares that are featured in it, whether these are apples-to-apples brands, or similar products at more affordable prices.

Essentially the 750 million+ hours of video watched by people everyday could become a constant stream of directly purchasable advertising – even live streamed – with minimal input required from brands and retailers (beyond stumping up their programmatic ad spend, much like they do with adwords now).

Not only could this be an immensely profitable new revenue stream for the big players like Google and Facebook (being the intermediaries that publish video, but also the ones who own the AI needed to make it shoppable), but it could be an exciting win-win-win for content creators too (those millionaire YouTube bloggers are likely to get even wealthier if they get an affiliate cut), and of course brands and retailers stand to increase their revenues too.

Now, let’s take a look at what else visual-focused AI and machine learning can do for eCommerce.

3. Machine Learning, AI and Image Recognition

Machine learning, deep neural networks, AI…without going into the differences, they’re all being applied to images, and specifically how to better commercialise them.

Essentially this means that shopping via images is a massive growth market, but not just via social media sites – absolutely everywhere.

In theory these advances mean you can turn your Vogue into a shoppable catalogue at the click of the camera button; high street stores could let you buy from their window displays without stepping into the store as you just click the mannequin and purchase the outfit it’s wearing; and anyone walking down the street could be your muse if you take a snap of them and the underlying AI tells you where to purchase what they’re wearing.

The most obviously application for this surge in smart images is fashion of course, but there’s no reason this couldn’t work just as well for food, gadgets, homeware and other popular categories for online retail.

Let’s explore the applications in more detail.

4. Shoppable Mags, Mobile High Streets

This technology is a potential game-changer for media companies and publishers, not just retailers. If their ads – even in print – can be snapped and bought in just a couple of clicks, then so long as they can embed some kind of tracking fingerprint into the exchange, they will be able to show the ROI of their ads to brands in a direct and compelling way, and stand a chance of competing with Facebook or Google again.

In other words, shoppable images could potentially be a lifesaver for the declining paper print industry.

However speaking of Google, they’re a player that’s likely to compete fiercely in this market – as it does in just about any emerging market – because they already have a vast database of images and one of the most advanced AI and machine learning programmes in existence.

Much like they’ve dominated text search (and stand in good stead to do the same with voice, as we’ve looked at in my previous post the voice seach and eCommerce), they will want a large slice of the visual commerce pie too.

And there’s a good chance that user-behaviour, and their existing infrastructure, will allow them to do it.

For example if someone snaps a magazine advert, it’s pretty likely they would then default to Google to find where they can buy that particular outfit or similar outfits (at a more affordable price point), and that’s where a combined organic-adwords results listing could work so well for Google.

However, there is an enticing opportunity for another third party, combining AI and an API with major online retailers, to cut Google out completely, by offering to take you directly to the store that sells the product at the best price without having to do the search.

In this example, there could be two options: ‘buy exact item i.e. brand specific’ or ‘buy best matching alternative at [price range].’

This seems like a huge opportunity for somebody to tackle.

5. APIs

One thing hampering the rapid advancement of visual commerce is its complexity and expense, and the spotty coverage it therefore has amongst retailers.

However many vendors are now coming onto the market, offering a centralising service that can be leveraged by commercial partners via a simple API integration.

Here are just some of these vendors: Slyce, Cortexica, CloudSight, Sentient, clarifai, yotpo, and visenze.

As larger retailers start to adopt these high-end technologies, there is then a downward pressure on smaller eCommerce companies to do the same, which is where the widespread adoption should eventually come from, as they are able to access services at an affordable rate by tapping into visual commerce as a service, and not as an expensive, self-developed piece of IP.

Says Practical eCommerce, “Larger retailers — Neiman Marcus, Nordstrom, J.C. Penney, John Lewis, Urban Outfitters, Home Depot — now offer visual search on their mobile apps. This has led consumers to expect smaller companies to support it, too. Additionally, visual search vendors (such as the heretofore mentioned Slyce and Cortexica) now integrate with popular ecommerce platforms, to make it easier for smaller retailers.”)

6. Upselling – Recommended add-ons

Yet another huge commercial opportunity enabled by AI-driven visual commerce are contextually relevant add-ons.

So if you shop a winter camel coat, retailers will be able to offer you much more accurate and interesting add-ons, from clothes to scarves to bags etc.

While this kind of add-on does already exist, it should become much better at matching your style and interests once machine learning is more embedded into the process.

Much like how Google Translate leapfrogged its accuracy almost overnight by using neural networks to better understand contexts and fuzzy relationships, so too should upsells became far more relevant and effective. 

7. Conversion rates

Trust and social proof have been shown to be incredibly powerful in eCommerce for a long time, from the underlying psychological triggers explained by Robert Cialdini, through to more empirical proof such as shown by this study.

As visual commerce continues to be more widely adopted, the use of User Generated Content libraries (UGC) will become even more prevalent, allowing potential buyers to see the fashion (or other goods) in the real world, which will help drive new levels of trust and social proof.

And where it is currently in use, it’s showing great results. 

8. VR – ‘Trying it on’

Trust and reducing barrier to purchase should be taken to a whole new level with the advent of Virtual Reality (VR), because customers will then able to virtually try on anything, view it on a model of their body (replicated to their specific size and shape), and view it from a 360 degree angle.

Courtesy: VR Scout

Vendors such as Marxent, and dozens of creative agencies and VR studios are currently bringing this kind of shopping experience into the market, with brands such as Rag & Bone, Topshop, Dior and Tommy Hilfiger setting the trend as early adopters.

Results are thin on the ground, but with any nascent technology this isn’t surprising, and as VR headsets drop in price and penetrate the market, this could become a major driver of increased sales and reduced returns, as consumers can much more accurately assess if they’ll like an item or not. 

9. AR – Spatial awareness

Augmented Reality (AR), should have a similar affect on homeware and furniture retailers, as users at home can ‘place’ the items they’re considering buying into their home, overlaying it onto the existing environment to see how it looks, and if it fits.

Again, this is something that is already being pioneered by the likes of Wayfair and Ikea, and will likely become more prevalent now Google has released Tango, a working version of its Android OS specifically designed for enabling better AR experiences (where retail purchases and visualising them in the home is a leading use case.) 

10. Expanding market access – for the visually impaired  

Advances in image recognition are also being used to help people who are blind or visually impaired, with Facebook launching its Automatic Alternative Text (AAT) tool last year to describe what is in images to those who can’t see.

Apps like Aipoly and TapTapSee are also now available to help ‘guide’ blind and visually impaired people around by describing their environments to them via their smartphone’s camera and deploying AI to audibly interpret what it sees back to them.

These are incredible applications for helping improve the lives of millions of people, and the technology also has a commercial application too. Given how much eCommerce is now driven by images (directly and indirectly), allowing those with limited or no sight to better understand the context of online purchases may well open eCommerce up to another new market.


Visual commerce means far more than Pinterest buy buttons.

With the advances in machine learning and AI, images and videos are rapidly becoming interactive, or direct conduits for purchasing goods and findings others of interest.

As always with disruptive innovations, this spells a huge opportunity for those willing to roll up their sleeves and embrace what is still a nascent market; and huge challenges for anyone who falls behind.

What’s your plan to explore visual commerce in 2017?