In a visual world, most online retailers still rely on consumers to define and enter their own search keywords. But using images to help shoppers sharpen their queries could push them to buy more, spend more, and feel happier about their purchases.
“For a long time, consumers have been asking things like, ‘When can I find a coat that I like by simply snapping a picture of somebody wearing that coat on the subway?”
A new study of a Chinese e-commerce site’s image-powered search tool offers lessons for companies trying to harness artificial intelligence (AI) to drive sales and highlights new paths for smaller sellers competing on online marketplaces.
“For a long time, consumers have been asking things like, ‘When can I find a coat that I like by simply snapping a picture of somebody wearing that coat on the subway?’” explains Chiara Farronato, the Glenn and Mary Jane Creamer Associate Professor of Business at HBS, who is a coauthor of the new research. “Companies are finally coming to this realization and adjusting online search tools accordingly.”
The market for AI technology in retail—and the promise of a more intuitive shopping experience—is projected to grow 32 percent a year, topping $85 billion by 2032, according to Fortune Business Insights. Consumers increasingly expect to find the goods they want easily, and a satisfying search feature could give retailers an advantage.
The tool Farronato studied responds to a shopper’s textual search for something general, like “headphones,” with suggested refinements like “noise-cancelling headphones,” aided by images and text.
Farronato wrote the paper with Lu Fang and Zhe Yuan of Zhejiang University; Yanyou Chen of the University of Toronto; and Yitong Wang, an employee at the Chinese ecommerce company that shared its data for the research. Farronato hopes to eventually study the impact of searching solely with images.
Embedded inside a retailer’s new rollout
In 2021, the authors collaborated with one of the largest e-commerce platforms in China as it debuted the text-and-image search tool. To test it, the platform randomly provided text-and-image search refinement suggestions to only half of roughly 500,000 users, chosen randomly.
A customer in the test group searching for the general term “headphones” would receive refinement suggestions, such as “Bluetooth headphones” or “over-ear headphones” with accompanying product images.
The team found that the images nudged shoppers to explore those narrower search queries. Some 5.5 percent of the test group searched for narrower queries suggested by the new refinement tool, compared to 1 percent of the control group, who did not have access to the same picture-text refinements. Those shoppers often stayed on the site—and may even have purchased a product—but did not search for narrower queries.
Six months later, happier customers
The refinement tool didn’t increase purchasing—at first.
“It takes time for the search tool to actually increase search effectiveness.”
Fast forward to six months later, and the results were markedly different: The test group completed 1.6 percent more orders and spent 3.2 percent more on products versus the group that didn’t have access to the new tool. Consumers using the text-and-image tool were also more likely to rate their purchase positively and less likely to return it. In fact, both star ratings and return rates improved by over 3 percent.
“It takes time for the search tool to actually increase search effectiveness in a way that allows both consumers to buy more products conditional on the same search effort and for the platform to better monetize their relationship with the users,” explains Farronato.
Giving less-well-known sellers a boost
Improving search functionality also boosts visibility for less popular products and sellers, particularly those that don’t rank among the top sellers displayed on an online marketplace. Consumers spent as much as 6 percent more on products ranked below the top 1,000 bestsellers when using the refined search. Some of the already popular products also benefit—but often by less or not much at all, effectively helping smaller sellers compete with more prominent ones.
“Now that consumers are able to find more niche products, sellers can start actually expanding their product assortment to include some of them.”
For sellers that don’t typically rank high in searches, more visibility may mean more market power—and potential for less pressure to cut prices in order to compete. For top sellers, that visibility from smaller competitors may mean more vigorous competition.
“So, there may be some adjustments on what kind of prices sellers set, holding constant the products sold,” Farronato says. “In addition, now that consumers are able to find more niche products, sellers can start actually expanding their product assortment to include some of them.”
Branching off for better choice
Farronato likens the refining process to starting off at a tree’s trunk. Guided search using visuals and more specific text helps consumers move quickly along to smaller branches to find exactly what they may want.
“There’s some level of curation that mostly prioritizes top sellers, and top sellers may be coming from, say, one type of headphone. But the moment you start searching for over-ear headphones or Bluetooth headphones, that’s when you start surfacing products that the general search wouldn’t have come up with to the same extent,” Farronato says.
She continues: “You were in the trunk at the beginning. But when you go finer and finer, you end up finding leaves that you wouldn’t have found otherwise.”
You Might Also Like:
Feedback or ideas to share? Email the Working Knowledge team at hbswk@hbs.edu.
Image: Illustration created by HBSWK using images from AdobeStock/avaicon and AdobeStock/Digital Bazaar
“Harvard Business School is the graduate business school of Harvard University, a private research university in Boston, Massachusetts. It is consistently ranked among the top business schools in the world and offers a large full-time MBA program, management-related doctoral programs, and executive education programs.”
Please visit the firm link to site
