Phony stars and false testimonials are rampant online. But is this really a problem, how did we get here, and what responsibility to e-commerce platforms have in addressing the problem? Assistant professor Shreyas Sekar explores what the future of reviews looks like, and why platforms like Amazon should probably take the issue of fake reviews more seriously.
Phony stars and false testimonials are rampant online. But is this really a problem, how did we get here, and what responsibility to e-commerce platforms have in addressing the problem? Assistant professor Shreyas Sekar explores what the future of reviews looks like, and why platforms like Amazon should probably take the issue of fake reviews more seriously.
Show notes:
[0:00] When you see thousands of version of the same product you’re looking to buy online, how do you parse through the options to make a selection? This is where reviews come in.
[0:46] Shreyas Sekar, an assistant professor at the Rotman School of Management/University of Toronto Scarborough studies how consumers make choices online, and how these choices can be manipulated.
[0:59] More than 29 million Canadians made an online purchase in 2022 – and with that comes inevitable fraud.
[1:23] Fake reviews are likely a billion-dollar industry.
[2:08] A quick brief on how Amazon shows you products in your search results, and why results near the top of the search are more likely to get purchased.
[3:31] Reviews play a crucial role in helping sort the rankings, with products with more plentiful and favourable reviews landing higher in the results.
[3:57] Estimates peg phony reviews and algorithm manipulation between four and 40 per cent. Shreyas suspects it’s somewhere in the middle.
[4:14] Why is this a problem?
[5:29] Consumers are likely also buying products on false information, resulting in a substandard experience, likely costing $150 million a year.
[6:07] This is likely to erode trust in the entire online buying ecosystem.
[6:49] So how did we get here? Let’s look first at how we used to shop a few decades ago.
[7:17] Amazon changed the game, first by offering online shopping, but second helping popularize the “marketplace.”
[8:02] What is a marketplace?
[8:44] How did the marketplace create an even greater reliance on reviews?
[9:14] And, in the current system, e-commerce platforms have little incentives to fight this issue.
[9:36] Fake reviews have gotten very sophisticated.
[10:58] So what can consumers and small businesses do?
[12:25] How can AI be utilized to help solve the problem of fake reviews? Shreyas has two suggestions. First, it can create a Coles’ Notes for consumers.
[12:56] Second, it can be used to create some randomness.
[14:11] Why does it even matter? “Algorithms play a huge role in our life, which means that we have to critically examine this pipeline, in terms of the different ways in which humans can manipulate the algorithms to do their bidding.
But when you trick the algorithm, you're not just tricking one consumer, you're tricking 1000s of future consumers, who are going to rely on this algorithm for the purchasing decision.”
Shreyas Sekar: So I'm in the market for wireless earphones. I go to Amazon and search for wireless earphones. I see 1,000s of products that are all extremely identical - they all have the same price range $60 to $120. They all have the same color. They even look almost identical. As a consumer, I have no way of telling these products apart. So there needs to be a concrete way for consumers to identify (one) which are the high quality products and (two) is this product appropriate for the needs that I have in mind. And I think that's where reviews come in.
So you're a seller, you know that reviews are the only way in which you can boost your sales. What do you do? Well, if I'm not getting the real reviews fast enough or at the rate that I want, then I'm going to start thinking about clever ways to outsource people to write five star reviews for me.
Hi so, My name is Shreyas Sekar. I am an assistant professor of operations management at the Rotman School of Management and the University of Toronto, Scarborough. My research focuses on online marketplaces and how people manipulate these different platforms.
Megan Haynes: Canadians spend a lot of their time – and money online. In 2022, more than 29 million people made an online purchase, and we spent more than $3.2 billion with e-commerce platforms.
Across North America, nearly $90 billion in profit flowed through e-commerce transactions, with between 40 and 50 per cent of that going to Amazon.
And with all that money comes the inevitable fraud. But it might be a surprising type of fraud: People aren’t just selling fake goods or shoddy products. They’re peddling in fake stars, phony likes, false testimonials.
Fake reviews run rampant online, and according to the New York Times, it’s a billion-dollar industry that could be costing consumers and small businesses hundreds of millions of dollars each year.
So what exactly are fake reviews, how did we get to this point, and importantly, what can we do about it?
Welcome to the Executive Summary. I’m Megan Haynes, editor of the Rotman Insights Hub.
Musical interlude
MH: Considering the glut of options we’re shown we shop online, let’s start with how an e-commerce platform like Amazon serves you up choice selection when you’re making a purchase.
A retailer will show you the product, and the more prominently it’s displayed, the more likely you are to buy that specific item.
SS: There is this thing called the position effect, where the products that are displayed on the top have a higher chance of being clicked on by the consumers compared to the products that are displayed at the bottom. And this is a no brainer, right. We all have limited time. I'm not going to scroll through all 1,000 pages of search results. I'm going to click on what's at the top two, three positions. If you are located on page two, three, four or five, it's pretty much a death sentence for you.
MH: How an e-commerce platform determines what you see is based on algorithms. These algorithms are proprietary, but people do have a generally decent idea of how they sort data. We know that purchase volume – how many people are buying a product – will help that item climb the ranks. We know that clicks and consideration – how many people looked at a product and put it in their cart, even if they didn’t buy it, will help it climb the ranks.
We know the algorithm looks at similar consumer behaviour – say you have two customers who traditionally share similar purchase patterns: the algorithm knows that if person A buys one brand, person B is likely to buy the same brand.
But importantly, reviews play a big role in the algorithm. The better a product is reviewed, both in terms of volume and stars, the better that product’s chances are landing high in the search results. And this is where the problem comes in.
SS: This leads to something called data manipulation or data corruption. If the data that you're using is not reliable, then the results of this algorithm in other words, the algorithm that you use to push consumers to specific products are not gonna be reliable.
MH: Watchdog organizations like FakeSpot or Review Meta estimate that upwards of 40 per cent of reviews on platforms like Amazon are phony; Amazon says the number is closer to one to four per cent.
SS: Now, I don't have a magic eight ball that tells me what the right number is. But I suspect it's somewhere in the middle.
MH: That’s a problem, since between 80 to 90 per cent of North Americans have said they use reviews in helping them make a decision before a purchase, which means we’re increasingly relying on false information when deciding to buy.
And because people are increasingly turning to reviews in decision-making, it also means platforms are increasingly prioritizing that metric when building their algorithms, creating a negative cycle that’s harder to break.
SS: Unfortunately, the way the search ranking and recommendation algorithms work is they constantly reward the products that are doing well in the marketplace. In other words, that is a rich get richer effect where the products are getting lots of reviews are then recommended to more consumers, and then they end up getting more reviews again and again and again, and the cycle continues.
MH: Many small business owners also end up suffering. Even if they want to be ethical and let their great product stand on its own, many feel they have to pay to play the fake review game or risk being deprioritized in the search results, making getting the legitimate reviews they need to grow their businesses harder.
SS: Remember that fake reviews have just become table stakes, because now if everyone around you is getting fake reviews, if everyone around you is playing dirty, then you are being forced to play dirty.
MH: So what’s the big deal? Well, effectively, consumers are being deceived, which leads them to buying sub-standard products.
A recent study by researchers in the UK and California estimate that these phony testimonials likely cost consumers and businesses more than $150 million a year in terms of lost revenue due to returns or faultily made products.
SS: For every dollar that consumers spend on e-commerce platforms, fake reviews cause at least a 12 cent or 12 per cent loss and consumer welfare. In other words, by purchasing substandard products, you're losing out on 12 per cent of the experience that you would have gotten had you purchased a much better product.
MH: Longer-term, fake reviews are likely to erode trust.
SS: In the short term, this will probably increase the sales on the platform because there will be lots and lots of reviews and lots of hot products. Great. In the long term, though, remember that reviews are one of the main reasons why we rely on online marketplaces in the first place. As the review systems become unreliable, I'm going to stop relying on these reviews. And I'm going to think, '"Okay, it's not worth the trouble for me to spend hours and hours sifting through these reviews, trying to identify what's good and what's bad. I might as well just go to the store and purchase what I like."
Musical Interlude
MH: So how did we get here? Let’s go back in time – say the late ‘90s to early 2000s - and recall how we used to shop for products before the dawn of e-commerce.
SS: When you think of traditional retailers where the retailer carefully curates an assortment of products for you to see at the physical store. Now, as a consumer, your choice is simple. I go to the store, I see maybe three, four, 10 boxes of cereals. And I pick the one that I like, Great, I'm done.
MH: Amazon and the dot-com bubble ushered in a new era of shopping. We were no longer tied to a single bricks-and-mortar location; suddenly you could order a product from halfway around the world. Through the 2000s and 2010s, the share of e-commerce business grew at a steady clip. In 2007, e-commerce made up about 3.2 per cent of all retail purchases in Canada. By 2018, e-commerce jumped to about four per cent as a share of all retail. Enter the pandemic, and e-commerce skyrocketed, making up nearly 11 per cent of all retail sales in 2020.
And, as we all shifted at least some of our shopping online, Amazon was building its marketplace, a spot where third parties could sell their wares, rather than Amazon being the curator.
SS: Originally, a marketplace is just providing you with a shopping mall, right. It's renting the shopping mall to thousands and thousands of sellers who can all operate their little stores on this marketplace. Now, this is a much better model because Amazon is outsourcing all the risk and all the supply decisions to these individual sellers. And on top of that, they get to take a 15 to 20 to 25 per cent cut of every purchase that happens on the platform.
MH: By 2017, Amazon announced that for the first time, more than half of its sales were coming from third-party vendors, and since then other retailers including Walmart, Best Buy and the Bay have jumped on the marketplace trend in a big way. But all this extra choice creates a problem.
SS: So as a consumer, you're now looking at a million different people who are all selling the same thing. And you need a concrete way to identify which products are the best ones for whatever tasks that you have in mind. And I think this is the backdrop against which reviews are really started playing a huge role.
MH: Not only are reviews integral to e-commerce search algorithms, but products with better and more plentiful reviews are also more likely to be purchased. A five-star rating can boost sales between five and 10 per cent.
And since Amazon gets a cut from every purchase, what incentive is there to look into an item’s unexpected or sudden jump in stars? Well, until recently, very little. Fighting these reviews is costly, and it takes Amazon about 100 days to remove a phony one. Retailers don’t want to take down valid reviews, and these fakes have gotten very sophisticated.
SS: They can start mimicking real reviews, they can start using personal anecdotes to make your reviews seem authentic. They can even have like a mostly positive review with a few negative parts sprinkled in, just so that it mimics the shape and function of a real review.
MH: A 2020 paper [published in 2022] exposed a number of Facebook groups where merchants could connect with people offering up fake reviews in exchange for a few dollars.
The reviewer would purchase the product through amazon, leave a review as a quote-unquote verified purchaser, then return the product through back-channels for a refund and a little extra cash on top.
Since the paper’s release, Amazon, Facebook and the FTC in the US put in a concerted effort to shut these groups down; unfortunately, it’s likely just gone underground on private WhatsApp chats and discord groups. Shreyas says to truly tackle the fake review issue, it would likely cost retailers millions, and in the short term, the cost of fighting these problems might not be worthwhile for these e-commerce platforms – especially since they are still getting their pay-day.
SS: However, in the long term, I think this can be disastrous for the platform.
Musical interlude
MH: So what can we do about this issue?
Consumers can be more vigilant when it comes to product selection, though Shreyas acknowledges that a number of studies have shown that shoppers are really bad at recognizing fake reviews.
SS: So (one) I would suggest is to be critical and try to read between the lines So for example, when you read a review which is overwhelmingly positive, try to think about what is it the review is not touching on. (Two), before you purchase a have a checklist, try to think about what the pros or what the cons are, and try to come up with a more balanced assessment of each individual product. And (three) I think it's okay to hedge your bets. Don't just rely on reviews, but also try to look at other sources.
MH: Consumers should be aware that – since reviews are so integral to climbing the search ranking, newer products tend to attract more fakes. And there is likely a “sweet spot” where fake reviews likely thrive – probably within the $30 to $500 range.
SS: If you're too cheap, it's not worth the investment to get fake reviews. If you're too expensive, consumers are usually more circumspect in their purchase, so fake reviews may not be as impactful.
MH: Small businesses, meanwhile, should not be overly reliant on a single platform like Amazon.
Shreyas strongly recommends building communities around products and making sure they’re available on multiple platforms – like a digital shopfront through Shopify.
He also says it’s important that small businesses continue to exert pressure on the bigger retailers to address this issue.
But ultimately, the onus is on Amazon and other e-commerce platforms to take up this fight against fakes.
To Amazon’s credit, Shreyas says it’s exploring how AI can create a better review structure. It’s piloting an AI that will comb through the thousands of product reviews and provide a Coles' Notes version of what they say. This shifts the focus from the volume of reviews or the star rating
SS: It's about helping consumers find the right product, and trying to identify the pros and cons of each product that they're looking at.
MH: The second option Shreyas likes is to introduce some randomness to the order in which products are presented. Rather than rely on a single algorithm to serve up products to customers – as Amazon currently does – he suggests that it uses three or four algorithms. And each time you visit the website, you’re assigned a new one at random.
SS: Each algorithm is kind of finely trained on a different set of consumers. So what then happens is that these algorithms have somewhat diverse performance measures. And these algorithms may end up promoting very different products. Because of running not one, but many different algorithms, I don't just promote one type of product, I can have a more heterogeneous experience in how my consumers are treated. Two, because I choose between these algorithms at random, this means that it's not that easy for fake reviewers to manipulate the algorithm directly. They can only try their best. If you're using four different algorithms to randomize, then the fake review are only 1/4 as effective as they were before.
MH: Not only does this make it a little harder for reviewers to game the system, it can actually be very beneficial to companies like Amazon. Studies have shown that a bit of variety in our product selection helps increase the likelihood of a purchase.
In the short term, it might feel like this isn’t an issue worth tackling – so what if you bought an iffy toaster if it still works?
Shreyas warns however there is a longer-term risk to our buying ecosystem, so the issue of fake reviews – and algorithm manipulation - is worth dealing with.
SS: I think to me, personally, it's kind of a travesty that reviews have become so unreliable. Because reviews are such a fundamentally democratic mechanism that I think generally benefits society as a whole, right? Like in the absence of fake reviews, you know, reviews really do work. In other words, they help consumers find good products, they help spotlight good products. And they help smaller sellers compete against the bigger sellers.
Algorithms play a huge role in our life, which means that we have to critically examine this pipeline, in terms of the different ways in which humans can manipulate the algorithms to do their bidding.
But when you trick the algorithm, you're not just tricking one consumer, you're tricking 1000s of future consumers, who are gonna rely on this algorithm for the purchasing decision.
Musical outro
MH: This has been Rotman Executive Summary, a podcast bringing you the latest insights and innovative thinking from Canada's leading business school.
Special thanks to Assistant Professor Shreyas Sekar. Join us in a few weeks as we chat with Assistant Professor Avni Shah about reframing how we think about saving for retirement.
This episode was written and produced by Megan Haynes. It was recorded by Dan Mazzotta, and edited by Avery Moore Kloss.
For more innovative thinking, head over to the Rotman Insights Hub, and subscribe to this podcast on Spotify, Apple or Google Podcasts.
Thanks for tuning in.