Do Popups Affect Conversion Rates? What Pact's A/B Test Revealed
Pact, working with its retention agency New Standard, ran a controlled A/B test to answer the industry's biggest question about popups—and the results may surprise you.
Popups are one of the most debated elements in ecommerce. While some teams find them essential for retention, others worry they distract shoppers and hurt conversions.
Due to this, many brands limit or disable popups in high-intent moments, often without directly testing the impact.
At the same time, not every visitor converts in a single session. When popups are removed, non-converting visitors leave anonymously, and potential future sales are lost.
We spoke to Daniel Guerra, Head of Data at New Standard (a boutique agency focused on ecommerce retention), on how popups are perceived in the industry, what most brands get wrong, and how they should be approached differently as part of a broader conversion and retention strategy.
Popups don't hurt conversion when done right
Many ecommerce brands disable popups to protect conversion, assuming they distract high-intent shoppers.
This creates a hidden cost: visitors who don't convert immediately leave anonymously, limiting future retention and revenue.
Pact, working with its retention agency New Standard, ran an A/B test to see whether popups actually hurt conversion.
Results showed no conversion downside:
- Add-to-cart rates were nearly identical (9.44% with popups vs. 9.32% without).
- Purchase rates held steady (1.75% with popups vs. 1.76% without).
At the same time, popups captured additional email and SMS signups from visitors who would have otherwise left.
The takeaway is that when implemented passively and thoughtfully, popups don't disrupt conversion. They preserve future demand.
DECODING THE INDUSTRY BELIEF
That Popups Disrupt Conversion
Popups are often seen as a source of friction rather than value. Many teams believe they interrupt focus, pull shoppers away from the buying journey, and make it harder for site visitors to complete a purchase.
"So the fear from the brand is, if we have a popup, it's disruptive and instead of getting people to purchase on the spot, we either scare them away because we're showing them a popup and they're annoyed or they going to enter their email and we're going to lose the ability to convert them right on the spot and so ultimately it's going to drop conversion."
As a result, many brands treat popups as incompatible with conversion-focused experiences.
Instead of testing different approaches, popups are often turned off entirely during high-intent moments, based on the assumption that removing them is the safest way to protect conversion rates.
WHY DISABLING POPUPS
Can Hurt Your Performance
Without popups, brands lose the chance to engage visitors who don't convert right away. Many teams make this tradeoff to protect short-term conversion, without considering what's lost when those visitors leave with no way to re-engage them.
"According to us [retention specialists], popups are a holy grail as a tool to leverage our ability to collect more emails. It's probably one of the first things we do and a thing that we constantly optimize is the popup experience to make sure that we're collecting emails and creating value for every visitor, not just potential purchasers."
Not every interested visitor is ready to buy, but many are willing to start a relationship. When popups are disabled, those visitors leave anonymously. Even though the brand has already invested in bringing them to the site, it loses the chance to follow up or continue the conversation later.
Disabling popups doesn't simply remove friction. It removes optionality. It shifts all value creation onto a single session and narrows the brand's ability to convert interest over time. This is most risky for brands that rely on lifecycle channels, where conversion often happens after the first visit.
At the same time, popups don't need to hurt conversion. When they offer value in a simple, frictionless way (like auto-applying a discount at checkout), they can actually make it easier for first-time shoppers to complete a purchase.
TEST DESIGN
And Controls
To isolate the impact of popups, New Standard ran a controlled test on Pact's site, focused specifically on non-interference with conversion.
The test followed a few key principles:
Popups were shown only to traffic coming from ads.
No changes were made to ads, landing pages, pricing, or checkout.
Popups were passive, behavior-based, and easy to dismiss.
Popups never appeared during checkout or blocked purchase actions.
No aggressive or conflicting offers were introduced.
The goal wasn't to push purchases, but to allow lead capture only when it didn't compete with conversion.
The people who viewed the page and stayed on the page was actually higher for people who had a popup.
Rather than disrupting conversion, the popup created a second path—capturing future demand without taking attention away from buyers who were ready to purchase.
What the Data Shows
When popups are passive and well-timed, they do not disrupt conversion or checkout behavior.
The expected tradeoff between protecting conversion and capturing leads did not show up in practice.
Popups captured value from visitors who were unlikely to convert in that session, without pulling attention away from active buyers.
When done well, popups act as a retention bridge—preserving future demand instead of competing with acquisition.
Visitors ready to convert
"Minimal overlap"
Visitors open to signing up
HOW DTC BRANDS CAN
Approach Popups Differently
Many DTC brands still think of popups as a conversion lever. It either helps a purchase or hurts it in the moment. Pact's test suggests a different approach. Instead of focusing only on immediate conversion, popups can help preserve demand and support future retention.
Consumers expect popups, and in many cases, they actively look for them.
"Consumers are savvy and oftentimes wait for some type of offer with a popup. It's no longer a disruptive experience to their purchasing behavior."
For many visitors, a popup isn't a distraction. It's a signal that there may be additional value available if they're not ready to convert immediately.
Value doesn't have to mean a price cut. What matters more is whether the popup engages the consumer in a way that feels relevant and worth responding to.
"They're really not speaking to the consumer in a way that they want to be spoken to, and they're actually not providing value. And by value, I don't necessarily mean discount. I just think it needs to be something that entices the consumer to ultimately make the purchase."
Popups shouldn't push urgency. They should extend the conversation.
Education, reassurance, exclusivity, or well-timed incentives can all create value—depending on the shopper.
Pact's test reinforced that popups should not compete with the purchase moment. Instead, they work best when shown only after a visitor has shown some level of engagement.
This is where Smart Triggering becomes important. Instead of showing popups on page load, Smart Triggering uses engagement signals—like scroll depth, time on page, or exit intent—to decide when a popup makes sense.
Here's what this looks like in practice:
Never trigger popups at checkout.
Trigger based on engagement or intent, not arrival.
Make popups easy to dismiss.
When popups are passive, behavior-based, and optional, they create a second path for interested visitors without getting in the way of buyers ready to convert.
One of the biggest risks brands take is treating popups as binary. It's either on or off.
"No popup in my opinion, is simply just sacrificing your ability to communicate with the consumer in the future. It should always be testing some sort of value offer, messaging, creative, etc."
In reality, popups are one of the most flexible touchpoints to test messaging, value propositions, and creative.
There may be cases where popups don't add value. But disabling them should be a conclusion reached after testing, not the default starting point.
"If it's simply not resonating, then maybe you are in a vertical or an industry or a brand that maybe it's not going to be valuable. But I certainly wouldn't start with that test [of not having a popup onsite]. I would put that last."
The absence of a popup should be a conclusion reached after testing to understand what works—not a default assumption
The idea that popups automatically hurt conversion is overstated. Pact's test shows that when popups are passive, easy to dismiss, and kept out of checkout, they don't interfere with conversion. Instead, they help you capture interest that would otherwise be lost.
If you're rethinking how popups fit into your conversion and retention strategy, Alia supports this exact approach with behavior-based popups designed to preserve long-term value.
Your Success Story
Starts Here
Sit down with an expert to figure out how your brand can significantly increase conversions, sales, and signups with a custom Alia demo.
.webp)