💎 Growth Gems #98 - Monetization: Pricing experiments
Hey,
This week, I’m sharing gems on pricing experiments: when to test, defining success, alternatives, and more.
These insights come from Jake Mor, Keya Patel, David Barnard, and more!
Did I get carried away? Maybe 🤷
Read on the web so you don’t miss important insights.
Enjoy!
🥇 TOP GEM OF THE WEEK
Pricing experiments: when to test, defining success, personalization and discounts, and more
Gems from Jake Mor (Co-founder at Superwall), Keya Patel (Product Strategy Consultant, ex-Headspace), and David Barnard (Growth Advocate at RevenueCat) in Mastering mobile pricing experiments: Proven tactics for revenue growth
Who doesn’t want more revenue from their app? 💰
Improving monetization not only means more revenue per customer, it often also means unlocking acquisition volume.
One of the most common ways to improve monetization for a subscription app?
Pricing experiments. One of my favorite topics!
I had already shared some great subscription monetization insights from Jake (Growth Gems #65), and I recently enjoyed another Sub Club podcast episode with Keya.
So I got very excited when I saw the announcement for this Mastering mobile pricing experiments webinar from RevenueCat, and I even texted David to get the video ASAP…
Below are my favorite gems from this 65-minute webinar, along with comments and examples!
When to price test
💎 It’s important to price-test regularly: every six months or at least every year. So much changes in terms of your users’ psychology, what’s happening in the market, competitor apps, etc.
(05:58) by Keya Patel (Product Strategy Consultant, ex-Headspace)
Are you neglecting price tests? This quote by Naomi Ionita (Partner at Menlo-Ventures), shared in Growth Gems #82, is a good reminder: “Think about your pricing just like you do your roadmap”.
On the other hand, there is such a thing as over-indexing in pricing experiments. Move on to other areas when you start seeing diminishing returns (cf., the framework I shared in Growth Gems #92).
💎 To be able to detect the lift in revenue when running a price experiment, you should be at least around the 1k-5k start trials a month.
(08:43) by Jake Mor (Co-founder at Superwall)
The point is that there is a threshold to reach before it makes sense to price test. If you haven’t reached it, focus on radical tests (without A/B testing.
Keya explained that if you’re early stage, you should start by anchoring your price to your competitors’ or similar apps’ average price. Once you get bigger and have enough experimentation sample size, THEN you can test raising your price (but grandfather early bird users in!).
💎 If you have just launched, you should work on your product until your subscription retention flatlines (which indicates you’ve built a retentive product) before starting pricing experiments.
(09:55) by Jake Mor (Co-founder at Superwall)
I love this advice. Jake also mentioned he prefers working on pricing once it’s a low-hanging fruit.
Does it mean you need to wait more than a year? No. Well, maybe, but you could also start by offering only a monthly plan to help you see how people churn first (this is one of my favorite gems from Matthieu Rouif - CEO at PhotoRoom, shared in Growth Gems #53).
The downside of having only a monthly plan at first is less cash upfront than if you also had a yearly, which might make running paid acquisition campaigns challenging.
Before you get started (defining success)
💎 Things you need before running pricing experiments:
Understand which metrics you’re optimizing for: conversions, revenue, retention
Be on the same page internally on how you will make the decisions
Make sure that you have tooling allowing you to run experiments properly so you’re able to make the important decisions needed
Have enough sample size
(14:40) by Keya Patel (Product Strategy Consultant, ex-Headspace)
💎 Assigning a value to a free-retained user is very important. What is a user staying around without paying worth? This is particularly relevant for apps with a social component.
(17:07) by Jake Mor (Co-founder at Superwall)
Why social apps? Because they grow off a K-factor of organic growth, if D7 drops, it means less revenue.
If you’re in this case, then you’ll need to keep this as an additional input to your decision in addition to:
If you’re not A/B testing: the realized LTV per Customer comparison of the tested cohort vs. the previous period.
If you’re A/B testing: the realized LTV per Customer of each variant (if you’re A/B testing)
I love this feature. Now we just want a more extended timeframe so we can watch results over at least a year #featurerequest
If you have yearly plans in the mix, how can you compare? Glad you asked 👇
💎 You need indicators of success so you can predict what LTV might be. Example: track the “opt-out” rate (turn auto-renew off) within 24 hours to get a sense of how much the renewals will drop.
(26:25) by Jake Mor (Co-founder at Superwall)
If you’re A/B testing with yearly plans, looking at the volume and distribution of people turning auto-renew off is an interesting proxy to help with decisions.
David shared that according to RevenueCat data, many annual subscriptions that end up churning are subscriptions where auto-renew was turned off in the first month.
💎 Be very clear about your success metrics. Example:
1. Conversion rate so you can see immediate revenue and predict revenue.
2. Early signs of engagement or turning auto-renew off.(29:13) by Jake Mor (Co-founder at Superwall)
💎 Price A/B testing is almost a myth because there are never any constants, and nothing is ever exactly the same. It’s really multi-variant testing, where you change only one variable and then look at how much LTV goes up.
(22:55) by Jake Mor (Co-founder at Superwall)
Plus, Jake mentioned that billing issues and refunds can also get into play.
Seasonality may be another significant variable: a higher price might get you better results when there’s a higher demand, but it might not be the case the rest of the year.
This doesn’t mean you should not price tests, but it’s a good context to remember.
💎 When price testing, you need to decide if you’re optimizing more for retention or return on ad spend. Are you sacrificing things that will matter to the business 1 or 2 years from now?
(22:55) by David Barnard (Growth Advocate at RevenueCat)
Personalization and discounts
💎 Personalize pricing over trying to find a better price for everyone (finding an average). Use days since install to define when and how much to discount (e.g., a discount 5 days after install or when they close the paywall).
(24:20) by Jake Mor (Co-founder at Superwall)
Below is an example from Blinkist, shared last week on LinkedIn by Rosie Hoggmascall. While I believe they’re leaving money on the table by discounting so much, it’s a good way to understand the kind of rules you can implement.
💎 For users who did not convert immediately, you can use the opportunity to try to get the conversion to happen over the web after a certain amount of time.
(25:17) by Keya Patel (Product Strategy Consultant, ex-Headspace)
For several reasons (one being App Store fees), users acquired on the web tend to have a higher LTV. However, discounting is not a big enough reason to implement web purchases or web-to-app (here is a list of web-to-app considerations from Growth Gems #96).
💎 You have to make customers understand that every time they see your paywall, it’s the last time: it’s then or never again. You can also A/B test the 2nd paywall shown: start by understanding your conversion rate on that 2nd paywall view and test your hypotheses.
(51:21) by Jake Mor (Co-founder at Superwall)
Below are some examples of one-time offers leveraging the “last chance” effect (#2 and #3 courtesy of Marcus Burke and Steve P. Young).
However, it’s a balance to strike. Some apps don’t hesitate to make “best prices ever” claims that are untrue…
💎 Use the best price possible for the first paywall, but you can also A/B test the price shown on the 2nd paywall view: Start by understanding your conversion rate on that 2nd paywall view, and form a hypothesis based on that.
(51:35) by Jake Mor (Co-founder at Superwall)
💎 Send an email to everyone that is looking at the paywall. People with notifications turned on for their email app would see the email arrive at the top of the app: “We have a 30% off discount on our website right now”.
(56:40) by Jake Mor (Co-founder at Superwall)
I’ve never seen an example of this, but I like the idea. Even if people don’t have notifications turned on for their email app, they might still check out the email and convert on the web.
💎 There are other ways than heavy discounts to thank engaged customers and get them to renew: think about non-monetary benefits (content, thank you notes) that they could appreciate. Example: At Headspace, they’ve considered what it means to be an engaged user. Instead of giving a discount to users to thank them for sticking around one year or two years, they looked into offering specific meditation webinars with instructor/content users really like, thank you note from the founder, etc.
(34:25) by Keya Patel (Product Strategy Consultant, ex-Headspace)
Keya mentioned this in the context of retaining users. Still, you could also imagine those non-monetary benefits as incentives for users to go premium or to choose a longer plan.
💎 In 2020, Headspace started offering a subscription to people who got laid off. They hypothesized that mentioning this to other potential subscribers would make those with more disposable income more willing to pay for a subscription (and it worked) because they would contribute to a “social goods” initiative.
(38:08) by Keya Patel (Product Strategy Consultant, ex-Headspace)
Testing a new tier
💎 A lot of apps are failing to charge their best subscribers more. Some people are probably willing to pay triple.
(39:15) by Jake Mor (Co-founder at Superwall)
💎 To test a new tier, you do not need to change any business logic.
Display the new higher price point on the paywall, along with the previous one. Say that the higher price point includes all features, and the lower price point includes only some of them
Give access to everything for both tiers
Tag the entry points from each locked feature that triggers the paywall so you can see in your analytics which ones have more demand and are leading to the most revenue
(42:30) by Jake Mor (Co-founder at Superwall)
Jake gave this example: Imagine you have a fitness app with $5/month and $30/year on the paywall. Remove the monthly price, and instead add a $59.99/year as a higher tier with more features (including a “personalization feature”). If the “personalization feature” is the one that leads to more people converting to the $59.99/year plan on the paywall, then it should be part of the higher tier. Compare each premium feature this way to be able to rank them.
If you’re in a big organization, Jake recommends finding single contributors and giving them the autonomy to launch experiments like these.
Essentially, this is the quantitative complement to the “rank feature” qualitative analysis you could do in a willingness to pay research.
Only a few apps have multiple tiers. Citizen recently added one, and it was actually a lower tier (no free trial). A smart way to monetize a higher percentage of users? You can then play around with which plans you push to which users.
Brand challenges (or not)
💎 Doing price tests is not as big of a deal as you might think regarding brand and customer experience. At Headspace, there were less than a handful of people who were noticing price experiments were going on. It’s the larger seasonal campaigns with advertising that people would notice (e.g., through small Reddit threads). If someone complains about something via support, give them the best possible experience: explain you run experiments, match the price they’ve seen, etc.
(46:05) by Keya Patel (Product Strategy Consultant, ex-Headspace)
💎 Multiple prices showing in the App Store subscriptions is a non-issue. Developers always think it’s a big deal, but it’s not.
(25:17) by Jake Mor (Co-founder at Superwall)
Keya mentioned that customer support would let customers know those prices were deprecated. Plus, for the prices you’re not currently using for “price sequencing” (discounts over time or usage), you can increase the prices for new subscribers so they at least show with the same price in “See all plans”.
💎 If your app is big in specific communities, you can make the price raise an event: let the community know that the price raise is coming in X months, and you’ll grandfather in users, etc., and put them in the know. It will bring revenue and might bring retention in the long run.
(25:17) by Jake Mor (Co-founder at Superwall)
Most of the time, you should increase prices for new subscribers only (i.e., “grandfather in” existing subscribers). But for some verticals, and in some cases, it makes sense to raise prices for everyone. Here is a good article on the topic in Growth Croissant by Reid DeRamus (it includes a Napkin Math for Price Increases spreadsheet).
🎁 Bonus Gems
Here are some more insights you might find helpful on the price experiment topic!
💎 It is so underrated how hard the math is to figure out which prices work best. You can not run one test in June and the other in July because you won’t be comparing apples to apples. The best way to test prices is to choose a month where everything is as constant as possible (paywalls, no promotion emails) and do as many tests as possible. Then, check every 6 months to see how each cohort’s performance matures.
(50:55) by Jake Mor (Co-founder at Superwall) in Mobile Paywall Optimization: How to Increase In-App Subscription
Jake’s right that things always change, and comparing tests at different times might be unfair. At the same time, though:
Picking a month means you’re not considering annual seasonality
No promotional emails means that you’re not considering an important factor: a higher price might stop some people from purchasing initially, but you might be able to convert them with a promotion later on. Too many people look only at their in-app purchases when running tests, but you need a more holistic approach (web + app).
There’s no perfect solution, so it’s safer to roll out a test when it looks like a clear win (not just a couple of percentage points).
And keep monitoring cohorts! In the video, Jake made the analogy of “planting seeds” and then watching the trees grow over time (e.g., every 6 months).
💎 People that fit different use cases have different willingness to pay. So, before optimizing your paywall and prices, you need to understand your users' use case(s). Example: some Calm users want to sleep better, while others want to meditate.
(04:59) by Vahe Baghdasaryan (Sr. Growth Marketing Manager at CoinStats) in Subscription Optimization & Pricing Strategy
This goes back to the personalization and tiers topic: if you go with “one size fits all”, you’re optimizing for the biggest minority. It’s a great place to start, but you might want to explore further as your app matures.
💎 There is a ton of value in being able to price-discriminate in a way that allows you to touch as many points of the price sensitivity curve as possible to drive more revenue. This can be done through personalizing prices and/or breaking down the purchase experience into smaller bits.
(11:28) by Mehmet Yavuz Açıkalın (Head of Data, Games at Forte) in Building Offers Your Customers Can't Refuse
The gaming world is far more advanced in the monetization topic. The webinar above is a good introduction. I “mined” it for paid subscribers a while ago; reply if you want me to share the gems with you.
💎 Your prices don’t exist in isolation: people always compare them to make decisions. Example (on the extreme side): the infamous Wall Street Journal and The Economist “decoy” plans.
(15:53) by Sree Das in Product Pricing Principles by fmr Disney Product Manager
On the less extreme side (not just decoys), people will compare your plans (e.g., monthly, quarterly, monthly), and there are multiple ways to leverage this to influence their decision.
And before I leave, one of my favorite quotes that emphasizes the importance of monetization (again, from Thomas):
“UA is a business model competition” - Thomas Petit (Growth Consultant)
See you next time. Stay curious!
⛏️ Sylvain
🔗 Sources:
Mastering mobile pricing experiments: Proven tactics for revenue growth, a webinar by RevenueCat
Mobile Paywall Optimization: How to Increase In-App Subscription, an AppMasters episode
Subscription Optimization & Pricing Strategy, an AppMasters episode
Building Offers Your Customers Can't Refuse, talk at Mobile Growth Summit Deep Dives
Product Pricing Principles by fmr Disney Product Manager, a Product School webinar