Available on the Awesome plan for Early user program participants
The Optimize collection tool lets you test a variant of your review request email against your current version. Judge.me splits your outgoing emails between the two versions and automatically identifies a winner based on which one collects more reviews.
Considerations
Currently, Optimize collection does not work with Assign templates per product. Make sure to disable Send a different review request email per product before running the optimization.
Only one optimization can run at a time. Stop or apply a running optimization before starting a new one.
Optimizations test your initial review request email (not reminders).
Your review request emails must be turned on and use the Smart email template.
If you send review requests through a third-party integration like Shopify Flow or Mailchimp, the optimization feature is not available.
If you have more than one active review request email for the same language, you need to keep only one active before starting an optimization.
The legacy A/B testing and the Optimize collection features cannot run at the same time.
When the optimization is in progress, the report takes around 10 minutes to update after a new review is submitted.
How it works
When you start an optimization, Judge.me creates two versions of your initial review request email:
Default — your current review request email, unchanged.
Variant — a modified version with different subject, title, or content that you create.
Judge.me then splits your outgoing review request emails evenly between the two versions. As orders come in and emails are sent, the tool tracks how many reviews each version collects. When one version is statistically better, it's flagged as the Winner.
You can then apply the winning version to make it your permanent review request email, or stop the test at any time.
How to set it up
Start a new optimization
From your Judge.me admin, go to Settings > Optimize collection.
Under Start a new optimization, click Email content.
Under Start your variant from, choose a starting point for your variant:
Branded — a professional tone focused on your brand.
Conversational — a friendly, casual tone.
Personalized — a tone that speaks directly to the customer.
A copy of the default — starts with your current email so you can make smaller tweaks.
Edit the Subject, Title, and Content fields for your variant. The Preview panel on the right shows how the email will look.
Click Start optimization.
If your store supports multiple languages, use the language selector at the top to choose which language's email to optimize. Each language can be optimized separately.
Monitor your optimization
While an optimization is running, the Optimization in progress card shows:
Split — the percentage of emails each version is receiving.
Reviews collected — how many reviews each version has collected so far.
When one version is statistically better, a Winner badge appears next to it.
Apply or stop
You can end an optimization in two ways:
Apply — click Apply next to the version you want to keep. That version becomes your live review request email and starts sending to all new orders. The optimization moves to your archive.
Stop — click the ⋮ menu and select Stop. This ends the test without changing your email. Your original email continues sending as before.
Store metrics
At the top of the Optimize collection page, the Store metrics card gives you an overview of your store's performance:
Reviews — the number of reviews collected.
Orders — the number of orders that had review requests sent.
Reviews / Order — the ratio of reviews to orders, shown as a percentage.
Use the date picker to change the time period. Each metric also shows a comparison with the previous period. Store metrics are available on all plans.
Archive
Past optimizations appear in the Archive section at the bottom of the page. Each entry shows whether the optimization was completed or aborted, the date it ended, and which version was applied. Click the expand arrow to see the per-version review counts.
Resources
Get your free trial:
✨ Unlock Optimize collection to test different versions of your review request email and find the message that collects more reviews. Start your 15-day free trial today, and get access to all Judge.me features!
Join the Early user program
🚀 Optimize collection is currently available to Early user program participants. Join the program and get early access to new features before they’re available to everyone.
See our latest updates:
➡️ Check our Unboxed page to discover newly launched features and improvements from Judge.me. See what’s new and start using the latest updates to grow your store.