A/B testing for festivals - groups and progressive sends
Groups & A/B Testing
Section titled “Groups & A/B Testing”A/B testing for festival campaigns: the Groups function lets you split attendees into equal parts for testing subject lines, early bird offers and progressive sends. Optimise ticket sale conversion with scientific campaign testing.
What Are Groups Used For?
Section titled “What Are Groups Used For?”1. A/B Testing
Section titled “1. A/B Testing”Split your audience to test different versions and learn what works best.
Example:
2. Progressive Sends (Deliverability)
Section titled “2. Progressive Sends (Deliverability)”Improve your sender reputation by sending in small batches.
Example:
3. Offer Testing
Section titled “3. Offer Testing”Test different incentives with small groups before launching at scale.
Example:
How Groups Work
Section titled “How Groups Work”Key Characteristics
Section titled “Key Characteristics”✅ Consistent: A fan will always be in the same group ✅ Random: Fans are distributed randomly ✅ Balanced: Each group has approximately the same size ✅ Deterministic: Based on a hash of the user_id (does not change over time)
Example:
This is crucial for consistency in long-term A/B tests.
Group Range
Section titled “Group Range”You can split your segment into 1 to 100 groups.
How many groups should I create for A/B testing festival campaigns?
Section titled “How many groups should I create for A/B testing festival campaigns?”| Groups | Size per Group | Typical Use |
|---|---|---|
| 2 groups | 50% each | Classic A/B testing |
| 3-5 groups | 20-33% each | Multi-variant testing |
| 10 groups | 10% each | Weekly progressive sends |
| 20 groups | 5% each | Daily progressive sends |
| 100 groups | 1% each | Granular testing / very gradual rollouts |
Practical Use Cases
Section titled “Practical Use Cases”Case 1: A/B Test of Subject Lines
Section titled “Case 1: A/B Test of Subject Lines”Objective: Discover which subject line generates the best open rate.
Setup:
Measurement:
- Group 1: Open rate 42%, Click rate 12%
- Group 2: Open rate 31%, Click rate 9%
Decision: Group 1 subject is the winner → Use in future campaigns.
Case 2: Progressive Send Over 10 Days
Section titled “Case 2: Progressive Send Over 10 Days”Objective: Send a campaign to 30,000 fans over 10 days for better deliverability.
Setup:
Benefits:
- ✅ Better sender reputation
- ✅ Lower probability of spam folder
- ✅ You can adjust the message based on early results
- ✅ Reduces server load
Case 3: Price Test Across 4 Variants
Section titled “Case 3: Price Test Across 4 Variants”Objective: Find the optimal price for early bird.
Setup:
Measurement (example):
| Group | Price | Conversion | Total Revenue | Revenue/Fan |
|---|---|---|---|---|
| 1 | €50 | 15% (75 sales) | €3,750 | €7.50 |
| 2 | €45 | 22% (110 sales) | €4,950 | €9.90 ✅ |
| 3 | €40 | 28% (140 sales) | €5,600 | €11.20 |
| 4 | €35 | 35% (175 sales) | €6,125 | €12.25 |
Decision:
- If maximising revenue/fan: Group 4 (€35) is the winner
- If you need volume: Also Group 4
- If you want a profit/volume balance: Group 3 (€40)
Case 4: Gradual Rollout of a New Feature
Section titled “Case 4: Gradual Rollout of a New Feature”Objective: Launch a new mobile app feature gradually.
Setup:
Benefits:
- ✅ You catch bugs with a small audience first
- ✅ You reduce the impact of critical issues
- ✅ You can iterate based on early adopter feedback
Best Practices with Groups
Section titled “Best Practices with Groups”1. Minimum Size per Group
Section titled “1. Minimum Size per Group”For statistically significant results:
What is the minimum audience size for A/B testing festival emails?
Section titled “What is the minimum audience size for A/B testing festival emails?”| Test Type | Minimum Recommended Size |
|---|---|
| Email A/B test | 100+ fans per variant |
| Price test | 50+ fans per variant |
| Landing page test | 200+ fans per variant |
| Progressive send | No minimum (depends on your base) |
2. Test Duration
Section titled “2. Test Duration”- Email A/B test: 24-48 hours (wait for opens to stabilise)
- Conversion test: 7-14 days (full purchase decision cycle)
- Engagement test: 30+ days (behavioural pattern)
3. Variables to Test (One at a Time)
Section titled “3. Variables to Test (One at a Time)”❌ Bad — You change multiple things:
Problem: You will not know which variable caused the difference.
✅ Good — You change ONE variable:
Result: You know exactly which subject line works better.
4. Control Group
Section titled “4. Control Group”Always include a control group (no change) to measure the real impact.
Example:
Frequently Asked Questions
Section titled “Frequently Asked Questions”Do groups change over time?
Section titled “Do groups change over time?”No. A fan will always be in the same group (based on a hash of their user_id).
Example: If Fan X is in Group 5 today, they will still be in Group 5 in 6 months.
Can I select multiple groups for a campaign?
Section titled “Can I select multiple groups for a campaign?”Yes. You can send to:
- A single group: Group 1
- A range of groups: Groups 1-5
- Specific groups: Groups 1, 3, 7
How do I split 13,456 fans into 10 groups?
Section titled “How do I split 13,456 fans into 10 groups?”Nevent does it automatically:
- Groups 1-6: ~1,346 fans each
- Groups 7-10: ~1,345 fans each
The difference is minimal (±1 fan).
Can I see which fans are in each group?
Section titled “Can I see which fans are in each group?”Yes, in the segment preview you can filter by a specific group and view the list of fans.
Next Steps
Section titled “Next Steps”Now that you understand groups:
- See Full Use Cases — Real examples with groups
- Best Practices — Optimise your A/B tests
- FAQ — Answers to common questions
Ready for scientific testing? 🧪
Examples of A/B testing: subject lines, prices, offers with groups →