Advertising Testing Framework

Table of Contents

1. Testing Philosophy
2. A/B Testing Methodology
3. Multi-Variate Testing
4. Channel-Specific Testing
5. Creative Testing
6. Landing Page Testing
7. Audience Testing
8. Testing Calendar

---

Testing Philosophy

Core Principles

1. Always Be Testing Never run campaigns without active tests. Continuous improvement comes from continuous experimentation.

2. Test One Variable at a Time Isolate variables to understand what actually drives results. Multiple changes = unclear attribution.

3. Statistical Significance Required Don't call winners prematurely. Wait for 95% confidence and adequate sample size.

4. Document Everything Track hypotheses, results, and learnings. Build institutional knowledge over time.

5. Fail Fast, Scale Winners Kill losers quickly (7 days max). Scale winners aggressively.

6. Test Big Swings AND Small Tweaks

• Big swings: 20%+ impact potential (new creative, different angles)
• Small tweaks: 2-5% improvement (CTA color, headline wording)

Testing Hierarchy (Priority Order)

Tier 1 - Highest Impact:

1. Offer/Value Proposition
2. Creative Hook (first 3 seconds of video, hero image, headline)
3. Audience Targeting

Tier 2 - Medium Impact: 4. Call-to-Action 5. Ad Format 6. Landing Page Design

Tier 3 - Lower Impact (But Still Worth Testing): 7. Ad Copy Variations 8. CTA Button Color 9. Form Field Order 10. Smaller Design Elements

---

A/B Testing Methodology

Standard A/B Test Protocol

1. Hypothesis Formation

• Current State: "Our assessment completion rate is 45%"
• Hypothesis: "Changing the headline from benefit-focused to question-format will increase completion rate"
• Expected Outcome: "Completion rate will increase to 55%+"
• Rationale: "Questions create curiosity gap and engagement"

2. Test Design

• Control (A): Current headline
• Variant (B): New question-format headline
• Everything Else: Identical
• Traffic Split: 50/50
• Duration: Minimum 7 days or 1,000 conversions per variation (whichever comes first)

3. Sample Size Requirements

Minimum Sample Sizes:

• High-traffic pages: 1,000 conversions per variation
• Medium-traffic pages: 500 conversions per variation
• Low-traffic pages: 100 conversions per variation
• Minimum time: 7 days (capture full week including weekend behavior)

Sample Size Calculator: Use: ev.tools/ab-test-calculator

• Input current conversion rate
• Input expected improvement
• Output: Required sample size

4. Statistical Significance

Requirements:

• Minimum 95% confidence level
• p-value < 0.05
• Adequate sample size achieved
• Full week of data (accounts for day-of-week variations)

Don't:

• Call winners before statistical significance
• Stop test early because one is "obviously winning"
• Keep test running indefinitely (diminishing returns after 30 days)

5. Results Analysis

Document:

• Hypothesis (what you expected)
• Results (what actually happened)
• Confidence level
• Sample size
• Duration
• Winner
• Magnitude of lift
• Key learnings
• Next steps

Example:

Test: Homepage Headline A/B
Hypothesis: Question format will increase conversions
Results:
- Control (A): 3.2% conversion (n=5,243)
- Variant (B): 3.8% conversion (n=5,127)
- Lift: +18.75%
- Confidence: 97%
- Winner: Variant B
- Learning: Question-format headlines create curiosity gap and engagement
- Next Step: Test different question variations

6. Implementation

• Implement winner permanently
• Archive test results
• Share learnings with team
• Plan next test based on results

---

Multi-Variate Testing (MVT)

When to Use MVT

Use When:

• High traffic volume (10,000+ visitors/week)
• Multiple elements to test simultaneously
• Want to find optimal combination
• Have statistical expertise

Don't Use When:

• Low traffic (insufficient sample size)
• Just starting testing program
• Simple A/B test will suffice

MVT Example: Landing Page

Elements to Test:

• Headline: A1 vs A2 vs A3
• Hero Image: B1 vs B2 vs B3
• CTA Button: C1 vs C2 vs C3

Total Combinations: 3 × 3 × 3 = 27 variations

Traffic Requirements: 27,000+ conversions (1,000 per variation minimum)

Duration: Potentially weeks/months

Better Approach for Most: Sequential A/B tests:

1. Test headlines first (find winner)
2. Then test hero images (with winning headline)
3. Then test CTA buttons (with winning headline + image)

Result: 3 tests vs. 1 massive MVT, same insights, faster results

---

Channel-Specific Testing

Google Ads Testing

What to Test:

1. Ad Copy (RSAs - Responsive Search Ads)

• Test 3-5 headline variations
• Test 2-3 description variations
• Let Google optimize combinations
• Review asset performance report

Example Test:

• Headlines emphasizing: Cost savings vs. Speed vs. Quality
• Measure: CTR and conversion rate by asset
• Winner: Implement winning messaging across campaigns

2. Landing Page

• Test different destination URLs
• Measure: Conversion rate post-click
• Common test: Homepage vs. Dedicated landing page vs. Assessment page

3. Ad Extensions

• Test sitelink variations
• Test different callouts
• Measure: Extension CTR and conversion rate

4. Bidding Strategy

• Manual CPC vs. Maximize Conversions vs. Target CPA
• Test for 2-3 weeks minimum
• Measure: CPA and conversion volume

Testing Cadence:

• New ad copy: Weekly
• Landing page: Monthly
• Bidding strategy: Quarterly

---

Meta Ads Testing

What to Test:

1. Creative (Primary Priority)

Image Ads:

• Photo vs. Graphic vs. Screenshot
• Different visual styles
• With/without text overlay
• Different value propositions

Video Ads:

• Different hooks (first 3 seconds)
• Different lengths (15s vs. 30s vs 60s)
• Talking head vs. Animation vs. Screen recording
• Different storytelling angles

Test Setup:

• Create ad set
• Add 3-5 creative variations
• Let Meta optimize delivery
• Review after 7 days, kill lowest performers

2. Audience

• Interest-based vs. Lookalike vs. Broad
• Different interest combinations
• Lookalike % (1% vs. 3% vs. 5%)
• Demographic variations (age ranges, locations)

Test Setup:

• Duplicate campaign
• Change only audience
• Equal budget split
• Run for 14 days minimum

3. Placement

• Automatic vs. Manual
• Feed vs. Stories vs. Reels
• Facebook vs. Instagram
• Feed position

Test Setup:

• Duplicate campaign
• Manual placement selection
• Run for 7 days
• Measure CPM, CTR, CPA by placement

4. Primary Text

• Long-form (300 chars) vs. Short-form (125 chars)
• Question vs. Statement
• Problem-focused vs. Solution-focused
• With/without emojis

Testing Cadence:

• New creative: Every 2 weeks
• Audience: Monthly
• Copy variations: Weekly
• Placement: Quarterly

---

LinkedIn Ads Testing

What to Test:

1. Audience Targeting

• Job title variations
• Company size segments
• Industry verticals
• Seniority levels

Test Setup:

• Create separate campaigns per audience
• Identical creative and budget
• Run for 2 weeks (LinkedIn has lower volume)
• Measure: CTR, CPC, conversion rate

2. Ad Format

• Sponsored Content vs. Message Ads vs. Document Ads
• Single Image vs. Carousel vs. Video
• Different content types

3. Creative

• Professional vs. Casual tone
• Feature-focused vs. Benefit-focused
• Data-driven vs. Emotional
• Third-person vs. First-person

4. Offer

• Free assessment vs. Guide download vs. Demo
• Different lead magnets
• Different CTAs

Testing Cadence:

• Audience: Bi-weekly
• Format: Monthly
• Creative: Weekly
• Offer: Monthly

---

Twitter Ads Testing

What to Test:

1. Tweet Format

• Single tweet vs. Thread vs. Poll
• Image vs. Video vs. Text-only
• Different thread lengths (3 vs. 5 vs. 10 tweets)

2. Creative Angle

• Hot takes vs. Educational vs. Personal story
• Question vs. Statement
• Contrarian vs. Supportive

3. CTA

• Learn more vs. Try free vs. Read article
• Link in tweet vs. Quote tweet with link
• Thread ending CTA vs. Multiple CTAs throughout

Testing Cadence:

• New tweets: Daily/Weekly
• Format variations: Weekly
• Angle shifts: Bi-weekly

---

Creative Testing

Video Ad Testing Framework

Hook Testing (Most Important) Test first 3 seconds extensively:

• Question vs. Statement vs. Visual hook
• Problem vs. Solution opening
• Stat/number vs. Story opening

Example:

• Version A: "Are you struggling with brand positioning?"
• Version B: "$15,000. That's what agencies charge for brand strategy."
• Version C: "We analyzed 10,000 brand strategies..."

Test Setup:

• Same video after first 3 seconds
• Only hook differs
• Measure: 3-second view rate, completion rate, CTR

Length Testing

• 15-second vs. 30-second vs. 60-second
• Test same concept in different lengths
• Measure: Completion rate, CPA, ROAS

Format Testing

• Talking head vs. Screen recording vs. Animation
• Captions vs. No captions
• Music vs. No music

Image Ad Testing Framework

Visual Style

• Photography vs. Graphics vs. Screenshots
• Minimal vs. Detailed
• Dark vs. Light background
• With person vs. Without person

Text Overlay

• With vs. Without
• Different headline options
• Different amounts of text

Color Scheme

• Brand colors vs. High contrast
• Different background colors
• Different accent colors

Testing Matrix

Week	Creative Type	Variable	Variations
1	Video	Hook	3 versions
2	Image	Visual style	3 versions
3	Video	Length	15s vs 30s vs 60s
4	Carousel	Number of cards	3 vs 5 vs 7
5	Image	With/without person	2 versions
6	Video	Format	Talking head vs Animation
7	Image	Text overlay	With vs Without
8	Video	CTA placement	End only vs Throughout

---

Landing Page Testing

High-Impact Tests

1. Headline

• Test 5-10 variations
• Different formats (question, benefit, social proof, etc.)
• Measure: Bounce rate, conversion rate

2. Hero Section

• Image vs. Video
• Different CTAs
• With/without social proof

3. Form Length

• Number of fields
• Required vs. Optional fields
• Single-step vs. Multi-step

4. Social Proof Placement

• Above fold vs. Below fold
• Type: Numbers vs. Testimonials vs. Logos

5. CTA Button

• Color (yellow vs. blue vs. green)
• Text ("Get Started" vs. "Start Free" vs. "Try Now")
• Size and placement
• Single vs. Multiple CTAs

Testing Tools

Recommended:

• Optimizely: Enterprise-grade A/B testing
• VWO: Mid-market solution
• Google Optimize: Free (being sunset, but alternatives available)
• Unbounce: Landing page builder with built-in A/B testing

Sample Test Schedule

Month 1:

• Week 1: Headline test
• Week 2: Implement winner, test hero image
• Week 3: Implement winner, test CTA button
• Week 4: Implement winner, test form length

Month 2:

• Week 1: Test social proof placement
• Week 2: Test page length (short vs. long-form)
• Week 3: Test different value propositions
• Week 4: Consolidate learnings, implement all winners

Compound Effect:

• Headline: +15% lift
• Hero image: +8% lift
• CTA button: +5% lift
• Form length: +12% lift
• Total Potential: ~40% conversion rate improvement

---

Audience Testing

Meta Audience Testing

Test 1: Interest Categories

• Broad interests (Entrepreneurship, Marketing)
• Specific interests (Y Combinator, TechCrunch)
• Competitor interests
• Tool/software interests (Canva, Figma, etc.)

Test 2: Lookalike Percentages

• 1% (most similar)
• 2-3% (moderate similarity)
• 4-5% (broader reach)

Test 3: Demographics

• Age ranges (25-35 vs. 35-45 vs. 45-55)
• Locations (US vs. UK vs. Global English-speaking)
• Devices (Mobile vs. Desktop)

Test 4: Custom Audiences

• Website visitors (all vs. high-intent pages only)
• Video viewers (25% vs. 75% vs. 95%)
• Engagement (page likes, post engagement, etc.)

Google Ads Audience Testing

Test 1: In-Market Audiences

• Business Services
• Marketing Services
• Small Business Services
• Compare: In-market vs. No audience layering

Test 2: Affinity Audiences

• Business Professionals
• Technophiles
• Startup Enthusiasts
• Compare: Different affinity combinations

Test 3: Custom Intent

• Keywords they've searched
• URLs they've visited
• Apps they use
• Compare: Custom intent vs. Broad match

LinkedIn Audience Testing

Test 1: Job Function

• Marketing only
• Marketing + Sales
• Marketing + Business Development
• All business functions

Test 2: Seniority

• C-level only
• Director+ level
• Manager+ level
• All seniority levels

Test 3: Company Size

• 1-50 employees
• 51-200 employees
• 201-1000 employees
• Compare: Small vs. Mid-market focus

---

Testing Calendar

Monthly Testing Plan Template

Week 1:

• Launch new creative tests (3-5 variations)
• Review previous month's results
• Document learnings

Week 2:

• Check creative test performance
• Kill obvious losers
• Scale early winners

Week 3:

• Audience testing
• Landing page element test
• Review mid-month metrics

Week 4:

• Finalize monthly tests
• Implement winners
• Plan next month's tests

Quarterly Testing Roadmap

Q1 Focus: Foundation

• Establish baseline metrics
• Test core value propositions
• Find winning creative formats
• Identify best-performing audiences

Q2 Focus: Optimization

• Optimize winning campaigns
• Test variations of winners
• Expand successful audiences
• Improve conversion funnel

Q3 Focus: Scale

• Scale proven winners
• Test new channels
• Lookalike expansion
• Advanced targeting tests

Q4 Focus: Efficiency

• Maximize ROAS
• Test retention offers
• Holiday/seasonal messaging
• Annual planning based on learnings

---

Test Documentation Template

TEST NAME: [Descriptive name]
DATE: [Start date] - [End date]
CHANNEL: [Platform/channel]
OBJECTIVE: [What you're trying to improve]

HYPOTHESIS:
We believe that [change]
Will result in [expected outcome]
Because [rationale]

CONTROL (A):
[Description of current/control]

VARIANT (B):
[Description of test variant]

VARIANT (C):
[If applicable]

SUCCESS METRICS:
Primary: [Main KPI]
Secondary: [Supporting KPIs]

SAMPLE SIZE:
Required: [Calculated sample size]
Achieved: [Actual sample size]

RESULTS:
Control: [Metric] = [Value] (n=[sample size])
Variant: [Metric] = [Value] (n=[sample size])
Lift: [Percentage change]
Confidence: [Statistical confidence level]
Winner: [Control or Variant]

LEARNINGS:
- [Key takeaway 1]
- [Key takeaway 2]
- [Key takeaway 3]

NEXT STEPS:
- [Action item 1]
- [Action item 2]

CREATIVE ASSETS:
[Links to creative files, screenshots, etc.]

---

This testing framework provides systematic approach to continuous improvement across all advertising channels, ensuring data-driven decision-making and compound performance gains over time.