SEO Tests
SEO Utils allows you to run controlled experiments to measure the impact of your SEO changes using real Google Search Console data. The tool automatically tracks metrics like clicks, impressions, CTR, and rankings to help you make data-driven decisions.
You can test whether your SEO changes actually improve performance before rolling them out across your entire site. The tool supports three different test types, each designed for specific testing scenarios.
Test Types
SEO Utils supports three types of SEO tests:
Time-based Test: Compare performance before and after implementing a change
URL Switch Test: A/B test by alternating between two versions of the pages
Split Test: Run simultaneous tests on different page groups
How to Create an SEO Test
To get started, head to the SEO Tests in the left sidebar. Then, click the Create Test button.
The test creation process has three steps. We'll go through each step and explain the different fields for each test type.
Step 1: Configure Test
First, you need to set up the basic test information.
Common Fields (All Test Types):
Domain: Select the Google Search Console property you want to test
Test Type: Choose between Time-based, URL Switch, or Split Test
Test Name: Give your test a descriptive name
Description: Optional field to add more context about your test
Hypothesis: Clearly state what you expect to happen
Test Type Specific Fields:
For Time-based Tests:
Change Implemented Date: When the SEO change will be implemented. This date separates the "before" and "after" periods for comparison
For URL Switch Tests:
Switch Interval: How often to switch between versions (1-30 days)
Starting Version: Which version to show first (original or variant)
For Split Tests:
Traffic Split: Percentage of traffic for the variant group (10-90%)
Step 2: Select Pages
Next, you'll select which pages to include in your test.
Page Selection Process:
Add Filters: Use filters to find pages that match your criteria
Date Range: Select the date range for baseline data validation
This ensures your selected pages have enough historical data
Recommended: Use at least 30 days of baseline data
Find Pages: Click to search for matching pages in your GSC data
For Time-based Tests:
You can select or remove pages using checkboxes
For URL Switch Tests:
Configure URL pairs (original vs. alternate versions)
Example:
/product(original) vs./product-v2(alternate)The tool will automatically switch between these URLs based on your interval
For Split Tests:
Pages are automatically divided into Control and Variant groups
You can manually move pages between groups if needed
The tool ensures both groups have similar baseline performance
Tip: For Split Tests, SEO Utils automatically balances the groups to ensure similar baseline metrics, reducing test bias. If the current split isn't well-balanced, use the Shuffle button to generate a different distribution of pages between control and variant groups until you find a balanced split.
Step 3: Schedule Test
Finally, set when your test should run.
Scheduling Fields:
Start Date: When to begin tracking test metrics
End Date: Optional - when to stop the test (leave empty for ongoing tests)
For Time-based Tests only:
Change Implemented Date: When you plan to implement (or have implemented) the SEO change
This separates the "before" and "after" periods
Must be between the start and end dates
Allow at least 2-4 weeks for your test to gather statistically significant data. Google Search Console data has a 2-3 day delay, so factor this into your timeline.
When to Use Each Test Type
Time-based Test
Use when:
Making site-wide changes (like updating all title tags)
Testing changes that can't be reversed easily
You want to compare "before" and "after" performance
Historical comparison is more important than simultaneous testing
Example scenarios:
Adding schema markup to all product pages
Changing the URL structure site-wide
Implementing Core Web Vitals improvements
URL Switch Test
Use when:
You have two versions of the same page
You can programmatically switch between versions
You want to minimize the impact of external factors
Testing significant page redesigns
Example scenarios:
Testing two different page layouts
Comparing long-form vs. short-form content
Testing different internal linking structures
Split Test
Use when:
You have many similar pages (like product or category pages)
You want to test changes on a subset before full rollout
You need results faster than sequential testing
Testing template-level changes
Example scenarios:
Testing new title tag templates on product pages
Adding FAQ sections to half of your blog posts
Testing different meta description formats
Understanding Statistical Terms
SEO Utils uses statistical analysis to ensure your test results are reliable. Here are the key terms:
1. Confidence Score / Statistical Significance
What it means: The probability that the observed difference is real and not due to random variation.
95% confidence = Only 5% chance the results are due to randomness
99% confidence = Only 1% chance the results are due to randomness
How to use it: Wait for at least 95% confidence before making decisions based on test results.
2. P-Value
What it means: The probability of seeing these results if there was actually no difference between versions.
p < 0.05: Statistically significant (less than 5% chance of random occurrence)
p < 0.01: Highly significant (less than 1% chance of random occurrence)
How to use it: A p-value below 0.05 indicates your results are statistically significant.
3. Z-Score
What it means: How many standard deviations the test results are from the mean. It measures the magnitude of difference.
|z| > 1.96: Significant at 95% confidence level
|z| > 2.58: Significant at 99% confidence level
How to use it: Larger absolute z-scores indicate stronger evidence of a real difference.
4. Uplift
What it means: The percentage change in performance between control and variant groups.
Positive uplift: Variant performs better than control
Negative uplift: Control performs better than variant
Calculation: (Variant Metric - Control Metric) / Control Metric × 100
Important: High uplift doesn't always mean statistical significance. Always check the confidence score before making decisions.
Interpreting Results Correctly
When to Trust Your Results
✅ Reliable results when:
Statistical significance is 95% or higher
The test has run for at least 2-4 weeks
You have sufficient sample size (usually 1000+ clicks per group)
No major external events occurred during the test
⚠️ Be cautious when:
Significance is between 90-95%
Sample size is small (< 500 clicks)
Test duration is less than 2 weeks
Major algorithm updates occurred during testing
❌ Don't trust results when:
Significance is below 90%
Very small sample size (< 100 clicks)
Test ran for less than a week
Site had technical issues during the test
Monitoring Test Progress
SEO Utils provides real-time monitoring features:
Daily Metrics Updates: See how metrics change day by day
Trend Charts: Visualize performance trends over time
Pause/Resume Capability: Exclude problematic periods from analysis
Export Options: Download detailed results for further analysis
Best Practices
Define clear hypotheses: Be specific about what you expect to change and by how much
Ensure sufficient sample size: Include enough pages and traffic for reliable results
Run tests for adequate duration: At least 2-4 weeks, accounting for weekly patterns
Monitor external factors: Note any algorithm updates or seasonal changes during your test
Document your changes: Keep detailed records of exactly what was changed
SEO Tests integrate directly with your Google Search Console data, so there's no need for additional tracking setup. Just connect your GSC account and start testing!
Last updated