Refactor SEO automation into unified CLI application
Major refactoring to create a clean, integrated CLI application: ### New Features: - Unified CLI executable (./seo) with simple command structure - All commands accept optional CSV file arguments - Auto-detection of latest files when no arguments provided - Simplified output directory structure (output/ instead of output/reports/) - Cleaner export filename format (all_posts_YYYY-MM-DD.csv) ### Commands: - export: Export all posts from WordPress sites - analyze [csv]: Analyze posts with AI (optional CSV input) - recategorize [csv]: Recategorize posts with AI - seo_check: Check SEO quality - categories: Manage categories across sites - approve [files]: Review and approve recommendations - full_pipeline: Run complete workflow - analytics, gaps, opportunities, report, status ### Changes: - Moved all scripts to scripts/ directory - Created config.yaml for configuration - Updated all scripts to use output/ directory - Deprecated old seo-cli.py in favor of new ./seo - Added AGENTS.md and CHANGELOG.md documentation - Consolidated README.md with updated usage ### Technical: - Added PyYAML dependency - Removed hardcoded configuration values - All scripts now properly integrated - Better error handling and user feedback Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
This commit is contained in:
Kevin Bataille
423
guides/AI_RECOMMENDATIONS_AND_DIAGNOSTICS.md
Normal file
423
guides/AI_RECOMMENDATIONS_AND_DIAGNOSTICS.md
Normal file
@@ -0,0 +1,423 @@
|
||||
# AI Recommendations & Meta Description Diagnostics
|
||||
|
||||
## Part 1: AI Recommendations - --top-n Parameter
|
||||
|
||||
### Understanding the Default (10 posts)
|
||||
|
||||
By default, the analyzer:
|
||||
- **Analyzes ALL posts** (titles, meta descriptions, scores)
|
||||
- **Generates AI recommendations for only top 10** worst-scoring posts
|
||||
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py
|
||||
|
||||
Result:
|
||||
✓ Analyzes 368 posts (all of them)
|
||||
✓ AI recommendations: top 10 only
|
||||
✓ Cost: ~$0.10
|
||||
```
|
||||
|
||||
### Why Only 10?
|
||||
|
||||
**Cost Control:**
|
||||
|
||||
| Posts | Cost | Time | Use Case |
|
||||
|-------|------|------|----------|
|
||||
| 10 | $0.10 | 5 min | Quick analysis, focus on worst |
|
||||
| 20 | $0.20 | 8 min | More detailed, more cost |
|
||||
| 50 | $0.50 | 15 min | Comprehensive, moderate cost |
|
||||
| 100 | $1.00 | 25 min | Very thorough |
|
||||
| 368 | $3.60+ | 60+ min | All posts (within €50 budget) |
|
||||
|
||||
### Changing the AI Analysis Level
|
||||
|
||||
```bash
|
||||
# Analyze top 20 worst posts
|
||||
python scripts/multi_site_seo_analyzer.py --top-n 20
|
||||
|
||||
# Analyze top 50 worst posts
|
||||
python scripts/multi_site_seo_analyzer.py --top-n 50
|
||||
|
||||
# Analyze ALL 368 posts (comprehensive)
|
||||
python scripts/multi_site_seo_analyzer.py --top-n 368
|
||||
|
||||
# Analyze 0 posts (no AI, free)
|
||||
python scripts/multi_site_seo_analyzer.py --no-ai
|
||||
```
|
||||
|
||||
### Expected Results
|
||||
|
||||
**Command:** `--top-n 50`
|
||||
|
||||
```
|
||||
Analyzing 368 posts...
|
||||
CSV written with all 368 posts
|
||||
|
||||
Generating AI recommendations for top 50 posts...
|
||||
[1/50] Post with score 12 → AI generates recommendations
|
||||
[2/50] Post with score 18 → AI generates recommendations
|
||||
...
|
||||
[50/50] Post with score 72 → AI generates recommendations
|
||||
|
||||
CSV updated with 50 AI recommendations
|
||||
Cost: ~$0.50
|
||||
```
|
||||
|
||||
**Output CSV:**
|
||||
- Posts 1-50: AI recommendations filled in
|
||||
- Posts 51-368: AI recommendations empty
|
||||
- All posts have: title_score, meta_score, overall_score
|
||||
|
||||
### Workflow by Level
|
||||
|
||||
**Level 1: Quick Overview (--no-ai)**
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py --no-ai
|
||||
# See all scores, identify worst posts, no AI cost
|
||||
# Good for: Understanding what needs work
|
||||
```
|
||||
|
||||
**Level 2: Quick Wins (default --top-n 10)**
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py
|
||||
# Analyze top 10 worst, get AI recommendations
|
||||
# Good for: Getting started, low cost (~$0.10)
|
||||
```
|
||||
|
||||
**Level 3: Thorough Analysis (--top-n 50)**
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py --top-n 50
|
||||
# Analyze top 50 worst, comprehensive AI
|
||||
# Good for: Serious optimization effort (~$0.50)
|
||||
```
|
||||
|
||||
**Level 4: Complete Analysis (--top-n 368)**
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py --top-n 368
|
||||
# AI for every post
|
||||
# Good for: Complete overhaul, fits €50 budget (~$3.60)
|
||||
```
|
||||
|
||||
### Combined Options
|
||||
|
||||
```bash
|
||||
# Include drafts + AI for top 30
|
||||
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 30
|
||||
|
||||
# No AI (free, fast) + drafts
|
||||
python scripts/multi_site_seo_analyzer.py --include-drafts --no-ai
|
||||
|
||||
# All posts + AI for all + progressive CSV
|
||||
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 368
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Part 2: Meta Description Detection & Diagnostics
|
||||
|
||||
### The Problem
|
||||
|
||||
Meta descriptions aren't being found for some posts. This could be because:
|
||||
|
||||
1. **WordPress REST API not returning meta fields**
|
||||
2. **Meta fields stored in different plugin locations**
|
||||
3. **SEO plugin not properly exposing fields**
|
||||
|
||||
### Supported SEO Plugins
|
||||
|
||||
The script now looks for meta descriptions from:
|
||||
|
||||
| Plugin | Field Name |
|
||||
|--------|------------|
|
||||
| Yoast SEO | `_yoast_wpseo_metadesc` |
|
||||
| Rank Math | `_rank_math_description` |
|
||||
| All in One SEO | `_aioseo_description` |
|
||||
| Standard | `description` |
|
||||
| Alternative names | `_meta_description`, `metadesc` |
|
||||
|
||||
### Diagnostic Command
|
||||
|
||||
Check what meta fields are actually available on your site:
|
||||
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net
|
||||
```
|
||||
|
||||
**Output example:**
|
||||
```
|
||||
============================================================
|
||||
META FIELD DIAGNOSTIC
|
||||
============================================================
|
||||
|
||||
Site: https://www.mistergeek.net
|
||||
Checking available meta fields in first post...
|
||||
|
||||
Post: The Best VPN Services 2025
|
||||
|
||||
Available meta fields:
|
||||
• _yoast_wpseo_metadesc: Discover the best VPN services...
|
||||
• _yoast_wpseo_focuskw: best VPN
|
||||
• _yoast_wpseo_title: Best VPN Services 2025 | mistergeek
|
||||
• custom_field_1: some value
|
||||
|
||||
Full meta object:
|
||||
{
|
||||
"_yoast_wpseo_metadesc": "Discover the best VPN services...",
|
||||
"_yoast_wpseo_focuskw": "best VPN",
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
### Running Diagnostics on All 3 Sites
|
||||
|
||||
```bash
|
||||
# Mistergeek
|
||||
python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net
|
||||
|
||||
# Webscroll
|
||||
python scripts/multi_site_seo_analyzer.py --diagnose https://www.webscroll.fr
|
||||
|
||||
# HelloGeek
|
||||
python scripts/multi_site_seo_analyzer.py --diagnose https://www.hellogeek.net
|
||||
```
|
||||
|
||||
### What to Look For
|
||||
|
||||
**Good - Meta descriptions found:**
|
||||
```
|
||||
Available meta fields:
|
||||
• _yoast_wpseo_metadesc: Discover the best VPN...
|
||||
• _yoast_wpseo_focuskw: best VPN
|
||||
```
|
||||
✓ Meta descriptions will be detected
|
||||
|
||||
**Problem - No meta descriptions:**
|
||||
```
|
||||
Available meta fields:
|
||||
(No meta fields found)
|
||||
```
|
||||
✗ Either:
|
||||
- SEO plugin not installed
|
||||
- REST API not exposing meta
|
||||
- Custom field names not recognized
|
||||
|
||||
**Problem - Unknown field names:**
|
||||
```
|
||||
Available meta fields:
|
||||
• custom_meta_1: some value
|
||||
• my_seo_field: description text
|
||||
```
|
||||
✗ Custom field names - need to update script
|
||||
|
||||
---
|
||||
|
||||
## Fixing Missing Meta Descriptions
|
||||
|
||||
### Solution 1: Enable REST API for SEO Plugin
|
||||
|
||||
**For Yoast SEO:**
|
||||
1. Admin → Yoast SEO → Settings → Advanced
|
||||
2. Look for "REST API" option
|
||||
3. Enable "Show in REST API"
|
||||
4. Save
|
||||
|
||||
**For Rank Math:**
|
||||
1. Admin → Rank Math → General Settings
|
||||
2. Look for "REST API" option
|
||||
3. Enable REST API fields
|
||||
4. Save
|
||||
|
||||
**For All in One SEO:**
|
||||
1. Admin → All in One SEO → Settings
|
||||
2. Look for REST API option
|
||||
3. Enable REST API
|
||||
4. Save
|
||||
|
||||
### Solution 2: Add Custom Field Recognition
|
||||
|
||||
If your site uses custom field names, tell us and we'll add them:
|
||||
|
||||
```python
|
||||
# Example: if site uses "my_custom_description"
|
||||
meta_desc = (
|
||||
meta_dict.get('_yoast_wpseo_metadesc', '') or
|
||||
meta_dict.get('_rank_math_description', '') or
|
||||
meta_dict.get('my_custom_description', '') # ← Add this
|
||||
)
|
||||
```
|
||||
|
||||
Run diagnostic and send us the field name, we'll update the script.
|
||||
|
||||
### Solution 3: Manual Curl Request
|
||||
|
||||
Check API response directly:
|
||||
|
||||
```bash
|
||||
# Replace with your site and credentials
|
||||
curl -u "username:app_password" \
|
||||
"https://www.mistergeek.net/wp-json/wp/v2/posts?per_page=1&status=publish" | jq '.[] | .meta'
|
||||
|
||||
# Output will show all meta fields available
|
||||
```
|
||||
|
||||
### Solution 4: Check REST API is Enabled
|
||||
|
||||
Test if REST API works:
|
||||
|
||||
```bash
|
||||
# Should return post data
|
||||
curl https://www.mistergeek.net/wp-json/wp/v2/posts?per_page=1
|
||||
|
||||
# Should return 404 or empty if not available
|
||||
curl https://broken-site.com/wp-json/wp/v2/posts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow: Finding Missing Meta Descriptions
|
||||
|
||||
### Step 1: Run Diagnostic
|
||||
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net
|
||||
```
|
||||
|
||||
### Step 2: Check Output
|
||||
|
||||
Look for meta description field names in the output.
|
||||
|
||||
### Step 3: If Missing
|
||||
|
||||
**Option A: Enable in SEO Plugin**
|
||||
- Go to plugin settings
|
||||
- Enable REST API field exposure
|
||||
- Save
|
||||
|
||||
**Option B: Update Field Name**
|
||||
- If custom field is shown in diagnostic
|
||||
- Tell us the field name
|
||||
- We'll add it to the script
|
||||
|
||||
**Option C: Check WordPress**
|
||||
- Verify WordPress REST API is working
|
||||
- Check security plugins aren't blocking
|
||||
- Ensure user has read permissions
|
||||
|
||||
### Step 4: Re-run Analysis
|
||||
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 50
|
||||
```
|
||||
|
||||
Now meta descriptions should be found!
|
||||
|
||||
---
|
||||
|
||||
## Complete Examples
|
||||
|
||||
### Example 1: Quick Analysis (Cost: $0.10)
|
||||
|
||||
```bash
|
||||
# Default: all posts analyzed, AI for top 10
|
||||
python scripts/multi_site_seo_analyzer.py
|
||||
|
||||
Result:
|
||||
- 368 posts analyzed (titles, meta, scores)
|
||||
- 10 posts get AI recommendations
|
||||
- Cost: ~$0.10
|
||||
- Time: 5 minutes
|
||||
```
|
||||
|
||||
### Example 2: Comprehensive Analysis (Cost: $0.50)
|
||||
|
||||
```bash
|
||||
# Include drafts, AI for top 50 worst posts
|
||||
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 50
|
||||
|
||||
Result:
|
||||
- 368 posts analyzed (all, including drafts)
|
||||
- 50 posts get AI recommendations
|
||||
- Cost: ~$0.50
|
||||
- Time: 15 minutes
|
||||
```
|
||||
|
||||
### Example 3: Diagnostic + Complete Analysis
|
||||
|
||||
```bash
|
||||
# First, diagnose meta fields
|
||||
python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net
|
||||
|
||||
# Then run full analysis
|
||||
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 100
|
||||
|
||||
Result:
|
||||
- Understand meta situation first
|
||||
- 100 posts get AI recommendations
|
||||
- Cost: ~$1.00
|
||||
- Time: 30 minutes
|
||||
```
|
||||
|
||||
### Example 4: Free Analysis (No AI Cost)
|
||||
|
||||
```bash
|
||||
# Get all scores without AI
|
||||
python scripts/multi_site_seo_analyzer.py --no-ai
|
||||
|
||||
Result:
|
||||
- 368 posts analyzed
|
||||
- 0 posts get AI recommendations
|
||||
- Cost: $0.00
|
||||
- Time: 2 minutes
|
||||
- Then manually review CSV and optimize
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
### AI Recommendations (-–top-n)
|
||||
|
||||
```bash
|
||||
--no-ai # Cost: $0 | Time: 2 min | AI: 0 posts
|
||||
--top-n 10 # Cost: $0.10 | Time: 5 min | AI: 10 posts (default)
|
||||
--top-n 20 # Cost: $0.20 | Time: 8 min | AI: 20 posts
|
||||
--top-n 50 # Cost: $0.50 | Time: 15 min | AI: 50 posts
|
||||
--top-n 100 # Cost: $1.00 | Time: 25 min | AI: 100 posts
|
||||
--top-n 368 # Cost: $3.60 | Time: 60 min | AI: all posts
|
||||
```
|
||||
|
||||
### Meta Description Detection
|
||||
|
||||
```bash
|
||||
--diagnose URL # Check what meta fields are available
|
||||
```
|
||||
|
||||
If meta descriptions not found:
|
||||
1. Run diagnostic
|
||||
2. Check which field names are available
|
||||
3. Enable in SEO plugin settings OR
|
||||
4. Tell us custom field name and we'll add support
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Run diagnostic:**
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net
|
||||
```
|
||||
|
||||
2. **Check for meta descriptions** in output
|
||||
|
||||
3. **If missing:**
|
||||
- Enable REST API in SEO plugin, or
|
||||
- Share diagnostic output so we can add custom field support
|
||||
|
||||
4. **Run full analysis with desired AI level:**
|
||||
```bash
|
||||
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 50
|
||||
```
|
||||
|
||||
5. **Review results in CSV**
|
||||
|
||||
Ready? Run: `python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net`
|
||||
Reference in New Issue
Block a user