seo/guides/AI_RECOMMENDATIONS_AND_DIAGNOSTICS.md

# AI Recommendations & Meta Description Diagnostics

## Part 1: AI Recommendations - --top-n Parameter

### Understanding the Default (10 posts)

By default, the analyzer:
- **Analyzes ALL posts** (titles, meta descriptions, scores)
- **Generates AI recommendations for only top 10** worst-scoring posts

```bash
python scripts/multi_site_seo_analyzer.py

Result:
  ✓ Analyzes 368 posts (all of them)
  ✓ AI recommendations: top 10 only
  ✓ Cost: ~$0.10
```

### Why Only 10?

**Cost Control:**

| Posts | Cost | Time | Use Case |
|-------|------|------|----------|
| 10 | $0.10 | 5 min | Quick analysis, focus on worst |
| 20 | $0.20 | 8 min | More detailed, more cost |
| 50 | $0.50 | 15 min | Comprehensive, moderate cost |
| 100 | $1.00 | 25 min | Very thorough |
| 368 | $3.60+ | 60+ min | All posts (within €50 budget) |

### Changing the AI Analysis Level

```bash
# Analyze top 20 worst posts
python scripts/multi_site_seo_analyzer.py --top-n 20

# Analyze top 50 worst posts
python scripts/multi_site_seo_analyzer.py --top-n 50

# Analyze ALL 368 posts (comprehensive)
python scripts/multi_site_seo_analyzer.py --top-n 368

# Analyze 0 posts (no AI, free)
python scripts/multi_site_seo_analyzer.py --no-ai
```

### Expected Results

**Command:** `--top-n 50`

```
Analyzing 368 posts...
CSV written with all 368 posts

Generating AI recommendations for top 50 posts...
  [1/50] Post with score 12 → AI generates recommendations
  [2/50] Post with score 18 → AI generates recommendations
  ...
  [50/50] Post with score 72 → AI generates recommendations

CSV updated with 50 AI recommendations
Cost: ~$0.50
```

**Output CSV:**
- Posts 1-50: AI recommendations filled in
- Posts 51-368: AI recommendations empty
- All posts have: title_score, meta_score, overall_score

### Workflow by Level

**Level 1: Quick Overview (--no-ai)**
```bash
python scripts/multi_site_seo_analyzer.py --no-ai
# See all scores, identify worst posts, no AI cost
# Good for: Understanding what needs work
```

**Level 2: Quick Wins (default --top-n 10)**
```bash
python scripts/multi_site_seo_analyzer.py
# Analyze top 10 worst, get AI recommendations
# Good for: Getting started, low cost (~$0.10)
```

**Level 3: Thorough Analysis (--top-n 50)**
```bash
python scripts/multi_site_seo_analyzer.py --top-n 50
# Analyze top 50 worst, comprehensive AI
# Good for: Serious optimization effort (~$0.50)
```

**Level 4: Complete Analysis (--top-n 368)**
```bash
python scripts/multi_site_seo_analyzer.py --top-n 368
# AI for every post
# Good for: Complete overhaul, fits €50 budget (~$3.60)
```

### Combined Options

```bash
# Include drafts + AI for top 30
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 30

# No AI (free, fast) + drafts
python scripts/multi_site_seo_analyzer.py --include-drafts --no-ai

# All posts + AI for all + progressive CSV
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 368
```

---

## Part 2: Meta Description Detection & Diagnostics

### The Problem

Meta descriptions aren't being found for some posts. This could be because:

1. **WordPress REST API not returning meta fields**
2. **Meta fields stored in different plugin locations**
3. **SEO plugin not properly exposing fields**

### Supported SEO Plugins

The script now looks for meta descriptions from:

| Plugin | Field Name |
|--------|------------|
| Yoast SEO | `_yoast_wpseo_metadesc` |
| Rank Math | `_rank_math_description` |
| All in One SEO | `_aioseo_description` |
| Standard | `description` |
| Alternative names | `_meta_description`, `metadesc` |

### Diagnostic Command

Check what meta fields are actually available on your site:

```bash
python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net
```

**Output example:**
```
============================================================
META FIELD DIAGNOSTIC
============================================================

Site: https://www.mistergeek.net
Checking available meta fields in first post...

Post: The Best VPN Services 2025

Available meta fields:
  • _yoast_wpseo_metadesc: Discover the best VPN services...
  • _yoast_wpseo_focuskw: best VPN
  • _yoast_wpseo_title: Best VPN Services 2025 | mistergeek
  • custom_field_1: some value

Full meta object:
{
  "_yoast_wpseo_metadesc": "Discover the best VPN services...",
  "_yoast_wpseo_focuskw": "best VPN",
  ...
}
```

### Running Diagnostics on All 3 Sites

```bash
# Mistergeek
python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net

# Webscroll
python scripts/multi_site_seo_analyzer.py --diagnose https://www.webscroll.fr

# HelloGeek
python scripts/multi_site_seo_analyzer.py --diagnose https://www.hellogeek.net
```

### What to Look For

**Good - Meta descriptions found:**
```
Available meta fields:
  • _yoast_wpseo_metadesc: Discover the best VPN...
  • _yoast_wpseo_focuskw: best VPN
```
✓ Meta descriptions will be detected

**Problem - No meta descriptions:**
```
Available meta fields:
  (No meta fields found)
```
✗ Either:
  - SEO plugin not installed
  - REST API not exposing meta
  - Custom field names not recognized

**Problem - Unknown field names:**
```
Available meta fields:
  • custom_meta_1: some value
  • my_seo_field: description text
```
✗ Custom field names - need to update script

---

## Fixing Missing Meta Descriptions

### Solution 1: Enable REST API for SEO Plugin

**For Yoast SEO:**
1. Admin → Yoast SEO → Settings → Advanced
2. Look for "REST API" option
3. Enable "Show in REST API"
4. Save

**For Rank Math:**
1. Admin → Rank Math → General Settings
2. Look for "REST API" option
3. Enable REST API fields
4. Save

**For All in One SEO:**
1. Admin → All in One SEO → Settings
2. Look for REST API option
3. Enable REST API
4. Save

### Solution 2: Add Custom Field Recognition

If your site uses custom field names, tell us and we'll add them:

```python
# Example: if site uses "my_custom_description"
meta_desc = (
    meta_dict.get('_yoast_wpseo_metadesc', '') or
    meta_dict.get('_rank_math_description', '') or
    meta_dict.get('my_custom_description', '')  # ← Add this
)
```

Run diagnostic and send us the field name, we'll update the script.

### Solution 3: Manual Curl Request

Check API response directly:

```bash
# Replace with your site and credentials
curl -u "username:app_password" \
  "https://www.mistergeek.net/wp-json/wp/v2/posts?per_page=1&status=publish" | jq '.[] | .meta'

# Output will show all meta fields available
```

### Solution 4: Check REST API is Enabled

Test if REST API works:

```bash
# Should return post data
curl https://www.mistergeek.net/wp-json/wp/v2/posts?per_page=1

# Should return 404 or empty if not available
curl https://broken-site.com/wp-json/wp/v2/posts
```

---

## Workflow: Finding Missing Meta Descriptions

### Step 1: Run Diagnostic

```bash
python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net
```

### Step 2: Check Output

Look for meta description field names in the output.

### Step 3: If Missing

**Option A: Enable in SEO Plugin**
- Go to plugin settings
- Enable REST API field exposure
- Save

**Option B: Update Field Name**
- If custom field is shown in diagnostic
- Tell us the field name
- We'll add it to the script

**Option C: Check WordPress**
- Verify WordPress REST API is working
- Check security plugins aren't blocking
- Ensure user has read permissions

### Step 4: Re-run Analysis

```bash
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 50
```

Now meta descriptions should be found!

---

## Complete Examples

### Example 1: Quick Analysis (Cost: $0.10)

```bash
# Default: all posts analyzed, AI for top 10
python scripts/multi_site_seo_analyzer.py

Result:
- 368 posts analyzed (titles, meta, scores)
- 10 posts get AI recommendations
- Cost: ~$0.10
- Time: 5 minutes
```

### Example 2: Comprehensive Analysis (Cost: $0.50)

```bash
# Include drafts, AI for top 50 worst posts
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 50

Result:
- 368 posts analyzed (all, including drafts)
- 50 posts get AI recommendations
- Cost: ~$0.50
- Time: 15 minutes
```

### Example 3: Diagnostic + Complete Analysis

```bash
# First, diagnose meta fields
python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net

# Then run full analysis
python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 100

Result:
- Understand meta situation first
- 100 posts get AI recommendations
- Cost: ~$1.00
- Time: 30 minutes
```

### Example 4: Free Analysis (No AI Cost)

```bash
# Get all scores without AI
python scripts/multi_site_seo_analyzer.py --no-ai

Result:
- 368 posts analyzed
- 0 posts get AI recommendations
- Cost: $0.00
- Time: 2 minutes
- Then manually review CSV and optimize
```

---

## Summary

### AI Recommendations (-–top-n)

```bash
--no-ai          # Cost: $0    | Time: 2 min  | AI: 0 posts
--top-n 10       # Cost: $0.10 | Time: 5 min  | AI: 10 posts (default)
--top-n 20       # Cost: $0.20 | Time: 8 min  | AI: 20 posts
--top-n 50       # Cost: $0.50 | Time: 15 min | AI: 50 posts
--top-n 100      # Cost: $1.00 | Time: 25 min | AI: 100 posts
--top-n 368      # Cost: $3.60 | Time: 60 min | AI: all posts
```

### Meta Description Detection

```bash
--diagnose URL   # Check what meta fields are available
```

If meta descriptions not found:
1. Run diagnostic
2. Check which field names are available
3. Enable in SEO plugin settings OR
4. Tell us custom field name and we'll add support

---

## Next Steps

1. **Run diagnostic:**
   ```bash
   python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net
   ```

2. **Check for meta descriptions** in output

3. **If missing:**
   - Enable REST API in SEO plugin, or
   - Share diagnostic output so we can add custom field support

4. **Run full analysis with desired AI level:**
   ```bash
   python scripts/multi_site_seo_analyzer.py --include-drafts --top-n 50
   ```

5. **Review results in CSV**

Ready? Run: `python scripts/multi_site_seo_analyzer.py --diagnose https://www.mistergeek.net`