Initial commit: Clean SEO analysis system
This commit is contained in:
Kevin Bataille
474
README.md
Normal file
474
README.md
Normal file
@@ -0,0 +1,474 @@
|
||||
# WordPress SEO Automation Tool
|
||||
|
||||
Programmatically optimize SEO titles and meta descriptions across all WordPress posts using AI-powered generation and a CSV review workflow.
|
||||
|
||||
## Features
|
||||
|
||||
- **AI-Powered SEO Generation**: Uses OpenRouter API (Claude, GPT-4, Llama, etc.) to create optimized titles and descriptions
|
||||
- **Plugin Support**: Auto-detects and works with both Yoast SEO and Rank Math
|
||||
- **CSV Review Workflow**: Generate proposals, review in Excel/Sheets, approve changes before applying
|
||||
- **Safety Features**: Dry-run mode, rollback CSV generation, detailed logging
|
||||
- **SEO Best Practices**: Enforces 50-60 char titles, 150-160 char descriptions, keyword optimization
|
||||
- **Batch Processing**: Handle hundreds or thousands of posts efficiently
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Installation](#installation)
|
||||
- [WordPress Configuration](#wordpress-configuration)
|
||||
- [OpenRouter API Setup](#openrouter-api-setup)
|
||||
- [Usage](#usage)
|
||||
- [Workflow](#workflow)
|
||||
- [SEO Plugin Comparison](#seo-plugin-comparison)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [Cost Estimates](#cost-estimates)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- WordPress site with Yoast SEO or Rank Math plugin installed
|
||||
- Python 3.8 or higher
|
||||
- WordPress Application Password (for REST API access)
|
||||
- OpenRouter API key (for AI-powered generation)
|
||||
|
||||
## Installation
|
||||
|
||||
### 1. Clone or Download
|
||||
|
||||
```bash
|
||||
cd /Users/acid/Documents/seo
|
||||
```
|
||||
|
||||
### 2. Create Virtual Environment
|
||||
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
```
|
||||
|
||||
### 3. Install Dependencies
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 4. Configure Environment Variables
|
||||
|
||||
Copy the example environment file:
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Edit `.env` with your credentials:
|
||||
|
||||
```env
|
||||
WORDPRESS_URL=https://yoursite.com
|
||||
WORDPRESS_USERNAME=your_username
|
||||
WORDPRESS_APP_PASSWORD=your_application_password
|
||||
OPENROUTER_API_KEY=your_openrouter_api_key
|
||||
AI_MODEL=anthropic/claude-3.5-sonnet
|
||||
```
|
||||
|
||||
## WordPress Configuration
|
||||
|
||||
### Step 1: Create Application Password
|
||||
|
||||
1. Log in to WordPress Admin
|
||||
2. Go to **Users → Profile**
|
||||
3. Scroll to **Application Passwords** section
|
||||
4. Enter application name: "SEO Automation"
|
||||
5. Click **Add New Application Password**
|
||||
6. Copy the generated password (it will only be shown once)
|
||||
7. Add to `.env` file as `WORDPRESS_APP_PASSWORD`
|
||||
|
||||
### Step 2: Verify REST API Access
|
||||
|
||||
Test your authentication:
|
||||
|
||||
```bash
|
||||
curl --user "your_username:your_app_password" \
|
||||
https://yoursite.com/wp-json/wp/v2/posts?per_page=1&context=edit
|
||||
```
|
||||
|
||||
You should receive a JSON response with post data.
|
||||
|
||||
### Step 3: SEO Plugin Requirements
|
||||
|
||||
**For Yoast SEO:**
|
||||
- Yoast SEO Free or Premium installed and activated
|
||||
- Meta fields automatically accessible via REST API
|
||||
|
||||
**For Rank Math:**
|
||||
- Rank Math Free or Pro installed and activated
|
||||
- Meta fields automatically accessible via REST API
|
||||
|
||||
**Both plugins are supported** - the scripts auto-detect which one you're using.
|
||||
|
||||
## OpenRouter API Setup
|
||||
|
||||
### Why OpenRouter?
|
||||
|
||||
OpenRouter provides access to multiple AI models through a single API:
|
||||
- **Claude 3.5 Sonnet** (recommended): Best quality, $3/$15 per 1M tokens
|
||||
- **GPT-4 Turbo**: Strong performance, $10/$30 per 1M tokens
|
||||
- **Llama 3.1 70B**: Free tier available, $0/$0 per 1M tokens
|
||||
- **Gemini Pro 1.5**: Good balance, $1.25/$5 per 1M tokens
|
||||
|
||||
### Get API Key
|
||||
|
||||
1. Visit [https://openrouter.ai/](https://openrouter.ai/)
|
||||
2. Sign up or log in
|
||||
3. Go to **API Keys** section
|
||||
4. Create new API key
|
||||
5. Add to `.env` file as `OPENROUTER_API_KEY`
|
||||
|
||||
### Choose AI Model
|
||||
|
||||
Edit `AI_MODEL` in `.env`:
|
||||
|
||||
```env
|
||||
# Best quality (recommended)
|
||||
AI_MODEL=anthropic/claude-3.5-sonnet
|
||||
|
||||
# Budget option (free)
|
||||
AI_MODEL=meta-llama/llama-3.1-70b-instruct
|
||||
|
||||
# OpenAI
|
||||
AI_MODEL=openai/gpt-4-turbo
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Step 1: Generate SEO Proposals
|
||||
|
||||
Fetch all posts and generate AI-powered SEO suggestions:
|
||||
|
||||
```bash
|
||||
python fetch_posts_and_generate_seo.py
|
||||
```
|
||||
|
||||
**Options:**
|
||||
|
||||
```bash
|
||||
# Test with first 5 posts
|
||||
python fetch_posts_and_generate_seo.py --limit 5
|
||||
|
||||
# Specify output file
|
||||
python fetch_posts_and_generate_seo.py --output my_proposals.csv
|
||||
|
||||
# Use rule-based generation (no AI/API costs)
|
||||
python fetch_posts_and_generate_seo.py --no-ai
|
||||
```
|
||||
|
||||
This creates a CSV file in `output/` directory with proposals for all posts.
|
||||
|
||||
### Step 2: Review Proposals
|
||||
|
||||
1. Open the generated CSV file in Excel or Google Sheets
|
||||
2. Review each row:
|
||||
- Check `proposed_seo_title` (should be 50-60 chars)
|
||||
- Check `proposed_meta_description` (should be 150-160 chars)
|
||||
- Edit proposals if needed
|
||||
3. Set `status` column to `approved` for changes you want to apply
|
||||
4. Set `status` column to `rejected` for posts to skip
|
||||
5. Save the CSV file
|
||||
|
||||
**CSV Columns:**
|
||||
|
||||
| Column | Description |
|
||||
|--------|-------------|
|
||||
| post_id | WordPress post ID |
|
||||
| post_url | Post permalink |
|
||||
| post_title | Original post title |
|
||||
| current_seo_title | Current SEO title (from Yoast/Rank Math) |
|
||||
| current_meta_description | Current meta description |
|
||||
| proposed_seo_title | AI-generated SEO title |
|
||||
| proposed_meta_description | AI-generated meta description |
|
||||
| primary_keyword | Detected primary keyword |
|
||||
| title_length | Character count of proposed title |
|
||||
| description_length | Character count of proposed description |
|
||||
| title_validation | Validation message |
|
||||
| description_validation | Validation message |
|
||||
| generation_method | 'ai' or 'rule-based' |
|
||||
| status | Set to 'approved' to apply changes |
|
||||
| notes | Your notes (optional) |
|
||||
|
||||
### Step 3: Test with Dry Run
|
||||
|
||||
Before applying changes, test with dry-run mode:
|
||||
|
||||
```bash
|
||||
python apply_approved_changes.py --input output/seo_proposals_YYYYMMDD_HHMMSS.csv --dry-run
|
||||
```
|
||||
|
||||
This shows what would be updated without actually making changes.
|
||||
|
||||
### Step 4: Apply Approved Changes
|
||||
|
||||
Apply the approved changes to WordPress:
|
||||
|
||||
```bash
|
||||
python apply_approved_changes.py --input output/seo_proposals_YYYYMMDD_HHMMSS.csv
|
||||
```
|
||||
|
||||
The script will:
|
||||
1. Create a rollback CSV with original values
|
||||
2. Ask for confirmation
|
||||
3. Apply all approved changes
|
||||
4. Generate detailed log file
|
||||
|
||||
## Workflow
|
||||
|
||||
### Complete Workflow Diagram
|
||||
|
||||
```
|
||||
1. Generate Proposals
|
||||
└─> python fetch_posts_and_generate_seo.py
|
||||
└─> Fetches all posts from WordPress
|
||||
└─> Generates AI-powered SEO suggestions
|
||||
└─> Exports to CSV: output/seo_proposals_YYYYMMDD_HHMMSS.csv
|
||||
|
||||
2. Review & Edit
|
||||
└─> Open CSV in Excel/Google Sheets
|
||||
└─> Review proposed titles and descriptions
|
||||
└─> Edit as needed
|
||||
└─> Set status='approved' for changes to apply
|
||||
└─> Save CSV
|
||||
|
||||
3. Test (Optional)
|
||||
└─> python apply_approved_changes.py --input <csv> --dry-run
|
||||
└─> Simulates changes without applying
|
||||
|
||||
4. Apply Changes
|
||||
└─> python apply_approved_changes.py --input <csv>
|
||||
└─> Creates rollback CSV
|
||||
└─> Applies approved changes to WordPress
|
||||
└─> Generates log file
|
||||
|
||||
5. Verify
|
||||
└─> Check WordPress admin (post editor)
|
||||
└─> View source on frontend
|
||||
└─> Monitor search performance
|
||||
```
|
||||
|
||||
### Safety Features
|
||||
|
||||
- **Dry Run Mode**: Test without applying changes
|
||||
- **Rollback CSV**: Automatically created before applying changes
|
||||
- **Detailed Logging**: All operations logged to `output/application_log_YYYYMMDD_HHMMSS.txt`
|
||||
- **Validation**: Enforces character limits and checks for duplicates
|
||||
- **Confirmation Prompt**: Requires 'yes' confirmation before applying changes
|
||||
- **Rate Limiting**: Prevents overwhelming WordPress server
|
||||
|
||||
## SEO Plugin Comparison
|
||||
|
||||
### Should You Switch from Yoast to Rank Math?
|
||||
|
||||
**Current: Yoast SEO Free**
|
||||
- ✓ Market leader (12M users)
|
||||
- ✓ Reliable and well-tested
|
||||
- ✗ Only 1 focus keyword (vs unlimited in Rank Math)
|
||||
- ✗ No redirect manager (premium only, $118.80/year)
|
||||
- ✗ Limited schema support
|
||||
- ✗ No internal linking suggestions
|
||||
|
||||
**Alternative: Rank Math Free**
|
||||
- ✓ **Unlimited focus keywords** (vs 1 in Yoast Free)
|
||||
- ✓ **Redirect manager included** (premium in Yoast)
|
||||
- ✓ **20+ rich snippet types** (FAQ, Product, Recipe, etc.)
|
||||
- ✓ **Better performance** (40% less code)
|
||||
- ✓ **Internal linking suggestions**
|
||||
- ✓ **Google Trends integration**
|
||||
- ✓ **One-click Yoast migration** (preserves all data)
|
||||
- ✗ Smaller community (900K vs 12M users)
|
||||
|
||||
**Recommendation for FREE users:** Switch to Rank Math Free
|
||||
|
||||
**Migration Steps:**
|
||||
1. Install Rank Math plugin
|
||||
2. Run Setup Wizard → Import from Yoast
|
||||
3. All SEO data automatically transferred
|
||||
4. Deactivate (don't delete) Yoast as backup
|
||||
5. Test a few posts
|
||||
6. If satisfied, delete Yoast
|
||||
|
||||
**These scripts work with both plugins** - they auto-detect which one you're using.
|
||||
|
||||
## SEO Best Practices (2026)
|
||||
|
||||
### Title Optimization
|
||||
- **Length**: 50-60 characters (≤600 pixels in SERPs)
|
||||
- **Keyword placement**: Primary keyword in first 60 characters
|
||||
- **Uniqueness**: Every post must have unique title
|
||||
- **Compelling**: Written to improve click-through rate (CTR)
|
||||
- **Natural**: No keyword stuffing
|
||||
|
||||
### Meta Description Optimization
|
||||
- **Length**: 150-160 characters (optimal for SERP display)
|
||||
- **User intent**: Address what reader will learn/gain
|
||||
- **Keyword inclusion**: Primary keyword appears naturally
|
||||
- **Uniqueness**: Every post must have unique description
|
||||
- **Value proposition**: Highlight what makes content unique
|
||||
- **CTR focused**: Compelling language to encourage clicks
|
||||
|
||||
**Note**: Google rewrites 62%+ of meta descriptions, but they still matter for:
|
||||
- CTR when not overridden
|
||||
- Social media sharing (Open Graph)
|
||||
- Signaling relevance to search engines
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Error: "Authentication failed"
|
||||
|
||||
**Cause**: Invalid WordPress username or application password
|
||||
|
||||
**Solution**:
|
||||
1. Verify username is correct (not email address)
|
||||
2. Regenerate application password in WordPress
|
||||
3. Update `.env` file with new password
|
||||
4. Ensure no extra spaces in credentials
|
||||
|
||||
### Error: "Access forbidden"
|
||||
|
||||
**Cause**: User doesn't have permission to edit posts
|
||||
|
||||
**Solution**:
|
||||
1. Ensure user has Editor or Administrator role
|
||||
2. Check if REST API is disabled by security plugin
|
||||
3. Temporarily disable security plugins and test
|
||||
|
||||
### Error: "OpenRouter API key invalid"
|
||||
|
||||
**Cause**: Invalid or missing OpenRouter API key
|
||||
|
||||
**Solution**:
|
||||
1. Get API key from https://openrouter.ai/
|
||||
2. Update `OPENROUTER_API_KEY` in `.env`
|
||||
3. Ensure no extra quotes or spaces
|
||||
|
||||
### Error: "No posts found"
|
||||
|
||||
**Cause**: No published posts or authentication issue
|
||||
|
||||
**Solution**:
|
||||
1. Verify you have published posts in WordPress
|
||||
2. Check authentication is working (see WordPress Configuration)
|
||||
3. Try with `--limit 1` to test with single post
|
||||
|
||||
### SEO Plugin Not Detected
|
||||
|
||||
**Cause**: Plugin not installed or meta fields not exposed
|
||||
|
||||
**Solution**:
|
||||
1. Verify Yoast SEO or Rank Math is installed and activated
|
||||
2. Check if custom code blocks meta field access
|
||||
3. Scripts default to Yoast field names if detection fails
|
||||
|
||||
### AI Generation Fails
|
||||
|
||||
**Cause**: OpenRouter API error or rate limit
|
||||
|
||||
**Solution**:
|
||||
1. Check OpenRouter account has credits
|
||||
2. Try different AI model (switch to free Llama model)
|
||||
3. Use `--no-ai` flag for rule-based generation
|
||||
4. Check log files for specific error messages
|
||||
|
||||
## Cost Estimates
|
||||
|
||||
### OpenRouter API Costs
|
||||
|
||||
**Using Claude 3.5 Sonnet (Recommended):**
|
||||
- Average post: ~2000 tokens input + 200 tokens output
|
||||
- Cost per post: ~$0.009
|
||||
- **100 posts: ~$0.90**
|
||||
- **1000 posts: ~$9.00**
|
||||
|
||||
**Using Free Models:**
|
||||
- Llama 3.1 70B: **$0.00** (free tier)
|
||||
- No cost for generation
|
||||
|
||||
**Rule-Based Generation:**
|
||||
- No API costs
|
||||
- Use `--no-ai` flag
|
||||
- Lower quality but free
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
/Users/acid/Documents/seo/
|
||||
├── .env # Your credentials (git-ignored)
|
||||
├── .env.example # Example configuration
|
||||
├── .gitignore # Git ignore rules
|
||||
├── requirements.txt # Python dependencies
|
||||
├── config.py # Configuration loader
|
||||
├── seo_generator.py # SEO generation logic
|
||||
├── fetch_posts_and_generate_seo.py # Main fetching script
|
||||
├── apply_approved_changes.py # Application script
|
||||
├── README.md # This file
|
||||
└── output/ # Generated files
|
||||
├── seo_proposals_*.csv # Generated proposals
|
||||
├── rollback_*.csv # Backup files
|
||||
└── application_log_*.txt # Detailed logs
|
||||
```
|
||||
|
||||
## Development Notes
|
||||
|
||||
### Testing
|
||||
|
||||
**Test with small batch first:**
|
||||
|
||||
```bash
|
||||
# Generate proposals for 5 posts
|
||||
python fetch_posts_and_generate_seo.py --limit 5
|
||||
|
||||
# Review CSV and approve changes
|
||||
|
||||
# Dry run to verify
|
||||
python apply_approved_changes.py --input output/seo_proposals_*.csv --dry-run
|
||||
|
||||
# Apply to 5 posts
|
||||
python apply_approved_changes.py --input output/seo_proposals_*.csv
|
||||
```
|
||||
|
||||
**Verify changes:**
|
||||
1. Open WordPress post editor
|
||||
2. Check Yoast/Rank Math SEO box shows updated title and description
|
||||
3. View source on frontend: check `<title>` and `<meta name="description">` tags
|
||||
4. Test rollback CSV if needed
|
||||
|
||||
### Extending the Scripts
|
||||
|
||||
**Add custom validation:**
|
||||
- Edit `seo_generator.py` → `validate_seo_title()` and `validate_meta_description()`
|
||||
|
||||
**Change AI model:**
|
||||
- Edit `.env` → `AI_MODEL=openai/gpt-4-turbo`
|
||||
|
||||
**Customize prompts:**
|
||||
- Edit `seo_generator.py` → `_generate_with_ai()` method
|
||||
|
||||
**Add more meta fields:**
|
||||
- Edit scripts to include focus keywords, Open Graph tags, etc.
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
1. Check this README troubleshooting section
|
||||
2. Review log files in `output/` directory
|
||||
3. Test with `--dry-run` mode first
|
||||
4. Start with `--limit 5` for testing
|
||||
|
||||
## License
|
||||
|
||||
This tool is provided as-is for WordPress SEO optimization. Use responsibly and always backup your WordPress site before bulk updates.
|
||||
|
||||
## Changelog
|
||||
|
||||
### Version 1.0.0 (2026-02-15)
|
||||
- Initial release
|
||||
- AI-powered SEO generation via OpenRouter
|
||||
- Support for Yoast SEO and Rank Math
|
||||
- CSV review workflow
|
||||
- Safety features (dry-run, rollback, logging)
|
||||
- Auto-detection of SEO plugins
|
||||
Reference in New Issue
Block a user