craig
6eff12e8ac
SmartCrawl plugin reference (11 docs): - Overview, configuration, titles & meta, schema, sitemaps, social/OG, redirects, breadcrumbs, content analysis, hooks/filters, known issues SEO strategy playbook (6 docs): - Keyword research, per-page-type selection, on-page optimization, local SEO, technical SEO, content strategy Reusable templates (3): - Site audit checklist, keyword research worksheet, page optimization checklist Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
3.5 KiB
Sitemaps
SmartCrawl generates three types of sitemaps: XML, Google News, and Image. All are configured under the Sitemap module.
XML Sitemap
Configuration
| Setting | Recommended Value |
|---|---|
| Enable XML Sitemap | Yes |
| Post types included | Posts, Pages, Products, public custom post types |
| Taxonomies included | Categories (yes), Tags (only if indexed), custom taxonomies |
| Max URLs per file | Default (auto-splits into multiple files) |
| Notify search engines | Automatic |
Access
Your sitemap is at: https://yoursite.com/sitemap.xml
SmartCrawl auto-generates a sitemap index that links to individual sitemaps per post type and taxonomy.
Exclusions
Exclude specific content by:
- Post/page ID — Enter IDs in the exclusion list
- Custom URLs — Add specific URLs to exclude
- Post type — Uncheck entire post types from the include list
Important: If you set a page to noindex, also exclude it from the sitemap. SmartCrawl does not do this automatically.
Limitation
SmartCrawl limits sitemaps to 1,000 most recent posts per post type to prevent memory errors on large sites. If you have more than 1,000 posts, older content may not appear in the sitemap.
Google News Sitemap
Enable this only if the site publishes news-style content that qualifies for Google News.
Requirements for Google News
- Content must be timely, original news articles
- Published within the last 2 days (Google News requirement)
- Site should be registered with Google News Publisher Center
Configuration
- Enable the Google News Sitemap toggle
- Select which post types to include (usually just Posts)
- Verify at
https://yoursite.com/news-sitemap.xml
Image Sitemap
One-click enable. Tells Google about images on your pages for Google Images indexing.
When to Enable
- Sites with significant visual content (portfolios, galleries, products)
- Sites where image search traffic is valuable
- Always enable for WooCommerce sites (product images)
What It Includes
SmartCrawl adds image entries to your existing XML sitemap (not a separate file). Each page entry includes its images with:
- Image URL
- Image title
- Image caption (if set)
Sitemap Submission
Google Search Console
- Go to Search Console > Sitemaps
- Enter
sitemap.xmlin the "Add a new sitemap" field - Click Submit
- Verify status shows "Success" after Google fetches it
Bing Webmaster Tools
- Go to Bing Webmaster Tools > Sitemaps
- Submit
https://yoursite.com/sitemap.xml
robots.txt
Add a sitemap directive to robots.txt (use full absolute URL):
Sitemap: https://yoursite.com/sitemap.xml
After Making Changes
When you add/remove content or change sitemap settings:
- Use the "New Crawl" button in SmartCrawl's sitemap tab to notify Google
- Or wait for automatic notification (if set to Automatic mode)
- Verify the updated sitemap at
yoursite.com/sitemap.xml - Check Google Search Console > Sitemaps for any errors
Troubleshooting
| Issue | Fix |
|---|---|
| Sitemap returns 404 | Flush permalinks (Settings > Permalinks > Save) |
| Post type missing from sitemap | Check SmartCrawl Sitemap settings — ensure the post type is included |
| Noindexed pages appearing in sitemap | Manually exclude them by ID — SmartCrawl doesn't auto-exclude noindexed content |
| WPML duplicate URLs | Known issue — duplicate homepage URLs in sitemap when using WPML |
| Sitemap too large / memory error | SmartCrawl limits to 1,000 posts — contact WPMU DEV support for large sites |