Frequently Asked Questions (FAQ) & Troubleshooting

This document addresses common operational questions, error codes, and troubleshooting steps for IndexFlow.

Technical SEO & Quality Gate

Q1: Why does IndexFlow mark 301/302 redirects as gate failures (HTTP 301 / HTTP 302)?

Answer: IndexFlow’s prober intentionally uses a non-following HTTP policy (redirect::Policy::none()). Sitemaps submitted to search engines must only contain canonical, final destination URLs that return 200 OK. Submitting URLs that redirect wastes crawl budget, increases latency, and can result in canonical confusion.

Q2: Why did my URL fail with "Canonical URL mismatch"?

Answer: IndexFlow compares the inspected page URL against the declared <link rel="canonical" href="..."> tag. A failure occurs if:

1. The canonical points to a different domain, protocol (http vs https), or path.
2. The URL path has different casing (e.g. /Blog/Post vs /blog/post), as URL paths are case-sensitive according to RFC 3986.
3. The canonical tag is empty or points to a non-canonical parameter variant.

Note: IndexFlow automatically tolerates insignificant differences such as trailing slashes (except root), default ports (:80, :443), URL fragments (#section), and reordered query parameters.

Q3: Why am I getting "decompression bomb: inflated size exceeds 52428800 bytes"?

Answer: Google's sitemap protocol specifies a 50 MiB uncompressed limit per sitemap document. IndexFlow enforces this limit (MAX_UNCOMPRESSED_BYTES = 50 * 1024 * 1024) to protect server memory against malicious gzip bombs. If your sitemap exceeds 50 MiB when uncompressed, split it into smaller sitemaps linked by a <sitemapindex>.

Search Engine Integrations & Quotas

Q4: How does IndexFlow handle Google Search Console 429 quota exhaustion?

Answer:

1. Google caps the URL Inspection API at 2,000 requests per day per project.
2. If Google returns HTTP 429, the GscInspectWorker trips an immediate circuit breaker and halts inspection to prevent project suspension.
3. Best Practice: Use the Sync Google Impressions feature (POST /api/v1/pipeline/gsc-inspect/sync). This queries the Search Analytics API (which has generous limits) to bulk-mark already-indexed URLs, consuming 0 URL Inspection quota.

Q5: How does the Google Indexing API daily quota reset?

Answer: The Google Indexing API permits 200 URL notifications per day. IndexFlow calculates quota consumption over a rolling 24-hour window:

• When the 200-submission limit is reached, IndexFlow calculates the exact timestamp when the oldest submission in the window turns 24 hours old (next_free_at).
• sites.google_quota_paused_until is set to this timestamp.
• The worker automatically resumes processing once the window frees up new capacity.

Q6: Why is Bing returning UNKNOWN during inspection?

Answer: Bing Webmaster Tools enforces a strict 1.5 QPS rate limit. When an account exceeds this limit, Bing responds with ThrottleHost or ErrorCode: 5. When IndexFlow detects this, it:

1. Keeps the URL in bing_index_status = 'UNKNOWN'.
2. Sleeps for 2 seconds.
3. Retains the URL in queue for automatic retry on the next tick without counting it as a failure.

Q7: How do I bypass Cloudflare Bot Fight Mode or 403 Forbidden errors?

Answer:

1. Go to /settings in IndexFlow and add a secret token under Cloudflare WAF 放行密钥 / 自定义 User-Agent (e.g. MySeoSecretToken888).
2. In the Cloudflare Dashboard, navigate to Security -> WAF -> Custom Rules.
3. Create a rule:
   • Field: User Agent
   • Operator: contains
   • Value: MySeoSecretToken888
   • Action: Skip -> Select WAF Managed Rules, Bot Fight Mode, and Rate Limiting.

System & Database Operations

Q8: How do I reset a forgotten administrator password?

Answer: Since IndexFlow stores administrator accounts in SQLite with bcrypt hashing, you can reset the password directly via the sqlite3 CLI:

# Generate a new bcrypt hash (example password: "NewPassword123!")
# Or update the admin user to a known hash:
sqlite3 /var/lib/indexflow/indexflow.db \
  "UPDATE admin_users SET password_hash = '\$2b\$12\$e8O9yqL1jNqZ7qZ1e6dF0eXm8w8B9g0F.4qL5u8v7w6x5y4z3a2b1' WHERE username = 'admin';"

Alternatively, delete the database record to re-trigger the initial /setup wizard:

sqlite3 /var/lib/indexflow/indexflow.db "DELETE FROM admin_users;"

Q9: What should I do if I see "database is locked" errors in SQLite?

Answer: IndexFlow automatically initializes SQLite in WAL (Write-Ahead Logging) mode with busy_timeout = 5000ms, which allows concurrent readers alongside active writers. If you still encounter locks:

1. Ensure no external processes (such as external backup scripts or SQLite GUI tools) are holding an open write transaction.
2. Confirm the database file resides on a local NVMe/SSD filesystem, not a network-mounted volume (NFS/CIFS), as network filesystems often fail to support POSIX advisory locking.