Installation & Setup

This document covers system requirements, build instructions from source, containerized execution, and database initialization for self-hosting IndexFlow.

System Requirements

IndexFlow is designed to be lightweight and resource-efficient.

Minimum Hardware

• CPU: 1 vCPU (x86_64 or aarch64)
• RAM: 512 MB (1 GB recommended during compilation)
• Disk: 500 MB free disk space (depends on database size and number of tracked URLs)

Supported Operating Systems

• Linux: Ubuntu 20.04+, Debian 11+, Alpine Linux 3.18+, RHEL/Rocky Linux 8+
• macOS: 12+ (Apple Silicon & Intel)
• Windows: Windows 10/11 or Windows Server 2019+ (native or via WSL2)

Method 1: Building from Source (Recommended for Linux/macOS)

1. Install Dependencies

Ensure the Rust compiler, Cargo, OpenSSL, and SQLite development libraries are installed:

Ubuntu / Debian

sudo apt-get update
sudo apt-get install -y build-essential pkg-config libssl-dev sqlite3 libsqlite3-dev git
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source "$HOME/.cargo/env"

Alpine Linux

apk add --no-cache build-base pkgconfig openssl-dev sqlite-dev git curl
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source "$HOME/.cargo/env"

macOS

xcode-select --install
brew install openssl sqlite git

2. Clone the Repository

git clone https://github.com/IndexFlowing/IndexFlow-core.git
cd IndexFlow-core

3. Compile the Binary

Build an optimized release binary:

cargo build --release

The compiled binary will be located at:

target/release/indexflow-core

You can optionally install it into your system path:

sudo cp target/release/indexflow-core /usr/local/bin/

Method 2: Docker Deployment

You can run IndexFlow in a container without installing the Rust toolchain locally.

1. Dockerfile

Create a Dockerfile in your working directory:

# Build stage
FROM rust:1.75-bookworm AS builder
WORKDIR /app
COPY . .
RUN cargo build --release

# Runtime stage
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
    ca-certificates \
    libssl3 \
    libsqlite3-0 \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app
COPY --from=builder /app/target/release/indexflow-core /app/indexflow-core
COPY --from=builder /app/migrations /app/migrations

# Create directory for persistent SQLite storage
RUN mkdir -p /data

ENV SERVER_HOST=0.0.0.0
ENV SERVER_PORT=8080
ENV DATABASE_URL=sqlite:/data/indexflow.db?mode=rwc
ENV RUST_LOG=indexflow_core=info,tower_http=info

EXPOSE 8080
VOLUME ["/data"]

CMD ["/app/indexflow-core"]

2. Build and Run

# Build the Docker image
docker build -t indexflow:latest .

# Run the container with persistent volume
docker run -d \
  --name indexflow \
  -p 8080:8080 \
  -v $(pwd)/data:/data \
  -e JWT_SECRET="your-secure-jwt-secret-key-at-least-32-chars" \
  -e DRY_RUN="false" \
  indexflow:latest

Database Initialization & Auto-Migrations

IndexFlow uses SQLite as its embedded primary database. You do not need to install an external PostgreSQL or MySQL instance.

Automatic Migration Execution

When indexflow-core boots up, it automatically executes embedded SQL migrations via sqlx::migrate!("./migrations"):

1. 001_init.sql: Sets up core tables (site_config, urls, admin_users, submission_logs, health_checks).
2. 002_multi_site.sql: Upgrades schema to support multi-tenancy (sites table and site_id foreign keys).
3. 003_bing_inspection.sql: Adds Bing Webmaster inspection fields (bing_index_status, bing_coverage_state).
4. 004_sitemap_synced_at.sql: Adds crawl tracking timestamps.
5. 005_seo_warnings.sql: Adds Schema.org types, AI bot blocking, and OpenGraph audit fields.
6. 006_discovery_source.sql: Tracks URL discovery origin (sitemap vs gsc_orphan).
7. 007_custom_user_agent.sql: Adds Cloudflare WAF bypass token support.
8. 008_indexing_watchlist.sql: Implements URL monitoring watchlist and index_status_history.

SQLite WAL Mode

IndexFlow configures SQLite in Write-Ahead Logging (WAL) mode on startup:

PRAGMA journal_mode = WAL;
PRAGMA synchronous = NORMAL;
PRAGMA busy_timeout = 5000;

This enables concurrent reads while background workers write audit and submission logs without locking the database.

Verifying the Installation

Check the server's health status via HTTP:

curl -s http://localhost:8080/health | json_pp

Expected output:

{
  "db": "sqlite-wal",
  "service": "indexflow-core",
  "status": "healthy"
}