← Back to all projects

Proxy Router

Proxy Router project screenshot

A high-performance proxy aggregation and routing engine written in Go, designed to manage, validate, rotate, and route traffic through tens of thousands of proxies simultaneously. Features automatic proxy rotation, intelligent routing logic, high-throughput concurrency, REST API, Firefox extension, and multi-platform Docker support.

View Source

Technology

frontendhtmljavascript
backendgo
databasesqlitemysql
devopsdocker
toolspythonfirefoxsocks5

🌐 High-Throughput Proxy Router & Multi-Proxy Outbound Network Engine (Go-Based)

Long-Form Project Summary for KNWS Showcase

This project is a high-performance proxy aggregation and routing engine written primarily in Go, designed to download, manage, validate, rotate, and route traffic through tens of thousands of proxies simultaneously. It acts as a network router for multiple machines or applications, enabling them to send traffic through large rotating proxy pools for privacy, security, research, scraping, or load distribution.

The system is engineered for extreme concurrency, supporting thousands of simultaneous outbound connections, dynamically rotating proxies, and intelligent routing rules for speed, resiliency, and anonymity.

🎯 Project Goals

  1. Build a massively parallel proxy ingestion and routing system, capable of handling tens of thousands of proxies of different types (HTTP, HTTPS, SOCKS4/5).
  2. Allow multiple client machines or apps to route outbound traffic through a single unified router.
  3. Provide automatic proxy rotation for every outbound request, ensuring anonymisation and distributing load.
  4. Implement custom routing logic, such as IP-based routing overrides, sticky sessions, and fallback behaviour.
  5. Support high-throughput, low-latency connection forwarding suitable for large scraping systems or security use cases.
  6. Include additional tooling such as a Firefox extension, Python clients, and metrics APIs.
  7. Ship with multi-platform builds, Docker support, environment automation, and cross-platform deployability.

🧱 Core Features

🌍 1. Proxy Aggregation: Tens of Thousands of Sources

The engine can:

  • Download proxy lists from large numbers of public or private sources
  • Parse mixed formats (IP:Port, JSON, CSV, provider-specific formats)
  • Validate proxy health and speed
  • Maintain a massive internal proxy pool
  • Categorise proxies by protocol, speed, reliability, region, anonymity

The resulting proxy list is continuously refreshed and quality-ranked.

🔀 2. Automatic Proxy Rotation

Every outbound connection routed through the server will:

  • Use a unique proxy
  • Rotate on every request
  • Recycle only healthy proxies
  • Avoid banned/burned IPs
  • Optionally maintain sticky sessions where needed

The rotation logic ensures even distribution and avoids reuse patterns that can trigger rate limits or detection systems.

🌐 3. Acts as a Full Network Router for Multiple Clients

Your system serves as a network-level proxy router, where:

  • Multiple devices
  • Multiple apps
  • Entire scraping clusters

can point traffic at a single endpoint and have it transparently routed through rotating proxies.

Examples:

  • A local browser's requests
  • Backend servers
  • Cron jobs
  • LLMs or automation agents fetching data

This effectively turns your router into a traffic anonymisation gateway.

🧩 4. High-Performance Go Architecture

The engine uses:

  • Goroutines for extreme concurrency
  • Channels for safe, scalable async data flows
  • go-chi/chi for API routing
  • modernc.org/sqlite (pure-Go sqlite)
  • armon/go-socks5 for SOCKS5 server functionality
  • Custom HTTP CONNECT tunnelling
  • Memory-safe, low-overhead patterns

The architecture is optimised for handling thousands of simultaneous outbound requests.

📦 5. Multiple Proxy Types Supported

The router supports:

  • HTTP
  • HTTPS
  • SOCKS4
  • SOCKS5
  • Custom dialects (provider-specific)

This enables compatibility across the entire proxy ecosystem.

🧠 6. Intelligent Routing Logic

The system includes logic for:

  • Auto fallback when a proxy fails
  • Timeout statistics
  • Geolocation-aware routing
  • IP-based rulesets
  • Whitelisting/blacklisting
  • Special-case routing for specific domains
  • Sticky sessions (optional)
  • Per-client proxy isolation

Documentation exists in:

  • SPECIAL_IP_ROUTING_README.md
  • CUSTOM_PROXY_IMPLEMENTATION_SUMMARY.md

📊 7. Monitoring & Timeout Statistics API

A built-in stats API provides:

  • Success/failure rates
  • Average response times
  • Proxy health monitoring
  • Timeout counts
  • Connection throughput metrics

A Python client is bundled for integration with external systems.

🔌 8. Firefox Extension (Optional Component)

The repository includes a full Firefox extension:

  • Routes browser traffic through the proxy router
  • Toggles routing on/off
  • Displays proxy rotation indicators
  • Can apply custom rules per site

Ideal for testing, browsing through rotating proxies, or UI-based validation.

🐳 9. Dockerised Multi-Platform Build System

The project includes:

  • Multi-architecture builds (amd64, arm64)
  • Docker Compose files
  • Automated .env loader
  • Environment propagation into containers
  • Cross-platform build scripts in /builds/

Making deployment simple on:

  • Proxmox
  • Linux servers
  • macOS development machines
  • Raspberry Pi clusters

🛠️ Technology Stack

Primary Language

  • Go (Golang) – concurrency-heavy proxy routing logic

Libraries

  • go-chi/chi – REST API/HTTP routing

  • modernc.org/sqlite – pure-Go SQLite (no cgo required)

  • armon/go-socks5 – SOCKS5 server implementation

  • Custom Go modules for:

    • Proxy validation
    • Rotation
    • Connection multiplexing
    • Timeout logging

Supporting Components

  • Python client tools
  • Bash utilities
  • Firefox WebExtension
  • Docker + multi-platform builds
  • MySQL (infrastructure layers for user mgmt / logs)

Infrastructure

  • Runs as a local or remote router
  • Supports thousands of concurrent outbound sessions
  • Instant proxy rotation and failover
  • Works with any client supporting HTTP/SOCKS5 proxies

📈 Outcome & Impact

This project demonstrates deep knowledge in:

  • Network programming
  • High-concurrency Go architecture
  • Proxy technologies (HTTP, SOCKS4/5)
  • Traffic anonymisation & rotation patterns
  • Large-scale ingestion pipelines
  • Router behaviour & connection multiplexing
  • Multi-platform deployment
  • API design and client integration
  • Cross-language tooling (Go, Python, JS/Firefox extension)
  • Building secure, performance-critical infrastructure

The result is a battle-hardened, high-throughput proxy router capable of powering:

  • Web scraping farms
  • Security or penetration testing infrastructures
  • Research clusters
  • Privacy networks
  • Automated AI agents needing anonymised browsing
  • Local devices needing enhanced privacy

All running on KNWS-owned infrastructure with full control and no dependency on external proxy management services.