PATRICK RIBBSAETER
CH•
Engineering portfolioPRIVATE AI INFRASTRUCTURE
PRODUCTION ENGINEERING

Private AI Inference and Gateway Reliability

Engineered a private inference architecture with resource-aware model selection, loopback-only Ollama infrastructure, system-managed tunneling, and verified GhidraGPT integration.

Role

  • Private inference
  • LLM deployment
  • Hermes & Ollama architecture
  • Provider routing diagnostics
  • Secure networking

Category

AI Infrastructure

Engineering discipline

PRODUCTION ENGINEERING

Host Memory Limit:7.6 GiB RAM
Capacity Decision:4B profile selected for 7.6 GiB host
Qwen3 4B Profile:8,192 ctx · 4,096 output

Engineering objective

System challenge

A Hetzner-hosted AI environment was running Hermes gateway services, Ollama, Docker, communication integrations, scheduled jobs, and a large provider fallback chain. The requirement was to understand why the system could fall back unpredictably while preserving service availability and protecting secrets.

Constraints

Working within real limits

  • ✕Private Linux VPS with 7.6 GiB total host memory
  • ✕Ollama endpoint required to remain loopback-only (127.0.0.1) without public application port exposure
  • ✕Client environment (GhidraGPT 1.2.1 / Ghidra 12.1.2) required a secure, system-managed connection path
  • ✕Strict non-destructive audit requirement for Phase 1 (no service restarts, firewall mutations, or secret changes)
  • ✕Zero exposure of host IP addresses, API credentials, secret tokens, or internal filesystem paths

Architecture

Interactive Gateway & Inference Diagram

Interactive Architecture FlowPhase 1 (Audit) & Phase 2 (Verified Private Path)

1. GhidraGPT 1.2.1

Connected via System-Managed SSH Tunnel

↓

2. Hermes Gateway Router

Configured provider chain & drift isolated

Upstream Fallbacks

20-entry chain: Rate & quota limits

Verified Private Path

Loopback-only Ollama

↓

3. Qwen3 4B Profile

8,192 Context · 4,096 Output Limit · Fits 7.6 GiB RAM

Smoke Test: GHIDRAGPT_OK

Verified Private Path
Audited Fallback Chain
Loopback Tunnel
SVG moving signal · Supports prefers-reduced-motion

Resource Capacity Decision

Sizing Model Footprint Against VPS Memory

Resource Allocation & Decision Trade-OffHost Constraint vs. Model Warm-Up Requirements
Total Host RAM (Private VPS)7.6 GiB Total

Host memory running Hermes gateway, Ollama, Docker, containerd, and SSH

✕Qwen3 8B Deployment Profile~10.6 GB Required — REJECTED

Rejected during capacity evaluation because the approximately 10.6 GB warm-up requirement exceeded 7.6 GiB total host RAM and introduced an unacceptable memory-exhaustion and stability risk.

✓Qwen3 4B Deployment ProfileSELECTED & DEPLOYED
Context Limit8,192 tokens
Output Limit4,096 tokens
Endpoint127.0.0.1 Loopback
Smoke TestGHIDRAGPT_OK

Approach

How it was built

  1. 01

    Context and environment mapping: Mapped running services across Hermes gateway, Ollama, Docker, containerd, and SSH to establish a complete operational baseline on the 7.6 GiB VPS.

  2. 02

    Infrastructure diagnostics: Mapped provider configuration against active CLI state, analyzed the 20-entry routing chain, identified the canonical Hermes runtime, and documented network-control requirements while preserving live availability and credential confidentiality.

  3. 03

    Resource-aware model decision: Evaluated the Qwen3 8B warm-up profile requiring approximately 10.6 GB against 7.6 GiB total host RAM. Rejected the 8B profile because it exceeded host capacity and introduced an unacceptable memory-exhaustion risk, then selected a resource-safe Qwen3 4B profile configured with an 8,192-token context window and 4,096-token output limit.

  4. 04

    Controlled private delivery and tunneling: Connected GhidraGPT 1.2.1 (Ghidra 12.1.2) to the private Ollama endpoint through a loopback-only, system-managed SSH tunnel. Executed an end-to-end provider smoke test returning GHIDRAGPT_OK.

Technical validation

Implementation evidence

Observable engineering evidence. Discipline: PRODUCTION ENGINEERING

Host Memory Limit

7.6 GiB RAM

Capacity Decision

4B profile selected for 7.6 GiB host

Qwen3 4B Profile

8,192 ctx · 4,096 output

Ollama Binding

Loopback-only (127.0.0.1)

SSH Tunnel Path

System-managed loopback

Provider Smoke Test

GHIDRAGPT_OK

GhidraGPT Version

1.2.1

Ghidra Version

12.1.2

Fallback Inspection

20-entry chain audited

Audit Integrity

Zero secret/IP exposure

Outcome

What was delivered

  • Delivered a stable, resource-aware private inference path running Qwen3 4B that fits within 7.6 GiB host RAM

  • Isolated and documented configuration drift between Hermes primary provider configuration and active CLI state

  • Preserved network isolation by maintaining loopback-only Ollama endpoints backed by system-managed SSH tunneling

  • Verified end-to-end provider response returning GHIDRAGPT_OK for GhidraGPT 1.2.1 and Ghidra 12.1.2

  • Produced an operational hardening map for provider routing and network controls without disrupting live availability

Technologies

Stack and tools

Private inferenceLLM deploymentHermesOllamaQwen3 4BLinuxProvider routingSystems diagnosticsSecure networking

Next engineering system

AI product engineering

Ethereal Campaign Director

Have an ambitious AI system to build?

Let’s build something serious.

I work across AI architecture, product engineering, private inference, automation, SaaS infrastructure, and production hardening.

Discuss a projectConnect on LinkedInTechnical profile