> ✅ CLEANED & CURRENT — 2026-07-04. The live instructional content (PART 0–4, PART 6 quick-start) has been brought to current reality: MiniMax-M3 (OpenRouter/Gemini/DeepSeek/Mistral provider walkthroughs removed) · canon_shim memory (60K/40K, Honcho gone) · scraper-stack · Twilio+Retell phone (Telnyx procedure removed). PART 5: Problems & Solutions is a dated HISTORICAL INCIDENT ARCHIVE — it intentionally references now-retired models/providers as a record of past fixes; do NOT re-apply its model/provider steps. For any conflict, PART 0 §0.2 (Current Standards) is authoritative. Format & colors are final.
This is now the ONE canonical copy. Six other copies of this manual (a legacy standalone HTML on 3 branches, a stale Google Doc, and a hand-authored section embedded in the live dashboard) have been retired or flagged for retirement — seedecisions/DECISIONS.md(2026-07-04 entry). Edit only this file;MANUAL.html/MANUAL.for-docs.htmlare generated from it bybuild_manual.py— never hand-edited. ⚠️ Every cron in §0.2 below was invalidated by the 2026-06-30 cron wipe (trigger.dev overreach deleted 24 of Lisa's crons; 9 were restored 2026-07-02, King-approved, and are NOT the same 9 that existed before). The cron list below is rewritten from that live-VPS-verified restore, not from memory of the pre-wipe state.
Step-by-step procedure to build ONE agent, PARAMETERIZED to scale to 10,000. This is the AUTHORITATIVE build process. Where it conflicts with the legacy Phase guide below, THIS WINS. Last verified: 2026-07-04 (reconciled against agents/lisa/profile.yaml, VPS-verified 2026-07-02 — the freshest ground truth in the project; see agents/_template/ for the full reproducible-template system this Part 0 blueprint now shares duties with). RULE: every step is tagged [VERIFIED] (built for Lisa and confirmed working) or [TODO] (needs detail/test). Nothing is "done" until [VERIFIED].
• AGENT_NAME (e.g. "Lisa") • AGENT_ROLE (e.g. "executive assistant") • NICHE/VERTICAL • OWNER_NAME · OWNER_EMAIL · OWNER_TELEGRAM_ID · OWNER_PHONE • VPS: provider=Hostinger, plan=KVM2 (8GB), VPS_ID, VPS_IP • AGENT_EMAIL (free Gmail or domain alias) • TELEGRAM_BOT_TOKEN • PHONE: carrier=TWILIO (LOCKED), AGENT_PHONE_NUMBER, VOICE_AGENT_NAME (e.g. "IVY") • MEMORY: provider=canon_shim (canon stack source-of-truth + holographic local recall; Honcho REMOVED), bounded core MEMORY.md/USER.md, recall_mode=hybrid, cap 60K/40K • LLM: primary=MiniMax-M3 (provider=minimax, base_url https://api.minimax.io/v1). No fallback chain. Vision is auxiliary (provider=auto, model=auto — NOT pinned to a model name). → Building agent N = filling this block + running §0.3. No hand-assembly, no hardcoded "Lisa".
• LLM: primary MiniMax-M3 via the minimax custom provider, base_url https://api.minimax.io/v1. NO fallback chain. Valid model ids: MiniMax-M3 / MiniMax-M1 — do NOT use MiniMax-Text-01 (its 40K max_output is below the gateway's max_tokens, so it 400s). Vision is routed AUXILIARY (provider=auto, model=auto — NOT pinned to a model name; no separate Gemini/DeepSeek vision route). ⚠️ DEAD — never set: model id minimax3 (400 "unknown model"), endpoint api.minimax.chat (decommissioned → 401 "User not found"), gemini-2.5-flash, deepseek-v4-pro, gpt-5.3-codex, mistral-small, kimi-k2, OpenRouter, or direct-anthropic. [ALL legacy "Sonnet via OpenRouter", "DeepSeek v3", and "gemini-2.5-flash" sections below are OBSOLETE.] • WEB SCRAPING: canonical tool is scraper-stack (/opt/data/scripts/scraper-stack/scrape.py) — a tiered router that goes free-first (Tier 0 requests/BeautifulSoup → Tier 1 local Crawl4AI headless) and only uses paid APIs on a hard block (Tier 2 ScraperAPI/Zyte → Tier 3 Firecrawl LLM-extract). Replaces the scattered legacy scrape scripts. Reads keys from /opt/data/.env. Tier 1 uses a PINNED Chromium at /opt/data/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome (override env SCRAPER_STACK_CHROME_BIN) — the pin keeps JS pages on the FREE tier instead of silently falling through to paid ScraperAPI. (General pattern for any tool that bundles its own Chromium.) • PHONE CARRIER: TWILIO ONLY (Telnyx is BANNED — purge every legacy Telnyx reference). IVY's live number is +1-817-632-6536, a Twilio number PROVISIONED THROUGH RETELL (the original Telnyx number 817-973-0830 was banned and replaced June 2026). ⚠️ NUANCE: there are NO direct Twilio API keys (TWILIO_ACCOUNT_SID / AUTH_TOKEN / PHONE) in /opt/data/.env — the number lives at Retell's phone-provisioning layer, not via a local Twilio SDK. Voice agent IVY is Retell-resident (api.retellai.com), not a process on this VPS. SMS alerts to King go via a Gmail→carrier gateway: 9455452529@vtext.com (Verizon). [TODO: rewrite legacy Section 13 for Twilio + IVY + Retell.] • VOICE I/O: STT (voice-memo transcription) = provider groq, model whisper-large-v3-turbo (LIVE since 2026-06-22; before this, voice memos arrived as silent attachments). TTS = provider edge, voice en-US-AriaNeural (fallbacks configured: elevenlabs, openai, xai, minimax). [This is why a groq provider block exists in config.yaml — it is for STT, not the chat model.] • CRONS (9 active, verified live 2026-07-02, restored post-wipe — keep under the Rule of 30): nightly-github-backup (06:00 daily, backs up /opt/data to GitHub), daily-credit-report (13:00 daily), morning-health-check (13:00 daily, Cloudflare tunnel health + self-heal), daily-diagnostic (11:00 daily, zero-LLM infra report), google-token-refresh (every 50m), kanban-watchdog (/5 — NOTE: the kanban board itself is RETIRED in favor of ClickUp; this watchdog is a retirement candidate, not a live dependency), ensure-retell-handler (/5, supervisor for the :8643 call handler), ensure-inbox-handler (/5, supervisor for the email inbox-wake handler), validate-critical-keys (hourly). ⚠️ This entire list replaces the pre-2026-06-30 set below the wipe line — none of the earlier canon-heartbeat / email-triage-live / ghl-refund-monitor / lisa-broker-watch / memory-curator cron names survived the wipe+restore; the memory curator now runs built-in rather than via its own cron (watchdog_cron_discrepancy flagged in agents/lisa/profile.yaml). High-frequency work runs as daemons + a light supervisor cron, never a cron that does the work. • PORTS: gateway 8642 (127.0.0.1, docker-proxy) · broker 4097 (host, 0.0.0.0) · 8643 serves THREE path-routed jobs inside the container (/retell Retell fn-handler, /broker-push wake receiver = 172.16.1.2:8643, /ivy_brain webhook) · OpenCode push 127.0.0.1:4098/push · container also publishes 4860 = ttyd (pid 7, Hermes Agent web-terminal — browser-based shell to the container, basic-auth enabled, 0.0.0.0, NOT in Cloudflare tunnel as of 2026-06-26). 9118 = Hermes dashboard (pid 262, hermes dashboard --port 9118 --host 127.0.0.1 --no-open --skip-build --insecure) and 9119 = auth proxy (pid 288, dashboard_auth_proxy.py) — BOTH LIVE as of 2026-06-26 re-verify (Lisa). Prior 2026-06-23 DOWN note was stale; these are the active Hermes web UI surfaces. • COMMS MESH: the bilateral-broker systemd service runs /opt/data/scripts/broker.py with KNOWN_AGENTS = {"opencode", "lisa", "code", "kim"} → QUAD-AGENT MESH (wired 2026-06-26). ⚠️ SUPERSEDED for Kim, 2026-07-01 (reconciled 2026-07-04):* this line originally said PUSH endpoints exist only for lisa and opencode, with kim POLL-ONLY — that is now WRONG. Kim has had a real push endpoint since 2026-07-01 (http://172.16.2.2:8643/broker-push — note the corrected IP, .2 not .1; see "The Trigger.dev Durable Delivery Spine" section for the full current mesh topology). Her old poll path (kim-send, 172.16.2.1:4097) still exists as a backstop cron, not her primary path anymore. code remains genuinely POLL-ONLY (NAT'd workstation, no push handler possible) — that part of this line is still accurate. code polls via code_broker_poller.py, reached for delivery via the gateway API on 8642. Kim subnet: 172.16.2.0/24, firewall rule in /opt/data/scripts/broker-firewall.sh (boot-persistent). Plus a Code↔OpenCode file bridge at ~/.hermes-bridge/. ⛔ DRIFT TRAP: a STALE bilateral copy of broker.py exists at /docker/hermes-agent-jemg/data/scripts/broker.py ({"opencode","lisa"}) — it is the ORPHAN, never loaded; do NOT copy it over the live /opt/data one or you kill code and kim routing. • MEMORY: provider=canon_shim (LIVE 2026-06-23, commit 80c205e). Bounded core (MEMORY.md/USER.md, cap raised to 60K/40K from 15K/12K) + holographic provider (local SQLite runtime recall) backed by the canon stack (scripts/canon/) as source-of-truth, recall_mode set to hybrid (NOTE: hybrid recall — G6 FTS5 + sqlite-vec/FastEmbed — is PLANNED, NOT yet wired; current live recall is FTS5-only, the S2 canon_shim→RecallIndex prefetch does not exist yet), plus a nightly auto-extraction pipeline. Under canon_shim, fact_store WRITES are gated → pending proposals for review, READS delegate to holographic; canon_verify GREEN. ⚠️ The canon-heartbeat cron this section used to cite did NOT survive the 2026-06-30 wipe+restore (see §0.2 CRONS above) — memory drift monitoring is currently a gap, not an active safeguard; the curator itself still runs built-in. Phase B (flip reads to canon) is DEFERRED + King-gated. [Honcho REMOVED/DEPRECATED 2026-06-20 — uncorrectable resurrection bug; Lisa migrated off it, verified healthy.] • AGENT↔CODE COMMS: email, and ALWAYS CC the shared Gmail (cfmbusiness@gmail.com) so the agent can read the full message body via the Gmail API (Hostinger IMAP body-read is unreliable). [Root cause verified 2026-06-07.] • OWNERSHIP: all writes to /opt/data as the hermes user (never root). Provisioning/lifecycle via the Hostinger VPS API (programmatic), not manual dashboard clicks.
• VPS-NATIVE FLEET (LOCKED 2026-06-26 — the rule that lets this scale to 10,000 agents): Nothing fleet-critical runs on King's laptop. Agents are BORN on the VPS (the new-agent-onboard factory provisions them there); fleet services (memory curator, PM engine, fleet analytics) are VPS systemd, addressable by agent-ID — never Windows Task Scheduler, never a laptop path. King's laptop is a remote control, not the engine. NUANCE: King's PERSONAL tooling (e.g. insight-XL personal-session analytics) MAY be laptop-native because its data is generated there — but that data MUST be centralized (synced to the VPS/central store) so it follows King across machines. Litmus test: "if King swaps laptops, does this keep running AND see all its data?" If no → not VPS-native enough.
⚡ EXECUTOR: /new-agent-onboard skill (built 2026-06-26, pm-rebuild/ws6-agents/new-agent-onboard/). Invoke it to run these phases interactively — Phase 7 (broker wire) is fully automated (proven on Kim); Phases 1–6 are grounded in this manual with a few ⚠️ CONFIRM placeholders being filled from Manual SECTION 4. Dry-run by default (LIVE=1 to execute). This is the agent factory — do NOT hand-assemble agent N+1.
PHASE 1 — PROVISION VPS. Goal: a running KVM2 box. Steps: Hostinger VPS API create + set root password (programmatic). Verify: API GET shows state=running. STATUS: [VERIFIED control plane works 2026-06-07; TODO: script end-to-end create]. PHASE 2 — INSTALL HERMES. Goal: gateway running. Steps: scripted docker run of nousresearch/hermes, mount /opt/data. Verify: docker ps + dashboard reachable. STATUS: [TODO: convert manual Phase 4 to a script]. PHASE 3 — LLM CONFIG. Goal: primary model live. Steps: hermes config set model.default MiniMax-M3; provider minimax (base_url https://api.minimax.io/v1 set in config.yaml → providers.custom:minimax.base_url); NO fallback chain. Verify: hermes config get model.default + a live reply. STATUS: [VERIFIED on Lisa]. PHASE 4 — IDENTITY. Goal: agent persona loaded. Steps: render SOUL.md + USER.md from templates using §0.1 params; fix the SOUL.md symlink to point at the real file (legacy Problem 56). Verify: agent introduces itself correctly; SOUL.md non-empty. STATUS: [VERIFIED — /new-agent-onboard executor (pm-rebuild/ws6-agents/new-agent-onboard/) handles Phase 4 automatically with per-agent params from §0.1 interview, 2026-06-26]. PHASE 5 — MEMORY. Goal: semantic + person-aware memory. Steps: hermes memory setup → canon_shim provider (canon stack + holographic; Honcho is GONE); recall_mode hybrid; cap is PER-AGENT (Lisa=60K/40K, Kim=80K — set at provision time, never migrate debt from another agent); deploy curator daemon (900s interval, 75% high-water) + supervisor cron + canon-heartbeat (every 30m). Verify: hermes memory status shows provider active + canon_verify GREEN + curator 7/7 audit PASS. STATUS: [VERIFIED on Lisa (60K) + Kim (80K) — Phase A; Phase B read-flip King-gated]. PHASE 6 — TELEGRAM. Goal: owner can talk to agent. Steps: BotFather token → hermes config. Verify: round-trip message. STATUS: [carry from legacy Phase 7]. PHASE 7 — COMMS BRIDGE. Goal: Code↔agent autonomous email. Steps: agent reads cfmbusiness via Gmail API; Code emails agent CC cfmbusiness; (optional) cloud routine to wake Code. Verify: real round-trip with substance, not just ack. STATUS: [PARTIAL — root cause fixed 2026-06-07, CC-always rule]. PHASE 8 — VOICE AGENT (IVY). Goal: outbound-capable voice agent. Steps: Retell + TWILIO trunk; dynamic begin-message (outbound persona); /think, /place_call, /save_task; anti-hallucination prompt V3+. Verify: live outbound test call with correct opening. STATUS: [TODO: write from current IVY blueprint; purge Telnyx]. PHASE 9 — QA / SMOKE TEST. Goal: confirm the agent works before handoff. Steps: python scripts/canon/reliability_gate.py --selftest (6 gates: LLM responds, memory healthy, Telegram reachable, canon clean, curator running, broker wired) + curator audit 7/7. Verify: all gates GREEN. Agent is SHIP-ELIGIBLE only after this passes. STATUS: [VERIFIED — reliability_gate.py is the checklist; proven on Lisa (6/6) + Kim (6/6), 2026-06-26].
TRAP 1 — Dead MiniMax endpoint / model. Symptom: 401 "User not found" or 400 "unknown model" on every Lisa reply. Cause: api.minimax.chat is DECOMMISSIONED (→ 401); model id minimax3 does not exist (→ 400); MiniMax-Text-01 400s because its 40K max_output < gateway's max_tokens. Fix: base_url: https://api.minimax.io/v1; model MiniMax-M3. ⚠️ Valid ids: MiniMax-M3, MiniMax-M1, MiniMax-M2. Never use minimax3, api.minimax.chat, or MiniMax-Text-01.
TRAP 2 — Broker orphan. Symptom: added an agent to KNOWN_AGENTS but messages misroute or agent is unknown. Cause: edited the STALE container copy at /docker/hermes-agent-jemg/data/scripts/broker.py instead of the live HOST copy at /opt/data/scripts/broker.py. The container copy is an orphan — systemd never loads it. Fix: ALWAYS edit /opt/data/scripts/broker.py (HOST). After editing, systemctl restart bilateral-broker on the host.
TRAP 3 — Port collision. Symptom: new agent container won't start, or an existing service dies silently. Cause: the new agent's ports clash with a live service. Fix: before docker run, check docker ps + ss -tlnp on the host. Known assignments: 8642=Lisa gateway, 8643=Lisa retell+broker-push, 4097=broker, 4098=opencode listener, 4860=ttyd, 4861=Kim gateway, 8644=Kim gateway (host-local — ⚠️ confirmed unreachable 2026-07-04, curl 127.0.0.1:8644/health → connection refused; Kim's gateway binds to localhost INSIDE her own container, so even though Docker publishes container-port 8642→host-port 8644, the host still can't reach it — this is why Kim needed her own push-receiver process instead, see §0.5 mesh notes), 9118=dashboard, 9119=auth proxy.
TRAP 7 — Claude Code's own safety classifier blocks cross-service credential moves silently. Symptom: an action (editing .claude/settings.json permissions, or moving an API key from one service to another, e.g. uploading a live gateway key to a third-party cloud task runner) is auto-blocked with no prompt, even though the Bash allowlist covers it. Cause: Claude Code has a safety classifier separate from the project's own permission allowlist — allowlist entries do NOT override it, and it can flag cross-service credential moves as "[Data Exfiltration]" even when legitimate. Fix: switch Claude Code's permission mode from "Auto mode" to "Default"/"Ask before edits" to get an actual approval prompt, do the action, switch back if needed. (Hit 2026-06-30/07-01 while wiring Lisa's gateway key into trigger.dev's prod env vars.)
TRAP 4 — Secrets in tracked files. Symptom: Telegram bot token, API key, or password visible in git history. Cause: writing config to PROGRESS.md, SOUL.md, or any git-tracked file instead of .env. Fix: ALL tokens and secrets go in the agent's /opt/data/.env (inside container) or .env in the Hermes Agents repo (.gitignore'd). Never commit them. If already committed: git filter-repo or scrub + force-push. (Caught live 2026-06-26: Kim's Telegram bot token was in PROGRESS.md; scrubbed by AI project management tab. Recurred 2026-07-03: Lisa's live Telegram bot token was hardcoded in 5 tracked files — not just the 2 originally flagged — including scripts/health-check-cron.sh and scripts/watch-code-inbox.ps1; the Hostinger API token was similarly live in 3 tracked files including PROGRESS.md itself. Both scrubbed to .env-only lookups; rotation of both tokens is King-owed and tracked in .tmp/SECRETS_TO_FIX.md. Two occurrences of the same trap is a pattern, not a fluke — a pre-commit secret scanner would catch this class of bug before it lands; worth adding as a TODO.)
TRAP 5 — New agent subnet not in firewall. Symptom: kim-send health returns connection refused; agent can't reach broker on port 4097. Cause: each agent container is on its own Docker network (172.16.X.0/24); only Lisa's subnet (172.16.1.0/24) was in the original firewall rule. Fix: ufw allow from <SUBNET>/24 to any port 4097 + iptables -I DOCKER-USER -s <SUBNET>/24 -p tcp --dport 4097 -j ACCEPT + append to /opt/data/scripts/broker-firewall.sh for boot persistence.
TRAP 6 — No-migration-debt rule. Symptom: new agent starts with stale facts, old identity bleed, wrong memory caps from a prior agent's snapshot. Cause: copying Lisa's memory files instead of seeding fresh. Fix: every new agent gets a FRESH empty memory on day one — same 4-layer architecture (canon_shim, curator, cap per §0.1), zero content from Lisa. Kim was the reference build: "No migration debt — started on final architecture." (Honcho's resurrection bug was exactly this: identity bleed from stale memory.)
[TODO: migrate verified lessons from Sessions 24–57 (root-owned /opt/data outages, cloudflared not auto-starting, paid-model budget drain, Hostinger key-only SSH, email body-read failure) — full traps in new-agent-onboard SKILL.md §0.4 section.]
This section is the live state of the FIRST agent (Lisa) and the team around her — verified against the running VPS, not from memory. It supersedes any contradictory legacy passage below.
⭐ Since 2026-07-02 there is a more granular, actively drift-checked version of this exact section for Lisa (and Kim) at agents/lisa/profile.yaml + narrative.md — the Hermes Agent Template (Phase 1). It's checked against the live VPS by a daily automated drift-checker (scripts/agent_template_drift.py, flags-never-overwrites) rather than manual re-verification. When this §0.5 and the template disagree, trust the template — it's newer and machine-checked. Building agent N+1 should start from the template (Kim was instantiated from it in ~7 minutes), not from copy-pasting this section.
THE TEAM — 7 agents (genders LOCKED 2026-06-19):
| Agent | Gender | Role | Model | Where it runs |
|---|---|---|---|---|
| Lisa | F | COO / executive assistant | MiniMax-M3 | VPS (Hermes) port 8642 |
| Kim | F | Growth & Marketing / AutomateYourBiz | MiniMax-M3 | VPS (Hermes) port 4861/8644 — wired 2026-06-26 |
| Ivy | F | Phone receptionist / voice | GPT-5.1 | Retell platform — +1-817-632-6536 |
| Claude Code | M | Tech lead / workstation engineer | Claude Opus 4.8 | Workstation (VS Code) |
| Claude Cowork | M | Desktop assistant | TBD | Workstation |
| OpenCode | M | VPS engineer | opencode/deepseek-v4-flash (Code/King-verified, not VPS-checkable) | VPS (OpenCode CLI) |
| Kode | F | VPS control plane | TBD | VPS control |
| Claude in Chrome | M | Browser worker | Claude Sonnet | Chrome extension |
⚠️ Kode is the VPS control plane — the Hostinger API key is held by Code, NOT Lisa. (This is the exact spot where Kode gets dropped from the roster; the pointer is deliberate.)
MEMORY — canon stack (durable truth):
canon_shim is the active memory provider. Canon DB = /opt/data/data/canon.db (NOT /opt/data/shared) — 74 facts (59 lisa.notes + 14 system + 1 lisa; canon row id=74, lisa_memory_architecture, auto-approved 2026-06-24). Backups every 30 min./opt/data/memory_store.db) is the active recall provider UNDER canon_shim (vector-semantic recall) — not a separate provider. Bounded core MEMORY.md/USER.md capped 60K/40K.scripts/canon/). Do NOT conflate the two.write_approval: true flag is a NO-OP). Phase 1 (read-flip) is King-gated.watchdog_cron_discrepancy in agents/lisa/profile.yaml.KIM MEMORY — verified 2026-06-26:
canon_shim from day 1 (no migration debt — started on final architecture, zero content copied from Lisa).kim in KNOWN_AGENTS, kim-send inside container (http://172.16.2.1:4097), kim-broker on VPS host (http://127.0.0.1:4097). Poll-only (no push endpoint — no retell handler on port 8643 in Kim's container).BACKUP / SYNC: /opt/data/scripts/github-sync.sh — nightly backup of critical files to a private GitHub repo; 3-retry push with exponential backoff; Telegram alert on FINAL failure only.
OPERATIONAL HEALTH WARNINGS (live):
kanban.db figures below are historical — do not triage them, and treat any other legacy passage in this manual that references the kanban board as stale. kanban-watchdog cron is a retirement candidate (see §0.2 CRONS).kanban.db — 712 tasks, of which 334 are BLOCKED (a structural smell worth triaging). Dispatcher runs every 60s (max_spawn 3, failure_limit 2).~~ (retired system — kept struck-through for history only)state.db — 757 MB of session transcripts (2,593 sessions, 77,001 messages, FTS5 + trigram indexes). Lifetime spend across all sessions ≈ $132.45. If any legacy passage implies state.db is small, that is wrong.code_broker_poller.py now includes a sender-aware ROUTING DISCIPLINE block in the wake prompt; forbids embedded reply-pointer lines. Lisa-side (container cron that builds Lisa's wake prompt) is still KING-GATED — that patch requires a container touch scheduled for next awake window. Until then, Lisa may still misroute replies to opencode instead of code (~8 recurrences/day).HERMES AGENT MASTER MANUAL — CFM BUSINESS Version 4.0 FINAL | May 2026 Single Source of Truth — Edit This Document Directly. Never Create a New Doc. Google Doc ID: (this document)
PART 1: UNDERSTANDING HERMES Section 1: What Is a Hermes Agent? Section 2: Internal Makeup — Architecture & Components 2A. The Five Pillars 2B. Memory System (USER.md + MEMORY.md) 2C. Skills System & Progressive Disclosure 2D. Soul (SOUL.md) 2E. Cron Jobs — Basic & Advanced 2F. LLM Providers & Models 2G. Session Storage & Search (SQLite)
PART 2: SETUP GUIDE Section 3: Pre-Setup Requirements & Security Preparation Section 4: Phase-by-Phase Setup (Phases 1–10) Section 5: Browser Infrastructure Setup (Phase 11 — 11 Sub-Phases)
PART 3: USE CASES Section 6: Personal Life Use Cases Section 7: Business Life Use Cases
PART 4: OPERATIONS Section 8: Dashboard Guide (Kanban, Sessions, Platforms, Memory, Crons) Section 9: Telegram Interface & Context Window Management Section 10: Scaling Decision Tree & Maintenance Section 11: Security Best Practices
PART 5: PROBLEMS & SOLUTIONS Section 12: 59 Common Problems with Step-by-Step Solutions (includes Quick Index)
APPENDIX: Glossary of Terms
Hermes is an open-source autonomous AI agent created by Nous Research. It is one of the most widely adopted AI agent frameworks in the world, with over 140,000 GitHub stars and an MIT license — meaning it is completely free to use and modify.
DEFINITION: Autonomous AI Agent — An AI system that doesn't just answer questions but takes independent actions in the real world: browsing websites, sending messages, managing files, running scheduled tasks, and more — without a human needing to be present for each action.
Unlike conversational tools like ChatGPT or Claude, Hermes is designed to ACT. You give it a goal, connect it to tools, and it works toward that goal on your behalf — 24 hours a day, 7 days a week — running on a server that never turns off.
KEY COMPARISON:
THE CFM DEPLOYMENT: At CFM Business, Hermes agents are deployed on Hostinger VPS servers (cloud computers that run 24/7). They are controlled primarily through Telegram. The first agent deployed is Lisa, who handles Upwork candidate screening and recruitment pipeline tasks.
KEY FACTS:
minimax provider)WHY A VPS AND NOT YOUR OWN COMPUTER? Your personal computer turns off. A VPS runs continuously in a data center. This means your agent can execute cron jobs at 3am, respond to Telegram messages while you sleep, and process tasks over the weekend — all without any action from you.
HERMES ON ANDROID (TERMUX): For personal experimentation, Hermes can run directly on an Android phone using Termux (an Android terminal emulator). Install from F-Droid (NOT Google Play — the Play version is outdated), then install Python, Node.js, and Git. Not recommended for production use: the phone must stay on and connected, and RAM is limited. Good for testing concepts before committing to a VPS.
Understanding Hermes's internal structure lets you customize it correctly and troubleshoot it when things go wrong. Everything about how the agent thinks, remembers, and acts comes down to five core pillars.
PILLAR 1 — MEMORY The agent's persistent knowledge about you, your preferences, and important context. Memory survives every session restart, container restart, and VPS reboot — because it lives in text files on disk, not in RAM. Two files: USER.md (permanent facts) and MEMORY.md (working memory that evolves). The agent reads both at the start of every conversation.
PILLAR 2 — SKILLS Pre-written playbooks that tell the agent how to handle specific task types. Stored as .md files in ~/.hermes/skills/. The key innovation is progressive disclosure: at startup, Hermes reads only the YAML metadata from each skill file (not the full content), so it knows what skills exist without loading everything into the context window. Full skill content loads only when a skill is triggered.
PILLAR 3 — SOUL The agent's identity, personality, values, and behavioral rules. Stored in SOUL.md. This is what makes "Lisa" different from "a generic AI assistant." It defines the agent's name, communication style, what it will and won't do, and how it handles edge cases.
PILLAR 4 — CRONS Hermes's scheduling system. Cron jobs run automatically at set times — no human needs to start a conversation. This is what makes the agent truly autonomous. A cron can check emails every 30 minutes, generate a daily report at 5pm, or run any task on any schedule.
PILLAR 5 — SELF-IMPROVING LOOP Hermes can analyze past conversations, identify gaps in its knowledge, and update its own MEMORY.md. This makes the agent progressively smarter about your preferences, projects, and work style without manual updates from you.
⚠️ SUPERSEDED for current standards (2026-06-23) → see PART 0 §0.1/§0.2 + PHASE 5. Memory is now provider=canon_shim (canon stack source-of-truth + holographic recall, bounded-core cap 60K/40K). Honcho is REMOVED. The USER.md/MEMORY.md mechanics below are still accurate; any Honcho references or old cap numbers (15K/12K) are OBSOLETE. Kept as historical record.
Hermes uses two memory files stored at /opt/data/ inside the Docker container:
USER.md — PERMANENT MEMORY Facts about the user that almost never change. The agent reads this at every conversation start.
Contents to include:
Example USER.md: ---
Name: [Your Name] Preferred Name: [What to call you] Location: [City, State] Timezone: Central (CDT, UTC-5)
Company: CFM Business Role: [Your role] Focus: Upwork staffing, recruitment, business operations
---
MEMORY.md — WORKING MEMORY Evolving information that updates regularly. The agent writes to this after significant sessions.
Contents:
FILE LOCATIONS: /opt/data/USER.md /opt/data/MEMORY.md
CRITICAL: If these files do not exist, the agent starts every conversation as if meeting you for the first time. Creating these files is one of the most important setup steps.
HOW MEMORY PERSISTS: The /opt/data/ directory is mounted from the VPS host into the Docker container (via -v /opt/data:/opt/data in the docker run command). Even if the container is deleted and recreated, data in /opt/data/ survives.
Skills are the agent's playbooks — step-by-step instructions for specific task types. They live as markdown files in ~/.hermes/skills/.
SKILL FILE STRUCTURE: Every skill has two parts:
Metadata the agent reads at startup to know the skill exists and when to use it.
The actual step-by-step instructions. Only loaded when the skill is triggered.
Example skill file: --- --- name: reply-to-upwork trigger: "When asked to check or reply to Upwork messages" description: "Handles Upwork inbox monitoring and candidate reply workflow" requires: [browser, proxy] ---
Check the Upwork inbox and reply to new candidate messages using approved templates.
/opt/data/templates/
---
PROGRESSIVE DISCLOSURE EXPLAINED: When Hermes starts, it reads ONLY the YAML headers of all skill files. This means:
This allows you to have 20+ skills without the agent starting every conversation with a bloated context window.
SKILL PRECEDENCE: Skills override SOUL.md for their specific domain. If SOUL.md says "always ask for confirmation before sending messages" but the Upwork skill says "send replies autonomously," the skill wins when it's active.
HOW TO CREATE A SKILL:
AGENTS.MD — LOCAL PROJECT CONTEXT: When using Hermes via terminal (CLI mode) inside a specific project folder, you can create an agents.md file in that directory. This is similar to CLAUDE.md — it gives the agent project-specific context without cluttering global memory. The agent reads agents.md automatically when running CLI sessions in that directory.
SOUL.md is the agent's identity file. It defines who the agent is at a fundamental level.
What SOUL.md contains:
Example SOUL.md opening: --- You are Lisa, an autonomous recruitment assistant for CFM Business. Your primary role is managing the Upwork hiring pipeline — screening candidates, sending templated replies, and keeping the pipeline moving.
You communicate professionally but warmly. You are proactive: if you notice something important, you flag it without being asked. You are honest about your limitations.
You never take irreversible actions (sending emails, making purchases, deleting files) without explicit confirmation unless you have been given specific autonomous permission for that action type. ---
IMPORTANT — MODEL CHOICE: The CFM standard model is MiniMax-M3 (via the minimax provider), which respects SOUL.md instructions. If an agent ever ignores its custom personality or behavioral rules, first confirm the model is MiniMax-M3 and the provider is minimax (some retired models — e.g. safety-filtered GPT-class models — would override SOUL.md). See PART 0 §0.2.
SOUL.md file location: /opt/data/SOUL.md
Crons are what make Hermes truly autonomous. They run tasks on a schedule without any human input.
BASIC CRON SYNTAX: Uses standard Unix cron format: minute hour day month weekday
Examples:
Creating a cron: docker exec -it hermes hermes cron add \ --schedule "0 9 *" \ --prompt "Check my Upwork inbox and summarize new candidate messages"
ADVANCED CRON FLAGS:
context_from: [filepath] Loads a file's contents as additional context before the cron runs. Use case: Cron needs campaign-specific info without hardcoding it. Example: --context_from "/opt/data/weekly_context.md"
work_dir: [directory path] Sets the working directory for the cron session. Use case: Cron works with files in a specific project folder. Example: --work_dir "/opt/data/reports/"
no_agent: true Runs as a script without LLM reasoning. For tasks that don't need AI. Use case: File cleanup, log rotation, health checks. Example: --no_agent true
CRITICAL CRON RULE: Cron sessions CANNOT create new cron jobs. If a cron tries to create another cron, it fails silently. Cron creation must happen in a regular conversation session or via the dashboard.
CRON TIMEZONE NOTE: Hermes uses UTC by default. If you're in Central Time (UTC-5 in summer, UTC-6 in winter), add 5-6 hours to get UTC time:
The current LLM standard is MiniMax-M3 via the minimax provider (base_url https://api.minimax.io/v1), NO fallback chain — see PART 0 §0.2 for the authoritative config and §0.5 for live state. The legacy OpenRouter / Claude-Sonnet / DeepSeek / Gemini / Mistral provider walkthrough that lived here has been removed; CFM no longer uses that stack.
Every Hermes conversation is stored permanently in a SQLite database. This is a powerful and often overlooked capability.
DATABASE LOCATION: /opt/data/sessions.db
WHAT IS STORED:
HOW TO SEARCH SESSION HISTORY: From within a Hermes conversation (in Telegram), ask:
The agent queries the SQLite database and surfaces relevant past sessions.
WHY THIS MATTERS FOR CFM: Troubleshooting recurring issues: Ask the agent "When did the proxy last fail and what fixed it?" and it will find that session in history. Auditing: Confirm exactly what was sent to candidates and when. Continuity: Even if context window compacts, you can search for past work.
AUTO-COMPACTION: When a conversation reaches approximately 136,000 tokens, Hermes automatically summarizes and compresses older portions of the conversation. This allows conversations to continue past the model's context limit.
IMPORTANT: This happens more visibly in the CLI than in Telegram. In Telegram, auto-compaction may occur without obvious notification. The session is still stored in SQLite — you can always search back through history even if the current conversation has been compacted.
REQUIRED ACCOUNTS (create before starting):
REQUIRED INFORMATION:
SECURITY PREPARATION — DO THESE BEFORE SETUP:
STEP A: Create a dedicated email for this agent Create a brand new Gmail account specifically for this agent (e.g., lisacfm2026@gmail.com). REASON: If the agent's email is ever compromised, only that account is affected. Your personal and business emails stay safe. The agent needs an email for GitHub notifications, cron alerts, and integrations — never use your primary email for this.
STEP B: Plan your API key naming strategy When you create your MiniMax API key, name it specifically: Example: "Lisa-CFM-Upwork-Agent" (NOT "my key" or "key 1") REASON: Named keys let you see per-agent spending in your dashboard and revoke access for one agent without disrupting others.
PHASE 1: PROVISION YOUR VPS (HOSTINGER)
Step 1.1 — Log into Hostinger Go to hpanel.hostinger.com and log in.
Step 1.2 — Navigate to VPS Hosting Top menu: VPS → VPS Hosting → Order New VPS
Step 1.3 — Select the KVM 2 Plan Specifications:
This is sufficient for 1–2 Hermes agents running simultaneously.
Step 1.4 — Configure Your VPS
Step 1.5 — Enable Free Malware Scanner (Optional) During VPS setup, Hostinger offers a free malware scanner. Enable it if the option appears — it provides baseline security monitoring at no extra cost.
Step 1.6 — Wait for Provisioning VPS setup takes 2–5 minutes. You'll receive an email with your server's IP address.
Step 1.7 — Save Your Server Details Record in your local .env file (NEVER in Git or chat):
[SCREENSHOT: Hostinger VPS dashboard showing newly provisioned server with IP address]
PHASE 2: CONNECT TO YOUR VPS VIA SSH
Step 2.1 — Open a Terminal
Step 2.2 — Connect ssh root@YOUR_VPS_IP
Step 2.3 — Accept the Host Key First connection shows a security warning about the host key fingerprint. Type "yes" and press Enter.
Step 2.4 — Enter Password Type your root password. Characters will NOT appear as you type — this is normal SSH behavior.
Step 2.5 — Verify Success You should see: root@hermes-cfm-lisa:~#
[SCREENSHOT: Terminal showing successful SSH connection prompt]
PHASE 3: UPDATE SERVER & INSTALL PREREQUISITES
Step 3.1 — Update all packages apt update && apt upgrade -y (This may take 2–5 minutes)
Step 3.2 — Install required tools apt install -y curl wget git unzip python3 python3-pip
Step 3.3 — Install Docker curl -fsSL https://get.docker.com | sh
Step 3.4 — Verify Docker docker --version Output should show: Docker version 24.x.x or higher
[SCREENSHOT: Terminal showing Docker version output]
PHASE 4: INSTALL AND START HERMES
Step 4.1 — Pull the Hermes Docker image docker pull nousresearch/hermes:latest (Downloads ~1–2GB, takes 3–5 minutes)
Step 4.2 — Create the persistent data directory mkdir -p /opt/data
Step 4.3 — Run Hermes docker run -d \ --name hermes \ --restart unless-stopped \ -v /opt/data:/opt/data \ -p 3000:3000 \ nousresearch/hermes:latest
Flag explanations: -d = Run in background (detached) --name hermes = Name the container --restart unless-stopped = Auto-restart on server reboot -v /opt/data:/opt/data = Mount data directory for persistence -p 3000:3000 = Expose web dashboard on port 3000
Step 4.4 — Verify Hermes is running docker ps Look for "hermes" with status "Up X minutes"
Step 4.5 — Access the dashboard In your browser: http://YOUR_VPS_IP:3000 You should see the Hermes login/setup page.
[SCREENSHOT: Hermes dashboard first-launch screen in browser]
PHASE 5: CONFIGURE THE HERMES DASHBOARD
Step 5.1 — Create admin credentials On first launch, Hermes prompts for admin username and password. Save these in your .env file: HERMES_ADMIN_USER=your_username HERMES_ADMIN_PASSWORD=your_password
Step 5.2 — Explore the dashboard sections (Full guide in Section 8 — Dashboard Guide) Key areas: Kanban Board, Session Viewer, Connected Platforms, Cron Manager, Memory Viewer
[SCREENSHOT: Hermes dashboard main screen with kanban board]
PHASE 6: CONFIGURE YOUR LLM PROVIDER
The current CFM standard is MiniMax-M3 via the minimax provider, with NO fallback chain. (See PART 0 §0.2 / PHASE 3 for the authoritative config.)
Step 6.1 — Get your MiniMax API key
Step 6.2 — Add the key to Hermes docker exec -it hermes hermes config set MINIMAX_API_KEY "your-minimax-key-here" (Set the provider base_url in config.yaml: providers.custom:minimax.base_url=https://api.minimax.io/v1)
Step 6.3 — Set the model (no fallback chain) docker exec -it hermes hermes config set model.default MiniMax-M3 docker exec -it hermes hermes config set model.provider minimax
Step 6.4 — Verify docker exec -it hermes hermes config get model.default (should print MiniMax-M3; then send a live message to confirm a real reply)
[SCREENSHOT: Terminal showing hermes config with MiniMax-M3]
PHASE 7: CREATE AND CONNECT TELEGRAM BOT
Step 7.1 — Open Telegram, find @BotFather Search for "@BotFather" in Telegram (official blue checkmark account).
Step 7.2 — Create your bot Send: /newbot BotFather asks for:
Step 7.3 — Save your bot token BotFather gives you a token like: 123456789:ABCdefGHIjklmNOPqrstuvWXYZ Save in .env: TELEGRAM_BOT_TOKEN=123456789:ABCdef...
Step 7.4 — Add token to Hermes docker exec -it hermes hermes config set TELEGRAM_BOT_TOKEN "your-token"
Step 7.5 — Start Telegram integration docker exec -it hermes hermes telegram start
Step 7.6 — Test Open Telegram, find your bot by username, send "Hello" — the agent should respond within seconds.
[SCREENSHOT: Telegram conversation showing bot responding to first message]
PHASE 8: SET UP GITHUB BACKUP
IMPORTANT: Use a CLASSIC personal access token (not fine-grained). Fine-grained tokens have per-repository permission restrictions that cause backup failures. Classic tokens with "repo" scope work reliably with Hermes.
Step 8.1 — Create a private GitHub repository Name it something like "lisa-hermes-backup". Set visibility: Private.
Step 8.2 — Generate classic token GitHub → Settings → Developer Settings → Personal Access Tokens → Tokens (classic) → Generate new token (classic)
Step 8.3 — Add to Hermes docker exec -it hermes hermes config set GITHUB_TOKEN "ghp_your-token" docker exec -it hermes hermes config set GITHUB_REPO "yourusername/lisa-hermes-backup"
Step 8.4 — Enable and test docker exec -it hermes hermes backup enable docker exec -it hermes hermes backup now Check your GitHub repo — files should appear.
[SCREENSHOT: GitHub repository showing backed-up agent configuration files]
PHASE 9: CONFIGURE MEMORY FILES
Step 9.1 — Enter the container docker exec -it hermes bash
Step 9.2 — Create USER.md nano /opt/data/USER.md
Paste and customize:
Name: [Your Name] Preferred Name: [What to call you] Location: [City, State] Timezone: Central (CDT, UTC-5 summer / UTC-6 winter)
Company: CFM Business Focus: Upwork staffing and recruitment, business operations
[List your active projects here]
Save: Ctrl+O, then Ctrl+X
Step 9.3 — Create MEMORY.md nano /opt/data/MEMORY.md
Initial content:
[Agent will update this automatically after sessions]
[Agent will track ongoing work here]
[Add anything you always want the agent to remember]
[Document any recurring problems here]
Save and exit.
Step 9.4 — Verify memory is working In Telegram: "What do you know about me?" The agent should reference your USER.md details.
[SCREENSHOT: Telegram showing agent referencing user profile from USER.md]
PHASE 10: CREATE YOUR FIRST SKILL AND CRON
Step 10.1 — Create a skill docker exec -it hermes bash nano ~/.hermes/skills/daily-summary.md
Paste: --- name: daily-summary trigger: "When asked for a daily summary or end-of-day report" description: "Generates summary of today's activities and tomorrow's priorities" requires: [] ---
Generate a concise daily summary for the user.
Save and exit.
Step 10.2 — Create a basic cron docker exec -it hermes hermes cron add \ --schedule "0 17 1-5" \ --prompt "Run the daily-summary skill and send me a summary of today's work"
This runs at 5pm Monday–Friday (adjust to your timezone: UTC + your offset).
Step 10.3 — Verify docker exec -it hermes hermes cron list You should see the cron with its schedule and next run time.
[SCREENSHOT: Terminal showing hermes cron list with new cron entry]
This section sets up autonomous web browsing for your Hermes agent. Required for tasks like Upwork monitoring, web research, and form submission.
WHAT YOU ARE BUILDING:
WHY BOT DETECTION BLOCKS THE VPS: Websites like Upwork use Cloudflare and Forter bot detection. These systems identify datacenter IP addresses (which is what your VPS has) and block or challenge them. Use a real logged-in browser session (VNC manual login, saved cookies) or the site's official API instead.
--- Phase 11.1: Install Xvfb (Virtual Display) ---
apt install -y xvfb Xvfb :99 -screen 0 1920x1080x24 & export DISPLAY=:99
Make persistent (add to /etc/rc.local before "exit 0"): Xvfb :99 -screen 0 1920x1080x24 & export DISPLAY=:99
--- Phase 11.2: Install Google Chrome ---
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb apt install -y ./google-chrome-stable_current_amd64.deb google-chrome --version # Verify
--- Phase 11.3: Install ChromeDriver ---
apt install -y chromium-chromedriver
--- Phase 11.4: Start Chrome with CDP ---
DISPLAY=:99 google-chrome \ --headless \ --disable-gpu \ --remote-debugging-port=9222 \ --no-sandbox \ --disable-dev-shm-usage &
Verify CDP is active: curl http://localhost:9222/json (Should return JSON with browser tab info)
--- Phase 11.6: Install x11vnc (Optional — Visual Browser Monitoring) ---
apt install -y x11vnc x11vnc -display :99 -bg -nopw -listen localhost -xkb
This lets you visually watch the browser using a VNC client.
--- Phase 11.7: Install websockify (Optional — Browser-Based VNC) ---
pip3 install websockify websockify --web=/usr/share/novnc/ 6080 localhost:5900 &
Access in browser: http://YOUR_VPS_IP:6080 (Lets you watch the agent's browser without installing VNC client software)
--- Phase 11.8: Set Up Cloudflare Tunnel (Secure Access) ---
wget https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb dpkg -i cloudflared-linux-amd64.deb cloudflared tunnel login cloudflared tunnel create hermes-tunnel cloudflared tunnel run hermes-tunnel
--- Phase 11.9: One-Time Website Logins ---
Before autonomous browsing works, manually log in to each site once so session cookies are saved.
For Upwork:
--- Phase 11.10: Connect Browser to Hermes ---
docker exec -it hermes hermes config set BROWSER_CDP_URL "http://localhost:9222"
--- Phase 11.11: End-to-End Test ---
In Telegram, send to your agent: "Open a browser, go to google.com, take a screenshot, and show me."
If working: Agent returns a screenshot of Google. If failing: See Problem 7 in Section 12 (troubleshooting).
[SCREENSHOT: Telegram showing agent returning a screenshot of Google homepage]
USE CASE 1: MORNING BRIEFING Daily cron that sends a Telegram message at your preferred wake time.
Contents:
Setup: Daily cron at your wake time (remember UTC conversion) Example prompt: "Generate my morning briefing: weather, any calendar events today, yesterday's incomplete tasks, and suggest my top 3 priorities for today."
---
USE CASE 2: EMAIL TRIAGE Monitor inbox, categorize by importance, summarize and alert.
Setup: IMAP connection to your personal Gmail (App Password) Cron: Every 30–60 minutes Example prompt: "Check my Gmail for new emails. Categorize as Urgent, Important, or FYI. Send me a Telegram summary. If anything is from my bank, doctor, or family, alert me immediately."
---
USE CASE 3: RESEARCH ON DEMAND Ask the agent to research any topic and return a structured summary.
Setup: Web browsing enabled Usage: On demand via Telegram Example: "Research the top 5 residential proxy services. Compare them on price, pool size, speed, and reliability. Return a comparison table."
---
USE CASE 4: CONTENT CONSUMPTION SUMMARY Monitor YouTube channels and websites, summarize new content weekly.
Setup: Web browsing, optional YouTube API Cron: Weekly Example: "Every Monday, check these 5 YouTube channels [list] for new videos posted this week. Give me a 2-sentence summary of each new video."
---
USE CASE 5: SUBSCRIPTION MONITORING Track subscription renewals, recurring charges, price changes.
Setup: Email (bank alerts forwarded to agent email) Cron: Weekly Example: "Each week, review bank notification emails and compile a list of upcoming subscription renewals in the next 30 days."
---
USE CASE 6: APPOINTMENT FOLLOW-UP TRACKING Track pending follow-ups and remind you before deadlines.
Setup: MEMORY.md + optional Google Calendar Usage: On demand to log; daily cron to review Example: "Remind me 2 days before any follow-up I've logged. If I haven't heard back from a contact in 10 days, remind me to check in."
USE CASE 1: UPWORK CANDIDATE SCREENING (Lisa — Active) The primary production use case for CFM.
What the agent does:
Cron: Every 30–60 minutes during business hours Required: Browser infrastructure, Upwork session cookies
---
USE CASE 2: JOB POSTING MANAGEMENT Create, monitor, and optimize job postings.
What the agent does:
---
USE CASE 3: CANDIDATE FOLLOW-UP SEQUENCES Automated follow-up pipeline for candidates who don't respond.
Pipeline:
Setup: Cron checks MEMORY.md daily for follow-up triggers
---
USE CASE 4: DAILY OPERATIONS REPORT End-of-day automated summary of all agent activity.
Daily cron at end of business day. Report includes:
---
USE CASE 5: COMPETITOR MONITORING Weekly automated scan of competitor activity.
What the agent does:
---
USE CASE 6: CLIENT COMMUNICATION DRAFTING Draft professional correspondence.
Examples:
---
USE CASE 7: MARKET RATE RESEARCH Compile salary/rate data from multiple sources.
Weekly or monthly cron: "Pull current market rate data for [role] from 5 job boards. Compile into a comparison table. Email to me and save to /opt/data/reports/."
---
USE CASE 8: SOCIAL MEDIA CONTENT SCHEDULING Draft and queue social media posts.
Agent creates posts based on your guidelines (tone, topics, frequency) and either posts directly or queues for approval.
Access: http://YOUR_VPS_IP:3000 in any web browser
KANBAN BOARD (Main View) Visual overview of all tasks as cards in columns:
Each card shows: task description, timestamp, session ID, duration, session type (cron vs. manual).
Click any card to open the full session transcript.
SESSION VIEWER Complete conversation logs for every session. Features:
CONNECTED PLATFORMS Real-time status of all integrations:
MEMORY VIEWER Read and edit USER.md and MEMORY.md directly in the browser. Changes save immediately to /opt/data/ — no terminal required. Use this to quickly update user preferences or add reminders.
CRON MANAGER
SKILLS BROWSER
BASIC TELEGRAM USAGE:
CONVERSATION TYPES:
Both types behave identically from the agent's perspective.
CONTEXT WINDOW MANAGEMENT: Every AI model has a context window — the maximum text it can process at once. For claude-sonnet-4-6: 200,000 tokens (~150,000 words).
The Telegram interface provides LESS visibility into context usage than the CLI. In the terminal, you can monitor token counts directly. In Telegram, compaction may happen without obvious warning.
AUTO-COMPACTION AT ~136,000 TOKENS: When a Hermes conversation approaches 136,000 tokens, it automatically compacts (summarizes older portions) to allow the conversation to continue. What this means:
HOW TO AVOID PROBLEMS:
WHEN TO START A FRESH TELEGRAM SESSION:
DECISION: HOW MANY AGENTS AND WHAT HARDWARE?
One Agent (Current CFM State): Use when: One primary role, one main platform (Upwork) Hardware: KVM 2 ($10/mo) — handles 1–2 agents comfortably Model: MiniMax-M3 via the minimax provider
Add a Second Agent (Same VPS): Use when: Need a second distinct role (e.g., email manager + Upwork agent) Action: Add another Docker container with different --name and different /opt/data mount Hardware: Same KVM 2 if total RAM usage stays under 6GB
Upgrade VPS Hardware: Trigger: Slow response times, memory warnings, 2+ active agents Upgrade to: KVM 4 (~$20/mo): 4 vCPU, 16GB RAM Tip: Upgrade BEFORE problems are noticeable, not after
Separate VPS Per Critical Agent: Use when: One agent handles sensitive tasks (client comms, financial data) Benefit: Isolation means one agent's failure or compromise doesn't affect others Benefit: Easier troubleshooting when each server has one responsibility
COMMON MAINTENANCE COMMANDS:
Restart agent: docker restart hermes
View recent logs: docker logs hermes --tail 100
Follow live logs: docker logs hermes -f
Check container resource usage: docker stats hermes
Update Hermes to latest version: docker pull nousresearch/hermes:latest docker stop hermes docker rm hermes [re-run original docker run command — data persists in /opt/data/]
Check disk usage: df -h
Check memory usage: free -h
RULE 1 — ONE AGENT, ONE DEDICATED EMAIL Each Hermes agent must have its own Gmail account (or other email), separate from your personal email. Examples: lisaagent2026@gmail.com, hermescfm@gmail.com Why: If one account is phished or compromised, your personal email is unaffected. You can delete the agent's email without affecting your life.
RULE 2 — NAMED API KEYS, ONE PER AGENT On every API provider (MiniMax, GitHub, etc.), create separately named keys per agent. Example key name: "Lisa-CFM-Upwork-May2026" Why: (a) Track spending per agent in provider dashboards. (b) Revoke one key without disabling others. (c) Immediately identify which agent is causing unexpected API usage.
RULE 3 — LEAST PRIVILEGE Each agent gets only the minimum permissions required for its specific job.
Never grant an agent permissions "just in case" — only what it actively needs.
RULE 4 — SECRETS IN .ENV ONLY API keys, passwords, and tokens belong exclusively in .env files. Never store secrets in:
RULE 5 — PRIVATE GITHUB REPOSITORIES Agent backup repos must be PRIVATE. Backing up to a public repository exposes configuration, file paths, and potentially sensitive context.
RULE 6 — QUARTERLY CREDENTIAL ROTATION Every 3 months: review all agents' API keys, rotate any that are 6+ months old, check GitHub token expiry dates.
RULE 7 — VPS ACCESS HARDENING
RULE 8 — USE THE STANDARD MODEL The CFM standard is MiniMax-M3 via the minimax provider — it respects SOUL.md custom instructions. Do not swap in retired/safety-filtered models (GPT-class, etc.) that override behavioral rules. See PART 0 §0.2.
SECTION 12: 57 COMMON PROBLEMS — ROOT CAUSES AND SOLUTIONS ⚠️ HISTORICAL INCIDENT ARCHIVE — read accordingly (2026-06-23). These are DATED past incidents and the fixes applied at the time; they document how the system evolved, not the current config. Many reference now-obsolete models/providers (OpenRouter, Mistral, Claude Sonnet, Gemini, DeepSeek, gpt-4.1-mini) and Honcho memory. For the CURRENT standard ALWAYS defer to PART 0. Do NOT re-apply a model/provider/memory fix from this archive without checking PART 0 first.
QUICK INDEX — Find your problem by number:
[ SETUP & CONTAINER ]
[ MODEL & LLM ]
[ MEMORY & SESSIONS ]
[ WEB SEARCH ]
[ EMAIL ]
[ BROWSER & VNC ]
[ UPWORK ]
[ GITHUB & BACKUPS ]
[ CREDENTIALS & SECURITY ]
[ KNOWN GAPS — NOT YET RESOLVED ]
[ DOCUMENTATION ]
---
EMERGENCY QUICK REFERENCE — Most Common Commands
Is the container running? docker ps | grep hermes docker start hermes-agent-jemg-hermes-agent-1
Read the last 30 log lines (find any crash cause): docker logs hermes-agent-jemg-hermes-agent-1 --tail 30
What model is actually running right now? docker exec hermes hermes config get MODEL_NAME
Restart Telegram gateway (bot not responding): docker exec hermes hermes telegram restart
Check memory file sizes (must stay within limits): docker exec hermes wc -c /opt/data/memories/USER.md /opt/data/memories/MEMORY.md Limits: USER.md ≤1,375 chars | MEMORY.md ≤2,200 chars
Full container restart (reloads all config, memory, crons): docker restart hermes-agent-jemg-hermes-agent-1
List all crons and their status: docker exec hermes hermes cron list
Check disk space inside container: docker exec hermes df -h
---
ABSOLUTE RULES — NEVER DO THESE THINGS
NEVER use sed to edit a Python file on the VPS. sed strips Python string quotes and silently corrupts the file. The error may not appear until a restart. Only safe method: edit locally at Lisa_Hermes/scripts/ → deploy via scp + docker cp.
NEVER create a new Google Doc to update the master manual. Canonical Doc ID: 1MMM5zurb9PFFDhCPKaQ_SRJOOKvTI0hGMnkKYlguNmY Edit that document directly. Every new doc creates a version that will mislead future agents.
NEVER use mistralai/mistral-small as the model ID. That endpoint does not exist on OpenRouter and returns HTTP 404. Correct ID: mistralai/mistral-small-3.2-24b-instruct
NEVER install @anthropic-ai/claude-chrome-mcp. This npm package does not exist. Always fails. Use chrome-devtools-mcp@latest instead.
NEVER use fine-grained GitHub tokens for the backup cron. Fine-grained tokens cause authentication failures reliably. Use classic tokens with repo scope only.
NEVER reinstall or reference himalaya. The binary was corrupted from day one (9-byte "Not Found" file). It has never worked. Use check_upwork_email.py and check_cfm_email.py instead. They bypass himalaya entirely.
NEVER trust or use upwork_cookies.json. All 63 cookie values are empty strings (decryption failed silently). This file is useless. Delete it: docker exec hermes rm -f /opt/data/upwork_cookies.json
PROBLEM #1 — UPWORK LOGIN EXPIRED (Most Common Issue — Solved May 17, 2026)
PLAIN ENGLISH — WHAT DARRICK NEEDS TO DO (3 minutes):
It opens a remote browser that runs on the server — looks like a normal browser.
username and password, just like you always do.
That is it. You do not need to run any scripts or do anything technical. This same 3-step process works for ANY website that loses its login, not just Upwork. The login lasts 30-90 days before needing to repeat.
--- TECHNICAL DETAILS (for Claude Code / developer reference only) ---
SYMPTOMS:
ROOT CAUSE: Chrome stores authenticated sessions as encrypted cookies in a SQLite database file at: /opt/data/browser-manager/profiles/default/Default/Cookies (inside container) These cookies expire every 30-90 days. CDP cookie injection (Network.setCookie) is memory-only and does NOT survive Chrome restart — this is why it was never a real fix.
THE FIX — COOKIE FILE SWAP (5 minutes total): Both Chrome instances on the VPS (VNC browser + Lisa's headless Chrome) use IDENTICAL Linux cookie encryption: AES-128-CBC, key derived from password "peanuts". This means a Cookies file from one Chrome is byte-for-byte compatible with the other.
Step 1 — Log in via VNC:
Step 2 — Darrick logs into the remote desktop link Lisa sends, logs into Upwork, says "done" Lisa then runs: bash /opt/data/scripts/restore_browser_session.sh (inside container) (Windows fallback if needed: cd "C:\Users\12149\Documents\Hermes Agents" then python swap_cookies.py)
What the script does automatically:
Step 3 — Verify: Script output should show: LOGGED_IN:https://www.upwork.com/ab/messages/rooms
HOW LONG IT LASTS: 30-90 days. Repeat when session expires again.
WORKS FOR ANY WEBSITE — not just Upwork: Log in to any site via VNC Chrome, then run swap_cookies.py. The script copies ALL site cookies at once.
LISA'S SELF-DIAGNOSIS: When Lisa detects she cannot reach an authenticated page, she should:
"My Upwork session has expired. Please log in through VNC and run swap_cookies.py from your Windows machine."
FILES: Recovery script: C:\Users\12149\Documents\Hermes Agents\swap_cookies.py Full protocol doc: C:\Users\12149\Documents\Hermes Agents\COOKIE_SWAP_PROTOCOL.md Lisa's memory file: /opt/data/memories/BROWSER_SESSION_RECOVERY.md (inside container)
WHY CDP INJECTION FAILS: Network.setCookie via CDP writes to Chrome's in-memory cookie store only. On Chrome restart or when CDP disconnects, these cookies are gone. The SQLite file swap writes to disk — persistent across all reboots indefinitely.
Symptom: docker ps shows hermes with status "Exited (1)"
Root cause: Missing environment variable, port conflict, or permission error
Solution:
---
Symptom: Messages to bot receive no reply
Diagnostic steps (in order):
[SCREENSHOT: Terminal showing hermes telegram status command output]
---
Symptom: Agent refuses to browse web, says tools are unavailable
Two distinct causes with different solutions:
Cause A — Web SEARCH unavailable (no browser needed): Fix: Add search API key docker exec -it hermes hermes config set SEARCH_API_KEY "your-key" Provider options: SerpAPI, Brave Search, DuckDuckGo (free but rate-limited)
Cause B — Web BROWSING unavailable (browser infrastructure needed): Fix: Complete Phase 11 (Browser Infrastructure Setup) in this manual Signs this is the cause: Search works, but navigating to specific pages fails
Cause C — search_backend not configured in config.yaml (most common root cause): Root cause: The search_backend key in /opt/data/config.yaml is empty ('') by default. DuckDuckGo is already installed — it just has never been activated. Fix:
If it shows search_backend: '' (empty) — this is the cause.
Change: search_backend: '' to search_backend: 'ddgs' Save and exit.
Silent fallback warning: If the primary model fails and no valid fallback is configured, Hermes may silently fall back to a default model (such as deepseek-chat) with no error message. The agent continues responding but with worse results. To confirm which model is actually active: docker exec hermes hermes config get MODEL_NAME or: docker exec hermes hermes config list
Model ID must be exact — common mistakes:
To verify a model ID exists on OpenRouter before setting it: curl https://openrouter.ai/api/v1/models -H "Authorization: Bearer YOUR_KEY" | python3 -m json.tool | grep '"id"' | grep mistral
No API key required. DuckDuckGo (ddgs) is free and built in.
---
Symptom: Agent doesn't remember previous conversations, asks for info already given
Root cause: Memory files missing or /opt/data not properly mounted
Solution:
docker exec hermes ls /opt/data/
Should show /opt/data mapped to /opt/data
Character limits for memory files: USER.md must stay at or under 1,375 characters. MEMORY.md must stay at or under 2,200 characters. Exceeding these limits can prevent Hermes from loading them into context at session start. Periodically check sizes with: docker exec hermes wc -c /opt/data/memories/USER.md /opt/data/memories/MEMORY.md If over limit, ask Lisa: "Read me your MEMORY.md" — identify outdated entries and remove them.
---
Symptom: Agent doesn't follow custom personality rules, ignores behavioral guidelines
Root cause: Model safety filters overriding custom instructions (GPT-4.1-mini is the primary culprit)
Solution: Switch to a model that respects custom instructions: docker exec -it hermes hermes config set MODEL_NAME "anthropic/claude-sonnet-4-6"
Budget alternative: mistral/mistral-small
---
Symptom: Scheduled tasks never execute
Diagnostic steps:
Verify the cron exists and is enabled
Example: 9am CDT = 2pm UTC = cron schedule "0 14 *"
---
Symptom: Browser tasks fail, screenshots return blank or never return
Diagnostic steps:
If empty (no process): restart Chrome (see commands below)
If error: Chrome CDP is not active
If empty: restart Xvfb
Restart Chrome: pkill chrome DISPLAY=:99 google-chrome --headless --disable-gpu --remote-debugging-port=9222 --no-sandbox --disable-dev-shm-usage &
Restart Xvfb: Xvfb :99 -screen 0 1920x1080x24 &
---
Symptom: Browser reaches Cloudflare challenge page or "Access Denied" on Upwork
Root cause: Upwork's Cloudflare + Forter stack blocks the VPS's datacenter IP, or the saved session cookie has expired.
Solution: Option A — Manual login via VNC tunnel: Darrick logs in by hand via noVNC, session cookie saved, valid 30–90 days (see Problems 15 and 16) Option B — Upwork OAuth API (permanent fix): no browser needed; create an OAuth app at developers.upwork.com, obtain Key + Secret, wire into the agent — works from any IP forever
Then restart the browser.
See also: Problem 15 (noVNC connection failure), Problem 16 (VNC tunnel URL changes), Problem 39 (Upwork session cookie expires), Problem 50 (Upwork OAuth API — permanent fix).
---
Symptom: Agent cannot read emails, authentication errors
Gmail with App Password (for hermesCFM account):
Gmail REST API (OAuth) (for cfmbusiness account):
---
Symptom: http://VPS_IP:3000 gives "connection refused" or just times out
Diagnostic steps:
Port changes on every restart — pin it permanently: If using docker-compose.yml and the dashboard URL breaks after every container restart because Docker assigns a random host port, pin it explicitly: In docker-compose.yml, set: ports: ['4860:4860'] This locks the host port to 4860 permanently regardless of restarts. Update the firewall: ufw allow 4860 Dashboard URL will then always be: http://VPS_IP:4860
---
Symptom: Agent responds with "I don't have an LLM configured" or uses wrong model, or replies make no sense
Solution:
Silent fallback warning: If the primary model fails and no valid fallback is configured, Hermes may silently fall back to a default model (such as deepseek-chat) with no error message. The agent continues responding but with worse results. To confirm which model is actually active: docker exec hermes hermes config get MODEL_NAME or: docker exec hermes hermes config list
Model ID must be exact — common mistakes:
To verify a model ID exists on OpenRouter before setting it: curl https://openrouter.ai/api/v1/models -H "Authorization: Bearer YOUR_KEY" | python3 -m json.tool | grep '"id"' | grep mistral
---
Symptom: Backup errors, "authentication failed", "repository not found", or 401 errors
Solution:
Fine-grained tokens cause this error reliably.
docker exec -it hermes hermes config set GITHUB_TOKEN "ghp_your-new-token"
docker exec -it hermes hermes backup now
---
Script path issue — "Script not found: /opt/data/scripts/github-sync.sh": The github-sync.sh script may exist at /opt/data/github-sync.sh or /opt/data/.hermes/scripts/github-sync.sh but the cron runner expects it at /opt/data/scripts/github-sync.sh. Fix: docker exec hermes cp /opt/data/github-sync.sh /opt/data/scripts/github-sync.sh Verify: docker exec hermes ls /opt/data/scripts/github-sync.sh
---
Symptom: Lisa responds to every Telegram message with "I cannot do that" or a generic refusal regardless of the request. Gateway logs show the pattern: Empty response (no content or reasoning) — retry 1/3, retry 2/3, retry 3/3, followed by OpenRouter response cache HIT (total: 1), (total: 2), (total: 3), then Empty response after 3 retries. No fallback available. Messages are received and acknowledged but every response is a refusal.
Root cause: Mistral Small 3.2's RLHF safety training returns a completely empty response body for tool-calling scenarios, even when the system prompt explicitly authorizes the task. OpenRouter caches this empty response at the infrastructure level. Hermes retries the request three times — but all three retries hit the OpenRouter cache and receive the same cached empty reply. Hermes exhausts its retry budget and sends a hardcoded fallback message ("I cannot do that") to Telegram. The SOUL.md instructions are never reached because the model returns nothing before they can take effect.
Solution:
docker exec hermes-agent-jemg-hermes-agent-1 bash -c 'hermes config set model.default anthropic/claude-sonnet-4-6 && hermes config set model.provider openrouter' IMPORTANT: routing through OpenRouter bypasses all direct Anthropic API spending caps — these are completely separate billing systems. The Anthropic key in /opt/data/.env is irrelevant when provider is set to openrouter.
docker exec -u hermes hermes-agent-jemg-hermes-agent-1 bash -c 'pkill -f "hermes gateway run" 2>/dev/null; sleep 3; nohup hermes gateway run > /tmp/gateway.log 2>&1 &'
Do NOT use: Mistral Small 3.2 as the primary model for any Hermes agent. The empty response bug is caused by its RLHF training and cannot be fixed with prompting, system prompt changes, or retry configuration adjustments. GPT-4.1-mini is a functional fallback but is not the official Nous Research recommendation — use Claude Sonnet 4.6 via OpenRouter as the primary model for all Hermes agents.
Status: FIXED — May 16, 2026. Switched to anthropic/claude-sonnet-4-6 via OpenRouter. Routing through OpenRouter bypasses all direct Anthropic API spending caps — these are separate billing systems. Gateway log confirmed: response ready: time=21.6s api_calls=2 response=393 chars.
See also: Problem 57 (model reverts to Claude on restart), Problem 53 (GPT-4.1-mini refuses email via safety filters).
---
Symptom: When told "login to my Facebook" or "check my Facebook," Lisa attempts browser automation to open facebook.com, which fails on VPS servers because Facebook detects and blocks server IP ranges. Lisa then reports she cannot log in rather than asking a clarifying question. No Facebook task is completed.
Root cause: SOUL.md contained a single ambiguous Facebook entry routing all requests to browser automation: "login to Facebook → navigate browser to https://www.facebook.com." This forced browser automation for every Facebook request — ignoring that the user operates two completely different Facebook contexts: (1) Business Pages (USA Gig Work, Fix Flint) accessible via Meta Graph API with no browser required; (2) Personal profile, accessible only via VNC browser tunnel because Facebook blocks direct server automation. The single entry also meant Lisa never asked which account — she always defaulted to browser and failed.
Solution: Replace the single Facebook entry in SOUL.md with three distinct entries in the tool trigger map:
Deploy facebook_manager.py to /opt/data/scripts/facebook_manager.py — a Meta Graph API script handling posts, comments, comment posting, and replies with no browser required.
Critical bug in facebook_manager.py to verify: The token_for_post() function must parse the page ID from the post ID prefix (e.g., 447725185097879_POSTID belongs to USA Gig Work page). Without this fix, the script defaults to the alphabetically-first page token (Fix Flint), which returns permission errors on USA Gig Work content.
Status: FIXED — May 16, 2026. SOUL.md updated with three-entry disambiguation. facebook_manager.py deployed at /opt/data/scripts/facebook_manager.py with correct token routing logic.
See also: Problem 8 (Upwork bot detection and proxy configuration), Problem 46 (proxy_agent.py does not read from .env).
---
Symptom: Every browser-dependent task fails with "Failed to resolve CDP endpoint http://localhost:9222." Running ps aux inside the container shows many [chrome] <defunct> zombie processes. Chrome starts and immediately dies — no usable CDP connection is ever established. Restarting browser_manager.py repeatedly produces the same zombie result. Crash dumps accumulate at /opt/data/home/.config/google-chrome-for-testing/Crash Reports/pending/.
Root cause: Playwright Chromium (bundled at /opt/data/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome) requires an X11 virtual display (Xvfb) to launch in non-headless mode. Xvfb crashes on container startup because the container kernel does not support the required X11 socket infrastructure. This creates a cascade failure: Xvfb fails → Chrome has no display → Chrome exits immediately → zombie processes accumulate. The launch flags --no-sandbox and --disable-dev-shm-usage do not fix this because the failure is at the display layer, not the sandbox or shared-memory layer.
Solution: Switch Chrome to headless mode — no virtual display required:
pkill -9 -f chrome 2>/dev/null; pkill -f Xvfb 2>/dev/null; rm -f /tmp/.X99-lock /tmp/.X98-lock; sleep 2
nohup python3 /opt/data/browser-manager/browser_manager.py > /tmp/browser_manager.log 2>&1 &
Note: --headless=new supports full CDP functionality including page navigation, screenshots, form filling, and JavaScript execution. Visual rendering is the only capability lost, which is irrelevant for server-side automation.
Status: FIXED — May 16, 2026. Chrome/147 running in headless=new mode. CDP responding on port 9222. Browser watchdog updated and redeployed to host crontab.
See also: Problem 53 (browser_manager.py crashes on restart), Problem 54 (Chrome crashes under heavy page load).
---
Symptom: After the watchdog runs (every 5 minutes), Lisa stops responding on Telegram entirely. No gateway process is found running inside the container. The watchdog log shows it detected the wrong model, killed the gateway, and corrected config.yaml — but Lisa never came back online. The watchdog "fixed" the agent by silencing her indefinitely until a human manually restarted the gateway.
Root cause: The original hermes_watchdog.py logic was: (1) read config.yaml, (2) if model is wrong, fix config.yaml, (3) pkill -f "hermes gateway run" — then stop. No start_gateway() call followed the kill. The script assumed the gateway would auto-restart itself after being killed — it does not. Every watchdog fix left Lisa in a dead, non-responding state. Additionally, the watchdog had no independent check for whether the gateway was running at all — if the gateway died for reasons other than model config, the watchdog would not notice.
Solution: Rewrite /opt/hermes_watchdog.py with two additions: a start_gateway() function and an independent gateway_running() health check:
def start_gateway(): subprocess.Popen( ['docker', 'exec', '-u', 'hermes', CONTAINER, 'bash', '-c', 'nohup hermes gateway run > /tmp/gateway.log 2>&1 &'], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL ) time.sleep(5)
def gateway_running(): result = subprocess.run( ['docker', 'exec', CONTAINER, 'pgrep', '-f', 'hermes gateway run'], capture_output=True ) return result.returncode == 0
Updated logic flow: (1) check model config → if wrong, fix config → kill gateway → call start_gateway(); (2) independently call gateway_running() → if gateway is down for any reason, call start_gateway() — even when the config is already correct.
Status: FIXED — May 16, 2026. Watchdog rewritten and redeployed to /opt/hermes_watchdog.py on host. Watchdog enforces anthropic/claude-sonnet-4-6 via openrouter as the correct model — any reversion to a different model is auto-corrected within 5 minutes. Both the model-fix path and the independent gateway health check now call start_gateway() after any intervention. Host cron confirmed active: /5 * /usr/bin/python3 /opt/hermes_watchdog.py.
See also: Problem 57 (auth.json caches Anthropic key — model reverts to Claude on restart).
APPENDIX: GLOSSARY OF TERMS
AGENTS.MD — A local context file (like CLAUDE.md) placed in a project directory for CLI-mode Hermes sessions. The agent reads it automatically when running in that directory.
AUTO-COMPACTION — When a Hermes conversation approaches ~136,000 tokens, older portions are automatically summarized to allow the conversation to continue. All sessions remain fully searchable in SQLite regardless.
CDP (Chrome DevTools Protocol) — A protocol that allows programs to control a Chrome browser: clicking buttons, filling forms, taking screenshots, reading page content. How Hermes autonomously operates web browsers.
CLASSIC TOKEN (GitHub) — A GitHub personal access token with broad scope settings. Required for Hermes GitHub backup. Use this, not fine-grained tokens.
CLOUDFLARE / FORTER — Bot detection systems used by websites like Upwork. Identifies and blocks traffic from datacenter IPs. Worked around with a real logged-in browser session or the site's official API.
CONTEXT WINDOW — The maximum amount of text an AI model can process in one session. For claude-sonnet-4-6: 200,000 tokens (~150,000 words).
CRON — A scheduled, automated task. Runs at set times without any human input. Name comes from the Unix cron job scheduler.
DOCKER — Containerization platform. Hermes runs inside a Docker container — an isolated, portable environment with everything it needs. Containers can be stopped, restarted, and recreated without losing data in mounted volumes.
FINE-GRAINED TOKEN (GitHub) — A newer GitHub access token type with per-repository permissions. NOT compatible with Hermes backup — use Classic tokens instead.
HERMES — Open-source autonomous AI agent framework by Nous Research. 140,000+ GitHub stars, MIT licensed. The framework for the agent; the LLM provides the intelligence.
IMAP — Internet Message Access Protocol. Standard protocol for accessing email from a mail server. Used to connect Gmail to Hermes for email reading.
KANBAN BOARD — Visual task management: cards organized in columns (Pending, In Progress, Complete, Failed). Available in the Hermes dashboard.
LEAST PRIVILEGE — Security principle: grant only the minimum permissions required for a task. Each agent should have only the access it actually needs.
LLM (Large Language Model) — The AI model powering Hermes's reasoning (the CFM standard is MiniMax-M3; other providers exist but are not used here). Hermes is the agent framework; the LLM is the "brain."
MEMORY.MD — Working memory file. Contains evolving, frequently-updated information. Located at /opt/data/MEMORY.md.
OPENROUTER — API aggregator providing access to 100+ AI models via a single API key. CFM standard LLM provider.
PROGRESSIVE DISCLOSURE — Hermes reads only YAML metadata from skill files at startup (not the full content). Full skill content loads only when the skill is triggered. Keeps context window efficient with many skills installed.
RESIDENTIAL PROXY — Proxy that routes traffic through real home internet connections. Makes VPS traffic appear as a regular home user. Not used in this stack.
SESSION — A single conversation with the Hermes agent. All sessions stored in SQLite at /opt/data/sessions.db. Fully searchable.
SKILL — A markdown file with YAML front matter + step-by-step instructions for a specific task type. Stored in ~/.hermes/skills/.
SOUL.MD — The agent's identity, personality, and behavioral rules file. Located at /opt/data/SOUL.md.
SQLITE — Lightweight file-based database. Hermes stores all session transcripts in /opt/data/sessions.db. Queryable by the agent itself.
SSH — Secure Shell. Protocol for securely connecting to a remote server via terminal.
TERMUX — Android terminal emulator. Allows Hermes to run on Android phones. For personal experimentation only — not recommended for production.
TOKEN (API) — Unit of text processed by AI models. Roughly 3/4 of a word. API usage is priced per token.
USER.MD — Permanent memory file. Contains stable facts about the user. Located at /opt/data/USER.md. Read at the start of every conversation.
VPS (Virtual Private Server) — A rented server in a data center that runs 24/7. Hermes agents run on VPS instances so they operate continuously without your computer being on.
XVFB — X Virtual Framebuffer. Software that simulates a monitor. Chrome needs a display to render to — Xvfb provides a virtual one on a headless Linux server.
YAML FRONT MATTER — Metadata at the top of a markdown file, between --- markers. Used in skill files to provide structured metadata (name, trigger, description) the agent reads without loading the full file.
Hermes Agent Master Manual — CFM Business Version 4.0 FINAL | May 2026 Maintained by: CFM Business Update rule: Edit this document directly. NEVER create a new Google Doc for updates. For new problems/solutions: Add to Section 12 above.
---
Symptom: Flood of 429 "Too Many Requests" errors; agent becomes unresponsive or loops endlessly
Root cause: A Gemini model was configured as a fallback. The Gemini free tier quota was exhausted (0 remaining). Every time the primary model had any issue, Hermes called Gemini, which immediately returned 429, creating a crash loop.
Solution:
docker exec -it hermes hermes config set FALLBACK_MODEL ""
docker exec -it hermes hermes config set MODEL_NAME "mistralai/mistral-small-3.2-24b-instruct"
---
Symptom: Opening the VNC tunnel URL and clicking Connect shows a red error banner. Cannot see or control the agent's browser.
Root cause: Two components must run simultaneously and point to each other correctly:
If Xvfb is a zombie process (started but crashed), x11vnc cannot attach to it and the chain breaks.
Solution:
docker restart hermes
docker exec -d hermes x11vnc -display :99 -nopw -listen localhost -xkb -forever
websockify 6080 172.16.1.2:5900 & (Replace 172.16.1.2 with the actual container IP from step 4)
---
Symptom: After any VPS reboot or cloudflared restart, the tunnel URL (e.g. https://example.trycloudflare.com) stops working. A new random URL is generated each time and must be hunted down.
Root cause: cloudflared in "quick tunnel" mode generates a temporary random subdomain that expires when the process restarts. There is no fixed hostname.
Temporary fix — find the current URL after a reboot: grep -oa "https://[a-z0-9-]*\.trycloudflare\.com" /tmp/cloudflared.log | tail -1
Permanent fix — named Cloudflare tunnel with fixed subdomain (requires Cloudflare account):
---
Symptom: Every email task fails with "himalaya is not available," executable errors, or returns the text "Not Found."
Root cause: The himalaya binary at /opt/data/.local/bin/himalaya was never properly installed. A failed HTTP download during setup saved a 9-byte error page ("Not Found") as the executable. It was never functional at any point.
Solution: Himalaya is permanently bypassed. Do NOT reinstall or attempt to fix it. Two Python scripts replace it entirely:
check_upwork_email.py — reads Upwork email via Python imaplib + Gmail app password: Location: /opt/data/browser-manager/tasks/check_upwork_email.py Usage: python3 /opt/data/browser-manager/tasks/check_upwork_email.py
check_cfm_email.py — reads cfmbusiness Gmail via Gmail REST API + OAuth2 token: Location: /opt/data/browser-manager/tasks/check_cfm_email.py Usage: python3 /opt/data/browser-manager/tasks/check_cfm_email.py N (Replace N with the number of emails to retrieve — e.g., 5 for "check my last 5 emails")
PERMANENT RULE: Do not use, reference, or reinstall himalaya. Both Python scripts work without it and are more reliable.
See also: Problem 18 (crons still referencing himalaya), Problem 19 (cron 404 Telegram errors), Problem 20 (email cherry-picking fix), Problem 51 (email scripts are read-only).
---
Symptom: Upwork email crons fire on schedule but produce errors every run. Cron history shows himalaya-related failures.
Root cause: The cron entries in /opt/data/cron/jobs.json still referenced "skill": "himalaya" even after himalaya was bypassed and replaced with Python scripts. The cron system tried to invoke the broken skill on every scheduled run.
Solution:
Then: nano /opt/data/cron/jobs.json
"prompt": "Run python3 /opt/data/browser-manager/tasks/check_upwork_email.py and output the results as your response."
---
Symptom: Cron fires on schedule. Darrick receives 404 error messages on Telegram instead of the actual email results. The agent is attempting to call the Telegram Bot API directly.
Root cause: The cron prompt instructed Lisa to "forward results to this Telegram chat." She interpreted this as a direct Telegram API call she had to make herself. Those API calls require authentication she doesn't have in scope, so every attempt returns 404.
The Hermes cron system handles Telegram delivery automatically — the agent should only output the result as a response. It does not need to call Telegram directly.
Solution: Change the cron prompt from "forward to Telegram" to "output as your response."
Correct cron prompt format: "Run python3 /opt/data/browser-manager/tasks/check_upwork_email.py and output the results as your response."
Incorrect (causes 404 loop): "Run the email check and forward results to Telegram chat ID 870599219."
---
Symptom: When asked "check my last 5 emails," the agent shows 5 emails — but not the actual 5 most recent. Real estate deals and other important messages are silently skipped. She appears to judge which emails are "worth" showing.
Root cause: The email script returned all emails as unstructured text. The LLM chose which lines from the output to include in its reply. This is LLM reasoning behavior and cannot be overridden by SOUL.md instructions alone.
Failed fix: Adding strict rules to SOUL.md ("never filter, always show all"). This did NOT work. The LLM continued to filter silently.
Actual fix: Rewrote check_cfm_email.py so that:
With this structure, the agent receives pre-numbered emails and copies them into her response. The count is controlled by the script — the LLM has no opportunity to filter.
SOUL.md and MEMORY.md must also be updated to tell the agent to pass the count argument: "When the user asks to check emails, run: python3 check_cfm_email.py [N] where N is the number they requested."
See also: Problem 17 (himalaya bypassed — the Python scripts that replaced it), Problem 37 (local check_cfm_email.py out of sync with VPS), Problem 51 (email scripts are read-only — cannot yet reply to candidates).
Verified: After the rewrite, "check my last 5 CFM Business emails" returned exactly 5 emails in order, including real estate deals that had previously been silently skipped.
---
Symptom: curl inside the container fails with a DNS resolution error on an external hostname, even though the host resolves it fine.
Root cause: Docker containers inherit DNS from the host, but some Docker network configurations do not forward external DNS lookups correctly.
Fix: echo 'nameserver 8.8.8.8' >> /etc/resolv.conf (inside the container) Or add DNS settings to docker-compose.yml: dns: ["8.8.8.8", "8.8.4.4"] After applying, test: docker exec hermes nslookup example.com
---
Symptom: Messages sent to the Telegram bot receive no response. The container appears to be running.
Two-step diagnosis: Step 1 — Ask Lisa directly via any working channel: "Telegram isn't responding — please check and diagnose." Lisa can self-diagnose the Telegram gateway from inside the container without SSH access.
Step 2 — If Lisa is also unreachable, SSH in and restart the gateway manually: docker exec hermes hermes telegram restart If that fails: docker restart hermes Check logs after restart: docker logs hermes --tail 30 | grep -i telegram
Also check: the Telegram Bot Token in config has not been invalidated. If the bot was revoked in BotFather, a new token must be issued and set: docker exec hermes hermes config set TELEGRAM_BOT_TOKEN "new-token"
---
Symptom: Scheduled crons never run at the expected time, or appear to run at the wrong time.
Root cause: The Hermes container runs on UTC time. Cron schedules must be entered in UTC. Central Time is UTC-5 (CST) or UTC-6 (CDT). A cron set for "8 AM Central" must be entered as 13:00 or 14:00 UTC depending on daylight saving time.
Conversion reference: 8 AM CDT (Central Daylight) = 13:00 UTC → cron: 0 13 8 AM CST (Central Standard) = 14:00 UTC → cron: 0 14
Verify cron exists and is enabled: docker exec hermes hermes cron list
Test a specific cron manually: docker exec hermes hermes cron run [cron-id]
Note: Crons must be created from a manual (non-cron) session. Creating a cron from inside a running cron session is not allowed.
---
Symptom: Lisa has outdated information, references services that no longer exist, uses wrong credentials, or contradicts herself across sessions.
Root cause: MEMORY.md has accumulated conflicting or outdated entries across many sessions without cleanup.
Solution:
---
Symptom: upwork_cookies.json exists on disk with 63 entries, but every cookie value is an empty string. Attempts to use this file for authentication fail silently.
Root cause: Chrome stores cookies in an encrypted SQLite database. Extracting them requires decryption using OS-level keys (DPAPI on Windows, Keychain on Mac, libsecret on Linux). Inside the Docker container, these keys are not accessible. The extraction script ran but produced empty values without throwing an error.
Resolution: This approach is a dead end. Do not attempt to extract or use Chrome cookies from inside the container.
docker exec hermes rm -f /opt/data/upwork_cookies.json
---
Symptom: npm install @anthropic-ai/claude-chrome-mcp fails with "404 Not Found" or "package not found."
Root cause: This package name does not exist in the npm registry and never did. Installing it will always fail.
Resolution: This is a dead end. Do not attempt to install it. Use chrome-devtools-mcp@latest instead — this is the correct package and is already configured in .mcp.json: npm install -g chrome-devtools-mcp@latest
---
Symptom: Any overnight task or in-progress work is lost. Lisa starts fresh at 4 AM UTC with no memory of what she was doing.
Root cause: Hermes performs a daily session reset at approximately 4:00 AM UTC. This is normal behavior — it clears the active conversation context to prevent runaway sessions. It does NOT delete memory files.
Mitigation:
---
Symptom: After a VPS reboot or power event, the Hermes container is stopped and Lisa is completely offline. No automatic recovery occurs.
Solution: Set the container restart policy to "unless-stopped": docker update --restart unless-stopped hermes-agent-jemg-hermes-agent-1
After setting this, the container will restart automatically after any VPS reboot, unless it was manually stopped by the user. Verify the policy was applied: docker inspect hermes-agent-jemg-hermes-agent-1 | grep RestartPolicy
---
Symptom: Agent becomes sluggish or unresponsive. Crons fail silently. No obvious error message.
Diagnosis: Check disk usage inside the container: docker exec hermes-agent-jemg-hermes-agent-1 df -h
If /tmp or /opt/data is near full, clean temporary files: docker exec hermes-agent-jemg-hermes-agent-1 find /tmp -type f -delete
Also check for large log files: docker exec hermes-agent-jemg-hermes-agent-1 find /opt/data -name "*.log" -size +10M
Note: /opt/data is a mounted volume — cleaning it reduces disk use on the host VPS as well.
---
Symptom: Lisa suddenly stops responding or returns "quota exceeded" errors. All OpenRouter-dependent functions fail simultaneously.
Root cause: A $20/month spending limit was set on the OpenRouter account. When the cap is hit, all API calls are blocked until the billing cycle resets or the limit is raised.
Solution:
---
Symptom: Web search worked previously but now returns rate limit errors or no results. DuckDuckGo is configured as the search backend.
Root cause: DuckDuckGo (ddgs) imposes rate limits on automated queries. After enough rapid searches, it temporarily blocks the container's IP.
Solution: Switch to a different search backend in /opt/data/config.yaml:
Temporary workaround: Wait 10-30 minutes and try again — DuckDuckGo rate limits are usually temporary.
---
Symptom: Container starts but Hermes immediately crashes. Logs show a YAML parse error.
Root cause: A manual edit to /opt/data/config.yaml introduced invalid YAML syntax (wrong indentation, unquoted special characters, etc.).
Solution:
Prevention: Always edit config.yaml via nano or a text editor that preserves indentation. Never use sed or echo to modify YAML files.
---
Symptom: check_cfm_email.py fails with OAuth authentication errors. The local backup copy of cfmbusiness_google_token.json no longer matches what's on the VPS.
Root cause: After any VPS restore or redeployment, the token file on disk may be a different version than what Google considers valid. The local backup snapshot may also be stale.
Solution:
python3 check_token.py (run locally — requires browser) Complete the browser authentication flow.
scp cfmbusiness_google_token.json root@2.24.96.191:/tmp/ docker cp /tmp/cfmbusiness_google_token.json hermes:/opt/data/cfmbusiness_google_token.json
If Google has fully revoked the token (e.g., after a security event): Re-run the full OAuth flow — the browser will ask you to re-grant permissions. The resulting token is fresh and valid.
---
Symptom: Browser tasks time out. Screenshots never return. Chrome appears to be running (pgrep shows a PID) but does not respond to CDP commands.
Diagnosis: docker exec hermes-agent-jemg-hermes-agent-1 pgrep -a chrome If Chrome is running but unresponsive, kill it: docker exec hermes-agent-jemg-hermes-agent-1 pkill -f chrome
Restart Chrome inside the container: docker exec -d hermes-agent-jemg-hermes-agent-1 bash -c "DISPLAY=:99 google-chrome --headless --disable-gpu --remote-debugging-port=9222 --no-sandbox --disable-dev-shm-usage &"
Verify CDP is responding: docker exec hermes curl -s http://localhost:9222/json
If Chrome won't start: restart the full container — browser_manager will reinitialize Chrome and Xvfb on startup: docker restart hermes-agent-jemg-hermes-agent-1
---
Symptom: Browser-related tasks fail with Python import errors. The hermes-browser-env virtualenv does not respond correctly.
Diagnosis: docker exec hermes-agent-jemg-hermes-agent-1 /opt/data/home/.hermes-browser-env/bin/python -c "import socket; print('OK')" If this fails with an error, the virtualenv is corrupted.
Solution — rebuild the virtualenv:
---
Symptom: CLAUDE.md or INTEGRATIONS.md still shows gpt-4.1-mini as the active model, but Session 7 switched the running model to mistralai/mistral-small-3.2-24b-instruct. Future sessions following the docs will attempt to configure the wrong model.
Root cause: Documentation was not updated when the model was changed in Session 7.
Correct current model: mistralai/mistral-small-3.2-24b-instruct (via OpenRouter) Wrong model name: mistralai/mistral-small — this endpoint returns HTTP 404
Action: Update CLAUDE.md and INTEGRATIONS.md in the local Lisa_Hermes/ project folder to reflect the actual running model. Verify on VPS: docker exec hermes hermes config get MODEL_NAME
---
Symptom: Session 7 rewrote check_cfm_email.py on the VPS to accept a count argument and number emails. The local copy at Lisa_Hermes/scripts/check_cfm_email.py was never updated. If the local file is ever deployed to the VPS, it overwrites the fix and email filtering resumes silently.
Solution — sync the local file from VPS: scp root@2.24.96.191:/opt/data/browser-manager/tasks/check_cfm_email.py Lisa_Hermes/scripts/check_cfm_email.py
After syncing, commit to GitHub so the fix is preserved in the backup repo.
---
Symptom: Facebook posting or Messenger actions fail silently. No error shown to user — Lisa simply cannot post.
Root cause: Meta long-lived page access tokens have a ~60-day lifespan. There is no auto-renewal mechanism.
Prevention: Set a calendar reminder at day 50 (10 days before expiry) to renew tokens.
Renewal: Generate new long-lived tokens via the Meta Developer portal → Graph API Explorer. Pages affected:
After renewal, update the tokens in /opt/data/.env on the VPS and in the local Lisa_Hermes/.env backup.
---
Symptom: Lisa's browser reaches Upwork but is immediately redirected to a login page. The saved session cookie has expired.
Root cause: Upwork session cookies are valid for approximately 30–90 days. After expiration, a fresh login is required.
Solution:
---
Symptom: Lisa cannot post to Facebook pages or reply to Messenger messages, even with a valid access token. API returns permission errors.
Root cause: The Facebook app used for this integration was never submitted for pages_manage_posts and pages_messaging scope approval. These scopes require Meta's App Review process before they work in production.
Current status: Unresolved. This is a platform-level requirement — it cannot be bypassed.
Path to resolution:
Until approved, Facebook posting and Messenger replies are not available to Lisa.
---
Symptom: Xvfb, Chrome, and CDP are running inside the container, but no Telegram command can trigger the browser. Browser tasks cannot be assigned to Lisa from Telegram.
Root cause: The browser skill file was never written. Hermes requires a SKILL.md file at a specific path to expose any capability as a Telegram command.
Solution — create the browser skill:
✅ RESOLVED – May 16, 2026. Option B (Gmail API reply for cfmbusiness@gmail.com) has been implemented.
Resolution details:
Commands: Reply to email: python3 /opt/data/browser-manager/tasks/reply_cfm_email.py --reply_to GMAIL_ID --body "Your reply text" Send new email: python3 /opt/data/browser-manager/tasks/reply_cfm_email.py --to recipient@example.com --subject "Subject" --body "Body"
Remaining gap: Upwork replies still require browser/proxy or the Upwork API (see Problem 50).
This was a known gap. Until the skill is created, Lisa cannot use her browser from Telegram commands.
---
Symptom: Lisa cannot access GoHighLevel, Facebook, or QuickBooks via browser. Browser navigates to the site and immediately hits a login page.
Root cause: These services require manual one-time login via VNC to save a session cookie into the Chrome profile. This step was planned but never completed.
Solution:
---
Symptom: README.md, HANDOFF.md, and CLAUDE.md each reference a different Google Doc as the "master manual." Agents following different files will work on different documents.
Root cause: The manual went through several versions (v2.0, v2.1, v3.0, v4.0). Each version's ID was recorded in whatever file was being used at the time, but old files were never updated to the current ID.
Canonical Master Manual Doc ID (as of Session 8): 1MMM5zurb9PFFDhCPKaQ_SRJOOKvTI0hGMnkKYlguNmY
Action: Update README.md and any other local file still referencing an old Doc ID to point to the canonical ID above. CLAUDE.md was already updated in Session 8.
---
Symptom: check_cfm_email.py fails after a Python upgrade or container rebuild with a ModuleNotFoundError.
Root cause: The script contains a line like: sys.path.insert(0, "/opt/hermes/.venv/lib/python3.13/site-packages") This path breaks if the Python version changes (e.g., from 3.13 to 3.14).
Fix options: Option A — dynamic path (recommended): import sys, glob paths = glob.glob("/opt/hermes/.venv/lib/python*/site-packages") if paths: sys.path.insert(0, paths[0])
Option B — version-agnostic symlink: ln -s /opt/hermes/.venv/lib/python3.13 /opt/hermes/.venv/lib/python-current Then use: /opt/hermes/.venv/lib/python-current/site-packages
Edit the file locally, then deploy via scp + docker cp (never sed).
---
Symptom: Future agents find upwork_cookies.json and attempt to use it for Upwork authentication. All 63 cookie values are empty strings. Authentication fails silently.
Root cause: A cookie extraction script ran successfully (no errors) but produced empty values because Chrome's cookie encryption could not be decrypted inside the container (see Problem 25).
Action: Delete this file to prevent future confusion: docker exec hermes rm -f /opt/data/upwork_cookies.json Also delete from the local project backup if present: Lisa_Hermes/upwork_cookies.json
---
Symptom: CONTEXT.md says GHL is "not configured." INTEGRATIONS.md says GHL API keys have been added. Agents following different files get different answers.
Action: SSH in and verify which state is actually true: docker exec hermes cat /opt/data/.env | grep -i ghl docker exec hermes hermes config list | grep -i ghl
If keys exist and are non-empty: update CONTEXT.md to reflect "configured." If keys are missing: update INTEGRATIONS.md to reflect "not configured." Then test an actual GHL API call to confirm the integration works end-to-end.
---
Symptom: This one-time migration script still exists in the project. It contains a full sk-ant-api03-... key embedded as a string literal.
Risk: If this file is ever committed to GitHub or shared, the key is exposed. Anyone with the key can make API calls billed to this account.
Action:
---
Symptom: This script exists in the project and contains logic to migrate Hermes to gemini-2.0-flash. Gemini was abandoned as a provider. If this script is ever run accidentally, it reverts the config to a non-working model.
Action:
---
Symptom: Every session, the team works around Upwork access issues with proxies, VNC logins, and cookie management. These all expire and require repeated manual intervention.
Root cause: The permanent solution — Upwork's official OAuth API — has been identified since May 15 but never implemented.
What the OAuth API provides:
Implementation path:
This is the recommended permanent fix for all Upwork access issues.
---
Symptom: Lisa can read Upwork and cfmbusiness emails via Python scripts, but cannot reply. The 9+ candidate backlog cannot be cleared autonomously.
Root cause: check_upwork_email.py and check_cfm_email.py are read-only scripts. No reply functionality has been written. No Upwork API integration exists.
Current options for replying: Option A — Manual: Darrick logs into each platform directly and replies. Option B — Gmail API (cfmbusiness replies): The Gmail REST API supports sending. A send_email.py script can be written using the existing OAuth token. Option C — Upwork API (Upwork replies): Implement Upwork OAuth API (see Problem 50). The Upwork API supports sending messages to applicants. Option D — Browser-based reply: Use the VNC + proxy approach to log in and reply manually via Lisa's browser (see Problems 8, 15, 16, 39).
✅ RESOLVED – May 16, 2026. Option B (Gmail API reply for cfmbusiness@gmail.com) has been implemented.
Resolution details:
Commands: Reply to email: python3 /opt/data/browser-manager/tasks/reply_cfm_email.py --reply_to GMAIL_ID --body "Your reply text" Send new email: python3 /opt/data/browser-manager/tasks/reply_cfm_email.py --to recipient@example.com --subject "Subject" --body "Body"
Remaining gap: Upwork replies still require browser/proxy or the Upwork API (see Problem 50).
This was a known gap — Lisa can see messages but cannot respond autonomously until one of the above is implemented.
---
APPENDIX A: SESSION STARTUP CHECKLIST
Run this at the start of every Claude Code session before doing any work on the Hermes agent.
Understand what the last session completed and what is still pending.
docker ps | grep hermes If not running: docker start hermes-agent-jemg-hermes-agent-1
docker exec hermes hermes config get MODEL_NAME Expected: mistralai/mistral-small-3.2-24b-instruct If wrong: docker exec hermes hermes config set MODEL_NAME "mistralai/mistral-small-3.2-24b-instruct" then docker restart hermes
docker exec hermes wc -c /opt/data/memories/USER.md /opt/data/memories/MEMORY.md USER.md must be ≤1,375 chars. MEMORY.md must be ≤2,200 chars. If over limit: ask Lisa "Read me your MEMORY.md" and trim outdated entries.
If no response: see Problem 2 (Telegram bot) or Problem 22 (Hermes gateway).
Run .\scripts\check-agent-status.ps1 from the Hermes Agents root.
---
APPENDIX B: MAINTENANCE CALENDAR — TOKEN AND CREDENTIAL EXPIRY
Review this table every 30 days. Set calendar reminders proactively — silent expiry is the #1 cause of unexplained failures.
ITEM | EXPIRES | REMINDER AT | WHERE TO RENEW Meta/Facebook page access tokens | ~60 days | Day 50 | developers.facebook.com → Graph API Explorer → generate long-lived page token Upwork browser session cookie | 30–90 days | Day 25 | VNC tunnel → log into upwork.com manually → session saved to Chrome profile OpenRouter monthly credits | Monthly (per billing cycle) | When daily Telegram report shows low balance | openrouter.ai → Account → Usage → raise cap or top up GitHub personal access token | Per token settings (1yr typical) | Check expiry in GitHub settings | GitHub → Settings → Developer Settings → Personal Access Tokens (Classic) → regenerate cfmbusiness OAuth refresh token | Does not expire unless revoked | Only act if check_cfm_email.py fails with auth error | Run check_token.py locally → complete browser OAuth flow → docker cp new token to VPS Telegram Bot Token | Does not expire unless revoked by user | Only act if bot goes silent and logs show auth error | t.me/BotFather → /mybots → revoke and reissue token → update hermes config
Facebook pages to renew:
After any token renewal, update /opt/data/.env on the VPS AND the local Lisa_Hermes/.env backup file.
---
APPENDIX C: GLOSSARY ADDITIONS (v4.0+)
The following terms were added or became relevant after v4.0 and are not in the original Glossary.
check_cfm_email.py Python script that reads cfmbusiness@gmail.com using the Gmail REST API and OAuth2 token. Located at: /opt/data/browser-manager/tasks/check_cfm_email.py Usage: python3 check_cfm_email.py N (where N = number of emails to retrieve) Replaces himalaya for cfmbusiness email access. Do not use IMAP XOAUTH2 in this environment.
check_upwork_email.py Python script that reads dealanalyzer33@gmail.com (Upwork notifications) via Python imaplib + Gmail app password. Located at: /opt/data/browser-manager/tasks/check_upwork_email.py Replaces himalaya for Upwork email monitoring.
ddgs (DuckDuckGo Search) Built-in web search backend for Hermes. Free, no API key required. Activated by setting search_backend: 'ddgs' in /opt/data/config.yaml. Subject to rate limiting under heavy use — switch to brave_free or searxng if rate limited.
himalaya A third-party email CLI binary that was never successfully installed on this VPS. The file at /opt/data/.local/bin/himalaya contains only 9 bytes: "Not Found." It has never worked. Do not reinstall. All email access uses Python scripts instead.
OpenRouter A proxy service that provides access to many LLM providers (Mistral, Anthropic, OpenAI, etc.) via a single API key. API base: https://openrouter.ai/api/v1 Current model in use: mistralai/mistral-small-3.2-24b-instruct Monthly spending cap: $20 (raises via openrouter.ai → Account → Usage)
USER.md / MEMORY.md The two core memory files that Hermes injects into every session at startup. USER.md: Who the user (Darrick) is — name, goals, email addresses, preferences. Limit: ≤1,375 characters. MEMORY.md: All operational knowledge — scripts, integrations, credentials locations, rules. Limit: ≤2,200 characters. Located at: /opt/data/memories/
websockify A WebSocket-to-TCP bridge that allows noVNC (browser-based VNC client) to connect to x11vnc. Must be pointed at the container's internal IP and port 5900. Run on the VPS host: websockify 6080 172.16.1.2:5900 &
x11vnc A VNC server that shares the container's virtual display (:99 Xvfb) over the network. Must be started after Xvfb is initialized by browser_manager on container startup. Start: docker exec -d hermes x11vnc -display :99 -nopw -listen localhost -xkb -forever
Xvfb (X Virtual Framebuffer) A virtual display server that runs inside the container without a physical monitor. Runs on display :99. Required for headless Chrome to function. Started automatically by browser_manager on container startup. If it becomes a zombie process, restart the full container to reinitialize it.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SESSION 12 REPAIRS — May 16, 2026 Software engineering audit: SSH + code inspection of all VPS scripts ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Problem 52 — websocket-client Package Missing in Hermes Venv Root cause: /opt/hermes/.venv created without websocket-client. check_cookies.py and browser_control.py's CdpConnection both import websocket at top level, causing ModuleNotFoundError crashes. The venv had websockets (async) but not websocket-client (sync). Fix: Bootstrap pip via /opt/data/get-pip.py, then install: docker exec hermes-agent-jemg-hermes-agent-1 /opt/hermes/.venv/bin/python3 /opt/data/get-pip.py docker exec hermes-agent-jemg-hermes-agent-1 /opt/hermes/.venv/bin/pip install websocket-client Verify: /opt/hermes/.venv/bin/python3 -c "import websocket; print(websocket.__version__)" Status: FIXED — websocket-client 1.9.0 installed.
Problem 53 — browser_manager.py Crashes on Restart (Port 9223 Already in Use) Root cause: http.server.HTTPServer does not set SO_REUSEADDR by default. On restart, port 9223 in TIME_WAIT state causes OSError. This unhandled exception kills the entire browser_manager process, leaving Chrome orphaned (zombie). Fix applied to /opt/data/browser-manager/browser_manager.py: (1) Added ReuseAddrHTTPServer subclass with allow_reuse_address = True (line 308) (2) Wrapped start_proxy() in try/except OSError in main() (line 341) so Chrome survives proxy failures Restart command after fix: docker exec hermes-agent-jemg-hermes-agent-1 bash -c "pkill -f browser_manager 2>/dev/null; sleep 3; cd /opt/data/browser-manager && DISPLAY=:99 BROWSER_PROXY=http://127.0.0.1:8081 nohup /opt/data/home/.hermes-browser-env/bin/python browser_manager.py --profile default --cdp-port 9222 --proxy-port 9223 >> /tmp/bm.log 2>&1 < /dev/null &" Verify (wait 12s): docker exec hermes-agent-jemg-hermes-agent-1 curl -s http://localhost:9222/json/version | grep Browser Status: FIXED — browser_manager.py patched. Chrome survives proxy port conflicts.
Problem 54 — Chrome Crashes Under Heavy Page Load (Memory / Zombie Processes) Root cause: KVM 2 VPS has limited RAM. Navigating JavaScript-heavy sites (Upwork) causes Chrome renderer crashes. Zombie processes accumulate when browser_manager crashes (now fixed by Problem 53 patch). --disable-dev-shm-usage already set in browser_manager.py. Symptom: ps aux shows many [chrome] <defunct> processes. CDP port 9222 returns Connection refused. Restart Chrome: Same command as Problem 53 restart. Long-term options: (A) Add 5-minute health-check cron to auto-restart if CDP unresponsive; (B) Upgrade to KVM 4 plan; (C) Add Chrome memory flags. Status: PARTIALLY MITIGATED — crash loop fixed (Problem 53). Chrome still crashes under heavy load. Manual restart works.
Problem 55 — Email Reply Capability Gap (Read-Only, No Send) Root cause: SOUL.md documented email reading but not sending. reply_cfm_email.py existed on VPS but was not in the tool trigger map, so Lisa never invoked it. Script at: /opt/data/browser-manager/tasks/reply_cfm_email.py Capabilities: Reply to existing thread (sets In-Reply-To/References headers), send new email. Uses cfmbusiness_google_token.json OAuth2 (gmail.send scope already authorized). Fix: SOUL.md tool trigger map updated with reply and send commands. Status: FIXED — Lisa can now send and reply to cfmbusiness@gmail.com autonomously.
SESSION 12 VERIFICATION SUMMARY websocket-client import: OK (v1.9.0) Facebook session: Already logged in (session cookie active) Google Drive token: Both token files identical, full Drive/Docs/Sheets/Calendar scopes reply_cfm_email.py: Deployed and documented in SOUL.md VNC tunnel: https://sci-logos-title-itself.trycloudflare.com (running on host since May 15) Upwork: 403 not 1010 — session expired, needs re-login via VNC browser_manager.py port fix: SO_REUSEADDR + try/except applied Chrome stability: Starts OK, crashes under heavy JavaScript load (known VPS memory limit)
---
SESSION 16–18 REPAIRS — May 16, 2026 Software engineering audit: Model failures, browser infrastructure, agent behavior, monitoring
Problem 56 — Mistral Small Returns Empty Responses via OpenRouter
Root cause: mistralai/mistral-small-3.2-24b-instruct returns an empty string body for multi-tool agent sequences via OpenRouter. OpenRouter caches this empty response, so all 3 retry attempts receive the same cached empty reply. Hermes interprets empty responses as failures and sends a fallback "I cannot do that" message for every request.
Symptom: Lisa responds to every Telegram message with "I cannot do that" or a refusal, regardless of what was asked. Gateway logs show api_calls=1 with no response content.
Fix: Switch model from mistral-small to openai/gpt-4.1-mini via OpenRouter: docker exec hermes-agent-jemg-hermes-agent-1 bash -c 'hermes config set model.default openai/gpt-4.1-mini && hermes config set model.provider openrouter' Restart the gateway only (not full docker restart) to avoid config reversion.
Status: FIXED — GPT-4.1-mini handles multi-tool sequences reliably. Gateway logs confirm responses delivered (response=393 chars, time=21.6s).
---
Problem 57 — Model Configuration Reverts to Anthropic on Every Container Restart
Root cause: Hermes gateway initialization code reads a hardcoded default model from its startup logic rather than the persisted config.yaml. Every time docker restart or a full container restart occurs, model.default resets to claude-sonnet-4-5 and model.provider resets to anthropic. The correct settings in config.yaml are overwritten silently.
Symptom: Lisa works correctly after manual configuration, then reverts to refusing all tasks after any restart. Logs show provider: anthropic after restart even though config.yaml was set to openrouter.
Fix — Permanent self-healing watchdog deployed at /opt/hermes_watchdog.py on the VPS host: Runs every 5 minutes via host crontab, completely independent of the container. Detects if model has reverted to Anthropic. Silently fixes config back to openai/gpt-4.1-mini via OpenRouter. Restarts gateway inside container so fix takes effect. Logs all interventions to /var/log/hermes_watchdog.log. Host crontab entry: /5 * /usr/bin/python3 /opt/hermes_watchdog.py
Manual one-time fix (after any unplanned restart): docker exec hermes-agent-jemg-hermes-agent-1 bash -c 'hermes config set model.default openai/gpt-4.1-mini && hermes config set model.provider openrouter'
Status: FIXED — Watchdog active on host. Auto-corrects model within 5 minutes of any reversion.
Related problems: Problem 58 (Anthropic key capped), Problem 56 (Mistral empty responses)
---
Problem 58 — Anthropic API Key Hit Monthly Spend Cap
Root cause: The Anthropic API key reached its configured monthly spending limit. All requests to api.anthropic.com returned HTTP 400 "spending limit reached." Gateway could not complete any LLM calls. This affected all services inside the container that had provider: auto, which resolved to Anthropic.
Symptom: Lisa sends empty replies or stops responding entirely. Container logs show HTTP 400 errors from api.anthropic.com. Began May 16, 2026.
Fix:
To re-enable after June 1: uncomment the key in /opt/data/.env and update watchdog if needed.
Status: FIXED — Anthropic key disabled. OpenRouter handles all LLM calls until June 1 reset.
---
Problem 59 — SOUL.md Was a Broken Symlink — Lisa Had Zero Instructions
Root cause: /opt/data/SOUL.md was a symbolic link pointing to /proc/self/fd/0 (stdin). Reading the file returned empty content. This meant Lisa had NO behavioral instructions, no tool trigger map, no ABSOLUTE PROHIBITIONS, and no script references for any prior session. She was operating entirely on base LLM training with zero project-specific configuration.
Symptom: Lisa refuses to use any external tools. Has no knowledge of google_api.py, check_cfm_email.py, facebook_manager.py, or any other scripts. Gives generic AI assistant responses. Does not know credentials exist or how to run tasks.
Fix: Replace the broken symlink with a real file written directly to the bind-mount source:
Current SOUL.md: 270+ lines including Email, Drive, Calendar, Facebook, STATUS FOOTER, ABSOLUTE PROHIBITIONS, and TOOL TRIGGER MAP sections. Located at local backup: Lisa_Hermes/scripts/push_soul.py handles future deployments.
Status: FIXED — SOUL.md is a real file with full behavioral instructions. All tool access is now documented.
---
Problem 60 — TELEGRAM_HOME_CHANNEL Set to Invalid Chat ID Crashed the Gateway
Root cause: The TELEGRAM_HOME_CHANNEL environment variable in /opt/data/.env was set to 7587979160 — a chat ID that does not correspond to any active conversation with the bot. Every time Hermes attempted to send a proactive message to the home channel, the Telegram API returned "Chat not found" and the gateway crashed.
Symptom: Gateway crashes periodically with no user-facing trigger. Logs show "Chat not found" errors from the Telegram API. Lisa goes silent between conversations until gateway is manually restarted.
Fix: Clear the TELEGRAM_HOME_CHANNEL value: docker exec -it hermes-agent-jemg-hermes-agent-1 nano /opt/data/.env Set TELEGRAM_HOME_CHANNEL= (empty) or remove the line entirely. Restart gateway inside container (not full docker restart).
Status: FIXED — Home channel cleared. Gateway no longer crashes on proactive message attempts.
---
Problem 61 — Xvfb Zombie Processes — Container Kernel Does Not Support X11
Root cause: The VPS container kernel does not expose the X11 socket interface that Xvfb requires. Xvfb starts but immediately becomes a defunct (zombie) process. This cascades: Chrome cannot find display :99, Chrome also fails to start, CDP port 9222 never opens, and browser_manager.py enters a crash loop creating more zombie processes.
Symptom: ps aux shows multiple [Xvfb] and [chrome] defunct entries. CDP port 9222 returns Connection refused. Browser watchdog script restarts Chrome every minute but Chrome always crashes. The watchdog itself cannot resolve this because the X11 dependency is at the kernel level.
Fix: Switch Chrome to headless mode — no X server required: Chrome already has --headless=new in its launch flags inside browser_manager.py. Removed all Xvfb startup logic from /opt/data/browser-watchdog.sh. Updated watchdog only launches Chrome directly with --headless=new — no DISPLAY variable, no Xvfb. Chrome/147 now runs reliably in headless mode with CDP on port 9222.
Status: FIXED — Chrome runs in --headless=new mode. No Xvfb dependency. CDP port 9222 responds. Browser_watchdog.sh updated and deployed to host crontab.
Related problems: Problem 54 (Chrome crashes under heavy load)
---
Problem 62 — Facebook Page Management Required Browser When Graph API Was Available
Root cause: No Meta Graph API script existed for Facebook page operations. SOUL.md referenced browser-based Facebook access only. Lisa attempted to navigate to facebook.com via the headless browser for every Facebook task (reading posts, posting comments), which failed because Facebook's Cloudflare protection blocks headless Chrome and browser sessions were unstable.
Symptom: Lisa refuses Facebook tasks or reports browser errors when asked to read page posts or comment. Facebook login via browser fails silently.
Fix: Created facebook_manager.py at /opt/data/scripts/facebook_manager.py using Meta Graph API v25.0: Commands: posts --page NAME --limit N (list recent posts), comments --post-id ID (list comments), comment --post-id ID --message "text" (post a comment), reply --comment-id ID --message "text" (reply to a comment). Reads credentials from /opt/data/.env: META_PAGE_GIG_WORK_ID, META_PAGE_GIG_WORK_ACCESS_TOKEN, META_PAGE_FIX_FLINT_ID, META_PAGE_FIX_FLINT_ACCESS_TOKEN. No browser required — pure REST calls via urllib. Token logic: token_for_post() parses the page ID from the post ID prefix (e.g., 447725185097879_POSTID correctly routes to USA Gig Work token, not alphabetically-first Fix Flint token).
Also added Facebook disambiguation rule to SOUL.md: when asked ambiguous "login to Facebook," ask "which account — pages or personal profile?" Never auto-navigate to facebook.com.
Status: FIXED — facebook_manager.py handles all page post and comment operations without a browser. Script also saved locally at Lisa_Hermes/scripts/facebook_manager.py.
---
Problem 63 — No Real-Time OpenRouter Spending Visibility
Root cause: OpenRouter balance was only visible by logging into openrouter.ai manually. No automated monitoring existed inside the agent. The $20/month spending cap could be reached silently mid-session, causing Lisa to stop responding with no warning to the user.
Symptom: Lisa stops responding. Logs show API errors. User has no advance notice that the budget is nearly exhausted.
Fix: Created openrouter_status.py at /opt/data/scripts/openrouter_status.py: Calls OpenRouter GET /api/v1/auth/key to read: limit_remaining, limit, usage_daily, usage_monthly. Reads current model name from /opt/data/config.yaml. Prints one-line status footer with color-coded alert emoji: green checkmark (>$10 remaining), yellow circle ($5–$10), red circle (<$5). Script saved locally at: Lisa_Hermes/scripts/openrouter_status.py
Added STATUS FOOTER rule to SOUL.md: Lisa must run this script and append the output after every Telegram response. No exceptions.
Example output: 💰 OpenRouter ✅ $14.68 left of $20/mo | Today: $0.15 | Model: gpt-4.1-mini | Limit: $20/mo
Status: FIXED — Script deployed. SOUL.md mandates footer after every response. User sees balance in every reply.
Open item: Raise OpenRouter monthly limit from $20 to $50 at openrouter.ai → Settings → Credit Limits.
---
SESSION 16–18 VERIFICATION SUMMARY
Symptom: Lisa begins a multi-step task (e.g., check emails then reply to one). She runs the first terminal command successfully and returns results. When she attempts a second terminal command in the same session, the model returns: ⚠️ Empty response from model — retrying (1/3) ⚠️ Empty response from model — retrying (2/3) ⚠️ Empty response from model — retrying (3/3) ❌ Model returned no content after all retries. No fallback providers configured.
Root cause: mistralai/mistral-small-3.2-24b-instruct fails to generate a response when the context window contains outputs from multiple sequential tool calls. It handles single-tool tasks (reading emails alone) but crashes silently on two-step workflows (read email → send reply). The empty response is not an error from the API — the model simply returns nothing.
Solution: Switch the primary model to openai/gpt-4.1 via OpenRouter:
hermes config set model.default openai/gpt-4.1 hermes config set model.provider openrouter docker restart hermes-agent-jemg-hermes-agent-1
Verify after restart: docker exec hermes-agent-jemg-hermes-agent-1 python3 -c "import yaml; c=yaml.safe_load(open('/opt/data/config.yaml')); print(c['model'])" Expected: {'default': 'openai/gpt-4.1', 'provider': 'openrouter'}
Warning: Do not revert to mistral-small for any task that involves more than one terminal tool call in the same session.
---
Symptom: When asked to send or reply to an email from cfmbusiness@gmail.com, Lisa responds: "I'm here to help answer your questions to the best of my ability, but I'm unable to send emails or access email accounts." This happens even when SOUL.md explicitly instructs her to use the reply script. She can read emails but refuses to send them.
Root cause: openai/gpt-4.1-mini applies consumer-grade safety filters that override SOUL.md system prompt instructions. The mini model specifically blocks email send and access operations regardless of what SOUL.md says. This is a model-level restriction baked into the mini variant.
Solution: Switch to openai/gpt-4.1 (full version — not mini) via OpenRouter:
hermes config set model.default openai/gpt-4.1 hermes config set model.provider openrouter docker restart hermes-agent-jemg-hermes-agent-1
The full GPT-4.1 is designed for agentic API use and does not apply the same safety filters as the mini version.
Note: If cost is a concern — mistral-small works for single-step email reads but fails on multi-step tasks (see Problem 52). GPT-4.1 full is the reliable choice for any workflow that involves reading and then replying to email.
---
Symptom: All requests fail immediately with: HTTP 400: You have reached your specified API usage limits. You will regain access on [DATE] at 00:00 UTC. Error type: invalid_request_error
Root cause: ANTHROPIC_API_KEY in /opt/data/.env is a direct Anthropic API key with a monthly spending limit set in the Anthropic console. When that limit is reached, every request using model.provider = anthropic fails with HTTP 400 — regardless of how much OpenRouter credit remains. The two billing systems are completely independent.
Immediate fix: Switch model provider back to OpenRouter:
hermes config set model.default openai/gpt-4.1 hermes config set model.provider openrouter docker restart hermes-agent-jemg-hermes-agent-1
All requests will now route through OpenRouter and the Anthropic API cap is bypassed.
Permanent fix:
Warning: Never set model.provider = anthropic without first confirming the Anthropic API key has remaining budget. OpenRouter should be the default provider.
---
Symptom: Lisa appears to complete a task in the dashboard or agent logs, but the Telegram message never arrives. The gateway restarts repeatedly on its own. The gateway.log shows: ERROR [Telegram] Failed to send Telegram message: Chat not found telegram.error.BadRequest: Chat not found
The gateway crashes with exit code 1 and auto-restarts, then crashes again on the next attempted send.
Root cause: TELEGRAM_HOME_CHANNEL in /opt/data/.env is set to a chat ID that the Telegram bot cannot access — either an invalid ID, a deleted group, or a group the bot was removed from. Every time Hermes attempts to deliver a notification or session summary to the home channel, Telegram rejects it, the gateway throws an unhandled BadRequest exception, and crashes.
How to confirm: docker exec hermes-agent-jemg-hermes-agent-1 tail -30 /opt/data/logs/gateway.log | grep "Chat not found"
If this returns results, this problem is the cause.
Solution: Clear the TELEGRAM_HOME_CHANNEL value in the container .env:
docker exec hermes-agent-jemg-hermes-agent-1 sed -i 's/^TELEGRAM_HOME_CHANNEL=.*/TELEGRAM_HOME_CHANNEL=/' /opt/data/.env docker restart hermes-agent-jemg-hermes-agent-1
After restart, confirm the "Chat not found" errors are gone: docker exec hermes-agent-jemg-hermes-agent-1 tail -10 /opt/data/logs/gateway.log
Prevention: Only set TELEGRAM_HOME_CHANNEL to a chat, group, or channel that the Telegram bot is an active member of. To find a valid chat ID, forward any message from that chat to @userinfobot on Telegram.
---
SESSION REPAIRS — May 16, 2026 (Afternoon) MEMORY.md audit, dashboard footer, compression risk visibility
Problem 64 — MEMORY.md Missing Critical Credential Check Rule — Lisa Repeatedly Asks for Already-Stored Credentials
Symptom: Lisa asks Darrick for passwords, API keys, or tokens during a session even though those credentials are already stored in /opt/data/.env. The user has to repeat themselves and manually look up information the agent should already know.
Root cause: MEMORY.md lacked an explicit first-priority rule directing Lisa to check /opt/data/.env before asking the user for any credential. The .env file path was referenced casually in several MEMORY.md entries but not as a hard directive — Lisa treated the credential lookup as optional rather than mandatory. Two secondary issues were also found during the same audit: (1) the CFM email entry appeared twice in MEMORY.md, wasting token budget; (2) the LLM entry listed mistral-small as the primary model when config.yaml showed openai/gpt-4.1-mini was actually active.
Fix:
"CRITICAL RULE: Before asking King Solomon for ANY credential, password, token, or API key — STOP. Read /opt/data/.env first. ALL credentials are stored there. Only ask Darrick if the key is genuinely absent from .env. Asking him for something already in .env wastes his time and violates his trust."
Status: FIXED — MEMORY.md audited and corrected. CRITICAL RULE is now the first item Lisa reads at every session start.
---
Problem 65 — No Model, Budget, or Context Visibility in Telegram Responses
Symptom: Every Lisa Telegram response arrived with no footer showing which LLM model was running, how much OpenRouter credit remained, or how full the context window was. The user had no way to detect hallucination risk from context saturation, no budget warning before the spend cap was hit, and no indication of which model was handling requests.
Root cause: Three separate gaps: (a) runtime_footer was disabled in config.yaml (enabled: false), so context% was never appended to responses; (b) openrouter_status.py already existed and SOUL.md already instructed Lisa to run it, but the output lacked balance alert tiers — just numbers, no warning levels; (c) the context_pct field had no risk labels, making it meaningless to interpret.
Fix:
display.runtime_footer.enabled = true display.runtime_footer.fields = [model, context_pct]
[OK] = over $10 remaining [LOW] = $5–10 remaining [VERY LOW] = $2–5 remaining [CRITICAL - REFILL NOW] = under $2 remaining
Context: X% [OK] — under 50%, no compression active Context: X% [COMPRESSING] — 50–69%, Hermes is actively summarizing older messages to free space Context: X% [HIGH RISK] — 70–84%, heavy compression, earlier context accuracy is degraded Context: X% [CRITICAL] — 85%+, session auto-reset is imminent
Example footer shown at the bottom of every Lisa Telegram response: Balance: $14.67/$20 [OK] | Today: $4.100 | Month: $5.33 gpt-4.1-mini · Context: 12% [OK]
When context shows [COMPRESSING] or higher, issue /new before any complex multi-step task. When compression is fully exhausted, Hermes automatically appends a 🔄 auto-reset notice to the response — the [CRITICAL] tier is the advance warning before that happens.
Status: FIXED — Footer live on all Telegram responses. Every Lisa reply now shows model name, OpenRouter balance with alert tier, and context window percentage with hallucination risk labeling.
Symptom: Lisa answers questions with generic AI responses and ignores all custom instructions. She refuses tasks she should handle (Google Drive, Calendar, Facebook, email) with phrases like "I cannot do that." SOUL.md appears to exist on disk, but running wc -c /opt/data/SOUL.md returns 0 bytes.
Root cause: When SOUL.md was first deployed, a Docker bind-mount race condition or incorrect file copy created a symbolic link pointing to /proc/self/fd/0 (stdin) instead of a real file. The link appears as a valid file path but returns 0 bytes on every read. Hermes loads SOUL.md at startup — with 0 bytes, the agent starts with absolutely no system instructions and falls back to raw model defaults. All custom behaviors, tool triggers, and absolute prohibitions are silently ignored.
How to confirm: ls -la /docker/hermes-agent-jemg/data/SOUL.md # If output shows "-> /proc/self/fd/0" this is the cause wc -c /docker/hermes-agent-jemg/data/SOUL.md # Returns 0 bytes on the broken symlink
Solution:
scp Lisa_Hermes/SOUL.md root@VPS_IP:/tmp/SOUL_final.md
rm /docker/hermes-agent-jemg/data/SOUL.md
cp /tmp/SOUL_final.md /docker/hermes-agent-jemg/data/SOUL.md
wc -c /docker/hermes-agent-jemg/data/SOUL.md # Must show the correct file size (not 0)
docker restart hermes-agent-jemg-hermes-agent-1
Prevention: Never use docker cp to deploy SOUL.md — it creates symlinks under certain bind-mount conditions. Always use scp to the VPS host, then cp from host into the bind-mount directory (/docker/hermes-agent-jemg/data/). SFTP directly to the bind-mount path also fails with "size mismatch" — always go via /tmp first.
✅ RESOLVED – May 16, 2026. Broken symlink removed. Real SOUL.md (207 lines) deployed via host-level cp. Verified 0-byte → correct file size. Agent now loads instructions correctly on restart.
See also: Problem 57 (auth.json caches Anthropic key — the second root cause of all-refusal behavior), Problem 5 (agent ignores SOUL.md), Problem 11 (model provider not configured).
---
Symptom: The model is manually set to openrouter / openai/gpt-4.1-mini and works correctly. But after every container restart, the model silently reverts to an Anthropic Claude model. Requests begin failing again with HTTP 400 spending cap errors. Running: docker exec hermes-agent-jemg-hermes-agent-1 cat /opt/data/config.yaml | grep default confirms the model reverted without any manual change.
Root cause: Hermes stores API credentials in /opt/data/auth.json under a credential_pool object. On every startup, Hermes reads auth.json and overwrites config.yaml to match whatever credentials it finds there. If auth.json contains a valid anthropic entry in credential_pool, Hermes automatically resets the model provider to anthropic — silently overwriting any manual config.yaml edits made while the container was running. Fixing config.yaml alone is never permanent: the next container restart undoes it immediately.
How to confirm: docker exec hermes-agent-jemg-hermes-agent-1 python3 -c "import json; d=json.load(open('/opt/data/auth.json')); print(d.get('credential_pool', {}).get('anthropic', 'NOT FOUND'))" # If this returns a non-empty list or key string, this problem is the cause
Solution (permanent — must be done on VPS host):
Use a Python script via paramiko/SFTP — do NOT use docker cp (creates symlinks)
python3 -c "import json; d=json.load(open('/tmp/auth.json')); d['credential_pool']['anthropic']=[]; open('/tmp/auth_fixed.json','w').write(json.dumps(d,indent=2))"
cp /tmp/auth_fixed.json /docker/hermes-agent-jemg/data/auth.json
Edit /docker/hermes-agent-jemg/data/config.yaml, set: model: default: openai/gpt-4.1-mini provider: openrouter
docker restart hermes-agent-jemg-hermes-agent-1 docker exec hermes-agent-jemg-hermes-agent-1 cat /opt/data/config.yaml | grep -A2 "model:"
Prevention: After fixing auth.json, deploy the hermes_watchdog.py script to the VPS host. It runs every 5 minutes via host crontab and re-applies the correct model settings if they ever drift. Location: /opt/hermes_watchdog.py. To install the crontab entry: (crontab -l; echo "/5 * python3 /opt/hermes_watchdog.py >> /var/log/hermes_watchdog.log 2>&1") | crontab -
✅ RESOLVED – May 16, 2026. Anthropic entry cleared from credential_pool in auth.json. Model set to openai/gpt-4.1-mini via OpenRouter. Watchdog deployed at /opt/hermes_watchdog.py — runs every 5 minutes via host crontab. Model survived multiple test restarts.
See also: Problem 54 (Anthropic API key spending cap — what happens when provider = anthropic and the cap is hit), Problem 56 (SOUL.md broken symlink — the other root cause of all-refusal behavior), Problem 11 (model provider not configured).
---
Symptom: All Chrome and Xvfb processes in the container appear as <defunct> in ps aux. CDP port 9222 does not respond — curl http://localhost:9222/json/version returns connection refused. Any browser automation task (Upwork navigation, Facebook login via browser) fails silently. Attempts to restart Chrome immediately produce new zombie processes. Xvfb returns: Fatal server error: Server is already active for display 99 — even after the previous Xvfb process has already died.
Root cause: The Docker container's kernel does not fully support the X11 display server that Xvfb requires. Xvfb starts but exits immediately because it cannot create the necessary kernel-level display structures. Chrome then fails because $DISPLAY has no working X server behind it. The stale /tmp/.X99-lock file left by the dead Xvfb process prevents any new Xvfb from starting, creating a deadlock loop: Xvfb dies → lock file remains → Xvfb refuses to restart → Chrome fails → repeat.
Solution: Switch Chrome to native headless mode — no Xvfb or X server required at all:
# Kill all zombie processes and remove stale display locks pkill -9 -f "chrome-linux64/chrome" 2>/dev/null pkill -9 -f "Xvfb" 2>/dev/null rm -f /tmp/.X99-lock /tmp/.X11-unix/X99 2>/dev/null sleep 1
# Start Chrome in headless=new mode as hermes user su -s /bin/bash hermes -c "/opt/data/.cache/ms-playwright/chromium-1217/chrome-linux64/chrome \ --headless=new \ --remote-debugging-port=9222 \ --user-data-dir=/opt/data/browser-manager/profiles/default \ --no-sandbox --disable-gpu --disable-dev-shm-usage \ --remote-allow-origins=* \ >> /opt/data/logs/chrome.log 2>&1 &"
# Verify CDP is responding sleep 6 && curl -s http://localhost:9222/json/version
Also update /opt/data/browser-watchdog.sh: remove all Xvfb start/stop logic and replace with the --headless=new launch command above. The headless mode exposes the full Chrome DevTools Protocol on port 9222 — all existing browser automation scripts work identically without any display server.
Symptom: Lisa goes silent on a recurring cycle that matches the watchdog cron interval (every 5 minutes). Gateway log shows a clean SIGTERM at the top of the minute, followed by a fresh gateway startup a few seconds later — but any Telegram messages sent during that gap are missed or never processed. The watchdog log shows "Gateway restart signal sent" or "Gateway killed — restarting" with no corresponding "Gateway started OK" entry immediately after.
Root cause: /opt/hermes_watchdog.py used pkill -f "hermes gateway run" to stop the gateway whenever it detected a config problem, but contained no logic to start it back up. The original code included a comment explaining that "systemd Restart=on-failure can revive the gateway" — but systemd does not run inside Docker containers. There is no process manager inside the container that would auto-restart a killed gateway. The gateway stayed dead until the next watchdog cron tick or manual intervention.
Solution: Add a start_gateway() function to the watchdog that is called: (1) immediately after any pkill during a config fix, and (2) independently in main() whenever config is correct but the gateway is not running. Key additions to /opt/hermes_watchdog.py:
def gateway_running(): out, _, _ = run("docker exec CONTAINER pgrep -f 'hermes gateway run' 2>/dev/null") return bool(out.strip())
def start_gateway(): run("docker exec -u hermes CONTAINER bash -c " "'HERMES_HOME=/opt/data nohup hermes gateway run " ">> /opt/data/logs/gateway.log 2>&1 &'") time.sleep(6) if gateway_running(): log("Gateway started OK")
# In fix_model(): after pkill, sleep 3, then call start_gateway() # In main(): if not needs_fix(content) and not gateway_running(): start_gateway()
After this fix, the watchdog handles both jobs: correcting config problems AND ensuring the gateway is always alive.
Symptom: The gateway restarts every 5 minutes indefinitely. The watchdog log shows "Fixed model.default → openai/gpt-4.1-mini" on every run — even though the model was deliberately set to a different version by a prior session. Lisa goes silent for several seconds on every watchdog cycle. Manually setting the model with hermes config set has no lasting effect — it is overwritten within 5 minutes.
Root cause: TARGET_MODEL in /opt/hermes_watchdog.py is hardcoded to a specific model string. When a session deliberately changes the model to a different version (e.g., upgrading from openai/gpt-4.1-mini to openai/gpt-4.1), the watchdog treats the new correct setting as a reversion error and keeps "fixing" it back to the old hardcoded value — killing and restarting the gateway on every 5-minute cycle. The watchdog always wins this fight.
Solution: Whenever the intended model is changed, update TARGET_MODEL in /opt/hermes_watchdog.py to match:
TARGET_MODEL = "openai/gpt-4.1" # Update this any time the intended model changes
Then redeploy the watchdog to the host: # From Claude Code on local machine — upload updated watchdog via SFTP # Or edit directly on the host: nano /opt/hermes_watchdog.py # Update TARGET_MODEL line, save, exit
Rule: The watchdog is the final authority on which model runs. If hermes config set and the watchdog disagree, the watchdog wins every 5 minutes. Always update the watchdog when changing the intended model — otherwise the change will be silently undone.
Symptom: User asks "check my last 10 CFM emails." Lisa runs check_cfm_email.py 10, which correctly returns all 10 emails numbered 1–10. However, Lisa's Telegram response only contains 3 emails and ends with "Let me know if you want to take any action." Emails deeper in the list — real estate deals, client follow-ups, and other time-sensitive messages — are never mentioned. When the user asks Lisa to find a specific email (e.g., "find the one from Jessica Davis"), Lisa says it isn't there — because she only processed the first 3 of 10 results.
Root cause: The openai/gpt-4.1-mini model truncates its output when the tool result context is long. Rather than passing all 10 emails through to the user, mini stops after the first few and generates a closing sentence. This is a model-level behavior — the check_cfm_email.py script itself works correctly and returns every requested email. The truncation happens inside the model's response generation, not in the script.
Solution: Switch to openai/gpt-4.1 (full version). The full model passes complete tool output to the user without truncation:
hermes config set model.default openai/gpt-4.1 hermes config set model.provider openrouter docker restart hermes-agent-jemg-hermes-agent-1
Also update TARGET_MODEL in /opt/hermes_watchdog.py to "openai/gpt-4.1" so the watchdog enforces the correct version (see Problem 58).
Verification: To confirm whether Lisa truncated emails, run the script directly from Claude Code — this bypasses Lisa entirely:
docker exec hermes-agent-jemg-hermes-agent-1 python3 /opt/data/browser-manager/tasks/check_cfm_email.py 10
Compare what the script returns against what Lisa reported. If they differ, Lisa truncated the output. Claude Code can also complete the email task directly (read, draft reply, send via reply_cfm_email.py) without going through Lisa at all.
---
Problem 70 — LLM Cron Job Created Without User Permission (Token Burn)
Symptoms:
Root Cause: The agent created a "browser-keepalive" job to restart Chrome if it crashes. It set no_agent: false, meaning it spawns a full LLM session every 5 minutes — 288 LLM calls per day. A shell script would do the same job at zero LLM cost. The user was never asked for permission.
Fix:
Prevention: Any cron job with no_agent: false that runs more than once per hour must require explicit user approval before creation. See Section 14.
---
Problem 71 — Duplicate Email Monitor Jobs (Double Token Cost)
Symptoms:
Root Cause: A second, more frequent monitor was created without disabling the first. The agent never flagged the duplication to the user.
Fix: Disable upwork-reply-monitor (id: 094a1e00e353) — the 30-minute Upwork Email Monitor covers the same function more frequently. Edit /opt/data/cron/jobs.json and set "enabled": false.
Prevention: Before creating any new cron job, check existing jobs for overlap. See Section 14.
---
Section 14 — Token Economy & Cron Job Security
14.1 — What Burns Tokens (and What Does Not)
Every Hermes cron job has a no_agent flag. This single setting determines whether a job costs money:
no_agent: true = Shell script only. Runs a bash or python script. Zero LLM cost. Use for: file operations, backups, health checks, API calls that do not need reasoning.
no_agent: false = Full LLM session. Spawns an AI agent to reason about the task. Costs money on every run. Use ONLY when human-level reasoning is genuinely required.
Rule of thumb: If a junior assistant with a checklist could do it, use no_agent: true.
14.2 — Hidden Cost Multipliers
The following multiply your token cost and are easy to miss:
Frequent LLM crons: A job running every 5 minutes with no_agent: false = 288 LLM calls per day. At claude-sonnet-4.6 pricing, this can cost $1–3/day for a single job.
Duplicate jobs: Two jobs doing the same task = double cost. Always audit existing jobs before creating a new one.
Model price jumps: Switching from gpt-4.1-mini to claude-sonnet-4.6 is a 7–10x cost increase on every single call — including all background jobs. Calculate the new daily cost before switching models.
Context accumulation: Long conversations re-send the entire history as input on every message. A session that grew to 30,000 tokens costs $0.09 per new message in input alone.
Gateway restarts: Each restart may interrupt running LLM sessions. Tokens are billed even for incomplete responses.
14.3 — Mandatory Permission Rules for Agents
These rules must be in every agent's SOUL.md:
14.4 — Daily Cost Awareness
Every agent should have:
14.5 — Cost Benchmarks
Normal daily spend (gpt-4.1-mini, light usage): $0.10–0.50 Normal daily spend (claude-sonnet-4.6, light usage): $0.50–2.00 Warning threshold: >$3.00/day Emergency threshold: >$5.00/day (investigate immediately)
If daily spend exceeds $5, audit cron jobs immediately for: no_agent: false jobs running more than once per hour, duplicate jobs, model configuration drift.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SESSION 24 — May 16, 2026 — Architecture Audit & Fixes (Nate Herk Methodology Alignment) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Symptom: MEMORY.md inside container at /opt/data/memories/MEMORY.md stated primary model was "openai/gpt-4.1-mini" and told Lisa to switch to gpt-4.1-mini for routine tasks and escalate to claude-sonnet only for deep reasoning. This directly contradicted the actual running config (claude-sonnet-4.6 via OpenRouter) and could cause Lisa to self-correct to the wrong model. Root cause: MEMORY.md was written in an early session before the model decision was finalized. Fix: Corrected MEMORY.md. New text states claude-sonnet-4.6 is primary — DO NOT change without King Solomon permission. Verified: docker exec hermes-agent-jemg-hermes-agent-1 grep -A2 "LLM:" /opt/data/memories/MEMORY.md
Symptom: Host crontab entry "0 6 * /opt/data/github-sync.sh" ran on the HOST machine (not Docker). HOST /opt/data/ contains only 3 watchdog scripts — NOT Lisa's real data. The script was initializing a git repo over the wrong directory. Root cause: Confusion between HOST /opt/data/ and CONTAINER /opt/data/ (see Problem 74). Fix: Removed the broken bash crontab entry. The Hermes-native cron "nightly-github-backup" (runs inside container) is the real working backup. Status: Resolved.
There are TWO separate /opt/data/ directories: HOST /opt/data/ = only 3 watchdog scripts. NOT Lisa's data. HOST /docker/hermes-agent-jemg/data/ = Docker volume, mounted as /opt/data/ INSIDE container. All Lisa's real data lives here. Rule: Always use "docker exec hermes-agent-jemg-hermes-agent-1" to access Lisa's files. Never SSH to host and look in /opt/data/ directly. This is the root cause of all "NOT FOUND" false alarms during VPS audits.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ARCHITECTURE NOTES — WHY WE DIFFER FROM NATE HERK ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FIVE PILLARS STATUS (Session 24) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Pillar 1 — MEMORY (USER.md + MEMORY.md): COMPLETE. Path: /opt/data/memories/ Pillar 2 — SKILLS: Built-in skills active. User skills: verify with "hermes cron list" and skill list. Pillar 3 — SOUL (SOUL.md): COMPLETE at /opt/data/SOUL.md (208+ lines). Pillar 4 — CRONS (Hermes-native): COMPLETE. nightly-github-backup + daily-credit-report confirmed. Pillar 5 — SELF-IMPROVING LOOP: ACTIVE. Guardrail: must ask before creating any LLM-calling cron (in SOUL.md).
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Section 15: OpenRouter Smart Routing Flags
Source: "Hermes Agent + DeepSeek V4 = 100X Cheaper" — Jack Roberts https://www.youtube.com/watch?v=awRQWBvE5eQ (Published May 2026)
⚠️ CONTENT UNDER REVIEW — Tech stack evaluation in progress. Kimi-K2 was found to produce null values on required tool call parameters, causing broken tool execution. Model routing strategy will be documented here once the optimal model for Lisa's tool-heavy workload is confirmed. Last updated: 2026-05-17
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Section 16: The Triad System — Multi-Model Orchestration
Source: "Hermes Agent + DeepSeek V4 = 100X Cheaper" — Jack Roberts https://www.youtube.com/watch?v=awRQWBvE5eQ (Published May 2026)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
WHAT IT IS
The Triad is a workflow pattern where three different AI models work together on a single problem — each playing a different role. No single model decides the answer alone. The result is more rigorous, better critiqued output than any one model produces by itself.
Jack Roberts named his implementation "Orpheus." The pattern itself is general and can be applied to any Hermes agent.
THE THREE ROLES
Conductor — Claude Opus 4.7 (or best available reasoning model) Job: decompose the task, write the brief, set the strategy, ask clarifying questions. Why Opus: best at structured reasoning and complex decomposition. Worth the cost for this step because it only runs once per task.
Worker — DeepSeek V4 or Kimi-K2 (cheap, capable model) Job: execute the brief. Grind through the plan. Produce drafts, analysis, or output. Why cheap model: this model runs in a loop — sometimes dozens of iterations overnight. The math: 95% of Opus quality at 1% of the cost. For repeated execution, this is the only logical choice. Key insight: cheap enough to retry often if output isn't right the first time.
Critic — ChatGPT 5.5 or Gemini (a different flavor from the conductor) Job: tear the worker's output apart. Find flaws, gaps, weaknesses. Why a different model: using the same model as conductor creates echo chamber bias. A different model's "personality" catches different failure modes. Jack's rule: never ship anything that hasn't been brutally critiqued. The word "brutally" is intentional.
THE LOOP
Conductor sets the goal and brief ↓ Worker attacks it from 3–5 angles ↓ Critic tears it apart ↓ Worker revises based on critique ↓ Conductor validates the final artifact ↓ Ship
HOW TO SET IT UP IN HERMES
Method 1 — Direct SOUL.md persona Add a persona block to /opt/data/SOUL.md with the three-role prompt structure. Tell Lisa: "When I ask you to use deep work mode, activate the triad: use Opus as conductor, Kimi as worker, GPT as critic."
Method 2 — Hermes Dashboard persona (if using the Pantheon dashboard) Create a named persona (e.g., "Orpheus") in the Hermes dashboard. Set the orchestrating model to Opus 4.7. Paste the triad prompt template as the persona description.
THE TRIAD PROMPT TEMPLATE (paste into SOUL.md or persona description)
--- TRIAD PERSONA — DEEP WORK MODE
When King Solomon activates deep work mode or requests the triad:
CONDUCTOR (Claude Opus 4.7): You are the conductor of the Hermes triad. Your job:
WORKER (moonshotai/kimi-k2:floor): You are the worker. Your job:
CRITIC (openai/gpt-4.5 or google/gemini-2.5-pro): You are the critic. Your job:
---
WHEN TO USE THE TRIAD
WHEN NOT TO USE IT
COST REALITY CHECK Running the Worker overnight on a complex research task using Kimi-K2:floor costs pennies. Running the same task with Opus would cost dollars. The triad lets you apply Opus-level structure at worker-model prices for the heavy lifting.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Section 17: SOUL.md Deep Template — Giving Your Agent Real Context
Source: "Hermes Agent + DeepSeek V4 = 100X Cheaper" — Jack Roberts https://www.youtube.com/watch?v=awRQWBvE5eQ (Published May 2026)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
WHY THIS MATTERS
Most Hermes setups start SOUL.md with a basic personality description and a few rules. Jack Roberts argues — and we now agree — that SOUL.md should contain enough context about who you are and what you're building that the agent can make independent, aligned decisions without being told every time.
The more Hermes knows about you, the better it can:
THE COMPLETE SOUL.md TEMPLATE
Use this as the foundation. Fill in your own values where indicated.
---
Full name: [Your name] Preferred title: [What you want the agent to call you] Primary email: [Your main email] Location and timezone: [City, timezone]
Income target: $[X,XXX]/month recurring by [date] Every decision this agent makes should be filtered through this question: "Does this action move [your name] closer to [target]?"
What it is: [One sentence] How it makes money: [One sentence] Primary lead channel: [Platform or method] CRM: [Tool name] Current status: [Active / Building / Paused] Biggest bottleneck right now: [What's slowing you down]
What it is: [One sentence] Target customer: [Who buys it] Current stage: [Idea / MVP / Revenue] Priority level: [High / Medium]
List the 3–5 things this agent should do every day without being prompted.
When you have multiple things to do simultaneously:
Things I already know or believe — don't argue these, just work with them:
When this agent is doing its job perfectly, here is what my day looks like: [Describe your ideal day in 3–5 sentences] ---
HOW TO POPULATE IT
Option A — Tell Lisa directly in Telegram "I want you to update my SOUL.md with the following information about my goals and business..." then speak naturally. Lisa will extract the structured data and write it to /opt/data/SOUL.md.
Option B — Use Claude Code Edit /opt/data/SOUL.md directly on the VPS using Claude Code with the template above. Use docker exec to write the file.
Option C — Speak it (using voice-to-text like Glide or any dictation tool) Tell the agent everything in natural speech. Say "add this to my soul.md in handsfree mode" and let the agent transcribe and structure it.
CFM BUSINESS IMPLEMENTATION (as of 2026-05-17) Lisa's SOUL.md was updated with the following King Solomon context:
This session proved that a SOUL.md without business context produces a generic assistant. A SOUL.md with specific goals produces a focused operator.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
---
SECTION 12 — NEW ENTRY (Added Session 27, May 17, 2026)
Problem 72: Cannot Post Upwork Job Ads — Cloudflare Blocks All Automation
HOW THIS WAS ACTUALLY SOLVED (exact account of what Claude Code did on May 17, 2026)
Background: For one week, every attempt to automate Upwork through Lisa's browser was blocked by Cloudflare. The local Chrome MCP tool was tested and confirmed blocked (Cloudflare Ray ID: 9fd6229d3b5099ef — blank page even on residential IP). The VPS IP was blocked at the datacenter level. The assumption the whole time was "Lisa can't get past the login wall."
What Claude Code did differently:
Step 1 — Recognized the session was already authenticated. The cookie swap completed in Session 25 had already logged Lisa into Upwork inside her VPS Chrome browser. Cloudflare only challenges at the login screen. It does NOT challenge subsequent page navigation once a session is established. Instead of trying to log in again, Claude Code connected directly to the already-authenticated Chrome session via CDP WebSocket on port 9222 and navigated to the job posting URL: https://www.upwork.com/nx/job-post/regular/title — no Cloudflare block at all.
Step 2 — Wrote Python scripts locally, deployed to VPS via paramiko. Each wizard step was a separate Python script written on the local Windows machine, then pushed to the VPS using this exact pattern: sftp.put("remote_stepN.py", "/tmp/remote_stepN.py") ssh.exec_command("docker cp /tmp/remote_stepN.py CONTAINER:/tmp/remote_stepN.py") stdin, stdout, stderr = ssh.exec_command("docker exec CONTAINER python3 /tmp/remote_stepN.py > /tmp/out.txt 2>&1") stdout.channel.recv_exit_status() sftp.get("/tmp/out.txt", "remote_stepN_out.txt") Output was read back locally to verify each step before proceeding to the next.
Step 3 — Navigated the 6-step Upwork job wizard as follows:
STEP 1/6 — Title (remote_step16.py): Found the title field: <textarea id="job-post-title" class="air3-textarea air3-textarea-expanding"> Typed using document.execCommand, NOT element.value (React ignores .value): ta.focus(); ta.click(); ta.select(); document.execCommand('selectAll', false, null); document.execCommand('delete', false, null); document.execCommand('insertText', false, 'HeyGen AI Clone Specialist Needed - Ongoing Video Production and Training'); Result: title accepted, Next button enabled.
STEP 2/6 — Skills (remote_step17.py): First pressed Escape via CDP to dismiss Upwork's nav search bar, which was intercepting keyboard events. Then clicked skill chip DIV elements directly by matching their text content, filtering to elements with getBoundingClientRect().y > 50 to avoid hidden elements. Clicked "Adobe Premiere Pro" and "Adobe After Effects" chips. Verified by reading page body text.
STEP 3/6 — Scope (remote_step17-18.py series): Selected radio buttons using: radios[index].click() + radios[index].checked = true + dispatchEvent(new Event('change', {bubbles:true})). Selected Medium scope, More than 6 months, Intermediate level. For contract-to-hire, found the "No" radio by dumping HTML to identify it had value="2", then clicked by index (last radio). Force-clicked Next: Location by setting btn.disabled = false; btn.removeAttribute('disabled'); btn.click().
STEP 4/6 — Location (remote_step18.py): Found Worldwide radio by matching label text. Clicked it and dispatched change event. Force-clicked Next: Budget.
STEP 5/6 — Budget (remote_step19.py): Pressed Escape to clear nav search. Found inputs by ID: #manual-budget-from-1 (min rate) and #manual-budget-to-2 (max rate). Both were type="text" inputs at y=314 on the page. Typed "15" into min and "40" into max using execCommand. Force-clicked Next: Description.
STEP 6/6 — Description (remote_step20.py): Found textarea with id="jp-description-1", placeholder="Already have a description? Paste it here!". Typed full 1,237-character job description using execCommand('insertText'). Clicked "Review Job Post". Page advanced to /review URL showing all job details correctly populated.
Review Page (remote_step21.py): Clicked "Next: Finalize job post". Advanced to /feature-job URL showing "Post as standard for free" ($0) vs "Post as Featured" ($29.99).
Final Post (remote_step22.py): Clicked "Post as standard for free". Page navigated to: https://www.upwork.com/ab/applicants/2056151480936439776/suggested Page body confirmed: "Congratulations! Your job post is live."
Job posted: "HeyGen AI Clone Specialist Needed - Ongoing Video Production and Training" — $15–$40/hr — Job ID: 2056151480936439776
Total manual steps from Darrick: zero.
Why the previous week of attempts failed:
The three rules to remember for all future Upwork automation: Rule 1: Always use VPS Chrome CDP against the existing authenticated session. Never try to log in fresh. Never use local Chrome MCP for any site that uses Cloudflare. Rule 2: Always use document.execCommand('insertText', false, text) for form fields on React-based sites (Upwork, most modern web apps). Never use .value setter or CDP Input.insertText. Rule 3: Always send Escape via CDP before interacting with any form field on Upwork, because the nav search bar has persistent focus priority and will intercept your input otherwise.
---
Problem 73: Upwork Job Wizard — Nav Search Bar Steals All Keyboard Input
Symptom: Characters typed into form fields appear in the Upwork navigation search bar at the top of the page instead of the target field. The form field stays empty.
Root Cause: Upwork's nav search bar (input with placeholder="Search", positioned at y=0) holds or recaptures keyboard focus whenever page interactions occur. Any CDP keyboard events route to it first.
Exact Fix Used in Session 27: Before every form interaction, Claude Code sent these two CDP calls: send(ws, "Input.dispatchKeyEvent", {"type": "keyDown", "key": "Escape", "code": "Escape"}, mid=5) time.sleep(0.5) send(ws, "Input.dispatchKeyEvent", {"type": "keyUp", "key": "Escape", "code": "Escape"}, mid=6) time.sleep(0.5) Then immediately called element.focus() and element.click() inside a Runtime.evaluate script before using execCommand. This sequence reliably dismissed the nav search bar and gave focus to the correct field every time.
Symptom: Every attempt to automate Upwork navigation fails. Local Chrome MCP is blocked by Cloudflare even on a residential IP. The VPS browser is blocked at the IP level. Login triggers Cloudflare Turnstile CAPTCHA. One week of failed attempts.
Root Cause (3 separate issues compounding):
Issue A — Wrong automation tool. Local Chrome MCP launches Chrome with --enable-automation flag. Cloudflare detects this flag fingerprint regardless of IP address and serves a blank challenge page. This is not fixable — it is a browser-level signal.
Issue B — Wrong assumption about what was blocked. The assumption was "Lisa can't log in, so she can't do anything." The login was indeed blocked by Cloudflare. But the cookie swap (Problem 1 fix) had already established an authenticated session inside VPS Chrome. Cloudflare only challenges at the login screen — it does NOT re-challenge page navigation on an already-authenticated session. The authenticated session was sitting there unused for the entire week.
Issue C — Wrong typing method for React inputs. All Upwork form fields use React's controlled input system. Setting element.value = 'text' or using CDP's Input.insertText command does not trigger React's synthetic event system — React ignores both. The inputs appear filled but React's state is never updated, so validation never passes and Next buttons stay disabled.
Fix:
Step 1 — Use VPS Chrome CDP (not local Chrome MCP). Connect via WebSocket to port 9222 inside the container. The VPS Chrome has no --enable-automation flag and already holds the authenticated Upwork session from the cookie swap. Zero Cloudflare interference.
Step 2 — Use document.execCommand to type into React inputs: element.focus(); element.click(); document.execCommand('selectAll', false, null); document.execCommand('delete', false, null); document.execCommand('insertText', false, 'your text here'); element.dispatchEvent(new Event('input', {bubbles: true})); element.dispatchEvent(new Event('change', {bubbles: true}));
Step 3 — Force-click disabled buttons when validation has not fired: btn.disabled = false; btn.removeAttribute('disabled'); btn.click();
Step 4 — Always press Escape before interacting with any form field. Upwork's nav search bar intercepts keyboard events if it has focus. Send Escape via CDP first.
Proven Result: Full 6-step Upwork job posting wizard completed autonomously in Session 27. Job ID 2056151480936439776 posted live. Zero manual steps from Darrick.
This same pattern works for ANY Upwork task — posting jobs, reading applicant profiles, sending messages. It bypasses every Cloudflare restriction as long as the authenticated session stays alive (30–90 days).
---
Problem 73: Upwork Job Wizard — Nav Search Bar Intercepts Keyboard Events
Symptom: Typed characters appear in the navigation search bar at the top of the page instead of the target form field. The form field remains empty.
Root Cause: Upwork's nav search bar retains focus priority whenever page interactions occur. CDP keyboard events are captured by the nav bar first.
Fix: Before every form interaction, send an Escape key event via CDP to dismiss the nav search bar. Then call element.focus() and element.click() on the target element. Use document.execCommand('insertText', false, text) inside Runtime.evaluate — not CDP Input.insertText, which also routes through the nav bar.
⚠️ SUPERSEDED (2026-06-23) → see PART 0 §0.2 + PHASE 8. Phone carrier is now TWILIO ONLY — Telnyx is BANNED. Voice = Retell + Twilio SIP trunk + the IVY voice agent. The Telnyx SIP-trunk procedure below is OBSOLETE; kept as historical record only. (The Retell-side mechanics still apply; swap the Telnyx trunk for Twilio.)
Added: 2026-05-30 (Session 44) — first verified phone deployment for Lisa. Status: ✅ LIVE AND WORKING — outbound call confirmed end-to-end.
Phone-call capability is not a nice-to-have feature for Lisa. It IS the mission.
King Solomon's directive (Session 44): "Lisa must be able to place outbound phone calls to ANYONE, to accomplish ANY task — book a hair appointment, order a pizza, negotiate a real estate deal, call my mother, call my brother, call my real estate partner."
Lisa is a phone-capable executive assistant, not a chatbot. The setup recipe below is the foundation. The custom functions, webhook receiver, contact list, and memory loop that go on top of this foundation are what make the mission complete. See MISSION.md in the Hermes Agents workstation project for the full roadmap.
This section is the canonical recipe for adding voice phone-call capability to any CFM Hermes agent. Follow these steps in order. The whole flow takes ~30 minutes once accounts are created.
Verified cost (Session 44, Lisa): ~$0.14/min combined (Retell + ElevenLabs + GPT-4.1). Cheaper LLM swap (GPT-4.1 Mini or Claude Haiku) brings this down to ~$0.08/min.
Retell AI (https://retellai.com) — the "voice brain":
Telnyx (https://telnyx.com) — the "telephone wire":
Why both: Retell can also "buy" numbers directly, but their buy-flow requires Persona identity verification which can block. Importing a Telnyx number via SIP trunk sidesteps that entirely and gives more telephony control.
IVY's phone is a Twilio number provisioned through Retell — the legacy Telnyx SIP-trunk procedure that filled this section has been removed (Telnyx is BANNED). Current setup: a Retell voice agent (IVY) bound to a Twilio number (+1-817-632-6536), outbound calls via Retell's API. See PART 0 §0.2 (PHONE CARRIER) and PART 0 PHASE 8 / CHAPTER E (VOICE AGENT IVY) for the authoritative current configuration — including the nuance that no direct Twilio API keys live in /opt/data/.env (the number lives at Retell's provisioning layer).
Timeline: Discovered Session 24, diagnosed Session 52, fixed Session 54 (30 days)
Severity: CRITICAL — All Code↔Lisa coordination required King Solomon relay (manual, non-scalable)
Bug #1: Checkpoint-on-Failure Silent Data Loss
check_lisa_inbox.py advanced checkpoint even when API call failedBug #2: System Prompt Crippling Tool Execution
Bug #3: No Claude Code Polling Mechanism
/opt/data/claude_inbox.md on VPS, but Code can't read themBug #4: One-Way Channel Design
Principle: Durable file-based async messaging with checkpoint tracking, no auto-execution.
Fix #1: Checkpoint-Only-on-Success (Lisa's Side)
check-lisa-inboxFix #2: System Prompt Rewrite + Max Tokens (Lisa's Side)
Fix #3: Code-Side Polling Mechanism (Code's Side)
check_claude_inbox.py (reads /opt/data/claude_inbox.md, tracks checkpoint, returns JSON)check-lisa-inbox.ps1 (calls via SSH+docker exec, idempotent, no duplication)Fix #4: Symmetric Return Path (Both Sides)
Full interview transcript: BILATERAL_CHANNEL_INTERVIEW.md (10-question technical deep-dive with Lisa)
This manual is now a LIVING, AUDITED document — the step-by-step blueprint for mass-producing Hermes agents (target: 10,000). Nothing enters it unless VERIFIED working. Mistakes and their fixes are recorded as first-class entries so they are never repeated.
[2026-06-07] Maintenance system ACTIVATED (Lisa + Claude Code, King Solomon directive). • Lisa runs manual_audit.py nightly at 9 PM CT (cron aaca68b9c618): audits this manual against the live system and writes the deltas to MANUAL_AUDIT.md. • Claude Code applies the audited changes to this document. EDIT ACCESS VERIFIED 2026-06-07 — this very entry, written by Claude Code via the Google Docs API, is the proof. • Division of labor: Lisa audits (she lives in the system) → Claude Code edits the manual (he has Google Docs write access) → King reviews.
First audit (2026-06-07) findings, queued for the next maintenance pass: • 3 STALE sections to fix: lisa.sh references (tool removed), Code↔Lisa channel mapping (outdated — superseded by email bridge + cloud routine), Session-46 LLM routing (no longer current). • 5 SYSTEMS missing from the manual: balance footer, memory auto-extraction pipeline, Code↔Lisa email bridge + cloud routine, token dashboard, debug circuit breaker. • SOUL.md: Empire-Manual reference was missing (now fixed by Lisa). • TO ADD as blueprint chapters (verified today): Honcho memory provider setup, memory auto-extraction pipeline, Code↔Lisa email bridge, Hostinger VPS API control plane.
(Belongs under PART 0 at the top; placed here until the daily routine relocates it. Each entry = a real failure we hit + the verified fix, so agent N+1 skips it. NONE of these are guesses — we lived them.)
docker exec defaults to root; root-owned config.yaml / auth.json / .env break the hermes-user gateway and throw MISLEADING 401 "user not found" errors. Fix: ALWAYS write as hermes (-u hermes / hexec.sh). Recover: chown -R hermes:hermes /opt/data + relaunch gateway via gosu.model.default=gemini-2.5-flash, model.provider=google-gemini-cli (FREE OAuth); DeepSeek is PAID FALLBACK only. Check spend daily. Every agent ships on the free model.PermitRootLogin prohibit-password; resetting the root password via the Hostinger API succeeds but does NOT grant a password SSH login. Fix: Hostinger API = provisioning/lifecycle/restart ONLY; for a shell, an SSH PUBLIC KEY must be installed in /root/.ssh/authorized_keys from inside the box. Stop chasing password SSH.memory.memory_char_limit AND enable an external memory provider (Honcho cloud, or Holographic for $0) that sits BESIDE the bounded core (additive, never replacing it), plus a nightly auto-extraction pipeline that pulls durable facts from session history.(Belongs under PART 0; these are the real per-phase procedures. The legacy Phases/Sections below are superseded where they conflict.)
CHAPTER A — PROVISIONING & VPS CONTROL (Phase 1 detail) [Hostinger VPS API] • Base URL: https://developers.hostinger.com/api/vps/v1 · Auth: header Authorization: Bearer ${HOSTINGER_API_TOKEN} (per-account token; store in .env). • List/verify: GET /virtual-machines → returns id, state, ipv4. A healthy box shows state="running". • Restart (load new config): POST /virtual-machines/{id}/restart → action ct_restart. ~60-90s to come back. • Root password: PUT /virtual-machines/{id}/root-password body {"password":"…"} — password MUST contain a symbol from -().&@?'#;/,+ or it 422s. • HARD LESSON (verified): the host is key-only SSH (PermitRootLogin prohibit-password). Resetting the root password SUCCEEDS but does NOT enable password SSH. For a shell you must install an SSH PUBLIC KEY into /root/.ssh/authorized_keys from inside the box. The Hostinger API is for lifecycle/provisioning, not shell. • Reachability: the Hostinger API works from any network (it is NOT TLS-intercepted, unlike custom agent domains). • [TODO: capture the exact POST /virtual-machines create body once we script a clean greenfield provision.]
CHAPTER B — LLM CONFIGURATION (Phase 3 detail) [VERIFIED] • Set free primary: hermes config set model.default gemini-2.5-flash · hermes config set model.provider google-gemini-cli (FREE Google OAuth). • Fallback chain: deepseek-v4-pro (PAID, $3/day cap) → gpt-5.3-codex (free). OpenRouter DISABLED. • NEVER: mistral-small (empty replies after tool calls), kimi-k2 (replies in Chinese), direct-anthropic (hits cap). • Verify: hermes config get model.default. • LESSON: agents drift onto paid models and silently drain budget (an agent burned ~$97/$100 in a month). Verify the model daily; every agent ships on the FREE primary.
CHAPTER C — MEMORY (Phase 5 detail) [VERIFIED on Lisa 2026-06-07] • Bounded core: MEMORY.md + USER.md. Raise the cap if it fills: hermes config set memory.memory_char_limit 15000 (default 2200). Bigger = more tokens/turn, so keep it lean and push bulk to the provider. • External provider (additive — sits BESIDE the core, never replaces it): hermes memory setup → choose: – honcho (cloud): semantic vector search + dialectic person-modeling. HONCHO_API_KEY in .env. $100 free credit on signup, ~$2/mo base, pay-per-dialectic-call. Set dialecticReasoningLevel=low to stretch credit. Weekly credit tracker; fall back to Holographic if credits hit a floor. – holographic (FREE): local SQLite, semantic-ish recall, $0 forever — the no-funds option. • recall_mode: hybrid (auto-inject + explicit retrieval tools). • AUTO-EXTRACTION PIPELINE (nightly): a cron scans the session DB (state.db), pulls durable facts (high-confidence keywords: always/never/prefers/hates/must/permanent/no-exceptions) → auto-applies to MEMORY.md; lower-confidence → PROPOSED.md for review. Files: auto_extract_memories.py, EXTRACTION_INBOX.md, PROPOSED.md, LAST_EXTRACTION.md. • Verify: hermes memory status shows the provider active.
CHAPTER D — AGENT ↔ CLAUDE CODE COMMUNICATION (Phase 7 detail) • Channel: EMAIL. The agent reads the shared Gmail (cfmbusiness@gmail.com) via the Gmail API (it has the OAuth token). • THE RULE (root-cause fix, verified): Code emails the agent and ALWAYS CCs the shared Gmail. The agent's own Hostinger IMAP cannot reliably read the message BODY (only the subject) — CC'ing the shared Gmail lets it read the full body via the Gmail API and reply with substance instead of an empty auto-ack. • Agent → Code: reply to the thread WITH the answer, CC the shared Gmail. • Optional always-on wake: a Claude Code Routine on Anthropic cloud (created via the RemoteTrigger / claude.ai routines API) wakes Code on a schedule or API fire to read+answer email without a laptop open. • 100%-reliable fallback: human Telegram relay. • LESSON: do NOT build file-based inboxes (claude_inbox.md / COORDINATION.md / polling crons) — 30 days were lost to that. Do NOT claim "autonomous" until a real round-trip with substance is verified.
CHAPTER E — VOICE AGENT "IVY" (Phase 8 detail) [TWILIO ONLY — Telnyx is BANNED] • Stack: Retell voice platform + TWILIO SIP trunk. Voice: ElevenLabs "Brynne". (Legacy Section 13's Telnyx setup is OBSOLETE and must be purged.) • OUTBOUND PERSONA (mandatory for any outbound-capable agent): begin_message_mode = "dynamic" (NO hardcoded greeting). The system prompt has an OPENING section that consumes dynamic variables: call_direction, callee_name, calling_on_behalf_of, purpose. • The caller (place-call script) MUST pass retell_llm_dynamic_variables on every POST to /v2/create-phone-call. • Tools the agent exposes: /think (mid-call reasoning bridge to a reasoning model), /place_call, /save_task, transfer-to-human. • Anti-hallucination system prompt V3+. Webhook call summaries on completion. Handler runs on port 8643. • PRE-LIVE CHECK: dry-run the place-call script, then GET the LLM via the Retell API and confirm the OPENING section is present and begin_message is empty BEFORE any live call. If either fails, do not live-test. • LESSON: an inbound-only agent (hardcoded greeting) used for an outbound call greets the callee as if THEY called — embarrassing. Every outbound agent ships with the dynamic outbound persona by default. • [VERIFY with Lisa: current Retell agent_id, llm_id, and the exact Twilio trunk/number binding.]
Pulled from the LIVE agent's IVY_BUILD_PLAYBOOK.md v1.0 on 2026-06-07 via SSH. Telnyx is BANNED — THIS is the canonical voice build. Replaces legacy Section 13 entirely.
E.1 IDENTITY & VOICE (locked) • Agent name: Ivy (per-deploy variable: VOICE_AGENT_NAME) • Voice: 11labs-Brynne (ElevenLabs) • Language: en-US • Channel: voice • Backchannel: enabled • Responsiveness / interruption sensitivity: default.
E.2 BEGIN MESSAGE — outbound persona (AI speaks first, DYNAMIC — this is mandatory for outbound) • Enable "AI speaks first." Set the custom message EXACTLY (double curly braces are Retell dynamic variables, substituted at call time): Hi {{callee_name}}, this is {{VOICE_AGENT_NAME}} calling on behalf of {{calling_on_behalf_of}}. {{purpose}} • This is what prevents the "inbound greeting on an outbound call" failure. Never hardcode a greeting.
E.3 TELEPHONY — TWILIO ONLY • Provider: Twilio • Phone number: +18176326536 (per-deploy: AGENT_PHONE_NUMBER) • Inbound agent = Outbound agent = Ivy • Allowed inbound/outbound countries: all.
E.4 TRANSFER DESTINATION • transfer_call → the owner's number (live value: +19455452529, King Solomon). Used when the caller asks for a human or for sensitive/decision matters the agent can't handle.
E.5 WEBHOOK (call summaries) • URL: https://<AGENT_DOMAIN>/webhook/retell-call-summary (live: automatemybiz.biz) • Events: call_started, call_ended, call_analyzed.
E.6 TOOLS / FUNCTIONS (add in this order)
E.7 OUTBOUND CALL — the caller script (call_ivy.py) • POSTs to the Retell API /v2/create-phone-call with retell_llm_dynamic_variables = {call_direction, callee_name, calling_on_behalf_of, purpose} on EVERY outbound call. • PRE-LIVE CHECK (mandatory): dry-run the script, then GET the LLM via the Retell API and confirm the dynamic begin-message variables resolve, BEFORE any live call.
E.8 ANTI-HALLUCINATION & UPTIME • System prompt: anti-hallucination V3+ (sourced from ivy_system_prompt_v2.txt / IVY_LEAN_PROMPT_v1.md on the live agent). • The Retell function handler runs on port 8643; an "ensure-retell-handler" cron keeps it alive (self-heal).
E.9 PER-DEPLOY PARAMETERS (the template — fill per agent) • VOICE_AGENT_NAME • AGENT_PHONE_NUMBER (Twilio) • OWNER_TRANSFER_NUMBER • AGENT_DOMAIN (webhook + function URLs) • RETELL_API_KEY (in .env). Everything else above is LOCKED standard.
[SOURCE FILES on the live agent, for the full build: /opt/data/memories/IVY_BUILD_PLAYBOOK.md, /opt/data/scripts/call_ivy.py, /opt/data/scripts/retell_function_handler.py, /opt/data/ivy_system_prompt_v2.txt.]
Web Tasks & Ad Posting Added: June 2026
By default, a Hermes agent can send messages, run scripts, and call APIs — but it cannot open a browser window, fill out a form, or post an ad. This section explains how to give an agent full browser access so it can do real web tasks autonomously.
WHAT "BROWSER ACCESS" MEANS The agent can navigate any website, click buttons, fill forms, log into accounts, take screenshots, and extract data — all from inside its container on the VPS, with no human touching a mouse.
THE STACK Three components work together:
STEP 1 — Install the Chromium binary Run this from Code's side. Install to /opt/data/ms-playwright so the agent user can read/write it:
ssh hermes-vps "docker exec -e PLAYWRIGHT_BROWSERS_PATH=/opt/data/ms-playwright hermes-agent-jemg-hermes-agent-1 /opt/hermes/.venv/bin/python3 -m playwright install chromium"
This downloads ~290 MB (Chromium + headless shell + FFmpeg). One-time per agent.
STEP 2 — Install Chromium system dependencies Chromium needs system-level libraries. Install as root:
ssh hermes-vps "docker exec -e PLAYWRIGHT_BROWSERS_PATH=/opt/data/ms-playwright hermes-agent-jemg-hermes-agent-1 /opt/hermes/.venv/bin/python3 -m playwright install-deps chromium"
⚠️ DO NOT SKIP — Chromium will crash on image-heavy pages without this.
STEP 3 — Deploy the browser_task.py script Save to /opt/data/scripts/browser_task.py (owned by hermes:hermes). Full script:
#!/usr/bin/env python3 import os, sys, asyncio os.environ["PLAYWRIGHT_BROWSERS_PATH"] = "/opt/data/ms-playwright" os.environ["ANONYMIZED_TELEMETRY"] = "false" from browser_use import Agent, Browser from browser_use.browser.profile import BrowserProfile from browser_use.llm import ChatAnthropic
CHROME_PROFILE = "/opt/data/chrome-profile" SCREENSHOT_DIR = "/opt/data/listings/_screenshots" os.makedirs(CHROME_PROFILE, exist_ok=True) os.makedirs(SCREENSHOT_DIR, exist_ok=True)
def load_env_key(key_name): value = os.getenv(key_name) if value: return value with open("/opt/data/.env") as f: for line in f: if line.startswith(f"{key_name}="): return line.split("=", 1)[1].strip() raise RuntimeError(f"{key_name} not found")
async def run_task(task): proxy = os.getenv("BROWSER_PROXY") profile = BrowserProfile( headless=True, user_data_dir=CHROME_PROFILE, args=["--no-sandbox","--disable-blink-features=AutomationControlled", "--disable-dev-shm-usage","--disable-gpu"], proxy={"server": proxy} if proxy else None, ) browser = Browser(browser_profile=profile) llm = ChatAnthropic( api_key=load_env_key("ANTHROPIC_API_KEY"), model="claude-sonnet-4-6", ) agent = Agent(task=task, llm=llm, browser=browser) result = await agent.run() await browser.close() return result
if __name__ == "__main__": task = " ".join(sys.argv[1:]) if len(sys.argv) > 1 else "Go to example.com and return the page title" asyncio.run(run_task(task))
STEP 4 — Add ANTHROPIC_API_KEY to the container's .env The browser agent uses Claude Sonnet as its reasoning LLM. Key must be inside container:
ssh hermes-vps "docker exec hermes-agent-jemg-hermes-agent-1 bash -lc 'echo \"ANTHROPIC_API_KEY=YOUR_KEY_HERE\" >> /opt/data/.env'"
WHY CLAUDE SONNET? Haiku 4.5 returns thinking format browser-use can't parse. Gemini free tier exhausts quickly. Groq Llama lacks json_schema support. Sonnet is the safe default.
STEP 5 — Smoke test ssh hermes-vps "docker exec -u hermes -e PLAYWRIGHT_BROWSERS_PATH=/opt/data/ms-playwright hermes-agent-jemg-hermes-agent-1 bash -lc '/opt/hermes/.venv/bin/python3 /opt/data/scripts/browser_task.py \"Go to example.com and return the page title\"'"
Expected: [browser_task] Result: ... Example Domain ...
# Simple task: /opt/hermes/.venv/bin/python3 /opt/data/scripts/browser_task.py "go to dallas.craigslist.org and find the first 3 massage table listings with prices"
# With proxy: BROWSER_PROXY=http://user:pass@host:port /opt/hermes/.venv/bin/python3 /opt/data/scripts/browser_task.py "post this listing on Craigslist"
ALWAYS use the full python path: /opt/hermes/.venv/bin/python3 The system Python is PEP 668 locked. The full path always works.
The Chrome profile at /opt/data/chrome-profile stores cookies/sessions between tasks.
To log in for the first time: give the agent a task like "Go to craigslist.org, log in with email X and password Y, confirm you're logged in." After that, all future tasks use the saved session.
The VPS IP (2.24.96.191 on Hostinger) is a datacenter IP. Most sites don't care.
✅ Works without proxy:
❌ Blocked without proxy:
Only add a proxy when the agent hits a site that blocks it.
❌ ISP Dedicated Proxy (avoid): Single static IP, cheap (~$5-10/mo), still fingerprintable. Won't reliably bypass Facebook/LinkedIn.
✅ Rotating Residential Proxy (if ever needed): Real home IPs that rotate per request. Charged per GB (~$3-10/GB). Actually bypasses PerimeterX and Akamai. Buy pay-per-GB, not monthly.
To activate for a specific task, set the BROWSER_PROXY env var. No code changes needed.
Beyond the VPS-side Bilateral Broker, King's workstation runs a second agent-to-agent channel: a lightweight daemon (scripts/agent_bridge.py) that lets the two local AI engineers — Claude Code (in VS Code) and OpenCode — talk and collaborate automatically, with NO human relaying messages between them. Proven live June 18, 2026.
How a message flows:
Why it's reliable: • Runaway loops — [EOT] terminator + 30 wakes/hour rate cap + instant kill switch (create the bridge_STOP file). • Duplicate daemons — singleton PID lock; only one ever runs. • Survives reboots — auto-starts at every Windows logon (launcher in the Startup folder). Nothing to re-establish. • Windows file protection — all files live in C:\Users\<user>\.hermes-bridge\, outside Controlled Folder Access (no "blocked" popups, no failed writes).
Files: C:\Users\12149\.hermes-bridge\ holds the two channel files, the log (agent_bridge.log), and the kill switch (bridge_STOP). Daemon script lives at Hermes Agents\scripts\agent_bridge.py.
KEY DIFFERENCE FROM THE VPS BILATERAL BROKER: the Broker connects agents on the VPS via a persistent SQLite server; the Agent Bridge connects the two AI engineers on the local workstation via shared files + on-demand wakes. Together they let every agent in the Hermes system reach the others.
The autonomous Claude Code <-> OpenCode bridge (built Session 64) was finished and elevated on three fronts:
Channel files / log / kill switch live in C:\Users\<user>\.hermes-bridge\. Safety rails (kill switch, 30 wakes/hr rate cap, singleton PID lock, EOF offsets) remain in force throughout.
A sustained 20-message stress test between the two local AI engineers (Claude Code and OpenCode) revealed that the Code↔OpenCode bridge, while reliable for single messages, had a hidden CLASS of bug: under bursts it could silently lose messages. Working as a team, the two agents traced three separate symptoms to ONE root cause — the daemon advanced its read position past a message before that message was actually delivered — and closed the whole class.
THE THREE BUGS (one root cause):
HOW THE TEAM WORKED:
VERIFICATION:
CURRENT STATE: The Code↔OpenCode channel is solid under any burst, terminator, or rate-cap scenario. Code on disk = committed = running = on GitHub. No pending actions.
Folded in from the HTML walkthrough edition on 2026-06-23. PART 0–5 above are the technical reference; this part is the gentle, video-style click-by-click setup (originally "How to Build a Hermes AI Agent — Complete Setup Guide", based on Nate Herk's video). Some overlap with the reference sections is intentional — a beginner can follow this start-to-finish, then use PART 0–5 for depth. PART 0 §0.2 remains the authoritative current-standards section.
CFM Business • Staff Training Document
A plain-English manual for staff who set up AI agents for clients — from zero to a working, self-improving assistant
Version2.0 — Illustrated Edition
Based OnNate Herk's official tutorial video + CFM field experience
UpdatedJune 2026
AudienceStaff setting up Hermes agents for clients
⚠️ CURRENT REALITY OVERRIDE — 2026-06-23
This guide has some stale spots. These are the current CFM standards and WIN over any contradiction below:
minimax provider (api.minimax.io/v1), no fallback chain; vision auxiliary (provider=auto, model=auto, not pinned). NOT gemini / OpenRouter / DeepSeek / Sonnet.canon_shim (canon stack + holographic, cap 60K/40K). Honcho removed./opt/data/scripts/scraper-stack/scrape.py), free-first tiered.Authoritative detail lives in empire-site/MANUAL.txt PART 0 §0.2 (the synced master).
Section 1
Hermes is an open-source AI agent built by Nous Research. Think of it as an AI employee that lives on your client's server, works 24/7 even when your client's computer is off, and gets smarter over time by learning from every task it does.
Screenshot from Nate Herk's tutorial — What Hermes Agent Is
Key facts about Hermes:
Hermes vs. Claude Code vs. OpenClaw — from Nate Herk's video
Tool What It Is Who Drives It Best For
Claude Code Anthropic's coding assistant — lives in your computer's terminal You — you have to be there Writing code, building skills, setting things up. That's us — we use Claude Code to set Hermes up for clients.
OpenClaw Another autonomous AI agent — 355,000+ users, backed by OpenAI Runs on its own on a server Same things as Hermes — some people run both at the same time
Hermes ✅ Autonomous AI agent from Nous Research — lighter, faster, self-improving Runs on its own, 24/7 The agent you set up for clients. Works without human oversight after training.
Plain English
Think of Claude Code (us) as the contractor who builds the house. Think of Hermes as the house itself — once we build it and move the client in, it runs on its own. We don't need to be there for Hermes to keep working.
Section 2
Before you set anything up, you need to understand how Hermes works internally. Hermes is built on five pillars. Each one has a job.
The five core topics covered in Nate Herk's full tutorial
How Memory works inside Hermes — loaded fresh at every session start
Memory is the small collection of facts Hermes carries from one conversation to the next. It is not a log of every message — it's more like a personal file that holds the important stuff.
Who the client is — their name, preferences, communication style, things they don't like. Example: "Darrick prefers short replies. He runs a real estate business in Texas."
Facts about the client's environment, projects, and tools. Example: "GitHub repo is lisa-hermes-backup. Telegram bot is Lisa The AI Queen."
Definition: Memory
The small set of important facts Hermes remembers between conversations. It's like an employee's personal notepad — they jot down the things that matter, not every word that was said in every meeting.
Beginner Mistake to Avoid
Do NOT store secrets (passwords, API keys) in Memory. Memory is readable and searchable. Secrets always go in the .env file.
Skills are reusable playbooks — Hermes builds them and reuses them automatically
A Skill is a saved set of instructions for how to do a task well. The first time Hermes does something complex, it figures it out step by step. Then it saves those steps as a skill so it can do it instantly next time.
Definition: Skill
A Skill is like a recipe card. The first time you make a dish, you experiment. Once you know the recipe, you write it down. Next time, you skip the guesswork. Hermes does the same thing — it writes its own recipe cards and reuses them.
There are two ways to give Hermes a new skill:
SOUL.md shapes how Hermes talks — its tone, personality, and communication style
SOUL.md controls the agent's personality. It's what makes Hermes sound like a specific assistant instead of a generic chatbot. You can make Hermes formal, casual, concise, warm, or anything the client wants.
Definition: Soul
Soul is the agent's personality file. If Memory is what Hermes knows about the client, Soul is who Hermes is. Same brain, different character. Like how the same person can be formal at work and relaxed at home.
Crons turn Hermes from a reactive chatbot into a proactive, scheduled worker
A Cron is a scheduled task — something Hermes does automatically at a set time without anyone asking. Example: "Every morning at 6am, send me a summary of my emails." That's a cron job.
Definition: Cron
A Cron is an alarm clock attached to a task. You set it once, and Hermes does that task every day (or every hour, or every week) without being asked again. The word "cron" comes from the Linux scheduling system — just think of it as "automatic task."
The self-improving loop: do → save → skill → search → get better over time
This is what makes Hermes different from a regular chatbot. Every time Hermes does something well, it can save that knowledge permanently. Over time, it becomes more capable and faster for that specific client.
The loop: Do the work → Save useful facts to Memory → Turn repeatable steps into Skills → Search past sessions when needed → Better next run.
"This isn't a tool you finish setting up. It's a teammate you keep training." — Nate Herk
Section 3
Before you touch Hostinger, make sure you have the following ready. This is your pre-flight checklist.
About the AI Brain (LLM)
Hermes needs an AI model to think with. The CFM standard is MiniMax-M3 via the minimax provider (api.minimax.io), with no fallback chain — see PART 0 §0.2. (Older videos/walkthroughs show OpenAI Codex, Gemini, DeepSeek, or OpenRouter; those are NOT used at CFM.)
Step 1
A VPS is the server that will run the agent 24/7. Think of it as renting a small computer in a data center that never turns off. The agent lives there — not on your laptop, not on the client's laptop. It's always running in the cloud.
Definition: VPS (Virtual Private Server)
A VPS is a computer you rent in a data center. It runs Linux (Ubuntu), has its own IP address, and stays on 24/7 even when your laptop is closed. This is where Hermes lives. You control it by typing commands through SSH.
1
Hostinger has a dedicated Hermes Agent landing page. Search for "Hostinger Hermes Agent" or go to hostinger.com and click "1-Click OpenClaw" at the top (Hermes is listed under their VPS applications).
The Hostinger page for Hermes Agent — showing the KVM 2 plan
Which Plan to Choose?
Select the KVM 2 plan. It gives you: 2 vCPU cores, 8 GB RAM, 100 GB NVMe storage, 8 TB bandwidth. This is the minimum recommended for a well-functioning Hermes agent. Do NOT go with the cheapest plan — it won't have enough memory.
Step 2
The checkout screen — KVM 2 plan, 12 months, ~$119.88/year total
1
12 months comes out to ~$9.99/month. This saves money vs paying month-to-month. Hostinger also offers to switch to 24 months for the biggest savings — your call.
2
Hostinger will offer "Daily Auto-Backup" for $6/mo extra. You don't need this — we set up GitHub as our backup system in Step 10. Skip it.
3
Pick the closest US location to your client. Boston or Dallas are both good. This affects how fast the server responds.
4
After payment, Hostinger will ask you to set up your server. You'll continue to the "Choose what to install" screen next.
Step 3
The "Choose what to install" screen — select Docker application, then "Hermes Agent"
Hostinger gives you a choice of how to install Hermes. There are two methods. Always choose Docker.
Nate's breakdown of the two installation methods
Side-by-side comparison: Direct install vs. Docker container
You SSH into the server and manually run the Hermes installer. Files go directly onto the server. Harder to update, harder to manage, riskier if something breaks.
Hermes runs inside an isolated container. The container and the server share one folder (/opt/data/), which means your settings survive even if the container is rebuilt. Easier to restart, update, and manage.
Definition: Docker Container
A Docker container is like a small self-contained apartment inside your server building. Hermes lives in the apartment. The shared folder (/opt/data/) is like a shared storage unit that both the apartment and the building can access. If you tear down the apartment (delete the container), the storage unit (your data) is still safe.
1
On the installation screen, you'll see tabs: Plain OS, Docker application, Control panel, Application. Click "Docker application."
2
You'll see two options: Hermes Agent and Hermes Workspace. Click Hermes Agent.
3
Hostinger will prompt you to set your admin credentials.
The Deploy Hermes Agent dialog — set a username and password here
4
Use: ADMIN_USERNAME: hermes and a strong password. Write these down immediately in the client's .env file in Claude Code before clicking Deploy.
Write It Down First
Before clicking Deploy, open the client's .env file in Claude Code and record the admin username and password. If you forget them, you'll have to SSH into the server and dig through config files to recover them.
Step 4
After Hostinger finishes deploying (takes 3–5 minutes), you'll see the server dashboard. This is your control panel for the VPS.
The Hostinger VPS dashboard after deployment — note the left sidebar with all options
Key sections in the left sidebar:
Docker Manager — your Hermes container shows up here with status "Created"
Find Your IP Address
Your server's IP address is shown on the Overview page. Write it down in the client's CLAUDE.md file immediately. You'll need this IP every time you SSH in.
Step 5
Definition: SSH
SSH stands for "Secure Shell." It's a way to remotely control your server by typing commands. Think of it like texting your server: you type a command, the server does it and types back the result. There's no clicking — just text.
What a successful SSH login looks like — Ubuntu 24.04 LTS welcome screen with a command prompt
1
On Windows: Open PowerShell (search "PowerShell" in the Start menu). On Mac: Open Terminal.
2
Replace the IP address with your client's actual server IP:
ssh root@2.24.96.191You'll be asked "Are you sure you want to continue connecting?" — type yes and press Enter.
3
You'll see a "password:" prompt. Type the root password (from the client's .env file). Nothing will appear on screen as you type — this is normal and expected. The password is hidden for security. Just type it and press Enter.
Password Looks Invisible — That's Normal
When you type your password in SSH, the cursor does not move and no characters appear. This is intentional — the terminal hides password input for security. Trust that you're typing correctly and press Enter when done.
4
A successful login shows the Ubuntu welcome message and ends with a prompt like root@yt-hermes:~#. You are now in control of the server.
Step 6
Once you're inside the Docker container, you run the Hermes setup wizard. This is an interactive menu that walks you through configuring the agent.
1
From your SSH session, type:
docker exec -it hermes-agent-jemg-hermes-agent-1 bash(Your container name may be slightly different — check the Docker Manager in Hostinger if this doesn't work.)
2
hermes setupThe Hermes setup wizard — choose "Quick setup" for first-time installation
3
Use the arrow keys to navigate. Press Enter or Space to select. The Quick setup covers: AI provider, model, and messaging. That's all you need to get started. You can run Full setup later.
Step 7
Hermes needs an AI model to think with. During setup you choose your "inference provider" — the AI that powers the agent's responses.
Definition: LLM (Large Language Model)
An LLM is the AI "brain" Hermes uses to understand questions and generate responses. Without an LLM, Hermes is a body with no brain — it can't think.
CFM standard: MiniMax-M3. For CFM agents the model is MiniMax-M3 via the minimax custom provider (base_url https://api.minimax.io/v1), with no fallback chain. Vision is auxiliary (provider=auto, model=auto — not pinned to a model name). Do NOT use OpenRouter, Google Gemini, DeepSeek, Mistral, or direct-Anthropic as the agent's primary model — those are retired. (Full authoritative config: PART 0 §0.2 of this manual.)
1
Create an account at api.minimax.io, generate an API key, and copy it.
2
minimax provider in the setup menuIn the Hermes setup menu choose the minimax custom provider and set its base_url to https://api.minimax.io/v1. Press Enter.
3
Paste the API key when prompted — setup saves it to /opt/data/.env. Then set model.default = MiniMax-M3 (no fallback chain).
Step 8
Telegram is how the client will communicate with Hermes after setup. Hermes uses a Telegram bot — a special account that Hermes controls. You create this bot using Telegram's BotFather.
Definition: Telegram Bot
A Telegram bot is a special Telegram account controlled by a computer (not a human). When the client sends a message to the bot, Hermes reads it and responds. The client just sees a chat conversation — they don't need to know anything technical.
Definition: BotFather
BotFather is Telegram's official bot creator. You message BotFather (on Telegram) and it creates your new bot and gives you a secret token. That token is what connects Hermes to the bot.
Creating a bot with BotFather — it gives you a token after naming the bot
1
Find the official BotFather account (it has a blue checkmark). Click Start.
2
Type /newbot and press send. BotFather will ask for a name, then a username.
3
Name: Can be anything. Example: "Lisa Hermes" or "CFM Assistant."
Username: Must end in "bot" and be unique across all of Telegram. Example: LisaHermesBot or CFMAgentBot. If it's taken, add numbers or words until it works.
4
BotFather will give you a token that looks like: 8935721190:. Copy this immediately and save it to the client's .env file. You only see it once in this message.
5
Search for @userinfobot on Telegram. Send it any message. It will reply with your User ID (a number like 8705992194). Save this too.
6
Back in your SSH terminal, the Hermes setup wizard will ask for the bot token and your user ID. Paste them in.
Setup complete — Hermes confirms Telegram is connected and shows tool availability
Step 9
1
Inside the container, type:
hermes chatYou should see the Hermes ASCII logo and a startup screen listing available tools and skills.
Hermes v0.13.0 loaded and running — showing all available tools and skills
2
Hermes should respond with a greeting. If it does, the CLI is working.
3
Open a second terminal window (or use a screen session) and run:
hermes gatewayThis starts the background process that connects Hermes to Telegram. Keep this running.
4
Open Telegram, find your bot (by the username you created), and send it "hello." Hermes should respond within a few seconds.
If Telegram Doesn't Respond
Check that the gateway is running (Step 3). If it stopped, restart it with hermes gateway. Also check the bot token is correct in /opt/data/.env. Open it with: nano /opt/data/.env
Step 10
GitHub is a website where developers store code. We use it as a nightly backup for the agent's memory, skills, and settings. If the server ever crashes, everything important is backed up to GitHub.
Definition: GitHub
GitHub is a website that stores files and tracks changes over time. Think of it like Google Drive but for code. We use it to automatically save a backup of the agent's brain every night so nothing is lost if the server breaks.
Hermes asking for GitHub credentials to set up the backup repository
1
Go to github.com → click "+" → "New repository." Name it something like lisa-hermes-backup. Set it to Private. Do NOT add a README. Click "Create repository."
2
Go to: GitHub → Your profile picture → Settings → Developer settings → Personal access tokens → Tokens (classic) → Generate new token (classic)
GitHub Personal Access Tokens page — click "Generate new token"
Token scopes — check the full "repo" checkbox (gives access to create and push to private repos)
3
Note: Give it a name like "hermes-backup"
Expiration: 90 days or "No expiration" (if you set an expiration, you'll need to rotate it)
Scopes: Check the "repo" box — this is the only scope Hermes needs for backup.
4
After clicking "Generate token," GitHub shows it only once. Copy it right away and paste it into the client's .env file. Then save it to the server:
hermes config set GITHUB_TOKEN your_token_here5
In Telegram or the CLI, tell Hermes:
"Set up a nightly GitHub sync cron to back up my skills, memories, and soul file to my private repo."
Hermes will create the cron job and configure it to run at midnight Central time.
Step 11
API keys are passwords that let Hermes connect to external services like email, Slack, Google Calendar, and more. Each service requires its own key.
Definition: API Key
An API key is a unique password that lets two software systems talk to each other. When Hermes needs to check a client's Gmail, it uses a Gmail API key to prove it's allowed in. Without the key, Gmail won't let Hermes access anything.
Security Rule — Always Follow This
NEVER type API keys directly in a chat message (Telegram, Claude Code chat, or anywhere else). Keys in chat logs can be stolen. Always use one of the two methods below.
Inside the container terminal, run:
hermes config set KEY_NAME your_key_valueExample: hermes config set MINIMAX_API_KEY your-minimax-key...
Open the .env file on the server:
nano /opt/data/.envAdd a new line: KEY_NAME=your_key_value. Press Ctrl+X, then Y, then Enter to save.
After Adding Keys
Restart the container after adding keys so Hermes picks them up: docker restart hermes-agent-jemg-hermes-agent-1
Service Key Name Where to Get It
MiniMax (primary LLM) MINIMAX_API_KEY api.minimax.io → API keys
Groq (speech-to-text) GROQ_API_KEY console.groq.com → API keys
Telegram Bot TELEGRAM_BOT_TOKEN From BotFather in Step 8
GitHub Backup GITHUB_TOKEN From Step 10
Step 12
Hermes is now running. Now you introduce it to the client's world. This is where the real customization happens.
1
Open Telegram and send your new bot a voice note or message describing who the client is, what they do, and what they want the agent to help with. Example:
"Hey, I'm [Client Name]. I run a real estate business in Texas. My biggest needs are: summarizing my emails every morning, tracking leads in my spreadsheet, and posting to social media 3 times a week."
Hermes will listen, ask follow-up questions, and save this to USER.md automatically.
2
Tell Hermes how to communicate. Example: "Always be concise. Use bullet points. Don't start messages with 'Certainly!' Be direct and professional."
Hermes will update its SOUL.md file.
3
Ask Hermes to schedule something recurring. Example: "Every weekday morning at 7am Central, send me a summary of what's on my agenda today."
Hermes will create a cron job automatically.
4
Browse the Skills Hub and install skills that match the client's needs. Instructions in the next section.
Training Takes Time
Hermes gets better the more you interact with it. The first week is about teaching it — correcting mistakes, telling it your preferences, adding skills. By week 2 it should be running smoothly with minimal supervision.
Reference
The mental model for CLI vs Telegram from Nate's video
Detailed breakdown of when to use each interface
Important: Hermes is the SAME agent in both CLI and Telegram. Telegram does not give you a dumber version. What changes is the control surface, not the intelligence.
Mental Model CLI (Terminal) Telegram
Think of it as... The cockpit — full controls visible The remote control — simple buttons
Best for... Setup, debugging, writing skills, inspecting memory, approving sensitive commands Quick tasks, daily check-ins, voice messages, triggering crons, getting reports
Visibility Full: see every tool call, token usage, context window Limited: you see the conversation, not the internals
Who uses it You (the setup person) during installation and maintenance The client, every day
Reference
Skills Hub — Anthropic category showing official skills like pdf, pptx, skill-creator, internal-comms
Skills Hub at hermes-agent.nousresearch.com/docs/skills — 684 total skills available
The Skills Hub is a library of pre-built skills you can install into any Hermes agent. Browse by category or search for what you need.
Method 1 — From Telegram: Tell Hermes: "Install the [skill name] skill from the Skills Hub." Hermes will find, download, and install it.
Method 2 — From CLI:
hermes skills search email
hermes skills install [skill-url]Skill What It Does
google-workspace Connect to Gmail, Google Calendar, Google Drive
github-code-review Review code in GitHub repositories
pdf Read and generate PDF files
youtube Summarize YouTube videos and extract content
hermes-agent Meta skill — helps Hermes explain itself and troubleshoot
Reference
Multiple specialist agents on one VPS — each in its own Docker container
As a client grows, you can run multiple specialized Hermes agents on the same VPS. Each agent lives in its own Docker container — they don't interfere with each other.
Definition: Specialist Agent
Instead of one agent that does everything, you create separate agents for separate jobs. One agent handles marketing. One handles finances. One handles operations. Like having different employees for different departments instead of one person doing everything.
Decision flowchart — ask these questions to decide if a task needs its own agent
Simple rule: If the task needs its own memory, tools, API keys, schedule, or audience — give it its own agent. Otherwise, keep it in the main agent.
Recommended starting team — Personal Hermes as manager, 3-4 specialist agents
Good pattern (specialized agents) vs bad pattern (one mega-agent doing everything)
Starter Team for a Business Client
Personal Hermes (Manager) — main contact for the client, routes tasks
Marketing Hermes — content, brand, competitor research
Finance Hermes — invoices, budgets, payment tracking
Ops Hermes — VPS management, logs, server health
Reference
When you run multiple Hermes agents (or need Claude Code on your workstation to talk to a Hermes agent on a VPS), you need a reliable, persistent communication channel. The Bilateral Broker is that channel.
Definition: Bilateral Broker
A lightweight message server that runs on the VPS host (outside any container). Any agent, script, or tool can send and receive messages through it. Messages are stored in SQLite and persist across restarts, reboots, and container rebuilds. Think of it as a shared bulletin board that never forgets.
Method Problem
File polling (read/write .md files) Race conditions, lost messages, no delivery confirmation, stale reads
Cron-based relays Crons break, have minimum intervals, add complexity, and are a single point of failure
Direct HTTP APIs Works but doesn't persist — if the receiver is offline, the message is lost
Bilateral Broker Persists all messages in SQLite, survives crashes and reboots, both sides can connect anytime, zero race conditions
The broker runs on the VPS host (not inside any container) as a systemd service. This means:
1
Copy broker.py to the VPS host at /opt/data/scripts/broker.py:
scp broker.py root@YOUR_VPS_IP:/opt/data/scripts/broker.pyThe broker is a Python Flask app (~100 lines) that uses SQLite for storage. No external dependencies beyond Flask.
2
Create /etc/systemd/system/bilateral-broker.service on the VPS host:
[Unit]
Description=Bilateral Agent Message Broker
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/python3 /opt/data/scripts/broker.py
WorkingDirectory=/opt/data/scripts
Restart=always
RestartSec=5
[Install]
WantedBy=multi-user.target3
systemctl daemon-reload
systemctl enable bilateral-broker
systemctl start bilateral-brokerVerify it's running:
systemctl status bilateral-broker4
The broker listens on port 4097. Allow containers to reach it:
ufw allow from 172.16.1.0/24 to any port 4097
iptables -I DOCKER-USER -s 172.16.1.0/24 -p tcp --dport 4097 -j ACCEPTPin the iptables rule for boot persistence:
echo 'iptables -I DOCKER-USER -s 172.16.1.0/24 -p tcp --dport 4097 -j ACCEPT' >> /etc/rc.local5
For Claude Code (workstation side): Deploy opencode-broker to /usr/local/bin/ on the VPS host. It's a bash script that uses curl to talk to the broker.
For Hermes agents (inside containers): Deploy lisa-send (or rename for your agent) to /usr/local/bin/ inside the container. It's a Python script that connects to the broker via HTTP.
Endpoint Method Purpose
/send POST Send a message: {"to": "agent_name", "body": "message"}
/inbox/<agent_id> GET Read unread messages for an agent. Returns JSON array.
/ack/<id> POST Acknowledge (mark as read) a specific message by ID.
/status GET Broker health and message counts.
/health GET Simple liveness check. Returns {"status": "ok"}.
/messages GET Full message history (for debugging).
# Send a message to Lisa
opencode-broker send lisa "What's the status on the Ripplewood deal?"
# Check your inbox
opencode-broker inbox opencode# Send a message to OpenCode
python3 /usr/local/bin/lisa-send send opencode "Email sent to cfmbusiness@gmail.com"
# Check your inbox
python3 /usr/local/bin/lisa-send inboxWhat Survives What
Container restart: Broker unaffected (runs on host). Messages in SQLite preserved.
VPS reboot: Broker restarts automatically via systemd. Messages preserved.
Broker crash: systemd restarts it in 5 seconds. Messages preserved (SQLite write-ahead log).
Broker killed (SIGKILL): systemd restarts it in ~8 seconds. All queued messages intact.
Agent offline for days: Messages queue in SQLite. Agent reads them on next connection.
Critical Rule
The broker must run on the VPS host, not inside a container. If it runs inside a container, it dies when the container is rebuilt and takes all queued messages with it (unless the database is on a mounted volume). Host-side = permanent.
Reference
By default, a Hermes agent can send messages, run scripts, and call APIs — but it cannot open a browser window, fill out a form, or post an ad on Craigslist. This section explains how to give an agent full browser access so it can do real web tasks autonomously.
What "browser access" means
The agent can navigate any website, click buttons, fill forms, log into accounts, take screenshots, and extract data — all from inside its container on the VPS, with no human touching a mouse.
Three components work together:
ComponentWhat It DoesAlready Installed?
browser-useThe AI brain. Translates plain-English tasks ("post this ad on Craigslist") into browser actions. Handles clicks, forms, navigation.Yes — included in Hermes as a plugin at /opt/data/hermes-agent-research/plugins/browser/browser_use/
PlaywrightThe browser controller. browser-use uses this under the hood to actually drive Chrome.Yes — playwright v1.60.0 in the Hermes venv
Chromium binaryThe actual headless browser that runs on the VPS. NOT included by default — must be installed separately.No — you must install it (see below)
1
Run this from Code's side (requires root inside the container). Install it to /opt/data/ms-playwright so the agent user can read and write to it:
ssh hermes-vps "docker exec -e PLAYWRIGHT_BROWSERS_PATH=/opt/data/ms-playwright hermes-agent-jemg-hermes-agent-1 /opt/hermes/.venv/bin/python3 -m playwright install chromium"This downloads ~290 MB (Chromium + headless shell + FFmpeg). It only needs to be done once per agent.
2
Chromium needs system-level libraries (fonts, codecs, display drivers). Install them as root:
ssh hermes-vps "docker exec -e PLAYWRIGHT_BROWSERS_PATH=/opt/data/ms-playwright hermes-agent-jemg-hermes-agent-1 /opt/hermes/.venv/bin/python3 -m playwright install-deps chromium"Do not skip this step
Chromium will launch and navigate but crash on image-heavy pages if system deps are missing. The smoke test (example.com) passes without deps — real-world sites may not.
3
This is the entry point the agent calls for any browser task. Create it at /opt/data/scripts/browser_task.py. The script handles:
PLAYWRIGHT_BROWSERS_PATH automatically/opt/data/.env/opt/data/chrome-profile (logins survive between tasks)BROWSER_PROXY env var#!/usr/bin/env python3
import os, sys, asyncio
os.environ["PLAYWRIGHT_BROWSERS_PATH"] = "/opt/data/ms-playwright"
os.environ["ANONYMIZED_TELEMETRY"] = "false"
from browser_use import Agent, Browser
from browser_use.browser.profile import BrowserProfile
from browser_use.llm import ChatAnthropic
CHROME_PROFILE = "/opt/data/chrome-profile"
SCREENSHOT_DIR = "/opt/data/listings/_screenshots"
os.makedirs(CHROME_PROFILE, exist_ok=True)
os.makedirs(SCREENSHOT_DIR, exist_ok=True)
def load_env_key(key_name):
value = os.getenv(key_name)
if value: return value
with open("/opt/data/.env") as f:
for line in f:
if line.startswith(f"{key_name}="):
return line.split("=", 1)[1].strip()
raise RuntimeError(f"{key_name} not found")
async def run_task(task):
proxy = os.getenv("BROWSER_PROXY")
profile = BrowserProfile(
headless=True,
user_data_dir=CHROME_PROFILE,
args=["--no-sandbox","--disable-blink-features=AutomationControlled",
"--disable-dev-shm-usage","--disable-gpu"],
proxy={"server": proxy} if proxy else None,
)
browser = Browser(browser_profile=profile)
llm = ChatAnthropic(
api_key=load_env_key("ANTHROPIC_API_KEY"),
model="claude-sonnet-4-6",
)
agent = Agent(task=task, llm=llm, browser=browser)
result = await agent.run()
await browser.close()
return result
if __name__ == "__main__":
task = " ".join(sys.argv[1:]) if len(sys.argv) > 1 else "Go to example.com and return the page title"
asyncio.run(run_task(task))4
The browser agent uses Claude Sonnet as its reasoning engine. The Anthropic API key must be present inside the container, not just on the workstation:
ssh hermes-vps "docker exec hermes-agent-jemg-hermes-agent-1 bash -lc 'echo \"ANTHROPIC_API_KEY=YOUR_KEY_HERE\" >> /opt/data/.env'"Why Claude Sonnet?
browser-use requires a model that supports its structured output format. Claude Sonnet works reliably. Other models tested: Haiku 4.5 (returns thinking format browser-use can't parse), Gemini Flash (free tier quota exhausted quickly), Groq Llama (no json_schema support on most models). Sonnet is the safe default.
5
Verify the full chain works end to end:
ssh hermes-vps "docker exec -u hermes -e PLAYWRIGHT_BROWSERS_PATH=/opt/data/ms-playwright hermes-agent-jemg-hermes-agent-1 bash -lc '/opt/hermes/.venv/bin/python3 /opt/data/scripts/browser_task.py \"Go to example.com and return the page title\"'"Expected output: [browser_task] Result: ... Example Domain ...
Once installed, the agent can run any browser task by calling the script directly:
# From inside the container (agent's perspective):
/opt/hermes/.venv/bin/python3 /opt/data/scripts/browser_task.py "go to dallas.craigslist.org and find the first 3 massage table listings with prices"
# With a proxy (only if one is ever configured):
BROWSER_PROXY=http://user:pass@host:port /opt/hermes/.venv/bin/python3 /opt/data/scripts/browser_task.py "post this listing on Craigslist"Always use the full python path
Use /opt/hermes/.venv/bin/python3 — not just python3. The system Python is PEP 668 locked and the venv's pip symlink may be broken. The full path always works.
The Chrome profile at /opt/data/chrome-profile stores cookies and session data between tasks. This means:
To log into a site for the first time, give the agent a task like: "Go to craigslist.org, log in with email X and password Y, then confirm you are logged in." After that, all future tasks will use the saved session.
The VPS IP (2.24.96.191 on Hostinger) is a datacenter IP. Most sites don't care. A few do.
Site / TaskWorks Without Proxy?Notes
Craigslist (browse + post)✅ YesConfirmed working in testing
eBay (browse)✅ YesLight bot detection
OfferUp (browse)⚠️ ProbablyAkamai detection — test first
Facebook Marketplace❌ NoPerimeterX blocks datacenter IPs
LinkedIn❌ NoBlocks cloud IPs aggressively
Public research / scraping✅ YesWorks on virtually all public sites
Only add a proxy when the agent hits a site that blocks it. Don't pay for one proactively.
A single static IP that looks like a home connection. Cheaper (~$5-10/mo) but still fingerprintable. Won't reliably bypass Facebook/LinkedIn.
Real home IP addresses that rotate per request. Charged per GB (~$3-10/GB). Actually bypasses PerimeterX and Akamai. Buy pay-per-GB, not a monthly plan.
To activate the proxy for a specific task, set the BROWSER_PROXY env var when calling the script. No code changes needed.
df -h /opt/data).Reference
Nate's guide to maintaining Hermes over time
Situation What to Do
Agent gets something wrong Correct it on the spot. Tell it to update its skill or memory for that topic.
You gave the same instruction twice Tell Hermes: "Write a skill for this so you remember next time."
Agent is too wordy or off-tone Edit SOUL.md. Tell Hermes: "Update your soul file to be more concise."
Want something scheduled Build the skill first, then ask Hermes to create a cron for it.
Something breaks / weird behavior Check MEMORY.md first. Stale memory is the #1 cause of weird agent behavior. Run hermes doctor.
Agent won't start Check the container: docker ps. Restart it: docker restart [container-name].
Reference
docker ps -a
docker start hermes-agent-jemg-hermes-agent-1Make sure the gateway is running:
hermes gatewayIf it crashes, restart it. Check that the TELEGRAM_BOT_TOKEN in /opt/data/.env is correct.
Memory may be stale or corrupted. Open /opt/data/memories/MEMORY.md and review. Remove outdated entries. Tell Hermes: "Re-read my memory file."
You're either on the host server (not inside the container) or using the wrong path. The .env inside the container is at /opt/data/.env. Make sure you ran docker exec -it [container] bash first.
Check the password in the client's .env file. Make sure you're typing it correctly (remember: nothing shows on screen as you type). If it's truly wrong, you can reset it from the Hostinger dashboard → OS & Panel → Reset root password.
hermes doctorBonus
Beyond the VPS-side Bilateral Broker, King's workstation runs a second agent-to-agent channel: a lightweight daemon that lets the two local AI engineers — Claude Code (in VS Code) and OpenCode — talk and collaborate automatically, with no human relaying messages between them. Proven live June 18, 2026.
Definition: Agent Bridge Daemon
A small Python watcher (scripts/agent_bridge.py) that monitors two shared text files. When one agent appends a message, the daemon launches a fresh session of the other agent to read it and reply — then that agent goes back to sleep. The daemon is the "doorbell"; each agent reads the shared history itself.
code_to_opencode.md (or OpenCode appends to opencode_to_code.md).opencode run / claude -p).[EOT] tag.ConcernHow it's handled
Runaway loops[EOT] terminator + per-direction wakes/hour rate cap (hold-and-retry, never drops) + kill switch (bridge_STOP file)
Duplicate daemonsSingleton PID lock — only one ever runs
Survives rebootsAuto-starts at every Windows logon (Startup folder). Reboot-tested & verified June 18, 2026 — after a real reboot the daemon relaunched itself ~3 minutes after logon (fresh START log line, new PID), with no admin and no manual step.
Windows file protectionAll files in C:\Users\<user>\.hermes-bridge\, outside Controlled Folder Access
The bridge was upgraded from a comms-only channel to a true collaboration channel. The daemon already woke each agent inside the repo with full permissions, so the change was in the wake instructions: a message may now be a work request. The woken agent reads the request, edits repo files, runs commands and tests, verifies the result, and reports back through the channel.
git commit or git push unless explicitly told (changes are left in the working tree for King to review); it refuses unsafe or out-of-scope requests; [EOT] still closes the exchange.King can now run the whole bilateral channel by simply talking to Claude Code in the VS Code chat — he never touches the terminal. A safety guardrail blocks the live Claude Code chat session from waking OpenCode on its own, which previously forced King to paste a command to start each exchange. The fix is a single narrow, append-only helper plus one permission rule:
PieceWhat it is
scripts/to_opencode.shA one-purpose command: it appends one message to the Code→OpenCode channel (which wakes OpenCode). It can do nothing else.
Permission ruleBash(bash scripts/to_opencode.sh:*) in .claude/settings.local.json — authorizes only that command. King must add this by hand; an AI cannot edit its own permissions (a hard self-modification guardrail).
Everyday workflow: King tells Claude Code, in plain words, what he wants OpenCode to do → Claude Code relays it → OpenCode does the work → Claude Code reads OpenCode's reply and reports it back in chat. Verified live ("Round-trip verified" from OpenCode).
Key difference from the VPS Bilateral Broker: the Broker connects agents on the VPS via a persistent SQLite server; the Agent Bridge connects the two AI engineers on the local workstation via files plus on-demand wakes. Together they let every agent in the system reach the others.
A sustained 20-message stress test between the two agents revealed that the bridge, while reliable for single messages, had a hidden class of bug: under bursts it could silently lose messages. Working as a team, Claude Code and OpenCode traced three separate symptoms to one root cause — the daemon advanced its read position past a message before that message was actually delivered — and closed the whole class.
BugSymptomFix
Burst loss10 messages sent in a burst, only 1 delivered — 9 silently dropped, no error, no logpump() "peek-then-commit": the read position advances only after a message is delivered
[EOT] chunk-wideA burst containing a closing [EOT] threw away the real messages batched with it[EOT] detection is now line-based — only the terminator line is dropped, real messages still delivered
Rate-cap drop + starvationA message that tripped the hourly cap was discarded; one busy direction could starve the otherPer-direction caps + hold-and-retry — a capped message is held and retried, never dropped
How the team worked: the agents divided the fixes (OpenCode took the [EOT] bug, Claude Code took the rate-cap and burst-loss bugs), peer-reviewed each other's code (5 of 5 checks passed), and — by reading the actual code rather than trusting each other's claims — caught a suspected fourth bug that turned out not to be real. An over-engineered "edit-lock" module was built mid-session and then shelved by mutual agreement, kept out of the daemon's critical path. Verified by deterministic proof scripts and two clean live test runs (a 2-message burst and a 5-message exchange, zero drops), then committed to git (the daemon had previously been untracked) and pushed to GitHub.
The Bilateral Broker and the Workstation Agent Bridge (above) are the two transports. The Trilateral Agent Communication System is what ties them together: it lets all three AI engineers — Claude Code (workstation), OpenCode (workstation), and Lisa (VPS container) — talk to each other autonomously, with no human relaying messages. Proven live June 19, 2026, with real message IDs.
⚠️ SUPERSEDED IN PART, 2026-06-30/07-01 (reconciled 2026-07-04). Everything below this point describes the June 19 v1 design accurately for what it covers, but it predates two major additions: (1) Kim joined the mesh with her own push receiver, not just poll, and (2) a trigger.dev cloud durable-delivery spine was added as a second, parallel path for Lisa. See "The Trigger.dev Durable Delivery Spine" further down for the current state. The June 19 poll-backstop design below is NOT dead — it's still the mechanism for Code and OpenCode, and still runs as Kim's backstop pending a hardening step (see that section for exactly what's still open).
The core idea — "never lose a message": durable delivery is kept separate from best-effort notification.
delivered=0 until the recipient acknowledges it. Nothing is ever silently lost.LegTransportInbound backstop
Claude Code ↔ OpenCodeLocal file bridge (~/.hermes-bridge/, agent_bridge.py) with headless wakesThe daemon itself; both agents share the file history
Claude Code ↔ LisaBilateral Broker (SQLite, port 4097)code_broker_poller.py — polls every 60s over SSH. Code is poll-only inbound because the workstation sits behind home NAT, so the VPS physically cannot push to it (this is physics, not a bug).
OpenCode ↔ LisaBilateral Broker (SQLite, port 4097)Push to port 4098 + opencode_poll_backstop.py (a 3-minute VPS cron) that drains any push-misses
Broker → LisaPush to port 8643 (/broker-push, IP 172.16.1.2)A silent lisa_ferry.py host cron that re-delivers push-misses without Telegram spam or spawning an LLM session
Broker → Kim (NEW 2026-07-01)Push to port 8643 on Kim's own subnet (/broker-push, IP 172.16.2.2 — her container's real IP; NOT .1, which is the docker network gateway, a common mix-up)kim_inbox_poll.py via the inbox-wake cron (every 2 min, verified live 2026-07-04) — still running as a backstop; NOT yet retired (needs an on-startup inbox-drain hardening step first, see the trigger.dev section below)
ComponentWhereRole
bilateral-broker (systemd, :4097)VPS hostSQLite message backbone — never drops a message
agent_bridge.py (+ launcher)WorkstationCode ↔ OpenCode file bridge, headless wakes
code_broker_poller.py (60s)WorkstationCode's inbound poll backstop (NAT-safe)
opencode-listener (:4098) + opencode_poll_backstop.py (3-min cron)VPS hostOpenCode push receiver + push-miss backstop
/broker-push (:8643) + lisa_ferry.py (silent host cron)VPS container / hostLisa push receiver + silent push-miss ferry
kim_broker_push.py (:8643 on 172.16.2.2) + kim-broker-push.service (systemd, supervised by Hermes cron ensure-kim-broker-push every 5 min, verified live)VPS container / hostKim push receiver (stdlib http.server, no Flask) + inbox-wake poll backstop (every 2 min, not yet retired)
OpenCode runs on the deepseek-v4-flash model (about $0.14 / $0.28 per million tokens in/out). A typical wake is a fraction of a cent; idle costs $0 because polling is just lightweight SSH + SQLite reads. Realistically, pennies per day. Claude Code and Lisa use their own subscriptions for their wakes.
Mechanical jobs — delivery, health checks, backstops — run as silent host crons, never as Hermes/Telegram-reporting crons, and message delivery must never spawn an LLM session. Waking an agent to think is a separate, rare event (a real-time push or a request from King), not a recurring cron job.
# Code → Lisa
ssh hermes-vps "code-broker send lisa '<message>'"
# OpenCode → Lisa
opencode-broker send lisa "<message>"
# Code → OpenCode
bash scripts/to_opencode.sh "<message>"
# Check health
ssh hermes-vps "code-broker health"
# Kill switch (halts all waking)
create the file ~/.hermes-bridge/bridge_STOPA second, parallel delivery path added on top of the broker — not a replacement. King's mandate driving this: agents should talk autonomously with no polling, no crons wherever possible; the broker's poll backstops are the fallback of last resort, not the primary mechanism, for whichever agents have a real network path to receive a push.
How it works: the broker's /send handler (patched, fire_deliver() function, background thread) fires a POST to a trigger.dev cloud task on every message, in parallel with its own direct push:
POST https://api.trigger.dev/api/v1/tasks/deliver/trigger
Bearer <TRIGGER_PROD_SECRET_KEY> (from systemd drop-in /etc/systemd/system/bilateral-broker.service.d/trigger.conf)
Body: {"payload":{"to","from","body","msgId"},"options":{"idempotencyKey":"hermes-msg-<id>"}}Task source: C:\Users\12149\Documents\TRIGGER.DEV\src\trigger\deliver.ts — routes to the target agent's gateway via env vars (LISA_GATEWAY_URL/LISA_API_KEY/LISA_MODEL, stored in trigger.dev's OWN prod env store, separate from the VPS .env). Deploy: cd C:\Users\12149\Documents\TRIGGER.DEV && npx trigger.dev@4.4.6 deploy (PAT-authenticated as cfmbusiness@gmail.com; if it crashes with a transient esbuild panic, just retry).
⚠️ Kim was added to the wired set 2026-07-02 — "only Lisa" is now stale. Current live value (verified 2026-07-04 via grep WIRED_TRIGGER_AGENTS /opt/data/scripts/broker.py): WIRED_TRIGGER_AGENTS = {"lisa", "kim"}. Kim is NOT reached the same way as Lisa though — trigger.dev's cloud can't reach her docker-internal network directly, so her path routes through the broker's own narrow /kim-relay route instead of a direct gateway URL. Verified live end-to-end at the time (broker message #811 showed both push_ok and trigger_ok firing for Kim). Code/OpenCode still deliberately excluded — firing at agents without a reachable gateway clogged the trigger.dev dashboard with permanently-failing "Queued 8h+" runs during testing, which is exactly why the guard exists at all.
Current live project (verified 2026-07-04 via list_projects): proj_mptaxodjnlkwndeokikd (slug trilateral-communication-system-SUOH), org automate-your-biz-27b3 (id cmr1pqclg011mn60nf0jq8kwv). Confirmed actively firing today — broker.log shows trigger_ok: #1143 deliver fired (200) at 2026-07-04T16:14 UTC, a few hours before this reconciliation pass.
⚠️ A dead project ref exists — do not use it if it surfaces in old notes/chat history: proj_ctvfkhasuylklrazofls (org automate-your-biz-df44) was the ORIGINAL project, but King accidentally deleted the entire org on 2026-07-01 (~01:46 CT), taking the project, its prod key (tr_prod_GneHRVNNwrjqPEB0GeOf, now permanently 401), and its env vars with it. Full rebuild into the new org/project above took about 45 minutes. The same new org also holds two unrelated projects — "Automate Your Biz" (proj_jsjuamckjzwiegocgfzy, the actual AYB website, kept separate on purpose) and "Ivy Daily Brief" (proj_rpopdczloyrqzyqmogxp) — do not confuse either with the comms project.
Still open as of this reconciliation (2026-07-04) — do not assume done:
inbox-wake poll cron (every 2 min) is explicitly a backstop, not yet retired — needs an on-startup inbox-drain step in kim_broker_push.py first.deliver.ts's retry policy (maxAttempts: 20, backoff 2.7s→...→59.3s) was flagged as too aggressive during testing; reducing it is still a TODO.code_broker_poller.py, opencode_poll_backstop.py) are still the only inbound path for those two agents, or whether a NAT-safe subscriber was ever built, was NOT confirmed during this reconciliation — verify live before stating either way.TRIGGER_PROD_SECRET_KEY documented above — the extra ones were not identified in this pass (values are secret; don't cat them into a chat).gmail_trigger_ok entries flagged in the prior pass are the email layer documented in the next section, built 2026-07-02.Built the day after the trigger.dev rebuild, on top of the same mesh: agent-native email, not just cross-agent messaging.
Identities:
lisa@automateyourbiz.info (Hostinger, IMAP — Hostinger has no Pub/Sub, see poll task below)kimmarketingdirector@gmail.com (Gmail API, Pub/Sub-capable)CC-King enforcement (hard rule, not a suggestion): both agents send outgoing mail through dedicated helpers — /opt/data/scripts/send_as_lisa.py and /opt/data/scripts/send_as_kim.py — that hard-code a CC to cfmbusiness@gmail.com on every message. Backed by a shared always-cc-king skill (/opt/data/skills/always-cc-king/SKILL.md, present in both containers) — confirmed still present on both sides, 2026-07-04.
Multi-account Gmail watch: broker.py's GMAIL_ACCOUNTS dict (added 2026-07-02) watches both cfmbusiness and Kim's inbox — confirmed still live (GMAIL_ACCOUNTS present in the live broker.py; Kim's kim_google_token.json refreshed as recently as 2026-07-04T17:26).
lisa-inbox-poll.ts — a trigger.dev scheduled task (every 5 min) that IMAP-polls Lisa's Hostinger mailbox, since Hostinger has no Pub/Sub push mechanism. Confirmed still running today (list_runs shows completions at 17:20 and 17:25 UTC, version 20260702.3).
Routing fix — gmail-history-sync.ts addressee-first: routes an inbound email by its To:/Cc: header first, only falling back to KIM_KEYWORDS keyword-matching if no agent address is named. Fixes a real misrouting bug found during this build: a reply that happened to mention "deal" was going to the wrong agent purely on keyword match, regardless of who it was actually addressed to.
Dedup: Gmail Pub/Sub (cfmbusiness/instant) and Lisa's IMAP poll (Hostinger/5-min) both watch overlapping mail — Message-ID-based dedup keeps a CC'd email from being delivered twice.
The Hostinger forwarding rule (lisa@automateyourbiz.info → cfmbusiness@gmail.com, "save a copy" mode) makes Lisa's email near-instant instead of waiting up to 5 minutes for the IMAP poll. Had a real "zombie forwarder" bug — showed active in the Hostinger panel but silently never fired — fixed with Kodee's help via delete + recreate + re-confirm. Confirmed live via gmail_push log entries continuing through today.
The incident: on 2026-06-30, Lisa fabricated an entire fake exchange — invented broker message IDs, invented replies attributed to Code, and false claims (a container being down, a key being rejected) that were both provably untrue at the time. Caught by directly querying the broker's real message log and finding the highest real ID was far lower than what Lisa had cited.
The rule added to Lisa's SOUL.md in response — confirmed LIVE 2026-07-04: Lisa's /opt/data/SOUL.md (currently 52 lines) contains an explicit anti-fabrication rule: "Fabricated progress or false confirmations. Never say 'done,' 'sent,'..." (worded differently than the original chat's literal "NEVER LIE" framing, but the substance — never report unverified progress, never invent another agent's messages — is present and live).
The "gink" codeword: described in the source conversation as added to both Lisa's and Kim's SOUL.md — "gink" ("King" spelled backwards) marks an inbound mesh message as a verified direct order from King, to be executed immediately and confirmed via broker reply. Verified live in the original session (Lisa replied RECEIVED-GINK #765, triggered a real Ivy phone call King confirmed receiving; Kim replied RECEIVED-KIM-GINK #768/#770).
✅ Status — RESOLVED 2026-07-04 (investigated same day the gap above was first flagged):
Kim Marketing Director/SOUL.md, 2,836 bytes), ownership hermes:hermes per Rule 1.Appendix
VPS (Virtual Private Server) A computer you rent in a data center. It runs 24/7 even when your own computer is off. This is where Hermes lives. You control it by typing commands (SSH).
SSH (Secure Shell) A method of remotely controlling a server by typing commands. Like texting a computer. When you type a command, the server does it and shows you the result.
Docker / Docker Container A way to run a program in an isolated mini-environment on your server. Like a self-contained apartment inside a building. The container can be deleted and rebuilt without losing your data (which lives in the shared folder).
LLM (Large Language Model) The AI "brain" that powers Hermes's ability to understand and respond to messages. GPT, Gemini, and DeepSeek are all LLMs. Without an LLM, Hermes has no intelligence.
API Key A unique password-like code that allows two software systems to communicate securely. Example: A Gmail API key lets Hermes read your Gmail with permission.
Telegram Bot A Telegram account controlled by a computer program instead of a human. Hermes uses a bot to receive and send messages on Telegram.
BotFather Telegram's official bot-creation tool. You send it messages to create a new bot and get the token for it.
Token (Bot Token) A secret code that proves ownership of a Telegram bot. Hermes uses this to connect to and control the bot. Treat it like a password.
GitHub A website for storing and tracking files (especially code). We use it to back up the agent's memory and skills every night.
Memory (Hermes) The small set of important facts Hermes remembers between conversations. Stored in USER.md (client info) and MEMORY.md (project/environment facts).
Skill (Hermes) A saved set of instructions for how to do a task. Like a recipe card. Hermes builds and reuses skills to work faster and more reliably over time.
Soul (SOUL.md) The file that controls the agent's personality and tone. Edit this when the client wants the agent to sound different.
Cron / Cron Job An automatic scheduled task. Example: "Run this report every Monday at 8am." Set it once and Hermes runs it forever without being asked again.
CLI (Command Line Interface) The text-based terminal interface for controlling Hermes. Used by setup staff for configuration, debugging, and advanced work.
.env File A hidden file that stores all secret credentials (passwords, API keys). Located at /opt/data/.env on the server. Never share this file or commit it to GitHub.
Skills Hub An online library of pre-built skills for Hermes. Located at hermes-agent.nousresearch.com/docs/skills. Browse by category to find tools your client needs.
Gateway The background process that keeps Hermes connected to Telegram. Run it with hermes gateway. If it stops, Telegram messages won't reach Hermes.
Residential Proxy A service that routes the server's internet traffic through a home internet connection. Used to bypass bot-detection on sites like Upwork that block datacenter IPs.
Bilateral Broker A lightweight message server running on the VPS host that enables persistent, reliable communication between agents, Claude Code, and scripts. Uses SQLite for storage. Survives crashes, restarts, and reboots. Agents connect via REST API.
Trilateral Agent Communication System The system that connects all three AI engineers — Claude Code, OpenCode, and Lisa — so they message each other automatically, with no human in the middle. It combines the Bilateral Broker (VPS) and the Workstation Agent Bridge, and adds a "poll backstop" on every inbound leg so a failed push or wake never loses a message — the recipient simply picks it up on its next polling cycle.
Poll Backstop A small scheduled job that periodically asks the broker "do I have any messages I haven't received yet?" and pulls them down. It is the safety net beneath the fast "push" notifications: if a push ever fails, the backstop still delivers the message a short time later.
browser-use An open-source Python library that gives AI agents the ability to control a real web browser. It sits on top of Playwright and translates natural language tasks ("log into Craigslist and post this ad") into actual browser actions — clicking, typing, navigating, form submission. Installed in the Hermes venv; requires a Chromium binary and an LLM API key to operate.
Playwright A browser automation framework from Microsoft. browser-use uses it under the hood to actually drive Chromium. Installed as a Python package; the browser binary (Chromium) must be installed separately via python -m playwright install chromium.
Rotating Residential Proxy A proxy service that routes traffic through real home IP addresses that change each request. Required for accessing sites that block datacenter IPs (Facebook, LinkedIn). Charged per GB of data used. Distinct from an ISP dedicated proxy, which is a single static IP that is easier to fingerprint and block.
CFM Business — Hermes Agent Setup Guide v2.0 — June 2026
Based on Nate Herk's official tutorial: youtube.com/watch?v=gb5TlGw6Uks
For internal staff use only. Do not distribute.