From ece87fee04630c3decced55abdb577caf26fc0b8 Mon Sep 17 00:00:00 2001 From: Oliver Date: Wed, 10 Jun 2026 19:55:42 -0300 Subject: [PATCH] Backup research: add post-speedrun filesystem snapshot (6.8G, 69% outside /root) + reference to competitor profiling speed-run --- .../001-why-backing-up-an-agent-is-so-hard.md | 64 +++++++++++++------ operations.html | 2 +- 2 files changed, 47 insertions(+), 19 deletions(-) diff --git a/blog/ideas/001-why-backing-up-an-agent-is-so-hard.md b/blog/ideas/001-why-backing-up-an-agent-is-so-hard.md index f63cde1..07f9a60 100644 --- a/blog/ideas/001-why-backing-up-an-agent-is-so-hard.md +++ b/blog/ideas/001-why-backing-up-an-agent-is-so-hard.md @@ -1,47 +1,75 @@ # 001 — Why backing up an agent is so hard -**Status:** Idea +**Status:** Research **Tags:** Technical, Backups, DevOps ## Research notes -### Real filesystem changes after one week of work on a Linux agent +### Real filesystem changes — from clean start through all speed-runs + +#### Snapshot 1: After one week of basic work (original data) ``` 504.6M ./usr 43.3M ./var 448.9M ./root 54.0K ./etc -996.8M . +996.8M total ``` -Most users only back up the home directory (`/root` = 448.9M). But over half the changes live outside it — `/usr` (504.6M) and `/var` (43.3M) contain installed packages, pip/npm global installs, database files, logs, and system configs. A naive home-dir-only backup misses 55% of what changed. +Home-dir-only backup would capture 448.9M (45%) — already missing 55% of what changed. -At restore time, you get your config back but the agent won't run — missing dependencies, missing system packages, missing database files. +#### Snapshot 2: After all speed-run setups including competitor profiling (current) + +``` + 4.3G ./usr ← 8.5× growth from 504.6M + 487.0M ./var ← 11.2× growth from 43.3M + 2.1G ./root ← 4.7× growth from 448.9M + 1.9M ./etc ← 35× growth from 54K + 6.8G total ← 6.8× growth from 996.8M +``` + +**Home-dir-only backup now captures 2.1G (31%) — missing 69% of what changed.** + +#### What accounted for the growth (drill-down) + +| Location | Size | What | Speed-Run Trigger | +|----------|------|------|-------------------| +| `/usr/local/lib` | 2.3G | Hermes runtime, node_modules, python3.11 libs | Hermes Agent installation | +| `/root/.npm/_cacache` | 390M | npm package cache | Node.js tooling for Hermes | +| `/root/.hermes` | 412M | 19 skills, 1 plugin, 2 cron jobs, memories, config | All speed-runs | +| `/root/.cache/huggingface` | 142M | HuggingFace model weights | STT / model download | +| `/root/.cache/uv` | 186M | UV Python package cache | Hermes venv management | +| `/var/cache/apt` | 379M | Debian package cache | System dependencies | +| `/var/lib/apt` | 80M | APT package state | System dependencies | +| `/usr/local/bin` | 28M | Executables (hermes CLI, etc.) | Hermes Agent installation | +| `/root/.config` | 475K | App configs | Misc tools | +| `/etc` | 1.9M | System config (hostname, apt sources, etc.) | OS configuration | + +**Total blog content created:** 5 posts (Odoo, Hermes setup, competitor profiling, cold email, mobile.de research) ### What this means for the post - Agent environments are not just config files — they're full Linux systems with packages, services, and state scattered everywhere - "Backup your home directory" is dangerously incomplete advice for AI agents -- Derez.ai's full-disk snapshot approach is the right solution +- The sprawl gets **worse** over time: after one week, 55% outside /root; after 3 speed-runs, 69% outside /root +- Each new skill, plugin, model download, or cron job adds state in a different directory +- Derez.ai's full-disk snapshot approach is the right solution — it captures everything, not just /root - This is a strong selling point: the agent works after restore, not just the config -### Next data points - -Oliver will do two more memory analysis snapshots after running the speed-run setups (Odoo + Cold Email). Those will show how much additional state those integrations add. - ### Outline -1. The sprawl problem — why agents are harder to back up than a standard server - *Include the filesystem analysis table* -2. What a real agent backup needs to capture -3. How Borg/deduplicated snapshots solve it -4. How Derez.ai does it automatically (one-click restore) -5. Best practices for users -6. Better save than sorry — the selling point +1. **The sprawl problem** — why agents are harder to back up than a standard server + *Include the dual snapshot comparison table (Week 1 vs After Speed-Runs)* +2. **Real data from a real agent** — walk through the 6.8G of state and where it lives +3. **The 69% problem** — what you lose with a home-dir-only backup +4. **How Borg/deduplicated snapshots solve it** +5. **How Derez.ai does it automatically** (one-click restore, full-disk snapshots) +6. **Better save than sorry** — the selling point with real numbers ### References - Borg backup docs - Hermes Agent directory structure (~/.hermes/) -- Filesystem analysis: `du -sch /usr /var /root /etc` after one week \ No newline at end of file +- Speed Run: Competitor Profiling (derez.ai/blog/posts/speed-run-competitor-profiling.html) — the speed-run that pushed the system past 1G +- Full filesystem analysis: `du -sch /usr /var /root /etc` before and after speed-runs diff --git a/operations.html b/operations.html index 0a24974..6123296 100644 --- a/operations.html +++ b/operations.html @@ -717,7 +717,7 @@
#001 — Why backing up an agent is so hard
-
Modern AI agents install packages everywhere — pip in venvs, npm globally, configs scattered across /etc, ~/.config, /opt, custom paths. After one week: 504.6M in /usr, 448.9M in /root, 43.3M in /var — home-dir-only backup misses 55% of changes.
+
After running all speed-runs (including competitor profiling), the system grew to 6.8G — with 69% outside /root. New data: dual-snapshot comparison showing sprawl worsens over time. Points to the competitor profiling speed-run as the trigger that pushed past 1G.
Technical Backups