Backup research: add post-speedrun filesystem snapshot (6.8G, 69% outside /root) + reference to competitor profiling speed-run
This commit is contained in:
@@ -1,47 +1,75 @@
|
|||||||
# 001 — Why backing up an agent is so hard
|
# 001 — Why backing up an agent is so hard
|
||||||
|
|
||||||
**Status:** Idea
|
**Status:** Research
|
||||||
**Tags:** Technical, Backups, DevOps
|
**Tags:** Technical, Backups, DevOps
|
||||||
|
|
||||||
## Research notes
|
## Research notes
|
||||||
|
|
||||||
### Real filesystem changes after one week of work on a Linux agent
|
### Real filesystem changes — from clean start through all speed-runs
|
||||||
|
|
||||||
|
#### Snapshot 1: After one week of basic work (original data)
|
||||||
|
|
||||||
```
|
```
|
||||||
504.6M ./usr
|
504.6M ./usr
|
||||||
43.3M ./var
|
43.3M ./var
|
||||||
448.9M ./root
|
448.9M ./root
|
||||||
54.0K ./etc
|
54.0K ./etc
|
||||||
996.8M .
|
996.8M total
|
||||||
```
|
```
|
||||||
|
|
||||||
Most users only back up the home directory (`/root` = 448.9M). But over half the changes live outside it — `/usr` (504.6M) and `/var` (43.3M) contain installed packages, pip/npm global installs, database files, logs, and system configs. A naive home-dir-only backup misses 55% of what changed.
|
Home-dir-only backup would capture 448.9M (45%) — already missing 55% of what changed.
|
||||||
|
|
||||||
At restore time, you get your config back but the agent won't run — missing dependencies, missing system packages, missing database files.
|
#### Snapshot 2: After all speed-run setups including competitor profiling (current)
|
||||||
|
|
||||||
|
```
|
||||||
|
4.3G ./usr ← 8.5× growth from 504.6M
|
||||||
|
487.0M ./var ← 11.2× growth from 43.3M
|
||||||
|
2.1G ./root ← 4.7× growth from 448.9M
|
||||||
|
1.9M ./etc ← 35× growth from 54K
|
||||||
|
6.8G total ← 6.8× growth from 996.8M
|
||||||
|
```
|
||||||
|
|
||||||
|
**Home-dir-only backup now captures 2.1G (31%) — missing 69% of what changed.**
|
||||||
|
|
||||||
|
#### What accounted for the growth (drill-down)
|
||||||
|
|
||||||
|
| Location | Size | What | Speed-Run Trigger |
|
||||||
|
|----------|------|------|-------------------|
|
||||||
|
| `/usr/local/lib` | 2.3G | Hermes runtime, node_modules, python3.11 libs | Hermes Agent installation |
|
||||||
|
| `/root/.npm/_cacache` | 390M | npm package cache | Node.js tooling for Hermes |
|
||||||
|
| `/root/.hermes` | 412M | 19 skills, 1 plugin, 2 cron jobs, memories, config | All speed-runs |
|
||||||
|
| `/root/.cache/huggingface` | 142M | HuggingFace model weights | STT / model download |
|
||||||
|
| `/root/.cache/uv` | 186M | UV Python package cache | Hermes venv management |
|
||||||
|
| `/var/cache/apt` | 379M | Debian package cache | System dependencies |
|
||||||
|
| `/var/lib/apt` | 80M | APT package state | System dependencies |
|
||||||
|
| `/usr/local/bin` | 28M | Executables (hermes CLI, etc.) | Hermes Agent installation |
|
||||||
|
| `/root/.config` | 475K | App configs | Misc tools |
|
||||||
|
| `/etc` | 1.9M | System config (hostname, apt sources, etc.) | OS configuration |
|
||||||
|
|
||||||
|
**Total blog content created:** 5 posts (Odoo, Hermes setup, competitor profiling, cold email, mobile.de research)
|
||||||
|
|
||||||
### What this means for the post
|
### What this means for the post
|
||||||
|
|
||||||
- Agent environments are not just config files — they're full Linux systems with packages, services, and state scattered everywhere
|
- Agent environments are not just config files — they're full Linux systems with packages, services, and state scattered everywhere
|
||||||
- "Backup your home directory" is dangerously incomplete advice for AI agents
|
- "Backup your home directory" is dangerously incomplete advice for AI agents
|
||||||
- Derez.ai's full-disk snapshot approach is the right solution
|
- The sprawl gets **worse** over time: after one week, 55% outside /root; after 3 speed-runs, 69% outside /root
|
||||||
|
- Each new skill, plugin, model download, or cron job adds state in a different directory
|
||||||
|
- Derez.ai's full-disk snapshot approach is the right solution — it captures everything, not just /root
|
||||||
- This is a strong selling point: the agent works after restore, not just the config
|
- This is a strong selling point: the agent works after restore, not just the config
|
||||||
|
|
||||||
### Next data points
|
|
||||||
|
|
||||||
Oliver will do two more memory analysis snapshots after running the speed-run setups (Odoo + Cold Email). Those will show how much additional state those integrations add.
|
|
||||||
|
|
||||||
### Outline
|
### Outline
|
||||||
|
|
||||||
1. The sprawl problem — why agents are harder to back up than a standard server
|
1. **The sprawl problem** — why agents are harder to back up than a standard server
|
||||||
*Include the filesystem analysis table*
|
*Include the dual snapshot comparison table (Week 1 vs After Speed-Runs)*
|
||||||
2. What a real agent backup needs to capture
|
2. **Real data from a real agent** — walk through the 6.8G of state and where it lives
|
||||||
3. How Borg/deduplicated snapshots solve it
|
3. **The 69% problem** — what you lose with a home-dir-only backup
|
||||||
4. How Derez.ai does it automatically (one-click restore)
|
4. **How Borg/deduplicated snapshots solve it**
|
||||||
5. Best practices for users
|
5. **How Derez.ai does it automatically** (one-click restore, full-disk snapshots)
|
||||||
6. Better save than sorry — the selling point
|
6. **Better save than sorry** — the selling point with real numbers
|
||||||
|
|
||||||
### References
|
### References
|
||||||
|
|
||||||
- Borg backup docs
|
- Borg backup docs
|
||||||
- Hermes Agent directory structure (~/.hermes/)
|
- Hermes Agent directory structure (~/.hermes/)
|
||||||
- Filesystem analysis: `du -sch /usr /var /root /etc` after one week
|
- Speed Run: Competitor Profiling (derez.ai/blog/posts/speed-run-competitor-profiling.html) — the speed-run that pushed the system past 1G
|
||||||
|
- Full filesystem analysis: `du -sch /usr /var /root /etc` before and after speed-runs
|
||||||
|
|||||||
+1
-1
@@ -717,7 +717,7 @@
|
|||||||
</div>
|
</div>
|
||||||
<div class="blog-idea-body">
|
<div class="blog-idea-body">
|
||||||
<div class="blog-idea-title">#001 — Why backing up an agent is so hard</div>
|
<div class="blog-idea-title">#001 — Why backing up an agent is so hard</div>
|
||||||
<div class="blog-idea-desc">Modern AI agents install packages everywhere — pip in venvs, npm globally, configs scattered across /etc, ~/.config, /opt, custom paths. After one week: 504.6M in /usr, 448.9M in /root, 43.3M in /var — home-dir-only backup misses 55% of changes.</div>
|
<div class="blog-idea-desc">After running all speed-runs (including competitor profiling), the system grew to 6.8G — with 69% outside /root. New data: dual-snapshot comparison showing sprawl worsens over time. Points to the competitor profiling speed-run as the trigger that pushed past 1G.</div>
|
||||||
<div class="blog-idea-meta">
|
<div class="blog-idea-meta">
|
||||||
<span class="blog-idea-tag">Technical</span>
|
<span class="blog-idea-tag">Technical</span>
|
||||||
<span class="blog-idea-tag">Backups</span>
|
<span class="blog-idea-tag">Backups</span>
|
||||||
|
|||||||
Reference in New Issue
Block a user