Oliver/4server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Create AGENTS.md

Browse Source
This commit is contained in:
oliver
2026-08-14 09:31:39 -03:00
parent 589ca5e4be
commit fbde9a19ab
1 changed files with 308 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+308
View File
@@ -0,0 +1,308 @@
# AGENTS.md — 4server Project Reference
## What This Is
**4server** is a self-hosted infrastructure management platform. It runs a Docker-based Alpine Linux control container locally and manages a fleet of remote Alpine Linux servers. Servers host multi-tenant Odoo (17/18/19) and N8N instances as Docker containers, all behind a Traefik reverse proxy with auto-TLS.
---
## Repository Layout
```
4server/
├── alpine/ # Control container (Dockerfile + entrypoint scripts)
│ ├── Dockerfile # Alpine image with pssh, borg, sshfs, nvim, cryptsetup, etc.
│ ├── config # SSH config for all managed hosts (user: 4server)
│ ├── known_hosts # Pre-populated SSH known_hosts
│ ├── rex # Run a command on ALL hosts via pssh
│ ├── template # Deploy a template file to ALL hosts (with variable substitution)
│ ├── create_volume # Create and LUKS-format an encrypted volume
│ └── mount_volume # Mount an encrypted volume
│
├── app/ # Mounted as /app inside the control container
│ ├── host_vars/ # Per-host SSH keys, env files, and hosts list
│ │ ├── hosts # Plain-text list of hostnames for pssh/prsync
│ │ ├── create # Script to generate Ed25519 SSH key pairs
│ │ └── {host}/ # Directory per host: {host} (private key), {host}.pub, {host}.env
│ │
│ ├── sbin/ # Scripts deployed to /4server/sbin/ on every server
│ │ ├── api # FastAPI REST API (Python 3, uvicorn) — server management API
│ │ ├── helpers # Bash lib: get_contract_info() — loads UUID data from SQLite
│ │ ├── startContainer # Dispatch start by UUID type
│ │ ├── stopContainer # Stop a container
│ │ ├── nukeContainer # Destroy container (must be stopped first)
│ │ ├── backupContainer # Dispatch backup by UUID type
│ │ ├── backupAll # Backup all running NNN-NNN containers
│ │ ├── borgpush # Push /BACKUP/current* files to BorgBase
│ │ ├── stats # JSON: uptime, top, cgroup, mounts
│ │ ├── audit # JSON: full Docker + OS audit
│ │ ├── checkCalls # OpenRC service: monitors API calls
│ │ ├── cleanTmp # OpenRC service: cleans /4server/tmp
│ │ ├── cpu # OpenRC service: CPU monitoring
│ │ ├── gitPull/gitRevert # Git operations inside containers
│ │ ├── migrate # Data migration helpers
│ │ ├── sql / psql # PostgreSQL query helpers
│ │ ├── contractInfo # Print contract info for a UUID
│ │ ├── getContainers # List containers
│ │ ├── pullAllContainers # docker pull on all containers
│ │ ├── restartContainers # Restart all containers
│ │ ├── odoo_passwd # Reset Odoo user password
│ │ ├── start/ # Type-specific start scripts (n8n, ODOO_17/18/19)
│ │ ├── backup/ # Type-specific backup scripts (N8N, ODOO)
│ │ ├── nuke/ # Type-specific nuke scripts (n8n, ODOO_19)
│ │ ├── ODOO_19/ # Odoo 19 admin tools (restore, shell, updateModules, audit, etc.)
│ │ └── N8N/ # N8N admin tools (currently empty)
│ │
│ ├── templates/ # Config templates deployed via the `template` command
│ │ ├── docker-compose.yml # Base infra: postgres (beedb) + traefik
│ │ ├── traefik.yaml # Traefik config
│ │ ├── repositories # APK repository list
│ │ ├── hosts # /etc/hosts
│ │ ├── hostname # /etc/hostname
│ │ ├── extlinux.conf # Bootloader (variants for DO, Hostinger)
│ │ ├── nebula/ # Nebula VPN: config.yml, ca.crt, host.crt, host.key
│ │ ├── init.d/ # OpenRC service files: nebula, api, checkCalls, cleanTmp, cpu, ping_service
│ │ ├── ssh/ # SSH authorized_keys (id_ed25519.pub)
│ │ ├── etc/doas.d/ # doas config (4server.conf)
│ │ └── .profile # Shell profile for 4server user
│ │
│ ├── etc/traefik/certs/ # TLS certificates distributed to servers
│ ├── vault/ # Encrypted secrets (not committed)
│ │
│ ├── onboarding # New server bootstrap (Alpine packages, Nebula VPN, 4server user)
│ ├── hardening # Lock down SSH (no root, no password login)
│ ├── update # Full server update (packages, Docker infra, API, Traefik)
│ ├── update_sbin # Deploy /app/sbin/ to servers + restart services
│ ├── update_alpine # APK update/upgrade only
│ ├── restart # Restart core services
│ ├── backup_prune # Prune old Borg archives
│ ├── migrate # One-off DB migrations
│ └── download_sbin # Download binary tools (nebula, etc.)
│
├── exchange/ # Shared volume between host and control container (file drop)
├── docker-compose.yaml # Builds + runs the Alpine control container
├── mount # Helper to mount exchange volume
└── start # Start the control container
```
---
## Servers (Managed Fleet)
| Alias | IP | Port | Notes |
|------------|-----------------|------|--------------------|
| dev | 192.168.9.221 | 2222 | Development server |
| manchester | 192.168.9.20 | 22 | |
| boston | 192.168.9.16 | 22 | |
| mumbai | 192.168.9.17 | 22 | |
| meppel | 192.168.9.21 | 22 | |
| sydney | 192.168.9.22 | 22 | |
| saopaulo | 192.168.9.11 | 22 | |
SSH user on all servers: `4server`
Key per host: `/app/host_vars/{host}/{host}` (Ed25519)
Active hosts file: `/app/host_vars/hosts` (currently: `sydney2`)
---
## UUID Format
Every managed container has a UUID structured as: `{server}-{type}-{random}`
| Segment | Meaning | Known values |
|---------|-------------|-------------------------------------------|
| `aaa` | Server | `001` = manchester, `002` = boston |
| `bbb` | Image type | `001` = N8N, `002` = Odoo 18, `003` = Odoo 19, `004` = Odoo 17 |
The second segment (`bbb`) is used by `startContainer`, `nukeContainer`, `backupContainer` to dispatch to the right type-specific script.
---
## Core Tools (inside the Alpine control container)
### `rex <command>`
Runs a shell command on **all hosts** in `/app/host_vars/hosts` via `pssh`.
Example: `rex doas apk update`
### `prsync -h /app/host_vars/hosts -avz <src> <dst>`
Parallel rsync to all hosts. Used to deploy `/app/sbin/` and certs.
### `template <localfile> <remotefile>`
Renders `{{KEY}}` placeholders in `localfile` and copies it to `remotefile` on every host.
Available template variables (loaded from `host_vars/{host}/{host}.env` + key files):
- `NEBULA_CA`, `NEBULA_CRT`, `NEBULA_KEY`
- `API_KEY`, `HOSTNAME`
- `SSH_PRIVATE`, `SSH_PUBLIC`
---
## Server-Side Architecture (on each managed server)
```
/4server/
├── sbin/ # All management scripts (deployed from app/sbin/)
├── data/
│ ├── contracts.db # SQLite: container contracts table
│ ├── log/ # Logs for all services
│ ├── {UUID}/ # Per-container data directory
│ │ ├── odoo/ # Odoo filestore
│ │ ├── git/ # Git repos
│ │ ├── git-server/ # Bare git repos exposed via SSH
│ │ ├── logs/
│ │ ├── config/
│ │ ├── cc/ # Control channel (e.g. backup trigger file)
│ │ ├── etc/ # domain file, gitpath, etc.
│ │ ├── install/
│ │ └── .ssh/ # Container SSH keys
│ ├── traefik/etc/ # traefik.yaml + TLS certs
│ └── postgres/ # PostgreSQL data + pg_hba.conf
├── tmp/ # Temporary files (cleaned by cleanTmp service)
└── docker-compose.yml # Base infra (beedb + traefik)
/BACKUP/{UUID}/ # Per-container backup files (borgpush picks up current*)
```
### Persistent Docker Services (base infra)
| Container | Image | Role | IP (internal) |
|-----------|---------------|-------------------------------|----------------|
| `beedb` | postgres:16 | Shared PostgreSQL for all Odoo| 10.5.0.200 |
| `traefik` | traefik:3.6 | Reverse proxy, HTTPS (Let's Encrypt) | — |
Docker network: `4server_4projects` (bridge, `10.5.0.0/16`)
### Per-Container Docker Services
- Each UUID is a `docker run` container on the host's Docker network
- SSH port: `CONTAINERDBID + 2200` (forwarded to the host)
- Traefik routing via Docker labels (`traefik.http.routers.{UUID}.*`)
- Default domain: `{UUID}.odoo4projects.com`; optional custom domains from `/4server/data/{UUID}/etc/domain`
---
## Data Model: `contracts.db`
Table: `containers`
| Column | Description |
|----------------|----------------------------------------------------|
| `UUID` | Container identifier (aaa-bbb-... format) |
| `id` | Auto-increment row ID (`CONTAINERDBID`) |
| `image` | Image type code |
| `status` | Container status |
| `expires` | Contract expiry date |
| `created` | Creation timestamp |
| `secret_list` | Backslash-separated `key=value` pairs (passwords) |
| `git` | Git integration config |
| `backup_slots` | Number of backup slots |
| `hdd` | Disk quota |
| `workers` | Worker count |
| `domains_slots`| Number of allowed custom domains |
| `domains` | Active custom domains (comma-separated) |
| `affiliate` | JSON: UTM tracking data |
`secret_list` parsing: `key=value` pairs separated by `\`. Key `psql` = Odoo DB password (`ODOO_DB_PASSWORD`).
---
## Management API (`/4server/sbin/api`)
FastAPI Python 3 app, served by uvicorn, managed by OpenRC (`api` service).
- Auth: `X-API-Key` header (value from `/etc/4server` env file, key `API_KEY`)
- Version: `0.0.9`
Key endpoints (all protected):
| Endpoint | Description |
|----------|-------------|
| `GET /containers` | List all containers from SQLite |
| `POST /container/update` | Create/update a container record |
| `POST /container/start` | Start a container by UUID |
| `POST /container/stop` | Stop a container by UUID |
| `POST /container/nuke` | Destroy a container |
| `GET /container/info` | Full container info + Docker inspect |
| `POST /container/bump` | Bump/restart a container |
| `GET /container/quota` | Disk quota usage |
| `GET /images` | List available Docker images |
| `GET /cpu` | CPU log |
| `GET /stats` | Server runtime stats (JSON) |
| `GET /audit` | Full Docker + OS audit (JSON) |
| `POST /git` | Git pull/revert inside container |
| `POST /git/key` | Add SSH key to git-server |
| `GET /odoo/audit` | Odoo-specific audit |
| `GET /odoo/log` | Summarised Odoo log (last N lines, errors/warnings) |
| `POST /backup/import` | Import a backup archive |
| `POST /backup/move` | Move a backup file |
| `POST /backup/check` | Check backup integrity |
| `POST /backup/all` | Trigger backup of all containers |
| `POST /backup/borgpush` | Push backups to BorgBase |
---
## Backup
- **Per-container**: `backupContainer <UUID>` → dispatches to `backup/N8N` or `backup/ODOO`
- **All containers**: `backupAll` — touches `/4server/data/{UUID}/cc/backup` for each running NNN-NNN container
- **Off-site**: `borgpush` → BorgBase (`e7e9h45y.repo.borgbase.com`) via SSH key `~/.ssh/borg-backup`
- **Pruning**: `backup_prune` — keep 7 daily / 8 weekly / 12 monthly
---
## Security Model
- **Privilege escalation**: `doas` (not sudo). Config at `/etc/doas.d/4server.conf`
- **SSH**: Key-only (`PasswordAuthentication no`, `PermitRootLogin no`)
- **API**: Header-based API key (`X-API-Key`)
- **Secrets**: Stored in `secret_list` column of SQLite (base64-safe `\`-separated pairs)
- **VPN**: Nebula overlay mesh between all servers (certs per host in `host_vars/{host}/`)
- **TLS**: Traefik with Let's Encrypt production resolver
---
## Workflow Reference
### Add a new server
1. Add SSH key pair: `cd app/host_vars && ./create {hostname}`
2. Create `host_vars/{host}/{host}.env` with `HOSTNAME`, `API_KEY`, `NEBULA_CRT`, `NEBULA_KEY`
3. Add host to `host_vars/hosts`
4. Add SSH config block to `alpine/config`
5. Run `onboarding` from inside the control container
6. Run `hardening` to lock down SSH
### Deploy code changes to servers
```bash
# Inside control container (/app)
./update_sbin # deploy sbin/ scripts + restart api/checkCalls
```
### Deploy a config template
```bash
template templates/my.conf /etc/my.conf
```
### Run a command on all servers
```bash
rex doas rc-service api restart
```
### Start/stop a specific container (from server)
```bash
startContainer <UUID>
stopContainer <UUID>
```
---
## Key Technologies
| Layer | Technology |
|-------|------------|
| Control container | Alpine Linux + Docker Compose |
| Server OS | Alpine Linux (OpenRC init system) |
| Reverse proxy | Traefik v3.6 |
| Database | PostgreSQL 16 (shared, per-server) |
| App runtime | Odoo 17/18/19, N8N |
| Management API | Python 3 / FastAPI / uvicorn |
| Backup | BorgBackup → BorgBase |
| VPN | Nebula mesh |
| Parallel SSH | pssh / prsync |
| Container secrets | SQLite (`contracts.db`) |
| SSH auth | Ed25519 keys per host |