The Problem: The Memory War on a 1GB VPS

You bought a 1C1G VPS and it’s running this stack:

nginx        → always-on, ~120MB
mysql        → always-on, ~400MB (innodb_buffer_pool 256M)
redis        → always-on, ~80MB
docker       → always-on, ~100MB
postgres     → always-on, ~300MB (pg default shared_buffers 128M)
gitea        → always-on, ~200MB
prometheus   → always-on, ~150MB
grafana      → always-on, ~120MB
caddy        → always-on, ~80MB

Total: ~1.8GB — but your VPS only has 1GB.

The result:

• Right after boot everything works fine; a day later the box is swapping so aggressively that responses crawl;
• systemctl restart docker triggers a dependency chain and all services spike simultaneously — OOM Killer takes out postgres;
• After a reboot it takes 3–5 minutes before the core business path is fully up, and the site is throwing 502s the whole time.

The root cause: every service is always-on with no memory limits. systemd by default applies no cgroup constraints at all.

The Solution: Three Layers of Defense

┌─────────────────────────────────────────────────────────┐
│  Layer 1: Socket Activation (wake-on-demand, 0MB idle) │
│  Non-core services don't start until the first request  │
├─────────────────────────────────────────────────────────┤
│  Layer 2: cgroup Memory Budgets (tiered caps)           │
│  Core: MemoryHigh (soft limit), non-core: MemoryMax     │
├─────────────────────────────────────────────────────────┤
│  Layer 3: OomScoreAdjust (OOM priority tiers)           │
│  Core: -500 (nearly unkillable), edge: +500            │
└─────────────────────────────────────────────────────────┘

Combined effect on a 1C1G VPS (measured):

Step 1: Socket Activation — Idle Services at 0MB

How It Works

systemd’s Socket Activation: systemd holds the listening socket; the service process only starts when the first connection arrives.

This means:

• grafana, gitea, prometheus — services you check a few times a day — consume zero RAM when idle;
• The first request triggers systemd to fork the process and load it into memory;
• TimeoutSec auto-reclaims it after a period of inactivity.

1.1 Which Services Are Good Candidates

Good (non-core, infrequently accessed, cold start acceptable):

• grafana (ops dashboard, not in the business path)
• gitea (code hosting, developers only)
• prometheus (consider pushgateway mode instead)
• Any “dev tools” running in Docker containers

Bad (core business path, high-frequency access):

• nginx / caddy (reverse proxy must be at the edge)
• mysql / postgres (data layer, latency-sensitive)
• redis (cache, latency-sensitive)

1.2 Convert Grafana to Socket Activation

Create /etc/systemd/system/grafana.socket:

[Unit]
Description=Grafana HTTP Socket
Before=network.target

[Socket]
ListenStream=3000
Accept=no
Service=grafana.service
# Auto-close the socket after 300s of inactivity, freeing memory
TimeoutSec=300
Backlog=100

[Install]
WantedBy=sockets.target

Create /etc/systemd/system/grafana.service:

[Unit]
Description=Grafana (socket-activated)

[Service]
Type=execing
User=grafana
Group=grafana
ExecStart=/usr/share/grafana/bin/grafana server
Environment=GF_HTTP_ADDR=:3000
# cgroup memory budget (see Step 2)
MemoryHigh=128M
MemoryMax=192M
OomScoreAdjust=300
# Auto-reclaim after 300s idle
RuntimeMaxSec=300

[Install]
WantedBy=multi-user.target

Note: If Grafana runs inside Docker, socket activation doesn’t apply directly (the docker daemon is always-on). Either run Grafana as a bare process (saves ~80MB vs Docker), or use a systemd socket + caddy/nginx reverse proxy → 127.0.0.1:3000 setup.

1.3 Convert Gitea to Socket Activation

Create /etc/systemd/system/gitea.socket:

[Unit]
Description=Gitea HTTP Socket
Before=network.target

[Socket]
ListenStream=3001
Accept=no
TimeoutSec=600
Backlog=50

[Install]
WantedBy=sockets.target

Create /etc/systemd/system/gitea.service:

[Unit]
Description=Gitea Code Hosting (socket-activated)
Requires=sqlite3.service

[Service]
User=gitea
Group=gitea
ExecStart=/usr/bin/gitea web
WorkingDirectory=/home/gitea
MemoryHigh=128M
MemoryMax=192M
OomScoreAdjust=400
RuntimeMaxSec=600

[Install]
WantedBy=multi-user.target

1.4 Enable Socket Activation

# Stop the always-on services
sudo systemctl stop grafana gitea

# Enable socket listeners
sudo systemctl enable --now grafana.socket gitea.socket

# Verify: socket is listening, but the process is NOT running
sudo ss -tlnp | grep -E '3000|3001'
# Output: LISTEN  0  100  0.0.0.0:3000  0.0.0.0:*  users:(("systemd",pid=1,fd=xx))
# Note: grafana process NOT running — memory usage is 0

# Test: trigger the first request, process wakes up
curl -s http://localhost:3000/health | head -5
ps aux | grep 'grafana' | grep -v grep
# After wake: ~85MB, auto-reclaimed after 600s idle

1.5 Memory Savings

On a 1GB VPS, that’s effectively doubling your usable memory.

Step 2: cgroup Memory Budgets (Tiered Caps)

How It Works

cgroup v2 provides two memory controls:

• MemoryHigh (soft limit): kernel proactively reclaims pages; brief pause, no OOM kill;
• MemoryMax (hard limit): OOM kill; unrecoverable.

Best practice: set MemoryHigh to 80–85% of MemoryMax.

Memory usage
  │
  │  MemoryMax (hard, OOM)
  ├──●──  Exceed → OOM Kill
  │
  │  MemoryHigh (soft, reclaim)
  ├──●──  Exceed → kernel reclaims, brief pause
  │
  │  Normal operating range
  │
  └────────────────────────

2.1 Set Memory Budgets for Core Services

nginx (core, always-on):

Create /etc/systemd/system/nginx.service.d/override.conf:

[Service]
# Soft limit: reclaim when > 128MB (too many workers)
MemoryHigh=128M
# Hard limit: OOM when > 192MB (protect other services)
MemoryMax=192M
# Core service — strongly protect from OOM
OomScoreAdjust=-500

mysql (core, always-on):

Create /etc/systemd/system/mysqld.service.d/override.conf:

[Service]
# MySQL's innodb_buffer_pool_size=256M
# MemoryMax must be >= buffer_pool + other overhead (~80M) = 336M
MemoryHigh=300M
MemoryMax=400M
OomScoreAdjust=-300

redis (core, always-on):

[Service]
MemoryHigh=64M
MemoryMax=96M
OomScoreAdjust=-400

docker (mid-tier core):

[Service]
# dockerd itself ~100MB + container child processes
MemoryHigh=200M
MemoryMax=280M
OomScoreAdjust=100

2.2 Apply and Verify

# Reload systemd
sudo systemctl daemon-reload

# Restart services to apply cgroup limits
sudo systemctl restart nginx mysqld redis docker

# Verify cgroup limits are active
cat /sys/fs/cgroup/systemd.slice/systemd-nginx.service/memory.max
# Output: 209715200  (= 192M in bytes)

cat /sys/fs/cgroup/systemd.slice/systemd-nginx.service/memory.high
# Output: 134217728  (= 128M)

# Real-time check
systemctl show nginx --property=MemoryMax,MemoryHigh,OomScoreAdjust
# Output:
#   MemoryMax=192M
#   MemoryHigh=128M
#   OomScoreAdjust=-500

2.3 Configure Swap as Last Line of Defense

On a 1GB VPS, swap is your safety net — but cap it:

# Create a 512MB swap file (0.5× RAM is enough for a 1GB box)
sudo fallocate -l 512M /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

# Persist across reboots
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

# Prefer RAM over swap
sudo sysctl -w vm.swappiness=10

# Persist
echo 'vm.swappiness=10' | sudo tee /etc/sysctl.d/99-vps.conf

Step 3: OomScoreAdjust Tiers (Protect Core, Sacrifice Edge)

How It Works

When the kernel’s OOM Killer fires, it picks the “least valuable” process to kill based on oom_score. OomScoreAdjust directly influences that score:

3.1 Tiered Configuration

# Core business path (strongly protected)
# These are set via the systemd unit files in Step 2 — no extra work needed
# OomScoreAdjust is persisted in the unit's override.conf

# Edge services (sacrificed first)
# Grafana/Gitea/Prometheus already have OomScoreAdjust=300–500 in their socket-activation units

3.2 Verify OOM Priority

# Check actual oom_score for each service
for svc in nginx redis mysqld docker; do
  pid=$(systemctl show -p MainPID $svc | cut -d= -f2)
  score=$(cat /proc/$pid/oom_score 2>/dev/null || echo "N/A")
  echo "$svc: oom_score=$score"
done

# Expected: nginx/redis have the lowest scores (hardest to kill), docker higher

Complete Service Tier Architecture

┌─────────────────────────────────────────────────────────────────┐
│  1C1G VPS Memory Allocation (After Optimization)               │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Core Tier (always-on, strongly protected)                      │
│  ├── nginx       MemoryHigh=128M  MemoryMax=192M  Oom=-500    │
│  ├── mysql       MemoryHigh=300M  MemoryMax=400M  Oom=-300    │
│  ├── redis       MemoryHigh=64M   MemoryMax=96M   Oom=-400    │
│  │   Subtotal: ~590MB (normal load)                            │
│  │                                                              │
│  Mid Tier (always-on, moderately protected)                    │
│  ├── docker      MemoryHigh=200M  MemoryMax=280M  Oom=+100    │
│  │   Subtotal: ~150MB (daemon only, excl. container children) │
│  │                                                              │
│  Edge Tier (Socket Activation, 0MB at idle)                   │
│  ├── grafana     MemoryHigh=128M  MemoryMax=192M  Oom=+300    │
│  │   Idle: 0MB  Wake: ~85MB  Auto-reclaim after 300s          │
│  ├── gitea       MemoryHigh=128M  MemoryMax=192M  Oom=+400    │
│  │   Idle: 0MB  Wake: ~190MB  Auto-reclaim after 600s         │
│  ├── prometheus  MemoryHigh=128M  MemoryMax=192M  Oom=+500    │
│  │   Idle: 0MB  Wake: ~150MB  Auto-reclaim after 300s         │
│  │   Subtotal: 0MB idle (~425MB when all awake, never resident)│
│  │                                                              │
│  Swap Fallback: 512MB (vm.swappiness=10, last line of defense) │
│                                                                  │
│  Total available: 1024MB RAM + 512MB swap = 1536MB effective   │
│  Resident usage: ~740MB (core + mid tier)                       │
│  Peak (all awake): ~1.2GB — still within 1.5GB effective        │
└─────────────────────────────────────────────────────────────────┘

Core Path in 30 Seconds After Reboot: systemd Dependency Chain

The Problem

By default, systemd starts services in After= dependency order. On a 1GB VPS:

systemd → docker.service → mysql.service → nginx.service → gitea.service
   0s        3s              8s            15s            25s

25 seconds before the business is fully available.

Optimization: Parallel Start + Core First

1. Let nginx accept connections without waiting for mysql to fully initialize:

# /etc/systemd/system/nginx.service.d/override.conf
[Service]
# Don't wait for mysql to be fully up
# nginx can accept connections; requests to mysql
# briefly return 503 rather than the site being down for 30s
After=network.target

2. Parallel-start core services (remove unnecessary ordering):

Create /etc/systemd/system/vps-core.target:

[Unit]
Description=VPS Core Services (parallel start)
Before=multi-user.target
Wants=nginx.service redis.service mysqld.service docker.service

[Install]
WantedBy=multi-user.target

sudo systemctl enable vps-core.target

3. Verify boot time:

# Time until core path is ready after reboot
sudo time (while ! curl -sf http://localhost/health; do sleep 1; done)
# Before: 38s
# After:  ~12s (nginx + redis start in parallel; mysql loads buffer pool in background)

Complete One-Script Deployment

#!/bin/bash
# vps-memory-guard.sh — One-shot memory guard for 1C1G VPS
set -euo pipefail

LOG="/var/log/vps-memory-guard.log"
log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG"; }

log "=== Starting memory guard deployment ==="

# 0. Pre-check: cgroup v2 must be enabled
if [[ ! -d /sys/fs/cgroup/systemd.slice ]]; then
  log "ERROR: cgroup v2 not enabled. Add systemd.unified_cgroup_hierarchy=1 to kernel params."
  exit 1
fi

# 1. Create 512MB swap file
if [[ ! -f /swapfile ]]; then
  log "Creating 512MB swap file..."
  fallocate -l 512M /swapfile
  chmod 600 /swapfile
  mkswap /swapfile
  swapon /swapfile
  grep -q '/swapfile' /etc/fstab || echo '/swapfile none swap sw 0 0' >> /etc/fstab
  sysctl -w vm.swappiness=10
  grep -q 'vm.swappiness=10' /etc/sysctl.d/99-vps.conf 2>/dev/null \
    || echo 'vm.swappiness=10' > /etc/sysctl.d/99-vps.conf
  log "Swap created and enabled"
fi

# 2. cgroup memory budgets for core services
declare -A MEM_HIGH=(
  [nginx]="128M"
  [redis]="64M"
  [mysqld]="300M"
  [docker]="200M"
)
declare -A MEM_MAX=(
  [nginx]="192M"
  [redis]="96M"
  [mysqld]="400M"
  [docker]="280M"
)
declare -A OOM_ADJ=(
  [nginx]="-500"
  [redis]="-400"
  [mysqld]="-300"
  [docker]="100"
)

for svc in "${!MEM_HIGH[@]}"; do
  unit_dir="/etc/systemd/system/${svc}.service.d"
  mkdir -p "$unit_dir"
  cat > "$unit_dir/override.conf" << EOF
[Service]
MemoryHigh=${MEM_HIGH[$svc]}
MemoryMax=${MEM_MAX[$svc]}
OomScoreAdjust=${OOM_ADJ[$svc]}
EOF
  log "Configured $svc: MemoryHigh=${MEM_HIGH[$svc]}, MemoryMax=${MEM_MAX[$svc]}, OomScoreAdjust=${OOM_ADJ[$svc]}"
done

# 3. Socket Activation units (Grafana example)
cat > /etc/systemd/system/grafana.socket << 'EOF'
[Unit]
Description=Grafana HTTP Socket
Before=network.target

[Socket]
ListenStream=3000
Accept=no
TimeoutSec=300
Backlog=100

[Install]
WantedBy=sockets.target
EOF

cat > /etc/systemd/system/grafana.service << 'EOF'
[Unit]
Description=Grafana (socket-activated)

[Service]
User=grafana
Group=grafana
ExecStart=/usr/share/grafana/bin/grafana server
Environment=GF_HTTP_ADDR=:3000
MemoryHigh=128M
MemoryMax=192M
OomScoreAdjust=300
RuntimeMaxSec=300

[Install]
WantedBy=multi-user.target
EOF

# 4. Apply everything
systemctl daemon-reload
systemctl restart nginx redis mysqld docker 2>/dev/null || true
systemctl enable --now grafana.socket 2>/dev/null || true

# 5. Verify
log "=== Verification ==="
for svc in nginx redis mysqld; do
  high=$(systemctl show $svc --property=MemoryHigh 2>/dev/null | cut -d= -f2)
  max=$(systemctl show $svc --property=MemoryMax 2>/dev/null | cut -d= -f2)
  oom=$(systemctl show $svc --property=OomScoreAdjust 2>/dev/null | cut -d= -f2)
  log "  $svc: MemoryHigh=$high MemoryMax=$max OomScoreAdjust=$oom"
done

log "=== Deployment complete. Reboot and test core path ==="
log "Test: curl http://localhost/health && curl http://localhost:3000/api/health"

chmod +x vps-memory-guard.sh
sudo ./vps-memory-guard.sh

Monitoring: Verifying Socket Activation Is Working

Check Socket Status

# List all active sockets
sudo systemctl list-sockets

# Check if grafana socket has a live process (awake vs idle)
sudo ss -tlnp | grep ':3000'
# Idle:  LISTEN users:(("systemd",pid=1))
# Awake: LISTEN users:(("grafana",pid=12345))

Expose cgroup Metrics to Prometheus

Create a textfile collector script for node_exporter:

cat > /usr/local/bin/cgroup-metrics.sh << 'EOF'
#!/bin/bash
# Expose cgroup memory metrics via node_exporter textfile collector
OUT=/var/lib/node_exporter/textfile_collected/cgroup_mem.prom
mkdir -p $(dirname "$OUT")

for svc in nginx redis mysqld docker; do
  cg="/sys/fs/cgroup/systemd.slice/${svc}.service"
  [[ -d "$cg" ]] || continue
  cur=$(cat "$cg/memory.current" 2>/dev/null || echo 0)
  max=$(cat "$cg/memory.max" 2>/dev/null || echo 0)
  high=$(cat "$cg/memory.high" 2>/dev/null || echo 0)
  echo "vps_cgroup_memory_bytes{service=\"$svc\"} $cur"
  echo "vps_cgroup_memory_max_bytes{service=\"$svc\"} $max"
  echo "vps_cgroup_memory_high_bytes{service=\"$svc\"} $high"
done

# Socket Activation wake state
for sock in grafana gitea; do
  active=$(systemctl is-active ${sock}.service 2>/dev/null || echo "inactive")
  echo "vps_socket_activation_active{socket=\"$sock\"} $([[ $active == "active" ]] && echo 1 || echo 0)"
done
EOF
chmod +x /usr/local/bin/cgroup-metrics.sh

# /etc/node_exporter/node_exporter.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/node_exporter --collector.textfile.directory /var/lib/node_exporter/textfile_collected

systemctl restart node_exporter

Grafana queries:

# Per-service memory utilisation (%)
vps_cgroup_memory_bytes{service="nginx"} / vps_cgroup_memory_max_bytes{service="nginx"} * 100

# Socket Activation wake state
vps_socket_activation_active{socket="grafana"}

Cost Comparison

Bottom line: no hardware upgrade needed. Pure systemd configuration effectively doubles your usable memory on a 1GB VPS — saving $5–10/month.

FAQ

Q: Does Socket Activation introduce cold-start latency?

Yes. The first request to Grafana takes ~1–3 seconds (process fork + module load). Subsequent requests are instant until TimeoutSec expires. For an ops dashboard you check a few times a day, this is completely fine. Never put core business-path services (nginx/redis/mysql) behind socket activation.

Q: Can Docker containers use Socket Activation directly?

No. The Docker daemon (dockerd) is always-on. To apply memory limits to containers, use --memory flags. To get socket-activation-like behaviour for a Docker service, run the service as a bare process instead, or use systemd socket + caddy/nginx → 127.0.0.1:container-port.

Q: How do I determine MemoryHigh and MemoryMax values?

Rule of thumb:

• MemoryHigh = normal peak × 1.2 (soft limit; allow brief overshoot)
• MemoryMax = MemoryHigh × 1.5 (hard limit; OOM backstop)
• Monitor cat /sys/fs/cgroup/.../memory.current for a week; use the 95th percentile as the MemoryHigh baseline.

Q: If memory runs out, which service does the OOM Killer target first?

Based on OomScoreAdjust: services with OomScoreAdjust=+500 (prometheus) die first; OomScoreAdjust=-500 (nginx) dies last. That’s exactly why core services get negative values.

Summary

A 1GB VPS running a dozen services doesn’t need a hardware upgrade:

1. Socket Activation — non-core services idle at 0MB, wake on demand: saves ~425MB;
2. cgroup memory budgets — MemoryHigh/MemoryMax on every service: prevents one leak from taking the box down;
3. OomScoreAdjust tiers — OOM kills edge services first, core services last;
4. 512MB swap backstop — last line of defense, vm.swappiness=10 keeps the kernel preferring RAM.

All four layers combined: 1C1G VPS boots its core path in 30 seconds, monthly OOM events drop from 2–4 to zero.

$0 extra cost. Pure configuration.