Introduction
In VPS administration, security is often the most daunting challenge.
You install a new service, configure a firewall rule, and then… nothing. Are ports accidentally exposed? Does the system have unpatched critical vulnerabilities? Is SSH under brute-force attack? Do Docker containers have configuration flaws? These security issues won’t proactively bother you, but they could destroy your server at 3 AM.
Traditional security tools (Fail2ban, CrowdSec, OpenVAS) can tell you “there might be a risk,” but they can’t:
- Understand the cascading impact of interconnected vulnerabilities
- Assess risk priority based on your business context
- Provide actionable, step-by-step remediation plans
- Filter real threats from thousands of false positive alerts
AI-powered VPS security automation solves these problems. This guide walks you through building a complete AI security operations platform covering four core capabilities: vulnerability scanning, intrusion detection, security hardening, and automated remediation.
Why AI for Security Operations?
The Limits of Traditional Security
| Dimension | Traditional Tools | AI-Enhanced Approach |
|---|---|---|
| Vulnerability Scanning | Periodic scans, results require manual analysis | Real-time scanning + LLM-based impact assessment |
| Intrusion Detection | Rule-based matching, high false positive rate | Behavior analysis + LLM semantic understanding |
| Remediation Advice | Generic patch suggestions | Business-context-aware fix recommendations |
| Security Compliance | Manual checklist auditing | Automated audits + deviation alerts |
| Threat Intelligence | Manual CVE subscription | Real-time threat report analysis + risk scoring |
AI Security Operations Architecture
┌──────────────────────────────────────────────────────────────┐
│ AI Security Operations Platform │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Vuln Scanner │ │ Intrusion │ │ Config Audit │ │
│ │ (Nuclei) │ │ Detection │ │ (Lynis) │ │
│ │ │ │ (ClamAV + │ │ │ │
│ │ │ │ LLM) │ │ │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────────────┐ │
│ │ LLM Security Analysis Engine │ │
│ │ • Vulnerability correlation • Threat scoring │ │
│ │ • Fix prioritization • Auto script gen │ │
│ └─────────────────────┬───────────────────────────┘ │
│ │ │
│ ┌────────────┼────────────┐ │
│ ▼ ▼ ▼ │
│ ┌───────────┐ ┌───────────┐ ┌───────────┐ │
│ │ Telegram │ │ Email │ │ Auto-Fix │ │
│ │ Instant │ │ Daily │ │ Workflow │ │
│ │ Alerts │ │ Reports │ │ │ │
│ └───────────┘ └───────────┘ └───────────┘ │
└──────────────────────────────────────────────────────────────┘
Step 1: Automated Vulnerability Scanning
Vulnerability scanning is the first line of defense in security operations. Traditional scanners only list CVE IDs, while AI helps you understand how those vulnerabilities actually affect your specific workload.
Option A: Nuclei + LLM Risk Assessment (Recommended)
Nuclei is a fast and flexible vulnerability scanner supporting YAML templates. We combine it with a local LLM for intelligent risk assessment.
Install Nuclei
# Install nuclei
curl -sSfL https://raw.githubusercontent.com/projectdiscovery/nuclei/main/install.sh | sh -s -- -b /usr/local/bin
# Update templates
nuclei -update-templates
# Verify installation
nuclei --version
Create an AI Risk Assessment Workflow
Create a Python script that feeds Nuclei scan results to an LLM for analysis:
#!/usr/bin/env python3
"""Nuclei + LLM Security Risk Assessment Workflow"""
import json
import subprocess
import sys
from datetime import datetime
def run_nuclei_scan(targets: list) -> list:
"""Run Nuclei scan and return structured results"""
cmd = [
"nuclei",
"-l", "-", # Read targets from stdin
"-jsonl", # JSON lines output
"-severity", "critical,high,medium", # Focus on high-severity vulns
"-tags", "exposed,exposure,cves",
"-rate-limit", "50",
"-burst", "25",
]
process = subprocess.Popen(
cmd,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True
)
targets_input = "\n".join(targets) + "\n"
stdout, stderr = process.communicate(input=targets_input)
results = []
for line in stdout.strip().split("\n"):
if line:
try:
results.append(json.loads(line))
except json.JSONDecodeError:
continue
return results
def analyze_with_llm(scan_results: list, system_info: dict) -> dict:
"""Use LLM to analyze scan results and generate risk assessment report"""
# Build context
vulnerability_summary = []
for r in scan_results:
vulnerability_summary.append({
"template_id": r.get("template-id", ""),
"matched_at": r.get("matched-at", ""),
"severity": r.get("info", {}).get("severity", "unknown"),
"name": r.get("info", {}).get("name", ""),
"description": r.get("info", {}).get("description", ""),
"reference": r.get("info", {}).get("reference", []),
})
context = f"""
System Information:
- OS: {system_info.get('os', 'unknown')}
- Running Services: {json.dumps(system_info.get('services', []))}
- Open Ports: {json.dumps(system_info.get('open_ports', []))}
Discovered Vulnerabilities:
{json.dumps(vulnerability_summary, indent=2, ensure_ascii=False)}
"""
prompt = f"""You are a senior cybersecurity expert. Analyze the following VPS security scan results and provide:
1. **Risk Level Assessment** (High/Medium/Low): Based on actual business context, not just CVSS scores
2. **Vulnerability Correlation**: Are there chained exploitation risks between these vulnerabilities?
3. **Fix Priority Ranking**: Specific remediation steps with commands
4. **False Positive Identification**: Which results might be false positives and why
System Info:
{context}
Output in JSON format:
{{
"risk_level": "high|medium|low",
"summary": "Risk assessment summary (under 200 words)",
"prioritized_actions": [
{{
"priority": 1,
"action": "Specific remediation step",
"command": "Executable command",
"risk_of_not_fixing": "Risk if not fixed",
"estimated_time": "Estimated duration"
}}
],
"false_positive_candidates": ["Vuln ID or description"],
"additional_recommendations": ["Recommendation 1", "Recommendation 2"]
}}
"""
# Call Ollama local LLM
import requests
try:
response = requests.post(
"http://localhost:11434/api/generate",
json={
"model": "llama3.2",
"prompt": prompt,
"stream": False,
"format": {"type": "object"}
},
timeout=60
)
return json.loads(response.json()["response"])
except Exception as e:
print(f"LLM call failed: {e}", file=sys.stderr)
return {"error": str(e), "manual_review_required": True}
if __name__ == "__main__":
# Configure targets
targets = ["http://your-domain.com"]
# System info
system_info = {
"os": "Ubuntu 24.04",
"services": ["nginx", "docker", "ssh"],
"open_ports": [22, 80, 443]
}
# Run scan
print("Running vulnerability scan...")
results = run_nuclei_scan(targets)
print(f"Found {len(results)} potential vulnerabilities")
# AI analysis
print("Analyzing results with LLM...")
analysis = analyze_with_llm(results, system_info)
# Output report
print(json.dumps(analysis, indent=2, ensure_ascii=False))
# Optional: Send Telegram notification
# send_telegram_notification(analysis)
Add Scanning to Cron
# Edit crontab
crontab -e
# Run security scan daily at 2 AM
0 2 * * * /usr/local/bin/ai-security-scan.sh >> /var/log/ai-security-scan.log 2>&1
#!/bin/bash
# /usr/local/bin/ai-security-scan.sh
set -euo pipefail
LOG_DIR="/var/log/ai-security"
mkdir -p "$LOG_DIR"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
echo "[$TIMESTAMP] Starting AI security scan..."
# Execute scan
python3 /opt/security/ai-security-scan.py > "$LOG_DIR/report_${TIMESTAMP}.json"
# Send alert if high risk
RISK_LEVEL=$(python3 -c "
import json, sys
with open('$LOG_DIR/report_${TIMESTAMP}.json') as f:
data = json.load(f)
print(data.get('risk_level', 'unknown'))
" 2>/dev/null || echo "unknown")
if [ "$RISK_LEVEL" = "high" ]; then
curl -X POST "https://api.telegram.org/bot$TG_BOT_TOKEN/sendMessage" \
-d "chat_id=$TG_CHAT_ID" \
-d "text=$(echo "🚨 High-Risk Security Alert - AI scan detected critical vulnerabilities! Report: $LOG_DIR/report_${TIMESTAMP}.json" | base64 -w0)" \
--data-urlencode "parse_mode=HTML"
fi
echo "[$TIMESTAMP] Scan complete"
Option B: Docker Container Security Scanning
If you run Docker on your VPS, container security is equally important:
# Install Trivy container vulnerability scanner
apt-get update && apt-get install -y wget apt-transport-https gnupg lsb-release
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor > /usr/share/keyrings/trivy.gpg
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" > /etc/apt/sources.list.d/trivy.list
apt-get update && apt-get install -y trivy
# Scan all Docker images
trivy image --severity HIGH,CRITICAL --format table $(docker images -q)
# Scan Dockerfiles
trivy config Dockerfile
Pair Trivy’s scan results with LLM analysis to automatically generate fix scripts.
Step 2: Real-Time Intrusion Detection & Smart Alerts
Vulnerability scanning is a static check; intrusion detection provides real-time monitoring. But traditional IDS tools (like OSSEC, Snort) require extensive rule configuration, while AI enables intelligent, behavior-based detection.
Deploy ClamAV + LLM Malware Analysis
# Install ClamAV
apt-get install -y clamav clamav-daemon
# Update virus definitions
freshclam
# Scan a specific directory
clamscan -r --infected /var/www/
Intelligent Log-Based Intrusion Detection
Instead of deploying complex IDS, use an LLM to directly analyze system logs for semantic-level intrusion detection:
#!/usr/bin/env python3
"""AI-Driven System Log Intrusion Detection"""
import subprocess
import json
from datetime import datetime, timedelta
def collect_recent_logs(hours: int = 24) -> str:
"""Collect system logs from the last N hours"""
cmd = ["journalctl", "--since", f"{hours} hours ago", "-q"]
result = subprocess.run(cmd, capture_output=True, text=True)
return result.stdout
def analyze_log_threats(logs: str) -> dict:
"""Use LLM to analyze security threats in logs"""
# Pre-filter: extract only security-related logs
security_keywords = [
"failed", "unauthorized", "invalid", "denied",
"error", "attack", "brute", "injection", "overflow",
"escalation", "privilege", "suspicious", "malware"
]
relevant_logs = []
for line in logs.split("\n"):
if any(kw.lower() in line.lower() for kw in security_keywords):
relevant_logs.append(line)
if not relevant_logs:
return {"threats_found": False, "message": "No security threats detected"}
context = f"""
System detected the following security-related events in the last {24} hours:
{' '.join(relevant_logs[:50])} # Analyze up to 50 entries
"""
prompt = f"""You are a SOC (Security Operations Center) analyst. Analyze the following system logs and identify potential security threats:
{context}
Provide:
1. **Threat Type**: Brute force? SQL injection? Privilege escalation? Malware?
2. **Attack Source**: IP addresses, users, processes
3. **Attack Stage**: Reconnaissance, weaponization, exploitation, lateral movement
4. **Urgency**: Immediate action required / Monitor / Can ignore
5. **Recommended Actions**: Specific IP blocking, user review, process termination commands
Output in JSON format."""
import requests
try:
response = requests.post(
"http://localhost:11434/api/generate",
json={
"model": "llama3.2",
"prompt": prompt,
"stream": False
},
timeout=60
)
return json.loads(response.json()["response"])
except Exception as e:
return {"error": str(e)}
if __name__ == "__main__":
logs = collect_recent_logs(24)
analysis = analyze_log_threats(logs)
print(json.dumps(analysis, indent=2, ensure_ascii=False))
Auto-Block Malicious IPs
Combine LLM analysis results with Fail2ban/CrowdSec for automated response:
#!/bin/bash
# /usr/local/bin/auto-ban-malicious-ips.sh
# Extract IPs to block from AI analysis report
MALICIOUS_IPS=$(python3 -c "
import json
with open('/var/log/ai-security/last_analysis.json') as f:
data = json.load(f)
for ip in data.get('malicious_ips', []):
print(ip)
" 2>/dev/null)
for ip in $MALICIOUS_IPS; do
# Check if already blocked
if ! iptables -C INPUT -s "$ip" -j DROP 2>/dev/null; then
# Add block rule
iptables -I INPUT -s "$ip" -j DROP
echo "Blocked malicious IP: $ip"
# Add to Fail2ban
if command -v fail2ban-client &> /dev/null; then
fail2ban-client set sshd banip "$ip"
fi
fi
done
Step 3: Automated Security Configuration Auditing
Security hardening isn’t a one-time task — it’s a continuous process. Automated auditing ensures your VPS maintains optimal security posture.
Lynis Automated Auditing
Lynis is a powerful security auditing tool:
# Install Lynis
apt-get install -y lynis
# Run security audit
lynis audit system
# Generate audit report
lynis audit system --audit-mode auditors --profile /profiler/profiles/linux/audit.prf
# Feed results to LLM for analysis
#!/usr/bin/env python3
"""Lynis + LLM Security Audit Analysis"""
import subprocess
import json
import requests
def run_lynis_audit() -> str:
"""Run Lynis audit and capture output"""
result = subprocess.run(
["lynis", "audit", "system", "--quick", "--nointeractive"],
capture_output=True, text=True
)
return result.stdout + result.stderr
def analyze_lynis_output(output: str) -> dict:
"""Use LLM to analyze Lynis audit results"""
prompt = f"""You are a Linux security expert. The Lynis security audit tool produced the following results:
{output[:5000]} # Limit length
Analyze the following aspects:
1. **Overall Security Score**: Interpret Lynis's scoring
2. **Critical Items**: List the most urgent security issues to fix
3. **Fix Commands**: Provide specific remediation commands for each critical item
4. **Best Practice Recommendations**: Security hardening beyond Lynis's report
5. **Compliance Check**: Whether it meets CIS Benchmark requirements
Output in JSON format."""
try:
response = requests.post(
"http://localhost:11434/api/generate",
json={
"model": "llama3.2",
"prompt": prompt,
"stream": False,
"format": {
"type": "object",
"properties": {
"overall_score": {"type": "string"},
"critical_items": {
"type": "array",
"items": {
"type": "object",
"properties": {
"item": {"type": "string"},
"description": {"type": "string"},
"fix_command": {"type": "string"}
}
}
},
"additional_recommendations": {"type": "array", "items": {"type": "string"}},
"compliance_status": {"type": "string"}
}
}
},
timeout=60
)
return json.loads(response.json()["response"])
except Exception as e:
return {"error": str(e)}
if __name__ == "__main__":
output = run_lynis_audit()
analysis = analyze_lynis_output(output)
print(json.dumps(analysis, indent=2, ensure_ascii=False))
Automated Hardening Script
Apply recommended security fixes based on LLM analysis:
#!/bin/bash
# /usr/local/bin/apply-security-hardening.sh
# Apply recommended security hardening measures automatically
echo "🔒 Applying security hardening..."
# 1. SSH hardening
echo "[1/5] SSH security hardening..."
sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config
sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
sed -i 's/^#\?MaxAuthTries.*/MaxAuthTries 3/' /etc/ssh/sshd_config
sed -i 's/^#\?X11Forwarding.*/X11Forwarding no/' /etc/ssh/sshd_config
systemctl restart sshd
# 2. Firewall rules
echo "[2/5] Configuring UFW firewall..."
ufw default deny incoming
ufw default allow outgoing
ufw allow 22/tcp # SSH
ufw allow 80/tcp # HTTP
ufw allow 443/tcp # HTTPS
ufw --force enable
# 3. Kernel security parameters
echo "[3/5] Applying kernel security parameters..."
cat >> /etc/sysctl.conf << 'EOF'
# Network security hardening
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.send_redirects = 0
# Anti-spoofing
net.ipv4.conf.all.bootp_relay = 0
net.ipv4.conf.all.log_martians = 1
# Disable ICMP redirects
net.ipv4.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
EOF
sysctl -p
# 4. Automatic security updates
echo "[4/5] Configuring automatic security updates..."
apt-get install -y unattended-upgrades
cat > /etc/apt/apt.conf.d/50unattended-upgrades << 'EOF'
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}-security";
};
Unattended-Upgrade::Automatic-Reboot "true";
EOF
# 5. File permission checks
echo "[5/5] Fixing file permissions..."
find /etc -type f -name "*.conf" -perm /o+w -exec chmod o-w {} \; 2>/dev/null
find /etc -type f -name "*.cfg" -perm /o+w -exec chmod o-w {} \; 2>/dev/null
# Restart affected services
systemctl restart sshd
echo "✅ Security hardening complete!"
echo "Consider rebooting the server to apply all kernel-level security changes."
Step 4: Building an AI Security Operations Workflow
Integrate all components above into a complete security operations platform.
Docker Compose Orchestration
# /opt/security/ai-security-ops/docker-compose.yml
version: '3.8'
services:
# Security scanning service
security-scanner:
image: projectdiscovery/nuclei:latest
volumes:
- ./reports:/reports
command: >
bash -c "
echo 'http://localhost' |
nuclei -jsonl -tags exposed,exposure -output /reports/latest.jsonl
"
restart: unless-stopped
# LLM analysis service (if Ollama not deployed locally)
# Use local Ollama — no additional container needed
# Alerting service
alerting:
image: python:3.11-slim
volumes:
- ./scripts:/scripts
- ./reports:/reports
command: >
bash -c "
pip install requests &&
python3 /scripts/ai-alert-analyzer.py
"
environment:
- TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN}
- TELEGRAM_CHAT_ID=${TELEGRAM_CHAT_ID}
restart: unless-stopped
# Scheduled tasks
cronjob:
image: alpine:3.19
volumes:
- ./scripts:/scripts
- /var/log:/var/log:ro
entrypoint: ["crond", "-f"]
command: ["sh", "-c", "echo '0 2 * * * /scripts/scan.sh' | crontab - && crond -f"]
Security Dashboard (Optional)
For a visual security dashboard, use Grafana + Prometheus:
# /opt/security/grafana/prometheus/prometheus.yml
global:
scrape_interval: 30s
scrape_configs:
- job_name: 'security'
static_configs:
- targets: ['localhost:9100'] # node_exporter
- job_name: 'fail2ban'
static_configs:
- targets: ['localhost:9253'] # fail2ban exporter
Pair the dashboard with LLM analysis for AI-driven alert explanation in Grafana.
Complete Deployment Process
One-Click Setup Script
#!/bin/bash
# /opt/security/setup-ai-security.sh
# One-click deployment for AI VPS Security Operations Platform
set -euo pipefail
echo "🚀 Deploying AI-powered VPS Security Operations Platform..."
# 1. Update system
echo "📦 [1/6] Updating system..."
apt-get update && apt-get upgrade -y
# 2. Install core tools
echo "🔧 [2/6] Installing security tools..."
apt-get install -y \
nuclei trivy lynis clamav \
fail2ban crowdsec ufw \
jq curl wget
# 3. Create project directories
echo "📁 [3/6] Creating project directories..."
mkdir -p /opt/security/{scripts,reports,templates}
mkdir -p /var/log/ai-security
# 4. Configure Fail2ban
echo "⚙️ [4/6] Configuring Fail2ban..."
cat > /etc/fail2ban/jail.local << 'EOF'
[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 3
banaction = iptables-multiport
[sshd]
enabled = true
maxretry = 3
bantime = 7200
[docker]
enabled = true
port = http,https
filter = docker-auth
logpath = /var/log/docker/*.log
maxretry = 5
EOF
# 5. Create automation scripts
echo "📝 [5/6] Creating automation scripts..."
# Copy the scripts from earlier sections
# 6. Set up scheduled tasks
echo "⏰ [6/6] Setting up scheduled security tasks..."
(crontab -l 2>/dev/null; echo "0 2 * * * /opt/security/scripts/scan.sh >> /var/log/ai-security/cron.log 2>&1") | crontab -
echo ""
echo "🎉 AI Security Operations Platform deployed successfully!"
echo ""
echo "📋 Next steps:"
echo " 1. Ensure Ollama is running: ollama serve &"
echo " 2. Test scan: python3 /opt/security/scripts/ai-security-scan.py"
echo " 3. View reports: ls /var/log/ai-security/"
echo " 4. Apply hardening: bash /opt/security/scripts/apply-security-hardening.sh"
Security Operations Best Practices
1. Defense-in-Depth Strategy
Layer 1: Network → UFW/WAF + GeoIP filtering
Layer 2: Host → Fail2ban + Lynis auditing
Layer 3: Application → Container isolation + least privilege
Layer 4: Analytics → AI log analysis + threat correlation
Layer 5: Response → Auto-blocking + ticket generation
2. Recommended Scan Frequencies
| Scan Type | Frequency | Tool |
|---|---|---|
| Vulnerability Scan | Daily | Nuclei |
| Configuration Audit | Weekly | Lynis |
| Malware Scan | Daily | ClamAV |
| Intrusion Detection | Real-time | LLM log analysis |
| Container Security Scan | Every deploy | Trivy |
| Penetration Testing | Monthly | Manual/Half-auto |
3. Daily Security Checklist
#!/bin/bash
# Daily security checklist
DAILY_CHECKS=(
"Failed SSH logins: $(journalctl -u ssh --since '24 hours ago' | grep -c 'Failed password' || echo 0)"
"New cron jobs: $(find /etc/cron* -mtime -1 2>/dev/null | wc -l)"
"SUID files: $(find / -perm -4000 -type f 2>/dev/null | wc -l)"
"Active connections: $(ss -tnp | grep -c ESTAB || echo 0)"
"Stuck Docker containers: $(docker ps --filter status=created --format '{{.Names}}' 2>/dev/null | wc -l)"
)
for check in "${DAILY_CHECKS[@]}"; do
echo "🔍 $check"
done
Cost Analysis
| Approach | Monthly Cost | Notes |
|---|---|---|
| Self-Hosted AI Security | VPS cost ($5-20/mo) | Local Ollama, zero API fees |
| Cloud Security Services | $50-500+/mo | WAF + IDS + SIEM services |
| Manual Security Ops | Extremely high time cost | Easy to miss critical threats |
The core advantage of a self-hosted AI security platform: deploy once, use infinitely, with no per-call API fees.
Summary
AI-powered VPS security operations doesn’t replace traditional security tools — it makes them smarter:
- Vulnerability Scanning — Nuclei + LLM = Know which vulnerabilities truly matter
- Intrusion Detection — Logs + LLM = Identify real threats from noise
- Configuration Auditing — Lynis + LLM = Auto-fix recommendations, not just problem discovery
- Automated Response — Scripts + LLM = Rapid threat containment
The key insight: let AI do what it’s great at (pattern recognition, semantic understanding) and let humans do what they’re great at (final decision-making, business trade-offs).
📌 Key Takeaway: Security isn’t a one-time task — it’s a continuous process. Integrating AI into security operations gives small teams near-enterprise-grade Security Operations Center capabilities.
Further Reading
- CrowdSec Intrusion Prevention — Community-driven intrusion prevention system
- AI-Driven VPS Self-Healing — Automated diagnosis and server repair
- AI Log Analysis — LLM-based log anomaly detection
- N8N Automation Workflows — Integrate security scan results into automated workflows
