#49 — Compliance Check Automatique

PLANIFIÉ

Priorité: 🟠 HAUTE · Type: TYPE C · Conteneur: rgz-beat · Code: app/tasks/compliance.pyDépendances: #47 arcep-reporting

Description

Vérification automatique hebdomadaire (chaque lundi 02:00 UTC) de 14 points conformité critiques. La Celery task rgz.compliance.check valide l'infrastructure et génère un rapport JSON avec score 0-14. Si score <13 : dashboard admin devient ROUGE, SMS alerte envoyé immédiatement.

Les 14 points couvrent : sécurité (SSL, logs intégrité, audit trail), légal (APDP consentements, ARCEP préparation), opérationnel (backup, services running, pas d'orphelins), performance (P0 alerte count, latence). Chaque point est vérifié en <30s max (timeout). Un détail explique chaque échec pour aide au debugging.

Rapport accessible via endpoint /api/v1/compliance/report. Historique sauvegardé pour trending sur 12 mois.

Architecture Interne

1. Trigger Celery (weekly):
   └─> rgz.compliance.check
       ├─> Schedule: "0 2 * * 1" (Lundi 02:00 UTC)
       ├─> Queue: rgz.compliance
       ├─> Timeout: 900s (15min max)
       └─> Retry: 3x en cas d'erreur

2. Boucle des 14 checks:

   ┌─ CHECK 1: SSL Certificate Valid
   │  └─> Vérifier /etc/letsencrypt/live/*/cert.pem
   │      ├─> Lecture expiry: openssl x509 -enddate
   │      ├─> Comparaison: expiry_date > NOW + 30 days
   │      ├─ PASS: verde
   │      └─ FAIL: rouge (expiry <30j)

   ├─ CHECK 2: APDP Consentements
   │  └─> SELECT COUNT(*) FROM apdp_consents WHERE consent_given = true
   │      ├─> Ratio: (avec_consentement / total_subscribers) >= 80%
   │      ├─ PASS: >=80%
   │      └─ FAIL: <80%

   ├─ CHECK 3: Logs Intégrité (SHA-256)
   │  └─> Vérification immuable_logs (#45)
   │      ├─> Sample 1000 random logs
   │      ├─> Replay hash: SHA256(ts + data + prev_hash)
   │      ├─> Comparaison avec stored_hash
   │      ├─ PASS: 100% match
   │      └─ FAIL: 1+ mismatch détecté

   ├─ CHECK 4: Backup Récent
   │  └─> Vérifier dernière backup PostgreSQL (#75)
   │      ├─> File: /backups/pg_dump_latest.sql.gz
   │      ├─> Modified: < 24h
   │      ├─ PASS: backup <24h
   │      └─ FAIL: backup >24h ou missing

   ├─ CHECK 5: RADIUS Orphans
   │  └─> SELECT COUNT(*) FROM radius_sessions
   │      WHERE subscriber_id NOT IN (SELECT id FROM subscribers)
   │      ├─ PASS: count = 0
   │      └─ FAIL: count > 0 (orphelins à nettoyer)

   ├─ CHECK 6: Vouchers Expired Cleaned
   │  └─> SELECT COUNT(*) FROM vouchers WHERE status IN ('expired', 'revoked')
   │      ├─> Ratio: (expired+revoked) / total <= 1%
   │      ├─ PASS: <=1%
   │      └─ FAIL: >1% (cleanup needed)

   ├─ CHECK 7: Invoices Générées (SLA J+5)
   │  └─> Vérifier factures du mois dernier
   │      ├─> SELECT COUNT(*) FROM invoices WHERE created_at >= LAST_MONTH
   │      ├─ PASS: count = COUNT(transactions_month)
   │      └─ FAIL: invoices missing

   ├─ CHECK 8: ARCEP Rapports Préparés
   │  └─> Vérifier ArchepReport pour trimestre actuel
   │      ├─> SELECT 1 FROM arcep_reports WHERE quarter=Q2 AND year=2026
   │      ├─ PASS: exists
   │      └─ FAIL: missing

   ├─ CHECK 9: P0 Alerts Count
   │  └─> SELECT COUNT(*) FROM alerts WHERE priority = P0 AND is_active = true
   │      ├─ PASS: count = 0
   │      └─ FAIL: >0 alerte P0 active

   ├─ CHECK 10: Suricata Running
   │  └─> Vérifier conteneur + process suricata (#8)
   │      ├─> docker ps | grep rgz-ids
   │      ├─> prometheus: suricata_up metric
   │      ├─ PASS: 1 (running)
   │      └─ FAIL: 0 (stopped/error)

   ├─ CHECK 11: ELK Status Green
   │  └─> Requête Elasticsearch cluster health API (#40)
   │      ├─> GET _cluster/health
   │      ├─> Status: green (all shards replicated)
   │      ├─ PASS: green
   │      └─ FAIL: yellow ou red

   ├─ CHECK 12: DNS Sinkhole Actif
   │  └─> Tester requête DNS vers domaine non-whitelist (#34)
   │      ├─> nslookup example.com 10.0.0.9
   │      ├─> Vérifier réponse = IP portail (10.0.0.1)
   │      ├─ PASS: retourne IP portail
   │      └─ FAIL: résolution normal (sinkhole DOWN)

   ├─ CHECK 13: Audit Trail Continu
   │  └─> Vérifier audit_logs (#48) sans gap
   │      ├─> SELECT MIN(created_at), MAX(created_at) FROM audit_logs
   │      ├─> Diff: < 10min (logs venant de maintenant)
   │      ├─ PASS: derniers logs < 10min
   │      └─ FAIL: dernier log > 10min (middleware broken?)

   └─ CHECK 14: APDP Demandes Traitées <90j
      └─> SELECT COUNT(*) FROM apdp_forget_requests
          WHERE status = FAILED OR (status = PROCESSED AND anonymized_at > NOW - 90 DAYS)
          ├─ PASS: 0 failures OU tous les PROCESSED <90j
          └─ FAIL: >0 failures OU quelques demandes >90j

3. Scoring:
   └─> Pass = 1 point (max 14)
       └─> Score = SUM(pass_count)
       ├─ 14/14: excellent (dashboard GREEN)
       ├─ 13/14: acceptable (dashboard YELLOW)
       ├─ <13/14: critique (dashboard RED, SMS admin)

4. Stockage rapport:
   └─> INSERT compliance_checks(
           check_date,
           ssl_cert_valid, apdp_consents_ok, logs_integrity_ok, backup_recent,
           radius_orphans_count, vouchers_expired_cleaned, invoices_generated_ok,
           arcep_prepared, alerts_p0_count, suricata_running, elk_status,
           dns_sinkhole_ok, audit_trail_continuous, apdp_requests_processed_ok,
           score, details
       )
   └─> details = {
         "ssl_cert": {"status": "pass", "message": "Expires 2026-05-20"},
         "apdp_consents": {"status": "pass", "ratio": 0.95},
         "logs_integrity": {"status": "fail", "message": "1 hash mismatch detected", "log_ids": ["xxx"]},
         ...
       }

5. Alertes:
   ├─> Si score < 13:
   │   ├─ SMS admin: "⚠️ RGZ Compliance: score 12/14 — check panel"
   │   ├─ Email admin avec détail rapport
   │   └─ Update dashboard: toggle RED
   │
   └─> Pour chaque FAIL:
       ├─ Suricata DOWN → SMS immédiatement
       ├─ ELK RED → SMS immédiatement
       ├─ P0 alerte active → SMS immédiatement (mais c'est #58 incident escalation)
       └─ SSL <15j → SMS remind (pas FAIL, juste INFO)

6. Endpoint API:
   └─> GET /api/v1/compliance/report
       ├─> Récupérer dernier report
       └─> JSON: {score, items: [{check, status, message}], generated_at}

7. Historique et trending:
   └─> Garder last 52 rapports (1 an, weekly)
   └─> Grafana dashboard: score over time, failure trending

Configuration

Variables d'environnement

env

COMPLIANCE_CHECK_ENABLED=true         # Activation
COMPLIANCE_CHECK_SCHEDULE=0 2 * * 1   # Lundi 02:00 UTC
COMPLIANCE_CHECK_TIMEOUT=900          # 15min max
COMPLIANCE_ALERT_SMS_ENABLED=true     # Alertes SMS si score <13
COMPLIANCE_ALERT_ADMIN_PHONE=...      # Numéro admin
SSL_CERT_WARNING_DAYS=30              # Alert si expire <30j
BACKUP_MAX_AGE_HOURS=24               # Max age backup avant FAIL
AUDIT_TRAIL_MAX_GAP_MINUTES=10        # Max gap logs
APDP_FORGET_TIMEOUT_DAYS=90           # Max délai traitement demandes
COMPLIANCE_HISTORY_RETENTION_DAYS=730 # 2 ans d'historique

Model SQLAlchemy

python

# app/models/compliance.py

class ComplianceCheck(Base):
    __tablename__ = "compliance_checks"

    id = Column(UUID(as_uuid=True), primary_key=True, default=uuid4)
    check_date = Column(DateTime, nullable=False, index=True)

    # 14 checks
    ssl_cert_valid = Column(Boolean, nullable=False)
    apdp_consents_ok = Column(Boolean, nullable=False)
    logs_integrity_ok = Column(Boolean, nullable=False)
    backup_recent = Column(Boolean, nullable=False)
    radius_orphans_ok = Column(Boolean, nullable=False)
    vouchers_expired_cleaned = Column(Boolean, nullable=False)
    invoices_generated_ok = Column(Boolean, nullable=False)
    arcep_prepared = Column(Boolean, nullable=False)
    alerts_p0_ok = Column(Boolean, nullable=False)  # no P0 alerts
    suricata_running = Column(Boolean, nullable=False)
    elk_status_ok = Column(Boolean, nullable=False)
    dns_sinkhole_ok = Column(Boolean, nullable=False)
    audit_trail_continuous = Column(Boolean, nullable=False)
    apdp_requests_processed_ok = Column(Boolean, nullable=False)

    # Counts for detail
    radius_orphans_count = Column(Integer, default=0)
    alerts_p0_count = Column(Integer, default=0)
    vouchers_expired_percent = Column(Float, default=0)

    # Overall
    score = Column(Integer, nullable=False)  # 0-14
    details = Column(JSON, nullable=False)  # {check_name: {status, message, details}}
    generated_at = Column(DateTime, default=utcnow, nullable=False)
    created_at = Column(DateTime, default=utcnow, nullable=False)

    __table_args__ = (
        Index('ix_check_date', 'check_date'),
    )

Endpoints API

Méthode	Route	Réponse	Auth
GET	`/api/v1/compliance/report`	200 latest report	Admin
GET	`/api/v1/compliance/history?from=&to=`	200 `{items: [...], total}`	Admin
POST	`/api/v1/compliance/check/manual`	202 `{job_id}`	Admin

Commandes Utiles

bash

# Voir dernier rapport conformité
curl -X GET http://localhost:8000/api/v1/compliance/report \
  -H "Authorization: Bearer $ADMIN_TOKEN" | jq

# Voir historique (tendance)
curl -X GET 'http://localhost:8000/api/v1/compliance/history?from=2026-02-01&to=2026-02-21' \
  -H "Authorization: Bearer $ADMIN_TOKEN" | jq

# Lancer check manuel (ne pas attendre lundi)
curl -X POST http://localhost:8000/api/v1/compliance/check/manual \
  -H "Authorization: Bearer $ADMIN_TOKEN"

# Requête DB: voir scores tendance
psql -h localhost -U rgz_admin -d rgz_db -c \
  "SELECT check_date, score FROM compliance_checks ORDER BY check_date DESC LIMIT 10;"

# Voir détail un report
psql -h localhost -U rgz_admin -d rgz_db -c \
  "SELECT check_date, score, details FROM compliance_checks
   WHERE check_date = '2026-02-16 02:00:00' \G"

Implémentation TODO

[ ] Classe ComplianceCheckTask dans app/tasks/compliance.py
[ ] Fonction check_ssl_cert() (openssl x509 -enddate)
[ ] Fonction check_apdp_consents() (SELECT COUNT ratio)
[ ] Fonction check_logs_integrity() (vérif SHA-256 sample)
[ ] Fonction check_backup_recent() (file timestamp)
[ ] Fonction check_radius_orphans() (COUNT sans subscribers)
[ ] Fonction check_vouchers_expired() (ratio <1%)
[ ] Fonction check_invoices_generated() (count last month)
[ ] Fonction check_arcep_prepared() (SELECT trimestre actuel)
[ ] Fonction check_alerts_p0() (COUNT active)
[ ] Fonction check_suricata_running() (docker + prometheus metric)
[ ] Fonction check_elk_status() (cluster health API)
[ ] Fonction check_dns_sinkhole() (nslookup test)
[ ] Fonction check_audit_trail() (SELECT last log timestamp)
[ ] Fonction check_apdp_requests() (failures/timeout)
[ ] Celery task registration: rgz.compliance.check
[ ] Endpoint GET /api/v1/compliance/report (latest)
[ ] Endpoint GET /api/v1/compliance/history (trending)
[ ] Endpoint POST /api/v1/compliance/check/manual
[ ] SMS alerting si score <13
[ ] Grafana dashboard: compliance score over time
[ ] Tests: tester chaque check, vérifier score calcul

Dernière mise à jour: 2026-02-21

#49 — Compliance Check Automatique ​

Description ​

Architecture Interne ​

Configuration ​

Variables d'environnement ​

Model SQLAlchemy ​

Endpoints API ​

Commandes Utiles ​

Implémentation TODO ​