FM-KED-004 — Backup Failure Due to Excessive Backup Size

Severity: S1 — Critical Recovery Class: B — Standard Recovery Covered by Monthly Support: Yes 

 

 Description 

 Automatic daily backups fail because the generated database dump exceeds available disk capacity on the backup worker or application node. 

 This condition usually indicates uncontrolled growth of historical data and must be addressed immediately to restore backup continuity. 

 

 Typical Symptoms 

 Daily backup job not completed or failing repeatedly Backup files missing or incomplete Disk space exhaustion on backup or worker node Alerts indicating failed or skipped backup runs 

 

 Root Cause 

 Excessive volume of historical records retained in the database Insufficient disk capacity on the worker node where the Authorizer VM is deployed, or the Finmars Community Edition is installed 

 

 Diagnostic Checklist 

 Verify Backup Job Status 

 Confirm backup job execution logs Identify failure reason and timestamps 

 Check Disk Space on Backup Node 

 df -h 

 Estimate Database Dump Size 

 pg_dump --format=custom --file=/tmp/test.dump db_name

ls -lh /tmp/test.dump 

 

 Recovery Options 

 Choose one or a combination, depending on business requirements. 

 

 Option 1: Remove Obsolete Historical Records 

 Identify historical tables with excessive row counts Confirm data retention requirements Delete or archive obsolete records Re-run backup after cleanup 

 ⚠️ Data deletion is irreversible and must be explicitly approved. 

 

 Option 2: Increase Disk Capacity on Backup Worker Node 

 Extend disk size of the worker node Ensure sufficient free space for full backup generation Re-run the backup job and verify completion 

 This option preserves all historical data. 

 

 Preventive Notes 

 Define and enforce data retention policies Monitor backup file sizes over time Monitor free disk space on backup and worker nodes Periodically validate backup completion, not only existence 

 

 Responsibility Boundary 

 Finmars SCSA provides diagnostics, recommendations, and operational guidance. Infrastructure changes such as disk resizing may depend on customer approval or cloud provider action.