Secure Data Deletion

ISO 27018 requires secure, verifiable deletion of PII when it's no longer needed. This lesson covers deletion methods, verification procedures, and implementation strategies for cloud environments.

Why Secure Deletion Matters

ISO 27018 Requirement: "The public cloud PII processor shall securely delete or render unrecoverable PII at the end of the contract or upon customer request."

Key Concerns:

• Prevent data recovery after deletion
• Meet retention policy requirements
• Support "right to be forgotten"
• Enable contract termination cleanup
• Reduce data breach exposure

Deletion Methods

1. Logical Deletion (Soft Delete)

Description: Mark data as deleted without physical removal

Use Cases:

• Temporary deletion with recovery period
• Compliance with retention policies
• Audit trail preservation

Implementation:

interface SoftDeleteRecord {
  id: string;
  data: any;
  deleted_at?: Date;
  deleted_by?: string;
  permanent_deletion_date?: Date;
}

class SoftDeleteService {
  async softDelete(recordId: string, userId: string): Promise<void> {
    const permanentDeletionDate = new Date();
    permanentDeletionDate.setDate(permanentDeletionDate.getDate() + 30); // 30-day grace period

    await this.database.update('pii_records', recordId, {
      deleted_at: new Date(),
      deleted_by: userId,
      permanent_deletion_date: permanentDeletionDate
    });

    // Schedule permanent deletion
    await this.scheduler.schedule(permanentDeletionDate, 'permanent_delete', {
      recordId,
      tableName: 'pii_records'
    });

    await this.auditLog.record('SOFT_DELETE', {
      recordId,
      userId,
      scheduledPermanentDeletion: permanentDeletionDate
    });
  }

  // Query filter to exclude soft-deleted records
  async query(conditions: any): Promise<any[]> {
    return await this.database.query({
      ...conditions,
      deleted_at: null // Exclude soft-deleted
    });
  }
}

2. Physical Deletion (Hard Delete)

Description: Permanently remove data from storage

Use Cases:

• End of retention period
• Confirmed deletion request
• Contract termination
• "Right to be forgotten" compliance

Implementation:

class PhysicalDeleteService {
  async hardDelete(recordId: string): Promise<DeletionCertificate> {
    // 1. Delete from primary database
    const primaryDeleted = await this.deletePrimaryRecord(recordId);

    // 2. Delete from all replicas
    const replicasDel ted = await this.deleteFromReplicas(recordId);

    // 3. Delete from backups (mark for purge on next backup cycle)
    await this.markBackupsForPurge(recordId);

    // 4. Delete from search indexes
    await this.deleteFromSearchIndex(recordId);

    // 5. Delete from caches
    await this.deleteFromCache(recordId);

    // 6. Notify sub-processors
    await this.notifySubProcessorsToDelete(recordId);

    // 7. Generate deletion certificate
    const certificate = await this.generateDeletionCertificate({
      recordId,
      deletionDate: new Date(),
      method: 'physical_deletion',
      locationsDeleted: [
        'primary_database',
        ...replicasDe leted,
        'search_index',
        'cache'
      ],
      verificationHash: this.calculateVerificationHash(recordId)
    });

    // 8. Audit log
    await this.auditLog.record('HARD_DELETE', {
      recordId,
      certificate: certificate.id,
      timestamp: new Date()
    });

    return certificate;
  }

  private async deleteFromReplicas(recordId: string): Promise<string[]> {
    const replicas = await this.getReplicaLocations();
    const deleted: string[] = [];

    for (const replica of replicas) {
      await replica.delete(recordId);
      deleted.push(replica.name);
    }

    return deleted;
  }
}

3. Cryptographic Erasure

Description: Destroy encryption keys, making encrypted data unrecoverable

Use Cases:

• Cloud storage deletion
• Large-scale deletions
• High-security requirements
• Hardware disposal

Implementation:

class CryptographicErasure {
  async cryptoErase(customerId: string): Promise<void> {
    // 1. Identify all encryption keys for customer
    const keys = await this.keyManagement.getCustomerKeys(customerId);

    // 2. Verify no other customers use these keys
    const exclusive = await this.verifyExclusiveKeyUsage(keys, customerId);
    if (!exclusive) {
      throw new Error('Keys shared with other customers - cannot crypto-erase');
    }

    // 3. Delete encryption keys from key management system
    for (const key of keys) {
      await this.keyManagement.deleteKey(key.id);
      await this.auditLog.record('KEY_DELETED', {
        keyId: key.id,
        customerId,
        method: 'cryptographic_erasure'
      });
    }

    // 4. Mark encrypted data for cleanup
    // (Optional: data now unrecoverable but can be cleaned up for storage)
    await this.markEncryptedDataForCleanup(customerId);

    // 5. Certificate
    await this.generateErasureCertificate(customerId, keys);
  }

  // Key-per-customer architecture for crypto-erasure
  async encryptWithCustomerKey(
    customerId: string,
    data: Buffer
  ): Promise<EncryptedData> {
    const customerKey = await this.keyManagement.getOrCreateCustomerKey(
      customerId
    );

    return await this.encrypt(data, customerKey);
  }
}

4. Secure Overwrite

Description: Overwrite data multiple times with random data

Use Cases:

• Physical media disposal
• VM/container cleanup
• High-security requirements
• Regulatory compliance (DOD 5220.22-M)

Implementation:

class SecureOverwriteService {
  async secureOverwrite(
    filePath: string,
    passes: number = 3
  ): Promise<OverwriteResult> {
    const fileSize = await this.getFileSize(filePath);
    const blockSize = 4096; // 4KB blocks

    for (let pass = 0; pass < passes; pass++) {
      const pattern = this.getOverwritePattern(pass);

      for (let offset = 0; offset < fileSize; offset += blockSize) {
        const data = this.generatePattern(pattern, blockSize);
        await this.writeBlock(filePath, offset, data);
      }

      // Verify overwrite
      await this.verifyOverwrite(filePath, pattern);
    }

    // Final pass: random data
    await this.overwriteWithRandom(filePath);

    // Delete file
    await fs.unlink(filePath);

    return {
      filePath,
      passes,
      verified: true,
      timestamp: new Date()
    };
  }

  private getOverwritePattern(pass: number): OverwritePattern {
    // DOD 5220.22-M pattern
    const patterns = [
      'zeros',    // Pass 1: All zeros
      'ones',     // Pass 2: All ones
      'random'    // Pass 3: Random
    ];
    return patterns[pass % 3];
  }
}

Cloud-Specific Deletion

AWS Deletion

class AWSDeletionService {
  async deleteS3Objects(bucket: string, prefix: string): Promise<void> {
    const s3 = new AWS.S3();

    // List all objects
    const objects = await this.listAllObjects(bucket, prefix);

    // Delete in batches
    for (let i = 0; i < objects.length; i += 1000) {
      const batch = objects.slice(i, i + 1000);
      await s3.deleteObjects({
        Bucket: bucket,
        Delete: {
          Objects: batch.map(obj => ({ Key: obj.Key }))
        }
      }).promise();
    }

    // Verify deletion
    const remaining = await this.listAllObjects(bucket, prefix);
    if (remaining.length > 0) {
      throw new Error('Deletion verification failed');
    }
  }

  async deleteRDSSnapshots(dbInstanceId: string): Promise<void> {
    const rds = new AWS.RDS();
    const snapshots = await rds.describeDBSnapshots({
      DBInstanceIdentifier: dbInstanceId
    }).promise();

    for (const snapshot of snapshots.DBSnapshots) {
      await rds.deleteDBSnapshot({
        DBSnapshotIdentifier: snapshot.DBSnapshotIdentifier
      }).promise();
    }
  }

  async deleteEBSVolumes(volumeIds: string[]): Promise<void> {
    const ec2 = new AWS.EC2();

    for (const volumeId of volumeIds) {
      // Detach if attached
      await this.detachVolume(volumeId);

      // Delete volume
      await ec2.deleteVolume({ VolumeId: volumeId }).promise();
    }
  }
}

Azure Deletion

class AzureDeletionService {
  async deleteBlobStorage(
    containerName: string,
    blobPrefix: string
  ): Promise<void> {
    const blobService = azure.createBlobService();

    // List blobs
    const blobs = await this.listAllBlobs(containerName, blobPrefix);

    // Delete blobs
    for (const blob of blobs) {
      await blobService.deleteBlob(containerName, blob.name);
    }

    // Delete container if empty
    const isEmpty = await this.isContainerEmpty(containerName);
    if (isEmpty) {
      await blobService.deleteContainer(containerName);
    }
  }

  async deleteSQLDatabase(serverName: string, dbName: string): Promise<void> {
    // Delete database
    await this.sqlClient.databases.delete(resourceGroup, serverName, dbName);

    // Delete backups
    await this.deleteBackups(serverName, dbName);
  }
}

Google Cloud Deletion

class GCPDeletionService {
  async deleteGCSObjects(bucketName: string, prefix: string): Promise<void> {
    const storage = new Storage();
    const bucket = storage.bucket(bucketName);

    // Delete objects with prefix
    await bucket.deleteFiles({ prefix });

    // Verify deletion
    const [files] = await bucket.getFiles({ prefix });
    if (files.length > 0) {
      throw new Error('Deletion verification failed');
    }
  }

  async deleteCloudSQLInstance(instanceId: string): Promise<void> {
    const sqladmin = google.sqladmin('v1beta4');

    // Delete instance (includes all data and backups)
    await sqladmin.instances.delete({
      project: projectId,
      instance: instanceId
    });
  }
}

Backup and Archive Deletion

Backup Purge Strategy

class BackupPurgeService {
  async purgeFromBackups(recordId: string): Promise<void> {
    // 1. Identify backups containing the record
    const backups = await this.findBackupsContainingRecord(recordId);

    // 2. For each backup
    for (const backup of backups) {
      if (this.canModifyBackup(backup)) {
        // Remove specific record from backup
        await this.removeRecordFromBackup(backup, recordId);
      } else {
        // Mark backup for deletion at next cycle
        await this.markBackupForFullDeletion(backup);
      }
    }

    // 3. Verify purge
    await this.verifyRecordNotInBackups(recordId);
  }

  private async removeRecordFromBackup(
    backup: Backup,
    recordId: string
  ): Promise<void> {
    // Restore backup to temporary location
    const tempLocation = await this.restoreBackup(backup);

    // Delete specific record
    await this.deleteRecordFromTemp(tempLocation, recordId);

    // Create new backup without the record
    await this.createModifiedBackup(tempLocation, backup);

    // Delete old backup
    await this.deleteBackup(backup);

    // Cleanup temp
    await this.cleanupTemp(tempLocation);
  }
}

Deletion Verification

Verification Process

interface DeletionVerification {
  recordId: string;
  verificationType: 'primary' | 'replicas' | 'backups' | 'caches' | 'sub-processors';
  verified: boolean;
  timestamp: Date;
  verifiedBy: string;
  method: string;
}

class DeletionVerificationService {
  async verifyDeletion(recordId: string): Promise<DeletionReport> {
    const verifications: DeletionVerification[] = [];

    // 1. Verify primary database
    verifications.push(await this.verifyPrimaryDatabase(recordId));

    // 2. Verify replicas
    verifications.push(await this.verifyReplicas(recordId));

    // 3. Verify backups
    verifications.push(await this.verifyBackups(recordId));

    // 4. Verify caches
    verifications.push(await this.verifyCaches(recordId));

    // 5. Verify search indexes
    verifications.push(await this.verifySearchIndexes(recordId));

    // 6. Verify sub-processors
    verifications.push(await this.verifySubProcessors(recordId));

    const allVerified = verifications.every(v => v.verified);

    return {
      recordId,
      verifications,
      fullyDeleted: allVerified,
      timestamp: new Date()
    };
  }

  private async verifyPrimaryDatabase(recordId: string): Promise<DeletionVerification> {
    const exists = await this.database.exists('pii_records', { id: recordId });

    return {
      recordId,
      verificationType: 'primary',
      verified: !exists,
      timestamp: new Date(),
      verifiedBy: 'automated_verification',
      method: 'database_query'
    };
  }
}

Deletion Certificate

Generating Certificates

interface DeletionCertificate {
  certificateId: string;
  customerId: string;
  recordIds: string[];
  deletionMethod: 'physical' | 'cryptographic' | 'overwrite';
  deletionDate: Date;
  locationsDeleted: string[];
  verificationResults: DeletionVerification[];
  certificateHash: string;
  signedBy: string;
  digitalSignature: string;
}

class DeletionCertificateService {
  async generateCertificate(
    customerId: string,
    deletionReport: DeletionReport
  ): Promise<DeletionCertificate> {
    const certificate: DeletionCertificate = {
      certificateId: generateId(),
      customerId,
      recordIds: deletionReport.recordIds,
      deletionMethod: deletionReport.method,
      deletionDate: new Date(),
      locationsDeleted: deletionReport.locations,
      verificationResults: deletionReport.verifications,
      certificateHash: '',
      signedBy: 'Cloud Service Provider',
      digitalSignature: ''
    };

    // Calculate hash
    certificate.certificateHash = this.calculateHash(certificate);

    // Digital signature
    certificate.digitalSignature = await this.signCertificate(
      certificate.certificateHash
    );

    // Store certificate
    await this.storeCertificate(certificate);

    // Send to customer
    await this.sendCertificateToCustomer(customerId, certificate);

    return certificate;
  }

  async verifyCertificate(certificateId: string): Promise<boolean> {
    const certificate = await this.getCertificate(certificateId);
    const calculatedHash = this.calculateHash(certificate);

    return calculatedHash === certificate.certificateHash &&
           await this.verifySignature(certificate.digitalSignature, calculatedHash);
  }
}

Sub-processor Deletion

Ensuring Complete Deletion Chain

class SubProcessorDeletionService {
  async notifySubProcessorsToDelete(recordId: string): Promise<void> {
    const subProcessors = await this.getSubProcessorsWithAccess(recordId);

    const deletionRequests: Promise<void>[] = [];

    for (const sp of subProcessors) {
      deletionRequests.push(
        this.sendDeletionRequest(sp, recordId)
      );
    }

    await Promise.all(deletionRequests);

    // Verify deletion with sub-processors
    await this.verifySubProcessorDeletion(recordId, subProcessors);
  }

  private async sendDeletionRequest(
    subProcessor: SubProcessor,
    recordId: string
  ): Promise<void> {
    const response = await axios.post(
      `${subProcessor.apiUrl}/data-deletion`,
      {
        requestId: generateId(),
        recordId,
        deletionDeadline: addDays(new Date(), 30)
      },
      {
        headers: {
          'Authorization': `Bearer ${subProcessor.apiKey}`
        }
      }
    );

    await this.trackDeletionRequest(subProcessor.id, recordId, response.data);
  }
}

Automated Deletion Workflows

Retention-Based Auto-Deletion

class AutomatedDeletionService {
  async runDeletionJob(): Promise<DeletionJobResult> {
    const deletionDate = new Date();
    const results: DeletionJobResult = {
      jobId: generateId(),
      startTime: deletionDate,
      recordsProcessed: 0,
      recordsDeleted: 0,
      errors: []
    };

    // 1. Find records past retention period
    const recordsToDelete = await this.findExpiredRecords();

    // 2. Process deletions
    for (const record of recordsToDelete) {
      try {
        await this.deleteRecord(record.id);
        results.recordsDeleted++;
      } catch (error) {
        results.errors.push({
          recordId: record.id,
          error: error.message
        });
      }
      results.recordsProcessed++;
    }

    // 3. Log job results
    await this.logDeletionJob(results);

    return results;
  }

  private async findExpiredRecords(): Promise<any[]> {
    return await this.database.query(`
      SELECT id, retention_period, created_at
      FROM pii_records
      WHERE deleted_at IS NULL
      AND created_at + INTERVAL retention_period DAY < NOW()
    `);
  }
}

// Schedule daily
cron.schedule('0 2 * * *', async () => {
  await automatedDeletionService.runDeletionJob();
});

Compliance Checklist

Deletion Implementation:

• Multiple deletion methods available (soft, hard, crypto-erasure)
• Deletion covers all data locations (primary, replicas, backups, caches)
• Sub-processor deletion notifications implemented
• Deletion verification process in place
• Deletion certificates generated and provided
• Automated deletion based on retention policies
• Backup purge procedures documented
• "Right to be forgotten" requests supported
• Deletion audit trail comprehensive
• Deletion SLAs defined and monitored

Documentation:

• Deletion procedures documented
• Data flow diagrams show all storage locations
• Verification procedures documented
• Certificate generation process documented
• Sub-processor agreements include deletion obligations

Next Lesson: Sub-processor management - selecting, monitoring, and controlling third-party PII processors.