Sensitive Data Protection

Purpose

Define strategies and best practices for protecting sensitive data in logging systems while maintaining observability and compliance requirements.

Scope

In Scope:

• PII (Personally Identifiable Information) protection
• Authentication and authorization data security
• Financial and payment information protection
• Data masking and redaction techniques
• Compliance with privacy regulations (GDPR, CCPA, etc.)
• Secure log transmission and storage

Out of Scope:

• General data encryption techniques
• Infrastructure security measures
• Database security practices
• Network security protocols

Introduction

Protecting sensitive data in logs is critical for maintaining user privacy, regulatory compliance, and organizational security. This document provides guidelines for implementing data protection measures that balance observability needs with privacy requirements.

Sensitive Data Categories

Personally Identifiable Information (PII)

Examples:

• Names, addresses, phone numbers
• Email addresses
• Social security numbers
• Date of birth
• Government identification numbers

Protection Strategies:

• Complete exclusion from logs
• Hash-based pseudonymization
• Tokenization for correlation
• Masking with consistent patterns

Authentication and Authorization Data

Examples:

• Passwords and password hashes
• API keys and tokens
• Session identifiers
• OAuth tokens and secrets

Protection Strategies:

• Never log authentication secrets
• Use session IDs for correlation only
• Log authentication events without credentials
• Implement secure token handling

Financial Information

Examples:

• Credit card numbers
• Bank account information
• Payment processor tokens
• Transaction amounts (context-dependent)

Protection Strategies:

• PCI DSS compliance requirements
• Tokenization for payment data
• Masking card numbers (show only last 4 digits)
• Secure audit trail maintenance

Health Information

Examples:

• Medical records
• Health insurance information
• Biometric data
• Mental health information

Protection Strategies:

• HIPAA compliance requirements
• Complete exclusion from general logs
• Secure audit logging for access
• Role-based access controls

Data Protection Techniques

Data Masking

Character Masking:

{
  "level": "INFO",
  "message": "User login attempt",
  "email": "j***@example.com",
  "timestamp": "2024-01-15T10:30:00.000Z"
}

Pattern-Based Masking:

{
  "level": "INFO",
  "message": "Payment processed",
  "cardNumber": "****-****-****-1234",
  "amount": "$XX.XX"
}

Tokenization

User Identification:

{
  "level": "INFO",
  "message": "User action performed",
  "userToken": "usr_abc123def456",
  "action": "profile_update",
  "timestamp": "2024-01-15T10:30:00.000Z"
}

Session Tracking:

{
  "level": "DEBUG",
  "message": "Session activity",
  "sessionToken": "sess_xyz789uvw012",
  "pageView": "/dashboard"
}

Hashing and Pseudonymization

Consistent User Tracking:

{
  "level": "INFO",
  "message": "User behavior tracked",
  "userHash": "sha256:a1b2c3d4e5f6...",
  "action": "feature_used",
  "feature": "data_export"
}

IP Address Handling:

{
  "level": "WARN",
  "message": "Suspicious activity detected",
  "ipHash": "sha256:x1y2z3a4b5c6...",
  "attemptCount": 5
}

Implementation Strategies

Logging Library Configuration

Field-Level Protection:

// Example configuration
const loggerConfig = {
  sensitiveFields: ['password', 'ssn', 'creditCard', 'apiKey'],
  maskingStrategy: 'redact',
  hashingFields: ['email', 'userId'],
  tokenizeFields: ['sessionId', 'transactionId'],
}

Custom Serializers:

const sensitiveDataSerializer = {
  email: value => maskEmail(value),
  creditCard: value => maskCreditCard(value),
  ssn: () => '[REDACTED]',
}

Automated Detection

Pattern Recognition:

• Credit card number patterns
• Social security number formats
• Email address detection
• Phone number identification

Machine Learning Approaches:

• NLP-based PII detection
• Context-aware data classification
• Anomaly detection for data leaks
• Automated masking recommendations

Data Classification

Sensitivity Levels:

• Public: Can be logged without restriction
• Internal: Requires access controls
• Confidential: Requires masking or tokenization
• Restricted: Must not be logged

Classification Metadata:

{
  "level": "INFO",
  "message": "Data access event",
  "dataClassification": "confidential",
  "accessType": "read",
  "userRole": "analyst"
}

Compliance Considerations

GDPR (General Data Protection Regulation)

Requirements:

• Right to erasure (data deletion)
• Data minimization principles
• Purpose limitation
• Consent-based processing

Implementation:

• Log data retention policies
• User consent tracking
• Data subject request handling
• Regular compliance audits

CCPA (California Consumer Privacy Act)

Requirements:

• Consumer right to know
• Right to delete personal information
• Right to opt-out of sale
• Non-discrimination requirements

Implementation:

• Consumer request tracking
• Data inventory maintenance
• Opt-out mechanism logging
• Privacy policy compliance

HIPAA (Health Insurance Portability and Accountability Act)

Requirements:

• Minimum necessary standard
• Access logging and monitoring
• Breach notification requirements
• Administrative safeguards

Implementation:

• Access audit trails
• Minimum data logging
• Secure log storage
• Incident response procedures

PCI DSS (Payment Card Industry Data Security Standard)

Requirements:

• Cardholder data protection
• Access monitoring and testing
• Regular security testing
• Information security policies

Implementation:

• Payment data exclusion
• Tokenization strategies
• Access logging
• Regular compliance validation

Security Implementation

Secure Log Transmission

Encryption in Transit:

• TLS 1.3 for log shipping
• Certificate-based authentication
• Encrypted log forwarding
• Secure API communications

Message Integrity:

• Digital signatures for log entries
• Checksum validation
• Tamper detection mechanisms
• Audit trail verification

Access Controls

Role-Based Access:

• Principle of least privilege
• Segregation of duties
• Regular access reviews
• Automated provisioning/deprovisioning

Log Access Monitoring:

{
  "level": "AUDIT",
  "message": "Log access event",
  "accessor": "analyst_001",
  "logQuery": "SELECT * FROM logs WHERE date > '2024-01-01'",
  "accessTime": "2024-01-15T10:30:00.000Z",
  "dataClassification": "confidential"
}

Data Retention and Disposal

Retention Policies:

• Business requirement alignment
• Regulatory compliance periods
• Automated purging processes
• Secure deletion verification

Disposal Procedures:

• Cryptographic deletion
• Physical media destruction
• Verification of complete removal
• Disposal audit trails

Monitoring and Validation

Data Leak Detection

Automated Scanning:

• Regular log content analysis
• PII detection algorithms
• Pattern matching systems
• Anomaly detection for sensitive data

Alert Systems:

{
  "level": "CRITICAL",
  "message": "Potential data leak detected",
  "alertType": "sensitive_data_exposure",
  "detectionMethod": "automated_scan",
  "affectedLogs": ["app-2024-01-15.log"],
  "immediateAction": "quarantine_logs"
}

Compliance Auditing

Regular Assessments:

• Data protection effectiveness reviews
• Compliance gap analysis
• Third-party security audits
• Internal control testing

Audit Trail Maintenance:

• Data protection decisions
• Policy change documentation
• Incident response records
• Training completion tracking

Best Practices

Development Guidelines

Secure Coding Practices:

• Input validation for log data
• Secure default configurations
• Regular security training
• Code review for data protection

Testing Strategies:

• Automated sensitive data detection tests
• Penetration testing for log systems
• Compliance validation testing
• Data protection regression testing

Operational Excellence

Continuous Improvement:

• Regular policy updates
• Technology advancement integration
• Threat landscape adaptation
• Stakeholder feedback incorporation

Incident Response:

• Data breach response procedures
• Notification requirements
• Remediation strategies
• Lessons learned integration

Related Documents

• JSON Logging - Secure JSON format implementation
• Logging Standards - Overall logging security standards
• Contextual Information - Secure context handling
• Log Levels - Security-appropriate log levels