Sensitive Data Masking in Logs

# Sensitive Data Masking in Logs

## Overview

This feature ensures that sensitive data (secrets, API keys, passwords, private keys, tokens) never appears in application logs. All logging utilities automatically mask sensitive information while preserving debug usefulness.

## Implementation

### Core Components

1. **Data Masker Utility** (`src/utils/dataMasker.js`)
   - Centralized masking logic
   - Pattern-based detection of sensitive fields
   - Value-based detection (e.g., Stellar secret keys, JWT tokens)
   - Recursive object/array masking
   - Configurable partial masking for debugging

2. **Enhanced Log Utility** (`src/utils/log.js`)
   - Automatically masks all logged metadata
   - Special handling for error objects
   - Integrates seamlessly with existing code

3. **Logger Middleware** (`src/middleware/logger.js`)
   - Masks request/response data
   - Sanitizes headers, body, query params
   - File logging with masked data

## Sensitive Patterns Detected

### Authentication & Authorization
- `password`, `passwd`, `pwd`
- `secret`, `secretKey`, `secret_key`
- `private`, `privateKey`, `private_key`
- `token`, `accessToken`, `refreshToken`
- `apiKey`, `api_key`, `api-key`
- `authorization`, `auth`, `bearer`

### Stellar-Specific
- `senderSecret`, `sender_secret`
- `sourceSecret`, `source_secret`
- `destinationSecret`, `destination_secret`
- `seed`, `mnemonic`
- Stellar secret keys (pattern: `S[A-Z2-7]{55}`)

### Financial & PII
- `creditCard`, `cardNumber`, `cvv`
- `ssn`, `social_security`
- `taxId`, `tax_id`

### Encryption
- `encryptionKey`, `encryption_key`
- `cipher`, `iv`, `authTag`

### Session & Cookies
- `session`, `sessionId`, `cookie`
- `csrf`, `xsrf`

### Value-Based Detection
- Stellar secret keys: `S[A-Z2-7]{55}`
- JWT tokens: `eyJ...` format
- Long alphanumeric strings (potential API keys)

## Usage

### Automatic Masking

All existing logging automatically masks sensitive data:

```javascript
const log = require('../utils/log');

// Sensitive data is automatically masked
log.info('USER_AUTH', 'User login attempt', {
  username: 'john',
  password: 'secret123',  // Will be masked
  apiKey: 'abc123xyz'     // Will be masked
});

// Output: [2026-02-25T...] [INFO] [USER_AUTH] User login attempt {"username":"john","password":"[REDACTED]","apiKey":"[REDACTED]"}
```

### Request/Response Logging

The logger middleware automatically masks sensitive data in HTTP requests:

```javascript
// Request with sensitive data
POST /api/donate
Headers:
  x-api-key: secret-key-123        // Masked
  authorization: Bearer token123   // Masked

Body:
  amount: "100"                    // Preserved
  senderSecret: "SBZV..."          // Masked
  destination: "GBZV..."           // Preserved
```

### Manual Masking

You can also use the masker directly:

```javascript
const { maskSensitiveData } = require('../utils/dataMasker');

const data = {
  username: 'john',
  password: 'secret123',
  amount: '100'
};

const masked = maskSensitiveData(data);
// Result: { username: 'john', password: '[REDACTED]', amount: '100' }
```

### Partial Masking for Debugging

Enable partial masking to show first/last characters:

```javascript
// Set environment variable
process.env.LOG_SHOW_PARTIAL = 'true';

// Now logs will show partial values
// apiKey: "abc1********x789" instead of "[REDACTED]"
```

## Configuration

### Environment Variables

- `LOG_SHOW_PARTIAL=true` - Show partial values (first 4 and last 4 characters)
- `LOG_VERBOSE=true` - Log full request/response payloads (still masked)
- `LOG_TO_FILE=true` - Enable file logging (with masking)
- `LOG_DIR=/path/to/logs` - Custom log directory

### Custom Sensitive Patterns

Add custom patterns to mask:

```javascript
const { addSensitivePatterns } = require('../utils/dataMasker');

addSensitivePatterns(['customSecret', 'internalKey']);
```

## Testing

### Unit Tests

Run data masker tests:
```bash
npm test -- tests/dataMasker.test.js
```

### Integration Tests

Run logger masking tests:
```bash
npm test -- tests/logger-masking.test.js
```

### Test Coverage

- ✅ Sensitive key detection (case-insensitive)
- ✅ Sensitive value detection (Stellar keys, JWT tokens)
- ✅ Nested object masking
- ✅ Array masking
- ✅ Request/response sanitization
- ✅ Error object masking
- ✅ Stack trace sanitization
- ✅ Partial value masking
- ✅ Edge cases (null, undefined, circular references)

## Security Guarantees

### What is Masked
✅ All password fields
✅ All API keys and tokens
✅ All Stellar secret keys (by key name and value pattern)
✅ All authorization headers
✅ All encryption keys
✅ All session tokens
✅ JWT tokens
✅ Credit card numbers
✅ SSN and tax IDs

### What is Preserved
✅ Public keys (Stellar addresses starting with G)
✅ Transaction hashes
✅ Amounts and balances
✅ Usernames and emails
✅ Timestamps and IDs
✅ URLs and endpoints
✅ HTTP methods and status codes
✅ Non-sensitive metadata

## Examples

### Example 1: Donation Request

**Before Masking:**
```json
{
  "amount": "100.50",
  "destination": "GBZVMB3SEPB2ENHQVEQ5MJQXB2QZUQPQQ6QQZQPQQ6QQZQPQQ6QQZQPQ",
  "senderSecret": "SBZVMB3SEPB2ENHQVEQ5MJQXB2QZUQPQQ6QQZQPQQ6QQZQPQQ6QQZQPQ",
  "memo": "Donation for charity"
}
```

**After Masking:**
```json
{
  "amount": "100.50",
  "destination": "GBZVMB3SEPB2ENHQVEQ5MJQXB2QZUQPQQ6QQZQPQQ6QQZQPQQ6QQZQPQ",
  "senderSecret": "[REDACTED]",
  "memo": "Donation for charity"
}
```

### Example 2: Authentication Headers

**Before Masking:**
```json
{
  "content-type": "application/json",
  "x-api-key": "sk_live_abc123xyz789",
  "authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."
}
```

**After Masking:**
```json
{
  "content-type": "application/json",
  "x-api-key": "[REDACTED]",
  "authorization": "[REDACTED]"
}
```

### Example 3: Error with Sensitive Data

**Before Masking:**
```javascript
{
  name: "Error",
  message: "Transaction failed",
  details: {
    senderSecret: "SBZVMB3SEPB2ENHQVEQ5MJQXB2QZUQPQQ6QQZQPQQ6QQZQPQQ6QQZQPQ",
    amount: "100"
  }
}
```

**After Masking:**
```javascript
{
  name: "Error",
  message: "Transaction failed",
  details: {
    senderSecret: "[REDACTED]",
    amount: "100"
  }
}
```

## Best Practices

1. **Never log raw request/response objects** - Always use the logger middleware or log utility
2. **Use structured logging** - Pass metadata as objects, not strings
3. **Review logs regularly** - Audit logs to ensure no sensitive data leaks
4. **Test with real data patterns** - Use realistic test data to verify masking
5. **Add custom patterns** - If you add new sensitive fields, update the patterns
6. **Enable partial masking in dev** - Helps debugging without exposing full secrets
7. **Disable partial masking in production** - Use full redaction for maximum security

## Compliance

This implementation helps meet compliance requirements for:
- **PCI DSS** - Credit card data protection
- **GDPR** - Personal data protection
- **SOC 2** - Security logging requirements
- **HIPAA** - Healthcare data protection (if applicable)

## Troubleshooting

### Issue: Sensitive data still appears in logs

**Solution:** 
1. Check if the field name matches sensitive patterns
2. Add custom pattern: `addSensitivePatterns(['yourFieldName'])`
3. Verify you're using the log utility, not console.log directly

### Issue: Too much data is masked

**Solution:**
1. Review the sensitive patterns list
2. Rename fields to avoid false positives
3. Use more specific field names

### Issue: Need to see partial values for debugging

**Solution:**
Set `LOG_SHOW_PARTIAL=true` in development environment only

## Migration Guide

### Existing Code

No changes required! All existing logging code automatically benefits from masking:

```javascript
// This code doesn't need to change
log.info('DONATION', 'Processing donation', {
  amount: '100',
  senderSecret: 'SBZV...'  // Automatically masked
});
```

### New Code

Continue using the same logging patterns:

```javascript
const log = require('../utils/log');

// All metadata is automatically masked
log.info('SCOPE', 'Message', metadata);
log.warn('SCOPE', 'Warning', metadata);
log.error('SCOPE', 'Error', metadata);
```

## Performance Impact

- **Minimal overhead** - Masking adds ~1-2ms per log entry
- **No impact on production** - Masking only affects logging, not business logic
- **Efficient pattern matching** - Uses optimized regex and string operations
- **Lazy evaluation** - Only masks when logging is enabled

## Future Enhancements

- [ ] Configurable masking strategies (hash, encrypt, redact)
- [ ] Audit trail for masked data access
- [ ] Machine learning-based sensitive data detection
- [ ] Integration with secret management systems
- [ ] Automatic PII detection
- [ ] Compliance report generation

## Related Documentation

- [Logging Feature](./LOGGING_FEATURE.md)
- [Security Best Practices](../security/README.md)
- [Error Handling](../security/ERROR_HANDLING.md)