Memory Compress

Your agent's memory is drowning in daily logs. This fixes it.

I built this because my agent's MEMORY.md hit 15,000 words and daily logs kept piling up at 2,500 words/day. Needed a way to compress without losing the important stuff.

The key insight came from classical Chinese writing — ancient scholars compressed entire dynasties into single sentences. Same principle here:

1. Strip redundancy — mention it once, not three times
2. Keep only turning points — what changed, not what continued
3. Let structure carry meaning — bullet hierarchy > verbose paragraphs
4. Drop the process, keep the result — "failed 3 times, then X worked" > 3 failure descriptions

Result: 4-8x compression ratio, zero loss on key events. Zero dependencies.

Before: 2,500 words of raw daily log
After:    400 words of structured insight

Architecture

┌────────────────────────────────────────────────┐
│           THREE-LAYER MEMORY SYSTEM            │
├────────────────────────────────────────────────┤
│                                                │
│  Layer 1: IDENTITY (SOUL.md)                   │
│           Ultra-compressed, stable             │
│           Who you are. What matters.           │
│                                                │
│  Layer 2: CURATED MEMORY (MEMORY.md)     ◄──┐ │
│           4:1 compressed summaries          │ │
│           Key events + lessons + todos      │ │
│                                             │ │
│  Layer 3: RAW LOGS (memory/YYYY-MM-DD.md)   │ │
│           Full detail, everything        ───┘ │
│           ~2,500 words/day                     │
│                                                │
│  memory-compress: Layer 3 ──► Layer 2          │
└────────────────────────────────────────────────┘

Quick Start

# Compress today's log
node scripts/memory-compress.js memory/2026-03-14.md

# Specify output
node scripts/memory-compress.js memory/2026-03-14.md /tmp/compressed.md

# Append to long-term memory
node scripts/memory-compress.js memory/2026-03-14.md /tmp/today.md
cat /tmp/today.md >> MEMORY.md

How It Works

Smart Hybrid Extraction

Most memory tools choke on unstructured logs. This one doesn't.

Step 1 — Keyword Matching: Scans headers for 40+ patterns across Chinese & English:

• Events: 重大进展, breakthrough, milestone, decision…
• Lessons: 教训, 反思, insight, takeaway…
• Growth: 进化, evolution, improvement…
• Action items: 待办, 🔴, 🟡, todo, next step…

Step 2 — Fallback Extraction: When no keywords match (e.g. time-based headers like ## 08:44 Standup), automatically extracts all sections with top items. No data loss, ever.

Step 3 — Hybrid Mode: For multi-day files, matched sections use keyword extraction while unmatched sections use fallback. Both coexist. Nothing gets dropped.

The Classical Chinese Compression Philosophy

This isn't just "summarize shorter." It's a deliberate compression methodology:

Principle	What it means	Example
去重复 (Strip redundancy)	Mentioned once = enough	Don't repeat "WebSocket reconnection" across 3 sections
留转折 (Keep turning points)	Only what changed	"Switched from nginx to direct Node.js WSS" > 5 paragraphs of debugging
去过程 (Drop process)	Result > journey	"3 failures → fixed with X" > 3 failure descriptions
留白 (Leave blanks)	Let reader infer	Bullet hierarchy implies relationship
形式即内容 (Form is content)	Structure carries meaning	Nested lists > flat paragraphs

Output Format

## 2026-03-14 Key Experiences

### Key Events
- **Event title**
  - Detail 1
  - Detail 2

### Core Lessons
- Lesson learned

### Pending/Remaining
- 🔴 Urgent items
- 🟡 Important items

Batch Compression

# Compress last 7 days
for file in memory/2026-03-{08..14}.md; do
    [ -f "$file" ] && node scripts/memory-compress.js "$file" "/tmp/$(basename $file)"
done

Heartbeat Integration

Add to your maintenance cycle:

## Memory Maintenance (every 2-3 days)
1. Run: node scripts/memory-compress.js memory/YYYY-MM-DD.md /tmp/compressed.md
2. Review for accuracy
3. Append: cat /tmp/compressed.md >> MEMORY.md
4. Timestamp: date +%s > .last-memory-maintenance

Edge Cases

Scenario	Behavior
Empty file	Graceful skip
BOM-encoded	Auto-detected, stripped
Non-UTF-8	Warning + continues
Missing output dir	Auto-created
No markdown structure	Friendly message
Multi-day concatenated	Hybrid strategy, all days preserved

CLI

node scripts/memory-compress.js <log-file> [output-file]
node scripts/memory-compress.js --help