Data Structures and Operations Reference

This document describes all data structures in the Growth Marketing Experimentation System and the operations performed on each entity. Use this as a reference for backend database design and API endpoint planning.

---

Table of Contents

1. Entity Relationship Overview
2. Core Entities
   • Product
   • Hypothesis
   • CreativeVariant
   • MetricsSnapshot
   • Insight
   • Belief
3. Supporting Entities
   • ProductDefinition
   • ProductRegistry
   • EvidenceReference
   • KernelState
   • PlatformSpecs
4. Enums
5. Operations by Entity
6. API Endpoint Suggestions

---

Entity Relationship Overview

┌─────────────────────────────────────────────────────────────────┐
│                         PRODUCT                                  │
│  (Aggregate Root - isolated state per product)                   │
├─────────────────────────────────────────────────────────────────┤
│  ├── ProductDefinition (embedded)                                │
│  │                                                               │
│  └── KernelState (per-product)                                  │
│       ├── Hypotheses (1:N)                                      │
│       │    └── CreativeVariants (1:N, platform-specific)        │
│       │                                                          │
│       ├── Insights (1:N)                                        │
│       │                                                          │
│       └── Beliefs (1:N)                                         │
│            └── EvidenceReferences (1:N)                         │
│                                                                  │
│  MetricsSnapshots (linked to Hypothesis, platform-specific)     │
└─────────────────────────────────────────────────────────────────┘

Platform (lookup table - not per-product)
  └── PlatformSpecs

---

Core Entities

Product

The top-level aggregate root. Each product has isolated state (hypotheses, variants, beliefs).

Fields:

Field	Type	Required	Description
product_id	string	Yes	Unique identifier (UUID)
definition	ProductDefinition	Yes	Synthesized product info (embedded)
source_urls	string[]	No	URLs used to extract product info
additional_text	string	No	User-provided additional context
enabled_channels	string[]	No	Marketing channels enabled
created_at	ISO datetime	Yes	Creation timestamp
updated_at	ISO datetime	Yes	Last update timestamp

Storage: products/<product_id>/context.json

---

Hypothesis

The atomic unit of learning. A structured, testable marketing claim.

Fields:

Field	Type	Required	Description
hypothesis_id	string	Yes	Unique ID (format: "H-{timestamp}")
statement	string	Yes	The testable claim (max 500 chars)
independent_variable	string	Yes	Variable being tested (e.g., "headline", "image")
dependent_metric	string	Yes	Metric to measure (e.g., "ctr", "cpa")
audience_scope	string	Yes	Target audience definition
expected_direction	enum	Yes	"increase", "decrease", "change"
confidence_level	enum	Yes	"low", "medium", "high"
status	enum	Yes	See HypothesisStatus enum
created_at	ISO datetime	Yes	Creation timestamp
updated_at	ISO datetime	No	Last update timestamp
expected_magnitude	string	No	Expected size of effect (e.g., "20-30%")
conclusion	enum	No	"confirmed", "refuted", "inconclusive"
abandonment_reason	enum	No	Reason if abandoned
evidence_summary	string	No	Summary of test results
rationale	string	No	Why hypothesis was proposed
psychological_trigger	string	No	Psychological principle being tested
risk_factors	string	No	Potential risks
success_criteria	string	No	What success looks like
test_duration_suggestion	string	No	Suggested test duration
budget_suggestion	string	No	Suggested budget
creative_brief	string	No	Brief for creative development
data_quality_flags	string[]	No	Quality flags (e.g., "low_sample_size")
creative_variants	CreativeVariant[]	No	Associated variants (embedded)

Business Rules:

• Cannot be deleted (soft delete only via abandonment)
• Cannot activate without >= 3 variants per platform
• Conclusion is immutable once set
• Status transitions: proposed → approved → active → concluded/abandoned

Storage: Part of products/<product_id>/state.json under hypotheses dict

---

CreativeVariant

A specific ad creative used in an experiment. Platform-specific.

Fields:

Field	Type	Required	Description
variant_id	string	Yes	Unique ID (format: "V-{sequence}")
asset_type	string	Yes	Ad format (e.g., "single_image", "video")
asset_reference	string	Yes	Reference to creative asset
description	string	Yes	Brief description of variant approach
created_at	ISO datetime	Yes	Creation timestamp
platform_id	string	No	Platform identifier ("facebook", "linkedin")
primary_text	string	No	Main ad copy (platform: intro_text for LinkedIn)
headline	string	No	Ad headline
link_description	string	No	Description below headline
cta_button	string	No	Call-to-action button text
hook	string	No	Attention-grabbing opening
angle	string	No	Persuasion angle
rationale	string	No	Why variant was created
psychological_angle	string	No	Psychological lever used
target_emotion	string	No	Emotion being evoked
differentiation	string	No	How it differs from others
image_description	string	No	Ideal image description
image_style	string	No	Visual style direction
image_mood	string	No	Mood/atmosphere

Business Rules:

• Platform_id is required for multi-platform support
• Each platform must have >= 3 variants for hypothesis approval
• Embedded within Hypothesis (not a separate table)

---

MetricsSnapshot

Performance data for a hypothesis over a time period.

Fields:

Field	Type	Required	Description
hypothesis_id	string	Yes	FK to Hypothesis
period_start	ISO datetime	Yes	Start of measurement period
period_end	ISO datetime	Yes	End of measurement period
impressions	integer	Yes	Number of impressions
clicks	integer	Yes	Number of clicks
conversions	integer	Yes	Number of conversions
spend	float	Yes	Ad spend in dollars
platform_id	string	No	Platform the metrics are from

Business Rules:

• Immutable (frozen dataclass)
• clicks <= impressions
• conversions <= clicks
• period_end > period_start

Derived Metrics (computed, not stored):

• CTR = clicks / impressions
• CPA = spend / conversions
• CPC = spend / clicks

---

Insight

A validated learning extracted from concluded hypotheses.

Fields:

Field	Type	Required	Description
insight_id	string	Yes	Unique ID (format: "I-{timestamp}")
statement	string	Yes	The insight claim
insight_class	enum	Yes	"messaging", "audience", "channel_mechanics"
evidence_hypothesis_ids	string[]	Yes	References to supporting hypotheses
confidence	enum	Yes	"low", "medium", "high"
reusability	enum	Yes	"single_use", "limited", "broad"
status	enum	Yes	"proposed", "confirmed", "retired"
scope	string	Yes	Applicability scope
discovered_at	ISO datetime	Yes	Discovery timestamp
last_validated_at	ISO datetime	No	Last validation timestamp
decay_flag	boolean	No	Whether insight is decaying

Business Rules:

• Must reference at least one concluded hypothesis

Storage: Part of products/<product_id>/state.json under insights dict

---

Belief

A persistent claim that guides future decisions.

Fields:

Field	Type	Required	Description
belief_id	string	Yes	Unique ID (format: "B-{timestamp}")
claim	string	Yes	The belief statement
confidence	float	Yes	0.1 to 1.0
source	enum	Yes	"insight", "seed", "inherited"
scope	enum	Yes	"global", "segment_specific", "creative_specific"
lifecycle_state	enum	Yes	See BeliefLifecycleState enum
formed_at	ISO datetime	Yes	Formation timestamp
last_updated_at	ISO datetime	Yes	Last update timestamp
scope_detail	string	No	Additional scope info
evidence_for	EvidenceReference[]	No	Supporting evidence
evidence_against	EvidenceReference[]	No	Contradicting evidence

Business Rules:

• Confidence must be in [0.1, 1.0]
• Evidence references are append-only
• Decays over time without new evidence
• No untraceable beliefs (source is required)

Storage: Part of products/<product_id>/state.json under beliefs dict

---

Supporting Entities

ProductDefinition

Synthesized product information (embedded in Product).

Field	Type	Description
name	string	Product name
tagline	string	Short tagline
description	string	Product description
target_audience	string	Target audience description
value_propositions	string[]	Value propositions
key_benefits	string[]	Key benefits
brand_voice	string	Brand voice/tone
unique_selling_points	string[]	USPs
pain_points_addressed	string[]	Pain points addressed
price_positioning	string	Price positioning
call_to_action_suggestions	string[]	Suggested CTAs

---

ProductRegistry

Tracks all products and active selection.

Field	Type	Description
products	Dict[string, string]	product_id → display_name
active_product_id	string	Currently active product
version	string	Registry format version

Storage: products/registry.json

---

EvidenceReference

Reference to evidence from a hypothesis (embedded in Belief).

Field	Type	Description
hypothesis_id	string	FK to Hypothesis
strength	enum	"strong", "weak"
added_at	ISO datetime	When evidence was added

---

KernelState

Container for all per-product state.

Field	Type	Description
hypotheses	Dict[string, Hypothesis]	All hypotheses
insights	Dict[string, Insight]	All insights
beliefs	Dict[string, Belief]	All beliefs
is_halted	boolean	Emergency halt flag
halt_reason	string	Reason for halt

Storage: products/<product_id>/state.json

---

PlatformSpecs

Platform-specific constraints (lookup table).

Field	Type	Description
name	string	Display name (e.g., "Facebook Ads")
platform_id	string	Unique ID (e.g., "facebook")
character_limits	Dict	Field → {visible, max} limits
image_specs	Dict	Image requirements
cta_options	string[]	Available CTAs
ad_formats	string[]	Supported formats
objectives	string[]	Campaign objectives
audience_types	string[]	Targeting options
key_metrics	Dict	Metric benchmarks

Note: This is config/lookup data, not stored per-product.

---

Enums

HypothesisStatus

PROPOSED → APPROVED → ACTIVE → CONCLUDED
                           └→ ABANDONED

Values: proposed, approved, active, concluded, abandoned

HypothesisConclusion

Values: confirmed, refuted, inconclusive

ConfidenceLevel

Values: low, medium, high

InsightClass

Values: messaging, audience, channel_mechanics

InsightConfidence

Values: low, medium, high

BeliefScope

Values: global, segment_specific, creative_specific

BeliefLifecycleState

Values: emerging, established, strong, challenged, retired

EvidenceStrength

Values: strong, weak

AbandonmentReason

Values: spend_cap, time_limit, early_stop, policy_block, human_override

---

Operations by Entity

Product Operations

Operation	CLI Option	Method	Description
Create	0	create_product()	Create new product from URLs/text
List	0	list_products()	Get all products
Get	0	get_product()	Get single product details
Switch	0	switch_product()	Change active product
Update	0	update_product_context()	Update product definition
Delete	0	delete_product()	Delete product

Hypothesis Operations

Operation	CLI Option	Method	Description
Create	1	create_hypothesis()	Strategy agent proposes hypothesis
List	8	list_hypotheses()	Get all hypotheses (with filters)
Get	8	get_hypothesis()	Get single hypothesis with variants
Approve	3	approve_hypothesis()	Transition proposed → approved
Activate	4	activate_hypothesis()	Transition approved → active
Conclude	7	conclude_hypothesis()	Set conclusion and transition to concluded
Abandon	8	abandon_hypothesis()	Mark as abandoned with reason
Delete	14	delete_hypothesis()	Soft delete (mark as deleted)
Set Flag	11	set_data_quality_flag()	Add quality flag
Clear Flag	11	clear_data_quality_flag()	Remove quality flag

CreativeVariant Operations

Operation	CLI Option	Method	Description
Generate	2	generate_variants()	Creative agent generates for platform
List	9	(via get_hypothesis)	Get all variants for hypothesis
Get	9	(via get_hypothesis)	Get single variant details
Edit	9	update_variant()	Edit variant fields
Delete	9	delete_variant()	Remove variant from hypothesis
Regenerate	9	regenerate_variants()	Generate fresh variants
Export	15	export_variants_for_ads()	Export for platform (with filter)
Validate	-	validate_creative()	Validate against platform specs

MetricsSnapshot Operations

Operation	CLI Option	Method	Description
Ingest	5	ingest_metrics_manual()	Record metrics for platform
Get Latest	-	(via harness)	Get most recent snapshot
Aggregate	-	merge_metrics_snapshots()	Combine multiple snapshots

Insight Operations

Operation	CLI Option	Method	Description
Create	-	(future)	Create from concluded hypothesis
List	-	(future)	Get all insights
Confirm	-	(future)	Confirm proposed insight
Retire	-	(future)	Mark as retired

Belief Operations

Operation	CLI Option	Method	Description
List	9	list_beliefs()	Get all beliefs
Get	9	get_belief()	Get belief details
Update	-	(future)	Update confidence based on evidence
Decay	-	(automatic)	Confidence decays over time

System Operations

Operation	CLI Option	Method	Description
Halt	12	halt_system()	Emergency halt
Resume	12	resume_system()	Resume from halt
Summary	13	generate_daily_summary()	Generate report
Events	10	get_events()	Get event log

---

API Endpoint Suggestions

Based on the entities and operations above, here are suggested REST API endpoints:

Products

POST   /api/products                    # Create product
GET    /api/products                    # List products
GET    /api/products/:id                # Get product
PUT    /api/products/:id                # Update product
DELETE /api/products/:id                # Delete product
POST   /api/products/:id/activate       # Set as active product

Hypotheses

POST   /api/products/:pid/hypotheses                    # Create hypothesis
GET    /api/products/:pid/hypotheses                    # List hypotheses
GET    /api/products/:pid/hypotheses/:hid               # Get hypothesis
POST   /api/products/:pid/hypotheses/:hid/approve       # Approve
POST   /api/products/:pid/hypotheses/:hid/activate      # Activate
POST   /api/products/:pid/hypotheses/:hid/conclude      # Conclude
POST   /api/products/:pid/hypotheses/:hid/abandon       # Abandon
DELETE /api/products/:pid/hypotheses/:hid               # Delete
PUT    /api/products/:pid/hypotheses/:hid/flags         # Update flags

Variants

POST   /api/products/:pid/hypotheses/:hid/variants          # Generate variants
GET    /api/products/:pid/hypotheses/:hid/variants          # List variants
GET    /api/products/:pid/hypotheses/:hid/variants/:vid     # Get variant
PUT    /api/products/:pid/hypotheses/:hid/variants/:vid     # Update variant
DELETE /api/products/:pid/hypotheses/:hid/variants/:vid     # Delete variant
POST   /api/products/:pid/hypotheses/:hid/variants/export   # Export variants

Query params for variants:

• ?platform_id=facebook - Filter by platform

Metrics

POST   /api/products/:pid/hypotheses/:hid/metrics       # Ingest metrics
GET    /api/products/:pid/hypotheses/:hid/metrics       # Get metrics history

Analysis

POST   /api/products/:pid/hypotheses/:hid/analyze       # Analyst propose conclusion

System

POST   /api/products/:pid/halt                          # Halt system
POST   /api/products/:pid/resume                        # Resume system
GET    /api/products/:pid/summary                       # Get summary report
GET    /api/products/:pid/events                        # Get event log

Platforms (lookup/config)

GET    /api/platforms                                   # List available platforms
GET    /api/platforms/:id                               # Get platform specs

---

Database Schema Suggestions

Relational (PostgreSQL)

-- Products
CREATE TABLE products (
    product_id UUID PRIMARY KEY,
    name VARCHAR(255) NOT NULL,
    tagline TEXT,
    description TEXT,
    target_audience TEXT,
    brand_voice TEXT,
    price_positioning TEXT,
    source_urls JSONB,
    additional_text TEXT,
    enabled_channels JSONB,
    value_propositions JSONB,
    key_benefits JSONB,
    unique_selling_points JSONB,
    pain_points_addressed JSONB,
    call_to_action_suggestions JSONB,
    is_active BOOLEAN DEFAULT FALSE,
    created_at TIMESTAMPTZ NOT NULL,
    updated_at TIMESTAMPTZ NOT NULL
);

-- Hypotheses
CREATE TABLE hypotheses (
    hypothesis_id VARCHAR(50) PRIMARY KEY,
    product_id UUID REFERENCES products(product_id),
    statement TEXT NOT NULL,
    independent_variable VARCHAR(100) NOT NULL,
    dependent_metric VARCHAR(100) NOT NULL,
    audience_scope TEXT NOT NULL,
    expected_direction VARCHAR(20) NOT NULL,
    confidence_level VARCHAR(20) NOT NULL,
    status VARCHAR(20) NOT NULL,
    expected_magnitude VARCHAR(100),
    conclusion VARCHAR(20),
    abandonment_reason VARCHAR(50),
    evidence_summary TEXT,
    rationale TEXT,
    psychological_trigger TEXT,
    risk_factors TEXT,
    success_criteria TEXT,
    test_duration_suggestion VARCHAR(100),
    budget_suggestion VARCHAR(100),
    creative_brief TEXT,
    data_quality_flags JSONB,
    created_at TIMESTAMPTZ NOT NULL,
    updated_at TIMESTAMPTZ
);

-- Creative Variants
CREATE TABLE creative_variants (
    variant_id VARCHAR(50) PRIMARY KEY,
    hypothesis_id VARCHAR(50) REFERENCES hypotheses(hypothesis_id),
    platform_id VARCHAR(50),
    asset_type VARCHAR(50) NOT NULL,
    asset_reference TEXT,
    description TEXT NOT NULL,
    primary_text TEXT,
    headline TEXT,
    link_description TEXT,
    cta_button VARCHAR(100),
    hook TEXT,
    angle TEXT,
    rationale TEXT,
    psychological_angle TEXT,
    target_emotion VARCHAR(100),
    differentiation TEXT,
    image_description TEXT,
    image_style VARCHAR(100),
    image_mood VARCHAR(100),
    created_at TIMESTAMPTZ NOT NULL
);

-- Metrics
CREATE TABLE metrics_snapshots (
    id SERIAL PRIMARY KEY,
    hypothesis_id VARCHAR(50) REFERENCES hypotheses(hypothesis_id),
    platform_id VARCHAR(50),
    period_start TIMESTAMPTZ NOT NULL,
    period_end TIMESTAMPTZ NOT NULL,
    impressions INTEGER NOT NULL,
    clicks INTEGER NOT NULL,
    conversions INTEGER NOT NULL,
    spend DECIMAL(12,2) NOT NULL,
    created_at TIMESTAMPTZ DEFAULT NOW()
);

-- Platforms (lookup)
CREATE TABLE platforms (
    platform_id VARCHAR(50) PRIMARY KEY,
    name VARCHAR(100) NOT NULL,
    character_limits JSONB,
    image_specs JSONB,
    cta_options JSONB,
    ad_formats JSONB,
    objectives JSONB,
    audience_types JSONB,
    key_metrics JSONB
);

-- Indexes
CREATE INDEX idx_hypotheses_product ON hypotheses(product_id);
CREATE INDEX idx_hypotheses_status ON hypotheses(status);
CREATE INDEX idx_variants_hypothesis ON creative_variants(hypothesis_id);
CREATE INDEX idx_variants_platform ON creative_variants(platform_id);
CREATE INDEX idx_metrics_hypothesis ON metrics_snapshots(hypothesis_id);

---

Notes for Backend Implementation

1. Product Isolation: Each product's state is fully isolated. The current implementation uses file-based storage with separate directories per product.

2. Hypothesis-Variant Relationship: Variants are currently embedded in hypotheses. For a database, normalize to a separate table with FK.

3. Metrics History: Store all metrics snapshots for historical analysis and trend detection.

4. Platform is Config: Platform specs are configuration data, not user data. Can be stored in code or a config table.

5. Event Sourcing: Consider event sourcing for hypothesis state changes to maintain complete audit trail.

6. Soft Deletes: Hypotheses should use soft delete (abandonment) rather than hard delete.

7. Concurrency: Multiple users may work on same product - consider optimistic locking on hypothesis updates.