Run Phase - Visual Product Search

# Run Phase - Visual Product Search

## Overview

The "Run" phase implements **visual product search** - take a picture of an item and query the Couchbase Lite database using vector similarity to find matching products.

## Architecture

```
┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
│   Camera/Image  │────▶│  ML Model (CLIP) │────▶│  Vector Query   │
│     Input       │     │  Generate Vector │     │  Couchbase Lite │
└─────────────────┘     └──────────────────┘     └─────────────────┘
                                                         │
                                                         ▼
                                                 ┌─────────────────┐
                                                 │ Similar Products│
                                                 │   (Top N)       │
                                                 └─────────────────┘
```

## Key Components

### 1. Image Embedding Generation

Convert images to vectors using CLIP or similar models that can run on-device.

**Recommended Libraries:**
- **react-native-executorch** - On-device AI inference (CLIP model support)
- **react-native-fast-tflite** - TensorFlow Lite for React Native with GPU acceleration
- **@react-native-rag/executorch** - RAG utilities with ExecuTorch integration

### 2. Couchbase Lite Vector Search

Use `APPROX_VECTOR_DISTANCE()` function to find similar vectors.

**Vector Index Configuration:**
```typescript
const config = new VectorIndexConfiguration({
  expression: 'embedding',  // field containing the vector
  dimensions: 512,          // CLIP base model outputs 512 dimensions
  centroids: 100,           // ~sqrt(num_documents)
  metric: 'cosine',         // or 'euclidean'
});

await collection.createIndex('embedding_index', config);
```

**Query Example:**
```sql
SELECT META().id, *, APPROX_VECTOR_DISTANCE(embedding, $queryVector) AS distance
FROM catalog.vectors
WHERE APPROX_VECTOR_DISTANCE(embedding, $queryVector) < 0.5
ORDER BY APPROX_VECTOR_DISTANCE(embedding, $queryVector)
LIMIT 10
```

### 3. Hybrid Search

Combine vector search with full-text search for better results:

```sql
SELECT META().id, *
FROM catalog.products
WHERE MATCH(description, 'red shirt')
  AND APPROX_VECTOR_DISTANCE(embedding, $queryVector) < 0.5
ORDER BY APPROX_VECTOR_DISTANCE(embedding, $queryVector)
LIMIT 10
```

## Implementation Plan

### Phase 1: Camera Integration
- [ ] Install `expo-camera` or `react-native-vision-camera`
- [ ] Create camera capture screen
- [ ] Allow photo selection from gallery

### Phase 2: Image Embedding
- [ ] Install `react-native-executorch` with CLIP model
- [ ] Load CLIP image encoder (352 MB model)
- [ ] Generate 512-dim embedding from captured image

### Phase 3: Vector Query
- [ ] Create vector index on `catalog.vectors` collection
- [ ] Implement `APPROX_VECTOR_DISTANCE()` query
- [ ] Display top matching products

### Phase 4: UI/UX
- [ ] Create visual search screen with camera preview
- [ ] Show search results with similarity scores
- [ ] Add product detail view

## Dependencies to Add

```bash
# Camera
npx expo install expo-camera expo-image-picker

# ML/Embeddings (choose one approach)
# Option A: ExecuTorch (recommended for CLIP)
npm install react-native-executorch @react-native-rag/executorch

# Option B: TensorFlow Lite
npm install react-native-fast-tflite
```

## Model Options

### CLIP (Recommended)
- **Size**: ~352 MB
- **Dimensions**: 512
- **Inference Time**: ~50-70ms on modern devices
- **Pros**: Same embedding space for text and images
- **Source**: https://huggingface.co/software-mansion/react-native-executorch-clip-vit-base-patch32

### MobileNet V3
- **Size**: ~5-15 MB
- **Dimensions**: 1024
- **Inference Time**: ~20-30ms
- **Pros**: Smaller model size
- **Cons**: Image-only embeddings

## Couchbase Lite Vector Search Reference

### Documentation
- [CBL React Native Full-Text Search](https://cbl-reactnative.dev/full-text-search)
- [CBL Swift Vector Search Guide](https://docs.couchbase.com/couchbase-lite/current/swift/working-with-vector-search.html)

### Key Functions
- `APPROX_VECTOR_DISTANCE(vector-expr, target-vector, [metric])` - Find approximate nearest neighbors
- `MATCH(field, text)` - Full-text search (can combine with vector search)

### Vector Index Parameters
| Parameter | Description |
|-----------|-------------|
| `expression` | Field containing the vector |
| `dimensions` | Vector size (2-4096) |
| `centroids` | Buckets for clustering (~sqrt(docs)) |
| `metric` | Distance: cosine, euclidean, dot |
| `encoding` | Compression: None, SQ (8-bit), PQ |

## Data Flow

1. **User captures image** via camera or gallery
2. **CLIP model processes image** → 512-dim float array
3. **Query Couchbase Lite** using `APPROX_VECTOR_DISTANCE()`
4. **Display matching products** sorted by similarity

## Considerations

### Performance
- CLIP model is ~352 MB - consider lazy loading
- First inference may take 2x longer (model initialization)
- Cache model in memory for repeated searches

### Accuracy
- CLIP works best with clear, well-lit images
- Training vectors should use same model as query
- Consider using cosine similarity for normalized vectors

### Offline-First
- Model runs entirely on-device
- No network required for inference
- Vectors sync from Sync Gateway for catalog updates