CineChoice RAG Implementation Plan

🎯 Project Overview

Transform CineChoice into a sophisticated RAG (Retrieval Augmented Generation) movie recommendation system using vector embeddings, semantic search, and AI-powered recommendations.

🏗️ Architecture Overview

User Questions → Embedding → Vector Search → Movie Matching → AI Response
     ↓              ↓           ↓              ↓              ↓
  [Frontend] → [API Routes] → [Supabase] → [HuggingFace] → [Results]

📊 Technology Stack

Core Technologies:

• Frontend: Next.js 15, React 19, TypeScript, Tailwind CSS
• Backend: Next.js API Routes, TypeScript
• Vector Database: Supabase (PostgreSQL + pgvector)
• Embeddings: mixedbread-ai/mxbai-embed-large-v1 (1024 dimensions)
• Text Generation: Hugging Face Inference API
• Text Processing: LangChain RecursiveCharacterTextSplitter

Data Sources:

• public/content.js - 9 structured movie objects
• public/movies.txt - 12 detailed movie descriptions

🗄️ Database Schema Design

Table: movies

CREATE TABLE movies (
  id SERIAL PRIMARY KEY,
  title TEXT NOT NULL,
  release_year INTEGER,
  genre TEXT[],
  rating FLOAT,
  runtime TEXT,
  director TEXT,
  cast TEXT[],
  plot_summary TEXT,
  imdb_rating FLOAT,
  content_source TEXT, -- 'content.js' or 'movies.txt'
  raw_content TEXT,
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

Table: movie_chunks

CREATE TABLE movie_chunks (
  id SERIAL PRIMARY KEY,
  movie_id INTEGER REFERENCES movies(id) ON DELETE CASCADE,
  chunk_text TEXT NOT NULL,
  chunk_index INTEGER NOT NULL,
  embedding VECTOR(1024), -- mixedbread-ai/mxbai-embed-large-v1 dimensions
  metadata JSONB,
  created_at TIMESTAMP DEFAULT NOW()
);

-- Create vector similarity index
CREATE INDEX movie_chunks_embedding_idx ON movie_chunks 
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);

Table: user_queries (Optional - for analytics)

CREATE TABLE user_queries (
  id SERIAL PRIMARY KEY,
  session_id TEXT,
  questions JSONB,
  query_embedding VECTOR(1024),
  matched_movies INTEGER[],
  response_generated TEXT,
  created_at TIMESTAMP DEFAULT NOW()
);

🔧 Implementation Phases

Phase 1: Database Setup & Data Ingestion

Step 1.1: Supabase Configuration

• Set up Supabase project
• Enable pgvector extension
• Create database schema
• Set up environment variables
• Create database connection utilities

Step 1.2: Data Processing Pipeline

• Create data ingestion script
• Parse content.js and movies.txt
• Normalize and structure movie data
• Store movies in database

Step 1.3: Text Chunking

• Install LangChain dependencies
• Implement RecursiveCharacterTextSplitter
• Chunk movie descriptions and metadata
• Store chunks with references to parent movies

Step 1.4: Vector Embeddings

• Set up Hugging Face client
• Create embedding generation function
• Generate embeddings for all chunks
• Store embeddings in vector database

Phase 2: Search & Matching Functions

Step 2.1: Vector Search Function

-- Cosine similarity search function
CREATE OR REPLACE FUNCTION match_movie_chunks (
  query_embedding VECTOR(1024),
  match_threshold FLOAT DEFAULT 0.7,
  match_count INT DEFAULT 10
)
RETURNS TABLE (
  id INTEGER,
  movie_id INTEGER,
  chunk_text TEXT,
  similarity FLOAT,
  metadata JSONB
)
LANGUAGE SQL
AS $$
  SELECT 
    mc.id,
    mc.movie_id,
    mc.chunk_text,
    1 - (mc.embedding <=> query_embedding) AS similarity,
    mc.metadata
  FROM movie_chunks mc
  WHERE 1 - (mc.embedding <=> query_embedding) > match_threshold
  ORDER BY mc.embedding <=> query_embedding ASC
  LIMIT LEAST(match_count, 200);
$$;

Step 2.2: Movie Recommendation Function

-- Get movie recommendations with scores
CREATE OR REPLACE FUNCTION get_movie_recommendations (
  query_embedding VECTOR(1024),
  match_threshold FLOAT DEFAULT 0.7,
  movie_limit INT DEFAULT 5
)
RETURNS TABLE (
  movie_id INTEGER,
  title TEXT,
  avg_similarity FLOAT,
  matching_chunks_count INTEGER,
  movie_data JSONB
)
LANGUAGE SQL
AS $$
  SELECT 
    m.id,
    m.title,
    AVG(1 - (mc.embedding <=> query_embedding)) AS avg_similarity,
    COUNT(mc.id) AS matching_chunks_count,
    ROW_TO_JSON(m) AS movie_data
  FROM movies m
  JOIN movie_chunks mc ON m.id = mc.movie_id
  WHERE 1 - (mc.embedding <=> query_embedding) > match_threshold
  GROUP BY m.id, m.title
  ORDER BY avg_similarity DESC
  LIMIT movie_limit;
$$;

Phase 3: API Development

Step 3.1: Core API Endpoints

app/api/ingest/route.ts

• Data ingestion endpoint
• Chunking and embedding pipeline
• Database population

app/api/search/route.ts

• Query embedding generation
• Vector similarity search
• Return matched movies

app/api/recommend/route.ts

• Process user answers
• Generate recommendations
• AI-powered response generation

Step 3.2: Utility Functions

lib/embeddings.ts

export async function generateEmbedding(text: string): Promise<number[]> {
  const response = await hf.featureExtraction({
    model: "mixedbread-ai/mxbai-embed-large-v1",
    inputs: text,
  });
  return Array.from(response);
}

lib/chunking.ts

import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";

export async function chunkText(text: string): Promise<string[]> {
  const splitter = new RecursiveCharacterTextSplitter({
    chunkSize: 500,
    chunkOverlap: 50,
  });
  return await splitter.splitText(text);
}

lib/supabase.ts

export async function searchSimilarMovies(
  queryEmbedding: number[],
  threshold: number = 0.7,
  limit: number = 5
) {
  // Vector search implementation
}

Phase 4: Frontend Integration

Step 4.1: Enhanced QA Component

• Add loading states
• Real-time answer processing
• Progress indicators

Step 4.2: Results Page

• Movie recommendation display
• Similarity scores
• AI-generated explanations
• Movie details and links

Step 4.3: User Flow

1. User answers questions → QA Components
2. Submit answers → API processing
3. Generate embeddings → Vector search
4. Find similar movies → AI response generation
5. Display results → Results page

Phase 5: AI Response Generation

Step 5.1: Response Generation

• Context building from matched movies
• Prompt engineering for recommendations
• AI model integration for explanations
• Response formatting and styling

Step 5.2: Prompt Templates

const RECOMMENDATION_PROMPT = `
Based on the user's preferences: {user_answers}
And these matching movies: {matched_movies}
Generate a personalized movie recommendation explaining why these movies match their preferences.
`;

📦 Dependencies to Install

# Core dependencies
npm install @supabase/supabase-js
npm install @huggingface/inference
npm install langchain

# Type definitions
npm install -D @types/pg

🔧 Environment Variables

# Supabase
NEXT_PUBLIC_SUPABASE_URL=your_supabase_url
NEXT_PUBLIC_SUPABASE_ANON_KEY=your_supabase_anon_key
SUPABASE_SERVICE_ROLE_KEY=your_service_role_key

# Hugging Face
HUGGINGFACE_API_KEY=your_hf_api_key

🚀 Deployment & Production

Performance Optimizations:

• Embedding caching
• Query result caching
• Database connection pooling
• Vector index optimization

Monitoring:

• Query performance metrics
• Embedding generation costs
• User interaction analytics
• Error tracking and logging

📊 Success Metrics

1. Technical Metrics:

   • Query response time < 2 seconds
   • Embedding generation < 500ms
   • 95%+ similarity search accuracy

2. User Experience:

   • Relevant movie recommendations
   • Intuitive interface
   • Fast, responsive interactions

3. Business Metrics:

   • User engagement time
   • Recommendation click-through rates
   • Return user percentage

🧪 Testing Strategy

1. Unit Tests:

   • Embedding generation functions
   • Vector search utilities
   • Data processing pipelines

2. Integration Tests:

   • API endpoint functionality
   • Database operations
   • End-to-end user flows

3. Performance Tests:

   • Vector search speed
   • Concurrent user handling
   • Database query optimization

🔄 Next Steps

1. Immediate: Set up Supabase and create schema
2. Week 1: Implement data ingestion and chunking
3. Week 2: Vector embeddings and search functions
4. Week 3: API development and frontend integration
5. Week 4: AI response generation and optimization

---

This plan provides a comprehensive roadmap for building a production-ready RAG-powered movie recommendation system. Each phase builds upon the previous one, ensuring a solid foundation for an advanced AI-driven user experience.