Product Requirement Document

Web Data Processing Application

1. Executive Summary

Project Name: Web Data Processing Application
Version: 1.0
Date: August 2025
Document Owner: Development Team

Ứng dụng desktop WPF được thiết kế để tự động hóa quy trình scraping dữ liệu web, xử lý hình ảnh thông qua ChatGPT API, và cập nhật thông tin trở lại các trang web một cách hiệu quả.

2. Technology Stack

2.1 Core Technologies

• Framework: .NET 8.0 WPF (Windows Presentation Foundation)
• Language: C# 12.0
• Database: SQLite 3.x
• Web Automation: Playwright for .NET (khuyến nghị thay vì Selenium do hiệu suất tốt hơn)

2.2 Additional Libraries & Packages

• HTTP Client: HttpClient với Polly (retry policies)
• JSON Processing: System.Text.Json
• Database ORM: Entity Framework Core với SQLite provider
• Async UI: ReactiveUI hoặc CommunityToolkit.Mvvm
• UI Controls: ModernWpf hoặc Material Design In XAML
• Image Processing: SixLabors.ImageSharp (nếu cần xử lý ảnh)

3. Functional Requirements

3.1 Data Extraction Module

FR-001: Web Page Data Scraping

• Ứng dụng phải có khả năng nhập URL của trang web nguồn
• Tự động extract dữ liệu từ bảng được chỉ định trên trang web
• Dữ liệu cần lấy: Link hình ảnh, Link trang (page links)
• Hiển thị dữ liệu trong DataGrid có tên ProductDataGrid

Acceptance Criteria:

• User có thể nhập URL trang web nguồn
• Hệ thống detect và extract được bảng dữ liệu
• Dữ liệu hiển thị đúng format trong DataGrid
• Xử lý được các trường hợp lỗi (page không tồn tại, table không có dữ liệu)

3.2 AI Image Analysis Module

FR-002: ChatGPT API Integration

• Gửi từng hình ảnh qua ChatGPT API với prompt template được định sẵn
• Yêu cầu response format là JSON với schema cố định
• Cập nhật kết quả JSON vào các cột tương ứng trong DataGrid

JSON Response Schema Example:

{
  "title": "string",
  "short_description": "string",
  "long_description_html": "string with inline CSS",
  "image_filename": "string",
  "image_alt_text": "string",
  "page_title": "string",
  "meta_description": "string",
  "url_handle_primary": "string",
  "url_handle_alternative": "string"
}

Prompt Template Example:

Analyze this product image (likely apparel/clothing item) and create comprehensive SEO-optimized content. Return a JSON response with the following structure:

{
  "title": "Product title optimized for SEO (60-70 characters)",
  "short_description": "Brief description for Google Shopping (150-160 characters)",
  "long_description_html": "Detailed HTML description with inline CSS styling, include emojis, bullet points with benefits, size info, target audience. Use inline styles for formatting.",
  "image_filename": "SEO-friendly filename with dashes (no spaces, lowercase)",
  "image_alt_text": "Descriptive alt text for accessibility and SEO",
  "page_title": "SEO page title (50-60 characters)",
  "meta_description": "Meta description for search engines (150-160 characters)",
  "url_handle_primary": "Primary URL slug (lowercase, dashes)",
  "url_handle_alternative": "Alternative URL slug option"
}

Requirements:
- Focus on the main design/pattern/theme of the product
- Include relevant keywords for SEO
- Target audience should be clear
- Long description must be HTML with inline CSS styling
- Use emojis appropriately in descriptions
- Include size ranges if applicable
- Make it appealing to potential buyers
- All text should be engaging and conversion-focused

Return only valid JSON, no additional text or formatting.

Acceptance Criteria:

• Xử lý được nhiều format ảnh (JPG, PNG, WebP)
• Retry mechanism khi API call thất bại
• Progress indicator cho user
• Validation JSON response từ API

3.3 Data Management Module

FR-003: Database Operations

• Multiple row selection trong ProductDataGrid
• Save selected rows vào 2 bảng SQLite tương ứng
• Database schema design phù hợp với business logic

Database Tables:

-- Bảng Products
CREATE TABLE Products (
    Id INTEGER PRIMARY KEY AUTOINCREMENT,
    ImageLink TEXT NOT NULL,
    PageLink TEXT NOT NULL,
    Title TEXT,
    ShortDescription TEXT,
    LongDescriptionHtml TEXT,
    ImageFilename TEXT,
    ImageAltText TEXT,
    PageTitle TEXT,
    MetaDescription TEXT,
    UrlHandlePrimary TEXT,
    UrlHandleAlternative TEXT,
    CreatedDate DATETIME DEFAULT CURRENT_TIMESTAMP,
    UpdatedDate DATETIME DEFAULT CURRENT_TIMESTAMP
);

-- Bảng ProcessedImages  
CREATE TABLE ProcessedImages (
    Id INTEGER PRIMARY KEY AUTOINCREMENT,
    ProductId INTEGER,
    ImageLink TEXT NOT NULL,
    ProcessedDate DATETIME DEFAULT CURRENT_TIMESTAMP,
    ApiResponse TEXT, -- Full JSON response
    ProcessingStatus TEXT CHECK(ProcessingStatus IN ('Success', 'Failed', 'Pending')),
    FOREIGN KEY (ProductId) REFERENCES Products (Id)
);

3.4 Web Page Update Module

FR-004: Automated Page Updates

• Mở từng page link trong tab mới (tối đa 5 tab concurrent)
• Cập nhật thông tin từ DataGrid vào các vị trí được chỉ định trên page
• Tự động đóng tab sau khi hoàn thành
• Asynchronous processing với semaphore control

Page Update Mapping:

public class PageUpdateConfig
{
    public string FieldName { get; set; }
    public string CssSelector { get; set; }
    public UpdateType Type { get; set; } // Text, Attribute, InnerHtml
}

public enum UpdateType
{
    Text,
    Attribute, 
    InnerHtml
}

4. Non-Functional Requirements

4.1 Performance Requirements

NFR-001: Response time cho data extraction không quá 30 giây/page NFR-002: ChatGPT API calls phải có timeout 60 giây NFR-003: Concurrent page updates tối đa 5 tabs NFR-004: Application memory usage không vượt quá 500MB

4.2 Reliability Requirements

NFR-005: Application uptime 99% trong session sử dụng NFR-006: Auto-retry mechanism cho failed API calls (tối đa 3 lần) NFR-007: Data persistence với transaction support

4.3 Usability Requirements

NFR-008: Modern, intuitive WPF interface
NFR-009: Real-time progress indicators NFR-010: Error messages user-friendly NFR-011: Keyboard shortcuts cho common actions

4.4 Security Requirements

NFR-012: ChatGPT API key encryption trong config NFR-013: Input validation cho URLs NFR-014: SQL injection protection

5. User Interface Design

5.1 Main Window Layout

┌─────────────────────────────────────────────────────┐
│ [File] [Edit] [Tools] [Help]                        │
├─────────────────────────────────────────────────────┤
│ Source URL: [__________________________] [Extract]  │
├─────────────────────────────────────────────────────┤
│ ProductDataGrid                                     │
│ ┌─────────────────────────────────────────────────┐ │
│ │☐ ImageLink │ PageLink │ Title │ Description │...│ │
│ │☐ img1.jpg  │ page1    │       │             │   │ │
│ │☐ img2.jpg  │ page2    │       │             │   │ │
│ └─────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────┤
│ [Process Images] [Save Selected] [Update Pages]     │
├─────────────────────────────────────────────────────┤
│ Progress: [████████████████████████] 85%            │
│ Status: Processing image 17 of 20...                │
└─────────────────────────────────────────────────────┘

5.2 Configuration Window

• ChatGPT API settings
• Prompt template configuration
• Page update field mapping
• Database connection settings

6. Implementation Architecture

6.1 Application Architecture

Presentation Layer (WPF)
├── MainWindow.xaml
├── ConfigWindow.xaml  
└── ViewModels/

Business Logic Layer
├── Services/
│   ├── WebScrapingService
│   ├── ChatGptService
│   ├── DatabaseService
│   └── PageUpdateService
└── Models/

Data Access Layer
├── Repositories/
├── DbContext/
└── Entities/

Infrastructure Layer
├── Configuration/
├── Logging/
└── HttpClients/

6.2 Key Classes Design

// Main ViewModel
public class MainViewModel : ObservableObject
{
    public ObservableCollection<ProductItem> Products { get; set; }
    public string SourceUrl { get; set; }
    public double ProgressValue { get; set; }
    public string StatusMessage { get; set; }
    
    public IAsyncRelayCommand ExtractDataCommand { get; }
    public IAsyncRelayCommand ProcessImagesCommand { get; }
    public IAsyncRelayCommand SaveSelectedCommand { get; }
    public IAsyncRelayCommand UpdatePagesCommand { get; }
}

// Product Model
public class ProductItem : ObservableObject
{
    public bool IsSelected { get; set; }
    public string ImageLink { get; set; }
    public string PageLink { get; set; }
    public string Title { get; set; }
    public string ShortDescription { get; set; }
    public string LongDescriptionHtml { get; set; }
    public string ImageFilename { get; set; }
    public string ImageAltText { get; set; }
    public string PageTitle { get; set; }
    public string MetaDescription { get; set; }
    public string UrlHandlePrimary { get; set; }
    public string UrlHandleAlternative { get; set; }
}

// Services
public interface IWebScrapingService
{
    Task<List<ProductItem>> ExtractProductDataAsync(string url);
}

public interface IChatGptService  
{
    Task<ChatGptResponse> AnalyzeImageAsync(string imageUrl, string prompt);
}

public interface IPageUpdateService
{
    Task UpdatePagesAsync(List<ProductItem> products, int maxConcurrency = 5);
}

7. Development Phases

Phase 1: Core Infrastructure (2 weeks)

• Project setup với .NET 8.0 WPF
• Database setup với Entity Framework
• Basic UI layout
• Configuration management

Phase 2: Data Extraction (2 weeks)

• Playwright integration
• Web scraping functionality
• DataGrid binding
• Error handling

Phase 3: AI Integration (2 weeks)

• ChatGPT API integration
• JSON response processing
• Progress tracking
• Retry mechanisms

Phase 4: Database & UI (1 week)

• Save functionality
• Multi-selection support
• Advanced UI controls
• Input validation

Phase 5: Page Updates (2 weeks)

• Asynchronous page updates
• Concurrency control
• Status monitoring
• Final testing

Phase 6: Polish & Optimization (1 week)

• Performance optimization
• Bug fixes
• Documentation
• Deployment preparation

8. Risk Assessment

High Risks

• API Rate Limiting: ChatGPT API có thể limit requests
  • Mitigation: Implement exponential backoff, API key rotation
• Web Structure Changes: Target websites thay đổi cấu trúc
  • Mitigation: Flexible selectors, configuration-based mapping

Medium Risks

• Performance Issues: Large datasets có thể chậm
  • Mitigation: Pagination, virtualization, background processing
• Browser Compatibility: Playwright compatibility issues
  • Mitigation: Multi-browser support, fallback options

Low Risks

• UI Responsiveness: Heavy operations block UI
  • Mitigation: Proper async/await implementation

9. Testing Strategy

9.1 Unit Testing

• Business logic services
• Data models validation
• Database operations
• API client functionality

9.2 Integration Testing

• Web scraping với real websites
• ChatGPT API integration
• Database transactions
• End-to-end workflows

9.3 Performance Testing

• Concurrent operations stress test
• Memory usage profiling
• Large dataset processing
• API timeout handling

10. Deployment & Maintenance

10.1 Deployment Requirements

• Windows 10/11 (x64)
• .NET 8.0 Runtime
• Minimum 4GB RAM
• Internet connection for API calls

10.2 Configuration Files

<!-- appsettings.json -->
{
  "ChatGPT": {
    "ApiKey": "encrypted_key",
    "BaseUrl": "https://api.openai.com/v1",
    "Model": "gpt-4-vision-preview",
    "MaxTokens": 1000,
    "Temperature": 0.1
  },
  "Database": {
    "ConnectionString": "Data Source=app_data.db"
  },
  "Scraping": {
    "UserAgent": "Mozilla/5.0...",
    "TimeoutSeconds": 30,
    "MaxRetries": 3
  }
}

10.3 Monitoring & Logging

• Application logs với Serilog
• Performance counters
• Error tracking và reporting
• API usage statistics

11. Success Metrics

11.1 Functional Metrics

• Data Accuracy: 95% successful data extraction
• API Success Rate: 98% successful ChatGPT API calls
• Processing Speed: < 2 minutes per 100 items
• Update Success: 95% successful page updates

11.2 Technical Metrics

• Application Stability: Zero crashes trong 8h session
• Memory Efficiency: < 500MB peak usage
• Response Time: UI responsive < 100ms cho user actions

---

Document Version: 1.0
Last Updated: August 9, 2025
Next Review: September 9, 2025