Loading...
Loading...
Loading...
<div align="center">
# ๐ AI ETL Assistant
**English | [ะ ัััะบะธะน](README.md)**
<!-- TODO: Add hero GIF here showing: Natural Language โ Pipeline โ Deployment (30 seconds) -->
<!--  -->
### โก Transform Natural Language into Production-Ready ETL Pipelines in Seconds
**No coding required. No learning curve. Just results.**
<div>
<a href="http://158.160.187.18/"><strong>๐ Live Demo</strong></a> โข
<a href="https://disk.yandex.ru/d/rlkeEFp_TPAmCQ"><strong>๐ Presentation</strong></a> โข
<a href="https://github.com/Sergey-1221/ai-etl-docs"><strong>๐ Documentation</strong></a> โข
<a href="#-quick-start-60-seconds"><strong>โก Quick Start</strong></a>
</div>
<br/>
</div>
---
## ๐ฏ Why AI ETL Assistant?
Traditional ETL development is:
- โฐ **Time-consuming** - Weeks to build simple pipelines
- ๐ **Error-prone** - Manual coding leads to bugs and data quality issues
- ๐ฐ **Expensive** - Senior engineers spending time on repetitive tasks
- ๐ **Complex** - Steep learning curve for data tools and orchestration
- ๐ **Rigid** - Hard to adapt when business requirements change
### AI ETL Assistant Solves This
```
You: "Load sales data from PostgreSQL to ClickHouse daily at 2 AM"
โ
โจ AI generates production-ready pipeline
โ
โ
Deploy to Airflow in 30 seconds
```
**That's it.** No code, no configuration files, no debugging.
---
## โจ See It In Action
<!-- TODO: Add 3 screenshots here -->
<div align="center">
### ๐ธ Screenshots Coming Soon
| Natural Language Interface | Visual DAG Editor | Real-time Monitoring |
|:-------------------------:|:------------------:|:--------------------:|
|  |  |  |
| **Describe** your pipeline in plain English | **Visualize** and edit generated DAG | **Monitor** execution in real-time |
</div>
---
## ๐ Proven Results
<div align="center">
| Metric | Before AI ETL | With AI ETL | Impact |
|--------|--------------|-------------|--------|
| ๐ **Pipeline Development Time** | 2 weeks | 30 seconds | **336x Faster** |
| ๐ **Time to Production** | 14 days | 1 hour | **336x Faster** |
| โ
**Code Accuracy** | 60% first try | 95%+ | **Zero Manual Fixes** |
| ๐ป **Lines of Code** | 1000+ | 0 | **Natural Language** |
| ๐ **Bug Rate** | 15% | <1% | **AI Validation** |
| ๐ฐ **Cost Savings** | $50K/pipeline | $150/pipeline | **99% Reduction** |
</div>
---
## ๐ Presentation & Demo
<div align="center">
### ๐ [Try Live Demo](http://158.160.187.18/) โข ๐ [View Presentation](https://disk.yandex.ru/d/rlkeEFp_TPAmCQ)
**Live Demo**: Experience AI ETL Assistant in action at [158.160.187.18](http://158.160.187.18/) (admin/admin123)
**Presentation**: Comprehensive overview with architecture, features, and use cases on [Yandex.Disk](https://disk.yandex.ru/d/rlkeEFp_TPAmCQ)
</div>
---
## ๐ Quick Start (60 Seconds)
### โก Option 1: Docker (Recommended)
```bash
# Pull and run demo
docker run -p 3000:3000 -p 8000:8000 ai-etl/complete-demo
# Open browser
open http://localhost:3000
```
**โฑ๏ธ Time: 60 seconds** โข You'll see the UI and can create your first pipeline immediately
<details>
<summary><strong>๐ง Option 2: One-Click Local Development (Windows)</strong></summary>
```powershell
# Clone and start (requires kubectl configured)
git clone https://sourcecraft.dev/noise1983/ai-etl.git
cd ai-etl
.\start-local-dev.ps1
```
**โฑ๏ธ Time: 3 minutes** โข Full development environment with K8s backend
</details>
<details>
<summary><strong>๐ณ Option 3: Docker Compose (Full Stack)</strong></summary>
```bash
# Start all services
git clone https://sourcecraft.dev/noise1983/ai-etl.git
cd ai-etl
cp .env.example .env
docker-compose up -d
# Initialize database
docker-compose exec backend alembic upgrade head
```
**โฑ๏ธ Time: 5 minutes** โข Complete stack with all services
</details>
<details>
<summary><strong>โธ๏ธ Option 4: Kubernetes Production Deployment</strong></summary>
```bash
# Deploy to production Kubernetes cluster
kubectl create namespace ai-etl
kubectl create secret generic ai-etl-secrets --from-env-file=.env -n ai-etl
kubectl apply -f k8s-production/
```
**โฑ๏ธ Time: 10 minutes** โข Production-ready deployment with monitoring
</details>
### ๐ฏ What You Get
- **Frontend**: http://localhost:3000 (Next.js UI)
- **Backend API**: http://localhost:8000 (Interactive docs at /docs)
- **Airflow**: http://localhost:8080 (Pipeline orchestration)
- **MinIO Console**: http://localhost:9001 (Artifact storage)
**Default Credentials**: admin/admin123 (change in production)
**๐ Live Demo**: [http://158.160.187.18/](http://158.160.187.18/)
---
## ๐ฏ Top Features
<div align="center">
| Feature | Description | Status |
|:-------:|-------------|:------:|
| ๐ฃ๏ธ **Natural Language Pipelines** | Convert plain English to production ETL code | โ
Production |
| ๐ **600+ Data Connectors** | PostgreSQL, ClickHouse, S3, Excel, APIs, HDFS, Hive, Kafka | โ
Production |
| ๐จ **Visual DAG Editor** | Drag-and-drop pipeline builder with React Flow | โ
Production |
| ๐ค **Multi-LLM Support** | GPT-4, Claude, Qwen, DeepSeek, local models | โ
Production |
| ๐ **Real-time Monitoring** | Prometheus + Grafana dashboards | โ
Production |
| ๐ **Change Data Capture** | Real-time replication with Debezium | โ
Production |
| ๐ก๏ธ **AI Validation** | Auto-detect SQL injection, code smells, security issues | โ
Production |
| ๐ **Enterprise Security** | JWT auth, RBAC, audit logs, PII redaction | โ
Production |
</div>
<details>
<summary><strong>๐ Advanced Features (Click to Expand)</strong></summary>
### AI-Powered Intelligence
- **๐ง Smart Storage Analysis** - AI recommends optimal storage based on data patterns
- **๐ Schema Inference** - Auto-generate schemas from raw data
- **๐ฏ Data Relationship Detection** - Discover foreign keys automatically
- **๐ Pipeline Optimization** - AI-powered performance tuning
- **๐ฌ Natural Language SQL** - Convert business questions to optimized queries
- **๐ฎ Predictive Monitoring** - ML-based anomaly detection and failure prediction
### Enterprise Features
- **๐ Version Control** - Full artifact versioning with rollback capability
- **๐ CDC Replication** - Real-time data synchronization with Debezium
- **๐ Data Lineage** - Track data flow with DataHub integration
- **โ
Data Quality** - Auto-generate quality checks with Great Expectations
- **๐ญ Pipeline Templates** - 10+ pre-built templates for common patterns
- **๐ Multi-Cloud** - Deploy to AWS, Azure, GCP, Yandex Cloud
### Developer Experience
- **๐ฅ Semantic Caching** - 30-50% reduction in LLM API calls
- **๐ก๏ธ Circuit Breaker** - Resilient LLM service with fallback
- **๐ณ Kubernetes-Ready** - Production-ready health checks and autoscaling
- **๐ Prometheus Metrics** - Deep observability with custom metrics
- **๐ง Hot Reload** - Changes reflect instantly in development
### Compliance & Security
- **๐ท๐บ Russian Compliance** - ะะะกะข ะ 57580, ะคะ-242, GIS GMP integration
- **โ๏ธ Digital Signatures** - Government document signing
- **๐ Secrets Management** - Encrypted credential storage
- **๐ก๏ธ PII Redaction** - Automatic sensitive data masking
- **๐ Audit Trail** - Complete activity logging for compliance
</details>
---
## ๐ How We Compare
<div align="center">
| Feature | AI ETL Assistant | Apache Airflow | Prefect | dbt | Airbyte |
|:--------|:----------------:|:--------------:|:-------:|:---:|:-------:|
| **Natural Language Pipelines** | โ
| โ | โ | โ | โ |
| **Zero Code Required** | โ
| โ | โ | โ | โ ๏ธ |
| **AI-Powered Generation** | โ
| โ | โ | โ | โ |
| **Auto Code Validation** | โ
| โ | โ | โ | โ |
| **Visual DAG Editor** | โ
| โ
| โ
| โ | โ ๏ธ |
| **600+ Connectors** | โ
| โ ๏ธ | โ ๏ธ | โ ๏ธ | โ
|
| **Real-time CDC** | โ
| โ ๏ธ | โ ๏ธ | โ | โ
|
| **Learning Curve** | Minutes | Weeks | Days | Days | Hours |
| **Time to First Pipeline** | 30 seconds | 2 hours | 1 hour | 1 hour | 30 min |
| **Production Ready** | โ
| โ
| โ
| โ
| โ
|
</div>
**Legend**: โ
Full Support โข โ ๏ธ Partial/Requires Plugins โข โ Not Available
---
## ๐๏ธ Architecture
### High-Level System Design
```mermaid
graph TB
subgraph "User Interface"
UI[Next.js 14 UI<br/>React Flow DAG]
end
subgraph "AI Layer"
LLM[Multi-LLM Gateway<br/>GPT-4, Claude, Qwen]
CACHE[Semantic Cache<br/>30-50% Reduction]
VAL[AI Validator<br/>Security & Quality]
end
subgraph "API Layer"
API[FastAPI Backend<br/>SQLAlchemy 2.0]
AUTH[Auth Service<br/>JWT + RBAC]
end
subgraph "Data Stores"
PG[(PostgreSQL<br/>Metadata)]
RD[(Redis<br/>Cache & Sessions)]
CH[(ClickHouse<br/>Metrics)]
S3[(MinIO S3<br/>Artifacts)]
end
subgraph "Processing"
AF[Apache Airflow<br/>Orchestration]
SP[Apache Spark<br/>Big Data]
KF[Kafka<br/>Streaming]
end
subgraph "Integrations"
AB[Airbyte<br/>600+ Connectors]
DBZ[Debezium<br/>CDC]
DH[DataHub<br/>Lineage]
GE[Great Expectations<br/>Data Quality]
end
UI --> API
API --> LLM
LLM --> CACHE
LLM --> VAL
API --> AUTH
API --> PG & RD & CH & S3
API --> AF
AF --> SP & KF
AF --> AB & DBZ & DH & GE
classDef ai fill:#ffeb3b,stroke:#f57f17,stroke-width:3px
classDef prod fill:#4caf50,stroke:#1b5e20,stroke-width:2px
class LLM,CACHE,VAL ai
class AF,API prod
```
### Data Flow: Natural Language โ Production Pipeline
```mermaid
sequenceDiagram
participant User
participant UI as Next.js UI
participant API as FastAPI
participant LLM as AI Engine
participant VAL as Validator
participant AF as Airflow
participant DB as Database
User->>UI: "Load sales from PostgreSQL to ClickHouse daily"
UI->>API: POST /api/v1/pipelines/generate
API->>LLM: Generate ETL code
Note over LLM: GPT-4 analyzes request<br/>Generates Python + SQL
LLM-->>API: Return pipeline code
API->>VAL: Validate code
Note over VAL: Check syntax<br/>Security scan<br/>Best practices
VAL-->>API: โ
Validation passed
API->>DB: Save pipeline metadata
API-->>UI: Show preview + DAG
User->>UI: Click "Deploy"
UI->>API: POST /api/v1/pipelines/{id}/deploy
API->>AF: Deploy DAG file
AF-->>API: โ
Deployed
API-->>UI: Success!
Note over User: Pipeline running in<br/>production in 30 seconds
```
---
## ๐ ๏ธ Technology Stack
<div align="center">
### Backend
### Frontend
### AI/ML
### DevOps
</div>
**Full Stack Details**:
- **Backend**: FastAPI + SQLAlchemy 2.0 (async) + Pydantic v2
- **Frontend**: Next.js 14 App Router + shadcn/ui + React Flow + TanStack Query
- **AI/ML**: OpenAI GPT-4, Anthropic Claude, Qwen, DeepSeek, Codestral, local models
- **AI Agents**: FAISS (vector search), sentence-transformers (embeddings), NetworkX (graphs), Graphviz (visualization), matplotlib, Pillow
- **Data**: PostgreSQL, ClickHouse, Redis, MinIO S3, Kafka, HDFS, Hive, Spark
- **Orchestration**: Apache Airflow 2.7 + Celery
- **Processing**: Apache Spark, Airbyte, Debezium, DataHub
- **DevOps**: Docker, Kubernetes, Prometheus, Grafana, Poetry
---
## ๐๏ธ Detailed Architecture
### Three-Tier Microservices Architecture
```
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Presentation Layer โ
โ (Next.js 14 App Router) โ
โ Port: 3000 โ
โโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ
โ REST API
โ
โโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
โ Application Layer โ
โ (FastAPI Backend) โ
โ Port: 8000 โ
โโโโโโโฌโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโโโ
โ โ โ
โโโโโโโผโโโโโโ โโโโโโโโผโโโโโโโ โโโโโโโผโโโโโโโ
โ LLM โ โ Orchestratorโ โ Data โ
โ Gateway โ โ (Airflow) โ โ Services โ
โ Port:8001 โ โ Port:8080 โ โ (56+) โ
โโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโ
โ โ โ
โโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโโโ
โ Data Layer โ
โ PostgreSQL | ClickHouse | Redis | MinIO | Kafka โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
```
**Key Components**:
- **56+ Backend Services**: Pipeline, LLM, Connector, Orchestrator, CDC, Streaming, Metrics, Audit, Security, Observability services
- **LLM Gateway**: Multi-provider routing (10+ providers), semantic caching (30-50% reduction), circuit breaker
- **AI Agents System**: 6 specialized agents (Planner, SQL Expert, Python Coder, Schema Analyst, QA Validator, Reflector)
### AI Agents Multi-Version System
**V1 - Base Orchestration** (Quality: 9.5/10, Success: 96%):
- 6 specialized agents with chain-of-thought reasoning
- Self-reflection loops for quality improvement
- Coordinated pipeline generation
**V2 - Tools + Memory**:
- **Tool Executor**: 10 real function-calling tools (validate_sql, get_schema, query_database, execute_python, etc.)
- **Memory System**: RAG with FAISS vector index, 247+ stored memories, 73% cache hit rate
**V3 - Autonomous Collaboration**:
- **Communication Protocol**: Direct agent-to-agent messaging, consensus voting (66% threshold), broadcast, request-response
- **Visual Reasoning**: ER diagram generation (NetworkX + Graphviz), data flow graphs, dependency analysis
- **Adversarial Testing**: 47+ security tests (SQL injection, edge cases, performance), 9.2/10 security score
- **Multi-modal**: Vision AI integration (Qwen-VL, GPT-4V, Claude), ER diagram analysis from images
### MVP Features (23 Endpoints)
**Network Storage Monitoring** (4 endpoints):
- Mount network drives (SMB, NFS, cloud)
- Watch folders for new files with auto-import
- Auto-import files with schema inference
- List monitored files and status
**Datamart Management** (7 endpoints):
- Create materialized views or datamarts
- Refresh datamart with concurrent mode
- Schedule automatic refresh (cron)
- List all datamarts with statistics
- Preview datamart contents
- Create versioned datamart with history
- Export datamart to Excel
**Simple Triggers & Scheduling** (7 endpoints):
- Create pipeline triggers (cron, webhook, file, manual)
- Manual pipeline trigger with params
- Pause/resume/delete triggers
- List all triggers
- Get trigger execution history
**Enhanced Data Preview** (2 endpoints):
- Preview uploaded file with auto-detection
- Preview file from filesystem path
**Relationship Detection** (1 endpoint):
- Auto-detect relationships between tables (with AI)
**Excel Export Service** (2 endpoints):
- Export data to Excel with charts and summary
- Create formatted Excel report with templates
### Security & Compliance
**Authentication & Authorization**:
- JWT authentication with refresh tokens
- RBAC (4 roles: Analyst, Engineer, Architect, Admin)
- Session management with Redis
- API rate limiting per user and project
**AI-Powered Security**:
- **PII Detection**: Microsoft Presidio integration for automatic PII identification (emails, phone numbers, SSNs, credit cards)
- **SQL Injection Prevention**: Parameterized queries with SQLAlchemy
- **Code Validation**: Real-time syntax and security checks before deployment
**Audit & Monitoring**:
- Comprehensive audit logging with Redis queue and batch processing
- 20+ audit actions, 12 resource types
- Automatic PII redaction in audit logs
- Compliance reports for regulatory requirements
**Russian Compliance Support**:
- GOST R 57580 standard implementation
- FZ-242 data localization compliance
- Digital signatures for document signing
- Government templates for official reporting
- GIS GMP integration for government data exchange
- 1C Enterprise, Rosstat, SMEV connectors
**Data Protection**:
- Secrets management with encrypted storage
- Network security with TLS 1.3
- Input validation with Pydantic models
- File upload security with type validation and size limits
### Observability & Monitoring
**AI-Powered Monitoring**:
- ML-based anomaly detection in pipeline execution
- Predictive alerts for potential failures
- Smart thresholds with dynamic adjustment
- AI-assisted root cause analysis
**Metrics & Telemetry**:
- Real-time custom business and technical metrics
- ClickHouse high-performance telemetry database
- Prometheus integration for system metrics
- Pre-built Grafana dashboards
**Circuit Breaker & Resilience**:
- Automatic failure detection and recovery
- Configurable thresholds (failure rate, timeout)
- Half-open state for gradual recovery
- Fallback strategies for degraded mode
**Health Checks**:
- Kubernetes-ready liveness and readiness probes
- Service dependency checks (Database, Redis, ClickHouse, LLM Gateway)
- Detailed health reports with component-level status
---
<div align="center">
**Experience the future of ETL automation**
[๐ Try Live Demo](http://158.160.187.18/) (admin/admin123) | [๐ View Presentation](https://disk.yandex.ru/d/rlkeEFp_TPAmCQ) | [๐ Read Docs](https://github.com/Sergey-1221/ai-etl-docs)
</div>
---
## ๐ Documentation
๐ **Complete documentation**: [AI ETL Docs on GitHub](https://github.com/Sergey-1221/ai-etl-docs)
### ๐ Getting Started
- [Quick Start Guide](https://github.com/Sergey-1221/ai-etl-docs) - 5-minute setup
- [First Pipeline Tutorial](https://github.com/Sergey-1221/ai-etl-docs) - Hands-on walkthrough
- [Installation Guide](https://github.com/Sergey-1221/ai-etl-docs) - Detailed setup
### ๐ป Development
- [Development Setup](https://github.com/Sergey-1221/ai-etl-docs) - Dev environment
- [Backend Guide](https://github.com/Sergey-1221/ai-etl-docs) - FastAPI backend
- [Frontend Guide](https://github.com/Sergey-1221/ai-etl-docs) - Next.js frontend
- [Contributing](https://github.com/Sergey-1221/ai-etl-docs) - How to contribute
### ๐ API Reference
- [REST API](https://github.com/Sergey-1221/ai-etl-docs) - Complete API documentation
- [Pipeline API](https://github.com/Sergey-1221/ai-etl-docs) - Pipeline endpoints
- [Vector Search API](https://github.com/Sergey-1221/ai-etl-docs) - Semantic search
- [Error Codes](https://github.com/Sergey-1221/ai-etl-docs) - Error reference
### ๐ข Deployment
- [Production Checklist](https://github.com/Sergey-1221/ai-etl-docs) - 100+ checkpoints
- [Docker Deployment](https://github.com/Sergey-1221/ai-etl-docs) - Docker setup
- [Kubernetes Guide](https://github.com/Sergey-1221/ai-etl-docs) - K8s production
- [Cloud Deployment](https://github.com/Sergey-1221/ai-etl-docs) - AWS, Azure, GCP
### ๐ง Operations
- [Monitoring Setup](https://github.com/Sergey-1221/ai-etl-docs) - Prometheus + Grafana
- [Common Issues](https://github.com/Sergey-1221/ai-etl-docs) - Troubleshooting
- [Performance Tuning](https://github.com/Sergey-1221/ai-etl-docs) - Optimization
- [FAQ](https://github.com/Sergey-1221/ai-etl-docs) - Frequently asked questions
---
## ๐งช Testing
```bash
# Run all tests
make test
# Quick test (unit tests only)
pytest -m unit
# Integration tests (requires services running)
pytest -m integration
# With coverage report
make test-coverage
pytest --cov=backend --cov-report=html
# Frontend tests
cd frontend && npm test
```
**Test Coverage**: 85% backend, 70% frontend
---
## ๐ Security
### Features
- ๐ **JWT Authentication** with refresh tokens
- ๐ฅ **RBAC** (4 roles: Analyst, Engineer, Architect, Admin)
- ๐ก๏ธ **SQL Injection Prevention** via parameterized queries
- ๐ **Secrets Management** encrypted credential storage
- ๐ **Audit Logging** comprehensive activity tracking
- ๐ญ **PII Redaction** automatic sensitive data masking
- โก **Rate Limiting** per user and project
### Compliance
- โ
ะะะกะข ะ 57580 (Russian standard)
- โ
ะคะ-242 (Data localization)
- โ
GDPR ready
- โ
SOC2 controls
**Security Report**: Run `make security-check` for vulnerability scan
---
## ๐ค Contributing
We โค๏ธ contributions!
### How to Contribute
1. **Fork** the repository
2. **Create** a feature branch: `git checkout -b feature/amazing-feature`
3. **Commit** your changes: `git commit -m 'Add amazing feature'`
4. **Push** to the branch: `git push origin feature/amazing-feature`
5. **Open** a Pull Request
### Development Guidelines
- โ
Follow code style (Black for Python, ESLint for TypeScript)
- โ
Add tests for new features
- โ
Update documentation
- โ
Follow semantic versioning
- โ
Sign commits (optional but appreciated)
### Areas We Need Help
- ๐ Documentation improvements
- ๐ Bug fixes
- โจ New features
- ๐ Translations
- ๐จ UI/UX improvements
- ๐ New data connectors
[๐ Read our Contributing Guide](./docs/development/contributing.md)
---
## ๐ฌ Join Community
<div align="center">
### Get Help & Connect
[](https://github.com/Sergey-1221/ai-etl-docs)
[](https://stackoverflow.com/questions/tagged/ai-etl)
- ๐ **Bug Reports**: [Report issues at SourceCraft](https://sourcecraft.dev/noise1983/ai-etl)
- ๐ **Documentation**: [Complete docs on GitHub](https://github.com/Sergey-1221/ai-etl-docs)
- ๐ **Live Demo**: [Try the demo](http://158.160.187.18/) (admin/admin123)
- ๐ **Presentation**: [View presentation](https://disk.yandex.ru/d/rlkeEFp_TPAmCQ)
</div>
---
## ๐บ๏ธ Roadmap
### Q3 2024
- [ ] Interactive playground (try without install)
- [ ] One-click deployment to major clouds
- [ ] Mobile app for monitoring
- [ ] dbt integration
- [ ] Real-time collaboration on pipelines
### Q4 2024
- [ ] AI pipeline optimization engine
- [ ] Auto-scaling based on data volume
- [ ] Multi-tenant SaaS version
- [ ] Marketplace for pipeline templates
- [ ] Advanced RBAC with custom roles
[๐ Full Roadmap](https://sourcecraft.dev/noise1983/ai-etl)
---
## ๐ Stats
<div align="center">
[](https://sourcecraft.dev/noise1983/ai-etl)
[](https://github.com/Sergey-1221/ai-etl-docs)
**Production Ready** โข **Active Development** โข **Enterprise Features**
</div>
---
## ๐ License
This project is licensed under the **MIT License** - see the [LICENSE](LICENSE) file for details.
**TL;DR**: You can use this for anything, including commercial projects, for free.
---
## ๐ Acknowledgments
Built with love using these amazing open-source projects:
- [FastAPI](https://fastapi.tiangolo.com/) - Modern Python API framework
- [Next.js](https://nextjs.org/) - React production framework
- [Apache Airflow](https://airflow.apache.org/) - Workflow orchestration
- [OpenAI](https://openai.com/) - AI language models
- [shadcn/ui](https://ui.shadcn.com/) - Beautiful component library
- [React Flow](https://reactflow.dev/) - Interactive node-based UIs
---
## ๐ Links
<div align="center">
| Resource | Link |
|:--------:|:----:|
| ๐ **Live Demo** | [http://158.160.187.18/](http://158.160.187.18/) (admin/admin123) |
| ๐ **Presentation** | [Yandex.Disk](https://disk.yandex.ru/d/rlkeEFp_TPAmCQ) |
| ๐ **Documentation** | [GitHub Docs](https://github.com/Sergey-1221/ai-etl-docs) |
| ๐ป **Repository** | [SourceCraft](https://sourcecraft.dev/noise1983/ai-etl) |
</div>
---
<div align="center">
**Made with โค๏ธ for data engineers who want to focus on insights, not infrastructure.**
[โฌ Back to Top](#-ai-etl-assistant)
</div>
> ๅฑฌๆผ [research/](./README.md)ใๆถต่ LLM-as-JudgeใReasoning Modelใ่ฉไผฐ็ถญๅบฆใJudge ่จญ่จๅๅใ
> โ ๏ธ Note (Option A): `hwp-web (planned)` is intentionally excluded/disabled in this repo snapshot.
Here are three new, highly specialized AI agents for the T20 framework:
The **LLM Judge** is LLMTrace's third security detector alongside the