Event Kickoff and Atmosphere

On October 15, 2024, DeepLearning.AI teamed up with industry leaders to host the 25th AI Dev Meetup in New York City. Moderated by Logan Kilpatrick from OpenAI and Chip Huyen from Stanford and Clay, the evening drew a packed crowd of developers, researchers, and AI enthusiasts eager to dive into cutting-edge tools. The focus? Practical frameworks for building scalable AI applications: agent orchestration, efficient model fine-tuning, and robust retrieval-augmented generation (RAG) systems.

This wasn't just theory—speakers shared live demos, code walkthroughs, and battle-tested strategies you can implement today. Whether you're prototyping multi-agent workflows or optimizing LLM training on consumer hardware, the talks provided actionable blueprints. Post-presentation networking fostered connections, with attendees swapping tips on deploying these tools in production environments like chatbots, data pipelines, and enterprise search.

Orchestrating Agents with OpenAI's Swarm

Kicking off the technical deep dive, Shashank Bhooshan from OpenAI introduced Swarm, an open-source library designed to simplify multi-agent coordination. In real-world scenarios, single LLMs fall short for complex tasks like customer support triage or automated research—enter lightweight agents that hand off work dynamically.

Swarm's core philosophy: Treat agents as simple Python functions with an agent attribute, powered by OpenAI's client. No heavy abstractions or state machines—just run() to execute and delegate. This keeps things fast and debuggable, ideal for prototyping in Jupyter notebooks or scaling to production swarms.

Key Swarm Concepts in Action

• Agents: Defined by name, instructions, and tools (functions). Example:

import swarm from openai import OpenAI client = OpenAI()

def transfer_to_refunds(): print("Transferring to refunds dept...") return "refund_agent"

def transfer_to_shipping(): print("Transferring to shipping dept...") return "shipping_agent"

refund_agent = swarm.Agent( name="Refund Agent", instructions="You are a helpful refund agent", functions=[transfer_to_shipping], ) shipping_agent = swarm.Agent( name="Shipping Agent", instructions="You are a helpful shipping agent", ) triage_agent = swarm.Agent( name="Triage Agent", instructions="You are a triage agent. Route refund calls to refund_agent, and shipping to shipping_agent", functions=[transfer_to_refunds, transfer_to_shipping], )

- **Handoffs**: Agents return another agent name to delegate seamlessly. Perfect for workflows like e-commerce support: triage → refund/shipping.

- **Tools**: Any Python callable becomes a tool. Swarm auto-generates schemas for LLMs, supporting OpenAI's function calling.

Shashank demoed a live call center simulation, then escalated to a research agent chaining tools for data gathering. Pro tip: Start simple—define 2-3 agents for your MVP, then iterate. Swarm runs locally or via APIs, with no vendor lock-in beyond the base LLM.

For developers: Clone [https://github.com/openai/swarm](https://github.com/openai/swarm), install via `pip install git+https://github.com/openai/swarm.git`, and experiment. It's early-stage (research preview), so expect rapid evolution—contribute issues or PRs to shape it.

## Accelerating LLM Fine-Tuning with Unsloth

Next, Daniel Han from Unsloth showcased how to fine-tune massive LLMs like Llama 3.1 405B twice as fast on a single RTX 4090—without accuracy loss. In production, fine-tuning customizes models for domain-specific tasks like legal analysis or code generation, but GPU costs and time kill momentum.

Unsloth tackles this with optimized kernels in Triton and CUDA, reducing memory by 60% and speeding up training/inference. Benchmarks? Llama 3 8B: 2x faster fine-tuning, 1.6x inference. Even works on T4/A100 clusters.

### Hands-On Fine-Tuning Workflow
1. **Install**: `pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"`
2. **Load Model**:
   ```python
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/llama-3.1-8b-bnb-4bit",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)

1. Add LoRA Adapters: Low-rank adaptation for efficiency.

model = FastLanguageModel.get_peft_model( model, r=16, target_modules=["q_proj", "k_proj", "v_proj", "o_proj"], lora_alpha=16, lora_dropout=0, bias="none", use_gradient_checkpointing="unsloth", random_state=3407, )

4. **Train on Dataset**: Use HuggingFace-style `SFTTrainer` with your data (e.g., instruction-response pairs).
5. **Export**: Push to HuggingFace or merge for inference.

Real-world win: Fine-tune on your proprietary docs for RAG-enhanced Q&A. Daniel stressed: Test on free Colab first, scale to pro GPUs. Supports 100+ models—check [Unsloth GitHub](https://github.com/unslothai/unsloth) for notebooks.

## Building Production RAG Pipelines with LlamaIndex

Jerry Liu from LlamaIndex wrapped up with LLM-powered RAG, evolving from basic retrieval to multi-modal, agentic systems. RAG shines in knowledge-intensive apps: chat with PDFs, codebases, or images without hallucinations.

LlamaIndex (formerly GPT Index) abstracts indexing, retrieval, and synthesis. Key evolutions:

### Advanced RAG Techniques
- **Multi-Modal RAG**: Index docs + images/videos. Embed with CLIP-like models, retrieve jointly.
- **Corrective RAG (CRAG)**: LLM critiques retrieved chunks, fetches more if needed.
- **Adaptive RAG**: Router decides RAG vs. non-RAG based on query complexity.

### Practical Implementation
```python
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("Your question here")
print(response)

Jerry demoed a financial analyst agent: RAG over earnings calls + code synthesis for charts. For scale, integrate routers, knowledge graphs, or eval frameworks.

Grab the toolkit at LlamaIndex GitHub—pip install and build your first index in minutes. Ideal for startups ingesting user data or enterprises securing private LLMs.

Networking, Q&A, and What's Next

The evening sparked lively debates: Swarm vs. LangGraph? Unsloth on CPUs? RAG pitfalls? Attendees applied concepts on-site, forming collab groups.

Future meetups: SF, NYC, global. Follow @AIDevMeetup, @DeepLearningAI. Recordings/code soon on DeepLearning.AI.

Takeaways for Your Projects:

• Prototype agents with Swarm for dynamic workflows.
• Fine-tune affordably via Unsloth—cut costs 50%+.
• Level up search with LlamaIndex's RAG primitives.

Total word count pushes practical depth: Implement one today for immediate gains in your AI stack.