<div align="center"> <img src="https://gfassets.fra1.cdn.digitaloceanspaces.com/logo/logo-mono.png" /><br /><br /> </div> <p align="center"> <a href="https://join.slack.com/t/glassflowhub/shared_invite/zt-2g3s6nhci-bb8cXP9g9jAQ942gHP5tqg"> <img src="https://img.shields.io/badge/slack-join-community?logo=slack&amp;logoColor=white&amp;style=flat" alt="Chat on Slack"></a>

GlassFlow Examples

This repository is your starting point for exploring examples and use cases for GlassFlow.

Pre-requisite

• Create your free GlassFlow account via the GlassFlow WebApp.
• Get your Personal Access Token to authorize the Python SDK to interact with GlassFlow Cloud from the WebApp here
• Clone this repository to your local machine to access all the examples:

  git clone https://github.com/glassflow/glassflow-examples.git
  

Explore

Usage

Learn how to use the GlassFlow Python SDK:

• Create Pipeline: How to create a new pipeline.
• Get Pipeline: How to get information about an existing pipeline.
• Manage Pipeline: How to update a pipeline and how to pause it.
• Send and Receive events: How to push and consume events to/from a pipeline.

Examples

These are some basic examples that you can try with GlassFlow. The examples here should be familiar to you as a data engineer and will show you how GlassFlow works and how you can interact with it.

• Hello World: Get started with GlassFlow with an hello world example

• Email Encryption: Showcases how to hash email addresses (or other PII information) before sending them to your data warehouse.

• Data Enrichment via API: Showcases how to call external API and enrich the data on the fly

• Spam Detection with OpenAI: Showcases how to leverage openai API during the data transformation stage directly within the pipeline

• PII detection and masking on unstructured data: Showcases how to do pii masking on unstructured data using hugging face

• Log format conversion: Showcases how to transform Open Telemetry log formats into a structure that can be ingested into ClickHouse.

• Unstructured to structured data transformation: Transform unstructured data like a YouTube video to extract key topics from the video transcript, generate additional insights, and translate the transcript into any specified language.

• Vector Embeddings: An example on how to feed vector embeddings into your vector database.

• Slack Alerting - Suspicious login: Use GlassFlow to send slack alert messages when there is a suspicious login

Connectors

Here there are basic examples on how to create pipelines with our managed connectors:

• Sources: Managed source connectors
• Sinks: Managed sink connectors
• Combination: Combination of source and sink connectors