Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud

--- title: "Intro to tc Cloud Functors: A Graph-First Mental Model for the Modern Cloud" published: true description: This is the first part of a multipart series introducing _tc Cloud Functors_ tags: aws,serverless,devops,sre cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/98ecdssbss16lzbrrncd.png series: Introduction to tc-functors --- This is the first part of a multipart series introducing _[tc Cloud Functors](https://github.com/tc-functors)_ ## The _Monolith in the Desert_ Problem Sometimes I feel like the Forrest Gump of computing, having worked on or been around everything from the introduction of mini and microcomputers to starting an ISDN ISP in 1993, pioneering international server farms, and building early e-commerce sites through the earliest days of the cloud, working with AWS in 2008. (and so much more!)* Since then I've specialized in serverless infrastructure and systems. About three years ago, I joined a startup called [Informed](https://www.informediq.com), whose product is a very sophisticated Intelligent Document Processing pipeline for lenders. Informed had the classic situation of a startup that has found its market with a working product, but the implementation is a mirror of the _random walk_ they took to discover their market/product fit. They were processing about 8% of all US car loans, dealing with heavy bank regulations, and had the terrifying constraint of _infinite PII_. Yet, the tech stack was a typical startup tangle: a massive Ruby on Rails app sitting at the center of the universe, talking to just about every AWS service imaginable through Elastic Beanstalk, and using a single Postgres database as a message bus. I don't say this with disdain. It was a working system that had lots of quality engineering and had been very successful in delivering services to some of the top banks and generating profits. Most startups have this random wandering in the desert until they find their product/market fit, and their software tends to follow the same route. Unfortunately, it was also a nightmare of tech debt where adding a single feature felt like moving a mountain. We knew we had to move to a modular, event-driven architecture, but we weren't just talking about a couple of functions. We were looking at a future that now consists of **107 topologies** and **340 Lambdas** across Python, Ruby, and Node. Managing that kind of complexity for a small team of developers is the real hurdle. Traditional tools like `CloudFormation` which, frankly, I find to be a collection of frustrating _magic spells_. Using CloudFormation felt like trying to build a high-performance engine out of low-level, brittle bones. Even Terraform, while it _sucks less_, didn't have the right impedance match for the high-velocity _muscle_ of a distributed serverless system. We knew we wanted to move to a more unified and serverless style of infrastructure. The use case matched serverless perfectly, as its API driven, machine-to-machine work primarily. It's very bursty and follows the normal sine wave pattern of the US business hours. We also found that the existing tooling did not lend itself to building such complex serverless systems. Before we had `tc` our developers were spending 80% of their time manually _wiring_ infrastructure and only 20% writing logic; we had already lost. We needed to stop managing resources and start programming the cloud as a single computer. This frustration with manual wiring led us to a breakthrough: moving away from resource lists and toward a _graph-first_ mental model. Amazingly, company management approved the rewrite, and we actually were successful with first customers on it after 6 months and fully switched to the new system in 9 months. It was during this time that tc was developed based on our real needs and experiences. We continue to add new features as we discover the needs. ## The Mental Shift: From Resources to Graphs The _Aha! Moment_ happens when you realize most tools ask the wrong question. They focus on the _implementation_ (the _how_) rather than the _intent_ (the _what_). I scoured the _InterTubes_ for something better, but everything else was still resource-centric. | Question Asked | Primary Focus | Outcome | | --- | --- | --- | | Resource-First (Terraform/CloudFormation) | _What resources do you need?_ | A static list of primitives (buckets, roles) that must be manually wired. | | Function-First (Serverless Framework) | _What functions do you have?_ | A collection of compute units that still require manual infrastructure _glue_. | | Graph-First (tc) | _What is the graph of your system?_ | A logical topology where relationships are primary and infrastructure is derived algorithmically. | Think of it like the **React analogy**. Before React, we manually manipulated the DOM (the infrastructure). With React, we focus on component state and relationships; the _DOM manipulation_ is just a side effect handled by the framework. In the `tc` model, you describe how components connect, and the system handles the IAM roles, event rules, and subscriptions as derivative artifacts. It’s an inversion of control where the graph is the truth. To make this graph-based thinking concrete, we created a single unit of encapsulation: the **Cloud Functor**. ## What is a Cloud Functor? We borrowed the term _functor_ from _OCaml’s_ parameterized modules. In our world, a **Cloud Functor** is a first-class, composable unit of infrastructure. To understand it, just look at your file system as a similar model of hierarchy and other characteristics. **Namespaced**: Just like a directory, every functor has a unique identifier to keep things organized and domain-specific. **Sandboxed**: You can deploy isolated versions (dev, staging, feature-branch) in the same deployment account without collision. **Versioned**: We use Git tags and deployment manifests to _freeze_ implementations, making rollbacks as simple as pointing to a previous tag. **Isomorphic**: The definition is abstract; the same YAML renders consistently whether it's on your laptop or in production. The **File System Analogy** shows how you work through _Sub-Tree Autonomy_. You can `cd` into a deeply nested folder, say, `src/loan-app/extraction/ssn-id` and run `tc create` or `tc update` in that directory. The tool treats that subfolder as the _root of the universe_, ignoring the rest of the 340-Lambda monolith. This _zooming_ allows for infinite nesting in a fractal pattern; a top-level orchestrator delegates to sub-orchestrators, and every level looks the same. Every one of these functors is built from the same set of _atomic_ building blocks. Breaking away from the file system analogy, the top feature of Cloud Functors is their _Composability_. ## The 8 Atoms of the Cloud Universe We identified eight core entities that are sufficient to build any sophisticated serverless topology. | Entity | Cloud Mapping (AWS) | The _So What?_ (Benefit) | | --- | --- | --- | | Functions | Lambda / ECS | Pure business logic; side-effect free compute. | | States | Step Functions | The _managers_ handling flow control and error recovery. | | Events | EventBridge | The _nerves_ of the system; decoupled signals. | | Routes | API Gateway | Synchronous HTTP entry points. | | Mutations | AppSync (GraphQL) | Modern data modification for frontends. | | Queues | SQS | Buffering and load leveling between components. | | Channels | AppSync Events | Real-time, two-way communication via WebSockets. | | Pages | S3 & CloudFront | Static frontend assets (HTML/JS/CSS). | By defining these logical connections, the _spaghetti_ of infrastructure glue is derived algorithmically. This isn't just a convenience; it’s the secret to killing some of the most common security headaches in the cloud. ## Composability - The Real Superpower of tc Let's cut to the chase about the real superpower of `tc`: graph-based _composability_. Turns out, when you stop thinking in terms of disjointed infrastructure resources and start treating your cloud primitives: `functions`, `events`, `queues`, and `routes` as fundamental _atoms_, you can build some truly elegant, self-documenting systems. One simply defines the logical connections between these _namespaced_ and _sandboxed_ entities in a clean YAML graph, and the `tc composer` works its magic spells to infer all the complex IAM permissions and infrastructure glue for you. Because these relationships are elevated to explicit, first-class citizens rather than being scattered as obscure implementation details, the topology definition itself becomes a highly readable architecture diagram where no low-level infrastructure configuration leaks into your business logic. This means we can seamlessly compose these atomic entities into higher-order `Cloud Functors`, and then recursively compose *those* into massive enterprise systems, allowing my team to understand an entire topology of 30+ interconnected microservices at a glance without ever wading through the spaghetti code of traditional IaC. ## Algorithmic Security: Killing the _Confused Deputy_ In the trenches, I see developers _just getting it working_ by stripping away `SourceArn` or `ExternalId` conditions because IAM is hard. This creates the _confused deputy_ problem, where a service is tricked into acting on behalf of a malicious party. The `tc` **composer** solves this using **Algorithmic Security**: **Tested Templates**: The composer uses pre-validated, secure policy templates for every connection. It automatically adds `SourceArn` condition keys to trust policies. You literally cannot forget them because you don't write them. **Least Privilege by Default**: Since the graph knows Function A only talks to Queue B, the compiler generates a policy scoped strictly to those ARNs. No wildcards (`*`) allowed. Because you define the **topology** (logic) and not the **policy** (implementation), the compiler always regenerates the algorithmically secure version. While `tc` handles the _muscle_, we still need a skeleton to hold everything up. You still need someone with an understanding of IAM roles/policies to create the templates, but the enforcement is done by the tooling. ## The _Bones and Muscles_ Hybrid Architecture We don't try to replace Terraform (at least not yet!). Instead, we use a hybrid model that respects different infrastructure velocities: **The Bones (Terraform)**: This is your skeleton—heavy, static stuff like VPCs, RDS, and the **Event Bus**. You want your bones to be rigid and reliable. We will run Terraform to set up and maintain these static elements. We do not run Terraform regularly and don't really use it for drift management. Not only that, but we expect to eventually have tc be able to create and manage all AWS elements (and maybe other clouds someday). **The Muscles (tc)**: These are the high-velocity parts: Lambdas, Step Functions, and the **Rules/Subscriptions** on that bus. They attach to the bones and provide the actual motion. `tc` manages this fully **The Tendons (The Resolver)**: This is the magic. The `tc resolver` queries the _universe_ (AWS) to find the ARNs of the _bones_ as well as the state of the deployed functors and all the previously deployed services. `tc` talks directly to AWS via the AWS SDKs. The _resolver_ then uses that data to supply ARNs or other info needed to wire things up. ## The Developer Workflow: `Zooming` into Action The workflow is designed for speed. Because the `tc builder` uses **Docker buildx** for multi-architecture support (Linux/amd64), your builds are consistent regardless of your local machine or build by CI/CD in the Cloud. ```shell tc build # Package logic and ML models (Docker multi-arch) tc publish # Push assets like Layers and EFS models to S3 tc create # Provision the isolated, namespaced sandbox tc invoke # Test logic directly in the cloud via a REPL ``` The three pillars of this workflow are: **Sandboxing**: Collision-free environments (e.g., `dev-your-name`). Alice’s messages never end up in Bob’s queue. **Contextual Execution**: _Zoom_ into a sub-folder and iterate in seconds, ignoring the rest of the system. **Isomorphic Rendering**: The same YAML is used for your local dev sandbox and production. No more _it worked on my machine_. ## Wrap Up: The Cloud is the Computer At the end of the day, the goal is to stop being _infrastructure managers_ and start being _cloud programmers_. When we moved to this graph-first model at Informed, our implementation speed just zoomed. Being spoiled by Ruby and other high-level languages, I couldn't go back to manual wiring. We are now integrating LLMs/Agents with tc both for dev and targets, so things should go even faster. Turns out, once you stop fighting with low-level primitives and start thinking in graphs, building complex systems becomes far too easy. It’s truly a superpower. Stop thinking in resource lists. Start thinking in graphs. The cloud is your computer; it's time to start programming it properly. `tc Cloud Functors` is an open source project that can be found at https://github.com/tc-functors with documentation at https://tc-functors.org/.