From: aidotengineer

14.ai is developing an AI native customer support platform designed to interact directly with end users [00:00:05]. This platform relies on Large Language Models (LLMs) in production and must operate reliably under uncertain conditions [00:00:09].

Core Technology: Effect

To manage the complexity of building robust, type-safe, and composable systems, 14.ai utilizes Effect, a TypeScript library [00:00:16]. While TypeScript provides a strong foundation, it falls short when dealing with unreliable APIs, complex system dependencies, non-deterministic model outputs, or long-running workflows [00:00:27]. Effect offers tools to confidently handle such situations as the platform evolves [00:00:36].

Key features of Effect include:

  • Strong type guarantees across the stack [00:00:44]
  • Powerful composition primitives [00:00:48]
  • Built-in concurrency, streaming, interruptions, and retry mechanisms [00:00:50]
  • Structured error modeling [00:00:54]
  • A clean dependency injection system for easier testing and modernization [00:00:57]
  • Easy observability via OpenTelemetry [00:01:02]

Effect aids in building stable, testable, and maintainable code at scale [00:01:07]. It can be gradually adopted into an existing codebase and feels like a natural extension of TypeScript [00:01:14].

Platform Architecture

14.ai uses Effect across its entire stack [00:01:23]. The main components are:

  • React Front End Powers dashboards, the agents’ interface, knowledge management, insights, analytics, and SDKs [00:01:28].
  • Internal RPC Server Handles app logic, built on Effect RPC and a modified version of Tanstack Query on the front end [00:01:37].
  • Public API Server Uses Effect HTTP with autogenerated OpenAPI documentation from annotated schemas [00:01:47].
  • Data Processing Engine Syncs data from CRM, documents, and databases, processing it for real-time analytics and reporting [00:01:53].
  • Agent Workflows Written in a custom Domain Specific Language (DSL) built on Effect, allowing a mix of deterministic and non-deterministic behavior [00:02:00].
  • PostgreSQL Database Used for both data and vector storage, with Effect SQL handling queries [00:02:09].

Effect schemas are used throughout the system for runtime validation, encoding, decoding, type-safe input/output handling, and autogenerated documentation [00:02:15].

Agent Design and Workflows

The AI agents function as planners: they receive user input, devise a plan, select the appropriate action, workflow, or sub-agent, execute it, and repeat until the task is complete [00:02:28].

  • Actions Small, focused units of execution, similar to tool calls (e.g., fetching payment info, searching logs) [00:02:40].
  • Workflows Deterministic multi-step processes (e.g., cancelling a subscription involving collecting a reason, offering retention, checking eligibility, and performing cancellation) [00:02:51].
  • Sub-agents Group related actions and workflows into larger, domain-specific modules (e.g., a billing agent or a log retrieval agent) [00:03:03].

To model this complexity, a domain-specific language for workflows was built using Effect’s functional pipe-based system, enabling clear and composable expression of branching, sequencing, retries, state transitions, and memory [00:03:14].

Ensuring Reliability

Given that these systems power mission-critical operations, reliability is paramount [00:03:32].

  • LLM Provider Fallback If one LLM provider fails, the system falls back to another with similar performance characteristics (e.g., GPT-4 Mini falling back to Gemini Flash 2.0 for tool calling) [00:03:36]. This is modeled with retry policies that track state to avoid retrying failed providers [00:03:47].
  • Token Stream Duplication Answers are often streamed to the end user. Token streams are duplicated, sending one directly to the user and another for internal storage (e.g., for analytics) [00:03:55]. Effect facilitates this process [00:04:05].
  • Dependency Injection for Testing Heavy use of dependency injection allows for mocking LLM providers and simulating failure scenarios. This approach enables swapping service providers with mock versions without affecting the system’s internals [00:04:08].

Developer Experience

Effect provides an excellent developer experience for building agentic systems [00:04:25].

  • Schema-Centric Design Input, output, and error types are defined upfront with powerful encoding/decoding built-in, providing strong type safety guarantees and automatic documentation [00:04:32].
  • Dependency Injection Services are provided at the system’s entry point, enabling composability and easy mocking for testing [00:04:48]. Dependencies are type-level, guaranteeing at compile time that all required services are provided [00:04:59].
  • Modularity Services are modular and composable, allowing easy overriding of behavior or swapping implementations without affecting system internals [00:05:06].
  • Strong Guard Rails Effect helps prevent common mistakes, allowing engineers new to TypeScript to become productive quickly [00:05:14].

Lessons Learned and Adoption

While powerful, using Effect effectively requires discipline [00:05:31]. It’s easy to write clean code for the “happy path,” but one must be careful not to accidentally catch errors upstream or silently lose important failures [00:05:37]. Dependency injection can be challenging to grasp at scale, as tracing service provision across multiple layers or subsystems can be difficult [00:05:55].

Effect has a significant learning curve due to its extensive ecosystem of concepts and tools [00:06:10]. However, once past the initial bump, the benefits compound [00:06:17].

Effect helps build predictable and resilient systems but is not magic; developers still need to think [00:06:27]. A key advantage is its incremental adoption capability; one can start with a single service or endpoint and expand from there [00:06:37].

Effect is particularly useful for LLM and AI based systems where reliability and coping with non-determinism are crucial [00:06:47]. It provides tools for predictable and observable systems [00:06:53]. It brings the rigor of functional programming into real-world TypeScript in a practical way for production use, offering value even without being a functional programming purist [00:06:58].