TypeScript and Effect Library usage

From: aidotengineer
The Effect library is a TypeScript library designed for building robust, type-safe, and composable systems [00:00:16]. While TypeScript provides a strong foundation, it can fall short when dealing with unreliable APIs, complex inter-system dependencies, non-deterministic model outputs, or long-running workflows [00:00:27]. Effect provides tools to handle these situations confidently [00:00:36].

14.ai, a company building an AI-native customer support platform, uses Effect to manage the complexity of systems that interact directly with end-users and rely on LLMs in production under uncertain conditions [00:00:07].

Benefits of Effect

The Effect library offers several key features:

• Strong type guarantees across the stack [00:00:44]
• Powerful composition primitives [00:00:48]
• Built-in concurrency, streaming, interruptions, and retry mechanisms [00:00:50]
• Structured error modeling [00:00:54]
• A clean dependency injection system that facilitates testing and modernization [00:00:57]
• Easy observability via Open Telemetry [00:01:02]
• The ability for gradual adoption into existing codebases [00:01:14]
• It feels like a natural extension of TypeScript [00:01:16]

14.ai Architecture and Effect Integration

14.ai utilizes Effect across its entire stack [00:01:23]. Key components include:

• Front End: A React front end powers dashboards, the agents ID, knowledge management, insights, analytics, and SDKs [00:01:28].
• Internal RPC Server: Handles application logic, built on Effect RPC and a modified version of TanStack Query [00:01:39].
• Public API Server: Uses Effect HTTP, with Open API documentation automatically generated from annotated schemas [00:01:47].
• Data Processing Engine: Synchronizes data from CRMs, documents, and databases for real-time analytics and reporting [00:01:53].
• Agent Workflows: Written in a custom Domain Specific Language (DSL) built on Effect, enabling a mix of deterministic and non-deterministic behaviors [00:02:02].
• Database: PostgreSQL is used for both data and vector storage, with Effect SQL handling queries [00:02:09].
• Schema Validation: Everything is modeled using Effect Schemas, providing runtime validation, encoding, decoding, type-safe input/output handling, and auto-generated documentation [00:02:15].

Agentic Systems and Workflows

14.ai’s agents function as planners, taking user input, devising a plan, choosing actions, workflows, or sub-agents, executing them, and repeating until a task is complete [00:02:28].

• Actions: Small, focused units of execution, similar to tool calls, such as fetching payment information or searching logs [00:02:40].
• Workflows: Deterministic, multi-step processes, like canceling a subscription, which might involve collecting a reason, offering retention options, checking eligibility, and performing the cancellation [00:02:51].
• Sub-agents: Group related actions and workflows into larger, domain-specific modules, such as a billing agent or a log retrieval agent [00:03:03].

To model this complexity, a domain-specific language for workflows was built using Effect’s functional, pipe-based system. This allows for clear and composable expression of branching, sequencing, retries, state transitions, and memory [00:03:14].

Reliability and Testing

Reliability is paramount for mission-critical systems [00:03:32].

• Fallback Mechanisms: If one LLM provider fails, the system falls back to another with similar performance characteristics (e.g., GPT-4 mini to GD Flash 2.0 for tool calling) [00:03:36].
• Retry Policies: These policies model and track state to avoid retrying failed providers [00:03:47].
• Streaming: Answers are streamed to the end-user, with token streams duplicated for direct user delivery and internal storage (e.g., for analytics), a process facilitated by Effect [00:03:55].
• Testing: Heavy use of dependency injection allows mocking LLM providers and simulating failure scenarios. This approach enables easy swapping of services with mock versions without affecting system internals [00:04:08].

Developer Experience

Effect significantly enhances developer experience when building agentic systems:

• Schema-Centric: Inputs, outputs, and error types are defined upfront using schemas with built-in encoding/decoding, strong type safety, and automatic documentation [00:04:32].
• Dependency Injection: Services are provided at system entry points, are composable, and easily mockable for testing. Dependencies are guaranteed at compile time at the type level [00:04:48].
• Modularity: Services are modular and composable, allowing easy overriding of behavior or swapping implementations without internal system changes [00:05:06].
• Guard Rails: Effect helps prevent common mistakes, allowing engineers new to TypeScript to become productive quickly and making it harder to fall into bad patterns after the initial learning curve [00:05:14].

Lessons Learned

While powerful, using Effect effectively requires discipline [00:05:31].

• Error Handling: It’s easy to accidentally catch errors upstream or out of sight, silently losing important failures if not careful [00:05:49].
• Dependency Injection at Scale: While great in principle, tracing service provision, especially across multiple layers or subsystems, can become challenging [00:05:57].
• Learning Curve: Effect has a significant ecosystem with many concepts and tools, which can be overwhelming initially. However, once the initial bump is overcome, the benefits compound [00:06:10].

Effect helps build predictable and resilient systems but is not a magical solution; developers still need to think critically [00:06:27].

Incremental Adoption and Conclusion

A key advantage of Effect is its support for incremental adoption. Users can start with a single service or endpoint and build from there [00:06:37].

Effect is particularly useful for LLM and AI-based systems where reliability and coping with non-determinism are crucial [00:06:47]. It provides tools for predictable and observable systems [00:06:53]. It also brings the rigor of functional programming into real-world TypeScript in a practical way for production use, without requiring users to be functional programming purists [00:07:01].