--- title: Coming From ZIO description: Key differences between Effect and ZIO. sidebar: order: 3 --- If you are coming to Effect from ZIO, there are a few differences to be aware of. ## Environment In Effect, we represent the environment required to run an effect workflow as a **union** of services: **Example** This may be confusing to folks coming from ZIO, where the environment is represented as an **intersection** of services: ## Rationale The rationale for using a union to represent the environment required by an `Effect` workflow boils down to our desire to remove `Has` as a wrapper for services in the environment . To be able to remove `Has` from Effect, we had to think a bit more structurally given that TypeScript is a structural type system. In TypeScript, if you have a type `A & B` where there is a structural conflict between `A` and `B`, the type `A & B` will reduce to `never`. **Example** In previous versions of Effect, intersections were used for representing an environment with multiple services. The problem with using intersections is that there could be multiple services in the environment that have functions and properties named in the same way. To remedy this, we wrapped services in the `Has` type , so you would have `Has & Has` in your environment. In ZIO 2.0, the _contravariant_ `R` type parameter of the `ZIO` type became fully phantom, thus allowing for removal of the `Has` type. This significantly improved the clarity of type signatures as well as removing another "stumbling block" for new users. To facilitate removal of `Has` in Effect, we had to consider how types in the environment compose. By the rule of composition, contravariant parameters composed as an intersection are equivalent to covariant parameters composed together as a union for purposes of assignability. Based upon this fact, we decided to diverge from ZIO and make the `R` type parameter _covariant_ given `A | B` does not reduce to `never` if `A` and `B` have conflicts. From our example above: Representing `R` as a covariant type parameter containing the union of services required by an `Effect` workflow allowed us to remove the requirement for `Has`. ## Type Aliases In Effect, there are no predefined type aliases such as `UIO`, `URIO`, `RIO`, `Task`, or `IO` like in ZIO. The reason for this is that type aliases are lost as soon as you compose them, which renders them somewhat useless unless you maintain **multiple** signatures for **every** function. In Effect, we have chosen not to go down this path. Instead, we utilize the `never` type to indicate unused types. It's worth mentioning that the perception of type aliases being quicker to understand is often just an illusion. In Effect, the explicit notation `Effect` clearly communicates that only type `A` is being used. On the other hand, when using a type alias like `RIO`, questions arise about the type `E`. Is it `unknown`? `never`? Remembering such details becomes challenging. --- title: Effect vs fp-ts description: Comparison of Effect and fp-ts, covering features like typed services, resource management, concurrency, and stream processing. sidebar: order: 4 --- ## Key Developments - **Project Merger**: The fp-ts project is officially merging with the Effect-TS ecosystem. Giulio Canti, the author of fp-ts, is being welcomed into the Effect organization. For more details, see the . - **Continuity and Evolution**: Effect can be seen as the successor to fp-ts v2 and is effectively fp-ts v3, marking a significant evolution in the library's capabilities. ## FAQ ### Bundle Size Comparison Between Effect and fp-ts **Q: I compared the bundle sizes of two simple programs using Effect and fp-ts. Why does Effect have a larger bundle size?** A: It's natural to observe different bundle sizes because Effect and fp-ts are distinct systems designed for different purposes. Effect's bundle size is larger due to its included fiber runtime, which is crucial for its functionality. While the initial bundle size may seem large, the overhead amortizes as you use Effect. **Q: Should I be concerned about the bundle size difference when choosing between Effect and fp-ts?** A: Not necessarily. Consider the specific requirements and benefits of each library for your project. The **Micro** module in Effect is designed as a lightweight alternative to the standard `Effect` module, specifically for scenarios where reducing bundle size is crucial. This module is self-contained and does not include more complex features like `Layer`, `Ref`, `Queue`, and `Deferred`. If any major Effect modules are used, the effect runtime will be added to your bundle, negating the benefits of Micro. This makes Micro ideal for libraries that aim to implement Effect functionality with minimal impact on bundle size, especially for libraries that plan to expose `Promise`-based APIs. It also supports scenarios where a client might use Micro while a server uses the full suite of Effect features, maintaining compatibility and shared logic between different parts of an application. ## Comparison Table The following table compares the features of the Effect and libraries. | Feature | fp-ts | Effect | | ------------------------- | ----- | ------ | | Typed Services | ❌ | ✅ | | Built-in Services | ❌ | ✅ | | Typed errors | ✅ | ✅ | | Pipeable APIs | ✅ | ✅ | | Dual APIs | ❌ | ✅ | | Testability | ❌ | ✅ | | Resource Management | ❌ | ✅ | | Interruptions | ❌ | ✅ | | Defects | ❌ | ✅ | | Fiber-Based Concurrency | ❌ | ✅ | | Fiber Supervision | ❌ | ✅ | | Retry and Retry Policies | ❌ | ✅ | | Built-in Logging | ❌ | ✅ | | Built-in Scheduling | ❌ | ✅ | | Built-in Caching | ❌ | ✅ | | Built-in Batching | ❌ | ✅ | | Metrics | ❌ | ✅ | | Tracing | ❌ | ✅ | | Configuration | ❌ | ✅ | | Immutable Data Structures | ❌ | ✅ | | Stream Processing | ❌ | ✅ | Here's an explanation of each feature: ### Typed Services Both fp-ts and Effect libraries provide the ability to track requirements at the type level, allowing you to define and use services with specific types. In fp-ts, you can utilize the `ReaderTaskEither` type, while in Effect, the `Effect` type is available. It's important to note that in fp-ts, the `R` type parameter is contravariant, which means that there is no guarantee of avoiding conflicts, and the library offers only basic tools for dependency management. On the other hand, in Effect, the `R` type parameter is covariant and all APIs have the ability to merge dependencies at the type level when multiple effects are involved. Effect also provides a range of specifically designed tools to simplify the management of dependencies, including `Tag`, `Context`, and `Layer`. These tools enhance the ease and flexibility of handling dependencies in your code, making it easier to compose and manage complex applications. ### Built-in Services The Effect library has built-in services like `Clock`, `Random` and `Tracer`, while fp-ts does not provide any default services. ### Typed errors Both libraries support typed errors, enabling you to define and handle errors with specific types. However, in Effect, all APIs have the ability to merge errors at the type-level when multiple effects are involved, and each effect can potentially fail with different types of errors. This means that when combining multiple effects that can fail, the resulting type of the error will be a union of the individual error types. Effect provides utilities and type-level operations to handle and manage these merged error types effectively. ### Pipeable APIs Both fp-ts and Effect libraries provide pipeable APIs, allowing you to compose and sequence operations in a functional and readable manner using the `pipe` function. However, Effect goes a step further and offers a `.pipe` method on each data type, making it more convenient to work with pipelines without the need to explicitly import the `pipe` function every time. ### Dual APIs Effect library provides dual APIs, allowing you to use the same API in different ways . ### Testability The functional style of fp-ts generally promotes good testability of the code written using it, but the library itself does not provide dedicated tools specifically designed for the testing phase. On the other hand, Effect takes testability a step further by offering additional tools that are specifically tailored to simplify the testing process. Effect provides a range of utilities that improve testability. For example, it offers the `TestClock` utility, which allows you to control the passage of time during tests. This is useful for testing time-dependent code. Additionally, Effect provides the `TestRandom` utility, which enables fully deterministic testing of code that involves randomness. This ensures consistent and predictable test results. Another helpful tool is `ConfigProvider.fromMap`, which makes it easy to define mock configurations for your application during testing. ### Resource Management The Effect library provides built-in capabilities for resource management, while fp-ts has limited features in this area and they are less sophisticated. In Effect, resource management refers to the ability to acquire and release resources, such as database connections, file handles, or network sockets, in a safe and controlled manner. The library offers comprehensive and refined mechanisms to handle resource acquisition and release, ensuring proper cleanup and preventing resource leaks. ### Interruptions The Effect library supports interruptions, which means you can interrupt and cancel ongoing computations if needed. This feature gives you more control over the execution of your code and allows you to handle situations where you want to stop a computation before it completes. In Effect, interruptions are useful in scenarios where you need to handle user cancellations, timeouts, or other external events that require stopping ongoing computations. You can explicitly request an interruption and the library will safely and efficiently halt the execution of the computation. On the other hand, fp-ts does not have built-in support for interruptions. Once a computation starts in fp-ts, it will continue until it completes or encounters an error, without the ability to be interrupted midway. ### Defects The Effect library provides mechanisms for handling defects and managing **unexpected** failures. In Effect, defects refer to unexpected errors or failures that can occur during the execution of a program. With the Effect library, you have built-in tools and utilities to handle defects in a structured and reliable manner. It offers error handling capabilities that allow you to catch and handle exceptions, recover from failures, and gracefully handle unexpected scenarios. On the other hand, fp-ts does not have built-in support specifically dedicated to managing defects. While you can handle errors using standard functional programming techniques in fp-ts, the Effect library provides a more comprehensive and streamlined approach to dealing with defects. ### Fiber-Based Concurrency The Effect library leverages fiber-based concurrency, which enables lightweight and efficient concurrent computations. In simpler terms, fiber-based concurrency allows multiple tasks to run concurrently, making your code more responsive and efficient. With fiber-based concurrency, the Effect library can handle concurrent operations in a way that is lightweight and doesn't block the execution of other tasks. This means that you can run multiple computations simultaneously, taking advantage of the available resources and maximizing performance. On the other hand, fp-ts does not have built-in support for fiber-based concurrency. While fp-ts provides a rich set of functional programming features, it doesn't have the same level of support for concurrent computations as the Effect library. ### Fiber Supervision Effect library provides supervision strategies for managing and monitoring fibers. fp-ts does not have built-in support for fiber supervision. ### Retry and Retry Policies The Effect library includes built-in support for retrying computations with customizable retry policies. This feature is not available in fp-ts out of the box, and you would need to rely on external libraries to achieve similar functionality. However, it's important to note that the external libraries may not offer the same level of sophistication and fine-tuning as the built-in retry capabilities provided by the Effect library. Retry functionality allows you to automatically retry a computation or action when it fails, based on a set of predefined rules or policies. This can be particularly useful in scenarios where you are working with unreliable or unpredictable resources, such as network requests or external services. The Effect library provides a comprehensive set of retry policies that you can customize to fit your specific needs. These policies define the conditions for retrying a computation, such as the number of retries, the delay between retries, and the criteria for determining if a retry should be attempted. By leveraging the built-in retry functionality in the Effect library, you can handle transient errors or temporary failures in a more robust and resilient manner. This can help improve the overall reliability and stability of your applications, especially in scenarios where you need to interact with external systems or services. In contrast, fp-ts does not offer built-in support for retrying computations. If you require retry functionality in fp-ts, you would need to rely on external libraries, which may not provide the same level of sophistication and flexibility as the Effect library. It's worth noting that the built-in retry capabilities of the Effect library are designed to work seamlessly with its other features, such as error handling and resource management. This integration allows for a more cohesive and comprehensive approach to handling failures and retries within your computations. ### Built-in Logging The Effect library comes with built-in logging capabilities. This means that you can easily incorporate logging into your applications without the need for additional libraries or dependencies. In addition, the default logger provided by Effect can be replaced with a custom logger to suit your specific logging requirements. Logging is an essential aspect of software development as it allows you to record and track important information during the execution of your code. It helps you monitor the behavior of your application, debug issues, and gather insights for analysis. With the built-in logging capabilities of the Effect library, you can easily log messages, warnings, errors, or any other relevant information at various points in your code. This can be particularly useful for tracking the flow of execution, identifying potential issues, or capturing important events during the operation of your application. On the other hand, fp-ts does not provide built-in logging capabilities. If you need logging functionality in fp-ts, you would need to rely on external libraries or implement your own logging solution from scratch. This can introduce additional complexity and dependencies into your codebase. ### Built-in Scheduling The Effect library provides built-in scheduling capabilities, which allows you to manage the execution of computations over time. This feature is not available in fp-ts. In many applications, it's common to have tasks or computations that need to be executed at specific intervals or scheduled for future execution. For example, you might want to perform periodic data updates, trigger notifications, or run background processes at specific times. This is where built-in scheduling comes in handy. On the other hand, fp-ts does not have built-in scheduling capabilities. If you need to schedule tasks or manage timed computations in fp-ts, you would have to rely on external libraries or implement your own scheduling mechanisms, which can add complexity to your codebase. ### Built-in Caching The Effect library provides built-in caching mechanisms, which enable you to cache the results of computations for improved performance. This feature is not available in fp-ts. In many applications, computations can be time-consuming or resource-intensive, especially when dealing with complex operations or accessing remote resources. Caching is a technique used to store the results of computations so that they can be retrieved quickly without the need to recompute them every time. With the built-in caching capabilities of the Effect library, you can easily cache the results of computations and reuse them when needed. This can significantly improve the performance of your application by avoiding redundant computations and reducing the load on external resources. ### Built-in Batching The Effect library offers built-in batching capabilities, which enable you to combine multiple computations into a single batched computation. This feature is not available in fp-ts. In many scenarios, you may need to perform multiple computations that share similar inputs or dependencies. Performing these computations individually can result in inefficiencies and increased overhead. Batching is a technique that allows you to group these computations together and execute them as a single batch, improving performance and reducing unnecessary processing. ### Metrics The Effect library includes built-in support for collecting and reporting metrics related to computations and system behavior. It specifically supports . This feature is not available in fp-ts. Metrics play a crucial role in understanding and monitoring the performance and behavior of your applications. They provide valuable insights into various aspects, such as response times, resource utilization, error rates, and more. By collecting and analyzing metrics, you can identify performance bottlenecks, optimize your code, and make informed decisions to improve your application's overall quality. ### Tracing The Effect library has built-in tracing capabilities, which enable you to trace and debug the execution of your code and track the path of a request through an application. Additionally, Effect offers a dedicated for integrating with the OpenTelemetry observability framework. In contrast, fp-ts does not offer a similar tracing tool to enhance visibility into code execution. ### Configuration The Effect library provides built-in support for managing and accessing configuration values within your computations. This feature is not available in fp-ts. Configuration values are an essential aspect of software development. They allow you to customize the behavior of your applications without modifying the code. Examples of configuration values include database connection strings, API endpoints, feature flags, and various settings that can vary between environments or deployments. With the Effect library's built-in support for configuration, you can easily manage and access these values within your computations. It provides convenient utilities and abstractions to load, validate, and access configuration values, ensuring that your application has the necessary settings it requires to function correctly. By leveraging the built-in configuration support in the Effect library, you can: - Load configuration values from various sources such as environment variables, configuration files, or remote configuration providers. - Validate and ensure that the loaded configuration values adhere to the expected format and structure. - Access the configuration values within your computations, allowing you to use them wherever necessary. ### Immutable Data Structures The Effect library provides built-in support for immutable data structures such as `Chunk`, `HashSet`, and `HashMap`. These data structures ensure that once created, their values cannot be modified, promoting safer and more predictable code. In contrast, fp-ts does not have built-in support for such data structures and only provides modules that add additional APIs to standard data types like `Set` and `Map`. While these modules can be useful, they do not offer the same level of performance optimizations and specialized operations as the built-in immutable data structures provided by the Effect library. Immutable data structures offer several benefits, including: - Immutability: Immutable data structures cannot be changed after they are created. This property eliminates the risk of accidental modifications and enables safer concurrent programming. - Predictability: With immutable data structures, you can rely on the fact that their values won't change unexpectedly. This predictability simplifies reasoning about code behavior and reduces bugs caused by mutable state. - Sharing and Reusability: Immutable data structures can be safely shared between different parts of your program. Since they cannot be modified, you don't need to create defensive copies, resulting in more efficient memory usage and improved performance. ### Stream Processing The Effect ecosystem provides built-in support for stream processing, enabling you to work with streams of data. Stream processing is a powerful concept that allows you to efficiently process and transform continuous streams of data in a reactive and asynchronous manner. However, fp-ts does not have this feature built-in and relies on external libraries like RxJS to handle stream processing. --- title: Effect vs neverthrow description: Comparison of Effect and neverthrow, covering features like type safety and error handling. sidebar: order: 6 --- import { Tabs, TabItem } from "@astrojs/starlight/components" When working with error handling in TypeScript, both and Effect provide useful abstractions for modeling success and failure without exceptions. They share many concepts, such as wrapping computations in a safe container, transforming values with `map`, handling errors with `mapErr`/`mapLeft`, and offering utilities to combine or unwrap results. This page shows a side-by-side comparison of neverthrow and Effect APIs for common use cases. If you're already familiar with neverthrow, the examples will help you understand how to achieve the same patterns with Effect. If you're starting fresh, the comparison highlights the similarities and differences so you can decide which library better fits your project. neverthrow exposes **instance methods** `). Effect exposes **functions** on `Either` `) and supports a `pipe` style for readability and better tree shaking. ## Synchronous API ### ok **Example** ### err **Example** ### map **Example** ### mapErr **Example** ### unwrapOr **Example** ### andThen **Example** ### asyncAndThen **Example** ### orElse **Example** ### match **Example** ### asyncMap **Example** **Note**. In neverthrow, `asyncMap` works with Promises directly. In Effect, passing a Promise to combinators like `Effect.andThen` automatically lifts it into an `Effect`. If the Promise rejects, the rejection is turned into an `UnknownException`, which is why the error type is widened to `string | UnknownException`. ### combine **Example** ### combineWithAllErrors **Example** **Note**. There is no exact equivalent of `Result.combineWithAllErrors` in Effect. Use `Array.getLefts` to collect all errors and `Array.getRights` to collect all successes. ## Asynchronous API In the examples below we use `Effect.runPromise` to run an effect and return a `Promise`. You can also use other APIs such as `Effect.runPromiseExit`, which can capture additional cases like defects and interruptions. ### okAsync **Example** ### errAsync **Example** ### fromThrowable **Example** ### map **Example** ### mapErr **Example** ### unwrapOr **Example** ### andThen **Example** ### orElse **Example** ### match **Example** ### combine **Example** ### combineWithAllErrors **Example** ## Utilities ### fromThrowable **Example** ### safeTry **Example** **Note**. With `Either.gen`, you do not need to wrap the final value with `Either.right`. The generator's return value becomes the `Right`. You can also use an async generator function with `safeTry` to represent an asynchronous block. On the Effect side, the same pattern is written with `Effect.gen` instead of `Either.gen`. **Example** **Note**. With `Effect.gen`, you do not need to wrap the final value with `Effect.succeed`. The generator's return value becomes the `Success`. --- title: Effect vs Promise description: Comparison of Effect and Promise, covering features like type safety, concurrency, and error handling. sidebar: order: 5 --- import { Tabs, TabItem } from "@astrojs/starlight/components" In this guide, we will explore the differences between `Promise` and `Effect`, two approaches to handling asynchronous operations in TypeScript. We'll discuss their type safety, creation, chaining, and concurrency, providing examples to help you understand their usage. ## Comparing Effects and Promises: Key Distinctions - **Evaluation Strategy:** Promises are eagerly evaluated, whereas effects are lazily evaluated. - **Execution Mode:** Promises are one-shot, executing once, while effects are multi-shot, repeatable. - **Interruption Handling and Automatic Propagation:** Promises lack built-in interruption handling, posing challenges in managing interruptions, and don't automatically propagate interruptions, requiring manual abort controller management. In contrast, effects come with interruption handling capabilities and automatically compose interruption, simplifying management locally on smaller computations without the need for high-level orchestration. - **Structured Concurrency:** Effects offer structured concurrency built-in, which is challenging to achieve with Promises. - **Error Reporting :** Promises don't inherently provide detailed error reporting at the type level, whereas effects do, offering type-safe insight into error cases. - **Runtime Behavior:** The Effect runtime aims to remain synchronous as long as possible, transitioning into asynchronous mode only when necessary due to computation requirements or main thread starvation. ## Type safety Let's start by comparing the types of `Promise` and `Effect`. The type parameter `A` represents the resolved value of the operation: Here's what sets `Effect` apart: - It allows you to track the types of errors statically through the type parameter `Error`. For more information about error management in `Effect`, see . - It allows you to track the types of required dependencies statically through the type parameter `Context`. For more information about context management in `Effect`, see . ## Creating ### Success Let's compare creating a successful operation using `Promise` and `Effect`: ### Failure Now, let's see how to handle failures with `Promise` and `Effect`: ### Constructor Creating operations with custom logic: ## Thenable Mapping the result of an operation: ### map ### flatMap Chaining multiple operations: ## Comparing Effect.gen with async/await If you are familiar with `async`/`await`, you may notice that the flow of writing code is similar. Let's compare the two approaches: It's important to note that although the code appears similar, the two programs are not identical. The purpose of comparing them side by side is just to highlight the resemblance in how they are written. ## Concurrency ### Promise.all ### Promise.allSettled ### Promise.any ### Promise.race ## FAQ **Question**. What is the equivalent of starting a promise without immediately waiting for it in Effects? **Answer:** You can achieve this by utilizing `Effect.fork` and `Fiber.join`. --- title: Myths About Effect description: Debunking common misconceptions about Effect's performance, complexity, and use cases. sidebar: label: Myths order: 1 --- ## Effect heavily relies on generators and generators are slow! Effect's internals are not built on generators, we only use generators to provide an API which closely mimics async-await. Internally async-await uses the same mechanics as generators and they are equally performant. So if you don't have a problem with async-await you won't have a problem with Effect's generators. Where generators and iterables are unacceptably slow is in transforming collections of data, for that try to use plain arrays as much as possible. ## Effect will make your code 500x slower! Effect does perform 500x slower if you are comparing: to The reason is one operation is optimized by the JIT compiler to be a direct CPU instruction and the other isn't. In reality you'd never use Effect in such cases, Effect is an app-level library to tame concurrency, error handling, and much more! You'd use Effect to coordinate your thunks of code, and you can build your thunks of code in the best perfoming manner as you see fit while still controlling execution through Effect. ## Effect has a huge performance overhead! Depends what you mean by performance, many times performance bottlenecks in JS are due to bad management of concurrency. Thanks to structured concurrency and observability it becomes much easier to spot and optimize those issues. There are apps in frontend running at 120fps that use Effect intensively, so most likely effect won't be your perf problem. In regards of memory, it doesn't use much more memory than a normal program would, there are a few more allocations compared to non Effect code but usually this is no longer the case when the non Effect code does the same thing as the Effect code. The advice would be start using it and monitor your code, optimise out of need not out of thought, optimizing too early is the root of all evils in software design. ## The bundle size is HUGE! Effect's minimum cost is about 25k of gzipped code, that chunk contains the Effect Runtime and already includes almost all the functions that you'll need in a normal app-code scenario. From that point on Effect is tree-shaking friendly so you'll only include what you use. Also when using Effect your own code becomes shorter and terser, so the overall cost is amortized with usage, we have apps where adopting Effect in the majority of the codebase led to reduction of the final bundle. ## Effect is impossible to learn, there are so many functions and modules! True, the full Effect ecosystem is quite large and some modules contain 1000s of functions, the reality is that you don't need to know them all to start being productive, you can safely start using Effect knowing just 10-20 functions and progressively discover the rest, just like you can start using TypeScript without knowing every single NPM package. A short list of commonly used functions to begin are: - - - - - - - - - - - - - - - A short list of commonly used modules: - - - - - - - ## Effect is the same as RxJS and shares its problems This is a sensitive topic, let's start by saying that RxJS is a great project and that it has helped millions of developers write reliable software and we all should be thankful to the developers who contributed to such an amazing project. Discussing the scope of the projects, RxJS aims to make working with Observables easy and wants to provide reactive extensions to JS, Effect instead wants to make writing production-grade TypeScript easy. While the intersection is non-empty the projects have fundamentally different objectives and strategies. Sometimes people refer to RxJS in bad light, and the reason isn't RxJS in itself but rather usage of RxJS in problem domains where RxJS wasn't thought to be used. Namely the idea that "everything is a stream" is theoretically true but it leads to fundamental limitations on developer experience, the primary issue being that streams are multi-shot and mutable delimited continuations are known to be only good to represent single-shot effects . In short it means that writing in imperative style is practically impossible with stream primitives , forcing the developer to use declarative approaches such as pipe to represent all of their code. Effect has a Stream module , but the basic Effect type is single-shot and it is optimised to act as a smart & lazy Promise that enables imperative programming, so when using Effect you're not forced to use a declarative style for everything and you can program using a model which is similar to async-await. The other big difference is that RxJS only cares about the happy-path with explicit types, it doesn't offer a way of typing errors and dependencies, Effect instead consider both errors and dependencies as explicitely typed and offers control-flow around those in a fully type-safe manner. In short if you need reactive programming around Observables, use RxJS, if you need to write production-grade TypeScript that includes by default native telemetry, error handling, dependency injection, and more use Effect. ## Effect should be a language or Use a different language Neither solve the issue of writing production grade software in TypeScript. TypeScript is an amazing language to write full stack code with deep roots in the JS ecosystem and wide compatibility of tools, it is an industrial language adopted by many large scale companies. The fact that something like Effect is possible within the language and the fact that the language supports things such as generators that allows for imperative programming with custom types such as Effect makes TypeScript a unique language. In fact even in functional languages such as Scala the interop with effect systems is less optimal than it is in TypeScript, to the point that effect system authors have expressed wish for their language to support as much as TypeScript supports. --- title: Getting Started description: Learn how to use Effect's AI integration packages to define LLM interactions sidebar: order: 1 --- import { Aside, Steps, Tabs, TabItem } from "@astrojs/starlight/components" In this getting started guide, we will demonstrate how to generate a simple text completion using an LLM provider using the Effect AI integration packages. We’ll walk through: - Writing provider-agnostic logic to interact with an LLM - Declaring the specific LLM model to use for the interaction - Using a provider integration to make the program executable ## Installation First, we will need to install the base `@effect/ai` package to gain access to the core AI abstractions. In addition, we will need to install at least one provider integration package : ## Define an Interaction with a Language Model First let's define a simple interaction with a large language model : **Example** ## Select a Provider Next, we need to select which model provider we want to use: **Example** Before moving on, it is important that we understand the purpose of the `Model` data type. ## Understanding `Model` The `Model` data type represents a **provider-specific implementation** of one or more services, such as `LanguageModel` or `EmbeddingsModel`. It is the primary way that you can plug a real large language model into your program. An `Model` has three generic type parameters: - **ProviderName** - the name of the large language model provider that will be used - **Provides** - the services this model will provide when built - **Requires** - the services this model will require to be built This allows Effect to track which services the `Model` requires as well as which services the `Model` will provide. ### Creating n `Model` To create a `Model`, you can use the model-specific factory from one of Effect's provider integration packages. **Example** This creates a `Model` that: - **Provides** the `ProviderName` service, which allows introspection of the current provider in use by the program - **Provides** an OpenAI-specific implementation of the `LanguageModel` service using `"gpt-4o"` - **Requires** an `OpenAiClient` to be built ### Providing a `Model` Once you've created a `Model`, you can directly `Effect.provide` it to your Effect programs just like any other service: ### Benefits of `Model` There are several benefits to this approach: **Reusability** You can provide the same `Model` to as many programs as you like. **Example** **Flexibility** If we know that one model or provider performs better at a given task than another, we can freely mix and match models and providers together. For example, if we know Anthropic's Claude generates some really great dad jokes, we can mix it into our existing program with just a few lines of code: **Example** Because Effect performs type-level dependency tracking, we can see that an `AnthropicClient` is now required to make our program runnable. **Abstractability** An `Model` can also be `yield*`'ed to lift its dependencies into the calling Effect. This is particularly useful when creating services that depend on AI interactions, where you want to avoid leaking service-level dependencies into the service interface. For example, in the code below the `main` program is only dependent upon the `DadJokes` service. All AI requirements are abstracted away into `Layer` composition. **Example** ## Create a Provider Client To make our code executable, we must finish satisfying our program's requirements. Let's take another look at our program from earlier: We can see that our `main` program still requires us to provide an `OpenAiClient`. Each of our provider integration packages exports a client module that can be used to construct a client for that provider. **Example** In the code above, we use the `layerConfig` constructor from the `OpenAiClient` module to create a `Layer` which will produce an `OpenAiClient`. The `layerConfig` constructor allows us to read in configuration variables using Effect's . The provider clients also have a dependency on an `HttpClient` implementation to avoid any platform dependencies. This way, you can provide whichever `HttpClient` implementation is most appropriate for the platform your code is running upon. For example, if we know we are going to run this code in NodeJS, we can utilize the `NodeHttpClient` module from `@effect/platform-node` to provide an `HttpClient` implementation: ## Running the Program Now that we have a `Layer` which provides us with an `OpenAiClient`, we're ready to make our `main` program runnable. Our final program looks like the following: --- title: Introduction to Effect AI description: "Introduction to Effect's AI integrations, a set of packages for interacting with large language models" sidebar: label: Introduction order: 0 --- import { Aside } from "@astrojs/starlight/components" Welcome to the documentation for Effect's AI integration packages — a set of libraries designed to make working with large language models seamless, flexible, and provider-agnostic. These packages enable you to write programs that describe _what_ you want to do with an LLM — generating completions, handling chat interactions, running function calls — without having to commit to _how_ or _where_ those operations are executed. The core package, , provides a high-level, unified interface for modeling LLM interactions, independent of any specific provider. Once you're ready to run your program, you can plug in the services your program requires from our LLM provider integration packages. This separation of concerns allows you to: - Write clean, declarative business logic without worrying about provider-specific quirks - Easily swap between or combine providers at runtime or during testing - Take advantage of Effect’s features when building AI-driven workflows Whether you're building an intelligent agent, an interactive chat app, or a system that leverages LLMs for background tasks, Effect's AI packages offer the flexibility and control you need! Let’s dive in! ## Why Effect for AI? Integrating LLMs isn’t just about sending API requests — it’s handling streaming output, retries, rate limits, timeouts, and user-driven side effects, all while keeping your system stable and responsive. Effect provides simple, composable building blocks to model these workflows in a **safe**, **declarative**, and **composable** manner. By using Effect for your LLM interactions you'll benefit from: - 🧩 **Provider-Agnostic Architecture** Write your business logic once, and defer choosing the underlying provider until runtime - 🧪 **Fully Testable** Because LLM interactions are modeled via Effect services, you can mock, simulate, or snapshot responses just by providing an alternative implementation - 🧵 **Structured Concurrency** Run concurrent LLM calls, cancel stale requests, stream partial results, or race multiple providers — all safely managed by Effect’s structured concurrency model - 🔍 **Observability** Leverage Effect's built-in tracing, logging, and metrics to instrument your LLM interactions to gain deep insight into performance bottlenecks or failures in production ...and much more! ## Core Concepts Effect’s AI integrations are built around the idea of **provider-agnostic programming**. Instead of hardcoding calls to a specific LLM provider's API, you describe your interaction using the services provided by the base `@effect/ai` package. These services expose capabilities such as: - **Generating Text** – single-shot text generation - **Generating Embeddings** – vector representations of text for search or retrieval - **Tool Calling** – structured outputs and tool usage - **Streaming** – incremental output for memory efficiency and responsiveness Each of these services is defined as an _Effect service_ — meaning they can be injected, composed, and tested just like any other dependency in the Effect ecosystem. This decoupling lets you write your AI code as a pure description of what you want to happen, and resolve _how_ it happens later — whether by wiring up OpenAI, Anthropic, a mock service for tests, or even your own custom LLM backend. --- ## Packages Effect’s AI ecosystem is composed of several focused packages: ### `@effect/ai` Defines the core abstractions for interacting with LLM provider services. This package defines the generic services and helper utilities needed to build AI-powered applications in a provider-agnostic way. Use this package to: - Define your application's interaction with an LLM - Structure chat or completion flows using Effect - Build type-safe, declarative AI logic For detailed API documentation, see the . ### `@effect/ai-openai` Concrete implementations of services from `@effect/ai` backed by the . Supported services include: - `LanguageModel` ) - `EmbeddingsModel` ) For detailed API documentation, see the . ### `@effect/ai-anthropic` Concrete implementations of services from `@effect/ai` backed by the . Supported services include: - `LanguageModel` ) For detailed API documentation, see the . ### `@effect/ai-amazon-bedrock` Concrete implementations of services from `@effect/ai` backed by . Supported services include: - `LanguageModel` For detailed API documentation, see the . ### `@effect/ai-google` Concrete implementations of services from `@effect/ai` backed by . Supported services include: - `LanguageModel` For detailed API documentation, see the . --- title: Execution Planning description: Learn how to create structured execution plans for your LLM interactions sidebar: order: 2 --- Imagine that we've refactored our `generateDadJoke` program from our guide. Now, instead of handling all errors internally, the code can **fail with domain-specific issues** like network interruptions or provider outages: This is fine, but what if we want to: - Retry the program a fixed number of times on `NetworkError`s - Add some backoff delay between retries - Fallback to a different model provider if OpenAi is down How can we accomplish such logic? ## Planning LLM Interactions The `ExecutionPlan` module from Effect provides a robust method for creating **structured execution plans** for your Effect programs. Rather than making a single model call and hoping that it succeeds, you can use `ExecutionPlan` to describe how to handle errors, retries, and fallbacks in a clear, declarative way. This is especially useful when: - You want to fall back to a secondary model if the primary one is unavailable - You want to retry on transient errors - You want to control timing between retry attempts ## Creating Execution Plans To create an `ExecutionPlan`, we can use the `ExecutionPlan.make` constructor. **Example** This plan contains a single step which will: - Provide OpenAi's `"gpt-4o"` model as a `LanguageModel` for the program - Attempt to call OpenAi up to 3 times - Wait with an exponential backoff between attempts - Only re-attempt the call to OpenAi if the error is a `NetworkError` ## Adding Fallback Models To make your interactions with large language models resilient to provider outages, you can define a **fallback** models to use. This will allow the plan to automatically fallback to another model if the previous step in the execution plan fails. Use this when: - You want to make your model interactions resilient to provider outages - You want to potentially have multiple fallback models **Example** This plan contains two steps. **Step 1** The first step will: - Provide OpenAi's `"gpt-4o"` model as a `LanguageModel` for the program - Attempt to call OpenAi up to 3 times - Wait with an exponential backoff between attempts - Only attempt the call to OpenAi if the error is a `NetworkError` If all of the above logic fails to run the program successfully, the plan will try to run the program using the second step. **Step 2** The second step will: - Provide Anthropic's `"claude-4-sonnet-20250514"` model as a `LanguageModel` for the program - Attempt to call Anthropic up to 2 times - Wait with an exponential backoff between attempts - Only attempt the fallback if the error is a `ProviderOutage` ## End-to-End Usage The following is the complete program with the desired execution plan fully implemented: --- title: Tool Use description: Equip your LLM interactions with the ability to use tools to perform specific actions sidebar: order: 3 --- Language models are great at generating text, but often we need them to take **real-world actions**, such as querying an API, accessing a database, or calling a service. Most LLM providers support this through **tool use** , where you expose specific operations in your application that the model can invoke. Based on the input it receives, a model may choose to **invoke ** one or more tools to augment its response. Your application then runs the corresponding logic for the tool using the parameters provided by the model. You then return the result to the model, allowing it to include the output in its final response. The `Toolkit` simplifies tool integration by offering a structured, type-safe approach to defining tools. It takes care of all the wiring between the model and your application - all you have to do is define the tool and implement its behavior. ## Defining a Tool Let’s walk through a complete example of how to define, implement, and use a tool that fetches a dad joke from the API. ### 1. Define the Tool We start by defining a tool that the language model will have access to using the `Tool.make` constructor. This constructor accepts several parameters that allow us to fully describe the tool to the language model: - `description`: Provides an optional description of the tool - `success`: The type of value the tool will return if it succeeds - `failure`: The type of value the tool will return if it fails - `parameters`: The parameters that the tool should be called with **Example** Based on the above, a request to call the `GetDadJoke` tool: - Takes a single `searchTerm` parameter - Will return a string if it succeeds - Does not have any expected failure scenarios ### 2. Create a Toolkit Once we have a tool request defined, we can create a `Toolkit`, which is a collection of tools that the model will have access to. **Example** ### 3. Implement the Logic The `.toLayer` method on a `Toolkit` allows you to define the handlers for each tool in the toolkit. Because `.toLayer` takes an `Effect`, we can access services from our application to implement the tool call handlers. **Example** In the code above: - We access the `ICanHazDadJoke` service from our application - Register a handler for the `GetDadJoke` tool using `.handle` - Use the `.search` method on our `ICanHazDadJoke` service to search for a dad joke based on the tool call parameters The result of calling `.toLayer` on a `Toolkit` is a `Layer` that contains the handlers for all the tools in our toolkit. Because of this, it is quite simple to test a `Toolkit` by using `.toLayer` to create a separate `Layer` specifically for testing. ### 4. Give the Tools to the Model Once the tools are defined and implemented, you can pass them along to the model at request time. Behind the scenes, the model is given a structured description of each tool and can choose to call one or more of them when responding to input. **Example** ### 5. Bring It All Together To make the program executable, we must provide the implementation of our tool call handlers: **Example** ## Benefits **Type Safe** Every tool is fully described using Effect's `Schema`, including inputs, outputs, and descriptions. **Effect Native** Tool call behavior is defined using Effect, so they can leverage all the power of Effect. This is especially useful when you need to access other services to support the implementation of your tool call handlers. **Injectable** Because implementing the handlers for an `Toolkit` results in a `Layer`, providing alternate implementation of tool call handlers in different environments is as simple as providing a different `Layer` to your program. **Separation of Concerns** The definition of a tool call request is cleanly separated from both the implementation of the tool behavior, as well as the business logic that calls the model. --- title: Batching description: Optimize performance by batching requests and reducing redundant API calls, enhancing efficiency in data fetching and processing. sidebar: order: 9 --- import { Aside } from "@astrojs/starlight/components" In typical application development, when interacting with external APIs, databases, or other data sources, we often define functions that perform requests and handle their results or failures accordingly. ### Simple Model Setup Here's a basic model that outlines the structure of our data and possible errors: ### Defining API Functions Let's define functions that interact with an external API, handling common operations such as fetching todos, retrieving user details, and sending emails. While this approach is straightforward and readable, it may not be the most efficient. Repeated API calls, especially when many todos share the same owner, can significantly increase network overhead and slow down your application. ### Using the API Functions While these functions are clear and easy to understand, their use may not be the most efficient. For example, notifying todo owners involves repeated API calls which can be optimized. This implementation performs an API call for each todo to fetch the owner's details and send an email. If multiple todos have the same owner, this results in redundant API calls. ## Batching Let's assume that `getUserById` and `sendEmail` can be batched. This means that we can send multiple requests in a single HTTP call, reducing the number of API requests and improving performance. **Step-by-Step Guide to Batching** 1. **Declaring Requests:** We'll start by transforming our requests into structured data models. This involves detailing input parameters, expected outputs, and possible errors. Structuring requests this way not only helps in efficiently managing data but also in comparing different requests to understand if they refer to the same input parameters. 2. **Declaring Resolvers:** Resolvers are designed to handle multiple requests simultaneously. By leveraging the ability to compare requests , resolvers can execute several requests in one go, maximizing the utility of batching. 3. **Defining Queries:** Finally, we'll define queries that utilize these batch-resolvers to perform operations. This step ties together the structured requests and their corresponding resolvers into functional components of the application. ### Declaring Requests We'll design a model using the concept of a `Request` that a data source might support: A `Request` is a construct representing a request for a value of type `Value`, which might fail with an error of type `Error`. Let's start by defining a structured model for the types of requests our data sources can handle. Each request is defined with a specific data structure that extends from a generic `Request` type, ensuring that each request carries its unique data requirements along with a specific error type. By using tagged constructors like `Request.tagged`, we can easily instantiate request objects that are recognizable and manageable throughout the application. ### Declaring Resolvers After defining our requests, the next step is configuring how Effect resolves these requests using `RequestResolver`: A `RequestResolver` requires an environment `R` and is capable of executing requests of type `A`. In this section, we'll create individual resolvers for each type of request. The granularity of your resolvers can vary, but typically, they are divided based on the batching capabilities of the corresponding API calls. In this configuration: - **GetTodosResolver** handles the fetching of multiple `Todo` items. It's set up as a standard resolver since we assume it cannot be batched. - **GetUserByIdResolver** and **SendEmailResolver** are configured as batched resolvers. This setup is based on the assumption that these requests can be processed in batches, enhancing performance and reducing the number of API calls. ### Defining Queries Now that we've set up our resolvers, we're ready to tie all the pieces together to define our queries. This step will enable us to perform data operations effectively within our application. By using the `Effect.request` function, we integrate the resolvers with the request model effectively. This approach ensures that each query is optimally resolved using the appropriate resolver. Although the code structure looks similar to earlier examples, employing resolvers significantly enhances efficiency by optimizing how requests are handled and reducing unnecessary API calls. In the final setup, this program will execute only **3** queries to the APIs, regardless of the number of todos. This contrasts sharply with the traditional approach, which would potentially execute **1 + 2n** queries, where **n** is the number of todos. This represents a significant improvement in efficiency, especially for applications with a high volume of data interactions. ### Disabling Batching Batching can be locally disabled using the `Effect.withRequestBatching` utility in the following way: ### Resolvers with Context In complex applications, resolvers often need access to shared services or configurations to handle requests effectively. However, maintaining the ability to batch requests while providing the necessary context can be challenging. Here, we'll explore how to manage context in resolvers to ensure that batching capabilities are not compromised. When creating request resolvers, it's crucial to manage the context carefully. Providing too much context or providing varying services to resolvers can make them incompatible for batching. To prevent such issues, the context for the resolver used in `Effect.request` is explicitly set to `never`. This forces developers to clearly define how the context is accessed and used within resolvers. Consider the following example where we set up an HTTP service that the resolvers can use to execute API calls: We can see now that the type of `GetTodosResolver` is no longer a `RequestResolver` but instead it is: which is an effect that access the `HttpService` and returns a composed resolver that has the minimal context ready to use. Once we have such effect we can directly use it in our query definition: We can see that the Effect correctly requires `HttpService` to be provided. Alternatively you can create `RequestResolver`s as part of layers direcly accessing or closing over context from construction. **Example** This way is probably the best for most of the cases given that layers are the natural primitive where to wire services together. ## Caching While we have significantly optimized request batching, there's another area that can enhance our application's efficiency: caching. Without caching, even with optimized batch processing, the same requests could be executed multiple times, leading to unnecessary data fetching. In the Effect library, caching is handled through built-in utilities that allow requests to be stored temporarily, preventing the need to re-fetch data that hasn't changed. This feature is crucial for reducing the load on both the server and the network, especially in applications that make frequent similar requests. Here's how you can implement caching for the `getUserById` query: ## Final Program Assuming you've wired everything up correctly: With this program, the `getTodos` operation retrieves the todos for each user. Then, the `Effect.forEach` function is used to notify the owner of each todo concurrently, without waiting for the notifications to complete. The `repeat` function is applied to the entire chain of operations, and it ensures that the program repeats every 10 seconds using a fixed schedule. This means that the entire process, including fetching todos and sending notifications, will be executed repeatedly with a 10-second interval. The program incorporates a caching mechanism, which prevents the same `GetUserById` operation from being executed more than once within a span of 1 minute. This default caching behavior helps optimize the program's execution and reduces unnecessary requests to fetch user data. Furthermore, the program is designed to send emails in batches, allowing for efficient processing and better utilization of resources. ## Customizing Request Caching In real-world applications, effective caching strategies can significantly improve performance by reducing redundant data fetching. The Effect library provides flexible caching mechanisms that can be tailored for specific parts of your application or applied globally. There may be scenarios where different parts of your application have unique caching requirements—some might benefit from a localized cache, while others might need a global cache setup. Let’s explore how you can configure a custom cache to meet these varied needs. ### Creating a Custom Cache Here's how you can create a custom cache and apply it to part of your application. This example demonstrates setting up a cache that repeats a task every 10 seconds, caching requests with specific parameters like capacity and TTL . ### Direct Cache Application You can also construct a cache using `Request.makeCache` and apply it directly to a specific program using `Effect.withRequestCache`. This method ensures that all requests originating from the specified program are managed through the custom cache, provided that caching is enabled. --- title: Equivalence description: Define and customize equivalence relations for TypeScript values. sidebar: order: 0 --- The Equivalence module provides a way to define equivalence relations between values in TypeScript. An equivalence relation is a binary relation that is reflexive, symmetric, and transitive, establishing a formal notion of when two values should be considered equivalent. ## What is Equivalence? An `Equivalence` represents a function that compares two values of type `A` and determines if they are equivalent. This is more flexible and customizable than simple equality checks using `===`. Here's the structure of an `Equivalence`: ## Using Built-in Equivalences The module provides several built-in equivalence relations for common data types: | Equivalence | Description | | ----------- | ------------------------------------------- | | `string` | Uses strict equality for strings | | `number` | Uses strict equality for numbers | | `boolean` | Uses strict equality for booleans | | `symbol` | Uses strict equality for symbols | | `bigint` | Uses strict equality for bigints | | `Date` | Compares `Date` objects by their timestamps | **Example** ## Deriving Equivalences For more complex data structures, you may need custom equivalences. The Equivalence module lets you derive new `Equivalence` instances from existing ones with the `Equivalence.mapInput` function. **Example** The `Equivalence.mapInput` function takes two arguments: 1. The existing `Equivalence` you want to use as a base . 2. A function that extracts the value used for the equivalence check from your data structure => user.id` in this case). --- title: Order description: Compare, sort, and manage value ordering with customizable tools for TypeScript. sidebar: order: 2 --- The Order module provides a way to compare values and determine their order. It defines an interface `Order` which represents a single function for comparing two values of type `A`. The function returns `-1`, `0`, or `1`, indicating whether the first value is less than, equal to, or greater than the second value. Here's the basic structure of an `Order`: ## Using the Built-in Orders The Order module comes with several built-in comparators for common data types: | Order | Description | | -------- | ---------------------------------- | | `string` | Used for comparing strings. | | `number` | Used for comparing numbers. | | `bigint` | Used for comparing big integers. | | `Date` | Used for comparing `Date` objects. | **Example** ## Sorting Arrays You can sort arrays using these comparators. The `Array` module offers a `sort` function that sorts arrays without altering the original one. **Example** You can also use an `Order` as a comparator with JavaScript's native `Array.sort` method, but keep in mind that this will modify the original array. **Example** ## Deriving Orders For more complex data structures, you may need custom sorting rules. The Order module lets you derive new `Order` instances from existing ones with the `Order.mapInput` function. **Example** Imagine you have a list of `Person` objects, and you want to sort them by their names in ascending order. To achieve this, you can create a custom `Order`. The `Order.mapInput` function takes two arguments: 1. The existing `Order` you want to use as a base . 2. A function that extracts the value you want to use for sorting from your data structure => person.name` in this case). Once you have defined your custom `Order`, you can apply it to sort an array of `Person` objects: **Example** ## Combining Orders The Order module lets you combine multiple `Order` instances to create complex sorting rules. This is useful when sorting by multiple properties. **Example** Imagine you have a list of people, each represented by an object with a `name` and an `age`. You want to sort this list first by name and then, for individuals with the same name, by age. ## Additional Useful Functions The Order module provides additional functions for common comparison operations, making it easier to work with ordered values. ### Reversing Order `Order.reverse` inverts the order of comparison. If you have an `Order` for ascending values, reversing it makes it descending. **Example** ### Comparing Values These functions allow you to perform simple comparisons between values: | API | Description | | ---------------------- | -------------------------------------------------------- | | `lessThan` | Checks if one value is strictly less than another. | | `greaterThan` | Checks if one value is strictly greater than another. | | `lessThanOrEqualTo` | Checks if one value is less than or equal to another. | | `greaterThanOrEqualTo` | Checks if one value is greater than or equal to another. | **Example** ### Finding Minimum and Maximum The `Order.min` and `Order.max` functions return the minimum or maximum value between two values, considering the order. **Example** ### Clamping Values `Order.clamp` restricts a value within a given range. If the value is outside the range, it is adjusted to the nearest bound. **Example** ### Checking Value Range `Order.between` checks if a value falls within a specified inclusive range. **Example** --- title: Cache description: Optimize performance with cache for concurrent, compositional, and efficient value retrieval. sidebar: order: 1 --- In many applications, handling overlapping work is common. For example, in services that process incoming requests, it's important to avoid redundant work like handling the same request multiple times. The Cache module helps improve performance by preventing duplicate work. Key Features of Cache: | Feature | Description | | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | **Compositionality** | Allows overlapping work across different parts of the application while preserving compositional programming. | | **Unified Sync and Async Caches** | Integrates both synchronous and asynchronous caches through a unified lookup function that computes values either way. | | **Effect Integration** | Works natively with the Effect library, supporting concurrent lookups, failure handling, and interruption. | | **Cache Metrics** | Tracks key metrics like entries, hits, and misses, providing insights for performance optimization. | ## Creating a Cache A cache is defined by a lookup function that computes the value for a given key if it's not already cached: The lookup function takes a `Key` and returns an `Effect`, which describes how to compute the value . This `Effect` may require an environment , can fail with an `Error`, and succeed with a `Value`. Since it returns an `Effect`, it can handle both synchronous and asynchronous workflows. You create a cache by providing a lookup function along with a maximum size and a time-to-live for cached values. Once a cache is created, the most idiomatic way to work with it is the `get` method. The `get` method returns the current value in the cache if it exists, or computes a new value, puts it in the cache, and returns it. If multiple concurrent processes request the same value, it will only be computed once. All other processes will receive the computed value as soon as it is available. This is managed using Effect's fiber-based concurrency model without blocking the underlying thread. **Example** In this example, we call `timeConsumingEffect` three times concurrently with the same key. The cache runs this effect only once, so concurrent lookups will wait until the value is available: ## Concurrent Access The cache is designed to be safe for concurrent access and efficient under concurrent conditions. If two concurrent processes request the same value and it is not in the cache, the value will be computed once and provided to both processes as soon as it is available. Concurrent processes will wait for the value without blocking the underlying thread. If the lookup function fails or is interrupted, the error will be propagated to all concurrent processes waiting for the value. Failures are cached to prevent repeated computation of the same failed value. If interrupted, the key will be removed from the cache, so subsequent calls will attempt to compute the value again. ## Capacity A cache is created with a specified capacity. When the cache reaches capacity, the least recently accessed values will be removed first. The cache size may slightly exceed the specified capacity between operations. ## Time To Live A cache can also have a specified time to live . Values older than the TTL will not be returned. The age is calculated from when the value was loaded into the cache. ## Methods In addition to `get`, the cache provides several other methods: | Method | Description | | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `refresh` | Triggers a recomputation of the value for a key without removing the old value, allowing continued access. | | `size` | Returns the current size of the cache. The size is approximate under concurrent conditions. | | `contains` | Checks if a value associated with a specified key exists in the cache. Under concurrent access, the result is valid as of the check time but may change immediately after. | | `invalidate` | Evicts the value associated with a specific key. | | `invalidateAll` | Evicts all values from the cache. | --- title: Caching Effects description: Efficiently manage caching and memoization of effects with reusable tools. sidebar: order: 0 --- This section covers several functions from the library that help manage caching and memoization in your application. ## cachedFunction Memoizes a function with effects, caching results for the same inputs to avoid recomputation. **Example** ## once Ensures an effect is executed only once, even if invoked multiple times. **Example** ## cached Returns an effect that computes a result lazily and caches it. Subsequent evaluations of this effect will return the cached result without re-executing the logic. **Example** ## cachedWithTTL Returns an effect that caches its result for a specified duration, known as the `timeToLive`. When the cache expires after the duration, the effect will be recomputed upon next evaluation. **Example** ## cachedInvalidateWithTTL Similar to `Effect.cachedWithTTL`, this function caches an effect's result for a specified duration. It also includes an additional effect for manually invalidating the cached value before it naturally expires. **Example** --- title: Branded Types description: Use branded types to enforce type safety and refine data in TypeScript. sidebar: order: 3 --- In this guide, we will explore the concept of **branded types** in TypeScript and learn how to create and work with them using the Brand module. Branded types are TypeScript types with an added type tag that helps prevent accidental usage of a value in the wrong context. They allow us to create distinct types based on an existing underlying type, enabling type safety and better code organization. ## The Problem with TypeScript's Structural Typing TypeScript's type system is structurally typed, meaning that two types are considered compatible if their members are compatible. This can lead to situations where values of the same underlying type are used interchangeably, even when they represent different concepts or have different meanings. Consider the following types: Here, `UserId` and `ProductId` are structurally identical as they are both based on `number`. TypeScript will treat these as interchangeable, potentially causing bugs if they are mixed up in your application. **Example** In the example above, passing a `UserId` to `getProductById` does not produce a type error, even though it's logically incorrect. This happens because both types are considered interchangeable. ## How Branded Types Help Branded types allow you to create distinct types from the same underlying type by adding a unique type tag, enforcing proper usage at compile-time. Branding is accomplished by adding a symbolic identifier that distinguishes one type from another at the type level. This method ensures that types remain distinct without altering their runtime characteristics. Let's start by introducing the `BrandTypeId` symbol: This approach assigns a unique identifier as a brand to the `number` type, effectively differentiating `ProductId` from other numerical types. The use of a symbol ensures that the branding field does not conflict with any existing properties of the `number` type. Attempting to use a `UserId` in place of a `ProductId` now results in an error: **Example** The error message clearly states that a `number` cannot be used in place of a `ProductId`. TypeScript won't let us pass an instance of `number` to the function accepting `ProductId` because it's missing the brand field. Let's add branding to `UserId` as well: **Example** The error indicates that while both types use branding, the unique values associated with the branding fields ensure they remain distinct and non-interchangeable. ## Generalizing Branded Types To enhance the versatility and reusability of branded types, they can be generalized using a standardized approach: This design allows any type to be branded using a unique identifier, either a string or symbol. Here's how you can utilize the `Brand` interface, which is readily available from the Brand module, eliminating the need to craft your own implementation: **Example** However, creating instances of these types directly leads to an error because the type system expects the brand structure: **Example** You cannot directly assign a `number` to `ProductId`. The Brand module provides utilities to correctly construct values of branded types. ## Constructing Branded Types The Brand module provides two main functions for creating branded types: `nominal` and `refined`. ### nominal The `Brand.nominal` function is designed for defining branded types that do not require runtime validations. It simply adds a type tag to the underlying type, allowing us to distinguish between values of the same type but with different meanings. Nominal branded types are useful when we only want to create distinct types for clarity and code organization purposes. **Example** Attempting to assign a non-`ProductId` value will result in a compile-time error: **Example** ### refined The `Brand.refined` function enables the creation of branded types that include data validation. It requires a refinement predicate to check the validity of input data against specific criteria. When the input data does not meet the criteria, the function uses `Brand.error` to generate a `BrandErrors` data type. This provides detailed information about why the validation failed. **Example** **Example** Attempting to assign a non-`Int` value will result in a compile-time error: **Example** ## Combining Branded Types In some cases, you might need to combine multiple branded types. The Brand module provides the `Brand.all` API for this purpose: **Example** --- title: Control Flow Operators description: Learn to control execution flow in Effect programs using advanced constructs for conditional branching, iteration, and combining effects seamlessly. sidebar: order: 6 --- Even though JavaScript provides built-in control flow structures, Effect offers additional control flow functions that are useful in Effect applications. In this section, we will introduce different ways to control the flow of execution. ## if Expression When working with Effect values, we can use standard JavaScript if-then-else statements: **Example** Here we are using the data type to represent the absence of a valid value. **Example** You can also handle invalid inputs by using the error channel, which allows you to return an error when the input is invalid: ## Conditional Operators ### if Executes one of two effects based on a condition evaluated by an effectful predicate. Use `Effect.if` to run one of two effects depending on whether the predicate effect evaluates to `true` or `false`. If the predicate is `true`, the `onTrue` effect is executed. If it is `false`, the `onFalse` effect is executed instead. **Example** In this example, we simulate a virtual coin flip using `Random.nextBoolean` to generate a random boolean value. If the value is `true`, the `onTrue` effect logs "Head". If the value is `false`, the `onFalse` effect logs "Tail". ### when Conditionally executes an effect based on a boolean condition. `Effect.when` allows you to conditionally execute an effect, similar to using an `if ` expression, but with the added benefit of handling effects. If the condition is `true`, the effect is executed; otherwise, it does nothing. The result of the effect is wrapped in an `Option` to indicate whether the effect was executed. If the condition is `true`, the result of the effect is wrapped in a `Some`. If the condition is `false`, the result is `None`, representing that the effect was skipped. **Example** In this example, the data type is used to represent the presence or absence of a valid value. If the condition evaluates to `true` , the effect is executed and wrapped in a `Some`. Otherwise, the result is `None`. ### whenEffect Executes an effect conditionally, based on the result of another effect. Use `Effect.whenEffect` when the condition to determine whether to execute the effect depends on the outcome of another effect that produces a boolean value. If the condition effect evaluates to `true`, the specified effect is executed. If it evaluates to `false`, no effect is executed. The result of the effect is wrapped in an `Option` to indicate whether the effect was executed. If the condition is `true`, the result of the effect is wrapped in a `Some`. If the condition is `false`, the result is `None`, representing that the effect was skipped. **Example** The following function creates a random integer, but only if a randomly generated boolean is `true`. ### unless / unlessEffect The `Effect.unless` and `Effect.unlessEffect` functions are similar to the `when*` functions, but they are equivalent to the `if expression` construct. ## Zipping ### zip Combines two effects into a single effect, producing a tuple with the results of both effects. The `Effect.zip` function executes the first effect and then the second effect . Once both effects succeed, their results are combined into a tuple. **Example** By default, the effects are run sequentially. To run them concurrently, use the `{ concurrent: true }` option. **Example** In this concurrent version, both effects run in parallel. `task2` completes first, but both tasks can be logged and processed as soon as they're done. ### zipWith Combines two effects sequentially and applies a function to their results to produce a single value. The `Effect.zipWith` function is similar to , but instead of returning a tuple of results, it applies a provided function to the results of the two effects, combining them into a single value. By default, the effects are run sequentially. To run them concurrently, use the `{ concurrent: true }` option. **Example** ## Looping ### loop The `Effect.loop` function allows you to repeatedly update a state using a `step` function until a condition defined by the `while` function becomes `false`. It collects the intermediate states in an array and returns them as the final result. **Syntax** This function is similar to a `while` loop in JavaScript, with the addition of effectful computations: **Example** In this example, the loop starts with the state `1` and continues until the state exceeds `5`. Each state is incremented by `1` and is collected into an array, which becomes the final result. #### Discarding Intermediate Results The `discard` option, when set to `true`, will discard the results of each effectful operation, returning `void` instead of an array. **Example** In this example, the loop performs a side effect of logging the current index on each iteration, but it discards all intermediate results. The final result is `undefined`. ### iterate The `Effect.iterate` function lets you repeatedly update a state through an effectful operation. It runs the `body` effect to update the state in each iteration and continues as long as the `while` condition evaluates to `true`. **Syntax** This function is similar to a `while` loop in JavaScript, with the addition of effectful computations: **Example** ### forEach Executes an effectful operation for each element in an `Iterable`. The `Effect.forEach` function applies a provided operation to each element in the iterable, producing a new effect that returns an array of results. If any effect fails, the iteration stops immediately , and the error is propagated. The `concurrency` option controls how many operations are performed concurrently. By default, the operations are performed sequentially. **Example** In this example, we iterate over the array ``, applying an effect that logs the current index. The `Effect.as` operation transforms each value, resulting in an array ``. The final output is the result of collecting all the transformed values. #### Discarding Results The `discard` option, when set to `true`, will discard the results of each effectful operation, returning `void` instead of an array. **Example** In this case, the effects still run for each element, but the results are discarded, so the final output is `undefined`. ## Collecting ### all Combines multiple effects into one, returning results based on the input structure. Use `Effect.all` when you need to run multiple effects and combine their results into a single output. It supports tuples, iterables, structs, and records, making it flexible for different input types. If any effect fails, it stops execution and propagates the error. To change this behavior, you can use the option, which allows all effects to run and collect results as or . You can control the execution order using the . For instance, if the input is a tuple: the effects are executed sequentially, and the result is a new effect containing the results as a tuple. The results in the tuple match the order of the effects passed to `Effect.all`. Let's explore examples for different types of structures: tuples, iterables, objects, and records. **Example** **Example** **Example** **Example** #### Short-Circuiting Behavior The `Effect.all` function stops execution on the first error it encounters, this is called "short-circuiting". If any effect in the collection fails, the remaining effects will not run, and the error will be propagated. **Example** You can override this behavior by using the `mode` option. #### The `mode` option The `{ mode: "either" }` option changes the behavior of `Effect.all` to ensure all effects run, even if some fail. Instead of stopping on the first failure, this mode collects both successes and failures, returning an array of `Either` instances where each result is either a `Right` or a `Left` . **Example** Similarly, the `{ mode: "validate" }` option uses `Option` to indicate success or failure. Each effect returns `None` for success and `Some` with the error for failure. **Example** --- title: Simplifying Excessive Nesting description: Simplify nested code with Do simulation and generators. sidebar: label: Excessive Nesting order: 5 --- import { Steps } from "@astrojs/starlight/components" Suppose you want to create a custom function `elapsed` that prints the elapsed time taken by an effect to execute. ## Using plain pipe Initially, you may come up with code that uses the standard `pipe` , but this approach can lead to excessive nesting and result in verbose and hard-to-read code: **Example** To address this issue and make the code more manageable, there is a solution: the "do simulation." ## Using the "do simulation" The "do simulation" in Effect allows you to write code in a more declarative style, similar to the "do notation" in other programming languages. It provides a way to define variables and perform operations on them using functions like `Effect.bind` and `Effect.let`. Here's how the do simulation works: 1. Start the do simulation using the `Effect.Do` value: 2. Within the do simulation scope, you can use the `Effect.bind` function to define variables and bind them to `Effect` values: - `variableName` is the name you choose for the variable you want to define. It must be unique within the scope. - `effectValue` is the `Effect` value that you want to bind to the variable. It can be the result of a function call or any other valid `Effect` value. 3. You can accumulate multiple `Effect.bind` statements to define multiple variables within the scope: 4. Inside the do simulation scope, you can also use the `Effect.let` function to define variables and bind them to simple values: - `variableName` is the name you give to the variable. Like before, it must be unique within the scope. - `simpleValue` is the value you want to assign to the variable. It can be a simple value like a `number`, `string`, or `boolean`. 5. Regular Effect functions like `Effect.andThen`, `Effect.flatMap`, `Effect.tap`, and `Effect.map` can still be used within the do simulation. These functions will receive the accumulated variables as arguments within the scope: With the do simulation, you can rewrite the `elapsed` function like this: **Example** ## Using Effect.gen The most concise and convenient solution is to use , which allows you to work with when dealing with effects. This approach leverages the native scope provided by the generator syntax, avoiding excessive nesting and leading to more concise code. **Example** Within the generator, we use `yield*` to invoke effects and bind their results to variables. This eliminates the nesting and provides a more readable and sequential code structure. The generator style in Effect uses a more linear and sequential flow of execution, resembling traditional imperative programming languages. This makes the code easier to read and understand, especially for developers who are more familiar with imperative programming paradigms. --- title: Dual APIs description: Explore data-first and data-last variants of dual APIs in the Effect ecosystem. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" When you're working with APIs in the Effect ecosystem, you may come across two different ways to use the same API. These two ways are called the "data-last" and "data-first" variants. When an API supports both variants, we call them "dual" APIs. Here's an illustration of these two variants using `Effect.map`. ## Effect.map as a dual API The `Effect.map` function is defined with two TypeScript overloads. The terms "data-last" and "data-first" refer to the position of the `self` argument in the signatures of the two overloads: ### data-last In the first overload, the `self` argument comes **last**: This version is commonly used with the `pipe` function. You start by passing the `Effect` as the initial argument to `pipe` and then chain transformations like `Effect.map`: **Example** This style is helpful when chaining multiple transformations, making the code easier to follow in a pipeline format: ### data-first In the second overload, the `self` argument comes **first**: This form doesn't require `pipe`. Instead, you provide the `Effect` directly as the first argument: **Example** This version works well when you only need to perform a single operation on the `Effect`. --- title: Guidelines description: Best practices for running Effect applications and ensuring safe, explicit coding styles. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" ## Using runMain In Effect, `runMain` is the primary entry point for executing an Effect application on Node.js. **Example** The `runMain` function handles finding and interrupting all fibers. Internally, it observes the fiber and listens for `sigint` signals, ensuring a graceful shutdown of the application when interrupted . ### Versions for Different Platforms Effect provides versions of `runMain` tailored for different platforms: | Platform | Runtime Version | Import Path | | -------- | ------------------------ | -------------------------- | | Node.js | `NodeRuntime.runMain` | `@effect/platform-node` | | Bun | `BunRuntime.runMain` | `@effect/platform-bun` | | Browser | `BrowserRuntime.runMain` | `@effect/platform-browser` | ## Avoid Tacit Usage Avoid using tacit function calls, such as `Effect.map`, or using `flow` from the `effect/Function` module. In Effect, it's generally safer to write functions explicitly: rather than in a point-free style: While tacit functions may be appealing for their brevity, they can introduce a number of problems: - Using tacit functions, particularly when dealing with optional parameters, can be unsafe. For example, if a function has overloads, writing it in a tacit style may erase all generics, resulting in bugs. Check out this X thread for more details: . - Tacit usage can also compromise TypeScript's ability to infer types, potentially causing unexpected errors. This isn't just a matter of style but a way to avoid subtle mistakes that can arise from type inference issues. - Additionally, stack traces might not be as clear when tacit usage is employed. Avoiding tacit usage is a simple precaution that makes your code more reliable. --- title: Pattern Matching description: Simplify complex branching with pattern matching using the Match module. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" Pattern matching is a method that allows developers to handle intricate conditions within a single, concise expression. It simplifies code, making it more concise and easier to understand. Additionally, it includes a process called exhaustiveness checking, which helps to ensure that no possible case has been overlooked. Originating from functional programming languages, pattern matching stands as a powerful technique for code branching. It often offers a more potent and less verbose solution compared to imperative alternatives such as if/else or switch statements, particularly when dealing with complex conditions. Although not yet a native feature in JavaScript, there's an ongoing in its early stages to introduce pattern matching to JavaScript. However, this proposal is at stage 1 and might take several years to be implemented. Nonetheless, developers can implement pattern matching in their codebase. The `effect/Match` module provides a reliable, type-safe pattern matching implementation that is available for immediate use. **Example** ## How Pattern Matching Works Pattern matching follows a structured process: 1. **Creating a matcher**. Define a `Matcher` that operates on either a specific or . 2. **Defining patterns**. Use combinators such as `Match.when`, `Match.not`, and `Match.tag` to specify matching conditions. 3. **Completing the match**. Apply a finalizer such as `Match.exhaustive`, `Match.orElse`, or `Match.option` to determine how unmatched cases should be handled. ## Creating a matcher You can create a `Matcher` using either: - `Match.type`: Matches against a specific type. - `Match.value`: Matches against a specific value. ### Matching by Type The `Match.type` constructor defines a `Matcher` that operates on a specific type. Once created, you can use patterns like `Match.when` to define conditions for handling different cases. **Example** ### Matching by Value Instead of creating a matcher for a type, you can define one directly from a specific value using `Match.value`. **Example** ### Enforcing a Return Type You can use `Match.withReturnType` to ensure that all branches return a specific type. **Example** This example enforces that every matching branch returns a `string`. ## Defining patterns ### when The `Match.when` function allows you to define conditions for matching values. It supports both direct value comparisons and predicate functions. **Example** ### not The `Match.not` function allows you to exclude specific values while matching all others. **Example** ### tag The `Match.tag` function allows pattern matching based on the `_tag` field in a . You can specify multiple tags to match within a single pattern. **Example** ### Built-in Predicates The `Match` module provides built-in predicates for common types, such as `Match.number`, `Match.string`, and `Match.boolean`. These predicates simplify the process of matching against primitive types. **Example** | Predicate | Description | | ------------------------- | ----------------------------------------------------------------------------- | | `Match.string` | Matches values of type `string`. | | `Match.nonEmptyString` | Matches non-empty strings. | | `Match.number` | Matches values of type `number`. | | `Match.boolean` | Matches values of type `boolean`. | | `Match.bigint` | Matches values of type `bigint`. | | `Match.symbol` | Matches values of type `symbol`. | | `Match.date` | Matches values that are instances of `Date`. | | `Match.record` | Matches objects where keys are `string` or `symbol` and values are `unknown`. | | `Match.null` | Matches the value `null`. | | `Match.undefined` | Matches the value `undefined`. | | `Match.defined` | Matches any defined value. | | `Match.any` | Matches any value without restrictions. | | `Match.is` | Matches a specific set of literal values `). | | `Match.instanceOf` | Matches instances of a given class. | ## Completing the match ### exhaustive The `Match.exhaustive` method finalizes the pattern matching process by ensuring that all possible cases are accounted for. If any case is missing, TypeScript will produce a type error. This is particularly useful when working with unions, as it helps prevent unintended gaps in pattern matching. **Example** ### orElse The `Match.orElse` method defines a fallback value to return when no other patterns match. This ensures that the matcher always produces a valid result. **Example** ### option `Match.option` wraps the match result in an . If a match is found, it returns `Some`, otherwise, it returns `None`. **Example** ### either The `Match.either` method wraps the result in an , providing a structured way to distinguish between matched and unmatched cases. If a match is found, it returns `Right`, otherwise, it returns `Left`. **Example** --- title: Basic Concurrency description: Manage and control effect execution with concurrency, interruptions, and racing. sidebar: order: 0 --- import { Aside } from "@astrojs/starlight/components" ## Concurrency Options Effect provides options to manage how effects are executed, particularly focusing on controlling how many effects run concurrently. The `concurrency` option is used to determine the level of concurrency, with the following values: Let's explore each configuration in detail. ### Sequential Execution By default, if you don't specify any concurrency option, effects will run sequentially, one after the other. This means each effect starts only after the previous one completes. **Example** ### Numbered Concurrency You can control how many effects run concurrently by setting a `number` for `concurrency`. For example, `concurrency: 2` allows up to two effects to run at the same time. **Example** ### Unbounded Concurrency When `concurrency: "unbounded"` is used, there's no limit to the number of effects running concurrently. **Example** ### Inherit Concurrency When using `concurrency: "inherit"`, the concurrency level is inherited from the surrounding context. This context can be set using `Effect.withConcurrency`. If no context is provided, the default is `"unbounded"`. **Example** If you use `Effect.withConcurrency`, the concurrency configuration will adjust to the specified option. **Example** ## Interruptions All effects in Effect are executed by . If you didn't create the fiber yourself, it was created by an operation you're using or by the Effect system. A fiber is created any time an effect is run. When running effects concurrently, a fiber is created for each concurrent effect. To summarize: - An `Effect` is a higher-level concept that describes an effectful computation. It is lazy and immutable, meaning it represents a computation that may produce a value or fail but does not immediately execute. - A fiber, on the other hand, represents the running execution of an `Effect`. It can be interrupted or awaited to retrieve its result. Think of it as a way to control and interact with the ongoing computation. Fibers can be interrupted in various ways. Let's explore some of these scenarios and see examples of how to interrupt fibers in Effect. ### interrupt A fiber can be interrupted using the `Effect.interrupt` effect on that particular fiber. This effect models the explicit interruption of the fiber in which it runs. When executed, it causes the fiber to stop its operation immediately, capturing the interruption details such as the fiber's ID and its start time. The resulting interruption can be observed in the type if the effect is run with functions like . **Example** In this case, the program runs without any interruption, logging the start and completion of the task. **Example** Here, the fiber is interrupted after the log `"start"` but before the `"done"` log. The `Effect.interrupt` stops the fiber, and it never reaches the final log. ### onInterrupt Registers a cleanup effect to run when an effect is interrupted. This function allows you to specify an effect to run when the fiber is interrupted. This effect will be executed when the fiber is interrupted, allowing you to perform cleanup or other actions. **Example** In this example, we set up a handler that logs "Cleanup completed" whenever the fiber is interrupted. We then show three cases: a successful effect, a failing effect, and an interrupted effect, demonstrating how the handler is triggered depending on how the effect ends. ### Interruption of Concurrent Effects When running multiple effects concurrently, such as with `Effect.forEach`, if one of the effects is interrupted, it causes all concurrent effects to be interrupted as well. The resulting includes information about which fibers were interrupted. **Example** ## Racing ### race This function takes two effects and runs them concurrently. The first effect that successfully completes will determine the result of the race, and the other effect will be interrupted. If neither effect succeeds, the function will fail with a containing all the errors. This is useful when you want to run two effects concurrently, but only care about the first one to succeed. It is commonly used in cases like timeouts, retries, or when you want to optimize for the faster response without worrying about the other effect. **Example** **Example** **Example** If you want to handle the result of whichever task completes first, whether it succeeds or fails, you can use the `Effect.either` function. This function wraps the result in an type, allowing you to see if the result was a success or a failure : **Example** ### raceAll This function runs multiple effects concurrently and returns the result of the first one to succeed. If one effect succeeds, the others will be interrupted. If none of the effects succeed, the function will fail with the last error encountered. This is useful when you want to race multiple effects, but only care about the first one to succeed. It is commonly used in cases like timeouts, retries, or when you want to optimize for the faster response without worrying about the other effects. **Example** **Example** **Example** ### raceFirst This function takes two effects and runs them concurrently, returning the result of the first one that completes, regardless of whether it succeeds or fails. This function is useful when you want to race two operations, and you want to proceed with whichever one finishes first, regardless of whether it succeeds or fails. **Example** **Example** #### Disconnecting Effects The `Effect.raceFirst` function safely interrupts the "loser" effect once the other completes, but it will not resume until the loser is cleanly terminated. If you want a quicker return, you can disconnect the interrupt signal for both effects. Instead of calling: You can use: This allows both effects to complete independently while still terminating the losing effect in the background. **Example** ### raceWith This function runs two effects concurrently and calls a specified "finisher" function once one of the effects completes, regardless of whether it succeeds or fails. The finisher functions for each effect allow you to handle the results of each effect as soon as they complete. The function takes two finisher callbacks, one for each effect, and allows you to specify how to handle the result of the race. This function is useful when you need to react to the completion of either effect without waiting for both to finish. It can be used whenever you want to take action based on the first available result. **Example** --- title: Deferred description: Master asynchronous coordination with Deferred, a one-time variable for managing effect synchronization and communication. sidebar: order: 4 --- A `Deferred` is a specialized subtype of `Effect` that acts like a one-time variable with some unique characteristics. It can only be completed once, making it a useful tool for managing asynchronous operations and synchronization between different parts of your program. A deferred is essentially a synchronization primitive that represents a value that may not be available right away. When you create a deferred, it starts out empty. Later, it can be completed with either a success value `Success` or an error value `Error`: Once completed, it cannot be changed again. When a fiber calls `Deferred.await`, it will pause until the deferred is completed. While the fiber is waiting, it doesn't block the thread, it only blocks semantically. This means other fibers can still run, ensuring efficient concurrency. A deferred is conceptually similar to JavaScript's `Promise`. The key difference is that it supports both success and error types, giving more type safety. ## Creating a Deferred A deferred can be created using the `Deferred.make` constructor. This returns an effect that represents the creation of the deferred. Since the creation of a deferred involves memory allocation, it must be done within an effect to ensure safe management of resources. **Example** ## Awaiting To retrieve a value from a deferred, you can use `Deferred.await`. This operation suspends the calling fiber until the deferred is completed with a value or an error. ## Completing You can complete a deferred in several ways, depending on whether you want to succeed, fail, or interrupt the waiting fibers: | API | Description | | ----------------------- | --------------------------------------------------------------------------------------------------------------- | | `Deferred.succeed` | Completes the deferred successfully with a value. | | `Deferred.done` | Completes the deferred with an value. | | `Deferred.complete` | Completes the deferred with the result of an effect. | | `Deferred.completeWith` | Completes the deferred with an effect. This effect will be executed by each waiting fiber, so use it carefully. | | `Deferred.fail` | Fails the deferred with an error. | | `Deferred.die` | Defects the deferred with a user-defined error. | | `Deferred.failCause` | Fails or defects the deferred with a . | | `Deferred.interrupt` | Interrupts the deferred, forcefully stopping or interrupting the waiting fibers. | **Example** Completing a deferred produces an `Effect`. This effect returns `true` if the deferred was successfully completed, and `false` if it had already been completed previously. This can be useful for tracking the state of the deferred. **Example** ## Checking Completion Status Sometimes, you might need to check if a deferred has been completed without suspending the fiber. This can be done using the `Deferred.poll` method. Here's how it works: - `Deferred.poll` returns an `Option>`: - If the `Deferred` is incomplete, it returns `None`. - If the `Deferred` is complete, it returns `Some`, which contains the result or error. Additionally, you can use the `Deferred.isDone` function to check if a deferred has been completed. This method returns an `Effect`, which evaluates to `true` if the `Deferred` is completed, allowing you to quickly check its state. **Example** ## Common Use Cases `Deferred` becomes useful when you need to wait for something specific to happen in your program. It's ideal for scenarios where you want one part of your code to signal another part when it's ready. Here are a few common use cases: | **Use Case** | **Description** | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Coordinating Fibers** | When you have multiple concurrent tasks and need to coordinate their actions, `Deferred` can help one fiber signal to another when it has completed its task. | | **Synchronization** | Anytime you want to ensure that one piece of code doesn't proceed until another piece of code has finished its work, `Deferred` can provide the synchronization you need. | | **Handing Over Work** | You can use `Deferred` to hand over work from one fiber to another. For example, one fiber can prepare some data, and then a second fiber can continue processing it. | | **Suspending Execution** | When you want a fiber to pause its execution until some condition is met, a `Deferred` can be used to block it until the condition is satisfied. | **Example** In this example, a deferred is used to pass a value between two fibers. By running both fibers concurrently and using the deferred as a synchronization point, we can ensure that `fiberB` only proceeds after `fiberA` has completed its task. --- title: Fibers description: Understand fibers in Effect, lightweight virtual threads enabling powerful concurrency, structured lifecycles, and efficient resource management for responsive applications. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" Effect is a highly concurrent framework powered by fibers. Fibers are lightweight virtual threads with resource-safe cancellation capabilities, enabling many features in Effect. In this section, you will learn the basics of fibers and get familiar with some of the powerful low-level operators that utilize fibers. ## What Are Virtual Threads? JavaScript is inherently single-threaded, meaning it executes code in a single sequence of instructions. However, modern JavaScript environments use an event loop to manage asynchronous operations, creating the illusion of multitasking. In this context, virtual threads, or fibers, are logical threads simulated by the Effect runtime. They allow concurrent execution without relying on true multi-threading, which is not natively supported in JavaScript. ## How Fibers work All effects in Effect are executed by fibers. If you didn't create the fiber yourself, it was created by an operation you're using or by the Effect runtime system. A fiber is created any time an effect is run. When running effects concurrently, a fiber is created for each concurrent effect. Even if you write "single-threaded" code with no concurrent operations, there will always be at least one fiber: the "main" fiber that executes your effect. Effect fibers have a well-defined lifecycle based on the effect they are executing. Every fiber exits with either a failure or success, depending on whether the effect it is executing fails or succeeds. Effect fibers have unique identities, local state, and a status . To summarize: - An `Effect` is a higher-level concept that describes an effectful computation. It is lazy and immutable, meaning it represents a computation that may produce a value or fail but does not immediately execute. - A fiber, on the other hand, represents the running execution of an `Effect`. It can be interrupted or awaited to retrieve its result. Think of it as a way to control and interact with the ongoing computation. ## The Fiber Data Type The `Fiber` data type in Effect represents a "handle" on the execution of an effect. Here is the general form of a `Fiber`: This type indicates that a fiber: - Succeeds and returns a value of type `Success` - Fails with an error of type `Error` Fibers do not have an `Requirements` type parameter because they only execute effects that have already had their requirements provided to them. ## Forking Effects You can create a new fiber by **forking** an effect. This starts the effect in a new fiber, and you receive a reference to that fiber. **Example** In this example, the Fibonacci calculation is forked into its own fiber, allowing it to run independently of the main fiber. The reference to the `fib10Fiber` can be used later to join or interrupt the fiber. ## Joining Fibers One common operation with fibers is **joining** them. By using the `Fiber.join` function, you can wait for a fiber to complete and retrieve its result. The joined fiber will either succeed or fail, and the `Effect` returned by `join` reflects the outcome of the fiber. **Example** ## Awaiting Fibers The `Fiber.await` function is a helpful tool when working with fibers. It allows you to wait for a fiber to complete and retrieve detailed information about how it finished. The result is encapsulated in an value, which gives you insight into whether the fiber succeeded, failed, or was interrupted. **Example** ## Interruption Model While developing concurrent applications, there are several cases that we need to interrupt the execution of other fibers, for example: 1. A parent fiber might start some child fibers to perform a task, and later the parent might decide that, it doesn't need the result of some or all of the child fibers. 2. Two or more fibers start race with each other. The fiber whose result is computed first wins, and all other fibers are no longer needed, and should be interrupted. 3. In interactive applications, a user may want to stop some already running tasks, such as clicking on the "stop" button to prevent downloading more files. 4. Computations that run longer than expected should be aborted by using timeout operations. 5. When we have an application that perform compute-intensive tasks based on the user inputs, if the user changes the input we should cancel the current task and perform another one. ### Polling vs. Asynchronous Interruption When it comes to interrupting fibers, a naive approach is to allow one fiber to forcefully terminate another fiber. However, this approach is not ideal because it can leave shared state in an inconsistent and unreliable state if the target fiber is in the middle of modifying that state. Therefore, it does not guarantee internal consistency of the shared mutable state. Instead, there are two popular and valid solutions to tackle this problem: 1. **Semi-asynchronous Interruption **: Imperative languages often employ polling as a semi-asynchronous signaling mechanism, such as Java. In this model, a fiber sends an interruption request to another fiber. The target fiber continuously polls the interrupt status and checks whether it has received any interruption requests from other fibers. If an interruption request is detected, the target fiber terminates itself as soon as possible. With this solution, the fiber itself handles critical sections. So, if a fiber is in the middle of a critical section and receives an interruption request, it ignores the interruption and defers its handling until after the critical section. However, one drawback of this approach is that if the programmer forgets to poll regularly, the target fiber can become unresponsive, leading to deadlocks. Additionally, polling a global flag is not aligned with the functional paradigm followed by Effect. 2. **Asynchronous Interruption**: In asynchronous interruption, a fiber is allowed to terminate another fiber. The target fiber is not responsible for polling the interrupt status. Instead, during critical sections, the target fiber disables the interruptibility of those regions. This is a purely functional solution that doesn't require polling a global state. Effect adopts this solution for its interruption model, which is a fully asynchronous signaling mechanism. This mechanism overcomes the drawback of forgetting to poll regularly. It is also fully compatible with the functional paradigm because in a purely functional computation, we can abort the computation at any point, except during critical sections where interruption is disabled. ### Interrupting Fibers Fibers can be interrupted if their result is no longer needed. This action immediately stops the fiber and safely runs all finalizers to release any resources. Like `Fiber.await`, the `Fiber.interrupt` function returns an value that provides detailed information about how the fiber ended. **Example** By default, the effect returned by `Fiber.interrupt` waits until the fiber has fully terminated before resuming. This ensures that no new fibers are started before the previous ones have finished, a behavior known as "back-pressuring." If you do not require this waiting behavior, you can fork the interruption itself, allowing the main program to proceed without waiting for the fiber to terminate: **Example** There is also a shorthand for background interruption called `Fiber.interruptFork`. ## Composing Fibers The `Fiber.zip` and `Fiber.zipWith` functions allow you to combine two fibers into one. The resulting fiber will produce the results of both input fibers. If either fiber fails, the combined fiber will also fail. **Example** In this example, both fibers run concurrently, and the results are combined into a tuple. Another way to compose fibers is by using `Fiber.orElse`. This function allows you to provide an alternative fiber that will execute if the first one fails. If the first fiber succeeds, its result will be returned. If it fails, the second fiber will run instead, and its result will be returned regardless of its outcome. **Example** ## Lifetime of Child Fibers When we fork fibers, depending on how we fork them we can have four different lifetime strategies for the child fibers: 1. **Fork With Automatic Supervision**. If we use the ordinary `Effect.fork` operation, the child fiber will be automatically supervised by the parent fiber. The lifetime child fibers are tied to the lifetime of their parent fiber. This means that these fibers will be terminated either when they end naturally, or when their parent fiber is terminated. 2. **Fork in Global Scope **. Sometimes we want to run long-running background fibers that aren't tied to their parent fiber, and also we want to fork them in a global scope. Any fiber that is forked in global scope will become daemon fiber. This can be achieved by using the `Effect.forkDaemon` operator. As these fibers have no parent, they are not supervised, and they will be terminated when they end naturally, or when our application is terminated. 3. **Fork in Local Scope**. Sometimes, we want to run a background fiber that isn't tied to its parent fiber, but we want that fiber to live in the local scope. We can fork fibers in the local scope by using `Effect.forkScoped`. Such fibers can outlive their parent fiber , and they will be terminated when their life end or their local scope is closed. 4. **Fork in Specific Scope**. This is similar to the previous strategy, but we can have more fine-grained control over the lifetime of the child fiber by forking it in a specific scope. We can do this by using the `Effect.forkIn` operator. ### Fork with Automatic Supervision Effect follows a **structured concurrency** model, where child fibers' lifetimes are tied to their parent. Simply put, the lifespan of a fiber depends on the lifespan of its parent fiber. **Example** In this scenario, the `parent` fiber spawns a `child` fiber that repeatedly prints a message every second. The `child` fiber will be terminated when the `parent` fiber completes. This behavior can be extended to any level of nested fibers, ensuring a predictable and controlled fiber lifecycle. ### Fork in Global Scope You can create a long-running background fiber using `Effect.forkDaemon`. This type of fiber, known as a daemon fiber, is not tied to the lifecycle of its parent fiber. Instead, its lifetime is linked to the global scope. A daemon fiber continues running even if its parent fiber is terminated and will only stop when the global scope is closed or the fiber completes naturally. **Example** This example shows how daemon fibers can continue running in the background even after the parent fiber has finished. Even if the parent fiber is interrupted, the daemon fiber will continue running independently. **Example** In this example, interrupting the parent fiber doesn't affect the daemon fiber, which continues to run in the background. ### Fork in Local Scope Sometimes we want to create a fiber that is tied to a local , meaning its lifetime is not dependent on its parent fiber but is bound to the local scope in which it was forked. This can be done using the `Effect.forkScoped` operator. Fibers created with `Effect.forkScoped` can outlive their parent fibers and will only be terminated when the local scope itself is closed. **Example** In this example, the `child` fiber continues to run beyond the lifetime of the `parent` fiber. The `child` fiber is tied to the local scope and will be terminated only when the scope ends. ### Fork in Specific Scope There are some cases where we need more fine-grained control, so we want to fork a fiber in a specific scope. We can use the `Effect.forkIn` operator which takes the target scope as an argument. **Example** In this example, the `child` fiber is forked into the `outerScope`, allowing it to outlive the inner scope but still be terminated when the `outerScope` is closed. ## When do Fibers run? Forked fibers begin execution after the current fiber completes or yields. **Example** In the following example, the `changes` stream only captures a single value, `2`. This happens because the fiber created by `Effect.fork` starts **after** the value is updated. If you add a short delay with `Effect.sleep` or call `Effect.yieldNow`, you allow the current fiber to yield. This gives the forked fiber enough time to start and collect all values before they are updated. **Example** --- title: Latch description: A Latch synchronizes fibers by allowing them to wait until a specific event occurs, controlling access based on its open or closed state. sidebar: order: 8 --- A Latch is a synchronization tool that works like a gate, letting fibers wait until the latch is opened before they continue. The latch can be either open or closed: - When closed, fibers that reach the latch wait until it opens. - When open, fibers pass through immediately. Once opened, a latch typically stays open, although you can close it again if needed Imagine an application that processes requests only after completing an initial setup . You can create a latch in a closed state while the setup is happening. Any incoming requests, represented as fibers, would wait at the latch until it opens. Once the setup is finished, you call `latch.open` so the requests can proceed. ## The Latch Interface A `Latch` includes several operations that let you control and observe its state: | Operation | Description | | ---------- | -------------------------------------------------------------------------------------------------------- | | `whenOpen` | Runs a given effect only if the latch is open, otherwise, waits until it opens. | | `open` | Opens the latch so that any waiting fibers can proceed. | | `close` | Closes the latch, causing fibers to wait when they reach this latch in the future. | | `await` | Suspends the current fiber until the latch is opened. If the latch is already open, returns immediately. | | `release` | Allows waiting fibers to continue without permanently opening the latch. | ## Creating a Latch Use the `Effect.makeLatch` function to create a latch in an open or closed state by passing a boolean. The default is `false`, which means it starts closed. **Example** In this example, the latch starts closed. A fiber logs "open sesame" only when the latch is open. After waiting for one second, the latch is opened, releasing the fiber: ## Latch vs Semaphore A latch is good when you have a one-time event or condition that determines whether fibers can proceed. For example, you might use a latch to block all fibers until a setup step is finished, and then open the latch so everyone can continue. A with one lock is usually for mutual exclusion: it ensures that only one fiber at a time accesses a shared resource or section of code. Once a fiber acquires the lock, no other fiber can enter the protected area until the lock is released. In short: - Use a **latch** if you're gating a set of fibers on a specific event . - Use a **semaphore ** if you need to ensure only one fiber at a time is in a critical section or using a shared resource. --- title: PubSub description: Effortless message broadcasting and asynchronous communication with PubSub in Effect. sidebar: order: 6 --- import { Aside } from "@astrojs/starlight/components" A `PubSub` serves as an asynchronous message hub, allowing publishers to send messages that can be received by all current subscribers. Unlike a , where each value is delivered to only one consumer, a `PubSub` broadcasts each published message to all subscribers. This makes `PubSub` ideal for scenarios requiring message broadcasting rather than load distribution. ## Basic Operations A `PubSub` stores messages of type `A` and provides two fundamental operations: | API | Description | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `PubSub.publish` | Sends a message of type `A` to the `PubSub`, returning an effect indicating if the message was successfully published. | | `PubSub.subscribe` | Creates a scoped effect that allows subscription to the `PubSub`, automatically unsubscribing when the scope ends. Subscribers receive messages through a which holds published messages. | **Example** ## Creating a PubSub ### Bounded PubSub A bounded `PubSub` applies back pressure to publishers when it reaches capacity, suspending additional publishing until space becomes available. Back pressure ensures that all subscribers receive all messages while they are subscribed. However, it can lead to slower message delivery if a subscriber is slow. **Example** ### Dropping PubSub A dropping `PubSub` discards new values when full. The `PubSub.publish` operation returns `false` if the message is dropped. In a dropping pubsub, publishers can continue to publish new values, but subscribers are not guaranteed to receive all messages. **Example** ### Sliding PubSub A sliding `PubSub` removes the oldest message to make space for new ones, ensuring that publishing never blocks. A sliding pubsub prevents slow subscribers from impacting the message delivery rate. However, there's still a risk that slow subscribers may miss some messages. **Example** ### Unbounded PubSub An unbounded `PubSub` has no capacity limit, so publishing always succeeds immediately. Unbounded pubsubs guarantee that all subscribers receive all messages without slowing down message delivery. However, they can grow indefinitely if messages are published faster than they are consumed. Generally, it's recommended to use bounded, dropping, or sliding pubsubs unless you have specific use cases for unbounded pubsubs. **Example** ## Operators On PubSubs ### publishAll The `PubSub.publishAll` function lets you publish multiple values to the pubsub at once. **Example** ### capacity / size You can check the capacity and current size of a pubsub using `PubSub.capacity` and `PubSub.size`, respectively. Note that `PubSub.capacity` returns a `number` because the capacity is set at pubsub creation and never changes. In contrast, `PubSub.size` returns an effect that determines the current size of the pubsub since the number of messages in the pubsub can change over time. **Example** ### Shutting Down a PubSub To shut down a pubsub, use `PubSub.shutdown`. You can also verify if it has been shut down with `PubSub.isShutdown`, or wait for the shutdown to complete with `PubSub.awaitShutdown`. Shutting down a pubsub also terminates all associated queues, ensuring that the shutdown signal is effectively communicated. ## PubSub as an Enqueue `PubSub` operators mirror those of with the main difference being that `PubSub.publish` and `PubSub.subscribe` are used in place of `Queue.offer` and `Queue.take`. If you're already familiar with using a `Queue`, you’ll find `PubSub` straightforward. Essentially, a `PubSub` can be seen as a `Enqueue` that only allows writes: Here, the `Enqueue` type refers to a queue that only accepts enqueues . Any value enqueued here is published to the pubsub, and operations like shutdown will also affect the pubsub. This design makes `PubSub` highly flexible, letting you use it anywhere you need a `Enqueue` that only accepts published values. --- title: Queue description: Learn how to use Effect's Queue for lightweight, type-safe, and asynchronous workflows with built-in back-pressure. sidebar: order: 5 --- A `Queue` is a lightweight in-memory queue with built-in back-pressure, enabling asynchronous, purely-functional, and type-safe handling of data. ## Basic Operations A `Queue` stores values of type `A` and provides two fundamental operations: | API | Description | | ------------- | ---------------------------------------------------- | | `Queue.offer` | Adds a value of type `A` to the queue. | | `Queue.take` | Removes and returns the oldest value from the queue. | **Example** ## Creating a Queue Queues can be **bounded** or **unbounded** . Different types of queues handle new values differently when they reach capacity. ### Bounded Queue A bounded queue applies back-pressure when full, meaning any `Queue.offer` operation will suspend until there is space. **Example** ### Dropping Queue A dropping queue discards new values if the queue is full. **Example** ### Sliding Queue A sliding queue removes old values to make space for new ones when it reaches capacity. **Example** ### Unbounded Queue An unbounded queue has no capacity limit, allowing unrestricted additions. **Example** ## Adding Items to a Queue ### offer Use `Queue.offer` to add values to the queue. **Example** When using a back-pressured queue, `Queue.offer` suspends if the queue is full. To avoid blocking the main fiber, you can fork the `Queue.offer` operation. **Example** ### offerAll You can also add multiple items at once using `Queue.offerAll`. **Example** ## Consuming Items from a Queue ### take The `Queue.take` operation removes and returns the oldest item from the queue. If the queue is empty, `Queue.take` will suspend and only resume when an item is added. To prevent blocking, you can fork the `Queue.take` operation into a new fiber. **Example** ### poll To retrieve the queue's first item without suspending, use `Queue.poll`. If the queue is empty, `Queue.poll` returns `None`; if it has an item, it wraps it in `Some`. **Example** ### takeUpTo To retrieve multiple items, use `Queue.takeUpTo`, which returns up to the specified number of items. If there aren't enough items, it returns all available items without waiting for more. This function is particularly useful for batch processing when an exact number of items is not required. It ensures the program continues working with whatever data is currently available. If you need to wait for an exact number of items before proceeding, consider using . **Example** ### takeN Takes a specified number of elements from a queue. If the queue does not contain enough elements, the operation suspends until the required number of elements become available. This function is useful for scenarios where processing requires an exact number of items at a time, ensuring that the operation does not proceed until the batch is complete. **Example** ### takeAll To retrieve all items from the queue at once, use `Queue.takeAll`. This operation completes immediately, returning an empty collection if the queue is empty. **Example** ## Shutting Down a Queue ### shutdown The `Queue.shutdown` operation allows you to interrupt all fibers that are currently suspended on `offer*` or `take*` operations. This action also empties the queue and makes any future `offer*` and `take*` calls terminate immediately. **Example** ### awaitShutdown The `Queue.awaitShutdown` operation can be used to run an effect when the queue shuts down. It waits until the queue is closed and resumes immediately if the queue is already shut down. **Example** ## Offer-only / Take-only Queues Sometimes, you might want certain parts of your code to only add values to a queue or only retrieve values from a queue . Effect provides interfaces to enforce these specific capabilities. ### Enqueue All methods for adding values to a queue are defined by the `Enqueue` interface. This restricts the queue to only offer operations. **Example** ### Dequeue Similarly, all methods for retrieving values from a queue are defined by the `Dequeue` interface, which restricts the queue to only take operations. **Example** The `Queue` type combines both `Enqueue` and `Dequeue`, so you can easily pass it to different parts of your code, enforcing only `Enqueue` or `Dequeue` behaviors as needed. **Example** --- title: Semaphore description: Learn to use semaphores in Effect for precise control of concurrency, managing resource access, and coordinating asynchronous tasks effectively. sidebar: order: 7 --- import { Aside } from "@astrojs/starlight/components" A semaphore is a synchronization mechanism used to manage access to a shared resource. In Effect, semaphores help control resource access or coordinate tasks within asynchronous, concurrent operations. A semaphore acts as a generalized mutex, allowing a set number of **permits** to be held and released concurrently. Permits act like tickets, giving tasks or fibers controlled access to a shared resource. When no permits are available, tasks trying to acquire one will wait until a permit is released. ## Creating a Semaphore The `Effect.makeSemaphore` function initializes a semaphore with a specified number of permits. Each permit allows one task to access a resource or perform an operation concurrently, and multiple permits enable a configurable level of concurrency. **Example** ## withPermits The `withPermits` method lets you specify the number of permits required to run an effect. Once the specified permits are available, it runs the effect, automatically releasing the permits when the task completes. **Example** In this example, three tasks are started concurrently, but they run sequentially because the one-permit semaphore only allows one task to proceed at a time. **Example** In this example, we create a semaphore with five permits and use `withPermits` to allocate a different number of permits for each task: --- title: Configuration description: Efficiently manage application configurations with built-in types, flexible providers, and advanced features like defaults, validation, and redaction. sidebar: order: 5 --- import { Aside, Badge } from "@astrojs/starlight/components" Configuration is an essential aspect of any cloud-native application. Effect simplifies the process of managing configuration by offering a convenient interface for configuration providers. The configuration front-end in Effect enables ecosystem libraries and applications to specify their configuration requirements in a declarative manner. It offloads the complex tasks to a `ConfigProvider`, which can be supplied by third-party libraries. Effect comes bundled with a straightforward default `ConfigProvider` that retrieves configuration data from environment variables. This default provider can be used during development or as a starting point before transitioning to more advanced configuration providers. To make our application configurable, we need to understand three essential elements: - **Config Description**: We describe the configuration data using an instance of `Config`. If the configuration data is simple, such as a `string`, `number`, or `boolean`, we can use the built-in functions provided by the `Config` module. For more complex data types like , we can combine primitive configs to create a custom configuration description. - **Config Frontend**: We utilize the instance of `Config` to load the configuration data described by the instance . This process leverages the current `ConfigProvider` to retrieve the configuration. - **Config Backend**: The `ConfigProvider` serves as the underlying engine that manages the configuration loading process. Effect comes with a default config provider as part of its default services. This default provider reads the configuration data from environment variables. If we want to use a custom config provider, we can utilize the `Effect.withConfigProvider` API to configure the Effect runtime accordingly. ## Basic Configuration Types Effect provides several built-in types for configuration values, which you can use right out of the box: | Type | Description | | ---------- | -------------------------------------------------------------------------- | | `string` | Reads a configuration value as a string. | | `number` | Reads a value as a floating-point number. | | `boolean` | Reads a value as a boolean . | | `integer` | Reads a value as an integer. | | `date` | Parses a value into a `Date` object. | | `literal` | Reads a fixed literal . | | `logLevel` | Reads a value as a . | | `duration` | Parses a value as a time duration. | | `redacted` | Reads a **sensitive value**, ensuring it is protected when logged. | | `url` | Parses a value as a valid URL. | `string | number | boolean | null | bigint` **Example** Here's an example of loading a basic configuration using environment variables for `HOST` and `PORT`: If you run this without setting the required environment variables: you'll see an error indicating the missing configuration: To run the program successfully, set the environment variables as shown below: Output: ## Using Config with Schema You can define and decode configuration values using a schema. **Example** For more information, see the documentation. ## Providing Default Values Sometimes, you may encounter situations where an environment variable is missing, leading to an incomplete configuration. To address this, Effect provides the `Config.withDefault` function, which allows you to specify a default value. This fallback ensures that your application continues to function even if a required environment variable is not set. **Example** Running this program with only the `HOST` environment variable set: produces the following output: In this case, even though the `PORT` environment variable is not set, the program continues to run, using the default value of `8080` for the port. This ensures that the application remains functional without requiring every configuration to be explicitly provided. ## Handling Sensitive Values Some configuration values, like API keys, should not be printed in logs. The `Config.redacted` function is used to handle sensitive information safely. It parses the configuration value and wraps it in a `Redacted`, a specialized designed to protect secrets. When you log a `Redacted` value using `console.log`, the actual content remains hidden, providing an extra layer of security. To access the real value, you must explicitly use `Redacted.value`. **Example** When this program is executed: The output will look like this: As shown, when logging the `Redacted` value using `console.log`, the output is ``, ensuring that sensitive data remains concealed. However, by using `Redacted.value`, the true value can be accessed and displayed, providing controlled access to the secret. ### Wrapping a Config with Redacted By default, when you pass a string to `Config.redacted`, it returns a `Redacted`. You can also pass a `Config` to ensure that only validated values are accepted. This adds an extra layer of security by ensuring that sensitive data is properly validated before being redacted. **Example** ## Combining Configurations Effect provides several built-in combinators that allow you to define and manipulate configurations. These combinators take a `Config` as input and produce another `Config`, enabling more complex configuration structures. | Combinator | Description | | ---------- | ------------------------------------------------------------------------------------------------------------------- | | `array` | Constructs a configuration for an array of values. | | `chunk` | Constructs a configuration for a sequence of values. | | `option` | Returns an optional configuration. If the data is missing, the result will be `None`; otherwise, it will be `Some`. | | `repeat` | Describes a sequence of values, each following the structure of the given config. | | `hashSet` | Constructs a configuration for a set of values. | | `hashMap` | Constructs a configuration for a key-value map. | Additionally, there are three special combinators for specific use cases: | Combinator | Description | | ---------- | ------------------------------------------------------------------------ | | `succeed` | Constructs a config that contains a predefined value. | | `fail` | Constructs a config that fails with the specified error message. | | `all` | Combines multiple configurations into a tuple, struct, or argument list. | **Example** The following example demonstrates how to load an environment variable as an array of strings using the `Config.array` constructor. **Example** **Example** ## Operators Effect provides several built-in operators to work with configurations, allowing you to manipulate and transform them according to your needs. ### Transforming Operators These operators enable you to modify configurations or validate their values: | Operator | Description | | ------------ | --------------------------------------------------------------------------------------------------------- | | `validate` | Ensures that a configuration meets certain criteria, returning a validation error if it does not. | | `map` | Transforms the values of a configuration using a provided function. | | `mapAttempt` | Similar to `map`, but catches any errors thrown by the function and converts them into validation errors. | | `mapOrFail` | Like `map`, but the function can fail. If it does, the result is a validation error. | **Example** If we run this program with an invalid `NAME` value: The output will be: ### Fallback Operators Fallback operators are useful when you want to provide alternative configurations in case of errors or missing data. These operators ensure that your program can still run even if some configuration values are unavailable. | Operator | Description | | ---------- | ----------------------------------------------------------------------------------------------------- | | `orElse` | Attempts to use the primary config first. If it fails or is missing, it falls back to another config. | | `orElseIf` | Similar to `orElse`, but it switches to the fallback config only if the error matches a condition. | **Example** In this example, the program requires two configuration values: `A` and `B`. We set up two configuration providers, each containing only one of the required values. Using the `orElse` operator, we combine these providers so the program can retrieve both `A` and `B`. If we run this program: The output will be: ## Custom Configuration Types Effect allows you to define configurations for custom types by combining primitive configurations using and . For example, let's create a `HostPort` class, which has two fields: `host` and `port`. To define a configuration for this custom type, we can combine primitive configs for `string` and `number`: **Example** In this example, `Config.all` combines two primitive configurations, `Config` and `Config`, into a `Config<>`. The `Config.map` operator is then used to transform these values into an instance of the `HostPort` class. **Example** When you run this program, it will try to retrieve the values for `HOST` and `PORT` from your environment variables: If successful, it will print: ## Nested Configurations We've seen how to define configurations at the top level, whether for primitive or custom types. In some cases, though, you might want to structure your configurations in a more nested way, organizing them under common namespaces for clarity and manageability. For instance, consider the following `ServiceConfig` type: If you were to use this configuration in your application, it would expect the `HOST`, `PORT`, and `TIMEOUT` environment variables at the top level. But in many cases, you may want to organize configurations under a shared namespace—for example, grouping `HOST` and `PORT` under a `SERVER` namespace, while keeping `TIMEOUT` at the root. To do this, you can use the `Config.nested` operator, which allows you to nest configuration values under a specific namespace. Let's update the previous example to reflect this: Now, if you run your application with this configuration setup, it will look for the following environment variables: - `SERVER_HOST` for the host value - `SERVER_PORT` for the port value - `TIMEOUT` for the timeout value This structured approach keeps your configuration more organized, especially when dealing with multiple services or complex applications. ## Mocking Configurations in Tests When testing services, there are times when you need to provide specific configurations for your tests. To simulate this, it's useful to mock the configuration backend that reads these values. You can achieve this using the `ConfigProvider.fromMap` constructor. This method allows you to create a configuration provider from a `Map`, where the map represents the configuration data. You can then use this mock provider in place of the default one by calling `Effect.withConfigProvider`. **Example** This approach helps you create isolated tests that don't rely on external environment variables, ensuring your tests run consistently with mock configurations. ### Handling Nested Configuration Values For more complex setups, configurations often include nested keys. By default, `ConfigProvider.fromMap` uses `.` as the separator for nested keys. **Example** ### Customizing the Path Delimiter If your configuration data uses a different separator , you can change the delimiter using the `pathDelim` option in `ConfigProvider.fromMap`. **Example** ## ConfigProvider The `ConfigProvider` module in Effect allows applications to load configuration values from different sources. The default provider reads from environment variables, but you can customize its behavior when needed. ### Loading Configuration from Environment Variables The `ConfigProvider.fromEnv` function creates a `ConfigProvider` that loads values from environment variables. This is the default provider used by Effect unless another is specified. If your application requires a custom delimiter for nested configuration keys, you can configure `ConfigProvider.fromEnv` accordingly. **Example** The following example modifies the path delimiter and sequence delimiter for environment variables. To match the custom delimiter , set environment variables like this: Output: ### Loading Configuration from JSON The `ConfigProvider.fromJson` function creates a `ConfigProvider` that loads values from a JSON object. **Example** ### Using Nested Configuration Namespaces The `ConfigProvider.nested` function allows **grouping configuration values** under a namespace. This is helpful when structuring settings logically, such as grouping `SERVER`-related values. **Example** Since we defined `"SERVER"` as the namespace, the environment variables must follow this pattern: Output: ### Converting Configuration Keys to Constant Case The `ConfigProvider.constantCase` function transforms all configuration keys into constant case . This is useful when adapting environment variables to match different naming conventions. **Example** Since `constantCase` converts `"Port"` → `"PORT"` and `"Host"` → `"HOST"`, the environment variables must be set as follows: Output: ## Deprecations ### Secret _Deprecated since version 3.3.0: Please use for handling sensitive information going forward._ The `Config.secret` function was previously used to secure sensitive information in a similar way to `Config.redacted`. It wraps configuration values in a `Secret` type, which also conceals details when logged but allows access via `Secret.value`. **Example** When this program is executed: The output will look like this: --- title: BigDecimal description: The BigDecimal data type represents arbitrary-precision decimal numbers. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" In JavaScript, numbers are typically stored as 64-bit floating-point values. While floating-point numbers are fast and versatile, they can introduce small rounding errors. These are often hard to notice in everyday usage but can become problematic in areas like finance or statistics, where small inaccuracies may lead to larger discrepancies over time. By using the BigDecimal module, you can avoid these issues and perform calculations with a higher degree of precision. The `BigDecimal` data type can represent real numbers with a large number of decimal places, preventing the common errors of floating-point math . ## How BigDecimal Works A `BigDecimal` represents a number using two components: 1. `value`: A `BigInt` that stores the digits of the number. 2. `scale`: A 64-bit integer that determines the position of the decimal point. The number represented by a `BigDecimal` is calculated as: value x 10-scale. - If `scale` is zero or positive, it specifies the number of digits to the right of the decimal point. - If `scale` is negative, the `value` is multiplied by 10 raised to the power of the negated scale. For example: - A `BigDecimal` with `value = 12345n` and `scale = 2` represents `123.45`. - A `BigDecimal` with `value = 12345n` and `scale = -2` represents `1234500`. The maximum precision is large but not infinite, limited to 263 decimal places. ## Creating a BigDecimal ### make The `make` function creates a `BigDecimal` by specifying a `BigInt` value and a scale. The `scale` determines the number of digits to the right of the decimal point. **Example** ### fromBigInt The `fromBigInt` function creates a `BigDecimal` from a `bigint`. The `scale` defaults to `0`, meaning the number has no fractional part. **Example** ### fromString Parses a numerical string into a `BigDecimal`. Returns an `Option`: - `Some` if the string is valid. - `None` if the string is invalid. **Example** ### unsafeFromString The `unsafeFromString` function is a variant of `fromString` that throws an error if the input string is invalid. Use this only when you are confident that the input will always be valid. **Example** ### unsafeFromNumber Creates a `BigDecimal` from a JavaScript `number`. Throws a `RangeError` for non-finite numbers . **Example** ## Basic Arithmetic Operations The BigDecimal module supports a variety of arithmetic operations that provide precision and avoid the rounding errors common in standard JavaScript arithmetic. Below is a list of supported operations: | Function | Description | | ----------------- | -------------------------------------------------------------------------------------------------------------- | | `sum` | Adds two `BigDecimal` values. | | `subtract` | Subtracts one `BigDecimal` value from another. | | `multiply` | Multiplies two `BigDecimal` values. | | `divide` | Divides one `BigDecimal` value by another, returning an `Option`. | | `unsafeDivide` | Divides one `BigDecimal` value by another, throwing an error if the divisor is zero. | | `negate` | Negates a `BigDecimal` value . | | `remainder` | Returns the remainder of dividing one `BigDecimal` value by another, returning an `Option`. | | `unsafeRemainder` | Returns the remainder of dividing one `BigDecimal` value by another, throwing an error if the divisor is zero. | | `sign` | Returns the sign of a `BigDecimal` value . | | `abs` | Returns the absolute value of a `BigDecimal`. | **Example** Using `BigDecimal` for arithmetic operations helps to avoid the inaccuracies commonly encountered with floating-point numbers in JavaScript. For example: **Example** ## Comparison Operations The `BigDecimal` module provides several functions for comparing decimal values. These allow you to determine the relative order of two values, find the minimum or maximum, and check specific properties like positivity or integer status. ### Comparison Functions | Function | Description | | ---------------------- | ------------------------------------------------------------------------ | | `lessThan` | Checks if the first `BigDecimal` is smaller than the second. | | `lessThanOrEqualTo` | Checks if the first `BigDecimal` is smaller than or equal to the second. | | `greaterThan` | Checks if the first `BigDecimal` is larger than the second. | | `greaterThanOrEqualTo` | Checks if the first `BigDecimal` is larger than or equal to the second. | | `min` | Returns the smaller of two `BigDecimal` values. | | `max` | Returns the larger of two `BigDecimal` values. | **Example** ### Predicates for Comparison The module also includes predicates to check specific properties of a `BigDecimal`: | Predicate | Description | | ------------ | -------------------------------------------------------------- | | `isZero` | Checks if the value is exactly zero. | | `isPositive` | Checks if the value is positive. | | `isNegative` | Checks if the value is negative. | | `between` | Checks if the value lies within a specified range . | | `isInteger` | Checks if the value is an integer . | **Example** ## Normalization and Equality In some cases, two `BigDecimal` values can have different internal representations but still represent the same number. For example, `1.05` could be internally represented with different scales, such as: - `105n` with a scale of `2` - `1050n` with a scale of `3` To ensure consistency, you can normalize a `BigDecimal` to adjust the scale and remove trailing zeros. ### Normalization The `BigDecimal.normalize` function adjusts the scale of a `BigDecimal` and eliminates any unnecessary trailing zeros in its internal representation. **Example** ### Equality To check if two `BigDecimal` values are numerically equal, regardless of their internal representation, use the `BigDecimal.equals` function. **Example** --- title: Cause description: Comprehensive error analysis with Cause in Effect - track failures, defects, and interruptions with precise details. sidebar: order: 2 --- The type is polymorphic in error type `E`, allowing flexibility in handling any desired error type. However, there is often additional information about failures that the error type `E` alone does not capture. To address this, Effect uses the `Cause` data type to store various details such as: - Unexpected errors or defects - Stack and execution traces - Reasons for fiber interruptions Effect strictly preserves all failure-related information, storing a full picture of the error context in the `Cause` type. This comprehensive approach enables precise analysis and handling of failures, ensuring no data is lost. Though `Cause` values aren't typically manipulated directly, they underlie errors within Effect workflows, providing access to both concurrent and sequential error details. This allows for thorough error analysis when needed. ## Creating Causes You can intentionally create an effect with a specific cause using `Effect.failCause`. **Example** Some causes do not influence the error type of the effect, leading to `never` in the error channel: For instance, `Cause.die` does not specify an error type for the effect, while `Cause.fail` does, setting the error channel type accordingly. ## Cause Variations There are several causes for various errors, in this section, we will describe each of these causes. ### Empty The `Empty` cause signifies the absence of any errors. ### Fail The `Fail` cause represents a failure due to an expected error of type `E`. ### Die The `Die` cause indicates a failure resulting from a defect, which is an unexpected or unintended error. ### Interrupt The `Interrupt` cause represents a failure due to `Fiber` interruption and contains the `FiberId` of the interrupted `Fiber`. ### Sequential The `Sequential` cause combines two causes that occurred one after the other. For example, in an `Effect.ensuring` operation , if both the `try` and `finally` sections fail, the two errors are represented in sequence by a `Sequential` cause. **Example** ### Parallel The `Parallel` cause combines two causes that occurred concurrently. In Effect programs, two operations may run in parallel, potentially leading to multiple failures. When both computations fail simultaneously, a `Parallel` cause represents the concurrent errors within the effect workflow. **Example** ## Retrieving the Cause of an Effect To retrieve the cause of a failed effect, use `Effect.cause`. This allows you to inspect or handle the exact reason behind the failure. **Example** ## Guards To determine the specific type of a `Cause`, use the guards provided in the Cause module: - `Cause.isEmpty`: Checks if the cause is empty, indicating no error. - `Cause.isFailType`: Identifies causes that represent an expected failure. - `Cause.isDie`: Identifies causes that represent an unexpected defect. - `Cause.isInterruptType`: Identifies causes related to fiber interruptions. - `Cause.isSequentialType`: Checks if the cause consists of sequential errors. - `Cause.isParallelType`: Checks if the cause contains parallel errors. **Example** These guards allow you to accurately identify the type of a `Cause`, making it easier to handle various error cases in your code. Whether dealing with expected failures, unexpected defects, interruptions, or composite errors, these guards provide a clear method for assessing and managing error scenarios. ## Pattern Matching The `Cause.match` function provides a straightforward way to handle each case of a `Cause`. By defining callbacks for each possible cause type, you can respond to specific error scenarios with custom behavior. **Example** ## Pretty Printing Clear and readable error messages are key for effective debugging. The `Cause.pretty` function helps by formatting error messages in a structured way, making it easier to understand failure details. **Example** ## Retrieval of Failures and Defects To specifically collect failures or defects from a `Cause`, you can use `Cause.failures` and `Cause.defects`. These functions allow you to inspect only the errors or unexpected defects that occurred. **Example** --- title: Chunk description: Learn about Chunk, a high-performance immutable data structure in Effect, offering efficient operations like concatenation, slicing, and conversions. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" A `Chunk` represents an ordered, immutable collection of values of type `A`. While similar to an array, `Chunk` provides a functional interface, optimizing certain operations that can be costly with regular arrays, like repeated concatenation. ## Why Use Chunk? - **Immutability**: Unlike standard JavaScript arrays, which are mutable, `Chunk` provides a truly immutable collection, preventing data from being modified after creation. This is especially useful in concurrent programming contexts where immutability can enhance data consistency. - **High Performance**: `Chunk` supports specialized operations for efficient array manipulation, such as appending single elements or concatenating chunks, making these operations faster than their regular JavaScript array equivalents. ## Creating a Chunk ### empty Create an empty `Chunk` with `Chunk.empty`. **Example** ### make To create a `Chunk` with specific values, use `Chunk.make`. Note that the resulting chunk is typed as non-empty. **Example** ### fromIterable You can create a `Chunk` by providing a collection, either from an iterable or directly from an array. **Example** ### unsafeFromArray `Chunk.unsafeFromArray` creates a `Chunk` directly from an array without cloning. This approach can improve performance by avoiding the overhead of copying data but requires caution, as it bypasses the usual immutability guarantees. **Example** ## Concatenating To combine two `Chunk` instances into one, use `Chunk.appendAll`. **Example** ## Dropping To remove elements from the beginning of a `Chunk`, use `Chunk.drop`, specifying the number of elements to discard. **Example** ## Comparing To check if two `Chunk` instances are equal, use . This function compares the contents of each `Chunk` for structural equality. **Example** ## Converting Convert a `Chunk` to a `ReadonlyArray` using `Chunk.toReadonlyArray`. The resulting type varies based on the `Chunk`'s contents, distinguishing between empty, non-empty, and generic chunks. **Example** --- title: Data description: Define immutable data structures, ensure equality, and manage errors seamlessly with Effect's Data module. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" The Data module simplifies creating and handling data structures in TypeScript. It provides tools for **defining data types**, ensuring **equality** between objects, and **hashing** data for efficient comparisons. ## Value Equality The Data module provides constructors for creating data types with built-in support for equality and hashing, eliminating the need for custom implementations. This means that two values created using these constructors are considered equal if they have the same structure and values. ### struct In plain JavaScript, objects are considered equal only if they refer to the exact same instance. **Example** However, the `Data.struct` constructor allows you to compare values based on their structure and content. **Example** The comparison performed by `Equal.equals` is **shallow**, meaning nested objects are not compared recursively unless they are also created using `Data.struct`. **Example** To ensure nested objects are compared by structure, use `Data.struct` for them as well. **Example** ### tuple To represent your data using tuples, you can use the `Data.tuple` constructor. This ensures that your tuples can be compared structurally. **Example** ### array You can use `Data.array` to create an array-like data structure that supports structural equality. **Example** ## Constructors The module introduces a concept known as "Case classes", which automate various essential operations when defining data types. These operations include generating **constructors**, handling **equality** checks, and managing **hashing**. Case classes can be defined in two primary ways: - as plain objects using `case` or `tagged` - as TypeScript classes using `Class` or `TaggedClass` ### case The `Data.case` helper generates constructors and built-in support for equality checks and hashing for your data type. **Example** In this example, `Data.case` is used to create a constructor for `Person`. The resulting instances have built-in support for equality checks, allowing you to compare them directly using `Equal.equals`. **Example** This example demonstrates using `Data.case` to create nested data structures, such as a `Person` type containing an `Address`. Both `Person` and `Address` constructors support equality checks. Alternatively, you can use `Data.struct` to create nested data structures without defining a separate `Address` constructor. **Example** **Example** This example demonstrates a recursive structure using `Data.case` to define a binary tree where each node can contain other nodes. ### tagged When you're working with a data type that includes a tag field, like in disjoint union types, defining the tag manually for each instance can get repetitive. Using the `case` approach requires you to specify the tag field every time, which can be cumbersome. **Example** Here, we create a `Person` type with a `_tag` field using `Data.case`. Notice that the `_tag` needs to be specified for every new instance. To streamline this process, the `Data.tagged` helper automatically adds the tag. It follows the convention in the Effect ecosystem of naming the tag field as `"_tag"`. **Example** The `Data.tagged` helper allows you to define the tag just once, making instance creation simpler. ### Class If you prefer working with classes instead of plain objects, you can use `Data.Class` as an alternative to `Data.case`. This approach may feel more natural in scenarios where you want a class-oriented structure, complete with methods and custom logic. **Example** Here's how to define a `Person` class using `Data.Class`: One of the benefits of using classes is that you can easily add custom methods and getters. This allows you to extend the functionality of your data types. **Example** In this example, we add a `upperName` getter to the `Person` class to return the name in uppercase: ### TaggedClass If you prefer a class-based approach but also want the benefits of tagging for disjoint unions, `Data.TaggedClass` can be a helpful option. It works similarly to `tagged` but is tailored for class definitions. **Example** Here's how to define a `Person` class using `Data.TaggedClass`. Notice that the tag `"Person"` is automatically added: One benefit of using tagged classes is the ability to easily add custom methods and getters, extending the class's functionality as needed. **Example** In this example, we add a `upperName` getter to the `Person` class, which returns the name in uppercase: ## Union of Tagged Structs To create a disjoint union of tagged structs, you can use `Data.TaggedEnum` and `Data.taggedEnum`. These utilities make it straightforward to define and work with unions of plain objects. ### Definition The type passed to `Data.TaggedEnum` must be an object where the keys represent the tags, and the values define the structure of the corresponding data types. **Example** ### $is and $match The `Data.taggedEnum` provides `$is` and `$match` functions for convenient type guarding and pattern matching. **Example** ### Adding Generics You can create more flexible and reusable tagged unions by using `TaggedEnum.WithGenerics`. This approach allows you to define tagged unions that can handle different types dynamically. **Example** ## Errors In Effect, handling errors is simplified using specialized constructors: - `Error` - `TaggedError` These constructors make defining custom error types straightforward, while also providing useful integrations like equality checks and structured error handling. ### Error `Data.Error` lets you create an `Error` type with extra fields beyond the typical `message` property. **Example** You can yield an instance of `NotFound` directly in an , without needing to use `Effect.fail`. **Example** ### TaggedError Effect provides a `TaggedError` API to add a `_tag` field automatically to your custom errors. This simplifies error handling with APIs like or . ### Native Cause Support Errors created using `Data.Error` or `Data.TaggedError` can include a `cause` property, integrating with the native `cause` feature of JavaScript's `Error` for more detailed error tracing. **Example** --- title: DateTime description: Work with precise points in time using Effect's DateTime, supporting creation, comparison, and arithmetic operations for efficient time handling. sidebar: order: 5 --- import { Aside } from "@astrojs/starlight/components" Working with dates and times in JavaScript can be challenging. The built-in `Date` object mutates its internal state, and time zone handling can be confusing. These design choices can lead to errors when working on applications that rely on date-time accuracy, such as scheduling systems, timestamping services, or logging utilities. The DateTime module aims to address these limitations by offering: - **Immutable Data**: Each `DateTime` is an immutable structure, reducing mistakes related to in-place mutations. - **Time Zone Support**: `DateTime` provides robust support for time zones, including automatic daylight saving time adjustments. - **Arithmetic Operations**: You can perform arithmetic operations on `DateTime` instances, such as adding or subtracting durations. ## The DateTime Type A `DateTime` represents a moment in time. It can be stored as either a simple UTC value or as a value with an associated time zone. Storing time this way helps you manage both precise timestamps and the context for how that time should be displayed or interpreted. There are two main variants of `DateTime`: 1. **Utc**: An immutable structure that uses `epochMillis` to represent a point in time in Coordinated Universal Time . 2. **Zoned**: Includes `epochMillis` along with a `TimeZone`, allowing you to attach an offset or a named region to the timestamp. ### Why Have Two Variants? - **Utc** is straightforward if you only need a universal reference without relying on local time zones. - **Zoned** is helpful when you need to keep track of time zone information for tasks such as converting to local times or adjusting for daylight saving time. ### TimeZone Variants A `TimeZone` can be either: - **Offset**: Represents a fixed offset from UTC . - **Named**: Uses a named region that automatically accounts for region-specific rules like daylight saving time changes. ### TypeScript Definition Below is the TypeScript definition for the `DateTime` type: ## The DateTime.Parts Type The `DateTime.Parts` type defines the main components of a date, such as the year, month, day, hours, minutes, and seconds. ## The DateTime.Input Type The `DateTime.Input` type is a flexible input type that can be used to create a `DateTime` instance. It can be one of the following: - A `DateTime` instance - A JavaScript `Date` object - A numeric value representing milliseconds since the Unix epoch - An object with partial date - A string that can be parsed by JavaScript's ## Utc Constructors `Utc` is an immutable structure that uses `epochMillis` to represent a point in time in Coordinated Universal Time . ### unsafeFromDate Creates a `Utc` from a JavaScript `Date`. Throws an `IllegalArgumentException` if the provided `Date` is invalid. When a `Date` object is passed, it is converted to a `Utc` instance. The time is interpreted as the local time of the system executing the code and then adjusted to UTC. This ensures a consistent, timezone-independent representation of the date and time. **Example** The following example assumes the code is executed on a system in Italy : **Explanation**: - The local time **2025-01-01 04:00:00** is converted to **UTC** by subtracting the timezone offset . - As a result, the UTC time becomes **2025-01-01 03:00:00.000Z**. - `epochMillis` provides the same time as milliseconds since the Unix Epoch, ensuring a precise numeric representation of the UTC timestamp. ### unsafeMake Creates a `Utc` from a . **Example** The following example assumes the code is executed on a system in Italy : **Explanation**: - The local time **2025-01-01 04:00:00** is converted to **UTC** by subtracting the timezone offset . - As a result, the UTC time becomes **2025-01-01 03:00:00.000Z**. ### make Similar to , but returns an instead of throwing an error if the input is invalid. If the input is invalid, it returns `None`. If valid, it returns `Some` containing the `Utc`. **Example** The following example assumes the code is executed on a system in Italy : **Explanation**: - The local time **2025-01-01 04:00:00** is converted to **UTC** by subtracting the timezone offset . - As a result, the UTC time becomes **2025-01-01 03:00:00.000Z**. ## Zoned Constructors A `Zoned` includes `epochMillis` along with a `TimeZone`, allowing you to attach an offset or a named region to the timestamp. ### unsafeMakeZoned Creates a `Zoned` by combining a with an optional `TimeZone`. This allows you to represent a specific point in time with an associated time zone. The time zone can be provided in several ways: - As a `TimeZone` object - A string identifier - A numeric offset in milliseconds If the input or time zone is invalid, an `IllegalArgumentException` is thrown. **Example** The following example assumes the code is executed on a system in Italy : Here, the system's time zone is used to create the `Zoned` instance. **Example** The following example assumes the code is executed on a system in Italy : In this case, the `"Europe/Rome"` time zone is explicitly provided, resulting in the `Zoned` instance being tied to this named time zone. By default, the input date is treated as a UTC value and then adjusted for the specified time zone. To interpret the input date as being in the specified time zone, you can use the `adjustForTimeZone` option. **Example** The following example assumes the code is executed on a system in Italy : **Explanation** - **Without `adjustForTimeZone`**: The input date is interpreted as UTC and then adjusted to the specified time zone. For instance, `2025-01-01 04:00:00` in UTC becomes `2025-01-01T04:00:00.000+01:00` in CET . - **With `adjustForTimeZone: true`**: The input date is interpreted as being in the specified time zone. For example, `2025-01-01 04:00:00` in "Europe/Rome" is adjusted to its corresponding UTC time, resulting in `2025-01-01T03:00:00.000+01:00`. ### makeZoned The `makeZoned` function works similarly to but provides a safer approach. Instead of throwing an error when the input is invalid, it returns an `Option`. If the input is invalid, it returns `None`. If valid, it returns `Some` containing the `Zoned`. **Example** ### makeZonedFromString Creates a `Zoned` by parsing a string in the format `YYYY-MM-DDTHH:mm:ss.sss+HH:MM`. If the input string is valid, the function returns a `Some` containing the `Zoned`. If the input is invalid, it returns `None`. **Example** ## Current Time ### now Provides the current UTC time as a `Effect`, using the service. **Example** ### unsafeNow Retrieves the current UTC time immediately using `Date.now`, without the service. **Example** ## Guards | Function | Description | | ------------------ | ---------------------------------------------- | | `isDateTime` | Checks if a value is a `DateTime`. | | `isTimeZone` | Checks if a value is a `TimeZone`. | | `isTimeZoneOffset` | Checks if a value is a `TimeZone.Offset`. | | `isTimeZoneNamed` | Checks if a value is a `TimeZone.Named`. | | `isUtc` | Checks if a `DateTime` is the `Utc` variant. | | `isZoned` | Checks if a `DateTime` is the `Zoned` variant. | **Example** ## Time Zone Management | Function | Description | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | `setZone` | Creates a `Zoned` from `DateTime` by applying the given `TimeZone`. | | `setZoneOffset` | Creates a `Zoned` from `DateTime` using a fixed offset . | | `setZoneNamed` | Creates a `Zoned` from `DateTime` from an IANA time zone identifier or returns `None` if invalid. | | `unsafeSetZoneNamed` | Creates a `Zoned` from `DateTime` from an IANA time zone identifier or throws if invalid. | | `zoneUnsafeMakeNamed` | Creates a `TimeZone.Named` from a IANA time zone identifier or throws if the identifier is invalid. | | `zoneMakeNamed` | Creates a `TimeZone.Named` from a IANA time zone identifier or returns `None` if invalid. | | `zoneMakeNamedEffect` | Creates a `Effect` from a IANA time zone identifier failing with `IllegalArgumentException` if invalid | | `zoneMakeOffset` | Creates a `TimeZone.Offset` from a numeric offset in milliseconds. | | `zoneMakeLocal` | Creates a `TimeZone.Named` from the system's local time zone. | | `zoneFromString` | Attempts to parse a time zone from a string, returning `None` if invalid. | | `zoneToString` | Returns a string representation of a `TimeZone`. | **Example** ### zoneFromString Parses a string to create a `DateTime.TimeZone`. This function attempts to interpret the input string as either: - A numeric time zone offset - An IANA time zone identifier If the string matches an offset format, it is converted into a `TimeZone.Offset`. Otherwise, it attempts to create a `TimeZone.Named` using the input. If the input string is invalid, `Option.none` is returned. **Example** ## Comparisons | Function | Description | | -------------------------------------------- | ------------------------------------------------------------ | | `distance` | Returns the difference between two `DateTime`s. | | `distanceDurationEither` | Returns a `Left` or `Right` `Duration` depending on order. | | `distanceDuration` | Returns a `Duration` indicating how far apart two times are. | | `min` | Returns the earlier of two `DateTime` values. | | `max` | Returns the later of two `DateTime` values. | | `greaterThan`, `greaterThanOrEqualTo`, etc. | Checks ordering between two `DateTime` values. | | `between` | Checks if a `DateTime` lies within the given bounds. | | `isFuture`, `isPast`, `unsafeIsFuture`, etc. | Checks if a `DateTime` is in the future or past. | **Example** ## Conversions | Function | Description | | ---------------- | ----------------------------------------------------------------------- | | `toDateUtc` | Returns a JavaScript `Date` in UTC. | | `toDate` | Applies the time zone and converts to a JavaScript `Date`. | | `zonedOffset` | For a `Zoned` DateTime, returns the time zone offset in ms. | | `zonedOffsetIso` | For a `Zoned` DateTime, returns an ISO offset string like "+01:00". | | `toEpochMillis` | Returns the Unix epoch time in milliseconds. | | `removeTime` | Returns a `Utc` with the time cleared . | ## Parts | Function | Description | | -------------------------- | ---------------------------------------------------------------------- | | `toParts` | Returns time zone adjusted date parts . | | `toPartsUtc` | Returns UTC date parts . | | `getPart` / `getPartUtc` | Retrieves a specific part from the date. | | `setParts` / `setPartsUtc` | Updates certain parts of a date, preserving or ignoring the time zone. | **Example** ## Math | Function | Description | | ------------------ | ------------------------------------------------------------------------------------------ | | `addDuration` | Adds the given `Duration` to a `DateTime`. | | `subtractDuration` | Subtracts the given `Duration` from a `DateTime`. | | `add` | Adds numeric parts to a `DateTime`. | | `subtract` | Subtracts numeric parts. | | `startOf` | Moves a `DateTime` to the start of the given unit . | | `endOf` | Moves a `DateTime` to the end of the given unit. | | `nearest` | Rounds a `DateTime` to the nearest specified unit. | ## Formatting | Function | Description | | ------------------ | -------------------------------------------------------------------- | | `format` | Formats a `DateTime` as a string using the `DateTimeFormat` API. | | `formatLocal` | Uses the system's local time zone and locale for formatting. | | `formatUtc` | Forces UTC formatting. | | `formatIntl` | Uses a provided `Intl.DateTimeFormat`. | | `formatIso` | Returns an ISO 8601 string in UTC. | | `formatIsoDate` | Returns an ISO date string, adjusted for the time zone. | | `formatIsoDateUtc` | Returns an ISO date string in UTC. | | `formatIsoOffset` | Formats a `Zoned` as a string with an offset like "+01:00". | | `formatIsoZoned` | Formats a `Zoned` in the form `YYYY-MM-DDTHH:mm:ss.sss+HH:MM`. | ## Layers for Current Time Zone | Function | Description | | ------------------------ | -------------------------------------------------------------------- | | `CurrentTimeZone` | A service tag for the current time zone. | | `setZoneCurrent` | Sets a `DateTime` to use the current time zone. | | `withCurrentZone` | Provides an effect with a specified time zone. | | `withCurrentZoneLocal` | Uses the system's local time zone for the effect. | | `withCurrentZoneOffset` | Uses a fixed offset for the effect. | | `withCurrentZoneNamed` | Uses a named time zone identifier . | | `nowInCurrentZone` | Retrieves the current time as a `Zoned` in the configured time zone. | | `layerCurrentZone` | Creates a Layer providing the `CurrentTimeZone` service. | | `layerCurrentZoneOffset` | Creates a Layer from a fixed offset. | | `layerCurrentZoneNamed` | Creates a Layer from a named time zone, failing if invalid. | | `layerCurrentZoneLocal` | Creates a Layer from the system's local time zone. | **Example** --- title: Duration description: Work with precise time spans using Effect's Duration, supporting creation, comparison, and arithmetic operations for efficient time handling. sidebar: order: 6 --- The `Duration` data type data type is used to represent specific non-negative spans of time. It is commonly used to represent time intervals or durations in various operations, such as timeouts, delays, or scheduling. The `Duration` type provides a convenient way to work with time units and perform calculations on durations. ## Creating Durations The Duration module includes several constructors to create durations in different units. **Example** You can create durations using units such as nanoseconds, microsecond, milliseconds, seconds, minutes, hours, days, and weeks. For an infinite duration, use `Duration.infinity`. **Example** Another option for creating durations is using the `Duration.decode` helper: - `number` values are treated as milliseconds. - `bigint` values are treated as nanoseconds. - Strings must follow the format `"${number} ${unit}"`. **Example** ## Getting the Duration Value You can retrieve the value of a duration in milliseconds using `Duration.toMillis`. **Example** To get the value of a duration in nanoseconds, use `Duration.toNanos`. Note that `toNanos` returns an `Option` because the duration might be infinite. **Example** To get a `bigint` value without `Option`, use `Duration.unsafeToNanos`. However, it will throw an error for infinite durations. **Example** ## Comparing Durations Use the following functions to compare two durations: | API | Description | | ---------------------- | ---------------------------------------------------------------------------- | | `lessThan` | Returns `true` if the first duration is less than the second. | | `lessThanOrEqualTo` | Returns `true` if the first duration is less than or equal to the second. | | `greaterThan` | Returns `true` if the first duration is greater than the second. | | `greaterThanOrEqualTo` | Returns `true` if the first duration is greater than or equal to the second. | **Example** ## Performing Arithmetic Operations You can perform arithmetic operations on durations, like addition and multiplication. **Example** ## Conversions Converts a `Duration` to a human readable string. **Example** --- title: Either description: Represent exclusive values as Left or Right with the Either data type, enabling precise control flow in computations. sidebar: order: 7 --- import { Aside } from "@astrojs/starlight/components" The `Either` data type represents two exclusive values: an `Either` can be a `Right` value or a `Left` value, where `R` is the type of the `Right` value, and `L` is the type of the `Left` value. ## Understanding Either and Exit Either is primarily used as a **simple discriminated union** and is not recommended as the main result type for operations requiring detailed error information. is the preferred **result type** within Effect for capturing comprehensive details about failures. It encapsulates the outcomes of effectful computations, distinguishing between success and various failure modes, such as errors, defects and interruptions. ## Creating Eithers You can create an `Either` using the `Either.right` and `Either.left` constructors. Use `Either.right` to create a `Right` value of type `R`. **Example** Use `Either.left` to create a `Left` value of type `L`. **Example** ## Guards Use `Either.isLeft` and `Either.isRight` to check whether an `Either` is a `Left` or `Right` value. **Example** ## Pattern Matching Use `Either.match` to handle both cases of an `Either` by specifying separate callbacks for `Left` and `Right`. **Example** ## Mapping ### Mapping over the Right Value Use `Either.map` to transform the `Right` value of an `Either`. The function you provide will only apply to the `Right` value, leaving any `Left` value unchanged. **Example** ### Mapping over the Left Value Use `Either.mapLeft` to transform the `Left` value of an `Either`. The provided function only applies to the `Left` value, leaving any `Right` value unchanged. **Example** ### Mapping over Both Values Use `Either.mapBoth` to transform both the `Left` and `Right` values of an `Either`. This function takes two separate transformation functions: one for the `Left` value and another for the `Right` value. **Example** ## Interop with Effect The `Either` type works as a subtype of the `Effect` type, allowing you to use it with functions from the `Effect` module. While these functions are built to handle `Effect` values, they can also manage `Either` values correctly. ### How Either Maps to Effect | Either Variant | Mapped to Effect | Description | | -------------- | ------------------ | -------------------- | | `Left` | `Effect` | Represents a failure | | `Right` | `Effect` | Represents a success | **Example** ## Combining Two or More Eithers ### zipWith The `Either.zipWith` function lets you combine two `Either` values using a provided function. It creates a new `Either` that holds the combined value of both original `Either` values. **Example** If either of the `Either` values is `Left`, the result will be `Left`, holding the first encountered `Left` value: **Example** ### all To combine multiple `Either` values without transforming their contents, you can use `Either.all`. This function returns an `Either` with a structure matching the input: - If you pass a tuple, the result will be a tuple of the same length. - If you pass a struct, the result will be a struct with the same keys. - If you pass an `Iterable`, the result will be an array. **Example** If one or more `Either` values are `Left`, the first `Left` encountered is returned: **Example** ## gen Similar to , `Either.gen` provides a more readable, generator-based syntax for working with `Either` values, making code that involves `Either` easier to write and understand. This approach is similar to using `async/await` but tailored for `Either`. **Example** When any of the `Either` values in the sequence is `Left`, the generator immediately returns the `Left` value, skipping further operations: **Example** In this example, `Either.gen` halts execution as soon as it encounters the `Left` value, effectively propagating the error without performing further operations. The use of `console.log` in these example is for demonstration purposes only. When using `Either.gen`, avoid including side effects in your generator functions, as `Either` should remain a pure data structure. --- title: Exit description: Represent the result of an Effect workflow with Exit, capturing success values or failure causes. sidebar: order: 8 --- An `Exit` describes the result of running an `Effect` workflow. There are two possible states for an `Exit`: - `Exit.Success`: Contains a success value of type `A`. - `Exit.Failure`: Contains a failure of type `E`. ## Creating Exits The Exit module provides two primary functions for constructing exit values: `Exit.succeed` and `Exit.failCause`. These functions represent the outcomes of an effectful computation in terms of success or failure. ### succeed `Exit.succeed` creates an `Exit` value that represents a successful outcome. You use this function when you want to indicate that a computation completed successfully and to provide the resulting value. **Example** ### failCause `Exit.failCause` creates an `Exit` value that represents a failure. The failure is described using a object, which can encapsulate expected errors, defects, interruptions, or even composite errors. **Example** ## Pattern Matching You can handle different outcomes of an `Exit` using the `Exit.match` function. This function lets you provide two separate callbacks to handle both success and failure cases of an `Effect` execution. **Example** ## Exit vs Either Conceptually, `Exit` can be thought of as `Either>`. However, the type represents more than just expected errors of type `E`. It includes: - Interruption causes - Defects - The combination of multiple causes This allows `Cause` to capture richer and more complex error states compared to a simple `Either`. ## Exit vs Effect `Exit` is actually a subtype of `Effect`. This means that `Exit` values can also be considered as `Effect` values. - An `Exit`, in essence, is a "constant computation". - `Effect.succeed` is essentially the same as `Exit.succeed`. - `Effect.failCause` is the same as `Exit.failCause`. --- title: HashSet description: Learn about HashSet data structures - both immutable and mutable variants. sidebar: order: 9 --- import { Aside } from "@astrojs/starlight/components" A HashSet represents an **unordered** collection of **unique** values with efficient lookup, insertion and removal operations. The Effect library provides two versions of this structure: - - Immutable - - Mutable Both versions provide constant-time operations on average. The main difference is how they handle changes: one returns new sets, the other modifies the original. ### Why use HashSet? HashSet solves the problem of maintaining an **unsorted collection where each value appears exactly once**, with fast operations for checking membership and adding/removing values. Some common use cases include: - Tracking unique items - Efficiently testing for membership in a collection - Performing set operations like union, intersection, and difference - Eliminating duplicates from a collection ### When to use HashSet Instead of other collections Choose HashSet over other collections when: - You need to ensure elements are unique - You frequently need to check if an element exists in the collection - You need to perform set operations like union, intersection, and difference - The order of elements doesn't matter to your use case Choose other collections when: - You need to maintain insertion order - You need key-value associations - You need to frequently access elements by index ### Choosing between immutable and mutable variants Effect offers both immutable and mutable versions to support different coding styles and performance needs. **HashSet** This version never modifies the original set. Instead, it returns a new set for every change. Characteristics: - Operations return new instances instead of modifying the original - Previous states are preserved - Thread-safe by design - Ideal for functional programming patterns - Suitable for sharing across different parts of your application **MutableHashSet** This version allows direct updates: adding and removing values changes the set in place. Characteristics: - Operations modify the original set directly - More efficient when building sets incrementally - Requires careful handling to avoid unexpected side effects - Better performance in scenarios with many modifications - Ideal for localized use where mutations won't cause issues elsewhere ### When to use each variant Use **HashSet** when: - You need predictable behavior with no side effects - You want to preserve previous states of your data - You're sharing sets across different parts of your application - You prefer functional programming patterns - You need fiber safety in concurrent environments Use **MutableHashSet** when: - Performance is critical, and you need to avoid creating new instances - You're building a collection incrementally with many additions/removals - You're working in a controlled scope where mutation is safe - You need to optimize memory usage in performance-critical code ### Hybrid approach You can apply multiple updates to a `HashSet` in a temporary mutable context using `HashSet.mutate`. This allows you to perform several changes at once without modifying the original set. **Example** ## Performance characteristics Both `HashSet` and `MutableHashSet` offer similar average-time performance for core operations: | Operation | HashSet | MutableHashSet | Description | | -------------- | ------------ | -------------- | ------------------------------- | | Lookup | O average | O average | Check if a value exists | | Insertion | O average | O average | Add a value | | Removal | O average | O average | Remove a value | | Iteration | O | O | Iterate over all values | | Set operations | O | O | Union, intersection, difference | The main difference is how updates are handled: - **HashSet** returns a new set for each change. This can be slower if many changes are made in a row. - **MutableHashSet** updates the same set in place. This is usually faster when performing many changes. ## Equality and uniqueness Both `HashSet` and `MutableHashSet` use Effect's trait to determine if two elements are the same. This ensures that each value appears only once in the set. - **Primitive values** are compared by value, similar to the `===` operator. - **Objects and custom types** must implement the `Equal` interface to define what it means for two instances to be equal. If no implementation is provided, equality falls back to reference comparison. **Example** ### Simplifying Equality with Data and Schema Effect's and modules implement `Equal` for you automatically, based on structural equality. **Example** **Example** ## HashSet A `HashSet` is an **immutable**, **unordered** collection of **unique** values. It guarantees that each value appears only once and supports fast operations like lookup, insertion, and removal. Any operation that would modify the set returns a new `HashSet`, leaving the original unchanged. ### Operations | Category | Operation | Description | Time Complexity | | ------------ | -------------------------------------------------------- | ------------------------------------------- | --------------- | | constructors | | Creates an empty HashSet | O | | constructors | | Creates a HashSet from an iterable | O | | constructors | | Creates a HashSet from multiple values | O | | elements | | Checks if a value exists in the set | O avg | | elements | | Checks if any element satisfies a predicate | O | | elements | | Checks if all elements satisfy a predicate | O | | elements | | Checks if a set is a subset of another | O | | getters | | Gets an `Iterator` of all values | O | | getters | | Gets an `Array` of all values | O | | getters | | Gets the number of elements | O | | mutations | | Adds a value to the set | O avg | | mutations | | Removes a value from the set | O avg | | mutations | | Toggles a value's presence | O avg | | operations | | Computes set difference | O | | operations | | Computes set intersection | O | | operations | | Computes set union | O | | mapping | | Transforms each element | O | | sequencing | | Transforms and flattens elements | O | | traversing | | Applies a function to each element | O | | folding | | Reduces the set to a single value | O | | filtering | | Keeps elements that satisfy a predicate | O | | partitioning | | Splits into two sets by a predicate | O | **Example** **Example** ## MutableHashSet A `MutableHashSet` is a **mutable**, **unordered** collection of **unique** values. Unlike `HashSet`, it allows direct modifications, operations like `add`, `remove`, and `clear` update the original set instead of returning a new one. This mutability can improve performance when you need to build or update a set repeatedly, especially within a local or isolated scope. ### Operations | Category | Operation | Description | Complexity | | ------------ | --------------------------------------------------------------- | ----------------------------------- | ---------- | | constructors | | Creates an empty MutableHashSet | O | | constructors | | Creates a set from an iterable | O | | constructors | | Creates a set from multiple values | O | | elements | | Checks if a value exists in the set | O avg | | elements | | Adds a value to the set | O avg | | elements | | Removes a value from the set | O avg | | getters | | Gets the number of elements | O | | mutations | | Removes all values from the set | O | **Example** ## Interoperability with JavaScript Both `HashSet` and `MutableHashSet` implement the `Iterable` interface, so you can use them with JavaScript features like: - the spread operator - `for...of` loops - `Array.from` You can also extract values as an array using `.toValues`. **Example** --- title: Option description: Represent optional values with Option, supporting presence or absence and seamless operations like mapping, combining, and pattern matching. sidebar: order: 10 --- import { Aside } from "@astrojs/starlight/components" The `Option` data type represents optional values. An `Option` can either be `Some`, containing a value of type `A`, or `None`, representing the absence of a value. You can use `Option` in scenarios like: - Using it for initial values - Returning values from functions that are not defined for all possible inputs - Managing optional fields in data structures - Handling optional function arguments ## Creating Options ### some Use the `Option.some` constructor to create an `Option` that holds a value of type `A`. **Example** ### none Use the `Option.none` constructor to create an `Option` representing the absence of a value. **Example** ### liftPredicate You can create an `Option` based on a predicate, for example, to check if a value is positive. **Example** Here's how you can achieve this using `Option.none` and `Option.some`: **Example** Alternatively, you can simplify the above logic with `Option.liftPredicate`: ## Modeling Optional Properties Consider a `User` model where the `"email"` property is optional and can hold a `string` value. We use the `Option` type to represent this optional property: Here are examples of how to create `User` instances with and without an email: **Example** ## Guards You can check whether an `Option` is a `Some` or a `None` using the `Option.isSome` and `Option.isNone` guards. **Example** ## Pattern Matching Use `Option.match` to handle both cases of an `Option` by specifying separate callbacks for `None` and `Some`. **Example** ## Working with Option ### map The `Option.map` function lets you transform the value inside an `Option` without manually unwrapping and re-wrapping it. If the `Option` holds a value , the transformation function is applied. If the `Option` is `None`, the function is ignored, and the `Option` remains unchanged. **Example** When dealing with `None`, the mapping function is not executed, and the `Option` remains `None`: **Example** ### flatMap The `Option.flatMap` function is similar to `Option.map`, but it is designed to handle cases where the transformation might return another `Option`. This allows us to chain computations that depend on whether or not a value is present in an `Option`. Consider a `User` model that includes a nested optional `Address`, which itself contains an optional `street` property: In this model, the `address` field is an `Option
`, and the `street` field within `Address` is an `Option`. We can use `Option.flatMap` to extract the `street` property from `address`: **Example** If `user.address` is `Some`, `Option.flatMap` applies the function ` => address.street` to retrieve the `street` value. If `user.address` is `None`, the function is not executed, and `street` remains `None`. This approach lets us handle nested optional values concisely, avoiding manual checks and making the code cleaner and easier to read. ### filter The `Option.filter` function allows you to filter an `Option` based on a given predicate. If the predicate is not met or if the `Option` is `None`, the result will be `None`. **Example** Here's how you can simplify some code using `Option.filter` for a more idiomatic approach: Original Code Refactored Idiomatic Code Using `Option.filter`, we can write the same logic more concisely: ## Getting the Value from an Option To retrieve the value stored inside an `Option`, you can use several helper functions provided by the `Option` module. Here's an overview of the available methods: ### getOrThrow This function extracts the value from a `Some`. If the `Option` is `None`, it throws an error. **Example** ### getOrNull / getOrUndefined These functions convert a `None` to either `null` or `undefined`, which is useful when working with non-`Option`-based code. **Example** ### getOrElse This function allows you to specify a default value to return when the `Option` is `None`. **Example** ## Fallback ### orElse When a computation returns `None`, you might want to try an alternative computation that yields an `Option`. The `Option.orElse` function is helpful in such cases. It lets you chain multiple computations, moving on to the next if the current one results in `None`. This approach is often used in retry logic, attempting computations until one succeeds or all possibilities are exhausted. **Example** ### firstSomeOf You can also use `Option.firstSomeOf` to get the first `Some` value from an iterable of `Option` values: **Example** ## Interop with Nullable Types When dealing with the `Option` data type, you may encounter code that uses `undefined` or `null` to represent optional values. The `Option` module provides several APIs to make interaction with these nullable types straightforward. ### fromNullable `Option.fromNullable` converts a nullable value into an `Option`. If the value is `null` or `undefined`, it returns `Option.none`. Otherwise, it wraps the value in `Option.some`. **Example** If you need to convert an `Option` back to a nullable value, there are two helper methods: - `Option.getOrNull`: Converts `None` to `null`. - `Option.getOrUndefined`: Converts `None` to `undefined`. ## Interop with Effect The `Option` type works as a subtype of the `Effect` type, allowing you to use it with functions from the `Effect` module. While these functions are built to handle `Effect` values, they can also manage `Option` values correctly. ### How Option Maps to Effect | Option Variant | Mapped to Effect | Description | | -------------- | --------------------------------------- | ---------------------------------- | | `None` | `Effect` | Represents the absence of a value | | `Some` | `Effect` | Represents the presence of a value | **Example** ## Combining Two or More Options ### zipWith The `Option.zipWith` function lets you combine two `Option` values using a provided function. It creates a new `Option` that holds the combined value of both original `Option` values. **Example** If either of the `Option` values is `None`, the result will be `None`: **Example** ### all To combine multiple `Option` values without transforming their contents, you can use `Option.all`. This function returns an `Option` with a structure matching the input: - If you pass a tuple, the result will be a tuple of the same length. - If you pass a struct, the result will be a struct with the same keys. - If you pass an `Iterable`, the result will be an array. **Example** If any of the `Option` values are `None`, the result will be `None`: **Example** ## gen Similar to , `Option.gen` provides a more readable, generator-based syntax for working with `Option` values, making code that involves `Option` easier to write and understand. This approach is similar to using `async/await` but tailored for `Option`. **Example** When any of the `Option` values in the sequence is `None`, the generator immediately returns the `None` value, skipping further operations: **Example** In this example, `Option.gen` halts execution as soon as it encounters the `None` value, effectively propagating the missing value without performing further operations. The use of `console.log` in these example is for demonstration purposes only. When using `Option.gen`, avoid including side effects in your generator functions, as `Option` should remain a pure data structure. ## Equivalence You can compare `Option` values using the `Option.getEquivalence` function. This function allows you to specify how to compare the contents of `Option` types by providing an for the type of value they may contain. **Example** Suppose you have optional numbers and want to check if they are equivalent. Here's how you can use `Option.getEquivalence`: ## Sorting You can sort a collection of `Option` values using the `Option.getOrder` function. This function helps specify a custom sorting rule for the type of value contained within the `Option`. **Example** Suppose you have a list of optional numbers and want to sort them in ascending order, with empty values `) treated as the lowest: **Example** Consider a more complex case where you have a list of objects containing optional dates, and you want to sort them in descending order, with `Option.none` values at the end: --- title: Redacted description: Securely handle sensitive data with the Redacted module, preventing accidental exposure in logs while supporting safe value access and comparison. sidebar: order: 11 --- The Redacted module provides functionality for handling sensitive information securely within your application. By using the `Redacted` data type, you can ensure that sensitive values are not accidentally exposed in logs or error messages. ## make The `Redacted.make` function creates a `Redacted` instance from a given value `A`, ensuring the content is securely hidden. **Example** Using `Redacted.make` helps prevent sensitive information, such as API keys, from being accidentally exposed in logs or error messages. ## value The `Redacted.value` function retrieves the original value from a `Redacted` instance. Use this function carefully, as it exposes the sensitive data, potentially making it visible in logs or accessible in unintended ways. **Example** ## unsafeWipe The `Redacted.unsafeWipe` function erases the underlying value of a `Redacted` instance, making it inaccessible. This helps ensure that sensitive data does not remain in memory longer than needed. **Example** ## getEquivalence The `Redacted.getEquivalence` function generates an for `Redacted` values using an Equivalence for the underlying values of type `A`. This allows you to compare `Redacted` values securely without revealing their content. **Example** --- title: Error Accumulation description: Learn to manage errors effectively in Effect workflows with tools for sequential execution, error accumulation, and result partitioning. sidebar: order: 8 --- import { Aside } from "@astrojs/starlight/components" Sequential combinators such as , and have a "fail fast" policy when it comes to error management. This means that they stop and return immediately when they encounter the first error. Here's an example using `Effect.zip`, which stops at the first failure and only shows the first error: **Example** The `Effect.forEach` function behaves similarly. It applies an effectful operation to each element in a collection, but will stop when it hits the first error: **Example** However, there are cases where you may want to collect all errors rather than fail fast. In these situations, you can use functions that accumulate both successes and errors. ## validate The `Effect.validate` function is similar to `Effect.zip`, but it continues combining effects even after encountering errors, accumulating both successes and failures. **Example** ## validateAll The `Effect.validateAll` function is similar to the `Effect.forEach` function. It transforms all elements of a collection using the provided effectful operation, but it collects all errors in the error channel, as well as the success values in the success channel. ## validateFirst The `Effect.validateFirst` function is similar to `Effect.validateAll` but it returns the first successful result, or all errors if none succeed. **Example** Notice that `Effect.validateFirst` returns a single `number` as the success type, rather than an array of results like `Effect.validateAll`. ## partition The `Effect.partition` function processes an iterable and applies an effectful function to each element. It returns a tuple, where the first part contains all the failures, and the second part contains all the successes. **Example** This operator is an unexceptional effect, meaning the error channel type is `never`. Failures are collected without stopping the effect, so the entire operation completes and returns both errors and successes. --- title: Error Channel Operations description: Explore operations on the error channel in Effect, including error mapping, filtering, inspecting, merging, and flipping channels. sidebar: order: 9 --- import { Aside } from "@astrojs/starlight/components" In Effect you can perform various operations on the error channel of effects. These operations allow you to transform, inspect, and handle errors in different ways. Let's explore some of these operations. ## Map Operations ### mapError The `Effect.mapError` function is used when you need to transform or modify an error produced by an effect, without affecting the success value. This can be helpful when you want to add extra information to the error or change its type. **Example** Here, the error type changes from `string` to `Error`. ### mapBoth The `Effect.mapBoth` function allows you to apply transformations to both channels: the error channel and the success channel of an effect. It takes two map functions as arguments: one for the error channel and the other for the success channel. **Example** ## Filtering the Success Channel The Effect library provides several operators to filter values on the success channel based on a given predicate. These operators offer different strategies for handling cases where the predicate fails: | API | Description | | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `filterOrFail` | This operator filters the values on the success channel based on a predicate. If the predicate fails for any value, the original effect fails with an error. | | `filterOrDie` / `filterOrDieMessage` | These operators also filter the values on the success channel based on a predicate. If the predicate fails for any value, the original effect terminates abruptly. The `filterOrDieMessage` variant allows you to provide a custom error message. | | `filterOrElse` | This operator filters the values on the success channel based on a predicate. If the predicate fails for any value, an alternative effect is executed instead. | **Example** It's important to note that depending on the specific filtering operator used, the effect can either fail, terminate abruptly, or execute an alternative effect when the predicate fails. Choose the appropriate operator based on your desired error handling strategy and program logic. The filtering APIs can also be combined with to improve type safety and code clarity. This ensures that only valid types pass through. **Example** In the example above, a guard is used within the `filterOrFail` API to ensure that the `user` is of type `User` rather than `User | null`. If you prefer, you can utilize a pre-made guard like for simplicity and consistency. ## Inspecting Errors Similar to for success values, Effect provides several operators for inspecting error values. These operators allow developers to observe failures or underlying issues without modifying the outcome. ### tapError Executes an effectful operation to inspect the failure of an effect without altering it. **Example** ### tapErrorTag This function allows you to inspect errors that match a specific tag, helping you handle different error types more precisely. **Example** ### tapErrorCause This function inspects the complete cause of an error, including failures and defects. **Example** ### tapDefect Specifically inspects non-recoverable failures or defects in an effect causes). **Example** ### tapBoth Inspects both success and failure outcomes of an effect, performing different actions based on the result. **Example** ## Exposing Errors in The Success Channel The `Effect.either` function transforms an `Effect` into an effect that encapsulates both potential failure and success within an data type: This means if you have an effect with the following type: and you call `Effect.either` on it, the type becomes: The resulting effect cannot fail because the potential failure is now represented within the `Either`'s `Left` type. The error type of the returned `Effect` is specified as `never`, confirming that the effect is structured to not fail. This function becomes especially useful when recovering from effects that may fail when using : **Example** ## Exposing the Cause in The Success Channel You can use the `Effect.cause` function to expose the cause of an effect, which is a more detailed representation of failures, including error messages and defects. **Example** ## Merging the Error Channel into the Success Channel The `Effect.merge` function allows you to combine the error channel with the success channel. This results in an effect that never fails; instead, both successes and errors are handled as values in the success channel. **Example** ## Flipping Error and Success Channels The `Effect.flip` function allows you to switch the error and success channels of an effect. This means that what was previously a success becomes the error, and vice versa. **Example** --- title: Expected Errors description: Learn how Effect manages expected errors with precise error tracking, short-circuiting, and powerful recovery techniques. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" Expected errors are tracked at the type level by the in the "Error channel": This means that the `Effect` type captures not only what the program returns on success but also what type of error it might produce. **Example** In this example, we define a program that might randomly fail with an `HttpError`. The type of `program` tells us that it can either return a `string` or fail with an `HttpError`: In this case, we use a class to represent the `HttpError` type, which allows us to define both the error type and a constructor. When using `Data.TaggedError`, a `_tag` field is automatically added to the class This discriminant field will be useful when we discuss APIs like , which help in handling specific error types. ## Error Tracking In Effect, if a program can fail with multiple types of errors, they are automatically tracked as a union of those error types. This allows you to know exactly what errors can occur during execution, making error handling more precise and predictable. The example below illustrates how errors are automatically tracked for you. **Example** Effect automatically keeps track of the possible errors that can occur during the execution of the program as a union: indicating that it can potentially fail with either a `HttpError` or a `ValidationError`. ## Short-Circuiting When working with APIs like , , , and , it's important to understand how they handle errors. These APIs are designed to **short-circuit the execution** upon encountering the **first error**. What does this mean for you as a developer? Well, let's say you have a chain of operations or a collection of effects to be executed in sequence. If any error occurs during the execution of one of these effects, the remaining computations will be skipped, and the error will be propagated to the final result. In simpler terms, the short-circuiting behavior ensures that if something goes wrong at any step of your program, it won't waste time executing unnecessary computations. Instead, it will immediately stop and return the error to let you know that something went wrong. **Example** This code snippet demonstrates the short-circuiting behavior when an error occurs. Each operation depends on the successful execution of the previous one. If any error occurs, the execution is short-circuited, and the error is propagated. In this specific example, `task3` is never executed because an error occurs in `task2`. ## Catching All Errors ### either The `Effect.either` function transforms an `Effect` into an effect that encapsulates both potential failure and success within an data type: This means if you have an effect with the following type: and you call `Effect.either` on it, the type becomes: The resulting effect cannot fail because the potential failure is now represented within the `Either`'s `Left` type. The error type of the returned `Effect` is specified as `never`, confirming that the effect is structured to not fail. By yielding an `Either`, we gain the ability to "pattern match" on this type to handle both failure and success cases within the generator function. **Example** As you can see since all errors are handled, the error type of the resulting effect `recovered` is `never`: We can make the code less verbose by using the `Either.match` function, which directly accepts the two callback functions for handling errors and successful values: **Example** ### option Transforms an effect to encapsulate both failure and success using the data type. The `Effect.option` function wraps the success or failure of an effect within the `Option` type, making both cases explicit. If the original effect succeeds, its value is wrapped in `Option.some`. If it fails, the failure is mapped to `Option.none`. The resulting effect cannot fail directly, as the error type is set to `never`. However, fatal errors like defects are not encapsulated. **Example** ### catchAll Handles all errors in an effect by providing a fallback effect. The `Effect.catchAll` function catches any errors that may occur during the execution of an effect and allows you to handle them by specifying a fallback effect. This ensures that the program continues without failing by recovering from errors using the provided fallback logic. **Example** We can observe that the type in the error channel of our program has changed to `never`: indicating that all errors have been handled. ### catchAllCause Handles both recoverable and unrecoverable errors by providing a recovery effect. The `Effect.catchAllCause` function allows you to handle all errors, including unrecoverable defects, by providing a recovery effect. The recovery logic is based on the `Cause` of the error, which provides detailed information about the failure. **Example** ## Catching Some Errors ### either The function, which was previously shown as a way to catch all errors, can also be used to catch specific errors. By yielding an `Either`, we gain the ability to "pattern match" on this type to handle both failure and success cases within the generator function. **Example** We can observe that the type in the error channel of our program has changed to only show `ValidationError`: indicating that `HttpError` has been handled. If we also want to handle `ValidationError`, we can easily add another case to our code: We can observe that the type in the error channel has changed to `never`: indicating that all errors have been handled. ### catchSome Catches and recovers from specific types of errors, allowing you to attempt recovery only for certain errors. `Effect.catchSome` lets you selectively catch and handle errors of certain types by providing a recovery effect for specific errors. If the error matches a condition, recovery is attempted; if not, it doesn't affect the program. This function doesn't alter the error type, meaning the error type remains the same as in the original effect. **Example** In the code above, `Effect.catchSome` takes a function that examines the error and decides whether to attempt recovery or not. If the error matches a specific condition, recovery can be attempted by returning `Option.some`. If no recovery is possible, you can simply return `Option.none`. It's important to note that while `Effect.catchSome` lets you catch specific errors, it doesn't alter the error type itself. Therefore, the resulting effect will still have the same error type as the original effect: ### catchIf Recovers from specific errors based on a predicate. `Effect.catchIf` works similarly to , but it allows you to recover from errors by providing a predicate function. If the predicate matches the error, the recovery effect is applied. This function doesn't alter the error type, so the resulting effect still carries the original error type unless a user-defined type guard is used to narrow the type. **Example** It's important to note that for TypeScript versions < 5.5, while `Effect.catchIf` lets you catch specific errors, it **doesn't alter the error type** itself. Therefore, the resulting effect will still have the same error type as the original effect: In TypeScript versions >= 5.5, improved type narrowing causes the resulting error type to be inferred as `ValidationError`. #### Workaround For TypeScript versions < 5.5 If you provide a instead of a predicate, the resulting error type will be pruned, returning an `Effect`: ### catchTag Catches and handles specific errors by their `_tag` field, which is used as a discriminator. `Effect.catchTag` is useful when your errors are tagged with a `_tag` field that identifies the error type. You can use this function to handle specific error types by matching the `_tag` value. This allows for precise error handling, ensuring that only specific errors are caught and handled. The error type must have a `_tag` field to use `Effect.catchTag`. This field is used to identify and match errors. **Example** In the example above, the `Effect.catchTag` function allows us to handle `HttpError` specifically. If a `HttpError` occurs during the execution of the program, the provided error handler function will be invoked, and the program will proceed with the recovery logic specified within the handler. We can observe that the type in the error channel of our program has changed to only show `ValidationError`: indicating that `HttpError` has been handled. If we also wanted to handle `ValidationError`, we can simply add another `catchTag`: **Example** We can observe that the type in the error channel of our program has changed to `never`: indicating that all errors have been handled. ### catchTags Handles multiple errors in a single block of code using their `_tag` field. `Effect.catchTags` is a convenient way to handle multiple error types at once. Instead of using multiple times, you can pass an object where each key is an error type's `_tag`, and the value is the handler for that specific error. This allows you to catch and recover from multiple error types in a single call. **Example** This function takes an object where each property represents a specific error `_tag` , and the corresponding value is the error handler function to be executed when that particular error occurs. ## Effect.fn The `Effect.fn` function allows you to create traced functions that return an effect. It provides two key features: - **Stack traces with location details** if an error occurs. - **Automatic span creation** for when a span name is provided. If a span name is passed as the first argument, the function's execution is tracked using that name. If no name is provided, stack tracing still works, but spans are not created. A function can be defined using either: - A generator function, allowing the use of `yield*` for effect composition. - A regular function that returns an `Effect`. **Example** ### Exporting Spans for Tracing `Effect.fn` automatically creates . The spans capture information about the function execution, including metadata and error details. **Example** ### Using Effect.fn as a pipe Function `Effect.fn` also acts as a pipe function, allowing you to create a pipeline after the function definition using the effect returned by the generator function as the starting value of the pipeline. **Example** --- title: Fallback description: Learn techniques to handle failures and implement fallback mechanisms in Effect programs. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" This page explains various techniques for handling failures and creating fallback mechanisms in the Effect library. ## orElse `Effect.orElse` allows you to attempt to run an effect, and if it fails, you can provide a fallback effect to run instead. This is useful for handling failures gracefully by defining an alternative effect to execute if the first one encounters an error. **Example** ## orElseFail `Effect.orElseFail` allows you to replace the failure from one effect with a custom failure value. If the effect fails, you can provide a new failure to be returned instead of the original one. This function only applies to failed effects. If the effect succeeds, it will remain unaffected. **Example** ## orElseSucceed `Effect.orElseSucceed` allows you to replace the failure of an effect with a success value. If the effect fails, it will instead succeed with the provided value, ensuring the effect always completes successfully. This is useful when you want to guarantee a successful result regardless of whether the original effect failed. The function ensures that any failure is effectively "swallowed" and replaced by a successful value, which can be helpful for providing default values in case of failure. This function only applies to failed effects. If the effect already succeeds, it will remain unchanged. **Example** ## firstSuccessOf `Effect.firstSuccessOf` allows you to try multiple effects in sequence, and as soon as one of them succeeds, it returns that result. If all effects fail, it returns the error of the last effect in the list. This is useful when you have several potential alternatives and want to use the first one that works. This function is sequential, meaning that the `Effect` values in the iterable will be executed in sequence, and the first one that succeeds will determine the outcome of the resulting `Effect` value. **Example** In this example, we try to retrieve a configuration from different nodes. If the primary node fails, we fall back to other nodes until we find a successful configuration. --- title: Matching description: Learn to handle success and failure cases in Effect programs with tools for pattern matching, value ignoring, side effects, and precise failure analysis. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" In the Effect module, similar to other modules like and , we have a `Effect.match` function that allows us to handle different cases simultaneously. Additionally, Effect provides various functions to manage both success and failure scenarios in effectful programs. ## match `Effect.match` lets you define custom handlers for both success and failure scenarios. You provide separate functions to handle each case, allowing you to process the result if the effect succeeds, or handle the error if the effect fails. This is useful for structuring your code to respond differently to success or failure without triggering side effects. **Example** ## ignore `Effect.ignore` allows you to run an effect without caring about its result, whether it succeeds or fails. This is useful when you only care about the side effects of the effect and do not need to handle or process its outcome. **Example** ## matchEffect The `Effect.matchEffect` function is similar to , but it enables you to perform side effects in the handlers for both success and failure outcomes. This is useful when you need to execute additional actions, like logging or notifying users, based on whether an effect succeeds or fails. **Example** ## matchCause The `Effect.matchCause` function allows you to handle failures with access to the full of the failure within a fiber. This is useful for differentiating between different types of errors, such as regular failures, defects, or interruptions. You can provide specific handling logic for each failure type based on the cause. **Example** ## matchCauseEffect The `Effect.matchCauseEffect` function works similarly to , but it also allows you to perform additional side effects based on the failure cause. This function provides access to the complete of the failure, making it possible to differentiate between various failure types, and allows you to respond accordingly while performing side effects . **Example** --- title: Parallel and Sequential Errors description: Handle concurrent and sequential errors in Effect programs, capturing multiple failures and ensuring robust error management in concurrent and sequential workflows. sidebar: order: 10 --- import { Aside } from "@astrojs/starlight/components" When working with Effect, if an error occurs, the default behavior is to fail with the first error encountered. **Example** Here, the program fails with the first error it encounters, `"Oh uh!"`. ## Parallel Errors In some cases, you might encounter multiple errors, especially during concurrent computations. When tasks are run concurrently, multiple errors can happen at the same time. **Example** In this example, both the `fail` and `die` effects are executed concurrently. Since both fail, the program will report multiple errors in the output. ### parallelErrors Effect provides a function called `Effect.parallelErrors` that captures all failure errors from concurrent operations in the error channel. **Example** In this example, `Effect.parallelErrors` combines the errors from `fail1` and `fail2` into a single error. ## Sequential Errors When working with resource-safety operators like `Effect.ensuring`, you may encounter multiple sequential errors. This happens because regardless of whether the original effect has any errors or not, the finalizer is uninterruptible and will always run. **Example** In this example, both `fail` and the finalizer `die` result in sequential errors, and both are captured. --- title: Retrying description: Enhance resilience with Effect's retrying strategies, enabling robust handling of transient failures with customizable retry policies and fallback mechanisms. sidebar: order: 5 --- import { Aside } from "@astrojs/starlight/components" In software development, it's common to encounter situations where an operation may fail temporarily due to various factors such as network issues, resource unavailability, or external dependencies. In such cases, it's often desirable to retry the operation automatically, allowing it to succeed eventually. Retrying is a powerful mechanism to handle transient failures and ensure the successful execution of critical operations. In Effect retrying is made simple and flexible with built-in functions and scheduling strategies. In this guide, we will explore the concept of retrying in Effect and learn how to use the `retry` and `retryOrElse` functions to handle failure scenarios. We'll see how to define retry policies using schedules, which dictate when and how many times the operation should be retried. Whether you're working on network requests, database interactions, or any other potentially error-prone operations, mastering the retrying capabilities of effect can significantly enhance the resilience and reliability of your applications. ## retry The `Effect.retry` function takes an effect and a policy, and will automatically retry the effect if it fails, following the rules of the policy. If the effect ultimately succeeds, the result will be returned. If the maximum retries are exhausted and the effect still fails, the failure is propagated. This can be useful when dealing with intermittent failures, such as network issues or temporary resource unavailability. By defining a retry policy, you can control the number of retries, the delay between them, and when to stop retrying. **Example** ### Retrying n Times Immediately You can also retry a failing effect a set number of times with a simpler policy that retries immediately: **Example** ### Retrying Based on a Condition You can customize how retries are managed by specifying conditions. Use the `until` or `while` options to control when retries stop. **Example** ## retryOrElse The `Effect.retryOrElse` function attempts to retry a failing effect multiple times according to a defined policy. If the retries are exhausted and the effect still fails, it runs a fallback effect instead. This function is useful when you want to handle failures gracefully by specifying an alternative action after repeated failures. **Example** --- title: Sandboxing description: Master error handling in Effect with sandboxing, enabling detailed inspection and recovery from failures, defects, and interruptions. sidebar: order: 7 --- Errors are an inevitable part of programming, and they can arise from various sources like failures, defects, fiber interruptions, or combinations of these. This guide explains how to use the `Effect.sandbox` function to isolate and understand the causes of errors in your Effect-based code. ## sandbox / unsandbox The `Effect.sandbox` function allows you to encapsulate all the potential causes of an error in an effect. It exposes the full cause of an effect, whether it's due to a failure, defect, fiber interruption, or a combination of these factors. In simple terms, it takes an effect `Effect` and transforms it into an effect `Effect, R>` where the error channel now contains a detailed cause of the error. **Syntax** By using the `Effect.sandbox` function, you gain access to the underlying causes of exceptional effects. These causes are represented as a type of `Cause` and are available in the error channel of the `Effect` data type. Once you have exposed the causes, you can utilize standard error-handling operators like and to handle errors more effectively. These operators allow you to respond to specific error conditions. If needed, we can undo the sandboxing operation with `Effect.unsandbox`. **Example** --- title: Timing Out description: Set time limits on operations with Effect, ensuring tasks complete within specified durations and customizing behavior for timeouts. sidebar: order: 6 --- import { Aside } from "@astrojs/starlight/components" In programming, it's common to deal with tasks that may take some time to complete. Often, we want to enforce a limit on how long we're willing to wait for these tasks. The `Effect.timeout` function helps by placing a time constraint on an operation, ensuring it doesn't run indefinitely. ## Basic Usage ### timeout The `Effect.timeout` function employs a parameter to establish a time limit on an operation. If the operation exceeds this limit, a `TimeoutException` is triggered, indicating a timeout has occurred. **Example** Here, the task completes within the timeout duration, so the result is returned successfully. If the operation exceeds the specified duration, a `TimeoutException` is raised: ### timeoutOption If you want to handle timeouts more gracefully, consider using `Effect.timeoutOption`. This function treats timeouts as regular results, wrapping the outcome in an . **Example** In this example, the first task completes successfully, while the second times out. The result of the timed-out task is represented as `None` in the `Option` type. ## Handling Timeouts When an operation does not finish within the specified duration, the behavior of the `Effect.timeout` depends on whether the operation is "uninterruptible". 1. **Interruptible Operation**: If the operation can be interrupted, it is terminated immediately once the timeout threshold is reached, resulting in a `TimeoutException`. 2. **Uninterruptible Operation**: If the operation is uninterruptible, it continues until completion before the `TimeoutException` is assessed. ## Disconnection on Timeout The `Effect.disconnect` function provides a way to handle timeouts in uninterruptible effects more flexibly. It allows an uninterruptible effect to complete in the background, while the main control flow proceeds as if a timeout had occurred. Here's the distinction: **Without** `Effect.disconnect`: - An uninterruptible effect will ignore the timeout and continue executing until it completes, after which the timeout error is assessed. - This can lead to delays in recognizing a timeout condition because the system must wait for the effect to complete. **With** `Effect.disconnect`: - The uninterruptible effect is allowed to continue in the background, independent of the main control flow. - The main control flow recognizes the timeout immediately and proceeds with the timeout error or alternative logic, without having to wait for the effect to complete. - This method is particularly useful when the operations of the effect do not need to block the continuation of the program, despite being marked as uninterruptible. **Example** Consider a scenario where a long-running data processing task is initiated, and you want to ensure the system remains responsive, even if the data processing takes too long: In this example, the system detects the timeout after one second, but the long-running task continues and completes in the background, without blocking the program's flow. ## Customizing Timeout Behavior In addition to the basic `Effect.timeout` function, there are variations available that allow you to customize the behavior when a timeout occurs. ### timeoutFail The `Effect.timeoutFail` function allows you to produce a specific error when a timeout happens. **Example** ### timeoutFailCause `Effect.timeoutFailCause` lets you define a specific defect to throw when a timeout occurs. This is helpful for treating timeouts as exceptional cases in your code. **Example** ### timeoutTo `Effect.timeoutTo` provides more flexibility compared to `Effect.timeout`, allowing you to define different outcomes for both successful and timed-out operations. This can be useful when you want to customize the result based on whether the operation completes in time or not. **Example** ) --- title: Two Types of Errors description: Learn how Effect differentiates between expected and unexpected errors to enhance error tracking and recovery. sidebar: order: 0 --- Just like any other program, Effect programs may fail for expected or unexpected reasons. The difference between a non-Effect program and an Effect program is in the detail provided to you when your program fails. Effect attempts to preserve as much information as possible about what caused your program to fail to produce a detailed, comprehensive, and human readable failure message. In an Effect program, there are two possible ways for a program to fail: - **Expected Errors**: These are errors that developers anticipate and expect as part of normal program execution. - **Unexpected Errors**: These are errors that occur unexpectedly and are not part of the intended program flow. ## Expected Errors These errors, also referred to as _failures_, _typed errors_ or _recoverable errors_, are errors that developers anticipate as part of the normal program execution. They serve a similar purpose to checked exceptions and play a role in defining the program's domain and control flow. Expected errors **are tracked** at the type level by the `Effect` data type in the "Error" channel: it is evident from the type that the program can fail with an error of type `HttpError`. ## Unexpected Errors Unexpected errors, also referred to as _defects_, _untyped errors_, or _unrecoverable errors_, are errors that developers do not anticipate occurring during normal program execution. Unlike expected errors, which are considered part of a program's domain and control flow, unexpected errors resemble unchecked exceptions and lie outside the expected behavior of the program. Since these errors are not expected, Effect **does not track** them at the type level. However, the Effect runtime does keep track of these errors and provides several methods to aid in recovering from unexpected errors. --- title: Unexpected Errors description: Understand how Effect handles unexpected errors with tools to manage defects, terminate execution, and selectively recover from critical failures. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" There are situations where you may encounter unexpected errors, and you need to decide how to handle them. Effect provides functions to help you deal with such scenarios, allowing you to take appropriate actions when errors occur during the execution of your effects. ## Creating Unrecoverable Errors In the same way it is possible to leverage combinators such as to create values of type `Effect` the Effect library provides tools to create defects. Creating defects is a common necessity when dealing with errors from which it is not possible to recover from a business logic perspective, such as attempting to establish a connection that is refused after multiple retries. In those cases terminating the execution of the effect and moving into reporting, through an output such as stdout or some external monitoring service, might be the best solution. The following functions and combinators allow for termination of the effect and are often used to convert values of type `Effect` into values of type `Effect` allowing the programmer an escape hatch from having to handle and recover from errors for which there is no sensible way to recover. ### die Creates an effect that terminates a fiber with a specified error. Use `Effect.die` when encountering unexpected conditions in your code that should not be handled as regular errors but instead represent unrecoverable defects. The `Effect.die` function is used to signal a defect, which represents a critical and unexpected error in the code. When invoked, it produces an effect that does not handle the error and instead terminates the fiber. The error channel of the resulting effect is of type `never`, indicating that it cannot recover from this failure. **Example** ### dieMessage Creates an effect that terminates a fiber with a `RuntimeException` containing the specified message. Use `Effect.dieMessage` when you want to terminate a fiber due to an unrecoverable defect and include a clear explanation in the message. The `Effect.dieMessage` function is used to signal a defect, representing a critical and unexpected error in the code. When invoked, it produces an effect that terminates the fiber with a `RuntimeException` carrying the given message. The resulting effect has an error channel of type `never`, indicating it does not handle or recover from the error. **Example** ## Converting Failures to Defects ### orDie Converts an effect's failure into a fiber termination, removing the error from the effect's type. Use `Effect.orDie` when failures should be treated as unrecoverable defects and no error handling is required. The `Effect.orDie` function is used when you encounter errors that you do not want to handle or recover from. It removes the error type from the effect and ensures that any failure will terminate the fiber. This is useful for propagating failures as defects, signaling that they should not be handled within the effect. **Example** ### orDieWith Converts an effect's failure into a fiber termination with a custom error. Use `Effect.orDieWith` when failures should terminate the fiber as defects, and you want to customize the error for clarity or debugging purposes. The `Effect.orDieWith` function behaves like , but it allows you to provide a mapping function to transform the error before terminating the fiber. This is useful for cases where you want to include a more detailed or user-friendly error when the failure is propagated as a defect. **Example** ## Catching All Defects There is no sensible way to recover from defects. The functions we're about to discuss should be used only at the boundary between Effect and an external system, to transmit information on a defect for diagnostic or explanatory purposes. ### exit The `Effect.exit` function transforms an `Effect` into an effect that encapsulates both potential failure and success within an data type: This means if you have an effect with the following type: and you call `Effect.exit` on it, the type becomes: The resulting effect cannot fail because the potential failure is now represented within the `Exit`'s `Failure` type. The error type of the returned effect is specified as `never`, confirming that the effect is structured to not fail. By yielding an `Exit`, we gain the ability to "pattern match" on this type to handle both failure and success cases within the generator function. **Example** ### catchAllDefect Recovers from all defects using a provided recovery function. `Effect.catchAllDefect` allows you to handle defects, which are unexpected errors that usually cause the program to terminate. This function lets you recover from these defects by providing a function that handles the error. However, it does not handle expected errors ) or execution interruptions ). **Example** ## Catching Some Defects ### catchSomeDefect Recovers from specific defects using a provided partial function. `Effect.catchSomeDefect` allows you to handle specific defects, which are unexpected errors that can cause the program to stop. It uses a partial function to catch only certain defects and ignores others. However, it does not handle expected errors ) or execution interruptions ). The function provided to `Effect.catchSomeDefect` acts as a filter and a handler for defects: - It receives the defect as an input. - If the defect matches a specific condition , the function returns an `Option.some` containing the recovery logic. - If the defect does not match, the function returns `Option.none`, allowing the defect to propagate. **Example** --- title: Yieldable Errors description: Explore yieldable errors in Effect programming for seamless error handling in generator functions using custom and tagged error constructors. sidebar: order: 11 --- Yieldable Errors are special types of errors that can be yielded directly within a generator function using . These errors allow you to handle them intuitively, without needing to explicitly invoke . This simplifies how you manage custom errors in your code. ## Data.Error The `Data.Error` constructor provides a way to define a base class for yieldable errors. **Example** ## Data.TaggedError The `Data.TaggedError` constructor lets you define custom yieldable errors with unique tags. Each error has a `_tag` property, allowing you to easily distinguish between different error types. This makes it convenient to handle specific tagged errors using functions like or . **Example** --- title: Getting Started description: A guide to the Effect documentation and where to start. sidebar: order: 0 --- Welcome to the Effect guides. If you are new to Effect, start with the section, which introduces the core concepts in order. These guides explain each topic in more detail, and you can also use them as a reference while you build. ## How to Use These Docs The documentation is structured in a sequential manner, starting from the basics and progressing to more advanced topics. This allows you to follow along step-by-step as you build your Effect application. However, you have the flexibility to read the documentation in any order or jump directly to the pages that are relevant to your specific use case. You can switch between Effect versions using the version selector at the top of the page. To facilitate navigation within a page, you will find a table of contents on the right side of the screen. This allows you to easily jump between different sections of the page. ## Guide Sections The guides are grouped by area: - The guides cover typed errors, fallbacks, and retries. - The guides cover fibers, bounded parallelism, and racing. - The guides explain how to acquire and release resources safely. - The guides cover parsing, validating, and transforming data. - The guides cover logging, metrics, and tracing. - The guides show how to build data pipelines with backpressure. For a complete reference of every module, see the . ## Coding with LLMs The following article describe how to use Effect with LLMs: When using LLMs it is also very important to optimize the feedback loop to be as tight as possible, that can include writing custom linting rules to fit your style preferences and patterns, an example of a repository optimized for agentic coding can be found at: A key part of optimizing the feedback loop is to use the Effect LSP plugin, we advise using the latest "tsgo" implementation of it that can be found at: ## Join our Community If you have questions about anything related to Effect, you're always welcome to ask our community on . --- title: Building Pipelines description: Learn to create modular, readable pipelines for composing and sequencing operations in Effect, enabling clear and efficient data transformations. sidebar: order: 10 --- import { Aside } from "@astrojs/starlight/components" Effect pipelines allow for the composition and sequencing of operations on values, enabling the transformation and manipulation of data in a concise and modular manner. ## Why Pipelines are Good for Structuring Your Application Pipelines are an excellent way to structure your application and handle data transformations in a concise and modular manner. They offer several benefits: 1. **Readability**: Pipelines allow you to compose functions in a readable and sequential manner. You can clearly see the flow of data and the operations applied to it, making it easier to understand and maintain the code. 2. **Code Organization**: With pipelines, you can break down complex operations into smaller, manageable functions. Each function performs a specific task, making your code more modular and easier to reason about. 3. **Reusability**: Pipelines promote the reuse of functions. By breaking down operations into smaller functions, you can reuse them in different pipelines or contexts, improving code reuse and reducing duplication. 4. **Type Safety**: By leveraging the type system, pipelines help catch errors at compile-time. Functions in a pipeline have well-defined input and output types, ensuring that the data flows correctly through the pipeline and minimizing runtime errors. ## Functions vs Methods The use of functions in the Effect ecosystem libraries is important for achieving **tree shakeability** and ensuring **extensibility**. Functions enable efficient bundling by eliminating unused code, and they provide a flexible and modular approach to extending the libraries' functionality. ### Tree Shakeability Tree shakeability refers to the ability of a build system to eliminate unused code during the bundling process. Functions are tree shakeable, while methods are not. When functions are used in the Effect ecosystem, only the functions that are actually imported and used in your application will be included in the final bundled code. Unused functions are automatically removed, resulting in a smaller bundle size and improved performance. On the other hand, methods are attached to objects or prototypes, and they cannot be easily tree shaken. Even if you only use a subset of methods, all methods associated with an object or prototype will be included in the bundle, leading to unnecessary code bloat. ### Extensibility Another important advantage of using functions in the Effect ecosystem is the ease of extensibility. With methods, extending the functionality of an existing API often requires modifying the prototype of the object, which can be complex and error-prone. In contrast, with functions, extending the functionality is much simpler. You can define your own "extension methods" as plain old functions without the need to modify the prototypes of objects. This promotes cleaner and more modular code, and it also allows for better compatibility with other libraries and modules. ## pipe The `pipe` function is a utility that allows us to compose functions in a readable and sequential manner. It takes the output of one function and passes it as the input to the next function in the pipeline. This enables us to build complex transformations by chaining multiple functions together. **Syntax** In this syntax, `input` is the initial value, and `func1`, `func2`, ..., `funcN` are the functions to be applied in sequence. The result of each function becomes the input for the next function, and the final result is returned. Here's an illustration of how `pipe` works: It's important to note that functions passed to `pipe` must have a **single argument** because they are only called with a single argument. Let's see an example to better understand how `pipe` works: **Example** In the above example, we start with an input value of `5`. The `increment` function adds `1` to the initial value, resulting in `6`. Then, the `double` function doubles the value, giving us `12`. Finally, the `subtractTen` function subtracts `10` from `12`, resulting in the final output of `2`. The result is equivalent to `subtractTen))`, but using `pipe` makes the code more readable because the operations are sequenced from left to right, rather than nesting them inside out. ## map Transforms the value inside an effect by applying a function to it. **Syntax** `Effect.map` takes a function and applies it to the value contained within an effect, creating a new effect with the transformed value. **Example** Here's a practical example where we apply a service charge to a transaction amount: ## as Replaces the value inside an effect with a constant value. `Effect.as` allows you to ignore the original value inside an effect and replace it with a new constant value. **Example** ## flatMap Chains effects to produce new `Effect` instances, useful for combining operations that depend on previous results. **Syntax** In the code above, `transformation` is the function that takes a value and returns an `Effect`, and `myEffect` is the initial `Effect` being transformed. Use `Effect.flatMap` when you need to chain multiple effects, ensuring that each step produces a new `Effect` while flattening any nested effects that may occur. It is similar to `flatMap` used with arrays but works specifically with `Effect` instances, allowing you to avoid deeply nested effect structures. **Example** ### Ensure All Effects Are Considered Make sure that all effects within `Effect.flatMap` contribute to the final computation. If you ignore an effect, it can lead to unexpected behavior: In this case, the `Effect.sync` call is ignored and does not affect the result of `applyDiscount`. To handle effects correctly, make sure to explicitly chain them using functions like `Effect.map`, `Effect.flatMap`, `Effect.andThen`, or `Effect.tap`. ## andThen Chains two actions, where the second action can depend on the result of the first. **Syntax** Use `andThen` when you need to run multiple actions in sequence, with the second action depending on the result of the first. This is useful for combining effects or handling computations that must happen in order. The second action can be: 1. A value 2. A function returning a value 3. A `Promise` 4. A function returning a `Promise` 5. An `Effect` 6. A function returning an `Effect` **Example** Let's look at an example comparing `Effect.andThen` with `Effect.map` and `Effect.flatMap`: ### Option and Either with andThen Both and are commonly used for handling optional or missing values or simple error cases. These types integrate well with `Effect.andThen`. When used with `Effect.andThen`, the operations are categorized as scenarios 5 and 6 because both `Option` and `Either` are treated as effects in this context. **Example** You might expect the type of `program` to be `Effect, UnknownException, never>`, but it is actually `Effect`. This is because `Option` is treated as an effect of type `Effect`, and as a result, the possible errors are combined into a union type. **Example** Although one might expect the type of `program` to be `Effect, UnknownException, never>`, it is actually `Effect`. This is because `Either` is treated as an effect of type `Effect`, meaning the errors are combined into a union type. ## tap Runs a side effect with the result of an effect without changing the original value. Use `Effect.tap` when you want to perform a side effect, like logging or tracking, without modifying the main value. This is useful when you need to observe or record an action but want the original value to be passed to the next step. `Effect.tap` works similarly to `Effect.flatMap`, but it ignores the result of the function passed to it. The value from the previous effect remains available for the next part of the chain. Note that if the side effect fails, the entire chain will fail too. **Example** In this example, `Effect.tap` is used to log the transaction amount before applying the discount, without modifying the value itself. The original value remains available for the next operation . Using `Effect.tap` allows us to execute side effects during the computation without altering the result. This can be useful for logging, performing additional actions, or observing the intermediate values without interfering with the main computation flow. ## all Combines multiple effects into one, returning results based on the input structure. Use `Effect.all` when you need to run multiple effects and combine their results into a single output. It supports tuples, iterables, structs, and records, making it flexible for different input types. For instance, if the input is a tuple: the effects are executed in order, and the result is a new effect containing the results as a tuple. The results in the tuple match the order of the effects passed to `Effect.all`. By default, `Effect.all` runs effects sequentially and produces a tuple or object with the results. If any effect fails, it stops execution and propagates the error. See for more information on how to use `Effect.all`. **Example** ## Build your first pipeline Let's now combine the `pipe` function, `Effect.all`, and `Effect.andThen` to create a pipeline that performs a sequence of transformations. **Example** This pipeline demonstrates how you can structure your code by combining different effects into a clear, readable flow. ## The pipe method Effect provides a `pipe` method that works similarly to the `pipe` method found in . This method allows you to chain multiple operations together, making your code more concise and readable. **Syntax** This is equivalent to using the `pipe` **function** like this: The `pipe` method is available on all effects and many other data types, eliminating the need to import the `pipe` function and saving you some keystrokes. **Example** Let's rewrite an , this time using the `pipe` method. ## Cheatsheet Let's summarize the transformation functions we have seen so far: | API | Input | Output | | --------- | ----------------------------------------- | --------------------------- | | `map` | `Effect`, `A => B` | `Effect` | | `flatMap` | `Effect`, `A => Effect` | `Effect` | | `andThen` | `Effect`, \* | `Effect` | | `tap` | `Effect`, `A => Effect` | `Effect` | | `all` | `` | `Effect<, E, R>` | --- title: Creating Effects description: Learn to create and manage effects for structured handling of success, failure, and side effects in synchronous and asynchronous workflows. sidebar: order: 7 --- import { Aside } from "@astrojs/starlight/components" Effect provides different ways to create effects, which are units of computation that encapsulate side effects. In this guide, we will cover some of the common methods that you can use to create effects. ## Why Not Throw Errors? In traditional programming, when an error occurs, it is often handled by throwing an exception: However, throwing errors can be problematic. The type signatures of functions do not indicate that they can throw exceptions, making it difficult to reason about potential errors. To address this issue, Effect introduces dedicated constructors for creating effects that represent both success and failure: `Effect.succeed` and `Effect.fail`. These constructors allow you to explicitly handle success and failure cases while **leveraging the type system to track errors**. ### succeed Creates an `Effect` that always succeeds with a given value. Use this function when you need an effect that completes successfully with a specific value without any errors or external dependencies. **Example** The type of `success` is `Effect`, which means: - It produces a value of type `number`. - It does not generate any errors . - It requires no additional data or dependencies . ### fail Creates an `Effect` that represents an error that can be recovered from. Use this function to explicitly signal an error in an `Effect`. The error will keep propagating unless it is handled. You can handle the error with functions like or . **Example** The type of `failure` is `Effect`, which means: - It never produces a value . - It fails with an error, specifically an `Error`. - It requires no additional data or dependencies . Although you can use `Error` objects with `Effect.fail`, you can also pass strings, numbers, or more complex objects depending on your error management strategy. Using "tagged" errors can help identify error types and works well with standard Effect functions, like . **Example** ## Error Tracking With `Effect.succeed` and `Effect.fail`, you can explicitly handle success and failure cases and the type system will ensure that errors are tracked and accounted for. **Example** Here's how you can rewrite the function using Effect, making error handling explicit. In this example, the `divide` function indicates in its return type `Effect` that the operation can either succeed with a `number` or fail with an `Error`. This clear type signature helps ensure that errors are handled properly and that anyone calling the function is aware of the possible outcomes. **Example** Let's imagine another scenario where we use `Effect.succeed` and `Effect.fail` to model a simple user retrieval operation where the user data is hardcoded, which could be useful in testing scenarios or when mocking data: In this example, `exampleUserEffect`, which has the type `Effect`, will either produce a `User` object or an `Error`, depending on whether the user exists in the mocked database. For a deeper dive into managing errors in your applications, refer to the . ## Modeling Synchronous Effects In JavaScript, you can delay the execution of synchronous computations using "thunks". Thunks are useful for delaying the computation of a value until it is needed. To model synchronous side effects, Effect provides the `Effect.sync` and `Effect.try` constructors, which accept a thunk. ### sync Creates an `Effect` that represents a synchronous side-effectful computation. Use `Effect.sync` when you are sure the operation will not fail. The provided function must not throw errors; if it does, the error will be treated as a . This defect is not a standard error but indicates a flaw in the logic that was expected to be error-free. You can think of it similar to an unexpected crash in the program, which can be further managed or logged using tools like . This feature ensures that even unexpected failures in your application are not lost and can be handled appropriately. **Example** In the example below, `Effect.sync` is used to defer the side-effect of writing to the console. The side effect encapsulated within `program` won't occur until the effect is explicitly run section for more details). This allows you to define side effects at one point in your code and control when they are activated, improving manageability and predictability of side effects in larger applications. ### try Creates an `Effect` that represents a synchronous computation that might fail. In situations where you need to perform synchronous operations that might fail, such as parsing JSON, you can use the `Effect.try` constructor. This constructor is designed to handle operations that could throw exceptions by capturing those exceptions and transforming them into manageable errors. **Example** Suppose you have a function that attempts to parse a JSON string. This operation can fail and throw an error if the input string is not properly formatted as JSON: In this example: - `parse` is a function that creates an effect encapsulating the JSON parsing operation. - If `JSON.parse` throws an error due to invalid input, `Effect.try` catches this error and the effect represented by `program` will fail with an `UnknownException`. This ensures that errors are not silently ignored but are instead handled within the structured flow of effects. #### Customizing Error Handling You might want to transform the caught exception into a more specific error or perform additional operations when catching an error. `Effect.try` supports an overload that allows you to specify how caught exceptions should be transformed: **Example** You can think of this as a similar pattern to the traditional try-catch block in JavaScript: ## Modeling Asynchronous Effects In traditional programming, we often use `Promise`s to handle asynchronous computations. However, dealing with errors in promises can be problematic. By default, `Promise` only provides the type `Value` for the resolved value, which means errors are not reflected in the type system. This limits the expressiveness and makes it challenging to handle and track errors effectively. To overcome these limitations, Effect introduces dedicated constructors for creating effects that represent both success and failure in an asynchronous context: `Effect.promise` and `Effect.tryPromise`. These constructors allow you to explicitly handle success and failure cases while **leveraging the type system to track errors**. ### promise Creates an `Effect` that represents an asynchronous computation guaranteed to succeed. Use `Effect.promise` when you are sure the operation will not reject. The provided function returns a `Promise` that should never reject; if it does, the error will be treated as a . This defect is not a standard error but indicates a flaw in the logic that was expected to be error-free. You can think of it similar to an unexpected crash in the program, which can be further managed or logged using tools like . This feature ensures that even unexpected failures in your application are not lost and can be handled appropriately. **Example** The `program` value has the type `Effect` and can be interpreted as an effect that: - succeeds with a value of type `string` - does not produce any expected error - does not require any context ### tryPromise Creates an `Effect` that represents an asynchronous computation that might fail. Unlike `Effect.promise`, this constructor is suitable when the underlying `Promise` might reject. It provides a way to catch errors and handle them appropriately. By default if an error occurs, it will be caught and propagated to the error channel as an `UnknownException`. **Example** The `program` value has the type `Effect` and can be interpreted as an effect that: - succeeds with a value of type `Response` - might produce an error - does not require any context #### Customizing Error Handling If you want more control over what gets propagated to the error channel, you can use an overload of `Effect.tryPromise` that takes a remapping function: **Example** ## From a Callback Creates an `Effect` from a callback-based asynchronous function. Sometimes you have to work with APIs that don't support `async/await` or `Promise` and instead use the callback style. To handle callback-based APIs, Effect provides the `Effect.async` constructor. **Example** Let's wrap the `readFile` function from Node.js's `fs` module into an Effect-based API : In the above example, we manually annotate the types when calling `Effect.async`: because TypeScript cannot infer the type parameters for a callback based on the return value inside the callback body. Annotating the types ensures that the values provided to `resume` match the expected types. The `resume` function inside `Effect.async` should be called exactly once. Calling it more than once will result in the extra calls being ignored. **Example** ### Advanced Usage For more advanced use cases, the callback passed to Effect.async may return an Effect that will be executed if the fiber running this effect is interrupted. This can be used to perform cleanup when the operation is cancelled. **Example** In this example: - The `writeFileWithCleanup` function writes data to a file. - If the fiber running this effect is interrupted, the cleanup effect is executed. - This ensures that resources like open file handles are cleaned up properly when the operation is canceled. If the operation you're wrapping supports interruption, the `resume` function can receive an `AbortSignal` to handle interruption requests directly. **Example** ## Suspended Effects `Effect.suspend` is used to delay the creation of an effect. It allows you to defer the evaluation of an effect until it is actually needed. The `Effect.suspend` function takes a thunk that represents the effect, and it wraps it in a suspended effect. **Syntax** Let's explore some common scenarios where `Effect.suspend` proves useful. ### Lazy Evaluation When you want to defer the evaluation of an effect until it is required. This can be useful for optimizing the execution of effects, especially when they are not always needed or when their computation is expensive. Also, when effects with side effects or scoped captures are created, use `Effect.suspend` to re-execute on each invocation. **Example** In this example, `bad` is the result of calling `Effect.succeed` a single time, which increments the scoped variable but . `Effect.runSync` does not result in any new computation, because `Effect.succeed` has already been called. On the other hand, each time `Effect.runSync` is called, the thunk passed to `Effect.suspend` will be executed, outputting the scoped variable's most recent value. ### Handling Circular Dependencies `Effect.suspend` is helpful in managing circular dependencies between effects, where one effect depends on another, and vice versa. For example it's fairly common for `Effect.suspend` to be used in recursive functions to escape an eager call. **Example** The `blowsUp` function creates a recursive Fibonacci sequence without deferring execution. Each call to `blowsUp` triggers further immediate recursive calls, rapidly increasing the JavaScript call stack size. Conversely, `allGood` avoids stack overflow by using `Effect.suspend` to defer the recursive calls. This mechanism doesn't immediately execute the recursive effects but schedules them to be run later, thus keeping the call stack shallow and preventing a crash. ### Unifying Return Type In situations where TypeScript struggles to unify the returned effect type, `Effect.suspend` can be employed to resolve this issue. **Example** ## Cheatsheet The table provides a summary of the available constructors, along with their input and output types, allowing you to choose the appropriate function based on your needs. | API | Given | Result | | ----------------------- | ---------------------------------- | ----------------------------- | | `succeed` | `A` | `Effect` | | `fail` | `E` | `Effect` | | `sync` | ` => A` | `Effect` | | `try` | ` => A` | `Effect` | | `try` | ` => A`, `unknown => E` | `Effect` | | `promise` | ` => Promise` | `Effect` | | `tryPromise` | ` => Promise` | `Effect` | | `tryPromise` | ` => Promise`, `unknown => E` | `Effect` | | `async` | ` => void` | `Effect` | | `suspend` | ` => Effect` | `Effect` | For the complete list of constructors, visit the . --- title: Devtools description: Enhance your Effect development experience with the Effect Language Service and official VS Code/Cursor extension for advanced diagnostics, refactors, and intelligent code completion. sidebar: order: 3 --- import { Aside, Steps, Tabs, TabItem } from "@astrojs/starlight/components" Effect provides powerful development tools to enhance your coding experience and help you write safer, more maintainable code. These tools integrate directly into your editor, providing real-time feedback, intelligent refactors, and helpful diagnostics. ## Effect LSP The Effect LSP extends your editor with Effect-specific features. It analyzes your Effect code and provides intelligent assistance through diagnostics, quick info, completions, and automated refactors. It works in editors that supports the standard TypeScript LSP, such as Code, Cursor, Zed, NVim, etc. ### Installation To install the Effect Language Service in your project: 1. Install the package as a development dependency: For monorepos, we suggest to install the language service at the root level. For single-package projects, install it in the package directory. 2. Add the plugin to your `tsconfig.json`: 3. Ensure your editor uses the workspace TypeScript version: This step is critical for the language service to function properly. The plugin must run on the TypeScript version installed in your project, not the one bundled with your editor. 4. You're ready to play! Writing the following code in a file.ts inside your project, should result in an error diagnostic appearing, saying that Effect's must be yielded or assigned to a variable: ### Features The Effect Language Service provides a comprehensive set of features to enhance your development workflow: #### Intelligent Quick Info Hover over Effect values to see extended type information and detailed insights: - **Effect Types**: See comprehensive type information for Effect values - **Generator Parameters**: When hovering over `yield*` in `Effect.gen`, view detailed information about the yielded value - **Layer Composition**: Visualize layer dependencies with interactive graphs showing how layers compose together - **Service Dependencies**: Understand service requirements and their relationships at a glance #### Real-time Diagnostics Catch common mistakes and potential issues as you write code: - **Floating Effects**: Detect Effect values that aren't assigned or yielded, preventing silent bugs - **Layer Issues**: Catch layer requirement leaks and scope violations before runtime - **Unnecessary Code**: Identify redundant `Effect.gen` or `pipe` calls - **Error Handling**: Detect misuse of catch functions on Effects that cannot fail - **Version Conflicts**: Detect when multiple Effect versions are present in your project #### Smart Completions Speed up your coding with context-aware suggestions: - **Generator Boilerplate**: Quickly scaffold `Effect.gen` functions - **Scaffolds**: For `Effect.Service`, `Data.TaggedError` and friends. - **Self Parameters**: Auto-complete for `Self` parameters in service declarations #### Powerful Refactors Transform your code with intelligent automated refactors: - **Async to Effect**: Convert async functions to Effect using `gen` or `fn` syntax - **Error Generation**: Generate tagged errors from promise-based code - **Service Accessors**: Automatically implement service accessor functions - **Pipe Conversion**: Transform function calls to pipe syntax - **Pipe Styles**: Toggle between different pipe style formats - **Layer Magic**: Automatically compose layers with correct dependencies ### Configuration The Effect LSP provides also lots of configuration options such as changing severity or disabling diagnostic messages. To see the full list of options and features, please visit the . ### Build-Time Diagnostics While LSPs only activate during editing sessions, you may want to catch diagnostics during your build process. Usually that's done through linting rules, but since almost all of the Effect diagnostics relies on types, that would mean enabling type-aware linting, which means performing type checking again on the project files. To solve this, the Effect Language Service allows you to patch your local TypeScript installation, so diagnostics are emitted while performing type checking. To enable it run the following command to modify your local TypeScript installation: To make this automatic for all developers, add it to your `package.json`: This ensures the language service runs during compilation with the standard `tsc` command. ## VS Code / Cursor Extension The editor extension provides utilities in helping you debug your Effect applications. At the moment only Code and Code forks like Cursor are supported. ### Installation The extension can be installed by searching directly in your editor extension page or from the or the . ### Debugger Features With the Effect Extension, you'll find couple of new sections inside the Debug section of your editor that, once you pause execution, will . - **Context**: Allows you to inspect the context of the currently paused Effect Fiber. - **Span Stack**: Shows you the stack of telemetry spans that lead you into the execution of the currently paused Effect. - **Fibers**: List all the Effect Fibers running in your application, allows you to inspect informations such as interrupt-ability and allows to request interruption of them. - **Breakpoints**: Allows to enable "pause on defect"; letting your debugger pause when a Effect Fiber fails with a defect. ### Built-in Tracer and Metrics The built-in tracer and metrics view allows to quickly see Effect Spans and Metrics of your app without spinning up an entire telemetry service. To enable it, you need to install the following dependency in your project: You can then import and use the DevTools module in your Effect app: If you are using `@effect/opentelemetry` in your project, then it is important that you provide the DevTools layer before your tracing layers, so the tracer is patched correctly. Now start both your editor and your app. Inside the Effect panel, in the clients section you'll see a newly connected client. In the bottom of your editor, near your terminal, a new tab "Effect Tracer" will appear as well, showing visually your spans as they happen in real time. --- title: Importing Effect description: Get started with Effect by installing the package and importing essential modules and functions for building type-safe, modular applications. sidebar: order: 5 --- import { Aside, Tabs, TabItem } from "@astrojs/starlight/components" If you're just getting started, you might feel overwhelmed by the variety of modules and functions that Effect offers. However, rest assured that you don't need to worry about all of them right away. This page will provide a simple introduction on how to import modules and functions, and explain that installing the `effect` package is generally all you need to begin. ## Installing Effect If you haven't already installed the `effect` package, you can do so by running the following command in your terminal: By installing this package, you get access to the core functionality of Effect. For detailed installation instructions for platforms like Deno or Bun, refer to the guide, which provides step-by-step guidance. ## Importing Modules and Functions Once you have installed the `effect` package, you can start using its modules and functions in your projects. Importing modules and functions is straightforward and follows the standard JavaScript/TypeScript import syntax. To import a module or a function from the `effect` package, simply use the `import` statement at the top of your file. Here's how you can import the `Effect` module: Now, you have access to the Effect module, which is the heart of the Effect library. It provides various functions to create, compose, and manipulate effectful computations. ## Namespace imports In addition to importing the `Effect` module with a named import, as shown previously: You can also import it using a namespace import like this: Both forms of import allow you to access the functionalities provided by the `Effect` module. However an important consideration is **tree shaking**, which refers to a process that eliminates unused code during the bundling of your application. Named imports may generate tree shaking issues when a bundler doesn't support deep scope analysis. Here are some bundlers that support deep scope analysis and thus don't have issues with named imports: - Rollup - Webpack 5+ ## Functions vs Methods In the Effect ecosystem, libraries often expose functions rather than methods. This design choice is important for two key reasons: tree shakeability and extendibility. ### Tree Shakeability Tree shakeability refers to the ability of a build system to eliminate unused code during the bundling process. Functions are tree shakeable, while methods are not. When functions are used in the Effect ecosystem, only the functions that are actually imported and used in your application will be included in the final bundled code. Unused functions are automatically removed, resulting in a smaller bundle size and improved performance. On the other hand, methods are attached to objects or prototypes, and they cannot be easily tree shaken. Even if you only use a subset of methods, all methods associated with an object or prototype will be included in the bundle, leading to unnecessary code bloat. ### Extendibility Another important advantage of using functions in the Effect ecosystem is the ease of extendibility. With methods, extending the functionality of an existing API often requires modifying the prototype of the object, which can be complex and error-prone. In contrast, with functions, extending the functionality is much simpler. You can define your own "extension methods" as plain old functions without the need to modify the prototypes of objects. This promotes cleaner and more modular code, and it also allows for better compatibility with other libraries and modules. ## Commonly Used Functions As you start your adventure with Effect, you don't need to dive into every function in the `effect` package right away. Instead, focus on some commonly used functions that will provide a solid foundation for your journey into the world of Effect. In the upcoming guides, we will explore some of these essential functions, specifically those for creating and running `Effect`s and building pipelines. But before we dive into those, let's start from the very heart of Effect: understanding the `Effect` type. This will lay the groundwork for your understanding of how Effect brings composability, type safety, and error handling into your applications. So, let's take the first step and explore the fundamental concepts of the . --- title: Installation description: Set up a new Effect project across different platforms like Node.js, Deno, Bun, and Vite + React with step-by-step installation guides. sidebar: order: 2 --- import { Steps, Tabs, TabItem } from "@astrojs/starlight/components" Requirements: - TypeScript 5.4 or newer. - Node.js, Deno, and Bun are supported. ## Manual Installation ### Node.js Follow these steps to create a new Effect project for : 1. Create a project directory and navigate into it: 2. Initialize a TypeScript project: This creates a `package.json` file with an initial setup for your TypeScript project. 3. Initialize TypeScript: When running this command, it will generate a `tsconfig.json` file that contains configuration options for TypeScript. One of the most important options to consider is the `strict` flag. Make sure to open the `tsconfig.json` file and verify that the value of the `strict` option is set to `true`. 4. Install the necessary package as dependency: This package will provide the foundational functionality for your Effect project. Let's write and run a simple program to ensure that everything is set up correctly. In your terminal, execute the following commands: Open the `index.ts` file and add the following code: Run the `index.ts` file. Here we are using to run the `index.ts` file in the terminal: You should see the message `"Hello, World!"` printed. This confirms that the program is working correctly. ### Deno Follow these steps to create a new Effect project for : 1. Create a project directory and navigate into it: 2. Initialize Deno: 3. Install the necessary package as dependency: This package will provide the foundational functionality for your Effect project. Let's write and run a simple program to ensure that everything is set up correctly. Open the `main.ts` file and replace the content with the following code: Run the `main.ts` file: You should see the message `"Hello, World!"` printed. This confirms that the program is working correctly. ### Bun Follow these steps to create a new Effect project for : 1. Create a project directory and navigate into it: 2. Initialize Bun: When running this command, it will generate a `tsconfig.json` file that contains configuration options for TypeScript. One of the most important options to consider is the `strict` flag. Make sure to open the `tsconfig.json` file and verify that the value of the `strict` option is set to `true`. 3. Install the necessary package as dependency: This package will provide the foundational functionality for your Effect project. Let's write and run a simple program to ensure that everything is set up correctly. Open the `index.ts` file and replace the content with the following code: Run the `index.ts` file: You should see the message `"Hello, World!"` printed. This confirms that the program is working correctly. ### Vite + React Follow these steps to create a new Effect project for + : 1. Scaffold your Vite project, open your terminal and run the following command: This command will create a new Vite project with React and TypeScript template. 2. Navigate into the newly created project directory and install the required packages: Once the packages are installed, open the `tsconfig.json` file and ensure that the value of the `strict` option is set to true. 3. Install the necessary package as dependency: This package will provide the foundational functionality for your Effect project. Now, let's write and run a simple program to ensure that everything is set up correctly. Open the `src/App.tsx` file and replace its content with the following code: After making these changes, start the development server by running the following command: Then, press **o** to open the application in your browser. When you click the button, you should see the counter increment. This confirms that the program is working correctly. --- title: Running Effects description: Learn how to execute effects in Effect with various functions for synchronous and asynchronous execution, including handling results and managing error outcomes. sidebar: order: 8 --- import { Aside } from "@astrojs/starlight/components" To execute an effect, you can use one of the many `run` functions provided by the `Effect` module. ## runSync Executes an effect synchronously, running it immediately and returning the result. **Example** Use `Effect.runSync` to run an effect that does not fail and does not include any asynchronous operations. If the effect fails or involves asynchronous work, it will throw an error, and execution will stop where the failure or async operation occurs. **Example** ## runSyncExit Runs an effect synchronously and returns the result as an type, which represents the outcome of the effect. Use `Effect.runSyncExit` to find out whether an effect succeeded or failed, including any defects, without dealing with asynchronous operations. The `Exit` type represents the result of the effect: - If the effect succeeds, the result is wrapped in a `Success`. - If it fails, the failure information is provided as a `Failure` containing a type. **Example** If the effect contains asynchronous operations, `Effect.runSyncExit` will return an `Failure` with a `Die` cause, indicating that the effect cannot be resolved synchronously. **Example** ## runPromise Executes an effect and returns the result as a `Promise`. Use `Effect.runPromise` when you need to execute an effect and work with the result using `Promise` syntax, typically for compatibility with other promise-based code. **Example** If the effect succeeds, the promise will resolve with the result. If the effect fails, the promise will reject with an error. **Example** ## runPromiseExit Runs an effect and returns a `Promise` that resolves to an , which represents the outcome of the effect. Use `Effect.runPromiseExit` when you need to determine if an effect succeeded or failed, including any defects, and you want to work with a `Promise`. The `Exit` type represents the result of the effect: - If the effect succeeds, the result is wrapped in a `Success`. - If it fails, the failure information is provided as a `Failure` containing a type. **Example** ## runFork The foundational function for running effects, returning a "fiber" that can be observed or interrupted. `Effect.runFork` is used to run an effect in the background by creating a fiber. It is the base function for all other run functions. It starts a fiber that can be observed or interrupted. **Example** In this example, the `program` continuously logs "running..." with each repetition spaced 200 milliseconds apart. You can learn more about repetitions and scheduling in our guide. To stop the execution of the program, we use `Fiber.interrupt` on the fiber returned by `Effect.runFork`. This allows you to control the execution flow and terminate it when necessary. For a deeper understanding of how fibers work and how to handle interruptions, check out our guides on and . ## Synchronous vs. Asynchronous Effects In the Effect library, there is no built-in way to determine in advance whether an effect will execute synchronously or asynchronously. While this idea was considered in earlier versions of Effect, it was ultimately not implemented for a few important reasons: 1. **Complexity:** Introducing this feature to track sync/async behavior in the type system would make Effect more complex to use and limit its composability. 2. **Safety Concerns:** We experimented with different approaches to track asynchronous Effects, but they all resulted in a worse developer experience without significantly improving safety. Even with fully synchronous types, we needed to support a `fromCallback` combinator to work with APIs using Continuation-Passing Style . However, at the type level, it's impossible to guarantee that such a function is always called immediately and not deferred. ### Best Practices for Running Effects In most cases, effects are run at the outermost parts of your application. Typically, an application built around Effect will involve a single call to the main effect. Here’s how you should approach effect execution: - Use `runPromise` or `runFork`: For most cases, asynchronous execution should be the default. These methods provide the best way to handle Effect-based workflows. - Use `runSync` only when necessary: Synchronous execution should be considered an edge case, used only in scenarios where asynchronous execution is not feasible. For example, when you are sure the effect is purely synchronous and need immediate results. ## Cheatsheet The table provides a summary of the available `run*` functions, along with their input and output types, allowing you to choose the appropriate function based on your needs. | API | Given | Result | | ---------------- | -------------- | --------------------- | | `runSync` | `Effect` | `A` | | `runSyncExit` | `Effect` | `Exit` | | `runPromise` | `Effect` | `Promise` | | `runPromiseExit` | `Effect` | `Promise>` | | `runFork` | `Effect` | `RuntimeFiber` | You can find the complete list of `run*` functions . --- title: The Effect Type description: Understand the Effect type in the Effect ecosystem, which models immutable, lazy workflows with type-safe success, error, and requirement handling for effectful computations. sidebar: order: 6 --- import { Aside } from "@astrojs/starlight/components" The `Effect` type is a description of a workflow or operation that is **lazily** executed. This means that when you create an `Effect`, it doesn't run immediately, but instead defines a program that can succeed, fail, or require some additional context to complete. Here is the general form of an `Effect`: This type indicates that an effect: - Succeeds and returns a value of type `Success` - Fails with an error of type `Error` - May need certain contextual dependencies of type `Requirements` to execute Conceptually, you can think of `Effect` as an effectful version of the following function type: However, effects are not actually functions. They can model synchronous, asynchronous, concurrent, and resourceful computations. **Immutability**. `Effect` values are immutable, and every function in the Effect library produces a new `Effect` value. **Modeling Interactions**. These values do not perform any actions themselves, they simply model or describe effectful interactions. **Execution**. An `Effect` can be executed by the , which interprets it into actual interactions with the external world. Ideally, this execution happens at a single entry point in your application, such as the main function where effectful operations are initiated. ## Type Parameters The `Effect` type has three type parameters with the following meanings: | Parameter | Description | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Success** | Represents the type of value that an effect can succeed with when executed. If this type parameter is `void`, it means the effect produces no useful information, while if it is `never`, it means the effect runs forever . | | **Error** | Represents the expected errors that can occur when executing an effect. If this type parameter is `never`, it means the effect cannot fail, because there are no values of type `never`. | | **Requirements** | Represents the contextual data required by the effect to be executed. This data is stored in a collection named `Context`. If this type parameter is `never`, it means the effect has no requirements and the `Context` collection is empty. | ## Extracting Inferred Types By using the utility types `Effect.Success`, `Effect.Error`, and `Effect.Context`, you can extract the corresponding types from an effect. **Example** --- title: Using Generators description: Learn how to use generators in Effect for writing effectful code, enhancing control flow, handling errors, and simplifying asynchronous operations with a syntax similar to async/await. sidebar: order: 9 --- import { Aside, Tabs, TabItem, Badge } from "@astrojs/starlight/components" Effect offers a convenient syntax, similar to `async`/`await`, to write effectful code using . ## Understanding Effect.gen The `Effect.gen` utility simplifies the task of writing effectful code by utilizing JavaScript's generator functions. This method helps your code appear and behave more like traditional synchronous code, which enhances both readability and error management. **Example** Let's explore a practical program that performs a series of data transformations commonly found in application logic: Key steps to follow when using `Effect.gen`: - Wrap your logic in `Effect.gen` - Use `yield*` to handle effects - Return the final result If any of the effects that you handle inside of the generator with `yield*` fail, then the generator will stop and exit with that failure. ## Comparing Effect.gen with async/await If you are familiar with `async`/`await`, you may notice that the flow of writing code is similar. Let's compare the two approaches: It's important to note that although the code appears similar, the two programs are not identical. The purpose of comparing them side by side is just to highlight the resemblance in how they are written. ## Embracing Control Flow One significant advantage of using `Effect.gen` in conjunction with generators is its capability to employ standard control flow constructs within the generator function. These constructs include `if`/`else`, `for`, `while`, and other branching and looping mechanisms, enhancing your ability to express complex control flow logic in your code. **Example** ## How to Raise Errors The `Effect.gen` API lets you integrate error handling directly into your workflow by yielding failed effects. You can introduce errors with `Effect.fail`, as shown in the example below. **Example** ## The Role of Short-Circuiting When working with `Effect.gen`, it is important to understand how it handles errors. This API will stop execution at the **first error** it encounters and return that error. How does this affect your code? If you have several operations in sequence, once any one of them fails, the remaining operations will not run, and the error will be returned. In simpler terms, if something fails at any point, the program will stop right there and deliver the error to you. If you don't want to stop on an error, you can use the `Effect.either` method to encapsulate the error in the data type: see the . **Example** Even though execution never reaches code after a failure, TypeScript may still assume that the code below the error is reachable unless you explicitly return after the failure. For example, consider the following scenario where you want to narrow the type of a variable: **Example** In this example, TypeScript still considers `user` possibly `undefined` because there is no explicit return after the failure. To fix this, explicitly return right after calling `Effect.fail`: **Example** ## Passing `this` In some cases, you might need to pass a reference to the current object into the body of your generator function. You can achieve this by utilizing an overload that accepts the reference as the first argument: **Example** ## Adapter You may still come across some code snippets that use an adapter, typically indicated by `_` or `$` symbols. In earlier versions of TypeScript, the generator "adapter" function was necessary to ensure correct type inference within generators. This adapter was used to facilitate the interaction between TypeScript's type system and generator functions. **Example** With advances in TypeScript , the adapter is no longer necessary for type inference. While it remains in the codebase for backward compatibility, it is anticipated to be removed in the upcoming major release of Effect. --- title: Why Effect? description: Discover how Effect transforms TypeScript programming by using the type system to track errors, context, and success, offering practical solutions for building reliable, maintainable applications. sidebar: order: 1 --- Programming is challenging. When we build libraries and apps, we look to many tools to handle the complexity and make our day-to-day more manageable. Effect presents a new way of thinking about programming in TypeScript. Effect is an ecosystem of tools that help you build better applications and libraries. As a result, you will also learn more about the TypeScript language and how to use the type system to make your programs more reliable and easier to maintain. In "typical" TypeScript, without Effect, we write code that assumes that a function is either successful or throws an exception. For example: Based on the types, we have no idea that this function can throw an exception. We can only find out by reading the code. This may not seem like much of a problem when you only have one function in your codebase, but when you have hundreds or thousands, it really starts to add up. It's easy to forget that a function can throw an exception, and it's easy to forget to handle that exception. Often, we will do the "easiest" thing and just wrap the function in a `try/catch` block. This is a good first step to prevent your program from crashing, but it doesn't make it any easier to manage or understand our complex application/library. We can do better. One of the most important tools we have in TypeScript is the compiler. It is the first line of defense against bugs, domain errors, and general complexity. ## The Effect Pattern While Effect is a vast ecosystem of many different tools, if it had to be reduced down to just one idea, it would be the following: Effect's major unique insight is that we can use the type system to track **errors** and **context**, not only **success** values as shown in the divide example above. Here's the same divide function from above, but with the Effect pattern: With this approach, the function no longer throws exceptions. Instead, errors are handled as values, which can be passed along like success values. The type signature also makes it clear: - What success value the function returns . - What error can occur . - What additional context or dependencies are required . Additionally, tracking context allows you to provide additional information to your functions without having to pass in everything as an argument. For example, you can swap out implementations of live external services with mocks during your tests without changing any core business logic. ## Don't Re-Invent the Wheel Application code in TypeScript often solves the same problems over and over again. Interacting with external services, filesystems, databases, etc. are common problems for all application developers. Effect provides a rich ecosystem of libraries that provide standardized solutions to many of these problems. You can use these libraries to build your application, or you can use them to build your own libraries. Managing challenges like error handling, debugging, tracing, async/promises, retries, streaming, concurrency, caching, resource management, and a lot more are made manageable with Effect. You don't have to re-invent the solutions to these problems, or install tons of dependencies. Effect, under one umbrella, solves many of the problems that you would usually install many different dependencies with different APIs to solve. ## Solving Practical Problems Effect is heavily inspired by great work done in other languages, like Scala and Haskell. However, it's important to understand that Effect's goal is to be a practical toolkit, and it goes to great lengths to solve real, everyday problems that developers face when building applications and libraries in TypeScript. ## Enjoy Building and Learning Learning Effect is a lot of fun. Many developers in the Effect ecosystem are using Effect to solve real problems in their day-to-day work, and also experiment with cutting edge ideas for pushing TypeScript to be the most useful language it can be. You don't have to use all aspects of Effect at once, and can start with the pieces of the ecosystem that make the most sense for the problems you are solving. Effect is a toolkit, and you can pick and choose the pieces that make the most sense for your use case. However, as more and more of your codebase is using Effect, you will probably find yourself wanting to utilize more of the ecosystem! Effect's concepts may be new to you, and might not completely make sense at first. This is totally normal. Take your time with reading the docs and try to understand the core concepts - this will really pay off later on as you get into the more advanced tooling in the Effect ecosystem. The Effect community is always happy to help you learn and grow. Feel free to hop into our or discuss on ! We are open to feedback and contributions, and are always looking for ways to improve Effect. --- title: Micro for Effect Users description: Learn about the Micro module, a lightweight alternative to Effect for reducing bundle size while maintaining compatibility and functionality for TypeScript applications. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" The Micro module is designed as a lighter alternative to the standard Effect module, tailored for situations where it is beneficial to reduce the bundle size. This module is standalone and does not include more complex functionalities such as , , , and . This feature set makes Micro especially suitable for libraries that wish to utilize Effect functionalities while keeping the bundle size to a minimum, particularly for those aiming to provide `Promise`-based APIs. Micro also supports use cases where a client application uses Micro, and a server employs the full suite of Effect features, maintaining both compatibility and logical consistency across various application components. Integrating Micro adds a minimal footprint to your bundle, starting at **5kb gzipped**, which may increase depending on the features you use. ## Importing Micro Micro is a part of the Effect library and can be imported just like any other module: You can also import it using a namespace import like this: Both forms of import allow you to access the functionalities provided by the `Micro` module. However an important consideration is **tree shaking**, which refers to a process that eliminates unused code during the bundling of your application. Named imports may generate tree shaking issues when a bundler doesn't support deep scope analysis. Here are some bundlers that support deep scope analysis and thus don't have issues with named imports: - Rollup - Webpack 5+ ## Main Types ### Micro The `Micro` type uses three type parameters: which mirror those of the `Effect` type. ### MicroExit The `MicroExit` type is a streamlined version of the type, designed to capture the outcome of a `Micro` computation. It can either be successful, containing a value of type `A`, or it can fail, containing an error of type `E` wrapped in a `MicroCause`. ### MicroCause The `MicroCause` type is a streamlined version of the type. Similar to how `Cause` is a union of types, `MicroCause` comes in three forms: | Variant | Description | | ----------- | ------------------------------------------------------------------------------------------- | | `Die` | Indicates an unforeseen defect that wasn't planned for in the system's logic. | | `Fail` | Covers anticipated errors that are recognized and typically handled within the application. | | `Interrupt` | Signifies an operation that has been purposefully stopped. | ### MicroSchedule The `MicroSchedule` type is a streamlined version of the type. Represents a function that can be used to calculate the delay between repeats. The function takes the current attempt number and the elapsed time since the first attempt, and returns the delay for the next attempt. If the function returns `None`, the repetition will stop. ## How to Use This Guide Below, you'll find a series of comparisons between the functionalities of `Effect` and `Micro`. Each table lists a functionality of `Effect` alongside its counterpart in `Micro`. The icons used have the following meanings: - ⚠️: The feature is available in `Micro`, but with some differences from `Effect`. - ❌: The feature is not available in `Effect`. ## Creating Effects | Effect | Micro | ⚠️ | | ---------------------- | -------------------- | ------------------------------------ | | `Effect.try` | `Micro.try` | requires a `try` block | | `Effect.tryPromise` | `Micro.tryPromise` | requires a `try` block | | `Effect.sleep` | `Micro.sleep` | only handles milliseconds | | `Effect.failCause` | `Micro.failWith` | uses `MicroCause` instead of `Cause` | | `Effect.failCauseSync` | `Micro.failWithSync` | uses `MicroCause` instead of `Cause` | | ❌ | `Micro.make` | | | ❌ | `Micro.fromOption` | | | ❌ | `Micro.fromEither` | | ## Running Effects | Effect | Micro | ⚠️ | | ----------------------- | ---------------------- | -------------------------------------------------- | | `Effect.runSyncExit` | `Micro.runSyncExit` | returns a `MicroExit` instead of an `Exit` | | `Effect.runPromiseExit` | `Micro.runPromiseExit` | returns a `MicroExit` instead of an `Exit` | | `Effect.runFork` | `Micro.runFork` | returns a `MicroFiber` instead of a `RuntimeFiber` | ### runSyncExit The `Micro.runSyncExit` function is used to execute an Effect synchronously, which means it runs immediately and returns the result as a . **Example** ### runPromiseExit The `Micro.runPromiseExit` function is used to execute an Effect and obtain the result as a `Promise` that resolves to a . **Example** ### runFork The `Micro.runFork` function executes the effect and return a `MicroFiber` that can be awaited, joined, or aborted. You can listen for the result by adding an observer using the `addObserver` method. **Example** ## Building Pipelines | Effect | Micro | ⚠️ | | ------------------ | ----------------- | ------------------------------------------------------- | | `Effect.andThen` | `Micro.andThen` | doesn't handle `Promise` or ` => Promise` as argument | | `Effect.tap` | `Micro.tap` | doesn't handle ` => Promise` as argument | | `Effect.all` | `Micro.all` | no `batching` and `mode` options | | `Effect.forEach` | `Micro.forEach` | no `batching` option | | `Effect.filter` | `Micro.filter` | no `batching` option | | `Effect.filterMap` | `Micro.filterMap` | the filter is effectful | ## Expected Errors | Effect | Micro | ⚠️ | | ------------- | ------------ | ------------------------------------------ | | `Effect.exit` | `Micro.exit` | returns a `MicroExit` instead of an `Exit` | ## Unexpected Errors | Effect | Micro | | | ------ | -------------------- | --- | | ❌ | `Micro.catchCauseIf` | | ## Timing Out | Effect | Micro | | | ------ | --------------------- | --- | | ❌ | `Micro.timeoutOrElse` | | ## Requirements Management To access a service while using `Micro.gen`, you need to wrap the service tag using the `Micro.service` function: **Example** ## Scope | Effect | Micro | ⚠️ | | ------------ | ----------------- | ------------------------------------------- | | `Scope` | `MicroScope` | returns a `MicroScope` instead of a `Scope` | | `Scope.make` | `Micro.scopeMake` | returns a `MicroScope` instead of a `Scope` | ## Retrying | Effect | Micro | ⚠️ | | -------------- | ------------- | ------------------- | | `Effect.retry` | `Micro.retry` | different `options` | ## Repetition | Effect | Micro | ⚠️ | | --------------- | ------------------ | ------------------- | | `Effect.repeat` | `Micro.repeat` | different `options` | | ❌ | `Micro.repeatExit` | | ## Timing out | Effect | Micro | | | ------ | --------------------- | --- | | ❌ | `Micro.timeoutOrElse` | | ## Sandboxing | Effect | Micro | ⚠️ | | ---------------- | --------------- | ----------------------------------------------- | | `Effect.sandbox` | `Micro.sandbox` | returns a `MicroCause` instead of `Cause` | ## Error Channel Operations | Effect | Micro | ⚠️ | | ---------------------- | ------------------------ | ------------------------------------- | | ❌ | `Micro.filterOrFailWith` | | | `Effect.tapErrorCause` | `Micro.tapErrorCause` | `MicroCause` instead of `Cause` | | ❌ | `Micro.tapCauseIf` | | | `Effect.tapDefect` | `Micro.tapDefect` | `unknown` instead of `Cause` | ## Requirements Management | Effect | Micro | ⚠️ | | ---------------- | ---------------------- | ---------------------- | | `Effect.provide` | `Micro.provideContext` | only handles `Context` | | ❌ | `Micro.provideScope` | | | ❌ | `Micro.service` | | ## Scoping, Resources and Finalization | Effect | Micro | ⚠️ | | -------------------------- | ------------------------- | ---------------------------------------- | | `Effect.addFinalizer` | `Micro.addFinalizer` | `MicroExit` instead of `Exit` and no `R` | | `Effect.acquireRelease` | `Micro.acquireRelease` | `MicroExit` instead of `Exit` | | `Effect.acquireUseRelease` | `Micro.acquireUseRelease` | `MicroExit` instead of `Exit` | | `Effect.onExit` | `Micro.onExit` | `MicroExit` instead of `Exit` | | `Effect.onError` | `Micro.onError` | uses `MicroCause` instead of `Cause` | | ❌ | `Micro.onExitIf` | | ## Concurrency | Effect | Micro | ⚠️ | | ------------------- | ------------------ | -------------------------------------- | | `Effect.fork` | `Micro.fork` | `MicroFiber` instead of `RuntimeFiber` | | `Effect.forkDaemon` | `Micro.forkDaemon` | `MicroFiber` instead of `RuntimeFiber` | | `Effect.forkIn` | `Micro.forkIn` | `MicroFiber` instead of `RuntimeFiber` | | `Effect.forkScoped` | `Micro.forkScoped` | `MicroFiber` instead of `RuntimeFiber` | --- title: Getting Started with Micro description: Learn how to get started with the Micro module, a lightweight alternative to Effect for reducing bundle size while maintaining essential functionality in TypeScript applications. sidebar: label: Getting Started order: 0 --- import { Aside, Tabs, TabItem, Steps } from "@astrojs/starlight/components" The Micro module is designed as a lighter alternative to the standard Effect module, tailored for situations where it is beneficial to reduce the bundle size. This module is standalone and does not include more complex functionalities such as , , , and . This feature set makes Micro especially suitable for libraries that wish to utilize Effect functionalities while keeping the bundle size to a minimum, particularly for those aiming to provide `Promise`-based APIs. Micro also supports use cases where a client application uses Micro, and a server employs the full suite of Effect features, maintaining both compatibility and logical consistency across various application components. Integrating Micro adds a minimal footprint to your bundle, starting at **5kb gzipped**, which may increase depending on the features you use. ## Importing Micro Before you start, make sure you have completed the following setup: Install the `effect` library in your project. If it is not already installed, you can add it using npm with the following command: Micro is a part of the Effect library and can be imported just like any other module: You can also import it using a namespace import like this: Both forms of import allow you to access the functionalities provided by the `Micro` module. However an important consideration is **tree shaking**, which refers to a process that eliminates unused code during the bundling of your application. Named imports may generate tree shaking issues when a bundler doesn't support deep scope analysis. Here are some bundlers that support deep scope analysis and thus don't have issues with named imports: - Rollup - Webpack 5+ ## The Micro Type Here is the general form of a `Micro`: which mirror those of the `Effect` type: | Parameter | Description | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Success** | Represents the type of value that an effect can succeed with when executed. If this type parameter is `void`, it means the effect produces no useful information, while if it is `never`, it means the effect runs forever . | | **Error** | Represents the expected errors that can occur when executing an effect. If this type parameter is `never`, it means the effect cannot fail, because there are no values of type `never`. | | **Requirements** | Represents the contextual data required by the effect to be executed. This data is stored in a collection named `Context`. If this type parameter is `never`, it means the effect has no requirements and the `Context` collection is empty. | ## The MicroExit Type The `MicroExit` type is used to represent the result of a `Micro` computation. It can either be successful, containing a value of type `A`, or it can fail, containing an error of type `E` wrapped in a `MicroCause`. ## The MicroCause Type The `MicroCause` type describes the different reasons why an effect may fail. `MicroCause` comes in three forms: | Variant | Description | | ----------- | ------------------------------------------------------------------------------------------- | | `Die` | Indicates an unforeseen defect that wasn't planned for in the system's logic. | | `Fail` | Covers anticipated errors that are recognized and typically handled within the application. | | `Interrupt` | Signifies an operation that has been purposefully stopped. | ## Wrapping a Promise-based API with Micro This guide shows how to wrap a `Promise`-based API using the `Micro` library from Effect. We'll create a simple example that interacts with a hypothetical weather forecasting API, using Micro to handle structured error handling and execution flow. 1. **Create a Promise-based API Function** Start by defining a basic Promise-based function that simulates fetching weather data from an external service. 2. **Wrap the Promise with Micro** Now, wrap the `fetchWeather` function using Micro, converting the `Promise` to a Micro effect to manage both success and failure scenarios. Here, `Micro.promise` transforms the `Promise` returned by `fetchWeather` into a `Micro` effect. 3. **Running the Micro Effect** Once the function is wrapped, execute the Micro effect and handle the results. **Example** In the example above, `Micro.runPromise` is used to execute the `weatherEffect`, converting it back into a `Promise`, which can be managed using familiar asynchronous handling methods. For more detailed information on the effect's exit status, use `Micro.runPromiseExit`: **Example** 4. **Adding Error Handling** To further enhance the function, you might want to handle specific errors differently. Micro provides functions like `Micro.tryPromise` to handle anticipated errors gracefully. **Example** ## Expected Errors These errors, also referred to as _failures_, _typed errors_ or _recoverable errors_, are errors that developers anticipate as part of the normal program execution. They serve a similar purpose to checked exceptions and play a role in defining the program's domain and control flow. Expected errors **are tracked** at the type level by the `Micro` data type in the "Error" channel: ### either The `Micro.either` function transforms an `Micro` into an effect that encapsulates both potential failure and success within an data type: This means if you have an effect with the following type: and you call `Micro.either` on it, the type becomes: The resulting effect cannot fail because the potential failure is now represented within the `Either`'s `Left` type. The error type of the returned `Micro` is specified as `never`, confirming that the effect is structured to not fail. By yielding an `Either`, we gain the ability to "pattern match" on this type to handle both failure and success cases within the generator function. **Example** As you can see since all errors are handled, the error type of the resulting effect `recovered` is `never`: ### catchAll The `Micro.catchAll` function allows you to catch any error that occurs in the program and provide a fallback. **Example** We can observe that the type in the error channel of our program has changed to `never`: indicating that all errors have been handled. ### catchTag If your program's errors are **all tagged** with a `_tag` field that acts as a discriminator you can use the `Effect.catchTag` function to catch and handle specific errors with precision. **Example** In the example above, the `Micro.catchTag` function allows us to handle `HttpError` specifically. If a `HttpError` occurs during the execution of the program, the provided error handler function will be invoked, and the program will proceed with the recovery logic specified within the handler. We can observe that the type in the error channel of our program has changed to only show `ValidationError`: indicating that `HttpError` has been handled. ## Unexpected Errors Unexpected errors, also referred to as _defects_, _untyped errors_, or _unrecoverable errors_, are errors that developers do not anticipate occurring during normal program execution. Unlike expected errors, which are considered part of a program's domain and control flow, unexpected errors resemble unchecked exceptions and lie outside the expected behavior of the program. Since these errors are not expected, Effect **does not track** them at the type level. However, the Effect runtime does keep track of these errors and provides several methods to aid in recovering from unexpected errors. ### die The `Micro.die` function returns an effect that throws a specified error. This function is useful for terminating a program when a defect, a critical and unexpected error, is detected in the code. **Example** ### orDie The `Micro.orDie` function converts an effect's failure into a termination of the program, removing the error from the type of the effect. It is useful when you encounter failures that you do not intend to handle or recover from. **Example** ### catchAllDefect The `Micro.catchAllDefect` function allows you to recover from all defects using a provided function. **Example** It's important to understand that `Micro.catchAllDefect` can only handle defects, not expected errors or interruptions in execution . A defect refers to an error that cannot be anticipated in advance, and there is no reliable way to respond to it. As a general rule, it's recommended to let defects crash the application, as they often indicate serious issues that need to be addressed. However, in some specific cases, such as when dealing with dynamically loaded plugins, a controlled recovery approach might be necessary. For example, if our application supports runtime loading of plugins and a defect occurs within a plugin, we may choose to log the defect and then reload only the affected plugin instead of crashing the entire application. This allows for a more resilient and uninterrupted operation of the application. ## Fallback ### orElseSucceed The `Effect.orElseSucceed` function will replace the original failure with a success value, ensuring the effect cannot fail: **Example** ## Matching ### match The `Micro.match` function lets you handle both success and failure cases without performing side effects. You provide a handler for each case. **Example** ### matchEffect The `Micro.matchEffect` function, similar to `Micro.match`, allows you to handle both success and failure cases, but it also enables you to perform additional side effects within those handlers. **Example** ### matchCause / matchCauseEffect The `Micro.matchCause` and `Micro.matchCauseEffect` functions allow you to handle failures more precisely by providing access to the complete cause of failure within a fiber. This makes it possible to differentiate between various failure types and respond accordingly. **Example** ## Retrying ### retry The `Micro.retry` function allows you to retry a failing effect according to a defined policy. **Example** ## Timing out When an operation does not finish within the specified duration, the behavior of the `Micro.timeout` depends on whether the operation is "uninterruptible". 1. **Interruptible Operation**: If the operation can be interrupted, it is terminated immediately once the timeout threshold is reached, resulting in a `TimeoutException`. 2. **Uninterruptible Operation**: If the operation is uninterruptible, it continues until completion before the `TimeoutException` is assessed. ## Sandboxing The `Micro.sandbox` function allows you to encapsulate all the potential causes of an error in an effect. It exposes the full cause of an effect, whether it's due to a failure, defect or interruption. In simple terms, it takes an effect `Micro` and transforms it into an effect `Micro, R>` where the error channel now contains a detailed cause of the error. ## Inspecting Errors ### tapError Executes an effectful operation to inspect the failure of an effect without altering it. **Example** ### tapErrorCause This function inspects the complete cause of an error, including failures and defects. **Example** ### tapDefect Specifically inspects non-recoverable failures or defects in an effect causes). **Example** ## Yieldable Errors Yieldable Errors are special types of errors that can be yielded directly within a generator function using `Micro.gen`. These errors allow you to handle them intuitively, without needing to explicitly invoke `Micro.fail`. This simplifies how you manage custom errors in your code. ### Error The `Error` constructor provides a way to define a base class for yieldable errors. **Example** ### TaggedError The `TaggedError` constructor lets you define custom yieldable errors with unique tags. Each error has a `_tag` property, allowing you to easily distinguish between different error types. This makes it convenient to handle specific tagged errors using functions like `Micro.catchTag`. **Example** ## Requirements Management In the context of programming, a **service** refers to a reusable component or functionality that can be used by different parts of an application. Services are designed to provide specific capabilities and can be shared across multiple modules or components. Services often encapsulate common tasks or operations that are needed by different parts of an application. They can handle complex operations, interact with external systems or APIs, manage data, or perform other specialized tasks. Services are typically designed to be modular and decoupled from the rest of the application. This allows them to be easily maintained, tested, and replaced without affecting the overall functionality of the application. To create a new service, you need two things: - A unique identifier. - A type describing the possible operations of the service. Now that we have our service tag defined, let's see how we can use it by building a simple program. **Example** It's worth noting that the type of the `program` variable includes `Random` in the `Requirements` type parameter: This indicates that our program requires the `Random` service to be provided in order to execute successfully. To successfully execute the program, we need to provide an actual implementation of the `Random` service. **Example** ## Resource Management ### MicroScope In simple terms, a `MicroScope` represents the lifetime of one or more resources. When a scope is closed, the resources associated with it are guaranteed to be released. With the `MicroScope` data type, you can: - **Add finalizers**: A finalizer specifies the cleanup logic for a resource. - **Close the scope**: When the scope is closed, all resources are released, and the finalizers are executed. **Example** In the above example, finalizers are added to the scope, and when the scope is closed, the finalizers are **executed in the reverse order**. This reverse order is important because it ensures that resources are released in the correct sequence. For instance, if you acquire a network connection and then access a file on a remote server, the file must be closed before the network connection to avoid errors. ### addFinalizer The `Micro.addFinalizer` function is a high-level API that allows you to add finalizers to the scope of an effect. A finalizer is a piece of code that is guaranteed to run when the associated scope is closed. The behavior of the finalizer can vary based on the `MicroExit` value, which represents how the scope was closed—whether successfully or with an error. **Example** Next, let's explore how things behave in the event of a failure: **Example** ### Defining Resources We can define a resource using operators like `Micro.acquireRelease`, which allows us to create a scoped value from an `acquire` and `release` workflow. Every acquire release requires three actions: - **Acquiring Resource**. An effect describing the acquisition of resource. For example, opening a file. - **Using Resource**. An effect describing the actual process to produce a result. For example, counting the number of lines in a file. - **Releasing Resource**. An effect describing the final step of releasing or cleaning up the resource. For example, closing a file. The `Micro.acquireRelease` operator performs the `acquire` workflow **uninterruptibly**. This is important because if we allowed interruption during resource acquisition we could be interrupted when the resource was partially acquired. The guarantee of the `Micro.acquireRelease` operator is that if the `acquire` workflow successfully completes execution then the `release` workflow is guaranteed to be run when the `Scope` is closed. **Example** The `Micro.scoped` operator removes the `MicroScope` from the context, indicating that there are no longer any resources used by this workflow which require a scope. ### acquireUseRelease The `Micro.acquireUseRelease` function is a specialized version of the `Micro.acquireRelease` function that simplifies resource management by automatically handling the scoping of resources. The main difference is that `acquireUseRelease` eliminates the need to manually call `Micro.scoped` to manage the resource's scope. It has additional knowledge about when you are done using the resource created with the `acquire` step. This is achieved by providing a `use` argument, which represents the function that operates on the acquired resource. As a result, `acquireUseRelease` can automatically determine when it should execute the release step. **Example** ## Scheduling ### MicroSchedule The `MicroSchedule` type represents a function that can be used to calculate the delay between repeats. The function takes the current attempt number and the elapsed time since the first attempt, and returns the delay for the next attempt. If the function returns `None`, the repetition will stop. ### repeat The `Micro.repeat` function returns a new effect that repeats the given effect according to a specified schedule or until the first failure. **Example** **Example** ### Simulating Schedule Behavior This helper function, `dryRun`, demonstrates how different scheduling policies control repetition timing without executing an actual effect. By returning an array of delay intervals, it visualizes how a schedule would space repetitions. ### scheduleSpaced A schedule that repeats indefinitely, each repetition spaced the specified duration from the last run. **Example** ### scheduleExponential A schedule that recurs using exponential backoff, with each delay increasing exponentially. **Example** ### scheduleUnion Combines two schedules using union. The schedule recurs as long as one of the schedules wants to, using the minimum delay between recurrences. **Example** ### scheduleIntersect Combines two schedules using intersection. The schedule recurs only if both schedules want to continue, using the maximum delay between them. **Example** ## Concurrency ### Forking Effects One of the fundamental ways to create a fiber is by forking an existing effect. When you fork an effect, it starts executing the effect on a new fiber, giving you a reference to this newly-created fiber. The following code demonstrates how to create a single fiber using the `Micro.fork` function. This fiber will execute the function `fib` independently of the main fiber: **Example** ### Joining Fibers A common operation with fibers is joining them using the `Micro.fiberJoin` function. This function returns a `Micro` that will succeed or fail based on the outcome of the fiber it joins: **Example** ### Awaiting Fibers Another useful function for fibers is `Micro.fiberAwait`. This function returns an effect containing a `MicroExit` value, which provides detailed information about how the fiber completed. **Example** ## Interruptions All effects in Effect are executed by fibers. If you didn't create the fiber yourself, it was created by an operation you're using or by the Effect runtime system. A fiber is created any time an effect is run. When running effects concurrently, a fiber is created for each concurrent effect. To summarize: - A `Micro` is a higher-level concept that describes an effectful computation. It is lazy and immutable, meaning it represents a computation that may produce a value or fail but does not immediately execute. - A fiber, on the other hand, represents the running execution of a `Micro`. It can be interrupted or awaited to retrieve its result. Think of it as a way to control and interact with the ongoing computation. Fibers can be interrupted in various ways. Let's explore some of these scenarios and see examples of how to interrupt fibers in Effect. ### Interrupting Fibers If a fiber's result is no longer needed, it can be interrupted, which immediately terminates the fiber and safely releases all resources by running all finalizers. Similar to `.await`, `.interrupt` returns a `MicroExit` value describing how the fiber completed. **Example** ### Micro.interrupt A fiber can be interrupted using the `Micro.interrupt` effect on that particular fiber. **Example** In this case, the program runs without any interruption, logging the start and completion of the task. **Example** Here, the fiber is interrupted after the log `"start"` but before the `"done"` log. The `Effect.interrupt` stops the fiber, and it never reaches the final log. When a fiber is interrupted, the cause of the interruption is captured, including details like the fiber's ID and when it started. ### Interruption of Concurrent Effects When running multiple effects concurrently, such as with `Micro.forEach`, if one of the effects is interrupted, it causes all concurrent effects to be interrupted as well. **Example** ## Racing The `Effect.race` function allows you to run multiple effects concurrently, returning the result of the first one that successfully completes. **Example** If you want to handle the result of whichever task completes first, whether it succeeds or fails, you can use the `Micro.either` function. This function wraps the result in an type, allowing you to see if the result was a success or a failure : **Example** --- title: Logging description: Discover Effect's logging utilities for dynamic log levels, custom outputs, and fine-grained control over logs. sidebar: order: 0 --- import { Aside } from "@astrojs/starlight/components" Logging is an important aspect of software development, especially for debugging and monitoring the behavior of your applications. In this section, we'll explore Effect's logging utilities and see how they compare to traditional logging methods. ## Advantages Over Traditional Logging Effect's logging utilities provide several benefits over conventional logging approaches: 1. **Dynamic Log Level Control**: With Effect's logging, you have the ability to change the log level dynamically. This means you can control which log messages get displayed based on their severity. For example, you can configure your application to log only warnings or errors, which can be extremely helpful in production environments to reduce noise. 2. **Custom Logging Output**: Effect's logging utilities allow you to change how logs are handled. You can direct log messages to various destinations, such as a service or a file, using a . This flexibility ensures that logs are stored and processed in a way that best suits your application's requirements. 3. **Fine-Grained Logging**: Effect enables fine-grained control over logging on a per-part basis of your program. You can set different log levels for different parts of your application, tailoring the level of detail to each specific component. This can be invaluable for debugging and troubleshooting, as you can focus on the information that matters most. 4. **Environment-Based Logging**: Effect's logging utilities can be combined with deployment environments to achieve granular logging strategies. For instance, during development, you might choose to log everything at a trace level and above for detailed debugging. In contrast, your production version could be configured to log only errors or critical issues, minimizing the impact on performance and noise in production logs. 5. **Additional Features**: Effect's logging utilities come with additional features such as the ability to measure time spans, alter log levels on a per-effect basis, and integrate spans for performance monitoring. ## log The `Effect.log` function allows you to log a message at the default `INFO` level. **Example** The default logger in Effect adds several useful details to each log entry: | Annotation | Description | | ----------- | --------------------------------------------------------------------------------------------------- | | `timestamp` | The timestamp when the log message was generated. | | `level` | The log level at which the message is logged . | | `fiber` | The identifier of the executing the program. | | `message` | The log message content, which can include multiple strings or values. | | `span` | The duration of a span in milliseconds, providing insight into the timing of operations. | You can also log multiple messages at once. **Example** For added context, you can also include one or more instances in your logs, which provide detailed error information under an additional `cause` annotation: **Example** ## Log Levels ### logDebug By default, `DEBUG` messages **are not displayed**. To enable `DEBUG` logs, you can adjust the logging configuration using `Logger.withMinimumLogLevel`, setting the minimum level to `LogLevel.Debug`. **Example** ### logInfo The `INFO` log level is displayed by default. This level is typically used for general application events or progress updates. **Example** ### logWarning The `WARN` log level is displayed by default. This level is intended for potential issues or warnings that do not immediately disrupt the flow of the program but should be monitored. **Example** ### logError The `ERROR` log level is displayed by default. These messages represent issues that need to be addressed. **Example** ### logFatal The `FATAL` log level is displayed by default. This log level is typically reserved for unrecoverable errors. **Example** ## Custom Annotations You can enhance your log outputs by adding custom annotations using the `Effect.annotateLogs` function. This allows you to attach extra metadata to each log entry, improving traceability and providing additional context. ### Adding a Single Annotation You can apply a single annotation as a key/value pair to all log entries within an effect. **Example** In this example, all logs generated within the `program` will include the annotation `key=value`. ### Annotations with Nested Effects Annotations propagate to all logs generated within nested or downstream effects. This ensures that logs from any child effects inherit the parent effect's annotations. **Example** In this example, the annotation `key=value` is included in all logs, even those from the nested `anotherProgram` effect. ### Adding Multiple Annotations You can also apply multiple annotations at once by passing an object with key/value pairs. Each key/value pair will be added to every log entry within the effect. **Example** In this case, each log will contain both `key1=value1` and `key2=value2`. ### Scoped Annotations If you want to limit the scope of your annotations so that they only apply to certain log entries, you can use `Effect.annotateLogsScoped`. This function confines the annotations to logs produced within a specific scope. **Example** ## Log Spans Effect provides built-in support for log spans, which allow you to measure and log the duration of specific tasks or sections of your code. This feature is helpful for tracking how long certain operations take, giving you better insights into the performance of your application. **Example** ## Disabling Default Logging Sometimes, perhaps during test execution, you might want to disable default logging in your application. Effect provides several ways to turn off logging when needed. In this section, we'll look at different methods to disable logging in the Effect framework. **Example** One convenient way to disable logging is by using the `Logger.withMinimumLogLevel` function. This allows you to set the minimum log level to `None`, effectively turning off all log output. **Example** Another approach to disable logging is by creating a layer that sets the minimum log level to `LogLevel.None`, effectively turning off all log output. **Example** You can also disable logging by creating a custom runtime that includes the configuration to turn off logging: ## Loading the Log Level from Configuration To dynamically load the log level from a and apply it to your program, you can use the `Logger.minimumLogLevel` layer. This allows your application to adjust its logging behavior based on external configuration. **Example** ## Custom loggers In this section, you'll learn how to define a custom logger and set it as the default logger in your application. Custom loggers give you control over how log messages are handled, such as routing them to external services, writing to files, or formatting logs in a specific way. ### Defining a Custom Logger You can define your own logger using the `Logger.make` function. This function allows you to specify how log messages should be processed. **Example** In this example, the custom logger logs messages to the console with the log level and message formatted as ` Message`. ### Using a Custom Logger in a Program Let's assume you have the following tasks and a program where you log some messages: To replace the default logger with your custom logger, you can use the `Logger.replace` function. After creating a layer that replaces the default logger, you provide it to your program using `Effect.provide`. **Example** When you run the above program, the following log messages are printed to the console: ## Built-in Loggers Effect provides several built-in loggers that you can use depending on your logging needs. These loggers offer different formats, each suited for different environments or purposes, such as development, production, or integration with external logging services. Each logger is available in two forms: the logger itself, and a layer that uses the logger and sends its output to the `Console` . For example, the `structuredLogger` logger generates logs in a detailed object-based format, while the `structured` layer uses the same logger and writes the output to the `Console` service. ### stringLogger The `stringLogger` logger produces logs in a human-readable key-value style. This format is commonly used in development and production because it is simple and easy to read in the console. This logger does not have a corresponding layer because it is the default logger. Output: ### logfmtLogger The `logfmtLogger` logger produces logs in a human-readable key-value format, similar to the logger. The main difference is that `logfmtLogger` removes extra spaces to make logs more compact. Output: ### prettyLogger The `prettyLogger` logger enhances log output by using color and indentation for better readability, making it particularly useful during development when visually scanning logs in the console. Output: ### structuredLogger The `structuredLogger` logger produces logs in a detailed object-based format. This format is helpful when you need more traceable logs, especially if other systems analyze them or store them for later review. Output: | Field | Description | | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `message` | Either a single processed value or an array of processed values, depending on how many messages are logged. | | `logLevel` | A string that indicates the log level label . | | `timestamp` | An ISO 8601 timestamp for when the log was generated . | | `cause` | A string that shows detailed error information, or `undefined` if no cause was provided. | | `annotations` | An object where each key is an annotation label and the corresponding value is parsed into a structured format . | | `spans` | An object mapping each span label to its duration in milliseconds, measured from its start time until the moment the logger was called . | | `fiberId` | The identifier of the fiber that generated this log . | ### jsonLogger The `jsonLogger` logger produces logs in JSON format. This can be useful for tools or services that parse and store JSON logs. It calls `JSON.stringify` on the object created by the logger. Output: ## Combine Loggers ### zip The `Logger.zip` function combines two loggers into a new logger. This new logger forwards log messages to both the original loggers. **Example** --- title: Metrics in Effect description: Effect Metrics provides powerful monitoring tools, including counters, gauges, histograms, summaries, and frequencies, to track your application's performance and behavior. sidebar: label: Metrics order: 1 --- import { Aside } from "@astrojs/starlight/components" In complex and highly concurrent applications, managing various interconnected components can be quite challenging. Ensuring that everything runs smoothly and avoiding application downtime becomes crucial in such setups. Now, let's imagine we have a sophisticated infrastructure with numerous services. These services are replicated and distributed across our servers. However, we often lack insight into what's happening across these services, including error rates, response times, and service uptime. This lack of visibility can make it challenging to identify and address issues effectively. This is where Effect Metrics comes into play; it allows us to capture and analyze various metrics, providing valuable data for later investigation. Effect Metrics offers support for five different types of metrics: | Metric | Description | | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Counter** | Counters are used to track values that increase over time, such as request counts. They help us keep tabs on how many times a specific event or action has occurred. | | **Gauge** | Gauges represent a single numerical value that can fluctuate up and down over time. They are often used to monitor metrics like memory usage, which can vary continuously. | | **Histogram** | Histograms are useful for tracking the distribution of observed values across different buckets. They are commonly used for metrics like request latencies, allowing us to understand how response times are distributed. | | **Summary** | Summaries provide insight into a sliding window of a time series and offer metrics for specific percentiles of the time series, often referred to as quantiles. This is particularly helpful for understanding latency-related metrics, such as request response times. | | **Frequency** | Frequency metrics count the occurrences of distinct string values. They are useful when you want to keep track of how often different events or conditions are happening in your application. | ## Counter In the world of metrics, a Counter is a metric that represents a single numerical value that can be both incremented and decremented over time. Think of it like a tally that keeps track of changes, such as the number of a particular type of request received by your application, whether it's increasing or decreasing. Unlike some other types of metrics ), where we're interested in the value at a specific moment, with counters, we care about the cumulative value over time. This means it provides a running total of changes, which can go up and down, reflecting the dynamic nature of certain metrics. Some typical use cases for counters include: - **Request Counts**: Monitoring the number of incoming requests to your server. - **Completed Tasks**: Keeping track of how many tasks or processes have been successfully completed. - **Error Counts**: Counting the occurrences of errors in your application. ### How to Create a Counter To create a counter, you can use the `Metric.counter` constructor. **Example** Once created, the counter can accept an effect that returns a `number`, which will increment or decrement the counter. **Example** ### Counter Types You can specify whether the counter tracks a `number` or `bigint`. ### Increment-Only Counters If you need a counter that only increments, you can use the `incremental: true` option. **Example** In this configuration, the counter only accepts positive values. Any attempts to decrement will have no effect, ensuring the counter strictly counts upwards. ### Counters With Constant Input You can configure a counter to always increment by a fixed value each time it is invoked. **Example** ## Gauge In the world of metrics, a Gauge is a metric that represents a single numerical value that can be set or adjusted. Think of it as a dynamic variable that can change over time. One common use case for a gauge is to monitor something like the current memory usage of your application. Unlike counters, where we're interested in cumulative values over time, with gauges, our focus is on the current value at a specific point in time. Gauges are the best choice when you want to monitor values that can both increase and decrease, and you're not interested in tracking their rates of change. In other words, gauges help us measure things that have a specific value at a particular moment. Some typical use cases for gauges include: - **Memory Usage**: Keeping an eye on how much memory your application is using right now. - **Queue Size**: Monitoring the current size of a queue where tasks are waiting to be processed. - **In-Progress Request Counts**: Tracking the number of requests currently being handled by your server. - **Temperature**: Measuring the current temperature, which can fluctuate up and down. ### How to Create a Gauge To create a gauge, you can use the `Metric.gauge` constructor. **Example** Once created, a gauge can be updated by passing an effect that produces the value you want to set for the gauge. **Example** ### Gauge Types You can specify whether the gauge tracks a `number` or `bigint`. ## Histogram A Histogram is a metric used to analyze how numerical values are distributed over time. Instead of focusing on individual data points, a histogram groups values into predefined ranges, called **buckets**, and tracks how many values fall into each range. When a value is recorded, it gets assigned to one of the histogram's buckets based on its range. Each bucket has an upper boundary, and the count for that bucket is increased if the value is less than or equal to its boundary. Once recorded, the individual value is discarded, and the focus shifts to how many values have fallen into each bucket. Histograms also track: - **Total Count**: The number of values that have been observed. - **Sum**: The sum of all the observed values. - **Min**: The smallest observed value. - **Max**: The largest observed value. Histograms are especially useful for calculating percentiles, which can help you estimate specific points in a dataset by analyzing how many values are in each bucket. This concept is inspired by , a well-known monitoring and alerting toolkit. Histograms are particularly useful in performance analysis and system monitoring. By examining how response times, latencies, or other metrics are distributed, you can gain valuable insights into your system's behavior. This data helps you identify outliers, performance bottlenecks, or trends that may require optimization. Common use cases for histograms include: - **Percentile Estimation**: Histograms allow you to approximate percentiles of observed values, like the 95th percentile of response times. - **Known Ranges**: If you can estimate the range of values in advance, histograms can organize the data into predefined buckets for better analysis. - **Performance Metrics**: Use histograms to track metrics like request latencies, memory usage, or throughput over time. - **Aggregation**: Histograms can be aggregated across multiple instances, making them ideal for distributed systems where you need to collect data from different sources. **Example** In this example, we define a histogram with linear buckets, where the values range from `0` to `100` in increments of `10`. Additionally, we include a final bucket for values greater than `100`, referred to as the "Infinity" bucket. This configuration is useful for tracking numeric values, like request latencies, within specific ranges. The program generates random numbers between `1` and `120`, records them in the histogram, and then prints the histogram's state, showing the count of values that fall into each bucket. ### Timer Metric In this example, we demonstrate how to use a timer metric to track the duration of specific workflows. The timer captures how long certain tasks take to execute, storing this information in a histogram, which provides insights into the distribution of these durations. We generate random values to simulate varying wait times, record the durations in the timer, and then print out the histogram's state. **Example** ## Summary A Summary is a metric that gives insights into a series of data points by calculating specific percentiles. Percentiles help us understand how data is distributed. For instance, if you're tracking response times for requests over the past hour, you may want to examine key percentiles such as the 50th, 90th, 95th, or 99th to better understand your system's performance. Summaries are similar to histograms in that they observe `number` values, but with a different approach. Instead of immediately sorting values into buckets and discarding them, a summary holds onto the observed values in memory. However, to avoid storing too much data, summaries use two parameters: - **maxAge**: The maximum age a value can have before it's discarded. - **maxSize**: The maximum number of values stored in the summary. This creates a sliding window of recent values, so the summary always represents a fixed number of the most recent observations. Summaries are commonly used to calculate **quantiles** over this sliding window. A **quantile** is a number between `0` and `1` that represents the percentage of values less than or equal to a certain threshold. For example, a quantile of `0.5` is the **median** value, while `0.95` would represent the value below which 95% of the observed data falls. Quantiles are helpful for monitoring important performance metrics, such as latency, and for ensuring that your system meets performance goals . The Effect Metrics API also allows you to configure summaries with an **error margin**. This margin introduces a range of acceptable values for quantiles, improving the accuracy of the result. Summaries are particularly useful in cases where: - The range of values you're observing is not known or estimated in advance, making histograms less practical. - You don't need to aggregate data across multiple instances or average results. Summaries calculate their results on the application side, meaning they focus on the specific instance where they are used. **Example** In this example, we will create a summary to track response times. The summary will: - Hold up to `100` samples. - Discard samples older than `1 day`. - Have a `3%` error margin when calculating quantiles. - Report the `10%`, `50%`, and `90%` quantiles, which help track response time distributions. We'll apply the summary to an effect that generates random integers, simulating response times. ## Frequency Frequencies are metrics that help count the occurrences of specific values. Think of them as a set of counters, each associated with a unique value. When new values are observed, the frequency metric automatically creates new counters for those values. Frequencies are particularly useful for tracking how often distinct string values occur. Some example use cases include: - Counting the number of invocations for each service in an application, where each service has a logical name. - Monitoring how frequently different types of failures occur. **Example** In this example, we'll create a `Frequency` to observe how often different error codes occur. This can be applied to effects that return a `string` value: ## Tagging Metrics Tags are key-value pairs you can add to metrics to provide additional context. They help categorize and filter metrics, making it easier to analyze specific aspects of your application's performance or behavior. When creating metrics, you can add tags to them. Tags are key-value pairs that provide additional context, helping in categorizing and filtering metrics. This makes it easier to analyze and monitor specific aspects of your application. ### Tagging a Specific Metric You can tag individual metrics using the `Metric.tagged` function. This allows you to add specific tags to a single metric, providing detailed context without applying tags globally. **Example** Here, the `request_count` metric is tagged with `"environment": "production"`, allowing you to filter or analyze metrics by this tag later. ### Tagging Multiple Metrics You can use `Effect.tagMetrics` to apply tags to all metrics within the same context. This is useful when you want to apply common tags, like the environment , across multiple metrics. **Example** If you only want to apply tags within a specific , you can use `Effect.tagMetricsScoped`. This limits the tag application to metrics within that scope, allowing for more precise tagging control. --- title: Supervisor description: Effect's Supervisor manages fiber lifecycles, enabling tracking, monitoring, and controlling fibers' behavior within an application. sidebar: order: 3 --- A `Supervisor` is a utility for managing fibers in Effect, allowing you to track their lifecycle and producing a value of type `A` that reflects this supervision. Supervisors are useful when you need insight into or control over the behavior of fibers within your application. To create a supervisor, you can use the `Supervisor.track` function. This generates a new supervisor that keeps track of its child fibers, maintaining them in a set. This allows you to observe and monitor their status during execution. You can supervise an effect by using the `Effect.supervised` function. This function takes a supervisor as an argument and returns an effect where all child fibers forked within it are supervised by the provided supervisor. This enables you to capture detailed information about these child fibers, such as their status, through the supervisor. **Example** In this example, we'll periodically monitor the number of fibers running in the application using a supervisor. The program calculates a Fibonacci number, spawning multiple fibers in the process, while a separate monitor tracks the fiber count. --- title: Tracing in Effect description: Explore tracing in distributed systems to track request lifecycles across services using spans and traces for debugging and performance optimization. sidebar: label: Tracing order: 2 --- import { Tabs, TabItem, Steps, Aside } from "@astrojs/starlight/components" Although logs and metrics are useful to understand the behavior of individual services, they are not enough to provide a complete overview of the lifetime of a request in a distributed system. In a distributed system, a request can span multiple services and each service can make multiple requests to other services to fulfill the request. In such a scenario, we need to have a way to track the lifetime of a request across multiple services to diagnose what services are the bottlenecks and where the request is spending most of its time. ## Spans A **span** represents a single unit of work or operation within a request. It provides a detailed view of what happened during the execution of that specific operation. Each span typically contains the following information: | Span Component | Description | | ---------------- | ------------------------------------------------------------------ | | **Name** | Describes the specific operation being tracked. | | **Timing Data** | Timestamps indicating when the operation started and its duration. | | **Log Messages** | Structured logs capturing important events during the operation. | | **Attributes** | Metadata providing additional context about the operation. | Spans are key building blocks in tracing, helping you visualize and understand the flow of requests through various services. ## Traces A trace records the paths taken by requests as they propagate through multi-service architectures, like microservice and serverless applications. Without tracing, it is challenging to pinpoint the cause of performance problems in a distributed system. A trace is made of one or more spans. The first span represents the root span. Each root span represents a request from start to finish. The spans underneath the parent provide a more in-depth context of what occurs during a request . Many Observability back-ends visualize traces as waterfall diagrams that may look something like this: ! Waterfall diagrams show the parent-child relationship between a root span and its child spans. When a span encapsulates another span, this also represents a nested relationship. ## Creating Spans You can add tracing to an effect by creating a span using the `Effect.withSpan` API. This helps you track specific operations within the effect. **Example** Instrumenting an effect with a span does not change its type. If you start with an `Effect`, the result remains an `Effect`. ## Printing Spans To print spans for debugging or analysis, you'll need to install the required tracing tools. Here’s how to set them up for your project. ### Installing Dependencies Choose your package manager and install the necessary libraries: ### Printing a Span to the Console Once the dependencies are installed, you can set up span printing using OpenTelemetry. Here's an example showing how to print a span for an effect. **Example** ### Understanding the Span Output The output provides detailed information about the span: | Field | Description | | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `traceId` | A unique identifier for the entire trace, helping trace requests or operations as they move through an application. | | `parentId` | Identifies the parent span of the current span, marked as `undefined` in the output when there is no parent span, making it a root span. | | `name` | Describes the name of the span, indicating the operation being tracked . | | `id` | A unique identifier for the current span, distinguishing it from other spans within a trace. | | `timestamp` | A timestamp representing when the span started, measured in microseconds since the Unix epoch. | | `duration` | Specifies the duration of the span, representing the time taken to complete the operation . | | `attributes` | Spans may contain attributes, which are key-value pairs providing additional context or information about the operation. In this output, it's an empty object, indicating no specific attributes in this span. | | `status` | The status field provides information about the span's status. In this case, it has a code of 1, which typically indicates an OK status | | `events` | Spans can include events, which are records of specific moments during the span's lifecycle. In this output, it's an empty array, suggesting no specific events recorded. | | `links` | Links can be used to associate this span with other spans in different traces. In the output, it's an empty array, indicating no specific links for this span. | ### Span Capturing an Error Here's how a span looks when the effect encounters an error: **Example** In this example, the span's status code is `2`, indicating an error. The message in the status provides more details about the failure. ## Adding Annotations You can provide extra information to a span by utilizing the `Effect.annotateCurrentSpan` function. This function allows you to attach key-value pairs, offering more context about the execution of the span. **Example** ## Logs as events In the context of tracing, logs are converted into "Span Events." These events offer structured insights into your application's activities and provide a timeline of when specific operations occurred. Each span can include events, which capture specific moments during the execution of a span. In this example, a log message `"Hello"` is recorded as an event within the span. Key details of the event include: | Field | Description | | ------------------------ | ------------------------------------------------------------------------------------------------- | | `name` | The name of the event, which corresponds to the logged message . | | `attributes` | Key-value pairs that provide additional context about the event, such as `fiberId` and log level. | | `time` | The timestamp of when the event occurred, shown in a high-precision format. | | `droppedAttributesCount` | Indicates how many attributes were discarded, if any. In this case, no attributes were dropped. | ## Nesting Spans Spans can be nested to represent a hierarchy of operations. This allows you to track how different parts of your application relate to one another during execution. The following example demonstrates how to create and manage nested spans. **Example** The parent-child relationship is evident in the span output, where the `parentId` of the `child` span matches the `id` of the `parent` span. This structure helps track how operations are related within a single trace. ## Tutorial: Visualizing Traces In this tutorial, we will guide you through visualizing traces generated by a sample Effect application. The sample application has also been configured to export traces and/or metrics via HTTP using . To visualize the traces being exported by our application, we will use a Docker image that contains a preconfigured OpenTelemetry backend based on the , , , , and . ### Tools Explained Let's understand the tools we'll be using in simple terms: - **Docker**: Docker allows us to run applications in containers. Think of a container as a lightweight and isolated environment where your application can run consistently, regardless of the host system. It's a bit like a virtual machine but more efficient. - **Prometheus**: Prometheus is a monitoring and alerting toolkit. It collects metrics and data about your applications and stores them for further analysis. This helps in identifying performance issues and understanding the behavior of your applications. - **Loki**: Loki is a log aggregation system inspired by Prometheus. It does not index the contents of the logs, but rather a set of labels for each log stream. - **Grafana**: Grafana is a visualization and analytics platform. It helps in creating beautiful and interactive dashboards to visualize your application's data. You can use it to graphically represent metrics collected by Prometheus. - **Tempo**: Tempo is a distributed tracing system that allows you to trace the journey of a request as it flows through your application. It provides insights into how requests are processed and helps in debugging and optimizing your applications. ### Getting Docker To get Docker, follow these steps: 1. Visit the Docker website at . 2. Download Docker Desktop for your operating system and install it. 3. After installation, open Docker Desktop, and it will run in the background. ### Simulating Traces 1. **Start the OpenTelemetry Backend** Before we begin generating and exporting traces from our sample application, we will need to get our OpenTelemetry backend running in Docker. This can be done using the following command: 2. **Install Dependencies** We also need to install a few additional dependencies, as well as the latest version of `effect`: 3. **Simulate Traces** Now, let's simulate traces using a sample Node.js application. The following code simulates a set of tasks and generates traces for each task. It also sets up a `Layer` which will export traces from our application to our OpenTelemetry backend over HTTP in OTLP format. 4. **Visualize Traces** Open your web browser and go to `http://localhost:3000/explore`. You should see the Grafana Tempo TraceQL interface. ! To get a list of all available traces, we can select the `"Search"` query type to get a list of all available traces. ! Clicking the generated Trace ID will allow us to inspect the details of the trace. ! ## Integrations ### Sentry To send span data directly to Sentry for analysis, replace the default span processor with Sentry's implementation. This allows you to use Sentry as a backend for tracing and debugging. **Example** --- title: Welcome to Effect description: Effect is a TypeScript library for building production-grade software — typed error handling, structured concurrency, resource safety, and observability from one composable core. sidebar: label: Welcome order: 0 --- Effect is a TypeScript library for building production-grade software: typed error handling, structured concurrency, resource safety, and observability, all from one composable core. ## Why Effect? ### Typed data, untyped programs TypeScript is excellent at describing your data, but it says almost nothing about your programs: a function's signature doesn't tell you what can fail, what dependencies it needs, or whether it can be safely retried, timed out, or interrupted. As applications grow, teams end up hand-rolling those guarantees with ad-hoc try/catch, unstructured promises, and libraries that don't compose. ### Programs as values Effect closes that gap with a single building block: the `Effect` type, a value that fully describes a program, including its success value, its possible errors, and its requirements. Because programs are values, they compose: retries, timeouts, concurrency, resource handling, and tracing are operators you apply, not architectures you rebuild.
Out of the box
Typed errors
Failures live in the signature and are handled like data.
Retries and scheduling
Backoff policies you compose instead of loops you write.
Structured concurrency
Bounded parallel work that cleans up after itself.
Resource safety
Acquisition and release are guaranteed, even when things fail.
Dependency injection
Services wired through the type system, trivial to swap in tests.
Observability
Tracing, metrics, and structured logging built into the runtime.
Streaming
Backpressured streams that use the same operators as everything else.
Schema validation
Parse and transform data with types that match reality.
Configuration
Typed config from the environment, validated at startup, secrets redacted.
A consistent ecosystem
HTTP, SQL, CLI, AI, and platform packages built on the same core.
### Built for the AI era The AI era raises the stakes. When coding agents write a growing share of your software, the bottleneck moves from writing code to trusting it. Effect makes a program's failure modes, dependencies, and lifecycle visible to the compiler, turning runtime surprises into precise feedback an agent can act on. And when the thing you are building is itself an AI application, flaky providers, retries, streaming, and rate limits are exactly the problems Effect solves out of the box. ## Your Learning Path Effect rewards learning in order: each step builds on the one before it. Following this spine takes most developers a few focused days; everything else branches off from it.
  1. Understand the core idea An Effect is a value that describes a program: what it produces, how it can fail, and what it needs to run. Everything else builds on this one type.
  2. Set up your project Install the library and configure TypeScript. Effect is a single dependency with zero required tooling.
  3. Write your first program Create effects, compose them with generators, and run them at the edge of your app.
  4. Handle errors the Effect way Errors are typed values, not surprises. Learn expected vs unexpected failures, fallbacks, and retries.
  5. Go concurrent Run effects in parallel with bounded concurrency, race them, and let structured concurrency clean up after you.
Here's the kind of program you'll be writing within the first hour: ## Keep Going
Guides{" "} The in-depth documentation: every topic from pipelines to PubSub, structured to read in order or dip into. Reference{" "} The complete API reference for every Effect module, versioned per release. Playground{" "} Run Effect in your browser with a full editor, no local setup needed. Ideal as a second tab while you learn.
## Join our Community The Effect community lives on : the core team and experienced users are there daily, and no question goes unanswered. --- title: Command description: Learn how to create, run, and manage commands with custom arguments, environment variables, and input/output handling in Effect. sidebar: order: 1 --- The `@effect/platform/Command` module provides a way to create and run commands with the specified process name and an optional list of arguments. ## Creating Commands The `Command.make` function generates a command object, which includes details such as the process name, arguments, and environment. **Example** This command object does not execute until run by an executor. ## Running Commands You need a `CommandExecutor` to run the command, which can capture output in various formats such as strings, lines, or streams. **Example** ### Output Formats You can choose different methods to handle command output: | Method | Description | | ------------- | ---------------------------------------------------------------------------------------- | | `string` | Runs the command returning the output as a string | | `lines` | Runs the command returning the output as an array of lines | | `stream` | Runs the command returning the output as a stream of `Uint8Array` chunks | | `streamLines` | Runs the command returning the output as a stream of lines | ### exitCode If you only need the exit code of a command, use `Command.exitCode`. **Example** ## Custom Environment Variables You can customize environment variables in a command by using `Command.env`. This is useful when you need specific variables for the command's execution. **Example** In this example, the command runs in a shell to ensure environment variables are correctly processed. ## Feeding Input to a Command You can send input directly to a command's standard input using the `Command.feed` function. **Example** ## Fetching Process Details You can access details about a running process, such as `exitCode`, `stdout`, and `stderr`. **Example** ## Streaming stdout to process.stdout To stream a command's `stdout` directly to `process.stdout`, you can use the following approach: **Example** --- title: FileSystem description: Explore file system operations for reading, writing, and managing files and directories in Effect. sidebar: order: 2 --- The `@effect/platform/FileSystem` module provides a set of operations for reading and writing from/to the file system. ## Basic Usage The module provides a single `FileSystem` , which acts as the gateway for interacting with the filesystem. **Example** The `FileSystem` interface includes the following operations: | Operation | Description | | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **access** | Check if a file can be accessed. You can optionally specify the level of access to check for. | | **copy** | Copy a file or directory from `fromPath` to `toPath`. Equivalent to `cp -r`. | | **copyFile** | Copy a file from `fromPath` to `toPath`. | | **chmod** | Change the permissions of a file. | | **chown** | Change the owner and group of a file. | | **exists** | Check if a path exists. | | **link** | Create a hard link from `fromPath` to `toPath`. | | **makeDirectory** | Create a directory at `path`. You can optionally specify the mode and whether to recursively create nested directories. | | **makeTempDirectory** | Create a temporary directory. By default, the directory will be created inside the system's default temporary directory. | | **makeTempDirectoryScoped** | Create a temporary directory inside a scope. Functionally equivalent to `makeTempDirectory`, but the directory will be automatically deleted when the scope is closed. | | **makeTempFile** | Create a temporary file. The directory creation is functionally equivalent to `makeTempDirectory`. The file name will be a randomly generated string. | | **makeTempFileScoped** | Create a temporary file inside a scope. Functionally equivalent to `makeTempFile`, but the file will be automatically deleted when the scope is closed. | | **open** | Open a file at `path` with the specified `options`. The file handle will be automatically closed when the scope is closed. | | **readDirectory** | List the contents of a directory. You can recursively list the contents of nested directories by setting the `recursive` option. | | **readFile** | Read the contents of a file. | | **readFileString** | Read the contents of a file as a string. | | **readLink** | Read the destination of a symbolic link. | | **realPath** | Resolve a path to its canonicalized absolute pathname. | | **remove** | Remove a file or directory. By setting the `recursive` option to `true`, you can recursively remove nested directories. | | **rename** | Rename a file or directory. | | **sink** | Create a writable `Sink` for the specified `path`. | | **stat** | Get information about a file at `path`. | | **stream** | Create a readable `Stream` for the specified `path`. | | **symlink** | Create a symbolic link from `fromPath` to `toPath`. | | **truncate** | Truncate a file to a specified length. If the `length` is not specified, the file will be truncated to length `0`. | | **utimes** | Change the file system timestamps of the file at `path`. | | **watch** | Watch a directory or file for changes. | | **writeFile** | Write data to a file at `path`. | | **writeFileString** | Write a string to a file at `path`. | **Example** ## Mocking the File System In testing environments, you may want to mock the file system to avoid performing actual disk operations. The `FileSystem.layerNoop` provides a no-operation implementation of the `FileSystem` service. Most operations in `FileSystem.layerNoop` return a **failure** or a **defect** . However, you can override specific behaviors by passing an object to `FileSystem.layerNoop` to define custom return values for selected methods. **Example** --- title: Introduction to Effect Platform description: Build cross-platform applications with unified abstractions for Node.js, Deno, Bun, and browsers using @effect/platform. sidebar: label: Introduction order: 0 --- import { Aside, Tabs, TabItem, Badge } from "@astrojs/starlight/components" `@effect/platform` is a library for building platform-independent abstractions in environments such as Node.js, Deno, Bun, and browsers. With `@effect/platform`, you can integrate abstract services like or into your program. When assembling your final application, you can provide specific for the target platform using the corresponding packages: - `@effect/platform-node` for Node.js or Deno - `@effect/platform-bun` for Bun - `@effect/platform-browser` for browsers ### Stable Modules The following modules are stable and their documentation is available on this website: | Module | Description | Status | | --------------------------------------------------- | ---------------------------------------------------------- | ----------------------------------------- | | | Provides a way to interact with the command line. | | | | A module for file system operations. | | | | Manages key-value pairs for data storage. | | | | Utilities for working with file paths. | | | | Log messages to a file using the FileSystem APIs. | | | | Run your program with built-in error handling and logging. | | | | Tools for terminal interaction. | | ### Unstable Modules Some modules in `@effect/platform` are still in development or marked as experimental. These features are subject to change. | Module | Description | Status | | -------------------------------------------------------------------------------------------------- | ----------------------------------------------- | ------------------------------------------- | | | Provide a declarative way to define HTTP APIs. | | | | A client for making HTTP requests. | | | | A server for handling HTTP requests. | | | | A module for socket-based communication. | | | | A module for running tasks in separate workers. | | For the most up-to-date documentation and details, please refer to the official of the package. ## Installation To install the **beta** version: ## Getting Started with Cross-Platform Programming Here's a basic example using the `Path` module to create a file path, which can run across different environments: **Example** ### Running the Program in Node.js or Deno First, install the Node.js-specific package: Update the program to load the Node.js-specific context: **Example** Finally, run the program in Node.js using `tsx`, or directly in Deno: ### Running the Program in Bun To run the same program in Bun, first install the Bun-specific package: Update the program to use the Bun-specific context: **Example** Run the program in Bun: --- title: KeyValueStore description: Manage key-value pairs with asynchronous, consistent storage, supporting in-memory, file system, and schema-based implementations. sidebar: order: 3 --- The `@effect/platform/KeyValueStore` module provides a robust and effectful interface for managing key-value pairs. It supports asynchronous operations, ensuring data integrity and consistency, and includes built-in implementations for in-memory, file system-based, and schema-validated stores. ## Basic Usage The module exposes a single , `KeyValueStore`, which acts as the gateway for interacting with the store. **Example** The `KeyValueStore` interface includes the following operations: | Operation | Description | | -------------------- | -------------------------------------------------------------------- | | **get** | Returns the value as `string` of the specified key if it exists. | | **getUint8Array** | Returns the value as `Uint8Array` of the specified key if it exists. | | **set** | Sets the value of the specified key. | | **remove** | Removes the specified key. | | **clear** | Removes all entries. | | **size** | Returns the number of entries. | | **modify** | Updates the value of the specified key if it exists. | | **modifyUint8Array** | Updates the value of the specified key if it exists. | | **has** | Check if a key exists. | | **isEmpty** | Check if the store is empty. | | **forSchema** | Create a `SchemaStore` for the specified schema. | **Example** ## Built-in Implementations The module includes two built-in implementations of the `KeyValueStore` interface. Both are provided as that you can inject into your effectful programs. | Implementation | Description | | --------------------- | ------------------------------------------------------------------------------------------------------- | | **In-Memory Store** | `layerMemory` provides a simple, in-memory key-value store, ideal for lightweight or testing scenarios. | | **File System Store** | `layerFileSystem` offers a file-based store for persistent storage needs. | ## Working with Non-String Values By default, `KeyValueStore` works with `string` and `Uint8Array` values. To store other types such as objects, numbers, or booleans, use the `forSchema` method to create a `SchemaStore`. A `SchemaStore` uses a to validate and convert values. Internally, it serializes data using `JSON.stringify` and deserializes it with `JSON.parse`. **Example** --- title: Path description: Perform file path operations such as joining, resolving, and normalizing across platforms. sidebar: order: 4 --- The `@effect/platform/Path` module provides a set of operations for working with file paths. ## Basic Usage The module provides a single `Path` , which acts as the gateway for interacting with paths. **Example** The `Path` interface includes the following operations: | Operation | Description | | -------------------- | -------------------------------------------------------------------------- | | **basename** | Returns the last part of a path, optionally removing a given suffix. | | **dirname** | Returns the directory part of a path. | | **extname** | Returns the file extension from a path. | | **format** | Formats a path object into a path string. | | **fromFileUrl** | Converts a file URL to a path. | | **isAbsolute** | Checks if a path is absolute. | | **join** | Joins multiple path segments into one. | | **normalize** | Normalizes a path by resolving `.` and `..` segments. | | **parse** | Parses a path string into an object with its segments. | | **relative** | Computes the relative path from one path to another. | | **resolve** | Resolves a sequence of paths to an absolute path. | | **sep** | Returns the platform-specific path segment separator . | | **toFileUrl** | Converts a path to a file URL. | | **toNamespacedPath** | Converts a path to a namespaced path . | **Example** --- title: PlatformLogger description: Log messages to a file using the FileSystem APIs. sidebar: order: 6 --- Effect's logging system generally writes messages to the console by default. However, you might prefer to store logs in a file for easier debugging or archiving. The `PlatformLogger.toFile` function creates a logger that sends log messages to a file on disk. ### toFile Creates a new logger from an existing string-based logger, writing its output to the specified file. If you include a `batchWindow` duration when calling `toFile`, logs are batched for that period before being written. This can reduce overhead if your application produces many log entries. Without a `batchWindow`, logs are written as they arrive. Note that `toFile` returns an `Effect` that may fail with a `PlatformError` if the file cannot be opened or written to. Be sure to handle this possibility if you need to react to file I/O issues. **Example** This logger requires a `FileSystem` implementation to open and write to the file. For Node.js, you can use `NodeFileSystem.layer`. In the following example, logs are written to both the console and a file. The console uses the pretty logger, while the file uses the logfmt format. **Example** --- title: Runtime description: Run your program with built-in error handling and logging. sidebar: order: 6 --- ## Running Your Main Program with runMain `runMain` helps you execute a main effect with built-in error handling, logging, and signal management. You can concentrate on your effect while `runMain` looks after finalizing resources, logging errors, and setting exit codes. - **Exit Codes** If your effect fails or is interrupted, `runMain` assigns a suitable exit code . - **Logs** By default, it records errors. This can be turned off if needed. - **Pretty Logging** By default, error messages are recorded using a "pretty" format. You can switch this off when required. - **Interrupt Handling** If the application receives `SIGINT` or a similar signal, `runMain` will interrupt the effect and still run any necessary teardown steps. - **Teardown Logic** You can rely on the default teardown or define your own. The default sets an exit code of `1` for a non-interrupted failure. ### Usage Options When calling `runMain`, pass in a configuration object with these fields : - `disableErrorReporting`: If `true`, errors are not automatically logged. - `disablePrettyLogger`: If `true`, it avoids adding the "pretty" logger. - `teardown`: Provide a custom function for finalizing the program. If missing, the default sets exit code `1` for a non-interrupted failure. **Example** **Example** **Example** **Example** **Example** --- title: Terminal description: Interact with standard input and output to read user input and display messages on the terminal. sidebar: order: 7 --- The `@effect/platform/Terminal` module provides an abstraction for interacting with standard input and output, including reading user input and displaying messages on the terminal. ## Basic Usage The module provides a single `Terminal` , which serves as the entry point to reading from and writing to standard input and standard output. **Example** ## Writing to standard output **Example** ## Reading from standard input **Example** ## Example: Number guessing game This example demonstrates how to create a complete number-guessing game by reading input from the terminal and providing feedback to the user. The game continues until the user guesses the correct number. **Example** --- title: Default Services description: Learn about the default services in Effect, including Clock, Console, Random, ConfigProvider, and Tracer, and how they are automatically provided for your programs. sidebar: order: 1 --- Effect comes equipped with five pre-built services: When we employ these services, there's no need to explicitly provide their implementations. Effect automatically supplies live versions of these services to our effects, sparing us from manual setup. **Example** As you can observe, even if our program utilizes both `Clock` and `Console`, the `Requirements` parameter, representing the services required for the effect to execute, remains set to `never`. Effect takes care of handling these services seamlessly for us. ## Overriding Default Services Sometimes, you might need to replace the default services with custom implementations. Effect provides built-in utilities to override these services using `Effect.with` and `Effect.withScoped`. - `Effect.with`: Overrides a service for the duration of the effect. - `Effect.withScoped`: Overrides a service within a scope and restores the original service afterward. | Function | Description | | --------------------------------- | ------------------------------------------------------------------------------ | | `Effect.withClock` | Executes an effect using a specific `Clock` service. | | `Effect.withClockScoped` | Temporarily overrides the `Clock` service and restores it when the scope ends. | | `Effect.withConfigProvider` | Executes an effect using a specific `ConfigProvider` service. | | `Effect.withConfigProviderScoped` | Temporarily overrides the `ConfigProvider` service within a scope. | | `Effect.withConsole` | Executes an effect using a specific `Console` service. | | `Effect.withConsoleScoped` | Temporarily overrides the `Console` service within a scope. | | `Effect.withRandom` | Executes an effect using a specific `Random` service. | | `Effect.withRandomScoped` | Temporarily overrides the `Random` service within a scope. | | `Effect.withTracer` | Executes an effect using a specific `Tracer` service. | | `Effect.withTracerScoped` | Temporarily overrides the `Tracer` service within a scope. | **Example** --- title: Layer Memoization description: Learn how layer memoization optimizes performance in Effect by reusing layers and controlling their instantiation. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" Layer memoization allows a layer to be created once and used multiple times in the dependency graph. If we use the same layer twice: then the `L1` layer will be allocated only once. ## Memoization When Providing Globally One important feature of an Effect application is that layers are shared by default. This means that if the same layer is used twice, and if we provide the layer globally, the layer will only be allocated a single time. For every layer in our dependency graph, there is only one instance of it that is shared between all the layers that depend on it. **Example** For example, assume we have the three services `A`, `B`, and `C`. The implementation of both `B` and `C` is dependent on the `A` service: Although both `BLive` and `CLive` layers require the `ALive` layer, the `ALive` layer is instantiated only once. It is shared with both `BLive` and `CLive`. ## Acquiring a Fresh Version If we don't want to share a module, we should create a fresh, non-shared version of it through `Layer.fresh`. **Example** ## No Memoization When Providing Locally If we don't provide a layer globally but instead provide them locally, that layer doesn't support memoization by default. **Example** In the following example, we provided the `ALive` layer two times locally, and Effect doesn't memoize the construction of the `ALive` layer. So, it will be initialized two times: ## Manual Memoization We can memoize a layer manually using the `Layer.memoize` function. It will return a scoped effect that, if evaluated, will return the lazily computed result of this layer. **Example** --- title: Managing Layers description: Learn how to use layers in Effect to manage service dependencies and build efficient, clean dependency graphs for your applications. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" In the page, you learned how to create effects which depend on some service to be provided in order to execute, as well as how to provide that service to an effect. However, what if we have a service within our effect program that has dependencies on other services in order to be built? We want to avoid leaking these implementation details into the service interface. To represent the "dependency graph" of our program and manage these dependencies more effectively, we can utilize a powerful abstraction called "Layer". Layers act as **constructors for creating services**, allowing us to manage dependencies during construction rather than at the service level. This approach helps to keep our service interfaces clean and focused. Let's review some key concepts before diving into the details: | Concept | Description | | ----------- | ------------------------------------------------------------------------------------------------------------------------- | | **service** | A reusable component providing specific functionality, used across different parts of an application. | | **tag** | A unique identifier representing a **service**, allowing Effect to locate and use it. | | **context** | A collection storing services, functioning like a map with **tags** as keys and **services** as values. | | **layer** | An abstraction for constructing **services**, managing dependencies during construction rather than at the service level. | ## Designing the Dependency Graph Let's imagine that we are building a web application. We could imagine that the dependency graph for an application where we need to manage configuration, logging, and database access might look something like this: - The `Config` service provides application configuration. - The `Logger` service depends on the `Config` service. - The `Database` service depends on both the `Config` and `Logger` services. Our goal is to build the `Database` service along with its direct and indirect dependencies. This means we need to ensure that the `Config` service is available for both `Logger` and `Database`, and then provide these dependencies to the `Database` service. ## Avoiding Requirement Leakage When constructing the `Database` service, it's important to avoid exposing the dependencies on `Config` and `Logger` within the `Database` interface. You might be tempted to define the `Database` service as follows: **Example** Here, the `query` function of the `Database` service requires both `Config` and `Logger`. This design leaks implementation details, making the `Database` service aware of its dependencies, which complicates testing and makes it difficult to mock. To demonstrate the problem, let's create a test instance of the `Database` service: **Example** Because the `Database` service interface directly includes dependencies on `Config` and `Logger`, it forces any test setup to include these services, even if they're irrelevant to the test. This adds unnecessary complexity and makes it difficult to write simple, isolated unit tests. Instead of directly tying dependencies to the `Database` service interface, dependencies should be managed at the construction phase. We can use **layers** to properly construct the `Database` service and manage its dependencies without leaking details into the interface. ## Creating Layers The `Layer` type is structured as follows: A `Layer` represents a blueprint for constructing a `RequirementsOut` . It requires a `RequirementsIn` as input and may result in an error of type `Error` during the construction process. | Parameter | Description | | ----------------- | -------------------------------------------------------------------------- | | `RequirementsOut` | The service or resource to be created. | | `Error` | The type of error that might occur during the construction of the service. | | `RequirementsIn` | The dependencies required to construct the service. | By using layers, you can better organize your services, ensuring that their dependencies are clearly defined and separated from their implementation details. For simplicity, let's assume that we won't encounter any errors during the value construction . Now, let's determine how many layers we need to implement our dependency graph: | Layer | Dependencies | Type | | -------------- | ---------------------------------------------------------- | ------------------------------------------ | | `ConfigLive` | The `Config` service does not depend on any other services | `Layer` | | `LoggerLive` | The `Logger` service depends on the `Config` service | `Layer` | | `DatabaseLive` | The `Database` service depends on `Config` and `Logger` | `Layer` | When a service has multiple dependencies, they are represented as a **union type**. In our case, the `Database` service depends on both the `Config` and `Logger` services. Therefore, the type for the `DatabaseLive` layer will be: ### Config The `Config` service does not depend on any other services, so `ConfigLive` will be the simplest layer to implement. Just like in the page, we must create a tag for the service. And because the service has no dependencies, we can create the layer directly using the `Layer.succeed` constructor: Looking at the type of `ConfigLive` we can observe: - `RequirementsOut` is `Config`, indicating that constructing the layer will produce a `Config` service - `Error` is `never`, indicating that layer construction cannot fail - `RequirementsIn` is `never`, indicating that the layer has no dependencies Note that, to construct `ConfigLive`, we used the `Config.of` constructor. However, this is merely a helper to ensure correct type inference for the implementation. It's possible to skip this helper and construct the implementation directly as a simple object: ### Logger Now we can move on to the implementation of the `Logger` service, which depends on the `Config` service to retrieve some configuration. Just like we did in the page, we can yield the `Config` tag to "extract" the service from the context. Given that using the `Config` tag is an effectful operation, we use `Layer.effect` to create a layer from the resulting effect. Looking at the type of `LoggerLive`: we can observe that: - `RequirementsOut` is `Logger` - `Error` is `never`, indicating that layer construction cannot fail - `RequirementsIn` is `Config`, indicating that the layer has a requirement ### Database Finally, we can use our `Config` and `Logger` services to implement the `Database` service. Looking at the type of `DatabaseLive`: we can observe that the `RequirementsIn` type is `Config | Logger`, i.e., the `Database` service requires both `Config` and `Logger` services. ## Combining Layers Layers can be combined in two primary ways: **merging** and **composing**. ### Merging Layers Layers can be combined through merging using the `Layer.merge` function: When we merge two layers, the resulting layer: - requires all the services that both of them require . - produces all services that both of them produce . For example, in our web application above, we can merge our `ConfigLive` and `LoggerLive` layers into a single `AppConfigLive` layer, which retains the requirements of both layers and the outputs of both layers : ### Composing Layers Layers can be composed using the `Layer.provide` function: Sequential composition of layers implies that the output of one layer is supplied as the input for the inner layer, resulting in a single layer with the requirements of the outer layer and the output of the inner. Now we can compose the `AppConfigLive` layer with the `DatabaseLive` layer: We obtained a `MainLive` layer that produces the `Database` service: This layer is the fully resolved layer for our application. ### Merging and Composing Layers Let's say we want our `MainLive` layer to return both the `Config` and `Database` services. We can achieve this with `Layer.provideMerge`: ## Providing a Layer to an Effect Now that we have assembled the fully resolved `MainLive` for our application, we can provide it to our program to satisfy the program's requirements using `Effect.provide`: Note that the `runnable` requirements type is `never`, indicating that the program does not require any additional services to run. ## Converting a Layer to an Effect Sometimes your entire application might be a Layer, for example, an HTTP server. You can convert that layer to an effect with `Layer.launch`. It constructs the layer and keeps it alive until interrupted. **Example** ## Tapping The `Layer.tap` and `Layer.tapError` functions allow you to perform additional effects based on the success or failure of a layer. These operations do not modify the layer's signature but are useful for logging or performing side effects during layer construction. - `Layer.tap`: Executes a specified effect when the layer is successfully acquired. - `Layer.tapError`: Executes a specified effect when the layer fails to acquire. **Example** ## Error Handling When constructing layers, it is important to handle potential errors. The Effect library provides tools like `Layer.catchAll` and `Layer.orElse` to manage errors and define fallback layers in case of failure. ### catchAll The `Layer.catchAll` function allows you to recover from errors during layer construction by specifying a fallback layer. This can be useful for handling specific error cases and ensuring the application can continue with an alternative setup. **Example** ### orElse The `Layer.orElse` function provides a simpler way to fall back to an alternative layer if the initial layer fails. Unlike `Layer.catchAll`, it does not receive the error as input. Use this when you only need to provide a default layer without reacting to specific errors. **Example** ## Simplifying Service Definitions with Effect.Service The `Effect.Service` API provides a way to define a service in a single step, including its tag and layer. It also allows specifying dependencies upfront, making service construction more straightforward. ### Defining a Service with Dependencies The following example defines a `Cache` service that depends on a file system. **Example** ### Using the Generated Layers The `Effect.Service` API automatically generates layers for the service. | Layer | Description | | ---------------------------------- | --------------------------------------------------------------------------------- | | `Cache.Default` | Provides the `Cache` service with its dependencies already included. | | `Cache.DefaultWithoutDependencies` | Provides the `Cache` service but requires dependencies to be provided separately. | ### Accessing the Service A service created with `Effect.Service` can be accessed like any other Effect service. **Example** Since this example uses `Cache.Default`, it interacts with the real file system. If the file does not exist, it results in an error. ### Injecting Test Dependencies To test the program without depending on the real file system, we can inject a test file system using the `Cache.DefaultWithoutDependencies` layer. **Example** ### Mocking the Service Directly Alternatively, you can mock the `Cache` service itself instead of replacing its dependencies. **Example** ### Alternative Ways to Define a Service The `Effect.Service` API supports multiple ways to define a service: | Method | Description | | --------- | -------------------------------------------------- | | `succeed` | Provides a static implementation of the service. | | `sync` | Defines a service using a synchronous constructor. | | `effect` | Defines a service with an effectful constructor. | | `scoped` | Creates a service with lifecycle management. | **Example** This is the simplest way to define a service. It is useful when you want to provide a constant value for the service. **Example** **Example** The `Scoped.Default` layer does not require `Scope` as a dependency, since `Scoped` itself manages its lifecycle. ### Enabling Direct Method Access By setting `accessors: true`, you can call service methods directly using the service tag instead of first extracting the service. **Example** ### Effect.Service vs Context.Tag Both `Effect.Service` and `Context.Tag` are ways to model services in the Effect ecosystem. They serve similar purposes but target different use-cases. | Feature | Effect.Service | Context.Tag | | ----------------------------------- | ------------------------------------------------------- | ----------------------------------------- | | Tag creation | Generated for you | You declare the tag manually | | Default implementation | **Required** - supplied inline | **Optional** - can be supplied later | | Ready-made layers | Automatically generated | You build layers yourself | | Best suited for | Application code with a clear runtime implementation | Library code or dynamically-scoped values | | When no sensible default exists | Not ideal; you would still have to invent one | Preferred | **Key points** - **Less boilerplate:** `Effect.Service` is syntactic sugar over `Context.Tag` plus the accompanying layer and helpers. - **Default required:** A class that extends `Effect.Service` must declare **one** of the built-in constructors . This baseline implementation becomes part of `MyService.Default`, so any code that imports the service can run without providing extra layers. That is handy for app-level services where a sensible runtime implementation exists . If your service is inherently contextual or you are writing a library that should not assume an implementation, prefer `Context.Tag`: you publish only the tag and let callers supply the layer that makes sense in their environment. - **The class _is_ the tag:** When you create a class with `extends Effect.Service`, the class constructor itself acts as the tag. You can provide alternate implementations by supplying a value for that class when wiring layers: --- title: Managing Services description: Learn how to manage reusable services in Effect, handle dependencies efficiently, and ensure clean, decoupled architecture in your applications. sidebar: order: 0 --- import { Aside, Tabs, TabItem } from "@astrojs/starlight/components" In the context of programming, a **service** refers to a reusable component or functionality that can be used by different parts of an application. Services are designed to provide specific capabilities and can be shared across multiple modules or components. Services often encapsulate common tasks or operations that are needed by different parts of an application. They can handle complex operations, interact with external systems or APIs, manage data, or perform other specialized tasks. Services are typically designed to be modular and decoupled from the rest of the application. This allows them to be easily maintained, tested, and replaced without affecting the overall functionality of the application. When diving into services and their integration in application development, it helps to start from the basic principles of function management and dependency handling without relying on advanced constructs. Imagine having to manually pass a service around to every function that needs it: This approach becomes cumbersome and unmanageable as your application grows, with services needing to be passed through multiple layers of functions. To streamline this, you might consider using an environment object that bundles various services: However, this introduces a new complexity: you must ensure that the environment is correctly set up with all necessary services before it's used, which can lead to tightly coupled code and makes functional composition and testing more difficult. ## Managing Services with Effect The Effect library simplifies managing these dependencies by leveraging the type system. Instead of manually passing services or environment objects around, Effect allows you to declare service dependencies directly in the function's type signature using the `Requirements` parameter in the `Effect` type: This is how it works in practice when using Effect: **Dependency Declaration**: You specify what services a function needs directly in its type, pushing the complexity of dependency management into the type system. **Service Provision**: `Effect.provideService` is used to make a service implementation available to the functions that need it. By providing services at the start, you ensure that all parts of your application have consistent access to the required services, thus maintaining a clean and decoupled architecture. This approach abstracts away manual service handling, letting developers focus on business logic while the compiler ensures all dependencies are correctly managed. It also makes code more maintainable and scalable. Let's walk through managing services in Effect step by step: 1. **Creating a Service**: Define a service with its unique functionality and interface. 2. **Using the Service**: Access and utilize the service within your application’s functions. 3. **Providing a Service Implementation**: Supply an actual implementation of the service to fulfill the declared requirements. ## How It Works Up to this point, our examples with the Effect framework have dealt with effects that operate independently of external services. This means the `Requirements` parameter in our `Effect` type signature has been set to `never`, indicating no dependencies. However, real-world applications often need effects that rely on specific services to function correctly. These services are managed and accessed through a construct known as `Context`. The `Context` serves as a repository or container for all services an effect may require. It acts like a store that maintains these services, allowing various parts of your application to access and use them as needed. The services stored within the `Context` are directly reflected in the `Requirements` parameter of the `Effect` type. Each service within the `Context` is identified by a unique "tag," which is essentially a unique identifier for the service. When an effect needs to use a specific service, the service's tag is included in the `Requirements` type parameter. ## Creating a Service To create a new service, you need two things: 1. A unique **identifier**. 2. A **type** describing the possible operations of the service. **Example** Let's create a service for generating random numbers. 1. **Identifier**. We'll use the string `"MyRandomService"` as the unique identifier. 2. **Type**. The service type will have a single operation called `next` that returns a random number. The exported `Random` value is known as a **tag** in Effect. It acts as a representation of the service and allows Effect to locate and use this service at runtime. The service will be stored in a collection called `Context`, which can be thought of as a `Map` where the keys are tags and the values are services: Let's summarize the concepts we've covered so far: | Concept | Description | | ----------- | ------------------------------------------------------------------------------------------------------ | | **service** | A reusable component providing specific functionality, used across different parts of an application. | | **tag** | A unique identifier representing a **service**, allowing Effect to locate and use it. | | **context** | A collection storing service, functioning like a map with **tags** as keys and **services** as values. | ## Using the Service Now that we have our service tag defined, let's see how we can use it by building a simple program. **Example** In the code above, we can observe that we are able to yield the `Random` tag as if it were an effect itself. This allows us to access the `next` operation of the service. In the code above, we can observe that we are able to flat-map over the `Random` tag as if it were an effect itself. This allows us to access the `next` operation of the service within the `Effect.andThen` callback. It's worth noting that the type of the `program` variable includes `Random` in the `Requirements` type parameter: This indicates that our program requires the `Random` service to be provided in order to execute successfully. If we attempt to execute the effect without providing the necessary service we will encounter a type-checking error: **Example** To resolve this error and successfully execute the program, we need to provide an actual implementation of the `Random` service. In the next section, we will explore how to implement and provide the `Random` service to our program, enabling us to run it successfully. ## Providing a Service Implementation In order to provide an actual implementation of the `Random` service, we can utilize the `Effect.provideService` function. **Example** In the code above, we provide the `program` we defined earlier with an implementation of the `Random` service. We use the `Effect.provideService` function to associate the `Random` tag with its implementation, an object with a `next` operation that generates a random number. Notice that the `Requirements` type parameter of the `runnable` effect is now `never`. This indicates that the effect no longer requires any service to be provided. With the implementation of the `Random` service in place, we are able to run the program without any further requirements. ## Extracting the Service Type To retrieve the service type from a tag, use the `Context.Tag.Service` utility type. **Example** ## Using Multiple Services When we require the usage of more than one service, the process remains similar to what we've learned in defining a service, repeated for each service needed. **Example** Let's examine an example where we need two services, namely `Random` and `Logger`: The `program` effect now has a `Requirements` type parameter of `Random | Logger`: indicating that it requires both the `Random` and `Logger` services to be provided. To execute the `program`, we need to provide implementations for both services: **Example** Alternatively, instead of calling `provideService` multiple times, we can combine the service implementations into a single `Context` and then provide the entire context using the `Effect.provide` function: **Example** ## Optional Services There are situations where we may want to access a service implementation only if it is available. In such cases, we can use the `Effect.serviceOption` function to handle this scenario. The `Effect.serviceOption` function returns an implementation that is available only if it is actually provided before executing this effect. To represent this optionality it returns an of the implementation. **Example** To determine what action to take, we can use the `Option.isNone` function provided by the Option module. This function allows us to check if the service is available or not by returning `true` when the service is not available. In the code above, we can observe that the `Requirements` type parameter of the `program` effect is `never`, even though we are working with a service. This allows us to access something from the context only if it is actually provided before executing this effect. When we run the `program` effect without providing the `Random` service: We see that the log message contains `-1`, which is the default value we provided when the service was not available. However, if we provide the `Random` service implementation: We can observe that the log message now contains a random number generated by the `next` operation of the `Random` service. ## Handling Services with Dependencies Sometimes a service in your application may depend on other services. To maintain a clean architecture, it's important to manage these dependencies without making them explicit in the service interface. Instead, you can use **layers** to handle these dependencies during the service construction phase. **Example** Consider a scenario where multiple services depend on each other. In this case, the `Logger` service requires access to a configuration service . To handle these dependencies in a structured way and prevent them from leaking into the service interfaces, you can use the `Layer` abstraction. For more details on managing dependencies with layers, refer to the page. --- title: Introduction description: Common patterns for safe resource management sidebar: order: 0 --- In long-running applications, managing resources efficiently is essential, particularly when building large-scale systems. If resources like socket connections, database connections, or file descriptors are not properly managed, it can lead to resource leaks, which degrade application performance and reliability. Effect provides constructs that help ensure resources are properly managed and released, even in cases where exceptions occur. By ensuring that every time a resource is acquired, there is a corresponding mechanism to release it, Effect simplifies the process of resource management in your application. ## Finalization In many programming languages, the `try` / `finally` construct ensures that cleanup code runs regardless of whether an operation succeeds or fails. Effect provides similar functionality through `Effect.ensuring`, `Effect.onExit`, and `Effect.onError`. ### ensuring The `Effect.ensuring` function guarantees that a finalizer effect runs whether the main effect succeeds, fails, or is interrupted. This is useful for performing cleanup actions such as closing file handles, logging messages, or releasing locks. If you need access to the effect's result, consider using . **Example** ### onExit `Effect.onExit` allows you to run a cleanup effect after the main effect completes, receiving an value that describes the outcome. - If the effect succeeds, the `Exit` holds the success value. - If it fails, the `Exit` includes the error or failure cause. - If it is interrupted, the `Exit` reflects that interruption. The cleanup step itself is uninterruptible, which can help manage resources in complex or high-concurrency cases. **Example** ### onError This function lets you attach a cleanup effect that runs whenever the calling effect fails, passing the cause of the failure to the cleanup effect. You can use it to perform actions such as logging, releasing resources, or applying additional recovery steps. The cleanup effect will also run if the failure is caused by interruption, and it is uninterruptible, so it always finishes once it starts. **Example** ## acquireUseRelease Many real-world operations involve working with resources that must be released when no longer needed, such as: - Database connections - File handles - Network requests Effect provides `Effect.acquireUseRelease`, which ensures that a resource is: 1. **Acquired** properly. 2. **Used** for its intended purpose. 3. **Released** even if an error occurs. **Syntax** **Example** --- title: Scope description: Learn how Effect simplifies resource management with Scopes, ensuring efficient cleanup and safe resource handling in long-running applications. sidebar: order: 1 --- import { Aside, Tabs, TabItem } from "@astrojs/starlight/components" The `Scope` data type is a core construct in Effect for managing resources in a safe and composable way. A scope represents the lifetime of one or more resources. When the scope is closed, all the resources within it are released, ensuring that no resources are leaked. Scopes also allow the addition of **finalizers**, which define how to release resources. With the `Scope` data type, you can: - **Add finalizers**: A finalizer specifies the cleanup logic for a resource. - **Close the scope**: When the scope is closed, all resources are released, and the finalizers are executed. **Example** In the above example, finalizers are added to the scope, and when the scope is closed, the finalizers are **executed in the reverse order**. This reverse order is important because it ensures that resources are released in the correct sequence. For instance, if you acquire a network connection and then access a file on a remote server, the file must be closed before the network connection to avoid errors. ## addFinalizer The `Effect.addFinalizer` function is a high-level API that allows you to add finalizers to the scope of an effect. A finalizer is a piece of code that is guaranteed to run when the associated scope is closed. The behavior of the finalizer can vary based on the value, which represents how the scope was closed—whether successfully or with an error. **Example** In this example, we use `Effect.addFinalizer` to add a finalizer that logs the exit state after the scope is closed. The finalizer will execute when the effect finishes, and it will log whether the effect completed successfully or failed. The type signature: shows that the workflow requires a `Scope` to run. You can provide this `Scope` using the `Effect.scoped` function, which creates a new scope, runs the effect within it, and ensures the finalizers are executed when the scope is closed. **Example** In this case, the finalizer is executed even when the effect fails. The log output reflects that the finalizer runs after the failure, and it logs the failure details. **Example** ) This example shows how a finalizer behaves when the effect is interrupted. The finalizer runs after the interruption, and the exit status reflects that the effect was stopped mid-execution. ## Manually Create and Close Scopes When you're working with multiple scoped resources within a single operation, it's important to understand how their scopes interact. By default, these scopes are merged into one, but you can have more fine-grained control over when each scope is closed by manually creating and closing them. Let's start by looking at how scopes are merged by default: **Example** In this case, the scopes of `task1` and `task2` are merged into a single scope, and when the program is run, it outputs the tasks and their finalizers in a specific order. If you want more control over when each scope is closed, you can manually create and close them: **Example** In this example, we create two separate scopes, `scope1` and `scope2`, and extend the scope of each task into its respective scope. When you run the program, it outputs the tasks and their finalizers in a different order. You might wonder what happens when a scope is closed, but a task within that scope hasn't completed yet. The key point to note is that the scope closing doesn't force the task to be interrupted. **Example** ## Defining Resources ### acquireRelease The `Effect.acquireRelease` function allows you to define resources that are acquired and safely released when they are no longer needed. This is useful for managing resources such as file handles, database connections, or network sockets. To use `Effect.acquireRelease`, you need to define two actions: 1. **Acquiring the Resource**: An effect describing the acquisition of the resource, e.g., opening a file or establishing a database connection. 2. **Releasing the Resource**: The clean-up effect that ensures the resource is properly released, e.g., closing the file or the connection. The acquisition process is **uninterruptible** to ensure that partial resource acquisition doesn't leave your system in an inconsistent state. The `Effect.acquireRelease` function guarantees that once a resource is successfully acquired, its release step is always executed when the `Scope` is closed. **Example** In the code above, the `Effect.acquireRelease` function creates a resource workflow that requires a `Scope`: This means that the workflow needs a `Scope` to run, and the resource will automatically be released when the scope is closed. You can now use the resource by chaining operations using `Effect.andThen` or similar functions. We can continue working with the resource for as long as we want by using `Effect.andThen` or other Effect operators. For example, here's how we can read the contents: **Example** To ensure proper resource management, the `Scope` should be closed when you're done with the resource. The `Effect.scoped` function handles this for you by creating a `Scope`, running the effect, and then closing the `Scope` when the effect finishes. **Example** ### Example Pattern: Sequencing Operations In certain scenarios, you might need to perform a sequence of chained operations where the success of each operation depends on the previous one. However, if any of the operations fail, you would want to reverse the effects of all previous successful operations. This pattern is valuable when you need to ensure that either all operations succeed, or none of them have any effect at all. Let's go through an example of implementing this pattern. Suppose we want to create a "Workspace" in our application, which involves creating an S3 bucket, an ElasticSearch index, and a Database entry that relies on the previous two. To begin, we define the domain model for the required : - `S3` - `ElasticSearch` - `Database` Next, we define the three create actions and the overall transaction for the workspace. We then create simple service implementations to test the behavior of our Workspace code. To achieve this, we will utilize to construct test These layers will be able to handle various scenarios, including errors, which we can control using the `FailureCase` type. Let's examine the test results for the scenario where `FailureCase` is set to `undefined` : In this case, all operations succeed, and we see a successful result with `right`. Now, let's simulate a failure in the `Database`: The console output will be: You can observe that once the `Database` error occurs, there is a complete rollback that deletes the `ElasticSearch` index first and then the associated `S3` bucket. The result is a failure with `left)`. Let's now make the index creation fail instead: In this case, the console output will be: As expected, once the `ElasticSearch` index creation fails, there is a rollback that deletes the `S3` bucket. The result is a failure with `left)`. --- title: Introduction to Runtime description: Learn how Effect's runtime system executes concurrent programs, manages resources, and handles configuration with flexibility and efficiency. sidebar: label: Runtime order: 6 --- The `Runtime` data type represents a runtime system that can **execute effects**. To run an effect, `Effect`, we need a `Runtime` that contains the required resources, denoted by the `R` type parameter. A `Runtime` consists of three main components: - A value of type `Context` - A value of type `FiberRefs` - A value of type `RuntimeFlags` ## What is a Runtime System? When we write an Effect program, we construct an `Effect` using constructors and combinators. Essentially, we are creating a blueprint of a program. An `Effect` is merely a data structure that describes the execution of a concurrent program. It represents a tree-like structure that combines various primitives to define what the effect should do. However, this data structure itself does not perform any actions, it is solely a description of a concurrent program. To execute this program, the Effect runtime system comes into play. The `Runtime.run*` functions are responsible for taking this blueprint and executing it. When the runtime system runs an effect, it creates a root fiber, initializing it with: - The initial - The initial `FiberRefs` - The initial effect It then starts a loop, executing the instructions described by the `Effect` step by step. You can think of the runtime as a system that takes an and its associated context `Context` and produces an result. Runtime Systems have a lot of responsibilities: | Responsibility | Description | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------ | | **Executing the program** | The runtime must execute every step of the effect in a loop until the program completes. | | **Handling errors** | It handles both expected and unexpected errors that occur during execution. | | **Managing concurrency** | The runtime spawns new fibers when `Effect.fork` is called to handle concurrent operations. | | **Cooperative yielding** | It ensures fibers don't monopolize resources, yielding control when necessary. | | **Ensuring resource cleanup** | The runtime guarantees finalizers run properly to clean up resources when needed. | | **Handling async callbacks** | The runtime deals with asynchronous operations transparently, allowing you to write async and sync code uniformly. | ## The Default Runtime When we use like `Effect.runPromise` or `Effect.runFork`, we are actually using the **default runtime** without explicitly mentioning it. These functions are designed as convenient shortcuts for executing our effects using the default runtime. Each of the `Effect.run*` functions internally calls the corresponding `Runtime.run*` function, passing in the default runtime. For example, `Effect.runPromise` is just an alias for `Runtime.runPromise`. Both of the following executions are functionally equivalent: **Example** In both cases, the program runs using the default runtime, producing the same output. The default runtime includes: - An empty - A set of `FiberRefs` that include the - A default configuration for `RuntimeFlags` that enables `Interruption` and `CooperativeYielding` In most scenarios, using the default runtime is sufficient for effect execution. However, there are cases where it's helpful to create a custom runtime, particularly when you need to reuse specific configurations or contexts. For example, in a React app or when executing operations on a server in response to API requests, you might create a `Runtime` by initializing a `Layer`. This allows you to maintain a consistent context across different execution boundaries. ## Locally Scoped Runtime Configuration In Effect, runtime configurations are typically **inherited** from their parent workflows. This means that when we access a runtime configuration or obtain a runtime inside a workflow, we are essentially using the configuration of the parent workflow. However, there are cases where we want to temporarily **override the runtime configuration for a specific part** of our code. This concept is known as locally scoped runtime configuration. Once the execution of that code region is completed, the runtime configuration **reverts** to its original settings. To achieve this, we make use of the `Effect.provide` function, which allow us to provide a new runtime configuration to a specific section of our code. **Example** In this example, we create a simple logger using `Logger.replace`, which replaces the default logger with a custom one that logs messages without timestamps or levels. We then use `Effect.provide` to apply this custom logger to the program. To ensure that the runtime configuration is only applied to a specific part of an Effect application, we should provide the configuration layer exclusively to that particular section. **Example** In this example, we demonstrate how to apply a custom logger configuration only to a specific section of the program. The default logger is used for most of the program, but when we apply the `Effect.provide` call, it overrides the logger within that specific nested block. After that, the configuration reverts to its original state. ## ManagedRuntime When developing an Effect application and using `Effect.run*` functions to execute it, the application is automatically run using the default runtime behind the scenes. While it’s possible to adjust specific parts of the application by providing locally scoped configuration layers using `Effect.provide`, there are scenarios where you might want to **customize the runtime configuration for the entire application** from the top level. In these cases, you can create a top-level runtime by converting a configuration layer into a runtime using the `ManagedRuntime.make` constructor. **Example** In this example, we first create a custom configuration layer called `appLayer`, which replaces the default logger with a simple one that logs messages to the console. Next, we use `ManagedRuntime.make` to turn this configuration layer into a runtime. ### Effect.Tag When working with runtimes that you pass around, `Effect.Tag` can help simplify the access to services. It lets you define a new tag and embed the service shape directly into the static properties of the tag class. **Example** In this setup, the fields of the service are turned into static properties of the `Notifications` class, making it easier to access them. This allows you to interact with the service directly: **Example** In this example, the `action` effect depends on the `Notifications` service. This approach allows you to reference services without manually passing them around. Later, you can create a `Layer` that provides the `Notifications` service and build a `ManagedRuntime` with that layer to ensure the service is available where needed. ### Integrations The `ManagedRuntime` simplifies the integration of services and layers with other frameworks or tools, particularly in environments where Effect is not the primary framework and access to the main entry point is restricted. For example, in environments like React or other frameworks where you have limited control over the main application entry point, `ManagedRuntime` helps manage the lifecycle of services. Here's how to manage a service's lifecycle within an external framework: **Example** --- title: Built-In Schedules description: Explore built-in scheduling patterns in Effect for efficient timed repetitions and delays. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" To demonstrate the functionality of different schedules, we will use the following helper function that logs each repetition along with the corresponding delay in milliseconds, formatted as: **Helper** ## Infinite and Fixed Repeats ### forever A schedule that repeats indefinitely, producing the number of recurrences each time it runs. **Example** ### once A schedule that recurs only once. **Example** ### recurs A schedule that repeats a specified number of times, producing the number of recurrences each time it runs. **Example** ## Recurring at specific intervals You can define schedules that control the time between executions. The difference between `spaced` and `fixed` schedules lies in how the interval is measured: - `spaced` delays each repetition from the **end** of the previous one. - `fixed` ensures repetitions occur at **regular intervals**, regardless of execution time. ### spaced A schedule that repeats indefinitely, each repetition spaced the specified duration from the last run. It returns the number of recurrences each time it runs. **Example** The first delay is approximately 100 milliseconds, as the initial execution is not affected by the schedule. Subsequent delays are approximately 200 milliseconds apart, demonstrating the effect of the `spaced` schedule. ### fixed A schedule that recurs at fixed intervals. It returns the number of recurrences each time it runs. If the action run between updates takes longer than the interval, then the action will be run immediately, but re-runs will not "pile up". **Example** ## Increasing Delays Between Executions ### exponential A schedule that recurs using exponential backoff, with each delay increasing exponentially. Returns the current duration between recurrences. **Example** ### fibonacci A schedule that always recurs, increasing delays by summing the preceding two delays . Returns the current duration between recurrences. **Example** --- title: Cron description: Explore cron scheduling in Effect for executing actions at specific times and intervals. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" The Cron module lets you define schedules in a style similar to . It also supports partial constraints , time zone awareness through the module, and robust error handling. This module helps you: - **Create** a `Cron` instance from individual parts. - **Parse and validate** cron expressions. - **Match** existing dates to see if they satisfy a given cron schedule. - **Find** the next occurrence of a schedule after a given date. - **Iterate** over future dates that match a schedule. - **Convert** a `Cron` instance to a `Schedule` for use in effectful programs. ## Creating a Cron You can define a cron schedule by specifying numeric constraints for seconds, minutes, hours, days, months, and weekdays. The `make` function requires you to define all fields representing the schedule's constraints. **Example** - `seconds`, `minutes`, and `hours`: Define the time of day. - `days` and `months`: Specify which calendar days and months are valid. - `weekdays`: Restrict the schedule to specific days of the week. - `tz`: Optionally define the time zone for the schedule. If any field is left empty , it is treated as having "no constraints," allowing any valid value for that part of the date. ## Parsing Cron Expressions Instead of manually constructing a `Cron`, you can use UNIX-like cron strings and parse them with `parse` or `unsafeParse`. ### parse The `parse` function safely parses a cron string into a `Cron` instance. It returns an , which will contain either the parsed `Cron` or a parsing error. **Example** ### unsafeParse The `unsafeParse` function works like , but instead of returning an , it throws an exception if the input is invalid. **Example** ## Checking Dates with match The `match` function allows you to determine if a given `Date` ) satisfies the constraints of a cron schedule. If the date meets the schedule's conditions, `match` returns `true`. Otherwise, it returns `false`. **Example** ## Finding the Next Run The `next` function determines the next date that satisfies a given cron schedule, starting from a specified date. If no starting date is provided, the current time is used as the starting point. If `next` cannot find a matching date within a predefined number of iterations, it throws an error to prevent infinite loops. **Example** ## Iterating Over Future Dates To generate multiple future dates that match a cron schedule, you can use the `sequence` function. This function provides an infinite iterator of matching dates, starting from a specified date. **Example** ## Converting to Schedule The Schedule module allows you to define recurring behaviors, such as retries or periodic events. The `cron` function bridges the `Cron` module with the Schedule module, enabling you to create schedules based on cron expressions or `Cron` instances. ### cron The `Schedule.cron` function generates a that triggers at the start of each interval defined by the provided cron expression or `Cron` instance. When triggered, the schedule produces a tuple `` representing the timestamps of the cron interval window. **Example** --- title: Examples description: Explore practical examples for scheduling, retries, timeouts, and periodic task execution in Effect. sidebar: order: 5 --- These examples demonstrate different approaches to handling timeouts, retries, and periodic execution using Effect. Each scenario ensures that the application remains responsive and resilient to failures while adapting dynamically to various conditions. ## Handling Timeouts and Retries for API Calls When calling third-party APIs, it is often necessary to enforce timeouts and implement retry mechanisms to handle transient failures. In this example, the API call retries up to two times in case of failure and will be interrupted if it takes longer than 4 seconds. **Example** ## Retrying API Calls Based on Specific Errors Sometimes, retries should only happen for certain error conditions. For example, if an API call fails with a `401 Unauthorized` response, retrying might make sense, while a `404 Not Found` error should not trigger a retry. **Example** ## Retrying with Dynamic Delays Based on Error Information Some API errors, such as `429 Too Many Requests`, include a `Retry-After` header that specifies how long to wait before retrying. Instead of using a fixed delay, we can dynamically adjust the retry interval based on this value. **Example** This approach ensures that the retry delay adapts dynamically to the server's response, preventing unnecessary retries while respecting the provided `Retry-After` value. ## Running Periodic Tasks Until Another Task Completes There are cases where we need to repeatedly perform an action at fixed intervals until another longer-running task finishes. This pattern is common in polling mechanisms or periodic logging. **Example** --- title: Introduction description: Learn the fundamentals of scheduling in Effect, including composable recurrence patterns and handling retries and repetitions. sidebar: order: 0 --- # Scheduling Scheduling is an important concept in Effect that allows you to define recurring effectful operations. It involves the use of the `Schedule` type, which is an immutable value that describes a scheduled pattern for executing effects. The `Schedule` type is structured as follows: A schedule operates by consuming values of type `In` and producing values of type `Out`. It determines when to halt or continue the execution based on input values and its internal state. The inclusion of a `Requirements` parameter allows the schedule to leverage additional services or resources as needed. Schedules are defined as a collection of intervals spread out over time. Each interval represents a window during which the recurrence of an effect is possible. ## Retrying and Repetition In the realm of scheduling, there are two related concepts: and . While they share the same underlying idea, they differ in their focus. Retrying aims to handle failures by executing an effect again, while repetition focuses on executing an effect repeatedly to achieve a desired outcome. When using schedules for retrying or repetition, each interval's starting boundary determines when the effect will be executed again. For example, in retrying, if an error occurs, the schedule defines when the effect should be retried. ## Composability of Schedules Schedules are composable, meaning you can combine simple schedules to create more complex recurrence patterns. Operators like `Schedule.union` or `Schedule.intersect` allow you to build sophisticated schedules by combining and modifying existing ones. This flexibility enables you to tailor the scheduling behavior to meet specific requirements. --- title: Repetition description: Explore repetition in Effect for executing actions multiple times with control over retries, failures, and conditions. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" Repetition is a common requirement when working with effects in software development. It allows us to perform an effect multiple times according to a specific repetition policy. ## repeat The `Effect.repeat` function returns a new effect that repeats the given effect according to a specified schedule or until the first failure. **Example** **Example** ### Skipping First Execution If you want to avoid the first execution and only run the action according to a schedule, you can use `Effect.schedule`. This allows the effect to skip the initial run and follow the defined repeat policy. **Example** ## repeatN The `repeatN` function returns a new effect that repeats the specified effect a given number of times or until the first failure. The repeats are in addition to the initial execution, so `Effect.repeatN` executes `action` once initially and then repeats it one additional time if it succeeds. **Example** ## repeatOrElse The `repeatOrElse` function returns a new effect that repeats the specified effect according to the given schedule or until the first failure. When a failure occurs, the failure value and schedule output are passed to a specified handler. Scheduled recurrences are in addition to the initial execution, so `Effect.repeat` executes `action` once initially and then repeats it an additional time if it succeeds. **Example** ## Repeating Based on a Condition You can control the repetition of an effect by a condition using either a `while` or `until` option, allowing for dynamic control based on runtime outcomes. **Example** --- title: Schedule Combinators description: Learn how to combine and customize schedules in Effect to create complex recurrence patterns, including union, intersection, sequencing, and more. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" Schedules define stateful, possibly effectful, recurring schedules of events, and compose in a variety of ways. Combinators allow us to take schedules and combine them together to get other schedules. To demonstrate the functionality of different schedules, we will use the following helper function that logs each repetition along with the corresponding delay in milliseconds, formatted as: **Helper** ## Composition Schedules can be composed in different ways: | Mode | Description | | ---------------- | -------------------------------------------------------------------------------------------------- | | **Union** | Combines two schedules and recurs if either schedule wants to continue, using the shorter delay. | | **Intersection** | Combines two schedules and recurs only if both schedules want to continue, using the longer delay. | | **Sequencing** | Combines two schedules by running the first one fully, then switching to the second. | ### Union Combines two schedules and recurs if either schedule wants to continue, using the shorter delay. **Example** The `Schedule.union` operator selects the shortest delay at each step, so when combining an exponential schedule with a spaced interval, the initial recurrences will follow the exponential backoff, then settle into the spaced interval once the delays exceed that value. ### Intersection Combines two schedules and recurs only if both schedules want to continue, using the longer delay. **Example** The `Schedule.intersect` operator enforces both schedules' constraints. In this example, the schedule follows an exponential backoff but stops after 5 recurrences due to the `Schedule.recurs` limit. ### Sequencing Combines two schedules by running the first one fully, then switching to the second. **Example** The first schedule runs until completion, after which the second schedule takes over. In this example, the effect initially executes 5 times with no delay, then continues every 1 second. ## Adding Randomness to Retry Delays The `Schedule.jittered` combinator modifies a schedule by applying a random delay within a specified range. When a resource is out of service due to overload or contention, retrying and backing off doesn't help us. If all failed API calls are backed off to the same point of time, they cause another overload or contention. Jitter adds some amount of randomness to the delay of the schedule. This helps us to avoid ending up accidentally synchronizing and taking the service down by accident. suggests that `Schedule.jittered` is an effective way to introduce randomness in retries. **Example** The `Schedule.jittered` combinator introduces randomness to delays within a range. For example, applying jitter to an exponential backoff ensures that each retry occurs at a slightly different time, reducing the risk of overwhelming the system. ## Controlling Repetitions with Filters You can use `Schedule.whileInput` or `Schedule.whileOutput` to limit how long a schedule continues based on conditions applied to its input or output. **Example** `Schedule.whileOutput` filters repetitions based on the output of the schedule. In this example, the schedule stops once the output exceeds `2`, even though `Schedule.recurs` allows up to 5 repetitions. ## Adjusting Delays Based on Output The `Schedule.modifyDelay` combinator allows you to dynamically change the delay of a schedule based on the number of repetitions or other output conditions. **Example** The delay modification applies dynamically during execution. In this example, the first three repetitions follow the original `1-second` spacing. After that, the delay drops to `100 milliseconds`, making subsequent repetitions occur more frequently. ## Tapping `Schedule.tapInput` and `Schedule.tapOutput` allow you to perform additional effectful operations on a schedule's input or output without modifying its behavior. **Example** `Schedule.tapOutput` runs an effect before each recurrence, using the schedule's current output as input. This can be useful for logging, debugging, or triggering side effects. --- title: Advanced Usage description: Learn advanced techniques for defining and extending data schemas, including recursive and mutually recursive types, optional fields, branded types, and schema transformations. sidebar: order: 5 --- import { Aside } from "@astrojs/starlight/components" ## Declaring New Data Types ### Primitive Data Types To declare a schema for a primitive data type, such as `File`, you can use the `Schema.declare` function along with a type guard. **Example** To enhance the default error message, you can add annotations, particularly the `identifier`, `title`, and `description` annotations . These annotations will be utilized by the messaging system to return more meaningful messages. - **Identifier**: a unique name for the schema - **Title**: a brief, descriptive title - **Description**: a detailed explanation of the schema's purpose **Example** ### Type Constructors Type constructors are generic types that take one or more types as arguments and return a new type. To define a schema for a type constructor, you can use the `Schema.declare` function. **Example** ### Adding Compilers Annotations When defining a new data type, some compilers like or may not know how to handle the new type. This can result in an error, as the compiler may lack the necessary information for generating instances or producing readable output: **Example** In the above example, attempting to generate arbitrary values for the `FileFromSelf` schema fails because the compiler lacks necessary annotations. To resolve this, you need to provide annotations for generating arbitrary data: **Example** For more details on how to add annotations for the Arbitrary compiler, refer to the documentation. ## Branded types TypeScript's type system is structural, which means that any two types that are structurally equivalent are considered the same. This can cause issues when types that are semantically different are treated as if they were the same. **Example** In the above example, `UserId` and `Username` are both aliases for the same type, `string`. This means that the `getUser` function can mistakenly accept a `Username` as a valid `UserId`, causing bugs and errors. To prevent this, Effect introduces **branded types**. These types attach a unique identifier to a type, allowing you to differentiate between structurally similar but semantically distinct types. **Example** By defining `UserId` as a branded type, the `getUser` function can accept only values of type `UserId`, and not plain strings or other types that are compatible with strings. This helps to prevent bugs caused by accidentally passing the wrong type of value to the function. There are two ways to define a schema for a branded type, depending on whether you: - want to define the schema from scratch - have already defined a branded type via and want to reuse it to define a schema ### Defining a brand schema from scratch To define a schema for a branded type from scratch, use the `Schema.brand` function. **Example** Note that you can use `unique symbol`s as brands to ensure uniqueness across modules / packages. **Example** ### Reusing an existing branded constructor If you have already defined a branded type using the module, you can reuse it to define a schema using the `Schema.fromBrand` function. **Example** ### Utilizing Default Constructors The `Schema.brand` function includes a default constructor to facilitate the creation of branded values. ## Property Signatures A `PropertySignature` represents a transformation from a "From" field to a "To" field. This allows you to define mappings between incoming data fields and your internal model. ### Basic Usage A property signature can be defined with annotations to provide additional context about a field. **Example** A `PropertySignature` type contains several parameters, each providing details about the transformation between the source field and the target field . Let's take a look at what each of these parameters represents: | Parameter | Description | | ------------ | ------------------------------------------------------------------------------------------------------------------- | | `age` | Key of the "To" field | | `ToToken` | Indicates field requirement: `"?:"` for optional, `":"` for required | | `ToType` | Type of the "To" field | | `FromKey` | Indicates the source field key, typically the same as "To" field key unless specified | | `FromToken` | Indicates source field requirement: `"?:"` for optional, `":"` for required | | `FromType` | Type of the "From" field | | `HasDefault` | Indicates if there is a constructor default value | In the example above, the `PropertySignature` type for `age` is: This means: | Parameter | Description | | ------------ | -------------------------------------------------------------------------- | | `age` | Key of the "To" field | | `ToToken` | `":"` indicates that the `age` field is required | | `ToType` | Type of the `age` field is `number` | | `FromKey` | `never` indicates that the decoding occurs from the same field named `age` | | `FromToken` | `":"` indicates that the decoding occurs from a required `age` field | | `FromType` | Type of the "From" field is `string` | | `HasDefault` | `false`: indicates there is no default value | Sometimes, the source field may have a different name from the field in your internal model. You can map between these fields using the `Schema.fromKey` function. **Example** When you map from `"AGE"` to `"age"`, the `PropertySignature` type changes to: ### Optional Fields #### Basic Optional Property The syntax: creates an optional property within a schema, allowing fields to be omitted or set to `undefined`. ##### Decoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `undefined` | remains `undefined` | | `i: I` | transforms to `a: A` | ##### Encoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `undefined` | remains `undefined` | | `a: A` | transforms back to `i: I` | **Example** ##### Exposed Values You can access the original schema type using the `from` property. **Example** #### Optional with Nullability The syntax: creates an optional property within a schema, treating `null` values the same as missing values. ##### Decoding | Input | Output | | ----------------- | ------------------------------- | | `` | remains `` | | `undefined` | remains `undefined` | | `null` | transforms to `` | | `i: I` | transforms to `a: A` | ##### Encoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `undefined` | remains `undefined` | | `a: A` | transforms back to `i: I` | **Example** ##### Exposed Values You can access the original schema type using the `from` property. **Example** #### Optional with Exactness The syntax: creates an optional property while enforcing strict typing. This means that only the specified type is accepted. Any attempt to decode `undefined` results in an error. ##### Decoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `undefined` | `ParseError` | | `i: I` | transforms to `a: A` | ##### Encoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `a: A` | transforms back to `i: I` | **Example** ##### Exposed Values You can access the original schema type using the `from` property. **Example** #### Combining Nullability and Exactness The syntax: allows you to define an optional property that enforces strict typing while also treating `null` as equivalent to a missing value. ##### Decoding | Input | Output | | ----------------- | ------------------------------- | | `` | remains `` | | `null` | transforms to `` | | `undefined` | `ParseError` | | `i: I` | transforms to `a: A` | ##### Encoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `a: A` | transforms back to `i: I` | **Example** ##### Exposed Values You can access the original schema type using the `from` property. **Example** ### Representing Optional Fields with never Type When creating a schema to replicate a TypeScript type that includes optional fields with the `never` type, like: the handling of these fields depends on the `exactOptionalPropertyTypes` setting in your `tsconfig.json`. This setting affects whether the schema should treat optional `never`-typed fields as simply absent or allow `undefined` as a value. **Example** When this feature is turned off, you can employ the `Schema.optional` function. This approach allows the field to implicitly accept `undefined` as a value. **Example** When this feature is turned on, the `Schema.optionalWith` function is recommended. It ensures stricter enforcement of the field's absence. ### Default Values The `default` option in `Schema.optionalWith` allows you to set default values that are applied during both decoding and object construction phases. This feature ensures that even if certain properties are not provided by the user, the system will automatically use the specified default values. The `Schema.optionalWith` function offers several ways to control how defaults are applied during decoding and encoding. You can fine-tune whether defaults are applied only when the input is completely missing, or even when `null` or `undefined` values are provided. #### Basic Default This is the simplest use case. If the input is missing or `undefined`, the default value will be applied. **Syntax** | Operation | Behavior | | ------------ | ---------------------------------------------------------------- | | **Decoding** | Applies the default value if the input is missing or `undefined` | | **Encoding** | Transforms the input `a: A` back to `i: I` | **Example** ##### Exposed Values You can access the original schema type using the `from` property. **Example** #### Default with Exactness When you want the default value to be applied only if the field is completely missing , you can use the `exact` option. **Syntax** | Operation | Behavior | | ------------ | ------------------------------------------------------ | | **Decoding** | Applies the default value only if the input is missing | | **Encoding** | Transforms the input `a: A` back to `i: I` | **Example** #### Default with Nullability In cases where you want `null` values to trigger the default behavior, you can use the `nullable` option. This ensures that if a field is set to `null`, it will be replaced by the default value. **Syntax** | Operation | Behavior | | ------------ | -------------------------------------------------------------------------- | | **Decoding** | Applies the default value if the input is missing or `undefined` or `null` | | **Encoding** | Transforms the input `a: A` back to `i: I` | **Example** #### Combining Exactness and Nullability For a more strict approach, you can combine both `exact` and `nullable` options. This way, the default value is applied only when the field is `null` or missing, and not when it's explicitly set to `undefined`. **Syntax** | Operation | Behavior | | ------------ | ----------------------------------------------------------- | | **Decoding** | Applies the default value if the input is missing or `null` | | **Encoding** | Transforms the input `a: A` back to `i: I` | **Example** ### Optional Fields as Options When working with optional fields, you may want to handle them as types. This approach allows you to explicitly manage the presence or absence of a field rather than relying on `undefined` or `null`. #### Basic Optional with Option Type You can configure a schema to treat optional fields as `Option` types, where missing or `undefined` values are converted to `Option.none` and existing values are wrapped in `Option.some`. **Syntax** ##### Decoding | Input | Output | | ----------------- | --------------------------------- | | `` | transforms to `Option.none` | | `undefined` | transforms to `Option.none` | | `i: I` | transforms to `Option.some` | ##### Encoding | Input | Output | | ------------------- | ------------------------------- | | `Option.none` | transforms to `` | | `Option.some` | transforms back to `i: I` | **Example** ##### Exposed Values You can access the original schema type using the `from` property. **Example** #### Optional with Exactness The `exact` option ensures that the default behavior of the optional field applies only when the field is entirely missing, not when it is `undefined`. **Syntax** ##### Decoding | Input | Output | | ----------------- | --------------------------------- | | `` | transforms to `Option.none` | | `undefined` | `ParseError` | | `i: I` | transforms to `Option.some` | ##### Encoding | Input | Output | | ------------------- | ------------------------------- | | `Option.none` | transforms to `` | | `Option.some` | transforms back to `i: I` | **Example** ##### Exposed Values You can access the original schema type using the `from` property. **Example** #### Optional with Nullability The `nullable` option extends the default behavior to treat `null` as equivalent to `Option.none`, alongside missing or `undefined` values. **Syntax** ##### Decoding | Input | Output | | ----------------- | --------------------------------- | | `` | transforms to `Option.none` | | `undefined` | transforms to `Option.none` | | `null` | transforms to `Option.none` | | `i: I` | transforms to `Option.some` | ##### Encoding | Input | Output | | ------------------- | ------------------------------- | | `Option.none` | transforms to `` | | `Option.some` | transforms back to `i: I` | **Example** ##### Exposed Values You can access the original schema type using the `from` property. **Example** #### Combining Exactness and Nullability When both `exact` and `nullable` options are used together, only `null` and missing fields are treated as `Option.none`, while `undefined` is considered an invalid value. **Syntax** ##### Decoding | Input | Output | | ----------------- | --------------------------------- | | `` | transforms to `Option.none` | | `undefined` | `ParseError` | | `null` | transforms to `Option.none` | | `i: I` | transforms to `Option.some` | ##### Encoding | Input | Output | | ------------------- | ------------------------------- | | `Option.none` | transforms to `` | | `Option.some` | transforms back to `i: I` | **Example** ##### Exposed Values You can access the original schema type using the `from` property. **Example** ## Optional Fields Primitives ### optionalToOptional The `Schema.optionalToOptional` API allows you to manage transformations from an optional field in the input to an optional field in the output. This can be useful for controlling both the output type and whether a field is present or absent based on specific criteria. One common use case for `optionalToOptional` is handling fields where a specific input value, such as an empty string, should be treated as an absent field in the output. **Syntax** In this function: - The `from` parameter specifies the input schema, and `to` specifies the output schema. - The `decode` and `encode` functions define how the field should be interpreted on both sides: - `Option.none` as an input argument indicates a missing field in the input. - Returning `Option.none` from either function will omit the field in the output. **Example** Consider an optional field of type `string` where empty strings in the input should be removed from the output. You can simplify the decoding logic with `Option.filter`, which filters out unwanted values in a concise way. **Example** ### optionalToRequired The `Schema.optionalToRequired` API lets you transform an optional field into a required one, with custom logic to handle cases when the field is missing in the input. **Syntax** In this function: - `from` specifies the input schema, while `to` specifies the output schema. - The `decode` and `encode` functions define the transformation behavior: - Passing `Option.none` to `decode` means the field is absent in the input. The function can then return a default value for the output. - Returning `Option.none` in `encode` will omit the field in the output. **Example** This example demonstrates how to use `optionalToRequired` to provide a `null` default value when the `nullable` field is missing in the input. During encoding, fields with a value of `null` are omitted from the output. You can streamline the decoding and encoding logic using `Option.getOrElse` and `Option.liftPredicate` for concise and readable transformations. **Example** ### requiredToOptional The `requiredToOptional` API allows you to transform a required field into an optional one, applying custom logic to determine when the field can be omitted. **Syntax** With `decode` and `encode` functions, you control the presence or absence of the field: - `Option.none` as an argument in `decode` means the field is missing in the input. - `Option.none` as a return value from `encode` means the field will be omitted in the output. **Example** In this example, the `name` field is required but treated as optional if it is an empty string. During decoding, an empty string in `name` is considered absent, while encoding ensures a value . You can streamline the decoding and encoding logic using `Option.liftPredicate` and `Option.getOrElse` for concise and readable transformations. **Example** ## Extending Schemas Schemas in `effect` can be extended in multiple ways, allowing you to combine or enhance existing types with additional fields or functionality. One common method is to use the `fields` property available in `Struct` schemas. This property provides a convenient way to add fields or merge fields from different structs while retaining the original `Struct` type. This approach also makes it easier to access and modify fields. For more complex cases, such as extending a struct with a union, you may want to use the `Schema.extend` function, which offers flexibility in scenarios where direct field spreading may not be sufficient. ### Spreading Struct fields Structs provide access to their fields through the `fields` property, which allows you to extend an existing struct by adding additional fields or combining fields from multiple structs. **Example** **Example** **Example** ### The extend function The `Schema.extend` function provides a structured method to expand schemas, especially useful when direct isn't sufficient—such as when you need to extend a struct with a union of other structs. Supported extensions include: - `Schema.String` with another `Schema.String` refinement or a string literal - `Schema.Number` with another `Schema.Number` refinement or a number literal - `Schema.Boolean` with another `Schema.Boolean` refinement or a boolean literal - A struct with another struct where overlapping fields support extension - A struct with in index signature - A struct with a union of supported schemas - A refinement of a struct with a supported schema - A `suspend` of a struct with a supported schema - A transformation between structs where the "from" and "to" sides have no overlapping fields with the target struct **Example** **Example** This example demonstrates an attempt to extend a struct with another struct that contains overlapping field names, resulting in an error due to conflicting types. **Example** In this example, we extend two refinements, `Integer` and `Positive`, creating a schema that enforces both integer and positivity constraints. ## Renaming Properties ### Renaming a Property During Definition To rename a property directly during schema creation, you can utilize the `Schema.fromKey` function. **Example** **Example** Using `Schema.optional` automatically returns a `PropertySignature`, making it unnecessary to explicitly use `Schema.propertySignature` as required for renaming required fields in the previous example. ### Renaming Properties of an Existing Schema For existing schemas, the `Schema.rename` API offers a way to systematically change property names across a schema, even within complex structures like unions, though in case of structs you lose the original field types. **Example** **Example** ## Recursive Schemas The `Schema.suspend` function is designed for defining schemas that reference themselves, such as in recursive data structures. **Example** In this example, the `Category` schema references itself through the `subcategories` field, which is an array of `Category` objects. **Example** ### A Helpful Pattern to Simplify Schema Definition As we've observed, it's necessary to define an interface for the `Type` of the schema to enable recursive schema definition, which can complicate things and be quite tedious. One pattern to mitigate this is to **separate the field responsible for recursion** from all other fields. **Example** ### Mutually Recursive Schemas You can also use `Schema.suspend` to create mutually recursive schemas, where two schemas reference each other. In the following example, `Expression` and `Operation` form a simple arithmetic expression tree by referencing each other. **Example** ### Recursive Types with Different Encoded and Type Defining a recursive schema where the `Encoded` type differs from the `Type` type adds another layer of complexity. In such cases, we need to define two interfaces: one for the `Type` type, as seen previously, and another for the `Encoded` type. **Example** Let's consider an example: suppose we want to add an `id` field to the `Category` schema, where the schema for `id` is `NumberFromString`. It's important to note that `NumberFromString` is a schema that transforms a string into a number, so the `Type` and `Encoded` types of `NumberFromString` differ, being `number` and `string` respectively. When we add this field to the `Category` schema, TypeScript raises an error: This error occurs because the explicit annotation `Schema.Schema` is no longer sufficient and needs to be adjusted by explicitly adding the `Encoded` type: --- title: Schema Annotations description: Learn how to enhance schemas with annotations for better customization, error handling, documentation, and concurrency control in your Effect-based applications. sidebar: label: Annotations order: 8 --- One of the key features of the Schema design is its flexibility and ability to be customized. This is achieved through "annotations." Each node in the `ast` field of a schema has an `annotations: Record` field, which allows you to attach additional information to the schema. You can manage these annotations using the `annotations` method or the `Schema.annotations` API. **Example** ## Built-in Annotations The following table provides an overview of common built-in annotations and their uses: | Annotation | Description | | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `identifier` | Assigns a unique identifier to the schema, ideal for TypeScript identifiers and code generation purposes. Commonly used in tools like to clarify output. Examples include `"Person"`, `"Product"`. | | `title` | Sets a short, descriptive title for the schema, similar to a JSON Schema title. Useful for documentation or UI headings. It is also used by to enhance readability of error messages. | | `description` | Provides a detailed explanation about the schema's purpose, akin to a JSON Schema description. Used by to provide more detailed error messages. | | `documentation` | Extends detailed documentation for the schema, beneficial for developers or automated documentation generation. | | `examples` | Lists examples of valid schema values, akin to the examples attribute in JSON Schema, useful for documentation and validation testing. | | `default` | Defines a default value for the schema, similar to the default attribute in JSON Schema, to ensure schemas are pre-populated where applicable. | | `message` | Customizes the error message for validation failures, improving clarity in outputs from tools like and during decoding or validation errors. | | `jsonSchema` | Specifies annotations that affect the generation of documents, customizing how schemas are represented. | | `arbitrary` | Configures settings for generating test data. | | `pretty` | Configures settings for generating output. | | `equivalence` | Configures settings for evaluating data . | | `concurrency` | Controls concurrency behavior, ensuring schemas perform optimally under concurrent operations. Refer to for detailed usage. | | `batching` | Manages settings for batching operations to enhance performance when operations can be grouped. | | `parseIssueTitle` | Provides a custom title for parsing issues, enhancing error descriptions in outputs from . See for more information. | | `parseOptions` | Allows overriding of parsing options at the schema level, offering granular control over parsing behaviors. See for application details. | | `decodingFallback` | Provides a way to define custom fallback behaviors that trigger when decoding operations fail. Refer to for detailed usage. | ## Concurrency Annotation For more complex schemas like `Struct`, `Array`, or `Union` that contain multiple nested schemas, the `concurrency` annotation provides a way to control how validations are executed concurrently. Here's a shorter version presented in a table: | Value | Description | | ------------- | --------------------------------------------------------------- | | `number` | Limits the maximum number of concurrent tasks. | | `"unbounded"` | All tasks run concurrently with no limit. | | `"inherit"` | Inherits concurrency settings from the parent context. | | `undefined` | Tasks run sequentially, one after the other . | **Example** In this example, we define three tasks that simulate asynchronous operations with different durations. Since no concurrency is specified, the tasks are executed sequentially, one after the other. **Example** By adding a `concurrency` annotation set to `"unbounded"`, the tasks can now run concurrently, meaning they don't wait for one another to finish before starting. This allows faster execution when multiple tasks are involved. ## Handling Decoding Errors with Fallbacks The `DecodingFallbackAnnotation` allows you to handle decoding errors by providing a custom fallback logic. This annotation enables you to specify fallback behavior when decoding fails, making it possible to recover gracefully from errors. **Example** In this basic example, when decoding fails , the fallback value is returned instead of an error. **Example** In this advanced example, when a decoding error occurs, the schema logs the issue and then returns a fallback value. This demonstrates how you can incorporate logging and other side effects during error handling. ## Custom Annotations In addition to built-in annotations, you can define custom annotations to meet specific requirements. For instance, here's how to create a `deprecated` annotation: **Example** To make your new custom annotation type-safe, you can use a module augmentation. In the next example, we want our custom annotation to be a boolean. **Example** You can retrieve custom annotations using the `SchemaAST.getAnnotation` helper function. **Example** --- title: Schema to Arbitrary description: Generate random test data that adheres to schema constraints using Arbitrary, with options for transformations, filters, and custom generation. sidebar: label: Arbitrary order: 15 --- import { Aside } from "@astrojs/starlight/components" The `Arbitrary.make` function allows for the creation of random values that align with a specific `Schema`. This function returns an `Arbitrary` from the library, which is particularly useful for generating random test data that adheres to the defined schema constraints. **Example** To make the output more realistic, see the section. ## Filters When generating random values, `Arbitrary` tries to follow the schema's constraints. It uses the most appropriate `fast-check` primitives and applies constraints if the primitive supports them. For instance, if you define an `age` property as: the arbitrary generation will use: to produce values within that range. ### Patterns To generate efficient arbitraries for strings that must match a certain pattern, use the `Schema.pattern` filter instead of writing a custom filter: **Example** By using `Schema.pattern`, arbitrary generation will rely on `FastCheck.stringMatching`, which is more efficient and directly aligned with the defined pattern. When multiple patterns are used, they are combined into a union. For example: This approach ensures all patterns have an equal chance of generating values when using `FastCheck.stringMatching`. ## Transformations and Arbitrary Generation When generating arbitrary data, it is important to understand how transformations and filters are handled within a schema: **Example** **Explanation:** - `schema1`: Takes into account `Schema.maxLength` since it is applied after the `Schema.Trim` transformation, but ignores the `Schema.NonEmptyString` as it precedes the transformations. - `schema2`: Adheres fully to all filters because they are correctly sequenced after transformations, preventing the generation of undesired data. ### Best Practices To ensure consistent and valid arbitrary data generation, follow these guidelines: 1. **Apply Filters First**: Define filters for the initial type . 2. **Apply Transformations**: Add transformations to convert the data. 3. **Apply Final Filters**: Use filters for the transformed type . This setup ensures that each stage of data processing is precise and well-defined. **Example** Avoid haphazard combinations of transformations and filters: Prefer a structured approach by separating transformation steps from filter applications: **Example** ## Customizing Arbitrary Data Generation You can customize how arbitrary data is generated using the `arbitrary` annotation in schema definitions. **Example** The annotation allows access the complete export of the fast-check library . This setup enables you to return an `Arbitrary` that precisely generates the type of data desired. ### Integration with Fake Data Generators When using mocking libraries like , you can combine them with `fast-check` to generate realistic data for testing purposes. **Example** --- title: Basic Usage description: Learn to define and work with basic schemas, including primitives, literals, unions, and structs, for effective data validation and transformation. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" ## Primitives The Schema module provides built-in schemas for common primitive types. | Schema | Equivalent TypeScript Type | | ----------------------- | -------------------------- | | `Schema.String` | `string` | | `Schema.Number` | `number` | | `Schema.Boolean` | `boolean` | | `Schema.BigIntFromSelf` | `BigInt` | | `Schema.SymbolFromSelf` | `symbol` | | `Schema.Object` | `object` | | `Schema.Undefined` | `undefined` | | `Schema.Void` | `void` | | `Schema.Any` | `any` | | `Schema.Unknown` | `unknown` | | `Schema.Never` | `never` | **Example** ## asSchema To make it easier to work with schemas, built-in schemas are exposed with shorter, opaque types when possible. The `Schema.asSchema` function allows you to view any schema as `Schema`. **Example** For example, while `Schema.String` is defined as a class with a type of `typeof Schema.String`, using `Schema.asSchema` provides the schema in its extended form as `Schema`. ## Unique Symbols You can create a schema for unique symbols using `Schema.UniqueSymbolFromSelf`. **Example** ## Literals Literal schemas represent a . You can use them to specify exact values that a type must have. Literals can be of the following types: - `string` - `number` - `boolean` - `null` - `bigint` **Example** **Example** ### Union of Literals You can create a union of multiple literals by passing them as arguments to the `Schema.Literal` constructor: **Example** If you want to set a custom error message for the entire union of literals, you can use the `override: true` option for more details) to specify a unified message. **Example** ### Exposed Values You can access the literals defined in a literal schema using the `literals` property: ### The pickLiteral Utility You can use `Schema.pickLiteral` with a literal schema to narrow down its possible values. **Example** Sometimes, you may need to reuse a literal schema in other parts of your code. Below is an example demonstrating how to do this: **Example** In this example, `FruitCategory` serves as the source of truth for the different fruit categories. We reuse it to create a subtype of `Fruit` called `SweetAndCitrusFruit`, ensuring that only the specified categories are allowed. This approach helps maintain consistency throughout your code and provides type safety if the category definition changes. ## Template literals In TypeScript, allow you to embed expressions within string literals. The `Schema.TemplateLiteral` constructor allows you to create a schema for these template literal types. **Example** **Example** Documentation) Let's look at a more complex example. Suppose you have two sets of locale IDs for emails and footers. You can use the `Schema.TemplateLiteral` constructor to create a schema that combines these IDs: ### Supported Span Types The `Schema.TemplateLiteral` constructor supports the following types of spans: - `Schema.String` - `Schema.Number` - Literals: `string | number | boolean | null | bigint`. These can be either wrapped by `Schema.Literal` or used directly - Unions of the above types - Brands of the above types **Example** ### TemplateLiteralParser The `Schema.TemplateLiteral` constructor, while useful as a simple validator, only verifies that an input conforms to a specific string pattern by converting template literal definitions into regular expressions. Similarly, employs regular expressions directly for the same purpose. Post-validation, both methods require additional manual parsing to convert the validated string into a usable data format. To address these limitations and eliminate the need for manual post-validation parsing, the `Schema.TemplateLiteralParser` API has been developed. It not only validates the input format but also automatically parses it into a more structured and type-safe output, specifically into a **tuple** format. The `Schema.TemplateLiteralParser` constructor supports the same types of as `Schema.TemplateLiteral`. **Example** ## Native enums The Schema module provides support for native TypeScript enums. You can define a schema for an enum using `Schema.Enums`, allowing you to validate values that belong to the enum. **Example** ### Exposed Values Enums are accessible through the `enums` property of the schema. You can use this property to retrieve individual members or the entire set of enum values. ## Unions The Schema module includes a built-in `Schema.Union` constructor for creating "OR" types, allowing you to define schemas that can represent multiple types. **Example** ### Union Member Evaluation Order When decoding, union members are evaluated in the order they are defined. If a value matches the first member, it will be decoded using that schema. If not, the decoding process moves on to the next member. If multiple schemas could decode the same value, the order matters. Placing a more general schema before a more specific one may result in missing properties, as the first matching schema will be used. **Example** ### Union of Literals While you can create a union of literals by combining individual literal schemas: **Example** You can simplify the process by passing multiple literals directly to the `Schema.Literal` constructor: **Example** If you want to set a custom error message for the entire union of literals, you can use the `override: true` option for more details) to specify a unified message. **Example** ### Nullables The Schema module includes utility functions for defining schemas that allow nullable types, helping to handle values that may be `null`, `undefined`, or both. **Example** ### Discriminated unions in TypeScript are a way of modeling complex data structures that may take on different forms based on a specific set of conditions or properties. They allow you to define a type that represents multiple related shapes, where each shape is uniquely identified by a shared discriminant property. In a discriminated union, each variant of the union has a common property, called the discriminant. The discriminant is a literal type, which means it can only have a finite set of possible values. Based on the value of the discriminant property, TypeScript can infer which variant of the union is currently in use. **Example** In the `Schema` module, you can define a discriminated union similarly by specifying a literal field as the discriminant for each type. **Example** In this example, the `Schema.Literal` constructor sets up the `kind` property as the discriminant for both `Circle` and `Square` schemas. The `Shape` schema then represents a union of these two types, allowing TypeScript to infer the specific shape based on the `kind` value. ### Transforming a Simple Union into a Discriminated Union If you start with a simple union and want to transform it into a discriminated union, you can add a special property to each member. This allows TypeScript to automatically infer the correct type based on the value of the discriminant property. **Example** For example, let's say you've defined a `Shape` union as a combination of `Circle` and `Square` without any special property: To make your code more manageable, you may want to transform the simple union into a discriminated union. This way, TypeScript will be able to automatically determine which member of the union you're working with based on the value of a specific property. To achieve this, you can add a special property to each member of the union, which will allow TypeScript to know which type it's dealing with at runtime. Here's how you can the `Shape` schema into another schema that represents a discriminated union: **Example** The previous solution works perfectly and shows how we can add properties to our schema at will, making it easier to consume the result within our domain model. However, it requires a lot of boilerplate. Fortunately, there is an API called `Schema.attachPropertySignature` designed specifically for this use case, which allows us to achieve the same result with much less effort: **Example** ### Exposed Values You can access the individual members of a union schema represented as a tuple: ## Tuples The Schema module allows you to define tuples, which are ordered collections of elements that may have different types. You can define tuples with required, optional, or rest elements. ### Required Elements To define a tuple with required elements, you can use the `Schema.Tuple` constructor and simply list the element schemas in order: **Example** ### Append a Required Element You can append additional required elements to an existing tuple by using the spread operator: **Example** ### Optional Elements To define an optional element, use the `Schema.optionalElement` constructor. **Example** ### Rest Element To define a rest element, add it after the list of required or optional elements. The rest element allows the tuple to accept additional elements of a specific type. **Example** You can also include other elements after the rest: **Example** ### Annotations Annotations are useful for adding metadata to tuple elements, making it easier to describe their purpose or requirements. This is especially helpful for generating documentation or JSON schemas. **Example** ### Exposed Values You can access the elements and rest elements of a tuple schema using the `elements` and `rest` properties: **Example** ## Arrays The Schema module allows you to define schemas for arrays, making it easy to validate collections of elements of a specific type. **Example** ### Mutable Arrays By default, `Schema.Array` generates a type marked as `readonly`. To create a schema for a mutable array, you can use the `Schema.mutable` function, which makes the array type mutable in a **shallow** manner. **Example** ### Exposed Values You can access the value type of an array schema using the `value` property: **Example** ## Non Empty Arrays The Schema module also provides a way to define schemas for non-empty arrays, ensuring that the array always contains at least one element. **Example** ### Exposed Values You can access the value type of a non-empty array schema using the `value` property: **Example** ## Records The Schema module provides support for defining record types, which are collections of key-value pairs where the key can be a string, symbol, or other types, and the value has a defined schema. ### String Keys You can define a record with string keys and a specified type for the values. **Example** ### Symbol Keys Records can also use symbols as keys. **Example** ### Union of Literal Keys Use a union of literals to restrict keys to a specific set of values. **Example** ### Template Literal Keys Records can use template literals as keys, allowing for more complex key patterns. **Example** ### Refined Keys You can refine the key type with additional constraints. **Example** Refinements on keys act as filters rather than causing a decoding failure. If a key does not meet the constraints , it is removed from the decoded output instead of triggering an error. **Example** If you want decoding to fail when a key does not meet the constraints, you can set to `"error"`. **Example** ### Transforming Keys The `Schema.Record` API does not support transformations on key schemas. Attempting to apply a transformation to keys will result in an `Unsupported key schema` error: **Example** To modify record keys, you must apply transformations outside of `Schema.Record`. A common approach is to use to adjust keys during decoding. **Example** ### Mutable Records By default, `Schema.Record` generates a type marked as `readonly`. To create a schema for a mutable record, you can use the `Schema.mutable` function, which makes the record type mutable in a **shallow** manner. **Example** ### Exposed Values You can access the `key` and `value` types of a record schema using the `key` and `value` properties: **Example** ## Structs ### Property Signatures The `Schema.Struct` constructor defines a schema for an object with specific properties. **Example** This example defines a struct schema for an object with the following properties: - `name`: a string - `age`: a number ### Index Signatures The `Schema.Struct` constructor can optionally accept a list of key/value pairs representing index signatures, allowing you to define additional dynamic properties. **Example** **Example** You can achieve the same result using `Schema.Record`: ### Multiple Index Signatures You can define **one** index signature per key type . Defining multiple index signatures of the same type is not allowed. **Example** Defining multiple index signatures of the same key type will cause an error. **Example** ### Conflicting Index Signatures When defining schemas with index signatures, conflicts can arise if a fixed property has a different type than the values allowed by the index signature. This can lead to unexpected TypeScript behavior. **Example** The TypeScript compiler flags this as an error when defining the type manually: This happens because TypeScript does not allow an index signature to contradict a fixed property. #### Workaround for Conflicting Index Signatures When working with schemas, a conflict can occur if a fixed property has a different type than the values allowed by an index signature. This situation often arises when dealing with external APIs that do not follow strict TypeScript conventions. To prevent conflicts, you can separate the fixed properties from the indexed properties and handle them as distinct parts of the schema. **Example** Consider an object where: - `"a"` is a fixed property of type `string`. - All other keys store numbers, which conflict with `"a"`. To avoid this issue, we can separate the properties into two distinct types: By using and , you can preprocess the input data before validation. This approach ensures that fixed properties and index signature properties are treated independently. ### Exposed Values You can access the fields and records of a struct schema using the `fields` and `records` properties: **Example** ### Mutable Structs By default, `Schema.Struct` generates a type with properties marked as `readonly`. To create a mutable version of the struct, use the `Schema.mutable` function, which makes the properties mutable in a **shallow** manner. **Example** ## Tagged Structs In TypeScript tags help to enhance type discrimination and pattern matching by providing a simple yet powerful way to define and recognize different data types. ### What is a Tag? A tag is a literal value added to data structures, commonly used in structs, to distinguish between various object types or variants within tagged unions. This literal acts as a discriminator, making it easier to handle and process different types of data correctly and efficiently. ### Using the tag Constructor The `Schema.tag` constructor is specifically designed to create a property signature that holds a specific literal value, serving as the discriminator for object types. **Example** In the example above, `Schema.tag` attaches a `_tag` property to the `User` struct schema, effectively labeling objects of this struct type as "User". This label is automatically applied when using the `make` method to create new instances, simplifying object creation and ensuring consistent tagging. ### Simplifying Tagged Structs with TaggedStruct The `Schema.TaggedStruct` constructor streamlines the process of creating tagged structs by directly integrating the tag into the struct definition. This method provides a clearer and more declarative approach to building data structures with embedded discriminators. **Example** In this example: - The `_tag` property is optional when constructing an instance with `make`, allowing the schema to automatically apply it. - When decoding unknown data, `_tag` is required to ensure correct type identification. This distinction between instance construction and decoding is useful for preserving the tag’s role as a type discriminator while simplifying instance creation. If you need `_tag` to be applied automatically during decoding as well, you can create a customized version of `Schema.TaggedStruct`: **Example** ### Multiple Tags While a primary tag is often sufficient, TypeScript allows you to define multiple tags for more complex data structuring needs. Here's an example demonstrating the use of multiple tags within a single struct: **Example** This example defines a product schema with a primary tag and an additional category tag , adding further specificity to the data structure. ## instanceOf When you need to define a schema for your custom data type defined through a `class`, the most convenient and fast way is to use the `Schema.instanceOf` constructor. **Example** The `Schema.instanceOf` constructor is just a lightweight wrapper of the API, which is the primitive in `effect/Schema` for declaring new custom data types. ### Private Constructors Note that `Schema.instanceOf` can only be used for classes that expose a **public constructor**. If you try to use it with classes that, for some reason, have marked the constructor as `private`, you'll receive a TypeScript error: **Example** In such cases, you cannot use `Schema.instanceOf`, and you must rely on like this: **Example** ### Validating Fields of the Instance To validate the fields of a class instance, you can use a . This approach combines instance validation with additional checks on the instance's fields. **Example** ## Picking The `pick` static function available on each struct schema can be used to create a new `Struct` by selecting specific properties from an existing `Struct`. **Example** The `Schema.pick` function can be applied more broadly beyond just `Struct` types, such as with unions of schemas. However it returns a generic `SchemaClass`. **Example** ## Omitting The `omit` static function available in each struct schema can be used to create a new `Struct` by excluding particular properties from an existing `Struct`. **Example** The `Schema.omit` function can be applied more broadly beyond just `Struct` types, such as with unions of schemas. However it returns a generic `Schema`. **Example** ## partial The `Schema.partial` function makes all properties within a schema optional. **Example** By default, the `Schema.partial` operation adds `undefined` to the type of each property. If you want to avoid this, you can use `Schema.partialWith` and pass `{ exact: true }` as an argument. **Example** ## required The `Schema.required` function ensures that all properties in a schema are mandatory. **Example** In this example, both `a` and `b` are made required, even though they were initially defined as optional. ## keyof The `Schema.keyof` operation creates a schema that represents the keys of a given object schema. **Example** --- title: Class APIs description: Learn to define and extend schemas using classes, incorporating validation, custom logic, and advanced features like equality checks and transformations. sidebar: order: 11 --- import { Aside } from "@astrojs/starlight/components" When working with schemas, you have a choice beyond the constructor. You can leverage the power of classes through the `Schema.Class` utility, which comes with its own set of advantages tailored to common use cases: Classes offer several features that simplify the schema creation process: - **All-in-One Definition**: With classes, you can define both a schema and an opaque type simultaneously. - **Shared Functionality**: You can incorporate shared functionality using class methods or getters. - **Value Hashing and Equality**: Utilize the built-in capability for checking value equality and applying hashing ). ## Definition To define a class using `Schema.Class`, you need to specify: - The **type** of the class being created. - A unique **identifier** for the class. - The desired **fields**. **Example** In this example, `Person` is both a schema and a TypeScript class. Instances of `Person` are created using the defined schema, ensuring compliance with the specified fields. **Example** ### Class Schemas are Transformations Class schemas a struct schema into a schema that represents a class type. - When decoding, a plain object is converted into an instance of the class. - When encoding, a class instance is converted back into a plain object. **Example** ### Defining Classes Without Fields When your schema does not require any fields, you can define a class with an empty object. **Example** ### Defining Classes With Filters Filters allow you to validate input when decoding, encoding, or creating an instance. Instead of specifying raw fields, you can pass a `Schema.Struct` with a filter applied. **Example** ## Validating Properties via Class Constructors When you define a class using `Schema.Class`, the constructor automatically checks that the provided properties adhere to the schema's rules. ### Defining and Instantiating a Valid Class Instance The constructor ensures that each property, like `id` and `name`, adheres to the schema. For instance, `id` must be a number, and `name` must be a non-empty string. **Example** ### Handling Invalid Properties If invalid properties are provided during instantiation, the constructor throws an error, explaining why the validation failed. **Example** The error clearly specifies that the `name` field failed to meet the `NonEmptyString` requirement. ### Bypassing Validation In some scenarios, you might want to bypass the validation logic. While not generally recommended, the library provides an option to do so. **Example** ## Automatic Hashing and Equality in Classes Instances of classes created with `Schema.Class` support the trait through their integration with . This enables straightforward value comparisons, even across different instances. ### Basic Equality Check Two class instances are considered equal if their properties have identical values. **Example** ### Nested or Complex Properties The `Equal` trait performs comparisons at the first level. If a property is a more complex structure, such as an array, instances may not be considered equal, even if the arrays themselves have identical values. **Example** To achieve deep equality for nested structures like arrays, use `Schema.Data` in combination with `Data.array`. This enables the library to compare each element of the array rather than treating it as a single entity. **Example** ## Extending Classes with Custom Logic Schema classes provide the flexibility to include custom getters and methods, allowing you to extend their functionality beyond the defined fields. ### Adding Custom Getters A getter can be used to derive computed values from the fields of the class. For example, a `Person` class can include a getter to return the `name` property in uppercase. **Example** ### Adding Custom Methods In addition to getters, you can define methods to encapsulate more complex logic or operations involving the class's fields. **Example** ## Leveraging Classes as Schema Definitions When you define a class with `Schema.Class`, it serves both as a schema and as a class. This dual functionality allows the class to be used wherever a schema is required. **Example** ### Exposed Values The class also includes a `fields` static property, which outlines the fields defined during the class creation. **Example** ## Adding Annotations Defining a class with `Schema.Class` is similar to creating a schema that converts a struct schema into a schema representing the class type. For example, consider the following class definition: Under the hood, this definition creates a transformation schema that maps: to a schema representing the `Person` class: So, defining a schema with `Schema.Class` involves three schemas: - The "from" schema - The "to" schema - The "transformation" schema You can annotate each of these schemas by passing a tuple as the second argument to the `Schema.Class` API. **Example** If you do not want to annotate all three schemas, you can pass `undefined` for the ones you wish to skip. **Example** By default, the unique identifier used to define the class is also applied as the default `identifier` annotation for the Class Schema. **Example** ## Recursive Schemas The `Schema.suspend` combinator is useful when you need to define a schema that depends on itself, like in the case of recursive data structures. In this example, the `Category` schema depends on itself because it has a field `subcategories` that is an array of `Category` objects. **Example** **Example** ### Mutually Recursive Schemas Sometimes, schemas depend on each other in a mutually recursive way. For instance, an arithmetic expression tree might include `Expression` nodes that can either be numbers or `Operation` nodes, which in turn reference `Expression` nodes. **Example** ### Recursive Types with Different Encoded and Type Defining recursive schemas where the `Encoded` type differs from the `Type` type introduces additional complexity. For instance, if a schema includes fields that transform data , the `Encoded` and `Type` types may not align. In such cases, we need to define an interface for the `Encoded` type. Let's consider an example: suppose we want to add an `id` field to the `Category` schema, where the schema for `id` is `NumberFromString`. It's important to note that `NumberFromString` is a schema that transforms a string into a number, so the `Type` and `Encoded` types of `NumberFromString` differ, being `number` and `string` respectively. When we add this field to the `Category` schema, TypeScript raises an error: This error occurs because the explicit annotation `S.suspend: S.Schema => Category` is no longer sufficient and needs to be adjusted by explicitly adding the `Encoded` type: **Example** As we've observed, it's necessary to define an interface for the `Encoded` of the schema to enable recursive schema definition, which can complicate things and be quite tedious. One pattern to mitigate this is to **separate the field responsible for recursion** from all other fields. **Example** ## Tagged Class variants You can also create classes that extend and from the `effect/Data` module. **Example** ## Extending existing Classes The `extend` static utility allows you to enhance an existing schema class by adding **additional** fields and functionality. This approach helps in building on top of existing schemas without redefining them from scratch. **Example** Note that you can only add additional fields when extending a class. **Example** This error occurs because allowing fields to be overwritten is not safe. It could interfere with any getters or methods defined on the class that rely on the original definition. For example, in this case, the `upperName` getter would break if the `name` field was changed to a number. ## Transformations You can enhance schema classes with effectful transformations to enrich or validate entities, particularly when working with data sourced from external systems like databases or APIs. **Example** The following example demonstrates adding an `age` field to a `Person` class. The `age` value is derived asynchronously based on the `id` field. The decision of which API to use, either `transformOrFail` or `transformOrFailFrom`, depends on when you wish to execute the transformation: 1. Using `transformOrFail`: - The transformation occurs at the end of the process. - It expects you to provide a value of type `{ age: Option }`. - After processing the initial input, the new transformation comes into play, and you need to ensure the final output adheres to the specified structure. 2. Using `transformOrFailFrom`: - The new transformation starts as soon as the initial input is handled. - You should provide a value `{ age?: number }`. - Based on this fresh input, the subsequent transformation `Schema.optionalWith` is executed. - This approach allows for immediate handling of the input, potentially influencing the subsequent transformations. --- title: Default Constructors description: Create values that conform to schemas effortlessly using default constructors for structs, records, filters, and branded types, with options for validation, default values, and lazy evaluation. sidebar: order: 12 --- import { Aside } from "@astrojs/starlight/components" When working with data structures, it can be helpful to create values that conform to a schema with minimal effort. For this purpose, the Schema module provides default constructors for various schema types, including `Structs`, `Records`, `filters`, and `brands`. Default constructors are **unsafe**, meaning they **throw an error** if the input does not conform to the schema. If you need a safer alternative, consider using , which returns a result indicating success or failure instead of throwing an error. **Example** ## Structs Struct schemas allow you to define objects with specific fields and constraints. The `make` function can be used to create instances of a struct schema. **Example** In some cases, you might need to bypass validation. While not recommended in most scenarios, `make` provides an option to disable validation. **Example** ## Records Record schemas allow you to define key-value mappings where the keys and values must meet specific criteria. **Example** ## Filters Filters allow you to define constraints on individual values. **Example** ## Branded Types Branded schemas add metadata to a value to give it a more specific type, while still retaining its original type. **Example** When using default constructors, it is helpful to understand the type of value they produce. For instance, in the `BrandedNumberSchema` example, the return type of the constructor is `number & Brand<"MyNumber">`. This indicates that the resulting value is a `number` with additional branding information, `"MyNumber"`. This behavior contrasts with the filter example, where the return type is simply `number`. Branding adds an extra layer of type information, which can assist in identifying and working with your data more effectively. ## Error Handling in Constructors Default constructors are considered "unsafe" because they throw an error if the input does not conform to the schema. This error includes a detailed description of what went wrong. The intention behind default constructors is to provide a straightforward way to create valid values, such as for tests or configurations, where invalid inputs are expected to be exceptional cases. If you need a "safe" constructor that does not throw errors but instead returns a result indicating success or failure, you can use `Schema.validateEither`. **Example** ## Setting Default Values When creating objects, you might want to assign default values to certain fields to simplify object construction. The `Schema.withConstructorDefault` function lets you handle default values, making fields optional in the default constructor. **Example** In this example, all fields are required when creating a new instance. **Example** Here, the `age` field is optional because it has a default value of `0`. ### Nested Structs and Shallow Defaults Default values in schemas are shallow, meaning that defaults defined in nested structs do not automatically propagate to the top-level constructor. **Example** This behavior occurs because the `Schema` interface does not include a type parameter to carry over default constructor types from nested structs. To work around this limitation, extract the constructor for the nested struct and apply it to its fields directly. This ensures that the nested defaults are respected. **Example** ### Lazy Evaluation of Defaults Defaults are lazily evaluated, meaning that a new instance of the default is generated every time the constructor is called: **Example** In this example, the `timestamp` field generates a new value for each instance. ### Reusing Defaults Across Schemas Default values are also "portable", meaning that if you reuse the same property signature in another schema, the default is carried over: **Example** ### Using Defaults in Classes Default values can also be applied when working with the `Class` API, ensuring consistency across class-based schemas. **Example** --- title: Effect Data Types description: Transform and manage various data types with schemas for enhanced JSON serialization, including support for options, eithers, sets, maps, durations, and sensitive redacted data. sidebar: order: 13 --- import { Aside } from "@astrojs/starlight/components" ## Interop With Data The module in the Effect ecosystem simplifies value comparison by automatically implementing the and traits. This eliminates the need for manual implementations, making equality checks straightforward. **Example** By default, schemas like `Schema.Struct` do not implement the `Equal` and `Hash` traits. This means that two decoded objects with identical values will not be considered equal. **Example** The `Schema.Data` function can be used to enhance a schema by including the `Equal` and `Hash` traits. This allows the resulting objects to support value-based equality. **Example** ## Config The `Schema.Config` function allows you to decode and manage application configuration settings using structured schemas. It ensures consistency in configuration data and provides detailed feedback for decoding errors. **Syntax** This function takes two arguments: - `name`: Identifier for the configuration setting. - `schema`: Schema describing the expected data type and structure. It returns a object that integrates with your application's configuration system. The Encoded type `I` must extend `string`, so the schema must be able to decode from a string, this includes schemas like `Schema.String`, `Schema.Literal`, or `Schema.NumberFromString`, possibly with refinements applied. Behind the scenes, `Schema.Config` follows these steps: 1. **Fetch the value** using the provided name . 2. **Decode the value** using the given schema. If the value is invalid, decoding fails. 3. **Format any errors** using , which helps produce readable and detailed error messages. **Example** To test the configuration, execute the following commands: **Test** **Test** **Test** ## Option ### Option The `Schema.Option` function is useful for converting an `Option` into a JSON-serializable format. **Syntax** ##### Decoding | Input | Output | | ---------------------------- | ----------------------------------------------------------------------------------- | | `{ _tag: "None" }` | Converted to `Option.none` | | `{ _tag: "Some", value: I }` | Converted to `Option.some`, where `I` is decoded into `A` using the inner schema | ##### Encoding | Input | Output | | ---------------- | ----------------------------------------------------------------------------------------------- | | `Option.none` | Converted to `{ _tag: "None" }` | | `Option.some` | Converted to `{ _tag: "Some", value: I }`, where `A` is encoded into `I` using the inner schema | **Example** ### OptionFromSelf The `Schema.OptionFromSelf` function is designed for scenarios where `Option` values are already in the `Option` format and need to be decoded or encoded while transforming the inner value according to the provided schema. **Syntax** #### Decoding | Input | Output | | ---------------- | ----------------------------------------------------------------------------------- | | `Option.none` | Remains as `Option.none` | | `Option.some` | Converted to `Option.some`, where `I` is decoded into `A` using the inner schema | #### Encoding | Input | Output | | ---------------- | ----------------------------------------------------------------------------------- | | `Option.none` | Remains as `Option.none` | | `Option.some` | Converted to `Option.some`, where `A` is encoded into `I` using the inner schema | **Example** ### OptionFromUndefinedOr The `Schema.OptionFromUndefinedOr` function handles cases where `undefined` is treated as `Option.none`, and all other values are interpreted as `Option.some` based on the provided schema. **Syntax** #### Decoding | Input | Output | | ----------- | ----------------------------------------------------------------------------------- | | `undefined` | Converted to `Option.none` | | `I` | Converted to `Option.some`, where `I` is decoded into `A` using the inner schema | #### Encoding | Input | Output | | ---------------- | ---------------------------------------------------------------------- | | `Option.none` | Converted to `undefined` | | `Option.some` | Converted to `I`, where `A` is encoded into `I` using the inner schema | **Example** ### OptionFromNullOr The `Schema.OptionFromUndefinedOr` function handles cases where `null` is treated as `Option.none`, and all other values are interpreted as `Option.some` based on the provided schema. **Syntax** #### Decoding | Input | Output | | ------ | ----------------------------------------------------------------------------------- | | `null` | Converted to `Option.none` | | `I` | Converted to `Option.some`, where `I` is decoded into `A` using the inner schema | #### Encoding | Input | Output | | ---------------- | ---------------------------------------------------------------------- | | `Option.none` | Converted to `null` | | `Option.some` | Converted to `I`, where `A` is encoded into `I` using the inner schema | **Example** ### OptionFromNullishOr The `Schema.OptionFromNullishOr` function handles cases where `null` or `undefined` are treated as `Option.none`, and all other values are interpreted as `Option.some` based on the provided schema. Additionally, it allows customization of how `Option.none` is encoded . **Syntax** #### Decoding | Input | Output | | ----------- | ----------------------------------------------------------------------------------- | | `undefined` | Converted to `Option.none` | | `null` | Converted to `Option.none` | | `I` | Converted to `Option.some`, where `I` is decoded into `A` using the inner schema | #### Encoding | Input | Output | | ---------------- | -------------------------------------------------------------------------- | | `Option.none` | Converted to `undefined` or `null` based on user choice | | `Option.some` | Converted to `I`, where `A` is encoded into `I` using the inner schema | **Example** ### OptionFromNonEmptyTrimmedString The `Schema.OptionFromNonEmptyTrimmedString` schema is designed for handling strings where trimmed empty strings are treated as `Option.none`, and all other strings are converted to `Option.some`. #### Decoding | Input | Output | | ----------- | ------------------------------------------------------- | | `s: string` | Converted to `Option.some`, if `s.trim.length > 0` | | | Converted to `Option.none` otherwise | #### Encoding | Input | Output | | ------------------------ | ----------------- | | `Option.none` | Converted to `""` | | `Option.some` | Converted to `s` | **Example** ## Either ### Either The `Schema.Either` function is useful for converting an `Either` into a JSON-serializable format. **Syntax** ##### Decoding | Input | Output | | ------------------------------ | ----------------------------------------------------------------------------------------------- | | `{ _tag: "Left", left: LI }` | Converted to `Either.left`, where `LI` is decoded into `LA` using the inner `left` schema | | `{ _tag: "Right", right: RI }` | Converted to `Either.right`, where `RI` is decoded into `RA` using the inner `right` schema | ##### Encoding | Input | Output | | ------------------ | ----------------------------------------------------------------------------------------------------------- | | `Either.left` | Converted to `{ _tag: "Left", left: LI }`, where `LA` is encoded into `LI` using the inner `left` schema | | `Either.right` | Converted to `{ _tag: "Right", right: RI }`, where `RA` is encoded into `RI` using the inner `right` schema | **Example** ### EitherFromSelf The `Schema.EitherFromSelf` function is designed for scenarios where `Either` values are already in the `Either` format and need to be decoded or encoded while transforming the inner valued according to the provided schemas. **Syntax** ##### Decoding | Input | Output | | ------------------ | ----------------------------------------------------------------------------------------------- | | `Either.left` | Converted to `Either.left`, where `LI` is decoded into `LA` using the inner `left` schema | | `Either.right` | Converted to `Either.right`, where `RI` is decoded into `RA` using the inner `right` schema | ##### Encoding | Input | Output | | ------------------ | ----------------------------------------------------------------------------------------------- | | `Either.left` | Converted to `Either.left`, where `LA` is encoded into `LI` using the inner `left` schema | | `Either.right` | Converted to `Either.right`, where `RA` is encoded into `RI` using the inner `right` schema | **Example** ### EitherFromUnion The `Schema.EitherFromUnion` function is designed to decode and encode `Either` values where the `left` and `right` sides are represented as distinct types. This schema enables conversions between raw union types and structured `Either` types. **Syntax** ##### Decoding | Input | Output | | ----- | ----------------------------------------------------------------------------------------------- | | `LI` | Converted to `Either.left`, where `LI` is decoded into `LA` using the inner `left` schema | | `RI` | Converted to `Either.right`, where `RI` is decoded into `RA` using the inner `right` schema | ##### Encoding | Input | Output | | ------------------ | --------------------------------------------------------------------------------- | | `Either.left` | Converted to `LI`, where `LA` is encoded into `LI` using the inner `left` schema | | `Either.right` | Converted to `RI`, where `RA` is encoded into `RI` using the inner `right` schema | **Example** ## Exit ### Exit The `Schema.Exit` function is useful for converting an `Exit` into a JSON-serializable format. **Syntax** ##### Decoding | Input | Output | | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `{ _tag: "Failure", cause: CauseEncoded }` | Converted to `Exit.failCause`, where `CauseEncoded` is decoded into `Cause` using the inner `failure` and `defect` schemas | | `{ _tag: "Success", value: SI }` | Converted to `Exit.succeed`, where `SI` is decoded into `SA` using the inner `success` schema | ##### Encoding | Input | Output | | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `Exit.failCause` | Converted to `{ _tag: "Failure", cause: CauseEncoded }`, where `Cause` is encoded into `CauseEncoded` using the inner `failure` and `defect` schemas | | `Exit.succeed` | Converted to `{ _tag: "Success", value: SI }`, where `SA` is encoded into `SI` using the inner `success` schema | **Example** ### Handling Defects in Serialization Effect provides a built-in `Defect` schema to handle JavaScript errors and other types of unrecoverable defects. - When decoding, it reconstructs `Error` instances if the input has a `message` and optionally a `name` and `stack`. - When encoding, it converts `Error` instances into plain objects that retain only essential properties. This is useful when transmitting errors across network requests or logging systems where `Error` objects do not serialize by default. **Example** ### ExitFromSelf The `Schema.ExitFromSelf` function is designed for scenarios where `Exit` values are already in the `Exit` format and need to be decoded or encoded while transforming the inner valued according to the provided schemas. **Syntax** ##### Decoding | Input | Output | | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `Exit.failCause` | Converted to `Exit.failCause`, where `Cause` is decoded into `Cause` using the inner `failure` and `defect` schemas | | `Exit.succeed` | Converted to `Exit.succeed`, where `SI` is decoded into `SA` using the inner `success` schema | ##### Encoding | Input | Output | | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `Exit.failCause` | Converted to `Exit.failCause`, where `Cause` is decoded into `Cause` using the inner `failure` and `defect` schemas | | `Exit.succeed` | Converted to `Exit.succeed`, where `SA` is encoded into `SI` using the inner `success` schema | **Example** ## ReadonlySet ### ReadonlySet The `Schema.ReadonlySet` function is useful for converting a `ReadonlySet` into a JSON-serializable format. **Syntax** ##### Decoding | Input | Output | | ------------------ | ----------------------------------------------------------------------------------- | | `ReadonlyArray` | Converted to `ReadonlySet`, where `I` is decoded into `A` using the inner schema | ##### Encoding | Input | Output | | ---------------- | ------------------------------------------------------------------------ | | `ReadonlySet` | `ReadonlyArray`, where `A` is encoded into `I` using the inner schema | **Example** ### ReadonlySetFromSelf The `Schema.ReadonlySetFromSelf` function is designed for scenarios where `ReadonlySet` values are already in the `ReadonlySet` format and need to be decoded or encoded while transforming the inner values according to the provided schema. **Syntax** ##### Decoding | Input | Output | | ---------------- | ----------------------------------------------------------------------------------- | | `ReadonlySet` | Converted to `ReadonlySet`, where `I` is decoded into `A` using the inner schema | ##### Encoding | Input | Output | | ---------------- | ---------------------------------------------------------------------- | | `ReadonlySet` | `ReadonlySet`, where `A` is encoded into `I` using the inner schema | **Example** ## ReadonlyMap The `Schema.ReadonlyMap` function is useful for converting a `ReadonlyMap` into a JSON-serializable format. ### ReadonlyMap **Syntax** ##### Decoding | Input | Output | | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ReadonlyArray` | Converted to `ReadonlyMap`, where `KI` is decoded into `KA` using the inner `key` schema and `VI` is decoded into `VA` using the inner `value` schema | ##### Encoding | Input | Output | | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ReadonlyMap` | Converted to `ReadonlyArray`, where `KA` is decoded into `KI` using the inner `key` schema and `VA` is decoded into `VI` using the inner `value` schema | **Example** ### ReadonlyMapFromSelf The `Schema.ReadonlyMapFromSelf` function is designed for scenarios where `ReadonlyMap` values are already in the `ReadonlyMap` format and need to be decoded or encoded while transforming the inner values according to the provided schemas. **Syntax** ##### Decoding | Input | Output | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ReadonlyMap` | Converted to `ReadonlyMap`, where `KI` is decoded into `KA` using the inner `key` schema and `VI` is decoded into `VA` using the inner `value` schema | ##### Encoding | Input | Output | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `ReadonlyMap` | Converted to `ReadonlyMap`, where `KA` is decoded into `KI` using the inner `key` schema and `VA` is decoded into `VI` using the inner `value` schema | **Example** ### ReadonlyMapFromRecord The `Schema.ReadonlyMapFromRecord` function is a utility to transform a `ReadonlyMap` into an object format, where keys are strings and values are serializable, and vice versa. **Syntax** #### Decoding | Input | Output | | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------ | | `{ readonly : VI }` | Converts to `ReadonlyMap`, where `x` is decoded into `KA` using the `key` schema and `VI` into `VA` using the `value` schema | #### Encoding | Input | Output | | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | `ReadonlyMap` | Converts to `{ readonly : VI }`, where `KA` is encoded into `x` using the `key` schema and `VA` into `VI` using the `value` schema | **Example** ## HashSet ### HashSet The `Schema.HashSet` function provides a way to map between `HashSet` and an array representation, allowing for JSON serialization and deserialization. **Syntax** #### Decoding | Input | Output | | ------------------ | --------------------------------------------------------------------------------------------------- | | `ReadonlyArray` | Converts to `HashSet`, where each element in the array is decoded into type `A` using the schema | #### Encoding | Input | Output | | ------------ | ------------------------------------------------------------------------------------------------------------- | | `HashSet` | Converts to `ReadonlyArray`, where each element in the `HashSet` is encoded into type `I` using the schema | **Example** ### HashSetFromSelf The `Schema.HashSetFromSelf` function is designed for scenarios where `HashSet` values are already in the `HashSet` format and need to be decoded or encoded while transforming the inner values according to the provided schema. **Syntax** #### Decoding | Input | Output | | ------------ | ------------------------------------------------------------------------------------------ | | `HashSet` | Converts to `HashSet`, decoding each element from type `I` to type `A` using the schema | #### Encoding | Input | Output | | ------------ | ------------------------------------------------------------------------------------------ | | `HashSet` | Converts to `HashSet`, encoding each element from type `A` to type `I` using the schema | **Example** ## HashMap ### HashMap The `Schema.HashMap` function is useful for converting a `HashMap` into a JSON-serializable format. **Syntax** | Input | Output | | ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | `ReadonlyArray` | Converts to `HashMap`, where `KI` is decoded into `KA` and `VI` is decoded into `VA` using the specified schemas | #### Encoding | Input | Output | | ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | | `HashMap` | Converts to `ReadonlyArray`, where `KA` is encoded into `KI` and `VA` is encoded into `VI` using the specified schemas | **Example** ### HashMapFromSelf The `Schema.HashMapFromSelf` function is designed for scenarios where `HashMap` values are already in the `HashMap` format and need to be decoded or encoded while transforming the inner values according to the provided schemas. **Syntax** #### Decoding | Input | Output | | ----------------- | ------------------------------------------------------------------------------------------------------------------------ | | `HashMap` | Converts to `HashMap`, where `KI` is decoded into `KA` and `VI` is decoded into `VA` using the specified schemas | #### Encoding | Input | Output | | ----------------- | ------------------------------------------------------------------------------------------------------------------------ | | `HashMap` | Converts to `HashMap`, where `KA` is encoded into `KI` and `VA` is encoded into `VI` using the specified schemas | **Example** ## SortedSet ### SortedSet The `Schema.SortedSet` function provides a way to map between `SortedSet` and an array representation, allowing for JSON serialization and deserialization. **Syntax** #### Decoding | Input | Output | | ------------------ | ----------------------------------------------------------------------------------------------------- | | `ReadonlyArray` | Converts to `SortedSet`, where each element in the array is decoded into type `A` using the schema | #### Encoding | Input | Output | | -------------- | --------------------------------------------------------------------------------------------------------------- | | `SortedSet` | Converts to `ReadonlyArray`, where each element in the `SortedSet` is encoded into type `I` using the schema | **Example** ### SortedSetFromSelf The `Schema.SortedSetFromSelf` function is designed for scenarios where `SortedSet` values are already in the `SortedSet` format and need to be decoded or encoded while transforming the inner values according to the provided schema. **Syntax** #### Decoding | Input | Output | | -------------- | -------------------------------------------------------------------------------------------- | | `SortedSet` | Converts to `SortedSet`, decoding each element from type `I` to type `A` using the schema | #### Encoding | Input | Output | | -------------- | -------------------------------------------------------------------------------------------- | | `SortedSet` | Converts to `SortedSet`, encoding each element from type `A` to type `I` using the schema | **Example** ## Duration The `Duration` schema family enables the transformation and validation of duration values across various formats, including `hrtime`, milliseconds, and nanoseconds. ### Duration Converts an hrtime into a `Duration`. **Example** ### DurationFromSelf The `DurationFromSelf` schema is designed to validate that a given value conforms to the `Duration` type. **Example** ### DurationFromMillis Converts a `number` into a `Duration` where the number represents the number of milliseconds. **Example** ### DurationFromNanos Converts a `BigInt` into a `Duration` where the number represents the number of nanoseconds. **Example** ### clampDuration Clamps a `Duration` between a minimum and a maximum value. **Example** ## Redacted ### Redacted The `Schema.Redacted` function is specifically designed to handle sensitive information by converting a `string` into a object. This transformation ensures that the sensitive data is not exposed in the application's output. **Example** It's important to note that when successfully decoding a `Redacted`, the output is intentionally obscured to prevent the actual secret from being revealed in logs or console outputs. **Example** In the example below, if the input string does not meet the criteria , the error message generated might inadvertently expose sensitive information included in the input. #### Mitigating Exposure Risks To reduce the risk of sensitive information leakage in error messages, you can customize the error messages to obscure sensitive details: **Example** ### RedactedFromSelf The `Schema.RedactedFromSelf` schema is designed to validate that a given value conforms to the `Redacted` type from the `effect` library. **Example** It's important to note that when successfully decoding a `Redacted`, the output is intentionally obscured to prevent the actual secret from being revealed in logs or console outputs. --- title: Schema to Equivalence description: Generate and customize equivalence checks for data structures based on schema definitions. sidebar: label: Equivalence order: 17 --- The `Schema.equivalence` function allows you to generate an based on a schema definition. This function is designed to compare data structures for equivalence according to the rules defined in the schema. **Example** ## Equivalence for Any, Unknown, and Object When working with the following schemas: - `Schema.Any` - `Schema.Unknown` - `Schema.Object` - `Schema.Struct` the most sensible form of equivalence is to use `Equal.equals` from the module, which defaults to reference equality . This is because these types can hold almost any kind of value. **Example** ## Customizing Equivalence Generation You can customize the equivalence logic by providing an `equivalence` annotation in the schema definition. The `equivalence` annotation takes any type parameters provided and two values for comparison, returning a boolean based on the desired condition of equivalence. **Example** --- title: Error Formatters description: Format and customize error messages during schema decoding and encoding using TreeFormatter or ArrayFormatter. sidebar: order: 10 --- When working with Effect Schema, errors encountered during decoding or encoding operations can be formatted using two built-in methods: `TreeFormatter` and `ArrayFormatter`. These formatters help structure and present errors in a readable and actionable manner. ## TreeFormatter The `TreeFormatter` is the default method for formatting errors. It organizes errors in a tree structure, providing a clear hierarchy of issues. **Example** In this example: - `{ readonly name: string; readonly age: number }` describes the schema's expected structure. - `` identifies the specific field causing the error. - `is missing` explains the issue for the `"name"` field. ### Customizing the Output You can make the error output more concise and meaningful by annotating the schema with annotations like `identifier`, `title`, or `description`. These annotations replace the default TypeScript-like representation in the error messages. **Example** Adding a `title` annotation replaces the schema structure in the error message with the more human-readable "Person" making it easier to understand. ### Handling Multiple Errors By default, decoding functions like `Schema.decodeUnknownEither` report only the first error. To list all errors, use the `{ errors: "all" }` option. **Example** ### ParseIssueTitle Annotation The `parseIssueTitle` annotation allows you to add dynamic context to error messages by generating titles based on the value being validated. For instance, it can include an ID from the validated object, making it easier to identify specific issues in complex or nested data structures. **Annotation Type** **Return Value**: - If the function returns a `string`, the `TreeFormatter` uses it as the title unless a `message` annotation is present . - If the function returns `undefined`, the `TreeFormatter` determines the title based on the following priority: 1. `identifier` annotation 2. `title` annotation 3. `description` annotation 4. Default TypeScript-like schema representation **Example** ## ArrayFormatter The `ArrayFormatter` provides a structured, array-based approach to formatting errors. It represents each error as an object, making it easier to analyze and address multiple issues during data decoding or encoding. Each error object includes properties like `_tag`, `path`, and `message` for clarity. **Example** In this example: - `_tag`: Indicates the type of error . - `path`: Specifies the location of the error in the data . - `message`: Describes the issue . ### Handling Multiple Errors By default, decoding functions like `Schema.decodeUnknownEither` report only the first error. To list all errors, use the `{ errors: "all" }` option. **Example** ## React Hook Form If you are working with React and need form validation, `@hookform/resolvers` offers an adapter for `effect/Schema`, which can be integrated with React Hook Form for enhanced form validation processes. This integration allows you to leverage the powerful features of `effect/Schema` within your React applications. For more detailed instructions and examples on how to integrate `effect/Schema` with React Hook Form using `@hookform/resolvers`, you can visit the official npm package page: --- title: Error Messages description: Customize and enhance error messages for schema decoding with default, refined, and custom messages. sidebar: order: 9 --- ## Default Error Messages By default, when a parsing error occurs, the system automatically generates an informative message based on the schema's structure and the nature of the error for more informations). For example, if a required property is missing or a data type does not match, the error message will clearly state the expectation versus the actual input. **Example** **Example** **Example** ### Enhancing Clarity in Error Messages with Identifiers In scenarios where a schema has multiple fields or nested structures, the default error messages can become overly complex and verbose. To address this, you can enhance the clarity and brevity of these messages by utilizing annotations such as `identifier`, `title`, and `description`. **Example** ### Refinements When a refinement fails, the default error message indicates whether the failure occurred in the "from" part or within the predicate defining the refinement: **Example** In the first example, the error message indicates a "from side" refinement failure in the `name` property, specifying that a string was expected but received `null`. In the second example, a "predicate" refinement failure is reported, indicating that a non-empty string was expected for `name` but an empty string was provided. ### Transformations Transformations between different types or formats can occasionally result in errors. The system provides a structured error message to specify where the error occurred: - **Encoded Side Failure:** Errors on this side typically indicate that the input to the transformation does not match the expected initial type or format. For example, receiving a `null` when a `string` is expected. - **Transformation Process Failure:** This type of error arises when the transformation logic itself fails, such as when the input does not meet the criteria specified within the transformation functions. - **Type Side Failure:** Occurs when the output of a transformation does not meet the schema requirements on the decoded side. This can happen if the transformed value fails subsequent validations or conditions. **Example** ## Custom Error Messages You have the capability to define custom error messages specifically tailored for different parts of your schema using the `message` annotation. This allows developers to provide more context-specific feedback which can improve the debugging and validation processes. Here's an overview of the `MessageAnnotation` type, which you can use to craft these messages: | Return Type | Description | | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `string` | Provides a static message that directly describes the error. | | `Effect` | Utilizes dynamic messages that can incorporate results from **synchronous** processes or rely on **optional** dependencies. | | Object | Allows you to define a specific error message along with a boolean flag . This flag determines if the custom message should supersede any default or nested custom messages, providing precise control over the error output displayed to users. | **Example** **Example** ### General Guidelines for Messages The general logic followed to determine the messages is as follows: 1. If no custom messages are set, the default message related to the innermost schema where the operation failed is used. 2. If custom messages are set, then the message corresponding to the **first** failed schema is used, starting from the innermost schema to the outermost. However, if the failing schema does not have a custom message, then **the default message is used**. 3. As an opt-in feature, **you can override guideline 2** by setting the `override` flag to `true`. This allows the custom message to take precedence over all other custom messages from inner schemas. This is to address the scenario where a user wants to define a single cumulative custom message describing the properties that a valid value must have and does not want to see default messages. Let's see some practical examples. ### Scalar Schemas **Example** ### Refinements This example demonstrates setting a custom message on the last refinement in a chain of refinements. As you can see, the custom message is only used if the refinement related to `maxLength` fails; otherwise, default messages are used. **Example** When setting multiple custom messages, the one corresponding to the **first** failed predicate is used, starting from the innermost refinement to the outermost: **Example** You have the option to change the default behavior by setting the `override` flag to `true`. This is useful when you want to create a single comprehensive custom message that describes the required properties of a valid value without displaying default messages. **Example** ### Transformations In this example, `IntFromString` is a transformation schema that converts strings to integers. It applies specific validation messages based on different scenarios. **Example** ### Compound Schemas The custom message system becomes especially handy when dealing with complex schemas, unlike simple scalar values like `string` or `number`. For instance, consider a schema comprising nested structures, such as a struct containing an array of other structs. Let's explore an example demonstrating the advantage of default messages in handling decoding errors within such nested structures: **Example** ### Effectful messages Error messages can go beyond simple strings by returning an `Effect`, allowing them to access dependencies, such as an internationalization service. This approach lets messages dynamically adjust based on external context or services. Below is an example illustrating how to create effect-based messages. **Example** ### Missing messages You can provide custom messages for missing fields or tuple elements using the `missingMessage` annotation. **Example** In this example, a custom message is defined for a missing `name` property in the `Person` schema. **Example** Here, each element in the `Point` tuple schema has a specific custom message if the element is missing. --- title: Filters description: Define custom validation logic with filters to enhance data validation beyond basic type checks. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" Developers can define custom validation logic beyond basic type checks, giving more control over how data is validated. ## Declaring Filters Filters are declared using the `Schema.filter` function. This function requires two arguments: the schema to be validated and a predicate function. The predicate function is user-defined and determines whether the data satisfies the condition. If the data fails the validation, an error message can be provided. **Example** Note that the filter does not alter the schema's `Type`: Filters add additional validation constraints without modifying the schema's underlying type. ## The Predicate Function The predicate function in a filter follows this structure: where The filter's predicate can return several types of values, each affecting validation in a different way: | Return Type | Behavior | | ----------------------------- | ------------------------------------------------------------------------------------------------ | | `true` or `undefined` | The data satisfies the filter's condition and passes validation. | | `false` | The data does not meet the condition, and no specific error message is provided. | | `string` | The validation fails, and the provided string is used as the error message. | | `ParseResult.ParseIssue` | The validation fails with a detailed error structure, specifying where and why it failed. | | `FilterIssue` | Allows for more detailed error messages with specific paths, providing enhanced error reporting. | | `ReadonlyArray` | An array of issues can be returned if multiple validation errors need to be reported. | ## Adding Annotations Embedding metadata within the schema, such as identifiers, JSON schema specifications, and descriptions, enhances understanding and analysis of the schema's constraints and purpose. **Example** ## Specifying Error Paths When validating forms or structured data, it's possible to associate specific error messages with particular fields or paths. This enhances error reporting and is especially useful when integrating with libraries like . **Example** In this example, we define a `MyForm` schema with two password fields . We use `Schema.filter` to check that both passwords match. If they don't, an error message is returned, specifically associated with the `confirm_password` field. This makes it easier to pinpoint the exact location of the validation failure. The error is formatted in a structured way using `ArrayFormatter`, allowing for easier post-processing and integration with form libraries. ## Multiple Error Reporting The `Schema.filter` API supports reporting multiple validation issues at once, which is especially useful in scenarios like form validation where several checks might fail simultaneously. **Example** In this example, we define a `MyForm` schema with fields for password validation and optional name/surname fields. The `Schema.filter` function checks if the passwords match and ensures that either a name or surname is provided. If either validation fails, the corresponding error message is associated with the relevant field and both errors are returned in a structured format. ## Exposed Values For schemas with filters, you can access the base schema using the `from` property: ## Built-in Filters ### String Filters Here is a list of useful string filters provided by the Schema module: ### Number Filters Here is a list of useful number filters provided by the Schema module: ### ReadonlyArray Filters Here is a list of useful array filters provided by the Schema module: ### Date Filters ### BigInt Filters Here is a list of useful `BigInt` filters provided by the Schema module: ### BigDecimal Filters Here is a list of useful `BigDecimal` filters provided by the Schema module: ### Duration Filters Here is a list of useful filters provided by the Schema module: --- title: Getting Started description: Learn how to define schemas, extract types, and handle decoding and encoding. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" You can import the necessary types and functions from the `effect/Schema` module: **Example** **Example** ## Defining a schema One common way to define a `Schema` is by utilizing the `Struct` constructor. This constructor allows you to create a new schema that outlines an object with specific properties. Each property in the object is defined by its own schema, which specifies the data type and any validation rules. **Example** This `Person` schema describes an object with a `name` and `age` property: ## Extracting Inferred Types ### Type Once you've defined a schema , you can extract the inferred type `Type` in two ways: 1. Using the `Schema.Type` utility 2. Accessing the `Type` field directly on your schema **Example** The resulting type will look like this: Alternatively, you can extract the `Person` type using the `interface` keyword, which may improve readability and performance in some cases. **Example** Both approaches yield the same result, but using an interface provides benefits such as performance advantages and improved readability. ### Encoded In a `Schema`, the `Encoded` type can differ from the `Type` type, representing the format in which data is encoded. You can extract the `Encoded` type in two ways: 1. Using the `Schema.Encoded` utility 2. Accessing the `Encoded` field directly on the schema **Example** The resulting type is: Note that `age` is of type `string` in the `Encoded` type of the schema and is of type `number` in the `Type` type of the schema. Alternatively, you can define the `PersonEncoded` type using the `interface` keyword, which can enhance readability and performance. **Example** Both approaches yield the same result, but using an interface provides benefits such as performance advantages and improved readability. ### Context In a `Schema`, the `Context` type represents any external data or dependencies that the schema requires to perform encoding or decoding. You can extract the inferred `Context` type in two ways: 1. Using the `Schema.Context` utility. 2. Accessing the `Context` field on the schema. **Example** ### Schemas with Opaque Types When defining a schema, you may want to create a schema with an opaque type. This is useful when you want to hide the internal structure of the schema and only expose the type of the schema. **Example** To create a schema with an opaque type, you can use the following technique that re-declares the schema: Alternatively, you can use the Class APIs section for more details). Note that the technique shown above becomes more complex when the schema is defined such that `Type` is different from `Encoded`. **Example** In this case, the field `"age"` is of type `string` in the `Encoded` type of the schema and is of type `number` in the `Type` type of the schema. Therefore, we need to define **two** interfaces and use both to redeclare our final schema `Person`. ## Readonly Types by Default It's important to note that by default, most constructors exported by `effect/Schema` return `readonly` types. **Example** For instance, in the `Person` schema below: the resulting inferred `Type` would be: ## Decoding When working with unknown data types in TypeScript, decoding them into a known structure can be challenging. Luckily, `effect/Schema` provides several functions to help with this process. Let's explore how to decode unknown values using these functions. | API | Description | | ---------------------- | ----------------------------------------------------------------------------------- | | `decodeUnknownSync` | Synchronously decodes a value and throws an error if parsing fails. | | `decodeUnknownOption` | Decodes a value and returns an type. | | `decodeUnknownEither` | Decodes a value and returns an type. | | `decodeUnknownPromise` | Decodes a value and returns a `Promise`. | | `decodeUnknown` | Decodes a value and returns an . | ### decodeUnknownSync The `Schema.decodeUnknownSync` function is useful when you want to parse a value and immediately throw an error if the parsing fails. **Example** ### decodeUnknownEither The `Schema.decodeUnknownEither` function allows you to parse a value and receive the result as an , representing success or failure . This approach lets you handle parsing errors more gracefully without throwing exceptions. **Example** ### decodeUnknown If your schema involves asynchronous transformations, the `Schema.decodeUnknownSync` and `Schema.decodeUnknownEither` functions will not be suitable. In such cases, you should use the `Schema.decodeUnknown` function, which returns an . **Example** In the code above, the first approach using `Schema.decodeUnknownEither` results in an error indicating that the transformation cannot be resolved synchronously. This occurs because `Schema.decodeUnknownEither` is not designed for async operations. The second approach, which uses `Schema.decodeUnknown`, works correctly, allowing you to handle asynchronous transformations and return the expected result. ## Encoding The `Schema` module provides several `encode*` functions to encode data according to a schema: | API | Description | | --------------- | ------------------------------------------------------------------------------------------------------- | | `encodeSync` | Synchronously encodes data and throws an error if encoding fails. | | `encodeOption` | Encodes data and returns an type. | | `encodeEither` | Encodes data and returns an type representing success or failure. | | `encodePromise` | Encodes data and returns a `Promise`. | | `encode` | Encodes data and returns an . | **Example** Note that during encoding, the number value `30` was converted to a string `"30"`. ### Handling Unsupported Encoding In certain cases, it may not be feasible to support encoding for a schema. While it is generally advised to define schemas that allow both decoding and encoding, there are situations where encoding a particular type is either unsupported or unnecessary. In these instances, the `Forbidden` issue can signal that encoding is not available for certain values. **Example** Here is an example of a transformation that never fails during decoding. It returns an containing either the decoded value or the original input. For encoding, it is reasonable to not support it and use `Forbidden` as the result. **Explanation** - **Decoding**: The `SafeDecode` function ensures that decoding never fails. It wraps the decoded value in an , where a successful decoding results in a `Right` and a failed decoding results in a `Left` containing the original input. - **Encoding**: The encoding process uses the `Forbidden` error to indicate that encoding a `Left` value is not supported. Only `Right` values are successfully encoded. ## ParseError The `Schema.decodeUnknownEither` and `Schema.encodeEither` functions returns a : where `ParseError` is defined as follows : In this structure, `ParseIssue` represents an error that might occur during the parsing process. It is wrapped in a tagged error to make it easier to catch errors using . The result `Either` contains the inferred data type described by the schema . A successful parse yields a `Right` value with the parsed data `Type`, while a failed parse results in a `Left` value containing a `ParseError`. ## Parse Options The options below provide control over both decoding and encoding behaviors. ### Managing Excess properties By default, any properties not defined in the schema are removed from the output when parsing a value. This ensures the parsed data conforms strictly to the expected structure. If you want to detect and handle unexpected properties, use the `onExcessProperty` option , which allows you to raise an error for excess properties. This can be helpful when you need to validate and catch unanticipated properties. **Example** To retain extra properties, set `onExcessProperty` to `"preserve"`. **Example** ### Receive all errors The `errors` option enables you to retrieve all errors encountered during parsing. By default, only the first error is returned. Setting `errors` to `"all"` provides comprehensive error feedback, which can be useful for debugging or offering detailed validation feedback. **Example** ### Managing Property Order The `propertyOrder` option provides control over the order of object fields in the output. This feature is particularly useful when the sequence of keys is important for the consuming processes or when maintaining the input order enhances readability and usability. By default, the `propertyOrder` option is set to `"none"`. This means that the internal system decides the order of keys to optimize parsing speed. The order of keys in this mode should not be considered stable, and it's recommended not to rely on key ordering as it may change in future updates. Setting `propertyOrder` to `"original"` ensures that the keys are ordered as they appear in the input during the decoding/encoding process. **Example** **Example** ### Customizing Parsing Behavior at the Schema Level The `parseOptions` annotation allows you to customize parsing behavior at different schema levels, enabling you to apply unique parsing settings to nested schemas within a structure. Options defined within a schema override parent-level settings and apply to all nested schemas. **Example** **Detailed Output Explanation:** In this example: - The main schema is configured to display all errors. Hence, you will see errors related to both the `d` field and any errors from the `a` subschema. - The subschema is set to display only the first error. Although both `b` and `c` fields are missing, only the first missing field is reported. ## Type Guards The `Schema.is` function provides a way to verify if a value conforms to a given schema. It acts as a , taking a value of type `unknown` and determining if it matches the structure and type constraints defined in the schema. Here's how the `Schema.is` function works: 1. **Schema Definition**: Define a schema to describe the structure and constraints of the data type you expect. For instance, `Schema`, where `Type` is the target type you want to validate against. 2. **Type Guard Creation**: Use the schema to create a user-defined type guard, ` => u is Type`. This function can be used at runtime to check if a value meets the requirements of the schema. **Example** The generated `isPerson` function has the following signature: ## Assertions While type guards verify whether a value conforms to a specific type, the `Schema.asserts` function goes further by asserting that an input matches the schema type `Type` . If the input does not match the schema, it throws a detailed error, making it useful for runtime validation. **Example** The `assertsPerson` function generated from the schema has the following signature: ## Managing Missing Properties When decoding, it's important to understand how missing properties are processed. By default, if a property is not present in the input, it is treated as if it were present with an `undefined` value. **Example** In this example, although the key `"a"` is not present in the input, it is treated as `{ a: undefined }` by default. If you need your validation logic to differentiate between genuinely missing properties and those explicitly set to `undefined`, you can enable the `exact` option. **Example** For the APIs `Schema.is` and `Schema.asserts`, however, the default behavior is to treat missing properties strictly, where the default for `exact` is `true`: **Example** ## Naming Conventions The naming conventions in `effect/Schema` are designed to be straightforward and logical, **focusing primarily on compatibility with JSON serialization**. This approach simplifies the understanding and use of schemas, especially for developers who are integrating web technologies where JSON is a standard data interchange format. ### Overview of Naming Strategies **JSON-Compatible Types** Schemas that naturally serialize to JSON-compatible formats are named directly after their data types. For instance: - `Schema.Date`: serializes JavaScript Date objects to ISO-formatted strings, a typical method for representing dates in JSON. - `Schema.Number`: used directly as it maps precisely to the JSON number type, requiring no special transformation to remain JSON-compatible. **Non-JSON-Compatible Types** When dealing with types that do not have a direct representation in JSON, the naming strategy incorporates additional details to indicate the necessary transformation. This helps in setting clear expectations about the schema's behavior: For instance: - `Schema.DateFromSelf`: indicates that the schema handles `Date` objects, which are not natively JSON-serializable. - `Schema.NumberFromString`: this naming suggests that the schema processes numbers that are initially represented as strings, emphasizing the transformation from string to number when decoding. The primary goal of these schemas is to ensure that domain objects can be easily serialized and deserialized for transmission over network connections, thus facilitating their transfer between different parts of an application or across different applications. ### Rationale While JSON's ubiquity justifies its primary consideration in naming, the conventions also accommodate serialization for other types of transport. For instance, converting a `Date` to a string is a universally useful method for various communication protocols, not just JSON. Thus, the selected naming conventions serve as sensible defaults that prioritize clarity and ease of use, facilitating the serialization and deserialization processes across diverse technological environments. --- title: Introduction to Effect Schema description: "Introduction to `effect/Schema`, a module for defining, validating, and transforming data schemas." sidebar: label: Introduction order: 0 --- import { Aside } from "@astrojs/starlight/components" Welcome to the documentation for `effect/Schema`, a module for defining and using schemas to validate and transform data in TypeScript. The `effect/Schema` module allows you to define a `Schema` that provides a blueprint for describing the structure and data types of your data. Once defined, you can leverage this schema to perform a range of operations, including: | Operation | Description | | --------------- | ------------------------------------------------------------------------------------ | | Decoding | Transforming data from an input type `Encoded` to an output type `Type`. | | Encoding | Converting data from an output type `Type` back to an input type `Encoded`. | | Asserting | Verifying that a value adheres to the schema's output type `Type`. | | Standard Schema | Generate a . | | Arbitraries | Generate arbitraries for testing. | | JSON Schemas | Create JSON Schemas based on defined schemas. | | Equivalence | Create based on defined schemas. | | Pretty printing | Support pretty printing for data structures. | ## Requirements - TypeScript 5.4 or newer. - The `strict` flag enabled in your `tsconfig.json` file. - The `exactOptionalPropertyTypes` flag enabled in your `tsconfig.json` file. ### The exactOptionalPropertyTypes Option The `effect/Schema` module takes advantage of the `exactOptionalPropertyTypes` option of `tsconfig.json`. This option affects how optional properties are typed ). **Example** Here, notice that the type of `name` is "exact" , which means the type checker will catch any attempt to assign an invalid value . **Example** If, for some reason, you can't enable the `exactOptionalPropertyTypes` option , you can still use `effect/Schema`. However, there will be a mismatch between the types and the runtime behavior: In this case, the type of `name` is widened to `string | undefined`, which means the type checker won't catch the invalid value . However, during decoding, you'll encounter an error, indicating that `undefined` is not allowed. ## The Schema Type A schema is an immutable value that describes the structure of your data, and it is represented by the `Schema` type. Here is the general form of a `Schema`: The `Schema` type has three type parameters with the following meanings: | Parameter | Description | | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Type** | Represents the type of value that a schema can succeed with during decoding. | | **Encoded** | Represents the type of value that a schema can succeed with during encoding. By default, it's equal to `Type` if not explicitly provided. | | **Requirements** | Similar to the type, it represents the contextual data required by the schema to execute both decoding and encoding. If this type parameter is `never` , it means the schema has no requirements. | **Examples** - `Schema` represents a schema that decodes to `string`, encodes to `string`, and has no requirements. - `Schema` represents a schema that decodes to `number` from `string`, encodes a `number` to a `string`, and has no requirements. ## Understanding Schema Values **Immutability**. `Schema` values are immutable, and every function in the `effect/Schema` module produces a new `Schema` value. **Modeling Data Structure**. These values do not perform any actions themselves, they simply model or describe the structure of your data. **Interpretation by Compilers**. A `Schema` can be interpreted by various "compilers" into specific operations, depending on the compiler type . ## Understanding Decoding and Encoding When working with data in TypeScript, you often need to handle data coming from or being sent to external systems. This data may not always match the format or types you expect, especially when dealing with user input, data from APIs, or data stored in different formats. To handle these discrepancies, we use **decoding** and **encoding**. | Term | Description | | ------------ | ------------------------------------------------------------------------------------------------------------ | | **Decoding** | Used for parsing data from external sources where you have no control over the data format. | | **Encoding** | Used when sending data out to external sources, converting it to a format that is expected by those sources. | For instance, when working with forms in the frontend, you often receive untyped data in the form of strings. This data can be tampered with and does not natively support arrays or booleans. Decoding helps you validate and parse this data into more useful types like numbers, dates, and arrays. Encoding allows you to convert these types back into the string format expected by forms. Below is a diagram that shows the relationship between encoding and decoding using a `Schema`: We'll break down these concepts using an example with a `Schema`. This schema serves as a tool to transform a `string` into a `Date` and vice versa. ### Encoding When we talk about "encoding," we are referring to the process of changing a `Date` into a `string`. To put it simply, it's the act of converting data from one format to another. ### Decoding Conversely, "decoding" entails transforming a `string` back into a `Date`. It's essentially the reverse operation of encoding, where data is returned to its original form. ### Decoding From Unknown Decoding from `unknown` involves two key steps: 1. **Checking:** Initially, we verify that the input data matches the expected structure. In our specific case, this means ensuring that the input is indeed a `string`. 2. **Decoding:** Following the successful check, we proceed to convert the `string` into a `Date`. This process completes the decoding operation, where the data is both validated and transformed. ### Encoding From Unknown Encoding from `unknown` involves two key steps: 1. **Checking:** Initially, we verify that the input data matches the expected structure. In our specific case, this means ensuring that the input is indeed a `Date`. 2. **Encoding:** Following the successful check, we proceed to convert the `Date` into a `string`. This process completes the encoding operation, where the data is both validated and transformed. ## The Rule of Schemas When working with schemas, there's an important rule to keep in mind: your schemas should be crafted in a way that when you perform both encoding and decoding operations, you should end up with the original value. In simpler terms, if you encode a value and then immediately decode it, the result should match the original value you started with. This rule ensures that your data remains consistent and reliable throughout the encoding and decoding process. --- title: Schema to JSON Schema description: Convert schema definitions into JSON Schema for data validation and interoperability. sidebar: label: JSON Schema order: 16 --- The `JSONSchema.make` function allows you to generate a JSON Schema from a schema. **Example** The following example defines a `Person` schema with properties for `name` and `age` . It then generates the corresponding JSON Schema. The `JSONSchema.make` function aims to produce an optimal JSON Schema representing the input part of the decoding phase. It does this by traversing the schema from the most nested component, incorporating each refinement, and **stops at the first transformation** encountered. **Example** Consider modifying the `age` field to include both a refinement and a transformation. Only the refinement is reflected in the JSON Schema. In this case, the JSON Schema reflects the integer refinement but does not include the transformation that clamps the value. ## Targeting a Specific JSON Schema Version By default, `JSONSchema.make` generates a JSON Schema compatible with **Draft 07**. You can change the target schema version by passing an options object with a `target` property. The supported targets are: - `"jsonSchema7"` - JSON Schema Draft 07 - `"jsonSchema2019-09"` - JSON Schema Draft 2019-09 - `"jsonSchema2020-12"` - JSON Schema Draft 2020-12 - `"openApi3.1"` - OpenAPI 3.1 Changing the target can affect the generated output. For example, tuple schemas use `items` and `additionalItems` in Draft 07, whereas Draft 2020-12 uses `prefixItems` and `items`. **Example** ## Specific Outputs for Schema Types ### Literals Literals are transformed into `enum` types within JSON Schema. **Example** **Example** ### Void ### Any ### Unknown ### Object ### String ### Number ### Boolean ### Tuples ### Arrays ### Non Empty Arrays Represents an array with at least one element. **Example** ### Structs ### Records ### Mixed Structs with Records Combines fixed properties from a struct with dynamic properties from a record. **Example** ### Enums ### Template Literals ### Unions Unions are expressed using `anyOf` or `enum`, depending on the types involved: **Example** **Example** ## Identifier Annotations You can add `identifier` annotations to schemas to improve structure and maintainability. Annotated schemas are included in a `$defs` object in the root of the JSON Schema and referenced from there. **Example** By using identifier annotations, schemas can be reused and referenced more easily, especially in complex JSON Schemas. ## Standard JSON Schema Annotations Standard JSON Schema annotations such as `title`, `description`, `default`, and `examples` are supported. These annotations allow you to enrich your schemas with metadata that can enhance readability and provide additional information about the data structure. **Example** ### Adding annotations to Struct properties To enhance the clarity of your JSON schemas, it's advisable to add annotations directly to the property signatures rather than to the type itself. This method is more semantically appropriate as it links descriptive titles and other metadata specifically to the properties they describe, rather than to the generic type. **Example** ## Recursive and Mutually Recursive Schemas Recursive and mutually recursive schemas are supported, however it's **mandatory** to use `identifier` annotations for these types of schemas to ensure correct references and definitions within the generated JSON Schema. **Example** In this example, the `Category` schema refers to itself, making it necessary to use an `identifier` annotation to facilitate the reference. ## Customizing JSON Schema Generation When working with JSON Schema certain data types, such as `bigint`, lack a direct representation because JSON Schema does not natively support them. This absence typically leads to an error when the schema is generated. **Example** Attempting to generate a JSON Schema for unsupported types like `bigint` will lead to a missing annotation error: To address this, you can enhance the schema with a custom `jsonSchema` annotation, defining how you intend to represent such types in JSON Schema: **Example** ### Refinements When defining a refinement , you can include a JSON Schema annotation to describe the refinement. This annotation is added as a "fragment" that becomes part of the generated JSON Schema. If a schema contains multiple refinements, their respective annotations are merged into the output. **Example** The `jsonSchema` annotation is defined as a generic object, allowing it to represent non-standard extensions. This flexibility leaves the responsibility of enforcing type constraints to the user. If you prefer stricter type enforcement or need to support non-standard extensions, you can introduce a `satisfies` constraint on the object literal. This constraint should be used in conjunction with the typing library of your choice. **Example** In the following example, we've used the `@types/json-schema` package to provide TypeScript definitions for JSON Schema. This approach not only ensures type correctness but also enables autocomplete suggestions in your IDE. For schema types other than refinements, you can override the default generated JSON Schema by providing a custom `jsonSchema` annotation. The content of this annotation will replace the system-generated schema. **Example** ## Specialized JSON Schema Generation with Schema.parseJson The `Schema.parseJson` function provides a unique approach to JSON Schema generation. Instead of defaulting to a schema for a plain string, which represents the "from" side of the transformation, it generates a schema based on the structure provided within the argument. This behavior ensures that the generated JSON Schema reflects the intended structure of the parsed data, rather than the raw JSON input. **Example** --- title: Schema to Pretty Printer description: Generate formatted string representations of values based on schemas. sidebar: label: Pretty Printer order: 18 --- The `Pretty.make` function is used to create pretty printers that generate a formatted string representation of values based on a schema. **Example** ## Customizing Pretty Printer Generation You can customize how the pretty printer formats output by using the `pretty` annotation within your schema definition. The `pretty` annotation takes any type parameters provided and formats the value into a string. **Example** --- title: Schema Projections description: Create new schemas by extracting and customizing the Type or Encoded components of existing schemas. sidebar: label: Projections order: 6 --- Sometimes, you may want to create a new schema based on an existing one, focusing specifically on either its `Type` or `Encoded` aspect. The Schema module provides several functions to make this possible. ## typeSchema The `Schema.typeSchema` function is used to extract the `Type` portion of a schema, resulting in a new schema that retains only the type-specific properties from the original schema. This excludes any initial encoding or transformation logic applied to the original schema. **Function Signature** **Example** ## encodedSchema The `Schema.encodedSchema` function enables you to extract the `Encoded` portion of a schema, creating a new schema that matches the original properties but **omits any refinements or transformations** applied to the schema. **Function Signature** **Example** ## encodedBoundSchema The `Schema.encodedBoundSchema` function is similar to `Schema.encodedSchema` but preserves the refinements up to the first transformation point in the original schema. **Function Signature** The term "bound" in this context refers to the boundary up to which refinements are preserved when extracting the encoded form of a schema. It essentially marks the limit to which initial validations and structure are maintained before any transformations are applied. **Example** --- title: Schema to Standard Schema description: Generate Standard Schema V1. sidebar: label: Standard Schema order: 14 --- import { Aside } from "@astrojs/starlight/components" The `Schema.standardSchemaV1` API allows you to generate a object from an Effect `Schema`. **Example** ## Sync vs Async Validation The `Schema.standardSchemaV1` API creates a schema whose `validate` method attempts to decode and validate the provided input synchronously. If the underlying `Schema` includes any asynchronous components , then validation will necessarily return a `Promise` instead. **Example** ## Defects If an unexpected defect occurs during validation, it is reported as a single issue without a `path`. This ensures that unexpected errors do not disrupt schema validation but are still captured and reported. **Example** --- title: Schema Transformations description: Transform and manipulate data with schema-based transformations, including type conversions, validations, and custom processing. sidebar: label: Transformations order: 7 --- import { Aside } from "@astrojs/starlight/components" Transformations are important when working with schemas. They allow you to change data from one type to another. For example, you might parse a string into a number or convert a date string into a `Date` object. The and functions help you connect two schemas so you can convert data between them. ## transform `Schema.transform` creates a new schema by taking the output of one schema and making it the input of another schema . Use this when you know the transformation will always succeed. If it might fail, use instead. ### Understanding Input and Output "Output" and "input" depend on what you are doing : **When decoding:** - The source schema `Schema` produces a `SourceType`. - The target schema `Schema` expects a `TargetEncoded`. - The decoding path looks like this: `SourceEncoded` → `TargetType`. If `SourceType` and `TargetEncoded` differ, you can provide a `decode` function to convert the source schema's output into the target schema's input. **When encoding:** - The target schema `Schema` produces a `TargetEncoded`. - The source schema `Schema` expects a `SourceType`. - The encoding path looks like this: `TargetType` → `SourceEncoded`. If `TargetEncoded` and `SourceType` differ, you can provide an `encode` function to convert the target schema's output into the source schema's input. ### Combining Two Primitive Schemas In this example, we start with a schema that accepts `"on"` or `"off"` and transform it into a boolean schema. The `decode` function turns `"on"` into `true` and `"off"` into `false`. The `encode` function does the reverse. This gives us a `Schema`. **Example** The `decode` function above never fails by itself. However, the full decoding process can still fail if the input does not fit the source schema. For example, if you provide `"wrong"` instead of `"on"` or `"off"`, the source schema will fail before calling `decode`. **Example** ### Combining Two Transformation Schemas Below is an example where both the source and target schemas transform their data: - The source schema is `Schema.NumberFromString`, which is `Schema`. - The target schema is `BooleanFromString` , which is `Schema`. This example involves four types and requires two conversions: - When decoding, convert a `number` into `"on" | "off"`. For example, treat any positive number as `"on"`. - When encoding, convert `"on" | "off"` back into a `number`. For example, treat `"on"` as `1` and `"off"` as `-1`. By composing these transformations, we get a schema that decodes a string into a boolean and encodes a boolean back into a string. The resulting schema is `Schema`. **Example** **Example** In this example, we convert an array into a `ReadonlySet`. The `decode` function takes an array and creates a new `ReadonlySet`. The `encode` function converts the set back into an array. We also provide the schema of the array items so they are properly validated. ### Non-strict option In some cases, strict type checking can create issues during data transformations, especially when the types might slightly differ in specific transformations. To address these scenarios, `Schema.transform` offers the option `strict: false`, which relaxes type constraints and allows more flexible transformations. **Example** Let's consider the scenario where you need to define a constructor `clamp` that ensures a number falls within a specific range. This function returns a schema that "clamps" a number to a specified minimum and maximum range: In this example, `Number.clamp` returns a `number` that might not be recognized as the specific `A` type, which leads to a type mismatch under strict checking. There are two ways to resolve this issue: 1. **Using Type Assertion**: Adding a type cast can enforce the return type to be treated as type `A`: 2. **Using the Non-Strict Option**: Setting `strict: false` in the transformation options allows the schema to bypass some of TypeScript's type-checking rules, accommodating the type discrepancy: ## transformOrFail While the function is suitable for error-free transformations, the `Schema.transformOrFail` function is designed for more complex scenarios where **transformations can fail** during the decoding or encoding stages. This function enables decoding/encoding functions to return either a successful result or an error, making it particularly useful for validating and processing data that might not always conform to expected formats. ### Error Handling The `Schema.transformOrFail` function utilizes the ParseResult module to manage potential errors: | Constructor | Description | | --------------------- | ------------------------------------------------------------------------------------------------ | | `ParseResult.succeed` | Indicates a successful transformation, where no errors occurred. | | `ParseResult.fail` | Signals a failed transformation, creating a new `ParseError` based on the provided `ParseIssue`. | Additionally, the ParseResult module provides constructors for dealing with various types of parse issues, such as: | Parse Issue Type | Description | | ---------------- | --------------------------------------------------------------------------------------------- | | `Type` | Indicates a type mismatch error. | | `Missing` | Used when a required field is missing. | | `Unexpected` | Used for unexpected fields that are not allowed in the schema. | | `Forbidden` | Flags the decoding or encoding operation being forbidden by the schema. | | `Pointer` | Points to a specific location in the data where an issue occurred. | | `Refinement` | Used when a value does not meet a specific refinement or constraint. | | `Transformation` | Flags issues that occur during transformation from one type to another. | | `Composite` | Represents a composite error, combining multiple issues into one, helpful for grouped errors. | These tools allow for detailed and specific error handling, enhancing the reliability of data processing operations. **Example** A common use case for `Schema.transformOrFail` is converting string representations of numbers into actual numeric types. This scenario is typical when dealing with user inputs or data from external sources. Both `decode` and `encode` functions not only receive the value to transform , but also the that the user sets when using the resulting schema, and the `ast`, which represents the low level definition of the schema you're transforming. ### Async Transformations In modern applications, especially those interacting with external APIs, you might need to transform data asynchronously. `Schema.transformOrFail` supports asynchronous transformations by allowing you to return an `Effect`. **Example** Consider a scenario where you need to validate a person's ID by making an API call. Here's how you can implement it: ### Declaring Dependencies In cases where your transformation depends on external services, you can inject these services in the `decode` or `encode` functions. These dependencies are then tracked in the `Requirements` channel of the schema: **Example** ## One-Way Transformations with Forbidden Encoding In some cases, encoding a value back to its original form may not make sense or may be undesirable. You can use `Schema.transformOrFail` to define a one-way transformation and explicitly return a `Forbidden` parse error during the encoding process. This ensures that once a value is transformed, it cannot be reverted to its original form. **Example** Consider a scenario where you need to hash a user's plain text password for secure storage. It is important that the hashed password cannot be reversed back to plain text. By using `Schema.transformOrFail`, you can enforce this restriction, ensuring a one-way transformation from plain text to a hashed password. ## Composition Combining and reusing schemas is often needed in complex applications, and the `Schema.compose` combinator provides an efficient way to do this. With `Schema.compose`, you can chain two schemas, `Schema` and `Schema`, into a single schema `Schema`: **Example** ### Non-strict Option When composing schemas, you may encounter cases where the output of one schema does not perfectly match the input of the next, for example, if you have `Schema` and `Schema` where `C` differs from `B`. To handle these cases, you can use the `{ strict: false }` option to relax type constraints. **Example** ## Effectful Filters The `Schema.filterEffect` function enables validations that require asynchronous or dynamic scenarios, making it suitable for cases where validations involve side effects like network requests or database queries. For simple synchronous validations, see . **Example** ## String Transformations ### split Splits a string by a specified delimiter into an array of substrings. **Example** ### Trim Removes whitespace from the beginning and end of a string. **Example** ### Lowercase Converts a string to lowercase. **Example** ### Uppercase Converts a string to uppercase. **Example** ### Capitalize Converts the first character of a string to uppercase. **Example** ### Uncapitalize Converts the first character of a string to lowercase. **Example** ### parseJson The `Schema.parseJson` constructor offers a method to convert JSON strings into the `unknown` type using the underlying functionality of `JSON.parse`. It also employs `JSON.stringify` for encoding. **Example** To further refine the result of JSON parsing, you can provide a schema to the `Schema.parseJson` constructor. This schema will validate that the parsed JSON matches a specific structure. **Example** In this example, `Schema.parseJson` uses a struct schema to ensure the parsed JSON is an object with a numeric property `a`. This adds validation to the parsed data, confirming that it follows the expected structure. ### StringFromBase64 Decodes a base64 encoded string into a UTF-8 string. **Example** ### StringFromBase64Url Decodes a base64 encoded string into a UTF-8 string. **Example** ### StringFromHex Decodes a hex encoded string into a UTF-8 string. **Example** ### StringFromUriComponent Decodes a URI-encoded string into a UTF-8 string. It is useful for encoding and decoding data in URLs. **Example** ## Number Transformations ### NumberFromString Transforms a string into a number by parsing the string using the `parse` function of the `effect/Number` module. It returns an error if the value can't be converted . The following special string values are supported: "NaN", "Infinity", "-Infinity". **Example** ### clamp Restricts a number within a specified range. **Example** ### parseNumber Transforms a string into a number by parsing the string using the `parse` function of the `effect/Number` module. It returns an error if the value can't be converted . The following special string values are supported: "NaN", "Infinity", "-Infinity". **Example** ## Boolean Transformations ### Not Negates a boolean value. **Example** ## Symbol transformations ### Symbol Converts a string to a symbol using `Symbol.for`. **Example** ## BigInt transformations ### BigInt Converts a string to a `BigInt` using the `BigInt` constructor. **Example** ### BigIntFromNumber Converts a number to a `BigInt` using the `BigInt` constructor. **Example** ### clampBigInt Restricts a `BigInt` within a specified range. **Example** ## Date transformations ### Date Converts a string into a **valid** `Date`, ensuring that invalid dates, such as `new Date`, are rejected. **Example** ## BigDecimal Transformations ### BigDecimal Converts a string to a `BigDecimal`. **Example** ### BigDecimalFromNumber Converts a number to a `BigDecimal`. **Example** ### clampBigDecimal Clamps a `BigDecimal` within a specified range. **Example** --- title: Sink Concurrency excerpt: Learn how to enhance performance with concurrent sink operations, such as combining results or racing to capture the first completion. sidebar: label: Concurrency order: 3 --- This section covers concurrent operations that allow multiple sinks to run simultaneously. These can be valuable for enhancing task performance when concurrent execution is desired. ## Combining Results with Concurrent Zipping To run two sinks concurrently and combine their results, use `Sink.zip`. This operation executes both sinks concurrently and combines their outcomes into a tuple. **Example** ## Racing Sinks: First Completion Wins The `Sink.race` operation allows multiple sinks to compete for completion. The first sink to finish provides the result. **Example** --- title: Creating Sinks description: Discover how to create and use various sinks for processing streams, including counting, summing, collecting, folding, and handling success or failure. sidebar: order: 1 --- In stream processing, sinks are used to consume and handle elements from a stream. Here, we'll explore various sink constructors that allow you to create sinks for specific tasks. ## Common Constructors ### head The `Sink.head` sink retrieves only the first element from a stream, wrapping it in `Some`. If the stream has no elements, it returns `None`. **Example** ### last The `Sink.last` sink retrieves only the last element from a stream, wrapping it in `Some`. If the stream has no elements, it returns `None`. **Example** ### count The `Sink.count` sink consumes all elements of the stream and counts the number of elements fed to it. ### sum The `Sink.sum` sink consumes all elements of the stream and sums incoming numeric values. ### take The `Sink.take` sink takes the specified number of values from the stream and results in a data type. ### drain The `Sink.drain` sink ignores its inputs, effectively discarding them. ### timed The `Sink.timed` sink executes the stream and measures its execution time, providing the . ### forEach The `Sink.forEach` sink executes the provided effectful function for every element fed to it. ## Creating Sinks from Success and Failure Just as you can define streams to hold or manipulate data, you can also create sinks with specific success or failure outcomes using the `Sink.fail` and `Sink.succeed` functions. ### Succeeding Sink This example creates a sink that doesn’t consume any elements from its upstream source but instead immediately succeeds with a specified numeric value: **Example** ### Failing Sink In this example, the sink also doesn’t consume any elements from its upstream source. Instead, it fails with a specified error message of type `string`: **Example** ## Collecting ### Collecting All Elements To gather all elements from a data stream into a , use the `Sink.collectAll` sink. The final output is a chunk containing all elements from the stream, in the order they were emitted. **Example** ### Collecting a Specified Number To collect a fixed number of elements from a stream into a , use `Sink.collectAllN`. This sink stops collecting once it reaches the specified limit. **Example** ### Collecting While Meeting a Condition To gather elements from a stream while they satisfy a specific condition, use `Sink.collectAllWhile`. This sink collects elements until the provided predicate returns `false`. **Example** ### Collecting into a HashSet To accumulate stream elements into a `HashSet`, use `Sink.collectAllToSet`. This ensures that each element appears only once in the final set. **Example** ### Collecting into HashSets of a Specific Size For controlled collection into a `HashSet` with a specified maximum size, use `Sink.collectAllToSetN`. This sink gathers unique elements up to the given limit. **Example** ### Collecting into a HashMap For more complex collection scenarios, `Sink.collectAllToMap` lets you gather elements into a `HashMap` with a specified keying and merging strategy. This sink requires both a key function to define each element's grouping and a merge function to combine values sharing the same key. **Example** In this example, we use ` => n % 3` to determine map keys and ` => a + b` to merge elements with the same key: ### Collecting into a HashMap with Limited Keys To accumulate elements into a `HashMap` with a maximum number of keys, use `Sink.collectAllToMapN`. This sink collects elements until it reaches the specified key limit, requiring a key function to define the grouping of each element and a merge function to combine values with the same key. **Example** ## Folding ### Folding Left If you want to reduce a stream into a single cumulative value by applying an operation to each element in sequence, you can use the `Sink.foldLeft` function. **Example** ### Folding with Termination Sometimes, you may want to fold elements in a stream but stop the process once a specific condition is met. This is known as "short-circuiting." You can accomplish this with the `Sink.fold` function, which lets you define a termination condition. **Example** ### Folding Until a Limit To accumulate elements until a specific count is reached, use `Sink.foldUntil`. This sink folds elements up to the specified limit and then stops. **Example** ### Folding with Weighted Elements In some scenarios, you may want to fold elements based on a defined "weight" or "cost," accumulating elements until a specified maximum cost is reached. You can accomplish this with `Sink.foldWeighted`. **Example** In the example below, each element has a weight of `1`, and the folding resets when the accumulated weight hits `3`. --- title: Introduction description: Learn the role of Sink in stream processing, handling element consumption, error management, result production, and leftover elements. sidebar: order: 0 --- In stream processing, a `Sink` is a construct designed to consume elements generated by a `Stream`. Here's an overview of what a `Sink` does: - It consumes a varying number of `In` elements, which may include zero, one, or multiple elements. - It can encounter errors of type `E` during processing. - It produces a result of type `A` once processing completes. - It can also return a remainder of type `L`, representing any leftover elements. To process a stream using a `Sink`, you can pass it directly to the `Stream.run` function: **Example** The type of `sink` is as follows: Here's the breakdown: - `Chunk`: The final result produced by the sink after processing elements of numbers). - `number` : The type of elements that the sink will consume from the stream. - `number` : The type of leftover elements, if any, that are not consumed. - `never` : Indicates that this sink does not produce any errors. - `never` : Shows that no dependencies are required to operate this sink. --- title: Leftovers description: Learn how to handle unconsumed elements in streams, collecting or ignoring leftovers for efficient data processing. sidebar: order: 4 --- In this section, we'll look at handling elements left unconsumed by sinks. Sinks may process only a portion of the elements from an upstream source, leaving some elements as "leftovers." Here's how to collect or ignore these remaining elements. ## Collecting Leftovers If a sink doesn't consume all elements from the upstream source, the remaining elements are called leftovers. To capture these leftovers, use `Sink.collectLeftover`, which returns a tuple containing the result of the sink operation and any unconsumed elements. **Example** ## Ignoring Leftovers If leftover elements are not needed, you can ignore them using `Sink.ignoreLeftover`. This approach discards any unconsumed elements, so the sink operation focuses only on the elements it needs. **Example** --- title: Sink Operations description: Explore operations to transform, filter, and adapt sinks, enabling custom input-output handling and element filtering in stream processing. sidebar: label: Operations order: 2 --- In previous sections, we learned how to create and use sinks. Now, let's explore some operations that let you transform or filter sink behavior. ## Adapting Sink Input At times, you may have a sink that works with one type of input, but your current stream uses a different type. The `Sink.mapInput` function helps you adapt your sink to a new input type by transforming the input values. While `Sink.map` changes the sink's output, `Sink.mapInput` changes the input it accepts. **Example** Suppose you have a `Sink.sum` that calculates the sum of numbers. If your stream contains strings rather than numbers, `Sink.mapInput` can convert those strings into numbers, allowing `Sink.sum` to work with your stream: ## Transforming Both Input and Output When you need to transform both the input and output of a sink, `Sink.dimap` provides a flexible solution. It extends `mapInput` by allowing you to transform the input type, perform the operation, and then transform the output to a new type. This can be useful for complete conversions between input and output types. **Example** ## Filtering Input Sinks can also filter incoming elements based on specific conditions with `Sink.filterInput`. This operation allows the sink to process only elements that meet certain criteria. **Example** In the example below, elements are collected in chunks of three, but only positive numbers are included: --- title: Ref description: Learn how to manage state in concurrent applications using Effect's Ref data type. Master mutable references for safe, controlled state updates across fibers. sidebar: order: 0 --- import { Aside } from "@astrojs/starlight/components" When we write programs, it is common to need to keep track of some form of state during the execution of the program. State refers to any data that can change as the program runs. For example, in a counter application, the count value changes as the user increments or decrements it. Similarly, in a banking application, the account balance changes as deposits and withdrawals are made. State management is crucial to building interactive and dynamic applications. In traditional imperative programming, one common way to store state is using variables. However, this approach can introduce bugs, especially when the state is shared between multiple components or functions. As the program becomes more complex, managing shared state can become challenging. To overcome these issues, Effect introduces a powerful data type called `Ref`, which represents a mutable reference. With `Ref`, we can share state between different parts of our program without relying on mutable variables directly. Instead, `Ref` provides a controlled way to handle mutable state and safely update it in a concurrent environment. Effect's `Ref` data type enables communication between different fibers in your program. This capability is crucial in concurrent programming, where multiple tasks may need to access and update shared state simultaneously. In this guide, we will explore how to use the `Ref` data type to manage state in your programs effectively. We will cover simple examples like counting, as well as more complex scenarios where state is shared between different parts of the program. Additionally, we will show how to use `Ref` in a concurrent environment, allowing multiple tasks to interact with shared state safely. Let's dive in and see how we can leverage `Ref` for effective state management in your Effect programs. ## Using Ref Here is a simple example using `Ref` to create a counter: **Example** **Example** ## Using Ref in a Concurrent Environment We can also use `Ref` in concurrent scenarios, where multiple tasks might be updating shared state at the same time. **Example** For this example, let's update the counter concurrently: ## Using Ref as a Service You can pass a `Ref` as a to share state across different parts of your program. **Example** Note that we use `Effect.provideServiceEffect` instead of `Effect.provideService` to provide an actual implementation of the `MyState` service because all the operations on the `Ref` data type are effectful, including the creation `Ref.make`. ## Sharing State Between Fibers You can use `Ref` to manage shared state between multiple fibers in a concurrent environment. **Example** Let's look at an example where we continuously read names from user input until the user enters `"q"` to exit. First, let's introduce a `readLine` utility to read user input : Next, we implement the main program to collect names: Now that we have learned how to use the `Ref` data type, we can use it to manage the state concurrently. For example, assume while we are reading from the console, we have another fiber that is trying to update the state from a different source. Here, one fiber reads names from user input, while another fiber concurrently adds preset names at regular intervals: --- title: SubscriptionRef description: Learn how to manage shared state with SubscriptionRef in Effect, enabling multiple observers to subscribe to and react to state changes efficiently in concurrent environments. sidebar: order: 2 --- A `SubscriptionRef` is a specialized form of a . It allows us to subscribe and receive updates on the current value and any changes made to that value. You can perform all standard operations on a `SubscriptionRef`, such as `get`, `set`, or `modify` to interact with the current value. The key feature of `SubscriptionRef` is its `changes` stream. This stream allows you to observe the current value at the moment of subscription and receive all subsequent changes. Every time the stream is run, it emits the current value and tracks future updates. To create a `SubscriptionRef`, you can use the `SubscriptionRef.make` constructor, specifying the initial value: **Example** `SubscriptionRef` is particularly useful for modeling shared state when multiple observers need to react to changes. For example, in functional reactive programming, the `SubscriptionRef` could represent a portion of the application state, and various observers would update in response to state changes. **Example** In the following example, a "server" continually updates a shared value, while multiple "clients" observe the changes: The `server` function operates on a regular `Ref` and continuously updates the value. It doesn't need to know about `SubscriptionRef` directly. Next, let's define a `client` that subscribes to changes and collects a specified number of values: Similarly, the `client` function only works with a `Stream` of values and doesn't concern itself with the source of these values. To tie everything together, we start the server, launch multiple client instances in parallel, and then shut down the server when we're finished. We also create the `SubscriptionRef` in this process. This setup ensures that each client observes the current value when it starts and receives all subsequent changes to the value. Since the changes are represented as streams, you can easily build more complex programs using familiar stream operators. You can transform, filter, or merge these streams with other streams to achieve more sophisticated behavior. --- title: SynchronizedRef description: Master concurrent state management with SynchronizedRef in Effect, a mutable reference that supports atomic, effectful updates to shared state in concurrent environments. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" `SynchronizedRef` serves as a mutable reference to a value of type `A`. With it, we can store **immutable** data and perform updates **atomically** and effectfully. The distinctive function in `SynchronizedRef` is `updateEffect`. This function takes an effectful operation and executes it to modify the shared state. This is the key feature setting `SynchronizedRef` apart from `Ref`. In real-world applications, `SynchronizedRef` is useful when you need to execute effects, such as querying a database, and then update shared state based on the result. It ensures that updates happen sequentially, preserving consistency in concurrent environments. **Example** In this example, we simulate fetching user ages concurrently and updating a shared state that stores the ages: --- title: Consuming Streams description: Learn techniques for consuming streams, including collecting elements, processing with callbacks, and using folds and sinks. sidebar: order: 2 --- When working with streams, it's essential to understand how to consume the data they produce. In this guide, we'll walk through several common methods for consuming streams. ## Using runCollect To gather all the elements from a stream into a single `Chunk`, you can use the `Stream.runCollect` function. ## Using runForEach Another way to consume elements of a stream is by using `Stream.runForEach`. It takes a callback function that receives each element of the stream. Here's an example: In this example, we use `Stream.runForEach` to log each element to the console. ## Using a Fold Operation The `Stream.fold` function is another way to consume a stream by performing a fold operation over the stream of values and returning an effect containing the result. Here are a couple of examples: In the first example , we use `Stream.runFold` to calculate the sum of all elements. In the second example , we use `Stream.runFoldWhile` to calculate the sum but only until a certain condition is met. ## Using a Sink To consume a stream using a Sink, you can pass the `Sink` to the `Stream.run` function. Here's an example: In this example, we use a `Sink` to calculate the sum of the elements in the stream. --- title: Creating Streams description: Learn various methods for creating Effect streams, from basic constructors to handling asynchronous data sources, pagination, and schedules. sidebar: order: 1 --- In this section, we'll explore various methods for creating Effect `Stream`s. These methods will help you generate streams tailored to your needs. ## Common Constructors ### make You can create a pure stream by using the `Stream.make` constructor. This constructor accepts a variable list of values as its arguments. ### empty Sometimes, you may require a stream that doesn't produce any values. In such cases, you can use `Stream.empty`. This constructor creates a stream that remains empty. ### void If you need a stream that contains a single `void` value, you can use `Stream.void`. This constructor is handy when you want to represent a stream with a single event or signal. ### range To create a stream of integers within a specified range `` , you can use `Stream.range`. This is particularly useful for generating a stream of sequential numbers. ### iterate With `Stream.iterate`, you can generate a stream by applying a function iteratively to an initial value. The initial value becomes the first element produced by the stream, followed by subsequent values produced by `f`, `f)`, and so on. ### scoped `Stream.scoped` is used to create a single-valued stream from a scoped resource. It can be handy when dealing with resources that require explicit acquisition, usage, and release. ## From Success and Failure Much like the `Effect` data type, you can generate a `Stream` using the `fail` and `succeed` functions: ## From Chunks You can construct a stream from a `Chunk` like this: Moreover, you can create a stream from multiple `Chunk`s as well: ## From Effect You can generate a stream from an Effect workflow by employing the `Stream.fromEffect` constructor. For instance, consider the following stream, which generates a single random number: This method allows you to seamlessly transform the output of an Effect into a stream, providing a straightforward way to work with asynchronous operations within your streams. ## From Asynchronous Callback Imagine you have an asynchronous function that relies on callbacks. If you want to capture the results emitted by those callbacks as a stream, you can use the `Stream.async` function. This function is designed to adapt functions that invoke their callbacks multiple times and emit the results as a stream. Let's break down how to use it in the following example: The `StreamEmit.Emit` type represents an asynchronous callback that can be called multiple times. This callback takes a value of type `Effect, Option, R>`. Here's what each of the possible outcomes means: - When the value provided to the callback results in a `Chunk` upon success, it signifies that the specified elements should be emitted as part of the stream. - If the value passed to the callback results in a failure with `Some`, it indicates the termination of the stream with the specified error. - When the value passed to the callback results in a failure with `None`, it serves as a signal for the end of the stream, essentially terminating it. To put it simply, this type allows you to specify how your asynchronous callback interacts with the stream, determining when to emit elements, when to terminate with an error, or when to signal the end of the stream. ## From Iterables ### fromIterable You can create a pure stream from an `Iterable` of values using the `Stream.fromIterable` constructor. It's a straightforward way to convert a collection of values into a stream. ### fromIterableEffect When you have an effect that produces a value of type `Iterable`, you can employ the `Stream.fromIterableEffect` constructor to generate a stream from that effect. For instance, let's say you have a database operation that retrieves a list of users. Since this operation involves effects, you can utilize `Stream.fromIterableEffect` to convert the result into a `Stream`: This enables you to work seamlessly with effects and convert their results into streams for further processing. ### fromAsyncIterable Async iterables are another type of data source that can be converted into a stream. With the `Stream.fromAsyncIterable` constructor, you can work with asynchronous data sources and handle potential errors gracefully. In this code, we define an async iterable and then create a stream named `stream` from it. Additionally, we provide an error handler function to manage any potential errors that may occur during the conversion. ## From Repetition ### Repeating a Single Value You can create a stream that endlessly repeats a specific value using the `Stream.repeatValue` constructor: ### Repeating a Stream's Content `Stream.repeat` allows you to create a stream that repeats a specified stream's content according to a schedule. This can be useful for generating recurring events or values. ### Repeating an Effect's Result Imagine you have an effectful API call, and you want to use the result of that call to create a stream. You can achieve this by creating a stream from the effect and repeating it indefinitely. Here's an example of generating a stream of random numbers: ### Repeating an Effect with Termination You can repeatedly evaluate a given effect and terminate the stream based on specific conditions. In this example, we're draining an `Iterator` to create a stream from it: ### Generating Ticks You can create a stream that emits `void` values at specified intervals using the `Stream.tick` constructor. This is useful for creating periodic events. ## From Unfolding/Pagination In functional programming, the concept of `unfold` can be thought of as the counterpart to `fold`. With `fold`, we process a data structure and produce a return value. For example, we can take an `Array` and calculate the sum of its elements. On the other hand, `unfold` represents an operation where we start with an initial value and generate a recursive data structure, adding one element at a time using a specified state function. For example, we can create a sequence of natural numbers starting from `1` and using the `increment` function as the state function. ### Unfold #### unfold The Stream module includes an `unfold` function defined as follows: Here's how it works: - **initialState**. This is the initial state value. - **step**. The state function `step` takes the current state `s` as input. If the result of this function is `None`, the stream ends. If it's `Some<>`, the next element in the stream is `A`, and the state `S` is updated for the next step process. For example, let's create a stream of natural numbers using `Stream.unfold`: #### unfoldEffect Sometimes, we may need to perform effectful state transformations during the unfolding operation. This is where `Stream.unfoldEffect` comes in handy. It allows us to work with effects while generating streams. Here's an example of creating an infinite stream of random `1` and `-1` values using `Stream.unfoldEffect`: #### Additional Variants There are also similar operations like `Stream.unfoldChunk` and `Stream.unfoldChunkEffect` tailored for working with `Chunk` data types. ### Pagination #### paginate `Stream.paginate` is similar to `Stream.unfold` but allows emitting values one step further. For example, the following stream emits `0, 1, 2, 3` elements: Here's how it works: - We start with an initial value of `0`. - The provided function takes the current value `n` and returns a tuple. The first element of the tuple is the value to emit , and the second element determines whether to continue `) or stop `). #### Additional Variants There are also similar operations like `Stream.paginateChunk` and `Stream.paginateChunkEffect` tailored for working with `Chunk` data types. ### Unfolding vs. Pagination You might wonder about the difference between the `unfold` and `paginate` combinators and when to use one over the other. Let's explore this by diving into an example. Imagine we have a paginated API that provides a substantial amount of data in a paginated manner. When we make a request to this API, it returns a `ResultPage` object containing the results for the current page and a flag indicating whether it's the last page or if there's more data to retrieve on the next page. Here's a simplified representation of our API: Our goal is to convert this paginated API into a stream of `RowData` events. For our initial attempt, we might think that using the `Stream.unfold` operation is the way to go: However, this approach has a drawback, it doesn't include the results from the last page. To work around this, we perform an extra API call to include those missing results: While this approach works, it's clear that `Stream.unfold` isn't the most friendly option for retrieving data from paginated APIs. It requires additional workarounds to include the results from the last page. This is where `Stream.paginate` comes to the rescue. It provides a more ergonomic way to convert a paginated API into an Effect stream. Let's rewrite our solution using `Stream.paginate`: ## From Queue and PubSub In Effect, there are two essential asynchronous messaging data types: and . You can easily transform these data types into `Stream`s by utilizing `Stream.fromQueue` and `Stream.fromPubSub`, respectively. ## From Schedule We can create a stream from a `Schedule` that does not require any further input. The stream will emit an element for each value output from the schedule, continuing for as long as the schedule continues: --- title: Error Handling in Streams description: Learn how to handle errors in streams, ensuring robust recovery, retries, and graceful error management for reliable stream processing. sidebar: label: Error Handling order: 3 --- ## Recovering from Failure When working with streams that may encounter errors, it's crucial to know how to handle these errors gracefully. The `Stream.orElse` function is a powerful tool for recovering from failures and switching to an alternative stream in case of an error. **Example** In this example, `s1` encounters an error, but instead of terminating the stream, we gracefully switch to `s2` using `Stream.orElse`. This ensures that we can continue processing data even if one stream fails. There's also a variant called `Stream.orElseEither` that uses the data type to distinguish elements from the two streams based on success or failure: The `Stream.catchAll` function provides advanced error handling capabilities compared to `Stream.orElse`. With `Stream.catchAll`, you can make decisions based on both the type and value of the encountered failure. In this example, we have a stream, `s1`, which may encounter two different types of errors. Instead of a straightforward switch to an alternative stream, as done with `Stream.orElse`, we employ `Stream.catchAll` to precisely determine how to handle each type of error. This level of control over error recovery enables you to choose different streams or actions based on the specific error conditions. ## Recovering from Defects When working with streams, it's essential to be prepared for various failure scenarios, including defects that might occur during stream processing. To address this, the `Stream.catchAllCause` function provides a robust solution. It enables you to gracefully handle and recover from any type of failure that may arise. **Example** In this example, `s1` may encounter a defect, but instead of crashing the application, we use `Stream.catchAllCause` to gracefully switch to an alternative stream, `s2`. This ensures that your application remains robust and continues processing data even in the face of unexpected issues. ## Recovery from Some Errors In stream processing, there may be situations where you need to recover from specific types of failures. The `Stream.catchSome` and `Stream.catchSomeCause` functions come to the rescue, allowing you to handle and mitigate errors selectively. If you want to recover from a particular error, you can use `Stream.catchSome`: To recover from a specific cause, you can use the `Stream.catchSomeCause` function: ## Recovering to Effect In stream processing, it's crucial to handle errors gracefully and perform cleanup tasks when needed. The `Stream.onError` function allows us to do just that. If our stream encounters an error, we can specify a cleanup task to be executed. ## Retry a Failing Stream Sometimes, streams may encounter failures that are temporary or recoverable. In such cases, the `Stream.retry` operator comes in handy. It allows you to specify a retry schedule, and the stream will be retried according to that schedule. **Example** In this example, the stream asks the user to input a number, but if an invalid value is entered , it fails with "NaN." However, we use `Stream.retry` with an exponential backoff schedule, which means it will retry after a delay of increasing duration. This allows us to handle temporary errors and eventually collect valid input. ## Refining Errors When working with streams, there might be situations where you want to selectively keep certain errors and terminate the stream with the remaining errors. You can achieve this using the `Stream.refineOrDie` function. **Example** In this example, `stream` initially fails with a generic `Error`. However, we use `Stream.refineOrDie` to filter and keep only errors of type `SyntaxError`. Any other errors will be terminated, while `SyntaxErrors` will be retained in `refinedStream`. ## Timing Out When working with streams, there are scenarios where you may want to handle timeouts, such as terminating a stream if it doesn't produce a value within a certain duration. In this section, we'll explore how to manage timeouts using various operators. ### timeout The `Stream.timeout` operator allows you to set a timeout on a stream. If the stream does not produce a value within the specified duration, it terminates. ### timeoutFail The `Stream.timeoutFail` operator combines a timeout with a custom failure message. If the stream times out, it fails with the specified error message. ### timeoutFailCause Similar to `Stream.timeoutFail`, `Stream.timeoutFailCause` combines a timeout with a custom failure cause. If the stream times out, it fails with the specified cause. ### timeoutTo The `Stream.timeoutTo` operator allows you to switch to another stream if the first stream does not produce a value within the specified duration. --- title: Introduction to Streams description: Learn the fundamentals of streams, a powerful tool for emitting multiple values, handling errors, and working with finite or infinite sequences in your applications. sidebar: label: Introduction order: 0 --- In this guide, we'll explore the concept of a `Stream`. A `Stream` is a program description that, when executed, can emit **zero or more values** of type `A`, handle errors of type `E`, and operates within a context of type `R`. ## Use Cases Streams are particularly handy whenever you're dealing with sequences of values over time. They can serve as replacements for observables, node streams, and AsyncIterables. ## What is a Stream? Think of a `Stream` as an extension of an `Effect`. While an `Effect` represents a program that requires a context of type `R`, may encounter an error of type `E`, and always produces a single result of type `A`, a `Stream` takes this further by allowing the emission of zero or more values of type `A`. To clarify, let's examine some examples using `Effect`: In each case, the `Effect` always ends with **exactly one value**. There is no variability; you always get one result. ## Understanding Streams Now, let's shift our focus to `Stream`. A `Stream` represents a program description that shares similarities with `Effect`, it requires a context of type `R`, may signal errors of type `E`, and yields values of type `A`. However, the key distinction is that it can yield **zero or more values**. Here are the possible scenarios for a `Stream`: - **An Empty Stream**: It can end up empty, representing a stream with no values. - **A Single-Element Stream**: It can represent a stream with just one value. - **A Finite Stream of Elements**: It can represent a stream with a finite number of values. - **An Infinite Stream of Elements**: It can represent a stream that continues indefinitely, essentially an infinite stream. Let's see these scenarios in action: In summary, a `Stream` is a versatile tool for representing programs that may yield multiple values, making it suitable for a wide range of tasks, from processing finite lists to handling infinite sequences. --- title: Operations description: Explore essential operations for manipulating and managing data in streams, including tapping, mapping, filtering, merging, and more, to effectively process and transform streaming data. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" In this guide, we'll explore some essential operations you can perform on streams. These operations allow you to manipulate and interact with stream elements in various ways. ## Tapping The `Stream.tap` operation allows you to run an effect on each element emitted by the stream, observing or performing side effects without altering the elements or return type. This can be useful for logging, monitoring, or triggering additional actions with each emission. **Example** For example, `Stream.tap` can be used to log each element before and after a mapping operation: ## Taking Elements The "taking" operations in streams let you extract a specific set of elements, either by a fixed number, condition, or position within the stream. Here are a few ways to apply these operations: | API | Description | | ----------- | ----------------------------------------------------- | | `take` | Extracts a fixed number of elements. | | `takeWhile` | Extracts elements while a certain condition is met. | | `takeUntil` | Extracts elements until a certain condition is met. | | `takeRight` | Extracts a specified number of elements from the end. | **Example** ## Streams as an Alternative to Async Iterables When working with asynchronous data sources, such as async iterables, you often need to consume data in a loop until a certain condition is met. Streams provide a similar approach and offer additional flexibility. With async iterables, data is processed in a loop until a break or return statement is encountered. To replicate this behavior with Streams, consider these options: | API | Description | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `takeUntil` | Takes elements from a stream until a specified condition is met, similar to breaking out of a loop. | | `toPull` | Returns an effect that continuously pulls data chunks from the stream. This effect can fail with `None` when the stream is finished or with `Some` error if it fails. | **Example** ## Mapping ### Basic Mapping The `Stream.map` operation applies a specified function to each element in a stream, creating a new stream with the transformed values. **Example** ### Mapping to a Constant Value The `Stream.as` method allows you to replace each success value in a stream with a specified constant value. This can be useful when you want all elements in the stream to emit a uniform value, regardless of the original data. **Example** ### Effectful Mapping For transformations involving effects, use `Stream.mapEffect`. This function applies an effectful operation to each element in the stream, producing a new stream with effectful results. **Example** To handle multiple effectful transformations concurrently, you can use the option. This option allows a specified number of effects to run concurrently, with results emitted downstream in their original order. **Example** ### Stateful Mapping `Stream.mapAccum` is similar to `Stream.map`, but it applies a transformation with state tracking, allowing you to map and accumulate values within a single operation. This is useful for tasks like calculating a running total in a stream. **Example** ### Mapping and Flattening The `Stream.mapConcat` operation is similar to `Stream.map`, but it goes further by mapping each element to zero or more elements and then flattening the entire stream. This is particularly useful for transforming each element into multiple values. **Example** ## Filtering The `Stream.filter` operation allows you to pass through only elements that meet a specific condition. It's a way to retain elements in a stream that satisfy a particular criteria while discarding the rest. **Example** ## Scanning Stream scanning allows you to apply a function cumulatively to each element in the stream, emitting every intermediate result. Unlike `reduce`, which only provides a final result, `scan` offers a step-by-step view of the accumulation process. **Example** If you need only the final accumulated value, you can use : **Example** ## Draining Stream draining lets you execute effectful operations within a stream while discarding the resulting values. This can be useful when you need to run actions or perform side effects but don't require the emitted values. The `Stream.drain` function achieves this by ignoring all elements in the stream and producing an empty output stream. **Example** ## Detecting Changes in a Stream The `Stream.changes` operation detects and emits elements that differ from their preceding elements within a stream. This can be useful for tracking changes or deduplicating consecutive values. **Example** ## Zipping Zipping combines elements from two streams into a new stream, pairing elements from each input stream. This can be achieved with `Stream.zip` or `Stream.zipWith`, allowing for custom pairing logic. **Example** In this example, elements from the two streams are paired sequentially. The resulting stream ends when one of the streams is exhausted. **Example** Here, `Stream.zipWith` applies custom logic to each pair, combining elements in a user-defined way. ### Handling Stream Endings If one input stream ends before the other, you might want to zip with default values to avoid missing pairs. The `Stream.zipAll` and `Stream.zipAllWith` operators provide this functionality, allowing you to specify defaults for either stream. **Example** In this example, when the second stream completes, the first stream continues with "x" as a default value for the second stream. **Example** With `Stream.zipAllWith`, custom logic determines how to combine elements when either stream runs out, offering flexibility to handle these cases. ### Zipping Streams at Different Rates When combining streams that emit elements at different speeds, you may not want to wait for the slower stream to emit. Using `Stream.zipLatest` or `Stream.zipLatestWith`, you can zip elements as soon as either stream produces a new value. These functions use the most recent element from the slower stream whenever a new value arrives from the faster stream. **Example** ### Pairing with Previous and Next Elements | API | Description | | ------------------------ | --------------------------------------------------------- | | `zipWithPrevious` | Pairs each element of a stream with its previous element. | | `zipWithNext` | Pairs each element of a stream with its next element. | | `zipWithPreviousAndNext` | Pairs each element with both its previous and next. | **Example** ### Indexing Stream Elements The `Stream.zipWithIndex` operator is a helpful tool for indexing each element in a stream, pairing each item with its respective position in the sequence. This is particularly useful when you want to keep track of the order of elements within a stream. **Example** ## Cartesian Product of Streams The Stream module includes a feature for computing the _Cartesian Product_ of two streams, allowing you to create combinations of elements from two different streams. This is helpful when you need to pair each element from one set with every element of another. In simple terms, imagine you have two collections and want to form all possible pairs by picking one item from each. This pairing process is the Cartesian Product. In streams, this operation generates a new stream that includes every possible pairing of elements from the two input streams. To create a Cartesian Product of two streams, the `Stream.cross` operator is available, along with similar variants. These operators combine two streams into a new stream of all possible element combinations. **Example** ## Partitioning Partitioning a stream involves dividing it into two distinct streams based on a specified condition. The Stream module offers two functions for this purpose: `Stream.partition` and `Stream.partitionEither`. Let's look at how these functions work and the best scenarios for their use. ### partition The `Stream.partition` function takes a predicate as input and divides the original stream into two substreams. One substream will contain elements that meet the condition, while the other contains those that do not. Both resulting substreams are wrapped in a `Scope` type. **Example** ### partitionEither In some cases, you might need to partition a stream using a condition that involves an effect. For this, the `Stream.partitionEither` function is ideal. This function uses an effectful predicate to split the stream into two substreams: one for elements that produce `Either.left` values and another for elements that produce `Either.right` values. **Example** ## Grouping When processing streams of data, you may need to group elements based on specific criteria. The Stream module provides two functions for this purpose: `groupByKey`, `groupBy`, `grouped` and `groupedWithin`. Let's review how these functions work and when to use each one. ### groupByKey The `Stream.groupByKey` function partitions a stream based on a key function of type ` => K`, where `A` is the type of elements in the stream, and `K` represents the keys for grouping. This function is non-effectful and groups elements by simply applying the provided key function. The result of `Stream.groupByKey` is a `GroupBy` data type, representing the grouped stream. To process each group, you can use `GroupBy.evaluate`, which takes a function of type ` => Stream.Stream<...>`. This function operates across all groups and merges them together in a non-deterministic order. **Example** In the following example, we use `Stream.groupByKey` to group exam scores by the tens place and count the number of scores in each group: ### groupBy For more complex grouping requirements where partitioning involves effects, you can use the `Stream.groupBy` function. This function accepts an effectful partitioning function and returns a `GroupBy` data type, representing the grouped stream. You can then process each group by using `GroupBy.evaluate`, similar to `Stream.groupByKey`. **Example** In the following example, we group names by their first letter and count the number of names in each group. Here, the partitioning operation is set up as an effectful operation: ### grouped The `Stream.grouped` function is ideal for dividing a stream into chunks of a specified size, making it easier to handle data in smaller, organized segments. This is particularly helpful when processing or displaying data in batches. **Example** ### groupedWithin The `Stream.groupedWithin` function allows for flexible grouping by creating chunks based on either a specified maximum size or a time interval, whichever condition is met first. This is especially useful for working with data where timing constraints are involved. **Example** In this example, `Stream.groupedWithin` groups the stream into chunks whenever either 18 elements accumulate or 1.5 seconds elapse since the last chunk was created. ## Concatenation In stream processing, you may need to combine the contents of multiple streams. The Stream module offers several operators to achieve this, including `Stream.concat`, `Stream.concatAll`, and `Stream.flatMap`. Let's look at how each of these operators works. ### Simple Concatenation The `Stream.concat` operator is a straightforward method for joining two streams. It returns a new stream that emits elements from the first stream followed by elements from the second stream . This is helpful when you want to combine two streams in a specific sequence. **Example** ### Concatenating Multiple Streams If you have multiple streams to concatenate, `Stream.concatAll` provides an efficient way to combine them without manually chaining multiple `Stream.concat` operations. This function takes a of streams and returns a single stream containing the elements of each stream in sequence. **Example** ### Advanced Concatenation with flatMap The `Stream.flatMap` operator allows for advanced concatenation by creating a stream where each element is generated by applying a function of type ` => Stream<...>` to each output of the source stream. This operator then concatenates all the resulting streams, effectively flattening them. **Example** If you need to perform the `flatMap` operation concurrently, you can use the option to control how many inner streams run simultaneously. Additionally, you can use the `switch` option to implement a "switch" behavior where previous streams are automatically cancelled when new elements arrive from the source stream. This is particularly useful when you only need the most recent result and want to conserve resources by cancelling outdated operations. **Example** The `switch` option is especially valuable for scenarios like search functionality, real-time data processing, or any situation where you want to discard previous operations when new input arrives. ## Merging Sometimes, you may want to interleave elements from two streams and create a single output stream. In such cases, `Stream.concat` isn't suitable because it waits for the first stream to complete before consuming the second. For interleaving elements as they become available, `Stream.merge` and its variants are designed for this purpose. ### merge The `Stream.merge` operation combines elements from two source streams into a single stream, interleaving elements as they are produced. Unlike `Stream.concat`, `Stream.merge` does not wait for one stream to finish before starting the other. **Example** ### Termination Strategy When merging two streams, it's important to consider the termination strategy, especially if each stream has a different lifetime. By default, `Stream.merge` waits for both streams to terminate before ending the merged stream. However, you can modify this behavior with `haltStrategy`, selecting from four termination strategies: | Termination Strategy | Description | | -------------------- | -------------------------------------------------------------------- | | `"left"` | The merged stream terminates when the left-hand stream terminates. | | `"right"` | The merged stream terminates when the right-hand stream terminates. | | `"both"` | The merged stream terminates only when both streams have terminated. | | `"either"` | The merged stream terminates as soon as either stream terminates. | **Example** ### mergeWith In some cases, you may want to merge two streams while transforming their elements into a unified type. `Stream.mergeWith` is designed for this purpose, allowing you to specify transformation functions for each source stream. **Example** ## Interleaving ### interleave The `Stream.interleave` operator lets you pull one element at a time from each of two streams, creating a new interleaved stream. If one stream finishes first, the remaining elements from the other stream continue to be pulled until both streams are exhausted. **Example** ### interleaveWith For more complex interleaving, `Stream.interleaveWith` provides additional control by using a third stream of `boolean` values to dictate the interleaving pattern. When this stream emits `true`, an element is taken from the left-hand stream; otherwise, an element is taken from the right-hand stream. **Example** ## Interspersing Interspersing adds separators or affixes in a stream, useful for formatting or structuring data in streams. ### intersperse The `Stream.intersperse` operator inserts a specified delimiter element between each pair of elements in a stream. This delimiter can be any chosen value and is added between each consecutive pair. **Example** ### intersperseAffixes For more complex needs, `Stream.intersperseAffixes` provides control over different affixes at the start, between elements, and at the end of the stream. **Example** ## Broadcasting Broadcasting a stream creates multiple downstream streams that each receive the same elements from the source stream. This is useful when you want to send each element to multiple consumers simultaneously. The upstream stream has a `maximumLag` parameter that sets the limit for how much it can get ahead before slowing down to match the speed of the slowest downstream stream. **Example** In the following example, we broadcast a stream of numbers to two downstream consumers. The first calculates the maximum value in the stream, while the second logs each number with a delay. The upstream stream's speed adjusts based on the slower logging stream: ## Buffering Effect streams use a pull-based model, allowing downstream consumers to control the rate at which they request elements. However, when there's a mismatch in the speed between the producer and the consumer, buffering can help balance their interaction. The `Stream.buffer` operator is designed to manage this, allowing the producer to keep working even if the consumer is slower. You can set a maximum buffer capacity using the `capacity` option. ### buffer The `Stream.buffer` operator queues elements to allow the producer to work independently from the consumer, up to a specified capacity. This helps when a faster producer and a slower consumer need to operate smoothly without blocking each other. **Example** Different buffering options let you tailor the buffering strategy based on your use case: | **Buffering Type** | **Configuration** | **Description** | | ------------------- | -------------------------------------------- | ------------------------------------------------------------- | | **Bounded Queue** | `{ capacity: number }` | Limits the queue to a fixed size. | | **Unbounded Queue** | `{ capacity: "unbounded" }` | Allows an unlimited number of buffered items. | | **Sliding Queue** | `{ capacity: number, strategy: "sliding" }` | Keeps the most recent items, discarding older ones when full. | | **Dropping Queue** | `{ capacity: number, strategy: "dropping" }` | Keeps the earliest items, discarding new ones when full. | ## Debouncing Debouncing is a technique used to prevent a function from firing too frequently, which is particularly useful when a stream emits values rapidly but only the last value after a pause is needed. The `Stream.debounce` function achieves this by delaying the emission of values until a specified time period has passed without any new values. If a new value arrives during the waiting period, the timer resets, and only the latest value will eventually be emitted after a pause. **Example** ## Throttling Throttling is a technique for regulating the rate at which elements are emitted from a stream. It helps maintain a steady data output pace, which is valuable in situations where data processing needs to occur at a consistent rate. The `Stream.throttle` function uses the to control the rate of stream emissions. **Example** In this configuration: - Each chunk processed uses one token => 1`). - Tokens are replenished at a rate of one token every 100 milliseconds . ### Shape Strategy The "shape" strategy moderates data flow by delaying chunk emissions until they comply with specified bandwidth constraints. This strategy ensures that data throughput does not exceed defined limits, allowing for steady and controlled data emission. **Example** ### Enforce Strategy The "enforce" strategy strictly regulates data flow by discarding chunks that exceed bandwidth constraints. **Example** ### burst option The `Stream.throttle` function offers a burst option that allows for temporary increases in data throughput beyond the set rate limits. This option is set to greater than 0 to activate burst capability . The burst capacity provides additional tokens in the token bucket, enabling the stream to momentarily exceed its configured rate when bursts of data occur. **Example** In this setup, the stream starts with a bucket containing 5 tokens, allowing the first five chunks to be emitted instantly. The additional burst capacity of 2 accommodates further emissions momentarily, allowing for handling of subsequent data more flexibly. Over time, as the bucket refills according to the throttle configuration, additional elements are emitted, demonstrating how the burst capability can manage uneven data flows effectively. ## Scheduling When working with streams, you may need to introduce specific time intervals between each element's emission. The `Stream.schedule` combinator allows you to set these intervals. **Example** In this example, we've used the `Schedule.spaced` schedule to introduce a one-second gap between each emission in the stream. --- title: Resourceful Streams description: Learn how to manage resources in streams with safe acquisition and release, finalization for cleanup tasks, and ensuring post-finalization actions for robust resource handling in streaming applications. sidebar: order: 5 --- In the Stream module, you'll find that most of the constructors offer a special variant designed for lifting a scoped resource into a `Stream`. When you use these specific constructors, you're essentially creating streams that are inherently safe with regards to resource management. These constructors, before creating the stream, handle the resource acquisition, and after the stream's usage, they ensure its proper closure. Stream also provides us with `Stream.acquireRelease` and `Stream.finalizer` constructors that share similarities with `Effect.acquireRelease` and `Effect.addFinalizer`. These tools empower us to perform cleanup or finalization tasks before the stream concludes its operation. ## Acquire Release In this section, we'll explore an example that demonstrates the use of `Stream.acquireRelease` when working with file operations. In this code snippet, we're simulating file operations using the `open` function. The `Stream.acquireRelease` function is employed to ensure that the file is correctly opened and closed, and we then process the lines of the file using the acquired resource. ## Finalization In this section, we'll explore the concept of finalization in streams. Finalization allows us to execute a specific action before a stream ends. This can be particularly useful when we want to perform cleanup tasks or add final touches to a stream. Imagine a scenario where our streaming application needs to clean up a temporary directory when it completes its execution. We can achieve this using the `Stream.finalizer` function: In this code example, we start with our application logic represented by the `application` stream. We then use `Stream.finalizer` to define a finalization step, which deletes a temporary directory and logs a message. This ensures that the temporary directory is cleaned up properly when the application completes its execution. ## Ensuring In this section, we'll explore a scenario where we need to perform actions after the finalization of a stream. To achieve this, we can utilize the `Stream.ensuring` operator. Consider a situation where our application has completed its primary logic and finalized some resources, but we also need to perform additional actions afterward. We can use `Stream.ensuring` for this purpose: In this code example, we start with our application logic represented by the `Application Logic.` message. We then use `Stream.finalizer` to specify the finalization step, which logs `Finalizing the stream`. After that, we use `Stream.ensuring` to indicate that we want to perform additional tasks after the stream's finalization, resulting in the message `Performing additional tasks after stream's finalization`. This ensures that our post-finalization actions are executed as expected. --- title: TestClock description: Control time during testing with Effect's TestClock, simulating time passage, delays, and recurring effects without waiting for real time. sidebar: order: 0 --- import { Aside } from "@astrojs/starlight/components" In most cases, we want our unit tests to run as quickly as possible. Waiting for real time to pass can slow down our tests significantly. Effect provides a handy tool called `TestClock` that allows us to **control time during testing**. This means we can efficiently and predictably test code that involves time without having to wait for the actual time to pass. ## How TestClock Works Imagine `TestClock` as a wall clock that only moves forward when we adjust it manually using the `TestClock.adjust` and `TestClock.setTime` functions. The clock time does not progress on its own. When we adjust the clock time, any effects scheduled to run at or before that time will execute. This allows us to simulate time passage in tests without waiting for real time. **Example** A key point is forking the fiber where `Effect.sleep` is invoked. Calls to `Effect.sleep` and related methods wait until the clock time matches or exceeds the scheduled time for their execution. By forking the fiber, we retain control over the clock time adjustments. ## Testing Recurring Effects Here's an example demonstrating how to test an effect that runs at fixed intervals using the `TestClock`: **Example** In this example, we test an effect that runs at regular intervals. An unbounded queue is used to manage the effects, and we verify the following: 1. No effect occurs before the specified recurrence period. 2. An effect occurs after the recurrence period. 3. The effect executes exactly once. It's important to note that after each recurrence, the next occurrence is scheduled to happen at the appropriate time. Adjusting the clock by 60 minutes places exactly one value in the queue; adjusting by another 60 minutes adds another value. ## Testing Clock This example demonstrates how to test the behavior of the `Clock` using the `TestClock`: **Example** ## Testing Deferred The `TestClock` also impacts asynchronous code scheduled to run after a specific time. **Example** --- title: Equal description: Implement value-based equality checks for improved data integrity and predictable behavior in TypeScript. sidebar: order: 0 --- The Equal module provides a simple and convenient way to define and check for equality between two values in TypeScript. Here are some key reasons why Effect exports an Equal module: 1. **Value-Based Equality**: JavaScript's native equality operators check for equality by reference, meaning they compare objects based on their memory addresses rather than their content. This behavior can be problematic when you want to compare objects with the same values but different references. The Equal module offers a solution by allowing developers to define custom equality checks based on the values of objects. 2. **Custom Equality**: The Equal module enables developers to implement custom equality checks for their data types and classes. This is crucial when you have specific requirements for determining when two objects should be considered equal. By implementing the `Equal` interface, developers can define their own equality logic. 3. **Data Integrity**: In some applications, maintaining data integrity is crucial. The ability to perform value-based equality checks ensures that identical data is not duplicated within collections like sets or maps. This can lead to more efficient memory usage and more predictable behavior. 4. **Predictable Behavior**: The Equal module promotes more predictable behavior when comparing objects. By explicitly defining equality criteria, developers can avoid unexpected results that may occur with JavaScript's default reference-based equality checks. ## How to Perform Equality Checking in Effect In Effect it's advisable to **stop using** JavaScript's `===` and `==` operators and instead rely on the `Equal.equals` function. This function can work with any data type that implements the `Equal` interface. Some examples of such data types include , , , and . When you use `Equal.equals` and your objects do not implement the `Equal` interface, it defaults to using the `===` operator for object comparison: **Example** In this example, `a` and `b` are two separate objects with the same contents. However, `===` considers them different because they occupy different memory locations. This behavior can lead to unexpected results when you want to compare values based on their content. However, you can configure your models to ensure that `Equal.equals` behaves consistently with your custom equality checks. There are two alternative approaches: 1. **Implementing the `Equal` Interface**: This method is useful when you need to define your custom equality check. 2. **Using the Data Module**: For simple value equality, the module provides a more straightforward solution by automatically generating default implementations for `Equal`. Let's explore both. ### Implementing the Equal Interface To create custom equality behavior, you can implement the `Equal` interface in your models. This interface extends the `Hash` interface from the module. **Example** In the above code, we define a custom equality function `` and a hash function `` for the `Person` class. The `Hash` interface optimizes equality checks by comparing hash values instead of the objects themselves. When you use the `Equal.equals` function to compare two objects, it first checks if their hash values are equal. If not, it quickly determines that the objects are not equal, avoiding the need for a detailed property-by-property comparison. Once you've implemented the `Equal` interface, you can utilize the `Equal.equals` function to check for equality using your custom logic. **Example** In this code, the equality check returns `true` when comparing `alice` to a new `Person` object with identical property values and `false` when comparing `alice` to `bob` due to their differing property values. ### Simplifying Equality with the Data Module Implementing both `Equal` and `Hash` can become cumbersome when all you need is straightforward value equality checks. Luckily, the module provides a simpler solution. It offers APIs that automatically generate default implementations for both `Equal` and `Hash`. **Example** In this example, we use the function to create structured data objects and check their equality using `Equal.equals`. The Data module simplifies the process by providing a default implementation for both `Equal` and `Hash`, allowing you to focus on comparing values without the need for explicit implementations. The Data module isn't limited to just structs. It can handle various data types, including tuples, arrays, and records. If you're curious about how to leverage its full range of features, you can explore the . ## Working with Collections JavaScript's built-in `Set` and `Map` can be a bit tricky when it comes to checking equality: **Example** Even though the two elements in the set have the same values, the set contains two elements. Why? JavaScript's `Set` checks for equality by reference, not by values. To perform value-based equality checks, you'll need to use the `Hash*` collection types available in the `effect` package. These collection types, such as and , provide support for the `Equal` interface. ### HashSet When you use the `HashSet`, it correctly handles value-based equality checks. In the following example, even though you're adding two objects with the same values, the `HashSet` treats them as a single element. **Example** **Note**: It's crucial to use elements that implement the `Equal` interface, either by implementing custom equality checks or by using the Data module. This ensures proper functionality when working with `HashSet`. Without this, you'll encounter the same behavior as the native `Set` data type: **Example** In this case, without using the Data module alongside `HashSet`, you'll experience the same behavior as the native `Set` data type. The set contains two elements because it checks for equality by reference, not by values. ### HashMap When working with the `HashMap`, you have the advantage of comparing keys by their values instead of their references. This is particularly helpful in scenarios where you want to associate values with keys based on their content. **Example** In this code snippet, `HashMap` is used to create a map where the keys are objects constructed with `Data.struct`. These objects contain identical values, which would usually create separate entries in a regular JavaScript `Map` because the default comparison is reference-based. `HashMap`, however, uses value-based comparison, meaning the two objects with identical content are treated as the same key. Thus, when we add both objects, the second key-value pair overrides the first, resulting in a single entry in the map. --- title: Hash description: Optimize equality checks with efficient hashing for faster comparisons in collections like hash sets and maps. sidebar: order: 1 --- The `Hash` interface is closely tied to the interface and serves a supportive role in optimizing equality checks by providing a mechanism for hashing. Hashing is an important step in the efficient determination of equality between two values, particularly when used with data structures like hash tables. ## Role of Hash in Equality Checking The primary purpose of the `Hash` interface is to provide a quick and efficient way to determine if two values are definitely not equal, thereby complementing the interface. When two values implement the interface, their hash values are compared first: - **Different Hash Values**: If the hash values are different, it is guaranteed that the values themselves are different. This quick check allows the system to avoid a potentially expensive equality check. - **Same Hash Values**: If the hash values are the same, it does not guarantee that the values are equal, only that they might be. In this case, a more thorough comparison using the interface is performed to determine actual equality. This method dramatically speeds up the equality checking process, especially in collections where quick look-up and insertion times are crucial, such as in hash sets or hash maps. ## Implementing the Hash Interface Consider a scenario where you have a custom `Person` class, and you want to check if two instances are equal based on their properties. By implementing both the `Equal` and `Hash` interfaces, you can efficiently manage these checks: **Example** Explanation: - The `` method determines equality by comparing the `id`, `name`, and `age` fields of `Person` instances. This approach ensures that the equality check is comprehensive and considers all relevant attributes. - The `` method computes a hash code using the `id` of the person. This value is used to quickly differentiate between instances in hashing operations, optimizing the performance of data structures that utilize hashing. - The equality check returns `true` when comparing `alice` to a new `Person` object with identical property values and `false` when comparing `alice` to `bob` due to their differing property values. --- title: Batching description: Optimize performance by batching requests and reducing redundant API calls, enhancing efficiency in data fetching and processing. sidebar: order: 9 --- import { Aside } from "@astrojs/starlight/components" In typical application development, when interacting with external APIs, databases, or other data sources, we often define functions that perform requests and handle their results or failures accordingly. ### Simple Model Setup Here's a basic model that outlines the structure of our data and possible errors: ### Defining API Functions Let's define functions that interact with an external API, handling common operations such as fetching todos, retrieving user details, and sending emails. While this approach is straightforward and readable, it may not be the most efficient. Repeated API calls, especially when many todos share the same owner, can significantly increase network overhead and slow down your application. ### Using the API Functions While these functions are clear and easy to understand, their use may not be the most efficient. For example, notifying todo owners involves repeated API calls which can be optimized. This implementation performs an API call for each todo to fetch the owner's details and send an email. If multiple todos have the same owner, this results in redundant API calls. ## Batching Let's assume that `getUserById` and `sendEmail` can be batched. This means that we can send multiple requests in a single HTTP call, reducing the number of API requests and improving performance. **Step-by-Step Guide to Batching** 1. **Declaring Requests:** We'll start by transforming our requests into structured data models. This involves detailing input parameters, expected outputs, and possible errors. Structuring requests this way not only helps in efficiently managing data but also in comparing different requests to understand if they refer to the same input parameters. 2. **Declaring Resolvers:** Resolvers are designed to handle multiple requests simultaneously. By leveraging the ability to compare requests , resolvers can execute several requests in one go, maximizing the utility of batching. 3. **Defining Queries:** Finally, we'll define queries that utilize these batch-resolvers to perform operations. This step ties together the structured requests and their corresponding resolvers into functional components of the application. ### Declaring Requests We'll design a model using the concept of a `Request` that a data source might support: A `Request` is a construct representing a request for a value of type `Value`, which might fail with an error of type `Error`. Let's start by defining a structured model for the types of requests our data sources can handle. Each request is defined with a specific data structure that extends from a generic `Request` type, ensuring that each request carries its unique data requirements along with a specific error type. By using tagged constructors like `Request.tagged`, we can easily instantiate request objects that are recognizable and manageable throughout the application. ### Declaring Resolvers After defining our requests, the next step is configuring how Effect resolves these requests using `RequestResolver`: A `RequestResolver` is capable of executing requests of type `A`. The resolver passed to `Effect.request` must have no requirements of its own; see for how to resolve services before constructing one. In this section, we'll create individual resolvers for each type of request. The granularity of your resolvers can vary, but typically, they are divided based on the batching capabilities of the corresponding API calls. In this configuration: - **GetTodosResolver** handles the fetching of multiple `Todo` items. It's set up as a standard resolver since we assume it cannot be batched. - **GetUserByIdResolver** and **SendEmailResolver** are configured as batched resolvers. This setup is based on the assumption that these requests can be processed in batches, enhancing performance and reducing the number of API calls. ### Defining Queries Now that we've set up our resolvers, we're ready to tie all the pieces together to define our queries. This step will enable us to perform data operations effectively within our application. By using the `Effect.request` function, we integrate the resolvers with the request model effectively. This approach ensures that each query is optimally resolved using the appropriate resolver. Although the code structure looks similar to earlier examples, employing resolvers significantly enhances efficiency by optimizing how requests are handled and reducing unnecessary API calls. In the final setup, this program will execute only **3** queries to the APIs, regardless of the number of todos. This contrasts sharply with the traditional approach, which would potentially execute **1 + 2n** queries, where **n** is the number of todos. This represents a significant improvement in efficiency, especially for applications with a high volume of data interactions. ### Resolvers with Context In complex applications, resolvers often need access to shared services or configurations to handle requests effectively. However, maintaining the ability to batch requests while providing the necessary context can be challenging. Here, we'll explore how to manage context in resolvers to ensure that batching capabilities are not compromised. When creating request resolvers, it's crucial to manage the context carefully. Providing too much context or providing varying services to resolvers can make them incompatible for batching. To prevent such issues, the context for the resolver used in `Effect.request` is explicitly set to `never`. This forces developers to clearly define how the context is accessed and used within resolvers. Consider the following example where we set up an HTTP service that the resolvers can use to execute API calls: We can see now that the type of `GetTodosResolver` is no longer a `RequestResolver` but instead it is: which is an effect that access the `HttpService` and returns a composed resolver that has the minimal context ready to use. Once we have such effect we can directly use it in our query definition: We can see that the Effect correctly requires `HttpService` to be provided. Alternatively you can create `RequestResolver`s as part of layers direcly accessing or closing over context from construction. **Example** This way is probably the best for most of the cases given that layers are the natural primitive where to wire services together. ## Caching While we have significantly optimized request batching, there's another area that can enhance our application's efficiency: caching. Without caching, even with optimized batch processing, the same requests could be executed multiple times, leading to unnecessary data fetching. Caching is configured on the resolver. `RequestResolver.withCache` wraps a resolver in a bounded, in-memory cache keyed by request equality. Any query built on top of that resolver with `Effect.request` uses the same caching behavior automatically. Here's how you can implement caching for the `getUserById` query: ## Final Program Assuming you've wired everything up correctly: With this program, the `getTodos` operation retrieves the todos for each user. Then, the `Effect.forEach` function is used to notify the owner of each todo concurrently, without waiting for the notifications to complete. The `repeat` function is applied to the entire chain of operations, and it ensures that the program repeats every 10 seconds using a fixed schedule. This means that the entire process, including fetching todos and sending notifications, will be executed repeatedly with a 10-second interval. Since `getUserById` is built on top of `cachedGetUserByIdResolver`, the program reuses cached results for a `GetUserById` request as long as its resolver's cache hasn't evicted it , reducing unnecessary requests to fetch user data. Furthermore, the program is designed to send emails in batches, allowing for efficient processing and better utilization of resources. ## Customizing Request Caching `RequestResolver.withCache` takes a `strategy` option controlling which entry is evicted once the cache reaches `capacity`: Cache entries created this way don't expire by time. If you need time-to-live-based expiry, or want cache lookups exposed as a first-class `Cache` ) instead of a plain `RequestResolver`, use instead. It accepts the same `capacity` option plus an optional `timeToLive`. --- title: Equivalence description: Define and customize equivalence relations for TypeScript values. sidebar: order: 0 --- The Equivalence module provides a way to define equivalence relations between values in TypeScript. An equivalence relation is a binary relation that is reflexive, symmetric, and transitive, establishing a formal notion of when two values should be considered equivalent. ## What is Equivalence? An `Equivalence` represents a function that compares two values of type `A` and determines if they are equivalent. This is more flexible and customizable than simple equality checks using `===`. Here's the structure of an `Equivalence`: ## Using Built-in Equivalences The module provides several built-in equivalence relations for common data types: | Equivalence | Description | | ----------------------- | ------------------------------------------- | | `String` | Uses strict equality for strings | | `Number` | Uses strict equality for numbers | | `Boolean` | Uses strict equality for booleans | | `strictEqual` | Uses strict equality for symbols | | `BigInt` | Uses strict equality for bigints | | `Date` | Compares `Date` objects by their timestamps | **Example** ## Deriving Equivalences For more complex data structures, you may need custom equivalences. The Equivalence module lets you derive new `Equivalence` instances from existing ones with the `Equivalence.mapInput` function. **Example** The `Equivalence.mapInput` function takes two arguments: 1. The existing `Equivalence` you want to use as a base . 2. A function that extracts the value used for the equivalence check from your data structure => user.id` in this case). --- title: Order description: Compare, sort, and manage value ordering with customizable tools for TypeScript. sidebar: order: 2 --- The Order module provides a way to compare values and determine their order. It defines an interface `Order` which represents a single function for comparing two values of type `A`. The function returns `-1`, `0`, or `1`, indicating whether the first value is less than, equal to, or greater than the second value. Here's the basic structure of an `Order`: ## Using the Built-in Orders The Order module comes with several built-in comparators for common data types: | Order | Description | | -------- | ---------------------------------- | | `String` | Used for comparing strings. | | `Number` | Used for comparing numbers. | | `BigInt` | Used for comparing big integers. | | `Date` | Used for comparing `Date` objects. | **Example** ## Sorting Arrays You can sort arrays using these comparators. The `Array` module offers a `sort` function that sorts arrays without altering the original one. **Example** You can also use an `Order` as a comparator with JavaScript's native `Array.sort` method, but keep in mind that this will modify the original array. **Example** ## Deriving Orders For more complex data structures, you may need custom sorting rules. The Order module lets you derive new `Order` instances from existing ones with the `Order.mapInput` function. **Example** Imagine you have a list of `Person` objects, and you want to sort them by their names in ascending order. To achieve this, you can create a custom `Order`. The `Order.mapInput` function takes two arguments: 1. The existing `Order` you want to use as a base . 2. A function that extracts the value you want to use for sorting from your data structure => person.name` in this case). Once you have defined your custom `Order`, you can apply it to sort an array of `Person` objects: **Example** ## Combining Orders The Order module lets you combine multiple `Order` instances to create complex sorting rules. This is useful when sorting by multiple properties. **Example** Imagine you have a list of people, each represented by an object with a `name` and an `age`. You want to sort this list first by name and then, for individuals with the same name, by age. ## Additional Useful Functions The Order module provides additional functions for common comparison operations, making it easier to work with ordered values. ### Reversing Order `Order.flip` inverts the order of comparison. If you have an `Order` for ascending values, reversing it makes it descending. **Example** ### Comparing Values These functions allow you to perform simple comparisons between values: | API | Description | | ------------------------ | -------------------------------------------------------- | | `isLessThan` | Checks if one value is strictly less than another. | | `isGreaterThan` | Checks if one value is strictly greater than another. | | `isLessThanOrEqualTo` | Checks if one value is less than or equal to another. | | `isGreaterThanOrEqualTo` | Checks if one value is greater than or equal to another. | **Example** ### Finding Minimum and Maximum The `Order.min` and `Order.max` functions return the minimum or maximum value between two values, considering the order. **Example** ### Clamping Values `Order.clamp` restricts a value within a given range. If the value is outside the range, it is adjusted to the nearest bound. **Example** ### Checking Value Range `Order.isBetween` checks if a value falls within a specified inclusive range. **Example** --- title: Cache description: Optimize performance with cache for concurrent, compositional, and efficient value retrieval. sidebar: order: 1 --- In many applications, handling overlapping work is common. For example, in services that process incoming requests, it's important to avoid redundant work like handling the same request multiple times. The Cache module helps improve performance by preventing duplicate work. Key Features of Cache: | Feature | Description | | --------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | **Compositionality** | Allows overlapping work across different parts of the application while preserving compositional programming. | | **Unified Sync and Async Caches** | Integrates both synchronous and asynchronous caches through a unified lookup function that computes values either way. | | **Effect Integration** | Works natively with the Effect library, supporting concurrent lookups, failure handling, and interruption. | | **Cache Inspection** | Provides functions like `Cache.size`, `Cache.keys`, and `Cache.entries` to inspect the current state of the cache. | ## Creating a Cache A cache is defined by a lookup function that computes the value for a given key if it's not already cached: The lookup function takes a `Key` and returns an `Effect`, which describes how to compute the value . This `Effect` may require an environment , can fail with an `Error`, and succeed with a `Value`. Since it returns an `Effect`, it can handle both synchronous and asynchronous workflows. You create a cache by providing a lookup function along with a maximum size and a time-to-live for cached values. Once a cache is created, the most idiomatic way to work with it is `Cache.get`. `Cache.get` returns the current value in the cache if it exists, or computes a new value, puts it in the cache, and returns it. If multiple concurrent processes request the same value, it will only be computed once. All other processes will receive the computed value as soon as it is available. This is managed using Effect's fiber-based concurrency model without blocking the underlying thread. **Example** In this example, we call `timeConsumingEffect` three times concurrently with the same key. The cache runs this effect only once, so concurrent lookups will wait until the value is available: ## Concurrent Access The cache is designed to be safe for concurrent access and efficient under concurrent conditions. If two concurrent processes request the same value and it is not in the cache, the value will be computed once and provided to both processes as soon as it is available. Concurrent processes will wait for the value without blocking the underlying thread. If the lookup function fails or is interrupted, the error will be propagated to all concurrent processes waiting for the value. Failures are cached to prevent repeated computation of the same failed value. If interrupted, the key will be removed from the cache, so subsequent calls will attempt to compute the value again. ## Capacity A cache is created with a specified capacity. When the cache reaches capacity, the least recently accessed values will be removed first. The cache size may slightly exceed the specified capacity between operations. ## Time To Live A cache can also have a specified time to live . Values older than the TTL will not be returned. The age is calculated from when the value was loaded into the cache. ## Functions In addition to `Cache.get`, the `Cache` module provides several other functions for working with a cache. Each takes the cache as its first argument, for example `Cache.refresh`: | Function | Description | | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Cache.refresh` | Triggers a recomputation of the value for a key without removing the old value, allowing continued access. | | `Cache.size` | Returns the current size of the cache. The size is approximate under concurrent conditions. | | `Cache.has` | Checks if a value associated with a specified key exists in the cache. Under concurrent access, the result is valid as of the check time but may change immediately after. | | `Cache.invalidate` | Evicts the value associated with a specific key. | | `Cache.invalidateAll` | Evicts all values from the cache. | --- title: Caching Effects description: Efficiently manage caching and memoization of effects with reusable tools. sidebar: order: 0 --- This section covers several functions from the library that help manage caching and memoization in your application. ## Memoizing a Function To memoize an effectful function, create a `Cache` whose `lookup` is the function to memoize, then call `Cache.get` for each input. The cache stores one result per input, so calling the function again with the same input reuses the cached result instead of recomputing it. **Example** ## once Ensures an effect is executed only once, even if invoked multiple times. **Example** ## cached Returns an effect that computes a result lazily and caches it. Subsequent evaluations of this effect will return the cached result without re-executing the logic. **Example** ## cachedWithTTL Returns an effect that caches its result for a specified duration, known as the `timeToLive`. When the cache expires after the duration, the effect will be recomputed upon next evaluation. **Example** ## cachedInvalidateWithTTL Similar to `Effect.cachedWithTTL`, this function caches an effect's result for a specified duration. It also includes an additional effect for manually invalidating the cached value before it naturally expires. **Example** --- title: Branded Types description: Use branded types to enforce type safety and refine data in TypeScript. sidebar: order: 3 --- In this guide, we will explore the concept of **branded types** in TypeScript and learn how to create and work with them using the Brand module. Branded types are TypeScript types with an added type tag that helps prevent accidental usage of a value in the wrong context. They allow us to create distinct types based on an existing underlying type, enabling type safety and better code organization. ## The Problem with TypeScript's Structural Typing TypeScript's type system is structurally typed, meaning that two types are considered compatible if their members are compatible. This can lead to situations where values of the same underlying type are used interchangeably, even when they represent different concepts or have different meanings. Consider the following types: Here, `UserId` and `ProductId` are structurally identical as they are both based on `number`. TypeScript will treat these as interchangeable, potentially causing bugs if they are mixed up in your application. **Example** In the example above, passing a `UserId` to `getProductById` does not produce a type error, even though it's logically incorrect. This happens because both types are considered interchangeable. ## How Branded Types Help Branded types allow you to create distinct types from the same underlying type by adding a unique type tag, enforcing proper usage at compile-time. Branding is accomplished by adding a symbolic identifier that distinguishes one type from another at the type level. This method ensures that types remain distinct without altering their runtime characteristics. Let's start by introducing the `BrandTypeId` symbol: This approach assigns a unique identifier as a brand to the `number` type, effectively differentiating `ProductId` from other numerical types. The use of a symbol ensures that the branding field does not conflict with any existing properties of the `number` type. Attempting to use a `UserId` in place of a `ProductId` now results in an error: **Example** The error message clearly states that a `number` cannot be used in place of a `ProductId`. TypeScript won't let us pass an instance of `number` to the function accepting `ProductId` because it's missing the brand field. Let's add branding to `UserId` as well: **Example** The error indicates that while both types use branding, the unique values associated with the branding fields ensure they remain distinct and non-interchangeable. ## Generalizing Branded Types To enhance the versatility and reusability of branded types, they can be generalized using a standardized approach: This design allows any type to be branded using a unique identifier, either a string or symbol. Here's how you can utilize the `Brand` interface, which is readily available from the Brand module, eliminating the need to craft your own implementation: **Example** However, creating instances of these types directly leads to an error because the type system expects the brand structure: **Example** You cannot directly assign a `number` to `ProductId`. The Brand module provides utilities to correctly construct values of branded types. ## Constructing Branded Types The Brand module provides two main functions for creating branded types: `nominal` and `make`. ### nominal The `Brand.nominal` function is designed for defining branded types that do not require runtime validations. It simply adds a type tag to the underlying type, allowing us to distinguish between values of the same type but with different meanings. Nominal branded types are useful when we only want to create distinct types for clarity and code organization purposes. **Example** Attempting to assign a non-`ProductId` value will result in a compile-time error: **Example** ### refined The `Brand.make` function creates a branded type constructor that validates its input. The validation function returns `true` for a valid value or describes why the value is invalid. Calling the constructor with invalid input throws a `BrandError`. Its `option`, `result`, and `is` methods provide non-throwing alternatives. **Example** **Example** Attempting to assign a non-`Int` value will result in a compile-time error: **Example** ## Combining Branded Types In some cases, you might need to combine multiple branded types. The Brand module provides the `Brand.all` API for this purpose: **Example** --- title: Control Flow Operators description: Learn to control execution flow in Effect programs using advanced constructs for conditional branching, iteration, and combining effects seamlessly. sidebar: order: 6 --- Even though JavaScript provides built-in control flow structures, Effect offers additional control flow functions that are useful in Effect applications. In this section, we will introduce different ways to control the flow of execution. ## if Expression When working with Effect values, we can use standard JavaScript if-then-else statements: **Example** Here we are using the data type to represent the absence of a valid value. **Example** You can also handle invalid inputs by using the error channel, which allows you to return an error when the input is invalid: ## Conditional Operators ### when Executes an effect conditionally, based on the result of another effect. Use `Effect.when` when the condition to determine whether to execute the effect depends on the outcome of another effect that produces a boolean value. If the condition effect evaluates to `true`, the specified effect is executed. If it evaluates to `false`, no effect is executed. The result of the effect is wrapped in an `Option` to indicate whether the effect was executed. If the condition is `true`, the result of the effect is wrapped in a `Some`. If the condition is `false`, the result is `None`, representing that the effect was skipped. **Example** The following function creates a random integer, but only if a randomly generated boolean is `true`. ## Zipping ### zip Combines two effects into a single effect, producing a tuple with the results of both effects. The `Effect.zip` function executes the first effect and then the second effect . Once both effects succeed, their results are combined into a tuple. **Example** By default, the effects are run sequentially. To run them concurrently, use the `{ concurrent: true }` option. **Example** In this concurrent version, both effects run in parallel. `task2` completes first, but both tasks can be logged and processed as soon as they're done. ### zipWith Combines two effects sequentially and applies a function to their results to produce a single value. The `Effect.zipWith` function is similar to , but instead of returning a tuple of results, it applies a provided function to the results of the two effects, combining them into a single value. By default, the effects are run sequentially. To run them concurrently, use the `{ concurrent: true }` option. **Example** ## Looping ### whileLoop The `Effect.whileLoop` function allows you to repeatedly update a state using a `step` function until a condition defined by the `while` function becomes `false`. It collects the intermediate states in an array and returns them as the final result. **Syntax** This function is similar to a `while` loop in JavaScript, with the addition of effectful computations: **Example** In this example, the loop starts with the state `1` and continues until the state exceeds `5`. Each state is incremented by `1` and is collected into an array, which becomes the final result. #### Discarding Intermediate Results The `discard` option, when set to `true`, will discard the results of each effectful operation, returning `void` instead of an array. **Example** In this example, the loop performs a side effect of logging the current index on each iteration, but it discards all intermediate results. The final result is `undefined`. ### forEach Executes an effectful operation for each element in an `Iterable`. The `Effect.forEach` function applies a provided operation to each element in the iterable, producing a new effect that returns an array of results. If any effect fails, the iteration stops immediately , and the error is propagated. The `concurrency` option controls how many operations are performed concurrently. By default, the operations are performed sequentially. **Example** In this example, we iterate over the array ``, applying an effect that logs the current index. The `Effect.as` operation transforms each value, resulting in an array ``. The final output is the result of collecting all the transformed values. #### Discarding Results The `discard` option, when set to `true`, will discard the results of each effectful operation, returning `void` instead of an array. **Example** In this case, the effects still run for each element, but the results are discarded, so the final output is `undefined`. ## Collecting ### all Combines multiple effects into one, returning results based on the input structure. Use `Effect.all` when you need to run multiple effects and combine their results into a single output. It supports tuples, iterables, structs, and records, making it flexible for different input types. If any effect fails, it stops execution and propagates the error. To change this behavior, you can use the option, which allows all effects to run and collect results as . You can control the execution order using the . For instance, if the input is a tuple: the effects are executed sequentially, and the result is a new effect containing the results as a tuple. The results in the tuple match the order of the effects passed to `Effect.all`. Let's explore examples for different types of structures: tuples, iterables, objects, and records. **Example** **Example** **Example** **Example** #### Short-Circuiting Behavior The `Effect.all` function stops execution on the first error it encounters, this is called "short-circuiting". If any effect in the collection fails, the remaining effects will not run, and the error will be propagated. **Example** You can override this behavior by using the `mode` option. #### The `mode` option The `{ mode: "result" }` option changes the behavior of `Effect.all` to ensure all effects run, even if some fail. Instead of stopping on the first failure, this mode collects both successes and failures, returning an array of `Result` instances. **Example** --- title: Simplifying Excessive Nesting description: Simplify nested code with Do simulation and generators. sidebar: label: Excessive Nesting order: 5 --- import { Steps } from "@astrojs/starlight/components" Suppose you want to create a custom function `elapsed` that prints the elapsed time taken by an effect to execute. ## Using plain pipe Initially, you may come up with code that uses the standard `pipe` , but this approach can lead to excessive nesting and result in verbose and hard-to-read code: **Example** To address this issue and make the code more manageable, there is a solution: the "do simulation." ## Using the "do simulation" The "do simulation" in Effect allows you to write code in a more declarative style, similar to the "do notation" in other programming languages. It provides a way to define variables and perform operations on them using functions like `Effect.bind` and `Effect.let`. Here's how the do simulation works: 1. Start the do simulation using the `Effect.Do` value: 2. Within the do simulation scope, you can use the `Effect.bind` function to define variables and bind them to `Effect` values: - `variableName` is the name you choose for the variable you want to define. It must be unique within the scope. - `effectValue` is the `Effect` value that you want to bind to the variable. It can be the result of a function call or any other valid `Effect` value. 3. You can accumulate multiple `Effect.bind` statements to define multiple variables within the scope: 4. Inside the do simulation scope, you can also use the `Effect.let` function to define variables and bind them to simple values: - `variableName` is the name you give to the variable. Like before, it must be unique within the scope. - `simpleValue` is the value you want to assign to the variable. It can be a simple value like a `number`, `string`, or `boolean`. 5. Regular Effect functions like `Effect.andThen`, `Effect.flatMap`, `Effect.tap`, and `Effect.map` can still be used within the do simulation. These functions will receive the accumulated variables as arguments within the scope: With the do simulation, you can rewrite the `elapsed` function like this: **Example** ## Using Effect.gen The most concise and convenient solution is to use , which allows you to work with when dealing with effects. This approach leverages the native scope provided by the generator syntax, avoiding excessive nesting and leading to more concise code. **Example** Within the generator, we use `yield*` to invoke effects and bind their results to variables. This eliminates the nesting and provides a more readable and sequential code structure. The generator style in Effect uses a more linear and sequential flow of execution, resembling traditional imperative programming languages. This makes the code easier to read and understand, especially for developers who are more familiar with imperative programming paradigms. --- title: Dual APIs description: Explore data-first and data-last variants of dual APIs in the Effect ecosystem. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" When you're working with APIs in the Effect ecosystem, you may come across two different ways to use the same API. These two ways are called the "data-last" and "data-first" variants. When an API supports both variants, we call them "dual" APIs. Here's an illustration of these two variants using `Effect.map`. ## Effect.map as a dual API The `Effect.map` function is defined with two TypeScript overloads. The terms "data-last" and "data-first" refer to the position of the `self` argument in the signatures of the two overloads: ### data-last In the first overload, the `self` argument comes **last**: This version is commonly used with the `pipe` function. You start by passing the `Effect` as the initial argument to `pipe` and then chain transformations like `Effect.map`: **Example** This style is helpful when chaining multiple transformations, making the code easier to follow in a pipeline format: ### data-first In the second overload, the `self` argument comes **first**: This form doesn't require `pipe`. Instead, you provide the `Effect` directly as the first argument: **Example** This version works well when you only need to perform a single operation on the `Effect`. --- title: Guidelines description: Best practices for running Effect applications and ensuring safe, explicit coding styles. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" ## Using runMain In Effect, `runMain` is the primary entry point for executing an Effect application on Node.js. **Example** The `runMain` function handles finding and interrupting all fibers. Internally, it observes the fiber and listens for `sigint` signals, ensuring a graceful shutdown of the application when interrupted . ### Versions for Different Platforms Effect provides versions of `runMain` tailored for different platforms: | Platform | Runtime Version | Import Path | | -------- | ------------------------ | -------------------------- | | Node.js | `NodeRuntime.runMain` | `@effect/platform-node` | | Bun | `BunRuntime.runMain` | `@effect/platform-bun` | | Browser | `BrowserRuntime.runMain` | `@effect/platform-browser` | ## Avoid Tacit Usage Avoid using tacit function calls, such as `Effect.map`, or using `flow` from the `effect/Function` module. In Effect, it's generally safer to write functions explicitly: rather than in a point-free style: While tacit functions may be appealing for their brevity, they can introduce a number of problems: - Using tacit functions, particularly when dealing with optional parameters, can be unsafe. For example, if a function has overloads, writing it in a tacit style may erase all generics, resulting in bugs. Check out this X thread for more details: . - Tacit usage can also compromise TypeScript's ability to infer types, potentially causing unexpected errors. This isn't just a matter of style but a way to avoid subtle mistakes that can arise from type inference issues. - Additionally, stack traces might not be as clear when tacit usage is employed. Avoiding tacit usage is a simple precaution that makes your code more reliable. --- title: Pattern Matching description: Simplify complex branching with pattern matching using the Match module. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" Pattern matching is a method that allows developers to handle intricate conditions within a single, concise expression. It simplifies code, making it more concise and easier to understand. Additionally, it includes a process called exhaustiveness checking, which helps to ensure that no possible case has been overlooked. Originating from functional programming languages, pattern matching stands as a powerful technique for code branching. It often offers a more potent and less verbose solution compared to imperative alternatives such as if/else or switch statements, particularly when dealing with complex conditions. Although not yet a native feature in JavaScript, there's an ongoing in its early stages to introduce pattern matching to JavaScript. However, this proposal is at stage 1 and might take several years to be implemented. Nonetheless, developers can implement pattern matching in their codebase. The `effect/Match` module provides a reliable, type-safe pattern matching implementation that is available for immediate use. **Example** ## How Pattern Matching Works Pattern matching follows a structured process: 1. **Creating a matcher**. Define a `Matcher` that operates on either a specific or . 2. **Defining patterns**. Use combinators such as `Match.when`, `Match.not`, and `Match.tag` to specify matching conditions. 3. **Completing the match**. Apply a finalizer such as `Match.exhaustive`, `Match.orElse`, or `Match.option` to determine how unmatched cases should be handled. ## Creating a matcher You can create a `Matcher` using either: - `Match.type`: Matches against a specific type. - `Match.value`: Matches against a specific value. ### Matching by Type The `Match.type` constructor defines a `Matcher` that operates on a specific type. Once created, you can use patterns like `Match.when` to define conditions for handling different cases. **Example** ### Matching by Value Instead of creating a matcher for a type, you can define one directly from a specific value using `Match.value`. **Example** ### Enforcing a Return Type You can use `Match.withReturnType` to ensure that all branches return a specific type. **Example** This example enforces that every matching branch returns a `string`. ## Defining patterns ### when The `Match.when` function allows you to define conditions for matching values. It supports both direct value comparisons and predicate functions. **Example** ### not The `Match.not` function allows you to exclude specific values while matching all others. **Example** ### tag The `Match.tag` function allows pattern matching based on the `_tag` field in a . You can specify multiple tags to match within a single pattern. **Example** ### Built-in Predicates The `Match` module provides built-in predicates for common types, such as `Match.number`, `Match.string`, and `Match.boolean`. These predicates simplify the process of matching against primitive types. **Example** | Predicate | Description | | ------------------------- | ----------------------------------------------------------------------------- | | `Match.string` | Matches values of type `string`. | | `Match.nonEmptyString` | Matches non-empty strings. | | `Match.number` | Matches values of type `number`. | | `Match.boolean` | Matches values of type `boolean`. | | `Match.bigint` | Matches values of type `bigint`. | | `Match.symbol` | Matches values of type `symbol`. | | `Match.date` | Matches values that are instances of `Date`. | | `Match.record` | Matches objects where keys are `string` or `symbol` and values are `unknown`. | | `Match.null` | Matches the value `null`. | | `Match.undefined` | Matches the value `undefined`. | | `Match.defined` | Matches any defined value. | | `Match.any` | Matches any value without restrictions. | | `Match.is` | Matches a specific set of literal values `). | | `Match.instanceOf` | Matches instances of a given class. | ## Completing the match ### exhaustive The `Match.exhaustive` method finalizes the pattern matching process by ensuring that all possible cases are accounted for. If any case is missing, TypeScript will produce a type error. This is particularly useful when working with unions, as it helps prevent unintended gaps in pattern matching. **Example** ### orElse The `Match.orElse` method defines a fallback value to return when no other patterns match. This ensures that the matcher always produces a valid result. **Example** ### option `Match.option` wraps the match result in an . If a match is found, it returns `Some`, otherwise, it returns `None`. **Example** ### result The `Match.result` method wraps the result in a , providing a structured way to distinguish between matched and unmatched cases. If a match is found, it returns `Success`, otherwise, it returns `Failure`. **Example** --- title: Basic Concurrency description: Manage and control effect execution with concurrency, interruptions, and racing. sidebar: order: 0 --- import { Aside } from "@astrojs/starlight/components" ## Concurrency Options Effect provides options to manage how effects are executed, particularly focusing on controlling how many effects run concurrently. The `concurrency` option is used to determine the level of concurrency, with the following values: Let's explore each configuration in detail. ### Sequential Execution By default, if you don't specify any concurrency option, effects will run sequentially, one after the other. This means each effect starts only after the previous one completes. **Example** ### Numbered Concurrency You can control how many effects run concurrently by setting a `number` for `concurrency`. For example, `concurrency: 2` allows up to two effects to run at the same time. **Example** ### Unbounded Concurrency When `concurrency: "unbounded"` is used, there's no limit to the number of effects running concurrently. **Example** ## Interruptions All effects in Effect are executed by . If you didn't create the fiber yourself, it was created by an operation you're using or by the Effect system. A fiber is created any time an effect is run. When running effects concurrently, a fiber is created for each concurrent effect. To summarize: - An `Effect` is a higher-level concept that describes an effectful computation. It is lazy and immutable, meaning it represents a computation that may produce a value or fail but does not immediately execute. - A fiber, on the other hand, represents the running execution of an `Effect`. It can be interrupted or awaited to retrieve its result. Think of it as a way to control and interact with the ongoing computation. Fibers can be interrupted in various ways. Let's explore some of these scenarios and see examples of how to interrupt fibers in Effect. ### interrupt A fiber can be interrupted using the `Effect.interrupt` effect on that particular fiber. This effect models the explicit interruption of the fiber in which it runs. When executed, it causes the fiber to stop its operation immediately, capturing the interruption details such as the fiber's ID and its start time. The resulting interruption can be observed in the type if the effect is run with functions like . **Example** In this case, the program runs without any interruption, logging the start and completion of the task. **Example** Here, the fiber is interrupted after the log `"start"` but before the `"done"` log. The `Effect.interrupt` stops the fiber, and it never reaches the final log. ### onInterrupt Registers a cleanup effect to run when an effect is interrupted. This function allows you to specify an effect to run when the fiber is interrupted. This effect will be executed when the fiber is interrupted, allowing you to perform cleanup or other actions. **Example** In this example, we set up a handler that logs "Cleanup completed" whenever the fiber is interrupted. We then show three cases: a successful effect, a failing effect, and an interrupted effect, demonstrating how the handler is triggered depending on how the effect ends. ### Interruption of Concurrent Effects When running multiple effects concurrently, such as with `Effect.forEach`, if one of the effects is interrupted, it causes all concurrent effects to be interrupted as well. The resulting includes information about which fibers were interrupted. **Example** ## Racing ### race This function takes two effects and runs them concurrently. The first effect that successfully completes will determine the result of the race, and the other effect will be interrupted. If neither effect succeeds, the function will fail with a containing all the errors. This is useful when you want to run two effects concurrently, but only care about the first one to succeed. It is commonly used in cases like timeouts, retries, or when you want to optimize for the faster response without worrying about the other effect. **Example** **Example** **Example** If you want to handle the result of whichever task completes first, whether it succeeds or fails, you can use the `Effect.result` function. This function wraps the result in a type, allowing you to see if the result was a success or a failure : **Example** ### raceAll This function runs multiple effects concurrently and returns the result of the first one to succeed. If one effect succeeds, the others will be interrupted. If none of the effects succeed, the function will fail with the last error encountered. This is useful when you want to race multiple effects, but only care about the first one to succeed. It is commonly used in cases like timeouts, retries, or when you want to optimize for the faster response without worrying about the other effects. **Example** **Example** **Example** ### raceFirst This function takes two effects and runs them concurrently, returning the result of the first one that completes, regardless of whether it succeeds or fails. This function is useful when you want to race two operations, and you want to proceed with whichever one finishes first, regardless of whether it succeeds or fails. **Example** **Example** #### Observing the Winner The optional `onWinner` callback receives the winning fiber and its index . The callback is observational: the result of `raceFirst` is still determined by the first effect to complete. **Example** --- title: Deferred description: Master asynchronous coordination with Deferred, a one-time variable for managing effect synchronization and communication. sidebar: order: 4 --- A `Deferred` is a specialized subtype of `Effect` that acts like a one-time variable with some unique characteristics. It can only be completed once, making it a useful tool for managing asynchronous operations and synchronization between different parts of your program. A deferred is essentially a synchronization primitive that represents a value that may not be available right away. When you create a deferred, it starts out empty. Later, it can be completed with either a success value `Success` or an error value `Error`: Once completed, it cannot be changed again. When a fiber calls `Deferred.await`, it will pause until the deferred is completed. While the fiber is waiting, it doesn't block the thread, it only blocks semantically. This means other fibers can still run, ensuring efficient concurrency. A deferred is conceptually similar to JavaScript's `Promise`. The key difference is that it supports both success and error types, giving more type safety. ## Creating a Deferred A deferred can be created using the `Deferred.make` constructor. This returns an effect that represents the creation of the deferred. Since the creation of a deferred involves memory allocation, it must be done within an effect to ensure safe management of resources. **Example** ## Awaiting To retrieve a value from a deferred, you can use `Deferred.await`. This operation suspends the calling fiber until the deferred is completed with a value or an error. ## Completing You can complete a deferred in several ways, depending on whether you want to succeed, fail, or interrupt the waiting fibers: | API | Description | | ----------------------- | --------------------------------------------------------------------------------------------------------------- | | `Deferred.succeed` | Completes the deferred successfully with a value. | | `Deferred.done` | Completes the deferred with an value. | | `Deferred.complete` | Completes the deferred with the result of an effect. | | `Deferred.completeWith` | Completes the deferred with an effect. This effect will be executed by each waiting fiber, so use it carefully. | | `Deferred.fail` | Fails the deferred with an error. | | `Deferred.die` | Defects the deferred with a user-defined error. | | `Deferred.failCause` | Fails or defects the deferred with a . | | `Deferred.interrupt` | Interrupts the deferred, forcefully stopping or interrupting the waiting fibers. | **Example** Completing a deferred produces an `Effect`. This effect returns `true` if the deferred was successfully completed, and `false` if it had already been completed previously. This can be useful for tracking the state of the deferred. **Example** ## Checking Completion Status Sometimes, you might need to check if a deferred has been completed without suspending the fiber. This can be done using the `Deferred.poll` method. Here's how it works: - `Deferred.poll` returns an `Option>`: - If the `Deferred` is incomplete, it returns `None`. - If the `Deferred` is complete, it returns `Some`, which contains the result or error. Additionally, you can use the `Deferred.isDone` function to check if a deferred has been completed. This method returns an `Effect`, which evaluates to `true` if the `Deferred` is completed, allowing you to quickly check its state. **Example** ## Common Use Cases `Deferred` becomes useful when you need to wait for something specific to happen in your program. It's ideal for scenarios where you want one part of your code to signal another part when it's ready. Here are a few common use cases: | **Use Case** | **Description** | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Coordinating Fibers** | When you have multiple concurrent tasks and need to coordinate their actions, `Deferred` can help one fiber signal to another when it has completed its task. | | **Synchronization** | Anytime you want to ensure that one piece of code doesn't proceed until another piece of code has finished its work, `Deferred` can provide the synchronization you need. | | **Handing Over Work** | You can use `Deferred` to hand over work from one fiber to another. For example, one fiber can prepare some data, and then a second fiber can continue processing it. | | **Suspending Execution** | When you want a fiber to pause its execution until some condition is met, a `Deferred` can be used to block it until the condition is satisfied. | **Example** In this example, a deferred is used to pass a value between two fibers. By running both fibers concurrently and using the deferred as a synchronization point, we can ensure that `fiberB` only proceeds after `fiberA` has completed its task. --- title: Fibers description: Understand fibers in Effect, lightweight virtual threads enabling powerful concurrency, structured lifecycles, and efficient resource management for responsive applications. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" Effect is a highly concurrent framework powered by fibers. Fibers are lightweight virtual threads with resource-safe cancellation capabilities, enabling many features in Effect. In this section, you will learn the basics of fibers and get familiar with some of the powerful low-level operators that utilize fibers. ## What Are Virtual Threads? JavaScript is inherently single-threaded, meaning it executes code in a single sequence of instructions. However, modern JavaScript environments use an event loop to manage asynchronous operations, creating the illusion of multitasking. In this context, virtual threads, or fibers, are logical threads simulated by the Effect runtime. They allow concurrent execution without relying on true multi-threading, which is not natively supported in JavaScript. ## How Fibers work All effects in Effect are executed by fibers. If you didn't create the fiber yourself, it was created by an operation you're using or by the Effect runtime system. A fiber is created any time an effect is run. When running effects concurrently, a fiber is created for each concurrent effect. Even if you write "single-threaded" code with no concurrent operations, there will always be at least one fiber: the "main" fiber that executes your effect. Effect fibers have a well-defined lifecycle based on the effect they are executing. Every fiber exits with either a failure or success, depending on whether the effect it is executing fails or succeeds. Effect fibers have unique identities, local state, and a status . To summarize: - An `Effect` is a higher-level concept that describes an effectful computation. It is lazy and immutable, meaning it represents a computation that may produce a value or fail but does not immediately execute. - A fiber, on the other hand, represents the running execution of an `Effect`. It can be interrupted or awaited to retrieve its result. Think of it as a way to control and interact with the ongoing computation. ## The Fiber Data Type The `Fiber` data type in Effect represents a "handle" on the execution of an effect. Here is the general form of a `Fiber`: This type indicates that a fiber: - Succeeds and returns a value of type `Success` - Fails with an error of type `Error` Fibers do not have an `Requirements` type parameter because they only execute effects that have already had their requirements provided to them. ## Forking Effects You can create a new fiber by **forking** an effect. This starts the effect in a new fiber, and you receive a reference to that fiber. **Example** In this example, the Fibonacci calculation is forked into its own fiber, allowing it to run independently of the main fiber. The reference to the `fib10Fiber` can be used later to join or interrupt the fiber. ## Joining Fibers One common operation with fibers is **joining** them. By using the `Fiber.join` function, you can wait for a fiber to complete and retrieve its result. The joined fiber will either succeed or fail, and the `Effect` returned by `join` reflects the outcome of the fiber. **Example** ## Awaiting Fibers The `Fiber.await` function is a helpful tool when working with fibers. It allows you to wait for a fiber to complete and retrieve detailed information about how it finished. The result is encapsulated in an value, which gives you insight into whether the fiber succeeded, failed, or was interrupted. **Example** ## Interruption Model While developing concurrent applications, there are several cases that we need to interrupt the execution of other fibers, for example: 1. A parent fiber might start some child fibers to perform a task, and later the parent might decide that, it doesn't need the result of some or all of the child fibers. 2. Two or more fibers start race with each other. The fiber whose result is computed first wins, and all other fibers are no longer needed, and should be interrupted. 3. In interactive applications, a user may want to stop some already running tasks, such as clicking on the "stop" button to prevent downloading more files. 4. Computations that run longer than expected should be aborted by using timeout operations. 5. When we have an application that perform compute-intensive tasks based on the user inputs, if the user changes the input we should cancel the current task and perform another one. ### Polling vs. Asynchronous Interruption When it comes to interrupting fibers, a naive approach is to allow one fiber to forcefully terminate another fiber. However, this approach is not ideal because it can leave shared state in an inconsistent and unreliable state if the target fiber is in the middle of modifying that state. Therefore, it does not guarantee internal consistency of the shared mutable state. Instead, there are two popular and valid solutions to tackle this problem: 1. **Semi-asynchronous Interruption **: Imperative languages often employ polling as a semi-asynchronous signaling mechanism, such as Java. In this model, a fiber sends an interruption request to another fiber. The target fiber continuously polls the interrupt status and checks whether it has received any interruption requests from other fibers. If an interruption request is detected, the target fiber terminates itself as soon as possible. With this solution, the fiber itself handles critical sections. So, if a fiber is in the middle of a critical section and receives an interruption request, it ignores the interruption and defers its handling until after the critical section. However, one drawback of this approach is that if the programmer forgets to poll regularly, the target fiber can become unresponsive, leading to deadlocks. Additionally, polling a global flag is not aligned with the functional paradigm followed by Effect. 2. **Asynchronous Interruption**: In asynchronous interruption, a fiber is allowed to terminate another fiber. The target fiber is not responsible for polling the interrupt status. Instead, during critical sections, the target fiber disables the interruptibility of those regions. This is a purely functional solution that doesn't require polling a global state. Effect adopts this solution for its interruption model, which is a fully asynchronous signaling mechanism. This mechanism overcomes the drawback of forgetting to poll regularly. It is also fully compatible with the functional paradigm because in a purely functional computation, we can abort the computation at any point, except during critical sections where interruption is disabled. ### Interrupting Fibers Fibers can be interrupted if their result is no longer needed. This action immediately stops the fiber and safely runs all finalizers to release any resources. Unlike `Fiber.await`, which returns an value describing how a fiber completed, `Fiber.interrupt` returns `Effect`. It still waits until the fiber has fully terminated before resuming, but it does not hand back the fiber's `Exit`. **Example** By default, the effect returned by `Fiber.interrupt` waits until the fiber has fully terminated before resuming. This ensures that no new fibers are started before the previous ones have finished, a behavior known as "back-pressuring." If you do not require this waiting behavior, you can fork the interruption itself, allowing the main program to proceed without waiting for the fiber to terminate: **Example** For fire-and-forget interruption, a fiber also exposes the immediate `interruptUnsafe` method. Unlike `Fiber.interrupt`, it does not wait for the fiber's finalizers to complete. ## Composing Fiber Results Fiber handles are not combined directly. Join each fiber to obtain an `Effect`, then compose those effects with operators such as `Effect.zip`. **Example** In this example, both fibers run concurrently, and the results are combined into a tuple. The same principle applies to fallback behavior: join both fibers and recover from the failure of the first joined effect. **Example** ## Lifetime of Child Fibers When we fork fibers, depending on how we fork them we can have four different lifetime strategies for the child fibers: 1. **Fork With Automatic Supervision**. If we use the ordinary `Effect.forkChild` operation, the child fiber will be automatically supervised by the parent fiber. Child fiber lifetimes are tied to the lifetime of their parent fiber. This means that these fibers will be terminated either when they end naturally, or when their parent fiber is terminated. 2. **Fork in Global Scope **. Sometimes we want to run long-running background fibers that aren't tied to their parent fiber, and also we want to fork them in a global scope. Any fiber that is forked in global scope will become daemon fiber. This can be achieved by using the `Effect.forkDetach` operator. As these fibers have no parent, they are not supervised, and they will be terminated when they end naturally, or when our application is terminated. 3. **Fork in Local Scope**. Sometimes, we want to run a background fiber that isn't tied to its parent fiber, but we want that fiber to live in the local scope. We can fork fibers in the local scope by using `Effect.forkScoped`. Such fibers can outlive their parent fiber , and they will be terminated when they complete or their local scope is closed. 4. **Fork in Specific Scope**. This is similar to the previous strategy, but we can have more fine-grained control over the lifetime of the child fiber by forking it in a specific scope. We can do this by using the `Effect.forkIn` operator. ### Fork with Automatic Supervision Effect follows a **structured concurrency** model, where child fibers' lifetimes are tied to their parent. Simply put, the lifespan of a fiber depends on the lifespan of its parent fiber. **Example** In this scenario, the `parent` fiber spawns a `child` fiber that repeatedly prints a message every second. The `child` fiber will be terminated when the `parent` fiber completes. This behavior can be extended to any level of nested fibers, ensuring a predictable and controlled fiber lifecycle. ### Fork in Global Scope You can create a long-running background fiber using `Effect.forkDetach`. This type of fiber, known as a daemon fiber, is not tied to the lifecycle of its parent fiber. Instead, its lifetime is linked to the global scope. A daemon fiber continues running even if its parent fiber is terminated and will only stop when the global scope is closed or the fiber completes naturally. **Example** This example shows how daemon fibers can continue running in the background even after the parent fiber has finished. Even if the parent fiber is interrupted, the daemon fiber will continue running independently. **Example** In this example, interrupting the parent fiber doesn't affect the daemon fiber, which continues to run in the background. ### Fork in Local Scope Sometimes we want to create a fiber that is tied to a local , meaning its lifetime is not dependent on its parent fiber but is bound to the local scope in which it was forked. This can be done using the `Effect.forkScoped` operator. Fibers created with `Effect.forkScoped` can outlive their parent fibers and will only be terminated when the local scope itself is closed. **Example** In this example, the `child` fiber continues to run beyond the lifetime of the `parent` fiber. The `child` fiber is tied to the local scope and will be terminated only when the scope ends. ### Fork in Specific Scope There are some cases where we need more fine-grained control, so we want to fork a fiber in a specific scope. We can use the `Effect.forkIn` operator which takes the target scope as an argument. **Example** In this example, the `child` fiber is forked into the `outerScope`, allowing it to outlive the inner scope but still be terminated when the `outerScope` is closed. ## When do Fibers run? Forked fibers begin execution after the current fiber completes or yields. **Example** In the following example, the `changes` stream only captures a single value, `2`. This happens because the fiber created by `Effect.forkChild` starts **after** the value is updated. If you add a short delay with `Effect.sleep` or call `Effect.yieldNow`, you allow the current fiber to yield. This gives the forked fiber enough time to start and collect all values before they are updated. **Example** --- title: Latch description: A Latch synchronizes fibers by allowing them to wait until a specific event occurs, controlling access based on its open or closed state. sidebar: order: 8 --- A Latch is a synchronization tool that works like a gate, letting fibers wait until the latch is opened before they continue. The latch can be either open or closed: - When closed, fibers that reach the latch wait until it opens. - When open, fibers pass through immediately. Once opened, a latch typically stays open, although you can close it again if needed Imagine an application that processes requests only after completing an initial setup . You can create a latch in a closed state while the setup is happening. Any incoming requests, represented as fibers, would wait at the latch until it opens. Once the setup is finished, you call `latch.open` so the requests can proceed. ## The Latch Interface A `Latch` includes several operations that let you control and observe its state: | Operation | Description | | ---------- | -------------------------------------------------------------------------------------------------------- | | `whenOpen` | Runs a given effect only if the latch is open, otherwise, waits until it opens. | | `open` | Opens the latch so that any waiting fibers can proceed. | | `close` | Closes the latch, causing fibers to wait when they reach this latch in the future. | | `await` | Suspends the current fiber until the latch is opened. If the latch is already open, returns immediately. | | `release` | Allows waiting fibers to continue without permanently opening the latch. | ## Creating a Latch Use the `Latch.make` function to create a latch in an open or closed state by passing a boolean. The default is `false`, which means it starts closed. **Example** In this example, the latch starts closed. A fiber logs "open sesame" only when the latch is open. After waiting for one second, the latch is opened, releasing the fiber: ## Latch vs Semaphore A latch is good when you have a one-time event or condition that determines whether fibers can proceed. For example, you might use a latch to block all fibers until a setup step is finished, and then open the latch so everyone can continue. A with one lock is usually for mutual exclusion: it ensures that only one fiber at a time accesses a shared resource or section of code. Once a fiber acquires the lock, no other fiber can enter the protected area until the lock is released. In short: - Use a **latch** if you're gating a set of fibers on a specific event . - Use a **semaphore ** if you need to ensure only one fiber at a time is in a critical section or using a shared resource. --- title: PubSub description: Effortless message broadcasting and asynchronous communication with PubSub in Effect. sidebar: order: 6 --- import { Aside } from "@astrojs/starlight/components" A `PubSub` serves as an asynchronous message hub, allowing publishers to send messages that can be received by all current subscribers. Unlike a , where each value is delivered to only one consumer, a `PubSub` broadcasts each published message to all subscribers. This makes `PubSub` ideal for scenarios requiring message broadcasting rather than load distribution. ## Basic Operations A `PubSub` stores messages of type `A` and provides two fundamental operations: | API | Description | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `PubSub.publish` | Sends a message of type `A` to the `PubSub`, returning an effect indicating if the message was successfully published. | | `PubSub.subscribe` | Creates a scoped effect that allows subscription to the `PubSub`, automatically unsubscribing when the scope ends. Subscribers receive messages through a which holds published messages. | **Example** ## Creating a PubSub ### Bounded PubSub A bounded `PubSub` applies back pressure to publishers when it reaches capacity, suspending additional publishing until space becomes available. Back pressure ensures that all subscribers receive all messages while they are subscribed. However, it can lead to slower message delivery if a subscriber is slow. **Example** ### Dropping PubSub A dropping `PubSub` discards new values when full. The `PubSub.publish` operation returns `false` if the message is dropped. In a dropping pubsub, publishers can continue to publish new values, but subscribers are not guaranteed to receive all messages. **Example** ### Sliding PubSub A sliding `PubSub` removes the oldest message to make space for new ones, ensuring that publishing never blocks. A sliding pubsub prevents slow subscribers from impacting the message delivery rate. However, there's still a risk that slow subscribers may miss some messages. **Example** ### Unbounded PubSub An unbounded `PubSub` has no capacity limit, so publishing always succeeds immediately. Unbounded pubsubs guarantee that all subscribers receive all messages without slowing down message delivery. However, they can grow indefinitely if messages are published faster than they are consumed. Generally, it's recommended to use bounded, dropping, or sliding pubsubs unless you have specific use cases for unbounded pubsubs. **Example** ## Operators On PubSubs ### publishAll The `PubSub.publishAll` function lets you publish multiple values to the pubsub at once. **Example** ### capacity / size You can check the capacity and current size of a pubsub using `PubSub.capacity` and `PubSub.size`, respectively. Note that `PubSub.capacity` returns a `number` because the capacity is set at pubsub creation and never changes. In contrast, `PubSub.size` returns an effect that determines the current size of the pubsub since the number of messages in the pubsub can change over time. **Example** ### Shutting Down a PubSub To shut down a pubsub, use `PubSub.shutdown`. You can also verify if it has been shut down with `PubSub.isShutdown`, or wait for the shutdown to complete with `PubSub.awaitShutdown`. Shutting down a pubsub also terminates all associated queues, ensuring that the shutdown signal is effectively communicated. --- title: Queue description: Learn how to use Effect's Queue for lightweight, type-safe, and asynchronous workflows with built-in back-pressure. sidebar: order: 5 --- A `Queue` is a lightweight in-memory queue with built-in back-pressure, enabling asynchronous, purely-functional, and type-safe handling of data. ## Basic Operations A `Queue` stores values of type `A` and provides two fundamental operations: | API | Description | | ------------- | ---------------------------------------------------- | | `Queue.offer` | Adds a value of type `A` to the queue. | | `Queue.take` | Removes and returns the oldest value from the queue. | **Example** ## Creating a Queue Queues can be **bounded** or **unbounded** . Different types of queues handle new values differently when they reach capacity. ### Bounded Queue A bounded queue applies back-pressure when full, meaning any `Queue.offer` operation will suspend until there is space. **Example** ### Dropping Queue A dropping queue discards new values if the queue is full. **Example** ### Sliding Queue A sliding queue removes old values to make space for new ones when it reaches capacity. **Example** ### Unbounded Queue An unbounded queue has no capacity limit, allowing unrestricted additions. **Example** ## Adding Items to a Queue ### offer Use `Queue.offer` to add values to the queue. **Example** When using a back-pressured queue, `Queue.offer` suspends if the queue is full. To avoid blocking the main fiber, you can fork the `Queue.offer` operation. **Example** ### offerAll You can also add multiple items at once using `Queue.offerAll`. **Example** ## Consuming Items from a Queue ### take The `Queue.take` operation removes and returns the oldest item from the queue. If the queue is empty, `Queue.take` will suspend and only resume when an item is added. To prevent blocking, you can fork the `Queue.take` operation into a new fiber. **Example** ### poll To retrieve the queue's first item without suspending, use `Queue.poll`. If the queue is empty, `Queue.poll` returns `None`; if it has an item, it wraps it in `Some`. **Example** ### takeUpTo To retrieve multiple items, use `Queue.takeBetween`, which returns up to the specified number of items. If there aren't enough items, it returns all available items without waiting for more. This function is particularly useful for batch processing when an exact number of items is not required. It ensures the program continues working with whatever data is currently available. If you need to wait for an exact number of items before proceeding, consider using . **Example** ### takeN Takes a specified number of elements from a queue. If the queue does not contain enough elements, the operation suspends until the required number of elements become available. This function is useful for scenarios where processing requires an exact number of items at a time, ensuring that the operation does not proceed until the batch is complete. **Example** ### takeAll To retrieve all items from the queue at once, use `Queue.takeAll`. This operation completes immediately, returning an empty collection if the queue is empty. **Example** ## Shutting Down a Queue ### shutdown The `Queue.shutdown` operation allows you to interrupt all fibers that are currently suspended on `offer*` or `take*` operations. This action also empties the queue and makes any future `offer*` and `take*` calls terminate immediately. **Example** ### await The `Queue.await` operation waits until the queue reaches its `Done` state. Because `Queue.shutdown` completes the queue with an interrupt cause, an effect that is awaiting shutdown is itself interrupted, not resolved as a normal success. To run cleanup logic regardless of that outcome, use `Effect.onExit` instead of `Effect.andThen`, and use `Fiber.await` instead of `Fiber.join` so the interruption does not propagate to the fiber that joins it. **Example** ## Offer-only / Take-only Queues Sometimes, you might want certain parts of your code to only add values to a queue or only retrieve values from a queue . Effect provides interfaces to enforce these specific capabilities. ### Enqueue All methods for adding values to a queue are defined by the `Enqueue` interface. This restricts the queue to only offer operations. **Example** ### Dequeue Similarly, all methods for retrieving values from a queue are defined by the `Dequeue` interface, which restricts the queue to only take operations. **Example** The `Queue` type combines both `Enqueue` and `Dequeue`, so you can easily pass it to different parts of your code, enforcing only `Enqueue` or `Dequeue` behaviors as needed. **Example** --- title: Semaphore description: Learn to use semaphores in Effect for precise control of concurrency, managing resource access, and coordinating asynchronous tasks effectively. sidebar: order: 7 --- import { Aside } from "@astrojs/starlight/components" A semaphore is a synchronization mechanism used to manage access to a shared resource. In Effect, semaphores help control resource access or coordinate tasks within asynchronous, concurrent operations. A semaphore acts as a generalized mutex, allowing a set number of **permits** to be held and released concurrently. Permits act like tickets, giving tasks or fibers controlled access to a shared resource. When no permits are available, tasks trying to acquire one will wait until a permit is released. ## Creating a Semaphore The `Semaphore.make` function initializes a semaphore with a specified number of permits. Each permit allows one task to access a resource or perform an operation concurrently, and multiple permits enable a configurable level of concurrency. **Example** ## withPermits The `withPermits` method lets you specify the number of permits required to run an effect. Once the specified permits are available, it runs the effect, automatically releasing the permits when the task completes. **Example** In this example, three tasks are started concurrently, but they run sequentially because the one-permit semaphore only allows one task to proceed at a time. **Example** In this example, we create a semaphore with five permits and use `withPermits` to allocate a different number of permits for each task: --- title: Configuration description: Describe, load, validate, and test application configuration with Config and ConfigProvider. sidebar: order: 5 --- Effect separates the **description** of a configuration from the source that supplies its values: - A `Config` describes how to load and decode a value of type `T`. It is also an `Effect`, so it can be yielded directly inside `Effect.gen`. - A `ConfigProvider` supplies the raw values. The default provider reads environment variables. Keeping these concerns separate lets application code define its requirements once and choose a different provider for production, local development, or tests. ## Defining and Parsing a Config Use the constructors in the `Config` module for individual values and `Config.all` to combine them. **Example** Calling `config.parse` is useful when the provider is already available explicitly, especially in tests. In application code, a `Config` can instead be yielded as an Effect. It then uses the `ConfigProvider` installed in the Effect context. When no provider is installed explicitly, Effect uses `ConfigProvider.fromEnv`. **Example** ## Built-in Config Values Effect provides convenience constructors for common scalar values: | Constructor | Result | | ------------------------- | -------------------------------------------------------- | | `string` | A string | | `nonEmptyString` | A non-empty string | | `finite` | A finite number | | `int` | An integer | | `port` | An integer between 1 and 65,535 | | `boolean` | A boolean | | `literal` | One literal value | | `literals` | One of several literal values | | `duration` | A `Duration` | | `date` | A `Date` | | `url` | A `URL` | | `logLevel` | A | | `redacted` | A | For ordinary numeric settings, prefer `Config.finite`. `Config.number` also accepts non-finite values such as `NaN` and `Infinity`, which are rarely valid configuration values. `Config.boolean` accepts the case-sensitive strings `true`, `false`, `yes`, `no`, `on`, `off`, `1`, `0`, `y`, and `n`. ## Using Config with Schema Use `Config.schema` when a setting needs a custom type, validation, or a structured representation. The provider supplies the encoded representation and the resulting `Config` produces the schema's `Type`. **Example** Schemas can also describe an entire configuration object. **Example** `Config.schema` derives a tree-shaped string encoding from the schema. This allows the same structured config to read `SERVER_HOST` and `SERVER_PORT` from an environment provider, or `{ server: { host, port } }` from `ConfigProvider.fromUnknown`. See for more details. ### Arrays and Records `Config.Array` and `Config.Record` are schemas for values that may be supplied either structurally or as one separated string. Pass them to `Config.schema`; they are not `Config` combinators. **Example** `Config.Record` similarly accepts either a record or a string such as `"service.name=api,service.version=1.0"`. Both schemas accept options for customizing their separators. Plain `Schema.Array` and `Schema.Record` remain useful when the provider must supply a structural array or object rather than a separated scalar value. ## Combining Configs `Config.all` combines configs into a tuple or a named object while preserving the input shape. **Example** `Config.nested` prepends a string or path to every lookup performed by that config. With `ConfigProvider.fromEnv`, path segments are joined with `_`. For APIs that accept either an already-built `Config` or a nested record of configs, `Config.Wrap` and `Config.unwrap` provide the corresponding input type and conversion. ## Defaults and Optional Values `Config.withDefault` supplies a value only when none of the relevant input is present. **Example** Invalid input is not treated as missing. For example, `PORT=not-a-port` still fails instead of silently producing `8080`. The same rule applies to a combined config: if part of the group is supplied, a missing or invalid sibling makes the group incomplete rather than replacing the whole group with a default. Use `Config.option` when absence should produce an `Option` instead. **Example** ### Falling Back After Any Config Error `Config.orElse` is broader than `Config.withDefault`: it tries another config after any `ConfigError`, including invalid input. **Example** Because `orElse` can recover from validation errors, use it only when replacing invalid input is intentional. Use `withDefault` for the common case where only absence should trigger a fallback. ## Transforming Values Use `Config.map` for a transformation that cannot fail. **Example** For validation and parsing, prefer expressing the rules in a schema and using `Config.schema`. `Config.mapOrFail` is available when a transformation instead needs to return an `Effect`. ## Handling Sensitive Values `Config.redacted` wraps a string in `Redacted`, whose string representation does not reveal the value. Accessing the secret requires an explicit call to `Redacted.value`. **Example** To decode a value and then wrap it, combine `Config.schema` with `Schema.RedactedFromValue`. ## Config Providers The `ConfigProvider` module includes providers for several common sources: | Constructor | Source | | -------------------- | ---------------------------------------------------------- | | `fromEnv` | Environment variables | | `fromUnknown` | An in-memory JavaScript value, including parsed JSON | | `fromDotEnvContents` | The contents of a `.env` file | | `fromDotEnv` | A `.env` file read through the `FileSystem` service | | `fromDir` | A directory tree, including mounted ConfigMaps and Secrets | | `make` | A custom backing store | `fromEnv` merges `process.env` and `import.meta.env` by default. Passing `{ env }` replaces that source, which makes it convenient for tests and non-Node runtimes. Both `fromEnv` and `fromUnknown` treat an empty string as missing by default. Pass `{ preserveEmptyStrings: true }` when an empty string is a meaningful value. ### Loading an In-Memory Object Use `ConfigProvider.fromUnknown` for a JavaScript value or parsed JSON object. Object keys and array indexes become configuration path segments. **Example** ### Loading .env Files and Directories `ConfigProvider.fromDotEnvContents` parses an already-loaded string. `ConfigProvider.fromDotEnv` reads `.env` from the current directory by default and returns an Effect that requires the `FileSystem` service. `ConfigProvider.fromDir` reads each file as a leaf value and each directory as a record. This is useful for configuration mounted as files, such as Kubernetes ConfigMaps and Secrets. It requires the `FileSystem` and `Path` services. ### Transforming Provider Paths Provider combinators change how all configs look up their paths: - `ConfigProvider.nested` prefixes every lookup. - `ConfigProvider.constantCase` converts string path segments to `CONSTANT_CASE`. - `ConfigProvider.mapInput` performs an arbitrary path transformation. **Example** ### Combining Providers `ConfigProvider.orElse` consults the fallback only when the primary provider has no value at a requested path. Unlike `Config.orElse`, it does not recover from source errors or schema validation errors. **Example** ### Installing a Provider Use `ConfigProvider.layer` to replace the provider used by configs yielded as Effects. `ConfigProvider.layerAdd` instead adds a fallback to the current provider; pass `{ asPrimary: true }` to make the added provider take precedence. **Example** For a single config, `config.parse` is simpler and does not change the provider for the rest of the Effect program. --- title: BigDecimal description: The BigDecimal data type represents arbitrary-precision decimal numbers. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" In JavaScript, numbers are typically stored as 64-bit floating-point values. While floating-point numbers are fast and versatile, they can introduce small rounding errors. These are often hard to notice in everyday usage but can become problematic in areas like finance or statistics, where small inaccuracies may lead to larger discrepancies over time. By using the BigDecimal module, you can avoid these issues and perform calculations with a higher degree of precision. The `BigDecimal` data type can represent real numbers with a large number of decimal places, preventing the common errors of floating-point math . ## How BigDecimal Works A `BigDecimal` represents a number using two components: 1. `value`: A `BigInt` that stores the digits of the number. 2. `scale`: A 64-bit integer that determines the position of the decimal point. The number represented by a `BigDecimal` is calculated as: value x 10-scale. - If `scale` is zero or positive, it specifies the number of digits to the right of the decimal point. - If `scale` is negative, the `value` is multiplied by 10 raised to the power of the negated scale. For example: - A `BigDecimal` with `value = 12345n` and `scale = 2` represents `123.45`. - A `BigDecimal` with `value = 12345n` and `scale = -2` represents `1234500`. The maximum precision is large but not infinite, limited to 263 decimal places. ## Creating a BigDecimal ### make The `make` function creates a `BigDecimal` by specifying a `BigInt` value and a scale. The `scale` determines the number of digits to the right of the decimal point. **Example** ### fromBigInt The `fromBigInt` function creates a `BigDecimal` from a `bigint`. The `scale` defaults to `0`, meaning the number has no fractional part. **Example** ### fromString Parses a numerical string into a `BigDecimal`. Returns an `Option`: - `Some` if the string is valid. - `None` if the string is invalid. **Example** ### unsafeFromString The `fromStringUnsafe` function is a variant of `fromString` that throws an error if the input string is invalid. Use this only when you are confident that the input will always be valid. **Example** ### unsafeFromNumber Creates a `BigDecimal` from a JavaScript `number`. Throws a `RangeError` for non-finite numbers . **Example** ## Basic Arithmetic Operations The BigDecimal module supports a variety of arithmetic operations that provide precision and avoid the rounding errors common in standard JavaScript arithmetic. Below is a list of supported operations: | Function | Description | | ----------------- | -------------------------------------------------------------------------------------------------------------- | | `sum` | Adds two `BigDecimal` values. | | `subtract` | Subtracts one `BigDecimal` value from another. | | `multiply` | Multiplies two `BigDecimal` values. | | `divide` | Divides one `BigDecimal` value by another, returning an `Option`. | | `divideUnsafe` | Divides one `BigDecimal` value by another, throwing an error if the divisor is zero. | | `negate` | Negates a `BigDecimal` value . | | `remainder` | Returns the remainder of dividing one `BigDecimal` value by another, returning an `Option`. | | `remainderUnsafe` | Returns the remainder of dividing one `BigDecimal` value by another, throwing an error if the divisor is zero. | | `sign` | Returns the sign of a `BigDecimal` value . | | `abs` | Returns the absolute value of a `BigDecimal`. | **Example** Using `BigDecimal` for arithmetic operations helps to avoid the inaccuracies commonly encountered with floating-point numbers in JavaScript. For example: **Example** ## Comparison Operations The `BigDecimal` module provides several functions for comparing decimal values. These allow you to determine the relative order of two values, find the minimum or maximum, and check specific properties like positivity or integer status. ### Comparison Functions | Function | Description | | ------------------------ | ------------------------------------------------------------------------ | | `isLessThan` | Checks if the first `BigDecimal` is smaller than the second. | | `isLessThanOrEqualTo` | Checks if the first `BigDecimal` is smaller than or equal to the second. | | `isGreaterThan` | Checks if the first `BigDecimal` is larger than the second. | | `isGreaterThanOrEqualTo` | Checks if the first `BigDecimal` is larger than or equal to the second. | | `min` | Returns the smaller of two `BigDecimal` values. | | `max` | Returns the larger of two `BigDecimal` values. | **Example** ### Predicates for Comparison The module also includes predicates to check specific properties of a `BigDecimal`: | Predicate | Description | | ------------ | -------------------------------------------------------------- | | `isZero` | Checks if the value is exactly zero. | | `isPositive` | Checks if the value is positive. | | `isNegative` | Checks if the value is negative. | | `between` | Checks if the value lies within a specified range . | | `isInteger` | Checks if the value is an integer . | **Example** ## Normalization and Equality In some cases, two `BigDecimal` values can have different internal representations but still represent the same number. For example, `1.05` could be internally represented with different scales, such as: - `105n` with a scale of `2` - `1050n` with a scale of `3` To ensure consistency, you can normalize a `BigDecimal` to adjust the scale and remove trailing zeros. ### Normalization The `BigDecimal.normalize` function adjusts the scale of a `BigDecimal` and eliminates any unnecessary trailing zeros in its internal representation. **Example** ### Equality To check if two `BigDecimal` values are numerically equal, regardless of their internal representation, use the `BigDecimal.equals` function. **Example** --- title: Cause description: Comprehensive error analysis with Cause in Effect - track failures, defects, and interruptions with precise details. sidebar: order: 2 --- The type is polymorphic in error type `E`, allowing flexibility in handling any desired error type. However, there is often additional information about failures that the error type `E` alone does not capture. To address this, Effect uses the `Cause` data type to store various details such as: - Unexpected errors or defects - Stack and execution traces - Reasons for fiber interruptions Effect strictly preserves all failure-related information, storing a full picture of the error context in the `Cause` type. This comprehensive approach enables precise analysis and handling of failures, ensuring no data is lost. Though `Cause` values aren't typically manipulated directly, they underlie errors within Effect workflows, providing access to both concurrent and sequential error details. This allows for thorough error analysis when needed. ## Creating Causes You can intentionally create an effect with a specific cause using `Effect.failCause`. **Example** Some causes do not influence the error type of the effect, leading to `never` in the error channel: For instance, `Cause.die` does not specify an error type for the effect, while `Cause.fail` does, setting the error channel type accordingly. ## Cause Variations There are several causes for various errors, in this section, we will describe each of these causes. ### Empty The `Empty` cause signifies the absence of any errors, represented by an empty `reasons` array . ### Fail The `Fail` reason represents a failure due to an expected error of type `E`. A `Cause` holding only this reason is created with `Cause.fail`. ### Die The `Die` reason indicates a failure resulting from a defect, which is an unexpected or unintended error. A `Cause` holding only this reason is created with `Cause.die`. ### Interrupt The `Interrupt` reason represents a failure due to `Fiber` interruption and contains the numeric id of the interrupted `Fiber`. A `Cause` holding only this reason is created with `Cause.interrupt`. ### Combining Causes `Cause` stores its failure reasons in a flat `reasons` array. Reasons that occurred sequentially and reasons that occurred concurrently use the same representation. Use `Cause.combine` to merge two causes into one. **Example** ## Retrieving the Cause of an Effect To retrieve the cause of a failed effect, use `Effect.exit` and inspect the `cause` field of a `Failure`. This allows you to inspect or handle the exact reason behind the failure. **Example** ## Guards To determine what happened inside a `Cause`, the Cause module provides two kinds of guards: cause-level predicates that check whether a `Cause` contains a certain kind of reason, and reason-level guards that narrow an individual entry of `cause.reasons`. - `Cause.hasFails`: Checks if the cause contains at least one expected failure. - `Cause.hasDies`: Checks if the cause contains at least one unexpected defect. - `Cause.hasInterrupts`: Checks if the cause contains at least one fiber interruption. - `Cause.hasInterruptsOnly`: Checks if every reason in the cause is an interruption. - `Cause.isFailReason`: Narrows a `Reason` to `Fail`. - `Cause.isDieReason`: Narrows a `Reason` to `Die`. - `Cause.isInterruptReason`: Narrows a `Reason` to `Interrupt`. An empty cause is checked with `cause.reasons.length === 0`; there is no dedicated `isEmpty` function. **Example** These guards allow you to accurately identify the reasons behind a `Cause`, making it easier to handle various error cases in your code. Whether dealing with expected failures, unexpected defects, or interruptions, these guards provide a clear method for assessing and managing error scenarios. ## Formatting Reasons To respond to specific error scenarios with custom behavior, iterate `cause.reasons` and switch on each reason's `_tag`. **Example** ## Pretty Printing Clear and readable error messages are key for effective debugging. The `Cause.pretty` function helps by formatting error messages in a structured way, making it easier to understand failure details. **Example** ## Retrieval of Failures and Defects Filter `cause.reasons` with `Cause.isFailReason` and `Cause.isDieReason` to inspect only the expected errors or unexpected defects that occurred. **Example** --- title: Chunk description: Learn about Chunk, a high-performance immutable data structure in Effect, offering efficient operations like concatenation, slicing, and conversions. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" A `Chunk` represents an ordered, immutable collection of values of type `A`. While similar to an array, `Chunk` provides a functional interface, optimizing certain operations that can be costly with regular arrays, like repeated concatenation. ## Why Use Chunk? - **Immutability**: Unlike standard JavaScript arrays, which are mutable, `Chunk` provides a truly immutable collection, preventing data from being modified after creation. This is especially useful in concurrent programming contexts where immutability can enhance data consistency. - **High Performance**: `Chunk` supports specialized operations for efficient array manipulation, such as appending single elements or concatenating chunks, making these operations faster than their regular JavaScript array equivalents. ## Creating a Chunk ### empty Create an empty `Chunk` with `Chunk.empty`. **Example** ### make To create a `Chunk` with specific values, use `Chunk.make`. Note that the resulting chunk is typed as non-empty. **Example** ### fromIterable You can create a `Chunk` by providing a collection, either from an iterable or directly from an array. **Example** ### unsafeFromArray `Chunk.fromArrayUnsafe` creates a `Chunk` directly from an array without cloning. This approach can improve performance by avoiding the overhead of copying data but requires caution, as it bypasses the usual immutability guarantees. **Example** ## Concatenating To combine two `Chunk` instances into one, use `Chunk.appendAll`. **Example** ## Dropping To remove elements from the beginning of a `Chunk`, use `Chunk.drop`, specifying the number of elements to discard. **Example** ## Comparing To check if two `Chunk` instances are equal, use . This function compares the contents of each `Chunk` for structural equality. **Example** ## Converting Convert a `Chunk` to a `ReadonlyArray` using `Chunk.toReadonlyArray`. The resulting type varies based on the `Chunk`'s contents, distinguishing between empty, non-empty, and generic chunks. **Example** --- title: Data description: Define immutable data structures, ensure equality, and manage errors seamlessly with Effect's Data module. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" The Data module simplifies creating and handling data structures in TypeScript. It provides tools for **defining data types**, ensuring **equality** between objects, and **hashing** data for efficient comparisons. ## Value Equality Plain JavaScript objects, arrays, tuples, `Map`s, and `Set`s get structural equality with `Equal.equals` by default. No special constructor is required. See for the full explanation. This means that two plain values are considered equal if they have the same structure and values. ### struct In plain JavaScript, objects are considered equal only if they refer to the exact same instance. **Example** However, `Equal.equals` lets you compare the same two objects based on their structure and content. **Example** The comparison performed by `Equal.equals` is **deep**: nested objects are compared recursively, with no extra work required. **Example** A different nested value makes the objects unequal, as you would expect. **Example** ### tuple Plain arrays used as tuples are compared structurally as well. **Example** ### array Plain arrays support structural equality too. **Example** ## Constructors The module introduces a concept known as "Case classes", which automate various essential operations when defining data types. These operations include generating **constructors**, handling **equality** checks, and managing **hashing**. Case classes can be defined in two primary ways: - as plain objects, using an ordinary factory function when you want a reusable constructor; equality and hashing come for free - as TypeScript classes using `Class` or `TaggedClass`, when you want a class-oriented structure with methods and custom logic ### Constructor functions A plain factory function that returns an object literal gives you a reusable constructor. Since plain objects have structural equality by default, no special helper is needed for equality or hashing. **Example** In this example, a plain arrow function creates a constructor for `Person`. The resulting instances are plain objects, so they already support equality checks. You can compare them directly using `Equal.equals`. **Example** This example demonstrates nested data structures, such as a `Person` type containing an `Address`. Both `Person` and `Address` constructors return plain objects, so equality checks work out of the box. Since nested plain objects are also compared structurally by default, you don't even need a separate `Address` constructor. An inline object literal works just as well. **Example** **Example** This example demonstrates a recursive structure defining a binary tree where each node can contain other nodes. ### Tagged constructor functions When you're working with a data type that includes a tag field, like in disjoint union types, defining the tag manually for each instance can get repetitive. **Example** Here, we create a `Person` type with a `_tag` field. Notice that the `_tag` needs to be specified for every new instance. To streamline this process, write a constructor function that adds the tag automatically. It follows the convention in the Effect ecosystem of naming the tag field as `"_tag"`. **Example** This way you define the tag just once, making instance creation simpler. ### Class If you prefer working with classes instead of plain objects, you can use `Data.Class` as an alternative to a constructor function. This approach may feel more natural in scenarios where you want a class-oriented structure, complete with methods and custom logic. **Example** Here's how to define a `Person` class using `Data.Class`: One of the benefits of using classes is that you can easily add custom methods and getters. This allows you to extend the functionality of your data types. **Example** In this example, we add a `upperName` getter to the `Person` class to return the name in uppercase: ### TaggedClass If you prefer a class-based approach but also want the benefits of tagging for disjoint unions, `Data.TaggedClass` can be a helpful option. It works similarly to `tagged` but is tailored for class definitions. **Example** Here's how to define a `Person` class using `Data.TaggedClass`. Notice that the tag `"Person"` is automatically added: One benefit of using tagged classes is the ability to easily add custom methods and getters, extending the class's functionality as needed. **Example** In this example, we add a `upperName` getter to the `Person` class, which returns the name in uppercase: ## Union of Tagged Structs To create a disjoint union of tagged structs, you can use `Data.TaggedEnum` and `Data.taggedEnum`. These utilities make it straightforward to define and work with unions of plain objects. ### Definition The type passed to `Data.TaggedEnum` must be an object where the keys represent the tags, and the values define the structure of the corresponding data types. **Example** ### $is and $match The `Data.taggedEnum` provides `$is` and `$match` functions for convenient type guarding and pattern matching. **Example** ### Adding Generics You can create more flexible and reusable tagged unions by using `TaggedEnum.WithGenerics`. This approach allows you to define tagged unions that can handle different types dynamically. **Example** ## Errors In Effect, handling errors is simplified using specialized constructors: - `Error` - `TaggedError` These constructors make defining custom error types straightforward, while also providing useful integrations like equality checks and structured error handling. ### Error `Data.Error` lets you create an `Error` type with extra fields beyond the typical `message` property. **Example** You can yield an instance of `NotFound` directly in an , without needing to use `Effect.fail`. **Example** ### TaggedError Effect provides a `TaggedError` API to add a `_tag` field automatically to your custom errors. This simplifies error handling with APIs like or . ### Native Cause Support Errors created using `Data.Error` or `Data.TaggedError` can include a `cause` property, integrating with the native `cause` feature of JavaScript's `Error` for more detailed error tracing. **Example** --- title: DateTime description: Work with precise points in time using Effect's DateTime, supporting creation, comparison, and arithmetic operations for efficient time handling. sidebar: order: 5 --- import { Aside } from "@astrojs/starlight/components" Working with dates and times in JavaScript can be challenging. The built-in `Date` object mutates its internal state, and time zone handling can be confusing. These design choices can lead to errors when working on applications that rely on date-time accuracy, such as scheduling systems, timestamping services, or logging utilities. The DateTime module aims to address these limitations by offering: - **Immutable Data**: Each `DateTime` is an immutable structure, reducing mistakes related to in-place mutations. - **Time Zone Support**: `DateTime` provides robust support for time zones, including automatic daylight saving time adjustments. - **Arithmetic Operations**: You can perform arithmetic operations on `DateTime` instances, such as adding or subtracting durations. ## The DateTime Type A `DateTime` represents a moment in time. It can be stored as either a simple UTC value or as a value with an associated time zone. Storing time this way helps you manage both precise timestamps and the context for how that time should be displayed or interpreted. There are two main variants of `DateTime`: 1. **Utc**: An immutable structure that uses `epochMilliseconds` to represent a point in time in Coordinated Universal Time . 2. **Zoned**: Includes `epochMilliseconds` along with a `TimeZone`, allowing you to attach an offset or a named region to the timestamp. ### Why Have Two Variants? - **Utc** is straightforward if you only need a universal reference without relying on local time zones. - **Zoned** is helpful when you need to keep track of time zone information for tasks such as converting to local times or adjusting for daylight saving time. ### TimeZone Variants A `TimeZone` can be either: - **Offset**: Represents a fixed offset from UTC . - **Named**: Uses a named region that automatically accounts for region-specific rules like daylight saving time changes. ### TypeScript Definition Below is the TypeScript definition for the `DateTime` type: ## The DateTime.Parts Type The `DateTime.Parts` type defines the main components of a date, such as the year, month, day, hours, minutes, and seconds. ## The DateTime.Input Type The `DateTime.Input` type is a flexible input type that can be used to create a `DateTime` instance. It can be one of the following: - A `DateTime` instance - A JavaScript `Date` object - A numeric value representing milliseconds since the Unix epoch - An object with partial date - A string that can be parsed by JavaScript's ## Utc Constructors `Utc` is an immutable structure that uses `epochMilliseconds` to represent a point in time in Coordinated Universal Time . ### unsafeFromDate Creates a `Utc` from a JavaScript `Date`. Throws an `IllegalArgumentError` if the provided `Date` is invalid. When a `Date` object is passed, it is converted to a `Utc` instance. The time is interpreted as the local time of the system executing the code and then adjusted to UTC. This ensures a consistent, timezone-independent representation of the date and time. **Example** The following example assumes the code is executed on a system in Italy : **Explanation**: - The local time **2025-01-01 04:00:00** is converted to **UTC** by subtracting the timezone offset . - As a result, the UTC time becomes **2025-01-01 03:00:00.000Z**. - `epochMilliseconds` provides the same time as milliseconds since the Unix Epoch, ensuring a precise numeric representation of the UTC timestamp. ### unsafeMake Creates a `Utc` from a . **Example** The following example assumes the code is executed on a system in Italy : **Explanation**: - The local time **2025-01-01 04:00:00** is converted to **UTC** by subtracting the timezone offset . - As a result, the UTC time becomes **2025-01-01 03:00:00.000Z**. ### make Similar to , but returns an instead of throwing an error if the input is invalid. If the input is invalid, it returns `None`. If valid, it returns `Some` containing the `Utc`. **Example** The following example assumes the code is executed on a system in Italy : **Explanation**: - The local time **2025-01-01 04:00:00** is converted to **UTC** by subtracting the timezone offset . - As a result, the UTC time becomes **2025-01-01 03:00:00.000Z**. ## Zoned Constructors A `Zoned` includes `epochMilliseconds` along with a `TimeZone`, allowing you to attach an offset or a named region to the timestamp. ### unsafeMakeZoned Creates a `Zoned` by combining a with an optional `TimeZone`. This allows you to represent a specific point in time with an associated time zone. The time zone can be provided in several ways: - As a `TimeZone` object - A string identifier - A numeric offset in milliseconds If the input or time zone is invalid, an `IllegalArgumentError` is thrown. **Example** The following example assumes the code is executed on a system in Italy : Here, the system's time zone is used to create the `Zoned` instance. **Example** The following example assumes the code is executed on a system in Italy : In this case, the `"Europe/Rome"` time zone is explicitly provided, resulting in the `Zoned` instance being tied to this named time zone. By default, the input date is treated as a UTC value and then adjusted for the specified time zone. To interpret the input date as being in the specified time zone, you can use the `adjustForTimeZone` option. **Example** The following example assumes the code is executed on a system in Italy : **Explanation** - **Without `adjustForTimeZone`**: The input date is interpreted as UTC and then adjusted to the specified time zone. For instance, `2025-01-01 04:00:00` in UTC becomes `2025-01-01T04:00:00.000+01:00` in CET . - **With `adjustForTimeZone: true`**: The input date is interpreted as being in the specified time zone. For example, `2025-01-01 04:00:00` in "Europe/Rome" is adjusted to its corresponding UTC time, resulting in `2025-01-01T03:00:00.000+01:00`. ### makeZoned The `makeZoned` function works similarly to but provides a safer approach. Instead of throwing an error when the input is invalid, it returns an `Option`. If the input is invalid, it returns `None`. If valid, it returns `Some` containing the `Zoned`. **Example** ### makeZonedFromString Creates a `Zoned` by parsing a string in the format `YYYY-MM-DDTHH:mm:ss.sss+HH:MM`. If the input string is valid, the function returns a `Some` containing the `Zoned`. If the input is invalid, it returns `None`. **Example** ## Current Time ### now Provides the current UTC time as a `Effect`, using the service. **Example** ### unsafeNow Retrieves the current UTC time immediately using `Date.now`, without the service. **Example** ## Guards | Function | Description | | ------------------ | ---------------------------------------------- | | `isDateTime` | Checks if a value is a `DateTime`. | | `isTimeZone` | Checks if a value is a `TimeZone`. | | `isTimeZoneOffset` | Checks if a value is a `TimeZone.Offset`. | | `isTimeZoneNamed` | Checks if a value is a `TimeZone.Named`. | | `isUtc` | Checks if a `DateTime` is the `Utc` variant. | | `isZoned` | Checks if a `DateTime` is the `Zoned` variant. | **Example** ## Time Zone Management | Function | Description | | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | | `setZone` | Creates a `Zoned` from `DateTime` by applying the given `TimeZone`. | | `setZoneOffset` | Creates a `Zoned` from `DateTime` using a fixed offset . | | `setZoneNamed` | Creates a `Zoned` from `DateTime` from an IANA time zone identifier or returns `None` if invalid. | | `setZoneNamedUnsafe` | Creates a `Zoned` from `DateTime` from an IANA time zone identifier or throws if invalid. | | `zoneMakeNamedUnsafe` | Creates a `TimeZone.Named` from a IANA time zone identifier or throws if the identifier is invalid. | | `zoneMakeNamed` | Creates a `TimeZone.Named` from a IANA time zone identifier or returns `None` if invalid. | | `zoneMakeNamedEffect` | Creates a `Effect` from a IANA time zone identifier failing with `IllegalArgumentError` if invalid | | `zoneMakeOffset` | Creates a `TimeZone.Offset` from a numeric offset in milliseconds. | | `zoneMakeLocal` | Creates a `TimeZone.Named` from the system's local time zone. | | `zoneFromString` | Attempts to parse a time zone from a string, returning `None` if invalid. | | `zoneToString` | Returns a string representation of a `TimeZone`. | **Example** ### zoneFromString Parses a string to create a `DateTime.TimeZone`. This function attempts to interpret the input string as either: - A numeric time zone offset - An IANA time zone identifier If the string matches an offset format, it is converted into a `TimeZone.Offset`. Otherwise, it attempts to create a `TimeZone.Named` using the input. If the input string is invalid, `Option.none` is returned. **Example** ## Comparisons | Function | Description | | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | | `distance` | Returns the difference between two `DateTime`s as a signed `Duration` . | | `min` | Returns the earlier of two `DateTime` values. | | `max` | Returns the later of two `DateTime` values. | | `isGreaterThan`, `isGreaterThanOrEqualTo`, etc. | Checks ordering between two `DateTime` values. | | `between` | Checks if a `DateTime` lies within the given bounds. | | `isFuture`, `isPast`, `isFutureUnsafe`, etc. | Checks if a `DateTime` is in the future or past. | **Example** ## Conversions | Function | Description | | ---------------- | ----------------------------------------------------------------------- | | `toDateUtc` | Returns a JavaScript `Date` in UTC. | | `toDate` | Applies the time zone and converts to a JavaScript `Date`. | | `zonedOffset` | For a `Zoned` DateTime, returns the time zone offset in ms. | | `zonedOffsetIso` | For a `Zoned` DateTime, returns an ISO offset string like "+01:00". | | `toEpochMillis` | Returns the Unix epoch time in milliseconds. | | `removeTime` | Returns a `Utc` with the time cleared . | ## Parts | Function | Description | | -------------------------- | ---------------------------------------------------------------------- | | `toParts` | Returns time zone adjusted date parts . | | `toPartsUtc` | Returns UTC date parts . | | `getPart` / `getPartUtc` | Retrieves a specific part from the date. | | `setParts` / `setPartsUtc` | Updates certain parts of a date, preserving or ignoring the time zone. | **Example** ## Math | Function | Description | | ------------------ | ------------------------------------------------------------------------------------------ | | `addDuration` | Adds the given `Duration` to a `DateTime`. | | `subtractDuration` | Subtracts the given `Duration` from a `DateTime`. | | `add` | Adds numeric parts to a `DateTime`. | | `subtract` | Subtracts numeric parts. | | `startOf` | Moves a `DateTime` to the start of the given unit . | | `endOf` | Moves a `DateTime` to the end of the given unit. | | `nearest` | Rounds a `DateTime` to the nearest specified unit. | ## Formatting | Function | Description | | ------------------ | -------------------------------------------------------------------- | | `format` | Formats a `DateTime` as a string using the `DateTimeFormat` API. | | `formatLocal` | Uses the system's local time zone and locale for formatting. | | `formatUtc` | Forces UTC formatting. | | `formatIntl` | Uses a provided `Intl.DateTimeFormat`. | | `formatIso` | Returns an ISO 8601 string in UTC. | | `formatIsoDate` | Returns an ISO date string, adjusted for the time zone. | | `formatIsoDateUtc` | Returns an ISO date string in UTC. | | `formatIsoOffset` | Formats a `Zoned` as a string with an offset like "+01:00". | | `formatIsoZoned` | Formats a `Zoned` in the form `YYYY-MM-DDTHH:mm:ss.sss+HH:MM`. | ## Layers for Current Time Zone | Function | Description | | ------------------------ | -------------------------------------------------------------------- | | `CurrentTimeZone` | The service key for the current time zone. | | `setZoneCurrent` | Sets a `DateTime` to use the current time zone. | | `withCurrentZone` | Provides an effect with a specified time zone. | | `withCurrentZoneLocal` | Uses the system's local time zone for the effect. | | `withCurrentZoneOffset` | Uses a fixed offset for the effect. | | `withCurrentZoneNamed` | Uses a named time zone identifier . | | `nowInCurrentZone` | Retrieves the current time as a `Zoned` in the configured time zone. | | `layerCurrentZone` | Creates a Layer providing the `CurrentTimeZone` service. | | `layerCurrentZoneOffset` | Creates a Layer from a fixed offset. | | `layerCurrentZoneNamed` | Creates a Layer from a named time zone, failing if invalid. | | `layerCurrentZoneLocal` | Creates a Layer from the system's local time zone. | **Example** --- title: Duration description: Work with precise time spans using Effect's Duration, supporting creation, comparison, and arithmetic operations for efficient time handling. sidebar: order: 6 --- The `Duration` data type represents non-negative spans of time. It is commonly used for timeouts, delays, and scheduling, and provides operations for working with different time units. ## Creating Durations The Duration module includes several constructors to create durations in different units. **Example** You can create durations using units such as nanoseconds, microsecond, milliseconds, seconds, minutes, hours, days, and weeks. For an infinite duration, use `Duration.infinity`. **Example** Another option for creating durations is using the `Duration.fromInputUnsafe` helper: - `number` values are treated as milliseconds. - `bigint` values are treated as nanoseconds. - Strings must follow the format `"${number} ${unit}"`. **Example** ## Getting the Duration Value You can retrieve the value of a duration in milliseconds using `Duration.toMillis`. **Example** To get the value of a duration in nanoseconds, use `Duration.toNanos`. Note that `toNanos` returns an `Option` because the duration might be infinite. **Example** To get a `bigint` value without `Option`, use `Duration.toNanosUnsafe`. However, it will throw an error for infinite durations. **Example** ## Comparing Durations Use the following functions to compare two durations: | API | Description | | ------------------------ | ---------------------------------------------------------------------------- | | `isLessThan` | Returns `true` if the first duration is less than the second. | | `isLessThanOrEqualTo` | Returns `true` if the first duration is less than or equal to the second. | | `isGreaterThan` | Returns `true` if the first duration is greater than the second. | | `isGreaterThanOrEqualTo` | Returns `true` if the first duration is greater than or equal to the second. | **Example** ## Performing Arithmetic Operations You can perform arithmetic operations on durations, like addition and multiplication. **Example** ## Conversions Converts a `Duration` to a human readable string. **Example** --- title: Exit description: Represent the result of an Effect workflow with Exit, capturing success values or failure causes. sidebar: order: 8 --- An `Exit` describes the result of running an `Effect` workflow. There are two possible states for an `Exit`: - `Exit.Success`: Contains a success value of type `A`. - `Exit.Failure`: Contains a failure of type `E`. ## Creating Exits The Exit module provides two primary functions for constructing exit values: `Exit.succeed` and `Exit.failCause`. These functions represent the outcomes of an effectful computation in terms of success or failure. ### succeed `Exit.succeed` creates an `Exit` value that represents a successful outcome. You use this function when you want to indicate that a computation completed successfully and to provide the resulting value. **Example** ### failCause `Exit.failCause` creates an `Exit` value that represents a failure. The failure is described using a object, which can encapsulate expected errors, defects, interruptions, or even composite errors. **Example** ## Pattern Matching You can handle different outcomes of an `Exit` using the `Exit.match` function. This function lets you provide two separate callbacks to handle both success and failure cases of an `Effect` execution. **Example** ## Exit vs Result Conceptually, `Exit` can be thought of as `Result>`. However, the type represents more than just expected errors of type `E`. It includes: - Interruption causes - Defects - The combination of multiple causes This allows `Cause` to capture richer and more complex error states compared to a simple `Result`. ## Exit vs Effect `Exit` is actually a subtype of `Effect`. This means that `Exit` values can also be considered as `Effect` values. - An `Exit`, in essence, is a "constant computation". - `Effect.succeed` is essentially the same as `Exit.succeed`. - `Effect.failCause` is the same as `Exit.failCause`. --- title: HashSet description: Learn about HashSet data structures - both immutable and mutable variants. sidebar: order: 9 --- import { Aside } from "@astrojs/starlight/components" A HashSet represents an **unordered** collection of **unique** values with efficient lookup, insertion and removal operations. The Effect library provides two versions of this structure: - - Immutable - - Mutable Both versions provide constant-time operations on average. The main difference is how they handle changes: one returns new sets, the other modifies the original. ### Why use HashSet? HashSet solves the problem of maintaining an **unsorted collection where each value appears exactly once**, with fast operations for checking membership and adding/removing values. Some common use cases include: - Tracking unique items - Efficiently testing for membership in a collection - Performing set operations like union, intersection, and difference - Eliminating duplicates from a collection ### When to use HashSet Instead of other collections Choose HashSet over other collections when: - You need to ensure elements are unique - You frequently need to check if an element exists in the collection - You need to perform set operations like union, intersection, and difference - The order of elements doesn't matter to your use case Choose other collections when: - You need to maintain insertion order - You need key-value associations - You need to frequently access elements by index ### Choosing between immutable and mutable variants Effect offers both immutable and mutable versions to support different coding styles and performance needs. **HashSet** This version never modifies the original set. Instead, it returns a new set for every change. Characteristics: - Operations return new instances instead of modifying the original - Previous states are preserved - Thread-safe by design - Ideal for functional programming patterns - Suitable for sharing across different parts of your application **MutableHashSet** This version allows direct updates: adding and removing values changes the set in place. Characteristics: - Operations modify the original set directly - More efficient when building sets incrementally - Requires careful handling to avoid unexpected side effects - Better performance in scenarios with many modifications - Ideal for localized use where mutations won't cause issues elsewhere ### When to use each variant Use **HashSet** when: - You need predictable behavior with no side effects - You want to preserve previous states of your data - You're sharing sets across different parts of your application - You prefer functional programming patterns - You need fiber safety in concurrent environments Use **MutableHashSet** when: - Performance is critical, and you need to avoid creating new instances - You're building a collection incrementally with many additions/removals - You're working in a controlled scope where mutation is safe - You need to optimize memory usage in performance-critical code ### Hybrid approach You can apply multiple updates to a `HashSet` by chaining immutable operations with `pipe`. This allows you to perform several changes at once without mutating the original set. **Example** ## Performance characteristics Both `HashSet` and `MutableHashSet` offer similar average-time performance for core operations: | Operation | HashSet | MutableHashSet | Description | | -------------- | ------------ | -------------- | ------------------------------- | | Lookup | O average | O average | Check if a value exists | | Insertion | O average | O average | Add a value | | Removal | O average | O average | Remove a value | | Iteration | O | O | Iterate over all values | | Set operations | O | O | Union, intersection, difference | The main difference is how updates are handled: - **HashSet** returns a new set for each change. This can be slower if many changes are made in a row. - **MutableHashSet** updates the same set in place. This is usually faster when performing many changes. ## Equality and uniqueness Both `HashSet` and `MutableHashSet` use Effect's trait to determine if two elements are the same. This ensures that each value appears only once in the set. - **Primitive values** are compared by value, similar to the `===` operator. - **Objects and custom types** must implement the `Equal` interface to define what it means for two instances to be equal. If no implementation is provided, equality falls back to reference comparison. **Example** ### Simplifying Equality with Plain Objects and Schema Plain objects implement automatically, based on structural equality. No wrapper is needed. Values decoded with `Schema` already benefit from this, as shown below. **Example** **Example** ## HashSet A `HashSet` is an **immutable**, **unordered** collection of **unique** values. It guarantees that each value appears only once and supports fast operations like lookup, insertion, and removal. Any operation that would modify the set returns a new `HashSet`, leaving the original unchanged. ### Operations | Category | Operation | Description | Time Complexity | | ------------ | -------------------------------------------------------- | ------------------------------------------- | --------------- | | constructors | | Creates an empty HashSet | O | | constructors | | Creates a HashSet from an iterable | O | | constructors | | Creates a HashSet from multiple values | O | | elements | | Checks if a value exists in the set | O avg | | elements | | Checks if any element satisfies a predicate | O | | elements | | Checks if all elements satisfy a predicate | O | | elements | | Checks if a set is a subset of another | O | | getters | | Gets the number of elements | O | | mutations | | Adds a value to the set | O avg | | mutations | | Removes a value from the set | O avg | | operations | | Computes set difference | O | | operations | | Computes set intersection | O | | operations | | Computes set union | O | | mapping | | Transforms each element | O | | folding | | Reduces the set to a single value | O | | filtering | | Keeps elements that satisfy a predicate | O | `HashSet` is directly `Iterable`, so use `Array.from` or `self` to access its values. For transformations and traversals, compose the operations from the `Iterable` module; to split a set, use two `HashSet.filter` calls. **Example** **Example** ## MutableHashSet A `MutableHashSet` is a **mutable**, **unordered** collection of **unique** values. Unlike `HashSet`, it allows direct modifications, operations like `add`, `remove`, and `clear` update the original set instead of returning a new one. This mutability can improve performance when you need to build or update a set repeatedly, especially within a local or isolated scope. ### Operations | Category | Operation | Description | Complexity | | ------------ | --------------------------------------------------------------- | ----------------------------------- | ---------- | | constructors | | Creates an empty MutableHashSet | O | | constructors | | Creates a set from an iterable | O | | constructors | | Creates a set from multiple values | O | | elements | | Checks if a value exists in the set | O avg | | elements | | Adds a value to the set | O avg | | elements | | Removes a value from the set | O avg | | getters | | Gets the number of elements | O | | mutations | | Removes all values from the set | O | **Example** ## Interoperability with JavaScript Both `HashSet` and `MutableHashSet` implement the `Iterable` interface, so you can use them with JavaScript features like: - the spread operator - `for...of` loops - `Array.from` You can also extract values as an array using `Array.from`. **Example** --- title: Option description: Represent optional values with Option, supporting presence or absence and seamless operations like mapping, combining, and pattern matching. sidebar: order: 10 --- import { Aside } from "@astrojs/starlight/components" The `Option` data type represents optional values. An `Option` can either be `Some`, containing a value of type `A`, or `None`, representing the absence of a value. You can use `Option` in scenarios like: - Using it for initial values - Returning values from functions that are not defined for all possible inputs - Managing optional fields in data structures - Handling optional function arguments ## Creating Options ### some Use the `Option.some` constructor to create an `Option` that holds a value of type `A`. **Example** ### none Use the `Option.none` constructor to create an `Option` representing the absence of a value. **Example** ### liftPredicate You can create an `Option` based on a predicate, for example, to check if a value is positive. **Example** Here's how you can achieve this using `Option.none` and `Option.some`: **Example** Alternatively, you can simplify the above logic with `Option.liftPredicate`: ## Modeling Optional Properties Consider a `User` model where the `"email"` property is optional and can hold a `string` value. We use the `Option` type to represent this optional property: Here are examples of how to create `User` instances with and without an email: **Example** ## Guards You can check whether an `Option` is a `Some` or a `None` using the `Option.isSome` and `Option.isNone` guards. **Example** ## Pattern Matching Use `Option.match` to handle both cases of an `Option` by specifying separate callbacks for `None` and `Some`. **Example** ## Working with Option ### map The `Option.map` function lets you transform the value inside an `Option` without manually unwrapping and re-wrapping it. If the `Option` holds a value , the transformation function is applied. If the `Option` is `None`, the function is ignored, and the `Option` remains unchanged. **Example** When dealing with `None`, the mapping function is not executed, and the `Option` remains `None`: **Example** ### flatMap The `Option.flatMap` function is similar to `Option.map`, but it is designed to handle cases where the transformation might return another `Option`. This allows us to chain computations that depend on whether or not a value is present in an `Option`. Consider a `User` model that includes a nested optional `Address`, which itself contains an optional `street` property: In this model, the `address` field is an `Option
`, and the `street` field within `Address` is an `Option`. We can use `Option.flatMap` to extract the `street` property from `address`: **Example** If `user.address` is `Some`, `Option.flatMap` applies the function ` => address.street` to retrieve the `street` value. If `user.address` is `None`, the function is not executed, and `street` remains `None`. This approach lets us handle nested optional values concisely, avoiding manual checks and making the code cleaner and easier to read. ### filter The `Option.filter` function allows you to filter an `Option` based on a given predicate. If the predicate is not met or if the `Option` is `None`, the result will be `None`. **Example** Here's how you can simplify some code using `Option.filter` for a more idiomatic approach: Original Code Refactored Idiomatic Code Using `Option.filter`, we can write the same logic more concisely: ## Getting the Value from an Option To retrieve the value stored inside an `Option`, you can use several helper functions provided by the `Option` module. Here's an overview of the available methods: ### getOrThrow This function extracts the value from a `Some`. If the `Option` is `None`, it throws an error. **Example** ### getOrNull / getOrUndefined These functions convert a `None` to either `null` or `undefined`, which is useful when working with non-`Option`-based code. **Example** ### getOrElse This function allows you to specify a default value to return when the `Option` is `None`. **Example** ## Fallback ### orElse When a computation returns `None`, you might want to try an alternative computation that yields an `Option`. The `Option.orElse` function is helpful in such cases. It lets you chain multiple computations, moving on to the next if the current one results in `None`. This approach is often used in retry logic, attempting computations until one succeeds or all possibilities are exhausted. **Example** ### firstSomeOf You can also use `Option.firstSomeOf` to get the first `Some` value from an iterable of `Option` values: **Example** ## Interop with Nullable Types When dealing with the `Option` data type, you may encounter code that uses `undefined` or `null` to represent optional values. The `Option` module provides several APIs to make interaction with these nullable types straightforward. ### fromNullable `Option.fromNullishOr` converts a nullable value into an `Option`. If the value is `null` or `undefined`, it returns `Option.none`. Otherwise, it wraps the value in `Option.some`. **Example** If you need to convert an `Option` back to a nullable value, there are two helper methods: - `Option.getOrNull`: Converts `None` to `null`. - `Option.getOrUndefined`: Converts `None` to `undefined`. ## Interop with Effect `Option` implements the `Yieldable` trait, so it can be yielded directly inside `Effect.gen`. To pass an `Option` to an Effect combinator such as `Effect.all`, convert it explicitly with `Effect.fromOption`. ### How Option Maps to Effect | Option Variant | Mapped to Effect | Description | | -------------- | ----------------------------------- | ---------------------------------- | | `None` | `Effect` | Represents the absence of a value | | `Some` | `Effect` | Represents the presence of a value | **Example** ## Combining Two or More Options ### zipWith The `Option.zipWith` function lets you combine two `Option` values using a provided function. It creates a new `Option` that holds the combined value of both original `Option` values. **Example** If either of the `Option` values is `None`, the result will be `None`: **Example** ### all To combine multiple `Option` values without transforming their contents, you can use `Option.all`. This function returns an `Option` with a structure matching the input: - If you pass a tuple, the result will be a tuple of the same length. - If you pass a struct, the result will be a struct with the same keys. - If you pass an `Iterable`, the result will be an array. **Example** If any of the `Option` values are `None`, the result will be `None`: **Example** ## gen Similar to , `Option.gen` provides a more readable, generator-based syntax for working with `Option` values, making code that involves `Option` easier to write and understand. This approach is similar to using `async/await` but tailored for `Option`. **Example** When any of the `Option` values in the sequence is `None`, the generator immediately returns the `None` value, skipping further operations: **Example** In this example, `Option.gen` halts execution as soon as it encounters the `None` value, effectively propagating the missing value without performing further operations. The use of `console.log` in these example is for demonstration purposes only. When using `Option.gen`, avoid including side effects in your generator functions, as `Option` should remain a pure data structure. ## Equivalence You can compare `Option` values using the `Option.makeEquivalence` function. This function allows you to specify how to compare the contents of `Option` types by providing an for the type of value they may contain. **Example** Suppose you have optional numbers and want to check if they are equivalent. Here's how you can use `Option.makeEquivalence`: ## Sorting You can sort a collection of `Option` values using the `Option.makeOrder` function. This function helps specify a custom sorting rule for the type of value contained within the `Option`. **Example** Suppose you have a list of optional numbers and want to sort them in ascending order, with empty values `) treated as the lowest: **Example** Consider a more complex case where you have a list of objects containing optional dates, and you want to sort them in descending order, with `Option.none` values at the end: --- title: Redacted description: Securely handle sensitive data with the Redacted module, preventing accidental exposure in logs while supporting safe value access and comparison. sidebar: order: 11 --- The Redacted module provides functionality for handling sensitive information securely within your application. By using the `Redacted` data type, you can ensure that sensitive values are not accidentally exposed in logs or error messages. ## make The `Redacted.make` function creates a `Redacted` instance from a given value `A`, ensuring the content is securely hidden. **Example** Using `Redacted.make` helps prevent sensitive information, such as API keys, from being accidentally exposed in logs or error messages. ## value The `Redacted.value` function retrieves the original value from a `Redacted` instance. Use this function carefully, as it exposes the sensitive data, potentially making it visible in logs or accessible in unintended ways. **Example** ## unsafeWipe The `Redacted.wipeUnsafe` function erases the underlying value of a `Redacted` instance, making it inaccessible. This helps ensure that sensitive data does not remain in memory longer than needed. **Example** ## getEquivalence The `Redacted.makeEquivalence` function generates an for `Redacted` values using an Equivalence for the underlying values of type `A`. This allows you to compare `Redacted` values securely without revealing their content. **Example** --- title: Result description: Represent exclusive values as Success or Failure with the Result data type, enabling precise control flow in computations. sidebar: order: 7 --- import { Aside } from "@astrojs/starlight/components" The `Result` data type represents two exclusive values: a `Result` can be a `Success` value or a `Failure` value, where `A` is the type of the `Success` value, and `E` is the type of the `Failure` value. ## Understanding Result and Exit Result is primarily used as a **simple discriminated union** and is not recommended as the main result type for operations requiring detailed error information. is the preferred **result type** within Effect for capturing comprehensive details about failures. It encapsulates the outcomes of effectful computations, distinguishing between success and various failure modes, such as errors, defects and interruptions. ## Creating Results You can create a `Result` using the `Result.succeed` and `Result.fail` constructors. Use `Result.succeed` to create a `Success` value of type `A`. **Example** Use `Result.fail` to create a `Failure` value of type `E`. **Example** ## Guards Use `Result.isFailure` and `Result.isSuccess` to check whether a `Result` is a `Failure` or `Success` value. **Example** ## Pattern Matching Use `Result.match` to handle both cases of a `Result` by specifying separate callbacks for `Failure` and `Success`. **Example** ## Mapping ### Mapping over the Success Value Use `Result.map` to transform the `Success` value of a `Result`. The function you provide will only apply to the `Success` value, leaving any `Failure` value unchanged. **Example** ### Mapping over the Failure Value Use `Result.mapError` to transform the `Failure` value of a `Result`. The provided function only applies to the `Failure` value, leaving any `Success` value unchanged. **Example** ### Mapping over Both Values Use `Result.mapBoth` to transform both the `Failure` and `Success` values of a `Result`. This function takes two separate transformation functions: one for the `Failure` value and another for the `Success` value. **Example** ## Interop with Effect `Result` implements the `Yieldable` trait, so it can be yielded directly inside `Effect.gen`. To pass a `Result` to an Effect combinator such as `Effect.all`, convert it explicitly with `Effect.fromResult`. ### How Result Maps to Effect | Result Variant | Mapped to Effect | Description | | -------------- | ------------------ | -------------------- | | `Failure` | `Effect` | Represents a failure | | `Success` | `Effect` | Represents a success | **Example** ## Combining Two or More Results ### Combining with flatMap and map Combine two `Result` values with a provided function by chaining `Result.flatMap` and `Result.map`. This creates a new `Result` that holds the combined value of both original `Result` values. **Example** If either of the `Result` values is a `Failure`, the result will be a `Failure`, holding the first encountered `Failure` value: **Example** ### all To combine multiple `Result` values without transforming their contents, you can use `Result.all`. This function returns a `Result` with a structure matching the input: - If you pass a tuple, the result will be a tuple of the same length. - If you pass a struct, the result will be a struct with the same keys. - If you pass an `Iterable`, the result will be an array. **Example** If one or more `Result` values are a `Failure`, the first `Failure` encountered is returned: **Example** ## gen Similar to , `Result.gen` provides a more readable, generator-based syntax for working with `Result` values, making code that involves `Result` easier to write and understand. This approach is similar to using `async/await` but tailored for `Result`. **Example** When any of the `Result` values in the sequence is a `Failure`, the generator immediately returns the `Failure` value, skipping further operations: **Example** In this example, `Result.gen` halts execution as soon as it encounters the `Failure` value, effectively propagating the error without performing further operations. The use of `console.log` in these example is for demonstration purposes only. When using `Result.gen`, avoid including side effects in your generator functions, as `Result` should remain a pure data structure. --- title: Error Accumulation description: Accumulate every typed error or preserve both failures and successes. sidebar: order: 8 --- Combinators such as `Effect.all` and `Effect.forEach` fail fast by default. Validation often needs different behavior: evaluate every input and report all problems together. ## validate `Effect.validate` applies an effectful function to every element. If all elements succeed, it returns all success values. If any fail, it returns every error as a non-empty array and discards the successes. **Example** Every element is evaluated. Use `{ concurrency }` to control parallelism and `{ discard: true }` when only validation, not the success values, is needed. ## partition `Effect.partition` also evaluates every element, but never fails. It returns ``, preserving both sides. **Example** Like `validate`, `partition` accepts a `concurrency` option. --- title: Error Channel Operations description: Transform, filter, inspect, expose, and flip Effect error channels. sidebar: order: 9 --- Error-channel operators change or observe an Effect's failure behavior without requiring an immediate recovery. ## Transforming Channels ### mapError `Effect.mapError` transforms a typed error while leaving the success value unchanged. **Example** ### mapBoth `Effect.mapBoth` transforms the success and error channels in one operation. The eager variants `mapErrorEager` and `mapBothEager` are optimizations for mappings that can be evaluated immediately. ## Filtering the Success Channel `Effect.filterOrFail` keeps a success value when it satisfies a predicate and otherwise creates a typed failure. **Example** A user-defined type guard narrows the success type. Use `Effect.filterOrElse` when a failed predicate should run another Effect instead of producing a value directly. ## Inspecting Failures Tap operators run an observation Effect and preserve the original outcome. If the observation itself fails, that new failure is composed with the original outcome. ### tapError `Effect.tapError` observes every typed error. ### tapErrorTag `Effect.tapErrorTag` observes only one member of a tagged error union without handling it. ### tapCause `Effect.tapCause` observes the complete `Cause`, including typed failures, defects, interruptions, and multiple reasons. ### tapDefect `Effect.tapDefect` observes defects only. It does not run for ordinary typed failures. Use an ordinary `Effect.tap` after `tapError` when both successful values and typed failures need separate observations. ## Moving Failures into the Success Channel `Effect.result` exposes typed failures as `Result.Failure` values and `Effect.exit` exposes the complete outcome, including the full `Cause`. See for `Effect.result` and for `Effect.exit`. When both a typed error and success should become the same success type, recover with `Effect.catch`: ## Flipping the Channels `Effect.flip` swaps the typed error and success channels. `flip` is useful for focused transformations of an error channel, but `mapError` or a catch operator usually communicates the intent more directly. --- title: Expected Errors description: Create, track, expose, and recover from typed errors. sidebar: order: 1 --- Expected errors are represented in the error channel of : Because the error type is explicit, callers can see which failures are possible and decide which ones to recover from. ## Creating Expected Errors `Effect.fail` creates an Effect that fails with `error`. Use `Effect.failSync` when constructing the error should be deferred until the Effect runs. **Example** Errors built with `Data.Error` or `Data.TaggedError` are also yieldable directly inside `Effect.gen`. See . ## Tracking Multiple Error Types When effects with different error types are composed, Effect tracks their union. Sequential composition short-circuits on the first failure. Operations after the failure are not evaluated. ## Exposing Errors as Values Sometimes the caller needs to inspect both outcomes without recovering into another Effect. ### result `Effect.result` moves the typed error into a in the success channel: **Example** `Effect.result` handles typed failures only. Defects and interruptions remain failures of the fiber. ### option `Effect.option` discards the error value, returning `Option.some` on success and `Option.none` on a typed failure. Use `result` when the error value matters and `option` only when every typed failure means absence. ## Catching Every Typed Error `Effect.catch` handles every typed error with a recovery Effect. It does not catch defects or interruptions. **Example** When the handler cannot fail, `Effect.catchEager` is an eager optimization for recovery Effects that can be evaluated immediately. Use when the handler needs the complete failure cause. ## Catching Selected Errors Selective catch operators preserve all unmatched errors in the error channel. ### catchTag `Effect.catchTag` handles one member of a tagged error union and removes that member from the resulting error type. **Example** `catchTag` also accepts a non-empty array of tags when several tagged errors share one handler. ### catchTags `Effect.catchTags` handles several tagged errors with a table of tag-specific handlers. **Example** ### catchIf `Effect.catchIf` selects errors with a predicate or type guard. ### catchFilter `Effect.catchFilter` uses the `Filter` module for reusable or composable selection logic. A narrowing filter also removes the handled subtype from the error channel. **Example** ## Catching Nested Error Reasons Some tagged errors contain another tagged error in a readonly `reason` field. `Effect.catchReason` handles one nested reason while keeping the parent error type for unmatched reasons; `Effect.catchReasons` handles several. **Example** Use `Effect.unwrapReason` when the nested reasons should replace the parent error in the error channel instead of being handled immediately. --- title: Fallback description: Recover from typed failures with fallback effects and values. sidebar: order: 3 --- Fallback operators recover from typed failures. Defects and interruptions remain unchanged. ## catch `Effect.catch` receives the error and returns a fallback Effect. If the source succeeds, the fallback is not evaluated. **Example** Use `Effect.catchTag`, `Effect.catchIf`, or `Effect.catchFilter` when only part of the error channel should trigger the fallback. ## orElseSucceed `Effect.orElseSucceed` replaces any typed failure with a lazily evaluated success value and removes the typed error channel. **Example** This operator handles every typed error. If only absence or one specific error should use the default, narrow the error first with a selective catch operator. ## firstSuccessOf `Effect.firstSuccessOf` runs alternatives sequentially and stops at the first success. If every effect fails, it propagates the last error. **Example** Passing an empty iterable creates a defect with the message `"Received an empty collection of effects"`. --- title: Matching description: Handle success and failure outcomes with pure or effectful handlers. sidebar: order: 4 --- Matching consumes both channels of an Effect and produces one result. Use the ordinary variants for typed failures and the `Cause` variants when defects and interruptions must also be considered. ## match `Effect.match` handles a typed failure or success with pure functions. Defects and interruptions are not handled. **Example** ## matchEffect `Effect.matchEffect` is the effectful version: both handlers return Effects and may introduce new errors or requirements. **Example** ## matchCause and matchCauseEffect `Effect.matchCause` passes the complete `Cause` to `onFailure`, so it also handles defects and interruptions. `Effect.matchCauseEffect` is its effectful counterpart. **Example** ## ignore and ignoreCause `Effect.ignore` discards the success value and recovers from typed errors, producing `Effect`. Defects and interruptions are preserved. `Effect.ignoreCause` also discards every failure cause. Use it sparingly because it can hide defects. --- title: Parallel and Sequential Errors description: Understand how Effect represents multiple failure reasons in Cause. sidebar: order: 10 --- Most Effect combinators fail fast: once an effect fails, later work is not started and concurrent work is interrupted. Some operations can nevertheless produce several failure reasons, for example when concurrent fibers fail together or when both an operation and its finalizer fail. ## Flat Causes `Cause` contains a flat readonly array of `Reason` values: Reasons combined sequentially and reasons combined in parallel use the same `reasons` array representation. **Example** Use `Cause.hasFails`, `Cause.hasDies`, and `Cause.hasInterrupts` when only the kind of failure matters. Use `cause.reasons` or extractors such as `Cause.findError` and `Cause.findDefect` when the individual values are needed. ## Accumulating Domain Errors Multiple typed validation errors are usually better represented as data rather than as multiple `Cause` reasons. Use to collect every typed error, or to preserve both failures and successes. --- title: Retrying description: Retry transient typed failures with limits, conditions, schedules, and fallbacks. sidebar: order: 5 --- Retrying is appropriate for transient failures such as temporary network or service unavailability. It is not a substitute for handling permanent errors, and defects or interruptions are never retried. ## retry `Effect.retry` reruns an Effect after a typed failure. The source Effect is always evaluated once before the retry policy is applied. **Example** `times` is the number of retries after the initial attempt. Therefore `{ times: 5 }` allows at most six executions of the source Effect. ### Retrying Selected Errors The options object can combine: - `while`: retry while the predicate is true; - `until`: stop retrying when the predicate is true; - `times`: limit the number of retries; - `schedule`: control timing and additional stopping conditions. Predicates may return either a boolean or an Effect. **Example** ### Using a Schedule A can define delays, backoff, jitter, and retry limits. `Schedule.recurs`, for example, permits three retries after the initial attempt. Use instead when repetition depends on successful values rather than errors. ## retryOrElse `Effect.retryOrElse` uses a Schedule and runs a fallback Effect when the schedule is exhausted. The fallback receives the final error and the schedule's output. **Example** --- title: Sandboxing description: Expose an Effect's complete Cause in its typed error channel. sidebar: order: 7 --- `Effect.sandbox` exposes the complete failure `Cause` in the typed error channel: Unlike an ordinary typed error, a `Cause` can contain typed failures, defects, interruptions, or several reasons at once. Causes are flat: inspect the readonly `reasons` array and use the reason guards from the `Cause` module. **Example** When a sandboxed effect has not otherwise recovered, convert its error channel back to the original failure model with `Effect.catch`: For one recovery step, `Effect.catchCause` is usually simpler because it provides the same `Cause` directly without changing the error type first. --- title: Timing Out description: Limit how long an Effect may run and customize the timeout outcome. sidebar: order: 6 --- Timeout operators race an Effect against a duration. If the timeout wins, the source Effect is interrupted before the timeout outcome is produced. ## timeout `Effect.timeout` represents a timeout as a typed `Cause.TimeoutError`. **Example** If the source fails before the timeout, its original error is preserved. If it succeeds in time, its success value is returned unchanged. ## timeoutOption `Effect.timeoutOption` represents only the timeout case as `Option.none`. A timely success becomes `Option.some`, while a typed failure from the source remains in the error channel. **Example** Use this operator only when a timeout genuinely means absence. It does not discard ordinary failures from the source Effect. ## timeoutOrElse `Effect.timeoutOrElse` switches to a lazily constructed fallback Effect when the timeout wins. The fallback may introduce its own success, error, and requirement types. **Example** ### Producing a Custom Error Return `Effect.fail` from the fallback when the timeout should use a domain-specific error. Returning `Effect.die` or `Effect.failCause)` from the fallback instead makes the timeout a defect. This should be reserved for situations where timing out violates an invariant. ## Interruption and Uninterruptible Work The timeout fiber interrupts the source when the duration expires. Most Effects respond immediately. An uninterruptible region defers that interruption until it becomes interruptible again, so the caller may wait longer than the configured duration. If work is intentionally allowed to outlive the caller, model that lifecycle explicitly by forking it into an appropriately supervised or detached fiber. Do not use detachment merely to make a timeout return early: detached work continues consuming resources after the caller has moved on. --- title: Two Types of Errors description: Understand the distinction between expected errors and unexpected defects in Effect. sidebar: order: 0 --- Effect distinguishes between errors that are part of a program's domain and unexpected problems that indicate a bug or broken invariant. ## Expected Errors Expected errors, also called **failures**, **typed errors**, or **recoverable errors**, are part of normal program execution. Examples include invalid input, a missing record, or a rejected request. They are tracked in the error channel of `Effect`: The type makes the possible failure visible to callers, who can recover from it with operators such as `Effect.catch` or `Effect.catchTag`. ## Unexpected Errors Unexpected errors, also called **defects**, are not part of the intended control flow. Examples include failed assertions, impossible states, and bugs in third-party code. Defects are not tracked in the `Effect` error channel. They are still retained by the runtime in the effect's `Cause`, together with typed failures and fiber interruptions. Defects usually should not be recovered from inside domain logic. At application boundaries, `Effect.exit`, `Effect.catchDefect`, or `Effect.catchCause` can be used to inspect or report them. --- title: Unexpected Errors description: Create, inspect, report, and selectively recover from defects. sidebar: order: 2 --- Unexpected errors, or **defects**, indicate bugs, violated invariants, or failures outside the program's expected domain. They are retained in the runtime `Cause`, but do not appear in the typed error channel. Defects normally should be reported and allowed to terminate the affected fiber. Recover from them only at boundaries where continuing is explicitly safe. ## Creating a Defect `Effect.die` creates an Effect that terminates with the supplied defect. Its typed error channel is `never`. **Example** Pass a string or, preferably, an `Error` with a useful message to `Effect.die`. Exceptions thrown while evaluating Effect callbacks such as `Effect.sync` are also represented as defects. ## Converting Typed Errors to Defects `Effect.orDie` converts every typed failure into a defect and removes the typed error channel. **Example** To customize the defect, transform the typed error first with `Effect.mapError` and then apply `Effect.orDie`. ## Inspecting the Complete Exit `Effect.exit` moves the complete outcome into the success channel: Unlike `Effect.result`, an `Exit` preserves the complete `Cause`, including defects and interruptions. **Example** This is useful at application boundaries, in tests, and when integrating with APIs that need an explicit value for every outcome. ## catchDefect `Effect.catchDefect` handles defects only. Typed failures and interruptions are left unchanged. **Example** ## catchCause `Effect.catchCause` handles the complete `Cause`, including typed failures, defects, interruptions, and multiple reasons. **Example** Prefer typed recovery operators such as `Effect.catch` and `Effect.catchTag` for domain errors. Use `catchDefect` or `catchCause` only where recovering from unexpected failures is intentional and safe. --- title: Yieldable Errors description: Define custom errors that can be yielded directly inside Effect.gen. sidebar: order: 11 --- Errors created with `Data.Error` and `Data.TaggedError` are yieldable. Inside , yielding one is equivalent to passing it to . ## Data.Error Use `Data.Error` when the error does not need a discriminant tag. **Example** ## Data.TaggedError `Data.TaggedError` adds a readonly `_tag` field. Tagged errors form discriminated unions that can be handled precisely with and . **Example** Use tagged errors for domain errors that callers may need to distinguish. The class is both the error's constructor and its TypeScript type. --- title: Getting Started description: A guide to the Effect documentation and where to start. sidebar: order: 0 --- Welcome to the Effect guides. If you are new to Effect, start with the section, which introduces the core concepts in order. These guides explain each topic in more detail, and you can also use them as a reference while you build. ## How to Use These Docs The documentation is structured in a sequential manner, starting from the basics and progressing to more advanced topics. This allows you to follow along step-by-step as you build your Effect application. However, you have the flexibility to read the documentation in any order or jump directly to the pages that are relevant to your specific use case. You can switch between Effect versions using the version selector at the top of the page. To facilitate navigation within a page, you will find a table of contents on the right side of the screen. This allows you to easily jump between different sections of the page. ## Guide Sections The guides are grouped by area: - The guides cover typed errors, fallbacks, and retries. - The guides cover fibers, bounded parallelism, and racing. - The guides explain how to acquire and release resources safely. - The guides cover parsing, validating, and transforming data. - The guides cover logging, metrics, and tracing. - The guides show how to build data pipelines with backpressure. For a complete reference of every module, see the . ## Coding with LLMs The following article describe how to use Effect with LLMs: When using LLMs it is also very important to optimize the feedback loop to be as tight as possible, that can include writing custom linting rules to fit your style preferences and patterns, an example of a repository optimized for agentic coding can be found at: A key part of optimizing the feedback loop is to use the Effect LSP plugin, we advise using the latest "tsgo" implementation of it that can be found at: ## Join our Community If you have questions about anything related to Effect, you're always welcome to ask our community on . --- title: Building Pipelines description: Learn to create modular, readable pipelines for composing and sequencing operations in Effect, enabling clear and efficient data transformations. sidebar: order: 10 --- import { Aside } from "@astrojs/starlight/components" Effect pipelines allow for the composition and sequencing of operations on values, enabling the transformation and manipulation of data in a concise and modular manner. ## Why Pipelines are Good for Structuring Your Application Pipelines are an excellent way to structure your application and handle data transformations in a concise and modular manner. They offer several benefits: 1. **Readability**: Pipelines allow you to compose functions in a readable and sequential manner. You can clearly see the flow of data and the operations applied to it, making it easier to understand and maintain the code. 2. **Code Organization**: With pipelines, you can break down complex operations into smaller, manageable functions. Each function performs a specific task, making your code more modular and easier to reason about. 3. **Reusability**: Pipelines promote the reuse of functions. By breaking down operations into smaller functions, you can reuse them in different pipelines or contexts, improving code reuse and reducing duplication. 4. **Type Safety**: By leveraging the type system, pipelines help catch errors at compile-time. Functions in a pipeline have well-defined input and output types, ensuring that the data flows correctly through the pipeline and minimizing runtime errors. ## Functions vs Methods The use of functions in the Effect ecosystem libraries is important for achieving **tree shakeability** and ensuring **extensibility**. Functions enable efficient bundling by eliminating unused code, and they provide a flexible and modular approach to extending the libraries' functionality. ### Tree Shakeability Tree shakeability refers to the ability of a build system to eliminate unused code during the bundling process. Functions are tree shakeable, while methods are not. When functions are used in the Effect ecosystem, only the functions that are actually imported and used in your application will be included in the final bundled code. Unused functions are automatically removed, resulting in a smaller bundle size and improved performance. On the other hand, methods are attached to objects or prototypes, and they cannot be easily tree shaken. Even if you only use a subset of methods, all methods associated with an object or prototype will be included in the bundle, leading to unnecessary code bloat. ### Extensibility Another important advantage of using functions in the Effect ecosystem is the ease of extensibility. With methods, extending the functionality of an existing API often requires modifying the prototype of the object, which can be complex and error-prone. In contrast, with functions, extending the functionality is much simpler. You can define your own "extension methods" as plain old functions without the need to modify the prototypes of objects. This promotes cleaner and more modular code, and it also allows for better compatibility with other libraries and modules. ## pipe The `pipe` function is a utility that allows us to compose functions in a readable and sequential manner. It takes the output of one function and passes it as the input to the next function in the pipeline. This enables us to build complex transformations by chaining multiple functions together. **Syntax** In this syntax, `input` is the initial value, and `func1`, `func2`, ..., `funcN` are the functions to be applied in sequence. The result of each function becomes the input for the next function, and the final result is returned. Here's an illustration of how `pipe` works: It's important to note that functions passed to `pipe` must have a **single argument** because they are only called with a single argument. Let's see an example to better understand how `pipe` works: **Example** In the above example, we start with an input value of `5`. The `increment` function adds `1` to the initial value, resulting in `6`. Then, the `double` function doubles the value, giving us `12`. Finally, the `subtractTen` function subtracts `10` from `12`, resulting in the final output of `2`. The result is equivalent to `subtractTen))`, but using `pipe` makes the code more readable because the operations are sequenced from left to right, rather than nesting them inside out. ## map Transforms the value inside an effect by applying a function to it. **Syntax** `Effect.map` takes a function and applies it to the value contained within an effect, creating a new effect with the transformed value. **Example** Here's a practical example where we apply a service charge to a transaction amount: ## as Replaces the value inside an effect with a constant value. `Effect.as` allows you to ignore the original value inside an effect and replace it with a new constant value. **Example** ## flatMap Chains effects to produce new `Effect` instances, useful for combining operations that depend on previous results. **Syntax** In the code above, `transformation` is the function that takes a value and returns an `Effect`, and `myEffect` is the initial `Effect` being transformed. Use `Effect.flatMap` when you need to chain multiple effects, ensuring that each step produces a new `Effect` while flattening any nested effects that may occur. It is similar to `flatMap` used with arrays but works specifically with `Effect` instances, allowing you to avoid deeply nested effect structures. **Example** ### Ensure All Effects Are Considered Make sure that all effects within `Effect.flatMap` contribute to the final computation. If you ignore an effect, it can lead to unexpected behavior: In this case, the `Effect.sync` call is ignored and does not affect the result of `applyDiscount`. To handle effects correctly, make sure to explicitly chain them using functions like `Effect.map`, `Effect.flatMap`, `Effect.andThen`, or `Effect.tap`. ## andThen Chains two actions, where the second action can depend on the result of the first. **Syntax** Use `andThen` when you need to run multiple actions in sequence, with the second action depending on the result of the first. This is useful for combining effects or handling computations that must happen in order. The second action can be: 1. An `Effect` 2. A function returning an `Effect` If you just need to transform the result into a plain value , use `Effect.map` instead. **Example** Let's look at an example comparing `Effect.andThen` with `Effect.map` and `Effect.flatMap`: ### Option and Result with andThen Both and are commonly used for handling optional or missing values or simple error cases. These types integrate well with `Effect.andThen`. When used with `Effect.andThen`, the operations are categorized as scenario 2 above , because both `Option` and `Result` implement the `Yieldable` trait and can be converted to an `Effect` with `Effect.fromOption`/`Effect.fromResult`. **Example** You might expect the type of `program` to be `Effect, UnknownError, never>`, but it is actually `Effect`. This is because `Option` is treated as an effect of type `Effect`, and as a result, the possible errors are combined into a union type. **Example** Although one might expect the type of `program` to be `Effect, UnknownError, never>`, it is actually `Effect`. This is because `Result` is treated as an effect of type `Effect`, meaning the errors are combined into a union type. ## tap Runs a side effect with the result of an effect without changing the original value. Use `Effect.tap` when you want to perform a side effect, like logging or tracking, without modifying the main value. This is useful when you need to observe or record an action but want the original value to be passed to the next step. `Effect.tap` works similarly to `Effect.flatMap`, but it ignores the result of the function passed to it. The value from the previous effect remains available for the next part of the chain. Note that if the side effect fails, the entire chain will fail too. **Example** In this example, `Effect.tap` is used to log the transaction amount before applying the discount, without modifying the value itself. The original value remains available for the next operation . Using `Effect.tap` allows us to execute side effects during the computation without altering the result. This can be useful for logging, performing additional actions, or observing the intermediate values without interfering with the main computation flow. ## all Combines multiple effects into one, returning results based on the input structure. Use `Effect.all` when you need to run multiple effects and combine their results into a single output. It supports tuples, iterables, structs, and records, making it flexible for different input types. For instance, if the input is a tuple: the effects are executed in order, and the result is a new effect containing the results as a tuple. The results in the tuple match the order of the effects passed to `Effect.all`. By default, `Effect.all` runs effects sequentially and produces a tuple or object with the results. If any effect fails, it stops execution and propagates the error. See for more information on how to use `Effect.all`. **Example** ## Build your first pipeline Let's now combine the `pipe` function, `Effect.all`, and `Effect.andThen` to create a pipeline that performs a sequence of transformations. **Example** This pipeline demonstrates how you can structure your code by combining different effects into a clear, readable flow. ## The pipe method Effect provides a `pipe` method that works similarly to the `pipe` method found in . This method allows you to chain multiple operations together, making your code more concise and readable. **Syntax** This is equivalent to using the `pipe` **function** like this: The `pipe` method is available on all effects and many other data types, eliminating the need to import the `pipe` function and saving you some keystrokes. **Example** Let's rewrite an , this time using the `pipe` method. ## Cheatsheet Let's summarize the transformation functions we have seen so far: | API | Input | Output | | --------- | ------------------------------------------------------------ | --------------------------- | | `map` | `Effect`, `A => B` | `Effect` | | `flatMap` | `Effect`, `A => Effect` | `Effect` | | `andThen` | `Effect`, `Effect \| A => Effect` | `Effect` | | `tap` | `Effect`, `A => Effect` | `Effect` | | `all` | `` | `Effect<, E, R>` | --- title: Creating Effects description: Learn to create and manage effects for structured handling of success, failure, and side effects in synchronous and asynchronous workflows. sidebar: order: 7 --- import { Aside } from "@astrojs/starlight/components" Effect provides different ways to create effects, which are units of computation that encapsulate side effects. In this guide, we will cover some of the common methods that you can use to create effects. ## Why Not Throw Errors? In traditional programming, when an error occurs, it is often handled by throwing an exception: However, throwing errors can be problematic. The type signatures of functions do not indicate that they can throw exceptions, making it difficult to reason about potential errors. To address this issue, Effect introduces dedicated constructors for creating effects that represent both success and failure: `Effect.succeed` and `Effect.fail`. These constructors allow you to explicitly handle success and failure cases while **leveraging the type system to track errors**. ### succeed Creates an `Effect` that always succeeds with a given value. Use this function when you need an effect that completes successfully with a specific value without any errors or external dependencies. **Example** The type of `success` is `Effect`, which means: - It produces a value of type `number`. - It does not generate any errors . - It requires no additional data or dependencies . ### fail Creates an `Effect` that represents an error that can be recovered from. Use this function to explicitly signal an error in an `Effect`. The error will keep propagating unless it is handled. You can handle the error with functions like or . **Example** The type of `failure` is `Effect`, which means: - It never produces a value . - It fails with an error, specifically an `Error`. - It requires no additional data or dependencies . Although you can use `Error` objects with `Effect.fail`, you can also pass strings, numbers, or more complex objects depending on your error management strategy. Using "tagged" errors can help identify error types and works well with standard Effect functions, like . **Example** ## Error Tracking With `Effect.succeed` and `Effect.fail`, you can explicitly handle success and failure cases and the type system will ensure that errors are tracked and accounted for. **Example** Here's how you can rewrite the function using Effect, making error handling explicit. In this example, the `divide` function indicates in its return type `Effect` that the operation can either succeed with a `number` or fail with an `Error`. This clear type signature helps ensure that errors are handled properly and that anyone calling the function is aware of the possible outcomes. **Example** Let's imagine another scenario where we use `Effect.succeed` and `Effect.fail` to model a simple user retrieval operation where the user data is hardcoded, which could be useful in testing scenarios or when mocking data: In this example, `exampleUserEffect`, which has the type `Effect`, will either produce a `User` object or an `Error`, depending on whether the user exists in the mocked database. For a deeper dive into managing errors in your applications, refer to the . ## Modeling Synchronous Effects In JavaScript, you can delay the execution of synchronous computations using "thunks". Thunks are useful for delaying the computation of a value until it is needed. To model synchronous side effects, Effect provides the `Effect.sync` and `Effect.try` constructors, which accept a thunk. ### sync Creates an `Effect` that represents a synchronous side-effectful computation. Use `Effect.sync` when you are sure the operation will not fail. The provided function must not throw errors; if it does, the error will be treated as a . This defect is not a standard error but indicates a flaw in the logic that was expected to be error-free. You can think of it similar to an unexpected crash in the program, which can be further managed or logged using tools like . This feature ensures that even unexpected failures in your application are not lost and can be handled appropriately. **Example** In the example below, `Effect.sync` is used to defer the side-effect of writing to the console. The side effect encapsulated within `program` won't occur until the effect is explicitly run section for more details). This allows you to define side effects at one point in your code and control when they are activated, improving manageability and predictability of side effects in larger applications. ### try Creates an `Effect` that represents a synchronous computation that might fail. In situations where you need to perform synchronous operations that might fail, such as parsing JSON, you can use the `Effect.try` constructor. This constructor is designed to handle operations that could throw exceptions by capturing those exceptions and transforming them into manageable errors. **Example** Suppose you have a function that attempts to parse a JSON string. This operation can fail and throw an error if the input string is not properly formatted as JSON: In this example: - `parse` is a function that creates an effect encapsulating the JSON parsing operation. - If `JSON.parse` throws an error due to invalid input, `Effect.try` catches this error and the effect represented by `program` will fail with an `UnknownError`. This ensures that errors are not silently ignored but are instead handled within the structured flow of effects. #### Customizing Error Handling You might want to transform the caught exception into a more specific error or perform additional operations when catching an error. `Effect.try` supports an overload that allows you to specify how caught exceptions should be transformed: **Example** You can think of this as a similar pattern to the traditional try-catch block in JavaScript: ## Modeling Asynchronous Effects In traditional programming, we often use `Promise`s to handle asynchronous computations. However, dealing with errors in promises can be problematic. By default, `Promise` only provides the type `Value` for the resolved value, which means errors are not reflected in the type system. This limits the expressiveness and makes it challenging to handle and track errors effectively. To overcome these limitations, Effect introduces dedicated constructors for creating effects that represent both success and failure in an asynchronous context: `Effect.promise` and `Effect.tryPromise`. These constructors allow you to explicitly handle success and failure cases while **leveraging the type system to track errors**. ### promise Creates an `Effect` that represents an asynchronous computation guaranteed to succeed. Use `Effect.promise` when you are sure the operation will not reject. The provided function returns a `Promise` that should never reject; if it does, the error will be treated as a . This defect is not a standard error but indicates a flaw in the logic that was expected to be error-free. You can think of it similar to an unexpected crash in the program, which can be further managed or logged using tools like . This feature ensures that even unexpected failures in your application are not lost and can be handled appropriately. **Example** The `program` value has the type `Effect` and can be interpreted as an effect that: - succeeds with a value of type `string` - does not produce any expected error - does not require any context ### tryPromise Creates an `Effect` that represents an asynchronous computation that might fail. Unlike `Effect.promise`, this constructor is suitable when the underlying `Promise` might reject. It provides a way to catch errors and handle them appropriately. By default if an error occurs, it will be caught and propagated to the error channel as an `UnknownError`. **Example** The `program` value has the type `Effect` and can be interpreted as an effect that: - succeeds with a value of type `Response` - might produce an error - does not require any context #### Customizing Error Handling If you want more control over what gets propagated to the error channel, you can use an overload of `Effect.tryPromise` that takes a remapping function: **Example** ## From a Callback Creates an `Effect` from a callback-based asynchronous function. Sometimes you have to work with APIs that don't support `async/await` or `Promise` and instead use the callback style. To handle callback-based APIs, Effect provides the `Effect.callback` constructor. **Example** Let's wrap the `readFile` function from Node.js's `fs` module into an Effect-based API : In the above example, we manually annotate the types when calling `Effect.callback`: because TypeScript cannot infer the type parameters for a callback based on the return value inside the callback body. Annotating the types ensures that the values provided to `resume` match the expected types. The `resume` function inside `Effect.callback` should be called exactly once. Calling it more than once will result in the extra calls being ignored. **Example** ### Advanced Usage For more advanced use cases, the callback passed to Effect.callback may return an Effect that will be executed if the fiber running this effect is interrupted. This can be used to perform cleanup when the operation is cancelled. **Example** In this example: - The `writeFileWithCleanup` function writes data to a file. - If the fiber running this effect is interrupted, the cleanup effect is executed. - This ensures that resources like open file handles are cleaned up properly when the operation is canceled. If the operation you're wrapping supports interruption, the `resume` function can receive an `AbortSignal` to handle interruption requests directly. **Example** ## Suspended Effects `Effect.suspend` is used to delay the creation of an effect. It allows you to defer the evaluation of an effect until it is actually needed. The `Effect.suspend` function takes a thunk that represents the effect, and it wraps it in a suspended effect. **Syntax** Let's explore some common scenarios where `Effect.suspend` proves useful. ### Lazy Evaluation When you want to defer the evaluation of an effect until it is required. This can be useful for optimizing the execution of effects, especially when they are not always needed or when their computation is expensive. Also, when effects with side effects or scoped captures are created, use `Effect.suspend` to re-execute on each invocation. **Example** In this example, `bad` is the result of calling `Effect.succeed` a single time, which increments the scoped variable but . `Effect.runSync` does not result in any new computation, because `Effect.succeed` has already been called. On the other hand, each time `Effect.runSync` is called, the thunk passed to `Effect.suspend` will be executed, outputting the scoped variable's most recent value. ### Handling Circular Dependencies `Effect.suspend` is helpful in managing circular dependencies between effects, where one effect depends on another, and vice versa. For example it's fairly common for `Effect.suspend` to be used in recursive functions to escape an eager call. **Example** The `blowsUp` function creates a recursive Fibonacci sequence without deferring execution. Each call to `blowsUp` triggers further immediate recursive calls, rapidly increasing the JavaScript call stack size. Conversely, `allGood` avoids stack overflow by using `Effect.suspend` to defer the recursive calls. This mechanism doesn't immediately execute the recursive effects but schedules them to be run later, thus keeping the call stack shallow and preventing a crash. ### Unifying Return Type In situations where TypeScript struggles to unify the returned effect type, `Effect.suspend` can be employed to resolve this issue. **Example** ## Cheatsheet The table provides a summary of the available constructors, along with their input and output types, allowing you to choose the appropriate function based on your needs. | API | Given | Result | | ----------------------- | ---------------------------------- | ------------------------- | | `succeed` | `A` | `Effect` | | `fail` | `E` | `Effect` | | `sync` | ` => A` | `Effect` | | `try` | ` => A` | `Effect` | | `try` | ` => A`, `unknown => E` | `Effect` | | `promise` | ` => Promise` | `Effect` | | `tryPromise` | ` => Promise` | `Effect` | | `tryPromise` | ` => Promise`, `unknown => E` | `Effect` | | `callback` | ` => void` | `Effect` | | `suspend` | ` => Effect` | `Effect` | For the complete list of constructors, visit the . --- title: Devtools description: Enhance your Effect development experience with the Effect Language Service and official VS Code/Cursor extension for advanced diagnostics, refactors, and intelligent code completion. sidebar: order: 3 --- import { Aside, Steps, Tabs, TabItem } from "@astrojs/starlight/components" Effect provides powerful development tools to enhance your coding experience and help you write safer, more maintainable code. These tools integrate directly into your editor, providing real-time feedback, intelligent refactors, and helpful diagnostics. ## Effect LSP The Effect Language Service extends your editor with Effect-specific features. It analyzes your Effect code and provides intelligent assistance through diagnostics, quick info, completions, and automated refactors. The language service ships as `@effect/tsgo`: a build of with the Effect diagnostics layer built in. It is a **superset** of plain `tsgo`; use `@effect/tsgo` as your sole TypeScript language server rather than running both side by side, since that produces duplicate diagnostics. It works in editors that support the standard TypeScript LSP, such as Code, Cursor, Zed, NVim, etc. ### Installation The quickest way to set up `@effect/tsgo` in your project is the interactive CLI: This guides you through adding the dependency, configuring your `tsconfig.json`, adjusting plugin options, and any editor configuration needed to activate the LSP. To set it up manually instead: 1. Install the package as a development dependency: For monorepos, we suggest installing it at the root level. For single-package projects, install it in the package directory. `@effect/tsgo` also requires a native TypeScript 7 installation: `typescript` version 7 or newer, such as `typescript@latest` or `typescript@next`, or an alias such as `@typescript/native`. 2. Add the plugin to your `tsconfig.json` : 3. Ensure your editor uses the workspace TypeScript version: This step is critical for the language service to function properly. The plugin must run on the TypeScript version installed in your project, not the one bundled with your editor. 4. You're ready to play! Writing the following code in a file.ts inside your project, should result in an error diagnostic appearing, saying that Effect's must be yielded or assigned to a variable: ### Features The Effect Language Service provides a comprehensive set of features to enhance your development workflow: #### Intelligent Quick Info Hover over Effect values to see extended type information and detailed insights: - **Effect Types**: See comprehensive type information for Effect values - **Generator Parameters**: When hovering over `yield*` in `Effect.gen`, view detailed information about the yielded value - **Layer Composition**: Visualize layer dependencies with interactive graphs showing how layers compose together - **Service Dependencies**: Understand service requirements and their relationships at a glance #### Real-time Diagnostics Catch common mistakes and potential issues as you write code: - **Floating Effects**: Detect Effect values that aren't assigned or yielded, preventing silent bugs - **Layer Issues**: Catch layer requirement leaks and scope violations before runtime - **Unnecessary Code**: Identify redundant `Effect.gen` or `pipe` calls - **Error Handling**: Detect misuse of catch functions on Effects that cannot fail - **Version Conflicts**: Detect when multiple Effect versions are present in your project #### Smart Completions Speed up your coding with context-aware suggestions: - **Generator Boilerplate**: Quickly scaffold `Effect.gen` functions - **Scaffolds**: For `Context.Service`, `Data.TaggedError` and friends. - **Self Parameters**: Auto-complete for `Self` parameters in service declarations #### Powerful Refactors Transform your code with intelligent automated refactors: - **Async to Effect**: Convert async functions to Effect using `gen` or `fn` syntax - **Error Generation**: Generate tagged errors from promise-based code - **Service Accessors**: Automatically implement service accessor functions - **Pipe Conversion**: Transform function calls to pipe syntax - **Pipe Styles**: Toggle between different pipe style formats - **Layer Magic**: Automatically compose layers with correct dependencies ### Configuration The Effect LSP provides also lots of configuration options such as changing severity or disabling diagnostic messages. To see the full list of options and features, please visit the . ### Build-Time Diagnostics While LSPs only activate during editing sessions, you may want to catch diagnostics during your build process. Usually that's done through linting rules, but since almost all of the Effect diagnostics relies on types, that would mean enabling type-aware linting, which means performing type checking again on the project files. To solve this, `@effect/tsgo` allows you to patch your local TypeScript installation, so diagnostics are emitted while performing type checking. To enable it run the following command to modify your local TypeScript installation: To make this automatic for all developers, add it to your `package.json`: This ensures the language service runs during compilation with the standard `tsc` command. ## VS Code / Cursor Extension The editor extension provides utilities in helping you debug your Effect applications. At the moment only Code and Code forks like Cursor are supported. ### Installation The extension can be installed by searching directly in your editor extension page or from the or the . ### Debugger Features With the Effect Extension, you'll find couple of new sections inside the Debug section of your editor that, once you pause execution, will . - **Context**: Allows you to inspect the context of the currently paused Effect Fiber. - **Span Stack**: Shows you the stack of telemetry spans that lead you into the execution of the currently paused Effect. - **Fibers**: List all the Effect Fibers running in your application, allows you to inspect informations such as interrupt-ability and allows to request interruption of them. - **Breakpoints**: Enables "pause on defect", letting your debugger pause when an Effect fiber fails with a defect. --- title: Importing Effect description: Get started with Effect by installing the package and importing essential modules and functions for building type-safe, modular applications. sidebar: order: 5 --- import { Aside, Tabs, TabItem } from "@astrojs/starlight/components" If you're just getting started, you might feel overwhelmed by the variety of modules and functions that Effect offers. However, rest assured that you don't need to worry about all of them right away. This page will provide a simple introduction on how to import modules and functions, and explain that installing the `effect` package is generally all you need to begin. ## Installing Effect If you haven't already installed the `effect` package, you can do so by running the following command in your terminal: By installing this package, you get access to the core functionality of Effect. For detailed installation instructions for platforms like Deno or Bun, refer to the guide, which provides step-by-step guidance. ## Importing Modules and Functions Once you have installed the `effect` package, you can start using its modules and functions in your projects. Importing modules and functions is straightforward and follows the standard JavaScript/TypeScript import syntax. To import a module or a function from the `effect` package, simply use the `import` statement at the top of your file. Here's how you can import the `Effect` module: Now, you have access to the Effect module, which is the heart of the Effect library. It provides various functions to create, compose, and manipulate effectful computations. ## Namespace imports In addition to importing the `Effect` module with a named import, as shown previously: You can also import it using a namespace import like this: Both forms of import allow you to access the functionalities provided by the `Effect` module. However an important consideration is **tree shaking**, which refers to a process that eliminates unused code during the bundling of your application. Named imports may generate tree shaking issues when a bundler doesn't support deep scope analysis. Here are some bundlers that support deep scope analysis and thus don't have issues with named imports: - Rollup - Webpack 5+ ## Functions vs Methods In the Effect ecosystem, libraries often expose functions rather than methods. This design choice is important for two key reasons: tree shakeability and extendibility. ### Tree Shakeability Tree shakeability refers to the ability of a build system to eliminate unused code during the bundling process. Functions are tree shakeable, while methods are not. When functions are used in the Effect ecosystem, only the functions that are actually imported and used in your application will be included in the final bundled code. Unused functions are automatically removed, resulting in a smaller bundle size and improved performance. On the other hand, methods are attached to objects or prototypes, and they cannot be easily tree shaken. Even if you only use a subset of methods, all methods associated with an object or prototype will be included in the bundle, leading to unnecessary code bloat. ### Extendibility Another important advantage of using functions in the Effect ecosystem is the ease of extendibility. With methods, extending the functionality of an existing API often requires modifying the prototype of the object, which can be complex and error-prone. In contrast, with functions, extending the functionality is much simpler. You can define your own "extension methods" as plain old functions without the need to modify the prototypes of objects. This promotes cleaner and more modular code, and it also allows for better compatibility with other libraries and modules. ## Commonly Used Functions As you start your adventure with Effect, you don't need to dive into every function in the `effect` package right away. Instead, focus on some commonly used functions that will provide a solid foundation for your journey into the world of Effect. In the upcoming guides, we will explore some of these essential functions, specifically those for creating and running `Effect`s and building pipelines. But before we dive into those, let's start from the very heart of Effect: understanding the `Effect` type. This will lay the groundwork for your understanding of how Effect brings composability, type safety, and error handling into your applications. So, let's take the first step and explore the fundamental concepts of the . --- title: Installation description: Set up a new Effect project across different platforms like Node.js, Deno, Bun, and Vite + React with step-by-step installation guides. sidebar: order: 2 --- import { Aside, Steps, Tabs, TabItem } from "@astrojs/starlight/components" Requirements: - TypeScript 5.9 or newer. TypeScript 7 is recommended for the best performance and compatibility with . - Node.js, Deno, and Bun are supported. ## Manual Installation ### Node.js Follow these steps to create a new Effect project for : 1. Create a project directory and navigate into it: 2. Initialize a TypeScript project: This creates a `package.json` file with an initial setup for your TypeScript project. 3. Initialize TypeScript: When running this command, it will generate a `tsconfig.json` file that contains configuration options for TypeScript. One of the most important options to consider is the `strict` flag. Make sure to open the `tsconfig.json` file and verify that the value of the `strict` option is set to `true`. 4. Install the necessary package as dependency: This package will provide the foundational functionality for your Effect project. Let's write and run a simple program to ensure that everything is set up correctly. In your terminal, execute the following commands: Open the `index.ts` file and add the following code: Run the `index.ts` file. Here we are using to run the `index.ts` file in the terminal: You should see the message `"Hello, World!"` printed. This confirms that the program is working correctly. ### Deno Follow these steps to create a new Effect project for : 1. Create a project directory and navigate into it: 2. Initialize Deno: 3. Install the necessary package as dependency: This package will provide the foundational functionality for your Effect project. Let's write and run a simple program to ensure that everything is set up correctly. Open the `main.ts` file and replace the content with the following code: Run the `main.ts` file: You should see the message `"Hello, World!"` printed. This confirms that the program is working correctly. ### Bun Follow these steps to create a new Effect project for : 1. Create a project directory and navigate into it: 2. Initialize Bun: When running this command, it will generate a `tsconfig.json` file that contains configuration options for TypeScript. One of the most important options to consider is the `strict` flag. Make sure to open the `tsconfig.json` file and verify that the value of the `strict` option is set to `true`. 3. Install the necessary package as dependency: This package will provide the foundational functionality for your Effect project. Let's write and run a simple program to ensure that everything is set up correctly. Open the `index.ts` file and replace the content with the following code: Run the `index.ts` file: You should see the message `"Hello, World!"` printed. This confirms that the program is working correctly. ### Vite + React Follow these steps to create a new Effect project for + : 1. Scaffold your Vite project, open your terminal and run the following command: This command will create a new Vite project with React and TypeScript template. 2. Navigate into the newly created project directory and install the required packages: Once the packages are installed, open the `tsconfig.json` file and ensure that the value of the `strict` option is set to true. 3. Install the necessary package as dependency: This package will provide the foundational functionality for your Effect project. Now, let's write and run a simple program to ensure that everything is set up correctly. Open the `src/App.tsx` file and replace its content with the following code: After making these changes, start the development server by running the following command: Then, press **o** to open the application in your browser. When you click the button, you should see the counter increment. This confirms that the program is working correctly. --- title: Running Effects description: Learn how to execute effects in Effect with various functions for synchronous and asynchronous execution, including handling results and managing error outcomes. sidebar: order: 8 --- import { Aside } from "@astrojs/starlight/components" To execute an effect, you can use one of the many `run` functions provided by the `Effect` module. ## runSync Executes an effect synchronously, running it immediately and returning the result. **Example** Use `Effect.runSync` to run an effect that does not fail and does not include any asynchronous operations. If the effect fails or involves asynchronous work, it will throw an error, and execution will stop where the failure or async operation occurs. **Example** ## runSyncExit Runs an effect synchronously and returns the result as an type, which represents the outcome of the effect. Use `Effect.runSyncExit` to find out whether an effect succeeded or failed, including any defects, without dealing with asynchronous operations. The `Exit` type represents the result of the effect: - If the effect succeeds, the result is wrapped in a `Success`. - If it fails, the failure information is provided as a `Failure` containing a type. **Example** If the effect contains asynchronous operations, `Effect.runSyncExit` will return an `Failure` with a `Die` cause, indicating that the effect cannot be resolved synchronously. **Example** ## runPromise Executes an effect and returns the result as a `Promise`. Use `Effect.runPromise` when you need to execute an effect and work with the result using `Promise` syntax, typically for compatibility with other promise-based code. **Example** If the effect succeeds, the promise will resolve with the result. If the effect fails, the promise will reject with an error. **Example** ## runPromiseExit Runs an effect and returns a `Promise` that resolves to an , which represents the outcome of the effect. Use `Effect.runPromiseExit` when you need to determine if an effect succeeded or failed, including any defects, and you want to work with a `Promise`. The `Exit` type represents the result of the effect: - If the effect succeeds, the result is wrapped in a `Success`. - If it fails, the failure information is provided as a `Failure` containing a type. **Example** ## runFork The foundational function for running effects, returning a "fiber" that can be observed or interrupted. `Effect.runFork` is used to run an effect in the background by creating a fiber. It is the base function for all other run functions. It starts a fiber that can be observed or interrupted. **Example** In this example, the `program` continuously logs "running..." with each repetition spaced 200 milliseconds apart. You can learn more about repetitions and scheduling in our guide. To stop the execution of the program, we use `Fiber.interrupt` on the fiber returned by `Effect.runFork`. This allows you to control the execution flow and terminate it when necessary. For a deeper understanding of how fibers work and how to handle interruptions, check out our guides on and . ## Synchronous vs. Asynchronous Effects There is no built-in way to determine in advance whether an effect will execute synchronously or asynchronously. Tracking this distinction would introduce several problems: 1. **Complexity:** Introducing this feature to track sync/async behavior in the type system would make Effect more complex to use and limit its composability. 2. **Safety Concerns:** Tracking asynchronous effects would not significantly improve safety. A callback-based API can invoke its callback immediately or defer it, and the type system cannot distinguish those behaviors reliably. ### Best Practices for Running Effects In most cases, effects are run at the outermost parts of your application. Typically, an application built around Effect will involve a single call to the main effect. Here’s how you should approach effect execution: - Use `runPromise` or `runFork`: For most cases, asynchronous execution should be the default. These methods provide the best way to handle Effect-based workflows. - Use `runSync` only when necessary: Synchronous execution should be considered an edge case, used only in scenarios where asynchronous execution is not feasible. For example, when you are sure the effect is purely synchronous and need immediate results. ## Cheatsheet The table provides a summary of the available `run*` functions, along with their input and output types, allowing you to choose the appropriate function based on your needs. | API | Given | Result | | ---------------- | -------------- | --------------------- | | `runSync` | `Effect` | `A` | | `runSyncExit` | `Effect` | `Exit` | | `runPromise` | `Effect` | `Promise` | | `runPromiseExit` | `Effect` | `Promise>` | | `runFork` | `Effect` | `RuntimeFiber` | You can find the complete list of `run*` functions . --- title: The Effect Type description: Understand the Effect type in the Effect ecosystem, which models immutable, lazy workflows with type-safe success, error, and requirement handling for effectful computations. sidebar: order: 6 --- import { Aside } from "@astrojs/starlight/components" The `Effect` type is a description of a workflow or operation that is **lazily** executed. This means that when you create an `Effect`, it doesn't run immediately, but instead defines a program that can succeed, fail, or require some additional context to complete. Here is the general form of an `Effect`: This type indicates that an effect: - Succeeds and returns a value of type `Success` - Fails with an error of type `Error` - May need certain contextual dependencies of type `Requirements` to execute Conceptually, you can think of `Effect` as an effectful version of the following function type: However, effects are not actually functions. They can model synchronous, asynchronous, concurrent, and resourceful computations. **Immutability**. `Effect` values are immutable, and every function in the Effect library produces a new `Effect` value. **Modeling Interactions**. These values do not perform any actions themselves, they simply model or describe effectful interactions. **Execution**. An `Effect` can be executed by the , which interprets it into actual interactions with the external world. Ideally, this execution happens at a single entry point in your application, such as the main function where effectful operations are initiated. ## Type Parameters The `Effect` type has three type parameters with the following meanings: | Parameter | Description | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Success** | Represents the type of value that an effect can succeed with when executed. If this type parameter is `void`, it means the effect produces no useful information, while if it is `never`, it means the effect runs forever . | | **Error** | Represents the expected errors that can occur when executing an effect. If this type parameter is `never`, it means the effect cannot fail, because there are no values of type `never`. | | **Requirements** | Represents the contextual data required by the effect to be executed. This data is stored in a collection named `Context`. If this type parameter is `never`, it means the effect has no requirements and the `Context` collection is empty. | ## Extracting Inferred Types By using the utility types `Effect.Success`, `Effect.Error`, and `Effect.Services`, you can extract the corresponding types from an effect. **Example** --- title: Using Generators description: Learn how to use generators in Effect for writing effectful code, enhancing control flow, handling errors, and simplifying asynchronous operations with a syntax similar to async/await. sidebar: order: 9 --- import { Aside, Tabs, TabItem, Badge } from "@astrojs/starlight/components" Effect offers a convenient syntax, similar to `async`/`await`, to write effectful code using . ## Understanding Effect.gen The `Effect.gen` utility simplifies the task of writing effectful code by utilizing JavaScript's generator functions. This method helps your code appear and behave more like traditional synchronous code, which enhances both readability and error management. **Example** Let's explore a practical program that performs a series of data transformations commonly found in application logic: Key steps to follow when using `Effect.gen`: - Wrap your logic in `Effect.gen` - Use `yield*` to handle effects - Return the final result If any of the effects that you handle inside of the generator with `yield*` fail, then the generator will stop and exit with that failure. ## Comparing Effect.gen with async/await If you are familiar with `async`/`await`, you may notice that the flow of writing code is similar. Let's compare the two approaches: It's important to note that although the code appears similar, the two programs are not identical. The purpose of comparing them side by side is just to highlight the resemblance in how they are written. ## Embracing Control Flow One significant advantage of using `Effect.gen` in conjunction with generators is its capability to employ standard control flow constructs within the generator function. These constructs include `if`/`else`, `for`, `while`, and other branching and looping mechanisms, enhancing your ability to express complex control flow logic in your code. **Example** ## How to Raise Errors The `Effect.gen` API lets you integrate error handling directly into your workflow by yielding failed effects. You can introduce errors with `Effect.fail`, as shown in the example below. **Example** ## The Role of Short-Circuiting When working with `Effect.gen`, it is important to understand how it handles errors. This API will stop execution at the **first error** it encounters and return that error. How does this affect your code? If you have several operations in sequence, once any one of them fails, the remaining operations will not run, and the error will be returned. In simpler terms, if something fails at any point, the program will stop right there and deliver the error to you. If you don't want to stop on an error, you can use the `Effect.result` method to encapsulate the error in the data type: see the . **Example** Even though execution never reaches code after a failure, TypeScript may still assume that the code below the error is reachable unless you explicitly return after the failure. For example, consider the following scenario where you want to narrow the type of a variable: **Example** In this example, TypeScript still considers `user` possibly `undefined` because there is no explicit return after the failure. To fix this, explicitly return right after calling `Effect.fail`: **Example** ## Passing `this` In some cases, you might need to pass a reference to the current object into the body of your generator function. You can achieve this by utilizing an overload that accepts the reference as the first argument: **Example** --- title: Why Effect? description: Discover how Effect transforms TypeScript programming by using the type system to track errors, context, and success, offering practical solutions for building reliable, maintainable applications. sidebar: order: 1 --- Programming is challenging. When we build libraries and apps, we look to many tools to handle the complexity and make our day-to-day more manageable. Effect presents a new way of thinking about programming in TypeScript. Effect is an ecosystem of tools that help you build better applications and libraries. As a result, you will also learn more about the TypeScript language and how to use the type system to make your programs more reliable and easier to maintain. In "typical" TypeScript, without Effect, we write code that assumes that a function is either successful or throws an exception. For example: Based on the types, we have no idea that this function can throw an exception. We can only find out by reading the code. This may not seem like much of a problem when you only have one function in your codebase, but when you have hundreds or thousands, it really starts to add up. It's easy to forget that a function can throw an exception, and it's easy to forget to handle that exception. Often, we will do the "easiest" thing and just wrap the function in a `try/catch` block. This is a good first step to prevent your program from crashing, but it doesn't make it any easier to manage or understand our complex application/library. We can do better. One of the most important tools we have in TypeScript is the compiler. It is the first line of defense against bugs, domain errors, and general complexity. ## The Effect Pattern While Effect is a vast ecosystem of many different tools, if it had to be reduced down to just one idea, it would be the following: Effect's major unique insight is that we can use the type system to track **errors** and **context**, not only **success** values as shown in the divide example above. Here's the same divide function from above, but with the Effect pattern: With this approach, the function no longer throws exceptions. Instead, errors are handled as values, which can be passed along like success values. The type signature also makes it clear: - What success value the function returns . - What error can occur . - What additional context or dependencies are required . Additionally, tracking context allows you to provide additional information to your functions without having to pass in everything as an argument. For example, you can swap out implementations of live external services with mocks during your tests without changing any core business logic. ## Don't Re-Invent the Wheel Application code in TypeScript often solves the same problems over and over again. Interacting with external services, filesystems, databases, etc. are common problems for all application developers. Effect provides a rich ecosystem of libraries that provide standardized solutions to many of these problems. You can use these libraries to build your application, or you can use them to build your own libraries. Managing challenges like error handling, debugging, tracing, async/promises, retries, streaming, concurrency, caching, resource management, and a lot more are made manageable with Effect. You don't have to re-invent the solutions to these problems, or install tons of dependencies. Effect, under one umbrella, solves many of the problems that you would usually install many different dependencies with different APIs to solve. ## Solving Practical Problems Effect is heavily inspired by great work done in other languages, like Scala and Haskell. However, it's important to understand that Effect's goal is to be a practical toolkit, and it goes to great lengths to solve real, everyday problems that developers face when building applications and libraries in TypeScript. ## Enjoy Building and Learning Learning Effect is a lot of fun. Many developers in the Effect ecosystem are using Effect to solve real problems in their day-to-day work, and also experiment with cutting edge ideas for pushing TypeScript to be the most useful language it can be. You don't have to use all aspects of Effect at once, and can start with the pieces of the ecosystem that make the most sense for the problems you are solving. Effect is a toolkit, and you can pick and choose the pieces that make the most sense for your use case. However, as more and more of your codebase is using Effect, you will probably find yourself wanting to utilize more of the ecosystem! Effect's concepts may be new to you, and might not completely make sense at first. This is totally normal. Take your time with reading the docs and try to understand the core concepts - this will really pay off later on as you get into the more advanced tooling in the Effect ecosystem. The Effect community is always happy to help you learn and grow. Feel free to hop into our or discuss on ! We are open to feedback and contributions, and are always looking for ways to improve Effect. --- title: Logging description: Discover Effect's logging utilities for dynamic log levels, custom outputs, and fine-grained control over logs. sidebar: order: 0 --- import { Aside } from "@astrojs/starlight/components" Logging is an important aspect of software development, especially for debugging and monitoring the behavior of your applications. In this section, we'll explore Effect's logging utilities and see how they compare to traditional logging methods. ## Advantages Over Traditional Logging Effect's logging utilities provide several benefits over conventional logging approaches: 1. **Dynamic Log Level Control**: With Effect's logging, you have the ability to change the log level dynamically. This means you can control which log messages get displayed based on their severity. For example, you can configure your application to log only warnings or errors, which can be extremely helpful in production environments to reduce noise. 2. **Custom Logging Output**: Effect's logging utilities allow you to change how logs are handled. You can direct log messages to various destinations, such as a service or a file, using a . This flexibility ensures that logs are stored and processed in a way that best suits your application's requirements. 3. **Fine-Grained Logging**: Effect enables fine-grained control over logging on a per-part basis of your program. You can set different log levels for different parts of your application, tailoring the level of detail to each specific component. This can be invaluable for debugging and troubleshooting, as you can focus on the information that matters most. 4. **Environment-Based Logging**: Effect's logging utilities can be combined with deployment environments to achieve granular logging strategies. For instance, during development, you might choose to log everything at a trace level and above for detailed debugging. In contrast, your production version could be configured to log only errors or critical issues, minimizing the impact on performance and noise in production logs. 5. **Additional Features**: Effect's logging utilities come with additional features such as the ability to measure time spans, alter log levels on a per-effect basis, and integrate spans for performance monitoring. ## log The `Effect.log` function allows you to log a message at the default `INFO` level. **Example** The default logger in Effect adds several useful details to each log entry: | Annotation | Description | | ----------- | --------------------------------------------------------------------------------------------------- | | `timestamp` | The timestamp when the log message was generated. | | `level` | The log level at which the message is logged . | | `fiber` | The identifier of the executing the program. | | `message` | The log message content, which can include multiple strings or values. | | `span` | The duration of a span in milliseconds, providing insight into the timing of operations. | You can also log multiple messages at once. **Example** For added context, you can also include one or more instances in your logs, which provide detailed error information under an additional `cause` annotation: **Example** ## Log Levels ### logDebug By default, `DEBUG` messages **are not displayed**. To enable them for an effect, provide the `References.MinimumLogLevel` context reference with the value `"Debug"`. **Example** ### logInfo The `INFO` log level is displayed by default. This level is typically used for general application events or progress updates. **Example** ### logWarning The `WARN` log level is displayed by default. This level is intended for potential issues or warnings that do not immediately disrupt the flow of the program but should be monitored. **Example** ### logError The `ERROR` log level is displayed by default. These messages represent issues that need to be addressed. **Example** ### logFatal The `FATAL` log level is displayed by default. This log level is typically reserved for unrecoverable errors. **Example** ## Custom Annotations You can enhance your log outputs by adding custom annotations using the `Effect.annotateLogs` function. This allows you to attach extra metadata to each log entry, improving traceability and providing additional context. ### Adding a Single Annotation You can apply a single annotation as a key/value pair to all log entries within an effect. **Example** In this example, all logs generated within the `program` will include the annotation `key=value`. ### Annotations with Nested Effects Annotations propagate to all logs generated within nested or downstream effects. This ensures that logs from any child effects inherit the parent effect's annotations. **Example** In this example, the annotation `key=value` is included in all logs, even those from the nested `anotherProgram` effect. ### Adding Multiple Annotations You can also apply multiple annotations at once by passing an object with key/value pairs. Each key/value pair will be added to every log entry within the effect. **Example** In this case, each log will contain both `key1=value1` and `key2=value2`. ### Scoped Annotations If you want to limit the scope of your annotations so that they only apply to certain log entries, you can use `Effect.annotateLogsScoped`. This function confines the annotations to logs produced within a specific scope. **Example** ## Log Spans Effect provides built-in support for log spans, which allow you to measure and log the duration of specific tasks or sections of your code. This feature is helpful for tracking how long certain operations take, giving you better insights into the performance of your application. **Example** ## Disabling Default Logging Sometimes, perhaps during test execution, you might want to disable default logging in your application. Effect provides several ways to turn off logging when needed. In this section, we'll look at different methods to disable logging in the Effect framework. **Example** One convenient way to disable logging is to provide `References.MinimumLogLevel` with the value `"None"`. **Example** Another approach to disable logging is by creating a layer that sets the minimum log level to `"None"`, effectively turning off all log output. **Example** You can also disable logging by creating a custom runtime that includes the configuration to turn off logging: ## Loading the Log Level from Configuration To load the log level from a and apply it to your program, map the configured value to a layer for `References.MinimumLogLevel`. **Example** ## Custom loggers In this section, you'll learn how to define a custom logger and set it as the default logger in your application. Custom loggers give you control over how log messages are handled, such as routing them to external services, writing to files, or formatting logs in a specific way. ### Defining a Custom Logger You can define your own logger using the `Logger.make` function. This function allows you to specify how log messages should be processed. **Example** In this example, the custom logger logs messages to the console with the log level and message formatted as ` Message`. ### Using a Custom Logger in a Program Let's assume you have the following tasks and a program where you log some messages: Create a `Logger.layer` containing the loggers that should receive messages, then provide it to the program with `Effect.provide`. **Example** When you run the above program, the following log messages are printed to the console: ## Built-in Loggers Effect provides several built-in loggers that you can use depending on your logging needs. These loggers offer different formats, each suited for different environments or purposes, such as development, production, or integration with external logging services. Each logger is available in two forms: the logger itself, and a layer that uses the logger and sends its output to the `Console` . For example, the `structuredLogger` logger generates logs in a detailed object-based format, while the `structured` layer uses the same logger and writes the output to the `Console` service. ### stringLogger The `stringLogger` logger produces logs in a human-readable key-value style. This format is commonly used in development and production because it is simple and easy to read in the console. This logger does not have a corresponding layer because it is the default logger. Output: ### logfmtLogger The `logfmtLogger` logger produces logs in a human-readable key-value format, similar to the logger. The main difference is that `logfmtLogger` removes extra spaces to make logs more compact. Output: ### prettyLogger The `prettyLogger` logger enhances log output by using color and indentation for better readability, making it particularly useful during development when visually scanning logs in the console. Output: ### structuredLogger The `structuredLogger` logger produces logs in a detailed object-based format. This format is helpful when you need more traceable logs, especially if other systems analyze them or store them for later review. Output: | Field | Description | | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `message` | Either a single processed value or an array of processed values, depending on how many messages are logged. | | `level` | A string that indicates the log level label . | | `timestamp` | An ISO 8601 timestamp for when the log was generated . | | `cause` | A string that shows detailed error information, or `undefined` if no cause was provided. | | `annotations` | An object where each key is an annotation label and the corresponding value is parsed into a structured format . | | `spans` | An object mapping each span label to its duration in milliseconds, measured from its start time until the moment the logger was called . | | `fiberId` | The identifier of the fiber that generated this log . | ### jsonLogger The `jsonLogger` logger produces logs in JSON format. This can be useful for tools or services that parse and store JSON logs. It calls `JSON.stringify` on the object created by the logger. Output: ## Combining Loggers ### Forwarding to Multiple Loggers A custom logger can invoke other loggers to forward each message to all of them. **Example** --- title: Metrics in Effect description: Effect Metrics provides powerful monitoring tools, including counters, gauges, histograms, summaries, and frequencies, to track your application's performance and behavior. sidebar: label: Metrics order: 1 --- import { Aside } from "@astrojs/starlight/components" In complex and highly concurrent applications, managing various interconnected components can be quite challenging. Ensuring that everything runs smoothly and avoiding application downtime becomes crucial in such setups. Now, let's imagine we have a sophisticated infrastructure with numerous services. These services are replicated and distributed across our servers. However, we often lack insight into what's happening across these services, including error rates, response times, and service uptime. This lack of visibility can make it challenging to identify and address issues effectively. This is where Effect Metrics comes into play; it allows us to capture and analyze various metrics, providing valuable data for later investigation. Effect Metrics offers support for five different types of metrics: | Metric | Description | | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Counter** | Counters are used to track values that increase over time, such as request counts. They help us keep tabs on how many times a specific event or action has occurred. | | **Gauge** | Gauges represent a single numerical value that can fluctuate up and down over time. They are often used to monitor metrics like memory usage, which can vary continuously. | | **Histogram** | Histograms are useful for tracking the distribution of observed values across different buckets. They are commonly used for metrics like request latencies, allowing us to understand how response times are distributed. | | **Summary** | Summaries provide insight into a sliding window of a time series and offer metrics for specific percentiles of the time series, often referred to as quantiles. This is particularly helpful for understanding latency-related metrics, such as request response times. | | **Frequency** | Frequency metrics count the occurrences of distinct string values. They are useful when you want to keep track of how often different events or conditions are happening in your application. | ## Counter In the world of metrics, a Counter is a metric that represents a single numerical value that can be both incremented and decremented over time. Think of it like a tally that keeps track of changes, such as the number of a particular type of request received by your application, whether it's increasing or decreasing. Unlike some other types of metrics ), where we're interested in the value at a specific moment, with counters, we care about the cumulative value over time. This means it provides a running total of changes, which can go up and down, reflecting the dynamic nature of certain metrics. Some typical use cases for counters include: - **Request Counts**: Monitoring the number of incoming requests to your server. - **Completed Tasks**: Keeping track of how many tasks or processes have been successfully completed. - **Error Counts**: Counting the occurrences of errors in your application. ### How to Create a Counter To create a counter, you can use the `Metric.counter` constructor. **Example** Once created, the counter can accept an effect that returns a `number`, which will increment or decrement the counter. **Example** ### Counter Types You can specify whether the counter tracks a `number` or `bigint`. ### Increment-Only Counters If you need a counter that only increments, you can use the `incremental: true` option. **Example** In this configuration, the counter only accepts positive values. Any attempts to decrement will have no effect, ensuring the counter strictly counts upwards. ### Counters With Constant Input You can configure a counter to always increment by a fixed value each time it is invoked. **Example** ## Gauge In the world of metrics, a Gauge is a metric that represents a single numerical value that can be set or adjusted. Think of it as a dynamic variable that can change over time. One common use case for a gauge is to monitor something like the current memory usage of your application. Unlike counters, where we're interested in cumulative values over time, with gauges, our focus is on the current value at a specific point in time. Gauges are the best choice when you want to monitor values that can both increase and decrease, and you're not interested in tracking their rates of change. In other words, gauges help us measure things that have a specific value at a particular moment. Some typical use cases for gauges include: - **Memory Usage**: Keeping an eye on how much memory your application is using right now. - **Queue Size**: Monitoring the current size of a queue where tasks are waiting to be processed. - **In-Progress Request Counts**: Tracking the number of requests currently being handled by your server. - **Temperature**: Measuring the current temperature, which can fluctuate up and down. ### How to Create a Gauge To create a gauge, you can use the `Metric.gauge` constructor. **Example** Once created, a gauge can be updated by passing an effect that produces the value you want to set for the gauge. **Example** ### Gauge Types You can specify whether the gauge tracks a `number` or `bigint`. ## Histogram A Histogram is a metric used to analyze how numerical values are distributed over time. Instead of focusing on individual data points, a histogram groups values into predefined ranges, called **buckets**, and tracks how many values fall into each range. When a value is recorded, it gets assigned to one of the histogram's buckets based on its range. Each bucket has an upper boundary, and the count for that bucket is increased if the value is less than or equal to its boundary. Once recorded, the individual value is discarded, and the focus shifts to how many values have fallen into each bucket. Histograms also track: - **Total Count**: The number of values that have been observed. - **Sum**: The sum of all the observed values. - **Min**: The smallest observed value. - **Max**: The largest observed value. Histograms are especially useful for calculating percentiles, which can help you estimate specific points in a dataset by analyzing how many values are in each bucket. This concept is inspired by , a well-known monitoring and alerting toolkit. Histograms are particularly useful in performance analysis and system monitoring. By examining how response times, latencies, or other metrics are distributed, you can gain valuable insights into your system's behavior. This data helps you identify outliers, performance bottlenecks, or trends that may require optimization. Common use cases for histograms include: - **Percentile Estimation**: Histograms allow you to approximate percentiles of observed values, like the 95th percentile of response times. - **Known Ranges**: If you can estimate the range of values in advance, histograms can organize the data into predefined buckets for better analysis. - **Performance Metrics**: Use histograms to track metrics like request latencies, memory usage, or throughput over time. - **Aggregation**: Histograms can be aggregated across multiple instances, making them ideal for distributed systems where you need to collect data from different sources. **Example** In this example, we define a histogram with linear buckets, where the values range from `0` to `100` in increments of `10`. Additionally, we include a final bucket for values greater than `100`, referred to as the "Infinity" bucket. This configuration is useful for tracking numeric values, like request latencies, within specific ranges. The program generates random numbers between `1` and `120`, records them in the histogram, and then prints the histogram's state, showing the count of values that fall into each bucket. ### Timer Metric In this example, we demonstrate how to use a timer metric to track the duration of specific workflows. The timer captures how long certain tasks take to execute, storing this information in a histogram, which provides insights into the distribution of these durations. We generate random values to simulate varying wait times, record the durations in the timer, and then print out the histogram's state. **Example** ## Summary A Summary is a metric that gives insights into a series of data points by calculating specific percentiles. Percentiles help us understand how data is distributed. For instance, if you're tracking response times for requests over the past hour, you may want to examine key percentiles such as the 50th, 90th, 95th, or 99th to better understand your system's performance. Summaries are similar to histograms in that they observe `number` values, but with a different approach. Instead of immediately sorting values into buckets and discarding them, a summary holds onto the observed values in memory. However, to avoid storing too much data, summaries use two parameters: - **maxAge**: The maximum age a value can have before it's discarded. - **maxSize**: The maximum number of values stored in the summary. This creates a sliding window of recent values, so the summary always represents a fixed number of the most recent observations. Summaries are commonly used to calculate **quantiles** over this sliding window. A **quantile** is a number between `0` and `1` that represents the percentage of values less than or equal to a certain threshold. For example, a quantile of `0.5` is the **median** value, while `0.95` would represent the value below which 95% of the observed data falls. Quantiles are helpful for monitoring important performance metrics, such as latency, and for ensuring that your system meets performance goals . Summaries are particularly useful in cases where: - The range of values you're observing is not known or estimated in advance, making histograms less practical. - You don't need to aggregate data across multiple instances or average results. Summaries calculate their results on the application side, meaning they focus on the specific instance where they are used. **Example** In this example, we will create a summary to track response times. The summary will: - Hold up to `100` samples. - Discard samples older than `1 day`. - Report the `10%`, `50%`, and `90%` quantiles, which help track response time distributions. We'll apply the summary to an effect that generates random integers, simulating response times. ## Frequency Frequencies are metrics that help count the occurrences of specific values. Think of them as a set of counters, each associated with a unique value. When new values are observed, the frequency metric automatically creates new counters for those values. Frequencies are particularly useful for tracking how often distinct string values occur. Some example use cases include: - Counting the number of invocations for each service in an application, where each service has a logical name. - Monitoring how frequently different types of failures occur. **Example** In this example, we'll create a `Frequency` to observe how often different error codes occur. This can be applied to effects that return a `string` value: ## Metric Attributes Attributes are key-value pairs you can add to metrics to provide additional context. They help categorize and filter metrics, making it easier to analyze specific aspects of your application's performance or behavior. Attributes may be attached to one metric or supplied through the context for a region of the program. ### Adding Attributes to a Specific Metric You can add attributes to an individual metric using `Metric.withAttributes`. **Example** Here, the `request_count` metric has the attribute `"environment": "production"`, allowing you to filter or analyze that metric series later. ### Providing Attributes to a Region The `Metric.CurrentMetricAttributes` context reference supplies attributes to every metric operation within an effect. This is useful when several metrics share context such as the deployment environment. **Example** --- title: Tracing in Effect description: Explore tracing in distributed systems to track request lifecycles across services using spans and traces for debugging and performance optimization. sidebar: label: Tracing order: 2 --- import { Tabs, TabItem, Steps, Aside } from "@astrojs/starlight/components" Although logs and metrics are useful to understand the behavior of individual services, they are not enough to provide a complete overview of the lifetime of a request in a distributed system. In a distributed system, a request can span multiple services and each service can make multiple requests to other services to fulfill the request. In such a scenario, we need to have a way to track the lifetime of a request across multiple services to diagnose what services are the bottlenecks and where the request is spending most of its time. ## Spans A **span** represents a single unit of work or operation within a request. It provides a detailed view of what happened during the execution of that specific operation. Each span typically contains the following information: | Span Component | Description | | ---------------- | ------------------------------------------------------------------ | | **Name** | Describes the specific operation being tracked. | | **Timing Data** | Timestamps indicating when the operation started and its duration. | | **Log Messages** | Structured logs capturing important events during the operation. | | **Attributes** | Metadata providing additional context about the operation. | Spans are key building blocks in tracing, helping you visualize and understand the flow of requests through various services. ## Traces A trace records the paths taken by requests as they propagate through multi-service architectures, like microservice and serverless applications. Without tracing, it is challenging to pinpoint the cause of performance problems in a distributed system. A trace is made of one or more spans. The first span represents the root span. Each root span represents a request from start to finish. The spans underneath the parent provide a more in-depth context of what occurs during a request . Many Observability back-ends visualize traces as waterfall diagrams that may look something like this: ! Waterfall diagrams show the parent-child relationship between a root span and its child spans. When a span encapsulates another span, this also represents a nested relationship. ## Creating Spans You can add tracing to an effect by creating a span using the `Effect.withSpan` API. This helps you track specific operations within the effect. **Example** Instrumenting an effect with a span does not change its type. If you start with an `Effect`, the result remains an `Effect`. ## Printing Spans To print spans for debugging or analysis, you'll need to install the required tracing tools. Here’s how to set them up for your project. ### Installing Dependencies Choose your package manager and install the necessary libraries: ### Printing a Span to the Console Once the dependencies are installed, you can set up span printing using OpenTelemetry. Here's an example showing how to print a span for an effect. **Example** ### Understanding the Span Output The output provides detailed information about the span: | Field | Description | | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `traceId` | A unique identifier for the entire trace, helping trace requests or operations as they move through an application. | | `parentId` | Identifies the parent span of the current span, marked as `undefined` in the output when there is no parent span, making it a root span. | | `name` | Describes the name of the span, indicating the operation being tracked . | | `id` | A unique identifier for the current span, distinguishing it from other spans within a trace. | | `timestamp` | A timestamp representing when the span started, measured in microseconds since the Unix epoch. | | `duration` | Specifies the duration of the span, representing the time taken to complete the operation . | | `attributes` | Spans may contain attributes, which are key-value pairs providing additional context or information about the operation. In this output, it's an empty object, indicating no specific attributes in this span. | | `status` | The status field provides information about the span's status. In this case, it has a code of 1, which typically indicates an OK status | | `events` | Spans can include events, which are records of specific moments during the span's lifecycle. In this output, it's an empty array, suggesting no specific events recorded. | | `links` | Links can be used to associate this span with other spans in different traces. In the output, it's an empty array, indicating no specific links for this span. | ### Span Capturing an Error Here's how a span looks when the effect encounters an error: **Example** In this example, the span's status code is `2`, indicating an error. The message in the status provides more details about the failure. ## Adding Annotations You can provide extra information to a span by utilizing the `Effect.annotateCurrentSpan` function. This function allows you to attach key-value pairs, offering more context about the execution of the span. **Example** ## Logs as events In the context of tracing, logs are converted into "Span Events." These events offer structured insights into your application's activities and provide a timeline of when specific operations occurred. Each span can include events, which capture specific moments during the execution of a span. In this example, a log message `"Hello"` is recorded as an event within the span. Key details of the event include: | Field | Description | | ------------------------ | ------------------------------------------------------------------------------------------------- | | `name` | The name of the event, which corresponds to the logged message . | | `attributes` | Key-value pairs that provide additional context about the event, such as `fiberId` and log level. | | `time` | The timestamp of when the event occurred, shown in a high-precision format. | | `droppedAttributesCount` | Indicates how many attributes were discarded, if any. In this case, no attributes were dropped. | ## Nesting Spans Spans can be nested to represent a hierarchy of operations. This allows you to track how different parts of your application relate to one another during execution. The following example demonstrates how to create and manage nested spans. **Example** The parent-child relationship is evident in the span output, where the `parentId` of the `child` span matches the `id` of the `parent` span. This structure helps track how operations are related within a single trace. ## Tutorial: Visualizing Traces In this tutorial, we will guide you through visualizing traces generated by a sample Effect application. The sample application has also been configured to export traces and/or metrics via HTTP using . To visualize the traces being exported by our application, we will use a Docker image that contains a preconfigured OpenTelemetry backend based on the , , , , and . ### Tools Explained Let's understand the tools we'll be using in simple terms: - **Docker**: Docker allows us to run applications in containers. Think of a container as a lightweight and isolated environment where your application can run consistently, regardless of the host system. It's a bit like a virtual machine but more efficient. - **Prometheus**: Prometheus is a monitoring and alerting toolkit. It collects metrics and data about your applications and stores them for further analysis. This helps in identifying performance issues and understanding the behavior of your applications. - **Loki**: Loki is a log aggregation system inspired by Prometheus. It does not index the contents of the logs, but rather a set of labels for each log stream. - **Grafana**: Grafana is a visualization and analytics platform. It helps in creating beautiful and interactive dashboards to visualize your application's data. You can use it to graphically represent metrics collected by Prometheus. - **Tempo**: Tempo is a distributed tracing system that allows you to trace the journey of a request as it flows through your application. It provides insights into how requests are processed and helps in debugging and optimizing your applications. ### Getting Docker To get Docker, follow these steps: 1. Visit the Docker website at . 2. Download Docker Desktop for your operating system and install it. 3. After installation, open Docker Desktop, and it will run in the background. ### Simulating Traces 1. **Start the OpenTelemetry Backend** Before we begin generating and exporting traces from our sample application, we will need to get our OpenTelemetry backend running in Docker. This can be done using the following command: 2. **Install Dependencies** We also need to install a few additional dependencies, as well as the latest version of `effect`: 3. **Simulate Traces** Now, let's simulate traces using a sample Node.js application. The following code simulates a set of tasks and generates traces for each task. It also sets up a `Layer` which will export traces from our application to our OpenTelemetry backend over HTTP in OTLP format. 4. **Visualize Traces** Open your web browser and go to `http://localhost:3000/explore`. You should see the Grafana Tempo TraceQL interface. ! To get a list of all available traces, we can select the `"Search"` query type to get a list of all available traces. ! Clicking the generated Trace ID will allow us to inspect the details of the trace. ! ## Integrations ### Sentry To send span data directly to Sentry for analysis, replace the default span processor with Sentry's implementation. This allows you to use Sentry as a backend for tracing and debugging. **Example** --- title: Tracking Fibers description: Track groups of fibers using FiberSet and FiberMap. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" Effect provides two structured-concurrency primitives for tracking groups of fibers: `FiberSet` for an unkeyed collection and `FiberMap` for a collection keyed by an arbitrary value. Both automatically remove fibers when they complete and interrupt any remaining fibers when their owning `Scope` closes. ## Tracking Fibers with FiberSet A `FiberSet` collects fibers so they can be observed, joined, or interrupted together. You add fibers to it with `FiberSet.run` or `FiberSet.add` , and inspect it with `FiberSet.size` or by iterating over it directly. **Example** In this example, we periodically monitor the number of fibers running in the application while calculating a Fibonacci number. The program forks two child fibers for every recursive step, adding each to a shared `FiberSet`, while a separate monitor fiber logs the set's size on a schedule until the calculation finishes. ## Tracking Fibers with FiberMap `FiberMap` behaves like `FiberSet`, but each fiber is tracked under a key. This is useful when you need to look up, replace, or interrupt a specific fiber later, for example, one fiber per connected client, keyed by client ID. Setting a new fiber under a key that's already in use interrupts the previous one first. For the full `FiberMap` API, see the module reference. --- title: Welcome to Effect description: Effect is a TypeScript library for building production-grade software — typed error handling, structured concurrency, resource safety, and observability from one composable core. sidebar: label: Welcome order: 0 --- Effect is a TypeScript library for building production-grade software: typed error handling, structured concurrency, resource safety, and observability, all from one composable core. ## Why Effect? ### Typed data, untyped programs TypeScript is excellent at describing your data, but it says almost nothing about your programs: a function's signature doesn't tell you what can fail, what dependencies it needs, or whether it can be safely retried, timed out, or interrupted. As applications grow, teams end up hand-rolling those guarantees with ad-hoc try/catch, unstructured promises, and libraries that don't compose. ### Programs as values Effect closes that gap with a single building block: the `Effect` type, a value that fully describes a program, including its success value, its possible errors, and its requirements. Because programs are values, they compose: retries, timeouts, concurrency, resource handling, and tracing are operators you apply, not architectures you rebuild.
Out of the box
Typed errors
Failures live in the signature and are handled like data.
Retries and scheduling
Backoff policies you compose instead of loops you write.
Structured concurrency
Bounded parallel work that cleans up after itself.
Resource safety
Acquisition and release are guaranteed, even when things fail.
Dependency injection
Services wired through the type system, trivial to swap in tests.
Observability
Tracing, metrics, and structured logging built into the runtime.
Streaming
Backpressured streams that use the same operators as everything else.
Schema validation
Parse and transform data with types that match reality.
Configuration
Typed config from the environment, validated at startup, secrets redacted.
A consistent ecosystem
HTTP, SQL, CLI, AI, and platform packages built on the same core.
### Built for the AI era The AI era raises the stakes. When coding agents write a growing share of your software, the bottleneck moves from writing code to trusting it. Effect makes a program's failure modes, dependencies, and lifecycle visible to the compiler, turning runtime surprises into precise feedback an agent can act on. And when the thing you are building is itself an AI application, flaky providers, retries, streaming, and rate limits are exactly the problems Effect solves out of the box. ## Your Learning Path Effect rewards learning in order: each step builds on the one before it. Following this spine takes most developers a few focused days; everything else branches off from it.
  1. Understand the core idea An Effect is a value that describes a program: what it produces, how it can fail, and what it needs to run. Everything else builds on this one type.
  2. Set up your project Install the library and configure TypeScript. Effect is a single dependency with zero required tooling.
  3. Write your first program Create effects, compose them with generators, and run them at the edge of your app.
  4. Handle errors the Effect way Errors are typed values, not surprises. Learn expected vs unexpected failures, fallbacks, and retries.
  5. Go concurrent Run effects in parallel with bounded concurrency, race them, and let structured concurrency clean up after you.
Here's the kind of program you'll be writing within the first hour: ## Keep Going
Guides{" "} The in-depth documentation: every topic from pipelines to PubSub, structured to read in order or dip into. Reference{" "} The complete API reference for every Effect module, versioned per release. Playground{" "} Run Effect in your browser with a full editor, no local setup needed. Ideal as a second tab while you learn.
## Join our Community The Effect community lives on : the core team and experienced users are there daily, and no question goes unanswered. --- title: FileSystem description: Explore file system operations for reading, writing, and managing files and directories in Effect. sidebar: order: 2 --- The `effect/FileSystem` module provides a set of operations for reading and writing from/to the file system. ## Basic Usage The module provides a single `FileSystem` , which acts as the gateway for interacting with the filesystem. **Example** The `FileSystem` interface includes the following operations: | Operation | Description | | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **access** | Check if a file can be accessed. You can optionally specify the level of access to check for. | | **copy** | Copy a file or directory from `fromPath` to `toPath`. Equivalent to `cp -r`. | | **copyFile** | Copy a file from `fromPath` to `toPath`. | | **chmod** | Change the permissions of a file. | | **chown** | Change the owner and group of a file. | | **exists** | Check if a path exists. | | **link** | Create a hard link from `fromPath` to `toPath`. | | **makeDirectory** | Create a directory at `path`. You can optionally specify the mode and whether to recursively create nested directories. | | **makeTempDirectory** | Create a temporary directory. By default, the directory will be created inside the system's default temporary directory. | | **makeTempDirectoryScoped** | Create a temporary directory inside a scope. Functionally equivalent to `makeTempDirectory`, but the directory will be automatically deleted when the scope is closed. | | **makeTempFile** | Create a temporary file. The directory creation is functionally equivalent to `makeTempDirectory`. The file name will be a randomly generated string. | | **makeTempFileScoped** | Create a temporary file inside a scope. Functionally equivalent to `makeTempFile`, but the file will be automatically deleted when the scope is closed. | | **open** | Open a file at `path` with the specified `options`. The file handle will be automatically closed when the scope is closed. | | **readDirectory** | List the contents of a directory. You can recursively list the contents of nested directories by setting the `recursive` option. | | **readFile** | Read the contents of a file. | | **readFileString** | Read the contents of a file as a string. | | **readLink** | Read the destination of a symbolic link. | | **realPath** | Resolve a path to its canonicalized absolute pathname. | | **remove** | Remove a file or directory. By setting the `recursive` option to `true`, you can recursively remove nested directories. | | **rename** | Rename a file or directory. | | **sink** | Create a writable `Sink` for the specified `path`. | | **stat** | Get information about a file at `path`. | | **stream** | Create a readable `Stream` for the specified `path`. | | **symlink** | Create a symbolic link from `fromPath` to `toPath`. | | **truncate** | Truncate a file to a specified length. If the `length` is not specified, the file will be truncated to length `0`. | | **utimes** | Change the file system timestamps of the file at `path`. | | **watch** | Watch a directory or file for changes. | | **writeFile** | Write data to a file at `path`. | | **writeFileString** | Write a string to a file at `path`. | **Example** ## Mocking the File System In testing environments, you may want to mock the file system to avoid performing actual disk operations. The `FileSystem.layerNoop` provides a no-operation implementation of the `FileSystem` service. Most operations in `FileSystem.layerNoop` return a **failure** or a **defect** . However, you can override specific behaviors by passing an object to `FileSystem.layerNoop` to define custom return values for selected methods. **Example** --- title: Introduction to Effect Platform description: Build cross-platform applications with unified abstractions for Node.js, Deno, Bun, and browsers using the platform modules built into Effect. sidebar: label: Introduction order: 0 --- import { Aside, Tabs, TabItem, Badge } from "@astrojs/starlight/components" The `effect` package includes platform modules for building platform-independent abstractions in environments such as Node.js, Deno, Bun, and browsers. With these modules, you can integrate abstract services like or into your program. When assembling your final application, you can provide specific for the target platform using the corresponding packages: - `@effect/platform-node` for Node.js or Deno - `@effect/platform-bun` for Bun - `@effect/platform-browser` for browsers ### Stable Modules The following modules are stable and their documentation is available on this website: | Module | Description | Status | | --------------------------------------------------- | ---------------------------------------------------------- | ----------------------------------------- | | | A module for file system operations. | | | | Utilities for working with file paths. | | | | Log messages to a file using the FileSystem APIs. | | | | Run your program with built-in error handling and logging. | | | | Tools for terminal interaction. | | ## Installation The platform modules are included in the `effect` package, so no additional installation is required. See for how to install `effect` itself. Platform-specific packages such as `@effect/platform-node` are only needed to run your program on a concrete platform, as shown in the sections below. ## Getting Started with Cross-Platform Programming Here's a basic example using the `Path` module to create a file path, which can run across different environments: **Example** ### Running the Program in Node.js or Deno First, install the Node.js-specific package: Update the program to load the Node.js-specific context: **Example** Finally, run the program in Node.js using `tsx`, or directly in Deno: ### Running the Program in Bun To run the same program in Bun, first install the Bun-specific package: Update the program to use the Bun-specific context: **Example** Run the program in Bun: --- title: Path description: Perform file path operations such as joining, resolving, and normalizing across platforms. sidebar: order: 4 --- The `effect/Path` module provides a set of operations for working with file paths. ## Basic Usage The module provides a single `Path` , which acts as the gateway for interacting with paths. **Example** The `Path` interface includes the following operations: | Operation | Description | | -------------------- | -------------------------------------------------------------------------- | | **basename** | Returns the last part of a path, optionally removing a given suffix. | | **dirname** | Returns the directory part of a path. | | **extname** | Returns the file extension from a path. | | **format** | Formats a path object into a path string. | | **fromFileUrl** | Converts a file URL to a path. | | **isAbsolute** | Checks if a path is absolute. | | **join** | Joins multiple path segments into one. | | **normalize** | Normalizes a path by resolving `.` and `..` segments. | | **parse** | Parses a path string into an object with its segments. | | **relative** | Computes the relative path from one path to another. | | **resolve** | Resolves a sequence of paths to an absolute path. | | **sep** | Returns the platform-specific path segment separator . | | **toFileUrl** | Converts a path to a file URL. | | **toNamespacedPath** | Converts a path to a namespaced path . | **Example** --- title: PlatformLogger description: Log messages to a file using the FileSystem APIs. sidebar: order: 6 --- Effect's logging system generally writes messages to the console by default. However, you might prefer to store logs in a file for easier debugging or archiving. The `Logger.toFile` function creates a logger that sends log messages to a file on disk. ### toFile Creates a new logger from an existing string-based logger, writing its output to the specified file. If you include a `batchWindow` duration when calling `toFile`, logs are batched for that period before being written. This can reduce overhead if your application produces many log entries. Without a `batchWindow`, logs are written as they arrive. Note that `toFile` returns an `Effect` that may fail with a `PlatformError` if the file cannot be opened or written to. Be sure to handle this possibility if you need to react to file I/O issues. **Example** This logger requires a `FileSystem` implementation to open and write to the file. For Node.js, you can use `NodeFileSystem.layer`. In the following example, logs are written to both the console and a file. The console uses the pretty logger, while the file uses the logfmt format. **Example** --- title: Runtime description: Run your program with built-in error handling and logging. sidebar: order: 6 --- ## Running Your Main Program with runMain `runMain` helps you execute a main effect with built-in error handling, logging, and signal management. You can concentrate on your effect while `runMain` looks after finalizing resources, logging errors, and setting exit codes. - **Exit Codes** If your effect fails or is interrupted, `runMain` assigns a suitable exit code . - **Logs** By default, it records errors. This can be turned off if needed. - **Interrupt Handling** If the application receives `SIGINT` or a similar signal, `runMain` will interrupt the effect and still run any necessary teardown steps. - **Teardown Logic** You can rely on the default teardown or define your own. The default sets an exit code of `1` for a non-interrupted failure. ### Usage Options When calling `runMain`, pass in a configuration object with these fields : - `disableErrorReporting`: If `true`, errors are not automatically logged. - `teardown`: Provide a custom function for finalizing the program. If missing, the default sets exit code `1` for a non-interrupted failure. **Example** **Example** **Example** **Example** --- title: Terminal description: Interact with standard input and output to read user input and display messages on the terminal. sidebar: order: 7 --- The `effect/Terminal` module provides an abstraction for interacting with standard input and output, including reading user input and displaying messages on the terminal. ## Basic Usage The module provides a single `Terminal` , which serves as the entry point to reading from and writing to standard input and standard output. **Example** ## Writing to standard output **Example** ## Reading from standard input **Example** ## Example: Number guessing game This example demonstrates how to create a complete number-guessing game by reading input from the terminal and providing feedback to the user. The game continues until the user guesses the correct number. **Example** --- title: Default Services description: Learn about the default services in Effect, including Clock, Console, Random, ConfigProvider, and Tracer, and how they are automatically provided for your programs. sidebar: order: 1 --- Effect comes equipped with live implementations of five services: `Clock`, `ConfigProvider`, `Console`, `Random`, and `Tracer`. When we employ these services, there's no need to explicitly provide their implementations. Effect automatically supplies live versions of these services to our effects, sparing us from manual setup. **Example** As you can observe, even if our program utilizes both `Clock` and `Console`, the `Requirements` parameter, representing the services required for the effect to execute, remains set to `never`. Effect takes care of handling these services seamlessly for us. ## Overriding Default Services Each default service is exposed as a `Context.Reference`: `Clock.Clock`, `ConfigProvider.ConfigProvider`, `Console.Console`, `Random.Random`, and `Tracer.Tracer`. Use `Effect.provideService` to run an effect with a different implementation. The override applies only to the effect being provided. Some modules also provide higher-level helpers for common overrides. For example, `Random.withSeed` installs a deterministic random-number generator for an effect. **Example** --- title: Layer Memoization description: Learn how layer memoization optimizes performance in Effect by reusing layers and controlling their instantiation. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" Layer memoization allows a layer to be created once and used multiple times in the dependency graph. If we use the same layer twice: then the `L1` layer will be allocated only once. ## Memoization When Providing Globally One important feature of an Effect application is that layers are shared by default. This means that if the same layer is used twice, and if we provide the layer globally, the layer will only be allocated a single time. For every layer in our dependency graph, there is only one instance of it that is shared between all the layers that depend on it. **Example** For example, assume we have the three services `A`, `B`, and `C`. The implementation of both `B` and `C` is dependent on the `A` service: Although both `BLive` and `CLive` layers require the `ALive` layer, the `ALive` layer is instantiated only once. It is shared with both `BLive` and `CLive`. ## Acquiring a Fresh Version If we don't want to share a module, we should create a fresh, non-shared version of it through `Layer.fresh`. **Example** ## No Memoization When Providing Locally If we don't provide a layer globally but instead provide them locally, that layer doesn't support memoization by default. **Example** In the following example, we provided the `ALive` layer two times locally, and Effect doesn't memoize the construction of the `ALive` layer. So, it will be initialized two times: ## Manual Memoization We can memoize a layer manually using a `Layer.MemoMap`. A `MemoMap` tracks which layers have already been built so that building the same layer again, against the same `MemoMap`, reuses the earlier result instead of re-running its acquisition effect. Create a `MemoMap` with `Layer.makeMemoMap`, then build a layer into a `Context` with `Layer.buildWithMemoMap`, passing the `MemoMap` and a `Scope`. **Example** --- title: Managing Layers description: Learn how to use layers in Effect to manage service dependencies and build efficient, clean dependency graphs for your applications. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" In the page, you learned how to create effects which depend on some service to be provided in order to execute, as well as how to provide that service to an effect. However, what if we have a service within our effect program that has dependencies on other services in order to be built? We want to avoid leaking these implementation details into the service interface. To represent the "dependency graph" of our program and manage these dependencies more effectively, we can utilize a powerful abstraction called "Layer". Layers act as **constructors for creating services**, allowing us to manage dependencies during construction rather than at the service level. This approach helps to keep our service interfaces clean and focused. Let's review some key concepts before diving into the details: | Concept | Description | | --------------- | ------------------------------------------------------------------------------------------------------------------------- | | **service** | A reusable component providing specific functionality, used across different parts of an application. | | **service key** | A unique identifier representing a **service**, allowing Effect to locate and use it. | | **context** | A collection of services, functioning like a map with **service keys** as keys and **services** as values. | | **layer** | An abstraction for constructing **services**, managing dependencies during construction rather than at the service level. | ## Designing the Dependency Graph Let's imagine that we are building a web application. We could imagine that the dependency graph for an application where we need to manage configuration, logging, and database access might look something like this: - The `Config` service provides application configuration. - The `Logger` service depends on the `Config` service. - The `Database` service depends on both the `Config` and `Logger` services. Our goal is to build the `Database` service along with its direct and indirect dependencies. This means we need to ensure that the `Config` service is available for both `Logger` and `Database`, and then provide these dependencies to the `Database` service. ## Avoiding Requirement Leakage When constructing the `Database` service, it's important to avoid exposing the dependencies on `Config` and `Logger` within the `Database` interface. You might be tempted to define the `Database` service as follows: **Example** Here, the `query` function of the `Database` service requires both `Config` and `Logger`. This design leaks implementation details, making the `Database` service aware of its dependencies, which complicates testing and makes it difficult to mock. To demonstrate the problem, let's create a test instance of the `Database` service: **Example** Because the `Database` service interface directly includes dependencies on `Config` and `Logger`, it forces any test setup to include these services, even if they're irrelevant to the test. This adds unnecessary complexity and makes it difficult to write simple, isolated unit tests. Instead of directly tying dependencies to the `Database` service interface, dependencies should be managed at the construction phase. We can use **layers** to properly construct the `Database` service and manage its dependencies without leaking details into the interface. ## Creating Layers The `Layer` type is structured as follows: A `Layer` represents a blueprint for constructing a `RequirementsOut` . It requires a `RequirementsIn` as input and may result in an error of type `Error` during the construction process. | Parameter | Description | | ----------------- | -------------------------------------------------------------------------- | | `RequirementsOut` | The service or resource to be created. | | `Error` | The type of error that might occur during the construction of the service. | | `RequirementsIn` | The dependencies required to construct the service. | By using layers, you can better organize your services, ensuring that their dependencies are clearly defined and separated from their implementation details. For simplicity, let's assume that we won't encounter any errors during the value construction . Now, let's determine how many layers we need to implement our dependency graph: | Layer | Dependencies | Type | | -------------- | ---------------------------------------------------------- | ------------------------------------------ | | `ConfigLive` | The `Config` service does not depend on any other services | `Layer` | | `LoggerLive` | The `Logger` service depends on the `Config` service | `Layer` | | `DatabaseLive` | The `Database` service depends on `Config` and `Logger` | `Layer` | When a service has multiple dependencies, they are represented as a **union type**. In our case, the `Database` service depends on both the `Config` and `Logger` services. Therefore, the type for the `DatabaseLive` layer will be: ### Config The `Config` service does not depend on any other services, so `ConfigLive` will be the simplest layer to implement. Just like in the page, we must create a service key for the service. And because the service has no dependencies, we can create the layer directly using the `Layer.succeed` constructor: Looking at the type of `ConfigLive` we can observe: - `RequirementsOut` is `Config`, indicating that constructing the layer will produce a `Config` service - `Error` is `never`, indicating that layer construction cannot fail - `RequirementsIn` is `never`, indicating that the layer has no dependencies Note that, to construct `ConfigLive`, we used the `Config.of` constructor. However, this is merely a helper to ensure correct type inference for the implementation. It's possible to skip this helper and construct the implementation directly as a simple object: ### Logger Now we can move on to the implementation of the `Logger` service, which depends on the `Config` service to retrieve some configuration. Just like we did in the page, we can yield the `Config` service key to "extract" the service from the context. Given that using the `Config` service key is an effectful operation, we use `Layer.effect` to create a layer from the resulting effect. Looking at the type of `LoggerLive`: we can observe that: - `RequirementsOut` is `Logger` - `Error` is `never`, indicating that layer construction cannot fail - `RequirementsIn` is `Config`, indicating that the layer has a requirement ### Database Finally, we can use our `Config` and `Logger` services to implement the `Database` service. Looking at the type of `DatabaseLive`: we can observe that the `RequirementsIn` type is `Config | Logger`, i.e., the `Database` service requires both `Config` and `Logger` services. ## Combining Layers Layers can be combined in two primary ways: **merging** and **composing**. ### Merging Layers Layers can be combined through merging using the `Layer.merge` function: When we merge two layers, the resulting layer: - requires all the services that both of them require . - produces all services that both of them produce . For example, in our web application above, we can merge our `ConfigLive` and `LoggerLive` layers into a single `AppConfigLive` layer, which retains the requirements of both layers and the outputs of both layers : ### Composing Layers Layers can be composed using the `Layer.provide` function: Sequential composition of layers implies that the output of one layer is supplied as the input for the inner layer, resulting in a single layer with the requirements of the outer layer and the output of the inner. Now we can compose the `AppConfigLive` layer with the `DatabaseLive` layer: We obtained a `MainLive` layer that produces the `Database` service: This layer is the fully resolved layer for our application. ### Merging and Composing Layers Let's say we want our `MainLive` layer to return both the `Config` and `Database` services. We can achieve this with `Layer.provideMerge`: ## Providing a Layer to an Effect Now that we have assembled the fully resolved `MainLive` for our application, we can provide it to our program to satisfy the program's requirements using `Effect.provide`: Note that the `runnable` requirements type is `never`, indicating that the program does not require any additional services to run. ## Converting a Layer to an Effect Sometimes your entire application might be a Layer, for example, an HTTP server. You can convert that layer to an effect with `Layer.launch`. It constructs the layer and keeps it alive until interrupted. **Example** ## Tapping The `Layer.tap` and `Layer.tapError` functions allow you to perform additional effects based on the success or failure of a layer. These operations do not modify the layer's signature but are useful for logging or performing side effects during layer construction. - `Layer.tap`: Executes a specified effect when the layer is successfully acquired. - `Layer.tapError`: Executes a specified effect when the layer fails to acquire. **Example** ## Error Handling When constructing layers, it is important to handle potential errors. `Layer.catch` can inspect an acquisition error and return a fallback layer. ### catch The `Layer.catch` function allows you to recover from errors during layer construction by specifying a fallback layer. This can be useful for handling specific error cases and ensuring the application can continue with an alternative setup. **Example** If the fallback does not need the error, ignore the argument passed to `Layer.catch`. **Example** ## Defining Services with Context.Service `Context.Service` can define a service key and its construction effect together as a single class. You can then expose layers as static fields on that class. ### Defining a Service with Dependencies The following example defines a `Cache` service that depends on a file system. **Example** ### Declaring the Service's Layers Declare a service's layers as static class fields, built from `this.make` with `Layer.effect`. The field names below are just a convention, not part of the API. | Static field | Description | | -------------------------------- | --------------------------------------------------------------------------------- | | `Cache.layer` | Provides the `Cache` service with its dependencies already included. | | `Cache.layerWithoutDependencies` | Provides the `Cache` service but requires dependencies to be provided separately. | ### Accessing the Service A service created with `Context.Service` can be accessed like any other Effect service. **Example** Since this example uses `Cache.layer`, it interacts with the real file system. If the file does not exist, it results in an error. ### Injecting Test Dependencies To test the program without depending on the real file system, we can inject a test file system using the `Cache.layerWithoutDependencies` layer. **Example** ### Mocking the Service Directly Alternatively, you can mock the `Cache` service itself instead of replacing its dependencies. **Example** ### Alternative Ways to Construct `make` `Context.Service`'s `make` field accepts any `Effect`, allowing services to use different construction styles: | Style | How to build `make` | | ------------ | ------------------------------------------------------------------------------------------------------------------------------ | | Static value | `Effect.succeed`: a constant implementation. | | Synchronous | `Effect.sync => ...)`: a synchronous constructor. | | Effectful | `Effect.gen { ... })` : a constructor that itself depends on other services. | | Scoped | `Effect.gen` using `Effect.acquireRelease`/`Effect.addFinalizer` inside `make`: lifecycle management, no separate option. | **Example** This is the simplest way to define a service. It is useful when you want to provide a constant value for the service. **Example** **Example** The `Scoped.layer` layer does not require `Scope` as a dependency, since `Scoped` itself manages its lifecycle. --- title: Managing Services description: Learn how to manage reusable services in Effect, handle dependencies efficiently, and ensure clean, decoupled architecture in your applications. sidebar: order: 0 --- import { Aside, Tabs, TabItem } from "@astrojs/starlight/components" In the context of programming, a **service** refers to a reusable component or functionality that can be used by different parts of an application. Services are designed to provide specific capabilities and can be shared across multiple modules or components. Services often encapsulate common tasks or operations that are needed by different parts of an application. They can handle complex operations, interact with external systems or APIs, manage data, or perform other specialized tasks. Services are typically designed to be modular and decoupled from the rest of the application. This allows them to be easily maintained, tested, and replaced without affecting the overall functionality of the application. When diving into services and their integration in application development, it helps to start from the basic principles of function management and dependency handling without relying on advanced constructs. Imagine having to manually pass a service around to every function that needs it: This approach becomes cumbersome and unmanageable as your application grows, with services needing to be passed through multiple layers of functions. To streamline this, you might consider using an environment object that bundles various services: However, this introduces a new complexity: you must ensure that the environment is correctly set up with all necessary services before it's used, which can lead to tightly coupled code and makes functional composition and testing more difficult. ## Managing Services with Effect The Effect library simplifies managing these dependencies by leveraging the type system. Instead of manually passing services or environment objects around, Effect allows you to declare service dependencies directly in the function's type signature using the `Requirements` parameter in the `Effect` type: This is how it works in practice when using Effect: **Dependency Declaration**: You specify what services a function needs directly in its type, pushing the complexity of dependency management into the type system. **Service Provision**: `Effect.provideService` is used to make a service implementation available to the functions that need it. By providing services at the start, you ensure that all parts of your application have consistent access to the required services, thus maintaining a clean and decoupled architecture. This approach abstracts away manual service handling, letting developers focus on business logic while the compiler ensures all dependencies are correctly managed. It also makes code more maintainable and scalable. Let's walk through managing services in Effect step by step: 1. **Creating a Service**: Define a service with its unique functionality and interface. 2. **Using the Service**: Access and utilize the service within your application’s functions. 3. **Providing a Service Implementation**: Supply an actual implementation of the service to fulfill the declared requirements. ## How It Works Up to this point, our examples with the Effect framework have dealt with effects that operate independently of external services. This means the `Requirements` parameter in our `Effect` type signature has been set to `never`, indicating no dependencies. However, real-world applications often need effects that rely on specific services to function correctly. These services are managed and accessed through a construct known as `Context`. The `Context` serves as a repository or container for all services an effect may require. It acts like a store that maintains these services, allowing various parts of your application to access and use them as needed. The services stored within the `Context` are directly reflected in the `Requirements` parameter of the `Effect` type. Each service within the `Context` is identified by a unique "service key," which is essentially a unique identifier for the service. When an effect needs to use a specific service, its service key is included in the `Requirements` type parameter. ## Creating a Service To create a new service, you need two things: 1. A unique **identifier**. 2. A **type** describing the possible operations of the service. **Example** Let's create a service for generating random numbers. 1. **Identifier**. We'll use the string `"MyRandomService"` as the unique identifier. 2. **Type**. The service type will have a single operation called `next` that returns a random number. The exported `Random` value is known as a **service key** in Effect. It acts as a representation of the service and allows Effect to locate and use this service at runtime. The service will be stored in a collection called `Context`, which can be thought of as a `Map` where the keys are service keys and the values are services: Let's summarize the concepts we've covered so far: | Concept | Description | | --------------- | ---------------------------------------------------------------------------------------------------------- | | **service** | A reusable component providing specific functionality, used across different parts of an application. | | **service key** | A unique identifier representing a **service**, allowing Effect to locate and use it. | | **context** | A collection of services, functioning like a map with **service keys** as keys and **services** as values. | ## Using the Service Now that we have our service key defined, let's see how we can use it by building a simple program. **Example** In the code above, we can observe that we are able to yield the `Random` service key as if it were an effect itself. This allows us to access the `next` operation of the service. In the code above, we can observe that we are able to flat-map over the `Random` service key as if it were an effect itself. This allows us to access the `next` operation of the service within the `Effect.andThen` callback. It's worth noting that the type of the `program` variable includes `Random` in the `Requirements` type parameter: This indicates that our program requires the `Random` service to be provided in order to execute successfully. If we attempt to execute the effect without providing the necessary service we will encounter a type-checking error: **Example** To resolve this error and successfully execute the program, we need to provide an actual implementation of the `Random` service. In the next section, we will explore how to implement and provide the `Random` service to our program, enabling us to run it successfully. ## Providing a Service Implementation In order to provide an actual implementation of the `Random` service, we can utilize the `Effect.provideService` function. **Example** In the code above, we provide the `program` we defined earlier with an implementation of the `Random` service. We use the `Effect.provideService` function to associate the `Random` service key with its implementation, an object with a `next` operation that generates a random number. Notice that the `Requirements` type parameter of the `runnable` effect is now `never`. This indicates that the effect no longer requires any service to be provided. With the implementation of the `Random` service in place, we are able to run the program without any further requirements. ## Extracting the Service Type To retrieve the service type from a service key, use the `Context.Service.Shape` utility type. **Example** ## Using Multiple Services When we require the usage of more than one service, the process remains similar to what we've learned in defining a service, repeated for each service needed. **Example** Let's examine an example where we need two services, namely `Random` and `Logger`: The `program` effect now has a `Requirements` type parameter of `Random | Logger`: indicating that it requires both the `Random` and `Logger` services to be provided. To execute the `program`, we need to provide implementations for both services: **Example** Alternatively, instead of calling `provideService` multiple times, we can combine the service implementations into a single `Context` and then provide the entire context using the `Effect.provide` function: **Example** ## Optional Services There are situations where we may want to access a service implementation only if it is available. In such cases, we can use the `Effect.serviceOption` function to handle this scenario. The `Effect.serviceOption` function returns an implementation that is available only if it is actually provided before executing this effect. To represent this optionality it returns an of the implementation. **Example** To determine what action to take, we can use the `Option.isNone` function provided by the Option module. This function allows us to check if the service is available or not by returning `true` when the service is not available. In the code above, we can observe that the `Requirements` type parameter of the `program` effect is `never`, even though we are working with a service. This allows us to access something from the context only if it is actually provided before executing this effect. When we run the `program` effect without providing the `Random` service: We see that the log message contains `-1`, which is the default value we provided when the service was not available. However, if we provide the `Random` service implementation: We can observe that the log message now contains a random number generated by the `next` operation of the `Random` service. ## Handling Services with Dependencies Sometimes a service in your application may depend on other services. To maintain a clean architecture, it's important to manage these dependencies without making them explicit in the service interface. Instead, you can use **layers** to handle these dependencies during the service construction phase. **Example** Consider a scenario where multiple services depend on each other. In this case, the `Logger` service requires access to a configuration service . To handle these dependencies in a structured way and prevent them from leaking into the service interfaces, you can use the `Layer` abstraction. For more details on managing dependencies with layers, refer to the page. --- title: Introduction description: Common patterns for safe resource management sidebar: order: 0 --- In long-running applications, managing resources efficiently is essential, particularly when building large-scale systems. If resources like socket connections, database connections, or file descriptors are not properly managed, it can lead to resource leaks, which degrade application performance and reliability. Effect provides constructs that help ensure resources are properly managed and released, even in cases where exceptions occur. By ensuring that every time a resource is acquired, there is a corresponding mechanism to release it, Effect simplifies the process of resource management in your application. ## Finalization In many programming languages, the `try` / `finally` construct ensures that cleanup code runs regardless of whether an operation succeeds or fails. Effect provides similar functionality through `Effect.ensuring`, `Effect.onExit`, and `Effect.onError`. ### ensuring The `Effect.ensuring` function guarantees that a finalizer effect runs whether the main effect succeeds, fails, or is interrupted. This is useful for performing cleanup actions such as closing file handles, logging messages, or releasing locks. If you need access to the effect's result, consider using . **Example** ### onExit `Effect.onExit` allows you to run a cleanup effect after the main effect completes, receiving an value that describes the outcome. - If the effect succeeds, the `Exit` holds the success value. - If it fails, the `Exit` includes the error or failure cause. - If it is interrupted, the `Exit` reflects that interruption. The cleanup step itself is uninterruptible, which can help manage resources in complex or high-concurrency cases. **Example** ### onError This function lets you attach a cleanup effect that runs whenever the calling effect fails, passing the cause of the failure to the cleanup effect. You can use it to perform actions such as logging, releasing resources, or applying additional recovery steps. The cleanup effect will also run if the failure is caused by interruption, and it is uninterruptible, so it always finishes once it starts. **Example** ## acquireUseRelease Many real-world operations involve working with resources that must be released when no longer needed, such as: - Database connections - File handles - Network requests Effect provides `Effect.acquireUseRelease`, which ensures that a resource is: 1. **Acquired** properly. 2. **Used** for its intended purpose. 3. **Released** even if an error occurs. **Syntax** **Example** --- title: Scope description: Learn how Effect simplifies resource management with Scopes, ensuring efficient cleanup and safe resource handling in long-running applications. sidebar: order: 1 --- import { Aside, Tabs, TabItem } from "@astrojs/starlight/components" The `Scope` data type is a core construct in Effect for managing resources in a safe and composable way. A scope represents the lifetime of one or more resources. When the scope is closed, all the resources within it are released, ensuring that no resources are leaked. Scopes also allow the addition of **finalizers**, which define how to release resources. With the `Scope` data type, you can: - **Add finalizers**: A finalizer specifies the cleanup logic for a resource. - **Close the scope**: When the scope is closed, all resources are released, and the finalizers are executed. **Example** In the above example, finalizers are added to the scope, and when the scope is closed, the finalizers are **executed in the reverse order**. This reverse order is important because it ensures that resources are released in the correct sequence. For instance, if you acquire a network connection and then access a file on a remote server, the file must be closed before the network connection to avoid errors. ## addFinalizer The `Effect.addFinalizer` function is a high-level API that allows you to add finalizers to the scope of an effect. A finalizer is a piece of code that is guaranteed to run when the associated scope is closed. The behavior of the finalizer can vary based on the value, which represents how the scope was closed, whether successfully or with an error. **Example** In this example, we use `Effect.addFinalizer` to add a finalizer that logs the exit state after the scope is closed. The finalizer will execute when the effect finishes, and it will log whether the effect completed successfully or failed. The type signature: shows that the workflow requires a `Scope` to run. You can provide this `Scope` using the `Effect.scoped` function, which creates a new scope, runs the effect within it, and ensures the finalizers are executed when the scope is closed. **Example** In this case, the finalizer is executed even when the effect fails. The log output reflects that the finalizer runs after the failure, and it logs the failure details. **Example** ) This example shows how a finalizer behaves when the effect is interrupted. The finalizer runs after the interruption, and the exit status reflects that the effect was stopped mid-execution. ## Manually Create and Close Scopes When you're working with multiple scoped resources within a single operation, it's important to understand how their scopes interact. By default, these scopes are merged into one, but you can have more fine-grained control over when each scope is closed by manually creating and closing them. Let's start by looking at how scopes are merged by default: **Example** In this case, the scopes of `task1` and `task2` are merged into a single scope, and when the program is run, it outputs the tasks and their finalizers in a specific order. If you want more control over when each scope is closed, you can manually create and close them: **Example** In this example, we create two separate scopes, `scope1` and `scope2`, and extend the scope of each task into its respective scope. When you run the program, it outputs the tasks and their finalizers in a different order. You might wonder what happens when a scope is closed, but a task within that scope hasn't completed yet. The key point to note is that the scope closing doesn't force the task to be interrupted. **Example** ## Defining Resources ### acquireRelease The `Effect.acquireRelease` function allows you to define resources that are acquired and safely released when they are no longer needed. This is useful for managing resources such as file handles, database connections, or network sockets. To use `Effect.acquireRelease`, you need to define two actions: 1. **Acquiring the Resource**: An effect describing the acquisition of the resource, e.g., opening a file or establishing a database connection. 2. **Releasing the Resource**: The clean-up effect that ensures the resource is properly released, e.g., closing the file or the connection. The acquisition process is **uninterruptible** to ensure that partial resource acquisition doesn't leave your system in an inconsistent state. The `Effect.acquireRelease` function guarantees that once a resource is successfully acquired, its release step is always executed when the `Scope` is closed. **Example** In the code above, the `Effect.acquireRelease` function creates a resource workflow that requires a `Scope`: This means that the workflow needs a `Scope` to run, and the resource will automatically be released when the scope is closed. You can now use the resource by chaining operations using `Effect.andThen` or similar functions. We can continue working with the resource for as long as we want by using `Effect.andThen` or other Effect operators. For example, here's how we can read the contents: **Example** To ensure proper resource management, the `Scope` should be closed when you're done with the resource. The `Effect.scoped` function handles this for you by creating a `Scope`, running the effect, and then closing the `Scope` when the effect finishes. **Example** ### Example Pattern: Sequencing Operations In certain scenarios, you might need to perform a sequence of chained operations where the success of each operation depends on the previous one. However, if any of the operations fail, you would want to reverse the effects of all previous successful operations. This pattern is valuable when you need to ensure that either all operations succeed, or none of them have any effect at all. Let's go through an example of implementing this pattern. Suppose we want to create a "Workspace" in our application, which involves creating an S3 bucket, an ElasticSearch index, and a Database entry that relies on the previous two. To begin, we define the domain model for the required : - `S3` - `ElasticSearch` - `Database` Next, we define the three create actions and the overall transaction for the workspace. We then create simple service implementations to test the behavior of our Workspace code. To achieve this, we will utilize to construct test These layers will be able to handle various scenarios, including errors, which we can control using the `FailureCase` type. Let's examine the test results for the scenario where `FailureCase` is set to `undefined` : In this case, all operations succeed, and we see a `Result.Success` containing the database entry. Now, let's simulate a failure in the `Database`: The console output will be: You can observe that once the `Database` error occurs, there is a complete rollback that deletes the `ElasticSearch` index first and then the associated `S3` bucket. The result is a `Result.Failure` containing the `DatabaseError`. Let's now make the index creation fail instead: In this case, the console output will be: As expected, once the `ElasticSearch` index creation fails, there is a rollback that deletes the `S3` bucket. The result is a `Result.Failure` containing the `ElasticSearchError`. --- title: Introduction to Runtime description: Learn how Effect's runtime system executes concurrent programs, manages resources, and handles configuration with flexibility and efficiency. sidebar: label: Runtime order: 6 --- The Effect **runtime system** is what turns an `Effect` blueprint into a running program: it supplies the `R` requirements, executes each step, and produces a result. `Effect.run*` functions execute an effect immediately using its runtime system. If you already have a `Context` for the effect's requirements, the `Effect.run*With` variants let you run with that context directly. For a reusable, top-level execution configuration, use `ManagedRuntime` . ## What is a Runtime System? When we write an Effect program, we construct an `Effect` using constructors and combinators. Essentially, we are creating a blueprint of a program. An `Effect` is merely a data structure that describes the execution of a concurrent program. It represents a tree-like structure that combines various primitives to define what the effect should do. However, this data structure itself does not perform any actions, it is solely a description of a concurrent program. To execute this program, the Effect runtime system comes into play. The `Effect.run*` functions are responsible for taking this blueprint and executing it. When the runtime system runs an effect, it creates a root fiber, initializing it with: - The initial - The initial fiber-local state - The initial effect It then starts a loop, executing the instructions described by the `Effect` step by step. You can think of the runtime as a system that takes an and its associated context `Context` and produces an result. Runtime Systems have a lot of responsibilities: | Responsibility | Description | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------ | | **Executing the program** | The runtime must execute every step of the effect in a loop until the program completes. | | **Handling errors** | It handles both expected and unexpected errors that occur during execution. | | **Managing concurrency** | The runtime spawns new fibers when `Effect.forkChild` is called to handle concurrent operations. | | **Cooperative yielding** | It ensures fibers don't monopolize resources, yielding control when necessary. | | **Ensuring resource cleanup** | The runtime guarantees finalizers run properly to clean up resources when needed. | | **Handling async callbacks** | The runtime deals with asynchronous operations transparently, allowing you to write async and sync code uniformly. | ## Running With an Explicit Context When we use like `Effect.runPromise` or `Effect.runFork`, we don't need to mention any runtime object at all. There is no separate "default runtime" value to look up or pass around. These functions execute an `Effect` directly, using an empty context and the . If your effect still has unmet requirements and you already have a `Context` satisfying them, for example, one built manually with `Context.make`, rather than through a `Layer`, you can run it directly with the matching `Effect.run*With` function, instead of first wrapping it with `Effect.provide`: **Example** In most scenarios, this direct style is sufficient for effect execution. However, there are cases where it's helpful to build a reusable runtime, particularly when you need to reuse specific configurations or contexts across many separate calls. For example, in a React app or when executing operations on a server in response to API requests, you might build a reusable runtime from a `Layer` using `ManagedRuntime`, covered next. This allows you to maintain a consistent context across different execution boundaries. ## Locally Scoped Runtime Configuration In Effect, runtime configurations are typically **inherited** from their parent workflows. This means that when we access a runtime configuration or obtain a runtime inside a workflow, we are essentially using the configuration of the parent workflow. However, there are cases where we want to temporarily **override the runtime configuration for a specific part** of our code. This concept is known as locally scoped runtime configuration. Once the execution of that code region is completed, the runtime configuration **reverts** to its original settings. To achieve this, we use `Effect.provide`, which allows us to provide a new runtime configuration to a specific section of our code. **Example** In this example, we create a layer containing a custom logger that logs messages without timestamps or levels. We then use `Effect.provide` to apply this logger layer to the program. To ensure that the runtime configuration is only applied to a specific part of an Effect application, we should provide the configuration layer exclusively to that particular section. **Example** In this example, we demonstrate how to apply a custom logger configuration only to a specific section of the program. The default logger is used for most of the program, but when we apply the `Effect.provide` call, it overrides the logger within that specific nested block. After that, the configuration reverts to its original state. ## ManagedRuntime When developing an Effect application and using `Effect.run*` functions to execute it, the application is automatically run using the default runtime behind the scenes. While it’s possible to adjust specific parts of the application by providing locally scoped configuration layers using `Effect.provide`, there are scenarios where you might want to **customize the runtime configuration for the entire application** from the top level. In these cases, you can create a top-level runtime by converting a configuration layer into a runtime using the `ManagedRuntime.make` constructor. **Example** In this example, we first create a custom configuration layer called `appLayer`, which replaces the default logger with a simple one that logs messages to the console. Next, we use `ManagedRuntime.make` to turn this configuration layer into a runtime. ### Context.Service When working with runtimes that you pass around, `Context.Service` can simplify access to services. It lets you define a service key and its shape together as a single class. **Example** Use `.use` to run a callback with the resolved service, or `yield* Notifications` inside `Effect.gen` to access it directly. This allows you to interact with the service directly: **Example** In this example, the `action` effect depends on the `Notifications` service. This approach allows you to reference services without manually passing them around. Later, you can create a `Layer` that provides the `Notifications` service and build a `ManagedRuntime` with that layer to ensure the service is available where needed. ### Integrations The `ManagedRuntime` simplifies the integration of services and layers with other frameworks or tools, particularly in environments where Effect is not the primary framework and access to the main entry point is restricted. For example, in environments like React or other frameworks where you have limited control over the main application entry point, `ManagedRuntime` helps manage the lifecycle of services. Here's how to manage a service's lifecycle within an external framework: **Example** --- title: Built-In Schedules description: Explore built-in scheduling patterns in Effect for efficient timed repetitions and delays. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" To demonstrate the functionality of different schedules, we will use the following helper function that logs each repetition along with the corresponding delay in milliseconds, formatted as: **Helper** ## Infinite and Fixed Repeats ### forever A schedule that repeats indefinitely, producing the number of recurrences each time it runs. **Example** ### once A schedule that recurs only once. **Example** ### recurs A schedule that repeats a specified number of times, producing the number of recurrences each time it runs. **Example** ## Recurring at specific intervals You can define schedules that control the time between executions. The difference between `spaced` and `fixed` schedules lies in how the interval is measured: - `spaced` delays each repetition from the **end** of the previous one. - `fixed` ensures repetitions occur at **regular intervals**, regardless of execution time. ### spaced A schedule that repeats indefinitely, each repetition spaced the specified duration from the last run. It returns the number of recurrences each time it runs. **Example** The first delay is approximately 100 milliseconds, as the initial execution is not affected by the schedule. Subsequent delays are approximately 200 milliseconds apart, demonstrating the effect of the `spaced` schedule. ### fixed A schedule that recurs at fixed intervals. It returns the number of recurrences each time it runs. If the action run between updates takes longer than the interval, then the action will be run immediately, but re-runs will not "pile up". **Example** ## Increasing Delays Between Executions ### exponential A schedule that recurs using exponential backoff, with each delay increasing exponentially. Returns the current duration between recurrences. **Example** ### fibonacci A schedule that always recurs, increasing delays by summing the preceding two delays . Returns the current duration between recurrences. **Example** --- title: Cron description: Explore cron scheduling in Effect for executing actions at specific times and intervals. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" The Cron module lets you define schedules in a style similar to . It also supports partial constraints , time zone awareness through the module, and robust error handling. This module helps you: - **Create** a `Cron` instance from individual parts. - **Parse and validate** cron expressions. - **Match** existing dates to see if they satisfy a given cron schedule. - **Find** the next occurrence of a schedule after a given date. - **Iterate** over future dates that match a schedule. - **Convert** a `Cron` instance to a `Schedule` for use in effectful programs. ## Creating a Cron You can define a cron schedule by specifying numeric constraints for seconds, minutes, hours, days, months, and weekdays. The `make` function requires you to define all fields representing the schedule's constraints. **Example** - `seconds`, `minutes`, and `hours`: Define the time of day. - `days` and `months`: Specify which calendar days and months are valid. - `weekdays`: Restrict the schedule to specific days of the week. - `tz`: Optionally define the time zone for the schedule. If any field is left empty , it is treated as having "no constraints," allowing any valid value for that part of the date. ## Parsing Cron Expressions Instead of manually constructing a `Cron`, you can use UNIX-like cron strings and parse them with `parse` or `parseUnsafe`. ### parse The `parse` function safely parses a cron string into a `Cron` instance. It returns a , which will contain either the parsed `Cron` or a parsing error. **Example** ### parseUnsafe The `parseUnsafe` function works like , but instead of returning a , it throws an exception if the input is invalid. **Example** ## Checking Dates with match The `match` function allows you to determine if a given `Date` ) satisfies the constraints of a cron schedule. If the date meets the schedule's conditions, `match` returns `true`. Otherwise, it returns `false`. **Example** ## Finding the Next Run The `next` function determines the next date that satisfies a given cron schedule, starting from a specified date. If no starting date is provided, the current time is used as the starting point. If `next` cannot find a matching date within a predefined number of iterations, it throws an error to prevent infinite loops. **Example** ## Iterating Over Future Dates To generate multiple future dates that match a cron schedule, you can use the `sequence` function. This function provides an infinite iterator of matching dates, starting from a specified date. **Example** ## Converting to Schedule The Schedule module allows you to define recurring behaviors, such as retries or periodic events. The `cron` function bridges the `Cron` module with the Schedule module, enabling you to create schedules based on cron expressions or `Cron` instances. ### cron The `Schedule.cron` function generates a that triggers at the start of each interval defined by the provided cron expression or `Cron` instance. When triggered, the schedule produces a tuple `` representing the timestamps of the cron interval window. **Example** --- title: Examples description: Explore practical examples for scheduling, retries, timeouts, and periodic task execution in Effect. sidebar: order: 5 --- These examples demonstrate different approaches to handling timeouts, retries, and periodic execution using Effect. Each scenario ensures that the application remains responsive and resilient to failures while adapting dynamically to various conditions. ## Handling Timeouts and Retries for API Calls When calling third-party APIs, it is often necessary to enforce timeouts and implement retry mechanisms to handle transient failures. In this example, the API call retries up to two times in case of failure and will be interrupted if it takes longer than 4 seconds. **Example** ## Retrying API Calls Based on Specific Errors Sometimes, retries should only happen for certain error conditions. For example, if an API call fails with a `401 Unauthorized` response, retrying might make sense, while a `404 Not Found` error should not trigger a retry. **Example** ## Retrying with Dynamic Delays Based on Error Information Some API errors, such as `429 Too Many Requests`, include a `Retry-After` header that specifies how long to wait before retrying. Instead of using a fixed delay, we can dynamically adjust the retry interval based on this value. **Example** This approach ensures that the retry delay adapts dynamically to the server's response, preventing unnecessary retries while respecting the provided `Retry-After` value. ## Running Periodic Tasks Until Another Task Completes There are cases where we need to repeatedly perform an action at fixed intervals until another longer-running task finishes. This pattern is common in polling mechanisms or periodic logging. **Example** --- title: Introduction description: Learn the fundamentals of scheduling in Effect, including composable recurrence patterns and handling retries and repetitions. sidebar: order: 0 --- # Scheduling Scheduling is an important concept in Effect that allows you to define recurring effectful operations. It involves the use of the `Schedule` type, which is an immutable value that describes a scheduled pattern for executing effects. The `Schedule` type is structured as follows: A schedule operates by consuming values of type `In` and producing values of type `Out`. It determines when to halt or continue the execution based on input values and its internal state. The inclusion of a `Requirements` parameter allows the schedule to leverage additional services or resources as needed. Schedules are defined as a collection of intervals spread out over time. Each interval represents a window during which the recurrence of an effect is possible. ## Retrying and Repetition In the realm of scheduling, there are two related concepts: and . While they share the same underlying idea, they differ in their focus. Retrying aims to handle failures by executing an effect again, while repetition focuses on executing an effect repeatedly to achieve a desired outcome. When using schedules for retrying or repetition, each interval's starting boundary determines when the effect will be executed again. For example, in retrying, if an error occurs, the schedule defines when the effect should be retried. ## Composability of Schedules Schedules are composable, meaning you can combine simple schedules to create more complex recurrence patterns. Operators like `Schedule.min` or `Schedule.max` allow you to build sophisticated schedules by combining and modifying existing ones. This flexibility enables you to tailor the scheduling behavior to meet specific requirements. --- title: Repetition description: Explore repetition in Effect for executing actions multiple times with control over retries, failures, and conditions. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" Repetition is a common requirement when working with effects in software development. It allows us to perform an effect multiple times according to a specific repetition policy. ## repeat The `Effect.repeat` function returns a new effect that repeats the given effect according to a specified schedule or until the first failure. **Example** **Example** ### Skipping First Execution If you want to avoid the first execution and only run the action according to a schedule, you can use `Effect.schedule`. This allows the effect to skip the initial run and follow the defined repeat policy. **Example** ## repeatN The `Effect.repeat` function returns a new effect that repeats the specified effect a given number of times or until the first failure. The repeats are in addition to the initial execution, so `Effect.repeat` executes `action` once initially and then repeats it one additional time if it succeeds. **Example** ## repeatOrElse The `repeatOrElse` function returns a new effect that repeats the specified effect according to the given schedule or until the first failure. When a failure occurs, the failure value and schedule output are passed to a specified handler. Scheduled recurrences are in addition to the initial execution, so `Effect.repeat)` executes `action` once initially and then repeats it an additional time if it succeeds. **Example** ## Repeating Based on a Condition You can control the repetition of an effect by a condition using either a `while` or `until` option, allowing for dynamic control based on runtime outcomes. **Example** --- title: Schedule Combinators description: Learn how to combine and customize schedules in Effect to create complex recurrence patterns, including union, intersection, sequencing, and more. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" Schedules define stateful, possibly effectful, recurring schedules of events, and compose in a variety of ways. Combinators allow us to take schedules and combine them together to get other schedules. To demonstrate the functionality of different schedules, we will use the following helper function that logs each repetition along with the corresponding delay in milliseconds, formatted as: **Helper** ## Composition Schedules can be composed in different ways: | Mode | Description | | ---------------- | -------------------------------------------------------------------------------------------------- | | **Union** | Combines two schedules and recurs if either schedule wants to continue, using the shorter delay. | | **Intersection** | Combines two schedules and recurs only if both schedules want to continue, using the longer delay. | | **Sequencing** | Combines two schedules by running the first one fully, then switching to the second. | ### Union Combines two schedules and recurs if either schedule wants to continue, using the shorter delay. **Example** The `Schedule.min` operator selects the shortest delay at each step, so when combining an exponential schedule with a spaced interval, the initial recurrences will follow the exponential backoff, then settle into the spaced interval once the delays exceed that value. ### Intersection Combines two schedules and recurs only if both schedules want to continue, using the longer delay. **Example** The `Schedule.max` operator enforces both schedules' constraints. In this example, the schedule follows an exponential backoff but stops after 5 recurrences due to the `Schedule.recurs` limit. ### Sequencing Combines two schedules by running the first one fully, then switching to the second. **Example** The first schedule runs until completion, after which the second schedule takes over. In this example, the effect initially executes 5 times with no delay, then continues every 1 second. ## Adding Randomness to Retry Delays The `Schedule.jittered` combinator modifies a schedule by applying a random delay within a specified range. When a resource is out of service due to overload or contention, retrying and backing off doesn't help us. If all failed API calls are backed off to the same point of time, they cause another overload or contention. Jitter adds some amount of randomness to the delay of the schedule. This helps us to avoid ending up accidentally synchronizing and taking the service down by accident. suggests that `Schedule.jittered` is an effective way to introduce randomness in retries. **Example** The `Schedule.jittered` combinator introduces randomness to delays within a range. For example, applying jitter to an exponential backoff ensures that each retry occurs at a slightly different time, reducing the risk of overwhelming the system. ## Controlling Repetitions with Filters Use `Schedule.while` to limit how long a schedule continues. Its predicate receives metadata containing both the schedule input and output. **Example** `Schedule.while` filters repetitions based on the output of the schedule. In this example, the schedule stops once the output exceeds `2`, even though `Schedule.recurs` allows up to 5 repetitions. ## Adjusting Delays Based on Output The `Schedule.modifyDelay` combinator allows you to dynamically change the delay of a schedule based on the number of repetitions or other output conditions. **Example** The delay modification applies dynamically during execution. In this example, the first three repetitions follow the original `1-second` spacing. After that, the delay drops to `100 milliseconds`, making subsequent repetitions occur more frequently. ## Tapping `Schedule.tap` performs an additional effectful operation without changing the schedule's behavior. Its callback receives metadata containing both the schedule input and output. **Example** `Schedule.tap` runs an effect before each recurrence, using the schedule's current output as input. This can be useful for logging, debugging, or triggering side effects. --- title: Advanced Usage description: Learn advanced techniques for defining and extending data schemas, including recursive and mutually recursive types, optional fields, branded types, and schema transformations. sidebar: order: 5 --- import { Aside } from "@astrojs/starlight/components" ## Declaring New Data Types ### Primitive Data Types To declare a schema for an opaque, non-generic data type, you can use `Schema.declare` with a type guard. The following example shows the low-level pattern with `File`. **Example** You can add `identifier`, `title`, and `description` annotations to make the declaration easier for people and schema interpreters to understand. `identifier` and `title` can also improve the default expected-value message. - **Identifier**: a unique name for the schema - **Title**: a brief, descriptive title - **Description**: a detailed explanation of the schema's purpose **Example** ### Type Constructors Type constructors are generic types that take one or more types as arguments and return a new type. To define a schema for a type constructor, you can use the `Schema.declare` function. **Example** ### Adding Interpreter Annotations When defining a new data type, schema interpreters such as or may not know how to handle the new type. This can result in an error, as the interpreter may lack the necessary information for generating instances or producing readable output: **Example** In the above example, attempting to generate arbitrary values for `FileSchema` fails because the interpreter lacks the necessary annotations. To resolve this, provide an annotation for generating arbitrary data: **Example** For more details on how to add annotations for the Arbitrary interpreter, refer to the documentation. ## Branded types TypeScript's type system is structural, which means that any two types that are structurally equivalent are considered the same. This can cause issues when types that are semantically different are treated as if they were the same. **Example** In the above example, `UserId` and `Username` are both aliases for the same type, `string`. This means that the `getUser` function can mistakenly accept a `Username` as a valid `UserId`, causing bugs and errors. To prevent this, Effect introduces **branded types**. These types attach a unique identifier to a type, allowing you to differentiate between structurally similar but semantically distinct types. **Example** By defining `UserId` as a branded type, the `getUser` function can accept only values of type `UserId`, and not plain strings or other types that are compatible with strings. This helps to prevent bugs caused by accidentally passing the wrong type of value to the function. There are two ways to define a schema for a branded type, depending on whether you: - want to define the schema from scratch - have already defined a branded type via and want to reuse it to define a schema ### Defining a brand schema from scratch To define a schema for a branded type from scratch, use the `Schema.brand` function. **Example** ### Reusing an existing branded constructor If you have already defined a branded type using the module, you can reuse it to define a schema using the `Schema.fromBrand` function. **Example** ### Utilizing Default Constructors The `Schema.brand` function includes a default constructor to facilitate the creation of branded values. ## Property Signatures Property signature combinators control a struct field independently on the encoded and decoded sides. They can make a key optional, allow `undefined`, provide defaults, attach key-level annotations, or rename the encoded key. ### Basic Usage A property signature can be defined with annotations to provide additional context about a field. **Example** Use `Schema.annotateKey` for field metadata. When the external representation uses a different key, use `Schema.encodeKeys` on the struct. **Example** ### Optional Fields #### Basic Optional Property `Schema.optional` makes a key optional and allows `undefined` when the key is present. ##### Decoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `undefined` | remains `undefined` | | `e: E` | transforms to `t: T` | ##### Encoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `undefined` | remains `undefined` | | `t: T` | transforms back to `e: E` | **Example** #### Optional with Nullability Combine `Schema.optional`, `Schema.NullOr`, and an optional-field transformation when `null` should be treated as a missing value. ##### Decoding | Input | Output | | ----------------- | ------------------------------- | | `` | remains `` | | `undefined` | remains `undefined` | | `null` | transforms to `` | | `e: E` | transforms to `t: T` | ##### Encoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `undefined` | remains `undefined` | | `t: T` | transforms back to `e: E` | **Example** #### Exact Optional Key `Schema.optionalKey` makes the key optional without adding `undefined` to its value type. If the key is present, its value must be accepted by the wrapped schema. ##### Decoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `undefined` | `SchemaError` | | `e: E` | transforms to `t: T` | ##### Encoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `t: T` | transforms back to `e: E` | **Example** #### Exact Optional Key with Nullability Combine `Schema.optionalKey`, `Schema.NullOr`, and an optional-field transformation when `null` should be treated as a missing key but `undefined` should still be rejected. ##### Decoding | Input | Output | | ----------------- | ------------------------------- | | `` | remains `` | | `null` | transforms to `` | | `undefined` | `SchemaError` | | `e: E` | transforms to `t: T` | ##### Encoding | Input | Output | | ----------------- | ------------------------- | | `` | remains `` | | `t: T` | transforms back to `e: E` | **Example** ### Representing Optional Fields with never Type When creating a schema to replicate a TypeScript type that includes optional fields with the `never` type, like: the handling of these fields depends on the `exactOptionalPropertyTypes` setting in your `tsconfig.json`. This setting affects whether the schema should treat optional `never`-typed fields as simply absent or allow `undefined` as a value. **Example** When this feature is turned off, you can employ the `Schema.optional` function. This approach allows the field to implicitly accept `undefined` as a value. **Example** When this feature is turned on, use `Schema.optionalKey` so that the field can only be absent. ### Default Values Use `Schema.withDecodingDefaultType` or `Schema.withDecodingDefaultTypeKey` to provide a decoded default. Constructor defaults are independent and can be added with `Schema.withConstructorDefault`. #### Basic Default This is the simplest use case. If the input is missing or `undefined`, the default value will be applied. | Operation | Behavior | | ------------ | ---------------------------------------------------------------- | | **Decoding** | Applies the default value if the input is missing or `undefined` | | **Encoding** | Transforms the input `t: T` back to `e: E` | **Example** #### Default for a Missing Key Use `Schema.withDecodingDefaultTypeKey` when the default should apply only if the key is missing, not when its value is `undefined`. | Operation | Behavior | | ------------ | ------------------------------------------------------ | | **Decoding** | Applies the default value only if the input is missing | | **Encoding** | Transforms the input `t: T` back to `e: E` | **Example** #### Default with Nullability Combine an optional nullable field with `SchemaGetter.transformOptional` when missing, `undefined`, and `null` should all produce the default. | Operation | Behavior | | ------------ | -------------------------------------------------------------------------- | | **Decoding** | Applies the default value if the input is missing or `undefined` or `null` | | **Encoding** | Transforms the input `t: T` back to `e: E` | **Example** #### Default for a Missing or Null Key Use an exact optional nullable field when missing and `null` should produce the default but `undefined` should be rejected. | Operation | Behavior | | ------------ | ----------------------------------------------------------- | | **Decoding** | Applies the default value if the input is missing or `null` | | **Encoding** | Transforms the input `t: T` back to `e: E` | **Example** ### Optional Fields as Options When working with optional fields, you may want to handle them as values. This approach allows you to explicitly manage the presence or absence of a field rather than relying on `undefined` or `null`. #### Basic Optional with Option Type `Schema.OptionFromOptional` converts a missing or `undefined` field to `Option.none` and an existing value to `Option.some`. ##### Decoding | Input | Output | | ----------------- | --------------------------------- | | `` | transforms to `Option.none` | | `undefined` | transforms to `Option.none` | | `e: E` | transforms to `Option.some` | ##### Encoding | Input | Output | | ------------------- | ------------------------------- | | `Option.none` | transforms to `` | | `Option.some` | transforms back to `e: E` | **Example** #### Exact Optional Key as an Option `Schema.OptionFromOptionalKey` converts a missing key to `Option.none` while rejecting `undefined` when the key is present. ##### Decoding | Input | Output | | ----------------- | --------------------------------- | | `` | transforms to `Option.none` | | `undefined` | `SchemaError` | | `e: E` | transforms to `Option.some` | ##### Encoding | Input | Output | | ------------------- | ------------------------------- | | `Option.none` | transforms to `` | | `Option.some` | transforms back to `e: E` | **Example** #### Optional with Nullability `Schema.OptionFromOptionalNullOr` also treats `null` as `Option.none`. ##### Decoding | Input | Output | | ----------------- | --------------------------------- | | `` | transforms to `Option.none` | | `undefined` | transforms to `Option.none` | | `null` | transforms to `Option.none` | | `e: E` | transforms to `Option.some` | ##### Encoding | Input | Output | | ------------------- | ------------------------------- | | `Option.none` | transforms to `` | | `Option.some` | transforms back to `e: E` | **Example** #### Exact Optional Nullable Key as an Option Combine `Schema.optionalKey`, `Schema.NullOr`, and `SchemaGetter.transformOptional` when missing and `null` should become `Option.none` but `undefined` should be rejected. ##### Decoding | Input | Output | | ----------------- | --------------------------------- | | `` | transforms to `Option.none` | | `undefined` | `SchemaError` | | `null` | transforms to `Option.none` | | `e: E` | transforms to `Option.some` | ##### Encoding | Input | Output | | ------------------- | ------------------------------- | | `Option.none` | transforms to `` | | `Option.some` | transforms back to `e: E` | **Example** ## Optional Field Transformations ### Optional to Optional Use `Schema.decodeTo` with `SchemaGetter.transformOptional` to transform an optional encoded field into an optional decoded field. This lets the transformation decide whether the key is present on either side. One common use case is treating a specific encoded value, such as an empty string, as an absent field in the decoded output. The decoding and encoding getters receive an `Option`: `None` means the key is absent, and returning `None` omits it from the output. **Example** Consider an optional field of type `string` where empty strings in the input should be removed from the output. You can simplify the decoding logic with `Option.filter`, which filters out unwanted values in a concise way. **Example** ### Optional to Required Use an optional schema on the encoded side and a required schema on the decoded side. `SchemaGetter.transformOptional` can supply a value when the encoded key is missing and omit selected values during encoding. **Example** This example provides a `null` value when the encoded field is missing. During encoding, a decoded `null` value omits the field. You can streamline the decoding and encoding logic using `Option.getOrElse` and `Option.liftPredicate` for concise and readable transformations. **Example** ### Required to Optional Use a required schema on the encoded side and an optional schema on the decoded side. The transformation can omit selected decoded values and must restore a required value during encoding. **Example** In this example, the `name` field is required but treated as optional if it is an empty string. During decoding, an empty string in `name` is considered absent, while encoding ensures a value . You can streamline the decoding and encoding logic using `Option.liftPredicate` and `Option.getOrElse` for concise and readable transformations. **Example** ## Extending Schemas Struct schemas expose their `fields`, which you can spread into a new struct or extend with `Schema.fieldsAssign`. Unions expose `mapMembers`, so the same field operation can be applied to every struct member. ### Spreading Struct fields Structs provide access to their fields through the `fields` property, which allows you to extend an existing struct by adding additional fields or combining fields from multiple structs. **Example** **Example** **Example** ### The fieldsAssign Function `Schema.fieldsAssign` is a concise form of `struct.mapFields)`. Use it directly on a struct, or map it over every member of a union. **Example** ## Renaming Properties ### Renaming a Property During Definition To use a different key in the encoded representation, apply `Schema.encodeKeys` after defining the struct. **Example** **Example** ### Renaming Properties of an Existing Schema For an existing struct, rename its decoded fields with `mapFields`, then use `Schema.encodeKeys` to preserve the original names in the encoded representation. For a union, apply the same operation to each member. **Example** **Example** ## Recursive Schemas The `Schema.suspend` function is designed for defining schemas that reference themselves, such as in recursive data structures. **Example** In this example, the `Category` schema references itself through the `subcategories` field, which is an array of `Category` objects. **Example** ### A Helpful Pattern to Simplify Schema Definition As we've observed, it's necessary to define an interface for the `Type` of the schema to enable recursive schema definition, which can complicate things and be quite tedious. One pattern to mitigate this is to **separate the field responsible for recursion** from all other fields. **Example** ### Mutually Recursive Schemas You can also use `Schema.suspend` to create mutually recursive schemas, where two schemas reference each other. In the following example, `Expression` and `Operation` form a simple arithmetic expression tree by referencing each other. **Example** ### Recursive Types with Different Encoded and Type Defining a recursive schema where the `Encoded` type differs from the `Type` type adds another layer of complexity. In such cases, we need to define two interfaces: one for the `Type` type, as seen previously, and another for the `Encoded` type. **Example** Let's consider an example with an `id` field defined by `Schema.FiniteFromString`. Its `Type` is `number`, while its `Encoded` type is `string`. When we add this field to the `Category` schema, TypeScript raises an error: This fails because `Schema.Codec` defaults the encoded type to `Category`. The recursive edge must also specify `CategoryEncoded`: --- title: Schema Annotations description: Learn how to enhance schemas with annotations for better customization, error handling, documentation, and concurrency control in your Effect-based applications. sidebar: label: Annotations order: 8 --- Schema AST nodes can carry optional metadata called annotations. Use the `.annotate` method or `Schema.annotate` for the decoded side, `Schema.annotateEncoded` for the encoded side, and `Schema.annotateKey` for a struct field or tuple element. **Example** ## Built-in Annotations The available annotations depend on the kind of schema node. These are the most common ones: | Annotation | Scope | Description | | ---------------------- | --------------------- | ------------------------------------------------------------------------------------------------------ | | `identifier` | schema | Stable name used by schema interpreters, including JSON Schema references and expected-value messages. | | `expected` | schema or check | Human-readable description used by the default error formatter. | | `title` | schema or key | Short display title, also understood by JSON Schema tooling. | | `description` | schema or key | Longer documentation for the represented value. | | `documentation` | schema or key | Additional developer-facing documentation. | | `examples` | schema or key | Example decoded values; this is metadata and does not validate them. | | `default` | schema or key | A documented default value; it does not change decoding or construction behavior. | | `message` | schema or check | Replaces the default message for matching failures. | | `messageMissingKey` | key | Replaces the error message when a required key is absent. | | `messageUnexpectedKey` | schema | Replaces the message for excess keys when `onExcessProperty` is `"error"`. | | `parseOptions` | schema | Overrides for that schema node. | | `toJsonSchema` | check | Describes a custom check to the interpreter. | | `toArbitrary` | schema or declaration | Customizes generation. | | `toFormatter` | declaration | Defines behavior for a custom declaration. | | `toEquivalence` | declaration | Defines behavior for a custom declaration. | | `toCodecJson` | declaration | Defines how a custom declaration is represented by the JSON codec interpreter. | ## Concurrency Parse Option For schemas like `Struct`, `Array`, or `Union` that contain multiple effectful schemas, the `concurrency` parse option controls how many parsing effects may run concurrently. Here's a shorter version presented in a table: | Value | Description | | ------------- | ---------------------------------------------- | | `number` | Limits the maximum number of concurrent tasks. | | `"unbounded"` | All tasks run concurrently with no limit. | | `undefined` | At most one task runs at a time . | **Example** In this example, we define three tasks that simulate asynchronous operations with different durations. Since no concurrency is specified, the tasks are executed sequentially, one after the other. **Example** By passing `{ concurrency: "unbounded" }` to the interpreter, the tasks can run concurrently instead of waiting for one another. ## Handling Decoding Errors with Fallbacks `Schema.catchDecoding` lets you recover from a decoding issue with fallback logic. This annotation enables you to specify fallback behavior when decoding fails, making it possible to recover gracefully from errors. **Example** In this basic example, when decoding fails , the fallback value is returned instead of an error. **Example** In this advanced example, when a decoding error occurs, the schema logs the issue and then returns a fallback value. This demonstrates how you can incorporate logging and other side effects during error handling. ## Custom Annotations In addition to built-in annotations, you can define custom annotations to meet specific requirements. For instance, here's how to create a `deprecated` annotation: **Example** To make your new custom annotation type-safe, you can use a module augmentation. In the next example, we want our custom annotation to be a boolean. **Example** You can retrieve custom annotations using the `Schema.resolveAnnotations` helper function. **Example** --- title: Schema to Arbitrary description: Derive fast-check arbitraries from schemas and customize generation with filters, candidates, and annotations. sidebar: label: Arbitrary order: 15 --- import { Aside } from "@astrojs/starlight/components" `Schema.toArbitrary` derives a `Arbitrary` that generates values of a schema's `Type`. **Example** Use `Schema.toArbitraryLazy` when the caller should provide the fast-check module. **Example** `Schema.Never` and declaration schemas without a `toArbitrary` annotation cannot be derived automatically. Impossible constraints and recursive schemas without a finite terminal path also fail immediately. ## Filters Generated values are always checked by the schema's type-side filters before they are returned. Built-in filters also provide metadata that lets derivation choose an efficient generator instead of relying only on fast-check discards. **Example** Length, range, integer, pattern, uniqueness, and other built-in constraints are translated to the corresponding fast-check constraints where possible. ### Opaque Filters and Reports A custom filter without arbitrary metadata is still correct because every generated value is checked. It may be inefficient when valid values are rare. Pass `{ report: true }` to find filters that could not guide generation. An `OpaqueFilter` warning means that the predicate is enforced but does not help construct the base arbitrary. **Example** Reports contain warnings only. Unsupported schemas, impossible constraints, invalid candidates, and invalid recursion still throw during derivation. ### Custom Filters with Constraints If a custom filter can partially describe its valid values using normal generation constraints, attach an `arbitrary.constraint` annotation. The predicate remains the final authority. **Example** The constraint avoids non-integers and numbers below `2`; the filter still checks primality. ### Custom Filters with Candidates Use a candidate when a filter cannot be expressed with the constraint vocabulary. Candidates are weighted alternatives to the base generator, and their values are still checked by every filter. **Example** The base generator has weight `1`; a candidate also defaults to `1` unless you provide another positive integer. ## Transformations `Schema.toArbitrary` generates the schema's `Type`, not its `Encoded`. For a codec, derivation therefore follows the type-side schema and its constraints. **Example** If you need encoded values, generate `Schema.toEncoded` instead. ## Schema-Level Overrides Use a `toArbitrary` annotation to replace the generator for a schema node. Put the override on the base schema when possible, before adding filters, so the filters remain independent final checks. **Example** Avoid placing an override after filters unless it intentionally handles those filters. For example, an override that always produces `""` cannot satisfy a preceding `Schema.isNonEmpty` check and will exhaust fast-check's discard budget. --- title: Basic Usage description: Learn to define and work with basic schemas, including primitives, literals, unions, and structs, for effective data validation and transformation. sidebar: order: 3 --- import { Aside } from "@astrojs/starlight/components" ## Primitives The Schema module provides built-in schemas for common primitive types. | Schema | Equivalent TypeScript Type | | ---------------------- | -------------------------- | | `Schema.String` | `string` | | `Schema.Finite` | `number` | | `Schema.Boolean` | `boolean` | | `Schema.BigInt` | `bigint` | | `Schema.Symbol` | `symbol` | | `Schema.ObjectKeyword` | `object` | | `Schema.Undefined` | `undefined` | | `Schema.Void` | `void` | | `Schema.Any` | `any` | | `Schema.Unknown` | `unknown` | | `Schema.Never` | `never` | **Example** ## revealCodec To make it easier to work with schemas, built-in schemas are exposed with shorter, opaque types when possible. The `Schema.revealCodec` function returns the same schema value widened to its full `Codec` view, prompting TypeScript to infer all four parameters without any runtime cost. **Example** For example, `Schema.String` has the concrete type `typeof Schema.String`. Passing it to `Schema.revealCodec` exposes its full view as `Codec`. ## Unique Symbols You can create a schema for unique symbols using `Schema.UniqueSymbol`. **Example** ## Literals Literal schemas represent a . You can use them to specify exact values that a type must have. Literals can be of the following types: - `string` - `number` - `boolean` - `null` - `bigint` **Example** **Example** ### Union of Literals You can create a union of multiple literals by passing them as arguments to the `Schema.Literals` constructor: **Example** You can annotate the entire union to replace its default error message ). **Example** ### Exposed Values You can access the literals defined in a literal schema using the `literals` property: ### Picking Literals You can use the `.pick` method of a `Schema.Literals` value to narrow down its possible values. **Example** Sometimes, you may need to reuse a literal schema in other parts of your code. Below is an example demonstrating how to do this: **Example** In this example, `FruitCategory` serves as the source of truth for the different fruit categories. We reuse it to create a subtype of `Fruit` called `SweetAndCitrusFruit`, ensuring that only the specified categories are allowed. This approach helps maintain consistency throughout your code and provides type safety if the category definition changes. ## Template literals In TypeScript, allow you to embed expressions within string literals. The `Schema.TemplateLiteral` constructor allows you to create a schema for these template literal types. **Example** **Example** Documentation) Let's look at a more complex example. Suppose you have two sets of locale IDs for emails and footers. You can use the `Schema.TemplateLiteral` constructor to create a schema that combines these IDs: ### Supported Span Types The `Schema.TemplateLiteral` constructor supports the following types of spans: - `Schema.String` - `Schema.Finite` - Literals: `string | number | boolean | null | bigint`. These can be either wrapped by `Schema.Literal` or used directly - Unions of the above types - Brands of the above types **Example** ### TemplateLiteralParser The `Schema.TemplateLiteral` constructor, while useful as a simple validator, only verifies that an input conforms to a specific string pattern by converting template literal definitions into regular expressions. Similarly, uses regular expressions directly for the same purpose. Post-validation, both approaches require additional manual parsing to convert the validated string into a usable data format. To address these limitations and eliminate the need for manual post-validation parsing, the `Schema.TemplateLiteralParser` API has been developed. It not only validates the input format but also automatically parses it into a more structured and type-safe output, specifically into a **tuple** format. The `Schema.TemplateLiteralParser` constructor supports the same types of as `Schema.TemplateLiteral`. **Example** ## Native enums The Schema module provides support for native TypeScript enums. You can define a schema for an enum using `Schema.Enum`, allowing you to validate values that belong to the enum. **Example** ### Exposed Values Enums are accessible through the `enums` property of the schema. You can use this property to retrieve individual members or the entire set of enum values. ## Unions The Schema module includes a built-in `Schema.Union` constructor for creating "OR" types, allowing you to define schemas that can represent multiple types. **Example** ### Union Member Evaluation Order When decoding, union members are evaluated in the order they are defined. If a value matches the first member, it will be decoded using that schema. If not, the decoding process moves on to the next member. If multiple schemas could decode the same value, the order matters. Placing a more general schema before a more specific one may result in missing properties, as the first matching schema will be used. **Example** ### Union of Literals While you can create a union of literals by combining individual literal schemas: **Example** You can simplify the process by passing multiple literals directly to the `Schema.Literals` constructor: **Example** You can annotate the entire union to replace its default error message ). **Example** ### Nullables The Schema module includes utility functions for defining schemas that allow nullable types, helping to handle values that may be `null`, `undefined`, or both. **Example** ### Discriminated unions in TypeScript are a way of modeling complex data structures that may take on different forms based on a specific set of conditions or properties. They allow you to define a type that represents multiple related shapes, where each shape is uniquely identified by a shared discriminant property. In a discriminated union, each variant of the union has a common property, called the discriminant. The discriminant is a literal type, which means it can only have a finite set of possible values. Based on the value of the discriminant property, TypeScript can infer which variant of the union is currently in use. **Example** In the `Schema` module, you can define a discriminated union similarly by specifying a literal field as the discriminant for each type. **Example** In this example, the `Schema.Literal` constructor sets up the `kind` property as the discriminant for both `Circle` and `Square` schemas. The `Shape` schema then represents a union of these two types, allowing TypeScript to infer the specific shape based on the `kind` value. ### Transforming a Simple Union into a Discriminated Union If you start with a simple union and want to transform it into a discriminated union, you can add a special property to each member. This allows TypeScript to automatically infer the correct type based on the value of the discriminant property. **Example** For example, let's say you've defined a `Shape` union as a combination of `Circle` and `Square` without any special property: To make your code more manageable, you may want to transform the simple union into a discriminated union. This way, TypeScript will be able to automatically determine which member of the union you're working with based on the value of a specific property. To achieve this, you can add a special property to each member of the union, which will allow TypeScript to know which type it's dealing with at runtime. Here's how you can the `Shape` schema into another schema that represents a discriminated union: **Example** The previous solution works but requires a lot of boilerplate. You can add the discriminant with `mapFields` and use `Schema.tagDefaultOmit` to provide its decoded default while omitting it during encoding: **Example** ### Exposed Values You can access the individual members of a union schema represented as a tuple: ## Tuples The Schema module allows you to define tuples, which are ordered collections of elements that may have different types. You can define tuples with required, optional, or rest elements. ### Required Elements To define a tuple with required elements, you can use the `Schema.Tuple` constructor and simply list the element schemas in order: **Example** ### Append a Required Element You can append additional required elements to an existing tuple by using the spread operator: **Example** ### Optional Elements To define an optional element, use the `Schema.optionalKey` constructor. **Example** ### Rest Element To define a rest element, add it after the list of required or optional elements. The rest element allows the tuple to accept additional elements of a specific type. **Example** You can also include other elements after the rest: **Example** ### Element Annotations Annotations are useful for adding metadata to tuple elements, making it easier to describe their purpose or requirements. This is especially helpful for generating documentation or JSON schemas. **Example** ### Exposed Values You can access the elements and rest elements of a tuple schema using the `elements` and `rest` properties: **Example** ## Arrays The Schema module allows you to define schemas for arrays, making it easy to validate collections of elements of a specific type. **Example** ### Mutable Arrays By default, `Schema.Array` generates a type marked as `readonly`. To create a schema for a mutable array, you can use the `Schema.mutable` function, which makes the array type mutable in a **shallow** manner. **Example** ### Exposed Values You can access the value type of an array schema using the `value` property: **Example** ## Non Empty Arrays The Schema module also provides a way to define schemas for non-empty arrays, ensuring that the array always contains at least one element. **Example** ### Exposed Values You can access the value type of a non-empty array schema using the `value` property: **Example** ## Records The Schema module provides support for defining record types, which are collections of key-value pairs where the key can be a string, symbol, or other types, and the value has a defined schema. ### String Keys You can define a record with string keys and a specified type for the values. **Example** ### Symbol Keys Records can also use symbols as keys. **Example** ### Union of Literal Keys Use a union of literals to restrict keys to a specific set of values. **Example** ### Template Literal Keys Records can use template literals as keys, allowing for more complex key patterns. **Example** ### Refined Keys You can refine the key type with additional constraints. **Example** Refinements on keys act as filters rather than causing a decoding failure. If a key does not meet the constraints , it is removed from the decoded output instead of triggering an error. **Example** If you want decoding to fail when a key does not meet the constraints, you can set to `"error"`. **Example** ### Transforming Keys The `Schema.Record` API does not support transformations on key schemas. Attempting to apply a transformation to keys will result in an `Unsupported key schema` error: **Example** To modify record keys, you must apply transformations outside of `Schema.Record`. A common approach is to use with `SchemaTransformation.transform` to adjust keys during decoding. **Example** ### Mutable Records By default, `Schema.Record` generates a type marked as `readonly`. To create a schema for a mutable record, you can use the `Schema.mutable` function, which makes the record type mutable in a **shallow** manner. **Example** ### Exposed Values You can access the `key` and `value` types of a record schema using the `key` and `value` properties: **Example** ## Structs ### Property Signatures The `Schema.Struct` constructor defines a schema for an object with specific properties. **Example** This example defines a struct schema for an object with the following properties: - `name`: a string - `age`: a number ### Index Signatures Use `Schema.StructWithRest` to combine a struct with one or more records representing index signatures. **Example** ### Multiple Index Signatures You can define **one** index signature per key type . Defining multiple index signatures of the same type is not allowed. **Example** Defining multiple index signatures of the same key type will cause an error. **Example** ### Conflicting Index Signatures When defining schemas with index signatures, conflicts can arise if a fixed property has a different type than the values allowed by the index signature. This can lead to unexpected TypeScript behavior. **Example** The TypeScript compiler flags this as an error when defining the type manually: This happens because TypeScript does not allow an index signature to contradict a fixed property. #### Workaround for Conflicting Index Signatures When working with schemas, a conflict can occur if a fixed property has a different type than the values allowed by an index signature. This situation often arises when dealing with external APIs that do not follow strict TypeScript conventions. To prevent conflicts, you can separate the fixed properties from the indexed properties and handle them as distinct parts of the schema. **Example** Consider an object where: - `"a"` is a fixed property of type `string`. - All other keys store numbers, which conflict with `"a"`. To avoid this issue, we can separate the properties into two distinct types: By using and `SchemaTransformation.transform`, you can preprocess the input data before decoding. This approach ensures that fixed properties and index signature properties are treated independently. ### Exposed Values You can access the fields and records of a struct schema using the `fields` and `records` properties: **Example** ### Mutable Structs By default, `Schema.Struct` generates a type with properties marked as `readonly`. To create a mutable version of the struct, use the `Schema.mutable` function, which makes the properties mutable in a **shallow** manner. **Example** ## Tagged Structs In TypeScript tags help to enhance type discrimination and pattern matching by providing a simple yet powerful way to define and recognize different data types. ### What is a Tag? A tag is a literal value added to data structures, commonly used in structs, to distinguish between various object types or variants within tagged unions. This literal acts as a discriminator, making it easier to handle and process different types of data correctly and efficiently. ### Using the tag Constructor The `Schema.tag` constructor is specifically designed to create a property signature that holds a specific literal value, serving as the discriminator for object types. **Example** In the example above, `Schema.tag` attaches a `_tag` property to the `User` struct schema, effectively labeling objects of this struct type as "User". This label is automatically applied when using the `make` method to create new instances, simplifying object creation and ensuring consistent tagging. ### Simplifying Tagged Structs with TaggedStruct The `Schema.TaggedStruct` constructor streamlines the process of creating tagged structs by directly integrating the tag into the struct definition. This method provides a clearer and more declarative approach to building data structures with embedded discriminators. **Example** In this example: - The `_tag` property is optional when constructing an instance with `make`, allowing the schema to automatically apply it. - When decoding unknown data, `_tag` is required to ensure correct type identification. This distinction between instance construction and decoding is useful for preserving the tag’s role as a type discriminator while simplifying instance creation. If you need `_tag` to be applied automatically during decoding as well, you can create a customized version of `Schema.TaggedStruct`: **Example** ### Multiple Tags While a primary tag is often sufficient, TypeScript allows you to define multiple tags for more complex data structuring needs. Here's an example demonstrating the use of multiple tags within a single struct: **Example** This example defines a product schema with a primary tag and an additional category tag , adding further specificity to the data structure. ## instanceOf When you need to define a schema for your custom data type defined through a `class`, the most convenient and fast way is to use the `Schema.instanceOf` constructor. **Example** The `Schema.instanceOf` constructor is just a lightweight wrapper of the API, which is the primitive in `effect/Schema` for declaring new custom data types. ### Private Constructors Note that `Schema.instanceOf` can only be used for classes that expose a **public constructor**. If you try to use it with classes that, for some reason, have marked the constructor as `private`, you'll receive a TypeScript error: **Example** In such cases, you cannot use `Schema.instanceOf`, and you must rely on like this: **Example** ### Validating Fields of the Instance To validate the fields of a class instance, you can use a . This approach combines instance validation with additional checks on the instance's fields. **Example** ## Picking Use `Struct.pick` with `Struct.mapFields` to create a new struct schema by selecting fields from an existing one. **Example** ## Omitting Use `Struct.omit` with `Struct.mapFields` to create a new struct schema by excluding fields from an existing one. **Example** ## Making Properties Optional Use `Struct.map` with `Schema.optional` to make every field in a struct optional. **Example** `Schema.optional` adds `undefined` to each field type. Use `Schema.optionalKey` instead when a field may be omitted but, when present, must contain a value accepted by its schema. **Example** ## Making Properties Required Use `Struct.map` with `Schema.requiredKey` to make every optional key in a struct required. **Example** In this example, both `a` and `b` are made required, even though they were initially defined as optional. --- title: Class APIs description: Define and extend schema-backed classes with validated constructors, methods, equality, and recursive fields. sidebar: order: 11 --- import { Aside } from "@astrojs/starlight/components" When working with schemas, you can use `Schema.Class` instead of a plain when your domain model benefits from class instances. Classes offer several features that simplify the schema creation process: - **Schema and class in one definition**: The class itself can be used anywhere a schema is expected. - **Validated construction**: The constructor and `make` check their input. - **Shared behavior**: Instances can expose methods and getters. - **Structural equality**: Instances can be compared with . ## Definition To define a class using `Schema.Class`, you need to specify: - The class type as the `Self` type parameter. - A stable identifier used in diagnostics and schema metadata. - The fields of the class. **Example** In this example, `Person` is both a schema and a TypeScript class. **Example** ### Class Schemas are Transformations Class schemas a struct schema into a schema that represents the class. - When decoding, a plain object is converted into an instance of the class. - When encoding, a class instance is converted back into a plain object. **Example** ### Defining Classes Without Fields When your schema does not require any fields, you can define a class with an empty object. **Example** ### Defining Classes With Filters Filters allow you to validate input when decoding, encoding, or creating an instance. Instead of specifying raw fields, you can pass a `Schema.Struct` with a filter applied. **Example** ## Validating Properties via Class Constructors When you define a class using `Schema.Class`, the constructor automatically checks that the provided properties adhere to the schema's rules. ### Defining and Instantiating a Valid Class Instance The constructor ensures that each property, like `id` and `name`, adheres to the schema. For instance, `id` must be a number, and `name` must be a non-empty string. **Example** ### Handling Invalid Properties If invalid properties are provided during instantiation, the constructor throws an error, explaining why the validation failed. **Example** The error clearly specifies that the `name` field failed to meet the `NonEmptyString` requirement. ### Bypassing Checks In some scenarios, you might want to bypass the validation logic. While not generally recommended, the library provides an option to do so. **Example** ## Structural Equality compares class instances structurally, including nested objects and arrays. **Example** ## Extending Classes with Custom Logic Schema classes provide the flexibility to include custom getters and methods, allowing you to extend their functionality beyond the defined fields. ### Adding Custom Getters A getter can be used to derive computed values from the fields of the class. For example, a `Person` class can include a getter to return the `name` property in uppercase. **Example** ### Adding Custom Methods In addition to getters, you can define methods to encapsulate more complex logic or operations involving the class's fields. **Example** ## Leveraging Classes as Schema Definitions When you define a class with `Schema.Class`, it serves both as a schema and as a class. This dual functionality allows the class to be used wherever a schema is required. **Example** ### Exposed Values The class also includes a `fields` static property, which outlines the fields defined during the class creation. **Example** ## Adding Annotations Pass annotations as the second argument after the fields or struct. The identifier passed to `Schema.Class` is also stored as the default `identifier` annotation. **Example** ## Recursive Schemas The `Schema.suspend` combinator is useful when you need to define a schema that depends on itself, like in the case of recursive data structures. In this example, the `Category` schema depends on itself because it has a field `subcategories` that is an array of `Category` objects. **Example** **Example** ### Mutually Recursive Schemas Sometimes, schemas depend on each other in a mutually recursive way. For instance, an arithmetic expression tree might include `Expression` nodes that can either be numbers or `Operation` nodes, which in turn reference `Expression` nodes. **Example** ### Recursive Types with Different Encoded and Type Defining recursive schemas where the `Encoded` type differs from the `Type` type requires an explicit encoded representation. For instance, `FiniteFromString` has `number` as its `Type` and `string` as its `Encoded`. In such cases, we need to define an interface for the `Encoded` type. Let's add an `id` field to the `Category` schema using `FiniteFromString`. When we add this field to the `Category` schema, TypeScript raises an error: The `Schema.Codec` annotation assumes that both the `Type` and `Encoded` are `Category`. Supply the recursive encoded type as the second parameter instead: **Example** As we've observed, it's necessary to define an interface for the `Encoded` of the schema to enable recursive schema definition, which can complicate things and be quite tedious. One pattern to mitigate this is to **separate the field responsible for recursion** from all other fields. **Example** ## Tagged Class Variants `Schema.TaggedClass` automatically adds a `_tag` field, while `Schema.TaggedError` also creates a yieldable `Error`. Both use the tag as their identifier by default. **Example** ## Extending Existing Classes The `extend` static utility allows you to enhance an existing schema class by adding **additional** fields and functionality. This approach helps in building on top of existing schemas without redefining them from scratch. **Example** --- title: Default Constructors description: Construct schema values with make, makeOption, makeEffect, validation options, and default values. sidebar: order: 12 --- import { Aside } from "@astrojs/starlight/components" Every schema exposes constructors for creating values of its `Type` while applying constructor defaults and type-side checks. Use `make` when a failure should throw, `makeOption` when you only need to know whether construction succeeded, and `makeEffect` when you need the `SchemaError` in the `Effect` error channel. **Example** ## Structs Struct schemas allow you to define objects with specific fields and constraints. The `make` function can be used to create instances of a struct schema. **Example** When the input is already trusted, `make` can skip schema checks. This is not recommended for untrusted values. **Example** ## Records Record schemas allow you to define key-value mappings where the keys and values must meet specific criteria. **Example** ## Filters Filters allow you to define constraints on individual values. **Example** ## Branded Types Branded schemas add metadata to a value to give it a more specific type, while still retaining its original type. **Example** When using default constructors, it is helpful to understand the type of value they produce. For instance, in the `BrandedNumberSchema` example, the return type of the constructor is `number & Brand<"MyNumber">`. This indicates that the resulting value is a `number` with additional branding information, `"MyNumber"`. This behavior contrasts with the filter example, where the return type is simply `number`. Branding adds an extra layer of type information, which can assist in identifying and working with your data more effectively. ## Error Handling in Constructors `make` is appropriate when invalid constructor input is exceptional. If failure is expected, use `makeOption` or `makeEffect` instead. `makeOption` returns `Option.some` on success and `Option.none` for schema issues. Use `makeEffect` when you need the full `SchemaError`. **Example** ## Setting Default Values When creating objects, you might want to assign default values to certain fields to simplify object construction. The `Schema.withConstructorDefault` function lets you handle default values, making fields optional in the default constructor. **Example** In this example, all fields are required when creating a new instance. **Example** Here, the `age` field is optional because it has a default value of `0`. ### Nested Defaults Constructor defaults compose through nested schemas. Inner defaults are resolved after the value for the outer field is supplied or defaulted. **Example** ### Lazy Evaluation of Defaults Defaults are lazily evaluated, meaning that a new instance of the default is generated every time the constructor is called: **Example** In this example, the `timestamp` field generates a new value for each instance. ### Reusing Defaults Across Schemas Default values are also "portable", meaning that if you reuse the same property signature in another schema, the default is carried over: **Example** ### Using Defaults in Classes Default values can also be applied when working with the `Class` API, ensuring consistency across class-based schemas. **Example** --- title: Effect Data Types description: Define schemas for Option, Result, Exit, Effect collections, Duration, Redacted values, and configuration. sidebar: order: 13 --- Effect provides schemas for its runtime data types, including `Option`, `Result`, `Exit`, hash collections, `Duration`, and `Redacted`. These schemas expect the corresponding runtime value on both sides. Their inner schemas can still transform the contained values. When you need a JSON-compatible representation, derive it with `Schema.toCodecJson`. **Example** ## Config Use `Config.schema` to read and decode configuration with a schema. The provider supplies the encoded representation, and the resulting `Config` produces the schema's `Type`. **Example** See for configuration providers, nesting, defaults, and secrets. ## Option `Schema.Option` describes `Option` values and applies `value` to the contents of `Some`. **Example** ### Options from Nullable and Optional Values The following schemas transform common nullable and optional representations into `Option` values: | Schema | Values decoded as `None` | Default encoding of `None` | | -------------------------- | ------------------------------------------ | -------------------------- | | `OptionFromUndefinedOr` | `undefined` | `undefined` | | `OptionFromNullOr` | `null` | `null` | | `OptionFromNullishOr` | `null` or `undefined` | `undefined` | | `OptionFromOptionalKey` | A missing property | A missing property | | `OptionFromOptional` | A missing property or `undefined` | A missing property | | `OptionFromOptionalNullOr` | A missing property, `null`, or `undefined` | A missing property | `OptionFromNullishOr` accepts an `onNoneEncoding` option of `null` or `undefined`. `OptionFromOptionalNullOr` accepts `"omit"`, `null`, or `undefined`. **Example** **Example** ## Result `Schema.Result` describes `Result` values and transforms the success and failure channels independently. **Example** Its default JSON representation uses `{ _tag: "Success", success }` and `{ _tag: "Failure", failure }`. **Example** ## Exit `Schema.Exit` describes `Exit` values. It applies the supplied schemas to successful values, expected failures, and defects. **Example** The JSON representation uses `{ _tag: "Success", value }` for success and `{ _tag: "Failure", cause }` for failure. **Example** `Schema.Defect` converts JSON-compatible defect data back into defects. Objects with `name`, `message`, and optionally `stack` are reconstructed as JavaScript errors. ## Collections Schemas for Effect collections expect collection values on both sides and apply the element schemas during decoding and encoding. Their JSON codecs use arrays of values or key-value entries. ### ReadonlySet **Example** ### ReadonlyMap **Example** ### HashSet **Example** ### HashMap **Example** ## Duration `Schema.Duration` validates existing `Duration` values. Use a transformation schema when the encoded value is a string, a number of milliseconds, or a bigint number of nanoseconds. | Schema | Encoded | Type | | --------------------------- | ---------- | ---------- | | `Schema.Duration` | `Duration` | `Duration` | | `Schema.DurationFromString` | `string` | `Duration` | | `Schema.DurationFromMillis` | `number` | `Duration` | | `Schema.DurationFromNanos` | `bigint` | `Duration` | **Example** The default JSON representation of `Schema.Duration` is a tagged object that preserves milliseconds, nanoseconds, and infinite durations. **Example** ## Redacted `Schema.Redacted` validates existing `Redacted` values and applies `value` to their hidden contents. Use `Schema.RedactedFromValue` to decode a raw value and wrap it in `Redacted`. **Example** The default JSON representation of `Schema.Redacted` exposes the encoded inner value. If a redacted value must never be serialized, set `disallowJsonEncode: true`. **Example** --- title: Schema to Equivalence description: Derive and customize equivalence checks from schema structure. sidebar: label: Equivalence order: 17 --- `Schema.toEquivalence` derives an for a schema's `Type`. Nested values are compared according to the corresponding nested schemas. **Example** For a struct, only fields described by the schema participate in the derived equivalence. Arrays and nested structs are compared recursively. ## Broad Schemas `Schema.Any`, `Schema.Unknown`, `Schema.ObjectKeyword`, and an empty `Schema.Struct` use `Equal.equals`. `Equal.equals` performs deep structural comparison for objects and arrays rather than defaulting to reference equality. **Example** ## Declarations Declarations are opaque to automatic derivation. Provide a `toEquivalence` annotation when the declaration needs behavior other than `Equal.equals`. **Example** Parameterized declarations receive the derived equivalence for each type parameter in the annotation callback. ## Overrides Use `Schema.overrideToEquivalence` to replace the equivalence derived for an existing schema. **Example** --- title: Error Formatters description: Format schema issues as readable strings or Standard Schema V1 issue arrays. sidebar: order: 10 --- The `SchemaIssue` module provides two built-in formatters: a human-readable string formatter and a structured Standard Schema V1 formatter. ## Default String Formatter `SchemaIssue.makeFormatterDefault` returns a multi-line string. `SchemaError.message` uses this formatter, so most applications can read the message directly from the error. **Example** In this example: - `` identifies the specific field causing the error. - `Missing key` describes the issue. ### Handling Multiple Errors By default, decoding functions like `Schema.decodeUnknownResult` report only the first error. To list all errors, use the `{ errors: "all" }` option. **Example** ## Standard Schema V1 Formatter `SchemaIssue.makeFormatterStandardSchemaV1` returns a Standard Schema V1 failure result. Each leaf issue becomes an object with a `message` and a full `path`, making the result convenient for forms and other structured consumers. **Example** In this example: - `path`: Specifies the location of the error in the data . - `message`: Describes the issue . ### Handling Multiple Errors By default, decoding functions like `Schema.decodeUnknownResult` report only the first error. To list all errors, use the `{ errors: "all" }` option. **Example** ### Customizing Messages Pass a `leafHook` to customize terminal issues while delegating the remaining cases to `SchemaIssue.defaultLeafHook`. **Example** ## React Hook Form If you are working with React, `@hookform/resolvers` provides an `effectTsResolver` adapter for React Hook Form. See the for setup instructions and an example. --- title: Error Messages description: Customize and enhance error messages for schema decoding with default, refined, and custom messages. sidebar: order: 9 --- ## Default Error Messages By default, a `SchemaError` formats its issue with a concise message and a path when the failure is nested ). For example, if a required property is missing or a value has the wrong type, the message states what was expected and where the failure occurred. **Example** **Example** **Example** ### Enhancing Clarity in Error Messages with Identifiers In scenarios where a schema has multiple fields or nested structures, the default error messages can become overly complex and verbose. To address this, you can enhance the clarity and brevity of these messages by utilizing annotations such as `identifier`, `title`, and `description`. **Example** ### Checks A check runs only after the base schema accepts the input. Base-type failures and check failures therefore have distinct messages. **Example** ### Transformations Transformations between different types or formats can occasionally result in errors. The system provides a structured error message to specify where the error occurred: - **Encoded Side Failure:** Errors on this side typically indicate that the input to the transformation does not match the expected initial type or format. For example, receiving a `null` when a `string` is expected. - **Transformation Process Failure:** This type of error arises when the transformation logic itself fails, such as when the input does not meet the criteria specified within the transformation functions. - **Type Side Failure:** Occurs when the output of a transformation does not meet the schema requirements on the decoded side. This can happen if the transformed value fails subsequent validations or conditions. **Example** ## Custom Error Messages Use the `message` annotation to replace the default message for a schema node or check. **Example** **Example** ### General Guidelines for Messages Attach `message` to the node whose failure you want to replace. For a specific check, pass the annotation to that check constructor; annotating a schema after `.check` targets its last check. If a different inner node fails, its own message or default formatting is used. ### Scalar Schemas **Example** ### Checks This example sets a custom message on the last check in a chain. The custom message is used only when `isMaxLength` fails; otherwise, default messages are used. **Example** When several checks have custom messages, the first failing check supplies the message: **Example** ### Transformations In this example, `IntFromString` is a transformation schema that converts strings to integers. It applies specific validation messages based on different scenarios. **Example** ### Compound Schemas The custom message system becomes especially handy when dealing with complex schemas, unlike simple scalar values like `string` or `number`. For instance, consider a schema comprising nested structures, such as a struct containing an array of other structs. Let's explore an example demonstrating the advantage of default messages in handling decoding errors within such nested structures: **Example** ### Missing messages You can provide custom messages for missing fields or tuple elements using the `messageMissingKey` annotation. **Example** In this example, a custom message is defined for a missing `name` property in the `Person` schema. **Example** Here, each element in the `Point` tuple schema has a specific custom message if the element is missing. --- title: Filters description: Define custom validation logic with filters to enhance data validation beyond basic type checks. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" Developers can define custom validation logic beyond basic type checks, giving more control over how data is validated. ## Declaring Filters Create a filter with `Schema.makeFilter`, then add it to a schema with `.check`. The predicate receives a decoded value and returns whether it satisfies the constraint, optionally providing one or more issues when it does not. **Example** Note that the filter does not alter the schema's `Type`: Filters add additional validation constraints without modifying the schema's underlying type. ## The Predicate Function The predicate function in a filter follows this structure: where The filter's predicate can return several types of values, each affecting validation in a different way: | Return Type | Behavior | | ---------------------------- | ------------------------------------------------------------------------------------------------ | | `true` or `undefined` | The data satisfies the filter's condition and passes validation. | | `false` | The data does not meet the condition, and no specific error message is provided. | | `string` | The validation fails, and the provided string is used as the error message. | | `SchemaIssue.Issue` | The validation fails with a detailed error structure, specifying where and why it failed. | | `FilterIssue` | Allows for more detailed error messages with specific paths, providing enhanced error reporting. | | `ReadonlyArray` | An array of issues can be returned if multiple validation errors need to be reported. | ## Adding Annotations Embedding metadata within the schema, such as identifiers, JSON schema specifications, and descriptions, enhances understanding and analysis of the schema's constraints and purpose. **Example** ## Specifying Error Paths When validating forms or structured data, it's possible to associate specific error messages with particular fields or paths. This enhances error reporting and is especially useful when integrating with libraries like . **Example** In this example, we define a `MyForm` schema with two password fields . We use `Schema.makeFilter` to check that both passwords match. If they don't, an error message is returned, specifically associated with the `confirm_password` field. This makes it easier to pinpoint the exact location of the validation failure. The error is formatted as a Standard Schema issue array with `SchemaIssue.makeFormatterStandardSchemaV1`, making it easier to post-process or pass to form libraries. ## Multiple Error Reporting The `Schema.makeFilter` API supports reporting multiple validation issues at once, which is especially useful in scenarios like form validation where several checks might fail simultaneously. **Example** In this example, we define a `MyForm` schema with fields for password validation and optional name/surname fields. The `Schema.makeFilter` function checks if the passwords match and ensures that either a name or surname is provided. If either validation fails, the corresponding error message is associated with the relevant field and both errors are returned in a structured format. ## Built-in Filters ### String Filters Here is a list of useful string filters provided by the Schema module: ### Number Filters Here is a list of useful number filters provided by the Schema module: ### ReadonlyArray Filters Here is a list of useful array filters provided by the Schema module: ### Date Filters ### BigInt Filters Here is a list of useful `BigInt` filters provided by the Schema module: ### BigDecimal Filters Here is a list of useful `BigDecimal` filters provided by the Schema module: ### Duration Filters Here is a list of useful filters provided by the Schema module: --- title: Schema to Formatter description: Generate formatted string representations of values based on schemas. sidebar: label: Formatter order: 18 --- `Schema.toFormatter` derives a human-readable formatter for values of a schema's `Type`. It recursively formats structs, arrays, unions, and declarations; it does not validate the value. This value formatter is distinct from the used for decoding and encoding failures. **Example** ## Customizing Formatter Generation Use `Schema.overrideToFormatter` to replace the formatter derived for an existing schema. **Example** Declarations can instead provide a `toFormatter` annotation when they are defined. Parameterized declarations receive the formatter derived for each type parameter. ## Intercepting AST Nodes Pass an `onBefore` hook to intercept selected AST nodes before the default formatter is derived. Return `undefined` to retain the default behavior. **Example** --- title: Getting Started description: Learn how to define schemas, extract types, and handle decoding and encoding. sidebar: order: 2 --- import { Aside } from "@astrojs/starlight/components" You can import the necessary types and functions from the `effect/Schema` module: **Example** **Example** ## Defining a schema One common way to define a `Schema` is by utilizing the `Struct` constructor. This constructor allows you to create a new schema that outlines an object with specific properties. Each property in the object is defined by its own schema, which specifies the data type and any validation rules. **Example** This `Person` schema describes an object with a `name` and `age` property: ## Extracting Inferred Types ### Type Once you've defined a schema, you can extract its inferred decoded type `T` in two ways: 1. Using the `Schema.Schema.Type` utility 2. Accessing the `Type` field directly on the schema **Example** The resulting type will look like this: Alternatively, you can extract the `Person` type using the `interface` keyword, which may improve readability and performance in some cases. **Example** Both approaches yield the same result, but using an interface provides benefits such as performance advantages and improved readability. ### Encoded For a schema viewed as a `Codec`, the encoded type `E` can differ from the decoded type `T`. You can extract the encoded type in two ways: 1. Using the `Schema.Codec.Encoded` utility 2. Accessing the `Encoded` field directly on the schema **Example** The resulting type is: Note that `age` is of type `string` in the `Encoded` type of the schema and is of type `number` in the `Type` type of the schema. Alternatively, you can define the `PersonEncoded` type using the `interface` keyword, which can enhance readability and performance. **Example** Both approaches yield the same result, but using an interface provides benefits such as performance advantages and improved readability. ### Services A `Codec` tracks its service requirements separately in each direction: `RD` contains the services required for decoding, while `RE` contains those required for encoding. You can extract both types in two ways: 1. Using the `Schema.Codec.DecodingServices` and `Schema.Codec.EncodingServices` utilities. 2. Accessing the `DecodingServices` and `EncodingServices` fields directly on the schema. **Example** ## Readonly Types by Default It's important to note that by default, most constructors exported by `effect/Schema` return `readonly` types. **Example** For instance, in the `Person` schema below: the resulting inferred `Type` would be: ## Decoding When working with unknown data types in TypeScript, decoding them into a known structure can be challenging. Luckily, `effect/Schema` provides several functions to help with this process. Let's explore how to decode unknown values using these functions. | API | Description | | ---------------------- | ---------------------------------------------------------------------------------- | | `decodeUnknownSync` | Synchronously decodes a value and throws an error if parsing fails. | | `decodeUnknownExit` | Decodes a value and returns an . | | `decodeUnknownOption` | Decodes a value and returns an type. | | `decodeUnknownResult` | Decodes a value and returns a type. | | `decodeUnknownPromise` | Decodes a value and returns a `Promise`. | | `decodeUnknownEffect` | Decodes a value and returns an . | ### decodeUnknownSync The `Schema.decodeUnknownSync` function is useful when you want to parse a value and immediately throw an error if the parsing fails. **Example** ### decodeUnknownResult The `Schema.decodeUnknownResult` function allows you to parse a value and receive the result as a , representing success or failure . This approach lets you handle parsing errors more gracefully without throwing exceptions. **Example** ### decodeUnknownEffect If a schema contains asynchronous transformations, the `Sync`, `Option`, `Result`, and `Exit` interpreters cannot execute them. Use `Schema.decodeUnknownEffect` or `Schema.decodeUnknownPromise` instead. **Example** In the code above, the first approach using `Schema.decodeUnknownExit` results in an error indicating that the transformation cannot be resolved synchronously. This occurs because `Schema.decodeUnknownExit` is not designed for async operations. The second approach, which uses `Schema.decodeUnknownEffect`, works correctly, allowing you to handle asynchronous transformations and return the expected result. ## Encoding The `Schema` module provides several `encode*` functions to encode data according to a schema: | API | Description | | --------------- | ----------------------------------------------------------------------------------------------------- | | `encodeSync` | Synchronously encodes data and throws an error if encoding fails. | | `encodeExit` | Encodes data and returns an . | | `encodeOption` | Encodes data and returns an type. | | `encodeResult` | Encodes data and returns a type representing success or failure. | | `encodePromise` | Encodes data and returns a `Promise`. | | `encodeEffect` | Encodes data and returns an . | **Example** Note that during encoding, the number value `30` was converted to a string `"30"`. ## SchemaError The `Schema.decodeUnknownResult` and `Schema.encodeResult` functions return a , with different success types for each direction: where `SchemaError` is defined as follows : In this structure, `SchemaIssue.Issue` represents an error that might occur during decoding or encoding. It is wrapped in a tagged error to make it easier to catch errors using . Decoding succeeds with the decoded type `T`, while encoding succeeds with the encoded type `E`. In either direction, a schema mismatch produces a `Failure` containing a `SchemaError`. ## Parse Options The options below provide control over both decoding and encoding behaviors. ### Managing Excess Properties By default, any properties not defined in the schema are removed from the output when parsing a value. This ensures the parsed data conforms strictly to the expected structure. If you want to detect and handle unexpected properties, use the `onExcessProperty` option , which allows you to raise an error for excess properties. This can be helpful when you need to validate and catch unanticipated properties. **Example** To retain extra properties, set `onExcessProperty` to `"preserve"`. **Example** ### Receiving All Errors The `errors` option enables you to retrieve all errors encountered during parsing. By default, only the first error is returned. Setting `errors` to `"all"` provides comprehensive error feedback, which can be useful for debugging or offering detailed validation feedback. **Example** ### Managing Property Order The `propertyOrder` option provides control over the order of object fields in the output. This feature is particularly useful when the sequence of keys is important for the consuming processes or when maintaining the input order enhances readability and usability. By default, the `propertyOrder` option is set to `"none"`. This means that the internal system decides the order of keys to optimize parsing speed. The order of keys in this mode should not be considered stable, and it's recommended not to rely on key ordering as it may change in future updates. Setting `propertyOrder` to `"original"` ensures that the keys are ordered as they appear in the input during the decoding/encoding process. **Example** **Example** ### Customizing Parsing Behavior at the Schema Level The `parseOptions` annotation allows you to customize parsing behavior at different schema levels, enabling you to apply unique parsing settings to nested schemas within a structure. Options defined within a schema override parent-level settings and apply to all nested schemas. **Example** **Detailed Output Explanation:** In this example: - The main schema is configured to display all errors. Hence, you will see errors related to both the `d` field and any errors from the `a` subschema. - The subschema is set to display only the first error. Although both `b` and `c` fields are missing, only the first missing field is reported. ## Type Guards The `Schema.is` function provides a way to verify if a value conforms to a given schema. It acts as a , taking a value of type `unknown` and determining if it matches the structure and type constraints defined in the schema. Here's how the `Schema.is` function works: 1. **Schema Definition**: Define a schema to describe the structure and constraints of the data type you expect. Its decoded type `T` is the target type checked by the type guard. 2. **Type Guard Creation**: Use the schema to create a user-defined type guard, ` => input is T`. This function can be used at runtime to check if a value meets the requirements of the schema. **Example** The generated `isPerson` function has the following signature: ## Assertions While type guards verify whether a value conforms to a specific type, the `Schema.asserts` function goes further by asserting that an input matches the decoded type `T` described by the schema. If the input does not match the schema, it throws a detailed error, making it useful for runtime validation. **Example** The `assertsPerson` wrapper has the following signature: ## Naming Conventions Schema names describe the decoded type and, when a transformation is involved, the encoded representation it is decoded from. Schemas whose decoded and encoded types are the same are generally named after that type: - `Schema.Finite` describes finite numbers in both directions. - `Schema.Date` describes `Date` values in both directions. For transformed schemas, a name of the form `TFromE` reads as “decode `E` into `T`”: - `Schema.FiniteFromString` decodes a `string` into a finite `number` and encodes the number back into a `string`. - `Schema.DateFromString` decodes an ISO-formatted `string` into a `Date` and encodes the `Date` back into a `string`. --- title: Introduction to Effect Schema description: "Introduction to `effect/Schema`, a module for defining, validating, and transforming data schemas." sidebar: label: Introduction order: 0 --- import { Aside } from "@astrojs/starlight/components" Welcome to the documentation for `effect/Schema`, a module for defining and using schemas to validate and transform data in TypeScript. The `effect/Schema` module allows you to define schema values that describe the structure and data types of your data. Once defined, you can leverage these schemas to perform a range of operations, including: | Operation | Description | | --------------- | ----------------------------------------------------------------------------------------------------------------- | | Decoding | Transforming data from an input type `Encoded` to an output type `Type`. | | Encoding | Converting data from an output type `Type` back to an input type `Encoded`. | | Asserting | Verifying that a value adheres to the schema's output type `Type`. | | Standard Schema | Generate a . | | Arbitraries | Generate for testing. | | JSON Schemas | Create for the encoded representation of schemas. | | Equivalence | Create an based on a schema. | | Formatting | Create a based on a schema. | ## Requirements - TypeScript 5.9 or newer. TypeScript 7 is recommended for the best performance and compatibility with . - The `strict` flag enabled in your `tsconfig.json` file. - The `exactOptionalPropertyTypes` flag enabled in your `tsconfig.json` file. ### The exactOptionalPropertyTypes Option The `effect/Schema` module takes advantage of the `exactOptionalPropertyTypes` option of `tsconfig.json`. This option affects how optional properties are typed ). **Example** With `exactOptionalPropertyTypes` enabled, `name` may be omitted, but when the property is present its value must be a `string`. TypeScript does not widen the property type to `string | undefined`, so the type checker rejects an explicit `{ name: undefined }`. **Example** If, for some reason, you can't enable the `exactOptionalPropertyTypes` option , you can still use `effect/Schema`. However, there will be a mismatch between the types and the runtime behavior: In this case, the type of `name` is widened to `string | undefined`, which means the type checker won't catch the invalid value . However, during decoding, you'll encounter an error, indicating that `undefined` is not allowed. ## Schema Views A schema is an immutable value that describes the structure of your data. The same schema value can be viewed through different interfaces, depending on which type-level information an API needs: | View | Type-level information retained | | ---------------------------------------------------------- | -------------------------------------------------------------------- | | `Top` | No specific type information; accepts any schema | | `Schema` | Decoded type | | `Decoder` | Decoded type and services required for decoding | | `Encoder` | Encoded type and services required for encoding | | `Codec` | Decoded type, encoded type, and services required in both directions | For example, the `Codec` view retains all four directional type parameters: These type parameters have the following meanings: | Parameter | Description | | -------------------- | ----------------------------------------------------------------------------------------------------------------- | | **Type** | The type of value produced by decoding. | | **Encoded** | The encoded representation accepted during decoding and produced during encoding. It defaults to `Type`. | | **DecodingServices** | The services required during decoding. It defaults to `never`, meaning that decoding has no service requirements. | | **EncodingServices** | The services required during encoding. It defaults to `never`, meaning that encoding has no service requirements. | **Examples** - `Schema` is a type-only view of any schema whose decoded type is `string`. - `Decoder` retains the decoded type but does not constrain the encoded type or encoding services. - `Encoder` retains the encoded type but does not constrain the decoded type or decoding services. - `Codec` is shorthand for `Codec`. - `Codec` represents a codec that decodes a `number` from a `string`, encodes a `number` to a `string`, and requires no services. ## Understanding Schema Values **Schema Values**. Schema values are immutable descriptions of data. Combinators that compose, refine, or transform a schema return a new schema without modifying the original. **Schema Interpreters**. A schema can be interpreted by different interpreters to produce operations such as decoding, encoding, formatting, and arbitrary generation. ## Understanding Decoding and Encoding When working with data in TypeScript, you often need to handle data coming from or being sent to external systems. This data may not always match the format or types you expect, especially when dealing with user input, data from APIs, or data stored in different formats. To handle these discrepancies, we use **decoding** and **encoding**. | Term | Description | | ------------ | ----------------------------------------------------------- | | **Decoding** | Converts a value from its encoded type `E` to its type `T`. | | **Encoding** | Converts a value from its type `T` to its encoded type `E`. | For example, consider an HTTP endpoint whose request and response bodies contain a finite number represented as a JSON string. After the request body is parsed as JSON, decoding converts `"42"` into the number `42`. Before sending the response, encoding converts `42` back into `"42"`, which can then be serialized as JSON. Below is a diagram that shows the relationship between encoding and decoding through the `Codec` view: The diagram shows the Effect-based interpreters because they preserve the `RD` and `RE` service requirements. The `Sync`, `Result`, `Exit`, `Option`, and `Promise` variants follow the same directions but can only be used when the corresponding operation requires no services. We'll illustrate these concepts using `Schema.FiniteFromString`, which can be viewed as a `Codec`. It decodes a `string` into a finite `number`, encodes a finite `number` as a `string`, and requires no services in either direction. ### Encoding When we talk about "encoding," we are referring to the process of changing a finite `number` into a `string`. To put it simply, it's the act of converting data from one format to another. ### Decoding Conversely, "decoding" entails transforming a `string` into a finite `number`. It's essentially the reverse operation of encoding, where data is returned to its original form. ### Decoding From Unknown Decoding from `unknown` involves two key steps: 1. **Checking:** Initially, we verify that the input data matches the expected structure. In our specific case, this means ensuring that the input is indeed a `string`. 2. **Decoding:** Following the successful check, we proceed to convert the `string` into a finite `number`. This process completes the decoding operation, where the data is both validated and transformed. ### Encoding From Unknown Encoding from `unknown` involves two key steps: 1. **Checking:** Initially, we verify that the input data matches the expected structure. In our specific case, this means ensuring that the input is indeed a finite `number`. 2. **Encoding:** Following the successful check, we proceed to convert the finite `number` into a `string`. This process completes the encoding operation, where the data is both validated and transformed. ## Round-Trip A highly desirable property of a schema is that an encoding-decoding round trip returns a value equivalent to the original: The round trip starts with encoding because `value` has type `T`: encoding produces an `E`, which is then decoded back into a `T`. This property is not guaranteed. Some transformations intentionally normalize or discard information during encoding or decoding. --- title: Schema to JSON Schema description: Export a schema's canonical JSON representation as JSON Schema Draft 2020-12. sidebar: label: JSON Schema order: 16 --- `Schema.toJsonSchemaDocument` generates a JSON Schema Draft 2020-12 document for a schema's canonical JSON representation. Internally, Effect first derives `Schema.toCodecJson` and then describes that codec's encoded side. The generated JSON Schema therefore matches the values accepted and produced by the canonical JSON codec, including the JSON representations of Effect data types. ## Basic Conversion **Example** The returned document contains: - `dialect`: the source dialect, always `"draft-2020-12"`. - `schema`: the root JSON Schema. - `definitions`: definitions referenced through `$ref`. JSON Schema generation is best-effort. Semantics that JSON Schema cannot express exactly may be approximated, and opaque declarations without a structural JSON codec produce an unconstrained schema. ## Canonical JSON Representations For codecs, the output describes the encoded JSON shape rather than the decoded `Type`. **Example** Declarations such as `Option`, `Duration`, and `BigInt` define canonical JSON codecs. Their generated JSON Schemas describe those representations. **Example** When defining a custom declaration, provide a `toCodecJson` annotation if it has a meaningful JSON representation. Both `Schema.toCodecJson` and `Schema.toJsonSchemaDocument` will then use the same shape. ## Other Drafts `Schema.toJsonSchemaDocument` always emits Draft 2020-12. Convert the resulting document with the `JsonSchema` module when another draft is required. **Example** `JsonSchema.toDocumentDraft04` similarly converts a document to Draft 04. ## Annotations The following standard JSON Schema annotations are emitted automatically: - `title` - `description` - `default` - `examples` - `readOnly` - `writeOnly` - `format` - `contentEncoding` - `contentMediaType` - `contentSchema` **Example** ### Annotating a Codec's Encoded Side Calling `.annotate` on a codec annotates its decoded side. Use `Schema.annotateEncoded` for metadata that belongs to the JSON representation. **Example** ### Custom Annotation Keys Use `includeAnnotationKey` to whitelist non-standard annotations such as editor metadata or vendor extensions. Standard keys are always included. **Example** ## Filters and Constraints Built-in filters contribute JSON Schema constraints such as `minLength`, `maximum`, `pattern`, and `uniqueItems`. **Example** For a custom filter, provide a `toJsonSchema` callback when its constraint has a JSON Schema equivalent. **Example** Set `generateDescriptions: true` to turn a check's `expected` annotation into a description when no description was supplied explicitly. ## Optional Properties An `optionalKey` property is omitted from `required`. A property defined with `optional` is also omitted from `required`, and its explicit `undefined` case is represented as `null` because JSON has no `undefined` value. **Example** ## References and Recursion An `identifier` annotation creates a definition and replaces uses of that schema with `$ref`. **Example** Recursive schemas require an identifier so their self-references can be emitted as `$ref`. **Example** ## Generation Options `Schema.toJsonSchemaDocument` accepts three options: - `additionalProperties`: `false` by default, `true` to allow extra properties, or a JSON Schema describing them. - `generateDescriptions`: generate missing check descriptions from `expected` annotations. - `includeAnnotationKey`: include selected non-standard annotation keys. **Example** ## JSON Strings `Schema.fromJsonString` accepts a JSON string and decodes its parsed contents with another schema. Its JSON Schema describes the outer string and marks its media type as JSON. **Example** --- title: Schema Projections description: Create new schemas by extracting and customizing the Type or Encoded components of existing schemas. sidebar: label: Projections order: 6 --- Sometimes, you may want to create a new schema based on an existing one, focusing specifically on either its `Type` or `Encoded` aspect. The Schema module provides several functions to make this possible. ## toType `Schema.toType` extracts the decoded side of a schema. The result has the original `Type` as both its `Type` and `Encoded`, requires no services, and discards the encoding path. **Function Signature** **Example** ## toEncoded `Schema.toEncoded` extracts the encoded side of a schema. The result has the original `Encoded` as both its `Type` and `Encoded`, requires no services, and discards the decoding path while preserving checks that apply to the encoded representation. **Function Signature** **Example** --- title: Schema to Standard Schema description: Generate Standard Schema V1. sidebar: label: Standard Schema order: 14 --- import { Aside } from "@astrojs/starlight/components" `Schema.toStandardSchemaV1` exposes an Effect schema through the interface. The resulting object can be passed to libraries that support the standard while retaining the original Effect schema APIs. **Example** ## Sync vs Async Validation The Standard Schema `validate` method first attempts to decode synchronously. If decoding encounters an asynchronous transformation or check, it returns a `Promise` instead. **Example** ## Defects If an unexpected defect occurs during validation, it is reported as a single issue without a `path`. This ensures that unexpected errors do not disrupt schema validation but are still captured and reported. **Example** Standard Schema failures use the same formatter described in . Pass `leafHook`, `checkHook`, or `parseOptions` to `Schema.toStandardSchemaV1` to customize its output. --- title: Schema Transformations description: Transform and manipulate data with schema-based transformations, including type conversions, validations, and custom processing. sidebar: label: Transformations order: 7 --- import { Aside } from "@astrojs/starlight/components" Transformations are important when working with schemas. They allow you to change data from one type to another. For example, you might parse a string into a number or convert a date string into a `Date` object. Use `Schema.decodeTo` to connect a source schema to a target schema. Provide `SchemaTransformation.transform` for an infallible conversion, or `SchemaGetter.transformOrFail` when either direction may fail or require services. ## Infallible Transformations `Schema.decodeTo` creates a new schema by connecting the decoded `Type` of a source schema to the `Encoded` type expected by a target schema. `SchemaTransformation.transform` supplies the two infallible conversion functions needed when those types differ. ### Understanding Input and Output "Output" and "input" depend on what you are doing : **When decoding:** - The source codec produces `SourceType` from `SourceEncoded`. - The custom `decode` function converts `SourceType` to `TargetEncoded`. - The target codec produces `TargetType` from `TargetEncoded`. - The complete decoding path is `SourceEncoded` → `TargetType`. If `SourceType` and `TargetEncoded` differ, you can provide a `decode` function to convert the source schema's output into the target schema's input. **When encoding:** - The target codec produces `TargetEncoded` from `TargetType`. - The custom `encode` function converts `TargetEncoded` to `SourceType`. - The source codec produces `SourceEncoded` from `SourceType`. - The complete encoding path is `TargetType` → `SourceEncoded`. If `TargetEncoded` and `SourceType` differ, you can provide an `encode` function to convert the target schema's output into the source schema's input. ### Combining Two Primitive Schemas In this example, we start with a schema that accepts `"on"` or `"off"` and transform it into a boolean schema. The `decode` function turns `"on"` into `true` and `"off"` into `false`. The `encode` function does the reverse. The resulting codec has `boolean` as its `Type` and `"on" | "off"` as its `Encoded` type. **Example** The `decode` function above never fails by itself. However, the full decoding process can still fail if the input does not fit the source schema. For example, if you provide `"wrong"` instead of `"on"` or `"off"`, the source schema will fail before calling `decode`. **Example** ### Combining Two Transformation Schemas Below is an example where both the source and target schemas transform their data: - The source schema is `Schema.FiniteFromString`, whose `Type` is `number` and whose `Encoded` type is `string`. - The target schema is `BooleanFromString`, whose `Type` is `boolean` and whose `Encoded` type is `"on" | "off"`. This example involves four types and requires two conversions: - When decoding, convert a `number` into `"on" | "off"`. For example, treat any positive number as `"on"`. - When encoding, convert `"on" | "off"` back into a `number`. For example, treat `"on"` as `1` and `"off"` as `-1`. By composing these transformations, we get a codec whose `Type` is `boolean` and whose `Encoded` type is `string`. **Example** **Example** In this example, we convert an array into a `ReadonlySet`. The `decode` function takes an array and creates a new `ReadonlySet`. The `encode` function converts the set back into an array. We also provide the schema of the array items so they are properly validated. ## Fallible Transformations Use `SchemaGetter.transformOrFail` inside `Schema.decodeTo` when decoding or encoding can fail, run asynchronously, or require Effect services. This function enables decoding/encoding functions to return either a successful result or an error, making it particularly useful for validating and processing data that might not always conform to expected formats. ### Error Handling The getter returns an `Effect` that succeeds with the converted value or fails with a `SchemaIssue.Issue`. Use a specific issue such as `SchemaIssue.InvalidValue`, `Pointer`, or `Composite` when you need structured error information. **Example** A transformation can normalize a broader input and report a domain-specific issue when no target value matches. The function passed to `SchemaGetter.transformOrFail` receives the value and the active . ### Async Transformations In modern applications, especially those interacting with external APIs, you might need to transform data asynchronously. `SchemaGetter.transformOrFail` supports this by returning an `Effect`. **Example** Consider a scenario where you need to validate a person's ID by making an API call. Here's how you can implement it: ### Declaring Dependencies When a transformation depends on services, they are tracked separately in the codec's `DecodingServices` and `EncodingServices` views. **Example** ## One-Way Transformations with Forbidden Encoding In some cases, encoding a value back to its original form may not make sense or may be undesirable. Use `SchemaGetter.forbidden` for that direction so the restriction is represented as a schema issue. **Example** Computing a digest loses the original content. This transformation decodes text to its SHA-256 digest and explicitly forbids encoding the digest back to the source text. ## Composition When the source codec's `Type` already matches the target codec's `Encoded` type, call `Schema.decodeTo` without a custom transformation. The result combines both decoding paths and both encoding paths. **Example** ## Effectful Filters Use `SchemaGetter.checkEffect` as part of a transformation for validations that require asynchronous work or services. For synchronous validation, use . **Example** ## String Transformations ### split Splits a string by a specified delimiter into an array of substrings. **Example** ### Trim Removes whitespace from the beginning and end of a string. **Example** ### Lowercase Converts a string to lowercase. **Example** ### Uppercase Converts a string to uppercase. **Example** ### Capitalize Converts the first character of a string to uppercase. **Example** ### Uncapitalize Converts the first character of a string to lowercase. **Example** ### JSON Strings `Schema.fromJsonString` creates a schema that decodes JSON text with `JSON.parse` and encodes values with `JSON.stringify`. Use `Schema.Unknown` when the parsed value can have any JSON-compatible shape. **Example** Pass a more specific schema to validate the parsed value. **Example** In this example, the struct ensures the parsed JSON is an object with a finite numeric property `a`. ### StringFromBase64 Decodes a base64 encoded string into a UTF-8 string. **Example** ### StringFromBase64Url Decodes a base64 encoded string into a UTF-8 string. **Example** ### StringFromHex Decodes a hex encoded string into a UTF-8 string. **Example** ### StringFromUriComponent Decodes a URI-encoded string into a UTF-8 string. It is useful for encoding and decoding data in URLs. **Example** ## Number Transformations ### FiniteFromString Transforms a string into a finite number. It returns an error if the value can't be converted or represents a non-finite number such as `NaN`, `Infinity`, or `-Infinity`. **Example** ## BigInt transformations ### BigIntFromString Converts a string to a `BigInt` using the `BigInt` constructor. **Example** ## Date transformations ### DateFromString Converts a string into a **valid** `Date`, ensuring that invalid dates, such as `new Date`, are rejected. **Example** ## BigDecimal Transformations ### BigDecimalFromString Converts a string to a `BigDecimal`. **Example** --- title: Creating Sinks description: Discover how to create and use various sinks for processing streams, including counting, summing, collecting, folding, and handling success or failure. sidebar: order: 1 --- In stream processing, sinks are used to consume and handle elements from a stream. Here, we'll explore various sink constructors that allow you to create sinks for specific tasks. ## Common Constructors ### head The `Sink.head` sink retrieves only the first element from a stream, wrapping it in `Some`. If the stream has no elements, it returns `None`. **Example** ### last The `Sink.last` sink retrieves only the last element from a stream, wrapping it in `Some`. If the stream has no elements, it returns `None`. **Example** ### count The `Sink.count` sink consumes all elements of the stream and counts the number of elements fed to it. ### sum The `Sink.sum` sink consumes all elements of the stream and sums incoming numeric values. ### take The `Sink.take` sink takes the specified number of values from the stream and returns them in an array. ### drain The `Sink.drain` sink ignores its inputs, effectively discarding them. ### timed The `Sink.timed` sink executes the stream and measures its execution time, providing the . ### forEach The `Sink.forEach` sink executes the provided effectful function for every element fed to it. ## Creating Sinks from Success and Failure Just as you can define streams to hold or manipulate data, you can also create sinks with specific success or failure outcomes using the `Sink.fail` and `Sink.succeed` functions. ### Succeeding Sink This example creates a sink that doesn’t consume any elements from its upstream source but instead immediately succeeds with a specified numeric value: **Example** ### Failing Sink In this example, the sink also doesn’t consume any elements from its upstream source. Instead, it fails with a specified error message of type `string`: **Example** ## Collecting ### Collecting All Elements To gather all elements from a data stream into an array, use the `Sink.collect` sink. The final output contains all elements from the stream in the order they were emitted. **Example** ### Collecting a Specified Number To collect a fixed number of elements from a stream into an array, use `Sink.take`. This sink stops collecting once it reaches the specified limit. **Example** ### Collecting While Meeting a Condition To gather elements from a stream while they satisfy a specific condition, use `Sink.takeWhile`. This sink collects elements until the provided predicate returns `false`. **Example** ### Collecting into a HashSet To accumulate stream elements into a native `Set`, fold them with `Sink.reduce`. This ensures that each element appears only once in the final set. **Example** ### Collecting into HashSets of a Specific Size For controlled collection into a `Set` with a specified maximum size, fold with `Sink.reduceWhile`, stopping once the set reaches the given limit. **Example** ### Collecting into a HashMap For more complex collection scenarios, fold elements into a native `Map` with `Sink.reduce`, using a key function to define each element's grouping and a merge function to combine values sharing the same key. **Example** In this example, we use ` => n % 3` to determine map keys and ` => a + b` to merge elements with the same key: ### Collecting into a HashMap with Limited Keys To accumulate elements into a native `Map` with a maximum number of keys, fold with `Sink.reduceWhile`, stopping once the map reaches the specified key limit. This requires a key function to define the grouping of each element and a merge function to combine values with the same key. **Example** ## Folding ### Reducing Elements If you want to reduce a stream into a single cumulative value by applying an operation to each element in sequence, you can use the `Sink.reduce` function. **Example** ### Folding with Termination Sometimes, you may want to fold elements in a stream but stop the process once a specific condition is met. This is known as "short-circuiting." You can accomplish this with the `Sink.fold` function, which lets you define a termination condition. **Example** ### Folding Until a Limit To accumulate elements until a specific count is reached, use `Sink.foldUntil`. This sink folds elements up to the specified limit and then stops. **Example** ### Folding with Weighted Elements In some scenarios, you may want to fold elements based on a defined "weight" or "cost," accumulating elements until a specified maximum cost is reached. You can accomplish this by building a custom sink out of `Sink.fold`, whose termination condition checks the accumulated cost instead of a plain element count. **Example** In the example below, each element has a weight of `1`, and the folding resets when the accumulated weight hits `3`. --- title: Introduction description: Learn the role of Sink in stream processing, handling element consumption, error management, result production, and leftover elements. sidebar: order: 0 --- In stream processing, a `Sink` is a construct designed to consume elements generated by a `Stream`. Here's an overview of what a `Sink` does: - It consumes a varying number of `In` elements, which may include zero, one, or multiple elements. - It can encounter errors of type `E` during processing. - It produces a result of type `A` once processing completes. - It can also return a remainder of type `L`, representing any leftover elements. To process a stream using a `Sink`, you can pass it directly to the `Stream.run` function: **Example** The type of `sink` is as follows: Here's the breakdown: - `Array`: The final result produced by the sink after processing elements. - `number` : The type of elements that the sink will consume from the stream. - `number` : The type of leftover elements, if any, that are not consumed. - `never` : Indicates that this sink does not produce any errors. - `never` : Shows that no dependencies are required to operate this sink. --- title: Leftovers description: Learn how to handle unconsumed elements in streams, collecting or ignoring leftovers for efficient data processing. sidebar: order: 4 --- In this section, we'll look at handling elements left unconsumed by sinks. Sinks may process only a portion of the elements from an upstream source, leaving some elements as "leftovers." Here's how to collect or ignore these remaining elements. ## Collecting Leftovers If a sink doesn't consume all elements from the upstream source, the remaining elements are called leftovers. `Sink.mapEnd` transforms both the sink result and its optional leftovers, so you can move the leftovers into the result returned by `Stream.run`. **Example** ## Ignoring Leftovers If leftover elements are not needed, you can ignore them using `Sink.ignoreLeftover`. This approach discards any unconsumed elements, so the sink operation focuses only on the elements it needs. **Example** --- title: Sink Operations description: Explore operations to transform, filter, and adapt sinks, enabling custom input-output handling and element filtering in stream processing. sidebar: label: Operations order: 2 --- In previous sections, we learned how to create and use sinks. Now, let's explore some operations that let you transform or filter sink behavior. ## Adapting Sink Input At times, you may have a sink that works with one type of input, but your current stream uses a different type. The `Sink.mapInput` function helps you adapt your sink to a new input type by transforming the input values. While `Sink.map` changes the sink's output, `Sink.mapInput` changes the input it accepts. **Example** Suppose you have a `Sink.sum` that calculates the sum of numbers. If your stream contains strings rather than numbers, `Sink.mapInput` can convert those strings into numbers, allowing `Sink.sum` to work with your stream: ## Transforming Both Input and Output When you need to transform both the input and output of a sink, combine `Sink.mapInput` with `Sink.map`. Together they let you transform the input type, perform the operation, and then transform the output to a new type. This can be useful for complete conversions between input and output types. **Example** ## Filtering Input You can filter the elements a sink ends up processing by filtering the stream that feeds it, before handing it off with `Stream.transduce`. This lets you combine a filter with a sink like `Sink.take` that only cares about elements meeting certain criteria. **Example** In the example below, elements are collected into arrays of three, but only positive numbers are included: --- title: Ref description: Learn how to manage state in concurrent applications using Effect's Ref data type. Master mutable references for safe, controlled state updates across fibers. sidebar: order: 0 --- import { Aside } from "@astrojs/starlight/components" When we write programs, it is common to need to keep track of some form of state during the execution of the program. State refers to any data that can change as the program runs. For example, in a counter application, the count value changes as the user increments or decrements it. Similarly, in a banking application, the account balance changes as deposits and withdrawals are made. State management is crucial to building interactive and dynamic applications. In traditional imperative programming, one common way to store state is using variables. However, this approach can introduce bugs, especially when the state is shared between multiple components or functions. As the program becomes more complex, managing shared state can become challenging. To overcome these issues, Effect introduces a powerful data type called `Ref`, which represents a mutable reference. With `Ref`, we can share state between different parts of our program without relying on mutable variables directly. Instead, `Ref` provides a controlled way to handle mutable state and safely update it in a concurrent environment. Effect's `Ref` data type enables communication between different fibers in your program. This capability is crucial in concurrent programming, where multiple tasks may need to access and update shared state simultaneously. In this guide, we will explore how to use the `Ref` data type to manage state in your programs effectively. We will cover simple examples like counting, as well as more complex scenarios where state is shared between different parts of the program. Additionally, we will show how to use `Ref` in a concurrent environment, allowing multiple tasks to interact with shared state safely. Let's dive in and see how we can leverage `Ref` for effective state management in your Effect programs. ## Using Ref Here is a simple example using `Ref` to create a counter: **Example** **Example** ## Using Ref in a Concurrent Environment We can also use `Ref` in concurrent scenarios, where multiple tasks might be updating shared state at the same time. **Example** For this example, let's update the counter concurrently: ## Using Ref as a Service You can pass a `Ref` as a to share state across different parts of your program. **Example** Note that we use `Effect.provideServiceEffect` instead of `Effect.provideService` to provide an actual implementation of the `MyState` service because all the operations on the `Ref` data type are effectful, including the creation `Ref.make`. ## Sharing State Between Fibers You can use `Ref` to manage shared state between multiple fibers in a concurrent environment. **Example** Let's look at an example where we continuously read names from user input until the user enters `"q"` to exit. First, let's introduce a `readLine` utility to read user input : Next, we implement the main program to collect names: Now that we have learned how to use the `Ref` data type, we can use it to manage the state concurrently. For example, assume while we are reading from the console, we have another fiber that is trying to update the state from a different source. Here, one fiber reads names from user input, while another fiber concurrently adds preset names at regular intervals: --- title: SubscriptionRef description: Learn how to manage shared state with SubscriptionRef in Effect, enabling multiple observers to subscribe to and react to state changes efficiently in concurrent environments. sidebar: order: 2 --- A `SubscriptionRef` is a specialized form of a . It allows us to subscribe and receive updates on the current value and any changes made to that value. You can perform all standard operations on a `SubscriptionRef`, such as `get`, `set`, or `modify` to interact with the current value. The key feature of `SubscriptionRef` is its `changes` stream. This stream allows you to observe the current value at the moment of subscription and receive all subsequent changes. Every time the stream is run, it emits the current value and tracks future updates. To create a `SubscriptionRef`, you can use the `SubscriptionRef.make` constructor, specifying the initial value: **Example** `SubscriptionRef` is particularly useful for modeling shared state when multiple observers need to react to changes. For example, in functional reactive programming, the `SubscriptionRef` could represent a portion of the application state, and various observers would update in response to state changes. **Example** In the following example, a "server" continually updates a shared value, while multiple "clients" observe the changes: The `server` function operates on a regular `Ref` and continuously updates the value. It doesn't need to know about `SubscriptionRef` directly. Next, let's define a `client` that subscribes to changes and collects a specified number of values: Similarly, the `client` function only works with a `Stream` of values and doesn't concern itself with the source of these values. To tie everything together, we start the server, launch multiple client instances in parallel, and then shut down the server when we're finished. We also create the `SubscriptionRef` in this process. This setup ensures that each client observes the current value when it starts and receives all subsequent changes to the value. Since the changes are represented as streams, you can easily build more complex programs using familiar stream operators. You can transform, filter, or merge these streams with other streams to achieve more sophisticated behavior. --- title: SynchronizedRef description: Master concurrent state management with SynchronizedRef in Effect, a mutable reference that supports atomic, effectful updates to shared state in concurrent environments. sidebar: order: 1 --- import { Aside } from "@astrojs/starlight/components" `SynchronizedRef` serves as a mutable reference to a value of type `A`. With it, we can store **immutable** data and perform updates **atomically** and effectfully. The distinctive function in `SynchronizedRef` is `updateEffect`. This function takes an effectful operation and executes it to modify the shared state. This is the key feature setting `SynchronizedRef` apart from `Ref`. In real-world applications, `SynchronizedRef` is useful when you need to execute effects, such as querying a database, and then update shared state based on the result. It ensures that updates happen sequentially, preserving consistency in concurrent environments. **Example** In this example, we simulate fetching user ages concurrently and updating a shared state that stores the ages: --- title: Consuming Streams description: Learn techniques for consuming streams, including collecting elements, processing with callbacks, and using folds and sinks. sidebar: order: 2 --- When working with streams, it's essential to understand how to consume the data they produce. In this guide, we'll walk through several common methods for consuming streams. ## Using runCollect To gather all the elements from a stream into an array, you can use the `Stream.runCollect` function. ## Using runForEach Another way to consume elements of a stream is by using `Stream.runForEach`. It takes a callback function that receives each element of the stream. Here's an example: In this example, we use `Stream.runForEach` to log each element to the console. ## Using runFold `Stream.runFold` consumes a stream by reducing its values and returns an effect containing the result. For early termination, use `Stream.runForEachWhile` and keep the accumulator local to an `Effect.suspend` block. In the first example, `Stream.runFold` calculates the sum of all elements. In the second, `Stream.runForEachWhile` stops after the accumulator exceeds `3`; the element that makes the predicate false has already been consumed, so the result is `6`. ## Using a Sink To consume a stream using a Sink, you can pass the `Sink` to the `Stream.run` function. Here's an example: In this example, we use a `Sink` to calculate the sum of the elements in the stream. --- title: Creating Streams description: Learn various methods for creating Effect streams, from basic constructors to handling asynchronous data sources, pagination, and schedules. sidebar: order: 1 --- In this section, we'll explore various methods for creating Effect `Stream`s. These methods will help you generate streams tailored to your needs. ## Common Constructors ### make You can create a pure stream by using the `Stream.make` constructor. This constructor accepts a variable list of values as its arguments. ### empty Sometimes, you may require a stream that doesn't produce any values. In such cases, you can use `Stream.empty`. This constructor creates a stream that remains empty. ### void If you need a stream that contains a single `void` value, you can use `Stream.succeed`. This is handy when you want to represent a stream with a single event or signal. ### range To create a stream of integers within a specified range `` , you can use `Stream.range`. This is particularly useful for generating a stream of sequential numbers. ### iterate With `Stream.iterate`, you can generate a stream by applying a function iteratively to an initial value. The initial value becomes the first element produced by the stream, followed by subsequent values produced by `f`, `f)`, and so on. ### scoped `Stream.scoped` is used to create a single-valued stream from a scoped resource. It can be handy when dealing with resources that require explicit acquisition, usage, and release. ## From Success and Failure Much like the `Effect` data type, you can generate a `Stream` using the `fail` and `succeed` functions: ## From Arrays You can construct a stream from an array like this: Moreover, you can create a stream from multiple arrays as well: ## From Effect You can generate a stream from an Effect workflow by employing the `Stream.fromEffect` constructor. For instance, consider the following stream, which generates a single random number: This method allows you to seamlessly transform the output of an Effect into a stream, providing a straightforward way to work with asynchronous operations within your streams. ## From Asynchronous Callback Imagine you have an asynchronous function that relies on callbacks. If you want to capture the results emitted by those callbacks as a stream, you can use the `Stream.callback` function. This function is designed to adapt functions that invoke their callbacks multiple times and emit the results as a stream. Let's break down how to use it in the following example: The function you pass to `Stream.callback` receives a `Queue` that you use to drive the stream from your asynchronous code. Here's what each of the possible outcomes means: - Calling `Queue.offerUnsafe` on the queue emits the given element as part of the stream. - Calling `Queue.fail` on the queue terminates the stream with the specified error. - Calling `Queue.endUnsafe`/`Queue.end` on the queue signals the end of the stream, terminating it successfully. To put it simply, this gives you full control over how your asynchronous callback interacts with the stream, determining when to emit elements, when to terminate with an error, or when to signal the end of the stream. ## From Iterables ### fromIterable You can create a pure stream from an `Iterable` of values using the `Stream.fromIterable` constructor. It's a straightforward way to convert a collection of values into a stream. ### fromIterableEffect When you have an effect that produces a value of type `Iterable`, you can employ the `Stream.fromIterableEffect` constructor to generate a stream from that effect. For instance, let's say you have a database operation that retrieves a list of users. Since this operation involves effects, you can utilize `Stream.fromIterableEffect` to convert the result into a `Stream`: This enables you to work seamlessly with effects and convert their results into streams for further processing. ### fromAsyncIterable Async iterables are another type of data source that can be converted into a stream. With the `Stream.fromAsyncIterable` constructor, you can work with asynchronous data sources and handle potential errors gracefully. In this code, we define an async iterable and then create a stream named `stream` from it. Additionally, we provide an error handler function to manage any potential errors that may occur during the conversion. ## From Repetition ### Repeating a Single Value You can create a stream that endlessly repeats a specific value using `Stream.forever)`: ### Repeating a Stream's Content `Stream.repeat` allows you to create a stream that repeats a specified stream's content according to a schedule. This can be useful for generating recurring events or values. ### Repeating an Effect's Result Imagine you have an effectful API call, and you want to use the result of that call to create a stream. You can achieve this by creating a stream from the effect and repeating it indefinitely. Here's an example of generating a stream of random numbers: ### Repeating an Effect with Termination You can repeatedly evaluate a given effect and terminate the stream based on specific conditions. In this example, we're draining an `Iterator` to create a stream from it: ### Generating Ticks You can create a stream that emits `void` values at specified intervals using the `Stream.tick` constructor. This is useful for creating periodic events. ## From Unfolding/Pagination In functional programming, the concept of `unfold` can be thought of as the counterpart to `fold`. With `fold`, we process a data structure and produce a return value. For example, we can take an `Array` and calculate the sum of its elements. On the other hand, `unfold` represents an operation where we start with an initial value and generate a recursive data structure, adding one element at a time using a specified state function. For example, we can create a sequence of natural numbers starting from `1` and using the `increment` function as the state function. ### Unfold #### unfold The Stream module includes an `unfold` function defined as follows: Here's how it works: - **initialState**. This is the initial state value. - **step**. The state function `step` takes the current state `s` as input and returns an effect. If the effect resolves to `undefined`, the stream ends. If it resolves to a tuple ``, the next element in the stream is `A`, and the state `S` is updated for the next step process. For example, let's create a stream of natural numbers using `Stream.unfold`: #### Unfolding with Effects Sometimes, we may need to perform effectful state transformations during the unfolding operation. Since the `step` function passed to `Stream.unfold` already returns an `Effect`, it can depend on any effectful computation, such as generating a random value, while producing the next element and state. Here's an example of creating an infinite stream of random `1` and `-1` values using `Stream.unfold`: ### Pagination #### paginate `Stream.paginate` is similar to `Stream.unfold` but allows emitting values one step further. For example, the following stream emits `0, 1, 2, 3` elements: Here's how it works: - We start with an initial value of `0`. - The provided function takes the current value `n` and returns a tuple. The first element of the tuple is the value to emit , and the second element determines whether to continue `) or stop `). ### Unfolding vs. Pagination You might wonder about the difference between the `unfold` and `paginate` combinators and when to use one over the other. `Stream.unfold` produces exactly one value per step, so it can't consume an API whose step naturally returns a batch of values at once. `Stream.paginate` is built for that shape: each step returns an array of values together with the next state, so a single call can emit zero, one, or many elements before deciding whether to continue. This is exactly the shape of a paginated API. Imagine a `fetchUsers` request that, given a cursor, returns a page of users and, if there's more data, the cursor for the next page: Each call to `fetchUsers` hands back a whole page of items, and `Stream.paginate` flattens every page into the resulting stream, stopping once `nextCursor` is `undefined`. Modeling this with `Stream.unfold` would require peeling one item off the current page at a time and keeping the rest around as extra state. `Stream.paginate` already handles that logic by accepting an array per step. ## From Queue and PubSub In Effect, there are two essential asynchronous messaging data types: and . You can easily transform these data types into `Stream`s by utilizing `Stream.fromQueue` and `Stream.fromPubSub`, respectively. ## From Schedule We can create a stream from a `Schedule` that does not require any further input. The stream will emit an element for each value output from the schedule, continuing for as long as the schedule continues: --- title: Error Handling in Streams description: Learn how to handle errors in streams, ensuring robust recovery, retries, and graceful error management for reliable stream processing. sidebar: label: Error Handling order: 3 --- ## Recovering from Failure When working with streams that may encounter errors, it's crucial to know how to handle these errors gracefully. The `Stream.catch` function is a powerful tool for recovering from failures and switching to an alternative stream in case of an error. **Example** In this example, `s1` encounters an error, but instead of terminating the stream, we gracefully switch to `s2` using `Stream.catch`. This ensures that we can continue processing data even if one stream fails. You can also distinguish elements from the two streams based on success or failure by tagging each side with the data type before merging them, mapping `s1`'s elements into `Result.succeed` and `s2`'s elements into `Result.fail`: The `Stream.catch` function provides advanced error handling capabilities compared to `Stream.catch`. With `Stream.catch`, you can make decisions based on both the type and value of the encountered failure. In this example, we have a stream, `s1`, which may encounter two different types of errors. Instead of a straightforward switch to an alternative stream, as done with `Stream.catch`, we employ `Stream.catch` to precisely determine how to handle each type of error. This level of control over error recovery enables you to choose different streams or actions based on the specific error conditions. ## Recovering from Defects When working with streams, it's essential to be prepared for various failure scenarios, including defects that might occur during stream processing. To address this, the `Stream.catchCause` function provides a robust solution. It enables you to gracefully handle and recover from any type of failure that may arise. **Example** In this example, `s1` may encounter a defect, but instead of crashing the application, we use `Stream.catchCause` to gracefully switch to an alternative stream, `s2`. This ensures that your application remains robust and continues processing data even in the face of unexpected issues. ## Recovery from Some Errors In stream processing, there may be situations where you need to recover from specific types of failures. The `Stream.catchFilter` and `Stream.catchCauseFilter` functions come to the rescue, allowing you to handle and mitigate errors selectively. If you want to recover from a particular error, you can use `Stream.catchFilter`: To recover from a specific cause, you can use the `Stream.catchCauseFilter` function: ## Running Cleanup on Failure `Stream.onError` runs an effect when the stream fails, then preserves the original failure. Use it for cleanup or diagnostics, not for recovery. ## Retry a Failing Stream Sometimes, streams may encounter failures that are temporary or recoverable. In such cases, the `Stream.retry` operator comes in handy. It allows you to specify a retry schedule, and the stream will be retried according to that schedule. **Example** In this example, the stream asks the user to input a number, but if an invalid value is entered , it fails with "NaN." However, we use `Stream.retry` with an exponential backoff schedule, which means it will retry after a delay of increasing duration. This allows us to handle temporary errors and eventually collect valid input. ## Refining Errors When working with streams, there might be situations where you want to selectively keep certain errors and terminate the stream with the remaining errors. You can achieve this by pattern matching on the error inside `Stream.catch`, re-failing with the errors you want to keep and dying on the rest. **Example** In this example, `stream` initially fails with a generic `Error`. However, `res` filters and keeps only errors of type `SyntaxError`, re-failing the stream with them. Any other error is turned into a defect via `Stream.die`, terminating the stream. ## Timing Out When working with streams, there are scenarios where you may want to handle timeouts, such as terminating a stream if it doesn't produce a value within a certain duration. In this section, we'll explore how to manage timeouts using various operators. ### timeout The `Stream.timeout` operator allows you to set a timeout on a stream. If the stream does not produce a value within the specified duration, it terminates. ### timeoutFail The `Stream.timeoutOrElse` operator combines a timeout with a custom failure message. If the stream times out, it fails with the specified error message. ### timeoutFailCause Similar to `Stream.timeoutOrElse`, `Stream.timeoutOrElse` combines a timeout with a custom failure cause. If the stream times out, it fails with the specified cause. ### timeoutTo The `Stream.timeoutOrElse` operator allows you to switch to another stream if the first stream does not produce a value within the specified duration. --- title: Introduction to Streams description: Learn the fundamentals of streams, a powerful tool for emitting multiple values, handling errors, and working with finite or infinite sequences in your applications. sidebar: label: Introduction order: 0 --- In this guide, we'll explore the concept of a `Stream`. A `Stream` is a program description that, when executed, can emit **zero or more values** of type `A`, handle errors of type `E`, and operates within a context of type `R`. ## Use Cases Streams are particularly handy whenever you're dealing with sequences of values over time. They can serve as replacements for observables, node streams, and AsyncIterables. ## What is a Stream? Think of a `Stream` as an extension of an `Effect`. While an `Effect` represents a program that requires a context of type `R`, may encounter an error of type `E`, and always produces a single result of type `A`, a `Stream` takes this further by allowing the emission of zero or more values of type `A`. To clarify, let's examine some examples using `Effect`: On success, each `Effect` produces exactly one value, even when that value is itself a collection. An effect may instead fail before producing its success value. ## Understanding Streams Now, let's shift our focus to `Stream`. A `Stream` represents a program description that shares similarities with `Effect`, it requires a context of type `R`, may signal errors of type `E`, and yields values of type `A`. However, the key distinction is that it can yield **zero or more values**. Here are the possible scenarios for a `Stream`: - **An Empty Stream**: It can end up empty, representing a stream with no values. - **A Single-Element Stream**: It can represent a stream with just one value. - **A Finite Stream of Elements**: It can represent a stream with a finite number of values. - **An Infinite Stream of Elements**: It can represent a stream that continues indefinitely, essentially an infinite stream. Let's see these scenarios in action: In summary, a `Stream` is a versatile tool for representing programs that may yield multiple values, making it suitable for a wide range of tasks, from processing finite lists to handling infinite sequences. --- title: Operations description: Explore essential operations for manipulating and managing data in streams, including tapping, mapping, filtering, merging, and more, to effectively process and transform streaming data. sidebar: order: 4 --- import { Aside } from "@astrojs/starlight/components" In this guide, we'll explore some essential operations you can perform on streams. These operations allow you to manipulate and interact with stream elements in various ways. ## Tapping The `Stream.tap` operation allows you to run an effect on each element emitted by the stream, observing or performing side effects without altering the elements or return type. This can be useful for logging, monitoring, or triggering additional actions with each emission. **Example** For example, `Stream.tap` can be used to log each element before and after a mapping operation: ## Taking Elements The "taking" operations in streams let you extract a specific set of elements, either by a fixed number, condition, or position within the stream. Here are a few ways to apply these operations: | API | Description | | ----------- | ----------------------------------------------------- | | `take` | Extracts a fixed number of elements. | | `takeWhile` | Extracts elements while a certain condition is met. | | `takeUntil` | Extracts elements until a certain condition is met. | | `takeRight` | Extracts a specified number of elements from the end. | **Example** ## Streams as an Alternative to Async Iterables When working with asynchronous data sources, such as async iterables, you often need to consume data in a loop until a certain condition is met. Streams provide a similar approach and offer additional flexibility. With async iterables, data is processed in a loop until a break or return statement is encountered. To replicate this behavior with Streams, consider these options: | API | Description | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `takeUntil` | Takes elements from a stream until a specified condition is met, similar to breaking out of a loop. | | `toPull` | Returns an effect that continuously pulls arrays of elements from the stream. This effect fails with a `Cause.Done` completion signal when the stream is finished, or with the stream's own error otherwise. | **Example** ## Mapping ### Basic Mapping The `Stream.map` operation applies a specified function to each element in a stream, creating a new stream with the transformed values. **Example** ### Mapping to a Constant Value The `Stream.map` method allows you to replace each success value in a stream with a specified constant value. This can be useful when you want all elements in the stream to emit a uniform value, regardless of the original data. **Example** ### Effectful Mapping For transformations involving effects, use `Stream.mapEffect`. This function applies an effectful operation to each element in the stream, producing a new stream with effectful results. **Example** To handle multiple effectful transformations concurrently, you can use the option. This option allows a specified number of effects to run concurrently, with results emitted downstream in their original order. **Example** ### Stateful Mapping `Stream.mapAccum` is similar to `Stream.map`, but it applies a transformation with state tracking, allowing you to map and accumulate values within a single operation. This is useful for tasks like calculating a running total in a stream. **Example** ### Mapping and Flattening The `Stream.flattenIterable` operation is similar to `Stream.map`, but it goes further by mapping each element to zero or more elements and then flattening the entire stream. This is particularly useful for transforming each element into multiple values. **Example** ## Filtering The `Stream.filter` operation allows you to pass through only elements that meet a specific condition. It's a way to retain elements in a stream that satisfy a particular criteria while discarding the rest. **Example** ## Scanning Stream scanning allows you to apply a function cumulatively to each element in the stream, emitting every intermediate result. Unlike `reduce`, which only provides a final result, `scan` offers a step-by-step view of the accumulation process. **Example** If you need only the final accumulated value, you can use : **Example** ## Draining Stream draining lets you execute effectful operations within a stream while discarding the resulting values. This can be useful when you need to run actions or perform side effects but don't require the emitted values. The `Stream.drain` function achieves this by ignoring all elements in the stream and producing an empty output stream. **Example** ## Detecting Changes in a Stream The `Stream.changes` operation detects and emits elements that differ from their preceding elements within a stream. This can be useful for tracking changes or deduplicating consecutive values. **Example** ## Zipping Zipping combines elements from two streams into a new stream, pairing elements from each input stream. This can be achieved with `Stream.zip` or `Stream.zipWith`, allowing for custom pairing logic. **Example** In this example, elements from the two streams are paired sequentially. The resulting stream ends when one of the streams is exhausted. **Example** Here, `Stream.zipWith` applies custom logic to each pair, combining elements in a user-defined way. ### Zipping Streams at Different Rates When combining streams that emit elements at different speeds, you may not want to wait for the slower stream to emit. Using `Stream.zipLatest` or `Stream.zipLatestWith`, you can zip elements as soon as either stream produces a new value. These functions use the most recent element from the slower stream whenever a new value arrives from the faster stream. **Example** ### Pairing with Previous and Next Elements | API | Description | | ------------------------ | --------------------------------------------------------- | | `zipWithPrevious` | Pairs each element of a stream with its previous element. | | `zipWithNext` | Pairs each element of a stream with its next element. | | `zipWithPreviousAndNext` | Pairs each element with both its previous and next. | **Example** ### Indexing Stream Elements The `Stream.zipWithIndex` operator is a helpful tool for indexing each element in a stream, pairing each item with its respective position in the sequence. This is particularly useful when you want to keep track of the order of elements within a stream. **Example** ## Cartesian Product of Streams The Stream module includes a feature for computing the _Cartesian Product_ of two streams, allowing you to create combinations of elements from two different streams. This is helpful when you need to pair each element from one set with every element of another. In simple terms, imagine you have two collections and want to form all possible pairs by picking one item from each. This pairing process is the Cartesian Product. In streams, this operation generates a new stream that includes every possible pairing of elements from the two input streams. To create a Cartesian Product of two streams, the `Stream.cross` operator is available, along with similar variants. These operators combine two streams into a new stream of all possible element combinations. **Example** ## Partitioning Partitioning a stream involves dividing it into two distinct streams based on a specified condition. The Stream module offers two functions for this purpose: `Stream.partition` and `Stream.partitionEffect`. Let's look at how these functions work and the best scenarios for their use. ### partition The `Stream.partition` function takes a `Filter` as input and divides the original stream into two substreams. One substream will contain elements that meet the condition, while the other contains those that do not. Both resulting substreams are wrapped in a `Scope` type. **Example** ### partitionEffect In some cases, you might need to partition a stream using a condition that involves an effect. For this, the `Stream.partitionEffect` function is ideal. This function uses an effectful `Filter` to split the stream into two substreams: one for elements that produce `Result.succeed` values and another for elements that produce `Result.fail` values. **Example** ## Grouping When processing streams of data, you may need to group elements based on specific criteria. The Stream module provides two functions for this purpose: `groupByKey`, `groupBy`, `grouped` and `groupedWithin`. Let's review how these functions work and when to use each one. ### groupByKey The `Stream.groupByKey` function partitions a stream based on a key function of type ` => K`, where `A` is the type of elements in the stream, and `K` represents the keys for grouping. This function is non-effectful and groups elements by simply applying the provided key function. The result of `Stream.groupByKey` is an ordinary `Stream` of `readonly ` pairs, representing the grouped stream. To process each group, you can use `Stream.flatMap`, passing a function of type ` => Stream.Stream<...>` together with `{ concurrency: "unbounded" }`. This function operates across all groups and merges them together in a non-deterministic order. **Example** In the following example, we use `Stream.groupByKey` to group exam scores by the tens place and count the number of scores in each group: ### groupBy For more complex grouping requirements where partitioning involves effects, you can use the `Stream.groupBy` function. This function accepts an effectful partitioning function and returns an ordinary `Stream` of `readonly ` pairs, representing the grouped stream. You can then process each group by using `Stream.flatMap`, similar to `Stream.groupByKey`. **Example** In the following example, we group names by their first letter and count the number of names in each group. Here, the partitioning operation is set up as an effectful operation: ### grouped The `Stream.grouped` function is ideal for dividing a stream into chunks of a specified size, making it easier to handle data in smaller, organized segments. This is particularly helpful when processing or displaying data in batches. **Example** ### groupedWithin The `Stream.groupedWithin` function allows for flexible grouping by creating chunks based on either a specified maximum size or a time interval, whichever condition is met first. This is especially useful for working with data where timing constraints are involved. **Example** In this example, `Stream.groupedWithin` groups the stream into chunks whenever either 18 elements accumulate or 1.5 seconds elapse since the last chunk was created. ## Concatenation In stream processing, you may need to combine the contents of multiple streams. The Stream module offers several operators to achieve this, including `Stream.concat`, `Stream.flatten`, and `Stream.flatMap`. Let's look at how each of these operators works. ### Simple Concatenation The `Stream.concat` operator is a straightforward method for joining two streams. It returns a new stream that emits elements from the first stream followed by elements from the second stream . This is helpful when you want to combine two streams in a specific sequence. **Example** ### Concatenating Multiple Streams If you have multiple streams to concatenate, `Stream.flatten` provides an efficient way to combine them without manually chaining multiple `Stream.concat` operations. This function takes a stream of streams and returns a single stream containing the elements of each stream in sequence. **Example** ### Advanced Concatenation with flatMap The `Stream.flatMap` operator allows for advanced concatenation by creating a stream where each element is generated by applying a function of type ` => Stream<...>` to each output of the source stream. This operator then concatenates all the resulting streams, effectively flattening them. **Example** If you need to perform the `flatMap` operation concurrently, you can use the option to control how many inner streams run simultaneously. Additionally, you can use the `switch` option to implement a "switch" behavior where previous streams are automatically cancelled when new elements arrive from the source stream. This is particularly useful when you only need the most recent result and want to conserve resources by cancelling outdated operations. **Example** The `switch` option is especially valuable for scenarios like search functionality, real-time data processing, or any situation where you want to discard previous operations when new input arrives. ## Merging Sometimes, you may want to interleave elements from two streams and create a single output stream. In such cases, `Stream.concat` isn't suitable because it waits for the first stream to complete before consuming the second. For interleaving elements as they become available, `Stream.merge` and its variants are designed for this purpose. ### merge The `Stream.merge` operation combines elements from two source streams into a single stream, interleaving elements as they are produced. Unlike `Stream.concat`, `Stream.merge` does not wait for one stream to finish before starting the other. **Example** ### Termination Strategy When merging two streams, it's important to consider the termination strategy, especially if each stream has a different lifetime. By default, `Stream.merge` waits for both streams to terminate before ending the merged stream. However, you can modify this behavior with `haltStrategy`, selecting from four termination strategies: | Termination Strategy | Description | | -------------------- | -------------------------------------------------------------------- | | `"left"` | The merged stream terminates when the left-hand stream terminates. | | `"right"` | The merged stream terminates when the right-hand stream terminates. | | `"both"` | The merged stream terminates only when both streams have terminated. | | `"either"` | The merged stream terminates as soon as either stream terminates. | **Example** ### mergeWith In some cases, you may want to merge two streams while transforming their elements into a unified type. `Stream.merge` combined with `Stream.map` on each source stream is designed for this purpose, allowing you to specify transformation functions for each source stream. **Example** ## Interleaving ### interleave The `Stream.interleave` operator lets you pull one element at a time from each of two streams, creating a new interleaved stream. If one stream finishes first, the remaining elements from the other stream continue to be pulled until both streams are exhausted. **Example** ### interleaveWith For more complex interleaving, `Stream.interleaveWith` provides additional control by using a third stream of `boolean` values to dictate the interleaving pattern. When this stream emits `true`, an element is taken from the left-hand stream; otherwise, an element is taken from the right-hand stream. **Example** ## Interspersing Interspersing adds separators or affixes in a stream, useful for formatting or structuring data in streams. ### intersperse The `Stream.intersperse` operator inserts a specified delimiter element between each pair of elements in a stream. This delimiter can be any chosen value and is added between each consecutive pair. **Example** ### intersperseAffixes For more complex needs, `Stream.intersperseAffixes` provides control over different affixes at the start, between elements, and at the end of the stream. **Example** ## Broadcasting Broadcasting a stream creates multiple downstream streams that each receive the same elements from the source stream. This is useful when you want to send each element to multiple consumers simultaneously. The upstream stream has a `capacity` parameter that sets the limit for how much it can get ahead before slowing down to match the speed of the slowest downstream stream. **Example** In the following example, we broadcast a stream of numbers to two downstream consumers. The first calculates the maximum value in the stream, while the second logs each number with a delay. The upstream stream's speed adjusts based on the slower logging stream: ## Buffering Effect streams use a pull-based model, allowing downstream consumers to control the rate at which they request elements. However, when there's a mismatch in the speed between the producer and the consumer, buffering can help balance their interaction. The `Stream.buffer` operator is designed to manage this, allowing the producer to keep working even if the consumer is slower. You can set a maximum buffer capacity using the `capacity` option. ### buffer The `Stream.buffer` operator queues elements to allow the producer to work independently from the consumer, up to a specified capacity. This helps when a faster producer and a slower consumer need to operate smoothly without blocking each other. **Example** Different buffering options let you tailor the buffering strategy based on your use case: | **Buffering Type** | **Configuration** | **Description** | | ------------------- | -------------------------------------------- | ------------------------------------------------------------- | | **Bounded Queue** | `{ capacity: number }` | Limits the queue to a fixed size. | | **Unbounded Queue** | `{ capacity: "unbounded" }` | Allows an unlimited number of buffered items. | | **Sliding Queue** | `{ capacity: number, strategy: "sliding" }` | Keeps the most recent items, discarding older ones when full. | | **Dropping Queue** | `{ capacity: number, strategy: "dropping" }` | Keeps the earliest items, discarding new ones when full. | ## Debouncing Debouncing is a technique used to prevent a function from firing too frequently, which is particularly useful when a stream emits values rapidly but only the last value after a pause is needed. The `Stream.debounce` function achieves this by delaying the emission of values until a specified time period has passed without any new values. If a new value arrives during the waiting period, the timer resets, and only the latest value will eventually be emitted after a pause. **Example** ## Throttling Throttling is a technique for regulating the rate at which elements are emitted from a stream. It helps maintain a steady data output pace, which is valuable in situations where data processing needs to occur at a consistent rate. The `Stream.throttle` function uses the to control the rate of stream emissions. **Example** In this configuration: - Each chunk processed uses one token => 1`). - Tokens are replenished at a rate of one token every 100 milliseconds . ### Shape Strategy The "shape" strategy moderates data flow by delaying chunk emissions until they comply with specified bandwidth constraints. This strategy ensures that data throughput does not exceed defined limits, allowing for steady and controlled data emission. **Example** ### Enforce Strategy The "enforce" strategy strictly regulates data flow by discarding chunks that exceed bandwidth constraints. **Example** ### burst option The `Stream.throttle` function offers a burst option that allows for temporary increases in data throughput beyond the set rate limits. This option is set to greater than 0 to activate burst capability . The burst capacity provides additional tokens in the token bucket, enabling the stream to momentarily exceed its configured rate when bursts of data occur. **Example** In this setup, the stream starts with a bucket containing 5 tokens, allowing the first five chunks to be emitted instantly. The additional burst capacity of 2 accommodates further emissions momentarily, allowing for handling of subsequent data more flexibly. Over time, as the bucket refills according to the throttle configuration, additional elements are emitted, demonstrating how the burst capability can manage uneven data flows effectively. ## Scheduling When working with streams, you may need to introduce specific time intervals between each element's emission. The `Stream.schedule` combinator allows you to set these intervals. **Example** In this example, we've used the `Schedule.spaced` schedule to introduce a one-second gap between each emission in the stream. --- title: Resourceful Streams description: Learn how to manage resources in streams with safe acquisition and release, finalization for cleanup tasks, and ensuring post-finalization actions for robust resource handling in streaming applications. sidebar: order: 5 --- Resources acquired by a stream must remain open for the whole period in which the stream is consumed. Compose `Effect.acquireRelease`, `Stream.fromEffect`, and `Stream.scoped` to tie the resource lifetime to the stream. Use `Stream.ensuring` when a stream only needs a finalizer. ## Acquire Release The following example acquires a file, emits its lines, and closes it when stream consumption ends. `Effect.acquireRelease` registers `file.close` in the scope created by `Stream.scoped`. The file therefore remains open while `Stream.fromIterableEffect` emits its contents. ## Finalization `Stream.ensuring` runs a finalizer after the stream's own finalizers, whether the stream succeeds, fails, or is interrupted. --- title: TestClock description: Control time during testing with Effect's TestClock, simulating time passage, delays, and recurring effects without waiting for real time. sidebar: order: 0 --- import { Aside } from "@astrojs/starlight/components" In most cases, we want our unit tests to run as quickly as possible. Waiting for real time to pass can slow down our tests significantly. Effect provides a handy tool called `TestClock` that allows us to **control time during testing**. This means we can efficiently and predictably test code that involves time without having to wait for the actual time to pass. ## How TestClock Works Imagine `TestClock` as a wall clock that only moves forward when we adjust it manually using the `TestClock.adjust` and `TestClock.setTime` functions. The clock time does not progress on its own. When we adjust the clock time, any effects scheduled to run at or before that time will execute. This allows us to simulate time passage in tests without waiting for real time. **Example** A key point is forking the fiber where `Effect.sleep` is invoked. Calls to `Effect.sleep` and related methods wait until the clock time matches or exceeds the scheduled time for their execution. By forking the fiber, we retain control over the clock time adjustments. ## Testing Recurring Effects Here's an example demonstrating how to test an effect that runs at fixed intervals using the `TestClock`: **Example** In this example, we test an effect that runs at regular intervals. An unbounded queue is used to manage the effects, and we verify the following: 1. No effect occurs before the specified recurrence period. 2. An effect occurs after the recurrence period. 3. The effect executes exactly once. It's important to note that after each recurrence, the next occurrence is scheduled to happen at the appropriate time. Adjusting the clock by 60 minutes places exactly one value in the queue; adjusting by another 60 minutes adds another value. ## Testing Clock This example demonstrates how to test the behavior of the `Clock` using the `TestClock`: **Example** ## Testing Deferred The `TestClock` also impacts asynchronous code scheduled to run after a specific time. **Example** --- title: Equal description: Implement value-based equality checks for improved data integrity and predictable behavior in TypeScript. sidebar: order: 0 --- The Equal module provides a simple and convenient way to define and check for equality between two values in TypeScript. Here are some key reasons why Effect exports an Equal module: 1. **Value-Based Equality by Default**: JavaScript's native equality operators check for equality by reference, meaning they compare objects based on their memory addresses rather than their content. This behavior can be problematic when you want to compare objects with the same values but different references. The `Equal.equals` function solves this for the common cases out of the box: plain objects, arrays, `Map`, `Set`, `Date`, and `RegExp` are compared structurally, without any extra setup. 2. **Custom Equality**: Sometimes structural comparison isn't what you want, for example when equality should depend on only part of a value. The Equal module enables developers to implement custom equality checks for their data types and classes. By implementing the `Equal` interface, developers can define their own equality logic. 3. **Data Integrity**: In some applications, maintaining data integrity is crucial. The ability to perform value-based equality checks ensures that identical data is not duplicated within collections like sets or maps. This can lead to more efficient memory usage and more predictable behavior. 4. **Predictable Behavior**: The Equal module promotes more predictable behavior when comparing objects. Structural values are compared consistently by content, custom types are compared using the logic they define, and you can still opt individual objects into reference equality when that's genuinely what you need. ## How to Perform Equality Checking in Effect In Effect it's advisable to **stop using** JavaScript's `===` and `==` operators and instead rely on the `Equal.equals` function. This function can work with any data type that implements the `Equal` interface. Some examples of such data types include , , , and . By default, `Equal.equals` performs a deep structural comparison. Plain objects, arrays, `Map`s, `Set`s, `Date`s, and `RegExp`s are compared by their contents, not their reference, even if they don't implement the `Equal` interface: **Example** In this example, `a` and `b` are two separate objects with the same contents. `===` would consider them different because they occupy different memory locations, but `Equal.equals` reports them as equal because plain objects are compared structurally by default. Structural comparison isn't always what you want, though. Sometimes equality should depend on only part of a value while ignoring the rest, or you may need `Equal.equals` to compare a specific object by reference instead of by value. There are two ways to change the default behavior: 1. **Implementing the `Equal` Interface**: This method is useful when you need to define custom equality logic. 2. **Opting Into Reference Equality**: Mark specific objects so `Equal.equals` compares them by reference instead of structurally. Let's explore both. ### Implementing the Equal Interface To create custom equality behavior, you can implement the `Equal` interface in your models. This interface extends the `Hash` interface from the module. **Example** Without a custom implementation, the default structural comparison would also take `updatedAt` into account, so two records for the same person captured at different times would count as different. The `` method above defines equality based on `id`, `name`, and `age` only, ignoring `updatedAt`. The `Hash` interface optimizes equality checks by comparing hash values instead of the objects themselves. When you use the `Equal.equals` function to compare two objects, it first checks if their hash values are equal. If not, it quickly determines that the objects are not equal, avoiding the need for a detailed property-by-property comparison. Once you've implemented the `Equal` interface, you can utilize the `Equal.equals` function to check for equality using your custom logic. **Example** In this code, the equality check returns `true` when comparing `alice` to another `Person` record with the same `id`, `name`, and `age` but a different `updatedAt`, because `updatedAt` doesn't participate in the comparison. It returns `false` when comparing `alice` to `bob` because their identifying fields differ. ### Opting Into Reference Equality Sometimes you want `Equal.equals` to compare a specific object or array by reference instead of by value, for example when identity matters more than content. The `Equal.byReference` function returns a proxy that opts a value out of structural comparison, without mutating the original: **Example** `Equal.byReferenceUnsafe` does the same thing without allocating a proxy, by marking the original object directly. The marking is irreversible for the lifetime of that object. The module still provides `Data.Class`, `Data.TaggedClass`, `Data.TaggedError`, and `Data.taggedEnum` for building tagged data types and errors. See the for details. ## Working with Collections JavaScript's built-in `Set` and `Map` can be a bit tricky when it comes to checking equality: **Example** Even though the two elements in the set have the same values, the set contains two elements. Why? JavaScript's `Set` checks for equality by reference, not by values. This is unaffected by Effect's `Equal` module, since the native `Set` never consults it. To perform value-based equality checks, you'll need to use the `Hash*` collection types available in the `effect` package. These collection types, such as and , use `Equal.equals` for comparisons. This means plain objects, arrays, and other structurally-comparable values are deduplicated automatically, without needing any extra setup. ### HashSet When you use the `HashSet`, it correctly handles value-based equality checks. In the following example, even though you're adding two objects with the same values, the `HashSet` treats them as a single element. **Example** **Note**: Structural comparison covers every enumerable property, though. If two values are meant to represent the same entity but carry extra fields that legitimately differ , `Equal.equals` treats them as unequal, and `HashSet` won't dedupe them: **Example** In this case, `HashSet` keeps both entries because `requestId` is part of the structural comparison. If you want deduplication based on only some fields, implement the `Equal` interface as shown so equality ignores the fields that don't matter. ### HashMap When working with the `HashMap`, you have the advantage of comparing keys by their values instead of their references. This is particularly helpful in scenarios where you want to associate values with keys based on their content. **Example** In this code snippet, `HashMap` is used to create a map where the keys are plain objects with identical values. A regular JavaScript `Map` would treat them as separate entries, because its default comparison is reference-based. `HashMap`, however, uses `Equal.equals` for comparisons, so plain objects with identical content are treated as the same key without any extra setup. Thus, when we add both objects, the second key-value pair overrides the first, resulting in a single entry in the map. --- title: Hash description: Optimize equality checks with efficient hashing for faster comparisons in collections like hash sets and maps. sidebar: order: 1 --- The `Hash` interface is closely tied to the interface and serves a supportive role in optimizing equality checks by providing a mechanism for hashing. Hashing is an important step in the efficient determination of equality between two values, particularly when used with data structures like hash tables. ## Role of Hash in Equality Checking The primary purpose of the `Hash` interface is to provide a quick and efficient way to determine if two values are definitely not equal, thereby complementing the interface. When two values implement the interface, their hash values are compared first: - **Different Hash Values**: If the hash values are different, it is guaranteed that the values themselves are different. This quick check allows the system to avoid a potentially expensive equality check. - **Same Hash Values**: If the hash values are the same, it does not guarantee that the values are equal, only that they might be. In this case, a more thorough comparison using the interface is performed to determine actual equality. This method dramatically speeds up the equality checking process, especially in collections where quick look-up and insertion times are crucial, such as in hash sets or hash maps. ## Implementing the Hash Interface Consider a scenario where you have a custom `Person` class, and you want to check if two instances are equal based on their properties. By implementing both the `Equal` and `Hash` interfaces, you can efficiently manage these checks: **Example** Explanation: - The `` method determines equality by comparing the `id`, `name`, and `age` fields of `Person` instances. This approach ensures that the equality check is comprehensive and considers all relevant attributes. - The `` method computes a hash code using the `id` of the person. This value is used to quickly differentiate between instances in hashing operations, optimizing the performance of data structures that utilize hashing. - The equality check returns `true` when comparing `alice` to a new `Person` object with identical property values and `false` when comparing `alice` to `bob` due to their differing property values.