|Sep 26, 2023
|Jul 31, 2023
|Jul 28, 2023
|Jul 28, 2023
#111 in Machine learning
Used in chidori
Star us on Github! Join us on Discord.
Check out high level docs
- 📖 Chidori
- ⚡️ Getting Started
- 🤔 About
- 🛣️ Roadmap
- Help us out!
Chidori is a reactive runtime for building AI agents. It provides a framework for building AI agents that are reactive, observable, and robust. It supports building agents with Node.js, Python, and Rust.
It is currently in alpha, and is not yet ready for production use. We are continuing to make significant changes in response to feedback.
- Built from the ground up for constructing agents
- Runtime written in Rust supporting Python and Node.js out of the box
- Build agents that actually work :emoji:
- LLM caching to minimize cost during development
- Optimized for long-running AI workflows
- Embedded code interpreter
- Time travel debugging
⚡️ Getting Started
You can use Chidori from Node.js, Python or Rust.
You will need to set the following environment variables if you depend on nodes that require them.
In the table below are examples for Node.js, Python and Rust. You'll need to scroll horizontally to view each.
The following examples show how to build a simple agent that fetches the top stories from Hacker News and call the OpenAI API to filter to AI related launches and then format that data into markdown. Results from the example are pushed into the Chidori database and can be visualized using the prompt-graph-ui project. We'll update this example with a pattern that makes those results more accessible soon.
At its core, Chidori brings a reactive runtime that orchestrates interactions between different agents and their components. The runtime is comprised of "nodes", which react to system changes they subscribe to, providing dynamic and responsive behavior in your AI systems. Nodes can encompass code, prompts, vector databases, custom code, services, or even complete systems.
Monitoring and Observability
Chidori ensures comprehensive monitoring and observability of your agents. We record all the inputs and outputs emitted by nodes, enabling us to explain precisely what led to what, enhancing your debugging experience and understanding of the system’s production behavior.
Branching and Time-Travel
With Chidori, you can take snapshots of your system and explore different possible outcomes from that point (branching), or rewind the system to a previous state (time-travel). This functionality improves error handling, debugging, and system robustness by offering alternative pathways and do-overs.
Code Interpreter Environments
Chidori comes with first-class support for code interpreter environments like Deno or Starlark. You can execute code directly within your system, providing quick startup, ease of use, and secure execution. We're continually working on additional safeguards against running untrusted code, with containerized nodes support coming soon.
- Reactive subscriptions between nodes
- Branching and time travel debugging, reverting execution of a graph
- Node.js, Python, and Rust support for building and executing graphs
- Simple local vector db for development
- Adding support for containerized nodes
- Allowing filtering in node queries
- Analysis tools for comparing executions
- Agent re-evaluation with feedback
- Definitive patterns for human in the loop agents
- Adding support for more vector databases
- Adding support for other LLM sources
- Adding support for more code interpreter environments
This is an early open source release and we're looking for collaborators from the community. A good place to start would be to join our discord!
Why Another AI Framework?
Chidori focuses on the specifics of how LLM+code execution operates rather than providing specific compositions of prompts. Other frameworks haven’t focused on this space, and it's an important one. We reduce accidental complexity in building systems for long-running agents; this helps developers build successful systems.
Chidori is the name of the lightning blade technique used by Kakashi in the Naruto anime series. It also happens to mean Thousand Birds in Japanese, which is a nice coincidence.
Well then why Thousand Birds?
Thousand Birds is a reference to flocks of birds (or a murmuration) and the emergent behavior that arises from their interactions. We think this is a good metaphor for the behavior of long running agents, the internal units of LLM execution within them, and the emergent behavior that arises from their interactions.
Rust is a great language for building systems, we like the type system and the guarantees provided by it. We also like the performance characteristics of Rust, and the ability to build a single binary that can be deployed anywhere. The Rust ecosystem makes it fairly easy to provide bindings to other languages, which is important for us to provide a good developer experience.
Our framework is inspired by the work of many others, including:
- Temporal.io - providing reliability and durability to workflows
- Eve - developing patterns for building reactive systems and reducing accidental complexity
- Timely Dataflow - efficiently streaming changes
- Langchain - developing tools and patterns for building with LLMs
Thousand Birds is under the MIT license. See the LICENSE for more information.
Help us out!
Please star the github repo and give us feedback in discord!