We just released Blue v0.9—our open-source framework for building and deploying applications with agentic workflows for the enterprise.
A Systems Perspective
In designing Blue, we followed a “systems perspective,” where we envisioned an architecture for an AI system—a compound AI architecture, not just a library—composed of a mix of LLM-based and “deterministic” agents and other key architectural components, such as data and agent registries and planners. Put together, this enables Agentic for Enterprise:
- performing complex tasks with workflows, providing essential control logic on outcomes
- accessing data from various databases, typical in an enterprise
- incorporating results with existing predictive models, built on enterprise data
- using existing services and APIs already prevalent within the enterprise
Agentic for Enterprise
From the get-go, our emphasis has been on building a framework that is tailored for the enterprise. This implies a number of requirements, such as scalability, observability, and configurability, quality of service, and more.
To achieve these requirements, a typical enterprise-scale software infrastructure has built-in control through data, application logic, and architecture. Such built-in controls are necessary for an enterprise to achieve reliable, quality outcomes that can be observed and analyzed. With the new “agentic” applications, these concerns simply carry over. Likewise, “agentic frameworks” must also comply and deliver.
Notably, there is substantial investment in the enterprise infrastructure: advanced AI models trained with proprietary data, fine-tuned LLMs, and plenty of services and APIs used in current applications, and of course a wide variety of data that has been collected over the years. “Agentic for Enterprise” also implies that frameworks need to integrate into and exploit what already exists in the enterprise infrastructure to support a wide variety of enterprise use cases.
In designing Blue, our aim has been to leverage what already exists in the enterprise infrastructure, i.e., existing APIs, models, and data in their original sources, formats, and systems. From there, we can couple them with new capabilities and controls to drive new agentic applications.
Redefining “Agents”
Our framework’s definition of “agent” differs from the norm. Unlike other frameworks where LLMs are the only agents and handle everything—from planning and orchestration to interfacing with services, tools, and data—our architecture puts LLMs within a larger software architecture, with components that are deterministic as well as parametric in nature.
Hence, we expanded the definition of agent to also include any computational construct (e.g. predictive models, application logic, query execution, etc.). Agency doesn’t necessarily come from competency, but it comes from the ability to take initiative in making decisions. Agents can make ad hoc, on-the-fly choices as well as prescribed ones via deterministic code.
Redefining “Coordination”
In existing frameworks, natural language is the de facto mode of communication among agents. However, language may introduce ambiguity and vagueness, specifically when accomplishing complex knowledge-intensive tasks that require planning and coordination. Hence, we extend “data” to include “structured data” from basic types, such as numbers, to more compound types, such as JSON. We believe incorporating more structure to data is essential to improving controllability.
Beyond data, coordination also necessitates control. In Blue, instructions are special messages that agents can generate to impose control. This allows agents to plan and coordinate workflows, as well as simply have other agents do something for them.
Redefining ‘Orchestration”
To help facilitate “Agentic for Enterprise,” we have introduced several concepts, including:
- Streams to facilitate data, control, and communication among agents
- Messages in streams to standardize data and instructions for agents
- Registries to capture metadata about data, agents, and more
- Session to provide context for computation
- Plans to represent workflows and execution
- and more…
At a high level, a session is created with a set of agents. Data comes in dynamically generated streams. For example, this could come from a user (agent) interacting, typing in text, or from an LLM generating text, and even from a sensor producing new readings. Agents in the session have agency to consume data in the streams, i.e. process data and instruction messages in streams, and in the process write data and instruction messages to new streams for other agents to process.
Sessions, in essence, provide the context of work. Sessions also have memory (key-value store) that agents can read/write from. Similarly, agents have their own memories (not shared) in the session scope. These additional memory constructs give sufficient power to build applications where agents can communicate and collaborate effectively.
In subsequent posts, we will go deeper into these concepts and explain how they help facilitate scalability, observability, configurability, and control, all critical components for agentic enterprise applications. To give a concrete example, a report generation use case might involve multiple data sources in an enterprise, different types of queries, visualizations, and summaries generated from these diverse data sources, plus agents checking facts. This might also incorporate experts as user agents who help to review, direct, and produce the final outcome: a report.
Why v0.9?
We have been working on this framework for nearly two years and have been through many iterations. We learned a lot from these earlier versions. The framework evolved and matured over the years, but we still believe we have not fully realized Blue’s potential. In other words, we have big plans for this framework, and we know some key areas to improve upon.
Yet, in a (very) fast-moving space, we also feel the urge to put the framework out sooner and share some of the architectural concepts with the community. We want to engage with the community to develop next versions. We would love to hear your opinion on our design, your use cases, and the agents you plan to build. We are also planning to get more direct contributions to the repository.
Together, we want to explore the design space of agentic orchestration systems, to support a broad variety of use cases:
- from conversational interactions to non-interactive use cases,
- from fixed workflows to ad hoc planned workflows,
- from purely textual interaction to interactive agents with user interfaces, visualizations, and beyond.
Blue is designed such that it can be configured to support many of these use cases.
What Can You Do?
Here are a few examples you can build with Blue:
- a set of agents that convert natural language to SQL, then execute and summarize results in natural language
- agents that produce interactive graphical user interfaces and visualizations with your data (e.g. self-service business intelligence)
- a conversational agent that interfaces with existing predictive models and APIs (e.g. job search agent with predictive models and data)
- agents that execute workflows processing text data, extracting and populating databases.
Upcoming Posts
We will follow up with a three-part blog, where we will tackle the key concepts in the architecture step by step. In the first part of the blog, we will talk about the enterprise, opportunities, and requirements that come with it, and discuss the rationale behind some of the conceptual designs of Blue. In the second part, we will primarily focus on “streams” as the key orchestration concept in Blue and argue why it’s the right abstraction for orchestrating data and control for agentic architectures. In the last part, we will introduce the overall blueprint for our agentic architecture and Blue’s implementation and then discuss some of the key design rationales.
Blue v1.0 is in the works and coming soon.
Stay tuned!
Written by: Eser Kandogan and Megagon Labs
