In previous posts, we introduced our agentic framework, Blue, and delved into “agentic for enterprise” requirements and how Blue supports enterprise integration through various registries in the Blue architecture.
In this blog post, we will explore streams, another key abstraction in Blue, to facilitate the execution of complex tasks with workflows. We will discuss how streams are an essential orchestration abstraction and argue why they are the right abstraction for orchestrating data and control for agentic workflows to address enterprise scalability requirements.
What are streams? What is stream processing?
In computer science, a stream is essentially a dynamic data structure, representing a sequence of data elements that can be processed as they arrive. For example, data from a sensor can be represented as a stream of readings. Stream processing is a programming paradigm with streams as the first-class objects for computation, with operators to distribute, consume, and process data in streams. Operators such as map, filter, split, and reduce can facilitate complex workflows to support a wide range of computations over data. In fact, stream processing has been applied in many aspects of computing, from operating systems and distributed systems to more applied areas such as data processing and analytics.
Streams in Blue
In Blue, streams are the key orchestration abstraction. Streams facilitate the distribution of data and control. As such, streams form the basis of communication among agents and other components in Blue.
Why streams?
Before diving into the specifics of streams in Blue, let’s consider why streams are the right abstraction for agentic workflows.
First and foremost, streams can express complex workflows efficiently and effectively, naturally breaking down work by data and tasks. Streams are also dynamic structures; both the data content and connections between streams can be created and updated at any time to provide scalability and flexibility in how data is processed.
Secondly, streams bring observability and controllability benefits. By explicitly representing the flow of data and control in a persistent data structure, streams can be examined for debugging and performance issues. Observability is critical for ensuring software verification as well as product design perspectives. For end users, it helps them gain confidence in system results.
From a production perspective, streams also bring the much-needed separation of concerns, subsequently improving reliability. Streams—acting as a communication bus among agents and other components, agents, and components—can be independently configured, deployed, and scaled, for performance and reliability reasons.
Lastly, streams are also a familiar concept in the LLM/AI world. In fact, feeding and consuming data from LLMs is pretty similar to how stream computing works. Many of the LLM APIs provide streaming output to facilitate the processing of data as it becomes available. Going beyond LLMs where text is the only data type, streams can build on and expand to more structured data in order to coordinate with agents. And so, streams bring structure to both content (reducing ambiguity in data semantics) and flow of data (reducing ambiguity in data processing), making the overall systems more controllable.
How do streams work in Blue?
A stream is essentially a sequence of data, which we call messages, in Blue. Streams have unique IDs, and tags to serve as descriptors. For example, a stream can be tagged USER to indicate that the contents of the streams originate from a user.
Each stream begins with a special message, BOS, representing the beginning of a stream, followed by any number of messages and then concluding with another special message, EOS, representing the end of the stream. BOS and EOS allow consumers (agents) of stream data to initialize and finalize computation or resources.
Stream messages
Each message in the stream is a tuple, comprising three components: k, representing the key or ID of a message, v representing data, and t representing the tag of a message. k can uniquely identify data, but can also be used as a key in computation, like for aggregation. v, data in the message, is standardized to represent structured data of various types, from core data types such as float and str to JSON as a way to represent compound data. t represents the semantics of data and can have multiple values, from basic DATA and CONTROL tags to more application-specific semantics. For example, a stream can contain a sequence of documents such as resumes, where each message is a resume, with resume ID, contents, and tags such as DOC, RESUME.
Most of the messages in Blue are data messages; they provide data for agents to consume and process. However, in Blue, there are also control messages, which are essentially instructions for an agent to process a stream. For example, an agent can output a message to ask another agent to compute something. This is similar to tool calling, except that it expands beyond tools to include agents. A clear distinction is that these are instructions (sort of like interpreted code) that are dynamically created, linking agents with each other. Furthermore, instructions are explicitly represented just like data is, in some ways like an instruction queue in computer systems. Control messages can be used by any agent to ask another agent to compute something, as well as planner agents that govern the overall flow of execution. Control messages can be included within any stream as well as dedicated instruction streams.
Control messages have the same tuple components, k to represent the key/id, v that has a specific structure to describe control message structure, in the case of agent execution instruction, the agent (A) to execute along with optional properties (P) and stream (S) to process, along with a t, that explicitly tags the message as a control and instruction message.
Streams and agents
In Blue, agents are the main consumers and producers of stream data. An agent defines a processor function that is executed on every message in the stream, optionally writing data into new streams. An agent can have multiple input and output parameters, meaning that input data streams map to a specific input parameter of the processor, and output data can be designated for a specific output parameter. For example, an intent classifier agent can have a named input parameter such as TEXT and an output parameter such as INTENT. Agents also have properties that can guide the data processing. For example, an intent classifier agent can define a model property to work with for intent identification.
Agents can connect to streams for processing through listeners declared for each input parameter. For example, an intent classifier can specify to listen to streams tagged as USER for input parameter TEXT. Additionally, agents can also initiate stream processing through explicit instructions in the streams, as we described earlier.
This model of agents as stream data processors promotes clarity and simplicity, making it easier to understand and manage the flow of data through an application.
Let’s describe how complex workflows can be built in this model.
Complex workflows
A workflow is essentially a sequence of tasks executing over data to achieve a specific goal. In agentic orchestration, this maps to organizing agents and the data they work with to collectively produce a desired outcome. A workflow can be represented as a DAG, directed-acyclic graph where nodes represent processors and edges represent connections between agents’ input and output data.
In Blue, nodes are agents with multiple inputs and outputs, and edges specifically designate matching inputs and outputs between agents. For example, an NL2SQL output stream for parameter SQL can be linked to SQL EXECUTOR agent input parameter SQL to execute.
Beyond just defining the sequence of execution, workflows can express complex processing pipelines by breaking down data and tasks. Streams are very suitable for expressing such complex workflows.
Simply put, sequences of data naturally map to iteration, and any filtering done by the agents maps to conditional statements. Output data is distributed to different streams to execute specific tasks that mimic procedure calls and, as such, they facilitate abstraction and scoping. Lastly, streams joined together later can aggregate results.
In Blue, streams’ natural ability to express complex workflows is further expanded, as streams can also contain control messages such as instructions for agent execution. Specific agents, such as planner agents, can utilize these constructs to create plans of sufficient complexity matching the expressive power of many programming languages.
Let’s walk through an example.
An Example: Real Estate Analytics
You are looking to buy a house. There are several neighborhoods you are interested in, many different houses, each with different pros and cons. Some have large lots, some are in districts with good schools, and some are close to public transportation. You have many personal and familial considerations too. You obviously have a budget, maybe kids that are in middle school, so education is important. Your partner works in the city, so you want to be close to the metro station.
How do we go about navigating such a complex workflow?
First, Blue can easily scale by distributing work and setting the right scope for each. You can divide up houses and distribute them to different streams. It is like having multiple real estate agents whom you can direct to focus on specific neighborhoods, specific houses. They can work in parallel and come back to present the results.
Secondly, each property is different so they need to be examined individually, carefully considering specific trade-offs. With Blue, you can create plans specific to each house, analyze, prioritize, and go deeper, with specific data to help in that situation. For one house, you can tap into land models to analyze lot conditions more deeply. For another house, you can use school-review databases, examining the school quality.
Lastly, Blue can put it all together, aggregating all the findings from these different agents. You can compare and contrast reports, examine trade-offs in house condition, and school quality. Blue can help you make a buying decision easily, explaining findings along the way.
How do streams differentiate Blue?
As the central orchestration abstraction, streams set Blue apart from other frameworks in regards to expressivity and control.
In most approaches, natural language is the only form of data agents consume and produce. As language is ambiguous, text-only approaches can be problematic for business applications. In Blue, on the other hand, data passed between agents is structured. This helps in regards to semantics and control.
Regarding the flow of data, and patterns of communication among agents, Blue excels. In fact, streams-based workflows allow Blue agentic workflows to match the computational power of many programming languages.
Workflows can be designed a priori (through explicit linking of agent inputs and outputs), on demand (through planners), and can be ad hoc based on agents’ declaration of interests (through listeners) to support complex knowledge-intensive tasks that require planning and coordination. Agents can generate special control messages to impose control. This allows agents to plan and coordinate workflows, like delegating to other agents.
Lastly, streams serve as context for agentic workflows. This is a key difference between Blue and other agentic frameworks where context can be specifically scoped for the task, as opposed to simply expanding context as processing continues, making agentic workflows increasingly slower and reducing quality as a result.
Conclusion
We strongly believe that stream processing is a key ingredient when making “agentic workflows” enterprise-ready. Streams support a wide range of workflows and support complexity while at the same time bringing about the right abstractions and scope for facilitating accuracy, scalability, and ease of use.
Next post and beyond
In the next post, we are going to put it all together, describe the overall architecture of the Blue agentic framework, and how the various components and abstractions we introduced thus far work together. We will also discuss unique aspects of the blue architecture and the advantages it brings over other agentic frameworks.
As we make progress with Blue, especially in areas of planners and advanced streaming functions, we will follow with additional posts.
For now, check out Blue v0.9 and start playing with it. If you have ideas, questions, or want to make contributions, reach out through our community.
Written by: Eser Kandogan & Megagon Labs
