My multi-talented bot built using BroadAI Agentic Framework
Chatbot, by far, is still an exciting interaction channel through which most of us like to engage with something that is artificially intelligent. Even the Turing tests involve chat-like interactions to disguise a machine as a human (of course, to validate human-like qualities from the machine, not for cheating!). Chatbot development has evolved tremendously from rule-based chatbots (non-AI) to intent-based bots (NLP-based).
The latest, GenerativeAI-based chatbots rely on extracting relevant prior knowledge using techniques such as semantic retrieval from the stored knowledgebases and grounding it with an LLM prompt to generate meaningful responses, albeit even allowing contextual human-like follow-ups. This is popularly known as the RAG (Retrieval Augmented Generation) approach. Although this approach may work best for scenarios that rely on knowledge sources such as policies, documentation, or textbooks (law, medicine, history, etc.), one challenge with RAG-based chatbots is that they are monotonous and high maintenance (for knowledge management). Such applications/bots also lack awareness about current events and real-time information.
So, in this blog, I plan to share the latest Agentic approach to building bots. The main idea behind this approach is to equip the system (chatbot, in this case) with multiple talents or skills and exploit GenAI intelligence to not only "generate" information but also perform higher intelligence functions, such as planning a systematic approach using one or more of its talents to find the solution to a given question.
Preface on BroadAI Framework
Before we get into the specifics of the approach taken here to build an intelligent chatbot, it might be essential to spend about 30 seconds understanding how the BroadAI framework works, which allows it to be the Agentic framework of choice used in this approach.
BroadAI is a framework that supports the latest iteration around building smart and intelligent AI systems. It takes a Multi-Agent Systems (MAS) approach, where an AI system can demonstrate the orchestration of complex skills when working in an environment. Multiple expert agents with specialized skills collectively work towards achieving a given goal.
Some critical features of the BroadAI Agentic framework are:
- Low-code developer accelerator for incorporating Generative AI in applications.
- Plug-and-play agentic approach. Develop your own BroadAI agents using BroadAI Agentic Framework or incorporate them from the Agent Store to add broader capabilities to your application.
- There is no need to be an expert in prompt engineering. Well-designed prompts are built into the BroadAI MAS framework.
- BroadAI framework effectively utilizes multiple agentic design patterns, viz. Planning, Reflection, Multi-Agent system, and Tool use.
- Build stateless and stateful applications by simply enabling a configuration flag.
The Big Picture
The final product can be seen on my website, https://mangeshpise.com, where you can engage with the chatbot by clicking on the 💬 chat icon.
The high-level architecture involves the primary website, hosted on a web server, and the landing page, which contains the chatbot. The chatbot sends the user's question to an API endpoint deployed as a serverless service, which performs the heavy lifting of taking the user's question and returning a response. The serverless service implements the BroadAI MAS framework along with multiple BroadAI Agents that, depending on their specific skills, integrate with LLMs, other API endpoints, or their own RAG infrastructure to generate a response. How each BroadAI Agents must be engaged is solely and dynamically determined by the BroadAI MAS framework based on the user's question. Thus, this agentic AI system can produce better outcomes for user interactions as it engages one or more skills incorporated within the system. Further, BroadAI allows one to easily extend the skillset by registering additional agents that can be downloaded from its agent store.
As shown in the diagram above, I used the FinancialAnalyst, NewsReporter, Randomizer, and Writer agents from BroadAI's Agent Store. At the same time, I built my agent, Mangesh, using BroadAI's Agentic framework. Mangesh agent has been designed to look up all of my published blogs from a Vector store and provide a point-of-view on the topic asked in the user's question. Notice the different skills these agents offer, which allows BroadAI to answer a broad range of questions.
To provide a sense of benefits with this approach, below are two examples of the interactions with this chatbot:
Help me draft a short email covering Intel’s business, stock trends, and latest financial performance. I’d also like to include more if there is any headlines involving them or other players in their industry.
Subject: Update on Intel Corporation
Dear Friends,I wanted to share some insights about Intel Corporation, a leading player in the semiconductor industry, led by CEO Mr. Patrick P. Gelsinger. Recently, Intel has been focusing on innovation and expanding its manufacturing capabilities, which is crucial for maintaining its competitive edge in the market. In terms of financial performance, Intel reported a revenue of approximately $54.2 billion for the fiscal year ending December 30, 2023. The company achieved a gross profit of around $21.7 billion, with a net income of about $1.7 billion, translating to an earnings per share (EPS) of $0.4. You can find more detailed financial information in their latest SEC filing [here](https://www.sec.gov/Archives/edgar/data/50863/000005086324000010/intc-20231230.htm). Looking at the stock trends, on September 20, 2024, Intel’s stock closed at $21.87, reflecting a change of approximately 4.69% from the previous day. The stock has experienced some fluctuations, with a recent high of $23.14 and a low of $20.35. It’s worth noting that the semiconductor industry is continually evolving, and Intel’s recent strategies are essential to watch as they navigate this competitive landscape.
Best, [Your Name]
The agentic approach to answering this question involved formulating a step-by-step plan that leveraged more than one available agent and their appropriate skills to come up with this response, as shown below:
Step 1. Gather Intel’s business and financial information.
FinancialAnalyst.companyProfile {“stock_symbol”:”INTC”}Step 2. Obtain Intel’s income statement for latest performance.
FinancialAnalyst.incomeStatement {“stock_symbol”:”INTC”}Step 3. Retrieve Intel’s latest stock trends.
FinancialAnalyst.historicalStockPrices {“stock_symbol”:”INTC”,”number_records”:1}Step 4. Collect recent news related Intel and its industry.
NewsReporter.getHeadlines {“country_code”:”US”,”category”:”business”}Step 5. Draft short email summarizing findings.
Writer.composeEmail {“situation”:”Summarizing Intel’s business, stock trends, and financial performance”,”outcome”:”Provide concise overview Intel’s current status and news.”}
This is another example that involved asking a question that needs reflecting on one of my blog posts about the deep-dive prompting technique (used to generate better-quality LLM responses). See the results below:
In English and Hindi, tell me what inspired you to develop the ‘deep-dive’ prompting technique, and how does it differ from other methods?
‘Deep-dive’ prompting technique is inspired by concepts such as zero-shot prompting, few-shot prompting, chain-thought prompting, and Retrieval-Augmented Generation (RAG). This technique differs from other methods as it allows for a more comprehensive exploration of topics, enabling a deeper understanding and context in responses.
गहरी-खुदाई करने की तकनीक ऐसे विचारों से प्रेरित थी जैसे जीरो-शॉट प्रॉम्प्टिंग, कुछ-शॉट प्रॉम्प्टिंग, चेन-थॉट प्रॉम्प्टिंग, और रिट्रीवल-ऑगमेंटेड जनरेशन (RAG)। यह तकनीक अन्य तरीकों से इस प्रकार भिन्न है कि यह विषयों की अधिक व्यापक खोज की अनुमति देती है, जिससे प्रतिक्रियाओं में गहरी समझ और संदर्भ संभव होता है।
Answering this request involved developing the following plan using the custom Mangesh Agent that can research my blogs and the Writer agent integrated from BroadAI's Agent Store, as shown below:
Step 1. Generate response in English about deep-dive prompting technique inspiration and differences from other methods
Mangesh.referPublications {“topic”:”deep-dive prompting technique inspiration and differences from other methods”}Step 2. Translate response into Hindi
Writer.translate {“originalText”:”The ‘deep-dive’ prompting technique was inspired by concepts such as zero-shot prompting, few-shot prompting, chain-thought prompting, and Retrieval-Augmented Generation (RAG). This technique differs from other methods by enabling a more comprehensive exploration of topics, allowing for deeper understanding and context in responses.”,”languageKey”:”hi”})
Low-Code approach to infusing GenAI in Applications
The heavy lifting mentioned above, which deals with determining the best approach to engage different agents and their skills to lead to an answer, is entirely encapsulated within the BroadAI MAS framework. So, from a developer’s standpoint, they need to pass the question to BroadAI and instantly receive a response.
The code looks something like below:
/* Import BroadAI MAS Framework */
import BroadAI from "broadai";
/* Import BroadAI Agents (custom agent Mangesh) */
import mangesh from "broadai-agent-mangesh";
/* Import BroadAI Agents (from Agent Store) */
import financialAnalyst from "broadai-agents/FinancialAnalyst";
import newsReporter from "broadai-agents/NewsReporter";
import randomizer from "broadai-agents/Randomizer";
import writer from "broadai-agents/Writer";
const _QUESTION_ = req.body['q']; // assuming user's question is passed as an API body parameter
// Create an instance of BroadAI MAS framework
// .............................................
let broadai = new BroadAI(
[mangesh.agent, financialAnalyst.agent, newsReporter.agent, randomizer.agent, writer.agent],
{ /* broadai configuration discussed later */ }
);
/* -- agent registration discussed later -- */
// *** MOST IMPORTANT PART OF BROADAI *** //
// Handoff user's question to BroadAI using .go() method
// ..........................................................
broadai.go(_QUESTION_, audit, conversationHistory)
.then((response) => {
res.json(response);
});Internally, the BroadAI MAS framework develops and executes the plan by identifying relevant use of agent skills. Refer to the examples shown previously to see what the plan steps entail. BroadAI MAS framework also takes the burden of identifying appropriate input values required for each skill to provide the necessary output. For example, from the first example above, the FinancialAnalyst agent’s companyProfile skill identifies the stock symbol as “INTC”, which it intelligently determines based on the company, “Intel”, found in the user’s question. Likewise, the NewsReporter agent’s getHeadlines skill requires country code (which it identifies as ‘US’) and news category (which it describes as ‘business’).
This low-code approach enables developers to focus on the application's core functionality. At the same time, the Generative AI aspects are encapsulated in a combination of the BroadAI MAS framework and the BroadAI Agentic framework that the agents leverage.
Double-click on Agents
The most advantageous feature of the BroadAI framework is the Agent Store. Although a limited number of agents are available at this point, you can develop and submit agents to be listed on their Agent Store. BroadAI provides a dedicated BroadAI Agentic framework to help you build your agents.
There is great potential to leverage BroadAI’s Agent Store as a monetization channel by publishing your agents. For example, the NewsReporter agent on the Agent Store requires users to sign up and obtain an API Key from newsapi.org. Likewise, one could build and publish their agents, granting access via API Keys.
BroadAI’s Agentic framework documentation provides a template to expedite building the agent. On a high level, creating an agent involves defining the skills an agent is expected to offer. An example of the agent I built for my use case is shown below:
/* Import BroadAI Agentic Framework */
const Agentic = require('broadai/agentic');
/* Define Agent */
module.exports = {
register: (broadaiConfig, agentConfig) => {
agentic = new Agentic(broadaiConfig);
params = agentConfig;
},
agent: {
"agent-name": "Mangesh",
"capability": "A digital-twin that has full knowledge of Mangesh Pise's experience and point-of-views expressed in his blogs.",
"skills": [
{
"skill-name": "referPublications",
"objective": "upon identifing given question is a request for point-of-view, generate a PoV using your published blogs",
"action": referPublications,
"parameters": {
"topic": "string | topic of conversation"
}
},
{
"skill-name": "referExperience",
"objective": "upon identifing given question is a job interview question, generate a professional interview-style response",
"action": referExperience,
"parameters": {
"question": "string | interview question"
}
}
]
}
};
/* Implement Skills */
const referPublications = (topic)=>{
...
};
const referExperience = (question)=>{
...
};BroadAI Agentic framework provides multiple utility functions that help implement the skills. For example, one of the utility functions, called callExternalResource(), helps call external APIs to retrieve real-time data. In contrast, another utility function, getGenAIResponse(), utilizes LLM to perform the desired task. You could contact the BroadAI team for help if you need support creating an agent.
Incorporating BroadAI Agents in BroadAI MAS
BroadAI Agents, either custom-built using the BroadAI Agentic framework or from the Agent Store, are incorporated in your application via the BroadAI MAS framework, some parts of which we saw earlier. It is a two-way handshake, where the first step informs the BroadAI MAS instance about agents that will be involved, and the second step registers the agent with the required parameters and authentication. See below for an example:
/** ***** ***** ***** ***** ***** ***** *****
This code is part of your application
***** ***** ***** ***** ***** ***** ***** **/
// Import BroadAI MAS Framework & BroadAI Agents
import BroadAI from "broadai";
import newsReporter from "broadai-agents/NewsReporter";
// STEP 1: Inform BroadAI MAS instance about agents that will be involved
// ........................................................................
let broadai = new BroadAI(
/* broadai agents list */
[newsReporter.agent],
{ /* broadai configuration */ }
);
// STEP 2: Register agents with required parameters and authentication
// .....................................................................
newsReporter.register(broadai.config, { "newsapi": env['newsapi'] }); // refer agent documentation
// FINAL STEP: Handoff user's question to BroadAI using .go() method
// ...................................................................
broadai.go(_QUESTION_, audit, conversationHistory)
.then((response) => {
res.json(response);
});This approach provides an added benefit that new capabilities can be added via additional agents as they are either built or published, analogous to the learning ability of any person. As a result, a modular agentic approach is enforced that completely eliminates the application code management burden from the core application developer, and AI capabilities are encapsulated within agents supported by a solid framework.
Conclusion
Chat-based applications are just one example of using an agentic approach to incorporate Generative AI capabilities into your applications. Agentic workflows can be highly advantageous in many other applications. A few examples of applications powered by multi-talented agents are provided on BroadAI’s demo page. Finally, I’d like to conclude by highlighting some key benefits of using the Agentic approach, and specifically the BroadAI framework, to develop modern applications:
1. Drive autonomy and decision-making
2. Build task-oriented and goal-driven systems
3. Improve context awareness
5. Take a multi-step approach with complex intelligent reasoning
6. Incorporate dynamic interaction with external tools and real-time information
7. Improve collaboration between multiple systems which can be considered as autonomous agents
Additional references:
https://broad-ai.github.io/ — BroadAI framework home page
https://youtu.be/v85IR2VoBNY — [Video] Why another Agentic framework?
https://youtu.be/_lAbdhE9FaY —[Video] BroadAI Multi-Agent System teaser
https://medium.com/@mangeshpise/what-exactly-are-agents-in-the-context-of-agentic-ai-fd9318fd8e1a — [PoV] What exactly are “Agents” in the context of Agentic AI?
