2: What makes an AI agent unique?

This article is part of a series that I started to write about AI agents. If you missed the first one, start here. Now let's delve into what makes an AI agent unique. Each AI agent is distinguished by three fundamental characteristics: Instructions, Knowledge, and Actions. These elements are the building blocks that define the behavior and capabilities of an AI agent.

Instructions

Instructions are the directives that set the context for the AI agent and guide its behavior. They are akin to the rules of engagement, providing a framework within which the AI agent operates. For instance, in a conversational AI agent, instructions could be designed to facilitate more than just short exchanges. They could be crafted to enable the agent to retain context across multiple turns of conversation, thereby creating a more human-like interaction

In the realm of large language models (LLMs), instructions are often provided through prompt engineering. This involves carefully crafting prompts to provide clear instructions and context to the model. The use of specific keywords, phrases, or formatting can guide the model's response. For example, a well-crafted prompt can significantly improve the accuracy of AI assistants' responses to our requests

Knowledge

Knowledge is the information that an AI agent uses to make decisions and generate responses. Large Language Models (LLMs) are trained on vast amounts of data, but they often lack domain-specific knowledge. Without this knowledge, they may not be able to answer specific user questions. To address this, we can feed these models with domain-specific knowledge at execution time using a method called Retrieval Augmented Generation (RAG)

RAG combines an information retrieval component with a text generator model. It retrieves a set of relevant documents given a source (e.g., Wikipedia), concatenates them as context with the original input prompt, and feeds this to the text generator, which produces the final output

This method allows the AI agent to access the latest information for generating reliable outputs without needing to retrain the entire model

Knowledge can come from various sources such as web data scraping, PDF documents, or help sections such as GitBook or Zendesk help The data should be in the form of text so that we can convert them into embeddings. These embeddings are then stored in a vector database for retrieval during generation.

For instance, an AI agent designed to assist with software development might be fed with knowledge from programming documentation, Stack Overflow threads, and GitHub repositories or even documentation from specific websites.

Actions

Actions are the outputs that the AI agent generates. Multiple agents can have the same knowledge but can perform different actions. These actions can range from simple text generation to image generation or they can be as complex as building a JSON as input and calling an API to get results Actions can also be inputs to another AI agent, creating a chain of AI agents each performing a specific task. For example, in the context of a React application, the action could be the generation of an image based on a textual prompt using the OpenAI DALL-E API

In another scenario, an AI agent could be designed to extract answers from PDF documents, which is an action that involves complex data extraction and processing

Actions can also be inputs to another AI agent, creating a chain of AI agents each performing a specific task. For instance, in a manufacturing process, one AI agent could be responsible for quality control, inspecting products and flagging any defects. The actions of this agent (i.e., the flagged defects) could then serve as inputs to another AI agent responsible for troubleshooting and repair.

. Summary

In conclusion, the uniqueness of an AI agent lies in the combination of its Instructions, Knowledge, and Actions. Instructions provide the context and guide the agent's behavior. Knowledge, often augmented by methods like RAG, equips the agent with the information it needs to generate responses. Actions are the outputs that the agent produces, which can range from text and image generation to complex data processing tasks. The interplay of these three elements defines the capabilities of an AI agent, making each one unique in its own right.