AutoGen is a framework that supports the development of LLM applications using multiple agents that can talk to each other to solve tasks. AutoGen agents are customizable, conversational, and seamlessly allow human participation. They can operate in various modes using LLM (Large Language Model), human input and tool combinations.
Main features
- AutoGen makes it easy to build next-generation LLM applications based onMulti-agent conversations. It simplifies the orchestration, automation and optimization of workflows for complex big data models. It maximizes the performance of LLM models and overcomes their weaknesses.
- It supports multiple conversation modes for complex workflows. With customizable and conversational agents, developers can use AutoGen to build a variety of conversation patterns involving conversation autonomy, number of agents, and agent conversation topology.
- It provides a range of working systems of varying complexity. These systems cover a wide range of applications in a variety of domains and complexities. They demonstrated how AutoGen can easily support different conversation modes.
- AutoGen provides enhanced LLM inference. It provides simple performance tuning, as well as utilities such as API unification and caching, as well as advanced usage patterns such as error handling, multi-configuration inference, contextual programming, and more.
AutoGen is powered by a collaborative research effort between Microsoft, Penn State University, and the University of Washington.
Quick Start
Install from pip pip install pyautogen: Find more options in the installation. For code execution, we strongly recommend installing the python docker package and using docker.
Multi-agent dialogue framework
Autogen supports next-generation LLM applications through a common multi-agent conversation framework. It provides customizable and conversational agents, integrating LLM, tools and people. By automating chats among multiple capable agents, one can easily have them perform tasks autonomously or collectively based on human feedback, including tasks that require the use of tools through code. For example,_
from autogen import AssistantAgent, UserProxyAgent, config_list_from_json
# Load LLM inference endpoints from an env variable or a file
# See https://microsoft.github.io/autogen/docs/FAQ#set-your-api-endpoints
# and OAI_CONFIG_LIST_sample.json
config_list = config_list_from_json(env_or_file="OAI_CONFIG_LIST")
assistant = AssistantAgent("assistant", llm_config={<!-- -->"config_list": config_list})
user_proxy = UserProxyAgent("user_proxy", code_execution_config={<!-- -->"work_dir": "coding"})
user_proxy.initiate_chat(assistant, message="Plot a chart of NVDA and TESLA stock price change YTD.")
# This initiates an automated chat between the two agents to solve the task
The following figure shows a sample dialog flow for AutoGen.
Enhanced large model inference
Autogen also helps maximize the utility of expensive LLMs like ChatGPT and GPT-4. It provides enhanced LLM inference along with powerful features such as tuning, caching, error handling, and templates. For example, you can optimize builds with LLM using your own tuning data, success metrics, and budgets.
# perform tuning
config, analysis = autogen.Completion.tune(
data=tune_data,
metric="success",
mode="max",
eval_func=eval_func,
inference_budget=0.05,
optimization_budget=3,
num_samples=-1,
)
# perform inference for a test instance
response = autogen.Completion.create(context=test_instance, **config)
Multi-agent dialogue framework
AutoGen provides a unified multi-agent dialogue framework as a high-level abstraction using the underlying model. It features powerful, customizable, and conversational agents that integrate LLM, tools, and people through automated agent chat. By automating chats among multiple capable agents, one can easily have them perform tasks autonomously or collectively based on human feedback, including tasks that require the use of tools through code.
The framework simplifies the orchestration, automation and optimization of complex LLMs workflows. It maximizes the performance of LLM models and overcomes their weaknesses. It enables building next-generation LLM applications based on multi-agent conversations with minimal effort.
Agent
AutoGen abstracts and implements conversational agents and is designed to solve tasks through inter-agent dialogue. Specifically, agents in AutoGen have the following salient features:
-
Conversable: Agents in AutoGen are conversational, meaning any agent can send and receive messages from other agents to initiate or continue a conversation
-
Customizable: Agents in AutoGen can be customized to integrate LLMs, people, tools, or a combination thereof.
The following image shows the built-in agents in AutoGen. Agent chat example
We designed a general ConversableAgent class for agents, which can talk to each other by exchanging messages to complete tasks together. Agents can communicate with other agents and perform actions. Different agents may perform different actions after receiving a message. Two representative subclasses are AssistantAgent and UserProxyAgent.
AssistantAgent is designed to act as an artificial intelligence assistant, using LLMs by default but requiring no human input or code execution. It can write Python code (in Python coding blocks) for the user to execute when it receives a message (usually a description of a task that needs to be solved). Under the hood, Python code is written in an LLM such as GPT-4. It can also receive execution results and recommend corrections or bug fixes. Its behavior can be changed by passing new system messages. LLM inference configuration can be configured through llm_config.
Conceptually, the UserProxyAgent is a proxy for humans, soliciting human input as the agent’s reply by default on every interaction, and also has the ability to execute code and call functions. Code execution is automatically triggered when the UserProxyAgent detects a block of executable code in a received message and no human user input is provided. Code execution can be disabled by setting the parameter code_execution_config to False. LLM-based responses are disabled by default. It can be enabled by setting the dictionary corresponding to the inference llm_config configuration. When set to a dictionary, LLM can be used to generate replies without executing code. llm_configUserProxyAgent
The auto-reply feature ConversableAgent allows for more autonomous multi-agent communication while retaining the possibility of human intervention. One can also easily extend it by registering a reply function using the register_reply() method.
In the following code, we create an AssistantAgent named “assistant” as the assistant, and a UserProxyAgent named “user_proxy” as the human user’s proxy. We will use these two agents later to solve the task.
from autogen import AssistantAgent, UserProxyAgent # create an AssistantAgent instance named "assistant" assistant = AssistantAgent(name="assistant") # create a UserProxyAgent instance named "user_proxy" user_proxy = UserProxyAgent(name="user_proxy")
Multi-agent dialogue
A basic two agent conversation example
Once the participating agents are properly built, a multi-agent session can be started via an initialization step, as shown in the following code:
# the assistant receives a message from the user, which contains the task description
user_proxy.initiate_chat(
assistant,
message="""What date is today? Which big tech stock has the largest year-to-date gain this year? How much is the gain?""",
)
After the initialization steps are completed, the conversation can proceed automatically. Here’s a visual illustration of how user_proxy and assistants work together to automatically solve the above tasks: Agent Chat Example
- The Assistant receives a message from user_proxy containing a task description.
- The assistant attempts to write Python code to solve the task and sends the response to user_proxy.
- Once the user_proxy receives the assistant’s response, it attempts to reply by requesting human input or preparing an automatically generated reply. If no human input is provided, user_proxy executes the code and uses the result as an autoresponder.
- The helper then generates further responses for user_proxy. The user_proxy can then decide whether to terminate the session. If not, repeat steps 3 and 4.
Supports multiple conversation modes
With varying degrees of autonomy and modes of human involvement
On the one hand, a fully autonomous dialogue is possible after the initialization step. AutoGen, on the other hand, can be used to implement human involvement in problem solving by configuring the human involvement level and mode (e.g., setting human_input_mode to ALWAYS ), as in many Human involvement is expected and/or required in the application.
Static and dynamic conversations
AutoGen inherently allows for dynamic conversations through conversation-driven control using programming and natural language. Dynamic sessions allow the agent topology to change based on the actual session flow under different input problem instances, whereas the flow of a static session always follows a predefined topology. Dynamic conversation mode is useful in complex applications where interaction modes cannot be scheduled in advance. AutoGen provides two general methods for implementing dynamic dialogue:
-
Autoresponder registered. With pluggable autoresponders, people can choose to engage in conversations with other agents based on the content and context of the current message. A working system demonstrating such a dynamic conversation can be found in this code sample, which demonstrates a dynamic group chat. In the system, we registered the auto-reply feature in the group chat manager and let the LLM decide who will speak next in the group chat settings.
-
LLM-based function call. In this approach, LLM decides whether to call a specific function based on the dialog state in each inference call. LLM can drive dynamic multi-agent conversations by sending messages to other agents within the called function. Working systems that demonstrate this type of dynamic dialogue can be found in multi-user mathematical problem-solving scenarios, where student assistants will automatically resort to experts using function calls.
Using AutoGen
The following image shows six examples of applications built using AutoGen. Application areas
1. Code generation, execution and debugging
Automate task solving with code generation, execution, and debugging
Automatic code generation, execution, debugging and human feedback
Automatically generate code and Q&A using retrieval-enhancing agents
2. Multi-agent collaboration (>3 agents)
- Automatically solve tasks using GPT-4 + multiple human users
- Automate task solving via group chat (with 3 group member agents and 1 manager agent)
- Automated data visualization via group chat (includes 3 group member agents and 1 manager agent)
- Automate complex tasks via group chat (with 6 group member agents and 1 manager agent)
- Automate task solving using coding and planning agents
3. Application
- GPT-4 agent automatically plays chess game and chats
- Automatically learn continuously from new data
- OptiGuide – Large-scale language models for supply chain optimization.
4. Tool usage
- Web Search: Solve tasks that require information from the Web
- Use the provided tools as functions
- Use the tools provided by Langchain as functions to solve tasks
- RAG: Group Chat with Retrieval Enhanced Generation (with 5 group member agents and 1 manager agent)
- An in-depth guide to OpenAI utility functions
5. Agent teaching
- Teach agents new skills through automated chat and reuse them
- Teach agents new facts, user preferences and skills beyond coding
Information source:
AutoGen Getting Started
AutoGen Multi-agent Conversation Framework
Other information
I currently have an account with w_x, yqpan1991. Regarding the latest AI consulting and information related to AI framework, everyone is welcome to pay attention and communicate. When adding an account, please note, from csdn