by NVIDIA
Connect and optimize teams of AI agents across any framework, data source, or tool, while providing profiling, observability, evaluation, and a UI for rapid development and debugging.
Nvidia Nemo Agent Toolkit is a lightweight, framework‑agnostic library that enables developers to link existing enterprise agents to various data sources, tools, and services. It offers reusable components, profiling, observability integrations, evaluation utilities, and an interactive UI, all while supporting the Model Context Protocol (MCP) for client‑server tool sharing.
uv
, and Python 3.11/3.12.uv venv
and activate it.uv sync
; with all plugins and extras: uv sync --all-groups --all-extras
).uv pip install -e '.[<plugin_name>]'
(e.g., langchain
).nat
CLI, e.g., nat run --config_file workflow.yaml --input "Your query"
.docs/source/workflows/mcp
folder.Q: Do I need to replace my existing LangChain or CrewAI code? A: No. The toolkit wraps around existing agents and can be added incrementally.
Q: Which Python versions are supported? A: Python 3.11 and 3.12.
Q: How can I add a custom tool?
A: Implement the tool as a regular Python function and register it in the workflow configuration under functions
.
Q: Is there a Docker image for the MCP server? A: The repository provides source‑only instructions; you can containerize it yourself using the standard Python base image.
Q: What licensing applies? A: Apache‑2.0.
NVIDIA NeMo Agent toolkit is a flexible, lightweight, and unifying library that allows you to easily connect existing enterprise agents to data sources and tools across any framework.
[!NOTE] NeMo Agent toolkit was previously known as the Agent Intelligence (AIQ) toolkit, and AgentIQ. The library was renamed to better reflect the purpose of the toolkit and to align with the NVIDIA NeMo family of products. The core technologies, performance and roadmap remain unchanged and the API is fully compatible with previous releases.
The rename is still in progress and references to the previous name may still be found in the codebase and documentation.
Framework Agnostic: NeMo Agent toolkit works side-by-side and around existing agentic frameworks, such as LangChain, LlamaIndex, CrewAI, and Microsoft Semantic Kernel, as well as customer enterprise frameworks and simple Python agents. This allows you to use your current technology stack without replatforming. NeMo Agent toolkit complements any existing agentic framework or memory tool you're using and isn't tied to any specific agentic framework, long-term memory, or data source.
Reusability: Every agent, tool, and agentic workflow in this library exists as a function call that works together in complex software applications. The composability between these agents, tools, and workflows allows you to build once and reuse in different scenarios.
Rapid Development: Start with a pre-built agent, tool, or workflow, and customize it to your needs. This allows you and your development teams to move quickly if you're already developing with agents.
Profiling: Use the profiler to profile entire workflows down to the tool and agent level, track input/output tokens and timings, and identify bottlenecks. While we encourage you to wrap (decorate) every tool and agent to get the most out of the profiler, you have the freedom to integrate your tools, agents, and workflows to whatever level you want. You start small and go to where you believe you'll see the most value and expand from there.
Observability: Monitor and debug your workflows with dedicated integrations for popular observability platforms such as Phoenix, Weave, and Langfuse, plus compatibility with OpenTelemetry-based observability platforms. Track performance, trace execution flows, and gain insights into your agent behaviors.
Evaluation System: Validate and maintain accuracy of agentic workflows with built-in evaluation tools.
User Interface: Use the NeMo Agent toolkit UI chat interface to interact with your agents, visualize output, and debug workflows.
Full MCP Support: Compatible with Model Context Protocol (MCP). You can use NeMo Agent toolkit as an MCP client to connect to and use tools served by remote MCP servers. You can also use NeMo Agent toolkit as an MCP server to publish tools via MCP.
With NeMo Agent toolkit, you can move quickly, experiment freely, and ensure reliability across all your agent-driven projects.
The following diagram illustrates the key components of NeMo Agent toolkit and how they interact. It provides a high-level view of the architecture, including agents, plugins, workflows, and user interfaces. Use this as a reference to understand how to integrate and extend NeMo Agent toolkit in your projects.
examples
directory of the source repository.Before you begin using NeMo Agent toolkit, ensure that you meet the following software prerequisites.
Clone the NeMo Agent toolkit repository to your local machine.
git clone git@github.com:NVIDIA/NeMo-Agent-Toolkit.git nemo-agent-toolkit
cd nemo-agent-toolkit
Initialize, fetch, and update submodules in the Git repository.
git submodule update --init --recursive
Fetch the data sets by downloading the LFS files.
git lfs install
git lfs fetch
git lfs pull
Create a Python environment.
uv venv --seed .venv
source .venv/bin/activate
Make sure the environment is built with Python version 3.11
or 3.12
. If you have multiple Python versions installed,
you can specify the desired version using the --python
flag. For example, to use Python 3.11:
uv venv --seed .venv --python 3.11
You can replace --python 3.11
with any other Python version (3.11
or 3.12
) that you have installed.
Install the NeMo Agent toolkit library.
To install the NeMo Agent toolkit library along with all of the optional dependencies. Including developer tools (--all-groups
) and all of the dependencies needed for profiling and plugins (--all-extras
) in the source repository, run the following:
uv sync --all-groups --all-extras
Alternatively to install just the core NeMo Agent toolkit without any plugins, run the following:
uv sync
At this point individual plugins, which are located under the packages
directory, can be installed with the following command uv pip install -e '.[<plugin_name>]'
.
For example, to install the langchain
plugin, run the following:
uv pip install -e '.[langchain]'
[!NOTE] Many of the example workflows require plugins, and following the documented steps in one of these examples will in turn install the necessary plugins. For example following the steps in the
examples/getting_started/simple_web_query/README.md
guide will install thenvidia-nat-langchain
plugin if you haven't already done so.
In addition to plugins, there are optional dependencies needed for profiling. To install these dependencies, run the following:
uv pip install -e '.[profiling]'
Verify the installation using the NeMo Agent toolkit CLI
nat --version
This should output the NeMo Agent toolkit version which is currently installed.
Ensure you have set the NVIDIA_API_KEY
environment variable to allow the example to use NVIDIA NIMs. An API key can be obtained by visiting build.nvidia.com
and creating an account.
export NVIDIA_API_KEY=<your_api_key>
Create the NeMo Agent toolkit workflow configuration file. This file will define the agents, tools, and workflows that will be used in the example. Save the following as workflow.yaml
:
functions:
# Add a tool to search wikipedia
wikipedia_search:
_type: wiki_search
max_results: 2
llms:
# Tell NeMo Agent toolkit which LLM to use for the agent
nim_llm:
_type: nim
model_name: meta/llama-3.1-70b-instruct
temperature: 0.0
workflow:
# Use an agent that 'reasons' and 'acts'
_type: react_agent
# Give it access to our wikipedia search tool
tool_names: [wikipedia_search]
# Tell it which LLM to use
llm_name: nim_llm
# Make it verbose
verbose: true
# Retry up to 3 times
parse_agent_response_max_retries: 3
Run the Hello World example using the nat
CLI and the workflow.yaml
file.
nat run --config_file workflow.yaml --input "List five subspecies of Aardvarks"
This will run the workflow and output the results to the console.
Workflow Result:
['Here are five subspecies of Aardvarks:\n\n1. Orycteropus afer afer (Southern aardvark)\n2. O. a. adametzi Grote, 1921 (Western aardvark)\n3. O. a. aethiopicus Sundevall, 1843\n4. O. a. angolensis Zukowsky & Haltenorth, 1957\n5. O. a. erikssoni Lönnberg, 1906']
We would love to hear from you! Please file an issue on GitHub if you have any feedback or feature requests.
We would like to thank the following open source projects that made NeMo Agent toolkit possible:
Please log in to share your review and rating for this MCP.
Discover more MCP servers with similar functionality and use cases
by zed-industries
Provides real-time collaborative editing powered by Rust, enabling developers to edit code instantly across machines with a responsive, GPU-accelerated UI.
by cline
Provides autonomous coding assistance directly in the IDE, enabling file creation, editing, terminal command execution, browser interactions, and tool extension with user approval at each step.
by continuedev
Provides continuous AI assistance across IDEs, terminals, and CI pipelines, offering agents, chat, inline editing, and autocomplete to accelerate software development.
by github
Enables AI agents, assistants, and chatbots to interact with GitHub via natural‑language commands, providing read‑write access to repositories, issues, pull requests, workflows, security data and team activity.
by block
Automates engineering tasks by installing, executing, editing, and testing code using any large language model, providing end‑to‑end project building, debugging, workflow orchestration, and external API interaction.
by RooCodeInc
An autonomous coding agent that lives inside VS Code, capable of generating, refactoring, debugging code, managing files, running terminal commands, controlling a browser, and adapting its behavior through custom modes and instructions.
by lastmile-ai
A lightweight, composable framework for building AI agents using Model Context Protocol and simple workflow patterns.
by firebase
Provides a command‑line interface to manage, test, and deploy Firebase projects, covering hosting, databases, authentication, cloud functions, extensions, and CI/CD workflows.
by gptme
Empowers large language models to act as personal AI assistants directly inside the terminal, providing capabilities such as code execution, file manipulation, web browsing, vision, and interactive tool usage.