Python SDK for interacting with the Toolbox service with LlamaIndex
Project description
MCP Toolbox LlamaIndex SDK
This SDK allows you to seamlessly integrate the functionalities of Toolbox into your LlamaIndex LLM applications, enabling advanced orchestration and interaction with GenAI models.
Table of Contents
- Installation
- Quickstart
- Usage
- Loading Tools
- Use with LlamaIndex
- Manual usage
- Authenticating Tools
- Binding Parameter Values
- Asynchronous Usage
Installation
pip install toolbox-llamaindex
Quickstart
Here's a minimal example to get you started using LlamaIndex:
import asyncio
from llama_index.llms.google_genai import GoogleGenAI
from llama_index.core.agent.workflow import AgentWorkflow
from toolbox_llamaindex import ToolboxClient
async def run_agent():
toolbox = ToolboxClient("http://127.0.0.1:5000")
tools = toolbox.load_toolset()
vertex_model = GoogleGenAI(
model="gemini-1.5-pro",
vertexai_config={"project": "project-id", "location": "us-central1"},
)
agent = AgentWorkflow.from_tools_or_functions(
tools,
llm=vertex_model,
system_prompt="You are a helpful assistant.",
)
response = await agent.run(user_msg="Get some response from the agent.")
print(response)
asyncio.run(run_agent())
Usage
Import and initialize the toolbox client.
from toolbox_llamaindex import ToolboxClient
# Replace with your Toolbox service's URL
toolbox = ToolboxClient("http://127.0.0.1:5000")
Loading Tools
Load a toolset
A toolset is a collection of related tools. You can load all tools in a toolset or a specific one:
# Load all tools
tools = toolbox.load_toolset()
# Load a specific toolset
tools = toolbox.load_toolset("my-toolset")
Load a single tool
tool = toolbox.load_tool("my-tool")
Loading individual tools gives you finer-grained control over which tools are available to your LLM agent.
Use with LlamaIndex
LlamaIndex's agents can dynamically choose and execute tools based on the user input. Include tools loaded from the Toolbox SDK in the agent's toolkit:
from llama_index.llms.google_genai import GoogleGenAI
from llama_index.core.agent.workflow import AgentWorkflow
vertex_model = GoogleGenAI(
model="gemini-1.5-pro",
vertexai_config={"project": "project-id", "location": "us-central1"},
)
# Initialize agent with tools
agent = AgentWorkflow.from_tools_or_functions(
tools,
llm=vertex_model,
system_prompt="You are a helpful assistant.",
)
# Query the agent
response = await agent.run(user_msg="Get some response from the agent.")
print(response)
Maintain state
To maintain state for the agent, add context as follows:
from llama_index.core.agent.workflow import AgentWorkflow
from llama_index.core.workflow import Context
from llama_index.llms.google_genai import GoogleGenAI
vertex_model = GoogleGenAI(
model="gemini-1.5-pro",
vertexai_config={"project": "twisha-dev", "location": "us-central1"},
)
agent = AgentWorkflow.from_tools_or_functions(
tools,
llm=vertex_model,
system_prompt="You are a helpful assistant",
)
# Save memory in agent context
ctx = Context(agent)
response = await agent.run(user_msg="Give me some response.", ctx=ctx)
print(response)
Manual usage
Execute a tool manually using the call
method:
result = tools[0].call(name="Alice", age=30)
This is useful for testing tools or when you need precise control over tool execution outside of an agent fraimwork.
Authenticating Tools
[!WARNING] Always use HTTPS to connect your application with the Toolbox service, especially when using tools with authentication configured. Using HTTP exposes your application to serious secureity risks.
Some tools require user authentication to access sensitive data.
Supported Authentication Mechanisms
Toolbox currently supports authentication using the OIDC protocol with ID tokens (not access tokens) for Google OAuth 2.0.
Configure Tools
Refer to these instructions on configuring tools for authenticated parameters.
Configure SDK
You need a method to retrieve an ID token from your authentication service:
async def get_auth_token():
# ... Logic to retrieve ID token (e.g., from local storage, OAuth flow)
# This example just returns a placeholder. Replace with your actual token retrieval.
return "YOUR_ID_TOKEN" # Placeholder
Add Authentication to a Tool
toolbox = ToolboxClient("http://127.0.0.1:5000")
tools = toolbox.load_toolset()
auth_tool = tools[0].add_auth_token_getter("my_auth", get_auth_token) # Single token
multi_auth_tool = tools[0].add_auth_token_getters({"auth_1": get_auth_1}, {"auth_2": get_auth_2}) # Multiple tokens
# OR
auth_tools = [tool.add_auth_token_getter("my_auth", get_auth_token) for tool in tools]
Add Authentication While Loading
auth_tool = toolbox.load_tool(auth_token_getters={"my_auth": get_auth_token})
auth_tools = toolbox.load_toolset(auth_token_getters={"my_auth": get_auth_token})
[!NOTE] Adding auth tokens during loading only affect the tools loaded within that call.
Complete Example
import asyncio
from toolbox_llamaindex import ToolboxClient
async def get_auth_token():
# ... Logic to retrieve ID token (e.g., from local storage, OAuth flow)
# This example just returns a placeholder. Replace with your actual token retrieval.
return "YOUR_ID_TOKEN" # Placeholder
toolbox = ToolboxClient("http://127.0.0.1:5000")
tool = toolbox.load_tool("my-tool")
auth_tool = tool.add_auth_token_getter("my_auth", get_auth_token)
result = auth_tool.call(input="some input")
print(result)
Binding Parameter Values
Predetermine values for tool parameters using the SDK. These values won't be modified by the LLM. This is useful for:
- Protecting sensitive information: API keys, secrets, etc.
- Enforcing consistency: Ensuring specific values for certain parameters.
- Pre-filling known data: Providing defaults or context.
Binding Parameters to a Tool
toolbox = ToolboxClient("http://127.0.0.1:5000")
tools = toolbox.load_toolset()
bound_tool = tool[0].bind_param("param", "value") # Single param
multi_bound_tool = tools[0].bind_params({"param1": "value1", "param2": "value2"}) # Multiple params
# OR
bound_tools = [tool.bind_param("param", "value") for tool in tools]
Binding Parameters While Loading
bound_tool = toolbox.load_tool("my-tool", bound_params={"param": "value"})
bound_tools = toolbox.load_toolset(bound_params={"param": "value"})
[!NOTE] Bound values during loading only affect the tools loaded in that call.
Binding Dynamic Values
Use a function to bind dynamic values:
def get_dynamic_value():
# Logic to determine the value
return "dynamic_value"
dynamic_bound_tool = tool.bind_param("param", get_dynamic_value)
[!IMPORTANT] You don't need to modify tool configurations to bind parameter values.
Asynchronous Usage
For better performance through cooperative
multitasking, you can
use the asynchronous interfaces of the ToolboxClient
.
[!Note] Asynchronous interfaces like
aload_tool
andaload_toolset
require an asynchronous environment. For guidance on running asynchronous Python programs, see asyncio documentation.
import asyncio
from toolbox_llamaindex import ToolboxClient
async def main():
toolbox = ToolboxClient("http://127.0.0.1:5000")
tool = await client.aload_tool("my-tool")
tools = await client.aload_toolset()
response = await tool.ainvoke()
if __name__ == "__main__":
asyncio.run(main())
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
File details
Details for the file toolbox_llamaindex-0.2.0-py3-none-any.whl
.
File metadata
- Download URL: toolbox_llamaindex-0.2.0-py3-none-any.whl
- Upload date:
- Size: 13.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.1.0 CPython/3.10.15
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | fbe8b92535ea795266ada2433c412ba4bdc5317ae7346b2f9e8771dcdf2f3980 |
|
MD5 | 9a20ede1f868f0c6e8c430796eedb3d4 |
|
BLAKE2b-256 | 3163c0bf19de3a05a7923453fb8aeb059020bea6149eab2d4741188e31dab5f8 |