Tool Calling with Heurist LLM Gateway

Hey there, fellow developers! 👋 Today, we’re announcing an exciting feature that’s going to supercharge your AI applications: Tool Calling in Large Language Models (LLMs), now available through Heurist’s LLM Gateway.

What’s the Buzz About?

• Heurist LLM Gateway: We’ve brought our OpenAI-compatible API gateway to the next level. The gateway lets you tap into open-source LLMs with just a few lines of code. No GPU setup or CUDA driver headaches headaches. Compatible with any applications using OpenAI API/SDK.
• Hermes-3-llama3.1–8b: Developed by Nous Research, this high-quality, fine-tuned version of LLaMA 3.1 natively supports tool calling. It excels in advanced agentic capabilities, improved reasoning, and enhanced multi-turn conversations. More Info.

Now, let’s see this in action with a practical example that showcases tool calling to extend your LLM’s capabilities.

Tool Calling in Action: Cryptocurrency Price Query Example

Imagine you’re building an AI assistant that needs to provide real-time cryptocurrency price information. Here’s how you can do it with Heurist’s LLM Gateway and tool calling:

from openai import OpenAI
import json

# Configure the Heurist client (OpenAI-compatible)
client = OpenAI(
    api_key="your_heurist_api_key_here",  # Get yours at https://dev-api-form.heurist.ai/
    base_url="https://llm-gateway.heurist.xyz"
)

def get_coin_price(token: str) -> float:
    """
    Simulates fetching the current price of a given cryptocurrency.
    In a production environment, replace this with an actual API call.
    
    Args:
        token (str): The name or symbol of the cryptocurrency.
    
    Returns:
        float: The current price of the specified cryptocurrency in USD.
    """
    # For demonstration purposes, we're returning static prices
    # In practice, you'd call a real-time crypto price API here
    prices = {
        "bitcoin": 45000.00,
        "ethereum": 3000.00,
        "dogecoin": 0.25,
    }
    return prices.get(token.lower(), 0.0)

# Define the tool
coin_price_tool = [
    {
        'type': 'function',
        'function': {
            'name': 'get_coin_price',
            'description': 'Get the current price of a specified cryptocurrency in USD',
            'parameters': {
                'type': 'object',
                'properties': {
                    'token': {
                        'type': 'string',
                        'description': 'The name or symbol of the cryptocurrency',
                    },
                },
                'required': ['token'],
            },
        }
    }
]

def query_coin_price_llm(prompt):
    messages = [{'role': 'user', 'content': prompt}]
    
    # First API call to determine if we need to use the coin price tool
    response = client.chat.completions.create(
        model="hermes-3-llama3.1-8b",
        messages=messages,
        temperature=0.01,
        tools=coin_price_tool,
        tool_choice="auto"
    )
    
    # Check if the model wants to use the coin price tool
    if response.choices[0].message.tool_calls:
        tool_call = response.choices[0].message.tool_calls[0]
        function_name = tool_call.function.name
        function_args = json.loads(tool_call.function.arguments)
        
        # Call the coin price function
        price = get_coin_price(**function_args)
        price_info = f"The current price of {function_args['token']} is ${price:.2f}"
        
        # Add the price info to the conversation
        messages.append(response.choices[0].message)
        messages.append({
            'role': 'tool',
            'content': price_info,
            'tool_call_id': tool_call.id
        })
        
        # Second API call to generate the final response
        final_response = client.chat.completions.create(
            model="hermes-3-llama3.1-8b",
            messages=messages,
            temperature=0.01
        )
        
        return final_response.choices[0].message.content
    else:
        # If no tool was called, return the initial response
        return response.choices[0].message.content

# Example usage
user_query = "What's the current price of Bitcoin?"
print(query_coin_price_llm(user_query))

Breaking It Down

• Setting Up: We’re using the OpenAI-compatible client to connect to Heurist’s LLM Gateway. This allows for easy switching between OpenAI and Heurist with minimal code changes.
• Coin Price Function: We’ve defined a get_coin_price function that simulates querying cryptocurrency prices. In a production environment, you'd replace this with a call to a real-time cryptocurrency price API.
• Tool Definition: We define the coin_price_tool that describes our get_coin_price function to the LLM. This tells the model when and how to use this external tool.
• Query Process:
• First, we send the user’s query to the LLM.
• If the LLM decides it needs price information, it calls our get_coin_price function.
• We then feed the price data back to the LLM.
• Finally, the LLM generates a response that incorporates the price information.

Get Started Today!

Ready to supercharge your AI applications with tool calling? Here’s how to get started:

1. Sign up for a Heurist API key at https://dev-api-form.heurist.ai/
2. Replace your_heurist_api_key_here in the code example with your actual credentials.
3. Start building!

More Examples

Head to https://github.com/heurist-network/dev-examples to find more examples of LLM gateway and tool calling.

Happy coding, and welcome to the decentralized AI revolution! 🚀