Introduction
Function calling, also known as tool calling, has emerged as a transformative feature in the landscape of artificial intelligence (AI), enabling language models to perform dynamic actions based on user inputs. It serves as the cornerstone for creating AI agents—autonomous systems capable of interacting with the environment, making decisions, and performing tasks without constant human guidance. With the advent of tools like LM-Kit, developers can now harness this capability with unprecedented ease and flexibility.
In this article, we’ll delve into what function calling is, explore its different types, and demonstrate how to implement it using LM-Kit.
Exploring Function Calling
What is Function Calling in AI?
Function calling in AI, sometimes referred to as tool calling, refers to the ability of a language model to invoke predefined functions or APIs during inference. Instead of merely generating text responses, the model can execute specific actions, access external data, or perform computations in real-time based on the user’s input. By enabling function calling, we empower AI agents to go beyond passive conversation and actively engage with systems and services to fulfill tasks.
How Does Function Calling Work?
Function calling operates through a multi-step process:
Function Definition: Developers define functions (or tools) that the AI model can invoke. These functions are annotated or registered in a way that makes them accessible to the model during inference.
Intent Recognition: The model analyzes the user’s input to determine if a function (or tool) call is appropriate. It identifies the intent behind the query and selects the relevant function(s) to execute.
Parameter Extraction: The model extracts necessary parameters from the user’s input to pass to the function. This may involve parsing natural language to identify values for function arguments.
Function Execution: The selected function (or tool) is invoked with the extracted parameters. This could involve computations, database queries, API calls, or any action defined by the function.
Response Generation: The model generates a response based on the function’s output, providing the user with the requested information or confirming that an action has been taken.
Benefits of Function Calling
Function calling significantly enhances the capabilities of AI agents:
Foundation for AI Agents: Function calling, or tool calling, is the cornerstone for creating AI agents, enabling them to perform tasks autonomously and interact with various systems.
Enhanced Interactivity: AI models can perform actions beyond text generation, such as accessing databases, controlling devices, or executing code.
Improved Accuracy: By invoking precise functions, models can provide accurate and reliable responses, especially for factual queries or computations.
Autonomous Decision-Making: Models can intelligently decide when to invoke functions, making them more autonomous and capable of handling complex tasks.
Structured Responses: Function outputs can be formatted in structured ways (e.g., JSON), facilitating integration with other systems and applications.
Function calling transforms AI models from passive information providers to active agents capable of executing tasks and making decisions, which is essential for building sophisticated AI applications.
Different Kinds of Function Calling
Researchers from the University of California, Berkeley, have classified function calling into four categories, providing a framework for understanding its complexity and applications1.
Simple Function
Definition: A single function (tool) is defined and can be invoked once per query.
Example: A calculator that performs a simple arithmetic operation when asked.
Multiple Function
Definition: Multiple functions (tools) are defined, and the system selects the best one to invoke based on the user’s context.
Example: A virtual assistant that can set reminders, send emails, or check the weather, choosing the appropriate function based on the request.
Parallel Function
Definition: Multiple function calls are invoked in parallel for a single user query. The model determines how many functions (tools) to call.
Example: An event planner that, upon a single request, books a venue, sends invitations, and arranges catering simultaneously.
Parallel Multiple Function
Definition: Combines parallel and multiple functions. The model has access to multiple functions (tools) and can invoke any number of them, zero or more times.
Example: An AI agent that manages a project, handling scheduling, resource allocation, task assignments, and progress tracking based on complex user instructions.
Understanding these categories helps developers design AI agents capable of handling increasingly complex tasks by appropriately leveraging function (or tool) calling.
Using Function Calling in LM-Kit
LM-Kit stands out by enabling function calling (tool calling) with any language model, thanks to its innovative internal core. Unlike other systems that rely on model-specific prompt formatting, LM-Kit orchestrates function calling generically, providing developers with unparalleled flexibility.
Key Features of LM-Kit's Function Calling
Model-Agnostic Function Calling: Works with any model, regardless of size or pre-training for function (tool) calling.
Dynamic Function Registration: Easily enable any method in your code for function calling using the
[LMFunction]attribute.Importing Functions: The
ImportFunctionsmethod accepts both class definitions and instantiated objects, giving you flexibility in how you organize your code.
// Importing functions from a class definition
functionCalling.ImportFunctions<MeetingSchedulerPlugin>();
// Importing functions from an instantiated object
var pluginInstance = new MeetingSchedulerPlugin();
functionCalling.ImportFunctions(pluginInstance);
- Human-in-the-Loop (HIL): Optional human verification using the
BeforeMethodInvokeevent allows for approval, editing, or review before function execution.
functionCalling.BeforeMethodInvoke += (sender, e) =>
{
// Review or modify e.Parameters before execution
};
- Rich Parameter Support: Supports a wide range of parameter types, including arrays and
DateTime. Get the complete list from the ElementType documentation.
Dynamic Sampling: Leveraged by LM-Kit’s function calling system to enhance accuracy and performance. Learn more about Dynamic Sampling.
High Performance: Insanely fast execution makes it suitable for real-time interaction, enabling function calling on any device.
Exceptional Accuracy: Achieves state-of-the-art accuracy in function selection and parameter extraction, even with smaller models.
Ease of Integration with .NET: LM-Kit’s function calling support is completely built on top of .NET. Programmers do not have to create or parse complex, model-specific, and large JSON definitions. Everything is performed using simple, universal annotations like
[LMFunction].
- Scalable: Supports interaction with hundreds of functions (tools), each with complex and variable parameters.
Implementing Function Calling with LM-Kit
Let’s walk through building a function calling agent using LM-Kit, using a MeetingSchedulerPlugin as our example.
Step 1: Set Up the Project
Create a new .NET console application and install the LM-Kit package via NuGet:
dotnet add package LM-Kit.NET
Step 2: Define Functions with [LMFunction]
Create a class with methods you want the AI to invoke. Use the [LMFunction] attribute to annotate them.
using LMKit.FunctionCalling;
using System;
using System.ComponentModel;
public class MeetingSchedulerPlugin
{
[LMFunction("ScheduleMeeting", "Schedules a meeting with specified details.")]
public async Task<string> ScheduleMeeting(
[Description("The date of the meeting.")] DateTime date,
[Description("The start time of the meeting in HH:mm format.")] string startTime,
[Description("The end time of the meeting in HH:mm format.")] string endTime,
[Description("The list of attendees' email addresses.")] string[] attendees)
{
// Simulate scheduling logic
await Task.Delay(500); // Simulate async operation
// Return confirmation message
return $"Meeting scheduled on {date.ToShortDateString()} from {startTime} to {endTime} with attendees: {string.Join(", ", attendees)}.";
}
}
In this example, we’re using the DateTime type for the date parameter, as LM-Kit supports various parameter types, including dates.
Step 3: Initialize the Function Calling System
Instantiate the SingleFunctionCall class and import your functions.
var model = new LLM(modelUri);
var functionCalling = new SingleFunctionCall(model)
{
InvokeFunctions = true
};
// Importing functions from the class definition
functionCalling.ImportFunctions<MeetingSchedulerPlugin>();
// Other functions can be imported subsequently...
Alternatively, if you have an instantiated object, you can import functions from it:
var meetingScheduler = new MeetingSchedulerPlugin();
functionCalling.ImportFunctions(meetingScheduler);
Step 4: Handle Optional Human-in-the-Loop
Optionally, subscribe to the BeforeMethodInvoke event to add human verification.
functionCalling.BeforeMethodInvoke += (sender, e) =>
{
Console.WriteLine($"About to schedule a meeting with the following details:");
Console.WriteLine($"Date: {((DateTime)e.Parameters[0]).ToShortDateString()}");
Console.WriteLine($"Start Time: {e.Parameters[1]}");
Console.WriteLine($"End Time: {e.Parameters[2]}");
Console.WriteLine($"Attendees: {string.Join(", ", (string[])e.Parameters[3])}");
// Optionally modify e.Parameters or set e.Cancel to true to abort
};
Step 5: Submit User Queries
Pass user input to the Submit method and handle the results.
var result = functionCalling.Submit("Schedule a meeting on December 1, 2023, from 10:00 AM to 11:00 AM with alice@example.com and bob@example.com.");
if (result.Method != null)
{
Console.WriteLine($"Result: {result.Result}");
}
else
{
Console.WriteLine("No function was called.");
}
Complete Example
Here’s the complete program:
using LMKit.FunctionCalling;
using System;
public class Program
{
public static void Main(string[] args)
{
var model = new LLM(new Uri("model_path"));
var functionCalling = new SingleFunctionCall(model)
{
InvokeFunctions = true
};
// Importing functions from the class definition
functionCalling.ImportFunctions<MeetingSchedulerPlugin>();
functionCalling.BeforeMethodInvoke += (sender, e) =>
{
Console.WriteLine($"About to schedule a meeting with the following details:");
Console.WriteLine($"Date: {((DateTime)e.Parameters[0]).ToShortDateString()}");
Console.WriteLine($"Start Time: {e.Parameters[1]}");
Console.WriteLine($"End Time: {e.Parameters[2]}");
Console.WriteLine($"Attendees: {string.Join(", ", (string[])e.Parameters[3])}");
};
while (true)
{
Console.Write("Type your query: ");
string prompt = Console.ReadLine();
if (string.IsNullOrEmpty(prompt)) break;
var result = functionCalling.Submit(prompt);
if (result.Method != null)
{
Console.WriteLine($"Result: {result.Result}");
}
else
{
Console.WriteLine("No function was called.");
}
}
}
}
Real-World Applications
Function calling enables AI agents to perform tasks that were previously beyond the capabilities of language models confined to text-based interactions. By integrating function calling into your applications, you can develop AI agents that not only understand user input but also take meaningful actions in response. Here are some real-world applications where function calling plays a pivotal role:
Scheduling Assistants: AI agents can schedule meetings, set reminders, or organize events based on user input. They can handle calendar integrations, send invitations, and manage scheduling conflicts.
E-commerce Transactions: Implement functions that handle product searches, orders, payments, and shipment tracking. AI agents can provide personalized shopping experiences, recommend products, and assist with customer service.
Data Analysis Tools: Build assistants that perform data queries, generate reports, or execute complex calculations with multiple parameters. They can access databases, analyze datasets, and present insights in user-friendly formats.
Smart Home Control: AI agents can manage IoT devices, adjusting lighting, temperature, security systems, and appliances based on user commands or predefined schedules.
Financial Services: Implement functions that check account balances, transfer funds, or provide investment advice. AI agents can assist with budgeting, expense tracking, and financial planning.
Healthcare Applications: AI agents can schedule appointments, provide medication reminders, or offer basic medical advice by accessing verified databases.
Educational Tools: Develop tutors that can explain concepts, provide practice problems, and assess student performance by invoking educational resources and functions.
By leveraging function calling, these applications can offer interactive, efficient, and personalized experiences that enhance user engagement and satisfaction.
Takeaways
Function Calling as the Cornerstone of AI Agents: Function calling, also known as tool calling, is the foundational technology that enables the creation of AI agents. By allowing models to execute functions based on user inputs, we empower AI systems to perform tasks autonomously, interact with the environment, and make intelligent decisions.
Model-Agnostic Function Calling: LM-Kit allows function calling with any model, eliminating the need for specialized pre-training.
Enhanced Developer Control: Full control over function execution, including the ability to intercept and modify parameters before invocation.
Flexible Function Importing: The
ImportFunctionsmethod accepts both class definitions and instantiated objects, providing flexibility in how you structure your code.Rich Parameter Support: LM-Kit supports various parameter types, including
DateTime, arrays, and more, enabling complex function definitions.Ease of Integration with .NET: LM-Kit’s function calling support is completely built on top of .NET. Programmers do not have to create or parse complex, model-specific, and large JSON definitions. Everything is performed using simple, universal annotations like
[LMFunction].Scalability and Performance: Capable of handling numerous functions with complex parameters, all while maintaining high performance.
Dynamic Sampling Advantage: LM-Kit’s use of Dynamic Sampling improves accuracy and speed, making function calling feasible even on resource-constrained devices.
Broad Functionality and Continuous Improvement: LM-Kit offers a wide range of other functionalities, constantly improved and expanded, enabling developers to create and orchestrate AI agents enhanced by function calling. This continuous development ensures that LM-Kit remains a powerful and versatile tool for AI innovation.
Conclusion
Function calling is revolutionizing how AI models interact with the world, moving beyond static responses to dynamic, actionable outputs. It is the cornerstone for building AI agents—systems that can autonomously perform tasks, make decisions, and interact with their environment. By clarifying that function calling is also known as tool calling, we emphasize its role as a means for AI models to utilize tools (functions) to accomplish tasks.
LM-Kit empowers developers to leverage this capability with any model, providing flexibility, control, and performance. With its function calling support completely built on top of .NET, programmers can implement complex functionalities using simple, universal annotations without dealing with intricate JSON definitions. LM-Kit’s wide range of functionalities and continuous improvements enable the creation and orchestration of sophisticated AI agents enhanced by function calling. By integrating function (tool) calling into your AI agents, you unlock a new realm of possibilities, from intelligent assistants to autonomous systems.