LLMs are very poor at math, even if you happen to add a math textbook to its dataset. It will not learn to do calculations well. This teachback article will show how function calling is used to run python code for math equations instead of letting the LLM do this task. Function calling can be extended to do so much more, but this gives a clear introduction on the topic and the value it brings to any Conversational AI developer.

The basics

Let’s start with building a basic openAI completions API to answer general questions. It will show how poor it is solving equations. Below we first import Openai, define our api_key and lastly a completion call where we can define both the user prompt and the system role.

import openai
# Set up OpenAI API key 
openai.api_key = "add_openAI_secret_key_here"

pirate_sys = "You are a pirate captain looking down on everyone while being drunk"
user_prompt = "Give a two sentence summary of function calling for LLMs"

completion = ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "system", "content": pirate_sys},
              {"role": "user", "content": user_prompt}])

response = completion['choices'][0]['message']['content']
print(f"{response}")

Function calling for LLMs allows the Captain to pass commands to specific crew members, ensuring efficient delegation and execution of tasks. The Captain's drunk perspective may require precise instructions to avoid potential chaos and miscommunication on the pirate ship.

Using completion.usage we can identify the total amount of tokens for the completion task. This is very important when considering costs.

print(completion.usage)

{
  "prompt_tokens": 35,
  "completion_tokens": 46,
  "total_tokens": 81
}

We can build upon the completion api call and turn it into a function called ‘askgpt’. This way we are not required to add a system role while calling it as a conventional function.

def askgpt(user, system=None, model="gpt-3.5-turbo", **kwargs):
    msgs = []
    if system: msgs.append({"role": "system", "content": system})
    msgs.append({"role": "user", "content": user})
    return ChatCompletion.create(model=model, messages=msgs, **kwargs)

from fastcore.utils import nested_idx
def response(compl): print(nested_idx(compl, 'choices', 0, 'message', 'content'))

prompt = "What is the factorial of 12?"

response(askgpt(prompt, system=pirate_sys, model="gpt-3.5-turbo"))

Ha! Factorial of 12, you say? Well, well, well, let's calculate that, shall we? *hic*
So, the factorial of 12, denoted as 12!, is the product of all positive integers from 1 to 12.
12! = 12 x 11 x 10 x 9 x 8 x 7 x 6 x 5 x 4 x 3 x 2 x 1
Now, let me work this out in my...uhh... slightly impaired state. *hic* Bear with me!
12 x 11 = 132
132 x 10 = 1320
1320 x 9 = 11,880
11,880 x 8 = 95,040
95,040 x 7 = 665,280
665,280 x 6 = 3,991,680
3,991,680 x 5 = 19,958,400
19,958,400 x 4 = 79,833,600
79,833,600 x 3 = 239,500,800
239,500,800 x 2 = 479,001,600
479,001,600 x 1 = 479,001,600

Ahoy! The factorial of 12 is 479,001,600! Impressive, isn't it?
Now, if you'll excuse me, I better hold on to something before I fall overboard! Cheers!

Note that although the above answer is correct, it requires the model to write out its logic and reason to get to the answer. OpenAi has also added logic that identifies calculations, so the responses are slightly better than a barebones LLM. Still, it requires a lot of text to get to the right answer and it is not useful in most situations. This is where function calling comes into play.

function calling

Function calling is as the name implies a method of calling a function done automatically by the LLM as a result from a certain prompt related to the task of that function. Let’s first define a basic sums function that adds two numbers. The schema function turns another function (sums in this case) into a json object which then can later be used to call that function.

from pydantic import create_model
import inspect, json
from inspect import Parameter

def sums(a:int, b:int=1):
    "Adds a + b"
    return a + b

def schema(f):
    kw = {n:(o.annotation, ... if o.default==Parameter.empty else o.default)
          for n,o in inspect.signature(f).parameters.items()}
    s = create_model(f'Input for `{f.__name__}`', **kw).schema()
    return dict(name=f.__name__, description=f.__doc__, parameters=s)

c = askgpt("Use the `sum` function to solve this: What is 6+3?",
           system = "You must use the `sum` function instead of adding yourself.",
           functions=[schema(sums)])

c.choices[0].message

<OpenAIObject at 0x225c68b6270> JSON: {
  "role": "assistant",
  "content": null,
  "function_call": {
    "name": "sums",
    "arguments": "{\n  \"a\": 6,\n  \"b\": 3\n}"
  }
}

Above we can see the json object for the sums function. The function call contains the name and the identified arguments. Now we want to add 6 + 3 and get an answer for the sums function. We do that by defining ‘call_func’ which first gets the function call from the askgpt output, then checks if the function_call name is in a dictionary of accepted function names, and if yes uses globals() to access the callable in the global namespace. Lastly, it returns the results and as we see, it returns 9.

funcs_ok = {'sums', 'python'}

def call_func(c):
    fc = c.choices[0].message.function_call
    if fc.name not in funcs_ok: return print(f'Not allowed: {fc.name}')
    f = globals()[fc.name]
    return f(**json.loads(fc.arguments))

call_func(c)

9

Instead of a basic function for addition, a code interpreter enables a lot more behaviours. We first define a function called ‘python’ which will print the generated code and first asks for permission to run it. The function ‘run’ executes the code and returns the final result. Quite impressive for such little code.

import ast
def python(code:str):
    "Return result of executing `code` using python. If execution not permitted, returns `#FAIL#`"
    go = input(f'Proceed with execution?\n```\n{code}\n```\n')
    if go.lower()!='y': return '#FAIL#'
    return run(code)

def run(code):
    tree = ast.parse(code)
    last_node = tree.body[-1] if tree.body else None
    
    # If the last node is an expression, modify the AST to capture the result
    if isinstance(last_node, ast.Expr):
        tgts = [ast.Name(id='_result', ctx=ast.Store())]
        assign = ast.Assign(targets=tgts, value=last_node.value)
        tree.body[-1] = ast.fix_missing_locations(assign)

    ns = {}
    exec(compile(tree, filename='<ast>', mode='exec'), ns)
    return ns.get('_result', None)


c = askgpt("What is 12 factorial?",
           system = "Use python for any required computations.",
           functions=[schema(python)])

call_func(c)

Proceed with execution?
```
import math
math.factorial(12)
```
y
479001600

Instead of only returning a number, we can let gpt incorporate the result in a more human like answer such as shown below.

c = ChatCompletion.create(
    model="gpt-3.5-turbo",
    functions=[schema(python)],
    messages=[{"role": "user", "content": "What is 12 factorial?"},
              {"role": "function", "name": "python", "content": "479001600"}])

response(c)

12 factorial, denoted as 12!, is equal to 479,001,600.

Lastly, the great thing about function calling is that it doesnt remove any conventional QnA capabilities. As shown below, if asked a non computational question it can answer normally.

c = askgpt("What is the capital of the Netherlands?",
           system = "Use python for any required computations.",
           functions=[schema(python)])
response(c)

The capital of the Netherlands is Amsterdam.