Building a GPT+Functions Chat App

As simply as possible

Yesterday I walked through a high level overview of how to build a chat application that uses functions. The amazing thing is that GPT doesn’t just use the functions naïvely, it can actually make multiple calls in a row to find the information it needs. For example, I just asked my stock pricing GPT app if Salesforce was more expensive than IBM, and here’s what happened:

Click on image to expand

First, GPT got the ticker symbols for each of the two stocks, IBM and Salesforce, then got the quotes for each of them, and then answered the question. That’s four function calls in a row to collect all the information it needed!

Today I want to go into the details of how you can build your own GPT+functions chat application. I’ll be referencing the code in this repository, so go ahead and grab a copy:

https://github.com/cmcguinness/ChatWithFuncs

After you clone it, you’ll want to set up a virtual environment with Python 3.11, and install the modules listed in requirements.txt.

---

By the way, if you’re looking for help in getting AI projects off the ground, particularly with regards to Salesforce, I’ve been working with Owls Head, a CRM focused SI, to set up an AI practice. You can ping me or them through their website.

---

API Keys

The program uses two services: OpenAI’s gpt-3.5-turbo, and Twelve Data’s stock quote APIs. You will need to sign up for both and have API keys to use. OpenAI offers initial free usage to new subscribers, and is dirt cheap after, and Twelve Data offers a low volume free account perfect for this project.

Before you can run the problem, you need to set environment variables that hold the keys, OPENAI_API_KEY and TWELVE_API_KEY.

The Main Loop

The main loop is in the file main.py, and it really doesn’t do much:

import gpt

print('Welcome to Stock GPT')

g = gpt.GPTlib(debug=True)

while True:
    q = input('USER:       ')
    a = g.ask_gpt(q)
    print('STOCK GPT: ', a)

Command line UIs … they’re back in style like Arts & Crafts¹ decorating.

The Stock APIs

The functions that call the stock APIs are in the file stocking.py, and consists of two primary routines: get_quote and lookup_ticker.

The code for get_quote is pretty simple:

def get_quote(ticker: str, return_form='str'):
    data_json = _call_twelve('quote', 'symbol', ticker)

    if return_form == 'json':
        return data_json

    return pretty_print(data_json)

The first line calls the API using a helper function to get the quote, then the code either returns the JSON result as is or converts it into a more readable string. Turns out GPT likes the readable string. Who knew.

The second function looks up a ticker symbol for a company name, and is a wee bit more complicated:

def lookup_ticker(name, return_form='str'):
    search_json = _call_twelve('symbol_search', 'symbol', name)

    # This retrieves a bunch of exchanges, but for demo purposes 
    # we only are interested in these two
    exchanges = ['NYSE', 'NASDAQ']

    for exchange_listing in search_json['data']:
        if exchange_listing['exchange'] in exchanges:
            if return_form == 'json':
                return exchange_listing

            return pretty_print(exchange_listing)

    return None

It, too, calls the API using the helper function, but what you get back is not a simple answer, but a list of 25 exchanges and (different) tickers for each exchange. So I loop through the list looking for either NYSE or NASDAQ and return just that exchanges information. As before, I either return the raw JSON or a formatted version.

You can look at the two helper functions (_call_twelve and pretty_print) in the source, they’re not hard to figure out.

The GPT Client

The GPT client is where most of the heavy lifting is done, and it consists of the following sections:

1. The System Prompt

2. The List of Functions

3. The low-level interface that calls GPT

4. Function handling code

5. The high-level interface for calling GPT with function support

Let’s look at each of these individually.

The System Prompt

The system prompt is used in GPT to describe what the agent is trying to do, in general. Mine is set up in the initialization logic for the class:

Not really poetry, but it seems to work. I could definitely have made it more concise. But it works, and if it ain’t broke …

The List of Functions

The way functions work with GPT is that, on every single call you also provide it a list of functions that are available².

The key idea is that you tell GPT the following for each function:

• What to call it (e.g., get_quote)

• What it does (and why it might be useful)

• What the arguments to the function are (aka parameters), what they represent, and what format they’re in.

• Which of the parameters are required.

And from that, GPT figures out when to call the functions.

The Low Level Interface

This is a method that collects all the information we have to send to GPT, sends it, and then collects the response. If it gets an error indicating that GPT is busy, it fakes a response that looks like GPT itself said it was busy:

What’s in messages? Stay tuned!

Function Handling Code

There’s two methods involved with handling functions. The first function reads the reply from GPT to figure out what function to call, and then calls it:

It’s messy, but simple. Note that “company” and “ticker” were the names I passed into GPT in the list of functions as being the names of parameters.

Also notice that I am the one who actually calls the function. GPT just says “you should call them”, and it’s on me to do it. But how does GPT get the benefit of my calling it? That’s all mixed into the second function. I’m going to review it in two sections. First section:

This method takes an inbound (to GPT) message (more on that later, I promise), sends it to GPT, and gets the response. When it gets the response, it can be one of two things: either it is text to display to the user, or it’s a request to call a function. In the snipped of code above, you can see at line 99 the code checks to see if the response is a function call and, if not, returns the results to the caller.

However, if it is a function call request, the code continues on:

This is a bit complicated, so let me explain:

• Lines 107 .. 114 generate a conciser version of the response for saving into history.

• Line 116-118 generate a debugging message that prints out the function call information.

• Lines 121 and 122 add the last response from GPT to the history and the current “inbound message”³.

• Line 125 calls the function requested.

• Lines 128-137 generate a history entry for the results of the function call and arrange for it to be sent back into GPT for it to further process.

• Line 140 does tail-recursion⁴: it calls the method anew with the results of having called the functions.

I realize that this can seem more confusing that before, so hold on a bit and let me get through the rest of the code first.

High-Level GPT Interface

This is the method our main loop calls. It passes in the string the user type, and it returns the final answer from GPT. This code does a bit of history management (next section, I promise!), calls the function-savvy method to call GPT, and when that returns, we have our answer from GPT to return.

History!

Somebody has to clean up after history

When you’re chatting with ChatGPT, it’s pretty good at remembering what you said before:

How does it do that? More importantly, how do we do that in our code?

The answer is brute force. Every time you (as in your code) call GPT, you also send in all the conversation that’s happened before. GPT itself doesn’t remember; there’s no sense of a session. If you don’t send in the history, it sounds like me when people ask me questions when I’m concentrating.

So that’s why you see the code taking great care to save user requests, GPT responses, function calls, and the return values from functions into a local history: it has to in order for the conversation to continue on.

But do you save everything from the dawn of time? You might, but at some point you run into a couple of problems if you do. First, the more history you send in, the slower GPT gets. It just has that much more stuff to process. The second, the more you send in, the more it costs.

So at some point, you have to trim your history back, like a house plant that’s gone wild. In my code, you see this:

# Trim back our history
self.history = self.history[-self.history_max:]

This will keep up to the last n entries on the list (currently 16). There are fancier approaches, I’m sure, but this works just fine.

Conclusion

Probably the best way to understand how this all works is to run the code in debug mode and watch how it behaves. If you’re not set up yet, I recommend getting Anaconda (https://www.anaconda.com/download), as it has everything you need including PyCharm, which I think is the best Python IDE.

Next week I’m going to talk about history a bit more, because it’s really important and there’s some neat things you can do with it.

1. Here you go: https://www.williams-sonoma.com/shop/tabletop-glassware-bar/tabletop-dinnerware-sets/williams-sonoma-morris-and-co/?cm_re=shopbanner-_-default-_-dinnerware_collections_morris_co

2. Might one change the list of functions dynamically based upon some contextual clues? Interesting thought.

3. This could definitely be simplified further.

4. The alternative is to wrap everything in some sort of giant while True loop, but I find tail-recursion is more elegant. Even if Python doesn’t believe in it.