Building Susan - AI chatbot that knows everything about your finances 🤖💸

One of the things I love about building product in 2024 is how easy it is to get high value features shipped to users and Susan is no exception.

What previously might've taken months (if not years!), can now be built in a matter of days or weeks.

At SortMe we recently launched "Ask Susan" to all our users.

Susan is an AI chatbot that can answer questions about financial life. Whether it's seeing how much you're spending on certain categories, what your bank account balances are, even things like how are your goals and net worth tracking along.

Susan is the perfect mix between fun, sassy but still actually useful. Letting users get quick insights into what's going on with their finances.

(credit: our very talented developer, Asep is responsible for developing most of the code behind Susan, I just get to help with the higher level design, product and technical strategy, but Asep was the brains behind the code!)

The Challenge

Launch an AI powered chat bot for users to ask questions about their finances.

The Tech

• OpenAI (deployed in our Azure account)

One of the biggest changers in AI recently was when OpenAI launched "function calls" in July 2023.

Function calls let your AI chatbot - whom was previously limited by the information you could feed it in the prompt or chat - was now able to request access to information and take actions.

The basic flow is this:

1. Your app tells the AI chatbot (or "Assistant") what function calls it has access to and provides a description of what the function call can do or return
   1. e.g: fetchBankAccounts - fetch the current user's list of bank accounts including the names, types, institutions and current balances.
2. User starts a chart with your AI chatbot asking how much money they have in their bank accounts
3. Your chatbot decides that to answer this question it needs to call fetchBankAccounts and it tells your app to do make that function call and return back the data to the chatbot
4. Your app makes that function call (e.g: makes the necessary database queries, API calls, whatever) and returns the data back to the chatbot
5. The chatbot then uses the returned data from the function call to answer the user's question.

Add a few more function calls such as "fetchTransactionHistory", "fetchGoals", "fetchBudget", etc, "engineer" some prompts, build the chatbot UI and away you go!

I won't bore anyone with the frontend, as it's simply a custom chat interface built in our NextJS webapp using Tailwind. In fact, getting the frontend working nicely probably took longer than the actual underlying chatbot service.

The Result

The result is Susan, your mostly friendly assistant! Ask her anything about your finances (that SortMe knows!) and she'll lookup your SortMe data and provide tailored answered just for you.

Interesting Learnings

Where to store the prompt and function call descriptions?

Initially, we were storing the opening prompt and function call descriptions in the database (using a Laravel NOVA Settings package). However, we quickly learned that we needed to tweak and iterate on the prompt and function call descriptions in a systematic way that let us see all the revision history and know when certain prompt changes went live.

We also needed a seamless way to manage different prompts between local, stage and prod environments.

To solve this we simple created a config/susan.php file (NOT using env()) which stores all our prompts and function call structure so now everything is tracked using source control.

If we want to tweak the prompt based on user conversations, we simply make a commit to tweak the prompt and let the CI/CD do the rest.

And to prevent against "prompt hacking"?

Something to be aware of is people trying to "prompt hack" (is this a real term? If not, it is now) your chatbot.

This is basically where the hacker will try to get more information than they're entitled to get. This could be something like getting the underling prompts and configuration of the chatbot. Or trying to access data from other users.

Thankfully, OpenAI does most of the heavy lifting here. OpenAI can detect when a user is trying to get the underlying prompts of your chatbot and it will block the chat request.

In the PHP SDK, this results in an exception being thrown that can easily be caught and handled accordingly.

For us, we catch this specific exception then return a friendly message:

Shoutout to one of our investors Cam for trying to do this the first day we launched Susan...

The other part is ensuring people can't access other user's data. This one is simple. Because the function calls (where the sensitive data is coming from) are actually functions written in our Laravel backend, we can simply scope the function calls ONLY to the authenticated user. This means the user whose data Susan uses, is limited based on the authentication token the user's browser is sending to the backend. It's impossible to get Susan to access another user's data.