Agent Based Modeling in Economics and Finance

A different way to look at things

While studying economics in undergrad, I was always struck by something: Economists don’t seem very good at predicting what will happen to the economy.

As Economics Explained, one of my favorite economics Youtubers says: “Nobody can predict the future, least of all economists”.

Why is that? And is there a better way?

The problem: people

I personally believe the original sin of economics is that we assume people (and organizations run by people) make optimal financial decisions. Of course, no equation will ever capture the complexity of the real economy, let alone human psychology. So, we need to assume something, or else we’d never get any research done.

Enter agent based modeling.

Agent based modeling definition

Instead of writing top-down formulas, such as Keynes’ GDP formula:

\(GDP = C + I + G + (X - M)\)

Agent based modeling (ABM) simulates the real world by creating an ecosystem of “agents” that act in their own self interest. The art of agent based modeling involves constructing agents that resemble the entities you’re trying to simulate, and environments that provide a realistic structure for those agents operate in.

Typically, each agent will have local “state”, which is data that only that specific agent can see. State can include how much money it has, what it’s behavioral rules are, offers it has made and received from other agents, and much more.

Environments contain rules that resemble real world laws, marketplaces, public state that all agents can see, and other things constructs that agents interact with each other through. Two ambitious environments that are meant to simulate an entire economy are:

• Agent based-stock flow consistent macroeconomics: Towards a benchmark model

• JMAB 2.0: A Critical Update for the Macroeconomic Agent-based Model JMAB

You can discover many many other agent based modeling projects in the excellent paper: Agent-Based Modeling in Economics and Finance: Past, Present, and Future, by Robert L. Axtell and J. Doyne Farmer. One particular standout mentioned in that paper is the CRISIS project, which used ABM to try to explain the global financial crisis.

ABMs aren’t perfect, either

ABM requires creating agents and environments. A key limitation to existing ABM is that their behavior is typically hard coded. For example, a given agent that is suppose to simulate a person looking for a job might simply take the highest salary. This way of doing things has two problems:

• It’s unrealistic. For example, lots of people take a job with a suboptimal salary to do something they find interesting, or somehow fits their lifestyle.

• If you ever want to change the environment to make it more complex, realistic, or something in the real world changes (such as laws, tax codes, etc), you have to go back and audit your agent code to see if it needs to be changed to fit to the new environment.

So basically, its slow, error prone, and unrealistic.

Simon Hess, the author of the JMAB 2.0 paper I referenced above, said to me in an email:

On the one hand, ABMs are appealing because of their more realistic micro-foundations and higher level of detail. On the other hand, they can make the model more complex, more difficult for others to understand, and more time-consuming to develop.

I completely agree with his assessment. In my early experiments with ABM, you’re basically writing complex computer algorithms, which are notoriously difficult to get right. At least in traditional finance, your formulas can be (semi) easily reviewed by other professionals.

Getting a qualified professional review complex computer code is expensive, time consuming, and in my experience, kind of useless. I can barely read my own code, let alone someone else’s. AI coding tools are helping with this, but they have a long way to go.

AI and reinforcement learning (RL) to the rescue

Here’s an idea: How about instead of giving agents brittle, rigid algorithms to follow, we give them a high level objective and set them loose in an environment?

This is exactly the idea behind reinforcement learning, and it’s led to terrific results in LLM training, robotics, and many other domains.

Put another way, we populate the environment with agents who desire something (or many things). These desires might be:

For a person:

• Increase consumption

• Increase wealth

• Work less

• Don’t go bankrupt

For a company:

• Increase dividend payments

• Increase stock price

• Lower costs

For a government:

• Increase GDP

• Keep inflation below 3%

• Make a rival’s GDP decrease

Each of these objectives can be defined with a simple reward function that look a lot like points in a video game. For example, a person might get 1 point for every consumer good they consume (maybe to a max of 70,000 points), .8 points for every dollar they accumulate as wealth, -.5 points for every hour they work, and -10,000 points for going bankrupt.

That’s all the “programming” we have to do for the agents. Now all we have to do is define an environment and set the agents loose. We don’t have to tell them “you must take the highest job offer, but weight that against working less using this formula”, they just figure it out themselves and try to maximize their points.

The catch

While investigating reinforcement learning (RL) frameworks, I found JaxMARL to be the highest performance, best-supported, and overall most impressive foundation to build on top of. Alex Rutherford, one of the main authors of the project, met with me and explained that while RL is incredibly powerful, designing reward functions is still a bit of a dark art.

I completely agree with his assessment.

My first project was using JaxMARL to train agents to play a currency trading game. The point of the game is to maximize your wealth at the end of the game. Initially, I found the agents incredibly easy to beat, until I changed one very simple thing about their reward function: Instead of telling the agents to maximize their wealth, I told them to maximize their the difference between their wealth and mean wealth held by other players in the game.

Essentially, don’t just worry about yourself - try to beat the other players in the game.

All of a sudden, I couldn’t beat the bots anymore.

Next steps

If a simple twist to a reward function can transform a subpar trading bot into an unbeatable opponent, what happens when we scale that logic up to an entire macroeconomic system?

My current project is re-implementing an environment inspired by JMAB and JMAB 2.0, but using RL instead of hard coded heuristics.

The idea is to move away from brittle hard coded logic and instead set these learning agents loose with nothing but basic, competing motivations. It’s going to involve a lot of trial, error, and wrestling with the "dark art" of reward functions, but I’m fascinated to see what kind of emergent behavior drops out of the simulation.

My hope is that creating realistic, low maintenance agents that can adapt to any environment you throw at them will significantly increase the velocity of economic agent based modeling research. Maybe it will cut the amount of code you have to write in half?

Stay tuned to see if a bunch of RL bots can figure out the economy any better than the economists.