Sci-fi economics
Overcoming the limits of a retrospective field
Above is a quote from Miles Brundage, former Head of Policy Research at OpenAI. It criticizes the economics profession for ignoring AI, and it is a sentiment I largely agree with. In this post, I argue that economists ignoring AI is part of a bigger issue with the field, in that it focuses on retrospective analysis instead of rigorous analyses of the future, which I call sci-fi economics.
Let’s take a step back and consider some of the most important economic questions. For students, what major should they choose? For employees, what company should they work for and what skills they should learn? For managers, what technologies they should invest in? For policymakers, what policies should they support?
In principle, economics has something to say about these questions. There are many excellent researchers studying labor markets, productivity, and so on. Economists have studied returns to specific majors and career trajectories, and policies to promote research and development. And these studies are quite interesting. But to apply the lessons from these studies to the future requires an unstated assumption, what is sometimes called temporal validity. In lay terms, this assumption posits that the future will be like the past. For example, if majoring in engineering had a high payoff in the past, there is an implicit assumption that this will continue to be so in the future. This can be justified to be a good assumption sometimes, but is often a bad one.
A simple theory is that if the world doesn’t change too much over time, then the choices that were good in the past will continue to be good in the future. But, especially in times of change, this theory can be a poor guide for making decisions. Skills taught in certain majors may quickly become obsolete, formerly large companies may go out of business, and technologies may get leapfrogged. Having a forward-looking view is critical for making good decisions at an individual and society wide scale.
Perhaps the most salient set of changes approaching us now relate to AI. With LLMs already being able to do chunks of white-collar work, it is likely that the organizational structure of industries and occupations will change. Even if not immediately, at least on timescales relevant to career, investment, and policy decisions (5 to 15 years). At the same time, the type of research favored by economists has little to say about these changes.
The reason is that the publishable unit in economics and many other social sciences is a retrospective analysis of an experiment or policy change. Consider, for example, “Generative AI at Work”, which is an influential and well-done study of what happened when call center workers were given access to a very early LLM. The paper finds that lower skilled workers increase their productivity more than higher skilled workers, and overall increase customer satisfaction.
Even though this paper will likely be published in a top journal, it has little to tell us about how AI will affect the economy, beyond that it will increase productivity, which is already well understood. Here are some reasons why. At the time of this blog post (2024), state-of-the-art LLMs are orders of magnitude better than the LLM evaluated in the paper. At the same time, end to end automation of customer service seems much more likely in the vast majority of use cases than a simple augmentation of customer service workers with AI. Lastly, whether AI benefits less or more skilled workers depends almost completely on context and organizational structure.
If economics does fewer retroactive studies, what should it focus on instead? It should think about possible futures and what will happen in those futures, what I call sci-fi economics.1 Sci-fi economics asks new questions, but does not necessarily need new tools. It considers new policies in economic environments that haven’t existed yet. Instead of assuming that an industry is structured in a particular way, it can imagine industries structured in completely different ways.
Let’s walk through an example of sci-fi economics that has nothing to do with AI. Suppose that the government is interested in promoting electric vehicle adoption. It can give refunds on electric vehicle purchases, it can build more charging stations, or it can subsidize domestic producers. It is useful to know which, if any, of these strategies work. For example, it could be that these efforts do barely anything to promote mass adoption of EVs or that they accelerate adoption by many years. In order to figure out which, we need to model the economic equilibrium. This modeling exercise is likely to be hard, and full of judgement calls. But it would be useful and interesting in a way that many economic studies are not.
An excellent example of sci-fi economics relating to AI is The Simple Macroeconomics of AI by Daron Acemoglu, even if I disagree with its model and conclusions. The paper tries to model how LLMs will affect total factor productivity (TFP) by using information about which tasks will be affected by AI and how much productivity will increase in those tasks. By clearly writing down his assumptions, Acemoglu creates an object that we can critique and debate. For example, we can argue that he too easily dismisses capital deepening,2 or that he doesn’t seriously model how competition will change due to AI. We also get a number, which we can evaluate ex-post to judge the veracity of the model. Importantly, the existence of this paper has made others more precise about their assumptions and forecasts.3
Zooming out a bit, there are many technological changes worth thinking rigorously about in addition to LLMs and electric cars. For example, Ozempic is likely to change the medical and dating industries in substantial ways. New longevity extending drugs are likely to change the wealth distribution of society in important ways. Autonomous vehicles will change where people live and how much housing is worth. Cheap space flight will enable new industries. But economists are mostly choosing to stay silent on these issues, and are losing relevance as a result.
Why don’t economists do more sci-fi economics? Many are careerist and worry that this type of work will never be published in a top journal. This can be viewed as a lack of courage and conviction. But by avoiding sci-fi, economists are ceding the debate to other less rigorous fields and communities. As a result, the useful tools economists have created for studying the world are likely to go unused, to the detriment of society.
Another type of sci-fi economics has a much more long-term and theoretical bent. For example, Krugman’s A Theory of Interstellar Trade consider interest rates when travel near the speed of light is possible. This is really fun work, but not exactly what I’m talking about.
Capital deepening refers to an increase in the amount of capital (like machinery, equipment, technology, or other productive assets) per worker in an economy.
See this article by Maximum Progress for a critique:
