We read the AI 2040 “Plan A” essay: a future in which the US and China agree to pace and monitor AI development, data centers are placed where they’re easy to bomb (hello, Mongolia), and 2040 arrives with abundance, a citizen’s dividend, and lie detectors. Before reading, the three of us stated our priors on two questions: will an international AI pacing and transparency treaty exist by 2030, and would one with teeth be a good idea? Phil started at two-thirds and 75%; Seth at 30% and 60%; Andrey at 20% and 49%.

Along the way: whether the essay’s growth forecast is load-bearing (Phil: it’s close to a wash, and Seth and Andrey push back), why a world where AI looks like nuclear weapons might be easier to coordinate on, Chad Jones’s risk-aversion argument, whether AI itself makes monitoring and commitment cheaper, why monitoring every data center sounds absurd to Andrey and like scrubbers on smokestacks to Phil, mind-reading and privacy, and Seth’s worry that a one-time lump-sum rebate of the universe ends in an extreme power-law distribution of wealth. Then Phil delivers a mini-lecture on renegotiation-proofness, the piece of game theory he thinks the essay is missing, and on why it may be safer to go faster if most risk is state risk rather than transition risk.

We close with posteriors, a discussion of Phil’s work on recursive self-improvement (RSI), and a lightning round: why Epoch instead of Stanford or Anthropic, his giving pledge and wild animal welfare, and how a Piketty question from Dwarkesh turned into one of the most-read things Phil has written.

Links & References

Phil’s work

• philiptrammell.com — Phil’s website · Epoch AI

• Charles I. Jones & Philip Trammell, “When GDP Misleads: Inferring Living Standards from the Value of a Statistical Life” — our earlier episode on the paper;

• Dwarkesh Patel & Philip Trammell, “Capital in the 22nd Century”

• Phil’s Epoch piece on parallelization constraints and his in-progress follow-up on whether RSI is imminent.

• Aschenbrenner & Trammell, “Existential Risk and Growth” (a Chad Jones tweak on state risk vs. transition risk) — Leopold’s Situational Awareness.

• Cunningham et al., “The Economics of Recursive Self-Improvement” (Elasticity Institute; Tom Cunningham led, Phil is a co-author)

The essay and the scenario

• AI 2040: Plan A — the global pacing/transparency treaty, data centers in Mongolia and Canada, the “citizen’s dividend”

• Dan Kokotajlo and the AI Futures Project, authors of the essay

• Lydia Nottingham’s reply to the “if everyone would just” framing of coordination plans.

Papers, concepts & people discussed

• Chad Jones, “The AI Dilemma: Growth versus Existential Risk” — willingness to take on existential risk as a function of wealth and mortality; our earlier Chad Jones episode

• Hall & Jones, on the value of life and the rise in health spending — people get more risk-averse as they get richer

• Armstrong, Bostrom & Shulman, “Racing to the Precipice” — why you want a big gap between the leader and the follower

• Renegotiation-proof equilibria and the grim-trigger strategy in a repeated prisoner’s dilemma.

• C.S. Lewis, The Abolition of Man — technology and the “middle of history” when we’re most free to plan the future

• Thomas Piketty, Capital in the Twenty-First Century and Brad DeLong’s early point about capital substituting for labor

• Gell-Mann amnesia, which Phil invokes on the verification discussion

• Tom Davidson’s work on takeoff speeds and automating AI R&D, e.g. Davidson & Eth); algorithmic efficiency, sample efficiency, and RLVR.

Previously on Justified Posteriors

• How much should we invest in AI safety? — existential risk vs. growth with Chad Jones

• Our “When GDP Misleads” episode.

Chapters

• (00:00) Cold open

• (01:22) Intro, and our priors on an international AI pacing treaty

• (08:18) The evidence: a walk through Plan A and its 2040 utopia (Mongolia included)

• (11:25) Is the growth forecast load-bearing? Immortality, the FDA, and a unipolar 1990s

• (19:06) AI as a normal technology, and could AI make coordination easier?

• (24:39) Monitoring every data center, and can the plan last until 2040?

• (31:02) Lie detectors, mind-reading, and privacy

• (34:38) Concentrated wealth: lump-sum rebates vs. power-law inequality

• (37:59) Renegotiation-proofness and Racing to the Precipice

• (42:14) Can competition deliver safety? Profits, moats, and a spirited disagreement

• (44:48) State risk vs. transition risk: is it safer to go faster?

• (46:29) Posteriors

• (53:03) Recursive self-improvement: is it imminent?

• (1:02:58) Lightning round: Epoch vs. Anthropic, the giving pledge, and the Piketty/Dwarkesh essay

Justified Posteriors is the podcast that updates its beliefs about the economics of AI and technology, hosted by Andrey Fradkin and Seth Benzell. If we changed your priors, subscribe and keep your posteriors justified.

Transcript:

Cold Open [00:00 - 01:22]

[00:00:00] Andrey: Do we think that we’re gonna have an international AI pacing and transparency treaty by 2030?

Phil: In the extreme case where AI was only disruptive, it would be easiest to coordinate a slowdown. What makes it hard is the prospect of abundance if we don’t.

Seth: Isn’t this a universe that’s inevitably setting itself up for an extreme concentration of wealth?

Phil: How you feel about that depends on whether you think there’s something intrinsically undesirable about wealth inequality. Not sure there is.

Andrey: One of the requirements of this plan is to create monitoring devices and monitoring software in every single data center in the world.

Seth: Will the Turing police come to my house if I have too nice a graphics card to play my video games?

Phil: It seems a bit inconsistent to be worried about that, but then not feel that AI would really lower the cost that would go into a regime to coordinate its development or deployment.

Andrey: What if Russia builds a data center and decides to just go for it?

Phil: You really want a big gap between leader and followers, that the leader can slow down for safety purposes. But if we’re all sharing information and the coordination is fragile, then when it unravels, we’re as neck and neck as can be. Their proposal to mitigate that, of course, is for us to destroy each other’s big data centers in Mongolia and Canada.

Andrey: Suppose countries actually are going to abide by it and it substantially slows down the pace of AI progress. Is that a good or a bad thing?

Intro and Our Priors on an AI Treaty [01:22 - 08:18]

[00:01:22] Seth: Welcome to Justified Posteriors. This week, we have on the incomparable Phil Trammell of Stanford and senior researcher at Epoch AI to talk to us about AI-driven economic growth. But first, we’ll get his views on AI 2040’s Plan A, a vision of a possible future where AI is globally regulated for the good of all. I’m Seth Benzell, always ready to pause my work for a good time, coming to you from Chapman University in sunny Southern California.

Andrey: And I’m Andrey Fradkin, coming to you from San Francisco, California. Very excited to have Phil on the show, who I consider one of the most original thinkers in this space, and also one of the most, morally upright. So welcome to the show, Phil.

Phil: Oh, thank you. I’ll try to live up to that.

Andrey: I did wanna kind of ask for our priors. And in particular, do we think that we’re gonna have an international AI pacing and transparency treaty by 2030?

Phil: I think it’s far from a sure thing, but I’ll, I’ll give it two-thirds.

Seth: Two-thirds probability. We get a sixty-six percent chance for Phil. I have to say, I put it closer to thirty percent. I think there’s a real probability, but I think there are some big, implementation problems, some of which are discussed in this, essay and some of which aren’t. I think there’s also big American and Chinese domestic policy issues. You know, I think my probability would be closer to two-thirds, maybe closer to fifty percent, if, you know, we were guaranteed to not have a, quote, America first president in 2028, but you always have to have some probability of that happening. I also want people to think about there’s kind of a... Some distributional aspects of what’s gonna be going on in this policy. I don’t think it’s a pure Pareto improvement, especially the kinds of people who might kind of really want acceleration of technologies by a couple of years might make a big difference to them. People who are maybe experiencing health challenges, or have some money in the game that’s betting on that, will definitely be a force in domestic policy in both countries for against slowdowns.

So I’d be closer to thirty percent.

Andrey: Yeah. And I think I’m more on the twenty percent range. I actually think there will be a bill that will be framed as an international AI pacing and transparency treaty. I just think it will have no bite, so I don’t count that. You know, a a lot of people have, you know, they look at the successes of international, policy agreements with regards to chemical or nuclear weapons. But I look at the failures, and in particular, the League of Nations or even the United Nations, and many other international bodies which make a lot of proclamations, but ultimately have no actual bite. And so I think what is more likely is a unilateral slowdown on behalf of the US simply due to political pressures from people who dislike, AI for a variety of reasons. So I think that’s much more likely than an international AI pacing agreement. But I’m curious, you know, as we discuss how, our beliefs will update. The second prior that we wanted to, put out there, this is a much more difficult prior in some sense, is do we think, such an international AI pacing and transparency treaty is desirable?

In particular, suppose there is one that has bite, so countries actually are going to abide by it, and it substantially slows down the pace of AI progress. Is that a good or a bad thing?

[00:05:11] Phil: Yeah. I think it is probably a good thing. It depends obviously on the details, and I can see a few ways that it would go wrong. But I think basically, there’s a lot of room for improvement here, and the burden of proof is really on someone to think that, like, coordinating to get close to the Pareto frontier actually makes things worse. I’m gonna say seventy-five percent.

Seth: I’m actually less far away from you going in than I thought I would be. I’m closer to sixty percent. I think on balance, you know, just framed as international pacing and transparency, you know, I’m... I think the most kind of broadly robust argument from that is something like an arms control logic, right? I th-- I would worry if I am she or president of the United States about someone attaining a really devastating cyber first strike capability. And we all know how about how, you know, if you have a first strike capability, but you don’t have mutually assured destruction, how kind of dangerous those dynamics can be, and kinda nobody wants that. So I do see there being a little bit of a positive logic for, hey, let’s make sure that no side gains access to a devastating first strike, right? But on the other hand, I guess my main concern is just that this will slow down growth too much. I guess we’ll learn more from the essay about what they think the passive path of economic growth looks like under, you know, kind of with the pause and then maybe implicitly without the pause.

But, you know, just ex ante, I mean, slowing down technological development, I mean, you gotta have... It’s a pretty high bar for you to clear for me before I’m really gung-ho about that.

Andrey: All right. And then for me, oh, it’s so it’s so detail-dependent for me. It really is. But I think I’m lower than you guys, maybe around fifty percent. And in particular, I worry that any such regulation is gonna overfit on a very specific technological paradigm, whereas there are gonna be lots of dangerous capabilities that are gonna be arising, but also very valuable capabilities are gonna be arising through other architectures, other ways of using AI. Even today’s AI capabilities are, you know, I think we have a lot of room to figure out how to use them. And so, controlling the pace of AI progress is maybe not how I’d frame it. I would more think about controlling the misuse of AI, regardless of what progress level it’s at, I think is a very interesting and difficult problem. And, you know, something to help with that, I think would be good, but I, but I also worry that it’s gonna have drastic costs, in terms of our privacy and in terms of our, autonomy from the government and also in terms of economic growth as well.

I would not support plan A, just to be clear.

Seth: Is it fair to call you forty-nine percent?

Andrey: Sure. You’re... Yes.

Seth: All right. Well, then, let’s see if after reading more about this plan and the future it imagines, Andrey and the rest of us change our minds. Let’s go to the evidence. Do, do. All right, Andrey.

The Evidence: Walking Through Plan A and Its 2040 Utopia [08:18 - 11:25]

[00:08:18] Andrey: Well, this is a very funny form of evidence, Seth. It is essentially a science fiction scenario-... That is painted by the authors of AI 2040. And so let’s kinda walk through exactly what’s going on. So the authors here are very bullish on future AI capabilities, perhaps unsurprisingly. Their kind of conceptual framework for AI’s impact on the economy is that AIs are like human laborers. They’re essentially like a drop-in replacement for a certain type of human labor. And we’re gonna have just a ton of these already, in the next couple of years to the point that AI becomes the most important issue in the twenty twenty-eight election. There is already gonna be massive white-collar disruption and a lot of discourse about the intelligence explosion. Whenever this newly elected government comes into power in the US, they’re gonna have to choose a path. And the path that they prefer is so-called plan A.

Seth: They prefer, meaning the authors of this essay.

Andrey: The authors of this essay prefer. Which, consists of, an agreement with China for declaring all compute resources, monitoring what they’re being used on, in particular, making sure that there, there’s not too much effort devoted to improving AI models. And the reasoning here is that the AI models are already gonna be really powerful in twenty twenty-nine. It’s just too dangerous to use them. We have no idea how to do alignment yet, and so they should just be very slowly paced. And furthermore, there are gonna be incentives, essentially, tit for tat incentives. So if someone breaks this agreement and starts doing a gigantic training run on a superintelligence, one of the proposals here is to put data centers in places that are not China and the US, for example, Mongolia. And then if China decides to break the agreement and, train a really superintelligent model, the US can bomb Mongolia, and it’s not gonna trigger a nuclear war.

Seth: The other way around. Wait, no, China bombs Mongolia to destroy the US data center, and we bomb Mexico to des-destroy the Chinese data center.

Andrey: My bad. Yeah, exactly. Yeah, exactly. Yeah, so it’s, so it’s, like, right there. It’s very easy to bomb, -... You know, for one country and not the other. I think that’s kind of the high-level vision. And then-

Seth: And in twenty forty, it’s utopia, and we all get a slice of the universe.

Andrey: There’s just so much of a boon from AI progress that we’re gonna get all these things that technologists have talked for, which is just abundance everywhere. We’re gonna get UBI/citizen’s dividend. We’re gonna have to figure out what to do after no one needs to work anymore. Yeah, the, I think that’s kind of the high-level version of the plan. I don’t know. Am I missing anything here?

Phil: I think you got the high-level points. I think the, like, details of the economic forecast are mostly separable from the plan to reduce AI risk. So I hope that the baby doesn’t get thrown out with the bathwater if you think that some of the economic claims are bathwater.

Is the Growth Forecast Load-Bearing? Immortality, the FDA, and a Unipolar 1990s [11:25 - 19:06]

[00:11:25] Seth: I wanna pause you on that one for a minute before we go onto your other insights because I was thinking about this too, and it really does... You know, the paper, kind of imagines fifty percent GDP growth by twenty thirty-two and just acceleration after that. And to me, it did feel a little bit load-bearing, right? Because it’s much easier to make this argument that we can afford a shutdown if in the background we’re having enormous growth in material resources, right?

Phil: No. I think it’s the other way around, yeah So like in the extreme case where AI was only disruptive, right? It was like nuclear weapons, it would be easiest to coordinate a slowdown. What makes it hard, as you were saying before, is the prospect of abundance if we don’t.

Seth: Right. Right. But I’m-- Well, let me give you an example of what I have in mind.

Phil: No, I know it can go the other way, that like- ... You, maybe we can afford to half slow down given the other half, you know, of the benefits. Yeah. Yeah. But that’s, and you’re still foregoing the up, foregoing the other half. It could go either way, but I think on balance, it’s, I think it’s close to a wash, which is kind of what I was saying. Like I think that they’re mostly separate. But yeah, if I had to guess, the more it’s like an arms reduction agreement, the better. Or the more, the more likely.

Seth: And I’m confused. It seems to me like it’s just a disruptive technology is more like the arms control agreement, which would make it more enforceable.

Phil: Yep. That’s exactly what I, what I meant to say. So the more we think that AI has power to destroy but not to create, the more likely it is that we’ll be able to coordinate on slowing it. Even despite the point that, the benefits of AI could, to some extent, pay for the slowdown.

Seth: Interesting. Andrey, where do you come down on that? I-- To me, I feel like I’m, I’m an optimist. I always wanna look at the happy vision, right? A vision where it’s like I have to choose between five percent GDP growth and, you know, X percent chance of the world blowing up. That’s a harder choice for me than, you know, choosing between fifty or a hundred percent GDP growth and I can give up... I feel like I can pay for-- I am richer to pay for the world blowing up, is my intuition. What’s yours?

Andrey: Yeah. I have a, I have a similar intuition, although I think like some of these trade-offs are not exactly gonna be very salient. From my perspective, I think the crux is gonna be actually on the debate over these things. Like if... I doubt that the debate is gonna be as stark as like, “Oh, if we go for this bill, we’re gonna, you know, we were gonna have fifty percent GDP growth and now we’re only gonna have five percent GDP growth.” Like I just can’t imagine that that’s what the debate is gonna be like. I mean, you know, we could all be wrong, but I think at least given the definition of GDP, I think fifty percent GDP growth is quite unlikely, in this short of a timeframe. So it doesn’t seem very plausible to me, but maybe that’s not even what we care about. And, you know, welfare is not GDP as, you know, we’ve discu- we’ve discussed in the past, specifically with relation to Phil’s papers.

[00:14:33] Seth: Andrey doesn’t think it’s load-bearing. Can I give you a twist on it by which it might be load-bearing? So, you know, Chad Jones has these papers about how much existential risk are you willing to take on as a function of kind of thinking about your own mortality, right? And so one of the arguments is that if you think that like we’re pretty-- and this is a stylized version of his arguments, which is like, if you think we’re pretty close to inventing immortality, that’s worth a lot to you, and you might be willing to take on a lot of existential risk today to make sure you get the immortality before you die, right? So I mean, how do you think that plays into this? I mean, I’m thinking about like the person in this future who’s like a retiree in twenty thirty-five and they’ve got cancer and they really wanna let it rip so they can make it to the miracle cure, and they’re gonna be out protesting on the streets.

Do you see them as being a force against such an agreement?

Phil: Well, they might be in a hospital. No, but I mean, I mean that. Like in general, th-this point that people who have life-threatening diseases will be a real political constituency ‘cause they’ll like organize to speed technological development. I think, that isn’t what seems to have happened with the FDA. I think there’ll be an even smaller share of the people that care about AI progress than they are people who care about the FDA. So yeah, I take your point, but I think making sure that like, you know, Jensen Huang gets compensated seems like the more important thing than making sure that the cancer victims get compensated.

Andrey: From a political perspective.

Phil: Not morally, but from a political account. Yeah. Yeah.

Andrey: It just seems like that’s not how policy is made. Like, I mean, I understand the Pareto improvement point, right? Like it, it’s the same point as the UBI point or even just the China shock point in that like, well, if we had known that the China shock would cause these displacements to these workers, then we could have easily compensated them and it would’ve, you know, everyone would’ve been better off under some set of circumstances. But I don’t think in history we’ve ever succeeded in enacting such a policy. So I don’t see why this time is different.

Seth: Right. That’s just wars, right? Wars are not Pareto improving, right? It’s their failures of bargaining. I guess, let me ask you a question, Phil. I know you’re not an international relations expert, so, but I wanna hear your take. Would the world be better off if the intelligence explosion had happened in like late nineties geopolitics during the unipolar moment for the United States? Is that like a better future where it’s just one democratic country basically gets to set world policy versus this kind of more multipolar world we live in?

Phil: I expect so. You know, just for the same reason that in models of technology races, risk is often lower when there’s a big gap between the leader and the follower, because they can sacrifice more for safety.

[00:17:35] Seth: Tell us what would, what would Chad Jones say, Phil?

Phil: You know, a point Chad Jones was first known for making was that, people get more risk-averse as they get richer, right? This is the punchline of Hall and Jones on the, value of life and the rise in health spending and life and growth as well, that we might sort of be more willing to put the brakes on technological development to get richer still once we’ve already gotten sort of rich enough. And these would both be reasons to think that if AI really is capable of delivering a lot of abundance, that’s actually an optimistic story for the feasibility of a plan to slow it, you know, for the sake of safety. As you pointed out, though, there are arguments pointing the other direction. There are ways in which we could get a lot richer because of some technology and yet be sort of less satiated, be kind of more keen for yet another proportional, improvement to the technology you know, as compared to a reduction in the chance of death.

And one of those channels is via life expectancy, right? So if you have something like linear utility in years of life, then, each doubling is actually more valuable to you than the last. On balance, my sort of central guess is that it roughly cancels out and that your forecasts about the GDP growth or the prosperity that AI will deliver are mostly separable from the question of whether we can coordinate a slow-down.

AI as a Normal Technology, and Could AI Make Coordination Easier? [19:06 - 24:39]

[00:19:06] Andrey: What if we’re in the AI as a normal technology world? I mean, we’re gonna continue having, you know, two to three percent GDP growth. You’re kind of assuming that the safety concerns are separable maybe from the economic growth concerns, but you might imagine that they’re kind of highly correlated. And in that case, if people see, like, look, we don’t have mass unemployment and we have pretty normal economic outcomes, it’s gonna be a lot harder to convince people that there is some grand safety risk happening.

Phil: There’s probably almost always a room for some freight of improvement, right? The question I think is whether it’s, like, worth the cost, worth the investment in doing all of that, you know, negotiating or implementing of monitoring systems or whatever, or like, you know, clever construction of incentive-compatible R&D funding regimes. But, like, even in AI as a normal technology world, well, normal technologies are global public goods, and I think we would wanna coordinate on funding AI more and getting in the way less of each other’s ability to make profit off of creating this public good, you know, by distillation or whatever. Lots of countries do have, you know, agreements to sync up their intellectual property regimes, for instance, and, I don’t think that’s implausible. As the stakes get higher, you are really pessimistic that we would ever kind of sort out something roughly efficient, but I don’t know. I think there are lots of examples in history of both within countries and across countries working out gains from trade, metaphorically speaking and-

Andrey: Most years, pairs of countries aren’t at war. Ninety-five percent of country pair years- ... They are not fighting each other.

Phil: Yeah. So I think it could go either way, but saying, “Oh, nothing like this has ever happened,” seem- seems a bit quaint. So that’s the first thing I wanted to say. The second thing is, I think there are a lot of ways in which AI, especially if capabilities advance a lot soon, which I think they will, but, you know, which is sort of the premise of this essay. There are a lot of ways in which it might especially accelerate the long historical process of coordination getting easier.

Andrey: Ah, cozy and singularity take. Here we go.

Phil: In general, when you think about, like, what tends to make cooperation easier, monitoring is a big one. And I mean, this is something we worry about with AI, right? That, like, you know, autocratic governments can use it to monitor its citizens. But, like, it seems a bit inconsistent to be worried about that but then not feel that, like, AI would really lower the cost, at least some of the cost that would go into a regime to coordinate its development or deployment. Another one that’s a little iffier but, maybe more powerful in some contexts is commitment. When we were in a Cold War with the Soviet Union, one way in which we tried to keep the peace was through a sort of dead-hand mechanism where, we would pre-commit to retaliating if the other party defected from the kind of agreement not to, not to use nuclear weapons. You’d want a lot of reliability, of course. But in principle, if you had a system that was reliable enough and, you know, intelligent enough to figure out when they’re actually being bombed or whatever and when it’s just a flock of geese, then commitment could also facilitate coordination.

And then just, like, it’s taking an even bigger step back. A common criticism that I’ll see people express about, this plan A or, like, other plans for, coordination between countries or companies is that it sort of, it unrea- realistically treats the group as a single agent. It’s this, “If everyone would just,” right? And there was this great reply I saw someone named Lydia Nottingham make, online, which is, you know, we might be on the verge of technology that will let everyone just for the first time. I’m sort of reminded of this passage from The Abolition of Man, in which, like, C.S. Lewis argues that technology, as technology advances, we get ever more ability to manipulate nature, but people are part of nature. And so there’ll come sort of, some sort of point at the middle of history when, we’re kind of most free to plan the future. And after that, things will be sort of locked in because our descendants were part of the, like, objects being planned by the ancestors.

And yeah, I mean, you just think about, like, you know, how uncoordinated all of the, like, hunter-gatherer tribes were, you know, twenty thousand years ago and how kind of, how many fewer actors there are that would need to coordinate today. There’s just, like, a couple big countries. Yeah, I don’t know. Maybe AI will facilitate the kind of the grand process toward a, toward getting toward the global frontier.

[00:23:47] Andrey: I guess, like, there’s a lot of optimism on your end on agreement between the parties for what the desired thing is, but, like, let me just say that, like, the various factions, in the AI safety world have not proven convincing of most informed, even of informed peo observers about, for example, the importance of, like, AI takeover risk.

Seth: Well, are they disagreeing over risk probabilities, or are they disagreeing over utility? I think Phil is optimistic about utility, maybe not beliefs. Is that fair, Phil?

Phil: No, I mean, I think Andrey has a great point that there’s a lot of persistent disagreement over what the risks are. I think there has been some convergence. That’ll continue to happen, so the question is whether it’ll happen quick-quickly enough for an agreement to be made before it’s too late.

Monitoring Every Data Center, and Whether the Plan Can Last to 2040 [24:39 - 31:02]

[00:24:39] Andrey: One of the requirements of this plan is to create monitoring devices and monitoring software in every single data center in the world. What do we think of that as an idea, right? Taking a step back, it just sounds absurd, right?

Seth: Will the Turing police come to my house if I have too nice a graphics card to play my video games?

Phil: First off, you say monitoring, like, what’s kinda nice is that we have all these results now and verification on how it’s possible to distinguish between, for instance, inference and training without revealing, like, what the inference is. So, I mean, the idea that, like, there’d be some infrastructure that would be a part of, like, any large plant that has a negative externality of some kind seems more analogous to, like, scrubbers on smokestacks or something.

Andrey: I would not trust other people’s software sitting on my computers and, you know, yes, in some abstract theoretical world, this software has the perfect properties, but generally, like, we know that software is highly imperfect.

Seth: It would use AI to prove to you that it was good software.

Andrey: We know that the history of, the countries involved includes vast amounts of hacking, and corporate espionage and-

Seth: But transparency is part of the argument, Andrey. Part of the argument is we wanna make this all transparent. I guess you’re worried about transparency in inference when we’re-- what we want is the transparency in training runs. Is that the distinction?

Andrey: I refuse the inference/training run dichotomy here. People are constantly overfitting on the last generation of how AI work is done. You could imagine inference being used to develop new AM-- AI methods that are then able to do very powerful things in new ways that are not detected as training runs.

Seth: Wouldn’t you prefer a world with a thousand human-level intelligences to one thousand X human intelligence? I mean, I think that’s implicit in this paper.

Phil: I wouldn’t, for sure. But it really depends on what the su-- what we know about the superhuman intelligences that exist. I think superhuman intelligence existing is inevitable, and it’s not even a good way to conceptualize things because there’s superhuman intelligence everywhere around us today.

Seth: Phil, what do you think of this inference, infer as much as you want, but, train under highly regulation and public circumstances part of the deal?

Phil: So yeah, first off, I do agree with you, Andrey, that the deal kind of assumes that the technology will be more stagnant than I think we can expect. I think what they might say is, well, if one of the things we’re coordinating on is stopping AI R&D, basically, except in, like, ways that w-we can all see and therefore respond to, and like, you know, like, the regulation can respond to it. If we’re banning everything else, then it’s fine to overfit to what we have today or what we have when the agreement’s made because, like, that’s how the technology will, remain. The argument for believing that is, as far as I can tell, is basically that no one will have an incentive to do any big R&D projects because the outputs will all be public, right? So it’s like, you know, if there’s no patent and no trade secrets, then, there’s no incentive to do R&D. But of course, we know that often R&D happens anyway, right?

Like, there’s open source software, there’s just hobby projects, there’s philanthropy.

[00:28:03] Seth: And even under oligopolistic competition, right, even if, you know, if you can make the pie go up, even if we’re gonna split it evenly between China and OpenAI, it’s still, you know, make pie go bigger.

Phil: Yeah. I guess the thought would be, though, that, like, if all the IP is public, then it’ll be a bit closer to perfect competition.

Seth: The moat will still be the enormous amount of compute, right? So there’ll still be a moat. It won’t be free, you know, everyone gets to have their frontier model.

Phil: Yeah. I mean, it might start out that way, but then things could spiral out of hand if, like, one of the ways in which we make progress allows for more distributed training runs, smaller training runs, and so on.

Andrey: Well, I mean, like, yeah, I mean, like, what if Russia builds a data center and decides to just go for it? I mean... And, and/or decides to sell compute to other places, and then they’re gonna say, “Well, you know, only ASML can produce, you know, lithographic devices,” and so on. But I-- But if you are truly bullish on the capabilities of AI even by ‘29, it should be pretty easy to get the AI to tell you how to create these things. And then once you’re able to create them, and then other, you know, nation-states certainly have incentives to train, even if it’s not profitable for their domestic, security, but also for maybe they wanna have some technology that they think is good for their citizens, you know?

Phil: Yeah, I think this is all consistent with what I was trying to say before, which is that it seems feasible to me, for a bit at first to really slow down progress and have the sort of arrangement not be overfit because, you know, you remove the main thing driving progress today, the possibility of profit, and, or at least you reduce that a lot. And also because as you were gesturing at, Andrey, the supply chain is currently so narrow that if, you know, you have the Netherlands on board and Taiwan on board, you know, and South Korea or whatever, then there’ll be that long lag before Russia can be producing data centers on its own or whatever. What I was not convinced by in the plan was that it could last all the way till 2040, right? That might be all right. I mean, it, there might still be some sort of differential Technological development where you slow down some risky projects and speed up some safer ones on the whole for five years or something.

Andrey: What’s your main reason for why you think it would fail before 2040?

Phil: Oh, well, I mean, you were getting at them. So, like, you know, and I, well, I said before, like, it’s a combination of the fact that technology can progress even after you’ve significantly weakened the profit motives and the fact that as technology progresses, the bottlenecks that the plan is sort of relying on, so the narrowness of the chip supply chain being one, can be relieved as advanced AI tells its prompters how to, you know, have a more rapid way of building an EUV machine or a way of, like, doing a successful distributed training run out of all the iPhones in people’s pockets or something like that.

Lie Detectors, Mind-Reading, and Privacy [31:02 - 34:38]

[00:31:02] Andrey: So another interesting part of their proposal or part of their vision is that we’re gonna have essentially panopticon-like technologies to detect whenever anyone lies.

Seth: Oh, yeah, that’s fun. By 2035, we solve politics with a lie detector.

Andrey: First of all, I do think that this kind of no privacy dystopia, and not, like, online privacy, but, like, literally, like, people can just read your mind all over the place dystopia is a possibility. I don’t see an obvious reason why AI wouldn’t be able to learn a lot about your thoughts and motivations from analyzing your face in some way. But I’m curious what you guys think.

Phil: Okay, I don’t know about the face. I mean, I think if it was just about the face, then it would seem annoying but not impossible to-

Seth: It could just put, inject a numbing agent, and then you’d be completely unreadable.

Andrey: Well, yeah. Obviously, yes, it could also have more advanced technologies than that.

Phil: We already got used to wearing masks for a few years, right? But more kind of concretely, someone I know, actually, Rio Popper is her name, she’s blind and has been working on a startup to basically allow mind reading, at least enough mind reading to, so that people with i-impairments like that can control a computer, control a robot with their mind. We currently have some ways that people can kind of crudely manipulate an object with their thought. I mean, obviously... Anyway, well, this isn’t my, this isn’t my area, but, like, they’re really aiming for, I think, something as flexible as an LLM is today. We just put in a natural language prompt, and it does it. You could just have a, you know, natural language thought, and some machine would go out and do it. But anyway, so that requires a lot of mind reading, and, if they succeed, I see no reason why they wouldn’t also have developed technology that would let people read each other’s minds, at least if the person consents to disclosing what’s in their mind.

Yeah, that seems less worrying than the face case ‘cause people at least retain the right to not disclose what they’re thinking. But in some ways not, because there’s a lot you can learn from the fact that someone doesn’t disclose.

Seth: Well, we would need to develop norms around it, right? We would develop... And then that, that’s what the essay imagines is that, you know, the politician will be able to say, you know, “Don’t read my mind to see if I’m gay, but you are allowed to read my mind to see if I’m gonna cheat on this, tax bill,” you know, or defect on policy.

Phil: Yeah. Yeah. Which I guess we already sort of have for, just, like, search and seizure, like, for private documents, right? You could infer things about someone’s tax fraud or about their sexuality from their, you know, files maybe, but-

Seth: And generally, we have good norms, you know, not imperfect norms, but we, we’re able to work with those.

Andrey: But I guess, you know, my thought is that this technology could potentially be applied without consent. That’s, I, it’s just hard for me to imagine that it wouldn’t be that way.

Seth: I would have my AI fight the AI that you have that’s trying to read my mind to protect my mind.

[00:34:04] Andrey: I imagine that, it might be very asymmetric, but... I mean, I mean, I, I’m, I’m optimistic, Seth, for your solution, but actually, and especially in more totalitarian, regimes, I don’t think they’re gonna have any qualms about forcing everyone. You know, at the end of the workday, “Did you work hard today?” And, you know. Yeah.

Seth: You worked seventy percent, thirty percent pay cut.

Andrey: Yes. I mean, that assumes that humans still have jobs, so I’m sorry for that. But,...

Concentrated Wealth: Lump-Sum Rebates vs. Power-Law Inequality [34:38 - 37:59]

[00:34:38] Seth: I have one sci-fi question. This is kind of, like, my last, question on this topic, which is, the paper wants to argue that it wants to prevent kind of two bad outcomes. One bad outcome is, like, rogue, out-of-control AI, and the other bad outcome is, like, a bad actor getting concentrated power over the future of humanity, right? And I think the essay does a pretty good job about, you know, explaining why it would be a good idea to do this in order to prevent, like, the rogue, out-of-control AI scenario. But I think it is much less compelling about the concentration of power, and I’ll tell you in particular the thing that I have in mind, right? Which is we’ve created all this enormous wealth in 2040 in the vision of this paper, and then there’s a question of, okay, how are we gonna distribute it? And the answer is basically we’re gonna do a lump sum rebate of the entire universe to everybody. Everybody gets one ten-billionth of the universe, right?

Setting aside kind of the fantasticness of that for a minute, you know, Chad Jones and others have worked on models of sort of wealth dynamics, over time. And in these models, what you get is if you, if people are allowed to, own and compound wealth freely, what you get is a power law distribution. A very extreme power law distribution of wealth emerges over time. And in particular, the extremeness of that wealth is increasing in the interest rate, right? This is a universe where the interest rate has gotta be some ungodly, you know, two X a year amount. And it decreases in the birth and death rate. And I think this is also a universe in which death rates come down a lot. So isn’t this a universe that’s inevitably setting itself up for an extreme concentration of wealth?

Phil: Yeah, I think, you know, absent a lot of redistribution, that’s what you should expect.

Seth: It’s gonna do a one-time redistribution, and then it expects it to solve equality forever. That’s the kind of naiveté I see.

Phil: Yeah, absent ongoing redistribution. Right. Yeah, I mean, so this was the thought that motivated this essay with Dwarkesh I wrote almost a year ago now. I guess how you feel about that depends on whether you think there’s something intrinsically undesirable about wealth inequality. I’m not sure there is, right? Or about power inequality, for that matter. I mean, I don’t share, you know, say, Piketty‘s view or a lot of other people’s views that wealth inequality has to have negative externalities. I think sometimes it does, but sometimes it has positive externalities and just kind of depends who who’s rich and what they’re doing with it. Once most income is capital income, those forces which were around all along will just be, like, magnified by the fact that they’re operating on, like, a larger sh-share of our income.

Seth: I agree with that. I think that puts us... You know, whether or not you agree with Piketty’s moral calculation, that puts us in, you know, exploding wealth inequality u-universe, right?

Phil: No, I... Well, I agree. Yeah. I think there’ll be a lot more wealth inequality, and there’ll be less, there’ll be less of the dynamism.

[00:37:38] Andrey: I mean, I view it as a good thing because I think as a human, I wouldn’t be happy living in a world where there was no dynamism. I think I would be profoundly unhappy if I was born and I had no chance to change my lot in life.

Seth: Phil, any last thoughts before we move on?

Renegotiation-Proofness and Racing to the Precipice [37:59 - 42:14]

[00:37:59] Phil: Two things that any other economist listening, could maybe take a look at are, first, you know, obviously th-th-there’s all this work on the game theory of, like, global public goods, right? Global coordination. And indeed, you know, a, a lot of it was developed to think about climate change and, you know, kind of never really got put to use as we were, as we were discussing before. But maybe people who have been, you know, spinning their wheels on that front can finally come riding in on horseback. But just for an example, it seem, it seems to me that, like, the authors of AI 2040, they know some game theory. What they know really well is, you know, AI and the latest results in verification and so on. But an example of a bit of game theory that seemed missing to me was, an analysis of whether the proposal was renegotiation-proof. Okay? So this is just one little, one little observation. It’s just, like, a bit of, you know, game theory catnip or something.

But for anyone who doesn’t know, I think this is, like, a, a really neat concept. A subgame perfect equilibrium is renegotiation-proof if it puts us on the Pareto frontier, not just on the equilibrium path, but in every subgame, right? So if someone defects, we move along the Pareto frontier in a direction that they dislike, right, and the other party likes, but there’s never an opportunity to renegotiate and sort of take advantage of further, you know, further possible improvements, even after a defection. So for instance, the grim trigger strategy profile in a repeated prisoner’s dilemma is not renegotiation-proof because we’re just cooperating all the time on the equilibrium path. But if anyone defects, then we just defect from then on, and then we’re back at the origin, and there’s room for gains again. The reason why renegotiation proofness seems important is that presumably the ability to propose some kind of agreement, right, that, and actually kind of have it take is part of the strategy profile that we each have.

Otherwise, we wouldn’t have worked out the agreement in the first place and gotten to the frontier in the first place. But you kind of forgot to include that in the game. What that means is that the cost of defection, once you remember that those other strategies you know, are actually available, might be a lot lower than you thought. Because if someone just defects, if we’re playing a repeated prisoner’s dilemma, someone defects, we go back to the origin, then a second later, someone could just be like, “Ah, why are we doing this to ourselves? Let’s just get back to the frontier.” And then, you know, maybe defection wasn’t so unappealing after all. So I think it really is important to think through this kind of thing. And yeah, it’s just, it’s just an example of how may-maybe people who have been working on this for years and decades could, team up with people like the AI Futures Project and, improve it on that front. There are ways in which I think it does kind of embed some deep insights about the game theory.

So the point that if we’re all sharing our information, one actual worry with that, pointed out in this old paper, “Racing to the Precipice” by Carl Shulman and Nick Bostrom and Stuart Armstrong, is that, as I was, as we were saying before, you really want a big gap between leader and followers, that the leader can slow down for safety purposes. But if we’re all sharing our information and the coordination is fragile, then when it unravels, we’re as neck and neck as can be. Their proposal to mitigate that, of course, is for, you know, for us to destroy each other’s big data centers in Mongolia and Canada.

[00:41:33] Seth: Which I think is really clever, right? Anyway, keep going.

Phil: Yeah, well, I’m not sure how credible it is, but, -... You know, it might be credible, and if so, then it sort of largely, overcomes that particular issue. Another thing that I was kind of surprised didn’t come up is that, yeah, people who tend to think that, you know, whatever risks there might be from AI, you know, normal kind of market, the kind of default market outcome is that the risks get solved because, you know, safety itself is valuable and companies can compete on safety. Is that, is that... I took... From earlier conversations, I thought that was something like your view, Andrey.

Can Competition Deliver Safety? Profits, Moats, and a Spirited Disagreement [42:14 - 44:48]

[00:42:14] Andrey: Certain sorts of safety will be provided by competition, I think. But I do think that extraordinarily rare Events, and especially to the extent that they can be enacted, affected by smaller companies, that’s-- I’m, I’m, I’m especially worried there. And then, and then, like, the science fiction, you know, escape scenario, which I don’t put, like... I put-- just think it’s very unlikely. Very unlikely. But I don’t think those would be well mitigated necessarily by competition potentially.

Phil: Okay. Well, at least for the kinds that would be, mitigated by competition, one downside of just sort of eliminating the ability to make profit off of AI R&D is that you’re also eliminating the ability to make profit off of, you know, improvements in reliability and so on.

Seth: Yeah, but I would say, I, I-- actually, that’s not a concern for me about Plan A, right? Because to the extent that I do think these companies will make profits. We discussed how one thing that might make them more profitable is banning these training runs, because then they just get to make the operating profits rather than these giant investments, and then maybe they transfer more of their R&D resources into those kinds of safety innovations.

Phil: No, but how are they doing that if you can’t do any, like, big experiments and you’re just serving inference?

Seth: Well, I mean, the... Again, you wouldn’t be able to do it using training runs. That wouldn’t be the kind of safety, but it seems like there’s other dimensions in which you could be more or less safe.

Phil: Okay. I mean, I think experiments usually, at least today and for m-many years, have basically been partial training runs. And so if anything that looks to a little on-ship governance, thing like a training run has to be done in the air-gapped, you know, data center that we can all observe and learn from, then you really are weakening the incentives to do some kind of experiments that would be valuable for safety purposes.

Andrey: I just wanna say, ‘cause it triggered me a little bit, Seth, is, like, banning training runs will not increase profits. That’s ridiculous. Intelligence is the most competitive market in world history as far as I’ve seen it. Without a model advantage, it goes to marginal cost pricing.

Seth: No. Well, there’s two, there’s... No, there’s two huge moats. Moat number one is you need the giant amount of compute, which not everyone’s able to raise that much capital.

Andrey: No. You don’t need that much compute to run inference. So rents will go, well, rents will go to a very different set of companies, which is the compu- the companies who own the compute or it’ll be nationalized. But the OpenAI and Anthropics don’t own the compute.

State Risk vs. Transition Risk: Is It Safer to Go Faster? [44:48 - 46:29]

[00:44:48] Phil: One of those papers, one of those childlike papers I wrote with Leopold Aschenbrenner years ago- ... And-

Seth: Leo.

Andrey: Situational awareness. Someone had situational awareness.

Phil: Well, he had situational awareness. I just had an idea to sort of tweak a Chad Jones paper, and then he did it. But, you know, the punchline of that paper was that insofar as you think that a lot of risk is what we’d call state risk as opposed to transition risk, so risk that, like, an advanced AI exists and is out there being deployed, but, like, you know, every year it has some chance of going rogue or something.

Seth: Isn’t it safer to accelerate, Phil?

Phil: Then, th-then it’s, then it’s safer to go faster, right. You need to think that mo-most of the risk is, transition risk for it to be safer to go slower. I think this can totally change based on the circumstances, but, at the moment it does seem to me that the risk is mostly kind of in training rather than deployment, just like, you know, the hugging face incident didn’t happen because, you know, someone used it in an unintended way. It happened in training. And so this is-- I mean, this can totally change, and this is definitely something I... That makes me worry about, like, going all in a plan, on a plan like plan A without any mechanism for modifying it as things change. But hopefully, one benefit of having written that paper is that people can know that I’m not just, like, a knee-jerk person who sort of says-... Like, whenever something seems scary, like, “Ah, just shut it all down.” And yet even so, on balance, I think that it would probably, again, you know, we’ll get to posteriors, but yeah, seventy-five percent or something, it’d be good on balance at the moment.

Posteriors [46:29 - 53:03]

[00:46:29] Seth: All right, so let’s do it. Posterior is Do, do. Andrey, remind us what we were thinking about.

Andrey: Yeah. So will there be a plan like this by twenty-thirty?

Seth: I’ll go first. I started at thirty percent, and I think reading this essay actually moved me towards it being more feasible. I think the thing that I thought was really clever that sort of updated me a little bit was this idea of locating the data centers in a way that would be particularly vulnerable to the other side’s attack. That isn’t a thought, thing I had thought about. Maybe that doesn’t literally mean, you know, the American data centers are in Mongolia and vice versa, but I think that if you cleverly set up your data centers to be vul-vulnerable to a mutually assured destruction, that’s the kind of thing that would help this go through. Now, that’s, you know, just one clever idea. In order to get this all to go through, you would need lots of clever solutions to lots of these different issues. Andrey talked about a major one, which is how do you do monitoring in a, in a way that’s trustworthy and reliable?

But I moved from maybe thirty percent this happens to, like, thirty-five percent it happens, given that, okay, I can see maybe there’s some clever ideas for some of these.

Andrey: I think that this is a m-much less likely plan than a typical coordination plan that I would have considered ex ante. I found it very interesting, thought, thought-provoking, potentially with good pieces, but i-just even the idea of, like, putting the data center somewhere else, well, we have a ton of data centers here, and, like, the amount of things that need to go right for us to remove the data centers from the United States is just insane. So even just that sort of plan, you know, like-

Seth: The point is that in an AI twenty-forty universe, this is a universe in which it’s the new data centers that are the important ones. It’s not the incumbent ones, right?

Phil: So new data centers that matter partly because they’re being built so fast that the latest ones will be a large share of the total. They’ll also be the, kind of the best. ‘cause technology’s improving so fast. And also, the old ones depreciate relatively quickly.

Seth: So, okay, so Phil, give us your posterior. So did engaging with this essay move you from your original thought of the feasibility of a Plan A-like plan?

Phil: I’ll be honest, it is a little hard to separate the prior from the posterior.

Seth: It’s the alarm for a spicy take coming around.

Phil: I guess, like Seth, this idea of putting the data centers in, you know, easy-to-bomb places had not occurred to me before reading it, and I think that does make it a little bit more likely. There’s also just seeing how it’s been received, and I don’t know which way that pushes. I think it’s, that probably makes it a bit less likely. My impression is people have been somewhat dismissive it, of it on the whole. But I guess finally, there’s just the point that, you know, Dan Kokotajlo and these people with uncanny track records have endorsed it, and that’s gotta count for something. So, I don’t know, small updates all around, but, I’d, I’d say I’m, I’m about whatever I said, two-thirds sure.

[00:49:43] Andrey: The second one, is it good or, is Plan A specifically good or desirable?

Phil: Yeah, I guess, some of the economics and game theory seemed half-baked to me. I’m worried about, like, the Gell-Mann thing, where, I notice all of the holes in the areas I, I’ve, that I’m more familiar with, and that all the stuff on verification, I mean, I read it all, you know, and I have friends that work on this-... And, you know, they seem to think well of it, but maybe there’s all kinds of flaws that I don’t see. So yeah, I still have this strong kinda prior that, it’ll... There’s just, the stakes are really high, and there’s a lot of room for improvement, and we’ll probably figure something out. But something exactly like this, maybe, my posterior’s a little lower than I went in with, so, we’ll say seventy percent. Again, my lack of updates hopefully is not symptomatic of just being the kind of person who doesn’t update. It’s just hard to separate prior from posterior when it’s, you know-

Andrey: It’s hard, especially when you know, I read this when it first came out.

Seth: I, on the other hand, am able to fully constrain my consciousness to part, pretty good thought expire.

Andrey: Using new AI technologies, the same ones that allow you to read people’s minds.

Seth: Exactly. It’s funny, Phil, I kinda update in the opposite direction, and I have to say, on the second question, it’s almost a little bit more vibes-based. The future that this, and I think this is, like, the power of science fiction, is that this really was able to put me in the shoes of what it would feel like to be in a real RSI economic takeoff world. And in that world, I can imagine people doing some really dumb stuff just because things are changing so fast. You could get a lot of policies that are worse than this. And so, like, my conditional belief that this is a good idea, conditional on we get, you know, fifty percent economic growth in six, seven years, is, like, maybe, like, ninety-plus percent, right? Now, I put a very low probability on that world, but it’s getting me to think about what you would want things to look like if we were in hard takeoff, moves me into this is, this is a good idea given, you know, the one, two-percent chance we end up there.

So maybe I move from a sixty percent chance this is a good idea to, like, a sixty-two, sixty-three percent chance this is a good idea.

Andrey: Yeah, I mean, I think, like, conditional on us being in economic growth takeoff land, sure. Although, I mean, I can imagine lots of ideas. This one seems acceptable, not too harmful. But I don’t, I just don’t think that that’s the world we’re gonna be living in a few years, even as capabilities grow. And knowing how politics works, plans like this will be proposed regardless of the economic situation we’re in, and I will probably not support something like this.

[00:52:47] Seth: So you started at forty-nine percent. Are-- do you update down, or are you still at-

Andrey: Yeah, I’m, I’m, I’m updating down.

Seth: You have to give us a number. I’m very-

Andrey: So forty-five

Seth: ... The premise of the show. Forty-five.

Recursive Self-Improvement: Is It Imminent? [53:03 - 01:02:58]

[00:53:03] Andrey: Phil, I did wanna ask you some... ‘Cause we haven’t really done an RSI episode, and I know it probably deserves a separate episode. But I know, you were a part of the Elasticity Institute collective that wrote a paper about, recursive self-improvement or self-sustaining, acceleration, as you like to call it. This idea very related to just very fast, progress in AI capabilities and on the importance of measuring it. So do you wanna give a little pitch for that and how it relates perhaps to some of these scenarios?

Phil: For sure. Whereas Tom Cunningham took the lead on that first note, I’m actually in charge of a follow-up that we’re writing at the moment. Is RSI imminent? Evaluating kind of the latest arguments and evidence for thinking that, you know, we will get a significant increase in the pace of AI capabilities improvement due to the automation of AI R&D over the next year or something. I mean, my case for just, like, prioritizing this question is that it’s upstream of so much else, right? Including whether it’s, like, urgent for us to get a plan to regulate or internationally coordinate on AI in place soon. The real thing I wanna pitch is that it’s actually probable. At face value, once AI can do AI R&D more or less as a human researcher does, you would expect a significant acceleration in the pace of AI R&D. A lot of arguments have been raised over the years pushing back on that point, and I’ve contributed to many of those arguments, you know?

I mean, you can find me in the... Well, I’ve, I’ve, I’ve, I wrote one recently over the summer about, like, the ease of parallelizing research. And, Tom Davidson‘s been kind of investigating this question for years, and I think I’ve given feedback and in some cases pushback on almost everything that he’s written. And just, you know, going through the arguments and, the current state of the arguments and the evidence, I think just that original obvious point holds up and the burden of proof is very much on someone to find a bottleneck that has not been identified and more or less refuted. So, I mean, I can go through all a dozen, but-

Seth: Well, let’s do two.

Phil: All right. So one that I just mentioned is this point that when formalizing the case for thinking that things would go really fast once we’ve automated AI R&D, the yardstick of improvement that people use to actually you know, quantify, like, how fast things would go is algorithmic efficiency, which more or less corresponds to the question, once we can simulate, you know, Alec Radford and Noam Shazeer, you know, a thousand times over in the data center, how long before we can simulate two thousand of them, right? Or, like, not how long, but, like, the, you know, the doubling in algorithmic efficiency means that you can do the same thing with half as much, compute. Whether you actually get twice as much done or anything close to it with twice as many simulated researchers, twice as much sort of virtual research input, depends on a kind of another dimension of technology, which is how well they can coordinate with each other, right? How well they’ve turned themselves into kind of a productive big firm that’s twice as big.

If Hugging Face teaches us anything, it’s that a little RL can go a long way, in terms of, facilitating coordination. Yeah, I mean, just kind of from first principles, you might expect that there are a lot of channels through which AIs can coordinate that people can’t. And so the rate of improvement on that front might be expected to be faster than the rate at which, you know, people can organize themselves into ever larger firms. Yeah. Something that I had thought for a while was that you need a lot of data for AI R&D, not just in the sense that you need the data to learn how to do different kinds of things, but in the sense that you need different kinds of data to learn how to learn about different kinds of things. One way in which training can improve over time is by getting more sample efficient, right? Being able to make do with ever less data. And, like, humans famously are more sample efficient than LLMs at the moment.

Depends what you mean exactly, but basically that’s-- I think that’s true. But of course, the rejoinder to that would be, once we have automated AI researchers, they’ll figure out how to get really sample efficient, and so they’ll be able to learn from as little data as, you know, a human learns from. And when you remember all of the data that there is on the internet, you know, all the YouTube videos in the world with a few pixels, with some, you know, axle of a truck going around. You know what I mean? Like, there’s just so much information out there, and if it could learn really sample efficiently, then it’ll be able to learn how to do all kinds of things. The rejoinder would be, well, it might be that feeding in all of, all the data on the internet, we can get to the point of an AI researcher that can figure out really sample efficient algorithms for learning the kind of information that’s on the internet with little data.

But that doesn’t necessarily transfer to the project of learning how to, like, be an effective lawyer on some case, some particular case where, like, the you know, the client’s case is private. You’re deployed in the law firm, and you need to learn quickly from that limited data set. You might have just sort of overfit to the internet data. It’s, you know, and so, like, you, you’ve come up with a learning algorithm that is efficient in one context, but not necessarily efficient in every other context. And something Tom Davidson wrote recently has largely convinced me that, even though while you’re not sample efficient, there’s not gonna be all that much transfer learning. Learning algorithms tend to transfer well. So if you learn how to learn efficiently on the, on one data set, that learning algorithm deployed to another data set will also tend to squeeze a lot of juice out of it.

[00:58:55] Seth: Right. So if I get, if I get good at, you know, predicting Reddit text, if you just drop me into a simulated lawyer environment, I’ll figure out how to be a good lawyer much quicker just because I’ve learned how to learn on predicting Reddit?

Phil: Exactly. It, it’s not that you’ve learned how people talk on Reddit, right? I’m talking about if you automate the AI R&D process, that means that you, you’ve closed the loop, right, on how to learn to talk like a Redditor with ever less Reddit text. That procedure will also help you learn how to be a good lawyer with relatively little legal text.

Seth: I guess it strikes me that, like, some of these arguments, so there’s one thing about, like, doing things more efficiently, but I think the parts where things get wonky is when, like, there’s a qualitatively new capability that emerges, right, of some sort. I mean, I mean, in the sci-fi scenario, it’s this artificially super intelligent, you know, agent that seeks power or whatever. And it, like-- ‘Cause if all that RSI was doing was just making existing inference very cheap, existing training very cheap, it would be really important, but I don’t think it, like, poses the sort of, like, catastrophic risk that maybe people are worried about, if that makes sense.

Phil: Oh, for sure. I think the old arguments for thinking that, you know, intelligence is intrinsically dangerous and, you know, once you get the RSI loop, then, you know, the kind of, quote, “default,” whatever that means, outcome is that all hell breaks loose, were bad arguments. And it’s just a real shame because, you know, it, it’s so easy to fall into the, like, you know, the fallacy that, what’s it called, the opposite of Bad argument is not necessarily a good argument or something. You know what I mean? Like if people feel inoculated against the idea that this will actually happen, even if for somewhat different reasons. You know, I mean, it turns out, I think it’s safe to say now that, just pre-training on the internet does not produce sort of scheming behavior basically, but that, you know, RLVR, you gotta be really careful ‘cause it often can induce scheming behavior. And that’s something we just learned through trial and error.

Seth: It’s an empirical fact. It’s not you can sit down and come up with the orthogonality thesis and, figure it out in Yudkowsky’s bedroom.

Phil: Right. Right. But anyway, I mean, the facts are coming in, I would say.

Andrey: All right. So you write this paper, let’s say RSI starting to happen in November of this year based on your best estimates. I don’t know if that’s true or not. What do we make of that? I mean, we already see the capabilities on all the benchmarks going up. We see compute efficiency going up already. This is already happening. So what do you expect to happen once we hit the exact threshold for RSI?

Phil: I think that capabilities will advance even more quickly. The, nature of the capabilities, the kind of direction of the technical change, put it that way, will be a, a little different in an important way, right? So it’ll be that, capabilities are improving not just because of increases in the inputs, right, the compute and the data and so on, or the data from real world deployment or whatever, but also more of the improvements will come in the form of efficiency improvements, right? The ability to do more with less data and less compute. And there’ll be a kind of interesting qualitative shift, which is that even if we stop increasing the inputs, right? So e-even if somehow we made everyone at, you know, everyone working in the AI industry go home and, stop building new data centers or whatever, capabilities could continue to advance in a way that’s kind of harder to observe.

Lightning Round: Epoch, the Giving Pledge, and the Piketty/Dwarkesh Essay [01:02:58 - 01:08:27]

[01:02:58] Andrey: Why did you choose to go to Epoch AI? Why not stay at Stanford or do something else?

Phil: I mean, it really just seems like AI is going to be the most important topic in the world for the next, I mean, forever maybe, but, certainly for the next few years. And that the academic kind of timelines of trying to write papers that you publish and, the academic expectation of quality versus quantity, you know, the expectation of only writing a few things but having them be very polished, just aren’t very suitable for the moment. So the chance to work with people who are really up to speed on AI, who, you know, get a lot of short things out without agonizing over them for a decade, and, you know, working up in Berkeley at Constellation with other people who are writing things like AI twenty forty, I get to talk to them at lunch, all just seemed m-more appealing than staying down in Palo Alto.

Andrey: Why not join Anthropic?

Phil: Yeah. So, I was talking with someone about joining the Anthropic Institute at one point, and it seemed to me that environment of, like, learning from other people was probably actually better, if anything, at Constellation than at Anthropic, but at least not a huge benefit to being at Anthropic. M-maybe I would feel differently if I actually worked there, but that was the sense that I had. And, the cost, of course, is that no matter what you say, people will call you a shill if you, yeah, if you’ve worked for the labs or even the institute.

Andrey: Why do you care what people say?

Phil: Oh, well, because, you know, I’m not a shill, and I want people to update accurately on whatever I have to say.

Andrey: All right. Fair enough.

Seth: You have, you have a giving pledge, which, we’re very, in awe of your ethical standards in pledging to give away a large share of your income moving forward above a certain consumption level. If we got a twenty forty plan A world, and we really did have, you know, fifty percent economic growth a year, would you allow yourself more consumption?

Phil: I mean, I think it would depend on the marginal utility of consumption for me versus for, you know, what wealth could do elsewhere. If all the problems in the world were solved and, animal welfare was taken care of, even wild animal welfare was taken care of, then sure.

Seth: But so you-- we need to solve wild animal welfare before you’re gonna lower your cons-- you’re gonna raise your consumption.

Phil: Yeah. I mean, well, solve it or somehow make it the case that it’s, like, permanently, definitely impossible to do anything about it.

Seth: Okay. A good answer. Solid answer.

Andrey: Another lightning round question. How did you and Dwarkesh get connected?

Phil: Through a mutual friend named Jess Riedel.

Andrey: And did that naturally lead you to write a blog post together? What was, what was that like?

Phil: Yeah. So, after we got put in touch, Dwarkesh reached out a few times to chat about, you know, the economics of AI. And at one of those chats, he said, “Hey, you know, I’ve been thinking about that old, Piketty book. Seems like he was kinda out to lunch when it comes to the past, but would be right in the future once capital really did become a gross substitute for labor. Do you have any thoughts on that?” So that was Dwarkesh’s for-- That was, that was him. I mean, you know. I said, “Thoughts? I’ve got slides.” ‘Cause, you know, this summer course that I’ve organized sort of a few times now on, among other things, the economics of AI in the long term, made precisely that point. And I think Brad DeLong made the point way back when the book first came out. Someone did. But o-otherwise, I think I, I’m not aware of anyone else who had. So, that was a happy coincidence, and, he said he would write something and send it to me for feedback.

And I said, “Well, given that I already have some content, why don’t we, just, collaborate on it?”

[01:06:51] Andrey: And what was the most surprising response to this essay? It’s probably the thing you’ve written that has been read the most, right?

Phil: Yeah. I mean, To me, that was the most surprising thing in itself. It shouldn’t have been, of course, because Dwarkesh has a big audience, and everyone has to have an opinion about inequality, and I don’t know. It should have been totally foreseeable. Anyway, the most surprising thing, I mean, there were things I just learned because, you know, of how, the best way to learn is to write something wrong on the internet, and then someone posts the right thing on the internet. So, I guess I had sort of thought going in that Piketty was wrong empirically about whether capital and labor were gross substitutes in the past, but that if we granted that absurd premise, he would at least be right, you know, about the implications of it for, you know, the dynamics of wealth inequality given one tax policy or another or something. And I think, yeah, he just-- even given that premise, the best tax policy is probably just a consumption tax, unless you care very explicitly about wealth inequality per se, as opposed to, you know, inequality in, consumption or income.

Which you might, but I think he wasn’t quite as explicit as would have been ideal about what premises you need to get to his policy conclusion. And not to blame him for our mistake, but like, I mean, the essay’s edited now, but when it-- when we first wrote it just kind of deferred to Piketty I think a bit too much about what follows given the premise.

Seth: Phil, you gave us so much good stuff.

Phil: Yeah. Thank you for your time.

Seth: Thank you.

Outro [01:08:27 - 01:08:52]

[01:08:27] Andrey: Well, thanks for joining us for another episode of Justified Posteriors. As a reminder, we have written down all of our priors and posteriors on our website, justifiedposteriors.com, so please check it out. And if you liked the episode, please share it with your friends.

Seth: And keep your posteriors justified.

Links

Our guest

• David Deming’s website and his Substack, Forked Lightning

Social skills

• Deming, “The Growing Importance of Social Skills in the Labor Market”, QJE (2017) — the paper we open with: social skills as reduced coordination costs in team production, tested in the NLSY and in occupational data, 1980–2012

• Becker & Murphy, “The Division of Labor, Coordination Costs, and Knowledge”, QJE (1992) — why the economy isn’t one big team

• Deming, Løken, Willén & Xu, “Meetings”, NBER WP 35706 (2026) — the Norway paper: meetings take ~12% of work hours and ~14% of firm wage bills, and meeting intensity correlates with wage growth

• Shahidi, Rusak, Manning, Fradkin & Horton, “The Coasean Singularity? Demand, Supply, and Market Design with AI Agents”, NBER WP 34468 — Andrey’s paper, which (note the question mark) does not predict that it will actually happen
  Yudkowsky, “Coherent Extrapolated Volition” (2004) — the rationalist term Andrey reaches for

• Deming, “Efficient social learning is the human advantage over AI” — the “last human skill” argument: sample-efficient humans hold a comparative advantage in sparse, high-context tasks, and nothing is higher-context than social life

AI at Harvard (plus admissions and the Ivy-Plus premium)

• Chetty, Deming & Friedman, “Diversifying Society’s Leaders? The Determinants and Causal Effects of Admission to Highly Selective Private Colleges” (NBER version) — waitlist design: attending an Ivy-Plus college rather than a flagship public raises the chance of reaching the top 1% by about 50% and nearly triples the chance of working at a prestigious firm. The paper also identifies athletic recruitment as one of three drivers of the high-income admissions advantage, relevant to my editorializing at 30:22.

• Derek Thompson, “Why Harvard’s Dean ‘Encourages’ Students to Use AI” — David’s longer interview on the same themes

• Yondr — the phone pouches David bought as a voluntary commitment device

• Panos Ipeirotis, “Fighting Fire with Fire: Scalable Personalized Oral Exams with an ElevenLabs Voice AI Agent” — the NYU professor’s AI oral examiner Andrey mentions (36 oral exams for about $15 total)

Grade inflation

• “70% of Faculty Vote to Overhaul Harvard Grading With A Cap” — Crimson on the 458–201 vote capping A’s at 20% plus four, starting fall 2027

Lightning round

• A note on my character sheet. Sharp-eyed Storytellers will notice that the social attributes I read out at 52:41 (Charisma, Manipulation, Composure) come from no published edition. Classic World of Darkness has Charisma / Manipulation / Appearance. The 2004 Storytelling System behind Chronicles of Darkness renamed Charisma to Presence and swapped Appearance for Composure. This was, of course, entirely intentional. I kept the new edition’s Power / Finesse / Resistance structure and restored the classic Charisma (Presence sounds like a meditation app). Call it a house rule.

• Chatterji, Cunningham, Deming, Hitzig, Ong, Shan & Wadman, “How People Use ChatGPT”, NBER WP 34255 (2025) — the source of “broad but shallow”

• Garicano, “Hierarchies and the Organization of Knowledge in Production”, JPE (2000) — the knowledge-hierarchy framework David thinks AI will reshape

• Baker, Gibbons & Murphy, “Relational Contracts and the Theory of the Firm”, QJE (2002), and Gibbons & Henderson, “Relational Contracts and Organizational Capabilities”, Organization Science (2012) — “highly rated but still underrated”

• Deming, “Early Childhood Intervention and Life-Cycle Skill Development: Evidence from Head Start”, AEJ: Applied (2009) — the Larry Katz labor-class paper that became David’s first publication

  
  Connections to previous Justified Posteriors episodes:
  

• JP: Seb Krier on AGI, the Coasean Singularity, and EDM — the optimistic case for agent-mediated bargaining at scale

• JP: Noah Smith on Blogging, AI Economics, and Elite Overproduction — Coasean utopia vs. the collapse of all meaning (answer: why not both?)

• JP: Does AI Cheapen Talk? (Bo Cowgill Pt. 1) — makes the point that cheap communication so far means more bad communication, not more good

• JP: Alex Imas — Demand Collapse, Bargaining with Machines, and Behavioral AI Economics and Imas, “What will be scarce?” — the demand-side “relational sector” argument David contrasts with his supply-side story

• Hackenburg et al., “Scaling language model size yields diminishing returns for single-message political persuasion”, PNAS (2025) — the “AI persuasion maxes out” result I ask about, and our episode on it, Scaling Laws Meet Persuasion

Chapters

(01:25) Social skills: coordination costs, the Coasean singularity, and “the last human skill” (20:39) AI at Harvard, teaching and selecting for social skills and the Ivy-Plus lottery ticket (46:05) Fighting grade inflation (52:41) Lightning round: Social Skills as RPG stats, “How People Use ChatGPT,” whether economists need better social skills, and Larry Katz

Transcript

David: I’m making an argument this is like the last human skill.

Andrey: So should Harvard just stop teaching hard skills and go all in on social skills?

David: No.

Andrey: Damn it. I thought I got you. [laughs]

Seth: Nobody’s going to have to communicate anything. The AI will tell each other, and then the AI will just tell me what to do, and no one will ever have to go to a meeting.

David: My guess is it will decrease the quantity of communication but increase the quality of it. AI will not have the meeting that tells you what the company’s strategic direction will be over the next two years, because there’s no right answer to that.

Andrey: If we all do have a lot more free time, then the most important thing that we should be doing is arguing over what the big AI system should be doing for us.

David: We spend a lot more time firing takes at each other and arguing about things, in part because most work is not necessary for human survival. Most of us are entertainers. It would seem crazy to our ancestors that we have podcasts.

Seth: David, I could not disagree more. Welcome to “Justified Posteriors.” Today, we’re joined by David Deming, top labor economist and, since 2025, dean of Harvard College, to talk about social skills in the age of AI and how Harvard is changing to meet the challenge. I’m Seth Benzell, always putting my social skills to the test here at Chapman University in sunny Southern California.

Andrey: I’m Andrey Fradkin, coming to you from San Francisco, California, and we’re very excited to have you on the show, David.

David: Thanks for having me, guys. I’m looking forward to it.

(01:25) Social skills: coordination costs, the Coasean singularity, and “the last human skill”

Seth: All right, so I guess the first thing we really want to talk to you about is you have so much really interesting, influential work about how social skills work in the labor market. And so we want to dive into how you think that that’s going to change in the age of AI, how Harvard can be part of that, maybe teach those social skills or select for them. But maybe a good place to start is you have this very influential paper, “The Growing Importance of Social Skills in the Labor Market.” Tell me what that found about change in demand for social skills from the 1980s to 2012.

David: Yeah, sure. So as you mentioned, the paper is called “The Growing Importance of Social Skills in the Labor Market,” and it’s really two things. The paper is an economic model of why social skills would actually be valuable in production and not just like, “Oh, I’m good at cocktail party conversation or I’m good at building social networks.” [laughs] It would actually be productive in the sense that it increases a worker’s marginal product of labor.

Seth: Is the idea that networking would be zero sum?

David: Yeah, potentially. Exactly. So if social skills are networking, it’s just bumping you in line in front of somebody else, where this is actually making you more productive in a competitive economy, right? And that’s not to be taken for granted. And so the model in the paper, the mechanism is, look, production requires specialization. So this is an old idea. It goes back to Becker and Murphy in some sense of-

Seth: It goes back to Smith.

David: Well, okay. Yes, it goes back to Smith originally, but the idea of coordination costs and the idea that basically... So let me just jump back. So Smith says, “Specialization, division of labor is important.” Becker and Murphy, 1992, says, “Look, if specialization makes the economy more productive, why isn’t the economy one big team?” Right? And they notice that if occupations and what people do on the job is actually not only limited by the extents of the market, which was Smith’s argument, it’s limited by coordination costs. And they say, “Look, the economy’s not one big team because it’s actually costly to get together, have a meeting, figure out who’s going to do what.” And the benefits of specialization are increasing and the costs are increasing as it gets more specialized, and there’s some interior condition. And then they explain why, for example, if you live in a small town, you have a family doctor that does everything, but if you live in a bigger town, you’ve got more specialization of doctors, radiologists, and cardiologists, and interventional cardiologists, and whatever, people who put stents in, whatever.

Seth: Passive cardiologists. [laughs]

David: Exactly. So that’s their argument. And so I just take that one step further and apply it to the concept of an individual’s skills. So the idea here is, if you and I are going to work together in a team, maybe we’re going to write a paper together, Seth or Andrey, like we’re going to do something together. And you’ve got some skills, I’ve got some skills, and we’ll arrive at some division of labor. Maybe you’re going to do the data analysis and I’m going to do the writing and whatever. [laughs] And we have some comparative advantage, so if I do something and you do the other thing, we’ll get more out of it. In the same way that two countries trading goods, the idea of Ricardian comparative advantage holds, is like two workers or three workers in our case, trading tasks. So we figure out who’s going to do what. We can do more when we specialize, but there’s some cost. Like maybe it’s a meeting that we have to get together and coordinate who’s going to do what. And so the question is, when do the benefits exceed the costs? And my idea was that social skills are like inverse coordination costs. So the better you are at working with others, the lower cost you pay to realize the benefits of specialization.

Seth: Do you think about that as like a taste? Like some people have a negative friction from interaction? Or is this more like a technology of communication? What do you think about what that penalty is?

David: I think of it as a skill, not a taste. And I think about it like the way I wrote it down in the model was like, if you’re familiar with trade models, there are these what they call iceberg trade costs. So it’s like literally the idea is if you ship a product from one country to the other, some share of it melts along the way. And so-

Seth: The Houthis blow it up.

David: Yeah, exactly. That could happen too. But this explains, these models of iceberg trade costs explain why the US mostly trades with Canada and Mexico, even though China’s a much bigger market. You just got to send things farther away. And so social skills are like inverse trade costs in the sense that we lose less output along the way if we have high social skills, and therefore we can reach finer grain degrees of specialization when we work together. And so I wrote down a model of this, generated a bunch of predictions, including the prediction that social skills will be valuable, but also that they’re more valuable when you have more to trade. So there’s kind of a complementarity between your expertise and your social skills. And so that’s the model side of it. And then I kind of tested it in some microdata using the NLSY, and then also used a bunch of broad labor market data to show that from 1980 to 2012, jobs that kind of require a lot of teamwork had grown faster and paid relatively higher wages over that period of time. And so that’s the paper. So it’s kind of a combination of a model and then a little bit of a theory and then some data that’s tested on it.

Andrey: What are these jobs that require a lot of teamwork?

David: Yeah. Well, I mean, you think about anything in the modern office. I just released a paper about meetings last week. We showed, at least in Norway, the average worker spends an hour a day in the meetings, and that includes workers who don’t have any meetings at all. So, like, if you’re a construction worker or a retail sales clerk or something, you don’t meet that often. People who work in office jobs tend to meet an average of two hours a day. So, just thinking about meetings aren’t the only way that social skills matter, but the ability to run a good meeting and to learn what you need to learn from that. We all have been in meetings that aren’t run well, and we know how painful that is. Maybe we always think of meetings as a cost, so we don’t appreciate the ones that are run well. But the way I like to think about it and what the model actually implies, which I love, is that higher social skills means you need to spend less time in meetings to get the same output. So it’s like the meeting is the cost you pay to achieve the gains from task trade.

Seth: Right, which would be different than kind of like a taste model where it’s like, “Oh, I’m just loving this meeting. Let’s do more meetings.”

David: Exactly. It’s actually, you can hate the meetings but also get value out of them [laughs] because they are the cost you’re paying. So, like, you’d like to minimize that cost. So to me, it’s a totally consistent worldview. It’s like meetings are necessary, but I don’t want to spend any more time in them than I have to to get done what I need to get done. I think that’s the way most people feel about them.

Andrey: I’m surprised that construction is one where you think it doesn’t require a lot of teamwork, because if one would be-

David: Oh, I just made that up. [laughs]

Andrey: You know? [laughs] Well, I mean, I imagine there’s a lot of coordination, and even if there aren’t formal meetings, people are working side by side, and so they’re meeting all the time, if that makes sense.

David: Yeah, no, I think that’s right. I mean, look, my point was just that the average of an hour a day is an average over all workers, and some people say they don’t really have regular meetings. For people in office jobs or jobs like ours, we probably spend a lot more time than an hour per day on meetings.

Seth: I guess Andrey’s raising a point here, which is like formal meetings versus sort of informal collaboration, right?

David: That’s right. Yeah, yeah, yeah. And so we asked some questions about that in the survey, but it’s a little hard to get at. You’re basically saying, what is the definition of a meeting? [laughs] And I don’t know. I mean, I guess anytime you’re spending, I would say anytime you spend coordinating. So, like, if you’re a software company, you’re trying to ship some code or ship a product, or if you’re a consulting company, you’re trying to land a client, and you’re spending a bunch of time meeting, figuring out who’s going to do what, who’s going to pitch, who’s going to write the deck. Like, that’s not really production. That’s coordination. And so anything that fits that bill would be a meeting in my book.

Andrey: A natural question here is, I’m hearing that we don’t need meetings anymore because the AIs will meet for us.

Seth: How will this change in the age of AI? Andrey, use the word. Use your term. [laughs] Use the magic words.

Andrey: [laughs] The Coasean singularity.

Seth: We’re going to have the Coasean singularity, David. Nobody’s going to have to communicate anything. The AI will tell each other, and then the AI will just tell me what to do, and no one will ever have to go to a meeting.

David: That might be true, but it also feels like kind of a partial equilibrium conclusion, doesn’t it? I mean-

Seth: Well, coordination costs go to zero. You tell me why that’s not general equilibrium.

David: Well, okay, so here’s a story I would have in mind. Again, I don’t know. Nobody knows the answer because this story’s being written as we speak. But here’s how I would think about how AI changes communication in the firm. So my guess is it will decrease the quantity of communication but increase the quality of it. And you see this in a lot of companies that are very AI forward. They basically have AI hooked up to Slack, which is essentially a within-firm communication technology. It’s like what’s going on in Slack. And when there’s an agent there that you can talk to, you can ask it the questions you would normally ask people, like, “Where’s the bathroom?” Or like, “Tell me where in the employee manual it talks about my health benefits.” Or, “I heard Seth talk about Project Glasswing in the bathroom. What’s he talking about? Who’s the lead on that project, and when is it shipping?” You know, like, you can ask factual questions to the AI.

Seth: If I’m talking about Project Glasswing in the bathroom, I’m not talking about a product that’s shipping. [laughs]

David: Okay, fine. [laughs] Fine. Fair enough. Fair enough. But there’s going to be... Look, another way to say it is some of what we communicate about is like there is some answer we’re trying to get to, and everyone knows what it is, and we just have to find the way. But a lot of what firms do and what societies do is communicate over spaces that are contested. So, like, your AI will not have the meeting that tells you what the company’s direction, strategic direction will be over the next two years, because there’s no right answer to that. It’s like something that has to be determined politically in a sense, like talking with people in the firm. And so that’s the sense in which I don’t see the Coasean singularity as an equilibrium, because the more we lower the cost of routine communication, like just figuring out who’s going to do what, the more space we will open up for communication about things for which there isn’t a right answer.

Seth: Well, I can think of some counterarguments. Andrey, may I go? [laughs]

Andrey: Yeah, go for it.

Seth: All right. Well, thing number one is when people talk about this Coasean singularity idea, they often have in mind political outcomes. They often think about, “Well, I have this long list of preferences about what I would like my hometown to be like. I don’t have the time to research what candidates or which arguments are going to get me there, so I just tell my chatbot agent, ‘Here’s my list of preferences,’ and then my chatbot agent goes to the Agora and then advocates for my angle.” Right? So it’s not clear to me that the fact that it’s political is going to be a barrier to this automation. And then I guess the second idea is just that, I mean, at least in the short term, it seems like we’re getting a lot, in some ways, lower quality communication. It’s so much cheaper to just produce communication that we get more bad versions of it rather than more good versions of it.

David: Which also doesn’t feel like a Coasean singularity. [laughs]

Seth: Well, that is counter. [laughs]

Andrey: To be clear, no one has predicted that it will actually happen. [laughs]

Seth: Andrey, what’s your prior?

Andrey: Well, very low. Less than 10%.

Seth: Very low. Aw, come on.

Andrey: Yeah.

Seth: I’m above that. [laughs]

David: I don’t really have a lot of certainty about how this will play out, so I just think it’s not obvious that it will lead to that. And again, like you mentioned, Seth, like let’s say you’ve got your AI doing this and going to the Agora. That all presupposes that you have your argument set and you know what the answer is, and you don’t want to listen to what other people have to say. Like-

Seth: Yes.

David: … part of political engagement means like... And maybe that’s true. Maybe that’s the way society will work. But I actually think you could imagine within a firm- Or some other organization spending more time debating.

Seth: Your AI will pay my AI and will say, “Seth says, I’m willing to accept a 100 dollars an hour of payment to hear your arguments, and there will be a price for communicating to me through my AI.”

Andrey: No, I mean, I do think there’s a vision of this. So I do think that, again, coherent extrapolated volition is a term that gets thrown around in the rationality sphere, but it will know how you would update based on new information. It can do that for you to some extent. You can imagine that. But I do think another version of the future is, if we all do have a lot more free time, then the most important thing that we should be doing is arguing over what the big AI system should be doing for us. And there’s probably going to be just the humanness of we’re going to value that humans are doing the deliberation and not the AIs. So it goes to kind of the relational sector argument.

Seth: Right. This is more like the demand for communication going up. This is the taste argument.

David: That is the taste argument. So I think that argument also holds. I don’t think it’s invalid, but you asked earlier about my paper. Mine was more of a supply side story, whereas I think there’s also a demand side story for relational work, and Alex Imas and others have talked about this, and I think that’s a very valid point of view too. I actually think one way to say it is, let’s look at the trend before AI and extrapolate it outward. We spend a lot more time as a society firing takes at each other and arguing about things, in part because most work is not necessary for human survival. If you think about how many hours do you need to work-

Seth: People would die without podcasts, David. [laughs] Sorry.

David: Well, I mean, it would seem crazy to our ancestors that we have podcasts, because it’d be like these guys are living a life of leisure, or like people make money off of it. That’s sort of my point.

Seth: What was Homer? Homer, “The Iliad?” That’s a long podcast series.

David: [laughs] That’s like one guy. That’s like one guy. Most people were not firing off takes. [laughs] You know what I mean? Or maybe it wasn’t one person, actually. It was a collection of people. But the point is, [laughs] the production of knowledge was much more elite than it is now. And I think it’s in part because what share of the jobs that we do and the work we engage in are actually necessary for the survival of the human race? It’s a vanishingly small share. And so in some ways, when people say, “Oh, AI is going to lead to the end of the work,” to me, I’m like, “It’s already here.” Most of us are entertainers.

Andrey: So, all right, great. So how’s Harvard? [laughs] How does Harvard feel about that take?

Seth: How does Harvard entertain us?

David: That’s a question for you to answer, not for me, how it entertains you.

Seth: [laughs] Well, before we get totally into Harvard, here’s the last question about extrapolating forward your results from a growing importance of social skills, which is you find that there’s an increasing sort of complementarity between having these hard math skills and the social skills. And again, your argument is that you’ve got this more thing to bargain, so you’re going to bargain it more. Do you see the age of AI allowing... Do you see that trend continuing, or do you see people with strong math skills being able to just use the AI if they’re weak on social skills to bring up the bottom, right?

David: So, in the paper, I think the empirical implementation was really about math skills or cognitive skills, but I actually think the lesson’s more general. It’s like you need to be good at something to be worth trading with in these models. But that doesn’t mean smart at math.

Seth: Oh, wait, wait, wait, wait. I thought comparative advantage means you don’t have to be good at anything. [laughs]

David: No, no, no. Comparative advantage means you don’t have to be good at everything. You have to be good at something to have something of value to trade.

Seth: Really?

David: Well, okay, so let’s be precise here. So, there’s a question of whether you can be employable, okay? Which is like, that’s your argument.

Seth: You can be a zero marginal product worker, right?

David: Yeah, but you’re asking about complementarity. So you’re asking about, in other words, the value of social skills is increasing in the value of other skills you bring to the table. So that’s what I mean to say. And so I think that generalizes beyond math skills. You think about your own... I mean, I’ll speak for myself. So most of the benefit, most of the good things that have happened to me in my professional life are downstream of becoming an expert in something that people care about. So I spent, I don’t know, 10 years toiling in obscurity, learning about human capital and other things, and you wake up and you’re an expert in some field in economics or something else that people actually want to read about. And then if you’re able to communicate about it, it generates much greater returns than if you’re not. But if you have nothing interesting to say, having great social skills is not very valuable. So that’s the sense in which complementarity is a much broader concept for these purposes than just math skills. Does that make sense?

Seth: Yes, makes perfect sense, right? So social skills is the way that you trade the other thing you have. It’s not good-

David: Yes

Seth: … in and of itself, unless we were going to move to a taste-based model in which, “Oh, I just love hanging out with that fun guy.” [laughs]

David: It could also be good in that sense, but that’s not the mechanism that I had in mind in my work. Yeah. Well, we can say more about social skills. So I do think you asked me the question about AI in the future of-

Seth: Yeah, yeah.

David: And so I think, again, on the supply side, so not relational work, but another way to say it is what is the comparative advantage of people over AI agents? And I argued recently that I think, and I believe this to be true still, that social learning and learning about how people interact together is actually likely to be the comparative advantage for people in a world of super intelligent AI. And I don’t think that’s because the AI is incapable of social reasoning. I think it’s because AI LLMs, their process of acquiring knowledge is just very, very different than how people learn about social networks. You know, because the social network is impossibly complex. So it’s like there’s millions and millions of possible states of a relatively small network, like who likes whom, who’s superior to whom, who’s... And so there’s just a lot of complexity, and humans don’t map the entire network and then make inferences about it. We basically build causal models in our heads of how people interact, and then we make predictions out of sample and extrapolate like, “Okay, Seth seems cool. He’s likely to get along with his other friend I have,” or whatever it is. And so when people reason, they don’t reason in a way that’s at all consistent with how LLMs do.

Seth: David, I could not disagree more. To me, it seems like having social skills- In the sense of being able to predict how complex social interactions will proceed is similar to the skill of writing a novel about a complex social interaction. Do you think that’s correct, or do you think that’s incorrect, and then that drives why you think LLMs will not have a comparative advantage in that?

David: Let me say what I think LLMs do and what I think people do and why I think they’re different. Okay? And then you can tell me how that fits into the novel framework.

Seth: Please.

David: So I think basically LLMs learn through sophisticated pattern recognition applied to massive amounts of data. So, like, you need to give the LLM... Like, LLMs are way less sample efficient learners than people are. Okay? But they make up for that in scales. Like, once an LLM learns, it can replicate easily and it can generalize somewhat, but it takes forever. Like, that’s why you have to feed it massive amounts of training data to get the same basic learning that people get much more naturally. But the learning is much more complete. So people don’t do that. Like, people basically learn by building, let’s say, game theoretic models of each other and reasoning backward. Like, the reason Seth behaves in this way, or the reason he made this face is because this is what he thinks about what I’m saying, and I know something about him, and so I can reason about that. So that’s all, like, guessing. It’s not building a complete model of you. And then we reason backward from that using incomplete information with pretty sparse data. It’s like we don’t actually know, and we’re conditioned through evolution to make inferences with very little data. And so I think the approach is very different. And so that’s the sense of the argument.

Seth: What do you make of these papers arguing that AIs are getting better and better at persuasion, but they kind of like max out? Do you think this is sort of representative, or do you predict AIs will sort of max out in their ability to persuade? Or do you think persuasion is something that AIs will continue to hill climb on?

David: Well, I think if it’s like written persuasion, like if you write an argument and I have to read it and decide whether I agree with it, they’ll probably get quite good at that.

Seth: And that’s a pretty small part of social skills is what you would say.

David: I would say, yeah. And again, I’m making an argument about comparative advantage. So I’m not saying that, like, people will always be better than LLMs at this. I’m saying that in a comparative advantage sense, this is relatively hard for at least the modern machine learning paradigm to do, and it’s relatively easy for people to do. So in a comparative advantage sense, I think, again, I’m not making a levels prediction about when AI will be better than people at social skills. Maybe it’ll happen someday. I’m making that argument this is like the last human skill.

(20:39) AI at Harvard, teaching and selecting for social skills, and the Ivy-Plus lottery ticket

Andrey: [laughs] That’s a hot take. Well, given it’s the last human skill, let us transition to a question of your institution.

David: Okay.

Andrey: So should Harvard just stop teaching hard skills and go all in on social skills?

David: No. [laughs]

Andrey: Damn it. I thought I got you. [laughs] So yeah. So how do you think about what Harvard should be teaching?

David: Yeah. So for exactly this complementarity story.

Andrey: Okay.

David: I think that’s the direct argument. So I think it’s you want to build those skills too, but you also want to develop people’s capacities to become an expert in something. And that requires often some technical knowledge. It requires some deep historical reading. It requires, I don’t know, like it depends on the nature of the expertise, but it requires careful study. I think that’s still really important. I just think of it as necessary but not sufficient for success in an AI world. So I think we need to be doing both.

Andrey: I kind of know how Harvard tries to teach non-social skills, but how do you think about Harvard teaching social skills? Is that something that should be more of an overt goal? Does it happen just because it does? Yeah.

David: Yeah, I think the hard thing about that is it’s like social skills, teamwork, leadership, management, it’s like those aren’t really content. Those are pedagogy. So it’s like you can’t say, “Fifth period, it’s social skills time,” and just like start reading out of a textbook. [laughs] It’s more about the way you educate. And so what I think we need to be doing is structuring learning around feedback and reflection much more than we do. So I think one of the reasons why not just Harvard students, but students at a lot of colleges engage more in extracurricular activities than they used to, and more in like work-based learning and project-based learning is because it’s more immersive and because it provides more feedback. Like, real-world feedback is the best way to learn. I think if all you do is turn in a couple of assignments and you get some cursory comments on your paper, you’re not actually getting enough feedback to learn. And so I’m not saying that it’s happening in all classrooms, but I think the general challenge is like how to bring more of that immersive, interactive learning into an academic setting. So I don’t know. It’s not going to be one answer for everybody, but I do think we have to reevaluate that. We have to ask, like, are we giving students enough feedback? Are we giving them enough opportunities to practice what they’ve learned and to build expertise through engagement with each other and with professors?

Andrey: Given that that’s an important part of the coursework, does that argue for changing the admission criteria in some way as well?

David: What do you have in mind?

Andrey: You know, this is a straw man, but you could imagine selecting for people who are the homecoming king and queen of a high school because they have such great social skills and then trying to teach them the harder skills, and then maybe they’ll be really productive, great human beings. I don’t know.

David: Well, I think it’s important to say, at least the way I think about social skills is it’s not really like who wins the popularity contest. So I think of that as conceptually different.

Seth: What do you call that? What’s the term for that? Is that charisma? That’s not a social skill?

David: Yeah, could be charisma. Yeah, I mean, it’s related. I mean, these things are not... Like, the correlation between these things is not zero, you know? But when I think about what it means to have good social skills, I don’t think about the person who is most popular or who makes great cocktail party conversation or who... That’s not what I have in mind. I have in mind the ability to work well with others. And that is a skill that requires self-knowledge, but also knowledge of others. So, like, if the three of us were going to write a paper together, we would arrive at a division of labor that is different than if I were to write a paper with one of my students or one of my advisors or what. So, like, there’s no one answer to it. It’s about adaptation to others. And flexibility. If I thought there were a really good way to measure not even just the skill, but like whether people are receptive to it and whether it can be built in some people more than others, I would absolutely want to try to find a way to select on that. I don’t think we’re there yet. So, like, I wouldn’t want to noise trade on things that we feel like are social skills but really aren’t.

Seth: Do you think Harvard should only admit on SAT scores?

David: No.

Seth: And the reason for that has something to do with social skills, right?

David: No.

Seth: Okay.

David: No, I don’t. And I think it’s broader than that. I think what not just Harvard, but any university is trying to do is admit a class of people who are the future leaders and contributors to society in a variety of ways. So I don’t think Harvard’s model should be or is like, we admit only people who score high on certain tests because that’s the only thing that matters. I think it’s more like we want people who make contributions to society, and those contributions take many forms. I think academic excellence is important, so I think it should be a very important part of our admissions, but I don’t think it should be the only part. And part of that too is honestly, like, Harvard is fortunate to be in the position of basically being able to admit, we could admit a class full of people who have close to perfect academics. And so you actually, in some cases, don’t need to choose. You can get people who are great academically but also have other skills, and many of our students fall in that category.

Andrey: I guess, like, one of the comments that you hear is that part of the problem is that SAT is just not hard enough. Like, it’s too top censored, right? Like, you could imagine a test that’s much more comprehensive, like they have in India, where you could really spread out the top talent. You’re curious what you think about that proposal.

David: Well, that’s another reason to not, I mean, Seth asked, should you admit just on SAT scores? That’s another reason not to do that because it’s top-coded. And that’s why we don’t do that, or one of the reasons why we don’t. I think there’s a lot of other ways to detect academic excellence beyond that. It’s not just tests. It’s also like what were your courses, what other enrichment oppor- you know, like in the case of math, like there’s well-known indicators of extreme math potential. And we have faculty read applications who are experts in fields, and so that’s part of the admissions process too, is like people who say, “Oh, this person’s going to win a Fields Medal, we think.” And like, that matters too, and that’s way above the scale of the SAT. And so, that’s a part of admissions at not just at Harvard, but many other schools, and I think it should be.

Seth: You made a really interesting comment, which is this idea of thinking about correlations between different social skills and maybe hard academic skills. And one thing I noticed in that your growing importance to social skills paper, I don’t have the number in front of me, but I believe you found approximately a 25% correlation between AFQT score, which is kind of a general IQ test, and social skills. If we’re moving to a world where social skills are more important, do you think that that kind of reorders the winners and losers in society? Or are those correlated enough that the people who would have been the winners in one society are going to be the winners in the other society?

David: Yeah, that’s a good question. It may reorder them. Not necessarily on raw skills, but I think the kinds of jobs. So I do think the winners and losers in the economy, not just in schools necessarily, will tilt more towards people who, I wouldn’t say just like have good social skills, but people who know how to sell and persuade and negotiate and navigate conflict. These are, I think, increasingly important skills that are kind of subsets of what you might call social skills. And I think they can be developed. So I think people are like, I think to me, like probably social skills are like any other thing, which is like some people are born with different endowments, but you can work on it. For me, like having children, especially daughters, has definitely built my social skills, no question about it. Having a job like this where I have to talk to people all day has definitely built my social skills. So I think you can work on those things, and I think you’ll see more people spend more time working on these things because it’ll be rewarded.

Seth: Well, tell me what that looks like in Harvard. Tell me what that looks like outside Harvard. Is this people going to Toastmasters? What is that?

David: Well, I would guess, this is a guess, I don’t have data on it. If you were to scan the syllabi of courses in any college and measure the change in assignments over the past 10 years, I would bet anything there’s more oral presentations, there’s more project-based assignments, and there’s fewer blue book exams or pen and paper tests, and probably even fewer papers. And so that I think is a recognition of this. And I also think a lot of the extracurricular activities that students engage in, certainly at Harvard, are like, they’re very interactive. So there’s a consulting club that basically has real clients that they make pitches to and write things for and they try to generate a book of business. And so, like, that’s something that they’re not really getting in the classroom, or at least not as much. So I think you see, when you start looking for it, you see all kinds of signs of the demand for this and people finding ways to satisfy it. So I think that’ll probably continue.

Andrey: All right. So Harvard has over 40 varsity sports. A large share of the student body in the undergraduate plays these sports. But at the same time, America’s reliance on college athletics or the prevalence of it has been criticized. What is the optimal amount of college athletics that you think colleges should have?

David: I don’t know. I think athletics are an important part of what we do. I can’t speak to other universities, but here at Harvard, I’m pretty proud of the fact that all of our athletes live in the same dorms with students and have the same college experiences. So I think we’re really about building student athletes. Our athletes are pretty high achievers. Not all of them, but most of them are. And they meet the Harvard admission standards. And so I think of it more as they’re great athletes and great students. And I do think athletics builds social skills, discipline, a sense of sacrifice. Like, there’s good things that come out of athletics. Whether that means we should have more or less on the margin, I’m not really sure. I don’t know. Like, I see both sides of it. Like, I see schools that have less athletics than Harvard but are also academically competitive, and that means they can, I don’t know, they have more degrees of freedom. But I also think our athletes do actually do exceptionally well on the job market, and some of our teams are pretty good, and that brings a lot of good things to the university as well, so.

Andrey: I’ll just editorialize a little bit because I assume you’re not at liberty to say. I think- Plenty of sports, not all sports, are a way for colleges to admit the children of very wealthy parents. That might even be optimal for a university, even though it is not fair. Okay. Editorialization complete. [laughs]

David: Thank you for registering your opinion.

Andrey: Now, you did not go to Harvard for your undergrad.

David: I did not. The Ohio State University.

Andrey: First of all, just a biographical question. Why did you end up going to OSU?

David: Well, I went to Ohio State. I moved to Cleveland with my family when I was 15 from Nashville, Tennessee. So, I attended Shaker Heights High School for three years, which is a public school on the east side of Cleveland. Actually has a storied history. It’s a racially integrated school, and through some deliberate gerrymandering of school districts in a way to preserve racial balance in the school that’s very interesting, it’s been written about. So, I went to Shaker, and then I went to Ohio State on an academic scholarship. They were running a program at that time. Basically, if you were a National Merit finalist, you could go there for free and they would pay you, I think, 3,000 dollars a year or something, which felt like a lot at the time. And so, I went there to save my family money, and also I went to the honors college there and had a great experience. And I got into some other schools. I think I got into Vanderbilt and Emory. I got some money to go to those schools, some financial aid, but it felt important to me to save my parents some dough, and I also thought I could get a good education, and I did.

Andrey: Awesome. So, our question to you is, how do you think about the Harvard College premium for undergrad? I know you’ve done some work on this. Maybe you can recap some of that work, and then we can talk about why do you think there is a premium? Where does it come from? Yeah.

David: Sure. Yeah, so I think you’re referring, Andrey, to a paper that I wrote with Raj Chetty and John Friedman about... The paper had two parts. Really was one is like who gets in and why, so the determinants of admissions at elite colleges. And the second one was like, what are the causal impacts of attending one of these schools? And in that second part, which is what you’re referring to, we found is that basically there was a small impact on mean earnings. It was not tiny, but it was not earth-shattering. So if you go to one of these schools... And just to say, our identification strategy, which was not perfect but pretty good, was like we looked at people who were on the waitlist. So we had data from several Ivy Plus colleges that declined to be identified. We first sort of used the waitlist to verify that just because you’re on the waitlist at one doesn’t mean you’re on another. We kind of looked at people who were kind of right on the margin of admissions, controlled for a lot of stuff, and showed what if you barely got off the waitlist and compared waitlist admits to waitlist rejects. So, what we found was that being admitted to a school off the waitlist, to an Ivy Plus school, relative to a public flagship like an Ohio State, some place that I went, led to a small increase in mean earnings, but a very large increase in the probability of being in the top 1% in your cohort. So, impacts on the top tail that were pretty large. It’s been a while, but I think it was like a 50% increase in the probability of being in the top tail of earnings, top 1%. And there’s kind of a fractal quality to it, so we don’t have enough statistical power to go all the way up, but when you just look at the distribution, it looks like the increases are all driven at the top. And if anything, there was a small increase in the probability of having zero earnings. So, that’s people who are in grad school or doing something else. So it was more of a fanning out, but an overall mean increase, but that’s driven by the top tail. So, that was what we found in the paper, and we think it’s very much connected to the advantage that people have in landing at what we would call an elite or a prestigious firm. And that has a loose definition, but you should think about it as like, if you go into finance and you go to Ohio State, you get a job at State Street Bank or KeyBank or something. And if you go into finance at Harvard, you go to Goldman Sachs. So they’re these big firms with national reputations where it’s very hard to get a job and they only recruit at a few schools. But it’s far from a guarantee. So I like to think about it like, if you get admitted to an Ivy Plus school, you get a lottery ticket and you have maybe a one in 10 chance of cashing it. But if you went to Ohio State, you don’t get any lottery tickets. So it’s not a guarantee, it’s just you get more tickets. And if you want to win the tournament in a high-status profession, not just finance, but technology or consulting or journalism, if you want to get a job at the New York Times rather than the Columbus Dispatch, it’s also a gateway to that. So, our story is that basically these colleges allow you to buy a lottery ticket or two in a tournament that few people win, but the pipeline, the opportunity set for winning that tournament greatly increases if you go to one of these schools.

Andrey: And is that just because of access or is that because of human capital formation or... Yeah, do you want to speculate on the mechanism there?

David: I could speculate, but just to be clear, our identification strategy doesn’t allow us to answer that question really. My own sense is it very much depends on how broadly you’re willing to define human capital. Okay? So, there’s a narrow sense in which human capital is just years of schooling, [laughs] or quality-adjusted years of schooling. That I think is probably not the mechanism. But I think it could be that you get an early start on learning a very specialized set of skills. And I’ll just give a concrete example. Again, I mentioned the consulting club. So these are clubs that where you actually learn how to be a consultant [laughs] while you’re in school. And so that’s human capital. It’s quite narrow human capital. But then also, so it’s kind of endogenous. The fact that these firms, like McKinsey hires a ton of people from Harvard, is endogenous to this. So it’s like you as a student, you know what they want, they know what you want, they get to know you because they think you’re the most talented. So it’s like there’s some soup of human capital development and networking and signaling that all goes together that I think is pretty hard to unpack. So that’s my own sense. What’s your sense?

Andrey: My sense? [laughs]

David: Yeah, I mean, well, the interpretation, you have just as much a claim on that as I do. I wrote the paper and I did the analysis, but that doesn’t mean I know what the mechanism is.

Andrey: I think in some places like grads, to the extent that Harvard undergrads end up accomplishing a lot in academic fields, they are able to get access to the top faculty earlier on. They’re able to essentially get their research accelerated. I think for things like banking and consulting, I think in addition to the human capital formation is peer effects. Peer effects on what is worth desiring as a career outcome. You know, if your friends are also very interested in banking, you’re going to be talking to them about the banking, and you’re going to do much better in the banking interviews, and you might even be a better banker as a result. So I think there’s a lot of that going on. In my experience as an econ major at a university that produces a lot of investment bankers, I don’t think actual knowledge of economics was the key determinant of who got those jobs, so yeah. [laughs]

David: Well, so that’s why I say I think it depends on what you mean by human capital. So I think I agree with your analysis. The way I would say it in my own experience, too, is having gone to a school like Ohio State, and then I came to Harvard to do my PhD, the big impact of Harvard on me, I mean, which led to human capital formation, was it raised my expectations for myself. Like, I had a classroom of people who were like, oh, I was like, “These people are serious. Like, I better get my act together.” You know? And it made me learn more. So is that human capital? Because I built more human capital causally because of that, but it was driven by a peer comparison. So I think it’s just hard to disentangle. I think the way we talk is human capital signaling networks. It’s just sort of hard to put any of it into that box so neatly. That’s my sense.

Seth: I’m going to try to have an AI take about this. I guess the AI take is, I think the idea of forming personal connections and being in a personal network with a bunch of people who are in a community will become relatively more important as a mechanism for getting these prestigious firm outcomes, where it’s really, in some ways, more important to have a community of people who trust and have a common background with each other than it is necessarily about skills. Because if we’re able to outsource more and more of our skills to the AI, it’ll be about tribe formation, and I think that’s a big part of what goes on at some of these elite schools.

David: Yep. I think it’s true.

Andrey: Yeah. So let’s talk a little bit about AI use policies at Harvard. Do you want to kind of lay out, first of all, your position on it, and then we can ask you some questions?

David: So I wrote a convocation speech that I delivered to students on the day before the first day of classes. I talked about the three eras of undergraduate education. My argument was that we’re entering a place because of AI, but not only AI, where we need to embrace a more communal aspect, and I talked a little bit about how that relates to AI policies. I sent an email to students the first day of class outlining my goals for the year at the college, which is something I did last year too, to say, like, “What are we trying to accomplish?” And one of them was trying to work with faculty to develop a set of advisory guidelines about AI use in classes. And I think they have to be guidelines and not hard policies, because the way you use AI in an intro to computer science class and a 10-person humanities seminar is just totally different, so I don’t think there’s a one-size-fits-all approach. But I do think we need to give more guidance to faculty and to students about this, because the state of nature here is that everyone’s using AI all the time for everything. And it’s actually quite easy for me as a 47-year-old man who built my skills in the pre-AI world to put AI down and write for myself. It’s actually not that difficult. But for students who are basically growing up in this era, so our college freshmen now have been through most of high school with this tool available, I think we can’t take for granted that it’s as easy for them. And I think what’s happening is, in a lot of classrooms, faculty are saying, “You’re not allowed to use AI,” or, “You’re allowed to use it for brainstorming, but not for writing,” as a loose guideline, but not in an enforceable way. And then you come and turn in a term paper, and students are just put in a difficult position of doing the right thing, like behaving with integrity, but feeling bad about it because they think their classmates are not behaving with integrity, and maybe their classmates are getting better grades and using AI on the sly, or violating the course policy. So you either do the wrong thing, or you do the right thing and feel bad about it. And so I think you have to be realistic about what’s happening out there right now. And many people will do the right thing, so it’s not that I think students are going to cheat all the time, but enough of them will, or at least they’ll feel the temptation to, that we have to design our assignments and courses to be AI aware, AI accepting, if we’re not willing to take the steps necessary to enforce it. So I think you could have, on one level, you can have a low-tech equilibrium where everything’s blue books and it’s controlled conditions, and that’s a totally fine way to do a lot of teaching and learning. And I think on the margin, we should be moving more courses toward that. Like in Ec 10, Jason Furman and David Laibson are teaching introductory econ here at Harvard, and they reduced the weight on take-home problem sets this year and increased the number of midterms. So that’s an AI adaptive thing in one direction. In the other direction, if you want to assign a paper and give it to people to take home, you just have to understand that AI is going to be there for them. And I think there’s a lot of ways to do this. I don’t have all the answers, Seth, or Andrey, but to think about it conceptually, I think as a professor, you need to get more into the process of writing with people if you want to be able to guarantee that people are doing the thinking on their own. So one thing I’m going to do in my class, which I’m teaching in the spring, is I’m going to ask students to write a first draft under controlled conditions that we know is AI-proof, they’ve done the thinking for them, and then they can use AI to help revise and draft under supervision. So it’s like, turn in a first draft, turn in a second draft, give me an outline first, tell me what you’re thinking. Oral exams, spot checks on, “Oh, you developed that argument. Tell me more about it.” So it’s really just getting into the guts of the thing, which I think will actually be better for student learning anyway. So that’s an example of how you could encourage AI or you could accept AI without giving over the thinking to the tool, which I’m very much against. But more than anything, I just want us to start talking about this more than we currently are and not just pretending like it’s not happening.

Andrey: Yeah, totally.

David: Because I talk to students all the time who are quite anguished about this. They want help.

Andrey: Two things you mentioned, like one, controlled setting. Do you think there’s a possibility of creating a library space or somewhere where people can go and access a computer guaranteed, no internet, no AI? Would that be useful for students? Like, you kind of clock in, you drop off your phone, you go sit there for a few hours, and you just write?

David: Yep. Some students do this on their own. I basically bought a bunch of these phone pouches, these Yondr phone pouches, that we’re going to have available on a voluntary basis for students who want to enter a space. And I don’t know if you know how they work, but you basically have a little box and you stick the thing on it, and you carry your phone with you, so it’s not me taking it from you, or anybody taking it from you. It’s just like a voluntary commitment device. So we’ll see, maybe that people want that and maybe they don’t. I’m just trying to experiment with different ways to just make it a little easier for students to do the thing that they want to do. And I was telling some students last night, like, I always feel torn between two principles. One is, there’s certain things that I would like for them to do, [laughs] but also they’re adults and they make their own decisions, and why should they listen to me? So I’m trying to make it a little easier for them to do the thing that they want to do while also not being too paternalistic.

Andrey: What about oral exams? So, I can imagine a world... One of the biggest kind of source of anguish for faculty is the fact that we have to both teach and evaluate, right? And that puts us in a very unusual, strange position to be cops. And cops in different ways. One, catching cheating, which is just not a very pleasant experience for everyone involved, but also students arguing about grades and so on. Let me imagine sitting a student down with a computer, and the computer is talking to the student in an oral exam fashion and is evaluating them. And that’s something that I can imagine can be done in most undergraduate classes for sure without faculty involvement. Have you considered this sort of mechanism? Do you think we’ll see it?

David: I’ve talked to faculty actually who are trying things like this. And if somebody handed me a tool like that that worked really well, I’d be pretty open to certainly to using it in my own class. I think basically the oral exam is a great technology for assessing mastery. That’s why we do oral defenses in PhD. And I don’t know how you guys do them or have done them, but I tend to just like, students will submit a research statement or paper and I just keep asking questions until they don’t know the answer, and then we talk about that. And that’s a very effective teaching and learning tool. The problem is it doesn’t scale very well, to your point. And so if you could use technology to scale it, I think that’d be great. Like a adaptive, personalized oral examiner would be a great way to learn. So I’d be pretty open to it. I haven’t seen anything that’s shovel-ready, but that doesn’t mean it doesn’t exist. Maybe somebody will watch this and send something to me and I’d be open to using it. Or maybe you have it. I don’t know.

Andrey: I don’t. But yeah, I think that’d be pretty great.

David: That’s disappointing.

Andrey: Yeah. [laughs] I think there’s a professor at NYU who I saw wrote a blog post about his-

David: Really?

Andrey: … design of such a technology.

David: I think it’s a cool idea. And it would be mildly surprising to me if in a few years people aren’t using something like this more. It just feels like an obvious win. Education way back was tutorial style. Exactly. So, why not go back to that?

Andrey: Well, I think one of the reasons we stopped doing it is it was kind of too subjective. There was too many ways for there to be bias. I mean, I have personal history of this. My parents were in the Soviet Union and are Jewish, and they never got the same oral exam that the normal students got.

David: That’s a good point. Yeah.

Andrey: Yeah. But I think with an AI, you’re a lot less worried about these sorts of things.

David: So, as with anything, technology that solves some problems creates other ones. But it’s an exciting area to push on, I think.

(46:05) Grade inflation

Andrey: Now we were talking about assessment. I would love to ask you a little bit about, there’s been this huge debate about grade inflation in American universities and at Harvard, how to deal with that. How do you think about this problem?

David: So at Harvard last year, the Faculty of Arts and Sciences voted, I think by a 70 to 30 margin, to cap the share of As, not As and A minuses, but just straight A grades in Harvard College classes, to 20% plus four. So what that means is, like in a five-person class, you can give 100% As, in a 10-person class, it basically asymptotes to 20% as the class gets larger. And also, less commented on, but just as important, voted on something called the average percentile rank, which is basically in order to calculate honors for students to break ties, you basically rank students internally in the class and then use that to calculate honors, not just whether you got an A or an A-minus. So it gives more gradation to people’s performance in a way that allows you to figure out who deserves some magna whatever. So those are the two things they passed. I should say, I was the dean when it happened, but this is a multi-year effort. I supported it. I was actually for it, but it wasn’t my initiative. It was something that the faculty worked on through a committee over the course of the year, and I kind of just shepherded it through. But I do think it was good. And when I say good, I mean better than nothing. So my view on this, and maybe this is just like, maybe you guys will be in a role like this one day, maybe not, but what I would say is you end up living in a second best thinking on the margin world a lot more than you’d like to. So I answer a lot of questions from students and faculty and other people who are sort of like, “Why didn’t you do X?” And I’m sort of like, “Well, X is might be what we wanted, but Y is what we could pass and it’s better than Z.” So like this, when you talk to people about what is their ideal grading system, people have lots of ideas. I think some of them were good, some of them weren’t very good, in my opinion, but everybody’s got their own ideas. And then you have to triangulate that against, what do you think students will tolerate and what will the faculty pass? And so what we did was turn this over to a committee of faculty who spent a bunch of time talking to students and thinking about it, and then they developed this proposal, and it was like yes or no. And so I kind of said to people, “Look, there are a lot of things that I might do differently about it, to be honest, but I think this is actually going to make things better than not doing it.” And that’s because we basically were... My view is if you look at the trend, we were basically increasing the share of As by three to five percentage points a year. And once that gets up to 100%, which it would in a decade, we’re functionally a pass-fail institution. And if everyone’s getting As, As have no meaning. And so that’s what I think we needed to prevent, and this is a way of doing that, even if it’s not the best way. A question that people asked a lot was, shouldn’t an A be a standard of mastery? You know? So it’s like, well, if everyone learns the material, everyone should get an A. And so actually there’s an answer to this in the handbook, which is that the grades of A range are reserved for assessment of mastery. So, like, anybody can get an A or an A minus, but the grade of A is rewarded for performance of extraordinary distinction. That’s the language in the handbook. And so that means if 60% of people are extraordinarily distinguished, then it doesn’t have any meaning. And so the faculty said, “Look, A-minus is for mastery. Anybody can get it if they meet the standard. But A is for both mastery and distinction.” So there’s at least a logic to it.

Andrey: So let’s think through the equilibrium here. It’s going to affect a few margins. Student effort in a class, right? Just presumably the amount of learning. Student choice of classes, and then employer interpretation of the signal of the grade. So have you started monitoring some of these? What’s going on after the policy has changed?

David: So on the employer side, we monitored it ahead of time. So one thing that Amanda Claybaugh and I did, and she’s the dean for undergraduate education, she really shepherded. She was in that role before I was, and so she really is responsible for a lot of this. She talked to a bunch of employers to get feedback on this, and I would say they were 100% in favor of it, actually. And what they said was, “Your grades don’t mean anything to us, and so we have to look for other signals of distinction,” basically. And we heard that from graduate school admissions directors, too. And they don’t just say this about Harvard. This is a problem at every school in the Ivy Plus or among selective schools, so it’s not unique to Harvard at all. But they’re basically, I think what became clear to me, Andrey, is that you would think if you give everyone As, it takes the pressure off, but actually the opposite is true because students feel like they have to be perfect and they also feel like an A is not good enough, so I got to do a thousand other things. And so employers were really saying, “Look, this is going to help us differentiate, and now we don’t need to ask for students to jump through a zillion hoops. The A means something again.” So that’s how they looked at it. I was actually surprised it was that high. I thought it would be more of a mixed bag, and that was quite influential to me in my own support of it was like, if employers are saying we need this, then I take that seriously.

Andrey: One of the things I’d be worried about is, yeah, employers can say that, but then when they’re comparing a Harvard and MIT student, or I mean, MIT doesn’t have as much grade inflation as Harvard, but a Yale student. The Yale students still have all As, and then if they don’t exactly adjust for the signal of the grade. I know a university that’s elite and has very little grade inflation is Caltech, and sometimes the students are very unhappy about how they get compared to other universities. [laughs]

David: Yeah, so I’d say two things about that. One is, you’re right, but that’s the kind of logic that gets you to never solving the problem. Because somebody has to go first. And actually, it’s an antitrust violation for us to coordinate on this together. So somebody has to be the first mover. That’s true. I mean, you laugh, but it’s true.

Andrey: Yeah, no, I know. It is.

David: We actually can’t.

Andrey: It’s true, yeah.

David: It’s non-price competition. Somebody has to do this, and so we just figured let’s leap first. And I hope that other, I mean, we didn’t coordinate, but I hope that other schools will follow our lead and go together. I think the other thing is, and Jason Furman made this point very well, is that if you’re worried about employer perceptions, you actually want a big change, not a small change, because then it’s salient to them. So we will write on the transcript, “In this year, we changed our grading structure,” and everybody will understand that, and that actually will mean something. Whereas if you just invisibly decrease the share of As by a few percentage points a year, you’re actually harming students more because people aren’t aware of it. So we kind of had to do this. So we had a very intentional strategy of big PR campaign, get it written up everywhere, get everybody talking about it so that when it happens, employers will know. And I think we accomplished that. Everybody and their mother wrote about it. I think there was five articles about grade inflation in the Wall Street Journal last year or something, you know.

(52:41) Lightning round

Seth: Lightning round. All right. In the World of Darkness role-playing system, social attributes are divided into three different categories. Charisma, which represents social power, manipulation, which represents social finesse, and composure, which represents social defense. If you were starting a new game plus in your life, how would you allocate attribute points across those attributes of social skill? Bonus points if you give me the pre-AI answer versus the post-AI answer.

David: Defense would be my number. I’d probably do 50% on defense, because I think it’s important to be able to think for oneself. So to me, that’d be the most important thing. I think charisma’s very important, so I’d probably go 35 to 40% charisma, and I’d take the rest for whatever, manipulation, which sounds kind of negative to me, but I don’t know. I’m sure it’s valuable.

Seth: Charisma is just like your brute force. That’s how people are paying attention to you. You have their attention, and then manipulation, you can be a little bit more subtle about how you affect them.

David: I’d go 10 manipulation, 40 charisma, 50 defense.

Seth: I love that answer. Does AI change that answer?

David: Yeah, I’m thinking about that. So I’m thinking about it on the fly, live. I’m doing a posterior updating right now. I don’t know if I would change it with AI, actually. It’s kind of hard because I’m in the AI world now. There’s a question of what I would have done five years ago. I’m not sure. I don’t think I would change it. Maybe that’s disappointing to you, but would you change yours?

Seth: No, you took that question remarkably seriously. I really appreciate it. [laughs]

David: I try to understand the assignment, you know?

Andrey: So, you’ve done a lot of descriptive work now on AI diffusion, both independently and with teams at OpenAI. There are a lot of results there. So what we wanted to ask you is, what surprised you the most in this work?

David: One thing that surprised me a lot actually was in the ChatGPT paper, we did this analysis of people’s conversations for different topics, and we mapped it to O*NET work tasks, and we knew something about people’s occupations. So the question is, do people who are doctors use AI differently than people who are lawyers or medical assistants or teachers or whatever, software developers? And I thought the answer would be yes, people are using it for their work, and they are using it for their work, but actually the top tasks are the same for all those jobs. And the way I would put it is AI use is broad but shallow. So it’s like everyone’s kind of using it to manipulate information, like summarize, synthesize, understand, write, and for decision support, like asking for advice and what should I do with X, or how do I fix my dishwasher, or give me advice about how to deal with a coworker. And it’s like you would think that if you’re a teacher, you use it to educate people, and if you’re a banker, you use it to do financial calculations, and that’s just not true. People are basically using it for the same things, but those things are broadly useful in every job. So I found that to actually be a quite hopeful story about whether this technology... And I mean, agents are different. This was pre-agents, this was chatbots. But it’s like there’s no way something like that is going to replace people because it’s so shallow. So I kind of updated positively in the direction of augmentation based on that evidence, and I had not expected it.

Andrey: Do you think that a lot of those questions sound related to social situations? So do you think that GenAI is augmenting people’s social skills?

Seth: Or substituting.

Andrey: Yeah, or substituting. [laughs]

David: I think it’s both. I think it’s doing both depending on the context. I think AI is pretty good for generic social advice. So, like, the AI is going to tell you, like, if you ask it, like, “Help me compose an email to my boss,” it’s like it’ll give you some general tips on how to do that well, but it doesn’t know whether your boss is a jerk or whether they’re sympathetic or how much your boss knows about you and the prior history. So, I think there’s a lot of context in social relationships, and so AI is pretty good at giving you a baseline generic boost, but it’s actually not very good at going one level up or filling in the gaps because you have a lot of... Part of it is the limitation of the user. It’s like if you don’t provide it with a context, it’s not going to give you what you need. And so even if the AI is super intelligent, it can’t read your mind, you know? And so I think that’s actually a human limitation, not an AI one, but it’s an important one.

Seth: My follow-up question is, what do we not know from all of these sort of descriptive papers about how people use AI that you would want to know in a next paper?

David: There’s a lot we don’t know.

Seth: Yeah?

David: There’s much more that we don’t know than we do.

Seth: What’s one thing we want to know that would help us make some useful policy?

David: Yeah, so I think people are very focused on which jobs are going to be hurt and helped by AI and which are going to disappear, and they use these exposure indices and things like that. And that’s valuable work, but I think there’s a kind of not quite macro mezzo level of this that is extremely important, which is how AI changes the organization of work. Not just who’s going to do what, but like the Luis Garicano and other work on knowledge hierarchies, like firms organize like pyramids where there’s a lot of people at the bottom and they go, “Is AI going to change that?” My instinct is it will change that. But you don’t get that from an individual survey, and you also don’t get that from a big macro look at AI. You have to actually get inside a company and ask, how is it changing when it starts to use AI and how do roles change, how do functions change? And I’ve seen some work on that, but I think there’s way more yet to be done, and my prior is that actually that’ll be the big thing that AI does. I think there’s a good chance AI will change the organization of the firm in a fundamental way.

Seth: I believe that.

Andrey: Yeah, we do as well. [laughs]

Seth: We definitely do. David, I believe the last conversation we had was long before your days as Harvard dean. It was around 2018, we had a conversation outside of MIT where we were talking about sort of blind spots in economics. And you told me that one thing, correct me if I misremember, that you thought of the way that economists think about price gouging and price caps during crises as being sort of illustrative of economic logic breaking down, or maybe a better way to put this is economists not having good social skills. Do you still think about limitations on price caps or hoarding and price gouging during crises as an example of something that economists get wrong? And maybe have you thought of any new blind spots you’ve discovered in economists’ thinking?

David: You know, that’s a good question, Seth. I don’t know if I’d fully agree with my past self. I mean, in the sense that I do think there’s some blind spots, but I’m not sure that’s the number one thing I would pick for sure, or even the number two or three thing. I think it’s more, I don’t know exactly what I said, but what I would say now anyway is an argument for why economists... Like, we can write it down in an economic model and understand the empirical impacts of this and how, for example, caps restrict supply, gouging is necessary to clear the market, and all that stuff. But I think the equilibrium impacts of that kind of behavior and that kind of thinking are often negative enough to offset whatever efficiency gains you get in the short run. So it’s like, I don’t know if this is what I said, but a lot of times there’s a political dimension to economic choices that are hard to model explicitly but actually really matter in the way people perceive whether these things are fair or not. So, like, there’s a kind of-

Seth: Right

David: … first best outcome you can write down, and there’s a second best outcome that is like the outcome of a negotiation that can end up leaving society better off. I don’t know if I said that.

Seth: No, you said something very similar, and then I said-

David: Okay, good

Seth: … “Well, if you allow people to gouge, then they’ll hoard in advance, and then there’ll be greater supplies if they know they can have a high price later.” And then you said, “Well, but there’s all these other bad general equilibrium effects,” right? But are those general equilibrium effects, are those really economic effects, or does that have to do with trust and relationships?

David: Well, I would say those are economic effects. Trust and relationships are. Like, here’s a line of research that I think is highly rated but still underrated, which is the work of like Bob Gibbons and Rebecca Henderson and others on relational contracts, because I think relational contracts basically run the world. Like, you can’t really write a contract between people that specifies all the things that could go wrong or that could go right. And so a reason, and this is why trust is important, is it’s like, by the letter of this contract, I could screw Seth and Andrey, but I don’t want to because I value my relationship with them. And when that breaks down, a lot of bad things start to happen because... And you see this with, I don’t know, the changes in what’s happening in government or what’s happening in American society, where it’s a lot of the things we thought of as being like hard rules are actually just norms that you could break, and then they’re broken. And that’s the breakdown of relational contracts. So anyway, I think that’s great work. Again, highly rated. It’s not like nobody knows about it, but still underrated.

Seth: Well put.

Andrey: One final question, and then we’ll let you go.

Seth: To your very important job of, presumably bossing Harvard undergrads around and- [laughs]

David: I don’t do much bossing around. [all laugh] That’s how I remember my undergrad, was the dean was always bossing. [all laugh]

Seth: Putting people on double secret probation. [laughs]

Andrey: Yeah.

Andrey: Who is the best teacher you ever had and why?

David: What a great question. I think Larry Katz is the best teacher I ever had. I don’t think it’s because he has the highest charisma lecturing. He doesn’t. He doesn’t, actually. I don’t think he would mind me saying that. We used to call him Encyclopedia Katz because he just has this incredible memory of papers. And so, when he lectures, it’s like drinking from a fire hose. It’s like this paper and this paper. But the thing about Larry is that I learned from the way he works more than just the lectures and the class itself, which is there are some scholars who do work that closes things down. So, there’s people who have a sense of intellectual closure, like this way of thinking is the right way and your way is the wrong way, and they tend to try to draw battle lines. And Larry is the opposite. He’s the most intellectually generous person I’ve ever met. He’s like, “I know all this stuff, but hey, if you have a new way to think about it, let’s talk about that.” And learning from him, I think it gave me a sense of openness. I have one way of thinking about this world, but it’s not necessarily the right way, and I’m always open to other. And also a kind of openness to tools. So, if you need to write a structural model to understand something, do that. If you need to do an experiment, do that. If you need to write a descriptive paper, do that. Say, “I’m interested in answering questions. What’s the right tool for the question?” Rather than like, “I want to use this tool. Let me find something to use it on.” I also learned that from Larry. And in a lot of ways, I’ve told this story before. I told this at his festschrift. I came to Harvard kind of through the back door. I got my PhD in public policy, and I didn’t have any of the math background that one would normally have, and I really struggled in my first year of classes. At the time, I thought I was dumb. But actually, now I know people had seen this stuff before, and it just took me longer to catch up, and I eventually did. But it didn’t feel great. And it wasn’t until I had the experience of working with Larry, and I basically wrote this paper in a labor class, which later became the first paper I ever published. It was about Head Start. And he sent me an email around Christmas time. I’ll never forget. And it was like a seven-page email. Like, “Here’s all the things you could do in your paper.” At the very end, he was like, “I think this has potential to be published.” And I was like, “Oh my God, I can do this.” And the amount of time this, and there was 60 people in the class. The amount of time that he spent on this and on everybody else, I’m sure he did that for everybody else too, was incredible. And there’s just no better public goods provider in the profession than Larry Katz, in my opinion. And so I learned so much from that. And you guys have asked me a lot of questions about being a dean, and yeah, I’m a dean, I’m a suit, whatever. In my heart, in my soul, I still identify as a researcher or a teacher. I got into it for that. But I also, when I took this job, I felt like I couldn’t say no because the institution’s been good to me, and I believe in what we do. And if you get called to serve, you serve. And it’s people like Larry who made me feel that way. It’s like this is actually an enterprise, and protecting it is worth doing, even if it’s kind of annoying sometimes and you have to have a lot of meetings. So, Larry was my best teacher.

Andrey: Wow, that was an amazing answer. Thank you so much. We’ve had so much fun. This is like a really-

David: Yeah, it was fun

Andrey: … meaty podcast, so thank you.

David: You guys asked the tough ones. That’s good.

Seth: Trying to keep you honest. And to our listeners out there, keep your posteriors justified.

Andrey: Thank you for joining us for another episode of “Justified Posteriors.” As a reminder, we’ve registered all of our priors and posteriors on our website, justifiedposteriors.com, so please check it out.

We came to steelman pro-worker AI and to find the tensions in it. The big question: is technology even directable, or is AGI the golden path to everything? Daron says technologists give him three reactions — “we’re already doing it,” “it’s impossible,” and “just wait for AGI” — and admits he can’t disprove any of them. From there: we discuss comparative advantage under superintelligence (Ricardo doesn’t save you if the other side is cheap enough), what he’d do if he were Dario (build a production line for applications, and move the knob from 95/5 toward 80/20 or 60/40), and why he doesn’t trust the market to sort this out.

We close with capital taxes and pensioners, whether Daron would join the AI-pause coalition (”slow down and pause are not the same thing”), what the US should learn from China on applications and classrooms, how he’d update The Simple Macroeconomics of AI for agentic AI. Lastly, we ask him what he really thinks of that Economist piece about him. “They’re the elite. I’m the working class.”

Links & References

Daron’s recent work

• Daron Acemoglu, David Autor & Simon Johnson, “Building Pro-Worker AI” — The Hamilton Project / Brookings, February 2026 (PDF · NBER w34854).

• Daron Acemoglu, “Why we must stop talking about artificial general intelligence — and instead build ‘pro-worker’ AI” — Nature, August 2026

• Daron Acemoglu, What Happened to Liberal Democracy? Remaking a Politics of Shared Prosperity — Penguin Random House, August 2026 (MIT News)

• Daron Acemoglu, “The Simple Macroeconomics of AI” — Economic Policy, 2025 (NBER w32487);

• Conversations with Tyler: Daron Acemoglu on Liberalism, Automation, and the Educated Elite — August 2026;

The task-based framework

• Daron Acemoglu & Pascual Restrepo, “The Race between Man and Machine: Implications of Technology for Growth, Factor Shares, and Employment” — American Economic Review, 2018;

• Daron Acemoglu & Pascual Restrepo, “Automation and New Tasks: How Technology Displaces and Reinstates Labor” — Journal of Economic Perspectives, 2019;

• Daron Acemoglu, Andrea Manera & Pascual Restrepo, “Does the US Tax Code Favor Automation?” — Brookings Papers on Economic Activity, 2020 (NBER w27052);

• Daron Acemoglu & Pascual Restrepo, “Automation and Rent Dissipation: Implications for Wages, Inequality, and Productivity” — NBER, 2024;

Other concepts, papers & people discussed

• Rich Sutton, “The Bitter Lesson” — the technologists’ argument that general, compute-heavy methods always beat specialized ones

• David Ricardo and comparative advantage — “Why did England trade with Portugal?” “Because nothing was very, very cheap.”

• Dell’Acqua, McFowland, Mollick et al., “Navigating the Jagged Technological Frontier” — the source of the “jagged frontier” (Ethan Mollick’s term, not Josh Gans’s)

• Bastani et al., “Generative AI Without Guardrails Can Harm Learning: Evidence from High School Mathematics” — PNAS, 2025; one of the classroom studies Daron has in mind

• Emmanuel Saez & Gabriel Zucman, “Wealth Inequality in the United States since 1913” vs. Smith, Zidar & Zwick, “Top Wealth in America: New Estimates under Heterogeneous Returns” — the wealth-inequality measurement debate

The Economist piece

• The Economist, “The world’s most influential economist is oddly unconvincing”

Previously on Justified Posteriors

• The Simple Macroeconomics of AI — our December 2024 episode on Daron’s paper.

Chapters

• (00:00) Cold open: three reactions from technologists

• (00:30) Intro — “help me or destroy me”

• (02:00) What is pro-worker technology? AI is not a TFP term

• (04:20) The general-equilibrium objection: new tasks in industry A, lower wages in industry B

• (08:56) Why care about the labor share if productivity is soaring? Three answers

• (10:04) Solow vs. AK: why not aim for growth forever with a labor share of zero?

• (11:37) Labor must be essential: dignity, self-worth, and “why would anybody listen to them?”

• (12:59) Is technology directable? Sutton’s bitter lesson and the AGI golden path

• (13:59) “We’re doing it already,” “it’s impossible,” “just wait for AGI” — and why Daron can’t disprove any of them

• (14:43) Is Anthropic’s “augmentation” pro-worker? Why the taxonomy can’t be read off the technology

• (15:37) AGI → ASI: nothing can be pro-worker; relational goods; Ricardo, Portugal, and the subsistence wage

• (18:29) If you were Dario: build a production line for applications, and move the knob from 95/5

• (20:26) Andrey pushes back: aren’t tons of companies already building applications?

• (21:36) The pre-agentic age had remarkably few usable applications; where the talent and compute are going

• (22:31) Will the market sort it out? “I haven’t convinced you that there is a need for proposals”

• (23:28) Nobody is making money: capital allocated on “future projections, vibes, and beliefs”; sovereign wealth looking for a place to park

• (25:32) If there’s an overabundance of capital, why isn’t pro-worker AI funded? Data centers and talent scarcity

• (26:47) Two standard market failures: the capital-biased tax code and excessive automation under labor-market rents

• (29:51) Who holds the capital? Life-cycle savings vs. the very rich; Saez–Zucman vs. the skeptics

• (31:10) Is the long-run supply elasticity of capital infinite? The data says moderate; OLG vs. representative agent

• (32:25) Having your cake and eating it too: equal treatment of capital and labor raises GDP in the calibration

• (33:56) How close is AGI? “Not that near,” and not the productivity-maximizing strategy either

• (34:23) Would Daron join the pause coalition? Safety vs. epsilon-from-AGI; hitting the brake to steer

• (37:45) China, classrooms, and the missing US regulatory infrastructure

• (39:11) Updating The Simple Macroeconomics of AI: agentic AI raises the numbers, diffusion is still in the money

• (40:28) The three real bottlenecks: engineers, organizations, and the jagged frontier

• (43:25) AI in Turkey: public use, lagging firms, and the state

• (44:44) Acemoglu memes

• (45:31) That Economist article: “if you’re going to have a hit piece against you, pray that it’s an incompetent one”
  

  Transcript

  ---

  
  Cold Open & Introductions [00:00 – 02:00]

  [00:00:00] Daron: Whenever I talk about pro-worker AI, which I’ve been doing quite often for about ten years, when I talk to CEOs or managers of companies, they are on board. Whenever I talk to technologists, I get three reactions. One is, “We love it. We’re doing it already.” Second, “It’s impossible.” Third, “Yes, just wait for it. We’ll get to AGI first.”

  [00:00:30] Seth: Welcome to Justified Posteriors. This week, we’re honored to host Nobel laureate Daron Acemoglu of MIT. He’ll answer the question “What happened to liberal democracy?” — the title of his new book. But first, we’re excited to learn about his evolving views on AI, and especially to pin him down on what pro-worker technology actually means. I’m Seth Benzell, always racing with the machine — and I don’t mean using a treadmill — at Chapman University in sunny Southern California.

  Andrey: And I’m Andrey Fradkin. Very excited to have Daron on the show. He was an inspiration for me going to economics grad school, so this is truly an honor. Coming to you from San Francisco. Welcome to the show.

  Daron: Thank you, Andrey. Thank you, Seth. This is my true pleasure and honor.

  [00:01:14] Seth: Thank you, Daron. Okay, so we have limited time, so we’re going to jump right into it. In a recent series of essays, including a paper with David Autor and Simon Johnson called “Building Pro-Worker AI,” and a new piece out in Nature about why we shouldn’t be talking about AGI, you’ve been calling for this thing called pro-worker AI. And of course, we love workers here on our podcast.

  Daron: We are all workers.

  Andrey: Yeah, it’s true.

  Seth: We love the workers here. But as we’ve been engaging with this, we want to help you steelman it for us, and help us understand some bits about it where we still think there may be some tensions.

  Daron: Help me or destroy me. Let’s go for it.

  ---

  What Is Pro-Worker Technology? [02:00 – 05:01]

  [00:02:00] Seth: All right, let’s do it. So the first question is: what is pro-worker technology?

  Daron: I would like to step back and say that, in general, in economics, sometimes we black-box what technology does. That’s not helpful, broadly speaking, but it’s particularly unhelpful when it comes to AI or other digital technologies.

  Seth: Right. AI is not a TFP term.

  [00:02:29] Daron: AI is not a TFP term, and it’s not a plain vanilla, garden-variety, labor-augmenting technology consistent with balanced growth — because we’ve assumed it has to be consistent with balanced growth. Most technologies make you better at one specific thing. They may make you less relevant by automating the specific things that you were doing, going back to the spinning and weaving machines, all the way to robots and other advanced machinery on the manufacturing floor, and now AI. They may also increase the new things that workers do, either because the machines that are being introduced require human input, or because the new technologies — directly or via organizational changes — create opportunities for workers to perform new tasks or gain new expertise.

  So my point, in general, is that these technologies have very different implications for worker wages, employment, the share of capital and labor in national product or sectoral value added, and, via their social effects, they have very distinct implications for liberal democracy, for social peace, and things like that. That’s why we should care about all of these things.

  [00:04:03] Seth: Great. One way you put it in your recent paper is that you define pro-worker technologies, including AI, as technologies that make human skills and expertise more valuable by expanding worker capabilities, which is aligned with what you just said.

  Daron: Exactly. So pro-worker AI is that last category, Seth. Exactly.

  Seth: All right. So staying there, there were a couple of different concepts that you brought up. You brought up the labor share being high as something that’s desirable. Making skills and expertise more valuable in these new tasks is something that’s desirable. And also this idea that when you create new tasks, it’s going to — maybe inevitably, maybe most of the time — get you these other things. But I’m sure you’ve thought this through. You can think of scenarios where you invent a new task for a worker in industry A, and then through some sort of general equilibrium effect, that’s going to reduce wages, or the wage bill, by more than that in industry B.

  ---

  General Equilibrium, Sectoral Reallocation, and the Labor Share [05:01 – 08:56]

  [00:05:01] Daron: Wow, you really want to go to the advanced material.

  Seth: Well, this is the general equilibrium econ of AI podcast.

  Daron: Boy, that’s the topic that I discuss only under very, very protected circumstances, where non-economists are not going to be bothered by it.

  Seth: Only economists in the room. I know you don’t get to put this in your Nature essay. Okay.

  [00:05:24] Daron: But let me again take a step back before answering your question, which is an excellent question, and you’re right. Let’s first say that everything I’m going to say is dependent on a particular framework. No conceptual framework, no coherent discussion. And I’m using the task-based model.

  Seth: You’re thinking about “The Race Between Man and Machine,” maybe.

  Daron: Yeah, that’s right. That’s one example of that framework. Somewhat more general versions of it have been developed since that paper. So in that framework, the very unambiguous prediction is that whenever you do automation in a sector, that reduces the labor share in that sector.

  Seth: In that sector.

  Daron: In that sector. And whenever you introduce new tasks, that increases the labor share in that sector.

  Seth: Very well put.

  Daron: And those actually receive ample empirical support as well.

  Seth: Yes. For example, your papers with Pascual about automation in the auto industry.

  [00:06:28] Daron: Well, David Autor’s papers, several other papers on robots, et cetera. The overwhelming number of them — and especially if you zero in on those that are well done — find these labor share results. Now, the general equilibrium effects of any technology, not just new tasks or automation, are different, and that’s something Pascual and I also address in some of our papers. It’s a very simple formula that you can have at the back of your mind if you want: any sectoral reallocation is going to change the aggregate labor share via the difference between the labor share of the expanding and the contracting sectors.

  Seth: Makes perfect sense.

  Daron: So in other words, if a labor-intensive sector contracts and a low-labor-share sector expands, that’s going to reduce the aggregate labor share. Hence, the scenario that you point out is entirely possible if, associated with new tasks, the sectors that have low labor share expand and the sectors that have high labor share contract.

  [00:07:39] Seth: So maybe one idea I have in mind here: you might imagine the movie business is invented. This creates all sorts of new tasks. I’m in LA these days. You can create all of these new tasks in the movie industry, which has a labor share that’s maybe eighty or seventy percent, and you’re crowding out the local theater, which had a labor share of ninety percent.

  Daron: One hundred percent. That’s possible. But again, that’s only relevant if there are really big sectoral reallocations —

  Seth: And really big differences.

  Daron: And really big differences. In one of our papers, Pascual and I did that decomposition for the aggregate US economy from the 1950s to the 2010s, and we found that those sectoral reallocations were on average neutral, meaning that the expanding and contracting sectors roughly had similar labor shares. That doesn’t mean that moving forward with AI it’s going to be the case. But from a recent economic history point of view — Baumol effects notwithstanding, which are very interesting and specifically relevant for some types of skills — for labor overall, that didn’t seem like a huge deal. Moving forward, that might be different.

  ---

  Why Care About the Labor Share? AK Growth, Dignity, and Liberal Democracy [08:56 – 12:59]

  [00:08:56] Andrey: I was going to push back a little on the labor share. I understand that there are reasons to care about it in and of itself, but presumably if productivity is increasing a lot, then the labor share can go down and people are a lot better off.

  Seth: Yeah — maybe we should be talking about reallocation to industries with better or worse median wages, right?

  [00:09:17] Daron: One hundred percent. I also agree with that. There are two different ways of thinking about the labor share, and then there’s a third reason to care, which I’ll come back to. One is that if we go from a sixty-six percent labor share to a sixty-two percent labor share and stay there, and in the process productivity is much higher — who cares? I’m willing to go with that. However, if there is a steady decline in the labor share, asymptoting, ultimately going down towards very, very low values, I think that is a cause for concern even if wages increase, because it means that labor is becoming more and more sidelined in the economy.

  [00:10:04] Seth: This is interesting. I had this as a topic we would talk about later — thinking about automation and capital accumulation dynamics in the long run. But I can tell a really positive story about the labor share going to zero, right? You think about moving from a Solow universe, where we have a cap on output because eventually growth asymptotes, versus an AK universe where you can get growth forever. So why not AK-universe growth forever as the thing we should be aiming for?

  Daron: That’s a much broader topic.

  Seth: Right. And so that’s going to be a political economy answer. Okay.

  [00:10:43] Daron: Before I come to that, I’ll just give a third reason, which I said to Andrey. From a practical point of view, if you want to think about what is going on with wages — which, Andrey, you said very much we should care about — then the labor share is a driver of that. If you want to see what’s going on with the wage, that can always be decomposed into something like a productivity effect, a displacement effect, and a reinstatement effect, and the labor share tells you about one of those components. So it’s also a diagnostic tool to think through, even if we don’t care about it ultimately.

  All right. Now, coming to your question, Seth. Since I’ve just published a book about liberal democracy —

  Seth: Ah, plug the book. Do it. It’s good.

  [00:11:37] Daron: Thank you. That’s great. All right, we can finish our interview now. I got what I wanted. So I think that a world in which labor becomes dispensable — where the AK model would be one version of that, where you don’t need labor for anything — would not be consistent with liberal democracy, or any type of society in which people are viewed as equals of those who control the machines, who control the AI models. It would be a very hierarchical, almost two-tier world. So I think there are a lot of problems with a world like that.

  Seth: You basically think wealth dynamics in the long run are always more unequal than labor distribution dynamics, something like that.

  Daron: Well, I think you need labor to be essential in some way, so that, A, the vast majority of people actually have some contribution to society, which is important for their self-worth, which is important for their dignity and all sorts of things. And it’s also critically important, as you anticipated, Seth, for the political economy. Because if they are dispensable, why would anybody listen to them?

  ---

  Is Technology Directable? Three Reactions from Technologists [12:59 – 18:29]

  [00:12:59] Seth: Okay, we’ll come back to political economy. Let’s stay on pro-worker AI. The second question I have here is one that, whenever I hear you talking about this, I want to jump into the podcast and say, “Ah, this is the question Andrey and I have.”

  Daron: Now is your chance.

  Seth: Here’s my chance. When you talk to technologists and you say something like, “We should direct AI to be more about healthcare AI instead of military AI,” the way they say it is, “AGI is on the golden path to getting whatever technology you want.” They’ll quote Sutton’s bitter lesson at you — this idea that in the long run, the generalized, heavy-compute technology will always beat the specialized technologies. Whatever individual expertise-enhancing, task-expanding technology you have in mind, the shortest path to that is AGI first and then that. So is technology directable in the sense that you have in mind, if that’s the case?

  [00:13:59] Daron: Whenever I talk about pro-worker AI, which I’ve been doing quite often for about ten years — when I talk to CEOs or managers of companies, they are on board. Whenever I talk to technologists, I get three reactions. One is, “We love it. We’re doing it already.” Second, “It’s impossible.” Third, “Yes, just wait for it. We’ll get to AGI first.” And all three of those, I cannot disprove. They might have some truth to them.

  Seth: But you can’t disprove anything. That’s what your book’s about.

  [00:14:43] Daron: Right. Exactly. All right, you got that one. So it may well be that some of the things that some of the companies are doing are already pro-worker, and I’m not seeing it. It’s a possibility. Though I must say, when I see reports from companies like Anthropic, et cetera, claiming, quote-unquote, “augmentation,” typically it’s not what I would call pro-worker AI. It’s using AI to do some of the things you were doing before a little bit more efficiently, or something like that. That’s just the standard labor-augmenting technology, which has very, very different implications.

  Seth: I think what it also suggests is that doing that taxonomy just from looking at the technology is extremely hard — that you really want to understand how wages and labor shares are changing in order to back out what kind of technology it was. But keep going.

  Daron: Exactly.

  [00:15:37] Daron: Second, the statement that this is impossible — the version that I think is not disprovable is that we are ultimately, and in the relatively near future, going to get to AGI. Once you get to AGI, you’re going to get to ASI, artificial superintelligence, and then at that point, or somewhere along that spectrum, whatever workers could do in new tasks, whatever they can do in any place, can be done better by machines — robots with AI, or AI by itself. So in that case, nothing can be pro-worker. So again, I cannot disprove that.

  Seth: Well, nothing can be pro-worker in terms of labor share?

  Daron: Nothing can be pro-worker in terms of anything.

  Andrey: Well, there are relational goods, right? There’s still going to be work to do —

  Daron: If it’s superintelligent, then it can also replicate those relational things very well.

  Andrey: So there’s that argument, but I actually think that’s a wedge point. Some people think that those jobs will remain even in a world of ASI, and others do not. I think there’s some debate there.

  Daron: I don’t see how they could remain, truly. If you define AGI and ASI in a coherent way — that it does everything better than humans — then it must be that it can do the relational things better than humans as well.

  [00:17:01] Seth: Can you explain “everything better”? This Ricardo guy told me once that we would always have a comparative advantage in something.

  Daron: Not necessarily. You would have a comparative advantage, but if something else has absolute advantage in everything and is extremely cheap, why is your comparative advantage of any use?

  Seth: Why did England trade with Portugal, even though Portugal was better at everything?

  Daron: Because nothing was very, very cheap.

  Seth: Right. It was just a little bit cheaper in Portugal.

  Daron: A little cheaper. So if you literally have ASI, at least in the way that I’m defining it, and the marginal cost of producing these ASI services is very, very low, the only way human labor can compete against that is by becoming very, very cheap — which humans cannot survive on. It will fall below the subsistence wage.

  Seth: Okay, fair enough. And so now, number three, the sexy one. Tell us about it.

  [00:17:55] Daron: Number three is essentially also a coherent argument: that ultimately, on the path to something like AGI, you can do things in different ways, but once you are at AGI, the best way of doing everything is using AGI. This does not say you cannot redirect AI. You can redirect AI, but it would be futile, because ultimately the best thing from a productive efficiency point of view would be to just let it rip with AGI and ASI, and then we’ll use them in whatever way we decide to use them.

  ---

  If You Were Dario: Frontier Models vs. Applications [18:29 – 22:31]

  [00:18:29] Andrey: So I have a question for you. Let’s say you’re Dario, and you get to choose what Anthropic works on next. How would you go about making that choice pro-worker?

  Seth: Dissolve Anthropic.

  Daron: No, no, I don’t think so. Anthropic is a very, very creative company. The problem we have right now in terms of productivity is that we don’t have enough applications. We don’t have applications that are custom-designed for specific tasks, that are easy to use, that can be integrated into different organizations and enable those organizations to do what they were doing more efficiently, or do new things. I think the first thing I would want to do is create an effective production line for applications. And then, in the context of those applications, you can relitigate the question that Seth raised, which is: what’s the best way of making these applications better, more effective, more usable in the short and medium term — short being four or five years, medium being ten to fifteen years? Is it to put all of your energy and best talent into the underlying Claude model, or to actually use some of that to improve those applications? And if so, what are the places where we can make those applications more pro-worker?

  [00:20:01] Seth: And your hypothesis is that if you did that analysis — if right now their knob is at ninety-five percent frontier model and five percent applications — maybe that knob should be at eighty percent.

  Daron: Eighty, seventy, sixty. We really want the applications. Right now, if you are very knowledgeable as a programmer, you can use Claude to do many, many things. But most companies aren’t.

  [00:20:26] Andrey: So I’m a little bit confused by this statement, in the following sense. I see tons of companies trying to build applications for AI. There is even an argument that they’re all using the Anthropic models. One very plausible thing is that Dario can make the model more capable, and then the application layer can develop it for specific applications. And now we can try to go through each company one by one and say, “Is this pro-worker or not?” I think that might be a little tricky, but certainly people are trying. I don’t have the same sense as you that people aren’t trying to make the technology pro-worker.

  Daron: Look, things are changing fast, and you guys are in San Francisco. I’m in the less de—

  Seth: Andrey’s in San Francisco. I’m in LA.

  Daron: LA, okay, fine. The West Coast, let’s say. Sorry. I’m in the less developed part of the world, on the East Coast.

  Seth: You’re off the technological frontier. “Diffusion’s more important than pushing the frontier,” someone not at the frontier would say.

  [00:21:36] Daron: Exactly. So I’m not seeing all of the different places where creative energy is going. But if you look at the pre-agentic age, up to the beginning of 2026, say, it was remarkable how few usable applications we had. Perhaps that’s changing now, and next year we’re going to see a lot of these applications. We’re going to have companies properly start integrating some of those technologies much more into their production line. Then you may be right, Andrey. But again, I have not seen that yet. If you look at where a lot of the spending is going, in terms of both talent spending and compute spending, it seems to be the development of the general model and the AGI agenda.

  ---

  Will the Market Sort It Out? Vibes-Based Capital and Two Market Failures [22:31 – 28:37]

  [00:22:31] Andrey: Just a follow-up on that: how do you think about the market forces here? There are going to be a ton of startups. There are already a ton of startups. Some of them are very focused on augmentation, some of them are more focused on substitution. But isn’t it likely that the market will determine which of these technologies wins? I know you have policy proposals to try to address this, and I’m curious whether you want to speak about those.

  Daron: Okay, that’s fantastic. No, I don’t want to talk about the proposals yet —

  Andrey: Oh.

  Daron: — because I haven’t convinced you that there is a need for proposals. That’s what I want to do first.

  Andrey: Sure.

  Daron: I’m an economist, as you might know.

  Seth: I thought you were a political commentator.

  Daron: Exactly. That’s it. Thank you. That’s below the belt, Seth, but it’s fine. I’ll get back to you.

  Seth: No, you’re getting hated by all the right people these days.

  [00:23:28] Daron: Thank you. Thank you. By John Warner [?]. So I’m very keen on the market, but it’s also important to understand the limitations of markets. I think there are two limitations of markets as it comes to that. Let me start with the second one, which isn’t the classic set of limitations related to imperfect information, monopoly power, et cetera. We don’t have the standard market deciding where money goes at the moment. None of the companies are making money. They’re all losing money. They’re losing a huge amount of money. The way that we’re deciding where funding goes is based on future projections, vibes, and beliefs.

  Seth: Hasn’t it always been the case? How is that different from how the market’s worked in the past?

  Daron: It hasn’t always been like this. Most of the time, try to take a loan from a bank as a small entrepreneur. Describe your dreams. You know how much money they will give you? Zero dollars.

  Seth: You’ve got to be in the San Francisco ecosystem to get money that way.

  [00:24:37] Daron: But yeah, those previous venture capitalists, Seth — just to close the loop on this — would do that, but they would withdraw the money after one or two years if you couldn’t show returns. Here, we have created almost an ideology that the best thing is to lose billions of dollars, hundreds of billions of dollars, and then somehow, at some future date, everything’s going to be sorted out. So that is a very different way in which the market allocates capital than what we are used to. And it would not have been possible, for example, if we did not have a huge amount of private wealth from sovereign funds and very, very wealthy individuals — and some of it from venture capital — that was looking for a place to be parked. So I think that ecosystem we have to bear in mind.

  Seth: Is AI a symptom of the global capital glut?

  Daron: I would definitely not say that it’s because of it, but the two are synergistic.

  [00:25:32] Andrey: The overabundance of capital suggests that if there was a profitable opportunity to make pro-worker AI, it could get funded. That’s one version of the argument. So you might be right that we’re over-investing in certain types of technologies, but there is a ton of capital to go into lots of technologies, right?

  Daron: That’s true. There is an overabundance of capital. But two things. First of all, a lot of that is now going into data centers, et cetera. So before the generative AI boom, there was an overabundance. Now that overabundance is more tempered. Second, talent is scarce. The smaller application companies hire some very talented people, but the cream of the crop go to Anthropic, Google, OpenAI, Meta, et cetera.

  Seth: Talent’s an interesting one too, because — and this is tied into the directability issue — people make a decision in their early twenties about the expertise they’re going to have for the rest of their lives. So there’s a sense in which you have to make your decision based on vibes about what this industry is going to look like for fifty years, and you might not be able to turn your expertise on a dime.

  [00:26:47] Daron: That’s right. I don’t know how important that is, though, because some of these AI skills are fairly portable. The very good talent that goes to Meta for hundreds of millions of dollars probably could have been employed in some applications doing a great job there as well.

  But just to close the circle: second, there are the more standard reasons why what the market decides may not be efficient, and there are two in particular that I would emphasize. One is that we have a tax code that creates a big bias in favor of capital instead of labor. That’s something I’ve written about with Andrea Manera and Pascual Restrepo. The market outcomes, conditional on that distortion, will not be optimal. Second — something I have written about with Pascual Restrepo — if there are labor market imperfections and workers are paid above their outside option, as in a search model, bargaining, efficiency wages, et cetera, then the equilibrium will have too much automation and too much demand for automation from companies, because companies automate according to the wage that workers receive. A social planner would use the opportunity cost of labor in that decision, and the wedge between the two can be quite large.

  Seth: I was thinking about that paper. It’s an interesting paper. I’m sorry if you address it in there, but you could tell the same story about the other side: there’s a financially constrained company, and there’s a wedge between the interest rate and their opportunity cost of capital. Do you have an argument that that’s less important, that I missed?

  Daron: No. If capital wedges take a similar form, that would also be relevant.

  ---

  Capital vs. Labor Taxes, Pensioners, and Who Holds the Wealth [28:37 – 33:56]

  [00:28:37] Seth: Okay, so let’s go back to this idea. One policy instrument it seems like you’re proposing is to rethink our capital-to-labor tax mix. This is my last question about pro-worker AI, and then we can get on to the next topic. A classic argument for wanting to favor capital over labor is that we think capital is supplied more elastically than labor, and in the long run you always want to tax the inelastic thing. So maybe you can engage with that. And then, relatedly: is a pro-worker policy an anti-pensioner policy? Professor Kotlikoff was my PhD advisor, and so I have had banged into my head that Social Security is a Ponzi scheme.

  Daron: I thought Pascual was too.

  Seth: Oh, I have too. That was my committee. It’s Kotlikoff and Pascual, dude. And so now you know exactly why I’m exactly this amount of insane. So is part of your argument that we’ve done too much in our political economy to favor the rentier elderly class, and we should be favoring labor incomes?

  [00:29:51] Daron: There are bigger beneficiaries of the favorable treatment of capital than the pensioners, last time I thought. When you look at who holds capital in this country —

  Seth: Well, you tell me. In a life-cycle savings model, it’s the old.

  Daron: Right. In a model with heterogeneous agents —

  Seth: It’s the patient.

  Daron: — it’s the people who get all of the capital at first, right.

  Seth: So it’s a mix of that.

  Daron: Yeah, it’s a mix of that.

  Seth: So why are some people wealthier than others? It’s the people who’ve been accumulating their capital for the longest, and that could either be inherited wealth, which maybe we frown at, or some of that’s just middle-class people saving for retirement. You can tell me which is empirically more important.

  [00:30:35] Daron: I don’t know. I think probably in the fifties and the sixties, life-cycle savings was very, very important. But as wealth inequality has increased, I think the numbers I’ve seen suggest that more and more of it is in the hands of the very rich. There are many measurement challenges when it comes to wealth inequality — big debates between Emmanuel Saez, Gabriel Zucman, and others on the one side, and those who think wealth inequality hasn’t increased that much on the other. I find it very difficult, given the level of income inequality and capital income inequality, that wealth inequality hasn’t increased. But it’s not my area of expertise.

  But let me take the question that you asked, which is: is it efficient to tax capital so low? That would depend on many things. Supply elasticities is one of them. Labor market and capital market imperfections is another. And how this actually affects production decisions is a third. Our simplest models impose that the long-run supply elasticity of capital is infinite. The data doesn’t support that. In the data, the supply elasticity of capital seems to be moderate, not huge.

  Seth: This is the elasticity with respect to the interest rate?

  Daron: Yeah, elasticity to interest rates, yes.

  Seth: Okay. Well, I just want to say that’s more consistent with an OLG model than a representative agent model.

  Daron: Yeah, OLG models would have that. Exactly. That’s right. Your training again, Seth. When I said “our standard models” — sorry, I was not locating myself in BU. I was thinking of representative-agent standard models. I should be more careful.

  [00:32:25] Seth: Okay, so anyway, that would be a policy that you’d be interested in, and you would be theoretically willing to trade a little bit of growth for a little bit of labor share. Is that putting words in your mouth?

  Daron: Okay. So both on the issue of AGI versus pro-worker AI and capital taxes versus labor taxes, I have tried to have my cake and eat it too. I’ve argued, in fact, that the growth negatives from these policies wouldn’t be that major. In the Brookings paper that I wrote with Andrea Manera and Pascual Restrepo, for example, we make a bunch of assumptions. We use a specific model, but a relatively flexible task-based model. We calibrate the parameters of that model from existing estimates along the lines of what we were just talking about — supply of capital to interest rates, labor supply elasticities, quasi-labor supply elasticities. And actually, we find that if you went to a more equal treatment of capital and labor, you would increase GDP, not reduce GDP. But push me, and I would say yes, I’m willing to trade off a little bit of growth for a fairer distribution — especially one which would be more consistent with liberal democracy, social cohesion, the dignity of workers, et cetera.

  Seth: Those things seem good.

  ---

  How Close Is AGI? Pause vs. Slow Down, and Lessons from China [33:56 – 39:11]

  [00:33:56] Daron: I am definitely not the person you should talk to if you want to understand how close we are to AGI. There are experts. But my assessment is that true AGI, along the lines of what we were talking about earlier, is not that near, and going after AGI right now is not necessarily the strategy that would maximize productivity either.

  Andrey: That’s a very interesting claim, because I think there’s another policy proposal out there, which is to pause AI development. But the stated purpose of that policy is that we’re so close to AGI and we’re not ready for it yet. So would you, for instrumental purposes, support a pause on development?

  Seth: Will you get in the broad tent?

  [00:34:45] Daron: Oh, man. There are two arguments out there among the slow-down crowd. One is that we are very close to AGI, and if we push a little bit more, we’re going to be at AGI and all bets are off. The second is that we are doing really badly on safety, even though we may or may not be that close to AGI. After all, Mythos was not AGI, but there was broad agreement that it should not be unleashed without any guardrails. So I think there is an argument on safety that does not necessarily rely on being epsilon away from AGI.

  But more directly to your question, Andrey: I have spent too much of my career studying political economy and the history of technology, where I see so many attempts at blocking technological change, that I put a high bar on saying you should block technology, stop innovation, et cetera. But slowing down temporarily in order to redirect is a different phenomenon. If you are driving two hundred miles an hour towards a cliff, you may want to first hit the brake temporarily to steer better. But no, I’m not in favor of stopping progress, so long as we democratically agree — based on expertise, and a democratic compromise and communication — that the technological direction is socially beneficial. Right now, we have not had that democratic debate. And even if we had the democratic debate, this is an international technology that’s going to affect seven billion people. The remaining six billion, or close to six billion, who don’t live in the US and China haven’t had any say on this. So I think there are more complex issues here.

  [00:36:36] Andrey: A related question. Let’s abstain from thinking about whether the investment in the data centers is justified or not. But if we implemented a hard pause at this moment, wouldn’t that crash the economy?

  Daron: Yes. There’s a problem. So that’s why, again, slow down and pause are not the same thing. I think we really need applications. Look at China. Despite what we are sometimes told from Silicon Valley, there are things to learn from China. They are behind in large language models and some applications of generative AI, but they are far ahead of the US in integrating AI into the production process, and getting it into the hands of consumers in a manner that’s safe. So there are things that we can do on the applications front — hopefully in a more democratic and more human-centered way than China — that would actually be a contribution to GDP and consumer surplus.

  Andrey: What do you mean by “they’re making it more safe for consumers” in China? I just want to try to understand where that’s coming from.

  [00:37:45] Daron: Meaning that because there is more government regulation, I think some of the blowback from consumers using it in ways that may not be warranted — for example, in the classroom — is going to be easier to control in China. The evidence that is now coming out from a couple of papers is that the use of ChatGPT and other generative AI models in the classroom has been pretty bad for student learning. And how are you going to deal with that? I think we in the US have no way of dealing with that, because we haven’t set up a regulatory structure, we haven’t developed norms about how AI should be used, whereas in China there is more of that. And the consumers are also more savvy, because they’ve been using e-commerce more actively. So there may be more ways of adjusting to some of these things in China than in the US, because of that lack of regulatory infrastructure here. But secondly, I think partly because of labor shortages, Chinese companies are much more interested in applications of AI integrated into the production process.

  ---

  Updating “The Simple Macroeconomics of AI”: Agentic AI and Three Bottlenecks [39:11 – 43:25]

  [00:39:11] Andrey: How have you updated your estimates of the economic effects of AI on US GDP per capita since “The Simple Macroeconomics of AI”?

  Seth: One of the first episodes of this pod. So we’re very much in your shadow.

  Daron: Oh my God. Well, look, I have not gone back to the data and updated them. But if I were to do them today, they would be higher, no doubt, because that did not incorporate agentic AI. Agentic AI is not AGI, in my opinion, but it’s pretty impressive. On the other hand, I think the diffusion assumptions made in that paper are still pretty much in the money. The diffusion assumption was that about a quarter of what can theoretically be done will be done economy-wide. Now, we’re actually well below that, because the applications aren’t there, as I said. With agentic AI, there’s much more that can be done. But look at the production process: there are very few companies — large companies, or even medium-sized companies — that have integrated AI deeply into their production process. So it’s going to be a race between the model advances and the applications.

  [00:40:28] Andrey: What do you think is the biggest bottleneck? A lot of people have proposed many bottlenecks. There are organizational bottlenecks; there’s “we don’t have enough computer chips”; “we don’t have enough energy.” Do you have one that you prefer?

  Daron: I think three. Three bottlenecks are really important. One is we don’t have enough engineers. Again, the Chinese experience is that any one of these technologies — robots or AI — integrated is so engineering-intensive. So many hours of actual hands-on engineering you need to do. We have a real engineering shortage in this country, and all of the people who actually understand generative AI are working in the generative AI companies. So there’s a real engineering shortage. Second, organizational constraints are real, absolutely one hundred percent. And they are different for large companies, which are bureaucratic, and small and medium-sized companies, which are stretched too thin. But a third — and this is where my uncertainty is, this is why I’ve always emphasized it’s so uncertain — is what a couple of people in industry and a couple of people in academia are now calling the jagged frontier of AI.

  Seth: Ah, the Josh Gans term.

  Daron: I thought it was Ethan Mollick, but it might be —

  Seth: Oh, he stole it from Ethan? Of course.

  Andrey: Yeah, he stole it from Ethan.

  [00:41:54] Daron: Okay. But the jagged frontier is also the reason why productivity improvements are going to be very difficult. So that’s why I wouldn’t say, “Oh, my earlier estimates are completely wrong.”

  Seth: It’s because sectors are gross complements with each other, or businesses are gross complements to each other.

  Daron: Each one of them has a different challenge. But the jagged frontier — especially if where the jaggedness is, is uncertain — means you still need the humans. So you need both the AI and the humans, so you’re not saving that much.

  Seth: It also sounds like we’re complementing humans, then. The story you just told about all these bottlenecks seems very optimistic about wages and labor share, right?

  Daron: Oh, absolutely. My estimates in that paper — again, all of it is uncertain — did not find huge negative effects at all on wages.

  [00:42:42] Seth: One last comment about uncertainty and “The Simple Macroeconomics of AI.” You talked on Tyler’s show about having a methodological point, an empirical point, and a rhetorical point. The rhetorical point that I felt wasn’t made, and maybe should have been, is that you’re multiplying four numbers together, each of which is highly uncertain. Maybe the rhetorical point should have just been: when you multiply, you add the variances, so the confidence intervals should actually be pretty big.

  Daron: One hundred percent. I was just leaving you to make that point.

  ---

  Turkey, Memes, and The Economist [43:25 – 47:56]

  [00:43:25] Andrey: Do you have any thoughts on AI in Turkey? We hear less about how non-American companies are adopting AI, or even how the government is adopting it.

  Daron: There are three layers there. One is that the public is using AI a lot. Because of press censorship, a lot of young people were on social media, on the internet, much more than you would have expected at that level of income, so that’s facilitated it. There’s a lot of use of AI. But second, with the exception of a few leading companies, most companies are behind the frontier, and they’re not really ready for integrating AI, or even for the AI age. Third, the government is using more and more data collection and AI — not frontier AI — and some of it is good and some of it is bad. The good stuff is that, while it’s not perfect, there are many more services that are now online. Turkey was famous for having to go from one office to another to get even a stamp on your passport. Some of these things have become much easier. On the other hand, Turkey has its own surveillance state.

  [00:44:44] Seth: All right, here’s a fun one. What do you think of Acemoglu memes, and what is your favorite meme? Do you get asked this on all the podcasts?

  Daron: No, it’s the first time I’m asked, Seth. And I don’t follow them, so I don’t have a favorite one.

  Seth: You don’t follow — oh, very respectable.

  Daron: I am touched that somebody has spent their time on these things.

  Seth: All right, I’m going to tell you my favorite three.

  Daron: Okay. Tell me. Please tell me.

  Seth: The law of large numbers has been renamed the law of Acemoglu citation count.

  Daron: Wow, that’s a good one.

  Seth: That was a good one. “Nations fail, Acemoglu does not.”

  Daron: I had heard that one. The first one I had not heard, actually.

  Seth: My actual favorite is “Prices take Daron Acemoglu for granted.”

  Daron: Oh, yes. I heard that one. That’s right. Okay.

  [00:45:31] Seth: Okay, but now a follow-up question. So you’ve gotten a lot of love, but you’ve also attracted some recent hate, including this really interesting Economist article — I don’t know if “interesting” is the word. Here’s kind of a meta question for you. One of the themes in What Happened to Liberal Democracy?, and a theme that I just see ambiently right now, is people increasingly concerned about superstars and very-top-percent inequality in general. Do you think that this ressentiment that is coming towards you is just general anxiety against superstars, and you’re just the economic superstar?

  Daron: No. Absolutely not. Look, my book sales fall way short of Yuval Noah Harari’s book sales. And the following week, they had a very, very long interview with Harari, eulogizing him. So it’s obviously not going after everybody who is doing a little bit of visible work.

  Seth: Sure, but I’ve never accused you of being a superstar in writing political books. You’re the superstar of economics.

  [00:46:49] Daron: I see. I don’t know. I don’t think so. But let me say the following. First of all, I see that as a sort of hit piece. But I also consider myself fortunate, because if you’re going to have a hit piece against you, you’d better pray that it’s an incompetent one. And that was a really incompetent one. They did not even go and interview the people they cite. What kind of journalism is that?

  Seth: It was very insinuating. “I had people behind his back say mean things about him.”

  Daron: And the criticisms are just laughable. So I’m baffled exactly what dynamics led to that, and what dynamics led to other frictions I’ve had with The Economist.

  Seth: But just coming back to the theme of your book, which is resentment of elites — do you see no connection there?

  Daron: No, I don’t. If “elite” is access, well, The Economist has a million times more access than I have. They’re the elite. I’m the working class.

  Seth: There we go. Everybody’s always punching up, Andrey.

  ---

  Closing [47:56 – 49:03]

  [00:47:56] Andrey: Any final thoughts for our listeners?

  Daron: Well, this was so much fun. I’ll come back again if I have anything else to say.

  Seth: Dude, we didn’t get through half of the things we wanted to talk about. I apologize we didn’t get to talk about your book a lot.

  Daron: That’s fine. I’ve had many podcasts about the book, so it’s okay. And my guess is that the topic of AI won’t quite go away, so we’ll have other opportunities.

  Seth: Perennial. Daron, thank you so, so much for your time. As I said before we got on the line, it’s been a real honor. We’ve always been working in your shadow, and we look forward to what you come out with next.

  Daron: Thank you, Seth. Thank you, Andrey. This was truly delightful. Thank you.

  [00:48:42] Andrey: Thank you for joining us for another episode of Justified Posteriors. As a reminder, we’ve registered all of our priors and posteriors on our website, justifiedposteriors.com, so please check it out.

Relevant Links

• Martin Beraja’s website and X account

• “Growth at the Resource Frontier” — with Noam Yuchtman, March 2026 (the Summer Institute paper)

• “Data-intensive Innovation and the State: Evidence from AI Firms in China” — with David Yang and Noam Yuchtman, Review of Economic Studies, 2023 (published version)

• “AI-tocracy” — with Andrew Kao, David Yang, and Noam Yuchtman, Quarterly Journal of Economics, 2023 (NBER version)

• “The Value of Organizational Learning Technologies” — with Eduard Talamás, March 2026

• “Inefficient Automation” — with Nathan Zorzi, Review of Economic Studies, 2024

• “Fast and Slow Technological Transitions” — with Rodrigo Adão and Nitya Pandalai-Nayar, JPE Macro, 2024

• “Exporting the Surveillance State via Trade in AI” — with Kao, Yang, and Yuchtman, 2025

• The New York Times Co. v. Microsoft Corp. et al. — the complaint that started the argument in London; OpenAI’s response

• The Green Mountain Boys and Ethan Allen — America’s tradition of appropriative innovation

• “Pacing the Frontier” — the July 2026 statement from frontier-lab employees on coordinated slowdown tools

Timestamps

(01:10) “Growth at the Resource Frontier” — what the property-rights tradition misses
(09:13) Why not just Coase it? DuPont, Conoco, and the plastics counterexample
(15:10) Ethan Allen, the Green Mountain Boys, and conflict at the literal frontier
(25:25) “Data-intensive Innovation and the State” — surveillance data and commercial AI
(34:54) “AI-tocracy”
(40:40) “The Value of Organizational Learning Technologies”
(48:19) If entrants were born old, US GDP would roughly double
(59:26) Lightning round

Transcript

Martin: The demand for surveillance that the Chinese state had was allowing firms to have access to large quantities of data that they wouldn’t have had otherwise, and hence allowed them to innovate more in this type of technology — facial recognition — that needed a lot of data to be successful.

Andrey: I’m gonna push back on this.

Martin: I like that Seth is the good cop and Andrey is the bad cop in this. [laughs]

Seth: Welcome to Justified Posteriors. Today we’re joined by Martin Beraja, a top economist at Berkeley who has exploded onto the scene with a raft of influential macroeconomics of AI papers. He’ll be talking to us about AI-enabled dictatorship, economic growth when no one knows how AI companies are allowed to use the data they hoover up, and so much more. I’m Seth Benzell, always ready to fight to appropriate a share of the world’s knowledge, coming to you from Chapman University in sunny Southern California.

Andrey: And I’m Andrey Fradkin, coming to you from San Francisco, California. Welcome to the show, Martin.

Martin: Hi, Seth, Andrey. Thank you for having me. It’s a pleasure.

(01:10) “Growth at the Resource Frontier” — what the property-rights tradition misses

Seth: It’s lovely to have you on. We saw you present some of this work at NBER Summer Institute, and we were already talking about having you on, but it was so exciting we just had to get you scheduled.

The first thing we wanted to talk to you about is the work you presented there — a paper in progress called “Growth at the Resource Frontier.” Maybe a good place to start us off: what do you think the previous literature misses about economic growth that this paper is designed to capture?

Martin: I would say that something we teach undergrads, and even later on, is the idea that having secure property rights is good for innovation and economic growth. There’s been plenty of evidence and plenty of theoretical models in that tradition. In fact, when you read reports from the IMF or the World Bank or any international organization — I come from Argentina, a middle-income country — a lot of those reports say, “The problem with these countries is that they do not have good property rights institutions.” That’s a very strong tradition in economics.

Then there’s another tradition, more on the innovation side, that asks what incentivizes innovators to innovate. The key idea there is that you need to give innovators some form of monopoly rents, either through patents or through trade secrets. They need to be able to exploit their ideas in the market, and those rents must not be competed away — otherwise they wouldn’t innovate. That idea, that rents in the product market drive innovation, is in basically all the models of innovation we have, in one way or another.

Seth: So what’s missing? [laughs]

Martin: When we started with this paper — it’s kind of the story — we began with a conversation with Noam Yuchtman, my co-author. He’s in London, and I think it was just about when the New York Times had filed its complaint against OpenAI, accusing them of appropriating all the Times articles and using them to train ChatGPT. So we’re debating this, and Noam was more like, “Obviously this is terrible that they appropriated all this.” I’m caricaturing the conversation a bit, but— [laughs]

Seth: No, I love that. Make your co-author the straw man. That’s good.

Martin: Yeah, there’s some drama. And obviously, it was illegal, blah, blah, blah. And I said, “Well, Noam, surely you think the innovation wasn’t just the algorithms behind the LLMs and the training, but also the idea that we could use all this public data” — in this case, public or not, that’s the question — “under some notion of fair use, to train LLMs at scale.” That was the big conceptual and technological breakthrough: that more data was enough, at the end of the day.

So we started there, thinking about what that conceptual breakthrough was. It was the idea that something previously not recognized as valuable — online text, images, whatever was out there — could suddenly become a new resource, namely training data for LLMs. And once we made that connection, we started to realize: hold on, this has been going on forever in human history. A lot of the history of human progress was really about expanding the resource frontier. In fact, historians name ages after their materials — the Iron Age, the Bronze Age—

Seth: Ah.

Martin: —the Stone Age—

Seth: Ah.

Martin: —and so on, up to more modern eras where knowledge became important and, through the Enlightenment, the idea that we could master nature.

Seth: If there were secure property rights over iron ore, the Hittites would have never conquered the Near East.

Martin: Right. Well, we’re gonna get there—

Seth: We’ll get there. [laughs]

Martin: We’ll get there. And then we say: oil was another example, where we brought oil into the resource frontier. The electromagnetic spectrum for radio and TV broadcasting. Data for digital advertising in more recent years. So we started coming up with all these stories and examples of expansions of the resource frontier being really important for human progress.

And once we were there — I think this was our moment of inspiration, if you want — we said, “What’s different about these new resources that previously were not in use and didn’t have any economic value is that no one has really bothered to establish who could own them, who could use them, who they belong to.” Property rights. And the reason was that most of the time it wasn’t worth doing it, or it’s just impossible to think in advance how a particular resource could be used and write all the contracts about exactly how that resource could be used.

Seth: Or maybe I’d add to that: it was also kind of non-excludable. If we’re thinking about electromagnetic spectrum, it’s not excludable until you can make electromagnetic waves, right?

Martin: Well, until you can exclude the frequencies in that case. You can say you can transmit on that frequency and not on that one, by regulation.

Seth: Okay. Fair enough. So excludable in terms of offense. Fair enough.

Andrey: It strikes me that the New York Times example doesn’t fit into this very neatly. One interpretation is that we do have great property rights for New York Times content. It has been recognized for ages that it is very valuable. That is what copyright is. It’s not like someone else can, for free, create newspapers with New York Times content, or display their archive. So what’s happening seems to me a bit more like Uber entering the taxi market without a medallion and hoping for the best.

Seth: It’s regulatory arbitrage, right?

Andrey: It’s regulatory arbitrage. Yeah.

Seth: Right. The New York Times story is: there are clear property rights, you’re not supposed to do this, but Sci-Hub exists and it gets on a torrent site.

Martin: Absolutely. This is why in the paper we don’t talk about the New York Times anymore — for the reason that I think it’s not the best example. That said, one way we end up thinking about it is: the Times articles were obviously valuable for consumption, for readers. What wasn’t understood at the time, because the technology didn’t exist, is whether one could use these articles as training data. And the price of an article as training data is very different from the price of an article for consumption.

So the ambiguity in the property rights is: did I still need to license these articles if I was going to use them for training LLMs? And the answer is, if it wasn’t at scale, yes — I could learn something from the Times articles and then teach it in class. That fell under fair use. So really the ambiguity here is about the concept of fair use in a world where intellectual goods can be used at scale to train large language models.

(09:13) Why not just Coase it? DuPont, Conoco, and the plastics counterexample

Andrey: So let’s think a little bit about the counterfactual. Let’s say OpenAI respected the property right. Do you think that would have substantially slowed down the progress of LLMs? Don’t we believe in Coase’s arguments—

Seth: Yeah, what about Coase?

Andrey: [laughs] —that we would come to an efficient agreement? I don’t know.

Martin: I do not. In this case, I do not. Once you understand that whenever you’re innovating to use new resources, and these resources do not have clearly defined property rights — which does not mean there were no property rights, just that they’re ambiguous enough that it could lead to ex post contestation if I try to appropriate them — you’re right to ask the question: why, instead of just taking and then hiring lawyers ex post, or figuring it out with the government, why not try to contract with the parties that have a potential claim?

Honestly, history has seen both. There are examples where innovators brought new resources into use but contracted with the claimants. One example is plastics. The way it worked was that DuPont, one of the main innovators behind plastics, ended up— So plastics are made out of waste from oil refining. That was one example where obviously the waste wasn’t valuable — hence the word “waste” — for refineries like Conoco. But the waste clearly belonged to Conoco. So it’s not that if DuPont had gone and taken that waste out of Conoco’s deposits, that would have been all right. They would have lost the lawsuit. Instead they contracted. They vertically integrated with Conoco to produce plastics with their waste.

So the question is: why wasn’t that the case in these other examples — the electromagnetic spectrum, where people just started transmitting; or data for digital advertising, where platforms just appropriated; or the New York Times, and so on?

Andrey: Those are completely different cases, right?

Martin: How so?

Andrey: Well, no one owned the electromagnetic spectrum.

Martin: Yeah.

Andrey: So that one is truly a case where we don’t have property rights at all. The case with the waste is pretty clear who owned it, so it was never a question — it was just a question of contract. And the New York Times, in my mind, it’s pretty clear who owns the rights to the content. It’s the New York Times. And then the data — consumers are willing to give up their data for nothing. Experiment after experiment shows they don’t value the data. I don’t see some sort of nefarious uncertainty-grabbing from the company. It’s more just that you collect data from users, even before the internet — you look at who’s coming into your store and what they like to buy.

Seth: I don’t think Martin’s got a problem in this model with there being a continuum from better-defined to less well-defined property rights.

Andrey: Yeah.

Seth: Theoretically, we didn’t need to pass a regulation about electromagnetic spectrum. Common law would have come to an answer. It just would have been a very uncertain answer. There would have been an answer before a judge eventually.

Andrey: Well, I think the electromagnetic spectrum is the clearest example for what you’re arguing. I’m just disputing that the other cases are what you’re arguing.

Seth: Fair enough. Martin, feel free to answer, but I think your answer would just be that it’s a continuum of how well-defined the right is.

Martin: I think different people will disagree on how obviously defined property rights over data were. In fact, we had a lot of conflict and litigation and regulation around it, I think precisely because of that ambiguity. But I agree there’s a continuum from very clearly defined to not so clearly defined, and these technologies fall in different parts of that spectrum.

Seth: Now let me ask the better version of Andrey’s question.

Martin: Yeah.

Seth: Clearly in some situations we have very well-defined property rights, and in some situations less well-defined ones. Is the spicy take of your paper that we might get more growth and innovation if we had less well-defined property rights?

Martin: Yes and no.

Seth: [laughs]

Martin: No, in the sense that the point of the paper is not— I don’t think we would ever say having secure property rights is bad for innovation.

Seth: I remember there’s a result in your paper that if you’re below the optimal rate of growth and monopoly rents aren’t out of control, you want the innovator to be able to appropriate. Did I misread that?

Martin: Correct. But that’s different. Obviously you want secure property rights ex post. Innovators must be able to innovate and then reap the returns from that innovation. What we’re saying here is really not normative. It’s a question about how you design institutions to be more or less permissive toward this type of appropriation.

Do you want institutions where, whenever we see anything that looks appropriative and conflictual, instantaneously you give the right to the counterclaimant and not the innovator? Versus institutions that are a bit more permissive, as they have been in the US for most of history, where again and again these instances of appropriation have been legitimized ex post by the courts and the regulators in some form. In that world, there’s a question of a continuum: where do you want to fall on the line of how much appropriation you allow? And the answer may not be zero. Every time we see anything appropriative, we’re just going to shut it down — that would be bad for growth.

(15:10) Ethan Allen, the Green Mountain Boys, and conflict at the literal frontier

Seth: I believe this is part of a long American tradition. I was recently visiting Vermont, where I learned about Ethan Allen and the Green Mountain Boys. Are you familiar with these guys?

Martin: I am not, no.

Seth: Okay.

Martin: I mean, you told me about it before, but— [laughs]

Seth: [laughs] I think this illustrates both the excitement of your approach and maybe a factor that’s left out of your model of conflict. Vermont was not a state. It was not one of the original 13 colonies. There was a dispute between New Hampshire and New York about whether the territory that is now Vermont would be part of the two states. New York sold land grants to the area, and New Hampshire sold land grants to the area.

Ethan Allen was a real character — free thinker, deist, wrote a book I was skimming the other day that I would describe as Hume, if Hume were a much worse writer [laughs], trying to talk about reason above faith. Anyway, he basically puts together a mob called the Green Mountain Boys and starts harassing people who had bought land under the New York land grants, because they themselves had bought under the much cheaper New Hampshire grants. So they’re basically a reckless mob attacking New Yorkers. Like most good things in America, this is a tradition that comes out of people trying to avoid New York state property taxes.

That institution, the Green Mountain Boys, during the Revolutionary War ends up joining the Revolutionary Army. They seize Fort Ticonderoga, an important early battle. So the innovation Ethan Allen made in reckless violence ended up contributing to the growth of America. Okay — so what do I think this example indicates?

Andrey: What do you mean, it contributed to the growth of America? You have no evidence of that.

Seth: What? They won an important battle, this mob.

Andrey: Yeah.

Seth: If he hadn’t made the investment in the complementary good of local civic defense, maybe we don’t have an America today.

Andrey: You don’t see the counterfactual.

Seth: Well, of course. Yeah, dude.

Andrey: [laughs] I know, I don’t understand.

Seth: Wow. Sorry, man. We don’t have an America.

Andrey: I’m just saying that if Vermont was part of New York and that was universally acknowledged, maybe the politicians in New York would have organized to defend. I feel like it’s a bit of a stretch of an argument.

Seth: Okay, well, believe it or not, that’s actually where I’m going with this, Andrey. It’s unclear whether you’d want to use Martin’s framework for thinking about this question of unsecured property rights literally at the frontier. This is the frontier. There are literally unsecured property rights. And do we want to think about unsecured property rights as leading to lawless violence, which is destructive?

One thing, Martin, that I didn’t see in this model is conflict as destructive. Usually when economists think about conflict, it’s destructive — all-pay auctions where we end up wasting resources. So I didn’t see that in the model. Is that a natural extension? We’re not seeing people literally show up with militias to fight over their electromagnetic spectrum, but paying for lawyers can certainly be destructive waste.

Martin: No, absolutely. There are a number of things once you go to the normative. That’s why the paper so far is about the idea, and more on the positive side — trying to make predictions and describe this process of growth. Once you get to the normative part, you need to start thinking about whether conflict is inherently destructive, whether there’s some destruction of resources in the process of conflict.

And the question Andrey was alluding to as well, which goes back to the traditions in intellectual property protection: if the resource is not just magically there but actually needs to be created somehow — and I think data is much closer to that, where there needs to be effort put into creating the content that later on may or may not be appropriated. All those things would move the optimal policy toward less lax permission for appropriation than you otherwise would have.

Now, does it move you all the way to no appropriation? In my mind, no. And so now it’s a question of quantifying these trade-offs, as we usually do in economics. The same way that for patents we don’t say patents should be infinitely lived, nor do we say there should be no patents whatsoever — there is some optimal patent length. I think this is similar.

Andrey: Just a question about that argument. Let’s say we did have defined property rights, but a company just chose to break the law anyway, with the rational calculation that eventually they’ll settle in court. That process might take some years, but by that point the uncertainty about the value of their seizure of the resource will have been resolved. Is that inconsistent with your model? So the property rights are defined, but you’re just going to say “screw it” and go for it anyway.

Martin: There we are much closer to the case where property rights are really well-defined, and then it looks much more like expropriation than appropriation. I guess that’s the difference in my mind. Whereas in the case of truly novel resources — and again, the world is complicated — that’s not the case, because the novelty is what makes the property rights ill-defined to begin with. So the question in reality is: to what extent is the resource truly novel and not owned by someone before?

Andrey: Maybe I’m anchoring back to the New York Times, which you said is not what you want to be talking about, but that’s what I want to be talking about. OpenAI probably sat in a room and said, “We’re probably gonna get sued. We’re gonna do it anyway. We’ll settle at a reasonable price, and we’ll have a better sense of how much it’s worth three years from now, once the model is trained.” To me that really doesn’t seem like uncertainty over property rights. It just seems like a rational calculation. There’s uncertainty over the final bargain.

Martin: Yeah. Imagine that all the value of OpenAI was just this training data. And imagine it wasn’t just the New York Times — imagine it was collective litigation where you had every single piece of data from the internet, everyone behind it. Now we’re talking about whether OpenAI could essentially be expropriated of their company. That’s what would be decided in court. If you’re thinking about that calculation, it’s very different.

Andrey: Yes. Yeah.

Martin: Of course, if you’re thinking about just one piece of data, like the New York Times, the argument you’d make in court is, “Look, the Times data wasn’t pivotal for training ChatGPT,” which is obviously true. So I should compensate these people somehow, but not with the full value of my company, because I have good substitutes — which I appropriated from somewhere else.

One example where that’s not the case, which is interesting, is YouTube videos. When you train on YouTube videos that were already subtitled, you wouldn’t say there are good substitutes for YouTube. Somehow that litigation has not happened. There’s no court case, YouTube against any of these companies, for obvious reasons.

Seth: Wait, what’s the obvious reason? [laughs]

Martin: Well, some of them own them. [laughs] Essentially, some of them own them.

Andrey: Why wouldn’t Google sue OpenAI over training on YouTube videos?

Martin: I don’t know. I imagine that could have knock-on effects where they don’t want that to become—

Seth: On net, Google wants appropriation. [laughs]

Martin: Yeah, so they don’t want to start suing each other, I imagine.

Seth: They should sue and then intentionally lose to set the case law their way. Galaxy brain.

Martin: [laughs] But look, you have the view that the New York Times case is clear-cut. I imagine OpenAI had the view that “we may pay something, but this is sufficiently valuable to us that it’s worth it.” That’s the rational calculation. So it’s really a question of how strong the penalties are ex post and how much they’ll have to pay.

There are plenty of reasons why they may choose not to contract. Some are obvious — first-mover advantage, or the fact that you’d have to contract with every possible claimant, which would be a nightmare and impossible. You’d never launch the product. And there’s another reason that’s plausible: deniability. Imagine you went out and tried to contract with the New York Times, Reddit, Wikipedia, and they said, “No, you can’t have our data.” And then you still go ahead and do it. That would travel badly in court ex post. They’d say, “Look, they tried to do this, we said no, they still did it.” Not good.

Andrey: It’s also interesting, because I do think they have now contracted with the New York Times, right?

Martin: They have now done so, yes, for the new stuff. But part of it is that in these bargaining situations, when the two parties don’t exactly know what it is that they have, the bargaining can become hard. The New York Times cannot really assess what the value of their own data is, and OpenAI at the time maybe didn’t know exactly what they had. They thought it was great — but how great?

Seth: So this is another thing that’s not explicitly modeled but you’d imagine as an elaboration. Crisis bargaining models suggest you fight when there’s disagreement about the underlying value of the thing you’re fighting over—

Martin: Yeah.

Seth: —and the likelihood of winning, right?

Martin: Yep. And that’s one reason you’d choose not to contract. There’s no way to contract, and then the decision becomes: either I don’t develop the technology, or I go ahead and appropriate, and we figure it out in court ex post.

(25:25) “Data-intensive Innovation and the State” — surveillance data and commercial AI

Andrey: Fascinating discussion. Now we’ll go to a domain where property rights are very different [laughs] — namely, China. You’ve done a bunch of work on AI in China. I’m wondering whether you can first summarize the main lessons you’ve learned from that stream of work.

Martin: This is work also with Noam Yuchtman, and with David Yang. We started off when David was visiting MIT. We were lucky — one of those random things. He was just in the office next door. So we started chatting. He of course knew a lot about China, being from there and having studied it. And we started thinking: this is a great place to study how the state and innovation are intertwined, because it has been so pervasive in modern history, and where we’d have good data.

The initial idea was to think about how the fact that the Chinese state cares about AI — and when I say AI here, this meant facial recognition, not the fancy things we have now — is influencing the direction of innovation. That was a very early idea, and we said, okay, this is too big a question. So then we started thinking more narrowly about what exactly we could talk about.

The first paper we wrote was on how Chinese facial-recognition AI firms with access to government data — in particular surveillance video feeds from cameras on the streets — actually got a competitive edge in innovation. We collected data on cameras on the streets to construct a measure of how much data these firms were getting access to, and on the contracts these firms had with municipal police departments. This was more of a positive paper, in the sense of saying: the demand for surveillance that the Chinese state had was allowing firms access to large quantities of data they wouldn’t have had otherwise, and hence allowed them to innovate more in this type of technology, facial recognition, that needed a lot of data to be successful.

Andrey: So I’m gonna push back on this.

Martin: I like that Seth is the good cop and Andrey is the bad cop in this. [laughs]

Andrey: So there’s the simplistic claim that this government data allows for much better facial recognition technology. If I think about it from first principles — yes, more data is better, but the bottleneck to facial recognition is probably more the mapping between an individual and the video, rather than just the video itself. That’s one part. And two, it doesn’t seem like it’s that hard of a problem. The reason the West wasn’t doing it was simply that it was illegal to do it. And in fact, I’m sure the West was doing it in its own way — the CIA, the NSA. It’s not clear this is a very difficult technology to begin with. So why don’t you speak to those?

Martin: Well, I can speak to the evidence that we find. [laughs] The question is: are there very good substitutes for this data in private markets? If the answer were yes, you wouldn’t see any effect from firms contracting with the state and getting access to this data versus not.

That said, we can’t really know — we never see how much data they were actually getting access to. We only saw proxies, like how large the surveillance network was in the municipal police departments they contracted with. Now, on whether they’re mapping a face to a name: we don’t know that, but I imagine the answer is yes, because these were municipal police departments that already had the list of people’s names, IDs, and so on.

Andrey: And what is your measure of innovation here? Is it just market success, or something else?

Martin: It’s very crude. It’s just numbers of products, really.

Andrey: So isn’t it — let’s say I win a contract with the local government. Now I have capital. I can continue my business. And, given how business is done in China, I probably have other ways of getting additional capital as a result of that. Maybe some kickbacks.

Martin: Were you referring to Andrey in the paper? [laughs] These are obviously the concerns about identification. The things we can control for are capital, for instance — we know how large the contracts were in monetary terms, so those are easy to control for.

Market access was something we really worried about: maybe if you get a contract, that gives you reputation, and the reason you start innovating more and producing more is because now you’re the firm that got a contract. We did a couple of things for that. First, look at firms that were already well-established and producing a lot before, for which these effects would be mitigated — we saw basically the same results. Second, look at second or third contracts instead of just the first, where reputation effects would be weaker.

Andrey: But how can you identify the causal effect of a second or third contract? They already got the first contract.

Martin: Oh, right. Of course, it’s not perfect. You need to control for getting the first contract in that case. All of these are imperfect ways of trying to look at the same thing.

Andrey: All right. So I want to take a step back. I’m the microeconomist here, but one of the themes of the show is how micro evidence doesn’t teach us as much as we hope.

Martin: [laughs]

Andrey: [laughs] And you’re a macroeconomist, so I have no problem with the scholarship, Martin.

Martin: But I dabbled in applied microeconomics.

Seth: Dabbled. Dabbled.

Andrey: Dabbled. Maybe we all dabble in the QJE. [laughs] One of the claims is that having access to government data allows you to innovate more with the data. In some sense, obviously true — but the question is how much. And looking at this specific sub-technology, how much does it really inform us? Like a lot of things related to AI, it’s just so obviously swamped by what has happened since. It’s kind of even funny to think that whatever was going on in 2017 in facial recognition has anything to do with AI. [laughs]

Seth: Right. You must admit, Martin, the title is a bit bold — AI-tocracy. If we’re just talking about facial recognition, that seems like a very small piece of the puzzle.

Martin: Right, and that was the second paper. So let me just say — I agree with you, Andrey, and I would say this about a lot of applied micro papers. Is the channel here obvious qualitatively? Yes. You would think more data is better for innovation in things where you need data to train the models. So the question was basically how much.

And there was something about testing this idea that data is non-rival, which in theory is obviously true, but in practice we didn’t know. We found that the effects were driven by firms innovating in commercial products, even when the data was surveillance feed. The typical example: you get a contract to provide services to a municipal police department to process their video surveillance feeds, and then you create a product for the commercial market that allows supermarkets to track people as they move along aisles, to know where you need to restock.

Seth: Wait, you’re telling me that the company that has influential political and police connections in China also has an advantage in the private market?

Martin: For producing these types of products, yes. [laughs]

Seth: [laughs] Well, maybe data has something to do with that. [laughs]

Martin: Maybe, or maybe it’s reputation. I agree with you. I think we did a good job trying to convince ourselves and others that it wasn’t the whole story. But as always, it’s not papers, it’s literatures. And then the autocracy paper — that’s the bolder paper.

(34:54) “AI-tocracy” — is AI a centralizing technology?

Andrey: So this is a claim that’s made a lot. I think Peter Thiel has famously made it: that AI is a centralizing technology, and this benefits authoritarian governments. Do you think that’s true? How big of an effect is this?

Seth: And who is the Antichrist? [laughs]

Martin: [laughs] So the bold claim — it’s a conjecture, I guess. And somehow I keep writing papers that go against everything Daron Acemoglu says, who was my colleague and whom I love. [laughs] The other one was also—

Andrey: He’s our next podcast guest.

Martin: Yes.

Seth: Next guest.

Martin: Okay, well, he knows. [laughs] That’s perfect sequencing, actually.

So: the tradition in economics is that autocratic regimes are bad for innovation. The reason being that property rights are insecure, innovators are afraid of being expropriated by the autocrat, and even if they become wealthy, they can be incarcerated or killed because they become a threat to the regime. All these things essentially tax innovation under autocracy. I very much agree with all of that.

In the AI-tocracy paper, what we said is: there’s a countervailing point, that sometimes we’ve seen autocracies actually innovate a lot, at least in particular technological domains. Russia is an obvious example.

Andrey: In terms of poisons for assassinations? [laughs]

Martin: [laughs] I would say in terms of space technology.

Andrey: Oh, okay. [laughs]

Martin: Imperial Germany is another example of a less-than-democratic regime — in that case it was more about chemicals. Those observations don’t quite fit the other story.

So the story we tell, and some of the evidence we find: first, we do see that in those places in China adopting more facial recognition technology, these are precisely the places where we were seeing more protests before. That’s a revealed-preference argument that the reason municipal police departments were adopting facial recognition is to squash different types of protests in those regions.

Seth: This is 1984. The only innovation that is allowed is innovation in technologies of social control.

Martin: Right. Now, if that’s the whole story, then you would see very narrow innovation — innovation in this particular technology would not spill over to other parts of the economy. The conjecture in the autocracy paper is that if there are indeed spillovers, either in terms of the data you accumulated, which was the point of the previous paper, or more traditional spillovers in terms of ideas, then innovating a lot for surveillance technology because the government demands it may end up boosting innovation more broadly in the rest of the economy.

And you get into a feedback loop where there’s more innovation in this other part of the economy, which again makes the surveillance technology better. So instead of autocratic regimes being detrimental to growth, you get that in fact they boost it. And instead of growth being destabilizing for autocratic regimes, they’re stabilizing, because they produce the very technology that allows them to repress dissent. That’s the story in that paper.

Andrey: I’m gonna make the same comment here. In some sense I buy the argument, but I just don’t update at all from the data. Okay, so there’s a protest and then those local places get more facial recognition. Sure. China doesn’t have any problems with facial recognition to begin with. I’m sure the federal government of China is using facial recognition a lot already, so it’s really a question about local police departments. And in terms of innovation in AI, it’s hard to imagine that the important dimension of innovation here is facial recognition.

Seth: Yeah, let me tell a story. You’re a democracy, right? That means you have a lot of political parties always trying to innovate in tools of social persuasion. So places that have close elections get more invested in tools of social persuasion, and then there are spillovers from that public good that boost adjacent innovation.

Martin: Again, as an applied microeconomist, one needs to be able to extrapolate a bit [laughs] from the particular context we’re studying to a broader type of story. Seth mentioned one. In China, obviously, control over social media and the digital world more generally is important, and AI has been a big part of that too, and will continue to be.

So the broader story is that innovations in technologies of social control that then spill over into other domains can create feedbacks by which the regime both becomes more stable and grows. And at which point the more traditional autocratic tax on innovation starts to kick in — that’s more of a quantitative question. I don’t know. The two things are both there, and it’d be impossible to quantify, I would say.

Seth: All right. Impossible to quantify.

Martin: Yeah.

Seth: Let’s move on to the next topic.

Andrey: [laughs] That’s a theme of our podcast.

Seth: The question that we care about is the one we can’t know.

(40:40) “The Value of Organizational Learning Technologies” — AI as a codification technology

Seth: So, Martin, excellent stuff. I’m now going to ask you about another paper that has a really clever theoretical argument about economic growth, but some challenges in bringing it to the data: your paper “The Value of Organizational Learning Technologies,” with Eduard Talamás. I’ve seen him present it. It’s a really cool paper. The reason I love it is that it’s attacking a really underappreciated angle of how AI may change firms. Everybody wants to talk about automation, but that’s a little overexposed compared to this idea that AI might help organizational learning, or what I might call managerial expertise. So how do you think about organizational learning or managerial expertise in this paper, and why do you think AI is going to help with it?

Martin: I’m really excited about this line of work too. I think there’s a lot to be done at the intersection of AI and org econ — where you start thinking about AI as not just a fancy robot. By the way, the title of the paper is now “The Value of Codifying Organizational Knowledge,” and it will become clear why in a second.

The idea is that organizations and firms take a long time to grow, to scale, because there’s a process of trial and error and experimentation they all have to go through until they figure out exactly what the right processes are, what the right products are, who the customer is, who the market is, how to expand, how to hire, how to price. All these business decisions take time to learn how to do well for the particular thing a business is producing and selling.

Seth: Let me pause you right there to understand why it takes firms a long time to grow. In our macroeconomic models we usually have some parameter for firms growing over time — a capital adjustment cost, which explains why average Q is different from marginal Q. When somebody says there is an adjustment cost in capital, are we talking about something like the learning you’re describing, or are we talking about the fact that when you decide to build a factory, it’s still going to be five years until you get the factory?

Martin: It’s neither of those two, actually. The factory taking five years is more like an adjustment cost — a time lag for building things. The way I think about this is more a form of learning-by-doing and experimentation that firms go through to increase their productivity. It doesn’t show up in the physical investments you see. It shows up in productivity, or in the quality of the products the firms have.

Seth: I guess what I’m pointing at is that the one parameter we have in a lot of models is just a capital adjustment cost. And what you’d say is: some of that’s organizational learning, and some of it is that it takes five years to build the factory.

Martin: Yeah, if you bundle everything under “capital, intangible and tangible,” then yes. But we do have models of that intangible buildup that have to do with learning as well.

Now, the question that comes up: different from capital — which you don’t need to build yourself, you can rent it in the market or buy it from a different firm — this idea of organizational capital or knowledge is something the firm needs to accumulate internally. It cannot just buy it from the market. That’s why it takes so many decades to build up that knowledge.

That said, we have had, in different eras, different technologies that can codify that knowledge and then diffuse it to the rest of the economy. One very early codification technology was manuals — scientific manuals. Someone figured out how to do a particular chemical process, and by writing the manual of how to do that, the knowledge accumulated through that person’s learning could be codified, because they could spell it out and transfer it to everyone else in the economy. Later on we had scientific management in factories. And later, actual management — the idea that management is a technology that can translate best practices across businesses.

And franchising. I love franchising as an example of a codification technology, because it’s literally that. McDonald’s took a long time to figure out exactly how to produce a Big Mac. But once they figured it out, they could write down the instructions for how you set up a new McDonald’s.

Seth: There’s a binder. You go and open up your franchise, you get the binder, and the binder gives you the step-by-step of what to do.

Martin: Exactly.

Seth: I think it’s optimistic if you think that chemistry instructions are at that level of instructiveness, but— [both laugh]

Martin: So now the new McDonald’s that opens up doesn’t need to go through the same process of learning. It can be born as an old, experienced McDonald’s.

That said, these codification technologies still remain pretty imperfect, because they require humans to go ahead and spell out explicitly and describe what’s going on. That process can be very hard. That’s why we still see learning curves in many industries.

So the conjecture in the paper is: AI is somewhat different from these other codification technologies, in that it does not require a human to explicitly say in natural language what’s going on. Instead, it can learn from instances and experiences of the firm that are captured in operational logs, emails, different parts of the organization that probably need to be digitized. And if new AI tools can learn from this type of operational data, then that is knowledge that can be codified and transferred to newcomers.

Seth: Let me pause you there again. You’re emphasizing the ability of AIs to replicate the knowledge that’s already inside the firm. But one of your thought experiments is: what if you could raise each firm’s productivity to its expected long-term productivity immediately? To me, that doesn’t sound like “we get some knowledge and then spread it faster.” That sounds like the AI actually helps us do the iteration and makes the right decision without the experimentation.

Martin: Great question. There are two parts to this, and that’s why I was building up.

One is a blunt experiment we do. Forget about AI. Imagine we could somehow codify all the knowledge that mature firms have and just give it to new entrants right away. Instead of taking ten years to build up, they’d start with it. There would be no learning curves at all. What’s the value of that? We can quantify that by looking at actual learning curves of firms in the US economy — in particular, how large a mature firm is compared to a new firm.

Seth: We’ll come back to how plausible this is.

Martin: Yes.

Seth: Keep going. [laughs]

(48:19) The number: if entrants were born old, US GDP would roughly double

Martin: So mature firms are about seven times larger, and they also exit much less, which is something else that contributes to increasing total output in the economy. So think about that ridiculous experiment. That number tells you: if you just did that, and new entrants were born old — like a forty-year-old firm — then US GDP would about double.

Seth: [whistles]

Martin: That’s a huge number, and we think of it as a bound.

Now, what does AI have to do with all of this, at least in the long run? Here’s where we talk about a way of implementing this in practice with an AI system. And it doesn’t need to learn from the forty-year-old firms. All you need is the following.

Imagine that each year you set up a system that learns from the data that one-year-old firms produce. Each year a new cohort of firms comes in. They’re one year in. They produce some data. Some of them succeeded, some failed and exited, some did really well. From all of that data, there must be some information about what are good business practices and bad business practices, things that work and things that don’t. Imagine — this is the conjecture, we don’t know this — that AI could distill that knowledge just one year ahead. I’m not talking about twenty or forty. One year ahead: what makes a startup grow past their first year versus just be a failure?

Now you pass that knowledge to the new entrants in the next cohort. Repeat. Now the new cohort enters with the experience of a one-year-old firm. So it grows again, and that new cohort goes from being like a one-year-old firm to like a two-year-old firm, but only in their first year. So now set up the system again. Learn from that new data, pass it on to the newcomers, and so on and so forth.

So just the existence of this technology that can learn from data of firms one year ahead eventually manages to codify all the organizational knowledge in the economy. And you get to that same ridiculous exercise, where maybe in fifty years the newcomers are born with the experience of very old firms in the economy.

Seth: I think you’re thinking a little too small ball here. People are talking about these AIs becoming superhuman. Why couldn’t the AI executive or manager make better decisions than the optimal firm from forty years ago?

Martin: Oh, absolutely. That’s why I say this is what we can quantify, instead of just speculate. This was a paper that was very different from the others. The others had some evidence and then a kind of crazy extrapolation; this is the opposite. Everyone wants to put a number on the macroeconomic impact of AI. If you think about it as a fancy robot, you can get away with some numbers, and there have been papers trying to do that. But if you think about it as what you’re describing — really increasing the productivity of firms in the long run, or improving science, or improving innovation—

Seth: Strategic decision-making.

Martin: —strategic decision-making [laughs] — I just don’t know what the number is, because we don’t have evidence on that, and we won’t for a long time. So our approach was to ask: what is the question we can answer somewhat convincingly, just from data that already exists in the economy, by knowing how important learning and knowledge are in the economy today?

Andrey: Just a question here. Let’s say it’s 2028. What in the data would we see to know if your story is first-order important or not?

Martin: Great question. 2028 may be too early, but let’s say 2030-plus, who knows. What you’d start seeing is a flattening of the age-size profiles. Right now, an entrant is about seven times smaller than a thirty-four-year-old firm. The profile is fairly flat initially, then it starts to kick off at around five to ten years. You’d want to see that profile flatten — old firms and new firms being somewhat similar in size.

Seth: And I love this. This is a decentralizing hypothesis, because a lot of the concentration in the economy comes from older firms.

Martin: Yeah.

Seth: So this is very interesting. If my mechanism is right — it just makes all the managers smarter — you’d expect that, if anything, to boost the big firms more.

Martin: Correct.

Seth: So we’ll bring you back on in five years to see which is happening. [laughs]

Andrey: The other story that would make that prediction a little tricky, and it’s not in your paper really, is that AI makes the optimal firm size smaller.

Seth: Well, it’s productivity. Productivity and size go together, but I think there’s a reading of this where it’s just about productivity.

Martin: Yeah. There’s another question about whether AI changes the span of control. Can a manager now have a firm that’s twice as large because we have ten agents and two workers? That’s another channel, which would lead to more concentration.

Seth: So, some data questions about these extrapolations. One concern Andrey raised about the facial recognition work in China is that maybe some of what you’re measuring isn’t a supply-side effect, it’s a demand-side effect. When I look at your list of businesses where you have the largest premium for being an established business, number two on the list is apparel. Number three is performing arts. Is this age advantage in productivity really about supply-side efficiency, or is it really about a reputation effect?

Martin: Great question. With the data we use, we can’t tell, because we’re using data on size. So really the question is: when we talk about knowledge — such a big word — is this thing creating size differences something AI can help codify and then spread?

If it’s something like status, I don’t think AI can do anything about that, because it’s sort of zero-sum. Either you have status or you don’t. Building reputation — I don’t know, maybe there is some way of building reputation, and if the AI can codify what are good versus bad ways of building reputation, that’s a type of knowledge that can be diffused. And then I’d say, yeah, that’s the right number.

So it need not be just something you’d associate with productivity, like processes inside the company or how you price — operational knowledge. It can be knowledge about how you build a brand. Is that captured in the operational logs and emails companies have? Then you’d say, well, that will not show up in productivity, it will show up in prices and quality. And if that knowledge can be transferred, then I’m fine with that story. But I think we’ll see.

Seth: Yeah, you just don’t want something that’s pure demand side. Anything that’s plausibly controlled from the supply side, you could tell an AI story about how you’d accelerate it.

I guess the final thing is: I led with this idea that maybe older firms are bigger just because it takes time to grow. There are frictions in getting the license to build the factory, capital adjustment. Is that really— I look at the top one on this list, couriers and messengers, and it seems pretty plausible that couriers face long adjustment costs because you’ve got to build out this network, and AI is not really going to accelerate that.

Martin: No, I agree. The main thing we push against is not just capital adjustment costs, but really financial frictions. Maybe small young firms are small not because they’re not productive and haven’t accumulated a lot of knowledge, but simply because they can’t scale up due to financial frictions.

Seth: They don’t have collateral, right?

Martin: Yeah, exactly. So we do an exercise where we imagine that the first ten years are really just a form of capital adjustment cost or financial friction — so a zero-year-old firm and a ten-year-old firm should be the same size, and the only reason they’re not is an adjustment cost. Even then, the numbers we come up with are still pretty sizable. Two is a very large number.

Seth: Final question here. If we get, say, five percentage points of economic growth a year because all of our tech friends are right and AI is a really powerful technology, what percentage of that additional GDP growth is coming from automation versus a mechanism like this? Do you think automation would be relatively more important in a productivity boom, or a mechanism like this?

Martin: You’re asking me to speculate now.

Andrey: Well, yeah, that’s what the listeners want to hear. [laughs]

Seth: That’s right. We want to know.

Martin: [laughs] This is what the audience demands.

In the short to medium run, I think automation, because it’s the easiest thing to implement. Just keep doing what we’re doing, but now with agents, especially in some industries. Where the channel I’m describing really kicks in is as you churn through new companies that can take advantage of setting up their whole processes to be able to learn in this way with these new tools. So we may see this filtering through the economy in ten, twenty, thirty years — not before that, until we really see the new cohorts coming in with a completely different organizational setup that can leverage this technology of codification.

But I would say the reason I started studying this side of AI is because I do think what makes it different is not that it’s an automation technology, or that it can automate cognitive rather than manual and routine tasks. That’s just a different matrix of exposures. I don’t find that the most exciting aspect of AI. The most exciting aspect to me — and we’re all experiencing this already — is that it can somehow codify knowledge from the experiences of others, first people, and then that knowledge becomes available to us or to other organizations, say through a chatbot. That seems really special about AI, and not something a robot could do before, where you needed to give the instructions by hand.

Andrey: Is it fair to say — Seth assumed we get to 5% GDP growth per capita.

Seth: I said conditional. It’s a conditional expectation. [laughs]

Martin: I should have pushed back on that to begin with.

Andrey: Is it fair to say that you don’t think we can get to that level with automation alone?

Martin: Automation of what? Of production? If it’s automation of production, yes. At the end of the day, a lot of this is also automation — you could call this the automation of codification. Before, you needed to actually go ahead, read stuff, document it, write it down, make it explicit. Whereas now you have this machine that can automatically read off all those experiences and come up with something that looks like — it’s not really instructions, but you can chat with the chatbot and it will tell you what to do. It is a form of automation, just not of a task of production, but of how you organize and run businesses.

(59:26) Lightning round, and the Pacing the Frontier letter

Andrey: All right. Let’s move on to our lightning round. First question.

Martin: [laughs] Can I pass? Do I get to pass?

Andrey: No.

Seth: No, don’t pass.

Martin: No passing. [laughs] Okay.

Andrey: What does the Argentina soccer team’s success — or this year’s lack thereof — teach us about efficient organizational learning?

Martin: [laughs] Somehow I haven’t connected those two dots. First of all, I’d say it was a pretty successful year. We made it to the final, so I’d say that’s successful. [laughs]

I think there’s been something about team building — how you manage to create a cohesive team — that our coach, Scaloni, was really good at, especially in the previous one. We were just talking about one of our colleagues, Damián, getting into football and thinking about whether we can use AI to figure out decisions on the pitch. When should we press? When should we stay back? When should we attack? These types of things. That’s the business of football in that case. This is just to get free tickets for the next World Cup. [laughs]

Andrey: Is there anything specific that the coach did that you think made a difference?

Martin: Two things. One, you hear that he really consulted with the players who knew — like Messi — about what the strategy should or should not be. I appreciated that. The second was that in the last World Cup, in the first match, when things didn’t work out, he realized early that he needed to sub in the younger players, disregarding names and reputation. Which is something I think he did not manage to do in this World Cup.

Andrey: Yeah, just from watching the World Cup, that seems like a very interesting consistent tension where some teams decided to bench their old stars and some decided to play them. I couldn’t figure out a pattern of what the optimal strategy was there.

Martin: But with AI, we will. [laughs]

Andrey: With AI, yeah, we’ll figure it out.

Seth: You’ve spent part of your academic career in San Francisco, part in Cambridge, Massachusetts, part in Chicago — all very highly charged intellectual climates. How would you say they differ in how they think about AI in the economy? And are you saltwater or freshwater?

Martin: What’s the name for when the rivers meet the sea? The edge?

Seth: Oh, you’re in the littorals? [laughs]

Martin: Yes. That water.

Seth: You’re in the floodplains. [laughs]

Martin: Yes. [laughs] And truly, I think I am — precisely because I’ve spent so much time in both.

One thing I’ve seen since I’ve been in Berkeley is how the agglomeration externalities are really here when it comes to AI. Even at MIT in Cambridge, where of course people are interested in it and studying it, I did not feel the externalities from the outside world. Whereas here in the Bay Area — just coming to your podcast, even when we’re online, dinners that others have organized, conferences, random encounters with people. You feel this around you in a way that in Cambridge, despite it being intellectually amazing and people being interested, that ecosystem is not quite there.

The only conjecture I have for that is that the venture capitalists are the ones that create the externalities. It’s not the engineers alone. It’s the venture capitalists who are interested in networking and making people come together. I don’t know if that’s your experience too.

Andrey: I think engineer founders are certainly part of this as well. But I’m wondering — one of the worries I have spending time in San Francisco is that maybe it’s biasing me in the wrong direction. We have an intuition that studying things from afar has some advantages, whereas once you’re in the middle of it, you’re hopelessly tied up with everything else going on. Have you thought about this?

Martin: I moved here because I wanted to be here for that reason. So I bought into the hype a bit too, and I think we all have to some extent — that this is an important moment in history, where this technology is going to change the economy a lot. Hence why we’re studying it.

I don’t know if I would say it would bias me. I think it would bias me in terms of overestimating how important it will be, and saying, “Maybe I look back ten years and I just wasted ten years of my academic career studying this thing that was not that important.”

Seth: Do you feel that way about your work before working on AI now?

Martin: [laughs] Are my co-authors listening to this? [laughs]

Seth: No, no one listens to this. Should you have been working on AI sooner? Is that your current sense?

Andrey: I mean, you were working on AI pretty early.

Martin: I was working on facial recognition since 2016 or ‘17, so I was pretty early. But my previous work was more on business cycles, monetary and fiscal policy.

Seth: Who cares about monetary policy these days? [laughs]

Martin: [laughs] Let’s talk about that. The reason I shifted is that I was lacking good ideas and new questions in those domains.

One thing I’d say is that every time I do research and write my paper, it’s not just about AI. AI is prompting a re-examination of certain more classical issues in economics. That’s what I find exciting. AI was what prompted me to study property rights. But then when you start thinking about property rights, you realize, “Well, it’s not like we understand everything perfectly.” And the same for org econ. Once I started thinking about AI and knowledge codification, I was like, “Ah, it’s not like our models are great models of this process that was going on even before AI.”

Andrey: Final question. You’ve thought about potentially inefficient automation. How big of a risk is this with AI? Are we going to automate too much on the margin, in terms of GDP losses or other metrics? Is this a first-order concern or not?

Martin: I don’t think so. And here I’m not going to speculate so much — this is one answer I’m perhaps the most confident about.

People are extrapolating a lot from the experience of robots, a technology that automated people in the lower parts of the income distribution who were both relatively poor and didn’t have the financial means to smooth out the transition to new jobs or to retirement.

Seth: And geographically concentrated.

Martin: And geographically concentrated. Absolutely.

In the “Inefficient Automation” paper, we get at the question of when we know that automation is not just bad for redistribution but also inefficient — inefficient in the sense that you could engineer a slowdown of the transition that would make everyone better off and no one worse off.

Based on that, the reason I’m more optimistic about AI is that when you look at the exposure, it’s hitting parts of the income distribution that are more middle-to-high income — cognitive workers in cognitive-intensive occupations — who you would not think of as poor, so the redistribution is going the opposite way, nor as financially vulnerable. They probably have the means to smooth out transitions to new occupations or into retirement. And they’re not that geographically concentrated. So all the things that made me think maybe for the robot transition we screwed it up and should have slowed things down a bit — those arguments are not that powerful in this case.

Except for some examples. Call centers is one where that’s obviously not the case. Truckers, probably another. But by and large those are specific examples, not the broad pattern.

That said, some of these groups are politically powerful. That may mean we see the same thing that happened with the forgiveness of college debt — another thing where the redistribution went the wrong way [laughs], but they managed to pass it because they were a politically powerful group. So now that makes me wonder whether the same thing will happen with AI, even when it’s not the right thing, just because of where the political power lies.

Seth: We almost closed on a happy note.

Martin: You can cut it there. [laughs]

Andrey: We could have cut there, but there was this recent Pacing the Frontier letter from the labs, essentially calling for a coordinated slowdown. Usually you don’t think about slowdowns in automation as a possible policy, but this does have that flavor — people asking for the government to essentially tell everyone not to train as much, or to slow—

Seth: Chill out. [laughs]

Andrey: Just chill out, guys. We have a lot to process here. And I don’t think it’s driven by what you’re saying. A lot of people are already talking about regulatory capture and stuff. I just don’t think it really comes from that place. But I do think that, more so than protectionism, we might just get slowed down for other reasons.

Martin: Yeah. And again, there are many reasons you may worry about. One is the automation of young people in entry-level jobs — I think the jury’s still out on that. And reasons that have to do with the social rather than the economic side: people asking chatbots for things that may be harmful to them, these types of things.

Seth: That was a really diplomatic way of saying child porn.

Martin: [laughs] Actually, I was thinking more about psychology-type things. [laughs]

Seth: Oh, okay. [laughs]

Martin: Asking for therapy or advice and things like that. And to me, that’s more like — we need some guardrails, and not so much halting or slowing down automation the way we have for other technologies.

Andrey: All right. Well, thanks for coming on the show, Martin. This was a really delightful episode, and we look forward to further conversations in the future.

Seth: Yeah, we’ll bring you back in five years to see if new firms are growing faster.

Martin: [laughs] No, you should bring me back with Daron, and then you can put us together. I think I can fight with Daron. [laughs] All right. It was great, guys.

Seth: Thank you so much.

We start with priors. Has real GDP per capita over- or underestimated welfare growth? Andrey’s case for over: China’s real GDP per capita is up over 70 times since the early 1950s, and life is not 70 times better.

The paper poses a dilemma, suppose you have two goods (food and string quartets) with different productivity growth. The counterintuitive result in the paper is that inventing the low-productivity good makes people better off while slowing measured GDP growth. The paper then proposes to use the value of a statistical life as a measure of welfare. It is the nominal price for being alive, so its growth rate, deflated by the marginal utility of consumption and pinned down by the intertemporal Euler equation, gives you the growth rate of lifetime utility. Seth likes it, while Andrey explains why dissatisfaction with the Euler equation was in the top three reasons he didn’t become a macroeconomist.

The results: welfare growth of 2.3% a year since 1940 — and a decline from 1980 to 2024. We get into how VSL is actually estimated (hedonic wage regressions identified off coal miners and other people with unusual preferences about dying), the 1% discount rate that arrives uncited in a one-sentence paragraph, whether mortality risk is being double-counted, and the robustness table where moving the interest rate by a single percentage point in either direction swings the 1940–2020 welfare gain.

Links & References

The paper

• Philip Trammell & Charles I. Jones, “When GDP Misleads: Inferring Living Standards from the Value of a Statistical Life”

• Charles I. Jones — Stanford GSB & Anthropic

• Philip Trammell — Stanford DEL & Epoch AI

• Charles I. Jones, “The AI Dilemma: Growth versus Existential Risk”

The value of a statistical life

• US DOT departmental guidance on VSL in economic analysis — the paper cites $13.7 million for 2024 (we said $14.5 million on air; see Corrections)

• Dora L. Costa & Matthew E. Kahn, “Changes in the Value of Life, 1940–1980” — Journal of Risk and Uncertainty, 2004;

  • Kahn responds to our episode defending his paper here:

Concepts discussed

• The intertemporal Euler equation — and Andrey’s objections: it fails for a large enough subset of people, individual Euler equations don’t aggregate into a linear functional form, and it implies Ricardian equivalence, which is “so provably false as to invalidate this entire approach”

• New goods and variety growth — the food-and-string-quartets example; why the moment of invention (price falling from infinity to finite) is the hard part of any variety adjustment; and the composite-smartphone problem in quality adjustment

• Robert Nozick’s experience machine — the eudaimonia button, and what it would mean for measured welfare

• Derek Parfit on personal identity and future Tuesday indifference — is your discount rate even something a social welfare function should respect?

• Charles I. Jones & Peter J. Klenow, “Beyond GDP? Welfare across Countries and Time” — AER 2016;

• Trammell’s bull case for AI welfare: not more stuff per person, but vastly more beings capable of having utility — a total-utilitarian argument that this paper’s single representative agent can’t represent

Previously on Justified Posteriors

• How much should we invest in AI safety? — our earlier Chad Jones episode (existential risk vs. growth)

Corrections

• VSL figure: On the episode we said the US DOT value of a statistical life was $14.5 million for 2024. Jones & Trammell cite $13.7 million for 2024 (DOT guidance; current DOT table also lists $13.7M for 2024 / $14.2M for 2025). The slip doesn’t affect the paper’s growth-rate results.

• China GDP multiple: On the episode we said China’s real GDP per capita was up over 50× and as high as 72× since 1952/1962. A cleaner figure: 2025 real GDP per capita was about 82× its 1962 level.

Chapters

• (00:00) Cold open: would you rather be middle-class today, or the king of China?

• (00:27) Intro — the paper, and why an AI podcast is covering a paper that never says “AI”

• (02:12) Priors: has GDP per capita been a good proxy for welfare?

• (03:47) Everything good is correlated with GDP — until you look closely

• (05:29) China, 1952 to today: 72× GDP per capita. Is life 72 times better?

• (06:32) Two concerns: diminishing returns, and growth in varieties

• (07:08) Over or under? Andrey’s split verdict on China and the US

• (07:43) 116% since 1980 — “they already had pinball machines”

• (08:51) Seth’s prior: diminishing returns dominate, and why the AI age might flip the sign

• (10:09) Haven’t we already had huge variety growth? Podcasts, Prairie Home Companion, and the eudaimonia button

• (11:15) Putting numbers on it: 95% and 80% that GDP still overstates

• (11:36) How much better is life since 1986? Andrey says 25%

• (12:12) The benchmark: life expectancy × log consumption, and 41% since 1980

• (13:25) The paper’s setup: food, string quartets, and a productivity gap

• (15:05) Why inventing the new good makes us better off and slows measured growth

• (16:37) Quality adjustment, and the composite-smartphone problem

• (17:08) The problem that exists even before invention: satiation

• (17:41) Varieties vs. abundance: the king of China, at length

• (19:01) Trammell’s actual bull case: more beings, more utility

• (19:44) The clever idea: the value of a statistical life as a nominal price for being alive

• (21:10) $14.5 million — but 14.5 million what?

• (21:50) The deflator problem, Weimar Germany, and the marginal utility of consumption

• (24:12) “Micro or macro?” — the Euler equation and the conditions it needs

• (25:54) Discount rate vs. mortality risk — is something being double-counted?

• (27:17) The magic equation, in two equivalent forms

• (28:59) “The intertemporal Euler is getting some intertemporal shade”

• (29:40) Ricardian equivalence, aggregation, and “I have told Chad this”

• (31:04) Seth’s defense: welfare on the left, nominal on the right

• (31:49) How VSL is actually measured: Costa & Kahn, and the death risk of coal mining

• (33:04) Identification off the highest-risk jobs — and the people such jobs attract

• (34:44) “It’s just made up”: the $14.5M highway-safety number

• (35:22) The other inputs: a 1% discount rate, uncited, and T-bills plus a convenience yield

• (35:53) The results: 2.3% a year since 1940

• (37:19) …and negative since 1980. Life peaked in 1984

• (38:05) 6.9× vs 2.4×: could you convince me life is seven times better than 1940?

• (39:27) “For whom?” — the representative agent, the 10th percentile, and changing demographics

• (41:19) Should a social welfare function respect your discount rate at all? Parfit and future Tuesday indifference

• (42:07) Posteriors

• (43:37) The robustness table: one percentage point, 16.1× or 3×

• (45:10) Sign-off

Transcript

Cold Open & Introductions [00:00 – 02:12]

[00:00:00] Seth: Would you rather be a middle-class person today, or the king of China 2,000 years ago? He has the strong intuition — I don’t want to put words in his mouth, but I think he’d agree with this — that you’d rather be an upper-middle-class American today than the king of China 2,000 years ago. I don’t know if I have that intuition. I can think of a lot of really fun things I could do if I was king of China.

[00:00:27] Seth: Welcome to Justified Posteriors, the podcast that updates its beliefs about the economics of AI and technology. I’m Seth Benzell, whose utility from your listenership grows at a rate completely unmeasured by traditional economic analysis, coming to you from the beautiful recording studio of Boston University in gloomy Boston, Massachusetts.

Andrey: And I’m Andrey Fradkin, surprisingly also coming to you from Boston University in — it’s actually not that gloomy — Boston, Massachusetts. Welcome, Seth, to Boston. We’re going to be talking about quite an important and interesting paper today. Do you want to introduce the paper?

[00:01:10] Seth: The title of the paper is “When GDP Misleads: Inferring Living Standards from the Value of a Statistical Life,” from friends of the show Phil Trammell and Chad Jones. And what could be more important than how good is life? On a scale of one to ten, how good is life, Andrey?

Andrey: Eleven.

Seth: Oh, beautiful. When I’m with you, Andrey, it’s always an eleven.

Andrey: So, as context for this paper: although the words “AI” do not show up in the paper, it is very much about AI, and we’ll get into it.

Seth: And by two co-authors who are very AI-brained.

Andrey: Yeah. And I should mention, I’ve talked with both Phil and Chad about this paper. They always like to bring it up in these sorts of discussions, so we’re very excited to be digging into it. As usual, let’s get to our priors.

Priors: Is GDP per Capita a Good Proxy for Welfare? [02:12 – 07:08]

[00:02:12] Seth: So the whole premise of this paper is we’re going to think about a new way of trying to measure how much better life has gotten. And a first question you might ask is, “Professor Fradkin, don’t we already know how good life is? It’s called GDP per capita, and we measure that pretty well.” The essay is going to give us some arguments about why it may not be such a good measure. But let me start off by asking you: has real GDP per capita done a good job being a proxy for welfare? Is it maybe an overestimate, underestimate? And how do you think about how that might change in the age of AI?

Andrey: Real GDP per capita is a number that we measure every year —

Seth: Hell of a number. They measure it every quarter, dude.

Andrey: — and it varies a lot across countries. But just looking at that one number doesn’t tell us how much better life has gotten over time. It’s just a snapshot. I don’t think there’s an obvious way to map a given GDP per capita to welfare or utility or something else broadly. But the argument that has been made is that differences in GDP per capita are capturing something that’s very relevant to how good life is across places.

Seth: Sure — index whatever year you want to one. Fair enough.

Andrey: The point is that it only matters for comparisons, not in terms of the raw number.

Seth: You’re not 70,000 units happy this year?

[00:03:47] Andrey: Exactly, that’s what I’m trying to say. So then we have to think about the change in GDP per capita. And as economists, as a profession, we’ve gone all around the world to policymakers everywhere saying how great of a measure this is. All these other things you might care about that are not captured in GDP — well, if you squint at it, they’re super correlated with GDP.

Seth: Give us some examples.

Andrey: Life expectancy —

Seth: That’s good.

Andrey: — measures of life satisfaction.

Seth: Cool. Education.

Andrey: Yeah, education. Well, is that good or bad?

Seth: You’ve seen what education did to this guy.

[00:04:25] Andrey: Too much education. So many good things are correlated with GDP per capita, and it’s something that we can compute across countries as well, and that allows us to think about who’s doing well and who’s not. One of the key debates on Twitter recently has been how much poorer is Europe than the United States? And if you look at the statistics, it’s pretty clear that Europe has substantially become poorer than the US, because it hasn’t grown as much as the US over the past decade and a half or so. People were skeptical of that, but I think their arguments were not so great. So I do think that GDP per capita is a very useful metric. Now, that said, it is not a perfect metric, and in some sense it wasn’t designed to measure how much welfare people have.

Seth: This is an early-twentieth-century thing that’s trying to think about the industrial capacity of a nation — how many tanks could different countries crank out if the time called for it.

[00:05:29] Andrey: Yeah. So it’s useful to think about GDP per capita growth and the numbers involved. One number I pulled up was GDP per capita growth in China from — I think 1962, or 1952 — to today. And that number, depending on which time series you use, is over 50. It can be as high as 72. So: has life gotten 72 times better in China from 1952 to today? It’s a very interesting question, isn’t it? Because life was pretty bad in China at the time. People were dying of starvation. There was political repression.

Seth: But this is not an economic history podcast.

Andrey: No, no. But I’m just saying, all these things affect welfare in a way that GDP per capita is only very loosely connected to.

[00:06:32] Seth: And maybe we’ll talk about this more in the limitations of this measure in a second, but you might imagine two kinds of concerns. One concern is diminishing returns to consumption — once you get enough corn that you’re not starving to death, does doubling the amount of corn make you worse off? And then there’s this other issue, which is growth in varieties, which maybe this real GDP adjustment is supposed to do something to talk about quality improvements, but maybe we don’t think it does a perfect job.

Andrey: You’re kind of getting into the paper already.

Seth: Fair enough. So give me your prior.

Over or Under? China, the US, and Pinball Machines [07:08 – 10:09]

[00:07:08] Andrey: Is GDP per capita a good proxy for welfare? That’s hard to have a prior on, because I don’t know what “good” means. But we do have more precise questions. Has it over- or underestimated the growth in welfare in the past? And my point was, in China, it’s probably overestimated the growth in welfare.

Seth: You don’t think Chinese lives have gotten 83 times better?

Andrey: 72 times.

Seth: 72 times better.

Andrey: I think they’ve gotten so much better, but 72 times is truly a staggering amount. But at the levels that we’re seeing in the US — maybe I think it’s underestimated.

[00:07:43] Seth: I can also give you the 1980 numbers. The 1980 number: we’ve had 116% real GDP per capita growth. Does that feel like an over- or an underestimate?

Andrey: That feels like an overestimate, honestly.

Seth: Life isn’t twice as good as it was in 1980. They already had pinball machines. Cocaine. Basic heart surgery.

[00:08:03] Andrey: So I think a lot of this hinges on things that you want more of as you get wealthier. It could be new goods — we have smartphones now, which are pretty great, despite the naysayers. And then medical advances. But even in the ‘80s we had antibiotics and other things, and while medicine has drastically improved, average life expectancy has not improved that much. One way to think about it is: if life expectancy doubled from the 1980s, that would be a pretty good argument that things are twice as good.

Seth: That’d be a pretty good way to get there.

Andrey: What do you think?

[00:08:51] Seth: Great question, Andrey. I would say since 1980, 116% does seem too big — as does 543% since 1940. So I’m thinking about the US in particular. And for me, the biggest factor there is diminishing marginal returns from consumption. Even setting aside these complications around new products, new healthcare, that would tend to go in the other direction. The major effect is: if you go from not having enough protein in your diet to having enough protein, that gets you a lot of the way towards satisficing.

As we think about moving towards the AI age, I think there’s more of a reason to think that we’re going to start underestimating. The way I would put it is: pretty confident we’ve overestimated in the past. Moving forward, there’s more of a concern that GDP per capita might underestimate, if we have tremendous growth in variety but it doesn’t show up in terms of more stuff — it shows up mostly in terms of better, more pleasant digital experiences, or medical enhancements that enhance your quality of life, but maybe people aren’t working as long. I would think it’s going to be less of an overestimate in the future.

Variety Growth, the Eudaimonia Button, and Putting Numbers On It [10:09 – 13:25]

[00:10:09] Andrey: Haven’t we had a lot of variety expansion since the 1980s? We’ve had the internationalization of trade, all the digital things that we’ve gotten — there’s so many different websites to read.

Seth: We got Fallout: New Vegas.

Andrey: We have podcasts. They didn’t exist. That’s a new product.

Seth: We have radio shows. We have Prairie Home Companion.

Andrey: What are these new varieties that we’re missing out on?

Seth: Well, dear listener — if you knew the new variety, you’d have it already.

Andrey: We’ve read a lot of science fiction, right? It’s not crazy to imagine new things.

Seth: Nozick’s experience machine, dude.

[00:10:45] Andrey: Well, this is kind of the crux of it, isn’t it? If we had a happy button, and we just pressed the happy button, then we’d be happy the entire time versus only some of the time.

Seth: It’s a eudaimonia button. You also get complete human flourishing. It’s not just happiness.

Andrey: Great. Perfect. So in that world, I do agree with you that we might be underestimating welfare gains. But given the argument that we’ve made, I do think it’s still going to overestimate welfare gains.

[00:11:15] Seth: All right. Do we want to put a probability on the probability with which it will overestimate in the AI age? Give me a number, percentage chance.

Andrey: 95%.

Seth: 95%. I still think on balance it will overestimate — it’ll just overestimate less. I’ll come in at, let’s say, 80%. All right. And now I’m going to ask you an even harder question, Andrey: how much has life gotten better since 1940, or since 1980? You can take either one.

Andrey: Maybe ‘86, since that’s when I was born.

Seth: Okay. Since 1986, since that halcyon year.

Andrey: That’s a hard question. And in the US — I wasn’t born in the US, dude.

Seth: Well, how much worse has it gotten in Belarus, dude?

Andrey: It’s still gotten better in Belarus. I can imagine maybe 25% better?

[00:12:12] Seth: 25% better. Okay. I’ll report — this is not the complex analysis that’s done in the paper we read, but maybe to ground my prediction: one analysis they do in the paper is they just take the change in life expectancy and multiply that by, basically, the logarithm of consumption per capita. One reason you might do that is that a common way of adjusting for decreasing marginal returns is to put it into some sort of production function, and a logarithmic utility function is a very common one. There’s various reasons you might like that one. And so if you plug those two growth rates in, you get an improvement in life goodness of 41% since 1980 in the US. And to me, that seems like it’s in the right order of magnitude.

Andrey: I think that’s in the right order of magnitude.

Seth: Okay. So, Phil — we go into this paper feeling like the state of the art does a pretty good job. I guess we’re going to be really curious to see what his innovations add.

The Paper’s Setup: Food and String Quartets [13:25 – 17:08]

[00:13:00] Seth: So, Andrey, the first part of this paper is laying out an argument why GDP per capita might not do such a great job. Tell me about that.

[00:13:25] Andrey: At a high level, they give us a very simple example where things don’t quite work out as you’d expect. The setting is you have two goods. Let’s say you have food, and you have string orchestras.

Seth: Or services. Anything you can think of that starts with an S that has low productivity growth.

Andrey: I don’t actually enjoy string orchestras. I’m not sophisticated enough. But let’s say you have two goods. What they do is they say there is productivity growth in food, and that productivity growth means that over time you’re getting more and more food per capita. If food was the only thing in the economy, then the growth rate of the economy and the growth rate of GDP per capita would just be in terms of food. So if that growth rate is 5% per year, then our GDP growth would be 5% per year.

Seth: Seems fine. What’s wrong with that?

[00:14:38] Andrey: Let’s say in addition to that, string orchestras are invented, and they have a productivity growth of only 1%. Because of people’s preferences, people have diminishing marginal utility from consuming both food and orchestras.

Seth: Speak for yourself. Bring me to that all-you-can-eat buffet, dude — I’ll explode.

[00:15:05] Andrey: That means that once this new good gets created, people are going to shift more and more of their consumption into this new good. And so when you measure GDP growth per capita, it’s going to be a spend-weighted average of a 5% growth rate of food and a 1% growth rate of string quartets. As a result, we’re going to have GDP per capita growth that’s substantially less than 5% when this new good is invented. But this is counterintuitive, since people are much better off — they now have two goods that they consume instead of one. They could have consumed as much food as they wanted, but they’ve chosen not to. So this creation of a new good results in a change in how we think about GDP per capita. It is an example where welfare and GDP per capita don’t move in the same direction.

Seth: Right. So they diverge. Inventing the new thing makes us better off, clearly, but it also clearly slows GDP growth. Whoa — paradox, my mind explodes.

Andrey: And to be clear, this is a well-known feature — though I think underappreciated, as they argue — of GDP per capita and these sorts of calculations.

Seth: And it’s a big challenge. The real challenge is that at the moment it’s invented, the price of the string quartet goes from infinity to non-infinity. It’s that moment of invention that’s the real challenge in adjusting for new varieties.

[00:16:37] Andrey: Just to take a step back — we haven’t even talked about adjusting for new varieties. But it’s related to adjusting for improvements in the quality of something. Going back to the smartphone: the smartphone does a bunch of things that other products were doing, like calculators back in the day, that people would pay a lot for. And then how do you create a composite smartphone out of previous things? Is that even possible?

Satiation, and Would You Rather Be the King of China? [17:08 – 19:44]

[00:17:08] Seth: One other detail I want to pull out of this example: there’s a problem even before the invention of the string quartets. If you think people get satisfied with the amount of food — if once you have your 2,000 calories a day you’re done — and we just keep doubling the amount of food, you’re going to start evolving this infinite divergence between welfare, which is stuck, and GDP per capita, which goes off to infinity.

Andrey: In fact, the previous Chad Jones paper we discussed on this podcast made this exact point.

[00:17:41] Seth: Right. One aside I want to make — hopefully this is not too much of a divergence — is that when talking to Phil personally about the value that comes from AI, he’s really of the camp that wants to emphasize that value over time comes from growth of varieties rather than growth in abundance. So the thought experiment he gave me, and this is one I’ve been thinking about a lot: would you rather be a middle-class person today, or the king of China 2,000 years ago? And he has the strong intuition — I don’t want to put words in his mouth, but I think he’d agree with this — that you’d rather be an upper-middle-class American today than the king of China 2,000 years ago. I don’t know if I have that intuition. I can think of a lot of really fun things I could do if I was king of China.

Andrey: Yeah, I wouldn’t want to be the king of China.

Seth: Because it’s too stressful?

Andrey: Well, there’s that part of it too — you might get killed at any moment. But also, I just don’t enjoy bossing people around. Just not my style.

Seth: Listen, when I see all the fun they got up to in those palaces — you guys will know what I would choose. Maybe I’ll leave those details up to your imagination.

[00:19:01] Andrey: I thought you were going to say — and sorry, Phil, for this side discussion — that Phil’s bull case on AI is that we’re going to create a lot more beings that are capable of having utility. And so then welfare goes up, and it goes up because we have these transhumanist beings that have infinite utility. Not each instance of it is getting infinite utility, but all of them have very high utilities, and we’re creating so many of them that that’s the bull case.

Seth: Well, he’s a total utilitarian. I think that’s a whole other argument, and something that really doesn’t show up in this paper at all, because we’re going to be focused on this one representative agent guy. So we’re not really going to be thinking about distributional questions, or summing over lots of people.

The Clever Idea: The Value of a Statistical Life [19:44 – 24:12]

[00:19:44] Seth: Maybe I can lay out what they show as the resolution of this challenge. This is their innovative idea, and I really do want to say this is a very, very clever idea. It’s a very Phil-and-Chad idea, because they take the economic theory very literally in a way that is both wonderful and can feel a little bit out of left field — but when you appreciate it, has some real deliciousness to it. This is a delicious little bit of theory, because it comes out really simple. Once you see it, you’re like, “Oh yeah, that is how you could calculate that” — but apparently nobody’s made this point before.

So what they point out is: believe it or not, people have this nominal measure of how valuable being alive is, and maybe we can do some funny calculations around this. It’s this thing called a value of a statistical life. What’s a value of a statistical life? You may have run into this if you think about policy analyses where people ask questions like, “How much should I spend on this road in order to prevent one car accident every year?” You can imagine other settings where you would need a number like this.

Andrey: Yeah, like environmental-related things.

Seth: Environmental is perfect.

Andrey: Pollution causes some marginal amount of deaths. Climate change as well.

[00:21:10] Seth: And so there’s this number. One obvious question is: presumably, if life is better, that number should be bigger. You should be willing to pay more to preserve a life that’s a good life. All right — let’s take this number really literally. That’ll be maybe one question we can come back to at the end: how literally we want to take this number. Apparently, according to the US Department of Transportation — do you know what that number is today, Andrey?

Andrey: Fourteen?

Seth: Fourteen and a half million dollars is the value of a human life. So, now you know.
[Correction: Jones & Trammell cite $13.7 million for 2024 from US DOT guidance — not $14.5 million.]

Andrey: In America.

[00:21:50] Seth: In America, in 2024. So we’re going to be talking about everything through 2024. So what’s the challenge here? The challenge is: fourteen and a half million what? So we’re back to this issue of needing a real deflator, a real way of converting fourteen and a half million whats into utils, into goodness. That’s the overall logic.

But maybe the level doesn’t tell us anything. So the next step in the logic is that maybe we can start thinking about the change in that number telling us something. And they have this really interesting paper that we’ll talk about later, that looks at different US policy proposals over time and measures the change in this value of a statistical life over time. So we’ve now got this number, which is the change in a value of a statistical life over time. Maybe the answer is as simple as: the growth in this nominal value of a statistical life is equal to the growth in lifetime utility. And that’s basically the answer. There’s going to be an important adjustment, but that’s the high-level answer.

So what is that adjustment? Now you might worry: there’s growth in the nominal value of a statistical life, and there’s this growth in lifetime utility, but we’re still back to the question of what the deflator should be. Because you can imagine there’s hyperinflation. In Weimar Germany, the value of a statistical life goes to a gazillion Deutsche Marks. Did life really get that much better? So you need some sort of deflator.

How do they get this deflator? They observe that your total lifetime goodness — your total lifetime utility — should be equal to your value of your statistical life, and they say the right deflation measure is the marginal utility of consumption. Because that tells you how good a dollar is, so that’s the right deflator. And then you’ve got this problem: what’s the marginal utility of consumption? That’s also a number we don’t have a good way of getting at. But, Andrey, if you’ve taken intermediate microeconomics, you know we do know something about the change in the marginal utility of consumption, from our good friend the Euler equation.

The Deflator Problem and the Euler Equation [24:12 – 28:59]

[00:24:12] Andrey: I’m confused by your calling it microeconomics rather than macroeconomics.

Seth: Listen, I always obey the Euler condition when I spend my money. I don’t know about you.

Andrey: Oof. So, the famous Euler condition. It’s this magical equation in macroeconomics that essentially is the equilibrium relationship between interest rates and how much you save and how much you consume today. In a simple model, it’s a very simple formula. It is only true under very many, very restrictive conditions.

[00:24:52] Seth: So what are the caveats here? First of all, even in the simplified model, you need people to not be at a boundary — they need to be saving some of their money, they need to not be saving 100% or saving zero. But conditional on you being a fully rational guy, you’re not saving 100%, you’re not saving zero. In the background here, this is a representative agent who doesn’t age. They just have a constant probability of dying in every year. So set that aside too.

In that case, imagine what would be the case if you cared the same amount about consuming today as consuming tomorrow, and there was no interest rate. As long as you have some diminishing marginal returns from consumption, you’d want to consume the same amount in every period. So in the simplified, simplified, simplified intertemporal Euler, you save the amount of money such that you eat the same every day of your life. There’s two adjustments to that. The first is the personal discount rate. What’s that, Andrey?

[00:25:54] Andrey: Well, it’s an interesting question. You don’t value consumption tomorrow as much as you value it today. But the question is why. It is empirically a true thing, I think, that people do not, and so economists have estimated what this discount rate is. But I’ve always interpreted at least part of the discount rate as being about mortality risk — you don’t care as much about consumption fifty years from now because you might be dead fifty years from now.

Seth: This isn’t that.

Andrey: This is very much not that. But I think putting it in there while trying to measure the value of statistical life — I’m wondering if there was potentially a double counting going on there.

[00:26:45] Seth: Yes. So one challenge here is we’ve got to think about what this discount rate is as separate from mortality risk. It’s like, if you knew for sure — let’s say, to simplify, you’re going to live forever — how much more would you care about consuming today versus consuming tomorrow? So that’s the first one. And then there’s another reason you might want to consume tomorrow instead of consuming today, which is —

Andrey: The interest rate.

Seth: The positive interest rate.

Andrey: You get the big bucks from putting money in your savings account. Or even in the stock market.

[00:27:17] Seth: Exactly. So as we mentioned before, if there was no time preference, you would want to consume everything that you got equally on every day. If you had $1,000 in lifetime income and you lived for ten years, you’d want to consume $100 every year. But we’ve got these two time factors pushing in different directions: you value today more than tomorrow, but you also want to save for the future because you get this interest rate.

So we combine the intertemporal Euler with our previous observation that the value of a statistical life times the marginal utility of consumption should tell you something about your lifetime utility. And we take logarithms, and we take the first derivative, and we get a magic, simple equation. The way that they write this down is: the change in the value of being alive is equal to the change in the value of a statistical life, plus your personal discount rate, minus the interest rate.

That way is a little confusing, so let me say it a slightly different way, which I think is more intuitive. I’ll put the preference things on one side and the nominal things on the other side. So you might equivalently say that the growth in lifetime utility, minus your personal discount rate, is equal to the growth in the nominal value of a human life, minus the interest rate. And that’s a nominal interest rate — theoretically, we can go out there and measure it in the world. And now we’ve got an equation we can bring to the data. Andrey, aren’t you so glad to learn how much life has gotten better with this easy, simple equation?

Why Andrey Doesn’t Buy It [28:59 – 31:49]

[00:28:59] Andrey: It is a very, very, very simple equation.

Seth: It’s a beautiful result. Let’s call it that. Let’s be honest.

Andrey: It is very clever. I’m sure we’ll come back to this a little bit, but when I go back to thinking about why I didn’t become a macroeconomist — how dissatisfied I was with the Euler equation had to be in the top three.

Seth: Wow. The intertemporal Euler is getting some intertemporal shade.

Andrey: It is just not how people make decisions.

Seth: You don’t try to smooth out your consumption over time?

Andrey: Not in this way.

Seth: I think about the interest rate when I decide whether I want to spend today or tomorrow.

[00:29:40] Andrey: I don’t think most people do. I think the evidence suggests that many people do not. And even when they do save, it’s not according to this equation. One of the implications of these Euler equations is, for example, the result of Ricardian equivalence.

Seth: I tax you today, and you know I’m going to spend on you in the future, so you don’t change your behavior at all.

Andrey: Yes — which is so provably false as to invalidate this entire approach.

Seth: Well, it might be wrong because the Euler equation’s wrong, or it might be because the representative agent Euler equation is wrong and there’s heterogeneity.

[00:30:09] Andrey: My understanding in this literature is that for a decent subset of people — for an important enough subset of people — the Euler equation does not seem to be correct. But there is a distributional thing as well, which is that you can’t aggregate unless you’re in an absurdist model.

Seth: Unless you’re a macroeconomist.

Andrey: Unless you’re a macroeconomist, you can’t aggregate a bunch of individual people’s Euler equations into a linear functional form in this way.

Seth: Don’t tell Chad that while he’s cooking, dude.

Andrey: I have told Chad this. I literally have. I’m sure I’m not the first person who’s made this point. I view a lot of Chad’s papers as being thought experiments, so this is one. And to be clear, I do think they’re making an obvious point — that the value of statistical life should be related to welfare. That I’m buying. But in this way, I’m not buying.

[00:31:04] Seth: To me, the beauty of the equation is how, when we line everything up perfectly, we get the welfare terms on the left-hand side and the nominal terms on the right-hand side. Just the beauty of the idea that we can learn about people’s true love and value of life from measuring these statistical aggregates, in a way that’s more sophisticated than “GDP go up” — I think it’s a cool insight. I think there’s going to be huge application issues. I think there’s going to be huge issues about who gets to decide what, and why the VSL is what it is. But I think that this analysis, if you could somehow do it at the individual level, would be a great starting point.

How VSL Is Actually Measured [31:49 – 35:53]

[00:31:49] Andrey: It would be much more plausible at the individual level. But even there, we then have to dig into how value of statistical life is actually measured. Taking a step back — how is value of statistical life measured? Is this a good time to discuss that?

Seth: Let’s bring it up in one second. I just want to make the point that at the individual level, you could imagine people making insurance or healthcare decisions that would allow you to infer the same number, which is: how much money am I willing to pay to reduce my cost of death a certain amount? I don’t think it’s so heroic to imagine that you could do this at the individual level.

Andrey: I think it’s extraordinarily heroic. But you’re proposing just different identification strategies for value of statistical life.

Seth: So how do they do it?

[00:32:40] Andrey: Costa and Kahn run a regression where they regress the wage of a person on a bunch of things, and one of them is the likelihood that you will die on the job. And there are certain jobs that entail a higher likelihood of death, like if you’re a coal miner or something like that.

Seth: And this is a 2004 paper. It’s a little bit out of date.

[00:33:04] Andrey: Yeah, it’s a 2004 paper. It doesn’t really matter — I don’t think any value of statistical life paper is very good. It’s just economists being clever, and it’s fine. But there’s an implicit assumption there that people who are making these decisions — for example, to be a coal miner — are rationally weighing, in a very specific statistical sense, their probability of death on the job when they’re picking their wage, and that the market clears in a way that reflects that.

And then you can have better measurements that would be much harder to get over time. One of the advantages of Costa and Kahn is that they’re able to do this for a very broad set of data. But if you’re focusing on very microeconomic decisions, or maybe even with survey data, that might be more plausible. This approach seems very problematic to me. Another reason it seems problematic is that the identification of this number is coming, presumably, from jobs that have a very high probability of death on the job.

Seth: And these attract the irrational.

Andrey: These attract the irrational. These attract people with unusual sorts of preferences. So what these numbers’ relationship is to the reality of this hypothetical number that people don’t even want to think about — to me, it’s very far.

[00:34:44] Seth: Do you like better the 2024 fourteen-and-a-half-million highway safety number? Is that a better number or a worse number?

Andrey: It’s just made up. I mean, it’s just made up.

Seth: This is the Doctor Manhattan meme.

Andrey: Fourteen and a half mil — why not? I don’t know what to make of that.

Seth: So what other numbers do we need to plug in in order to make this analysis?

Andrey: We need a discount rate, and we need the interest rate. The discount rate — there are standard ones you can use.

Seth: They choose 1%, uncited. “We use 1%,” period. It’s one-sentence paragraphs.

[00:35:22] Andrey: For interest rates, they use the T-bill plus some convenience yield. The numbers are sensitive to the interest rate. Now, if you take the Euler equation seriously, then the interest rate should be whatever interest rate people have risk-free access to, plus some other stuff. But if people were making decisions according to the Euler equation — what are they putting in that Euler?

Seth: If the one representative human was making decisions according to the Euler equation, and they had access to this interest rate, and they had this discount rate, and they had this value of a statistical life. Sure.

Andrey: And we measured them correctly over time.

The Results: 2.3% a Year — and a Decline After 1980 [35:53 – 39:27]

[00:35:53] Seth: What are the results?

Andrey: So the results — oh, let me put on my glasses.

Seth: Oh, shoot. Oh, shit. Shit’s getting real.

Andrey: So the results are that welfare has gone up by 2.3 —

Seth: 2.3 percent per year, on average.

Andrey: Percent per year, yeah.

Seth: Since 1940.

[00:36:35] Andrey: Since 1940, exactly — with the biggest gains coming in 1940 to 1950, which is an interesting thing to think about. Why could that be? I don’t know.

Seth: If you think about what goes into this measure, mechanically it comes from the fact that the interest rate is going up. The interest rate going up makes welfare look good, because to make VSL stay the same, it means that welfare had to have gone up.

Andrey: The value of statistical life went up a lot between 1940 and 1950.

Seth: Yeah, so the value of statistical life is going up a lot. And then maybe the other headline number is from 1980 to 2024.

Andrey: Where it’s actually going down.

Seth: So actually, life got worse since 1980.

Andrey: Life got worse. Woo.

Seth: Life peaked in 1984. What can I say?

Andrey: You and George Orwell.

[00:37:36] Seth: So it’s a high enough interest rate, while value of statistical life does not grow as much. Man — big if true, life got 0.5% worse per year since 1980.

Andrey: The other thing is: 2.3% growth in welfare — how does that compare to GDP per capita growth?

Seth: Over the same period, it’s actually in the ballpark, isn’t it?

Andrey: Hmm. Interesting.

Seth: Interesting.

[00:38:05] Andrey: So one thing to naturally go to is: when calculating utility in the kind of dumb way, where we literally take the GDP growth and we say that utility is log of C, log of consumption — then GDP per capita growth just mechanically, drastically understates — sorry, overstates — welfare growth.

Seth: Right, just because you have to double in order to have a linear increase.

Andrey: Yes. But here, welfare is growing at that rate and there’s no additional logging involved. So as a result, that means welfare has grown a lot more using this measure of calculating things than using the other measure.

[00:38:51] Seth: So just to report those numbers: since 1940, according to this baseline projection, they see lifetime utility going up by 6.9x. Whereas the measure that I kind of prefer since 1940, which they also report — life expectancy times a log utility function over consumption — goes up by 2.4x since 1940. Could you convince me that life is two and a half times better than 1940? Probably faster than you can convince me that it’s seven times better.

For Whom? Representative Agents and Whose Life [39:27 – 42:07]

[00:39:27] Andrey: For whom?

Seth: Maybe we should start thinking about caveats now, and about refinements. If somebody said, “Seth, invent your own social welfare function and tell me how much total welfare has changed for the average person in the United States since 1940” — probably where that analysis would have to start is: how much desperate poverty have you eliminated? How many people were horribly impoverished? How many people had messed-up tuberculosis? How many people faced racism that’s been addressed in some ways today? You wouldn’t really think about the average person. You’d be trying to think about the 10th and 20th percentile people.

Andrey: That’s right.

[00:40:16] Seth: What other sort of — at the high level, we talked about this being an analysis that has a single representative agent. We think there’s all sorts of heterogeneity that also matters at the theoretical level, because you imagine that people over the course of their life cycles are making savings decisions based on that.

Andrey: So this is actually one of the key caveats here: the distribution of demographics has changed a lot over time, and it’s completely unclear from Costa and Kahn how that should play into the statistical value of life. Because presumably your willingness to take a death risk when you’re 60 is very different than when you’re 15 — by the way, for many different reasons. Some of them rational and some of them irrational.

Seth: Exactly. What is a value — whose statistical life, if you will?

Andrey: Exactly.

[00:41:19] Seth: Does the 1% time discount rate bother you? I guess that’s a related issue.

Andrey: I haven’t thought through how I would try to — I haven’t read this literature, on what share of the discount rate is due to death risk. Another version of it is the Parfit risk: that you in the future are a different person, and so you don’t —

Seth: Right, I don’t want to give that guy money. He might donate to a political party I don’t like.

Andrey: Yeah — maybe according to Parfit, why would you care about yourself in 100 years? You’re a totally different person.

Seth: Exactly. Future Tuesday indifference. On Tuesday, I don’t care what happens to Tuesday Seth.

Andrey: Exactly.

Seth: Another way to put it, I would say, is that even if it’s an empirical regularity that people treat consumption in the future as not as good as consumption today, it’s unclear from a social welfare perspective whether we need to adopt that view.

Posteriors [42:07 – 45:37]

[00:42:07] Andrey: So do we want to go to our posterior?

Seth: We’ll proceed to our posteriors.

[00:42:16] Andrey: Honestly, I enjoyed reading through this paper. It’s very simple. To me, it really highlights the importance of thinking about new goods, and whether AI changes what new goods we get.

Seth: Which is funny. If Phil’s bull case on AI creating welfare is lots of agents that we just multiply, that’s almost like a linear technology in GDP — in which case all these adjustments don’t matter, because it’s just more GDP equals more welfare, one for one.

Andrey: Well, I think we wouldn’t traditionally count non-humans in GDP.

Seth: Yeah, but come on. We’re enlightened beings. We accept all sentient life here on this podcast, Andrey.

Andrey: Be careful. We might become even more effective altruists than we currently are.

Seth: We love our EA fans.

[00:43:08] Andrey: So it was useful to think about. But ultimately, my pessimism on our ability to measure value of statistical life, and my pessimism on the Euler equation, means that I have not updated. It was good to think through how much welfare could have increased over this time period — so there’s a good scaffolding for that — but I don’t believe this method at all.

Seth: And one thing we should point out is that they do a bunch of robustness analyses, where they try different interest rates —

Andrey: And it matters a ton.

[00:43:37] Seth: — and it goes from “life got 100% worse” to “life has gotten 1,000 times better.” To give you one example of a confidence interval: just changing the interest rate you discount by 1% in either direction changes the percentage increase in welfare from 1940 to 2020 from 16.1x to 3x. And that’s just from changing one of the parameters by one percentage point.

Andrey: So, not a lot of updates. What about you?

[00:44:10] Seth: Not a lot of updates. How do I want to think about this? Ultimately, this is a theoretical exercise that I guess I’m a little bit more optimistic about. You could go out and try to measure these at the individual level a little bit more seriously. I think that when somebody shows — or a society shows — this is how much more we’re willing to put in to keep you alive, we should be able to learn something about welfare from that. So I do think this is inspiring as a direction. But for our actual numerical intuitions about how much life has gotten better or worse — yeah, I’m still in the same place. GDP per capita has overestimated welfare in the past. Maybe we’ll overestimate it a little bit less in the AI age; 80% chance it will. Haven’t really moved from that. And my initial estimates about how much welfare has changed since 1940 or 1980 haven’t changed. But maybe I’m just a little bit more optimistic about the method.

[00:45:10] Andrey: Okay, cool. Well, thanks for joining us for another episode of Justified Posteriors. Please let us know if you enjoyed this episode, or any other topics you’d like us to cover, and make sure to like, comment, and subscribe.

Seth: And always keep your posteriors justified.

In this week’s episode of Justified Posteriors, we update our beliefs with evidence from an ambitious new paper estimating the impact of AI on software production productivity.

Demirer (friend of the show), Musolff, and Yang combine public GitHub records for over 100,000 developers with confidential Microsoft data to trace the effect of distinct generations of AI coding tools — autocomplete, sync agents, and async agents — on the code production hierarchy: lines of code, files, commits, pull requests, projects, and releases. The main empirical finding is attenuation of the effect of AI at each step. Enormous gains of 1000% productivity increases or more at the top of the chain translate into about a 30% increase in shipped releases. The model they use to explain this result is closely connected to Kremer’s O-ring logic, which regular listeners will recognize as from a few episodes back. Seth likes the spirit of the model, but feels it is overcomplicated for this context, for reasons he explains.

In addition to discussing the data analysis, Andrey and Seth have a good back-and-forth about what we can conclude from it and extrapolate to the economy more generally. The implication that grabbed Seth’s attention is a sentence in this paper’s abstract. Nested in the summary of the careful empirical exercise is an estimated elasticity of substitution of 0.25 between AI and human effort. Big if true! Seth points out the enormous long-run implications of humans being complements to AI in what seems to be the most AI-friendly of tasks: as AI gets cheaper, the human share of income goes up, wages skyrocket, and ultimately AI boosts jobs instead of taking them. That’s a huge real-world hook. Seth and Andrey discuss whether, and if so how much, we update our beliefs in this direction, with Andrey being careful to point out the difficulties of extrapolating from a partial equilibrium elasticity to long-run macro consequences.

This episode is sponsored by Revelio Labs — a great source of labor economics data for academics and firms. Now available on WRDS.

Priors → Posteriors

Prior 1: Does access to AI coding tools boost lines of code written by more than 100%?

• Seth: 95% → 99%. Seth came in confident and left more so. Great to see giant numbers.

• Andrey: 80% → 95%. A high yes, hedged because “which developers” and “which tools” do a lot of work in that sentence.

Prior 2: Does AI boost economic value by 50% or less of the factor by which it boosts lines of code?

• Seth: 95% → 97.5%. I have personally produced a great deal of economically worthless code lately.

• Andrey: 85% → 95%.

Prior 3: Are AI coding tools a gross complement to human labor?

Seth’s answer depends on the level of aggregation:
The average normie programmer 20%→20% (unchanged)
A human engineering department 33%→40%
A software company / open source project 60%→85–90%
The economy as a whole 33%→33% (unchanged)

Andrey: 75% complement at the sectoral level, and he’d put it as low as the programming department — because right now the code that comes out is not shippable without substantial human input. Posterior: still a complement, mildly supported. He declines, on the record and repeatedly, to extrapolate to the macroeconomy. No fun!

References

The paper under review

• Mert Demirer, Leon Musolff & Liyuan Yang, “Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools,” NBER Working Paper 35275 (May 2026).

• The authors’ own summary: “Writing code versus shipping code”, VoxEU, June 2026.

Prior work by the same team

• Kevin Zheyuan Cui, Mert Demirer, Sonia Jaffe, Leon Musolff, Sida Peng & Tobias Salz, “The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers,” Management Science (2026). 4,867 developers, roughly a 26% increase in completed tasks, larger gains for the less experienced.

Related Research and Prior Episodes

• METR, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” — the RCT Andrey refers to, in which 16 experienced developers took 19% longer with AI while believing they were 20% faster. arXiv version.

• METR’s own update, “We are Changing our Developer Productivity Experiment Design” (Feb 2026) — developers increasingly refuse to be randomized into working without AI, which biases the estimated speedup downward.

• Michael Kremer, “The O-Ring Theory of Economic Development,” Quarterly Journal of Economics 108(3), 1993. Our episode on it: Weak Links, Strong Predictions: Kremer’s O-Ring at 30.

• Josh Gans & Avi Goldfarb, “O-Ring Automation,” NBER Working Paper 34639 — and our conversation with Avi: Avi Goldfarb on Prediction Machines, O-Ring Tasks, and How AI is Reshaping Economics.

• Aaron Chatterji, Tom Cunningham, David Deming, Zoë Hitzig, Christopher Ong, Carl Shan & Kevin Wadman, “How People Use ChatGPT,” NBER Working Paper 34255 — a reference for Andrey’s point about substitution toward home production. Non-work usage grew to over 70% of messages; computer programming is a small share.

• Our episode on the same phenomenon in a different market: The Economics of Book Slop — more books, not obviously more valuable books. More apps, not obviously more downloads. Rhymes.

• The universal token budget: Alex Imas — Demand Collapse, Bargaining with Machines, and Behavioral AI Economics and Seb Krier on AGI, the Coasean Singularity, and EDM.

  

Table of Contents

• Introduction and Today’s Paper — [00:00]

• Priors: [02:58]

• The Evidence: Ideal and Possible Experiments— [18:34]

• The Evidence: Adoption Events and the Attenuating Waterfall — [24:56]

• The Evidence: Model and Simulation Results — [42:56]

• Aggregates, App Stores, and Posteriors — [56:00]

Subscribe now

Transcript

Introduction and Today’s Paper [00:00]

Seth: The abstract of the paper has the following sentence: blah, blah, blah, based on the results in the model, there’s an estimated elasticity of substitution of 0.25, so high complementarity between AI and human effort, which indicates strong complementarities. Wow. As the people say, big if true. Humans and AI, 0.25 complements. Everybody worried about AI taking all our jobs — wrong. All labor share to 100%.

Welcome to the Justified Posteriors podcast, the podcast that updates beliefs about the economics of AI and technology. I’m Seth Benzell, with a 0% productivity impact on my code writing, as measured by my podcast release schedule, coming to you from the Pocono Mountains of eastern Pennsylvania.

Andrey: And I’m Andrey Fradkin, coming to you from San Francisco, California. Justified Posteriors is sponsored by the fine folks at Revelio Labs, and please do sign up to our podcast and our Substack whenever you get the chance.

Seth: Today we’re talking about a really interesting empirical study investigating the impact of AI tool use — autocomplete, synchronous agents, asynchronous agents — on people’s productivity in writing code. This is that kind of hard empirical data that maybe has the potential to move our beliefs. So I’m cautiously optimistic that I’m going to learn a lot from this one.

Andrey: It’s the big question, in many ways. We have these tools. We’re using them. What do we get out of them? Are we really that much more productive? That is the question on everyone’s mind, especially since so many of these tools are very costly. There are people token maxing under the belief that the more tokens that are used, the more valuable the output will be.

Seth: People are going broke over the tokens. We discussed a universal token budget when Alex Imas was on — or maybe that was with Seb. What are people getting when they’re actually paying for them? The paper we read to look into this is called “Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools,” from Demirer, Leon Musolff, and Liyuan Yang. So, butchered every name, as is common for us.

Andrey: Mert is a co-author of the pod, so very excited to be reading a paper of his.

Seth: Despite being friends of the show, no punches pulled.

Andrey: We never pull any punches. As listeners may know, now is the time for our priors. So what do we think about this topic before we read the paper?

Priors: Three Claims, One Spicy [02:58]

Seth: Reading this abstract, it seems to make two pretty narrow claims and then one claim that’s a really big spicy one. The first claim is that AI is really productive for helping you write lines of code. The next, more detailed claim is that the translation function going from lines of code to subsequently more advanced stages of production has an attenuation effect. You write more code, then you get more economic stuff out of that code — but the benefit attenuates. And then finally, using a model and some simulation, the paper goes on to argue that this can tell us something about the degree of complementarity between human coders and AIs. So let’s hit those one by one.

First off, let me ask you this prior, Andrey. Do you think access to AI coding tools boosts the lines of code written by programmers by more than 100%?

Andrey: Yes. I put my prior at 80%.

Seth: 80%? That’s not a “yes, shut up.” Why only 80%?

Andrey: That’s a pretty high yes. I’m a good Bayesian. Like any empirical question, there are sub-questions about which developers we’re talking about and which specific agentic tools we’re talking about. I’m sure the answer varies by those. It probably increases lines of code written by non-programmers by a ton.

Seth: In percentage terms — from, you know, infinity.

Andrey: You start with zero and then you go to something. It’s a pretty big percentage increase. It’s really important what samples are being used. But what’s your prior, Seth?

Seth: Maybe I’m not a good Bayesian, but I came in pretty confident. I’ve got in mind all those people who have written one line of code in their lives now writing thousands of lines of code. If we look at the population as a whole, I’d be 95% confident going in that the amount of coding is going up by more than 100%.

Andrey: And just to be clear, my interpretation was not people who previously did not code.

Seth: Right. What we’re going to see is people who were using GitHub in the ex ante period, which raises you above some very basic level of coding in the pre-period. But even still, I’m pretty confident in this one.

Andrey: [06:08] Let me ask you a related question. If you just plotted the lines of code produced by developers over time, in aggregate, not per developer — would that increase by more than 100%?

Seth: Right, because now the guy who’s going from zero to 1,000 is a lot less important than the guy going from 1,000 to 10,000. If we’re saying the total amount, I would say yes — my guess is that it would double — but the confidence goes down. Probably that brings me to the 80 to 85% range.

[06:21] All right, prior number two. Now it’s this idea that maybe the benefits attenuate. We’re not actually getting 100% more program even if we’re getting more than 100% more code. So let me ask you, Andrey: would you say that AI boosts economic output or economic value by 50% or less of the factor by which it boosts the lines of code written?

Andrey: It boosts it by less than 50%. I’m very confident of that. I’ll even say 85%.

Seth: I again come in pretty confident, because I’ve seen some real economically worthless code written by AIs recently. I’m a producer of a lot of it myself. I come in in the 90 to 95% range. In my head I’m also thinking about the fact that when we’re talking about lines of code, we’re probably also talking about comments. AI is really good at commenting and leaving descriptive comments if you let it. That’s one way the code might end up less dense. But also just my experience of writing my own code versus seeing what the AI writes — when you say “go do A, B, and C,” you end up with a whole bunch of stuff that I would never have done myself because I’m lazy. So that’s another reason I’m confident: 95%.

Seth: [08:07] Big question, Andrey. The abstract of the paper has the following sentence, which I love: based on the results in the model, there’s an estimated elasticity of substitution of 0.25, so high complementarity between AI and human effort, which indicates strong complementarities. Wow. Big if true. Humans and AI, 0.25 complements. Everybody worried about AI taking all our jobs — wrong. All labor share to 100%, wages skyrocket. What’s your prior on AI coding’s complementarity to humans?

Andrey: In the prepared priors, you made the comment that this is a prior about gross complements.

Seth: [09:08] The way I think about complementarity — this is a good way to think about it if you’re not constantly dealing with elasticities of substitution — is that the elasticity in the middle is Cobb-Douglas. What Cobb-Douglas gives you is that no matter how cheap capital or labor is, you always spend a fixed share of your income on capital and a fixed share on labor. So put gross complements on one side of that and gross substitutes on the other. If you’re gross substitutes, then as your factor gets cheaper, you spend more of the budget on it. So if AI and humans were gross substitutes, that would mean that as AI gets better, the AI share of income goes up and the human share goes down. That’s the robots-take-our-jobs universe.

Gross complements would mean the opposite. As AI gets cheaper, we’ve got all of this jelly, and the human peanut butter becomes more and more valuable. The share of income going to humans would go up. So: Cobb-Douglas in the middle, and what this paper argues is that in their data, in their sample, AI coders and human coders are very much on the peanut butter and jelly side.

Andrey: To be clear, this paper is not about gross complements or substitutes. I view this paper as about holding all else equal, no readjustment — are they complements or substitutes? Which is a different story from what you’re talking about, which involves reallocation of resources.

Seth: Fair enough. But the argument would be that they’re measuring an underlying parameter such that if there were reallocation, you would get the effects I’m talking about. That’s my reading.

Andrey: This is not a macro model, Seth.

Seth: Why do you care about estimating micro parameters except to plug them into a macro model?

Andrey: My reading of the paper is that, given the way we’re currently organizing code production, this is their attempt at estimating the production function.

Seth: And then I extrapolated out into infinity.

Andrey: I don’t view this as a macro paper about a reallocation of labor and capital in the macroeconomy, if that makes sense.

Seth: Andrey, your comment is so sympathetic to the authors that I worry you may have been replaced by some sort of alien invader. But feel free to address it just on the basis of an estimate of a parameter in a production function. You don’t have to extrapolate anything from that.

Andrey: [12:15] The question is how much this paper informs our beliefs about the macro versus the micro, which we’ll talk about. But is AI coding a gross complement or substitute for human labor? My prior is that it is a complement, at the moment, with the current technology — which is a big caveat. The AIs, or at least how we’re using AIs, are not yet autonomous enough that humans can be taken out of the loop. As a result, humans still have a huge role to play in being in the loop on the coding production process. They’re essentially the bottleneck.

That said, it really depends on the human, and this goes in many directions. Now we have a lot of people who were previously not coding, coding, and presumably that code would never have been written had this technology not existed. I would venture a guess that these humans in their roles are becoming more productive — they’re highly complementary, since literally the code would not have existed without them. There are some programmers who are not very good programmers, who were writing slop code even before AI tools. I imagine those coders are substitutes. And then there are other coders who understand the business logic, architecture, and so on, who are at the moment highly complementary to the AI. How that all nets out — I think it’s still on the complements side in the aggregate for the sector at current technological levels.

Seth: But you can zoom out even more.

Andrey: I’m not a macroeconomist, so I don’t want to extrapolate to macro from just coding tools. There’s a lot more to the economy.

Seth: Ask a computer programmer. They say you get infinite code, and then the economy goes brr.

Andrey: Our singularitarian friends notwithstanding, I’d rather not speculate about that in this prior. And I didn’t give you a number. I need to give you a number, right? I would say it’s a complement with a prior of 75%.

Seth: And that’s at the sectoral level?

Andrey: Yes, at the sectoral level. And I have to be very precise — I’m not even including Fable in here, because we haven’t seen Fable in the wild.

Seth: [14:49] Well, beloved listeners, while you have one host who will not extrapolate wildly, I am always here for you. I agree with Andrey that the question depends on what level of the economy we’re looking at, so let me build up from narrow to aggregate.

If we’re thinking about the level of mediocre coders — the average coder — I would be very surprised if AI were a complement to the average computer programmer at this point. I’d say a 20% chance that AI is a complement to the average normie computer programmer.

Then if we move up to the level of a human engineering department, where maybe you’ve got one really sophisticated computer science architect who can complement the slop code being made, my probability of complementarity goes up to maybe one-third.

But then we hit the level Andrey was focused on, the software company. At the software company, it seems clear that being able to churn out slop code and have a software product that is useful for people involves complementarity with people who have business expertise, people who can figure out how to sell the thing — which is not being automated when we’re talking about pure software. So at the level of the software company, maybe a 60% chance of complementarity between the AI and the entire software company.

Andrey: I do want to clarify — you mischaracterized what I said. I was saying complements not at the software company level, but even at the programming department.

Seth: But you said more substitutable for the lower quality coders.

Andrey: Yes. But I’m not even talking about salespeople. I’m just talking about the fact that literally right now, if you’re coding, the code that is produced is not good enough to ship — Fable notwithstanding — without substantial human input.

Seth: Okay. So that’s the human engineering department level, and you’d say even at the individual programmer level there are complements. Possible. And then I zoom out one more time, from the sectoral level to the economy-wide level. What do people do with software, Andrey? They automate things, and they consume digital products that don’t have large labor inputs. So at the economy-wide level I’d say AI coding tools are a substitute for human labor — a 33% chance they’re a complement. That’s the hierarchy I’d split out. Any response?

Andrey: So let’s say you have someone at the insurance company reading claims. To this date there hasn’t been software that really automates them, but now it’s finally there, and so those insurance readers are donezo.

Seth: That would be my argument.

Andrey: In the short run.

Seth: As software gets better generally because of this, that’s automative, that’s a gross substitute for humans.

Andrey: Interesting.

Seth: So now we will see if this paper has a chance to change our views.

The Experiment We’d Run If We Could Clone the Earth [18:34]

Andrey: Before we get to that — if we could design an optimal experiment, how would we study this question?

Seth: I would make a clone of the planet Earth. Actually, I would make a thousand clones of the planet Earth, and in half of them Sam Altman would be born, and in half of them alt-Sam Altman would be born. Help me out here, Andrey. What’s the correct answer?

Andrey: Seth wants to do a macroeconomics experiment, which is a nice thought experiment, where we forget about the existence of — where the transformer does not get invented. Is that the counterfactual? I’m not even sure.

Seth: I go back in time, I assassinate Geoffrey Hinton. Ilya goes and sells ice cream somewhere.

Andrey: And then we see how macroeconomic productivity and labor shares change in those worlds where LLMs don’t exist versus the ones in which they do. That’s an interesting thought experiment. A hard one to approximate, obviously. I was thinking of something more mundane. We have a bunch of firms, and half of those firms are randomly not allowed to use LLM-based tools — and half of them are imperfect anyway because of business stealing, but let’s forget that for a second. And then we see whether they’re more productive when they have LLMs, and how they change their labor shares. What do you think about that experiment?

Seth: You pointed out the main concern I’d have, which is business stealing effects. So maybe these are in two separate countries where you do these splits.

Andrey: What about the following experiment? What if we gave some developers LLMs, and some developers were not allowed to use LLMs? What do we learn from that experiment?

Seth: The issue is that you can’t force anyone to use the LLMs. So you’d find the treatment effect on the group that chooses to adopt when they’re available.

Andrey: Isn’t that always true, though? You can’t force a company to use LLMs.

Seth: In medical settings you can inject somebody with something. They don’t have to be given a choice.

Andrey: Sure. But relative to our other thought experiments — why isn’t that satisfactory for what we’re interested in?

Seth: The reason it’s not satisfactory is that first you have to think about whether the randomization is at the level of the company or at the level of the programmer. You’d imagine those are different effects. If you’re randomizing at the level of the programmer, one issue is that when people can build up a whole new workflow at the company, that might have a different effect than a single person adopting.

Andrey: What about just the fact that you can’t get labor substitution at all?

Seth: If we’re doing it at the programmer level, you can’t fire a treated programmer.

Andrey: Exactly. You can’t study that reallocation of labor.

Seth: Right. If you’re randomly assigning tools to the As and not the Bs, and you think the long-run effect is that some of the As should actually not be programmers, then obviously that’s a mechanism you’re not going to capture.

Andrey: [22:50] So it does have limitations, but you can still at least see how much more productive, in this narrow sense, the treated are versus the control programmers. Now, METR famously tried to do this experiment about a year ago and made the claim that AI tools as of a year ago — I think this was Cursor — did not make programmers more productive. It made them less productive. Much respect to our friends at METR, but most of us did not believe that experiment at all.

Seth: What was the productivity outcome in that paper?

Andrey: Speed. How long it took them to solve some issues.

Seth: Speed is frequency inverted.

Andrey: Great insight, Seth.

Seth: Frequency is our productivity measure here today. Sorry.

Andrey: Designing and implementing such an experiment in a credible way — people have thought pretty hard about how to do it, and it’s essentially impossible. It’s just hard to get the validity of true programming tasks, true randomization, preventing some people from using LLMs in all ways. And then most importantly, maybe, statistical power. Maybe with millions of dollars you could do this, but to have validity you need so many observations. So it doesn’t seem like a fruitful way to study this. We’re kind of left with the tools of observational causal inference, where we have to find something in the data that looks like an experiment if you squint at it and use the right econometric techniques.

Seth: If only there were a social science field with a bucket of techniques for inferring causality from observational data.

Andrey: It’s true. So that’s what the authors set out to do in this frankly really ambitious, impressive paper, in my opinion.

Seth: Are you ready to go to the evidence, Andrey?

Andrey: Sure.

The Evidence: Adoption Events and the Attenuating Waterfall [24:56]

Seth: Maybe I can quiz you through the different elements. Let’s start with the setting. What’s the population we’re going to evaluate, and what’s the natural experiment?

Andrey: At a high level, we’re looking at contributors to open source projects on GitHub. The authors have other data as well — some proprietary data from Microsoft, which is pretty important for this empirical study — and they have some side data on non-open-source development projects. But broadly, that’s the main population.

And then the thought experiment. Credit to the authors, because this is a mistake a lot of people make: a lot of people will write a paper like, “what is the effect of AI?”

Seth: Oh my gosh, I am so frustrated by these papers that say “what’s the effect of AI.” I got a restaurant to use AI to write ads once.

Andrey: [26:03] What we’re hinting at is that there is no one AI. It’s not the one ring to rule them all. You have the elven rings, which are the coding agents. You have the dwarven rings, which are the slop generators. You have the human-shaped rings, which are—

Seth: Well, they were the men doomed to die, so that’s the AI for health research.

Andrey: But seriously — over the past two years, AI has been many, many different things depending on whether you’re at the frontier or not and what tools you’re using. That includes the quality of the models. It includes the tooling, and how autonomous the tooling is. The authors do a very careful job of thinking about different ways people have been able to code via AI over this period, including just autosuggest. Remember back in the day when you were typing in an IDE such as VS Code, and it filled it out for you because what you were doing was predictable.

Seth: I think the dwarven rings are the AIs for finance, because they used them to get rich.

Andrey: And the one ring is the backdoor that Sam Altman has to all the GPT models.

Seth: All right. Back to the paper.

Andrey: So then you have what they call sync agents. I understand why they made this distinction, but it’s a pretty strange one in my opinion. A sync agent is essentially what we think of as an agent. You have Claude Code open, you’re typing something into Claude Code, Claude Code does something and gives you something back, you iterate with it, and then in the end you tell it when to push to the repository.

Seth: Seems pretty agent-y to me, Andrey.

Andrey: And then they also have async agents, which are autonomously sitting on a repo, refactoring the code, implementing things, and creating pull requests. Maybe it’s useful at this point to think a bit about the production hierarchy of code, which I found fascinating.

Seth: [28:40] It’s provocative. I don’t know if I agree with every step, but it’s an interesting way of thinking about it. So what is the production hierarchy? We go from lines of code on one end, and then question mark question mark question mark, and then money at the other end. What’s in between?

Andrey: Well, I don’t think open source projects are supposed to make money, Seth, but just FYI. So: there’s lines of code. There’s how many distinct files are touched. Obviously those are fungible in many ways — you can write one really long script, or a bunch of short scripts. You have commits, which are what programmers consider a discrete change to the code base worthy of noting. Once again, something that’s fungible and perhaps arbitrary. Then you have a pull request, which is when you submit this into the production code base, and usually someone else is going to review that in a professionally managed project. Then you have repos touched, so how many distinct repositories you’re contributing to — different repositories are distinct software projects. And then lastly, releases of those projects.

Seth: And they’re going to tell us the effect of each of these different kinds of agents on each of these different outcomes?

Andrey: Before we get to the empirics, do you want to go through the theory model?

Seth: No. I know that’s how the paper lays it out, but I think it’s kind of silly the way they give you some theory, then some empirics, then some simulation. If I wrote this paper, I would do all of the empirics and then all of the model and simulations. I’d just as soon do it that way.

Andrey: [30:29] So you have this dataset. It has the GitHub data. It has data on what they call adoption events — when specific GitHub users start using GitHub autocomplete, which is obviously an earlier period, back in 2022 and 2023. Then there’s the sync agents. Importantly, GitHub Copilot is a main one here — not always considered the best agent by many, sorry to our friends at Microsoft. But they also have ways of figuring out whether someone ostensibly is using Claude Code. For example, there might be a claude.md file in the repository, suggesting someone is using Claude somewhere. And then for the async agents, in particular Codex, OpenAI’s tool actually has a prefix saying that Codex generated a particular set of changes in the repository.

Seth: And a human would never delete that before uploading code, would they?

Andrey: I don’t think they would. Extremely low probability. I have very many concerns about what these adoption events mean, and in particular the intensity of adoption is very tricky to understand. You have a claude.md file. That could be because you’re just playing around with Claude Code. It could be because you’re very actively using Claude Code.

Seth: You have a partner who’s using it.

Andrey: Did your collaborator use Claude and you didn’t? This is a very long paper, so maybe I missed something, but some of these are imperfect proxies. That’s one way to put it.

Seth: You could summarize that about a lot of this paper.

Andrey: But again, I really do want to defend the paper: this is kind of the only thing we’ve got. We can’t do an RCT. I think it’s quite implausible.

Seth: And the other things I’ve seen are country-level shocks, where the problems get twice as bad.

Andrey: [33:07] Now, what is the control group here? This is really tricky. The obvious causal inference issue is that when you adopt a tool, you might be adopting it for reasons related to your future expected need to use the tool to produce code.

Seth: But Andrey, they’re matched on observables.

Andrey: This is the only exercise we can do, so we’re going to have to roll with it. I’m just explaining what the main challenge is. I’m not trying to snipe at it like you are already.

Seth: I’m not sniping. I’m having fun, Andrey.

Andrey: They need to find people who didn’t adopt the tool but otherwise would have attempted essentially similar things had they adopted. One way to think about it: if this async agent did not come out, both people would be doing the same exact thing. It did come out. One of them chose to use it, one of them chose not to, in a completely random way. That’s what they’re trying to get to.

Seth: Ideally they’re working the same number of hours on a project of the same difficulty level.

Andrey: Even that’s not true. If they were in the control group, they would be working on the same project, because one of the things the coding tools allow you to do is work on more projects. So you can’t condition on the project.

Now, the authors are justifiably hesitant to literally match the behavior of someone who adopted to someone who didn’t adopt in the same time period. Why? It’s a reasonable assumption that many software developers have adopted these tools, and the measurement is quite imperfect for whether someone adopted or not. So the control group might have also been treated. What they do instead is find people they don’t observe adopting, and then look at their behavior literally a year before. That’s the behavior they compare to the behavior of the adopting developers.

It’s not a strategy I’ve ever seen before. It’s quite non-standard. Usually in this sort of design you look at literally the same time period. But I understand why they do it, because of these very serious measurement error issues. What do you think about that, Seth?

Seth: I think there is some logic there. The reason you’d want to use the same chronological time period is if you were worried there was some calendar-time shock to people’s writing of code. If in 2023 everybody’s writing more code than in 2022, that’s when you’d want to match up calendar dates. I don’t know if they do anything to deal with seasonality.

Andrey: Seasonality is not the issue. It’s more that maybe the previous year was COVID, which would be a serious problem as a comparison year.

Seth: A weird year. But given that coding is kind of in a steady state between 2022 and 2023, or 2023 and 2024, it does not seem that implausible to me.

Andrey: I think it’s reasonable. The authors have sanity checks. They look at other adoption events like Docker, which is not an AI tool, and see whether they detect that it matters — a placebo test. And they don’t. So they do reasonable checks that this empirical strategy doesn’t always just pick up effects. And the magnitudes involved — it’s always easier to study something—

Seth: When the magnitude is massive.

Andrey: Which we know has massive effects. It would be crazy not to think it has massive effects, as our priors suggest.

Seth: This is not a case where they’re picking up significance because they have so many observations. They find big numbers here.

Andrey: [37:12] Let’s go through the outcomes very quickly. Lines of code: they find a 17x increase in lines of code after the adoption of an asynchronous agent. They find a slightly lower one, about 10x, for the sync agent, and about 2x for just using autocomplete. So just massive increases in lines of code. Files touched increases by about 4x with the agents, but only 0.5x with autocomplete. And then we see this pattern of diminishing numbers down the production hierarchy—

Seth: An attenuation.

Andrey: —to commits, pull requests. By the time we get to releases, the adoption of either type of agent only increases releases by 30%, even though it increased lines of code by 17 times.

Seth: Right. So 30% from the agents, and only 10% from autocomplete.

Andrey: [38:16] The other really interesting empirical pattern is what happens over time after adoption. As all of us know, we can spend some time playing around with these tools.

Seth: We can make a lot of commits real fast.

Andrey: They find a pattern that’s true across all three tool types: you get an initial spike in activity, whether it’s commits or lines of code, and then it diminishes a bit over time, although sometimes it rises again toward the end of the period. This is one of these very tricky things for them to disentangle.

Seth: How big is the spike compared to the long-term effect?

Andrey: For commits, the spike is about a 175% increase, and then it diminishes to about a 100% increase, and then it goes back up a little after that. That’s the pattern for the sync agent.

Seth: So why do we think that is?

Andrey: There’s people exploring, or they had a specific need for it and that’s why they adopted. And one thing to think about is that a job like ours — sometimes we’re coding a lot, sometimes we’re not coding a lot. We’re not coding all the time, and that’s probably true for a lot of people, especially open source developers, who might have other jobs in addition to their open source contributions. So I view that as an activity bias.

There are also calendar effects that are really relevant. If Codex comes out in January, and everyone who adopts adopts in January for the most part, then you track them over time — and let’s say a new frontier model comes out in March that’s really, really good. Once you get to March, everyone’s going to start using the new frontier model. That’s going to increase your output. Now it looks like there’s a much larger effect later on, but that’s confounding model quality with the tool release. Which goes back to the point that there’s no one AI. What AI is is constantly changing.

Seth: Right. So there’s a tension here. You’d want to take the short-term impact more seriously because there’s less margin for parallel trends to have a problem in the short term than in the long term. But on the other hand, we think there are all these short-term effects from adoption — either experimenting, or “I adopted the tool because I need it right now.” They focus on the long-term impact rather than the short-term impact, which is interesting, because usually in these difference-in-differences papers you see the opposite. They’ll say, “the effect in the short term is where I’ve really got the natural experiment, and then the long term, who knows?”

Andrey: I want to push back on that. It just depends on which AI we’re interested in. Let’s say the parallel trends hold and we just have bigger effects once we have Opus 4.5. We might be more interested in the effects of agents with 4.5 than in agents with 4.0. They’re just different objects.

And the key parallel trends are from the previous year, remember. There’s no sense in which Opus 4.5 being released in 2025 affects the parallel trend assumption for 2024. What would affect the parallel trend assumption for 2024 is if the people in 2024 started using some new tool at the end of 2024 that wasn’t being captured by the adoption measure.

Seth: Right, but we already mentioned that concern, so it’s not an additional concern. It’s just unmeasured use again.

Andrey: [42:07] And then the final empirical thing: they split out the causal effects by your pre-period activity. So how much you coded—

Seth: We were talking about the effect on power users versus normies earlier.

Andrey: And you see the biggest effects by far for people who have not been coding much according to these measures—

Seth: And that’s in percentage terms.

Andrey: In percentage terms, exactly. But it really should affect our interpretation, because presumably here we care about professional programmers more than non-programmers. All right, Seth, why don’t you tell us about the model the authors propose?

The Model: A Seven-Layer CES Cake [42:56]

Seth: The model is a fascinating one. It does a lot of what you would hope from a micro model of how code gets produced and economic value gets created. It’s got a lot of the elements you’d want. As we go, I’m going to point out that maybe there are a few too many elements, given that they’re only going to be able to bring the model to the data in a pretty superficial way.

So what’s the model? One of our favorite functional forms: a constant elasticity of substitution production function at each level of production. Starting from lines of code and then to files and commits and pulls and repos and releases, they think about each of those levels as having its own production function that takes a little bit from the level above and combines it with what you’re adding at the current level. So there’s an elasticity of substitution between lines of code and some new effort brought in at the files level, and they’re combined to produce an input that goes down to the commit level. A seven-layer nested constant elasticity of substitution production function, for people following along at home.

The key parameters they’re interested in estimating are the elasticities of substitution — how much the previous level’s inputs combine with the current level’s inputs. And then they’re also interested in what I would call the share term: what share of input at each level comes from the level above versus the current level. In a Cobb-Douglas production function those would just be the power terms. But they’re not going to interpret it that way. They’re going to invent a new parameter and say that’s actually what they’re measuring. We’ll come back to that.

[45:19] At each level, the upstream level is combined with effective input at the current level. What’s effective input at the current level? The way they measure it is just the amount of additional stuff you get at that level. But they think you can make effective inputs in three different ways, each corresponding to a different way to interact with AI.

The first is a pure labor-augmenting way, and there they’re thinking about autocomplete. The way you write more lines of code with autocomplete is that it makes each hour a programmer spends writing code produce so much more code.

Then they think, well, asynchronous agents aren’t like that. The way they think about agents is that the AI is going to write so much code, and then that’s a Leontief complement with a human who needs to go through and review all of that code before it turns into effective input. So they call that a different sort of complementarity between AI and humans.

And then finally their last mode is pure substitution. Maybe if the asynchronous agent is good enough, you could just let it cook and write as many effective lines of code as you want without any input.

[46:36] And Andrey, in our previous episode — or maybe two episodes ago now — we used a phrase I really enjoyed: ex cathedra. We get this ex cathedra pronouncement that autocomplete enters the production function this way and synchronous agents enter the production function that way. It’s not going to end up being super important, but it’s meant to get you thinking about what patterns you might expect to see as the boosted productivity of lines of code attenuates. So that’s the basic setup, those are the parameters we’re interested in, and then the question is what we can conclude from this model, and finally they try to plug in some parameters and simulate it.

Andrey: [47:16] It is a useful model for thinking through the production process here, but I agree with you, the ex cathedra pronouncements are a bit hard to swallow in some dimensions. The fact that you can use an agent can affect all other parts of how you produce your code. Even the code that you’re writing, you might be writing differently. Or alternatively, just because you’re using an async agent doesn’t mean you’re not also using a sync agent at the same time and autocomplete at the same time.

Seth: I don’t think they say you couldn’t. The part that’s a limitation, or maybe a little heroic, is the idea that they can do this mapping to a production function a priori.

Andrey: They’ve set up the model so that the optimal production mechanism is a corner solution, meaning that you can’t both be writing code and having the AI agent write code, for example.

Seth: Right. And then when they simulate it, they simplify it even further, and they really only simulate it for autocomplete. Given that they cannot actually bring all of these parameters to the data, and given that they have such a rich empirical setting already — if I were writing this model, I would not have gotten into the nitty-gritty of “here are three different ways that AI can plug into the production function” that we cannot distinguish between in the best possible data.

Oh, and then this is the delicious part, Andrey. The delicious part is that after setting up this highly complex production function, they then say: we’re not going to solve this production function for the optimal reallocation problem, because that’s too hard. Why would you write down a model that’s so complex that you can’t bring it to the data and can’t optimize it? I would drop some of these terms. But that’s a nitpick, because at a motivational level it does hit the different vibes that people have, even if it’s not an empirical question at this point.

Andrey: I think the reason they did it is that they have seven outcome variables, and they needed a model that can rationalize those.

Seth: They rationalize it with way fewer parameters than they write down. They only calibrate two of these parameters.

Andrey: The model is general, but then they assume a bunch of the terms are identical — the layer substitutions are identical.

Seth: And for example, there’s a parameter in the model which is the number of hours you need to spend reviewing each AI-written bit of code. If you could actually measure that, put it in the model. But if you can’t, I don’t know why this is in the model. There’s this temptation to overcomplicate models of automation, and I’ve run into this with commenters on stuff I’ve written down, where it’s “your model is just a CES production function, but what if something entered this way? What if it were complementary in this weird, different way?” The point of CES is to write something super general and super flexible down. To add more flexibility to the already super flexible—

Andrey: Well, I don’t think CES is that flexible. I really don’t.

Seth: You’re right to say CES is not super flexible. The way I’d put it is that in a lot of settings, the only things you observe are the things you can estimate with CES, so that’s why you write it down.

Andrey: I think that’s what you’re trying to say, not that it’s flexible.

Seth: [51:47] It’s a model that hits all of the empirically measurable things. All right, friends of the show, love you guys, but a couple more parameters in here than I would have written down.

So what are the conclusions of this model? First of all, it’s intractable with reallocation, so we restrict attention to the partial equilibrium case where there’s no reallocation of efforts. What do they find? Two intuitive results. The first is that if you write infinity lines of code, you only get infinite output if there are gross substitutes at every link of the production hierarchy. That makes sense. So now we’ve got a story for why, even if you were really good at automating code, you wouldn’t have infinite output. Fair enough.

The second theoretical result is that as long as there isn’t perfect substitution, you’re going to have attenuation of the boost as you move down the production hierarchy. If you have a big effect at the top of the waterfall, the effects attenuate as you go down the waterfall. Again, intuitive, but it’s good to see the model deliver the result we’re going to see in the data.

[52:21] In the simulation, the goal is to estimate two parameters of the model. All the other parameters we’re going to forget about. The two we’re interested in are, first, the elasticity of substitution between production at a higher level and input at the current level, and second, what they call an elasticity of output between the higher level and the lower level. That’s not the way I would talk about it, because it’s not a primitive of the model — it’s going to depend on input levels. I would talk about it as a share term between production at the higher stage and the current level’s input. But given that, they estimate these two parameters. Call it what you will.

The parameter that’s really the interesting one is the elasticity of substitution between input at the higher level and the human inputs. Why can we say it’s human inputs? They restrict attention to the autocomplete natural experiment. So based on their ex cathedra pronouncements, autocomplete works by boosting lines of code at the lines-of-code level, but it’s not going to have any effect on productivity down the cascade. Autocomplete isn’t going to help you produce commits or releases. Given that, they estimate the parameter such that you get that attenuating effect, and finally we get the headline result: a 0.25 elasticity of substitution between upstream production and lines of code. How does that make you feel, Andrey?

Andrey: One way to think about it is that it’s just a translation of the fact that lines of code don’t result in that many new releases. There are a lot of things the model is obviously missing. For example, you can reallocate your effort, and that’s something the estimation is not capturing.

But look — what do I view this as? The meat of this paper is the empirical exercise, and this is a back-of-the-envelope exercise for the elasticity of substitution parameter. Given that, I’m okay with it. I’m not going to go to the bank and say it’s 0.25. So that’s not the part I worry about. I’m more worried that all these things under the code are fungible with each other. What is even a commit? A commit-to-PR ratio is not a constant thing. There’s not even a standard way — different coding teams have different ways of splitting up this code production process. I don’t know whether to think of this as a hierarchy or not. Maybe that’s the deeper criticism.

And then, as you point out, quality is really a missing element. Whether that code is comments, whether that code is unit tests that are maybe not that important — that might be something being produced with autosuggest. That’s very different from the other type of code being produced that was essential functionality.

Seth: Very well put, Andrey. Are we ready to move to our posteriors?

Aggregates, App Stores, and Posteriors [56:00]

Andrey: No — one other thing to point out. They do this back-of-the-envelope exercise, and the final thing they do is think about aggregate outcomes. Another naive way to study the effect of AI tools on coding is just to look at the total number of GitHub pull requests over time. If we thought coding was making people much more efficient, we’d expect that to spike.

What they find is a trend increase around the start of 2025, and their data doesn’t go long past 2026, so we don’t really know what happens after that. But that increase is substantially less than the per-programmer increase they find in the event studies. In particular, the increase in pull requests you can read as maybe a 50% increase.

What might be driving that discrepancy goes back to the experienced versus inexperienced developers. The developers producing the most code — their measured productivity isn’t increasing as much as that of casual, occasional users. And they’re also the ones generating the most code, so the overall effect is not as big as the event study estimates suggest.

The other thing to point out is that to the extent code is being used to generate apps, we do see increases in the number of apps being produced on, say, the iOS store — but we don’t seem to see an increase in downloads. So it’s not obvious, and it’s similar to our story with books, that a lot of these marginally produced apps are actually resulting in something people are using. To the extent that productivity is ultimately tied to value, we don’t seem to be seeing it yet.

That points to a broader observation: if you look around, you see a lot more code, but it’s not clear it’s resulted in amazingly better software for us yet. Or, another way to put it — and this is shown in the OpenAI paper on how people use ChatGPT — there’s a lot of substitution from buying things to home production, or even just producing things at home that you weren’t going to do before. You might ask ChatGPT for advice, or for tech support, and before you might have hired someone to do that, or alternatively not solved the problem in the first place. That’s probably where most of the AI value is coming from right now. Maybe the most impactful use of AI agents is in the use of producing Claude and Codex.

Seth: All super well-taken points, and the little empirical exercise on the App Store releases is a nice cherry on top. In the framework of their model, the explanation for why we get much less output of new products than lines of code is a complementarity with a human at one of those production process layers. But as you suggest, there are a lot of different things that bottleneck might be, or lots of reasons why economic value being created is not showing up in that final measure. So — one more reason to take with a grain of salt the idea that this gives us strong evidence of complementarity between AI coding agents and humans.

Andrey: So do you want to move on to the posteriors?

Seth: [1:00:07] For those of you playing along at home, now is your chance to think about how this conversation has changed your priors. This chance to contemplate your posteriors is sponsored by Revelio Labs. Revelio Labs is a leading provider of labor economics data and data services for companies, academics, and independent researchers. Andrey and I have been working in economics of AI, digitization, and automation for a long time, and we can confirm just how useful Revelio’s data is. Revelio’s team combines comprehensive micro-level data on employee professional profiles, job postings, and employee sentiment with standardizations, mappings, and enrichments available, all to make that data useful without making your modeling decisions for you. The data can be flexibly aggregated to company, market, or industry, and can be used to study questions ranging from career trajectories to occupational transformation to the returns to skills and the impact of AI on labor demand for tasks. Can’t imagine anyone who would be interested in that. And Revelio data is available on WRDS. So if you’re an academic with a good library, go see if you have access to their premier data already. And if you don’t, you can reach out to their excellent economics team and they’ll hook you up.

Okay, so Andrey, remind me — what were our posteriors?

Andrey: [1:01:32] What were our priors? Mine was 80% on the effect on developer lines of code. I’ve updated to probably 95% at that point. Does AI use boost the true economic value of software developers by less than 50% of the boost to lines of code? I’ve become more confident, so moving up to 95% there. Is AI coding a gross complement or substitute? I still think it’s a complement. I think this paper mildly supports that it is, at least in the relevant sense I was talking about. What about you?

Seth: [1:02:10] To this paper’s credit — and I had some sharp words about the model — I think this paper actually does move my priors considerably. That’s the best thing I can say about a paper.

On the first question, AI boosting lines of code written by greater than 100%, that moves me from 95% confidence to 99% confidence. It’s great to see these giant values.

For AI boosting economic output by less than 50% of the boost to lines of code, I’m going up from 95% to 97.5%. Again, this confirms my priors.

And then the most subtle question. This paper really only moves my view about complementarity between AI coding agents and humans at one very particular layer of the production process. I said there was maybe a 20% chance of complementarity between ordinary coders and AI coding agents. That’s unchanged. For human engineering departments, that’s going up from 33% to 40%.

For software companies, that’s where I move the most. I’d move from a 60% chance of complementarity at the software company or open source project level to 85 or 90%. That’s a big move for me. Big-time move in the priors. And finally, for the economy as a whole, I’m unchanged.

Andrey: Thanks for joining us for another episode of Justified Posteriors. Always remember to keep your posteriors justified, and to like, comment, and subscribe to our podcast. Thank you.

This essay is related to ongoing research with Stella Si, Giorgos Zervas, Mert Demirer, Aaron Kaye, and Nadav Tadelis. This analysis is vibe-coded and should be considered preliminary. More rigorous work on this topic is in progress.

Epistemic status: Possible.

A pivotal question for the AI industry is whether frontier models can sustain a large price premium over open-weight and other cheaper models. If they can, frontier labs may capture much of the industry’s profits. If models at different capability levels are close substitutes, more of the value may instead accrue to other parts of the AI stack.

The answer depends partly on whether users experience diminishing returns to model capability. If each additional increment of capability produces a smaller improvement in user value, willingness to pay for frontier models will also rise more slowly. That would compress frontier-lab margins and weaken the incentive for increasingly expensive training runs.

While many have speculated about competition between LLMs at different capability levels, few have tried to measure it. I present preliminary descriptive evidence consistent with diminishing returns to measured model capability among consumer users. The results suggest that frontier labs may ultimately need to earn rents from products and complementary services, not solely from access to the most capable model. I end by considering what those services might be.

Measuring intelligence

There are many definitions of intelligence. Some are so broad as to include nearly anything and imply that the maximum level of intelligence achieves omnipotence. The AI industry typically measures intelligence through benchmarks and model characteristics. Existing benchmarks become saturated over time, and new ones are continually released. New frontier models earn their status by making the benchmark numbers go up.

I take the LLM paradigm as given and use standard metrics of model capability and scale to illustrate my argument. As a result, this essay does not concern new types of AI models, including those that involve continuous or active learning, world models, or robotics.

I use three proxies for model capability and scale because no single metric is perfect.

• The number of model parameters: The most objective of the three metrics, because it concerns the properties of the neural network rather than its performance. For closed-source models, I use estimates of model size from the literature.

• The Artificial Analysis Intelligence Index: An average of many benchmark scores. It is a simple measure but risks becoming saturated.

• The Epoch Capabilities Index: An index that uses item-response theory to rank models. Importantly, it accounts for the fact that some benchmarks become saturated while others do not.

A note on scope: These measures capture raw model capabilities rather than the performance of products built around them. I therefore do not count scaffolding innovations such as Claude Code as model intelligence. Scaffolds can work with a variety of underlying models and are conventionally excluded when benchmarking frontier models.

Diminishing returns to intelligence

The simple intuition

Introductory economics students are often taught about diminishing marginal utility through the example of pizza. The first slice is amazing, the second is good, the third is fine, and the tenth is unpleasant. A similar phenomenon appears in business settings. Consider data science: improving an inventory forecasting model from 60% to 90% accuracy may be much more valuable than eking out the last few percentage points through extensive specialized modeling and PhD-level expertise. Diminishing marginal utility appears in many economic settings, and there is little reason to assume that model capability is an exception.

Why might measured model capability exhibit diminishing marginal returns? Once a model is capable enough to complete a task, additional capability may add little value. A given use case—say, a consumer chatbot—requires a model to attempt tasks of varying difficulty. As capability increases, the model can complete more of them correctly. But once most relevant tasks can be completed, other attributes, such as reliability, speed, and personalization, become more important. Benchmark saturation illustrates this threshold: beyond a certain capability level, nearly all leading models score close to perfectly. The empirical question I investigate is how much additional end-user utility comes from raw model capability rather than from these other attributes or from scaffolding.

Evidence from Arena AI

The ideal way to measure diminishing utility from model capability is to study people’s choices between outputs from models at different capability levels. Although comparable data from AI labs are not publicly available, Arena AI (formerly LMArena) collects users’ stated preferences over model responses and publishes a leaderboard. A user is shown two responses and votes for the one they prefer. These votes are used to calculate a publicly available Elo score. My analysis of Arena AI scores is illustrative, and I discuss its limitations at the end of the post.

Below, I plot the Arena AI score against the three proxies for model capability and scale: log parameters, the Artificial Analysis Intelligence Index, and the Epoch Capabilities Index (ECI).

Three observations stand out:

1. Greater measured capability is associated with higher user preference. The Arena AI Elo score increases with all three metrics.

2. The improvement associated with greater measured capability is gradual. A 10-point increase in the Artificial Analysis Intelligence Index is associated with an average increase of only 7 percentage points in the win probability implied by Elo. Users therefore still prefer the lower-scoring model’s response a substantial share of the time.

3. The relationship appears to exhibit diminishing returns, although the evidence varies by measure. In Panel A, log parameters are approximately linearly related to Elo. This implies diminishing returns to model scale: achieving the same increase in user preference requires roughly a tenfold increase in parameters. Panel B shows a similar flattening in the relationship between the Artificial Analysis Intelligence Index and Elo. The gains are large when moving from older Llama models (around 5 on the index) to GPT-4-class models (around 10), but the slope is modest between index values of 10 and 45. The most recent models—including Fable 5 and Sol—barely improve on the Elo scores of older, lower-scoring models such as Opus 4.6 and Grok 4.20. Panel C, which uses the ECI, provides weaker evidence of diminishing returns, although several current frontier models, including Fable 5 and GPT 5.6 Sol, fall below the fitted linear relationship.

Arena AI also breaks out its leaderboard by task category. The next figure shows how the preference advantage associated with greater capability varies across prompt categories. Each cell reports the average increase in win probability when moving from one capability bucket to the next. For example, the upper-left cell indicates that models with 10–100 billion parameters have a 9.2-percentage-point higher win rate, on average, than models with 1–10 billion parameters. Reading from left to right within each panel shows how this relationship changes at higher levels of the corresponding metric.

The shape of the returns to measured capability differs across categories. Math, for example, appears to exhibit relatively high returns compared with most other categories. By contrast, and perhaps surprisingly, coding queries appear to benefit relatively little from moving from the second-highest to the highest capability bucket. This heterogeneity suggests that diminishing returns may emerge at different capability levels for different use cases.

Reasons frontier models may remain valuable even with diminishing returns

Recursive self-Improvement

If end users experience diminishing marginal utility from additional intelligence within existing applications, frontier model developers must continually discover new and increasingly valuable uses for it. The most widely discussed (and meta) example of such a use case is recursive self-improvement (RSI), in which AI systems contribute to improving their own capabilities. If this process unfolds rapidly, it could generate gains in intelligence large enough to overwhelm ordinary diminishing returns. Even then, however, RSI would need to produce a persistently faster rate of improvement at the frontier than among models and firms attempting to catch up. A permanent speed advantage seems unlikely, given the difficulty of keeping secrets and the fact that even intelligent AI researchers face diminishing returns to research effort.

High-value problems and test-time compute

A second set of use cases involves exceptionally difficult problems and massive amounts of test-time compute. Noam Brown has emphasized that we have not yet seriously tried spending millions of dollars on a single inference, allowing a frontier model to reason for an extended period about one especially important problem. It remains unclear which problems would justify such expenditures, but leading candidates lie in areas such as biology and energy. An AI system capable of producing patentable cancer treatments or radically more efficient energy technologies could create enormous economic value. Model developers might capture that value through much higher inference prices, through licensing or intellectual property, or by vertically integrating into the industries in which the discoveries are applied. Anthropic’s recent acquisition of Coefficient Bio is suggestive of movement in this direction.

Zero-sum competition

Another set of use cases in which the returns to intelligence may not diminish sharply is when the user is engaged in a zero-sum competition. Whenever the best reasoner captures the prize, as in chess, financial trading, litigation, cybersecurity, or military strategy, what matters is winning and in many cases the best intelligence wins disproportionately. A small capability advantage may therefore remain valuable even when all participants are already capable. In such settings, demand for frontier intelligence may resemble an arms race: each participant must continue purchasing better reasoning simply to avoid falling behind.

The simplicity of one strong model and product

A final reason to use an expensive frontier model may, ironically, be cognitive overhead. Suppose a user performs a mix of tasks, some difficult and others relatively easy. The user may know that cheaper, less capable models could handle many of the easier tasks equally well, but may not want to spend time deciding which model to use for each request. Provided that the frontier model is not prohibitively expensive and comes packaged with a good interface, the user may therefore prefer to use it for everything, effectively paying a premium for convenience and consistency.

The size of the premium from such a bundle, however, is limited by the availability of increasingly capable routing technologies. If a routing system can identify the difficulty of each task and assign it to the least expensive model capable of completing it well, users can obtain much of the convenience of relying on a single frontier model without paying frontier prices for every query.

Even sophisticated routers are unlikely to eliminate the frontier premium entirely. Routing decisions are imperfect and users may therefore rationally favor systems that err toward more capable models, particularly when reliability matters a lot. In this case, frontier models may retain value as a form of task difficulty insurance.

Limitations of this analysis

This analysis is meant to be illustrative and to encourage AI labs and other companies to publish more work measuring these curves. Below are some reasons to be skeptical of my analysis.

Representativeness. Arena AI users are unlikely to be representative of the broader population of AI users, and the data comes from single-turn rather than multi-turn conversations. Business use cases may also exhibit different patterns from those observed here.

Causality. The analysis is descriptive and does not establish that greater model capability causes higher Arena scores. Models with different measured intelligence may also differ in response speed, style, or default reasoning budgets. If these characteristics affect user preferences and are correlated with measured capability, the observed relationship cannot be interpreted as the causal effect of capability alone.

Test-time compute. Arena Elo scores and intelligence metrics do not hold inference-time compute fixed. Many models have “thinking” or “reasoning” variants that spend more tokens (and dollars) before answering, and those settings often improve both benchmark scores and Arena win rates. A flatter Elo-versus-intelligence curve could therefore partly reflect differences in how much test-time compute each model was allowed, rather than diminishing returns to underlying capability alone. Benchmarks and Arena battles may also evaluate models under different default reasoning budgets, so the x- and y-axes are not always measured under the same compute regime.

Imperfect model matching. Linking Arena entries to Artificial Analysis, Epoch, and parameter estimates requires matching model names across sources that use inconsistent naming conventions. Matching can be imperfect for several reasons: the same weights may appear under slightly different labels (preview vs. release, date-stamped Arena IDs vs. catalog names); thinking and non-thinking configurations are sometimes scored separately on Arena but share a pooled model-group intelligence score elsewhere; closed-source parameter counts are estimates rather than confirmed figures; and some aliases are judgment calls when no exact identifier exists. When a match is wrong or only approximate, a point on the scatterplot pairs the wrong Elo with the wrong intelligence or size measure, which adds noise and can distort the apparent slope.

Comments welcome! Thanks to Tom Cunningham, Alex Imas, Abhishek Nagaraj, and Soumitra Shukla for early feedback.

Please cite:

@misc{fradkin_diminishing_returns,
  author       = {Fradkin, Andrey},
  title        = {Are There Diminishing Returns to AI Intelligence?}},
  howpublished = {Substack},
  year         = {2026},
  url          = {https://empiricrafting.substack.com/post/207306437}
}

We start in Washington: what politicians are actually asking about AI, the case for “no regrets” economic policy, and why the unemployment insurance system “is not prepared for someone to sneeze within 50 feet of it” — the tech, the financing, and Mississippi’s $200-a-week maximum benefit. Along the way: why almost no policy “pays for itself” (except funding the IRS), whether one hacker per state plus Claude Code can fix government IT, and the Anthropic finding that agentic coding rewards domain expertise, not coding skill.

Then the big empirical question: is AI already taking our jobs? Martha walks through the Budget Lab’s labor-market tracking — no sign of broad macro disruption yet — and why the narrative says otherwise: 1.7 million layoffs in a normal month, CEOs with an incentive to blame AI, and Challenger data attributing seven times more layoffs to AI than to tariffs (”this is implausible”). From there we turn to her Senate testimony and Atlantic essay on the national debt: deficits since 2015 are already costing new mortgage holders about $2,500 a year, and “AI will grow us out of it” is a bet — one the Budget Lab has actually modeled. We close with token taxes, sovereign wealth funds (”the great thing about the government — we can tax it”), what CEA is really like from the inside, and a lightning round that ends in a Red Rising roast.

Links & References

Martha’s work

• Martha Gimbel — The Budget Lab at Yale · budgetlab.yale.edu

• Martha’s Atlantic essay on how deficits are raising costs for households.

• Testimony before the Senate Finance Subcommittee on Fiscal Responsibility and Economic Growth — “The Fiscal Outlook: 2027–2036” hearing (March 11, 2026): debt held by the public ~99% of GDP in 2025 → 120% by 2036 → 175% by 2056

• Evaluating the Impact of AI on the Labor Market: Current State of Affairs — the Budget Lab’s occupational-mix tracking; no sign of broad AI disruption in the macro data yet

• What Might AI Adoption Mean for the Fiscal and Economic Outlook? — the AI-and-the-debt scenarios built on the Karger et al. expert forecasts; the fiscal gains are not a free lunch once you add support for displaced workers

• Abhi Gupta, The Impact of Deficits on Costs for Households

• The Budget Lab Small Macro Model (BLSMM) — the open, interactive macro model discussed in the R-vs-G section

• Long-term Impacts of the One Big Beautiful Bill Act — ~zero growth impact at 10 years, negative at 30 (crowding out)

• Coming soon from Martha: a token-tax piece in Tax Notes, and Budget Lab work on AI, capital taxation, and sovereign-wealth-fund economics — stay tuned

Concepts, papers & people discussed

• Anthropic, “Agentic coding and persistent returns to expertise” — the Claude Code study Martha cites: domain expertise, not coding background, predicts success with AI agents

• Challenger, Gray & Christmas layoff announcements — the data attributing ~7× more layoffs to AI than to tariffs in 2025

• Brynjolfsson, Chandar & Chen, “Canaries in the Coal Mine? Six Facts about the Recent Employment Effects of AI” — covered previously on the podcast.

• Olivier Blanchard, “Public Debt and Low Interest Rates” — the “if r is low, debt has no fiscal cost.”

• Danny Yagan and Neil Mehrotra — the Budget Lab’s designated R-vs-G thinkers

• The Windfall Trust — the AI economic scenario-planning (”war gaming”) exercises Andrey asks about

• The ROAD to Housing Act — “Congress is doing something obviously good. Fingers crossed.”

• Reinhart–Rogoff and the Excel error — Seth’s aside on debt-threshold doom predictions

• The Council of Economic Advisers and the Joint Economic Committee — partisan vs. unbiased, and the puppy-distribution test

• Trade-offs of the pandemic UI plus-up — the $600 flat add-on existed because state systems literally couldn’t compute 90% wage replacement

Sci-fi corner

• Becky Chambers, The Long Way to a Small, Angry Planet — Martha’s pick for the future we actually want (and a direct appeal: Becky, the people want to know what’s next)

• Adrian Tchaikovsky, Children of Time — “so good”; the one with the spiders

• Pierce Brown, Red Rising — “angry Harry Potter”; sorry, Basil

Our sponsor

• This episode is brought to you by Revelio Labs, providing data products useful for many questions about the economy.

Chapters

• (00:00) Intro & sponsor

• (00:59) What politicians are actually asking about AI — and the case for “no regrets” policy

• (02:30) D.C.’s misconceptions: politicians are still learning the tools

• (03:22) Scenario planning, war gaming, and automatic stabilizers

• (04:43) “The UI system is not prepared for someone to sneeze within 50 feet of it”

• (05:54) Three reasons UI is broken: the tech, the financing, and Mississippi’s $200/week max

• (08:53) Should we federalize it? The IRS, and why almost nothing “pays for itself” (the 7× rule)

• (11:28) Why politicians can’t do the obviously good thing: concentrated pain, diffuse benefits

• (12:46) Can one hacker per state + Claude Code fix government IT? DOGE, caves full of paper records, and returns to domain expertise

• (16:58) Is AI already affecting the labor market? What the macro data shows (and doesn’t)

• (19:05) Why the narrative says otherwise: youth outcomes, wrong numbers, and 1.7 million layoffs in a normal month

• (22:22) Challenger data: 7× more layoffs attributed to AI than tariffs — “this is implausible”

• (23:16) What is behind the entry-level slowdown? The low-hire, low-fire puzzle

• (25:16) Will we know it when we see it? Weavers, export controls, and the Napoleonic Wars

• (27:54) “I can’t travel to Earth 2”: the counterfactual problem, even at the Budget Lab

• (28:43) The fiscal outlook: debt from ~100% to 170%+ of GDP — and why thresholds aren’t the point

• (30:44) Fiscal crisis risk, sweet sweet T-bills, and the Blanchard low-rates argument

• (32:05) Why didn’t we issue 100-year bonds at 2%?

• (32:44) Crowding out: deficits since 2015 ≈ an extra mortgage payment every year

• (33:55) Housing affordability, interest rates, and mortgage lock-in

• (37:14) What deficit spending crowds out — and the diapers-bill standard for government spending

• (40:25) Fiscal gap accounting vs. talking so D.C. actually understands

• (44:26) “AI will grow us out of the debt”: R vs. G and the Budget Lab’s AI fiscal scenarios

• (47:05) Who pays when the robots work? Payroll taxes, capital taxation, and AI finding every loophole

• (48:33) Is capital more or less elastic in the AI age? Token taxes and the IRS

• (52:16) A sovereign wealth fund for AI? “The government doesn’t have to own things to get money from things”

• (55:41) Fairness and the nerd’s-nerd case for simplifying the tax code

• (56:47) Inside the JEC and CEA: partisan vs. unbiased, and the puppy test

• (1:00:34) What working at CEA feels like: second best, third best, fourth best

• (1:02:06) The CEA junior staff and their “extremely benevolent and well-reasoned rule”

• (1:03:34) Budget Lab vs. Penn Wharton vs. CBO: 30-year horizons and what you’re buying with paid family leave

• (1:06:38) Private vs. public data: Indeed, benchmarking, and the shutdown’s three contradictory hiring estimates

• (1:08:31) What data do we actually want from the AI labs?

• (1:09:41) Lightning round: the biggest bottleneck to AI productivity gains (Hollywood vs. healthcare)

• (1:10:53) Gun to your head: pre-distribution or redistribute-after?

• (1:12:59) Favorite sci-fi: Becky Chambers and the future we want

• (1:14:38) Worst sci-fi takes: Red Rising, Children of Time, and a message for Basil

• (1:15:35) Sign-off

Justified Posteriors is the podcast that updates its beliefs about the economics of AI and technology, hosted by Andrey Fradkin and Seth Benzell. If we changed your priors, subscribe, share it with a friend, and keep your posteriors justified.

Transcript

What Politicians Are Asking About AI [00:00 – 04:43]

[00:00:04] Seth: Welcome to Justified Posteriors, the podcast that updates beliefs about the economics of AI and technology. I’m Seth Benzell, building, with your support, a podcast community which is hopefully more fiscally sustainable than the federal government. Coming to you from the Pocono Mountains of eastern Pennsylvania.

Andrey: And I’m Andrey Fradkin, coming to you from San Francisco, California. We are sponsored by the fine folks at Revelio Labs, providing data products useful for many questions about the economy. And we’re very excited to have Martha Gimbel with us today. Martha is the executive director and co-founder of the Yale Budget Lab, and has worked in an enormous variety of impactful roles related to economic policy. Martha, welcome.

Martha: Thank you so much for having me.

Andrey: To get started: we know that you talk with a lot of politicians and staffers. What questions are they asking you about AI?

[00:01:10] Martha: Some of the questions that politicians, policymakers, and staffers are asking are the same ones everyone is asking, right? What is going to happen? How should we think about the economic impacts of this? What is plausible? What seems unlikely? Politicians — they’re just like us. They have the same questions everyone else does.

I think the other thing is that people are really trying to figure out how much of the potential future requires something that is different than what we’ve done before. And I don’t just mean “we know that workforce training hasn’t worked particularly well in the past, so we should update how we do it” — the things that you would do no matter what — but a true paradigm shift in how we do policy. And I think that politicians are still sort of confused about that. They don’t know how much they should be thinking about a totally different way of doing things. I should say, I’m one of the people who’s been on record saying that this is a time to do “no regrets” economic policy. We don’t know what’s going to happen, and so now is a great time to do things that are a good idea no matter what. But you shouldn’t wager policy on a specific version of the world.

Andrey: What are the most common misconceptions that politicians have about AI, in your experience?

Martha: I think part of it is that a lot of people in DC are still learning how to use these tools — as, to be clear, are all of us. They’re still kind of at the baby stages there. And also, think about a typical politician’s life — and I’m not trying to say this is everyone, but think about what they’re doing. They’re talking to constituents, they’re getting briefings from staff, they’re attending a committee hearing. They actually aren’t spending that much time sitting in front of their computer trying to generate memos or do analysis — they have staff who are doing that. So some of it is just people trying to make sure that they have the time to sit down and play with these things and figure out how they work.

[00:03:22] Andrey: Have you seen efforts by, let’s say, the Windfall Trust and other organizations to do scenario planning — essentially war gaming, but for AI and economic policy? Do you have an opinion on those sorts of exercises?

Martha: In general, the scenario planning is helpful and is a good way of approaching this. It’s also one that DC is very familiar with. The great thing about scenario planning, if you’re trying to think about influencing economic policy in DC, is you’re speaking policymakers’ language. It’s also something people think about a lot in economic policy already: we talk a lot about automatic stabilizers — things like unemployment insurance that kick in when the economy is doing worse, and then trigger off in various ways, or you don’t file for them when the economy is doing better. Thinking about these types of policies that can stretch to fit different scenarios is a place that people in DC really understand, and it has been a subject of a lot of debate — how do we structure these, et cetera — over at least the last 10 years.

The Safety Net Isn’t Ready for a Sneeze [04:43 – 08:53]

[00:04:43] Andrey: How well do you think the current set of automatic stabilizers will do if we have a shock — let’s say a white-collar unemployment shock?

Martha: One of the things that is frustrating to people like me is that you will sometimes see... I’m gonna call out someone’s tweet, but fortunately for them, I don’t remember their name, so they totally get to get away with it. Someone tweeted, “People don’t understand — the unemployment insurance system isn’t prepared for the tsunami of white-collar job loss that’s about to come at it.” And all of us who worked on unemployment insurance were like, “You don’t understand. The UI system is not prepared for someone to sneeze within 50 feet of it. What are you talking about? Wave of white-collar — it can’t handle anything.” So, you know, we can have a longer conversation about how well Silicon Valley and DC understand each other. The unemployment insurance system is not in good shape.

Andrey: What are the top three reasons why it’s not in good shape?

[00:05:59] Martha: Let’s do it. First of all is the tech. Just legitimately filing for benefits, getting them processed, all of those things, is really, really cumbersome. I should say, for some states, that is a feature, not a bug — it is a way to keep from paying out benefits. This is a thing I think a lot of people experienced during the pandemic. You might assume it has been fixed. It has not. And this flows through to UI policy: there are fixes you might want to make to the UI system, but the systems are so buggy that you can’t touch them. I had friends who were parachuting in to try to be helpful to the state UI systems during the pandemic, and if you’ll forgive a metaphor that doesn’t really work, they basically opened up the door to the closet where the code is kept, and then closed it slowly and said, “Okay — if no one breathes, it might not break down. No one look at the code, no one think about the code, and it will be fine.”

There was a lot of debate during the pandemic about the plus-up to UI benefits that we did, where we just added a certain amount onto people’s benefits. That meant some people were making more on UI than they had been when they were working, and people were asking, “Is this a good policy or a bad policy? How do we think about incentives to work?” Guys — the reason we did that was that the systems literally couldn’t handle saying we’re going to do a return of 90% of your previous wages. It seems very simple, but they couldn’t handle it. Which is a very long-winded way of saying the first issue is the tech.

[00:07:47] The second issue is the financing — how people are paying in for benefits. Trust funds and things like that are not always in the best shape. And third, I think people don’t realize how ungenerous a lot of these benefits are. It varies a lot by state, but in Mississippi, the maximum — not minimum, maximum — benefit that you can get is a little over $200 a week. That’s not a huge amount of money.

Seth: It’s not paying the mortgage.

Martha: It’s not gonna pay the mortgage. And particularly if you’re worried about higher-paid workers losing their jobs, it’s a huge problem. Which is all to say: no, I don’t think the social safety net is particularly well set up for broad technological disruption. I don’t think the social safety net is particularly well set up for a standard-issue recession. I don’t think the social safety net is well set up for a 4.3% unemployment rate, which we have right now. We have let these systems languish.

Why Nothing Pays for Itself (Except Funding the IRS) [08:53 – 12:46]

[00:08:53] Seth: Is the answer to move in the direction of federalization? Is this incompetent states and the government letting them run amok?

Martha: We have federalized programs that also don’t work particularly well. Part of this is just: you have to spend the money for things to work. And who knows if we’ve allocated that in the right way, et cetera. As another example, there was a lot of money that was supposed to be going to the IRS for modernization and increased enforcement, and that’s just been eaten away and eaten away and eaten away. Money for the IRS is particularly frustrating to budget nerds. To get a little bit nerdy about the budget for a second—

Andrey: Do it.

Martha: Everyone always thinks that their preferred policy will, quote-unquote, “pay for itself.” You see these things: if we spend a dollar on... I don’t know, name a policy you like.

Seth: Corporate tax reform.

Martha: Well, corporate tax reform might pay for itself, but I mean on the spending side.

Seth: Schools.

[00:10:00] Martha: Yeah — “additional dollars for schools will pay for themselves.” And from a societal perspective, that is often true: you will have a more highly educated populace, it affects crime, all of these different things. But in order for a policy to pay for itself from the budget perspective, it has to induce economic activity that is basically seven times the size of what it costs — because we don’t tax the whole amount.

Seth: Right, that’s just the share of GDP that’s taxed.

Martha: Exactly. And so it is almost impossible to have a policy that pays for itself. Funding the IRS does. If you think about what funding the IRS does: one, it makes things easier — everyone’s always annoyed during tax time when they have to call the IRS and they’re on hold for hours and hours. It also allows them to go after more complicated cases that require more resources, but also where you get larger amounts on the back end. And just to state explicitly: when they go after these cases and get this money, this is money that we as taxpayers were owed. People owed us this money. It’s our money. Pay it to us. It’s not a new tax being levied. But even so, the IRS has been starved of resources.

[00:11:28] Seth: Why is it impossible to get politicians to do the obviously good thing?

Martha: That is the eternal question in DC. First of all, just to state explicitly: the obviously good thing can be very bad for a small group of people, and they’re going to be much louder about it than everyone else. The benefits of improved tax collection are diffuse — we all benefit from that, but I don’t see it. No one sends me a thing that says “the IRS went after this many tax scofflaws this year, and that means...”

Seth: JD Vance is on it.

Martha: But for the people they go after, that’s a real concentrated pain. They have to pay more in taxes, they didn’t wanna pay that, and so they’re gonna be much louder about it — which is always a problem. And yeah, we’re gonna have to think about state governance and capacity issues. They’re also just not very sexy. You go to your constituents and you’re like, “I’m really making sure that the code that operates our UI system has been updated — please return me to office...” No one cares. But it’s important.

Can AI Fix Government? DOGE and Domain Expertise [12:46 – 16:58]

[00:12:46] Seth: Martha, some people have told me that AI makes coding way easier. Is there some sense in which AI will just solve this problem? Because we’ll just get one hacker in each state, plus, you know, Claude Code Opus, and we’ll knock one of these out in an afternoon?

Martha: To be clear, I am hopeful that AI will make some of these issues easier. I also think, as we all know, any kind of AI implementation is never that easy. You also have to think — with government, so much of this stuff has been neglected for so long. There were all those stories last year about a bunch of government retirement records which are still on paper in a cave, I think in West Virginia. This is a broader issue that comes up: people say, “Oh, we can just use AI to solve this,” or some other thing that seems easy, and it’s like — well, wait a minute. Let’s get in there. You have to actually understand the environment in which you’re operating. You have to understand the existing issues. For instance, if you’re the IRS, there are huge privacy concerns — you need to make sure that you’ve really locked up all the data. I can feel people rolling their eyes at me, but it’s a real concern.

[00:14:04] And frankly, you saw this with DOGE. There were people in DC who, frankly, were kind of excited about DOGE — “a bunch of people who understand technology and have a mandate to operate, and we might actually fix these things!” ...and we’ve closed down USAID. Okay. It’s not that people in DC are hostile to doing this. It’s just that it’s complicated, and you have to cut through a lot of red tape and barriers and a lot of years of doing things like putting records in a cave in... I apologize if the cave is not actually in West Virginia. I cannot remember where it is. But it’s somewhere.

Seth: I was at a conference where I actually talked to some DOGE-ers over this past weekend — I don’t know what they call themselves. Something like this is never values-free, because you can palpably see how excited they are to fire people, right? And so, yes, obviously if we can have AI do a lot of the work of the government, we’re gonna need fewer federal employees. But it doesn’t come across as credible.

[00:15:14] Martha: Or alternatively, you free up a bunch of people at the IRS to actually help people with their tax returns. This gets at the whole debate about AI and labor markets and what’s going to happen: if you’re freeing up certain capacity, do people lose their jobs, or do they switch into doing other types of things that are higher value? I also think — Anthropic actually just put out a piece on this last night, or this morning, I’m not sure — about what they’ve seen with the use of Claude Code: there’s this increasing return to domain expertise. If you just send people in to work on something who understand the tech but don’t understand the broader context in which they’re operating, it’s not nearly as effective. So I do think you really need to make sure that you’re having these conversations with people.

Not to keep picking on DOGE, but just as an example — I remember there was this story that they found all this real estate that wasn’t being used, and so they’re like, “We’re gonna sell it and put it up for auction, and isn’t this great?”

Seth: It’s free real estate.

Martha: It’s free real estate! We should do something with it! And look — there are all of these random sites in Northern Virginia that no one says what they’re for, and everyone’s like, “Take the list off the internet. Take it off the internet. If you don’t know what something is in Northern Virginia, go into a SCIF — a classified area — and ask someone. Don’t put it on the internet.” The list came down in five minutes. But it had already been put up.

Is AI Already Affecting the Labor Market? [16:58 – 23:16]

[00:16:58] Andrey: This is a natural transition point. Your lab has been one of the key groups documenting what’s going on in the labor market in real time. So — the big question that everyone’s asking: is AI already affecting the labor market, and if so, how? Is it gonna take our jobs? Or has it already taken our jobs and we don’t know yet?

Martha: I would say: I don’t know if it’s going to take your job. There is currently no sign that AI is causing broad disruption in the labor market at the macro level. This does not mean that no one has lost their job to AI yet. This does not mean that no one’s job has changed due to AI — I wanna give all these caveats here. What it means is: if you’re looking at the macro data about the economy, it doesn’t seem to be the case that AI is causing broad disruption at this time. I do want to say, I think this take sometimes gets me labeled as an AI Luddite or an AI skeptic—

Seth: God forbid.

[00:18:11] Martha: —which I think says something about where this conversation has been. This technology is still very, very new. The ChatGPT that was released in November 2022 is not the same as the tools that we have today. And it is interesting to me that people assume that after only a couple of years, we would have this kind of broad disruption that’s easily measurable. These things just take time. Have you ever met a compliance department? They have questions. So I think it’s really important to be clear about what we’re seeing and not seeing now, while also not dismissing the potential for disruption in the future.

Andrey: I totally agree, by the way. I think in the media, the way this gets reported is oftentimes frustrating. They’ll take companies at their word, at face value, when they say they’re firing due to AI. There are all sorts of prominent people already saying that the youth can no longer find jobs, when, as far as I can tell, that’s not at all what’s going on. Why do you think this is happening, and is there something to be done about it?

[00:19:40] Martha: First of all, I think there are just things that people get in their heads that feel right to them. In 2022, my friends kept asking me when the recession was going to be over, and I kept saying, “We’re not in a recession.” And they were like, “What are you talking about? We’re definitely in a recession.” I don’t know what to tell you — GDP growth is positive. So there’s the question of how people feel versus what is happening.

On the youth, and on the headlines: on the youth side, it is legitimately the case that you have seen some deterioration in outcomes for younger workers. This seems to be coming because overall we’ve had a slowdown in hiring, which, again, does not seem to be related to AI — but it’s not something that labor economists understand particularly well. And that’s just frustrating to people, understandably. I graduated in 2009. My friends and I had a really hard time finding jobs, but we knew why. It was very clear. Whereas now it’s like, “Oh man, things aren’t looking so great. No idea. Good luck out there — have a good time!” So people are searching for an explanation. I also think people have wrong numbers in their heads. I’ve heard senators talking about 20, 30% youth unemployment — which, to be clear, we do not have.

On the CEO announcements: part of this is just that people don’t have context in their heads. You’ll see these things like “X company lays off 10,000 people due to AI.” Well, first of all, we have a relatively low-layoff environment right now, and we still have 1.7 million layoffs every month. So 10,000 is just not that many. To be clear — if you are one of those people who’s gotten laid off, it is terrible. It is really hard for you, it is really hard for your family. I’m not dismissing that. But from an economy-wide perspective, it’s just not that big.

[00:21:40] There are also huge selection effects in what gets covered, and in who makes announcements. The hardware store down the street, if it starts hiring a bunch of people, doesn’t make an announcement about it. And if it fires a bunch of people, it doesn’t make an announcement either. But these big companies do, and so that gets a lot of coverage — but they’re not the totality of the US labor market. And again, there’s also just an incentive to ascribe any layoffs to AI. It gets covered weirdly kind of positively for your company — you’re forward-looking, you’re doing all the things.

Seth: I mean, not weirdly. You’re lowering costs, right? It should be positive.

Martha: Well, but also people say, “Oh, they’re a forward-looking company making investments in tech for the future.” There’s this data from this company called Challenger — which has some pros and cons — but what they do is gather layoff announcements, and they attribute the layoff announcements to a source. And between April and November of 2025, about seven times as many layoffs were due to AI, according to Challenger, than tariffs. This is implausible. It just is. But if you think about it — if you’re a CEO, which one is more fun to put in your announcement? There are huge incentive effects here, and I think people aren’t thinking those through, and they aren’t thinking through the scale, and they aren’t thinking through the selection effects in both who makes layoff announcements and whose layoff announcements get reported.

The Entry-Level Puzzle and the Limits of Attribution [23:16 – 28:43]

[00:23:16] Seth: But thinking about that attribution challenge — if you had to come up with a story for why there seems to be a differential job market for entry-level workers versus more advanced workers, would you point to work from home? Would you point to increased interest rates, and some sense in which the worker has to pay off sooner? What else would you point at?

Martha: The honest answer is I don’t know. There have been people pointing to work from home — there was a new paper out on that. Interest rates in general have probably slowed down hiring, which is going to affect young people more than older people, because older people tend to have jobs and young people tend to need their first job — so hiring is more important for them. But this kind of gets at the point: a bunch of economists have been debating why we’ve been in this low-hire, low-fire labor market for a couple of years now, and we’re just not sure. There’s been this question of — the unemployment rate’s been about 4.3% for a while. Are we at equilibrium? Is this a labor market that’s at that magical soft-landing, just-chugging-along point? In that case, you’re just not gonna have that much hiring — and that’s going to be bad for young people. That’s one theory. We can’t prove anything.

I think part of the reason why the “AI is holding back hiring for young people” theory has really taken off is because we don’t have an alternative. We don’t have an “oh, it’s actually this thing.” We don’t know, and so of course people are going with an explanation. I should also say: it is not insane to think that AI will affect young people first. That’s a very plausible outcome — it’s what we’ve seen in past technological change. It’s not some totally cuckoo theory that people are working with here. It’s just that it doesn’t really seem to be showing up clearly in the data.

[00:25:16] Andrey: There’s a paper that we’ve covered on the podcast, “Canaries in the Coal Mine,” and I don’t think we need to rehash that. But my sense has been that if it’s really starting to affect the labor market, we’ll know — we’ll see large shifts in overall employment rates. Is that kind of your sense of the AI impact on the labor market?

Martha: I am just constantly the person who’s like, “It’s hard, and who knows what we’ll know, and who knows where this is going?” This is why no one likes hanging out with economists. But if you think about the Industrial Revolution, and you think about the weavers who lost their jobs — that was really clearly due to technology, right? They didn’t need a Bureau of Labor Statistics to tell them they were losing their jobs to technological change. However, what I just said is actually not quite correct. One of the things going on at the time was the French Revolution and the Napoleonic Wars, and England actually put export controls on the textile industry. They couldn’t export.

Seth: The Continental System.

Martha: It’s a problem! So they sent the industry into recession, and then you had technological change on top of that. How much of the job loss that happened very quickly was about the technology? How much was about the export controls? How much was about some third factor that I don’t know about, because this isn’t my area of expertise? I don’t know. All of this gets very, very complicated — all of these factors going in and out, compounding and building on each other. The economy doesn’t happen in a vacuum. I think part of the problem here is that everyone wants to know right now where this is going, who’s going to be affected, and how and when. And so much of this is probably only going to be clear in retrospect. Which is not a fun answer.

[00:27:18] Andrey: And it might not even be clear then, right? There was this recent Wall Street Journal article that I think you were a part of, where people were asked what the net effect of AI on jobs is gonna be. I know I just said we’ll know — I think we’ll know in very specific cases, if certain occupations truly are automated. But I think we won’t know the general counterfactual. It’s not like we’re gonna be there five years from now and know the exact trajectory of the economy without AI. I don’t know how we’d ever get at that.

Martha: People ask me all the time, “Has hiring at Budget Lab been impacted because of AI? Are you hiring fewer research assistants?” And I say no — how people do their jobs has changed, the tasks that people do have changed, but the number of people that we’re hiring has not changed, as far as I can tell. But you shouldn’t ask me that question! I don’t know! I can only see the version of Budget Lab where AI happened. I can’t travel to Earth 2, which has no AI, and see how many research assistants we hired. I have no idea. We want this kind of certainty from people that is just really implausible.

The National Debt: Why the Trajectory Matters [28:43 – 33:55]

[00:28:43] Andrey: Okay, this is a good time to move on to our next topic. Seth, you wanna take it away?

Seth: I’m super excited to dig into this next topic, ‘cause you’ve been speaking a lot about it recently — you had a recent Atlantic article, and in March you spoke to the US Senate about the fiscal outlook over the 2027-to-2036 time horizon. You seem to be concerned about the national debt. Can you tell us a little bit about our current trajectory there?

Martha: Everyone always worries about debt and deficits when people are talking about spending — by which I include both literal spending and also tax cuts — on things they like. “Deficits are fine if it’s on the thing I like.” I often have this experience where I’ll talk to people about deficit spending and how it can increase interest rates, and I’ll give the example of a specific policy, and someone will go, “Well, wait a minute — but that’s a good policy. We should do that.” Sure. But if we think we should do it, we should pay for it.

[00:29:46] Seth: So the number that you put in your recent Atlantic article — and perhaps this was for your Senate testimony — was that the debt-to-GDP ratio in 2025 was almost 100% in terms of debt held by the public. By 2056 it is projected to reach over 170%. Why is that bad, Martha? Debt’s good.

Martha: So I actually think this is a really important thing. One, I actually don’t think that specific percentages are necessarily bad.

Seth: Ooh, you get snaps for that.

Martha: There has been a problem in the debt-hawk conversations in DC where people would go, “If we get to this level, it’s all gonna fall apart.” And then we hit that, and it’s still okay.

Seth: It turns out that was an Excel spreadsheet error. Reinhart and Rogoff.

[00:30:44] Martha: Yes. But you still have this thing where people said, “It would be really bad if we got to 100% debt to GDP” — and the sky has not yet fallen. I think there are two reasons to worry about this. One is that you do risk a fiscal crisis, where markets all of a sudden go, “Oh no, no, no — we don’t like it.” Those aren’t predictable. They could happen at any time, and they’re very, very expensive if they happen. You’re not gonna like what we have to do if we run into a fiscal crisis. The US is weird for all sorts of reasons — we are not even like the United Kingdom — and I think we’ve been really relying on that for the fact that we’re not gonna have a fiscal crisis. Markets don’t have a good place to go that’s not the United States.

Seth: There’s an infinite demand for T-bills. I mean, if they want the T-bills so much...

Martha: Yeah, those sweet, sweet T-bills.

Seth: Actually, the sophisticated version of that argument is the Olivier Blanchard argument from a bunch of years ago — the idea that if safe interest rates are super low, there’s some sense in which there is no fiscal burden to the debt. People just really, really are looking for safe investments. Give it to the market — you’re not actually hurting anybody.

Martha: Sure. And then now we have higher interest rates, and so everything is much more costly. Right? You make a bet.

[00:32:05] Seth: Why weren’t we issuing 100-year bonds 20 years ago — or whatever, 15 years ago, when we were at zero? Why did that never happen?

Martha: I am not a treasury person, so I should really not be talking about this. I’m going to anyway, and probably gonna get this wrong. My understanding was, one, it ended up being much more complicated than people thought it was going to be — you have to figure out market demand and all of these things. This is another example of: you have to talk to people with actual domain expertise. We can all sit here and say, “We should’ve been issuing these really long-term bonds,” and it’s actually much more complicated.

Seth: We were borrowing at negative real rates. It seems like a good idea. Okay. All right. Killjoy.

Martha: Sorry! Sorry. You should talk to the treasury markets room about how complicated these things get.

[00:32:44] I think one of the things that we’ve been trying to really emphasize at Budget Lab is there is this evidence that as debt-to-GDP goes up, it puts upward pressure on interest rates, because of what economists call crowding out. There are so many people who wanna buy so much debt; the Treasury Department issues more debt; all of us poor little people who wanna take out mortgages now have to compete against the great big US government. Some research by Abhi Gupta here at Budget Lab says that if you’re taking out a new mortgage today, Congress’s fiscal actions since 2015 have raised your average mortgage payment by about $2,500 a year. On an average mortgage, that means you’re basically making an extra payment each year. So congratulations to everyone taking out a new mortgage. And I think this gets to the point: there are costs to this. You don’t have to wait for a fiscal crisis for things to be a problem. There are still costs.

Mortgages, Housing, and Crowding Out [33:55 – 39:56]

[00:33:55] Seth: I wanted to pick you up on this emphasis, because in both your Atlantic piece and your Senate testimony, you really emphasized the cost of borrowing for homeowners. One takeaway I had was: this means that instead of talking about the fact that our national debt could fill all 32 NFL stadiums with two tiers of construction pallets filled with hundred-dollar bills — I love those — or that we’re getting hit with a meteorite the size of Texas made of hundred-dollar bills, we should be talking about how deficit spending is making it harder to pay our own bills. So help me understand something: is this actually the substantive argument, or is this a rhetorical move? Because when I think about it, on net, Americans are creditors. And if on net we’re creditors and have assets, don’t we on net want high interest rates — or at least the old people who actually vote?

Martha: The old people may be fine with higher interest rates. But we are still talking about a situation where there’s a huge emphasis, for instance, on housing affordability. One of the things that is hard about housing affordability is that one of the best tools we have for it is to increase housing supply.

Seth: We’re passing the Road to Housing Act as we speak, right?

Martha: I believe we are, yes. Has it passed?

Seth: Congress is doing something obviously good. Fingers crossed.

[00:35:18] Martha: But increasing housing supply takes time. This is not to say that we shouldn’t do it. But it’s not the case that we pass this bill, or a locality improves zoning, and housing affordability immediately improves. It takes time to build a house. They are building a house on my street right now. They have been building it forever. But interest rates are a lever we can pull to improve housing affordability.

Seth: How strong a lever is it? Doesn’t it just bid up the housing prices?

Andrey: Yeah, I don’t think it matters at all — I think it gets passed through.

Martha: You don’t think it matters at all? I mean, yeah, that’s fair — there’s supply and demand. But I do think it makes a difference. People don’t like high interest rates.

Seth: That’s not obvious to me. I think the old voters might like high interest rates.

[00:36:23] Martha: No, no, no. I think this is very, very clear: if you look at studies of consumer sentiment and things like that, people do not like higher interest rates. Fair point on “you bring down the interest rates and then housing prices will go up,” et cetera. But—

Andrey: Redistribution-wise, I think it really matters. Depending on whether people have flexible mortgages or not, and when they locked them in, it affects different people in very different ways.

Seth: And specifically generationally, as we’re pointing out.

Martha: Yes. And also, it can unlock housing supply, because you do have this issue right now—

Seth: There’s a lock-in issue. If you have a 2% mortgage rate...

Martha: ...why in God’s name would you move?

[00:37:14] Seth: If you were to ask me why high debt is bad in general equilibrium, I would have definitely talked about risking a fiscal crisis. I would have definitely talked about transferring from young debtors to old savers. But isn’t the number one crowding-out concern that we’re not getting the physical capital investment that we want? Isn’t that kind of the general-equilibrium story — wages are lower because interest rates are high?

Martha: Yes. And in general — if you’re thinking about the types of investment that will drive economic growth moving forward, I don’t know that issuing more treasuries so that we can pay more Social Security benefits to older people would be high on most economists’ lists.

Seth: It’s the boomers. It’s the boomers again.

Martha: It’s always the boomers. To be clear — we pay Social Security benefits for a reason. We don’t want old people to live in poverty. I should say elderly people. I’m gonna get in trouble with my mother; I just called her old.

[00:38:13] But again, there are reasons why we do government spending, which I think is really important. Sometimes when we have these conversations about government spending, there’s too much of a focus on “is this going to increase GDP growth by 20 basis points” or whatever. Sometimes government just does things because we live in a society and we should do them. There was a bill someone introduced in Congress, as an example — and I’m not saying I support the bill or don’t support the bill; I actually know very little about the bill — that would give money to new parents. And there was a criticism of the bill that said this amount of money is not going to increase fertility. And my frustration with that was: no one who introduced the bill thought it was going to increase fertility. They just think diapers are expensive and the government should help people pay for diapers. Now, you can agree with that or not — is that a good use of government funds? That’s a question. But the standard for government spending cannot always be “is this a magical thing that will solve all of our economic problems?”

At the same time, you can’t just dismiss the cost of the program. You have to think: this program pays for diapers, this program pays for roads, this program pays for missiles — how do we value those different things? And the solution is not to say “we’re just gonna deficit-spend ad nauseam,” because the deficit spending has costs.

Talking Fiscal Policy So D.C. Understands [39:56 – 44:26]

[00:39:56] Seth: Here’s another question for you, Martha. When we started this conversation, you said, “I’m not particularly concerned about the debt-to-GDP number — I’m interested in this continuum of risk of more crowding out, of important borrowing.”

Martha: I shouldn’t say that I’m not concerned about debt-to-GDP. I just mean it’s not that I think we’re gonna hit 115% of debt-to-GDP and it’s all over for all of us.

Seth: Can I ask a follow-up question then? There’s this popular other way of thinking about fiscal sustainability, which is fiscal gap accounting — taking the present discounted value of all revenues and the present discounted value of all expenditures. Would you favor us thinking about debt through that lens rather than the accounting number? There’s a correct answer.

[00:40:51] Martha: I think we can argue about different ways of thinking about this, but this actually gets at an overall point about economists in DC that I wanna make. We can get ourselves into these more complicated concepts — other people would say we should be thinking about the risk that R is greater than G, and how should we be plotting that, all the things—

Seth: That’s the next question.

Martha: There we go. I think sometimes we overcomplicate things, and I think that that is a problem. We are trying to bring these very complicated concepts to people, and I think it is useful to pull back and think about illustrative numbers, or back-of-the-envelope calculations, that help people understand what’s going on — without, and I’m not trying to pick on you here, retreating into our ivory towers and having these very interesting debates about how we think about R versus G. I’m not saying those debates aren’t important or that we shouldn’t be having them. But if we’re thinking about trying to drive these conversations in DC, it is really important to be talking about things in a way that people understand and that really speaks to them. Because otherwise we’re not going to have an impact.

Seth: So the correct answer was that fiscal gap accounting is the correct way to think about fiscal sustainability — but unfortunately, maybe the politicos aren’t caught up to that yet.

[00:42:45] Martha: If you wanna write a memo and send it to the Hill, you can do that. But I actually do think this is important. I used to be at the Council of Economic Advisers, and one of the things that we worked with senior economists on really closely was: you can overcomplicate this, and you need to be able to talk to people about this in a way that helps them really understand what is going on — what is the thing they have to worry about and what they don’t have to worry about. Someone was asking about our interest rate work — a totally reasonable question — what about the impacts of government spending on real disposable income? You’re talking about the impacts on mortgage costs, but there’s the other half of this: how does it affect people’s income? And there’s two answers there. One answer is, I would need to know what the actual policy is and run it through a full macro model, and that’s very complicated. The other answer is that we are trying to give people an illustrative calculation that helps them get their heads around something. I just think that that is a really important service to provide to policymakers.

Seth: To continue retreating into the ivory tower — and moving into R and G...

Martha: I just wanna state: if you wanna go full R-and-G, you actually need Danny Yagan from the Budget Lab, or Neil Mehrotra, who have done a lot of thinking about this. But yes, let’s do R and G.

Seth: We’ll ease into it.

Martha: Should we define R and G, or is that not necessary for this group?

Seth: Well, I’m gonna build up to it.

Can AI Grow Us Out of the Debt? [44:26 – 48:33]

[00:44:26] Seth: So I’ll tell you how we get there. The way I wanna get there is: Director Gimbel — or Martha, whatever you prefer — some people have suggested that actually it is foolish to worry about the debt-to-GDP ratio, or even the fiscal gap, because AI will boost G — will boost the growth rate — so much that we will simply grow out of the debt. So what do you say to those who call you fiscal hot-bed-wetters?

Martha: One: that’s a bet, and that’s a big bet. And if you wanna make that bet, you can make that bet — and to be clear, there are a lot of people who are making that bet right now. I think there are a couple of other things that go into this. One is, you’d likely also have some impact on interest rates, and we can debate how big that is—

Seth: There we go. This is where we go R and G. So, counter: sure, the growth rate will go up. But also, if we think that robots will bid up interest rates — people are gonna wanna invest in these robots and not invest in T-bills — that’ll make borrowing more expensive. So actually now we’re in a race between how much it boosts interest rates versus boosting growth rates. Do you have a stance on that? Does the Yale Budget Lab have a stance on that?

[00:45:50] Martha: We don’t. Or rather, I should say: we have a new macro model that people can play with online that has some feedback — not one-for-one — from growth into interest rates. But I think there’s another aspect of this, which is something people are starting to talk about more and more: yes, there’s G, yes, there’s R, but there’s also how much spending you have to do, and how much revenue you’re getting. We put something out about the fiscal impacts of AI using the Karger et al. forecasts — which is not a statement that those are correct; we were using those as inputs. And one of the things we point out is: yes, under their forecasts, debt-to-GDP improves — in some cases improves dramatically. But then you can start wearing away at that, depending on what you assume around government spending and government taxation.

Seth: Are you thinking about social support, or are you thinking about arms race?

Martha: I’m thinking about social support. They have forecasts of what’s gonna happen to the labor force. You can assume that we spend on displaced workers the way we spend on usual unemployed people — which is not very much money, by the way, per our previous conversation. Or you can assume that we spend on them more similar to how we spend on Social Security, which then has further deterioration. There are then questions around capital share and inequality, which impacts what taxation looks like. I will say, spoiler alert, that is something my team is thinking about right now. And it’s not just these questions around capital versus labor. It’s also questions like: how much of the payroll tax are you still paying? We have these gaps in our taxation system. I should also say: we tax capital less than we tax labor in this country, and we also have many loopholes for capital taxation. And if you think that AI can’t find every loophole in capital taxation possible, I have a bridge I’d like to sell you. I will sell you the IRS cafeteria where they have been keeping all of the paper records.

Taxing Capital in the AI Age: Token Taxes and Loopholes [48:33 – 52:16]

[00:48:33] Seth: Maybe let’s talk about tax policy for a minute there. I’m very curious to hear what your answer is. We know the traditional reason why we tax capital at a lower rate than labor is the idea that labor is inelastically supplied, but capital is very elastically supplied. Is AI gonna change that? If anything, it seems like capital would be even more elastic in that age.

Martha: I’m so glad you asked this question. I am so interested in this question. We obviously have no idea, but I think it’s fascinating.

Seth: It’s the country of geniuses on a cloud. You can put it anywhere you want.

Martha: Well, but also, it’s plausible that it becomes less elastic — if the returns are so much higher there. I don’t know; I can argue this either way. And that’s kind of what I feel like with these questions around elasticity and capital taxation: you can come up with whatever theory you want, and we’ll see what happens. I don’t think it’s clear. I do think it’s clear that we should deal with the loophole situation.

Seth: So, Martha, is consumption taxation the answer? Is a Georgist land value tax — as it’s always the answer — the answer? Can we get Rand Paul to finally support a VAT?

[00:49:55] Martha: If you could get Rand Paul to support a VAT, I think everyone in DC would be lining up to hire you as a lobbyist, because that is a level of persuasion I don’t think people realized was possible.

Andrey: We need the AIs to do that persuasion.

Seth: The super-persuasive AIs.

Martha: That’s right. You know, this kind of goes back to where we started, actually. One of the things that I worry about is that some of the conversations around taxation and AI are — people are trying to come up with a whole new different way of doing things, and they’re thinking about it theoretically, as opposed to working with the real-world constraints that we have. So there’s been a lot of discussion about a token tax, and I won’t get into the pros and cons of a token tax from an economic perspective—

Andrey: I will. A token is a—

Seth: You know this is a podcast for that, Martha.

Martha: I have a piece coming out on this in Tax Notes in like two weeks! Don’t spoil me.

Seth: We’re gonna scoop you.

Andrey: Okay, okay — I’ll let you have it. No, I just think that, I mean, obviously the details matter, but tokens are a meaningless unit. So I think it’s just crazy to think you’re gonna have a per-token tax.

[00:51:25] Martha: Yeah. There are so, so many things to say here. One of the things I will say is: you get the IRS to figure out how to implement a token tax. What are you talking about? And again, that’s not a shot at the IRS — they have a lot on their plate. Let’s really not try to overcomplicate this. I worry a lot that people are trying to come up with a new way of addressing problems. If the issue is that people are concerned about increasing returns to capital and us not taxing those properly — if that’s the thing you’re concerned about, we should figure out how to properly tax returns to capital.

Seth: Rather than invent some micro thing to target the tax at.

Martha: Yeah.

A Sovereign Wealth Fund? “We Can Tax It” [52:16 – 56:47]

[00:52:16] Seth: Let me ask you about one last pie-in-the-sky economic reform policy before we move on to our last topic.

Martha: I swear to God, if this is gonna be UBI...

Seth: It’s UBI-adjacent. This one, I’m gonna say, is actually in the Overton window, because perhaps you know that our president recently talked about having a stake in the big frontier labs. And when you talk to people about transformative AI, a lot of the discussion is around: well, we should have a national sovereign wealth fund, so the people can own the robots. The people will own the AI, and then we can share the wealth. Is that realistic at all, given the national debt? Or is there some universe where actually we should sell T-bills and then buy Anthropic stock as a government?

[00:53:08] Martha: I realize that I keep saying “oh man, the Budget Lab is working on a paper around this” — but this is one of the things that we’ve been trying to think through: what are the economics around this? So stay tuned. I will say, in general, I get itchy about government owning parts of corporations.

Seth: I mean, it’s free money, Martha. You just borrow at 2% and invest at 5%. What could go wrong?

Martha: I do love free money. Whomst among us doesn’t love free money? I think this actually gets at a broader thing, which is: this conversation, by definition, basically assumes that the benefits of this will be fully captured by — name your preferred AI lab here. What that policy doesn’t do: if some other organization is able to use AI to massively increase their profits, and fire all of their workers, and they don’t hire any other workers and it’s just pure capital returns — we don’t get any revenue from that. They’re still operating under the old system. So this goes back to this thing of: you’re assuming that the profits from this technology are going to end up in a very specific place. But we don’t know that.

Andrey: The steel man of this could be just that this is a hedge — not that, for sure, the big labs are gonna take all the rents.

[00:55:04] Martha: Sure. Or we could think about improving capital taxation, which is a good thing to do. We should think about improving how we tax capital in this country no matter what.

Seth: You know, the government doesn’t have to own things in order to get money from things.

Martha: Truly. This is the great thing about the government: we can tax it. ...Someone’s gonna pull that out — “this is the great thing about the government, we can tax it” — and put my name and a really scary picture of me on it.

Seth: That’s the TikTok clip. Exactly.

Martha: “This woman is coming for your...” But, you know.

By the way, one thing I will also say about this — and this is not really econ, but I’m gonna go with it anyway. One question that’s come up repeatedly in all of this, for a lot of people, is this question around fairness. I actually do think that one of the issues we have right now is that people feel like the system is not fair. And this comes up on the taxation side partly because there are so many loopholes and ways to avoid paying taxes on capital gains. One of the things that you get by simplifying the tax code, and making it much clearer who pays what for what reason, is that people go, “Oh, okay. That is this person’s contribution. This is my contribution. We all live in a society. We go from there.” In a situation where people feel like things are unfair, decreasing the complexity in the tax code actually can make a difference. I realize this is a nerd’s nerd’s take, but I actually do think it’s important.

Seth: This is the midwit meme. It’s like: just do proportional taxation... “no, we need crazy developed policy”... just do proportional taxation.

Inside the CEA and JEC: Partisan vs. Unbiased [56:47 – 63:28]

[00:56:47] Seth: All right. The last topic I wanted to ask you about, Martha: you’ve worked in so many fascinating positions inside and outside the government, adjacent to policymaking — from the Joint Economic Committee, to the Obama CEA, the Biden CEA, and now in this think-tank role running the Yale Budget Lab. Can you talk to us a little bit about, when we as outsiders read a report from something like the JEC versus the CEA, how much is that a real pure-economics document versus a political document? How should we read it as outsiders?

Martha: One thing I will say is, JEC in particular often has leadership changes — it can switch with every Congress, depending on who’s in control of the Senate or the House. CEA obviously switches over every four or eight years. So part of this is just gonna depend on who’s in charge.

I’m gonna get up on my high horse a little bit on this one. There was something that you saw people saying for a couple of years about the Council of Economic Advisers — that it was a nonpartisan place to work. CEA is not nonpartisan. You work for the President of the United States. You are within the Executive Office of the President. You are not nonpartisan. What CEA’s job is, is to be unbiased. You are the person whose job it is to sit in the room when they say, “We wanna give every American a puppy,” and you say, “That’s a very interesting policy. How will we distribute the puppies? How do we think about different preferences for puppies? What about allergies? How do we deal with the refuse problem?” Your job is to provide the unbiased analysis.

And one of the things I think you see when CEA is working really well is that people in DC can kind of tell. You’ll notice that they’ll send the CEA chair out to talk about certain topics, and then on other topics they just never get sent out to talk to the news media. That’s because you wanna send the economists out to talk about things where the economists think it’s a good idea — their credibility is really important. Different administrations are gonna approach this differently, but I think historically that’s the way CEA has worked, and that’s really important.

[00:59:40] Andrey: It also matters whether you have a charismatic person — I mean, Goolsbee was out there all the time in a way that I think others weren’t necessarily.

Martha: That’s also a personality thing. There are people who enjoy talking to the press and people who don’t. There are people who enjoy being public figures and people who don’t. There are people who are really good at it and people who are less good at it. But I do think it is important that you are only sending the CEA economists out on things that they legitimately feel they can speak to from the economic literature and the underlying economic reasoning. Because otherwise, what are we even doing here?

Andrey: I’m curious, on a personal level, working in these roles — what does it feel like? Do you feel like you’re making a difference? How does it compare to other types of jobs?

[01:00:46] Martha: I am biased here. I worked at CEA twice — under Jason Furman and under Cecilia Rouse. It is hard to get better bosses, so I had a great experience. Most of my closest friends are from when I was at CEA, each time, so that kind of speaks to what my experience was like.

I also think that working there can be hard, particularly for economists — and this gets at some of the conversations I was having with Seth earlier. A lot of the time, economists come in and say, “We have the one right way to do things.” And then comms freaks out: “You can never say that publicly.” And then the White House Counsel’s Office freaks out: “That is so illegal — please never tell anyone you even thought about that.” And then legislative affairs freaks out: “That will never get passed. Please don’t do that.” So you have to think really carefully: okay, this may be the optimal economic option, but what’s second best? Maybe what’s third best? If we get to the fourth-best thing, is that actually worse than doing nothing? You do have to operate under these real-world constraints. Some people find that incredibly frustrating, and some people find it incredibly interesting. I find it incredibly interesting.

Andrey: Very cool. Is there something you can talk about that you’re particularly proud of from your work at the CEA?

[01:02:14] Martha: Oh, man. I try not to do that. Mostly because your job is to be an advisor behind the scenes, and not to be someone who’s out there saying “I did this thing.” You’re working for a president. You’re working for a policymaker — they are the person.

The actual answer to your question, not to get overly soppy: particularly when I was at CEA the second time, I was in charge of the junior staff. And the CEA junior staff are kick-ass. I will say to all of the economists out there thinking about whether or not AI is going to take your job: AI may or may not take your job, but the former CEA junior staff are definitely taking your job. And they’re gonna be so much better at it than you are, and it will be fine. We just had such amazing people there, and it’s been really amazing to watch them do all the things they’re doing after leaving CEA. I look forward to their extremely benevolent and well-reasoned rule. I think it’s gonna be great.

Budget Lab vs. the Other Modeling Shops [63:28 – 66:38]

[01:03:34] Seth: My last question is about these different forecasting and policymaking groups. Do you see differences in high-level philosophy between how you at the Yale Budget Lab, or maybe the Penn Wharton Budget Model, or the Joint Committee on Taxation approach things? You’re all academic economists — are you all coming at it from the same direction, or are there big philosophical differences?

Martha: I don’t know that I would necessarily say philosophical differences. I should say, basically all of the modeling shops think of JCT and CBO as the one ring to rule them all. If we get an estimate that’s markedly different than theirs, generally we’re like, “Hey, staff, what on earth is going on here?” And sometimes there’s a reason — a difference in a parameter, et cetera.

One thing Budget Lab is interested in that I think is different is this very long-term approach. The costs and benefits of policies can look very different at 10 years versus 30 years. If you take the One Big Beautiful Bill Act: at 10 years, its average impact on growth is basically zero. If you go out 30 years, it’s negative — because of what we were talking about earlier: the rise in interest rates, the crowding out of private investment that slows down economic growth.

We are also interested in non-monetary economic benefits. As an example, you can look at paid family medical leave. Paid family medical leave is not gonna double GDP growth — that’s just not what it does. It enables people to spend time with a newborn, for instance, and one of the things we know is it has some impact on neonatal mortality. That’s a thing a lot of people care about, and it’s not an estimate that’s part of the traditional budget modeling process. I think allowing people to think about what is it that we’re buying with this policy — we are spending money for a reason; what are we getting; do we think what we’re getting for those dollars makes sense — is important. That’s been relatively easy in the past on tax policy, because what you’re trying to do with tax policy is redistribute, raise more revenue, et cetera. So that’s easy — or I should say easier; the poor people who do the tax modeling for me are screaming right now: “Martha, our jobs are not easy!” But particularly on the spending side: what is it that you’re buying, and how efficiently are you buying that thing?

Private Data, Public Data, and the AI Labs [66:38 – 69:41]

[01:06:38] Andrey: All right. You also have spent some time at Indeed—

Martha: Oh, no. To be clear, I loved Indeed, but now I’m terrified about where this is going.

Andrey: No, no — we were curious what you think the role of private versus government data will be.

Martha: I think private data is incredibly important. It allows you to get much more disaggregated. You can sometimes see things faster. You can see things that don’t show up in public-sector data, because they may not be measuring it. But I think sometimes people talk about private-sector data as a panacea — “oh, we won’t have to have public-sector data anymore.” I do not think that is correct. One, private-sector data shows you what one company sees. We are often interested in what is happening with the overall economy, so someone needs to do the aggregating to help us see that. People should also keep in mind that a lot of the private-sector data indices you see are benchmarked against official government data — “hey, do we have the right number of job postings for leisure and hospitality, given what we know about hiring in that industry from official government data?” So there’s really a symbiotic relationship there that is really important, and it’s important not to say, “Oh, we’ll just use private data and it’ll be fine.” You saw this, by the way, when the government was shut down last fall, and we had three different private-sector data estimates come out: one said “hiring is fine,” one said “we’re having no hiring,” and the other said “hiring’s fallen off a cliff.” And everyone was like, “Well... who knows?”

[01:08:31] Andrey: There’s been also a call for data sharing between the AI labs and the government. Obviously, all else equal, it would be great to have more data from the labs. Is there anything particularly interesting in terms of data that the labs have that we would want?

Martha: To be clear, per your point: I love data. Data is great. I personally should have access to all the data. I will say — if I think about the things that I’m wondering about in the labor market right now, what I kind of want to know is: if a company adopts AI, how quickly and how do they adjust their hiring? And that’s not something, as far as I’m aware, that the labs can answer. So I think there’s been a little bit too much focus on the data that the labs have, and not on starting with: what are the questions that we need to answer right now, and where does that data live?

Lightning Round: Bottlenecks and Redistribution [69:41 – 72:58]

[01:09:41] Andrey: Related to this point — this is also in our speed round — economists talk about bottlenecks to AI adoption leading to productivity growth. What do you think is the biggest bottleneck?

Martha: Oh, man. The biggest?

Andrey: You can do a big one. Your favorite.

Martha: Actually, I’m gonna cheat here, which is to say that this really depends on different industries and different occupations — and that that is a really important part of this overall conversation. It’s going to look different in different workplaces, and I think sometimes we talk about this as if it’s overly homogeneous. For instance, Hollywood is not particularly regulated. The biggest barrier I see there is questions around consumer sentiment: are consumers gonna revolt if they think a movie was made with AI? On the other hand, in healthcare, there are real regulatory and liability concerns, so that’s going to make a difference there. People need to do much more thinking about the dynamics in specific industries and occupations, rather than some of the broader economy-wide thinking.

[01:10:53] Seth: Fair. All right — so imagine some of those bottlenecks get unlocked. We start getting these productivity gains and some disruption to labor demand. Gun to your head—

Martha: No.

Seth: —an imaginary gun — are you on team pre-distribution, to save good jobs? Or are you on team let-’er-rip — second fundamental welfare theorem, get efficiency, and then redistribute after?

Martha: I think this gets at one of the things about working in policy versus being in academia. The economist answer is: let ‘er rip and then we redistribute, and we get the most economically efficient whatever. People don’t seem to like that very much.

Seth: Why don’t people love us? Martha, why don’t people love us?

Martha: It’s a real question! I actually legitimately do think this is one of those hard questions for economists working in policy. We can say, “Look, you get the most economically efficient outcome if you allow this thing to happen, and then obviously we can just redistribute on the back end.” The citizenry just doesn’t seem to like that. And so I think we need to do more thinking about how to balance those issues.

Seth: Okay, but gun to your head.

[01:12:18] Martha: I’m an economist, right? I have this instinct. But I’m just saying — again, this is part of the problem. We are the problem. People’s preferences are real, and we have to take into account people’s preferences, even if we are sitting there saying, “No, no, you don’t understand — GDP growth will be this much higher, and then we just redistribute it, and it’ll be fine, and your welfare will be higher.” People don’t like that.

Seth: All right, we should keep it halfway.

Martha: But also, people should let us redistribute as necessary, and not get mad at us for it. But that’s not how things work.

Sci-Fi Corner and Sign-Off [72:58 – 76:14]

[01:12:59] Andrey: Let’s wrap it up. What is your favorite sci-fi book or author?

Martha: Oh, that’s mean. That’s really mean. I should say, I was a huge sci-fi nerd as a kid, and obviously still am. Because we have been here talking about AI, I’m gonna say Becky Chambers — just because I think Becky Chambers is the version of the future that we all kind of want for ourselves. It’s these amazing space-going communities with these friendly AIs. It just seems very nice. So I’m gonna choose the optimistic one. And maybe we’ll get that.

Seth: Great recommendation. Do you have one book by her in particular?

Martha: I think you kind of have to start with — I’m gonna get the title wrong — A Long Way to a Small, Angry Planet, which is the first one. It’s just so, so great. There’s a follow-up that takes place on a space station, with a very cute child that they all start looking after together — I forget the name of that one, which is really bad, but I loved it. She’s working on a new thing right now, and I keep refreshing her website to see if she’s announced what it is yet. So if by some miracle this makes its way to Becky Chambers: the people wanna know what’s coming next. Please let us know.

Seth: Amazing. And definitely a better sci-fi take than some of the takes we’ve had on this podcast. I won’t name names.

Martha: I wanna know what the worst sci-fi take was. Okay, don’t name names — but I wanna know.

[01:14:38] Seth: I’ll name names. Basil Halperin said he loved Red Rising — which I just finished. Which is so mediocre.

Martha: No! Oh my God. I’m sorry — I love Basil, but that is... I’m legitimately very upset about this.

Seth: Next time you see him, you can tell him Seth really did not enjoy Red Rising.

Martha: Let me rephrase: Red Rising is a fun read you can get through in two hours. It’s terrible, but, like, sure.

Seth: Angry Harry Potter.

Martha: So angry. The angriest of Harry Potters. By the way, if I can do a second one, it would be Adrian Tchaikovsky’s Children of Time, which is so good.

Andrey: Great, great book.

Seth: Oh, that one’s great. With the spiders?

Martha: Yeah, with the spiders. It’s so good. Children of Time is great. People should read Children of Time.

[01:15:35] Seth: All right, Andrey, shall we wrap it on that? Everyone read Children of Time and...

Martha: A Long Way to a Small, Angry Planet.

Seth: And everyone out there, when you’re done reading, you should...

Andrey: The Yale Budget Lab?

Seth: Say the line. Keep your posteriors justified. We have a line, Andrey.

Andrey: I know our line. You gotta constantly be keeping your posteriors justified.

Seth: Never stops.

First up is Pope Leo XIV’s 42,000-word encyclical (that’s Pope-talk for letter) on artificial intelligence. Magnifica Humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence lays out 5 principles of Catholic social teaching, and then explains how this should shape Catholicism’s approach to AI.

We focus on two in particular. The first is subsidiarity, which Seth summarizes as Catholic federalism, the idea that most decisions should be made at as local a level as possible. We discuss both the economic argument for this, but also what the Pope adds to Hayek: Decentralization not for efficiency’s sake, but a kind of ennoblement, the dignity of deciding things locally. The second is the universal destination of goods, which the encyclical extends to “immaterial goods”. This leads to the positive argument of the Pope - that AI should be undertaken as a communal project with decentralized power and discussion, rather than a technocratic “Tower of Babel” that will lead to ruin and division.

Much of our disputation focuses on whether these principles actually resolve the important questions. Is the Pope rightfully cautious about an emerging technology, or was this an opportunity to take a stronger stand on what constitutes AI Sin? Interestingly, the Pope’s strongest stand is against transhumanism, which would be a plausible resolution to the dialectic of “Butlerian Jihad” vs. worship of a new machine god.

Then we pick up DeepMind’s “Positive Alignment” paper, and the economists get grumpier. Andrey complains that the paper is vacuous, failing to take a stand on actual practical goals or methods. But it sets us up for a good conversation about several issues: Such as liberalism of fear, a type of anti-utopian liberalism; whether “flourishing” is something you can A/B test towards; and where the ‘constitution’ metaphor behind Constitutional AI works vs. breaks down.

We also tease a joint project, “SCOTUS Bench,” a new benchmark for evaluating AIs’ ability to predict appeals court outcomes. Watch this space for more on that soon.

Related Links

• Scaling Laws — Alan and Kevin’s AI, law, and policy podcast at Lawfare

• Alan Rozenshtein on X: @ARozenshtein · Kevin Frazier on X: @KevinTFrazier

• Magnifica Humanitas — Pope Leo XIV’s first encyclical, “On Safeguarding the Human Person in the Time of Artificial Intelligence,” in full, straight from the Vatican

• “Positive Alignment: Artificial Intelligence for Human Flourishing” — the DeepMind-led paper (Laukkonen, Krier, et al.) arguing alignment should optimize toward flourishing, not just away from harm

• Claude’s Constitution — Anthropic’s ~20,000-word statement of Claude’s values and character, released under CC0

• “Claude’s Constitution,” with Amanda Askell — the Scaling Laws interview with the document’s primary author (the one we keep saying we’re jealous of)

• The Moral Machine — MIT Media Lab’s crowdsourced trolley-problem experiment: millions of judgments on the grandma-versus-criminals ratio

• Meta’s Oversight Board — the “Supreme Court of Facebook,” and Kevin’s cautionary tale in institutional design

• Andrew B. Hall — Stanford political economist on deliberative democracy, platform governance, and what went wrong with the Oversight Board

• The Anthropic Economic Index — the adoption data behind the “whole countries blacked out” point

• Judith Shklar, “The Liberalism of Fear” — the cruelty-first, anti-utopian liberalism Alan invokes against thick conceptions of the good

Timestamps

(00:00) Intro — two papers, four hosts
(01:47) Paper 1: Pope Leo XIV’s encyclical, Magnifica Humanitas (04:00) Subsidiarity, or “Catholic federalism”
(12:26) Does the Pope take AI seriously enough? Mind-body dualism and the ex cathedra problem
(15:34) The coming religious schism over AI personhood — and the Butlerian jihad
(18:06) Transhumanism and the dignity of human limits
(20:59) When is using AI a sin? Best-man speeches and eulogies
(25:05) The universal destination of goods — is AI access already universal?
(33:37) Is AI a centralizing technology? Dignity vs. efficiency
(36:37) Freedom vs. control, the labor market, and make-work
(41:10) Chess, the centaur era, and living after we’re no longer the best
(47:34) Sponsor: Revelio Labs (48:49) Paper 2: DeepMind’s “Positive Alignment”
(49:17) The liberalism of fear and thick vs. thin notions of the good
(53:53) Is positive alignment an empirical question? A/B-testing flourishing
(56:29) What would a useful positive-alignment paper actually do?
(58:09) Constitutional AI as a site for public participation
(1:00:47) The Moral Machine and trolley problems at scale
(1:01:08) Does the “constitution” metaphor hold? Virtue ethics and self-binding
(1:10:02) Running every Supreme Court case through the models
(1:10:53) Lessons from Meta’s Oversight Board
(1:15:09) Wrap-up

Our sponsor

This episode is brought to you by Revelio Labs, the leading provider of labor-economics data, available to academics on WRDS.

Transcript:

Seth (00:00:00): [upbeat music] Welcome to the Justified Posteriors podcast, the podcast that updates beliefs about the economics of AI and technology. I'm Seth Benzell, always positive and always aligned, coming to you from the Pocono Mountains of eastern Pennsylvania.

Andrey (00:00:23): And I'm Andrey Fradkin, coming to you from San Francisco, California. We're sponsored by Revelio Labs, fine purveyors of data products. And we're very excited to have Alan Rozenshtein and Kevin Frazier from the "Scaling Laws" podcast on the podcast today. Welcome.

Alan (00:00:43): Yeah, thanks for having us.

Andrey (00:00:45): Just for our listeners, why don't you tell us a little bit about "Scaling Laws?"

Kevin (00:00:51): Sure thing. So our main goal here is to provide robust and timely analysis of all AI policy questions. And that's an expansive ambit, and it's one that keeps us really, really busy because if it's not an executive order, then it's some big new policy idea from one of the labs, or it's some new economic report. But really what we try to do is dive into the weeds of policy and legal issues that are emerging in the AI space, given our backgrounds as law professors. But Alan's the one with the brain, so I'll let him fill in the details on earth.

Alan (00:01:30): No, that's a perfect description. Yeah. We just think that there's a lot of really interesting stuff happening at the intersection of AI, law, policy, especially around national security, which is the core focus of the publication that "Scaling Laws" is part of, which is Lawfare, and so we're trying to fight the good fight, and it's never a dull moment.

Kevin (00:01:47): That out of the way, I think we can dive into our first paper. Although, I think by any podcast standards, our first paper is lengthy to say the least, dealing with the-

Andrey (00:02:00): Mm

Kevin (00:02:01): ... pope's encyclical at 42,000 words, or for all those listening, about two and a half hours on my stationary bike. I don't know what that says about my biking skill-

Andrey (00:02:13): [laughing]

Kevin (00:02:14): ... or my reading ability, but it was a very tiring afternoon. But a very extensive, very important read from Pope Leo. And this has been covered by a lot of folks, but I don't think it's ever been covered by two lawyers and two economists at once.

Andrey (00:02:34): [laughing]

Kevin (00:02:36): My hunch is that this wasn't what Pope Leo was anticipating when he was sitting and putting a... I like to think of him writing with a quill and-

Andrey (00:02:45): [laughing]

Kevin (00:02:45): ... on some very old paper. But, I don't think he anticipated this podcast duo diving into his encyclical.

Seth (00:02:55): Well, hopefully, our analysis will be less a Tower of Babel of technocratic overreach, and more a blessed city of Jerusalem built together by our common efforts.

Kevin (00:03:06): Seth did his reading. Seth dove in. All right. Excellent. Good to hear. Well, I think at this point in time, we're talking in early June. By the time folks are listening to this, unless you've been living under-

Seth (00:03:18): There may be a new encyclical. [laughing]

Kevin (00:03:21): Somebody-

Seth (00:03:22): Pope maybe changed his mind.

Kevin (00:03:24): Yeah. Ugh. But there's so much to cover in this encyclical. Obviously, we could start with just the pope's analysis of the role of the Church and of social doctrine, which he gets into in extensive detail, and that covers about 20 to 30 pages. I think for the sake of our podcast, that's probably not our main forte in terms of analyzing the evolution of the Church's social doctrine. But I will let anyone intervene there if they're extremely fired up about that posture.

Andrey (00:04:00): [chuckles]

Kevin (00:04:00): But I do think that the first area for us to really explore, that both economists and lawyers can appreciate, is this idea of subsidiarity, which is really-

Andrey (00:04:12): Mm

Kevin (00:04:12): ... the notion that we have various institutions operating at various levels of jurisdiction, and that ultimately we want to devolve regulation or governance of an issue to the smallest capable actor. And that has a lot of resonance and a lot of power in the Church's teaching, which is to say you have this centralized entity, the Catholic Church, and yet we have parishes all over the world. And so how do we distinguish between the issues that Pope Leo needs to decide and the issues that parishes and then to, switch to a different context, local governments versus national government versus international government. How do we think about the allocation of responsibility there? So I would love to just hear the initial thoughts. I know Alan will have thoughts. But from an economic perspective, what is the relevance of a sort of subsidiarity principle to the governance of emerging technology?

Seth (00:05:15): Oh, well, Catholic federalism.

Andrey (00:05:16): Well, there-

Seth (00:05:17): I love it.

Kevin (00:05:18): Exactly.

Andrey (00:05:19): Let me take a little stab at it. I think this hearkens back to the central planning versus markets debate in the sense that we could have AI policy be governed at the national or even a supranational level. But there is a risk that those laws are not going to be well-suited to individuals with heterogeneous preferences and heterogeneous information about their needs and their constraints. And so to the extent that we can allow for governance to happen at a more local level, then that AI, the way in which it's used, is going to be More appropriate, more beneficial for everyone involved. So that's kind of the high-level thought here. But then, of course, with something like AI, you are worried about externalities of various types, right? So, the way in which one group decides to use the AI may affect everyone else, and then that kind of pushes things back up to the top because you need coordinating mechanisms, and that's pretty hard. But also, this is a very abstract discussion, and I always like to think about specific issues, specific AI policies about which we can think about.

Kevin (00:06:44): Seth-

Seth (00:06:45): Are you going to give us one? [laughing]

Andrey (00:06:47): Well, so an example might be by what constitution is the AI trained from.

Kevin (00:06:58): Sure.

Andrey (00:06:58): So, if every little village had its own Claude with a different constitution, there might be a scenario in which the constitution of one of the Claudes might say, "Help us dominate our neighbors." And that would obviously have [chuckles] a negative repercussion on the neighbors potentially. And I think there's this kind of a separate thing there now that I'm bringing it up, is that it seems at least, given today's technology, pretty implausible to have that many different Claudes. We don't know how to do that yet. The training runs and the post-training are pretty catered to one thing and very expensive, and so maybe we'll get there one day, but at the moment, it doesn't even seem very affordable.

Seth (00:07:49): Mm-hmm.

Andrey (00:07:49): Yeah. But I'm curious what you think.

Seth (00:07:52): Yeah. So I'll jump in here. I'll say that, first of all, we're all kind of maybe libertarian-leading guys. I don't know if that's fair to say for you, too. So as I was reading this, I was thinking, "Oh, those law fair guys are going to really like the subsidiarity. Of these five points, that's going to be the one they like." [chuckles] And I think Andre gave a good analysis of the economic take on why you would want subsidiarity. So, what is the pope adding to that that wouldn't be in Hayek's argument for decentralized planning? I think what he wants to add is there's a kind of an ennoblement. There's a kind of positive good vibes that come around from the local decision-making, even above and beyond the efficiency arguments for decentralized planning, which is I think what Andre was emphasizing. And this is definitely an essay that is sort of a little bit anti-putting efficiency above everything else. The second thing I will say is that it's sort of interesting reading this principle of subsidiarity. Obviously, we know the history of Catholicism and its various schisms, right? Obviously, sometimes-

Kevin (00:09:03): Yeah. I don't tend to think of it as a super decentralized religious and spiritual movement. I will say with a name like Rosenstein, I am neither qualified to opine on Catholic thought, nor do I have a ton invested in it. But-

Seth (00:09:16): Mm

Kevin (00:09:16): ... since we're talking about it-

Seth (00:09:18): Let's do it

Kevin (00:09:18): ... the Catholic Church did not strike me as the most let a thousand flowers bloom type institution.

Seth (00:09:26): It's not clear to me that establishing that the sacrament of communion is literally Jesus in the wafer. It's not clear that that had to be the centralized decision to be made. From the outside, it's not obvious what are the high-up decisions and what are the low-down local decisions. But I guess that's where I'm going with this, right? Which is, alongside this idea that you should decentralize things when possible is this very sort of Catholic teaching-based view on what are the important things to not decentralize. And those are some kind of foundational ideological commitments around prioritizing the poor and one of these principles I'm sure you'll get to soon is the universal destination of all goods, which seems to mean something like communal ownership. So, as we talk about this, one thing I'll be keeping an eye on is to what extent is subsidiarity in tension with this idea of there being a kind of a common good that the pope can tell you about.

Kevin (00:10:27): Yeah, and I really appreciated your flagging of the fact that so much of this seems to be grounded in the pope's insistence and hope for people to feel a sense of agency in the AI era and really leaning into the sort of humanity of this all. And so I think subsidiarity is a sort of end round circumvention to that point of saying, how can we find a way for people to feel like they have a mechanism by which to actually assert autonomy in this domain? But to your point, Andre, and something that I think from a technical standpoint is really interesting, is that even feasible in terms of policy development that reflects cultures around the world, even the entirety of the Catholic base, right, which we know spans from South America to Southeast Asia and everywhere in between. As things stand right now, there are whole countries that don't have access to Claude yet, right? And there are whole countries for whom I'm guessing that their performance in their language is probably pretty poor relative to English, for example. And so just from a technical standpoint, that's something that I don't know was as thoroughly addressed about the just technical feasibility of some of the ideals of subsidiarity and something like that.

Seth (00:11:49): Yeah. He talks about trying to bring ethics into the research lab. So maybe that's pointing towards what he wants us to be working on.

Andrey (00:12:00): Yeah. It's not really clear in some way who the audience ... of this piece is. It's unlikely that typical Catholics would read it, of course. It's a very long and dry document. People at the labs, I guess, could be reading it. I guess that might be just the main audience of this document, but I'm curious what you think.

Alan (00:12:26): Yeah. I think there are kind of two different parts of the document. Obviously, there's this whole long thing about Catholic social thought, which is interesting, but I think somewhat orthogonal to the discussions that AI watchers are interested in. On the AI side, it seems like he's making two different sets of arguments. One is a set of social arguments about the effect of AI and the dignity of labor and the need to spread resources equitably, and I think those are perfectly fine arguments. You can agree with them, you can disagree with them, but those seem like perfectly reasonable social and kind of political positions to take. The other side is his engagement with the technology itself, which I did find somewhat disappointing. Now, on the one hand, the fact that the Pope's first major written output is about AI is itself, I think, remarkable, and he should get a lot of credit for that. You can hardly accuse him of ignoring this epochal issue, but we shouldn't grade him on a curve, right? He is-

Andrey (00:13:33): [laughing]

Alan (00:13:34): He is trying to engage-

Andrey (00:13:36): Grade him on curves.

Alan (00:13:37): Well, I'm just saying, I'm really impressed that he's engaging on this issue, but he is engaged on the issue. Okay, so how well is he engaged on the issue? And on the actual issue of AI itself, I don't know. I think I'm stealing this from Matt Yglesias, who had this, I think, pretty good post on X, which was something like, "I get why he has to say this because he is, after all, the Pope, but the whole mind-body dualism is not very helpful in discussions of AI." I'm paraphrasing here. And what Yglesias is referring to, and I felt this as well when reading it, is there are these ex cathedra pronouncements in the encyclical-

Andrey (00:14:11): Literally ex cathedra.

Alan (00:14:13): Yeah, I guess literally, right? I guess literally. You are the Pope, after all. About how AI can never have moral responsibility, AI can never have thoughts, AI can never have feelings, AI can never have this, and AI can never have that. Which, look, I understand that is a highly intuitive view, and yes, I guess if you are literally the Pope, there are certain metaphysical commitments of your religion that perhaps might require you to take this position, and I don't think this is simply a Catholic position either. I suspect if you were to ask the chief rabbi of Israel or some high-level Islamic thinker, they might give you a sort of similar position on the metaphysics of all of this. I'd be very interested in what a Buddhist scholar would say. I suspect actually that might be the most fruitful engagement. A kind of tradition that really takes no self seriously, I think, might have a lot of very interesting things to say about the metaphysics of AI personhood. But I think in some ways, the problem to me with the encyclical is that it doesn't take AI seriously enough. It's not actually as AGI-pilled as I would want it to be because, and here I'm going to steal from friend of the Lawfare pod, Dean Ball, who made a very excellent point. The existential questions about AI are not actually about the distribution of resources in a post-scarcity economy, though those are very important, to be clear. It's about: what is the place of humanity when we are no longer the smartest and most sophisticated entities? That is the fundamental spiritual and existential question, and that is one which a posture of "AI will never have feelings, AI will never have moral responsibility" just tries to kind of drive out of polite conversation. But the problem with that is that, first of all, I think it's wrong. But even if it weren't wrong, it's actually not, I think, going to be responsive. I've had this idea that I think may be crazy, but I think may also be correct. We'll see. I think-

Andrey (00:16:20): Those are my favorite ideas.

Alan (00:16:21): Yeah, right. That the future religious schisms will not be between Christians and Muslims and Jews and Buddhists and whatever, but it's going to be people who think that AI has potential moral personhood, maybe even divine personhood. Certainly, if you're going to kneel at the altar of the machine god, that's a big deal. But even if you don't think that they're literally divine entities, if you're convinced that the AI that you're interacting with has so thoroughly passed the Turing test, that it has a kind of moral personhood, that's going to have profound religious implications. And then on the other hand, you're going to have a set of religious views, and I think you're going to have a lot of the incumbent religious bodies here. Which is why the Catholics and the Jews and the Muslims and the Hindus, it's the beginning of a joke. We'll all get together on this side that says, no, AI must be. We must have a kind of Butlerian jihad, to cite Frank Herbert here and the Dune series. We must take a kind of Butlerian jihad approach to machines, because otherwise, these machines, which are already so much smarter and more capable of us, if, my God, we allow them moral personhood, then we're no longer the apex dignity-holding entity on this planet. And while I don't expect the Catholic Church to be able to metabolize that terribly well or, frankly, I'm not picking on the Catholics here, any organized religion to be able to metabolize that particularly well, just denying that, not engaging with that, I think is not going to work in the long term. Certainly not by the time Claude 17 comes out.

Seth (00:18:06): I guess I would say I don't think that's 100% fair to the pope. I think he does have a big take that is related to the questions that you raise, which is he takes a very strong stance on transhumanism. So these questions that you're raising around will we have a machine God? Will we destroy the AI? Will there be some sort of intermediate result? One very common answer to those questions is, is we'll merge with the machine. We'll become immortal, embodied, Ms on the computer, or we'll become physical cyborgs, or we'll use all sorts of advanced eugenics techniques in order to become more than human. And so that is an answer some people have, and the pope very strongly comes out against that answer. He says it is the fact that we suffer and die and have miserable things happen to us and are limited by our nature is what makes us human, therefore do not do transhumanism. So I think you might not agree with the take, but he's got a take.

Andrey (00:19:13): I don't think that those are in contradiction to each other necessarily, in that I agree with Alan that I expect a blossoming of new religions to come about, and the existing religions certainly have a commitment to the primary role of humanity as it is today.

Seth (00:19:38): Right.

Andrey (00:19:38): And so a lot of this document is spent justifying why today's humans are essentially the relevant moral unit. So-

Seth (00:19:52): Made in the image of God and all that

Andrey (00:19:53): ... humans die, but that's what makes it good in the light of God. I also don't want to be speaking on behalf of the Catholics here. But, that was kind of my sense from it, and it was just don't go for efficiency. Humans aren't meant to be the smartest or the most efficient or anything like that. Even though they're imperfect, that is as it should be. So there is maybe a sliver there where we might have AIs, but as long as they don't have pretense to being moral beings, if they're designed in a way that tries to make them be less like that, then they could coexist with humans in a way that might be satisfactory to the Catholic Church.

Seth (00:20:41): Right. And then the natural follow-up question is, okay, all right, if you're going to throw out transhumanism and efficiency for the sake of efficiency, aren't you going to be outcompeted by the groups or the nations that do go full hog for AI-

Andrey (00:20:54): Yes

Seth (00:20:54): ... transhuman efficiency? And then he's got an answer on that. Do you want to pick that up?

Kevin (00:20:59): I wanted to hit on one key point, though, which is a critique that was in the "New York Times" on the fact that, and I found this pretty persuasive, one of the bigger omissions was the lack of specificity around when AI use may constitute a sin, and-

Seth (00:21:18): Mm

Kevin (00:21:18): ... when AI use may be something that is inherently and definitively bad. And something that I think stood out to me about that was when I talk to people about AI in a moral context and in a setting where we're trying to identify what are those red lines about how and when you use AI, that's where some of the most difficult conversations come up. I love to pose a very dumb hypothetical, which is imagine your best man wrote his best man speech with AI. Are you happy or mad? You're probably pissed off, in my opinion. Now, we could switch it and make the stakes even higher. We could say something like a eulogy. If you found out your eulogy was delivered by your homie, and they're like, "Yeah, I just used Claude, and it was really good." I'm probably rolling in my grave. What are those instances, though, in a moral context? Can you consult ChatGPT as a proxy for your pastor or your religious leader? Can you use ChatGPT for relationships? What kinds of relationships? How far can you take that relationship? And those are some of the questions I find that people have some of the most difficulty resolving, where the contestation by "The Times" was, hey, if there was a question or some lines to be drawn, we would kind of expect that the religious authority would be the one who draws those lines and says, "Yes, this is bad. Yes, this is good." Now you can go forth and use AI in a way that you feel less moral ambiguity. And I just thought that was a really interesting take because we have struggled, in my opinion, about how to draw red lines about when and why AI is used and when and why AI should not be used.

Seth (00:23:13): But isn't that the right answer here? The pope notes that the technology is moving fast. It seems like subsidiarity could help a lot with that question. Maybe one region develops the norms to not use the AI for eulogies, and another region develops another norm. Why is that the centrally planned pope should have an opinion on one?

Kevin (00:23:34): Well, I think that if we need to have moral clarity as to how and when to use AI, I'm not sure that subsidiarity is going to magically percolate those use cases in a way that is never hyper-relativistic. It's always going to be context-driven, which from the point of a faith, I think if you have no principle other than you do you based off of context, that kind of isn't a faith.

Seth (00:23:59): He's got five principles. We can give the five principles he gives. Which are subsidiarity. You're right, I could list them, but one of them-

Kevin (00:24:08): But that's not really about AI use on an individual basis. Subsidiarity doesn't change how I use AI

Seth (00:24:16): If you're Dario Amodei, it might change the way that you allow people to customize the AI.

Andrey (00:24:26): I do think it's very pragmatic of him not to go into specifics, for the reasons stated. But I do think that traditional Christian morality does bear on some of those questions, and in particular, you're not supposed to lie. So if you're giving your best man speech and the AI wrote it for you, that to me seems like it's a lie, no?

Kevin (00:24:56): Is it a lie?

Seth (00:24:57): Would depend on the norm.

Kevin (00:24:59): I don't know. But-

Seth (00:25:00): In some places, it'd be a norm to not disclose. In some places, vice versa.

Kevin (00:25:05): Well, let's switch to something even easier, which is whether we should make AI universally accessible to everyone, based off of this idea, as Seth mentioned, the universal destination of goods. So for folks who did not spend their entire Saturday or whatever-

Andrey (00:25:24): [chuckles]

Kevin (00:25:24): ... stationary bike ride reading the encyclical, here's what we are referring to. So this principle, and I'm quoting now from the encyclical, "Reminds us that the Earth's goods, soil, water, air, and natural resources are given by God to the entire human family to sustain the lives of all, and that every person has an inherent right to use of such goods both now and in the future." And the Pope then clarifies, "Certainly, there is a right to private property which has its own specific meaning and purpose, yet it is always subordinate to the universal destination of goods." According to John Paul II, this subordination is the golden rule of social conduct and the first principle of the whole ethical and social order. So that's a bit of a mic drop, or to go back to my earlier-

Andrey (00:26:20): [laughs]

Kevin (00:26:20): ... refrain, a quill drop by the Pope to say that this is the first principle of the whole ethical and social order. Coming from the Pope, that's a big statement, which is to say making sure everyone has access to these goods, to this knowledge, is incredibly profound to me because, again, if you go and you look at the research Anthropic's done around its economic index report, for example, you'll see whole countries that are just blacked out because there is no access to Claude yet. There is no sort of universal use of AI. And again, as I mentioned earlier, even if it were, the idea that the AI is culturally sensitive, or as robust or as reliable in a certain language, for example. That clearly hasn't been the case so far. And then just to make sure that listeners don't forget the fact that we still have a digital divide, we still have millions, if not billions, of people who don't have access to high-speed internet. So if we are going to realize this idea of the universal destination of, let's say, AI, we are very far behind in terms of just the infrastructure that would be required to make real on that. So I would love the economists' hot takes on this because I'm getting Lockean vibes are coming up. We can Kosian concerns. We could just start name-dropping tons of economists here. What are y'all thinking?

Andrey (00:27:57): Well, the first thing is just, yeah, it's a very socialist notion from the Pope, although he tries to thread it with, "We also respect private property." It's a bit weaselly in my opinion, lacking specifics. But I think, in particular with regards to AI, to me, it's a bit of a funny concern. AI is the fastest diffusing technology in the history of the world. It is almost universally accessible. Yes, Claude is blacked out in some countries. By the way, people in those countries have figured out a way to use Claude if you talk to them. Yes, not all languages are equal, but also, we have the best translation tools in history available. I'm not saying there isn't inequality to AI access, but it's actually you can get pretty good AI almost everywhere. And it'll become even more ubiquitous over the coming year, I'm sure. And in particular, Google is essentially serving AI to everyone. So sometimes I just find these concerns extraordinarily contrived. It's people who haven't actually thought about the specifics of the product diffusion, just making pronouncements from their chair about how everyone should get AI, whatever that is.

Seth (00:29:15): It's a really special chair, Andre.

Andrey (00:29:17): Yes.

Seth (00:29:18): It's not just any chair.

Andrey (00:29:18): It is. I know. [laughs]

Seth (00:29:21): [laughs] I'll second your comments there, Andre. I think you're exactly right about the speed of diffusion. The thing that I would add here is that the new thing that isn't just run-of-the-mill, let's split up the goods equally, is there's a take that he seems to be an innovation. It's unclear if he is drawing on prior teaching here, where he says that extends to immaterial goods as well. That cultural products, intellectual products, are also part of this common universal wheel. And I don't know, that kind of got my Ayn Rand hackles up thinking about if I have an idea in my own head, is that all of society's idea just because I just had it? I think that there is a little bit of an expansive view on what constitutes the goods in the universal destination of all goods here to include things that usually we wouldn't think of as even in communist states, things you would have to share.

Alan (00:30:27): Yeah. So look, I'll say, I think the pope is allowed whatever social teaching the pope-

Kevin (00:30:33): [laughing]

Alan (00:30:33): No, I'm not trying to be snarky about it. I know, I think that you're allowed to be a socialist, you're allowed to be a hardcore capitalist, you're allowed to be anything in between. I certainly don't feel like I have any particular wisdom or expertise to adjudicate between rival conceptions of the political economic good. I think what I think is interesting is to say, okay, let's take this seriously, and let's say that what we do want is this common good type distribution of resources. Does the world of really powerful AGI change the approach to doing that? And I just don't know the answer to that question. But I'd be curious what the economists say. If you are, in fact, a socialist, does the possibility of profound artificial intelligence change what your existing toolkit is?

Kevin (00:31:30): Well-

Alan (00:31:30): Or should be?

Kevin (00:31:31): That's-

Andrey (00:31:31): Yeah

Kevin (00:31:32): ... I want to hear from the economists, too, but I do want to make sure, I'm going to push back just on the AI adoption and the AI diffusion narrative. I do agree that if you are already on the internet and tech-savvy, you can find a way to use these tools. But I think across the whole of humanity, in a lot of the global majority, there's just still not the infrastructure even in place to do that reliably, and the idea that we'll have the infrastructure in place in the near future to have an equivalent access to AI as someone living in the Bay, for example, is just not going to be the case for years, if not decades, absent some crazy change. So in terms of whether or not this is an actual policy matter, I do think that if your goal is diffusion across a lot of communities, that is a huge barrier, and billions, if not trillions of dollars would have to be spent pretty quickly to actually accomplish that kind of universal use of AI. Aren't we just talking about Starlink plus cell phones? What other technologies, what other infrastructure do you have in mind? Even with that, the idea that is Elon going to make Starlink available for free to everyone around the world? And then-

Andrey (00:32:53): Of course not.

Kevin (00:32:54): Yeah.

Andrey (00:32:55): Well, Google will, and Facebook will, and Facebook is the internet in many places. And not everyone has a smartphone, but almost everyone has a smartphone, and they have Facebook on it. So I've been skeptical of digital divide narratives for the longest time, I think, and they've led to enormously bad policies like investments in extremely inefficient broadband solutions when Starlink makes it all obsolete. Just central planning gone wild, in my opinion. So-

Kevin (00:33:25): Which is the pope would back him up on. Pope's anti. This is a whole anti-technocratic centralization overreach essay. This is a very seeing like a state-pilled essay we read. And I think it's really interesting to kind of think about this question which you posed, which is, is there something about this new technology which is kind of essentially more centralizing? And I know a lot of ink has been spilled about, oh, well, here's various ways that AI will tend to allow us to do things independently or in small groups that you would have needed huge teams for, or maybe it'll make us weirder in ways that'll be more diverging and idiosyncratic, and that'll lead to smaller scale groups. But I got news for you guys. My reading of everything together is that at the end of the day, AI is a technology that tends to make more centralized concentrations more efficient. It can process vast amounts of data in order to make more centralized decisions. That was always the critique of centralized decision-making, is that you couldn't process everything. Well, we're starting to get to the place where you can process a lot more. And now I'm not going to argue for central planning, but it does make me think that this is an age coming up where the economic forces will be towards centralization, and that includes big foundation model companies like OpenAI and Anthropic, and it might include companies that we don't even know about yet that'll grow to immense size. But I think that's why it's so important for the pope to be arguing for subsidiarity, not from the perspective of this Hayekian efficiency argument, but from actually, there's some other reason we want to preserve it, and it has to do with dignity. It's not a neo-Brandicean concern. It's not that big companies are essentially evil. It's just that there's something ennobling about decentralization. I think that's particularly compelling, going back to this idea of having some degree of agency in an age in which you may have just a handful of companies dictating incredibly powerful decisions. Just if I accept your scenario, Andre, for example, where it's just Starlink and Meta and Google offer you internet, Apple gives you a phone, and then you're running OpenAI or Anthropic's AI. And so now you just have seven companies who dictate kind of the entirety of your economic existence, if not your informational existence. How then, to Seth's point, do we maintain some degree of, I have a degree of control over my own future and well-being in a world in which seven companies shape the ins and outs of how you wake up, what you do for work, what you read, so on and so forth. Maybe faith and a connection to humanity and an emphasis on agency is the only thing you can kind of stress to make people feel like they have some meaningful role in shaping their lives.

Andrey (00:36:37): Yeah

Seth (00:36:38): They also serve what we can like-

Andrey (00:36:39): So it's a pretty nihilistic take. I don't feel like I don't have agency over my life just because I use Google and Apple products. Those are products, those are bicycles of the mind. I can choose how I use them.

Seth (00:36:54): I'm with... Yeah.

Andrey (00:36:55): And I think that's true for AI models. Look, I understand that they have values baked into them and so forth, but in the end, I can do a variety of things with them across any ideological spectrum that I can practically think of. Yes, there are subtle biases and nudges and so on, but I don't feel like I've lost agency due to AI. I feel like I've gained agency due to AI. So I just think that this is a bit of a hypothetical, honestly.

Alan (00:37:26): Well, isn't all of this a hypothetical? That way [chuckles] we've-

Andrey (00:37:31): No, but it is just like this loss of agency. I found one part of the essay interesting, which is the part about the labor market. So he says: "The labor market is one area in which the risks associated with new technologies more clearly emerge. It is thus necessary to remember that economic freedom is not absolute. It must be measured against the common good and the dignity of every person." Blah, blah, blah, blah. "This is possible when it recognizes the creation of dignified, valuable jobs are an essential part of its proper service to society."

Seth (00:38:09): Right.

Andrey (00:38:09): So I think maybe more speaking to this point of dignity, the pope is arguing for a make work program, like in the style of the New Deal, to give people jobs just so they feel dignity.

Seth (00:38:21): The pope likes free distribution. The pope is not sucking wealthier theorem pilled.

Alan (00:38:26): No, and look, just to defend the pope for a second, I think his instinct that work is an immense source of dignity and that once people's material needs are met, as they are increasingly in the not just developed world, but in the developing world, that these questions of dignity and entity become really... They sort of begin to hedonically dominate. They really are where you get a lot of your utility from. The idea that AI will replace all potential, certainly white-collar work, is a huge threat. The problem is, what do you do about that? And that's where I feel like the insufficient AGI pilled-ness is the problem here.

Seth (00:39:09): Right.

Alan (00:39:09): Because a world in which people who use AI can outcompete by a factor of 10 or 100 or 1,000 to one, the economic productivity of those who don't, is not a world in which make work is going to work. I feel like people get very excited about the three make work jobs created during the New Deal, and then they're like, "Okay, that's an actual thing that can get done." Make work, paying people to dig ditches is not a thing that is sustainable, and people also see through that. So the real question is, I think not even an economic one-

Seth (00:39:43): It's not sustainable in a dignified sense, right? The idea is there'd be so much income

Alan (00:39:47): It's not sustainable in a dignified sense. Well, also, people just won't do it. Governments just won't tolerate this indefinitely. And so the question is-

Seth (00:39:55): Well, this is a real politic. Wait, no. Wait, I want to understand why you don't think it's sustainable. So I can understand why it wouldn't be sustainable in a dignified sense if you like, "Oh, it's actually make work, so I'm not getting dignity from it." But the idea is in this AGI scenario, there'll be so much income that we can support people who aren't actually contributing.

Alan (00:40:11): Yeah. I guess that's right.

Seth (00:40:13): Are you saying that like we'll-

Alan (00:40:14): No, I think no. So I'll take that back, because you see that a lot in petro states where a lot of the economy is propped up by sort of this kind of pointless public sector, this pointless bloated public sector. So fine, let me just going to go back to my dignity point, which is to say, at some point, people start seeing through it, and then the question is the one that I feel like the pope's encyclical keeps pushing off, and we're like, I'm actually more interested in the pope's kind of... I want the pope's sort of Catholic existential analysis more than I want the sort of political economy analysis, which is, just go back to the original question. In a world where we are no longer the most useful, most intelligent beings, how do we deal with that in a way that is productive and in a way that allows us to save face? And there are models-

Seth (00:41:10): Yeah

Alan (00:41:10): ... that we could look to. Chess, and I think chess is always an interesting example here. For a long time, machines couldn't play chess, and then there was a time when machines could beat humans, famously Deep Blue and Garry Kasparov, and then for about a decade, you had what was called the centaur era, where machines plus humans, these centaur teams, were the best, and then at some point, the humans just started causing problems. And so humans contribute nothing at this point to chess. I'm pretty sure an iPhone can defeat Magnus Carl... Like an old iPhone can defeat Magnus Carlsen at this point. And yet chess has never been more popular, and people watch Magnus Carlsen.

Seth (00:41:49): Right.

Alan (00:41:49): So there are examples where we have worked through the existential malaise of we're not the best anymore, but we still want to see humans do it. What I'm very curious about and where I think religion could play a really useful role is going domain by domain and helping us get to the other side of that transition. But to do that, you do have to take it seriously. You can't just continue to pretend that we are the best. No, no, we're not the best. That's the whole point. How do you live an existentially meaningful life when you're no longer the best? That's the question that I think is the most interesting one.

Seth (00:42:25): Well, and to add on to that, too, because I think that we need to distinguish, going to Andre's point earlier, between freedom and control. I think that there's a vast difference between freedom to

Kevin (00:42:37): Use AI to do anything, to look up anything, to pursue boundless knowledge, to, in theory, create anything or do anything, versus actual control over the infrastructure and the decisions and the entities that are shaping the preponderance of governance and the shape of the economy itself. And so you can have freedom, and you can have an increase in freedom, but you can have a decrease in control in terms of your actual ability to shape broader circumstances around you. And I think it's important not to confuse the two, because absent having some degree of control over those meta constraints, over those larger aspects of your life-

Andrey (00:43:17): Mm-hmm

Kevin (00:43:17): ... I do think it's hard to feel a sense of dignity, right? If you're born and to go to this make work idea, and you're told, "Hey, you have one of three jobs. Hey, great, you have freedom to choose your one of three jobs. Enjoy whichever. You can dig a ditch, or you can dig a well, or you can be in charge of high fives. But those are your three jobs."

Andrey (00:43:41): [laughing]

Kevin (00:43:43): Freedom, yay, but no real control over the nature of your life or the lives around you. And so I think that's the sort of dignity plus or-

Andrey (00:43:52): So I guess-

Kevin (00:43:53): ... plus

Andrey (00:43:55): ... I guess I have a question for you. So suppose that we use democratic mechanisms to govern AI labs. So people elect representatives, the representatives sit in a House of Representatives at Anthropic. They vote on the latest constitution. Do you think that people will actually feel more in control of their lives that way? Because to me, it's not very obvious, right?

Seth (00:44:24): Congress is not glowingly reviewed by US society.

Kevin (00:44:29): I think that if you give an American an outlet, they'll feel more control. That's my attempt at a riff of If You Give a Mouse a Cookie. Look at how-

Andrey (00:44:40): [chuckles]

Kevin (00:44:40): ... folks are making-

Andrey (00:44:41): Yeah, those books don't tend to end well. I have two small children, and I can tell you, those books are not optimistic stories. Those books are psychological horror films-

Kevin (00:44:50): Oh, Jesus

Andrey (00:44:50): ... in miniature.

Kevin (00:44:51): We'll save that for our next episode. But I think that if you look at how people have made use of town halls and permitting processes right now-

Seth (00:45:03): Right

Kevin (00:45:03): ... with respect to data centers, I think those people would feel like they are actively shaping, and they are actively shaping the AI infrastructure build-out. So I do think if there were an avenue for people to feel as though they or their neighbor could participate, that would meaningfully change how they felt about AI. Whereas right now, the absence of-

Seth (00:45:26): But it's bad. They're using their power for bad. [laughs]

Kevin (00:45:30): That's a different point. On the question of whether they would feel like-

Seth (00:45:34): Well, I think if we're doing political... I guess what I would say here is, I agree with the previous argument that this is an essay that's about the political economy and not about the existentialism. And so, yeah, if you're going to talk about the political economy, you should care about the economy part of the political economy, right? You want to get the voting power mixed the right way to create the prosperity that makes doing what you want to do possible. I think the pope would've been in an excellent position to do what you described before and talk about how a monastic life could be a model for thinking about what an AGI age would look like. Or, I think about the Messianic age in Judaism, where it's kind of a post-scarcity society, and everyone's devoting themselves to Torah study and mitzvah or whatever, right? That would've been a really interesting essay. This is a political economy essay. And then I do think you have to take a stance on, well, we should maybe put the power in a place where people are making better decisions.

Kevin (00:46:37): Well, I think-

Andrey (00:46:39): Well, there is an inevitable efficiency trade-off, right? If we think that the ASI is going to be very smart and is going to be well-aligned, the well-aligned is the questionable part, maybe. But if it's well-aligned, then we know the masses have their issues in terms of making decisions. And so-

Seth (00:46:58): Is that a Catholic pun, masses?

Andrey (00:47:00): [laughs] So yeah, we could delegate to the ASI to make our decisions. And I think this is, I guess, what it's being warned against, is it doesn't matter if the ASI is more efficient, that just the very fact that humans are in the loop in a real way is-

Kevin (00:47:20): Well, at 42,000 words-

Seth (00:47:21): That's the essay we got. Yeah.

Kevin (00:47:23): We could keep going until eternity, religious pun intended.

Andrey (00:47:30): [laughs]

Kevin (00:47:30): But assuming that we're not going to, why don't we transition here?

Seth (00:47:34): [upbeat music] For those of you playing along at home, now is your chance to think about how this conversation has changed your priors. This chance to contemplate your posteriors is sponsored by Revelio Labs. Revelio Labs is a leading provider of labor economics data and data services for companies, academics, and independent researchers. Andre and I have been working in economics of AI, digitization, and automation for a long time, and we can confirm just how useful Revelio's data is. Revelio's team combines comprehensive micro-level data on employee professional profiles, job postings, and employee sentiment with standardizations, mappings, and enrichments available, all to make that data useful without making your modeling decisions for you. The data can be flexibly aggregated to company, market, or industry, and can be used to study questions ranging from career trajectories to occupational transformation, to the returns to skills, and the impact of AI on labor demand for tasks. Can't imagine anyone who would be interested in that. And Revelio data is available on WRDS.

Kevin (00:48:39): So if you're an academic with a good library, go see if you have access to their premier data already. And if you don't, you can reach out to their excellent economics team, and they'll hook you up.

Andrey (00:48:49): Ooh, the next piece that we're considering is this piece by DeepMind, various DeepMind authors, Positive Alignment.

Kevin (00:48:57): Including the friend of the show.

Andrey (00:49:00): Yeah. Seb Krier, who refused to take ownership of this paper. [laughing] But yeah. Alan, Kevin, what'd you think?

Kevin (00:49:12): I'll let Alan start. I've been talking for way too long.

Alan (00:49:17): Yeah, look, I think it's an interesting point, and the inner social psychologist in me likes it, and thinks that the positive psychology turn should apply to alignment as well. And then the old school, Cold War liberal in me gets very nervous about these kind of positive conceptions of human flourishing, right? So, there's this idea that post-war liberalism became what's sometimes called the liberalism of fear, which is the idea that in the wake of all the totalitarian ideologies of the 20th century, Western liberals retreated to a kind of defensive crouch, where the point of liberalism was just to prevent the worst excesses of totalitarian and authoritarian ideologies. And you're not supposed to look to liberalism, or frankly any political philosophy, for a sort of a positive conception of the good.

Kevin (00:50:20): And that's also Rawls, right?

Alan (00:50:22): Yes. This is related to Rawls's idea of political liberalism. This has become unfashionable lately because something, something neoliberalism, something, something. But I still think there's a decent amount of wisdom in that. And so, whenever I read these proposals that AI systems should promote human flourishing, which to have bite is always... Which only has bite-

Kevin (00:50:53): On the other hand

Alan (00:50:54): ... if the systems are not doing what their human wants them to do in that moment. Otherwise, none of this would matter, right? That would just be covered by regular alignment. I get a little nervous. So again-

Kevin (00:51:07): I wish the essay even got there

Alan (00:51:08): ... at a high level of abstraction, these are my trade-offs. Sorry, what?

Kevin (00:51:11): [laughs] I wish the essay even got there. If the essay had made the point we need a thicker notion of the good in order to blah, blah, blah, it would've been something fun to argue about and get antsy in our liberal pants about. But it equivocates so much, and it's so wishy-washy about, "Yeah, we want a thicker notion of the good, but there are so many different notions of the good." So, I don't even think it ended up making me afraid as a liberal.

Alan (00:51:37): Yeah.

Andrey (00:51:37): I don't know, Kevin, do you have anything to add?

Kevin (00:51:39): Yeah. For me, this kind of does tie neatly, at least in part, about what we were discussing with the encyclical, which is we also still need a idea of what it does mean to positively align a model to anything, right? The selection of your social welfare function about whatever your objective is going to be is a really, really hard task, and I don't think that we necessarily know, again, to bring in the prior discussion as well. The idea of what the positive outcome is in any context that you are training a model on or setting a model to might vary wildly from one culture to another, and it may change-

Alan (00:52:25): Right

Kevin (00:52:25): ... over time. And so I appreciate the idea that we should not only look to avoiding negative outcomes and instead try to train models for some degree of positive behavior. But this brings to mind that folks, when Amanda Askell came on scaling laws and talked about Claude's Constitution, many people were not pleased when she said that her ideal was Claude acting like a good neighbor or a good traveler. Folks did not like that as the ambition for Claude. And I'm not saying that's good or bad or otherwise, I'm just saying it's really, really difficult to try to even encapsulate, envision what should a model do. What should that positive alignment be? What does human flourishing-

Alan (00:53:16): Right

Kevin (00:53:16): ... even look like? And that, to me, is where having more granular ability to train models will be profoundly important. Or at least to use some system prompt that directs your model quickly to whatever your community or your conception of human flourishing looks like, so that we can have that broader kind of polycentric approach to aligning models. But I don't think there's ever going to be one single approach, which is really difficult.

Andrey (00:53:51): Yeah. I totally agree with you. I think one of the things much of this discussion misses or it's swept under the rug is that they want to pose this as some philosophical dispute, but it is an empirical question. As someone who studied digital platforms for my entire career, whether something is good or bad, in many ways evaluated by running an AB test and seeing whether the outcomes that you're measuring are improving. And it's completely not obvious ex ante. And I think with anything in such a constitution, that's also likely to be the case. Now, I understand why you would take this approach before you have the data. You have to take a stand on some of this stuff. But in the end, I think a lot of this is an empirical question in addition to a philosophical question. I think I have a broader take on this work is I have no idea who the audience for this is. And I feel like DeepMind puts out a lot of these papers. They're very general. You can see that they're citing a lot. They're citing everyone, and everyone is, we have RLHF, and we have supervised-

Seth (00:55:01): 10 co-authors

Andrey (00:55:01): ... fine-tuning. Yeah. And let's put in a lot of people on this paper to say a bunch of vague stuff that means nothing. They don't take a stand on anything, essentially. And then people laud this sort of stuff online, I think because they read the abstract. It's like, "Oh, yeah, vaguely I agree that we should try to have the models help human flourishing." But then when you read this paper, even though grammatically it's correct, it is vacuous. It is such a waste of time for them to write it, or alternatively, I have no idea who the audience of this thing is. And this is not only criticism of DeepMind. So many of these AI policy people write this stuff. And I just found it just to be deeply uninteresting. Yeah.

Seth (00:55:58): It's for people who don't own a thesaurus and want lots of synonyms for flourishing in different languages. I have to say, reading this after the pope's encyclical made me more negative on the pope's encyclical. Because in it, he says something like, "No technology is neutral. We need to design technology from the ground up in the labs to have all of these five Catholic principles." And I read this, and it's like, do not let the computer scientists do ethics because they're bad at it. [laughs]

Kevin (00:56:29): So what would be your conception of a useful, positive alignment paper? If you were to have a one-on-one with these authors and say, "Y'all, look, A for effort, F in execution," based off of what I've heard. Maybe you all assign slightly different grades. What is your feedback? Speak to us, the AI policy community. How can we now go to our friends and say, "Hey, y'all, we talked with Seth. We talked with Andre. Apparently, we're getting it wrong. We're not doing it well." What is the feedback? What can we do to be better?

Andrey (00:57:10): Take a stand. If you think that you want a particular conception of positive alignment, tell me what metrics are indicative of that, and then propose a methodology, even if it's not immediately actionable, for how to measure whether a system is pushing us in that positive direction. Maybe do post-interaction surveys with users to see how satisfied they feel or how happy they are, and then you put them in an A/B test and compare different system prompts or... I'm just spitballing here. But give me something actionable instead of giving me a list of vagaries and then it's not even clear in this paper whether they consider that currently labs are already doing positive alignment or whether that's something that's new that needs to be done. Because they list a bunch of things in there that already sound a lot like positive alignment to me.

Seth (00:58:07): Right.

Andrey (00:58:08): So, yeah.

Seth (00:58:09): You guys talked to Askell, which I'm so jealous of. We read the Claude Constitution, and that's where I want people thinking about. That's the actual principles that we're building AIs on right now. What would it look like to get a different team together to have its own hierarchy of values in the Claude Constitution? Can we think about other ways of adjudicating whether an AI is following its principles? Yeah, because like Andre says, if you read the Claude Constitution, there's plenty of positive goals in there.

Kevin (00:58:42): Well, so this goes to my point earlier, where I do think that constitutional AI presents a really interesting nexus for public engagement and public participation that we have be the mechanism by which people do feel like they're in control or have some degree of oversight with respect to AI because even if it were-

Andrey (00:59:08): Mm

Kevin (00:59:09): ... a citizens assembly of 1,000 people across the US who are engaging, let's say, with Claude in shaping Claude's constitution with one another, so on and so forth. Well, now if I'm like, "Well, hey, I know my buddy Alan was a part of the latest constitutional convention for Claude," and I told him, "Hey, bro, you better make sure that Claude's a little bit more of a fan of the Texas Longhorns or whatever." [laughing]

Seth (00:59:36): Principle five.

Kevin (00:59:37): Maybe now I'm like, "Hey, at least I had some chance of influence," or I know somebody who influenced that person. Whereas right now, I don't think most folks even know anyone who lives in San Francisco because no one can afford it, with the exception of you, Andre, which I'm pleased to hear you're in town. [laughing] But right now-

Andrey (00:59:58): What are your secrets, man?

Kevin (00:59:59): Yeah.

Andrey (01:00:00): I'm on leave at Amazon. I think it's not a secret.

Kevin (01:00:02): Yeah. That helps. [laughing] But so that, to me, is a promising vehicle by which we can use that mechanism of what does positive alignment mean in practice. I like your point, Andre, of like, "Hey, go present some scenarios. Go do that A/B testing of saying, how do you want Claude to respond to this very difficult, let's say, even democratic context. Should the president or should the president not invest in this stake in Intel?" Big, huge question. Let's see what people say. Let's see what values we can then deduce from their answers, right, and have that sort of inverse constitutionalism, which could be really interesting. But-

Seth (01:00:47): Yeah, there was the MIT Moral Machine Project where they were doing... Did you ever see this? They did millions of trolley problems with people decentralized to see, would you rather run over one grandma or two criminals? Have you seen this?

Kevin (01:00:59): No, I'm going to have to check it out.

Seth (01:01:01): All right. You have to look that up.

Kevin (01:01:02): How many criminals guys versus grandmas?

Seth (01:01:05): Exactly. Well, it's the ratio. Can I actually ask you guys a question? This is kind of my big law question I was hoping to get an insight on, which is to us, this constitutional approach seems really promising, but to what extent do you think the constitutional part of constitutional AI, does that metaphor really hold versus where does that metaphor work versus where does it break down compared to something like the US Constitution?

Kevin (01:01:34): Well, I'll start briefly by saying Anthropic has been clear, in defense of Anthropic, to say they do not intend this to be a constitution qua the US Constitution, something that evokes the same legal understanding. So it's important to note that the labs have tried to distance the exact mapping of, let's say, the frontier safety framework or the model spec or the Constitution to a legally binding document. Now, with that said, I think that the idea that we're going to have high-level principles and values instilled within something that's going to have an enduring and stable influence on something makes a heck of a lot of sense to me in terms of a constitutional analogy there, which is to say, hey, if you actually go read the US Constitution, there are a lot of huge open questions, which is the reason Alan and I have jobs-

Alan (01:02:33): [laughing]

Kevin (01:02:33): ... is because the Constitution doesn't specify how the president has to react here or what Congress must do there. And so I think there's a lot to the idea of forcing people to think critically and deeply about what it is they want to see from something that's going to have a very real impact on society, on the economy, and on governance. So in that regard, I think it's really important. I also think that the idea that it needs to be grounded in the sort of consent and ideas of the users is a particularly compelling analogy and nexus as well because the Constitution right now is in a state of, I don't know what the best word to say here. If you look at Gen Z-

Alan (01:03:24): [laughing]

Kevin (01:03:24): ... there was a recent poll by, I actually don't know how to say this name. IPSOS. Is it IPSOS?

Seth (01:03:35): Ipsos.

Kevin (01:03:35): I don't know.

Seth (01:03:35): We say Ipsos.

Kevin (01:03:36): Okay, Ipsos. So Gen Z, they're like, "Yo, I know we have a constitution, but we don't really care."

Alan (01:03:45): [laughing]

Kevin (01:03:45): "It's fine that it's there, but it's not something we wake up thinking about. It's not something that we think is necessarily in line and true with the country's values or practices." So what are the mechanisms we can create now to give people a sense of ownership or at least a degree of connection to the Constitution, I think is an open question with respect to AI constitutions, right? What is that enduring mechanism that makes people feel like they were actually a part of that process? I'm not sure. I'm not sure if deliberative democracy gets you there, right? If we pick 100 random Claude users, stick them in a room in San Francisco, and say, "Come up with the Constitution," does that solve the issue of people feeling like they didn't have a chance to shape it? I don't know. But I think it raises a whole set of really interesting questions.

Seth (01:04:43): Can I ask-

Kevin (01:04:44): Yeah.

Seth (01:04:44): Okay, let me ask a couple of those follow-up questions. Or you go first, Alan. I want to hear what you think.

Alan (01:04:48): No, no. No, that's fine.

Seth (01:04:50): Oh okay. So follow-up question number one. When you talk about the US Constitution being kind of in tension right now, one of the ways it's sort of in tension is a lot of people feel like the Supreme Court is using the Constitution like a rubber band, right? And you can kind of read anything between the lines of it. Do you think that the AI Constitution is, in some sense, more reliable than a government constitution because there isn't a potentially politically motivated Supreme Court having to interpret it?

Alan (01:05:26): Well, not to get too down the weeds about the Supreme Court. I would say all Supreme Courts are about equally politically motivated and-

Seth (01:05:36): Correct

Alan (01:05:36): ... people just don't like who's on the current Supreme Court, which is a perfectly-

Seth (01:05:38): Sure

Alan (01:05:38): ... fine thing to not like. I think that the question, and this is an empirical question rather than a conceptual question, is how good is this constitutional approach when it comes to alignment, right? At the end of the day-

Seth (01:05:55): Right

Alan (01:05:55): ... I think if you strip away all of the analogies to the legal system, which I get why they made, but I don't think are terribly helpful. I think the bet that Anthropic is making is-

Seth (01:06:04): That was going to be my next question. [laughs]

Alan (01:06:05): Well, I think the bet that Anthropic is making is that a kind of almost virtue ethics approach to alignment, where instead of trying to program specific behavior or have lots of hypothetical cases, instead you try to program in a kind of disposition, and that is what's going to, in the long run, get you the best outcome. As someone who likes virtue ethics, generally, I hope it works. But that is the big question. Also, if you think the Supreme Court is political, which of course it is because it's a human institution, I'm not sure why you wouldn't think that Anthropic is also political, right? At the end of the day-

Seth (01:06:50): Sure

Alan (01:06:50): ... and this is one of the great problematic, I think, disanalogies between the Anthropic Constitution and a regular constitution. A regular constitution is meant to be difficult to amend. That's kind of the whole point, right? It's not meant to be "regular, normal law." But the Claude Constitution is just set by Anthropic, right? Now, maybe what they mean, well, we've trained the model ... on this constitution in a deep way that it is difficult to override that particular model, because this constitution-

Andrey (01:07:19): Corrigibility

Alan (01:07:20): ... is baked deeply in, right? Yeah, exactly. But that, of course, doesn't mean that Anthropic can't train the next model with a different constitution. And so, yeah. From an alignment perspective, the constitutional AI approach might be really great, and that would be an enormous accomplishment. From a self-binding perspective, which is really what constitutions are meant to do in democratic theory-

Andrey (01:07:45): Mm-hmm

Alan (01:07:45): ... I don't think the Constitution does all that much.

Andrey (01:07:48): Hmm.

Kevin (01:07:49): Right. So the interesting question there would be, do we begin to see if this does become a nexus for public policy intervention, which is to say, "Hey, we want to see that you have a constitution. We want to know that you published it. We want to know that there was some degree of user involvement," or so on and so forth. Do we begin to see things like Alan's alluding to? Are there recurring opportunities by which we would expect the Constitution either lasts for a certain period of time, or when it needs to be amended at a certain period of time, or how that's disclosed and how that goes about? This is what I broadly refer to as the future of governance studies, which is to think about the new questions we haven't even asked about when and how we try to kick the tires on some of these AI governance mechanisms. But it also begs the question of, well, how good is the AI to begin with on the whole, right? And what is it shaping and what is it changing? An AI constitution may matter if it is indeed, in particular, being adopted into government processes, right? One of the things that I-

Andrey (01:08:54): Yeah

Kevin (01:08:54): ... have been toying around with, to go to Andre's point earlier about A/B tests, coming up with a hypo, a fact pattern based off of, let's say, a weird new First Amendment-related law that Congress passes. Grok will interpret that one way and come out with one answer. Claude will come out with the opposite answer. And now if you think about a justice on the Supreme Court going and entering that fact pattern to one of those models or the other, and the anchoring bias that then would result from having consulted that AI first, that one AI or the other. Well, now I do care as a member of the public. I really want to know which model are you turning to first, what are the values that are being baked within that model, and then brought into the public context. I think that has huge governance implications. And I think AI constitutions, to the extent they are corrigible, to the extent that they are robust in different circumstances, could be a really interesting means for that governance mechanism.

Andrey (01:09:58): Incorrigible.

Kevin (01:09:59): Incorrigible, yes. Thank you.

Andrey (01:10:00): Yeah.

Alan (01:10:01): Yeah.

Andrey (01:10:02): There is an interesting thing that comes to mind just from that idea, was that probably for all proposed Supreme Court cases, someone should be doing the exercise of just running them through all the major models and seeing what happens and whether they agree or disagree. That could be a measure of, in some sense of like a deviation-

Kevin (01:10:23): I keep meaning to-

Alan (01:10:23): Type code that

Andrey (01:10:25): Supreme Court-

Kevin (01:10:26): Do it. Do it, Alan.

Andrey (01:10:28): [laughs]

Kevin (01:10:30): It would not be that hard. Honestly, I could do that-

Andrey (01:10:33): No, it would be pretty easy

Kevin (01:10:33): ... in a weekend.

Andrey (01:10:35): Oh, yeah. Yeah.

Kevin (01:10:36): Prove it.

Alan (01:10:37): Yeah.

Andrey (01:10:38): All right. That's our crossover experiment coming out of the pod.

Kevin (01:10:41): Yeah, crossover vibe check.

Alan (01:10:43): Yeah. We should do that, yeah.

Kevin (01:10:46): It would go viral so quick, bud. [laughs]

Andrey (01:10:53): So one thing I did want to mention, just that actually made me think of this. So a person who's been doing many such experiments is Andy Hall over at Stanford, and he's previously worked with Meta on the oversight board and their attempts at deliberative democracy. And I don't know enough about those details, but my understanding is it was all a big failure. And so I'm curious whether we can learn anything from that experience as we try to think about very similar problems with regards to the AI constitutions.

Kevin (01:11:26): I've tried my hand to be a bit of a student of the oversight board, and I think that the realization is that an institution has to be shaped to its context and not based off of good New York Times coverage when it initially gets announced.

Andrey (01:11:42): [laughs]

Kevin (01:11:42): And that, to me, was kind of the biggest failure of the oversight board, where in abstract, it sounds awesome to have the former prime minister of Denmark, and to have this free speech advocate-

Andrey (01:11:57): [laughs]

Kevin (01:11:57): ... and to have this great lawyer, and so on and so forth. You put them all together on this oversight board. But then when you kick the tires on the oversight board itself, the thing that became readily apparent was just it wasn't suited to the scale of Facebook's content moderation issues. And so its opinions were coming out months after a hot-

Alan (01:12:19): Yeah

Kevin (01:12:19): ... topic needed to be addressed.

Andrey (01:12:20): After the genocide already is over, they can ban the speech, right?

Kevin (01:12:24): So not wildly helpful, and then the fact that it was comprised of more or less the Western world, whereas many of the cases were coming from the global majority, just didn't look great, for lack of a better phrase. When you have weird, W-E-I-R-D, weird people governing the rest of the world in terms of content moderation, you're running into red flags, too. And then I think you also have to recognize that undue complexity is often going to undermine an institution. So there were all these considerations about, okay, well, once an opinion comes out, then Meta may or may not adopt that resolution, but in some cases, it's binding, but in other cases it's not. But Meta might get back to you months later and tell you they've changed it. And so there was a kind of convoluted nature as to what the actual dynamic was between the oversight board and Meta. And then, of course, the composition of the board itself was problematic. And then again, the resources that were afforded to the board in terms of being able to go through the full scale Of the number of appeals of content moderation decisions was always going to be inadequate. And so I think the lesson from the oversight board is trying to create a Supreme Court for anything is a bad idea unless you're just trying to recreate the Supreme Court. You have to be [laughing] attentive to the actual circumstances you're trying to govern. And that's hard because lawyers, the people who design these systems, and here's where I shit on my own profession, we're not known for being creative. We're not known for being imaginative. And so when you ask us, "Hey, what's a good new governing system for this bespoke crazy technology?" We're like, "Well, I don't know. A Supreme Court sounds like a great place to start." [laughing] And everyone's like, "Well, yeah. That seems spiffy." So I think that's the fundamental challenge is leaning into the fact that we can't just copy and paste the institutions we've relied on to govern somewhat related challenges in the AI context. We've got to be, to use Jack Clark's phrase, we need new regulatory technologies. We need new means of measuring and intervening and of just governing. But that requires thinking outside the box, and that's just not something that's on most law school curricula, unfortunately. But Alan, I don't know. Tell me if I'm wildly off base.

Alan (01:15:07): No, I think that's right.

Andrey (01:15:09): All right. Well, that was a really fascinating discussion. We actually had a lot more that we were hoping to cover, but given the long documents we had to discuss, I think it's not surprising that we only got through so much. So maybe there will be a part two someday. But really enjoyed this conversation. [upbeat music]

We start with her Digitalist Papers essay, which proposes a flexible, two-tier toolkit: AI Adjustment Insurance (extended unemployment benefits + retraining + wage insurance, modeled on Trade Adjustment Assistance) for the churn scenario, and a scalable Digital Dividend — a broad-based cash transfer funded by a small tax on the digital sector — for the world where the jobs don’t come back. Along the way: whether to make policy now or wait, what counts as the “status quo,” moral hazard in mass unemployment, the TAA wage-insurance result that repaid its own subsidy, and Andrey’s “we can’t afford UBI” pushback.

Then we get into her new model with Konrad Kording, (Artificial) Intelligence Saturation and the Future of Work”— why splitting the economy into an intelligence sector and a physical sector implies that output and wages saturate even as AI scales to infinity, the robots-vs-LLMs debate, and whether to just relabel “physical” as the non-automatable sector. We close with her DOJ years: defining monopsony, the transmigrante used-car collusion-and-murder case, the Penguin Random House–Simon & Schuster merger (yes, Stephen King testified), antitrust and AI, and a lightning round on ikigai, Camus, and Rawls vs. Mill.

Links & References

Ioana’s work

• marinescu.eu — Ioana’s website · Penn SP2 faculty page

• Ioana Marinescu, “Resilient by Design: Dual Safety Nets for Workers in the AI Economy” — The Digitalist Papers, Vol. 2: The Economics of Transformative AI (volume)

• Konrad Kording & Ioana Marinescu, “(Artificial) Intelligence Saturation and the Future of Work” — working paper (Brookings write-up & interactive tool). The model finds wage growth can reverse once roughly a third of intelligence tasks are automated.

• Ioana Marinescu, comments on Betsey Stevenson’s chapter — NBER, The Economics of Artificial Intelligence: An Agenda (the ikigai discussion)

Concepts, papers & people discussed

• Trade Adjustment Assistance (TAA) — the template for Ioana’s adjustment insurance; the wage-insurance component that got people back to work faster and was net fiscally positive

• Betsey Stevenson, “Artificial Intelligence, Income, Employment, and Meaning” — the post-AGI meaning / ikigai argument Ioana was commenting on

• “GPTs are GPTs” — Eloundou, Manning, Mishkin & Rock, GPTs are GPTs: An Early Look at the Labor Market Impact Potential of LLMs — the occupational LLM-exposure measure (”Eloundou et al. / Daniel Rock”) correlated with COVID-era telework

• Pascual Restrepo — job-market work on skill mismatch and structural unemployment during automation waves

• Daron Acemoglu & Pascual Restrepo, “Robots and Jobs: Evidence from US Labor Markets”.

• Albert Camus, The Myth of Sisyphus; ikigai (Japanese: “reason for being”)

• Baumol’s cost disease

• John Rawls and John Stuart Mill (Utilitarianism)

Antitrust & the DOJ

• The DOJ Antitrust Division, monopsony in the labor market, and the 2023 Merger Guidelines

• Judge blocks the Penguin Random House–Simon & Schuster merger (2022) on a labor theory of harm to authors — Stephen King testified for the government

• The transmigrante used-car export case — collusion (and worse) in the US-to-Latin America used-car trade

• Anthropic’s Economic Index and Economic Advisory Board

• Leopold Aschenbrenner’s Situational Awareness — the “we’ll have to nationalize it” argument referenced on consolidation

Previously on Justified Posteriors

• Our episode on the Anthropic Economic Index.

Our sponsor

• This episode is brought to you by Revelio Labs, the leading provider of labor-economics data, available to academics on WRDS.

Chapters

• (00:00) Intro & sponsor

• (00:47) The Digitalist Papers proposal: a flexible safety net for the AI labor shock — and why make policy now

• (03:48) Why unemployment insurance isn’t enough, and the Trade Adjustment Assistance template

• (05:51) What counts as the “status quo”? Banning AI vs. letting it run

• (07:42) How much to insure: moral hazard, mass unemployment, and the three parts of AI Adjustment Insurance

• (11:15) Skill mismatch (Restrepo), and how do you certify a layoff was “due to AI”?

• (14:45) Did TAA buy social buy-in for free trade? Underfunding — and the wage insurance that repaid its own subsidy

• (16:38) “Would Hillary be president?” General-equilibrium pushback and the ski-instructor problem

• (19:28) Will the new jobs still be there in two years? The lump-of-labor fallacy

• (22:09) Policy B: the Digital Dividend — unconditional, broad-based cash from a small digital-sector tax

• (23:52) How to fund it: a sales tax, a sovereign-style fund, and deliberately slowing diffusion a little

• (26:00) “We can’t afford UBI”: productivity growth, 0.5% vs. the deficit, and setting money aside ex ante

• (30:47) Taxing digital goods: VPNs, evasion, and land-value taxes

• (34:23) The motte-and-bailey worry, and the other reasons to like UBI

• (36:05) The new model: (Artificial) Intelligence Saturation — intelligence vs. physical sectors, and the telework × AI-exposure correlation

• (40:14) Gross complements: why output and wages saturate even with infinite intelligence

• (42:23) Won’t enough intelligence just automate the physical world? Robots vs. LLMs

• (45:52) “15% by 2030”: humanoid robots, cost, and bespoke vs. general-purpose machines

• (47:58) Baumol, the “humanness sector,” and relabeling physical as the non-automatable sector

• (48:52) The capital-share / profit-share puzzle: if they’re complements, why has the intelligence share risen?

• (50:25) The DOJ years: monopsony, and what the Antitrust Division actually does (mid-roll sponsor at 51:29)

• (54:52) “Assassinating rival CEOs”: the transmigrante collusion-and-murder case

• (58:12) Favorite cases: Stephen King, the publisher merger, and the chicken-farmer monopsony settlement

• (1:01:30) Antitrust and AI: foundation models, consolidation, and the natural-monopoly question

• (1:06:05) Slowing AI by allowing market power; Leopold, nationalization, and diminishing returns vs. the singularity

• (1:09:27) Substitutability, the AK economy, and short-run vs. long-run wages

• (1:10:59) Lightning round: ikigai, Camus, and the myth of Sisyphus

• (1:12:44) Can we build market-like mechanisms for ikigai? Loneliness and coordination costs

• (1:14:13) The Anthropic Economic Advisory Board and the Economic Index

• (1:15:21) What’s next: monopsony and industrial policy

• (1:17:59) Favorite philosopher: Rawls vs. John Stuart Mill

• (1:19:45) Sign-off

Justified Posteriors is the podcast that updates its beliefs about the economics of AI and technology, hosted by Andrey Fradkin and Seth Benzell. If we changed your priors, subscribe, share it with a friend, and keep your posteriors justified.

Intro & Sponsor [00:00 – 00:47]

[00:00:06] Seth: Welcome to Justified Posteriors, the podcast that updates beliefs about the economics of AI and technology. I’m Seth Benzell, excited to learn about what AI is other than what my bubbe says after I spill hot water on her, coming to you from Chapman University in sunny Southern California.

Andrey: And I’m Andrey Fradkin, coming to you from San Francisco, California. We’re very thankful to our sponsors at Revelio Labs, purveyors of fine data products. And we’re very excited to have Ioana Marinescu join us today. Welcome to the show, Ioana.

Ioana: Thank you. I’m so glad to be here.

Make Policy Now: A Flexible Safety Net [00:47 – 05:51]

[00:00:47] Andrey: To get started — you have this very provocative, interesting piece in the Digitalist Papers about various social policy solutions for transformative AI scenarios. Could you tell us about the piece?

Ioana: Absolutely. As part of doing this Digitalist piece, I was thinking, as somebody who has worked a lot on the social safety net: what do we do if AI leads to a lot of job loss, like many people are saying it would? We’ll talk later about the various scenarios, but assuming that’s at least a possibility we have to acknowledge, what would you want to have from a policy perspective?

And so I was really thinking hard about devising a flexible policy toolkit that will be able to address issues in the labor market no matter how big the shock is. That was the overarching theme of the policy design I’m proposing — just to start a discussion. I’ve tried to propose some helpful options, but it’s really with the idea of, let’s talk about doing something like this, what are the pros and cons.

[00:02:10] Andrey: So what are the options on the menu for — let’s say AI comes along, a lot of people lose their jobs. The first thing we should get started with: do you think we should be making policy today, or should we wait until something happens and then make policy?

Ioana: I think it’s very important to make policy today, but in a flexible way — meaning the policy cannot depend on some very specific detail of exactly how AI is going to impact the labor market, because we don’t know exactly what’s going to happen. It’s important to put the policy in place today because the political process is very long, so it may not be able to come online quickly enough when we really need it. That’s one reason.

The other is — and I work a lot on social insurance — for workers, they want to and should feel insured. “Whatever happens, we the government have got you covered.” If we don’t have that, and we’re just waiting for bad stuff to happen, that defeats the purpose of having a social safety net. That’s a core reason I think it’s good to have something in place sooner rather than later, even before all the effects of AI on the labor market have materialized.

Seth: Something that will automatically kick in.

Ioana: Exactly.

Andrey: And why is — we do have some programs like that, like unemployment insurance. Why is unemployment insurance not enough in its current form?

Ioana: Unemployment insurance is incredibly valuable, but if we have a big shock like AI, it’s going to affect a lot of people who will not necessarily lose their job forever, but simply have to change jobs — and that’s very costly. The whole purpose of the social safety net is to help people through those transitions.

The thing is, we have AI, and the way it’s being deployed is a policy choice. We could say we’re going to try to stop AI, but we’re not doing that — and I’m not saying we should or shouldn’t, just that it’s a policy choice. We’re saying we’re not going to stop AI, we’re going to let it be. But then some people are going to get hurt, at least in the short run, and we need to do something so those people have something to fall back on. Just like with trade: we decided to have free trade, we knew some people were going to get hurt and lose their wages, and we put in place policies like trade adjustment assistance — which inspired some of my proposals — to make sure the policy we’d chosen wouldn’t leave people on the side of the road. That policy hasn’t completely worked, because it was underfunded, but the big point is: the technology is exciting and has a lot of benefits, we’ve decided to deploy it quickly, and some people are going to bear a cost. We just want to make sure we help those people.

What Counts as the “Status Quo”? [05:51 – 07:42]

[00:05:51] Seth: I’m really excited to hear your specific ideas, but I’m curious about this framing of what counts as the status quo and what counts as the policy shock. In the case of trade, you might say the status quo is protectionism and the policy shock is allowing trade — so it makes sense to frame the social insurance relative to that: you shouldn’t be worse off relative to introducing trade. But with AI it’s not obvious. It seems like the policy shock would be to ban AI — AI would happen without the shock. So why not say, “If you banned AI, there should be social insurance to help the people who would have been better off if we’d allowed AI to go full throttle”? How do you think about what the status quo is here?

Ioana: I don’t know that the status quo is necessarily the correct reference point — that’s something you could debate. My point is rather that a lot of these technologies have a lot of ramifications, and there’s a decision we make about how we want to control it. Collectively, we’ve made the decision that we’re not going to try to control it too much in its effect on the labor market. And therefore we need to deal with the consequences of helping people who might get hurt, at least in the short run — even though, hopefully, in the long run it will be great for everybody, including them. So we can reassure people that it’s going to be fine. That’s part of the goal of the policy.

AI Adjustment Insurance: Moral Hazard and Three Components [07:42 – 14:45]

[00:07:42] Andrey: How do you know how much to insure? Full insurance would be very expensive, but it would also create a lot of moral hazard — we do want people making choices in anticipation of AI. If everyone just puts their head in the sand and pretends it’s not happening...

Seth: “The automation insurance is too good. I want to get automated — give me that automation insurance.”

Ioana: So the insurance is going to be incomplete. And I’ll talk in a moment about what I’m proposing, which is modeled on trade adjustment assistance. But it’s also important to know that some of the moral hazard issues in unemployment insurance — which is something I’ve studied a lot — are much less economically important during times of high unemployment.

If what happens is huge amounts of unemployment — not necessarily forever, but a lot of people needing to change jobs — then typically there are too many people looking for jobs relative to the number of vacancies. In that case, the fact that some people might put in less effort to find a job, sending fewer applications, is actually fine, because collectively they’re sending a lot of applications. They’re shooting themselves in the foot by competing too aggressively for the limited number of jobs.

[00:09:25] Ioana: So even if more generous unemployment insurance seems like it’s desensitizing people from looking hard for a job, the end effect doesn’t necessarily reduce the number of jobs found, because there are too many unemployed people relative to jobs. My work has shown that in prior situations like COVID. So in a situation like that, we should be far less worried about the disincentive effects of more generous unemployment benefits.

And maybe now I’ll come to the policy — I call it AI Adjustment Insurance. It includes more generous unemployment benefits, meaning they last longer (again modeled on trade adjustment assistance); additional training; and a third component, wage insurance. Wage insurance means that if you find a job with a lower wage than your prior job, the policy covers part of that gap. That actually encourages you to take a new job even if the wage is a little lower — so it’s directly pro-reallocation. The training and wage insurance push reallocation, which counteracts the concern that longer unemployment benefits might discourage job search.

Seth: Maybe one of your sources is Pascual Restrepo’s job-market paper — this idea that during an automation-driven unemployment wave you get a skill mismatch. Lots of people are applying, but they have the wrong skills, and that increases friction in the market. From an efficiency standpoint, it might not be the worst thing if some of the people with out-of-date skills aren’t looking for jobs.

Ioana: Exactly right. That would increase the friction, and this policy, if well-implemented, has the ability to manage that friction better.

[00:11:52] Seth: There are many reasons a person might lose their job. They could be losing it because they’re doing a bad job, or because of macroeconomic things that have nothing to do with AI — or it could literally be AI. Recently Coinbase claimed to fire a bunch of people because of AI, and a cynic might say their stock price was going down and crypto was struggling. Do you care to identify that? Is that an important part of the policy?

Ioana: Of course you need to identify it, and there are going to be inclusion and exclusion errors — it’s not foolproof. Some people who should be eligible won’t be deemed eligible, and vice versa. But I believe we can put in place a process with reasonable accuracy. That was the case with trade adjustment assistance: the company had to certify that the job was lost due to trade. You can question it, but there was a process. In the case of AI, similarly — and I haven’t fully thought this through; if someone wants to do it, let’s do it. People in these offices know a lot about how to do this.

An example: if you’re in an occupation that’s highly exposed, and there’s recently been investment in AI at the firm — buying new software that uses AI to do business services — that might plausibly amount to a layoff due to AI. It won’t be 100% accurate. You could do it at a micro level, trying to figure out whether this particular job got automated, or you could do a macro counterfactual simulation — in a different universe there would have been 30,000 more taxi-driver jobs, so you attribute some percentage of that to your loss. But you can’t do that if you need to decide right now whether this person gets the service. That’s interesting from a research perspective, but operationally we’d have to determine eligibility — maybe just being in an exposed occupation, though that might be too crude.

Did Trade Adjustment Assistance Work? [14:45 – 19:28]

[00:14:45] Seth: Before we move to your other policy idea — this trade adjustment policy that was supposed to get big societal buy-in for free trade was a glowing success, right? Everybody loves free trade...

Ioana: The policy didn’t do well because it was underfunded. The amount of funding is a strict cap decided by Congress, so they just couldn’t spend more. The number of people who received it at all is very small relative to the number exposed to trade. However, for those who did receive it — and remember, I’d argue it wasn’t enough — it worked well. There’s a really cool paper looking at the wage-insurance dimension.

So: I lost my job due to trade, I take a new job that pays less, and the wage insurance covers part of that gap for two years. What happened, which is fascinating, is that this component helped people return to work quicker — and it was fiscally net beneficial from the government’s point of view, because they returned to jobs that were no lower-paid than they’d otherwise have taken, started paying payroll taxes again, and essentially repaid their own subsidy over time. So at least based on that experience, it’s a highly effective way to support people through the transition. While the scale of TAA was too small, for those who got it, they benefited a good bit, and it was effective even from a fiscal perspective.

Seth: If TAA was more generous, would Hillary Clinton be president?

Ioana: Who knows? But I’m going to push back on the argument. Earlier you were making the case that with unemployment insurance, moral hazard isn’t an issue because in general equilibrium there aren’t enough jobs. But if we expanded the size of TAA, the general-equilibrium effects could also swamp the benefits — maybe only a small share of those people could effectively have found jobs, and if you gave the benefits to many more of them, they wouldn’t be able to take advantage.

Seth: You mean because the fiscal cost would become significant?

Ioana: No — they wouldn’t be able to find the jobs.

Seth: Right. If a small number of people get wage insurance and there are some jobs they can take, they take them. But if you give wage insurance to everyone, many wouldn’t be able to find jobs — or they’d cannibalize jobs from people who would have gotten them anyway. Say I worked at a factory, and now I decide to become a ski instructor, and you give me wage insurance for that. There have to be equilibrium effects.

Ioana: This is close to my heart, because I’m a big skier. For sure this increases competition for jobs wherever people decide to go. But from a micro perspective, wage insurance helps because people are now willing to expand to jobs that pay a little less. And mind you, it’s only for two years, so you need some commitment that the job is reasonable — and by that time you can increase your wage through returns to experience.

It’s similar to research on job-search assistance: you put people on benefits and help them — or even require them — to apply to more jobs. What the research shows is that if you do that for a lot of people within a given labor market, it stops being effective, because they’re trampling on each other’s toes.

Will the Jobs Come Back? The Lump-of-Labor Fallacy [19:28 – 22:09]

[00:19:28] Andrey: Is there an underlying assumption in your proposal that the occupations people move into won’t go away within those two years? This is one of the big challenges — we have enormous uncertainty about exactly which labor markets are going to be affected negatively, and maybe some positively. What do we do about that?

Ioana: I don’t think there’s any guarantee that in two years the places they go will be safe. But if we still have the policy and AI continues to provoke churn, they still have it to rely on and can find another job. Also — and this is less true for non-economists, but a lot of economists imagine there’s a fixed number of jobs, so if we lose a lot, there are only so few left and everyone’s fighting over them. That’s not how it works. Everything is connected, and especially if the technology improves production, there are positive spillover effects that make other jobs more productive. So there will also be a lot of job creation.

At the same time, we don’t know exactly what those jobs will be, and it could take several rounds of adjustment. I don’t want to rule out that it could be very bad, but I don’t think a massive net loss in the number of jobs is the most likely scenario. It could be very bad in the sense that a lot of people have to change jobs, which is difficult. But as someone who’s prudent and wants to be flexible — that’s what my second policy is designed to address. What if a lot of jobs are lost forever? Then we need something to fall back on.

The Digital Dividend [22:09 – 26:00]

[00:22:09] Andrey: Do you want to tell us about that one?

Ioana: The second policy addresses a situation where there’s durable mass unemployment — not just people needing to find a different job while other sectors grow, but a lot of jobs disappearing forever, not replaced by new ones, so some people are structurally, permanently unemployed. What do we do with them? This is especially important in the US, where most of the social safety net depends on you either having a job or looking for one. Even food stamps: as a so-called able-bodied adult, you can’t get food stamps unless you have a job or are looking. If there are just no jobs for a large number of people, there’s not much to fall back on.

So that’s what policy B — the Digital Dividend — is meant to address. The idea is a cash transfer that’s unconditional and broad-based. In the specific proposal I give it to everyone, but you could make it broad-based so a lot of people benefit. You might fund it with a tax on the digital sector. I don’t want to tax AI specifically, but all sectors that can immediately benefit from it — a broad-based tax, so it’s harder to avoid.

Seth: A profit tax? A consumption tax? An income tax?

Ioana: I was thinking a sales tax, just to make it easier — but this is something we can talk about.

Andrey: No sales tax on GPUs, or...?

Ioana: Just a sales tax on all digital companies. We can talk about other options — this is a beginning. The point is it would be very small, and the broader you make it, the smaller it can be while still creating revenue. You’d invest it in a fund, and the returns come back to people as cash.

Why this structure? The tax side can slightly slow the diffusion of the technology — and there are theory papers showing that if there are labor-market frictions and credit constraints, reallocation is painful for workers, so it can be optimal to slow diffusion a little. We’re not banning anything. And the revenue lets you pay people the cash benefit if we end up in the no-jobs situation. This policy can be expanded — that’s the flexibility. You can start very small, almost zero tax, but if we have mass unemployment you scale it up and grow the base to the whole economy.

Can We Afford UBI? [26:00 – 34:23]

[00:26:00] Andrey: All of us have had conversations with technologists who jump straight to UBI as the solution to all issues with AI, and this is one version of it. What I always tell them is that we can’t afford it — and we deeply can’t afford it. There’s some future where productivity gains are so large that the numbers pencil out, but I’ve yet to see a tractable, plausible version. If we did it today, the amount per person would be trivial. And for UBI to truly work — as something that lets people not work — it needs to be a massive transfer. It can’t be even a thousand dollars a month.

Ioana: I hear you. But where the technologists are consistent with themselves is that they often think AI is going to revolutionize productivity. If that’s true, then it will be possible to have a reasonably high UBI. And that’s the whole point of my proposal — it’s conditional, we scale it up as needed. I could even envision it not being completely universal, but it should be broad-based, so people have an income to fall back on if technology is deleting a lot of jobs.

Andrey: Let’s say a plausible scenario that a lot of economists believe: AI increases per-capita GDP growth by about 0.5 percentage points per year relative to baseline. We’re five years down the road, and in the limit it’ll be big enough to support everyone — but we’re not there yet, and a lot of people are out of jobs by that point. We might not get the super-productive world until well after we have economic displacement massive enough that wage subsidies won’t work.

Seth: And we need that 0.5 percentage points just to deal with the current deficits — we’re already counting on it. But here’s what’s natural to me: you start today and make plausible projections — what’s a world with another percentage point of GDP growth a year worth, what’s a world with two? — and you set aside a fraction of that in advance for a UBI or digital dividend. You make the policy now, rather than after the crazy thing happens. It has that automatic logic I really like.

Ioana: That’s exactly the point. I’m a little concerned that after a lot of people lose their jobs and the situation looks grim, it might be more difficult to say, “Now let’s have a big reshuffling of money to help these people,” especially when some people have made a lot of money. Whereas if we can commit more ex ante to putting money aside, it’s a bit of a veil-of-ignorance situation — we don’t know for sure who the winners and losers will be. So it can be socially easier to agree to put a parachute in place now, before you know whether you’re a winner or a loser. It’s a political-economy argument, but I think it’s important, because I really worry we get there without enough to support people, and the winners say, “Ah, too bad.”

[00:30:47] Seth: My questions are more on the tax side than the spending side. We’ve seen many efforts to tax digital goods, and they’ve had a lot of problems. Where was a Netflix video watched? Where was an ad viewed? People use VPNs; these companies have no physical locations and move easily. How convinced are you that we could actually raise significant revenue from a digital tax when my VPN is in the Cayman Islands, so I’m not paying a sales tax in America?

Ioana: That definitely needs to be worked out. With tax policy you always have to think about incidence and evasion — not necessarily illegal evasion, just ways around it. I haven’t done a detailed implementation calculation, but it’s probably feasible to find a version that works. You won’t eliminate evasion — that’s always true with taxes — but you want to think ahead of the incentives. That’s why we’re economists. It’s not like you slap on a tax and the money comes in; there are behavioral adjustments you need to foresee with a coherent design. In principle, it should be possible to raise a good chunk of money if we wanted to.

Seth: Related to that — it’s not crazy to think all the rents go to energy producers or even landowners of energy resources. And if we’re taxing them, we might disincentivize energy production, which raises the cost of living. Land-value tax solves all problems forever, of course — I do like land-value taxation.

Ioana: In the long run you want to tax the inelastic input, and land is the ultimate inelastic input — that’s something to think about in the long run. But the reason a digital tax can make sense in the short run is this idea of a small, moderate slowdown — not massive — that has the benefit of accumulating some capital to help people with later. As I said in my piece, I’d definitely expand the tax base at some later point, once we think the transition has happened.

The Motte-and-Bailey Worry [34:23 – 36:05]

[00:34:23] Seth: Are you worried about a motte-and-bailey situation? Your proposal is very modest, but I could see politicians using that logic to implement a massive tax-and-transfer scheme today under the pretense that it’s about the AI future — and I really worry we can’t afford it.

Ioana: That gets into the social welfare function — you could call it politics, or simply what we want — and as economists, it’s not really our job; as citizens we can have views. Different politicians legitimately have different ideas about what’s important. Within my piece, I was proposing the digital dividend as a solution to AI unemployment, but there are other reasons to like UBI. I’ve written about UBI before and discussed some of them. So maybe you also like UBI for those reasons — that’s a tenable view, and it’s also fine to disagree. There could be a push to go big right now, and if that’s ultimately what people want and they convince the rest of us, that’s just how the democratic process works.

Intelligence Saturation: The Model [36:05 – 42:23]

[00:36:05] Seth: Maybe this is a good time to transition into the new macroeconomic model you came out with, which is informing your beliefs about how radical the changes might be — “(Artificial) Intelligence Saturation and the Future of Work.” I love a title with parentheses in it. Just to lay it out for listeners: it’s a very neoclassical way of thinking about automation, but with new twists — a nested constant-elasticity-of-substitution model with an intelligence sector and a physical sector. Why is it important to think about an intelligence sector and a physical sector as complementary, rather than one thing?

Ioana: It’s important to distinguish them because of an empirical fact — it’s not yet in the paper, but I’ll add it in the next version. If you look across occupations at which were most teleworked during COVID versus their exposure to AI — the Eloundou et al. measure of LLM exposure — there’s a very strong correlation. The more an occupation was teleworked during COVID, the higher its exposure to AI, and conversely. By “physical” I mean an in-person job, where you need a physical human body. That doesn’t necessarily mean working with your hands — teaching in person is physical in my definition but not manual. It’s just an in-person job.

Seth: I was curious about those examples, because you gave the example of an in-person lawyer giving oral arguments — but didn’t we do those online during COVID?

Ioana: Some of it we did online, but for the economics, the important thing is how substitutable these things are. Online teaching is its own thing — it has a place and a function, but it’s not the same as in-person teaching. They’re differentiated products; you can’t easily replace one with the other. So based on the fact that AI exposure and the ability to do remote work are highly correlated, that justifies the distinction between physical and intelligence. There are also fundamental limitations of the physical world that are much more stringent than the limitations you meet scaling the virtual world. It’s much more difficult to expand human bodies and physical capital — that’s very slow — whereas you can scale up AI incredibly fast. It’s still not costless — data centers and so on — but you can do a lot, really quickly. This distinction is critical to understanding how AI could affect the economy.

[00:40:14] Seth: So you’ve got an intelligence sector growing really fast and a physical sector that maybe doesn’t grow as fast. Let’s roll with the assumption that the two are gross complements — you need both to have a lot of output; you can’t just have the peanut butter or the jelly. What are the conclusions of the model?

Ioana: The core conclusion is that if physical and intelligence are complements, then AI can grow the intelligence side incredibly — to infinite intelligence — but as long as the physical sector stays fixed (or grows much slower), the impact of growing AI saturates on both output and wages. By saturating, I mean output goes to a finite limit, a ceiling. Even with infinite intelligence and infinite AI, output is strictly bounded. Wages increase too, because they go together with output, but they hit a ceiling. That’s what we call intelligence saturation.

This is super important because a lot of technologists see the progress of AI and imagine the whole economy could expand at a similar rate. This makes the strong point that here’s a scenario I think is quite plausible: you could expand like crazy in AI and still only hit a ceiling in output.

Robots vs. LLMs [42:23 – 48:52]

[00:42:23] Andrey: I understand the thought experiment, but saying intelligence goes to infinity while the physical is a constraint is a little strange — it’s pretty clear that with enough intelligence we’ll figure out how to make robots work, even self-replicating, learning systems.

Seth: There are two different parameters in the model, right? You’ve got the output of the intelligence sector, and then the automatability of the physical sector. So Andrey’s intuition is: if we had a gazillion intelligence, don’t we fully automate the physical sector?

Ioana: I want to distinguish two things. One is whether humans can be replaced with robots. Robots are improving, but relatively slowly compared to AI. So comparatively, it’s much more advantageous to replace people on the AI-replaceable side than in applications where you need to be there in person. That’s not to say there’s no progress in robotics.

Andrey: I’m just representing the technologist viewpoint — that this is true for now.

Ioana: The argument I’m going to make is based on history — the history of technology specifically, and you can think history won’t be the same. We’ve had physical robots for the longest time. With the Industrial Revolution we created many more and improved them a lot, and even before LLMs they improved tremendously with machine learning and semi-autonomous systems. These are very real improvements, and they have replaced some jobs in manufacturing — Daron Acemoglu has papers on that. But it wasn’t like, “wow,” thanks to all that intelligence in the system, it still takes a lot to get there.

I’m totally willing to think this new technology can make them even better — but I’m skeptical it gets a lot, lot, lot better. It’s the saturation argument: there’s a fundamental limit. The cost of robots is fundamental — you have to use materials to make them and maintain them. We’ve tried for centuries to improve robots, and they have improved, but I don’t know how much more you can improve them with this new technology.

Andrey: A humanoid robot that’s smarter than a human seems like a pretty big improvement that’s plausible.

Seth: 15% chance, according to a recent survey of economists and AI researchers. By 2030.

Ioana: The question is partially the cost — not that it’s technically impossible, but what’s the cost of the whole thing relative to a human. At least in the medium run, I think it’s not very plausible you’ll bring that cost down a great deal. Also, a lot of these robots aren’t very versatile, unlike AI. That’s the cool thing with AI — it’s super general-purpose; it can use all the tools we had before. But most industrial robots are bespoke, meant for a particular application — that’s how you make them cheap and effective.

I feel somewhat confident that in the short-to-medium run it will be very difficult to make it cost-effective to have robots replace people in most jobs. Harder to tell the further out you go. However, I believe everything you could do on a computer could get automated in the medium term — possibly even the research I’m doing. That’s a different matter, because it’s all based on computers.

Andrey: Even if what I’m saying is true, the Baumol-style logic would still hold, right? If we still want human teachers even when robots are available...

Seth: Maybe don’t call it the physical sector — call it the humanness sector.

Ioana: You could relabel it. I think “physical” is relevant at least in the medium run, but to make it more future-proof, you could relabel the physical sector as the non-automatable sector, whatever that turns out to be. As long as there exists a non-automatable sector, that’s where people will work, and the mechanics of the model are identical.

The Capital-Share Puzzle [48:52 – 50:35]

[00:48:52] Seth: I love playing around with these neoclassical models — automatable part, non-automatable part. I’ve been doing it for a decade, and one challenge that pushes historically in a different direction: if intelligence and physical stuff are gross complements, you’d expect that as you get more intelligent stuff, its share of national income would go down. But over the last 50 years we’ve seen a huge explosion in education in the US, and yet the educated share of income has been going up. So how do I think about it actually looking more like gross substitutes?

Ioana: But we’ve also had the share of capital going up. Part of the last 30 years or so is the ICT revolution, which is somewhat similar to the prior version — really you could say it’s the same thing, just different stages of an AI revolution that’s at an early stage. During that, the share of capital has been going up, and research suggests—

Seth: Is it the share of capital, or the profit share? That’s an important question we’ll come to in a minute.

Inside the DOJ Antitrust Division [50:35 – 58:12]

[00:50:35] Andrey: You did this stint at the DOJ — you were on leave from being a professor. Could you tell us what led you to work there, and a bit about your work?

Ioana: At the time I was doing work on monopsony power in the labor market — the difference between wages and the marginal productivity of labor. The Biden administration commissioned a report on labor-market monopsony, and the people doing that called me up; that’s how I learned about the job. I thought, “That sounds really interesting — to do antitrust enforcement as an economist.” I said yes, and I was lucky enough to get the job.

[00:51:29] Seth: [Sponsor break] For those of you playing along at home, now is your chance to think about how this conversation has changed your priors. This chance to contemplate your posteriors is sponsored by Revelio Labs — a leading provider of labor-economics data and data services for companies, academics, and independent researchers. Revelio combines comprehensive micro-level data on employee profiles, job postings, and sentiment with standardizations, mappings, and enrichments, all to make the data useful without making your modeling decisions for you. It can be aggregated to company, market, or industry, and used to study everything from career trajectories to occupational transformation to the impact of AI on labor demand. Revelio data is available on WRDS — so if you’re an academic with a good library, go see if you have access already. And if not, reach out to their excellent economics team.

[00:52:43] Ioana: It was an incredible experience. What I did there is — I was the principal economist—

Andrey: Can you pause and define monopsony for our listeners?

Ioana: Monopsony power is the idea that employers are able to pay workers less than their marginal productivity. Under perfect competition, the wage exactly equals the marginal productivity of labor — whatever value the worker brings, the company pays them for it. With monopsony power, workers get paid less than what they bring to the company. One of the key reasons is the lack of competition among employers. Intuitively, in the extreme — a literal monopsony, only one employer — that employer doesn’t need to pay much to keep you. Whereas if there are many employers, they bid up the price of labor by competing for you, and in the competitive extreme you get paid your marginal product, because if someone underpays you, a neighboring employer recruits you away.

Andrey: Great — so you can continue.

Ioana: So when I was at the DOJ, I was the principal economist. In the antitrust division, the job is to enforce the antitrust laws, and they do two broad things. One is examining mergers and potentially blocking them if they lead to anti-competitive effects. The other is so-called conduct, which can include criminal conduct like literal collusion—

Seth: Assassinating rival CEOs.

Ioana: A situation kind of like that — which is unbelievable. You think it only happens in the movies, but it happens in real life. Economists don’t get too involved in those cases, because it’s more of a whodunit.

Andrey: No fun. Let us in — we want to be detectives.

Ioana: If you want to look this up, the keyword is transmigrante — a trade in used cars from the US being traded toward Latin America. There was unbelievable murder among the companies involved, around collusion. If you were a traitor to the scheme... yes. Anyway, that’s the monopsony power.

Andrey: That’s one way to get monopsony power.

Ioana: This is where the FBI goes — not really the province of economists. The other category is a bunch of behaviors by firms that hinder competition, the big one being monopolization — trying to remain or become a monopoly by kneecapping rivals. In my role I oversaw the expert analysis group, a team of about 50 PhDs, mostly economists and data scientists. Whenever we had a case against a company, there’d be data gathering and economic analysis to support arguments about why a behavior hinders competition and might, for example, increase prices — or, in a labor-market case, how employers hinder their employees’ ability to find another job, and so can pay them less.

For someone who worked on monopsony, being able to think about how mergers should be blocked if they lead to greater monopsony power was incredibly rewarding. Intuitively, if two employers merge, that reduces competition for workers and can lower wages or degrade non-wage dimensions of jobs. That’s now officially in the merger guidelines — which was unbelievable. How often do you do research and then get to implement the thing?

Favorite Cases, and Antitrust Meets AI [58:12 – 1:09:27]

[00:58:12] Andrey: Is there a monopsony case you worked on that you’re particularly excited about?

Ioana: There were a number. They’re described in papers we write every year reporting on finished cases, in the Review of Industrial Organization. One — I only caught the tail end — was a publisher merger. Big publishers were trying to merge, and the argument was that if they merged, authors trying to sell their books would get lower payments. The judge found it very convincing. We even had Stephen King testify about how the merger would reduce what he could get paid.

Andrey: He couldn’t afford all the cocaine he needed.

Ioana: Ultimately the merger was blocked, and they decided not to appeal. That was the first merger in the US blocked exclusively on a labor theory of harm — that it would lead to lower payment for authors.

The other is an interesting case around small farmers who raise chickens. They work for a processor as subcontractors — small farmers, not workers, but worker-like. They raise chicken and sell it to a big integrator. There was a contractual term that if you left to work for a different company, you’d have to pay a big chunk of cash to leave. We argued this significantly restricted competition for these farmers’ services and lowered their pay, because it’s hard to leave if you have to pay to do so. We won in the sense that there was a settlement — the company said, “Fine, we’re not doing it anymore.”

I did a lot of work on agriculture, because farming is often an area with few opportunities to sell labor or goods, so monopsony is prevalent there. This tells listeners that monopsony — whether there’s competition among buyers — isn’t just about workers. Workers are a big application, but it can also be a more B2B situation, where many small businesses or independent contractors sell to big buyers with market power.

[01:01:30] Andrey: Shifting slightly — something people are beginning to think about is antitrust and AI. Have you thought about that? Do you have opinions?

Ioana: It’s really important to stay vigilant in AI and antitrust. We’ve had prior tech giants the government has gone after — Microsoft, Google — and in these industries there can be an opportunity to monopolize. We’re not there right now, but that’s why we have watchdogs like the Antitrust Division. There’s been a lot of partnership and financing deals, which might ultimately lead to consolidation, and that should be watched in the ordinary course of antitrust enforcement.

Why does this matter? We want to maintain low prices and high-quality services for consumers and businesses — and a big part of AI is used by businesses. If you want this technology to lead to greater productivity through adoption, you want to keep it cheap and good. What usually happens with consolidation is the product gets worse and prices get higher than they’d be in a more competitive industry. It’s natural for companies to try to monopolize — that’s why we have the Antitrust Division and the FTC, and also so that companies considering certain steps recognize some might not be lawful and stay away from them, preserving competition.

Seth: That’s generally the argument, but sometimes we have natural monopolies. Some argue these big foundation-model builders — OpenAI, Anthropic — pouring giant amounts into training runs might be natural monopolies. Maybe we just want one company doing the one giant training run, and the right way to regulate isn’t competition policy but tax policy or some other government control. What do you think?

Ioana: It could be, but it’s not clear yet, because there are still many foundation models, including outside the US — the data is out there. You need data and power to train, but you can do it multiple times if you have the resources. There’s also a difference: in the US there’s a big focus on the biggest, fastest foundation models, but in places like China it’s much more focused on applications, and there you see a lot of competition. Some company might want to have it all — “Why don’t I have all the applications?” — and that’s why we have the antitrust authority to stop that. In some situations a utilities-type regulation can make sense, but I think we’re not there yet. For now I’d take the position that we should promote competition, and we’ll see where the dust settles. If you prematurely favor monopoly, that can actually hinder the development of the technology. So I’ll err on the side of competition.

Andrey: One interesting thing here: if you’re the person who thinks we should slow things down, then you should be rooting for more market power. Leopold famously argued the technology is so powerful we’re going to have to nationalize it — which again goes to an argument for more market power rather than less. I’m not saying I support this, just throwing it out as a slightly unusual difference from most industries.

Seth: Energy might be similar — or nuclear power would be a different analogy.

Ioana: The thing is, you can still use a model from elsewhere — whereas with energy there are huge costs of transmission lines, so it’s more limited geographically. And let me make a point related to my saturation paper that’s highly relevant. Assume there are decreasing returns — you add more intelligence, it’s helpful, but less and less helpful at the margin. If that’s the case, then the whole competition between countries is less—

Seth: Then America loses, because China is better at physical stuff.

Ioana: The point is rather that being first is not that important if you have diminishing returns. It’s important, but less so, because if you’re second, you’re just a little bit worse and can still do a lot of things almost as well. Whereas if it’s “the singularity, boom,” and then you’re far ahead of everyone, being there first really matters. So whether you think you’ll reach a point of explosion versus diminishing returns completely changes how you think about competition between countries — and even between different models.

Andrey: There’s a subtle point: you could have diminishing returns, but the nature of military conflict is a contest — you just need the max. So it’s very different from the economy.

Ioana: I feel less of an expert on military. I was talking more about economic might — if there are diminishing returns, it’s nice to be first, but it’s only—

Seth: Let’s talk about economic might, because your argument is even stronger than the one you’re making. In a universe where American innovations in AI spill over into Chinese innovation — which they do, with distillation and publicly written papers — if China has the advantage in the physical, we’d want a world less constrained by the physical. We’d want slower progress.

Ioana: That gets to things we discuss in our paper — how much substitutability there is between physical and intelligence from the workers’ point of view. The paper is about what happens to workers and equilibrium wages during automation versus after. During automation — assume you’re automating all intelligence tasks — low substitutability is a form of insurance for workers; it avoids some of the worst wage outcomes, especially at high levels of automation. But after, once we all work in the physical sector, more substitutability promotes higher wages and growth in the very long run. So the game is different depending on whether you’re in the short run, during automation, or after.

Seth: In the very long run, you want an AK economy.

Ioana: Exactly. In the long run it’s good, but in the short run it could be better not to have it, in terms of avoiding a wage decline.

Lightning Round [1:10:59 – 1:19:45]

[01:10:59] Seth: Lightning round. What’s the meaning of life? In your discussion of a Betsey Stevenson paper at a recent NBER session, you said that after automation takes all our jobs, something called ikigai will be more important. What is that?

Ioana: It’s the idea of having a sense of the inherent meaning of your everyday activities. This is something Betsey Stevenson proposed, and I was commenting on it and thinking about examples from philosophy. Ikigai is a Japanese concept, but we have other examples in Western philosophy — the French writer Camus, and the myth of Sisyphus. You imagine Sisyphus pushing a boulder up the hill; it rolls down, and he pushes it up again. It seems pointless. However, the myth says you have to imagine Sisyphus happy: he finds satisfaction in the repetition and transcends the fact that it’s repetitive, making sense of his life by becoming absorbed in it and seeing his freedom in embracing it. It’s a very inspiring way of thinking, because in the current regime we’re so obsessed — especially in economics — with making more stuff, versus paying attention to what we already have.

Seth: But then you immediately economist-brain it, because you have this amazing quote: “If there are no market-like mechanisms to encourage people to pursue their ikigai, just as wages incentivize people to work, a world without transformative AI”— sorry, a world with transformative AI but without work could undermine wellbeing. So how do we incentivize ikigai?

Ioana: That’s something I want to work on — so if anybody’s listening and wants to embark on this quest, I’m all for it. Maybe you’ve read about the epidemic of loneliness. Why aren’t people getting out and doing social activities? As economists, we think in terms of cost — there must be friction costs, coordination costs. The question is how you engineer a world where those costs are lower and people actually get out, meet friends, do their gardening or their rock-pushing, instead of sitting there contemplating how they’re doing.

Andrey: You climb the rocks rather than push them.

Seth: We’ve got a rock climber in the room. The social planner will assign you the rock, and you will experience ikigai pushing the rock.

Ioana: No, no — it’s going to be a market-like mechanism.

[01:14:13] Seth: Do you want to say anything about your work on the Anthropic Economic Advisory Board?

Ioana: I’m a member of Anthropic’s economic advisory board, advising them on the economic impact of AI. As you probably know, Anthropic releases data products publicly that measure how Claude is being used. Part of my role is to give feedback on what data would be helpful to release, what checks to do, how to show people what the data means and what its representativeness looks like. It’s been exciting to collaborate with one of the biggest AI companies and play this advisory role.

Andrey: And we’ve covered the Economic Index on this podcast — we had an entire episode about it.

Ioana: Oh, really? Nice.

Seth: What are you working on these days? What can we expect next?

Ioana: Right now I’m working on a project, back to monopsony — monopsony and industrial policy. Industrial policy is fashionable right now. If you subsidize a sector — the government pays to create more jobs there — one effect is that it raises wages in the other sector that competes for those workers. You increase employment here, and wages increase there. We want to demonstrate under what conditions you get a bigger or smaller spillover, and therefore why industrial policy can sometimes be justified through this argument — you’re paying to get more competition for jobs.

Seth: Although there would be a negative spillover from the taxes or regulation needed to support it.

Ioana: Absolutely. You put some cost in for the industrial policy, and one benefit is increasing wages in the non-subsidized sector. It’s also a way to redistribute between wages and profits — you decrease profits and increase wages in the non-subsidized sector. And profits are pretty hard to tax, so it’s an interesting instrument. We’re developing the theory — how much monopsony power yields what optimal size of subsidized sector — and thinking about applications. The big-picture point is that the public has often lost confidence in the government’s ability to redistribute effectively through tax-and-transfer. So if we can provide more jobs while also increasing wages in the other sector, that could be an interesting policy instrument — one that comes with its own costs, but worth understanding better.

Andrey: Final question: who’s your favorite philosopher?

Seth: And we love the way you pronounce Camus, so say it with that beautiful accent.

Ioana: Who’s my favorite philosopher? That’s surprisingly difficult. I might go with Rawls — John Rawls.

Seth: Beloved of liberals.

Ioana: He’s done incredible work, because he incorporated considerations from utilitarianism. Whether you agree with his take or not, he clarified a lot about the different theoretical frameworks for thinking about social justice. When I was growing up intellectually, it was very helpful. Actually, I learned about utilitarianism first — I read Mill. Oh, I love John Stuart Mill. Maybe he’s my favorite, actually — because the writing is amazing, and he has such a nuanced view of the world. His book on utilitarianism is amazing. They speak very nicely to each other — so maybe I have to say John Stuart Mill, which is fitting for an economist.

Andrey: Tyler Cowen thinks he’s the best economist ever. So you’re in good company.

[01:19:45] Seth: It’s been an absolute pleasure to have you on the podcast — I had so much fun.

Ioana: Thanks so much. It was great.

Andrey: This was awesome. Thank you.

Seth: All right, everyone out there — please like, share, subscribe, and keep your posteriors justified.

Kevin recently published a multi-book review of the economics of AI in the Journal of Economic Literature, and that’s where we start. Along the way we get into the gap between AI’s technical capability and its actual diffusion, the stages of how organizations adopt new technology, why the binding constraint on AI value is organizational integration (not prediction vs. judgment), what an AI-for-science research agenda should look like, the coffee test and the fence-post test, what forecasting surveys reveal about how economists and lab researchers actually differ, a dispatch from Kevin’s recent trip to China (spoiler: they are not AGI-pilled), the future of the academic paper, and a lightning round on comparative advantage in the age of AI.

A wide-ranging, opinionated, very fun conversation. Grab your Chinese peptides and settle in.

Links & References

Kevin’s work

• Kevin Bryan, “The Economic Impacts of Artificial Intelligence: A Multidisciplinary, Multi-book Review” — Journal of Economic Literature, 64(1), 2026.

• A Fine Theorem — Kevin’s research blog

• All Day TA — turn course content into a custom AI teaching assistant

• Creative Destruction Lab — the accelerator Kevin helps run (first AI accelerator in the world, 2016)

Books & essays discussed

• Leopold Aschenbrenner, Situational Awareness — the essay Kevin gives all his students (”read chapter one, believe chapter one”)

• Erik Brynjolfsson & Andrew McAfee, The Second Machine Age

• Ajay Agrawal, Joshua Gans & Avi Goldfarb, Prediction Machines and the follow-up Power and Prediction

• Joel Mokyr, The Gifts of Athena and A Culture of Growth — Kevin’s PhD advisor, “the Michael Jordan of progress world”

People & projects mentioned

• The Unjournal and Works in Progress — models for the “new journal”

• Chad Jones, Stanford GSB — growth theorist read seriously by people in industry

• Phil Trammell, GPI / Oxford — “Phil World,” the rapid-growth scenario

• The coffee test (attributed to Steve Wozniak) and Kevin’s own fence-post test as benchmarks for embodied AGI

Previously on Justified Posteriors

• Avi Goldfarb — Prediction Machines, O-Ring Tasks, and How AI is Reshaping Economics

• Alex Imas — Demand Collapse, Bargaining with Machines, and Behavioral AI Economics

Our sponsor

• This episode is brought to you by Revelio Labs, the leading provider of labor-economics data, available to academics on WRDS.

Chapters

• (00:00) Intro & sponsor

• (00:39) The JEL book review: what the economics-of-AI canon got right — and what the older books still beat the new ones on

• (03:19) Prediction vs. judgment, and the real bottleneck: organizational integration

• (05:52) Too pessimistic on the tech, too optimistic on diffusion — Waymo, Pearl Street, and the COVID vaccine

• (12:34) The four stages of how organizations actually adopt a new technology

• (15:42) Status-quo bias, banning Anthropic, and treating frontier AI like nuclear material

• (20:16) Why Situational Awareness beat the economists, and the book Kevin actually wants: AI for science

• (26:53) Forecasting AI: the surveys, and where economists and lab researchers do (and don’t) diverge

• (28:20) Benchmarks, the coffee test, and the fence-post test

• (35:53) Rapid-growth scenarios, labor-force participation, and “Phil World”

• (41:40) Scaling regularities: what economists should defer to technologists on — and what they shouldn’t

• (43:34) Why forecasts matter for policy and capital allocation

• (45:50) Dispatch from China: not AGI-pilled, “involution,” broken capital markets, EVs and self-driving

• (1:01:40) War, nationalization, the end of open source — and why everyone in China uses Claude

• (1:06:06) A Fine Theorem, the economics of blogging, and the rising value of taste

• (1:17:48) The economist as plumber: comparative advantage, RCTs, and what grad students should do

• (1:24:07) What the academic paper looks like in two years

• (1:28:22) San Francisco, ambition, and the permission structure for growth

• (1:32:56) Lightning round: favorite economists, All Day TA, and advice for econ grad students

Open & Intro [00:00 - 00:39]

[00:00:12] Seth: Welcome to the Justified Posteriors Podcast, the podcast that updates beliefs about the economics of AI and technology. I’m Seth Benzell, finally able to meet one of my theoretical heroes, coming to you from Chapman University in sunny Southern California.

Andrey: And I’m Andrey Fradkin, coming to you from San Francisco. Excited to have Kevin Bryan as our guest today. Kevin, welcome.

Kevin: Thanks for having me. Very excited.

Andrey: Kevin is a leading thinker in the field of progress, and in AI economics. He also has his own startup, All Day TA, and is prolific on Twitter — at times.

Kevin: At times.

The JEL Book Review: What the AI-Econ Canon Got Right [00:39 - 03:19]

Andrey: Kevin, you wrote an article reviewing several prominent books on AI. Why did you do this, and what did you learn from the exercise?

[00:01:13] Kevin: It’s pretty interesting. Economics of AI is not that new of a field — some of the canonical books on how economics thinks about AI go back to before large language models existed. Books like The Second Machine Age by Brynjolfsson and McAfee, and Prediction Machines by Agrawal, Gans, and Goldfarb. These are pre-LLM — written before the attention paper. So it’s interesting to look at what of the core ideas in the economics of AI have changed given the technological improvements.

On the technology side, I don’t think there have been massive surprises for people who were paying attention. At least since the scaling law paper, if you’d drawn the line on the graph, you’d have more or less predicted everything that happened. I remember reading Kurzweil — The Age of Intelligent Machines, The Age of Spiritual Machines — back in college, and those are just drawing different lines on the graph, in that case based on compute, and we’re getting very close to what actually happened.

Likewise on the economic side: given that the technological trajectory hasn’t changed much, I don’t think the underlying economics has changed as much as people might think. Where things might be bottlenecked, how technology improvements map into growth, the effects on labor markets — the fundamental microeconomics of AI’s predictions hold up pretty well. I found it interesting how few of the 2023, 2024, 2025 books had really advanced my understanding of the economics of AI compared to the older ones.

Prediction vs. Judgment, and the Real Bottleneck [03:19 - 05:52]

[00:03:19] Seth: Lots to unpack. We just had Avi Goldfarb on the podcast and pressed him on his Prediction Machines approach, where he distinguishes the AI that’s good at predicting from the human that’s good at judging. If any of these books would have changed after gen AI, it’d be that one. Don’t you think that book maybe gets something wrong?

Kevin: I think they’d agree — they wrote a follow-up in Power and Prediction. But the disagreement isn’t about the prediction-versus-judgment distinction. Even in the original book — and I remember talking to them about this in 2016, 2017 — judgment is a sliding scale. Take the umbrella example: I know my utility function on an umbrella, I know how much I dislike rain. I give the AI data, it looks at my face, sees light rain, heavy rain, and it can predict my utility function — in which case judgment is taken over by AI. Everyone understands that.

That said, on the scale of how easy it is to figure out the underlying utility function from data versus the predictions that go into it, I don’t think that’s changed. None of the major language models technologically can — or even attempt to — modify how they operate for me versus you. They store a little memory and RAG their way into remembering what you’re like, but there’s no attempt to fine-tune the model. We’d like to use continual learning, but we can’t yet. So the judgment aspect is still pretty binding even today.

Where I think there’s a difference — and where Ajay, Avi, and Josh would say they were wrong — is that the fundamental problem for AI’s creation of value isn’t prediction versus judgment. It’s the organizational integration problem. There’s overlap between the two, but we’d take the organizational and architectural bottlenecks more seriously now, partly because we’re applying AI to more complex tasks where those bottlenecks start to bite.

Too Pessimistic on Tech, Too Optimistic on Diffusion [05:52 - 12:34]

[00:05:52] Seth: You point this out with The Second Machine Age — Andy and Eric’s world-historical automated car ride.

Andrey: It’s weird to think that in some ways they’re a little too pessimistic about the technology, but a little too optimistic about social diffusion. The driverless cars going down the highway in California are a perfect example.

Kevin: Such a good example. We all talk to different audiences. When I talk to policy people, I tell them: “Whatever you think the capabilities of AI will be in the future — more than that.” This isn’t a sales pitch. Every single person inside the lab agrees. You have people high up in government who think about AI as the AI of today plus epsilon. And you want to ask: what did you see in the past 10 years that makes you think this is a good way to plan for the future?

[00:07:01] On the other hand, out in California they wildly underrate diffusion friction. I give the Waymo example: if diffusion is so easy, how come we rode in a Waymo 10 years ago? I’m in Toronto — Jeff Hinton’s city — and there’s not a single one. Clearly there’s some friction.

I remember a couple of years ago, Andrey was with us at one of the labs with a few other economists. They brought in a bunch of computer scientists and asked, “What’s the effect on GDP productivity in the short run?” And we said, “Through 2030, maybe 1% per year.” Actually we said less. And to be fair, from 2024 to 2026, we’ve been right so far. They said, “But why?” And we said, “We agree with you technologically.” At the time we saw technology that hadn’t come out yet that everyone now thinks is amazing. But on the production side you’ve got bottlenecks — you’re combining complements in some CES or Cobb-Douglas function, and you don’t need many bottlenecks for growth to stall quickly. Then there are social diffusion factors, regulatory factors, organizational architecture factors, like in Kim Clark and Rebecca Henderson’s work. Add all these up across every technology ever, and I just don’t see fast takeoff.

Honestly, I think it’s bad. I think we’ll be able to make personalized medicine very cheaply much more quickly than regulators will allow you to sell it. That’s a problem.

Andrey: The standard retort is one of two things. One: it’ll be so self-evidently good that people will find a way to take it — like a cure for cancer. There are already sub-treatments where rich people take them without FDA approval and claim they work. Two: we have autonomous zones where we let AI do whatever it wants, and they out-produce the rest of society.

Kevin: Great arguments both. Out here in San Francisco you’ve probably got a bunch of Chinese peptides in your fridge.

[00:10:20] But here’s the thing — the Chinese peptides are self-evidently really good at the single most costly part of the medical system. And yet can you legally buy them? Anywhere? You can’t even legally buy them in China. That self-evidence did not change the regulatory process. We just went through COVID. We had the vaccine in January 2020. Everything from then until it diffused was government. What more important thing to diffuse quickly could there be? Waymos — you sit in one for one minute and it’s obvious it’s driving more safely.

Andrey: People start crying. It’s so beautiful.

Kevin: And yet. We saw this historically too. If you want self-evidently useful: Pearl Street, 1882, Edison flips the switch — let there be light. And yet look at the diffusion rate for electric street lighting, especially after Chicago burns down.

The Four Stages of Organizational Adoption [12:34 - 15:42]

[00:11:49] Andrey: Let me retort. Diffusion of technology has accelerated over time. The smartphone diffused extraordinarily quickly. Aspects of AI have diffused even quicker — by standard adoption measures, almost everyone has used AI at least once.

Seth: And how much do regulations matter for the diffusion rate? If you ask my students, regulation against using AI hasn’t slowed their adoption at all.

[00:12:34] Kevin: We have an answer to this — it’s actually easy. This is why I like starting with a little organizational economics. Every technology I’m aware of, ever: first we adopt it when individuals can do existing tasks with the new thing more efficiently. That’s easy — that’s your student cheating on their exam, the coder using it for coding, me brainstorming with GPT to prep for a meeting.

The next step is a group or an organization doing an existing task using the new technology. That’s tougher. To give you a sense — how many organizations have changed their IT procurement policies given the change in how we make software? Find one.

The third stage: a new task that’s now efficient given the new technology, inside my organization. I haven’t run into a single large incumbent organization that has reached this level for anything important. The fourth is the hard one: a new task that’s only efficient because of the new technology, and that requires something on the outside — partners in the supply chain, regulators, someone to change. That’s Waymo. That’s containerized shipping. That’s UPC codes. The fundamental barrier there isn’t information or firm growth rates. It’s that the institutions are built around existing skills, promotion policies, and so on.

If AI made the optimal university 50% capital and 50% labor — bringing people in and out instead of having a tenure system — what year do you think we get that? You can maybe out-compete the incumbents, but I don’t think Harvard’s reputation goes away that quickly. And if we’re talking about governments, you can’t even out-compete them.

Status-Quo Bias, Banning Anthropic, AI as Nuclear Material [15:42 - 20:16]

[00:15:03] Seth: Sometimes you can out-compete governments — it depends how crazy a takeoff we’re talking about. I talk to Phil Trammell about scenarios where the world is too decadent and we don’t save enough, so we don’t get growth from AI. His comeback is always: then one country will accumulate and overwhelm all the others eventually.

Andrey: There’s also a timeline over which universities get out-competed — maybe not Harvard. Harvard’s a luxury good, and luxury goods have different economics. But if the ROI to college falls drastically, I don’t see college education remaining anything other than a luxury or niche thing.

Kevin: We see this in the X-inefficiency papers, or the steel mini-mill papers — quicker organizational change in response to existential threats. It’s almost worse when the organization has rents to share, because then who wants to be the manager who’s the jackass firing people? My favorite example: Blockbuster could have bought Netflix for tens of billions. If they had, you’d have never heard of Netflix — I don’t think the transition to streaming happens if retail-location experts are running it, and the relational contracts with the studios have to change wildly. Someone would have out-competed them eventually, but it would have taken longer.

Something like self-driving cars — let’s make a bet. Of the top 500 cities in the world in 2035, how many differentially regulate self-driving cars on safety in a substantial way compared to human drivers?

Andrey: All of them.

Kevin: All of them, of course. Outright ban self-driving cars — I wouldn’t be surprised if that’s double digits.

Andrey: Boston almost did it, as far as I can tell. Very close. Although it’s one of those things — kind of like Uber, which entered as a banned entity and got so much consumer goodwill that politicians had to allow it. I’m not sure that happens with self-driving by 2035, but it’s not obvious when it tips.

[00:18:21] Kevin: It’s also not obvious we don’t get differential regulation — say, regulation that makes self-driving cars subsidize the insurance rates of traditional cars. The world is status-quo biased. Institutions exist because they won the Darwinian struggle to survive, so they’re well-fit for the environment they operate in, which makes them inherently conservative. That’s not crazy — but at a time of big disruption like AI, you have to take it seriously.

How many high-up people in Silicon Valley thought the US government would ban Anthropic? I agree it’d be insane to do. Nonetheless, I’m not surprised. If you think any government is going to allow open sales of AI at the frontier in two years, you’re deluded — they’re going to treat it like nuclear material. If you don’t believe that, your vision of the world is way too technological and not nearly organizational enough.

Why Situational Awareness Beat the Economists; AI for Science [20:16 - 26:00]

[00:20:16] Seth: Let’s wrap up the JEL article. In some ways you review Situational Awareness in a positive light compared to what the economists wrote. But it’s a narrative essay, not an economics book. What’s the economics book you want to read, and why are economists stuck?

Kevin: Getting Situational Awareness into the book review took a little persuading — for one, it’s not a book. But if an economist asked me for the one book chapter that best explains what’s happened in the last few years, I’d say chapter one of Situational Awareness. I give it to all my students.

What I want to read — both as a book and as research — is the endogenous impact of AI on science, including via robotics and via self-improvement. That’s the whole game.

Seth: You point out The Second Machine Age misses this. It says “imagine a billion researchers,” and they imagine Africa getting the internet, but they don’t actually model it.

Kevin: Exactly. Think how many papers go: “Here’s 2025 AI. I run an experiment where I tell you to do something that takes 10 minutes of work with AI, and I measure the treatment effect.” Who cares? Nobody’s reading that paper in five years. What people will care about is: are we getting self-automated science? Is blue-collar work being affected? If AI can do most of the research on the next AI...

I always ask high-up people in the labs: what year do you think a Chinchilla-law-level result — in terms of its importance to developing the next model — comes from AI? The answers are between 2027 and 2029. I’ve never gotten 2030. That’s not all research done by AI, but it’s a substantial speed-up beyond just writing the code faster.

Andrey: We already see that in math — it’s proving things humans weren’t proving.

Kevin: If something like Navier-Stokes is proven, I’d put that in the set of a Chinchilla-law-level result done autonomously. And if it can do that, presumably it can do research on sensors, actuators, batteries — and then the robots improve more quickly, and we get automated labs. That’s the takeoff question. Everything I’ve said doesn’t actually imply a takeoff. It depends what the bottlenecks are. Measuring those bottlenecks — the production function for specific areas of science and robotics — is incredibly high value for knowing where to allocate resources.

Andrey: And if we correctly predict them, maybe they won’t exist. It’s a feedback loop.

Seth: Ooh, I love this.

Kevin: Think of a production function — it tells you how much capital and labor to maximize production. I want to know what tools.

Seth: Kevin, I just did it — it turns out it’s energy. I looked into the future, it’s energy. Are you saying the best book about AI economics is just a book about energy?

[00:24:48] Kevin: It’s plausible. I help run Creative Destruction Lab — we were the first AI accelerator in the world in 2016, and we also run, I think, the biggest space accelerator. I was just down in Texas with astronauts for the Artemis launch. When you hear Elon talk about AI and space, it’s on the one hand crazy, on the other hand basically unregulated, effectively unlimited energy — and for training, who cares about latency? It’s not totally crazy that one way we get around the energy bottleneck is solar sails and ideas like that. In which case we face other bottlenecks. But this is an empirical question, and one where you’d want energy economists and energy experts, not just labor economists.

Forecasting AI: Surveys, Economists vs. Labs [26:00 - 28:20]

Andrey: One thing where I feel very stupid: about six months ago people around here kept saying “energy shortage, energy shortage,” and I thought they were probably right but didn’t trade on it.

You’re also involved in a project — we discussed it a bit with Avi Goldfarb — figuring out what economists are forecasting about the future of the economy under different scenarios. Tell us about it.

[00:27:06] Kevin: There are actually two projects — one I’ve been involved with, one I’m an academic advisor on. They both also ask AI-lab researchers, superforecasters, and the general public. The most interesting thing, as far as I’m concerned: on technical projections, there’s really no gap between the economists and the people inside the labs. And on economic projections the gap is also pretty small.

If you go from Acemoglu to Dario Amodei in our sample, Acemoglu is like the 1st percentile and Dario’s like the 99th — and neither is really representative of economists or AI researchers. It’s important to put these projections on paper and see how we did. Some surveys are now old enough to check. The projections of everyone — economists and non-economists — on frontier math were low. We were thought to be crazy with some of these projections, and we still underestimated the rate of improvement on certain benchmarks.

People say “it’s a benchmark, they trained to it.” The problem: I wrote benchmarks for one of the big labs. You know how hard it is to write a benchmark the AIs can’t solve? I did some in March. I’m running out of questions I can ask them.

Seth: They know how many R’s there are in strawberry now.

Benchmarks: The Coffee Test & the Fence-Post Test [28:20 - 35:53]

[00:28:43] Kevin: I have some tricks, but who knows how long they’ll last. Honestly you need benchmarks that look like the coffee test — or my favorite, the fence-post test.

The fence-post test is mine: I can buy a general-purpose embodied AI that I can tell on a Saturday morning, when I want to sleep in, “Go to my backyard and dig that fence post.” Not a specific machine — a general one. Every human could in principle do it. I think we’re quite a ways from AI doing it at cost.

The coffee test — I think this comes from Wozniak — is that an embodied AI walks into three random houses it’s never seen, finds the ingredients and the mug, and makes a cup of coffee. Well within the capability of any normal person. Whoever came up with it said the year it’s possible is “never.” We’ve done surveys — the modal answer from researchers now is the early 2030s. I think that’s the kind of benchmark you need, because anything on paper or on a computer — what Shane Legg calls minimal AGI — the goose is cooked. We can’t write tests I’m confident AI won’t pass in that domain.

Seth: Andrey just wrote a test the AI was very bad at.

Andrey: It’s really bad at predicting how many tokens it’ll use for a given task and whether it can actually do it. It’s poorly calibrated.

Kevin: That’s a well-known one, included in some benchmarks on AI’s ability to self-reflect. But it’s in the set of things where if I think about it a bit, I don’t know any reason I can’t hill-climb to answering it — ergo it’ll get solved.

Andrey: To be clear, our paper’s call wasn’t “we need these for economic activity, so please RL on them.”

Kevin: That’s essentially what you need, though. Anything obvious you can hill-climb on is cooked. Anything non-obvious but complementary to you hill-climbing on it is cooked. I need something outside that set.

Seth: But it has to be at the intersection of hill-climbability and being economically valuable to hill-climb. Or do you think we’ll saturate everything even if it’s not valuable?

[00:31:46] Kevin: I don’t think there’s a difference. Once I use AI in adversarial or competitive settings, making a mistake 0.1% of the time screws you. Edge cases are really bad in adversarial settings, and lots of economic activity has that flavor.

Seth: There’s no such thing as an economically unimportant question.

Kevin: Right — if you give me the economically unimportant question, I’ll design the economic interaction to screw you on it. Before LLMs we had GANs — you could put a sticker on a stop sign that fools any model but looks identical to a human. There are good statistical reasons we’ll never fully solve that. My favorite one AI has trouble with: they took an outline map of Europe, filled in part of the Bay of Biscay as if it were land, put an arrow on it, and asked “what’s here?” If you know your geography you say, “that’s the Bay of Biscay, oddly colored like land.” It’s just a weird thing for the training data to see. Economically it’s not per se valuable — most maps you see are the real map. But if I were using AI in a financial system, I’d be super concerned about my inability to solve that.

Seth: It’s very important to be able to draw pictures of wine filled all the way to the brim.

Kevin: Especially for these evening podcasts, Seth.

[00:34:14] Andrey: Back to the forecast — one question about the composition of people. I’m a participant in your surveys. I wonder if the economists are all our friends and not the skeptics, like Acemoglu.

Seth: We’re gonna get him on.

Kevin: It’s not just our friends. The selection mechanisms differ between the two, but you have to have published something related to AI at some point — and plenty of people who’ve published on AI are quite skeptical. It’s not snowball sociology; the selection mechanism is completely public.

Andrey: But there’s self-selection into participating — I do it because I’m very interested in AI economics; some might not.

Kevin: For sure, that’s an issue. Forget econ — I was just at a faculty association meeting talking to the humanities people. It’s amazing: the AI is simultaneously destroying society and can’t do anything. Very hard to hold both at once.

Seth: A very prestigious combination.

Rapid-Growth Scenarios, Labor Force, and “Phil World” [35:53 - 41:40]

[00:35:53] Seth: You said there’s not much difference between economists and non-economists on economic predictions, but my recollection is there are substantive differences — like the fast-progress scenario, a percentage point of GDP growth per year difference. That’s sizable.

Kevin: That’s where the biggest difference is — the rapid-growth scenario: widespread inexpensive robots that can do basically everything. Call it “Phil World,” since we talked about Phil Trammell — friend of the podcast. In that world in 2050, saying 1% growth per year is a little crazy. It’s hard to write down a model with bottlenecks that strong.

Seth: Or there could be dis-saving — people taking their labor out of the economy. We asked about labor-force participation, and even there the gaps were off-trend by five or six points. Not enormous.

Kevin: That seems small for that magnitude of change. But the description of “rapid AI” was technological capability, not diffusion. One explanation: it’s possible to do this, and we ban robots.

Seth: For my prediction I included increased chance of war as something that reduces growth.

Kevin: We had a couple of respondents say zero GDP because we’re all turned into goo. We won’t say which of our friends. For the rapid scenario, the 25–75 bounds are stupidly high. But for the AI all three of us would expect, the error bounds aren’t enormous — people were generally on the same page across groups. Most of the difference was within-group until you get to 2050 and rapid AI.

[00:39:10] Seth: Give the listeners some numbers for the median scenario.

Kevin: The best comparison is something like CBO or IMF projections — on the order of one percentage point more productivity, one point more growth per year. Which adds up to a lot — let’s say it adds up to the single most important invention in human history. On labor-force participation, about half a percentage point more per year in the drop — substantial out to 2030, not quite as big by 2050. Big effects, but...

Seth: It’s not the singularity. One percentage point additional growth a year for 20 years is the difference between two high-income countries — not the difference between the Flintstones and the Jetsons.

Kevin: I understand the objection: you read Situational Awareness, and from 2001 to 2026 the AI-pilled people were right and everyone else was wrong, so don’t bet against their projections. Fine — on technical grounds, say they were right. My response: I’ll literally take, as my technical projection for 2030, whatever the modal response from researchers inside the labs is. On what grounds would I disagree? But how that maps into labor-force participation — there I wish some of these people would close their mouths.

Scaling Regularities & What Economists Should Defer On [41:40 - 43:34]

[00:41:14] Seth: Let me ask about a techno-social prediction. We have this regularity, the scaling law — which you said should really be called a scaling regularity, because we’re not sure it’s a law of nature. The relationship between error rate and number of parameters seems technical. But then there’s the sociotechnological leap — that scaling leads to scaling capability. Should economists defer to technical experts on that, or is it a socioeconomic prediction we should have an opinion about?

Kevin: That one’s in the middle — related to AI for science. The scaling regularities — let’s say four of them — we just take from the computer scientists. But what’s the production function of medicine? How important are improvements in predicting protein structure to making a new drug? That’s not economics in the sense that we don’t have the field expertise, but it’s also not biology and not computer science. We’re in the middle.

Seth: In Aschenbrenner there’s a figure: right now it’s high-schooler level, in a year college, then professor. Is that the first kind of prediction or the second?

Kevin: That’s the first. I take that from the computer scientists. I want field experts and economists to estimate the production function, and social scientists to work out the implications on other parts of the economy.

Why Forecasts Matter for Policy & Capital [43:34 - 45:50]

[00:43:34] Andrey: Let me retort. People are interested in forecasts, but I don’t think economists are very good at forecasting. And it’s not clear how useful the whole exercise is. I could build my own custom macro model to answer these surveys — how much value to society would there be? Or is this more an exercise in social consensus, to bring to policymakers and say “here’s the range of expected outcomes,” without caring about the specific forecasts?

Kevin: A bit of both. Take chapter three of Aschenbrenner. If I believe that forecast, the government should borrow literally everything it can and plow it into chip production — because if your growth rate is 10% a year, who cares? So it matters a lot for policy. On a micro level: I’m an executive at Google deciding whether to put money into AI math solvers or into bio — Anthropic just put Novartis’s CEO on their board. Which improvements lead to value more quickly? And at the organizational level, if I’m a university, I need to know which bottlenecks are in my control and where I can just free-ride and wait.

When you talk about China, I’ll tell you something interesting I learned there: they’re not AGI-pilled. I think that’s going to cause problems — but we’ll get to that.

[00:45:56] Andrey: The final thing: yes, Anthropic is going into bio, but you don’t need forecasts for that. Just look at the share of GDP in different sectors. Economists are valuable, smart people — but using AI for medicine is the most obvious thing in the world; I don’t need an economist to tell me that.

Kevin: The marginal value comes elsewhere: if I spend $10 million figuring out how to allocate $10 billion of capital, that’s really high value. And on policy — listen to how policymakers talk. Bad predictions about the labor market coming out of some labs are going to cause regulation. States are going to ban data centers. We’re going to tax all the compute before we get the cancer drug.

I was working on a theory problem this week: I care about the wage bill — I want AI to be as productive as possible without harming wages. So you take something like Chamley-Judd, add a wage-bill constraint, add informational constraints for the planner about which capital is AI, let it substitute and complement in various ways, and solve. The result on taxation looks nothing like anything being proposed right now. To know that’s the right way to think about it, you can’t just say “AI will be useful in the future.” No — they’re going to ban it.

Andrey: This political economy of AI is something I’m tracking very seriously now. It’s obvious we’ll have bans and regulations long before AI actually has effects. People already think AI is causing mass unemployment.

Kevin: They’re immune to the data. “Block laid off 40% of their workforce.” It’s a bad media environment, too. A reasonable hypothesis: the sector most harmed by digitization and then AI is journalists — so young journalists, especially culture journalists, are incredibly hostile to AI, and the world they influence ends up asking “unemployment’s 4.5%, why is everyone talking about this?” There was an article this week about young people who don’t want kids because it’s too expensive — and the first couple they showed were 25, owned a 2,000-square-foot house, and the husband’s hobby was golfing in Utah. It’s a bad epistemic environment, and it’s bad for AI because it makes people hostile to change — they feel they have to protect what they have, even though the economy roared.

China Trip: Not AGI-Pilled, Involution, Capital Markets [45:50 - 1:01:40]

[00:50:29] Andrey: Let’s get your take on your China trip. What was the occasion?

Seth: Is China AGI-pilled? Why or why not?

Kevin: We need that one for the clip at the start of the video. I studied diplomacy — my goal when I was younger was to join the Foreign Service. I worked in China briefly at the embassy in ‘05, around WTO session time, and I’m back there quite a bit. After COVID, the number of foreigners in China dropped, so the information flow is bad. A colleague calls it the G2 when it comes to AI: two countries, plus Google’s London outpost. Nothing else really matters for AI. So not knowing what’s going on in China is really important.

This year I brought a group — economists, a guy from Epoch AI, a trade lawyer. I wanted to understand robotics, especially in traditional industries. AI’s effect on most of the market won’t come through San Francisco or Hangzhou. We met Zhipu’s COO, journalists who work on AI policy, startup founders, cloud providers, the biggest angel fund. We also went to Dongbei, the northeast — the fastest-falling population region in the entire world, losing about 1% a year, maybe 100 million people. We went to the one city that’s hanging on.

[00:53:34] First thing: nobody we talked to was AGI-pilled. When you ask what the AI is for, it’s completely about process engineering of existing industry. That’s it. Why open source? Process engineering. Why build your own non-frontier stack? Process engineering. And they actually use it in industry — some examples looked better than what we see in the West. But no one talks like the San Francisco or London DeepMind folks: “in 2029 my robot flies through the air and shoots the robber and delivers my peptides.” It honestly felt like talking to government people — “AI’s capabilities in 2026 plus epsilon.”

Part of what’s going on is a word in Chinese they translate as “involution” — I always tell them it’s not a word in English; it actually comes from Clifford Geertz, the anthropologist. It means extreme competition. It’s very hard to make a profit in certain industries — a hundred entrants immediately when you start making money. So high-fixed-cost, payoff-in-the-future investments are really hard. You only see it from things like DeepSeek, where it’s a hedge fund and the guy spends his own money. Even companies that seem to be doing great — the independent AI producers, not the Alibabas and Tencents — are in massive financial trouble, because it’s too competitive.

Seth: Part of that’s the interest rate and capital-market environment, right? American AI companies can lose money for a long time — why can’t they access money for more runway?

Kevin: China’s biggest advantages are energy costs about half of ours, and a much stronger hardware ecosystem — your ability to experiment and prototype blows away North America’s. It’s probably not even worth running a battery or robotics-hardware company here; you’ll get swamped.

Seth: Unless you’ve got a government contract.

Kevin: True — we should probably build our own drones. But things that require big fixed costs and have long payoffs need deep capital markets that reallocate capital quickly, and China doesn’t have that. The VC market is worse than a decade ago — foreign VC basically left. Most companies get investment from state-linked banks or rich people out of pocket. DeepSeek is trying to raise $20 billion; if they were in San Francisco they’d start at ten times that.

[00:57:06] Andrey: Let me play devil’s advocate. So far most of the rewards go to frontier models — you can’t charge enough for non-frontier tokens. So DeepSeek doesn’t make sense unless it’s a government-funded national champion.

Kevin: If DeepSeek weren’t in China, with their leadership and computer scientists, they could have attracted the Chinese equivalent of Alec Radford and Ilya Sutskever and been in the race for the frontier. They can’t, because of the capital markets. This isn’t just AI — all sorts of industries face it. They can move quickly when the design already exists, but for “I’m doing something genuinely new,” they’re behind. Self-driving — they’re behind Tesla. Not Waymo, Tesla — even the frontier Chinese car companies.

Andrey: That’s crazy to me. I’d have thought they’d have a separate, more generous government lane.

Kevin: Look at what Google had to spend to build Waymo — no one else in North America pulled it off, because you needed to lose tens of billions and there wasn’t enough capital.

Andrey: In China labor is cheap, so the economics of an autonomous-vehicle service are worse there. But modern neural networks made AVs a lot easier — Google couldn’t really have done it before 2022.

Kevin: An executive at one of the new Chinese car companies told me that in China, Elon’s strategy is seen as smarter than Waymo’s — they think Waymo’s approach is out of date: LiDAR is cheap now, don’t map the roads. Maybe they’re right and catch up. On the cars themselves they’ve caught up — if their cars were sold in North America they’d take the market. And it’s not the traditional four — the Ford and GM of China are also screwed; the architectural shift to electric was too hard. It’s the new companies that would crush us. But they still haven’t caught up on self-driving.

[01:00:11] Seth: Follow-up on capital deployment — bringing Leopold back. He thinks the big frontier labs end up as nationalized projects. China can deploy a lot of capital toward national projects. Do you see this disadvantage reversing if we get one big national lab per country?

Kevin: Good question. Hasn’t happened yet. I think they’d have the same problem — China has hippies now. They have words like tang ping, “lie flat” — I’m not joining the rat race. They have guys like the people at Anthropic wearing sandals and reading the Whole Earth Catalog. Those people, in the US and China, aren’t going to work for some state-backed project. You can maybe state-back the energy rollout, but it wouldn’t attract some types of talent.

War, Nationalization, the End of Open Source — and Claude [1:01:40 - 1:06:06]

Kevin: The part of Situational Awareness that seems like it must happen — I wrote my PhD dissertation on early nuclear. Back then you literally weren’t allowed to publish your patents — state secret. We’re very close to wars where AI plays a major role. At that point, who’s going to let this stuff be independent? The government doesn’t let you sell missiles — they’ll let you sell to partners they approve, and that’s it.

Seth: Does that mean the end of open-source models above a certain size? Some sort of IAEA for AI? Turing police monitoring frontier labs under UN auspices?

Kevin: When people talk about UN regulation of AI — take a foreign-policy class. Neither China nor the US cares one whit what the UN says. There’s going to be an organization called the G2: the US president and the Chinese premier talking to each other. That’s how it’ll work.

Open source is interesting — it’s a little bit dying in China. The most well-known researchers at Alibaba quit. A couple of other well-known model makers are going to go bankrupt — it’s not obvious how you make money making open-source LLMs as an independent. I suspect Llama is the last big one Meta makes. Someone will make them — NVIDIA’s pretty clearly going to try, because it’s such an obvious complement. But you can imagine a world where open source becomes much less common.

[01:03:32] One interesting thing: talking to people in AI in China — not political people — every single person thinks Claude is the best model, and they all use Claude. Not domestic models. Even though it’s very hard to do that from China, on both the government and the Anthropic side. There’s no opinion that China is catching up on AI. The view is that not only is Claude ahead, but the one place they a bit believe the AGI pill is that inside OpenAI, Anthropic, and DeepMind they’re using these models to speed up product deployment — and China doesn’t have the same access to frontier models, which makes it tough.

I’m doing a thing for NBER on what chip bans would do to endogenous innovation in China — how to even model that isn’t obvious. The cynical answer is it’s whatever the marginal cost of buying chips from Kazakhstan is — one more plane flight.

Seth: I was reacting to the Jensen interview. We’re half a beard away from you being at that point.

Kevin: I should have worn the leather jacket — that’s the look now. Actually, the real move is the T-shirt from our machine-learning accelerator, before we called it AI, back in 2016. That’s the one you flex with.

A Fine Theorem, Blogging, and the Value of Taste [1:06:06 - 1:17:48]

[01:06:06] (For those playing along at home, now’s your chance to think about how this conversation has changed your priors — sponsored by Revelio Labs.)

Seth: Revelio Labs is a leading provider of labor-economics data and data services for companies, academics, and independent researchers. They combine comprehensive micro-level data on employee profiles, job postings, and sentiment with standardizations, mappings, and enrichments — flexibly aggregated to company, market, or industry — to study everything from career trajectories to occupational transformation to the impact of AI on labor demand. Their data is available on WRDS, so if you’re an academic with a good library, check whether you already have access. If not, reach out to their economics team.

[01:07:21] Seth: One thing you didn’t mention at the top: the reason you’re so close to my heart is your famous blog from the glory days of econ blogging, A Fine Theorem. When I started my PhD in 2012, getting excited about the big questions in economics and how theory can contribute, I found it so inspiring. So much of how you publish in econ now is: find a cute IV for one of a limited list of subjects, or — God willing — J-PAL backs you and you do an RCT. That may be useful, but it’s not what excited me about economics. Your blog was my north star for how technical theory can and should be communicated. So, snaps for how cool that was.

Kevin: Hold on — who do you think the Gen X is in this conversation?

Seth: Are you an elder millennial? Did I just mess up?

Kevin: I thought I looked young for my age — I’ve got the dimples.

Seth: As a generational-conflict theorist, the thing that struck me about the Dwarkesh–Jensen interview was Gen X shape-rotator Jensen and millennial wordcel Dwarkesh. So it wasn’t surprising you had the leather-jacket option.

Kevin: I’ll say the Gen X has excellent taste in music. I went to the Oasis reunion concert — probably the youngest person there. It was great.

[01:09:42] So, A Fine Theorem. It’s related to AI development, believe it or not — one important way new technologies diffuse is the development of complements. That site started as my PhD notes on the papers I was reading; it was just easier to keep them in a WordPress setup. Some people found it through RSS — that’s how you found things on the internet then. Now people find things through gated social media, group chats, podcasts. It was good timing for me. I was never that interested in running a podcast — someone asked me to do one on the economics of science years ago — writing just matches my background better.

It got a bit wild. I’d write about maybe a hundred papers a year, plus Clark Medals and Nobel Prizes. I had a reputation as the guy who reads everything across fields and isn’t shy about his opinions. The three craziest emails I’ve ever gotten: I proposed a reform to the NBA and the president of an NBA team emailed me to talk about it. And two different Nobel laureates read my notes after they won and wrote asking me to read through their Nobel speeches. That’s the coolest thing ever.

Seth: “Explain to me why my work was important.”

Kevin: It makes sense — you know your work, but not always how people see it or how it influences them. I go to conferences and students will say “I’m extending your paper from 15 years ago this way,” and I’ve completely forgotten about that corollary. They know more about it than I do. Tyler Cowen liking it led a lot of people to read it. At one point it got, I don’t know, a million views — crazy for a microeconomic-theory blog.

[01:12:51] Seth: I do think you’re quite good at writing it for an educated reader, not just as a paper. There’s a big latent market for this — previous guest Noah Smith works the same lane. We love Noah, but you can’t compare Noah to Kevin in terms of gravitas and depth.

Kevin: A lot of academics think their job is research and teaching — writing papers for other academics and maybe policy folks. But now I know who’s reading that stuff. I was writing about epistemic game theory, and serious people read it. My work on progress studies — I teach a class on progress with serious research behind it — there’s huge interest. I was at a conference with Chad Jones, the growth theorist, and there are people in industry reading Chad Jones papers seriously. The world is much more interested in serious work that answers serious questions than academics think. If they understood that, they’d be more careful with their work and would choose different topics — instead of “I’m writing this because journal editor X just got promoted.”

The Economist as Plumber: Comparative Advantage & RCTs [1:17:48 - 1:24:07]

[01:14:24] Seth: Let me ask about the how and who you write for. One theory behind this podcast: as the marginal cost of writing papers goes down, the marginal product of reading them can go up. Do you see AI increasing the relative importance of digesting and synthesizing research?

Kevin: The one-sentence version you hear — which I think is true — is that the marginal value of taste has gone up.

Seth: But what’s taste?

Kevin: There’s stuff that’s fun to consume — I watch YouTube golf like everyone my age, but I know I’m not learning anything; I should be watching topology videos. Taste is understanding why a thing matters. Show me 20 things written about chemistry and I can tell which is better written, but not which one matters. To have taste — in music, literature, economics, anything — you need a really strong epistemic base. AI can point out “this is a good paper,” but not “this is a good paper in line with your individual interests.” Maybe in a world with continual learning, where your AI is your assistant — but we’re not there.

Seth: But it was beyond what was interesting to you — somehow it was also inspiring to people like me in grad school.

Kevin: Right. I’m illiterate about music — play me some Bach and I barely know the difference. But once in a while a really good critic writes “listen to this part and you’ll hear this,” and suddenly I do hear it and understand why it’s interesting. That person couldn’t have just listened to that one piece or read one book — they need to understand the history of music. People have different areas where they can have taste. Mine is probably the intersection of theory, history, and history of thought — and that mixture isn’t very common.

[01:17:48] Seth: Let me pull out something you may have a distaste for — a quote from your review of the Banerjee–Duflo–Kremer prize. “The economist as plumber, famously popularized by Duflo, who rigorously diagnoses small problems and proposes solutions, is a fine job for a World Bank staffer, but a crazy use of the intelligence of our otherwise leading scholars.” React to that in the age of AI, where the market is flooded with “we estimated the productivity impact of AI adopted here on this date” papers. What should those people do instead?

Kevin: You’ll be surprised — because I believe in comparative advantage. I literally mean it’s good work for a World Bank economist; people should do that. I just don’t think Banerjee and Duflo should have been doing it. Same way Stantcheva’s taxation work was unbelievable, Clark-Medal-winning — and then she wrote a bunch of papers basically running a survey firm. The papers are interesting, but it’s not her comparative advantage; many people have more expertise in that area, and it’s not that complementary with the rest of her work.

Andrey: I’ll disagree. Both survey research and experiments required elite permission to do this type of work. There’s no objective, agreed-upon standard in social science for what we should work on. Having an MIT or Harvard economist legitimize it in a top-five journal lets a bunch of other people — for whom it is their comparative advantage — work on it. On the margin maybe they work too much on it. In marketing, where I sit, there was a perception that survey research with stated preferences was something we shouldn’t do — and now if a top economist says it’s okay, maybe we can.

Kevin: For sure — same way J-PAL was useful, and they won a Nobel for it, so they were rewarded. I agree on the permission structure. The question is what we do now with AI. In a sense it’s not great for me — being the smart-ass kid who’s really good at algebra is worthless now. I worked on a paper recently I’d been stumped on for years — a proof I couldn’t figure out. GPT-5.4 Pro was also stumped, but in its write-up it gave me a polytope-theory result I hadn’t seen, and I used it to prove the thing. I felt like a dad beating his teenage kid in basketball — super happy, but I know it might be the last time.

[01:21:56] If you’re a PhD student now and your specialty is being really good at solving models, you’re just not going to have a job — you’re not as good as the AI. But some things are incredible complements to AI: within-firm field experiments done with much higher ambition than now. Those will be very popular and not susceptible to replacement for a while.

Andrey: But didn’t you say we shouldn’t be working on this?

Kevin: I said we shouldn’t be doing RCTs — but I believe in comparative advantage, and we’ve changed the price of the factors. If we’re going to do this, what’s a bad idea is doing it atheoretically and ahistorically. Two things you need as a PhD student: your work has to be a complement to what AI can do, and your work has to have taste — you need to know what matters and why. A field experiment estimating a treatment effect no one cares about shows a lack of taste. Thinking you’ll get a job solving a model any AI can solve shows a lack of understanding of comparative advantage. High-paced managerial types are going to do better in academia than they used to, and some folks who were high-status will find nobody cares.

Andrey: I see how the human advantage is running RCTs versus writing macro models. But what’s the right approach for writing that AI book you want us to write?

Kevin: I still think you should write the macro model — your contribution just isn’t solving it. And your empirical paper needs to draw on and understand the macro models you’re building on. You should spend more time reading papers, not less, to develop taste.

Andrey: Or you shouldn’t read papers — you should talk to the AI about the papers. Or listen to this podcast.

Kevin: You should be listening to Justified Posteriors, brought to you by Jane Street.

Andrey: We’re manifesting Jane Street.

The Future of the Academic Paper [1:24:07 - 1:28:22]

[01:24:19] Kevin: Academic papers are an unbelievably entrenched system, but here’s where I’m trying to go — and I edit an AEA journal, so I talk to them about how we handle AI. In a couple of years, a paper is: all the lab notes, code, and data, open and in a format AI can read — that’s already in progress. Then a paper that ranges from the 40-page version to the five-page version to a “talk to the AI” version. If you go to my website, my papers already have a built-in Gemini Flash interface, because I assume people want to talk about the paper while reading it. It’s not just a PDF.

So every paper will have a partially AI-generated hundred-page version with all the information for the AI, the 40-page version, the five-page, the three-page, the interactive version — because the cost of writing the paper is so high relative to the cost of those manipulations. The idea of a paper as a fixed set of words is over. If that’s all it is, everyone’s going to talk to GPT about it anyway — we can do better.

Andrey: Does that mean writing goes down in importance? Someone like Chad Jones is such a crisp writer — that’s a key reason people read him.

Kevin: How AGI-pilled am I? The best academic writer in our profession is not Hemingway — let’s not be deluded; the average writer is terrible. People outside academia may not realize how much editing for readability happens in an academic article: the answer is zero. Maybe one or two sentences you’ll be asked to crisp up. It’s not The New Yorker — there’s no editor rewriting your paper for readability. The only reason people try to write well is that on the margin it raises your acceptance probability. Otherwise they write like a lawyer.

Andrey: It’s taste. It’s for themselves.

[01:27:05] Kevin: I had an idea — maybe we do it for AI. I wanted an innovation journal; there’s no good one, and innovation is very interdisciplinary. But no one will send a paper to a new journal, for tenure reasons. So how do I free-ride on the system? Create a journal that any already-published paper is eligible for. Have a board of 30 great innovation and AI economists; as soon as three say “if this were my field, top field journal, I’d have taken it,” it’s in the journal. We link to the working-paper version and hire a professional to write a 1,500-word, Quanta-Magazine-style article about why the paper matters.

Seth: Have you heard of the Unjournal, an EA project? It has some of these ideas.

Kevin: Yeah, the Unjournal’s a good one.

Andrey: Works in Progress is doing some of this too — taking academic research and making a great article about it.

Kevin: That’s why the innovation-econ world and the progress world have a lot in common — we’re all friends. This year I felt like a progress-world celebrity, because one of my PhD advisors was Joel Mokyr, and they love Mokyr in progress world — he’s like Michael Jordan.

Andrey: I tried reading A Culture of Growth and it’s unreadable. I’ll just put it out there.

Kevin: The Gifts of Athena is the one I recommend — though you have to work through 50 pages of prescriptive-versus-propositional knowledge with lambdas and sigmas. He’s still a better writer than the average economist — low bar.

San Francisco, Ambition & the Permission Structure [1:28:22 - 1:32:56]

[01:28:46] Kevin: My favorite thing about what’s going on in California — other than the incredible ambition — for folks who aren’t here, there’s all sorts of craziness; they make Seth and his EA beard look normal.

Seth: It hosts insects and shrimp that are having a lot of utility.

Kevin: You can save a little dinner for later up in the mustache. But the level of ambition — your average 21-year-old asking “what should I do with my life” aims this high. That’s not normal in most places, where the very smart people are type-A, “follow this rule and this process.” In academia we know tons of those people. It’s super refreshing. In progress world, every random person is like, “should I make money, or start a biohacking magazine that four people buy but I like doing?” — biohacking magazine. And I love it.

We have a guy in Toronto, Ben Perry, who runs a sort of “Toronto society,” also in progress world — he holds talks on what makes a beautiful city. He asked me to give one related to my progress course, on idiosyncratic factors that lead to progress — a pretty out-there talk. I show up and we’ve sold out a concert hall. People paid 30 bucks a ticket, there was music beforehand, and afterward people are in the hallway chatting about what they’re building. These people are all over.

[01:31:05] The remaining secret sauce of Silicon Valley is that everyone — all the way up and down the permission and capital structure — agrees the most ambitious people should have the power and the capital. That’s rare. During COVID, my university was closed and it was driving me crazy, so I went to teach in Senegal — the best university in French West Africa, teaching high-growth entrepreneurship. Great students. I asked what they wanted to do when they graduated, and they all wanted to work for the government. “You don’t want to start a company?” “If it doesn’t work out and I go bankrupt, I’m living on the street, and no one gives a 22-year-old money to start a company.” And that’s reasonable. But that societal structure makes growth impossible. That’s the thing you have to get right.

Lightning Round [1:32:56 - 1:38:08]

[01:32:56] Seth: How are we on time? Want to do a lightning round? And give yourself a chance to talk about All Day TA.

Kevin: Let’s do All Day TA as part of the lightning round, so I don’t feel like a sales call.

Seth: Lightning round, beginning. Favorite economist, living or dead?

Kevin: Dead: Paul Samuelson — awesome work. Living: Bengt Holmström, because when I walk around on the street his ideas are in my head all day.

Seth: All Day TA — what did you learn from being an entrepreneur?

Kevin: This is my company — we sell ed-tech to universities, a hundred-plus now, all over the world. I learned that for AI diffusion, institutional sales is so hard in traditional industries, and so unrelated to product quality, that the people who already own the gates into big institutions — the Salesforces, the Microsofts — are going to clean up in the AI world. People who think they’ll sell a great product and get around those gates are deluding themselves.

Seth: Did you learn a trick for selling to universities?

Kevin: Did it a hundred times. The technical stuff matters very little. You have to figure out who has the decision rights — often the head of IT — and they often have some idiosyncratic thing they want. Going through the professor has no power. With any institutional sale, the secret is knowing who can write the check and getting to that person quickly.

Seth: If you had to burn all of Kremer’s RCT work or his O-ring paper, which would you destroy?

Kevin: I love the O-ring paper. But if your experimental papers probably saved a million lives, I have to let you keep those. So we burn the O-ring. Also, we probably could have figured out the O-ring without Kremer.

Seth: But would it have been written that beautifully?

Kevin: No. It’s such a nice paper.

Seth: What advice do you have for folks in economics grad school today?

Kevin: You’re five years out — read Situational Awareness chapter one, and believe it. Whatever you think you’re doing in your job-market paper, ask: is that consistent with creating value in the world of Situational Awareness chapter one? If not, literally do anything else.

Seth: If you had a choice of joining a lab or going to econ grad school, what should someone choose?

Kevin: I don’t think there’s necessarily a conflict. But when my most ambitious 22-year-old students ask what to do, I say: it’s like being a writer in 1920 — get on a boat and go to Paris and don’t be stupid. You’re an ambitious 22-year-old: get in a van, drive to San Francisco, and don’t be stupid.

Seth: Seth, any more lightning rounds? No, I think we’ve covered it. Kevin, this was a completing-the-circle experience for me — your blog was so inspirational on my economic journey, and getting to talk to you and be treated as an equal was a very special moment.

Kevin: It’s nice you got to talk to me before I reached my full senescence — given whatever age you think I am. It’s really ruining my self-image.

Seth: I always think it’s so beautiful when millennials can get along with Gen Xers. It’s a special thing.

Kevin: You know the irony? The Gen Xer wouldn’t have cared — “who cares, man, don’t worry about it.” Only the millennial complains about being called the wrong generation.

Seth: That’s true. Thanks a lot, guys — keep up the good work on the podcast. I’m looking forward to the next guests.

[01:37:53] Seth: And to listeners at home — keep your posteriors justified.

The Forecasting Research Institute’s recent survey of leading economists, AI experts, and superforecasters quantifies several dimensions of disagreement. The figures below show that there is enormous disagreement among experts and that AI experts tend to forecast higher economic growth rates than economists. Tom Cunningham nicely describes a variety of forecasts in his blog post.

Some disagreement reflects genuinely different beliefs about AI progress and the transition path of economic adjustment. Disagreement is healthy, and reflects diversity in economists’ world models. But a deeper and more fundamental problem is that few people have strong incentives to produce granular, decision-relevant forecasts. Few have done the detailed work of trying to predict what will happen to the macroeconomy. Even fewer have worked at the granular level required to predict which industries will be affected and when, which jobs will be diminished and which will grow, and what will happen to wages in those jobs. The financial incentives are too weak for these experts to invest the effort.

Even structured modeling efforts illustrate how wide the uncertainty is. Epoch’s GATE model is a macro model of AI’s economic impact, built around an automation feedback loop between compute investment, AI capabilities, and task automation. But its outputs are highly sensitive to a handful of parameters whose true values are deeply uncertain. The model can produce qualitatively different trajectories depending on plausible parameter choices.

So we’re operating in a world of great uncertainty. This uncertainty is high-dimensional and will never be fully resolved. But we can, in principle, make good predictions about slices of the state space -- specific events that are both learnable: we can go out into the world and collect information about what will happen; and decision-relevant: the information is valuable for making better decisions. We think our best bet for doing so will be through prediction markets that, despite their shortcomings, are powerful technologies for incentivizing information acquisition and aggregating dispersed information.1 This is a modern rendition of a classic Hayekian case for prices.2

The Proposal

We propose that AI labs, Big Tech companies, and philanthropic donors contribute funding to create well-capitalized prediction markets to understand the likely impacts of AI. We need this information because individuals, firms, and policymakers are already making decisions right now – with or without this information.

Consider a college graduate deciding which industry to enter, or even a highschooler entering college deciding what to major in. They probably want some information about which jobs are likely to exist in the future and which jobs are likely to be less common. It would be very useful to have, based on the most recent up-to-date information, some signal for how to make this decision. Firms are in a similar position, when deciding which occupations to hire and at what level of seniority.

Policymakers face an even harder version of this problem. Decisions about retraining programs, extended unemployment insurance, and sector-specific support require knowing not just whether AI will displace workers, but which workers, in which industries, and how quickly. Retraining programs in particular are highly sensitive to these forecasts: training a displaced bookkeeper takes years, and the value of that investment depends on which destination occupations will still exist when the training is complete.

The FRI survey found that economists ranked job retraining as their most-supported policy response, with a forecasted boost to labor force participation in a rapid AI progress scenario, but designing such a program requires answering granular questions that current forecasts simply don’t address. Extended UI faces a similar problem: the case for sector-specific extensions depends on whether displacement is temporary (workers transitioning between jobs) or structural (occupations disappearing), and the appropriate duration depends on expected re-employment rates that vary enormously by industry.

How Prediction Markets Would Help and Why They Don’t Yet Exist

For prediction markets, we envision a variety of contracts. Start with the labor market: aggregate labor-force participation, and occupation-level employment and wages, are measured in official BLS data. For more detail on how these contracts could resolve, see the appendix on contract resolution using labor-market data. We would like to have a forecast of the future labor force participation rate and of the employment rate and wages across different occupations.

Suppose that we had prediction markets at different time horizons (1, 2, 5 years) for the total labor force size of different occupations in the United States, and analogous contracts on wages and employment rates. With an appropriate and liquid contract, we should have a good signal about the future labor market. If this exists, when a new graduate is choosing a career, they could use this information to think about which occupations have continued demand. Note that these contracts don’t forecast the impact of AI directly, but to the extent that AI is an important force in determining future employment rates, its impact will be a part of the forecast.

But labor outcomes are downstream of AI capabilities, and we’d also want contracts directly on capabilities themselves. The questions currently trading on Polymarket are coarse, such as “Which company has the best AI model by the end of June?”, and don’t track the underlying technical progress in a way that’s useful for forecasting economic impact. We’d want contracts on benchmarks (e.g., ARC-AGI-3) or capability thresholds (e.g., passing Steve Wozniak’s coffee test) at different horizons. These capability forecasts are valuable on their own and would also help calibrate the labor market contracts: if traders price in rapid capability gains but slow labor market changes, that discrepancy is, itself, informative about the market’s belief about passthrough from capabilities to labor market impacts.

The next question is: what will it take for these prediction markets to have meaningful information aggregation, and why don’t they exist yet?3

There are two reasons people might participate in prediction markets: speculation and hedging. So far, firms haven’t created enough demand to hedge for AI’s labor market impacts. Furthermore, for contracts that are longer-term, traders need to be compensated at least at the risk-free interest rate, meaning that incentives to trade are diminished for contracts further in the future. As a result, the chicken-and-egg problem hasn’t been broken, and we lack prediction markets on these questions. So the question becomes how to bootstrap them (see Andy Hall’s essay on some ideas).

What It Would Take

Prediction markets need to be seeded. The reason is that prediction markets have increasing returns to participation. With a thin market, prices are noisy, spreads are wide, and traders cannot enter and exit positions without moving the market. But without traders, the market never becomes informative. Seeding solves this coordination problem by paying for early liquidity, narrowing spreads, and making it worthwhile for those best-positioned to acquire information to do so.

There are two ways to seed prediction markets. The direct way is to provide liquidity by staking a market with capital on both sides of contracts, so that other participants have something to trade against. This is how most prediction markets historically got started, but it requires the seeder to take on inventory risk.

A more natural way is sponsorship. The way this works is that a principal (the entity that wants the information) deposits money into a contract. Sponsorship adds a way to bootstrap the market: the sponsor pays participants who improve liquidity, for example by posting orders that narrow spreads or add depth. If the sponsorship is large enough, then traders have incentives to acquire information and to participate in the market. Once there is already active trading volume, the market may sustain itself even without additional sponsorship funding. Sponsorship is already a feature on Polymarket and may be available soon on Kalshi. Technically, sponsorship is one way to subsidize liquidity provision. This connects to the market-design literature on automated market makers and scoring rules, especially Hanson’s logarithmic market scoring rule and Chen and Pennock’s bounded-loss market-maker framework, which formalizes the tradeoff between liquidity provision and worst-case loss (Hanson 2003, 2007; Chen and Pennock 2007).

It’s worth being clear about what sponsorship does and doesn’t do, because the FRI survey is instructive here. FRI paid economists to participate, and got back the most comprehensive elicitation of expert beliefs about AI’s economic impact that we have. But the economists essentially had no skin in the game. Sponsorship has a different structure. The sponsor pays for the market to exist, but the traders who actually set prices are putting their own capital at risk.

The natural candidates to sponsor these markets are large players, either the AI labs themselves, or firms with significant exposure to AI that are interested in getting better information. They have the budget, and they have the strongest reason to want this information to exist. We don’t have a good sense of how much sponsorship volume is necessary for good information aggregation. That’s an open question, and one we’re interested in experimenting with. That said, we can’t do this alone. A big player needs to participate, or at least fund research on how to create these contracts.

Thanks to Tom Cunningham, Andy Hall, John Horton, and Scott Kominers for comments.

Appendix: What Makes a Good Contract

Not every interesting question can be turned into a useful prediction market. A workable contract needs four things:

Verifiability. The resolution criterion has to be objectively checkable. “By 2030, will AI have caused Cheerios sales to double?” fails immediately because “caused” isn’t observable. You can’t separate the AI-driven counterfactual from everything else moving in the world. “By 2030, will the BLS-reported employment in occupation X be below Y?” is verifiable: the number gets published, and you read it.

Stability. The underlying data series has to keep being published and measured the same way over the life of the contract. Niche or proprietary statistics fail this. BLS occupation employment, unemployment rates, and labor force participation are highly stable. Anything that depends on a single company’s voluntary disclosure is not.

Robustness to gaming. The bigger the underlying quantity, the harder it is for any trader to move it. Someone with a few million dollars and an axe to grind could plausibly distort sales of a single product or the headcount of a small firm. They cannot plausibly distort BLS employment statistics for a major occupational category. Sector-level labor market contracts are robust for the same reason: the sector is too large to move by hiring or firing at the margin.

Attention. Even a verifiable, stable, robust contract is useless if no one cares enough to trade it. Cheerios sales fail this test too. Unemployment contracts have so far failed, but we think that this is for reasons that are fixable.

Appendix: Contract Resolution Using Labor-Market Data

For labor-market contracts, the natural resolution sources are large, official, regularly published data series, and not tailor-made AI-impact metrics. The Occupational Employment and Wage Statistics (OEWS) program under the Bureau of Labor Statistics (BLS) provides annual estimates on employment and wages for approximately 830 occupations at the national level, state, metropolitan, and nonmetropolitan areas, as well as for specified industries. This makes OEWS a natural source for occupation-level employment and wage contracts. Such contracts would not identify the causal effect of AI. Instead, they would rely on a set of observable labor-market outcomes — employment, wages, or labor-force quantities, which directly influence students, firms, and policymakers. For example, a contract could be settled on whether employment reported by BLS in a specified SOC occupation falls below a pre-specified threshold in a given release year.

References

Hayek, F. A. 1945. “The Use of Knowledge in Society.” American Economic Review.

Wolfers, Justin, and Eric Zitzewitz. 2004. “Prediction Markets.” Journal of Economic Perspectives.

Arrow, Kenneth J., et al. 2008. “The Promise of Prediction Markets.” Science.

Hanson, Robin. 2003. “Combinatorial Information Market Design.” Information Systems Frontiers 5 (1): 107–119.

Hanson, Robin. 2007. “Logarithmic Market Scoring Rules for Modular Combinatorial Information Aggregation.” Journal of Prediction Markets 1 (1): 3–15.

Chen, Yiling, and David M. Pennock. 2007. “A Utility Framework for Bounded-Loss Market Makers.” In Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI 2007), 49–56

In this episode of Justified Posteriors, we welcome Seb Krier — policy lead for AGI at Google DeepMind and excellent Twitter poster. Speaking in his personal capacity, Seb walks us through his understanding of AGI, why AI alignment has gone better than expected, the potential and limitations of a world where agents constantly barter on our behalf, and — of course — electronic music.

We also cover AI in London vs. New York, how Seb went from reading Marginal Revolution for 15 years to becoming a recurring character on it, and Seb’s side-splitting humor on mediocre AI conferences.

Related Links

• Seb Krier on X: @sebkrier

• Seb’s Substack, Technologik

• “Coasean Bargaining at Scale” — Seb’s essay at the Cosmos Institute (also republished here)

• “Musings on Recursive Self-Improvement” — Seb’s essay separating model-side RSI from societal-side

• “The Cyborg Era: What AI Means for Jobs” — Seb’s guest essay on Alex Imas’s Substack, defending the scaffolding view

• Anthropic’s Project Deal — the agent-bargaining experiment among Anthropic employees

• Fradkin & Krishnan, “MarketBench” — Andrey and Rohit experiment of LLMs bidding in procurement auctions as an investigation of the future of AI marketplaces and the companion writeup: Rohit Krishnan, “Agent, Know Thyself! (and bid accordingly)”

• Edge Esmeralda — Devon Zuegel’s pop-up village in Healdsburg, CA

• MATS — for junior economists looking to skill up on AI safety/governance

• Cosmos Institute and FIRE

• bianjie.systems — the art platform Seb is co-organizing a dinner with in NY (Seb’s announcement)

• Drexciya — James Stinson, Gerald Donald, and the Detroit electro-afrofuturism canon

Timestamps

(00:00) Intro
(01:16) What is AGI?
(07:30) In defense of scaffolding — Hayek, division of labor, and why one giant model won’t do it
(13:00) Markets for cognition: will agents bid in procurement auctions?
(18:40) Recursive self-improvement — separating the model side from the societal side (24:44) Alignment has gone better than 2017-Seb expected; prefer “intent following” (31:14) What economists should actually work on to inform AI labs
(33:32) What does a DeepMind policy lead’s day look like?
(38:20) AI Conferences
(41:52) Coasean bargaining at scale — the positive vision
(55:00) Inequality, property rights, and who gets the initial allocation
(01:03:00) The Helldivers 2 “Managed Democracy” dystopia as Coasean bargaining gone wrong
(01:09:00) Sponsor: Revelio Labs
(01:09:30) Lightning round

Transcript

00:00:00,100 --> 00:00:20,480 [Seth]

[upbeat music] Welcome to the Justified Posterior’s podcast, the podcast that updates beliefs about the economics of AI and technology. I’m Seth Benzell, the number two biggest fan, after Tyler Cowen, in the Seb Krier fan club.

00:00:20,480 --> 00:00:20,740 [Andrey]

[laughs]

00:00:20,740 --> 00:00:24,660 [Seth]

Coming to you from Chapman University in sunny southern California.

00:00:24,660 --> 00:00:34,120 [Andrey]

And I’m Andrey Fradkin, coming to you from San Francisco, California. And Justified Posterior’s is sponsored by the fine folks at Revelio Labs.

00:00:35,560 --> 00:00:45,600 [Andrey]

We’re very excited to have Seb Krier here with us today. He is the policy lead for AGI at Google DeepMind, and is,

00:00:46,840 --> 00:00:52,400 [Andrey]

dare I say, a thought leader in this space. Welcome to the show, Seb.

00:00:52,400 --> 00:00:54,200 [Seb Krier]

Thank you very much. It’s great to be here.

00:00:55,380 --> 00:00:58,160 [Seb Krier]

Yeah, I’m Seb, calling in from New York.

00:00:58,160 --> 00:01:00,320 [Andrey]

And we should remind our listeners that

00:01:01,340 --> 00:01:08,410 [Andrey]

Seb is, during this podcast, expressing his personal opinions, and is not speaking on behalf of DeepMind. All right.

00:01:08,410 --> 00:01:09,740 [Seb Krier]

Indeed. [laughs]

00:01:09,740 --> 00:01:11,060 [Andrey]

[laughs]

00:01:12,780 --> 00:01:13,900 [Andrey]

The usual caveat.

00:01:15,260 --> 00:01:16,760 [Andrey]

Seb, what is AGI?

00:01:18,080 --> 00:01:19,450 [Seb Krier]

What is AGI? [laughs]

00:01:19,450 --> 00:01:19,570 [Andrey]

[laughs]

00:01:19,570 --> 00:01:19,580 [Seth]

[laughs]

00:01:19,580 --> 00:01:19,780 [Seb Krier]

Great question.

00:01:19,780 --> 00:01:21,900 [Andrey]

We’re going to start with the big questions.

00:01:21,900 --> 00:01:22,880 [Seb Krier]

Yeah, might as well.

00:01:24,259 --> 00:01:54,840 [Seb Krier]

[sighs] I think there’s so many definitions out there of what AGI is, and I think most of them are kind of unsatisfactory in one way or another. I’ve seen stuff like many definitions are indexed on the societal transformations or economic impacts of the technology, which I don’t really like very much because it makes it very dependent on external factors whether or not we have AGI. If it’s banned, we don’t have AGI, and if it’s not banned, we have AGI. Is it?

00:01:54,840 --> 00:01:55,480 [Andrey]

[laughs]

00:01:55,480 --> 00:02:04,670 [Seb Krier]

And there are other tests, like if an AI makes $1 million or something, which I find is very weird because most humans do not make $1 million in the first place.

00:02:04,670 --> 00:02:05,080 [Andrey]

[laughs]

00:02:05,080 --> 00:02:11,359 [Seb Krier]

So the one I kind of like is actually Shane Legg’s definition-

00:02:11,360 --> 00:02:11,620 [Andrey]

Mm

00:02:11,620 --> 00:02:12,420 [Seb Krier]

... who’s at Deep Mind, who is

00:02:13,640 --> 00:02:16,980 [Seb Krier]

more of a capability-based definition, which is something along the lines of

00:02:18,420 --> 00:02:20,960 [Seb Krier]

an AI or a system that does most

00:02:22,380 --> 00:02:30,360 [Seb Krier]

standard cognitive tasks that people typically do. [lips smack] So it’s kind of the bar isn’t too low, and it’s also not too high either.

00:02:32,220 --> 00:02:35,480 [Seb Krier]

And so I think he’s got this definition of a minimal AGI,

00:02:36,580 --> 00:02:43,020 [Seb Krier]

and I think that we’re not exactly there yet. I would disagree with people saying that we have AGI today because I think

00:02:44,220 --> 00:02:48,900 [Seb Krier]

a lot of the systems we have, there’s many things that a human can do that they don’t really do very well.

00:02:48,900 --> 00:02:50,360 [Seth]

What’s the biggest gap that we’re missing?

00:02:52,020 --> 00:03:47,740 [Seb Krier]

I’d say there’s a few. One of them might be continual learning, or at least the ability to adapt and learn over time, and in different contexts and situations, just kind of update your own world model or whatever. If I think of a new joiner in a company, they’re not super useful the first day, but their value goes up over time because they learn all sorts of things. And so [lips smack] that might be one of them. A lot of the systems we have today, I think, are not very good at software, and you’re using graphical user interfaces and software and whatnot. If I ask an agent right now to go and use a music production software and make a track, I think they’d generally struggle. That doesn’t mean it’s impossible to solve or anything like that, but I think, in many respects, they’re not as general as you’d want them to be. And then the other bit also is, [lips smack] and of course they still make some silly mistakes here and there, but I think that’s getting it fixed. But the creativity point is one that I’m really interested in as well, in that I think they’re really good at kind of

00:03:48,780 --> 00:04:02,700 [Seb Krier]

exploiting maybe an existing paradigm or an existing knowledge and so on, and recombining knowledge and whatnot. But I think really coming up with new concepts and abstractions entirely is something I think humans can do, but I don’t see our current systems really doing either.

00:04:02,700 --> 00:04:10,060 [Andrey]

How do you measure whether humans can do creative tasks? One of the things that

00:04:11,200 --> 00:04:15,940 [Andrey]

strikes me as a bit of an unfair test in that,

00:04:17,060 --> 00:04:23,290 [Andrey]

let’s say you ask an LLM to write a poem or to write a story. It’s very-

00:04:23,290 --> 00:04:23,290 [Seth]

[laughs]

00:04:23,290 --> 00:04:32,050 [Andrey]

... times more entertaining than what a random human would write. So, do you have a benchmark for creativity?

00:04:32,050 --> 00:04:35,390 [Seth]

This is the meme where the robot asks Will Smith if he can compose an opera.

00:04:35,390 --> 00:05:14,700 [Seb Krier]

[laughs] Can you? Yeah, exactly. It depends, and you’re right. Obviously, most people aren’t creating new abstraction and concepts on a day-to-day level. But I imagine there’s still something qualitative about that kind of creativity that I think does get applied in everyone’s day-to-day life in various kind of ways. Maybe they’re not as big or significant as creating a symphony. But I don’t really have a strong test. There’s actually an interesting podcast that had Ben Goertzel and Yoshua, I think a few years ago, where they were saying something like, if you had a model that was trained knowing only classical music and West African drumming, could it come up with jazz in the first place, or recreate jazz?

00:05:16,460 --> 00:05:27,880 [Seb Krier]

And I quite like that test. And in principle, I can imagine it being possible. You could kind of decompose all sorts of different kind of elements and variables here and just get something jazz-like. But it still feels a bit...

00:05:29,580 --> 00:05:40,580 [Seb Krier]

It’s not the same as just coming up with the idea of jazz in the first place and saying, oh, I’m going to try these things out. And for whatever reason, I’m going to stick to that. And I don’t know. It’s-

00:05:40,580 --> 00:05:53,190 [Seth]

Recombination versus paradigm shifting. I’ve also heard one test people would want for AGI is, can you train the model on the 1900s corpus and it comes up with Einsteinian physics?

00:05:53,190 --> 00:05:53,200 [Seb Krier]

Yeah.

00:05:53,200 --> 00:05:54,720 [Seth]

That would be really impressive.

00:05:54,720 --> 00:06:36,151 [Seb Krier]

Yeah, I think actually Demis uses that test sometimes, or I think Pele Gritzer as well mentioned it before. And there are some people, I think David Duvenour and Nick Levine, I think, had this recent kind of language model talky that was trained up in, I think, the 1930s or something. And I tried to play around with it a lot. It was like, let’s try to get it to create something new, and it’s pretty tricky. Although they have apparently recently, some people kind of fine-tuned it on a very few examples of coding and gotten it to be good at coding. But for some reason, that doesn’t impress me maybe as much as other things I would’ve expected. It’s like [laughs] there’s the-I agree that the goalposts also kind of move a little bit over time, and it’s also maybe unfair of me. It’s like, oh, well, can it create a new programming language from scratch or something?

00:06:37,272 --> 00:06:43,052 [Seb Krier]

So it’s a tricky one to kind of square off, but it does still feel like there’s a lack of that kind of true creativity, at least in my

00:06:44,212 --> 00:06:45,072 [Seb Krier]

interactions with them.

00:06:46,392 --> 00:06:57,342 [Andrey]

I am really worried that it is a goalpost moving exercise here. We don’t have a benchmark for creativity and therefore,

00:06:58,432 --> 00:07:03,211 [Andrey]

all these claims are not quantitative in a way that I’d like. And let-

00:07:03,212 --> 00:07:10,612 [Seth]

Right. What about all those IS papers we see where one of the axes is creativity and we instrument for something? [laughs]

00:07:10,612 --> 00:07:11,032 [Andrey]

Yes.

00:07:13,132 --> 00:07:13,592 [Seth]

There’s a lot of bad measures of creativity.

00:07:13,592 --> 00:07:19,762 [Andrey]

Those are not creative, to be clear. I’m sure I’ve offended a ton of people. Sorry.

00:07:19,762 --> 00:07:20,992 [Seth]

It’s okay.

00:07:20,992 --> 00:07:56,432 [Seb Krier]

I think it’s fair. I agree that it’s a bit like... But I still feel like there’s, at least if part of the reason you’re going to create these systems is to come up with kind of also new sorts of theories and so on. And I think you can probably get that through good search and a lot of inference compute and trying out lots of different things. And I think there are many low-hanging fruits there, to be clear. So it’s not like I think, oh, we’ve hit some sort of wall or something. And I think there’s a lot that you can kind of get in terms of new knowledge and new creative knowledge from that. But I feel like there’s maybe something more needed. It’s maybe not that kind of magical or anything, right? Maybe you just need better scaffolding or better multi-agent systems. But

00:07:58,992 --> 00:08:02,072 [Seb Krier]

yeah, at least so far, I would say that I see a bit more creativity, say, in

00:08:03,652 --> 00:08:11,612 [Seb Krier]

humans so far as a collective. And maybe that’s, again, an unfair comparison. You don’t have a culture of AIs and AGIs to compare that against. So-

00:08:11,612 --> 00:08:11,682 [Andrey]

Yeah

00:08:11,682 --> 00:08:15,092 [Seb Krier]

... the right comparison is also a hard one to do.

00:08:15,092 --> 00:08:52,772 [Andrey]

So, you mentioned scaffolding, and I guess a question, you recently wrote about a defense of scaffolding, and I think just to frame things, some people you talk with, especially very AGI-pilled people, are like, “Scaffolding, it’s an epiphenomenon. It doesn’t matter. In the end, we are going to train a smarter model with more parameters and more training data, and it’s just going to do it out of the box. And so all these scaffolding hacks are just very temporary.” And then other people like yourself, I guess, argue the opposite. So what do you think about scaffolding?

00:08:54,832 --> 00:08:55,052 [Seb Krier]

Yeah.

00:08:56,572 --> 00:08:59,372 [Seb Krier]

The first thing is I’m definitely not sure. This is kind of

00:09:00,532 --> 00:09:39,672 [Seb Krier]

one of many hot takes, but I think, I guess there are a few reasons why I see it as, I think it’s going to stay over time. The first is that I think it’s plausible that as, I think scaling laws continue, I think you scale models and they get better over time and so on, but I think the inputs are expensive and grow over time. And I also think that it’s plausible that you might get more and more diminishing returns over time. And if that’s the case, I see the kind of utility of the scaffolding side and the harnesses as going up because you’re going to want to make more, you’ll want more bang for your buck kind of thing. You’re going to want to extract this intelligence and use this resource as efficiently as possible.

00:09:40,772 --> 00:09:51,532 [Seb Krier]

So that’s maybe one reason. The other one is a bit more, I guess, Hayekian in nature or something, in that I see a lot of, I think there’s a lot of local knowledge, a lot of

00:09:53,212 --> 00:10:18,592 [Seb Krier]

stuff that isn’t necessarily kind of codified. And I don’t really see one big giant AGI model now kind of perfectly guessing everything forever at infinite scales. And in a way, I see this as a little bit like a division of labor in that I think it’s actually more efficient to have this kind of integration layer that is closer to the local information or to the ground or to demand side that can better integrate this kind of cognitive resource

00:10:19,812 --> 00:10:23,632 [Seb Krier]

to satisfy and create value and satisfy whatever consumers and businesses want.

00:10:25,552 --> 00:10:31,352 [Seb Krier]

So to help with all the sorts of constraints and the context they’re dealing with, I think it’s very useful to have that.

00:10:33,712 --> 00:10:39,112 [Seb Krier]

Of course, I don’t think this necessarily also implies or means that you’re going to get complete, full decentralization or something.

00:10:40,772 --> 00:10:42,212 [Seb Krier]

Walmart gets huge

00:10:43,872 --> 00:10:48,872 [Seb Krier]

returns from the scale that they have, and you don’t have loads of businesses downstream kind of reselling their stuff.

00:10:51,252 --> 00:10:53,932 [Seb Krier]

But there’s two things. The first is that-

00:10:53,932 --> 00:10:56,812 [Seth]

We have bodegas reselling stuff from Walmart on the corner.

00:10:56,812 --> 00:11:18,992 [Seb Krier]

Actually, that’s a good point, yeah. And also, there are all sorts of other businesses kind of selling different things, right? If the task is generic and the demand is homogenous, then sure, maybe you can do more of that. But also, even Walmart relies on all sorts of kind of suppliers, local labor, compliance system, inventory systems, third parties, and whatnot, that help with this kind of integration and the delivery of these services.

00:11:18,992 --> 00:11:25,862 [Seth]

So if I may summarize your answer, you’re very Hayek-pilled, but maybe not as Bitterlesson-pilled as most.

00:11:25,862 --> 00:11:25,972 [Seb Krier]

Well,

00:11:27,212 --> 00:11:31,052 [Seb Krier]

I think I’m definitely Bitterlesson-pilled in the sense that I don’t think you should

00:11:33,652 --> 00:11:48,992 [Seb Krier]

try to kind of cement some sort of rules-based system you either devise or something and kind of hope that this just takes forever. If anything, I think the scaffold needs to be a lot more adaptive and evolve over time. In the same way as if you have a small startup and they have all sorts of kind of rules and,

00:11:50,332 --> 00:12:02,772 [Seb Krier]

sorry, not rules, different functions. When the startup grows and gets more capabilities, they also kind of change from the inside. So I think that, of course, if you have some sort of light GPT-type wrapper that kind of makes your system a little bit better, whatever, yeah, that was not going to

00:12:03,812 --> 00:12:23,652 [Seb Krier]

work out over time. But I think there are kind of scaffolds that help better integrate the wider environment, private data, deals with permissions or liability regimes or user preferences and whatnot. And also, at a somewhat higher level, kind of more coordination-type scaffolds maybe in terms of market interfaces, like clearing house equivalents or something.

00:12:24,516 --> 00:12:33,536 [Seth]

The third example you gave is maybe it’s not the super frontier model that are going to these scaffolds, but simpler models that are still very useful and cheaper to run with a scaffold.

00:12:33,536 --> 00:12:46,176 [Seb Krier]

Yeah, totally. Because I think you’re not going to need the enormous, super expensive brain for every single random task. And so it’ll make, for most kind of basic queries, people aren’t using Opus’s latent space or something as-

00:12:46,176 --> 00:12:46,186 [Seth]

[laughing]

00:12:46,186 --> 00:12:48,236 [Seb Krier]

... it’s a big waste in some sense.

00:12:48,236 --> 00:12:50,036 [Seth]

What toothbrush should I buy? [chuckles]

00:12:50,036 --> 00:12:51,196 [Seb Krier]

Yeah. Exactly.

00:12:51,196 --> 00:12:53,896 [Andrey]

Wait. That is an important question, Seth.

00:12:53,896 --> 00:12:54,516 [Seb Krier]

I mean-

00:12:54,516 --> 00:12:56,536 [Andrey]

I would definitely use Opus for that.

00:12:56,536 --> 00:12:57,385 [Seb Krier]

It’s funny because I’ve actually-

00:12:57,385 --> 00:12:59,696 [Seth]

Use all the collective intelligence of reality. [chuckles]

00:12:59,696 --> 00:13:02,266 [Seb Krier]

I have actually used Opus for that exact question not long ago-

00:13:02,266 --> 00:13:02,626 [Seth]

[laughing]

00:13:02,626 --> 00:13:06,256 [Seb Krier]

... in trying out this new electric toothbrush that I found out as a result. But,

00:13:07,636 --> 00:13:22,076 [Seb Krier]

so yeah, I agree there’s that and also there’s all sorts of ways in which actually kind of using tools or specialized kind of tools is just more effective and more efficient. Why would you expect a large model or something to kind of calculate things innately or something when you can just access a calculator? It’s a much better use of tokens.

00:13:22,076 --> 00:13:36,856 [Andrey]

But it should kind of know that the calculator is available and then use it when it’s there. So that’s the argument against scaffolding, or you’re giving it a general environment, but you’re not scaffolding it much. I think a curious thing is just,

00:13:38,376 --> 00:13:40,356 [Andrey]

it seems like most people who are using

00:13:41,416 --> 00:13:49,156 [Andrey]

scaffolded agents today are using them with essentially one of two scaffolds, with Cloud Code or Codex. And

00:13:50,236 --> 00:14:00,475 [Andrey]

those seem to be good enough maybe. I guess, do we see a lot of people customizing, a lot of people, whatever, companies customizing their scaffolds?

00:14:00,476 --> 00:14:03,856 [Seth]

CladBot, do the CladBots count as that, I guess?

00:14:03,856 --> 00:14:04,236 [Andrey]

Yeah.

00:14:05,396 --> 00:14:39,676 [Seb Krier]

They are a form of it. I don’t know. I think a lot of power users and people in our immediate communities use a lot of Cloud Code and Codex, and particularly software engineers. But I don’t think most legal departments and most kind of firms out there are necessarily using Cloud Code either. And it’s not clear to me that this is necessarily the optimal interface or, there may be better systems that are Cloud Code-like, or CLI-like perhaps in some way. But, so I don’t know, maybe they’re sufficient, but even these tools end up kind of calling on loads of other external APIs and tools and so on in how they

00:14:40,836 --> 00:14:57,576 [Seb Krier]

function. So if anything, these are actually scaffolds. You’re not kind of calling the model directly. There’s all sorts of different sub-agents behind the scenes. It’s not just a one-shot call. There’s quite a lot going on, which is in fact this more, I don’t know, dynamic scaffolding thing I was mentioning earlier, I guess.

00:14:58,976 --> 00:15:06,736 [Andrey]

Okay. The natural question here is, what is going to be the role of the market in coordinating-

00:15:06,736 --> 00:15:07,375 [Seb Krier]

Mm

00:15:07,375 --> 00:15:11,276 [Andrey]

... AI here? And I’ll just very shamelessly plug-

00:15:11,276 --> 00:15:11,285 [Seb Krier]

[chuckles]

00:15:11,285 --> 00:15:24,796 [Andrey]

... some recent work with Rohit Krishnan, where we’re kind of playing around with the idea of LLMs bidding in a procurement auction and seeing whether that results in more efficient use of AI.

00:15:26,696 --> 00:15:29,655 [Seb Krier]

Well, first of all, I need to properly read that again. But the-

00:15:29,655 --> 00:15:30,476 [Andrey]

[laughing]

00:15:30,476 --> 00:15:31,016 [Seb Krier]

In terms of,

00:15:32,496 --> 00:15:32,916 [Seb Krier]

I guess,

00:15:34,556 --> 00:15:46,396 [Seb Krier]

at a very high level, markets are good at just coordinating in general, including AI. And so, assuming they function as intended in it, you’ve got the pricing mechanism to get...

00:15:47,556 --> 00:15:49,396 [Seb Krier]

I don’t know. I expect that to kind of work as well with

00:15:50,476 --> 00:15:52,616 [Seb Krier]

matching, I guess, supply and demand or something.

00:15:54,016 --> 00:15:55,196 [Seb Krier]

The supply of this

00:15:56,216 --> 00:16:00,036 [Seb Krier]

raw resource of cognition or something, and the demand of all sorts of different businesses and users.

00:16:01,696 --> 00:16:05,516 [Seb Krier]

So maybe, at a very high level, I don’t know. What exactly do you mean by the role of the market or something here?

00:16:09,076 --> 00:16:21,356 [Andrey]

Obviously the market is involved in many parts of the AI vertical supply chain, right? From competition in chips. There’s competition between models. There might be also competition between

00:16:22,516 --> 00:16:28,576 [Andrey]

scaffolds, bundles of environments, scaffolds, and LLMs.

00:16:28,576 --> 00:17:06,496 [Seth]

I guess maybe it would be useful to juxtapose this versus, so what Andrey, one of the things he’s imagining is, I have a job. I post it to some sort of Upwork-like future platform. Different companies that host different AI models bid to do that job. “Oh, I think I can do that job with $1 of electricity and tokens,” versus another model, and then we get efficient allocation of intellectual tasks to models, right? So do we think that that’s going to be important, or is it going to be more like I ask the super model what the best model is, and I just get allocated in a non-market way? Might be one version of this question.

00:17:08,156 --> 00:17:18,836 [Seb Krier]

I guess intuitively, my mind goes to the former question. But, or there’s a little bit of both in some sense, because even in the former one, you’re going to be using the large model for some sort of

00:17:20,436 --> 00:17:26,686 [Seb Krier]

cognitively demanding task or something. It kind of depends what kind of quality of output you also need and want.

00:17:26,686 --> 00:17:26,706 [Seth]

[chuckles]

00:17:26,706 --> 00:17:27,056 [Seb Krier]

But then

00:17:28,376 --> 00:17:49,636 [Seb Krier]

you’re still going to be constrained by your own resources or something, and depending on what you have to spend, if you can get the output for cheaper by kind of relying on this kind of competitive marketplace of smaller models or something, not even smaller models, they might just be all be big and kind of just scaffolding different, you’re offering a slightly different thing. Why wouldn’t you go for that, and why wouldn’t that exist in the first place? Unless the very first-

00:17:49,636 --> 00:17:52,216 [Andrey]

Doesn’t exist yet, just to be clear.

00:17:52,216 --> 00:17:52,716 [Seb Krier]

Um-

00:17:52,716 --> 00:17:58,416 [Seth]

A, it doesn’t exist yet, and as Andrey proves, at least current models are bad at understanding their own capabilities.

00:17:58,416 --> 00:17:58,666 [Andrey]

Oh, yeah.

00:17:58,666 --> 00:18:00,496 [Seth]

Now maybe that’s going to be fixed.

00:18:00,496 --> 00:18:08,096 [Seb Krier]

Yeah. Oh, no, I agree. I think that we’re not there yet, right? I think, again, and that goes back to the earlier AGI question, is there’s all sorts of, then again, what’s the right comparator? But,

00:18:09,476 --> 00:18:21,316 [Seb Krier]

yeah, I don’t think we’re exactly there. Yeah, I think a lot of this will have to be built as well. The kind of an ability for a model to just better kind of operate in a more multi-agent environment, kind of have a better sense of

00:18:22,596 --> 00:18:32,556 [Seb Krier]

delegation. I think the kind of, yeah, industrial intelligence or something seems to be maybe more neglected, as opposed to just single-agent intelligence or something, if that makes sense.

00:18:32,556 --> 00:18:34,776 [Seth]

Do we need to bring the word cybernetics back?

00:18:34,776 --> 00:18:35,496 [Seb Krier]

Yeah.

00:18:35,496 --> 00:18:36,116 [Andrey]

[laughs]

00:18:36,116 --> 00:18:38,816 [Seb Krier]

Somewhat. [laughs]

00:18:40,756 --> 00:18:51,256 [Andrey]

All right. A little change in subject, but I know this has been in the discourse, the topic of recursive self-improvement, RSI.

00:18:51,256 --> 00:18:52,956 [Seth]

Ooh, very scary.

00:18:52,956 --> 00:18:54,896 [Andrey]

Jack Clark recently had an essay about it.

00:18:56,376 --> 00:18:58,876 [Andrey]

Seb, what is your take?

00:18:58,876 --> 00:18:59,206 [Seb Krier]

[chuckles]

00:19:00,316 --> 00:19:07,896 [Seb Krier]

What is my take? I don’t know. I think it depends what exactly we mean by recursive self-improvement.

00:19:09,096 --> 00:19:50,336 [Seb Krier]

I had a blog post not long ago, I guess, when trying to disentangle a little bit what I have in mind when I think about this. On the one hand, there’s the model getting recursively better through the usage of more AI and whatnot. And on the other hand, there’s the more kind of societal side of things, the transformation side, which I think very often, these two worlds are a little bit blurred in the discourse. It’s like, oh, you get RSI, and then X, Y, Z about the world or something. Things go really fast or they don’t go fast. And, I think these should be separated very neatly because on the model side, of course, I expect, already there’s a lot of AI being used everywhere to kind of create models. And I expect that to continue.

00:19:52,536 --> 00:19:55,976 [Seb Krier]

But it’s not clear to me that this necessarily now leads to a dynamic by which

00:19:57,156 --> 00:20:16,596 [Seb Krier]

the model now gets extremely or exponentially intelligent in a very short amount of time. It’s still kind of bottlenecked by all sorts of resources. And as I was saying earlier, I still see them as better at kind of paradigm exploitation than kind of exploration, which I think is the thing you might need to get to the next step. But, first of all, what do I know? But secondly,

00:20:17,616 --> 00:20:19,986 [Seb Krier]

the other thing is, yeah, on the societal side of things,

00:20:20,996 --> 00:20:29,756 [Seb Krier]

people sometimes talk about foom or hard takeoffs and whatnot, and these have very clear kind of real-life implications. It’s not just kind of a model of getting better in a

00:20:31,216 --> 00:20:34,576 [Seb Krier]

data center somewhere. And that side, I think, is where you have to think about

00:20:36,116 --> 00:21:27,056 [Seb Krier]

[lip smack] all the kind of usual bottlenecks, adoption, deployment, diffusion, the kind of productive integration of all these systems at scale, both in terms of manufacturing and so on and so forth. And, I guess it’s not clear to me that the shift from GPT-2 to GPT-3 or coming up with kind of, we’re just very classic kind of software engineering, meat and potatoes type tasks that you can just easily just automate away. It’s maybe one of these things that’s maybe easy to say ex post, but, I’m not sure. And certainly, my expectation is you’re going to get loads of gains in the coming years of kind of automating part of that pipeline. But that seems good. You just get better models, and that’s just overall helpful for all sorts of other things, even if you’re doing safety work and kind of governance work and whatnot, we benefit a lot from that cognitive resource, I guess.

00:21:27,056 --> 00:21:40,696 [Andrey]

What would happen in the world for you to change your mind? Is there any, let’s say that recursive self-improvement is actually kind of this much more profound change than you’re painting.

00:21:41,816 --> 00:21:42,036 [Andrey]

What

00:21:44,136 --> 00:21:45,696 [Andrey]

signs would there be, I guess? Yeah.

00:21:45,696 --> 00:21:51,656 [Seb Krier]

But to be clear, I’m not claiming it’s just business as usual, nothing to see here or whatever, right? I’m

00:21:52,796 --> 00:22:14,936 [Seb Krier]

kind of just claiming that some of the stronger versions of the claim aren’t kind of self-evident. And so I see a lot of this happening in some sense. Certainly, in 10 years, I expect to have larger kind of more, again, acceleration of economic growth and whatnot and kind of faster diffusion across the board. I certainly don’t expect diffusion to take the same amount of time as, say, electricity or these other technologies.

00:22:16,576 --> 00:22:23,236 [Seb Krier]

So it depends what exactly you mean, because what specifically am I looking to change my mind on?

00:22:23,296 --> 00:22:30,656 [Andrey]

Well, let’s say the scenarios of AI 2027, right? Presumably,

00:22:31,996 --> 00:22:45,176 [Andrey]

in 2027, you’ll see something that’s like, “Oh, wow, I was wrong. This is not going to be so gradual. This is going to be this sudden foom,” that you’re criticizing. Yeah.

00:22:45,176 --> 00:22:52,236 [Seb Krier]

The original foom or hard takeoff definition literally talks about this change happening within hours or days.

00:22:52,236 --> 00:22:53,236 [Andrey]

[chuckles]

00:22:53,236 --> 00:22:56,056 [Seb Krier]

Which is not even, it’s not what the 2027 scenario, I think, predicts.

00:22:56,056 --> 00:22:56,296 [Andrey]

Yes.

00:22:57,556 --> 00:23:00,446 [Seb Krier]

But the 2027 scenario, from what I remember, again, it’s been a bit of time now.

00:23:01,796 --> 00:23:08,816 [Seb Krier]

One thing with the scenarios there is that there’s the kind of misalignment assumption, and which I’m kind of uncertain about.

00:23:08,816 --> 00:23:09,255 [Andrey]

Mm.

00:23:09,256 --> 00:23:17,296 [Seb Krier]

And it also talks about a lot of progress in robotics, which I think is a bit further away. I think it’s close. We’re getting there, too.

00:23:19,116 --> 00:23:19,476 [Seb Krier]

But

00:23:21,156 --> 00:23:25,916 [Seb Krier]

I don’t know. Probably kind of AI, if in 2030, we start seeing AI is making all sorts of crazy

00:23:26,956 --> 00:24:06,196 [Seb Krier]

inventions, innovations in fields other than just kind of perhaps math and coding across the boards, and I’m like, okay, this is clearly-- And you get extremely fast adoption, too, right? You have entire businesses doing completely, it’s not business as usual, clearly, in the economy or something and wide adoption. But it’s hard to say because I expect all that to some degree, right? It’s not that I’m saying, “Oh, this is never going to happen.” I just think of it as a little bit more elongated and the implications of that being maybe not as like, we have Dyson spheres in five years or something like that, so. It’s more of a disagreement maybe on the extremes or the margins or something, but not so much at the core of the claim that yes, models are going to make models better and...

00:24:07,276 --> 00:24:27,536 [Seb Krier]

But, again, even having-- In fact, actually, here would be a thing. If Anthropic or DeepMind or something in 2037 have fewer and fewer employees, fewer people kind of just doing AI research, engineers and so on, you’re clearly seeing kind of that profession. Because of course, I can imagine these jobs to change, right? Maybe you’re kind of managing more agents or something. That

00:24:28,616 --> 00:24:35,966 [Seb Krier]

I expect. But the fact that you just need far fewer people to kind of do not only these large training runs, but the kind of

00:24:36,976 --> 00:24:43,476 [Seb Krier]

large training runs that give you just much, much better systems, then I think I’d be like, okay, this is going a little bit faster than maybe expected or something.

00:24:44,656 --> 00:24:51,676 [Andrey]

Okay. One thing you mentioned in that kind of hints at another hot take you have, which is about alignment.

00:24:51,676 --> 00:24:52,026 [Seb Krier]

Uh-huh.

00:24:54,596 --> 00:24:55,926 [Andrey]

What’s the deal with alignment?

00:24:57,196 --> 00:24:58,086 [Andrey]

[laughs]

00:24:58,086 --> 00:24:58,136 [Seb Krier]

[laughs]

00:24:58,136 --> 00:25:02,136 [Seth]

Is it hard? Is it easy? Is it different than we would’ve expected going in?

00:25:02,136 --> 00:25:19,646 [Seb Krier]

Yeah. It’s perhaps that. I think my take about alignment is something-- Well, first of all, I just don’t like the word. I think it’s a bit of an annoying word because it’s being used for all sorts of things. The AI says something that we just kind of don’t like, or you say, “Oh, it’s misaligned.” No one pre-registers what they expect the aligned behavior to be, and then just kind of tests.

00:25:19,646 --> 00:25:20,116 [Andrey]

[laughs]

00:25:20,116 --> 00:25:35,626 [Seb Krier]

But I think my general claim is maybe the fact that it’s been easier than we would’ve predicted a decade ago or so. Then when I first got into AI in 2017, that was partly as a result of reading things like “Superintelligence” by Bostrom.

00:25:35,626 --> 00:25:36,236 [Andrey]

Mm-hmm.

00:25:36,236 --> 00:25:48,496 [Seb Krier]

And you’d read these books, like Stuart Russell’s “Human Compatible” and others, that kind of had all these analogies like King Midas and you ask a system to optimize for goal X, and in pursuit of that goal, it does all sorts of other things that you don’t want it to do.

00:25:48,496 --> 00:25:51,916 [Seth]

Right. The paperclip maximizer, and we seem to not have those.

00:25:51,916 --> 00:25:57,476 [Seb Krier]

Yeah. It’s like one version of it or one variant of it. And certainly at the time you didn’t really have language models. A lot of these intuitions were kind of based off

00:25:58,596 --> 00:26:48,236 [Seb Krier]

reinforcement learning systems in very basic kind of game scenarios where they were actually given a single goal to optimize for. And this is not actually what we do, I think, with models. And you had these kind of examples, even the value loading problem was something discussed at the time where actually specifying these complicated nuanced human values in mathematical terms would be extremely hard. So even if you managed to tell a robot to clean the room, it would then just pick up a baby and put it in the trash or something. And I think it turns out a lot of this stuff is actually much easier. You have problems. You’ve got things like reward hacking. You’ve got AIs behaving in weird ways that we were not always kind of anticipating because of the ways they were post-trained. So my claim is not like, oh, again, it’s all fine, and safety is a scam or whatever. It’s more that it’s certainly much easier than, or at least we’re in a much better track than I would’ve at least guessed perhaps a decade ago. And secondly, I think it

00:26:49,916 --> 00:26:54,816 [Seb Krier]

just seems tractable. There’s a lot of progress in terms of chain-of-thought monitoring and all these other things. And

00:26:56,696 --> 00:26:57,796 [Seb Krier]

I also think that the

00:26:59,016 --> 00:27:05,825 [Seb Krier]

hard part is maybe more the kind of normative question of whose values and when, and what and everything. That’s the kind of thing that we’re looking into more. But

00:27:07,096 --> 00:27:13,696 [Seb Krier]

yeah, I prefer the word actually instruction following or intent following or something instead of alignment. And I think by and large, they’re actually pretty good at that.

00:27:14,796 --> 00:27:31,636 [Seb Krier]

So again, that doesn’t mean you have to dismiss all sorts of theories and all the kind of power optimization stuff. But I guess my immediate outcome is this goes rather well. Or if I am more concerned by other things like misuse, if you’d like, than kind of the AI’s being innately, inherently kind of internally misaligned.

00:27:31,636 --> 00:28:03,676 [Seth]

This really seems related to your take that intelligence is not at odds with being a tool, right? So a lot of people have this intuition where if you had a super-duper intelligent genie or oracle, it would develop even implicitly some sort of value or goal that orthogonality thesis might have nothing to do with what we want. But you’re more optimistic about the idea that the LLM doesn’t want anything. It’s incorrect to take the intentional stance towards an LLM.

00:28:03,676 --> 00:28:09,236 [Seb Krier]

Not incorrect. It’s actually kind of descriptively useful, even functionally sometimes to use that language.

00:28:10,796 --> 00:28:18,836 [Seb Krier]

But that’s the thing, right? I think we kind of lack the language to properly delineate and differentiate when it’s useful to use that or appropriately descriptive and when it’s not.

00:28:20,076 --> 00:28:41,496 [Seb Krier]

And so I agree that, of course, I think the take I had on this was something like, and I can imagine a tool being an agent and an agent being a tool. Or in principle, I can imagine something being hyper-capable and still being broadly instruction following rather than at a certain level of capability, aha, that’s when the goals change and things get... And it kind of depends on the type of system as well. I imagine not all

00:28:42,656 --> 00:28:45,116 [Seb Krier]

paths lead to the same kind of outcome. But,

00:28:46,256 --> 00:29:13,596 [Seb Krier]

so again, I can see plausible versions of the world where homo hundrio drives or something are a more salient feature of the way we kind of train models. Right now, it doesn’t seem to me very likely that this is a core feature that they have. But of course, it’s hard to kind of either prove or disprove, right? Because someone might just say, well, that’s because they’re very good at hiding this or something, or once they’re capable enough or whatever. So there’s always a bit of this kind of gotcha thing. It’s like deception. But

00:29:14,936 --> 00:29:39,896 [Seb Krier]

yeah. So in principle, I guess I can totally conceive of at least a superintelligence that is controllable, that is benign, that is at least subservient to the goals of humanity or a user or principle or whatever. That could still be used to cause enormous harm, but it’s just I don’t necessarily think the analogies of, oh, I think Tegmark was thinking, look at the zoo where the monkey’s going. I think these are just not really

00:29:41,736 --> 00:29:43,136 [Seb Krier]

helpful kind of analogies.

00:29:44,276 --> 00:30:02,396 [Seth]

Monkey at the zoo, but you’ve also got the monkey’s paw, right? Maybe the reason some prefer alignment to instruction following is we all know the story of, be careful what you wish for. You wish for something, and it’s under-specified, and you get the bad version of it because the AI doesn’t understand the context.

00:30:02,396 --> 00:30:08,336 [Seb Krier]

I think that’s why, yeah, I think maybe instruction following is maybe too... Intent following or something gets to it more.

00:30:09,936 --> 00:30:18,316 [Seb Krier]

But of course, that problem doesn’t go, even if it follows intent or something, you could still have all the problems because your intent is nefarious or whatever. So

00:30:19,436 --> 00:30:19,816 [Seb Krier]

I think the

00:30:21,356 --> 00:31:06,756 [Seb Krier]

way you deal with that is all sorts of, I don’t know how to conceptualize it, but in fact scaffolds. It’s a bit more this outside of the model or something. I’m kind of almost indexing on a world that will indeed have agents that are trained to be bad or whatever, or someone going to be instructed to do bad things. But just like with humans, you come up with all sorts of kind of systems, rules, laws, norms, kind of protocols that either discourage the kind of bad behavior, or punishes it, or makes it just not worthwhile or something. But I’m not going to put all my bets on the, oh, it has to be pure-hearted, and that will be sufficient. And then you just scale it forever, and it’s going to be an amazing goal. I just think that the way of seeing or thinking about AI is that I just find kind of a bit

00:31:08,096 --> 00:31:12,656 [Seb Krier]

too narrow, I guess. I think it’s important, it’s just insufficient, and it’s certainly not my main kind of a-- yeah.

00:31:14,946 --> 00:31:15,206 [Andrey]

Okay.

00:31:16,666 --> 00:31:20,086 [Andrey]

Our audience is very much composed of economists.

00:31:22,586 --> 00:31:30,506 [Andrey]

If you’re an economist and you’re very interested in AI, what sort of work would you be trying to do?

00:31:30,506 --> 00:31:32,146 [Seth]

Maybe to be useful to AI people-

00:31:32,146 --> 00:31:32,216 [Andrey]

Yes

00:31:32,216 --> 00:31:37,466 [Seth]

... in particular. What would you want, what did the DeepMind team want to read from economists?

00:31:37,466 --> 00:32:20,766 [Seb Krier]

I think kind of engaging with their assumptions or something, right? If you assume, let’s say, an AG-- and I think some do, to be fair. I actually think there’s a lot more, I think, discourse now going on between economists and AI people, whatever. But assuming that you do have AI systems that are interchangeable or almost quasi-fully substitutable with humans, that come up with good ideas, that are parallelizable and whatnot, what does that change to your kind of growth function and so on? So, maybe that’s useful. Right now, in the short term, at least, there’s all sorts of questions around labor, there’s questions around productivity or adoption. Clearly, there’s useful work to be done there. But I think in terms of AGI specifically, given that a lot of the field just thinks you’re going to get to AGI in the next five to 10 years,

00:32:22,746 --> 00:32:26,806 [Seb Krier]

what are the implications for taxation? What are the implications for

00:32:28,626 --> 00:32:37,786 [Seb Krier]

how that’ll affect different states across the world? I think I’m probably more worried about a call center in Hyderabad than I am about the white-collar worker in North America or something. So,

00:32:39,066 --> 00:32:57,306 [Seb Krier]

yeah. I think all these kind of questions, but just indexing more and making fewer, I guess, assumptions around the limits of capabilities. Because sometimes you see them kind of being implicitly snuck in somewhere or something of like, well, because AIs can’t do XYZ, therefore... And yeah, fine, but maybe they will do XYZ. And then what? How does that change your thinking? Yeah.

00:32:57,306 --> 00:32:59,506 [Seth]

Maybe more scenario planning than,

00:33:00,526 --> 00:33:04,746 [Seth]

here’s my median projection, or here is one projection I think is plausible.

00:33:04,746 --> 00:33:22,846 [Seb Krier]

Yeah. And embedding the kind of thoughtful models and thinking that economists have within these scenarios and making them more salient to the kind of computer scientists, right? Even when I brought up competitive advantage, people will be like, “Oh, but what if the AI is cheaper and better?” It’s like, well, that’s not the point. The opportunity cost point of competitive advantage, there’s a difference.

00:33:22,846 --> 00:33:23,286 [Andrey]

[laughs]

00:33:23,286 --> 00:33:31,786 [Seb Krier]

And again, there are answers to that as well, but I think just kind of better translating, I think, some of these insights to the AI tribe, the thing is useful.

00:33:32,846 --> 00:33:40,526 [Andrey]

So that’s very naturally leading us to this question about yourself. And you do lots of different things.

00:33:41,946 --> 00:33:50,426 [Andrey]

You’re prolific on Twitter, for sure. But also, you’re doing internal work for DeepMind. How do you allocate your time?

00:33:52,066 --> 00:33:52,166 [Seb Krier]

I don’t know.

00:33:52,166 --> 00:33:53,266 [Seth]

What percentage is Twitter?

00:33:53,266 --> 00:33:54,646 [Andrey]

Yeah. [laughs]

00:33:54,646 --> 00:34:04,686 [Seb Krier]

Twitter is actually not that much today. It must be an hour max or something, an hour and a half, two hours, maybe, something. But that is maybe much by others’ standards. But the-

00:34:04,686 --> 00:34:06,476 [Andrey]

[laughs] What is the optimal amount of Twitter? [laughs]

00:34:06,476 --> 00:34:29,866 [Seb Krier]

[laughs] Yeah. It’s the Pareto optimal. I guess, in my day-to-day work, it’s a mixture of proactive and reactive. Proactive in the sense that I think, oh, these questions of agents and cybersecurity and liability and whatnot, and biosecurity are kind of important things to look into, and therefore, there’s a lot of research that I do and colleagues do, and a lot of coordination across the org.

00:34:31,026 --> 00:34:39,486 [Seb Krier]

But there’s also more reactive stuff because we’re a policy team, and so there’s things happening in the external world like CA 53, the preemption debates.

00:34:40,546 --> 00:34:48,386 [Seb Krier]

So it’s a bit of a mix of that. And of course, all sorts of internal dynamics. But, yeah. I guess I’m curious about all sorts of other things, and so when I do have time, and I’ve kind of

00:34:50,006 --> 00:34:58,106 [Seb Krier]

completed the main quests, I try to keep some time for other stuff I’m interested in. I work with some research teams and kind of look into what they’re into. I’ll

00:34:59,266 --> 00:35:09,826 [Seb Krier]

find topics or themes that I think are maybe kind of neglected or underrated or I just don’t see out there as much, and like, “Oh, cool. We’re going to try to find out about this more.” But I think it’s just very kind of curiosity driven, and the allocation of time is

00:35:11,566 --> 00:35:16,705 [Seb Krier]

not super thought out. It’s more like, oh, I think these things are interesting, and I’m going to get into that for a bit. [laughs]

00:35:16,706 --> 00:35:22,306 [Andrey]

So it wasn’t a deliberate strategy of getting Tyler’s attention and adoration. [laughs]

00:35:22,306 --> 00:35:25,126 [Seb Krier]

No, not at all. Not at all. But I’m very-

00:35:25,126 --> 00:35:25,746 [Seth]

The long play

00:35:25,746 --> 00:35:30,565 [Seb Krier]

... very grateful for his... [laughs] For the meme. But-

00:35:30,566 --> 00:35:41,766 [Seth]

What kind of, but I know you can’t be specific, but for your sort of internal work, what does a work product look like? Are you participating in a meeting and giving hot takes? Are you writing internal memos? What is-

00:35:41,766 --> 00:35:42,026 [Seb Krier]

Yeah

00:35:42,026 --> 00:35:42,276 [Seth]

... in-

00:35:42,276 --> 00:35:56,406 [Seb Krier]

It’s a mixture. Obviously, meetings. Any large bureaucracy will have meetings. But I think a lot of analysis, memos to execs sometimes. Just research, managing researchers sometimes, depending on the project.

00:35:57,626 --> 00:36:04,106 [Seb Krier]

We’ll have a lot of coordination. Actually, I’m realizing through a lot of these kind of meetings, a lot of it is just kind of coordination and information transfer, right?

00:36:04,106 --> 00:36:04,146 [Andrey]

[laughs]

00:36:04,146 --> 00:36:07,006 [Seb Krier]

It’s maybe why I’m so obsessed with the Coasean bargaining thing. Just let-

00:36:07,006 --> 00:36:07,326 [Seth]

Ah

00:36:07,326 --> 00:36:08,546 [Seb Krier]

... the agents do it. But,

00:36:09,806 --> 00:36:34,116 [Seb Krier]

yeah. I think the day-to-day work is a lot of reading, a lot of meetings, a lot of writing, and distilling and translating information, I think, across different tribes also. So if I’m talking to legal people, like lawyers, about what’s going on in, say, the more technical side of the org, or if I’m speaking to the researchers about something that’s more... But yeah, there’s a lot of translating of concepts across different stakeholders, I guess.

00:36:34,116 --> 00:36:45,726 [Andrey]

So how does that work in an org like Google? Because I think in a lot of orgs, they’re really obsessed with KPIs and output metrics.

00:36:45,726 --> 00:36:46,156 [Seb Krier]

Mm-hmm.

00:36:46,156 --> 00:36:48,746 [Andrey]

And what you’re describing sounds very-

00:36:48,746 --> 00:36:49,706 [Seth]

Hot takes per meeting. [laughs]

00:36:49,706 --> 00:36:54,926 [Andrey]

Yeah. Very much amorphous, very hard to measure.

00:36:56,066 --> 00:36:56,196 [Seb Krier]

Yeah.

00:36:56,196 --> 00:37:00,606 [Andrey]

Obviously, you have a lot of external visibility, but is that

00:37:02,786 --> 00:37:07,846 [Andrey]

a problem? Or is that just it’s understood that that’s how this goes? Yeah.

00:37:07,846 --> 00:37:13,846 [Seb Krier]

I think the external stuff is kind of almost just very separate from the kind of day-to-day work side of things.

00:37:14,986 --> 00:37:23,366 [Seb Krier]

And yeah, internally, we do have KPIs or equivalents or whatever. I think they may be less numerical in nature. But you might still have some, develop a consistent position on

00:37:24,506 --> 00:37:30,819 [Seb Krier]

X issue or something in the next two, three months.And that requires a lot of research work, coordinating.

00:37:30,819 --> 00:37:32,929 [Seth]

Have 10 opinions. [laughs]

00:37:32,930 --> 00:37:38,100 [Seb Krier]

No, ideally they just want one. I think 10 opinions, that’s the issue. There are a lot of opinions out there. You’ve got to find the good ones.

00:37:38,100 --> 00:37:39,530 [Seth]

That’s the main problem with economists.

00:37:39,530 --> 00:37:42,350 [Seb Krier]

But [laughs] yeah. Exactly. Who was that quote?

00:37:43,830 --> 00:37:44,290 [Seth]

Truman.

00:37:44,290 --> 00:37:44,330 [Seb Krier]

Yeah.

00:37:44,330 --> 00:37:46,210 [Seth]

Truman begged for the one-handed economist.

00:37:46,270 --> 00:38:20,990 [Seb Krier]

Yeah, exactly. But, so I think, yeah, I think internally it’s just a kind of analysis or something. Say you’re thinking about, oh, agents and legal liability. How do these things work? What does the existing legal environment say and prescribe? What happens if something goes wrong? What are relevant factors? There’s a lot of that kind of thing. And I guess particularly within the DeepMind side, because when we’re on the frontier side, we’re thinking about the next five years as opposed to what’s going on right now. But yeah, the other side stuff is really just kind of out of personal interest and just me writing stuff, and they seem fine with it so far. [chuckles]

00:38:20,990 --> 00:38:26,510 [Andrey]

What about... So we’ll be at a conference together, the Post-AGI conference-

00:38:26,510 --> 00:38:26,830 [Seb Krier]

Ooh

00:38:26,830 --> 00:38:28,370 [Andrey]

... at Lighthaven, Berkeley.

00:38:28,370 --> 00:38:30,110 [Seth]

Ooh. Prestigious.

00:38:31,130 --> 00:38:32,990 [Andrey]

I don’t know if it’s prestigious.

00:38:34,550 --> 00:38:34,629 [Seth]

[laughs]

00:38:34,630 --> 00:38:45,730 [Andrey]

But you’ve gone to a few of these conferences, like the Curve is another fairly well-known one. What’s your take on these?

00:38:45,730 --> 00:38:54,750 [Seb Krier]

I think some are useful. The majority of conferences I go to, I don’t exactly find that life-transforming, I guess.

00:38:54,750 --> 00:38:57,610 [Andrey]

[laughs] You’re going to the wrong conference. [laughs]

00:38:57,610 --> 00:39:09,290 [Seb Krier]

I know. Can someone show me the... But I think, yeah, they obviously perform a social function to some degree, right? There’s a lot of meeting people, some networking or something, some kind of finding out new ideas. But

00:39:10,390 --> 00:39:20,310 [Seb Krier]

my issue with conferences, very often they’re just very tame. They’re very risk-averse. They’re very the same ideas you’ve-- Already if you can read it online or something, it depends on the conference. But,

00:39:21,510 --> 00:39:24,190 [Seb Krier]

although I have been to really good ones, too. There was this

00:39:25,570 --> 00:39:43,529 [Seb Krier]

IMF conference with Econ Ty, with I think Anton Korinek and others had organized. And that was great because that was a nice one where you had both the technologists and a lot of economists and loads of presentations, and you got to learn lots of new things. But, in general, I don’t see a huge... Beyond maybe showing, again, some hot takes here and there.

00:39:45,370 --> 00:39:49,990 [Seb Krier]

Yeah, some I assume are good conferences. [chuckles]

00:39:49,990 --> 00:40:00,670 [Seth]

I’m just the exception, but you had a great joke on your Twitter the other day about this, which is, Caveman panelist one, “Fire is bad.” Caveman panelist two, “Fire is good.”

00:40:00,670 --> 00:40:00,770 [Seb Krier]

Yeah.

00:40:00,770 --> 00:40:02,100 [Seth]

Caveman panelist three,

00:40:03,450 --> 00:40:07,120 [Seth]

“We need to balance the upsides and downsides of fire and use it wisely.”

00:40:07,120 --> 00:40:07,320 [Seb Krier]

Absolutely.

00:40:07,320 --> 00:40:09,620 [Seth]

Wild applause. [laughs]

00:40:09,620 --> 00:40:09,650 [Andrey]

[laughs]

00:40:09,650 --> 00:40:14,850 [Seb Krier]

Exactly. There’s a lot of that. That’s the energy that I’m getting very tired of because it’s-

00:40:14,850 --> 00:40:15,050 [Seth]

[laughs]

00:40:15,050 --> 00:40:21,700 [Seb Krier]

And I like playing the role of the wise centrist opinion, whatever. But it does get very-

00:40:21,700 --> 00:40:23,150 [Seth]

You do get wild applause.

00:40:23,150 --> 00:40:24,470 [Seb Krier]

Yeah. All the time. [chuckles]

00:40:26,490 --> 00:40:29,770 [Seb Krier]

But yeah, I think there’s a lot of that. I wish there were more

00:40:30,810 --> 00:40:35,090 [Seb Krier]

almost private Chatham House-y conferences, where you had people who highly disagreed with each other-

00:40:35,090 --> 00:40:35,210 [Andrey]

Mm

00:40:35,210 --> 00:40:36,770 [Seb Krier]

... but were polite and didn’t get at

00:40:37,950 --> 00:40:49,370 [Seb Krier]

each other’s throats. And you had more setups that actually allowed ideas to clash a bit more, in a civilized way, of course. But that would be a bit hard, but also much more interesting, I think, than

00:40:51,490 --> 00:40:55,390 [Seb Krier]

everyone broadly agreeing that it’s good to be good and it’s bad to be bad, and yeah. [chuckles]

00:40:55,390 --> 00:41:03,710 [Andrey]

I do feel like the Lighthaven conferences are quite good for this, in that there’s an enormous amount of free time and-

00:41:03,710 --> 00:41:04,130 [Seb Krier]

Mm-hmm

00:41:04,130 --> 00:41:07,770 [Andrey]

... free space that’s not where the talk is happening.

00:41:07,770 --> 00:41:07,940 [Seb Krier]

Yeah.

00:41:07,940 --> 00:41:10,630 [Andrey]

And so you do get a lot of this.

00:41:10,630 --> 00:41:11,040 [Seb Krier]

Well, yeah, I agree.

00:41:11,040 --> 00:41:21,090 [Andrey]

But I agree that many conferences are not like that, where you’re just packed. You have a conference hall, and you don’t have anywhere else to go, and it’s packed with talks. Yeah.

00:41:21,090 --> 00:41:21,710 [Seb Krier]

Yeah. No, totally.

00:41:21,710 --> 00:41:23,550 [Seth]

NBER Summer Institute. [laughs]

00:41:24,750 --> 00:41:28,330 [Andrey]

Seth, there is disagreement. Say what you will. At NBER-

00:41:28,330 --> 00:41:28,540 [Seth]

There is fire

00:41:28,540 --> 00:41:29,430 [Andrey]

... people throw down.

00:41:30,450 --> 00:41:31,430 [Andrey]

[laughs]

00:41:31,430 --> 00:41:37,720 [Seth]

[laughs] I’ve never seen a meaner comment than I have seen from a discussant at NBER Summer Institute. [laughs]

00:41:37,720 --> 00:41:52,570 [Seb Krier]

[laughs] The Progress Conference, for example, last year, was one that I thought was really good. That was at Lighthaven, in fact. I think the setup and the kind of people and the curation and so just made it something that I found quite engaging. [upbeat music]

00:41:52,570 --> 00:41:56,490 [Seth]

So you brought up this idea, as we were talking, about you

00:41:58,330 --> 00:42:21,049 [Seth]

think there are so many meetings in your organization because it’s so hard, yet so critical to transfer information. And there’s this Coasean idea that so much of why the economy works the way it does is just the idea of transaction costs, right? In addition to kind of this Hayekian idea of local information that’s hard to share.

00:42:21,050 --> 00:42:21,810 [Seb Krier]

Mm-hmm.

00:42:21,810 --> 00:42:23,960 [Seth]

You have a very influential essay

00:42:25,130 --> 00:42:30,230 [Seth]

that kind of maybe stole some of Andrey’s thunder, but is still an excellent essay-

00:42:30,230 --> 00:42:31,040 [Seb Krier]

[laughs]

00:42:31,040 --> 00:42:46,210 [Seth]

... about this idea of, well, what happens when AIs go out there and can micro-bargain costlessly with each other at high frequency over very, what might seem to us, small issues.

00:42:47,570 --> 00:42:57,440 [Seth]

Tell us maybe in a few sentences, what’s that vision and what’s the positive vision for why that would be good for society, for us to have AI agents constantly bargaining for us over stuff?

00:42:59,130 --> 00:43:01,810 [Seb Krier]

Yeah. I guess the idea is, as you mentioned, there’s all sorts of

00:43:03,990 --> 00:43:26,350 [Seb Krier]

transaction costs that mean that we don’t get to bargain on things that we would otherwise bargain for. And instead, you get these blunt rules and these solutions that kind of work, but come with all sorts of externalities or aren’t super efficient. And so the idea is, if you can actually do this kind of negotiation at scale for very little, and that’s a big assumption. That’s not a given either,

00:43:27,850 --> 00:43:35,586 [Seb Krier]

then you could solve all sorts of things thatAnd also just kind of problems that would otherwise not be even conceivable in the first place.

00:43:36,726 --> 00:43:41,186 [Seth]

One example you give, just so we can be a little bit more specific, is noise standards, right?

00:43:41,186 --> 00:43:41,456 [Seb Krier]

Right.

00:43:41,456 --> 00:43:57,226 [Seth]

So you can’t throw a loud party after 10:00 PM in such and such a place. But you think that maybe AI agents could come to a less coarse rule that is, get us more to the grand coalition of allocative efficiency than a coarse rule like that.

00:43:57,226 --> 00:44:01,166 [Seb Krier]

Yeah. To be fair, that’s probably a problem that no one really cares about except me because of like- [chuckles]

00:44:01,166 --> 00:44:02,086 [Seth]

No. Dude.

00:44:02,086 --> 00:44:03,645 [Andrey]

I care about it so much.

00:44:03,645 --> 00:44:04,626 [Seb Krier]

Oh, really? Okay, cool.

00:44:04,626 --> 00:44:04,746 [Andrey]

Yes.

00:44:04,746 --> 00:44:07,816 [Seb Krier]

Maybe that’s a good example then. But yeah, the idea here is,

00:44:09,146 --> 00:44:17,006 [Seb Krier]

my neighbor is throwing a party, and instead of there being some sort of rule that says you’re not allowed to throw parties after 11:00, he could maybe just compensate me for the noise or something.

00:44:18,326 --> 00:44:21,686 [Seb Krier]

Or in fact, that’s one of the key crux of the whole Coasean thing is maybe

00:44:24,186 --> 00:44:36,085 [Seb Krier]

I have to compensate him to stop his parties. And it kind of depends where the initial right is. But broadly, you could have these kind of, my whole neighborhood doesn’t want me to party, and they’re just giving me a small payment or the reverse, depending on where the initial allocation is.

00:44:37,226 --> 00:44:44,446 [Seb Krier]

But I think you could have all sorts of micro ways in which these transaction costs at scale help you get much better beneficial outcomes.

00:44:45,486 --> 00:44:48,486 [Seb Krier]

And so that would be the noise one would be like, okay.

00:44:50,406 --> 00:45:18,666 [Seb Krier]

And it’ll probably just also let people kind of regroup into the party people just going into the neighborhood where that’s just generally more party tolerant or something, and the kind of peace and quiet preferring people just... Because I think one of the points with the piece was that AI also helps you coordinate better. You can use this stuff to find people who have the same interests and preferences as you or something, and just then bargain or negotiate or whatnot in that way as well.

00:45:20,626 --> 00:45:27,386 [Seth]

So it’s not just bargaining over externalities that are negative, it’s maybe coordinating over positive externalities, right?

00:45:27,386 --> 00:45:27,526 [Seb Krier]

Yeah.

00:45:28,766 --> 00:45:51,746 [Seth]

What pieces do we need in the economy to make this a reality, and what time horizon are you thinking about? So obviously this is an idea that you could have a small version of, and then like the sci-fi, this is constantly, I’m allowed to speed in my car today because I really need to get to work because I’m late, and it’s bargaining with all the cars on the highway at ultra-high frequency. So what are the time horizons you have in mind, and what pieces do we need?

00:45:51,746 --> 00:46:21,786 [Seb Krier]

Honestly, I haven’t even thought about the timelines really. [laughing] For me, this was mostly kind of an aspirational thing of like, well, it looks like we could unlock some cool things, and because there’s all these-- It’d be nice to have a positive vision of how things might pan out. It certainly doesn’t mean that everything has to be negotiated and bargained over. But I could see a large proportion of things, certainly in everyday life, like I could just tell my aunt, “You don’t have to worry about your parking issues anymore. It’s just sorted now,” whatever. The agents are taking care of that. And so it kind of depends on what scale you’re talking about. Certainly having democracy at scale and

00:46:23,626 --> 00:46:29,086 [Seb Krier]

half automated and half made more efficient through these systems or something is something that I think is going to take a long time.

00:46:30,426 --> 00:46:47,986 [Seb Krier]

But I can see smaller versions of it happening. The smallest version I can think of is just even basic calendar management. Until recently, that would be impossible to get automated really well. But gradually now I could expect an agent to really know my preferences, the context in which I operate, the hierarchical

00:46:49,006 --> 00:47:12,466 [Seb Krier]

relationship with the people I’m organizing meetings with and whatnot, and the agent coming to tell me something like, “Okay, I’ve moved your meeting here to next week because this one’s not very important, and you don’t really care about it. But I’ve brought this meeting with your director forward because it’s urgent given what I’ve seen in that email. And also, I’ve set the location in this place because you’ll both be around that area in any event around then, given what I’ve gathered from talking to the other agent.”

00:47:13,946 --> 00:47:23,766 [Seb Krier]

Something like that I think I could easily see in the next couple of years. But something at a higher, things like automating part of a local authority and-

00:47:23,766 --> 00:47:26,786 [Seth]

So let’s stick to the noise example because that’s a core one.

00:47:26,786 --> 00:47:30,276 [Seb Krier]

Well, the noise one, I think takes a bit longer because it requires this kind of,

00:47:31,486 --> 00:47:44,496 [Seb Krier]

well, this adoption and integration of this technology at my local city council or whatever. And that’s one of the things that takes a long time. It’s not immediate, and it requires a shift in behaviors. People have to also then use these agents. It has to be legitimized. The

00:47:45,726 --> 00:47:48,386 [Seb Krier]

value proposition has to be made a bit clearer.

00:47:48,386 --> 00:47:48,586 [Seth]

Mm-hmm.

00:47:48,586 --> 00:48:01,666 [Seb Krier]

And there’s all the usual political issues of like, well, you’ve got these vested interests, and it’s not in some people’s interests for that whole thing to work out. In the same way as if you wanted to have some sort of YIMBY paradise, a lot of kind of planning authorities wouldn’t be very delighted. They would probably find ways to stop the

00:48:02,786 --> 00:48:03,166 [Seb Krier]

deployment of-

00:48:03,166 --> 00:48:07,046 [Seth]

But they could be compensated through micro-transactions, dude. [chuckles]

00:48:07,046 --> 00:48:11,866 [Seb Krier]

Exactly. Right? It’s a recursive Coasean improvement. But-

00:48:11,866 --> 00:48:15,566 [Andrey]

So, I guess just like a thought here.

00:48:15,566 --> 00:48:15,885 [Seb Krier]

Mm-hmm.

00:48:15,886 --> 00:48:19,106 [Andrey]

I’m sure you’ve heard of Edge Esmeralda-

00:48:19,106 --> 00:48:19,116 [Seb Krier]

Mm-hmm

00:48:19,116 --> 00:48:31,936 [Andrey]

... which is kind of this experimental new town. It seems just like we think that with new firms that are entering, they’re going to be able to organize in a way that’s going to better be able to take advantage of AI capabilities.

00:48:31,936 --> 00:48:32,726 [Seb Krier]

Mm-hmm.

00:48:32,726 --> 00:48:43,006 [Andrey]

To the extent that we can have newly incorporated cities or other jurisdictions, they might allow for agentic representation at the local council meeting-

00:48:43,006 --> 00:48:43,646 [Seb Krier]

Mm-hmm

00:48:43,646 --> 00:48:50,666 [Andrey]

... and other such things. And if that works really well, then it would actually

00:48:51,786 --> 00:48:56,166 [Andrey]

show other places what they’re missing out on. I’m not saying that the San Francisco-

00:48:56,166 --> 00:48:56,176 [Seb Krier]

Yeah

00:48:56,176 --> 00:48:59,076 [Andrey]

... City Council would do this. [laughs]

00:48:59,076 --> 00:48:59,586 [Seb Krier]

[laughs]

00:48:59,586 --> 00:49:07,265 [Andrey]

But some smaller cities might start doing this, or school boards. That’s a great example. I hear people have a lot of issues with those.

00:49:07,266 --> 00:49:30,866 [Seb Krier]

Totally. I think that’s kind of what I concluded in the essay, is that you’ve got to start in these kind of small proof of concepts and then kind of make the idea actively desirable and for people to kind of actually see the benefits of these things. And so I agree, The Edge Esmeralda, I think they’re trying something out with agents, and that’s going to be pretty exciting to see. And maybe then that kind of takes on, right? In the same way as a lot of conferences now have apps or something to...

00:49:31,886 --> 00:49:36,446 [Seb Krier]

I guess the more pessimistic version is, well, if you look at Estonia, they’ve been really good at

00:49:37,506 --> 00:49:42,666 [Seb Krier]

integrating technology and automating all sorts of parts of their government in really efficient and effective ways.

00:49:43,642 --> 00:49:44,622 [Seth]

Estonia?

00:49:44,622 --> 00:49:46,102 [Seb Krier]

Yeah. [laughs]

00:49:46,102 --> 00:49:46,982 [Seth]

[laughs]

00:49:46,982 --> 00:49:58,162 [Seb Krier]

And yet you don’t necessarily see neighboring countries have a strong incentive to do the same straight away. And I would expect the equivalent to take quite a long time, say, if you try to do the same thing in Brussels. So there are other-

00:49:58,162 --> 00:49:58,902 [Seth]

[laughs]

00:49:58,902 --> 00:49:59,401 [Seb Krier]

... dynamics.

00:49:59,401 --> 00:50:00,802 [Seth]

Anything takes longer in Brussels.

00:50:00,802 --> 00:50:03,082 [Seb Krier]

That’s true. It’s part and parcel of it.

00:50:03,082 --> 00:50:04,262 [Seth]

Except for the french fries.

00:50:04,262 --> 00:50:08,022 [Seb Krier]

I’d say, yeah. Which are indeed very good. But, I think

00:50:09,562 --> 00:50:25,742 [Seb Krier]

you’ll want some of these kind of smaller challenges, I guess, to really push for these things and put more pressure on the incumbent systems, whether it’s in the private sector or in the public sector. The only thing is, I expect it to be a bit slower in the public sector than it will be in the private sector.

00:50:27,382 --> 00:50:41,082 [Seb Krier]

Once a bunch of startups start using agents that, I don’t know, coordinates on meetings or whatever, maybe that’s going to take off as well with all the wider organizations much more quickly. Whereas, you still have parts of the German or UK government using fax machines today.

00:50:41,082 --> 00:50:59,042 [Andrey]

Well, there’s this example that we have in our paper where some agentic environments are designed for agents only, right? And so you might imagine that to work well within a firm where everyone has their calendar scheduling agent, and they all talk to each other. But I think

00:51:00,102 --> 00:51:07,482 [Andrey]

you can also imagine an environment where the agent has to be able to interface also with humans, right?

00:51:08,782 --> 00:51:10,042 [Andrey]

And those are quite different

00:51:11,762 --> 00:51:13,022 [Andrey]

capabilities

00:51:14,782 --> 00:51:18,842 [Andrey]

or tests of the system. And if you want maybe

00:51:20,062 --> 00:51:25,642 [Andrey]

more useful versions of this, they should also be able to interact with humans who are not using the agents.

00:51:25,642 --> 00:51:37,682 [Seb Krier]

True. And that probably acts also as a speed limiter, I guess, as well, right? If you have to then... But I agree. I think that’s going to be an important part, and I think that’s where protocol design and all these kind of things start coming into play.

00:51:39,022 --> 00:51:48,362 [Seb Krier]

And I liked actually in your paper the whole bowling shoe versus bring your own question as well. Because you’d expect some platforms to just say, “Well, just use our agent here” or something.

00:51:49,422 --> 00:51:54,362 [Seb Krier]

And it’s not clear to me that you’ll necessarily have one agent that you’re going to use across all platforms the same as something.

00:51:56,002 --> 00:52:06,842 [Seb Krier]

Maybe you probably want certain characteristics or certain knowledge to be more segmented. Maybe you want your agent doing high-frequency trading to be a bit different than your agent doing, I don’t know, social life coordination or something.

00:52:08,402 --> 00:52:14,622 [Seb Krier]

And it’s not clear whether it’s an agent that’s necessarily owned in some way by me or if it’s in fact...

00:52:15,882 --> 00:52:25,202 [Seb Krier]

So yeah, there’s all sorts of interesting design questions that aren’t fully clear to me just yet. And I think that would be the kind of thing I’d love to see more work on. Actually, I think also Anthropic had this project deal recently, which was interesting, too.

00:52:27,282 --> 00:52:29,662 [Seb Krier]

And it seems to have worked out quite nicely, I think, in their experiment.

00:52:31,242 --> 00:52:32,262 [Seth]

It is, yeah.

00:52:32,262 --> 00:52:34,242 [Andrey]

Yeah. You’re welcome to explain it. Yeah.

00:52:34,242 --> 00:52:41,262 [Seb Krier]

[laughs] Again, I don’t have deep knowledge of what they’ve done or anything. I think Christy glanced at the... But I think broadly, they

00:52:42,822 --> 00:52:51,922 [Seb Krier]

let part of their employees have access to agents to do some sort of negotiating and bargaining on their behalf for all sorts of commercial items or things they would want to trade.

00:52:53,102 --> 00:52:59,302 [Seb Krier]

And they also did some experiment where I think some had access to more capable models, others had access to slightly less capable models. And I think the

00:53:00,842 --> 00:53:01,842 [Seb Krier]

insight was something like

00:53:03,162 --> 00:53:11,622 [Seb Krier]

everyone was happy in the end, but actually people with the more capable models got better surplus or something. And those without the capable models didn’t realize the

00:53:12,642 --> 00:53:20,782 [Seb Krier]

gain they would have made had they had the more capable one. But in any event, everyone was happy. Isn’t it? It apparently worked out and people traded stuff.

00:53:20,782 --> 00:53:28,921 [Andrey]

Well, my understanding is the setting was people literally took photos. They inventoried the stuff they didn’t need in their home.

00:53:28,922 --> 00:53:29,242 [Seb Krier]

Yeah.

00:53:29,242 --> 00:53:29,932 [Andrey]

And then-

00:53:29,932 --> 00:53:31,522 [Seth]

[laughs]

00:53:31,522 --> 00:53:37,942 [Andrey]

... if you can get rid of it, you might already be happy if you can get rid of it, let alone get something in return for it.

00:53:37,942 --> 00:53:47,182 [Seb Krier]

It’s true. Well, yeah. That was the thing we were discussing earlier. I’d be happy to have an agent sell my records, but not buy them or some silly [chuckles] certain things.

00:53:47,182 --> 00:53:47,861 [Andrey]

Mm.

00:53:47,862 --> 00:53:51,862 [Seb Krier]

But, yeah. So I think I didn’t go too deep into that specific

00:53:53,022 --> 00:54:03,162 [Seb Krier]

paper or webpage or whatever they did on that. But there seems to be more experimentation along these lines. I think Rohit, of course, has his own kind of things. I had a MATS

00:54:04,322 --> 00:54:09,922 [Seb Krier]

stream with Yoav Shavit at OpenAI on kind of multi-agent coordination.

00:54:09,922 --> 00:54:13,992 [Andrey]

Can you explain what MATS is? Because I don’t think many in our audience know what that is.

00:54:13,992 --> 00:54:16,622 [Seb Krier]

Oh, it’s basically a fellowship.

00:54:18,662 --> 00:54:28,062 [Seb Krier]

You have mentors and mentees, and they work on all sorts of projects, and there’s close collaboration with labs. But it’s broadly on technical projects for AI safety and governance.

00:54:28,062 --> 00:54:34,122 [Andrey]

Is this a program that our junior economist listeners should try to apply to?

00:54:34,122 --> 00:54:59,482 [Seb Krier]

I think so, yeah. I don’t know if they have an econ stream. Well, they should if they don’t. And there is, I think, a wider kind of governance stream, so I wouldn’t be surprised. If they don’t have one, I would imagine there to be one eventually. But I think a lot of economists who’ve got this expertise in mechanism design or game theory, I think there’s a lot to offer here, even in the more classical AI safety type streams as well. So yeah, they should look into MATS for sure.

00:55:00,882 --> 00:55:13,182 [Seth]

Well, a lot of pickup on those answers. Very interesting stuff. I guess one question I have here that you already introduced is the idea of, okay, so maybe we’re going to get allocative efficiency from all this Coasean bargaining.

00:55:13,182 --> 00:55:13,222 [Seb Krier]

Mm-hmm.

00:55:13,222 --> 00:55:41,202 [Seth]

We’ll make sure that the lawnmower doesn’t get mowed at 2:00 a.m. when it’s really annoying for me. But there’s a distributional question here, too. So one concern that you already kind of aired out was the idea that people with better models who can pay a little bit more might be able to bargain harder and get a bigger share of the pie. Another distributional concern might be just the step zero. Which is for any sort of bargaining over

00:55:44,802 --> 00:55:48,561 [Seth]

an externality to happen, you need to assign that property right first.

00:55:48,562 --> 00:55:48,752 [Seb Krier]

Mm-hmm.

00:55:48,752 --> 00:55:52,382 [Seth]

And that could be a process that is very unequal. So,

00:55:53,822 --> 00:55:59,962 [Seth]

feel free to take up either of those concerns. Is Coasean bargaining going to lead to a dystopic inequality hellhole?

00:56:01,490 --> 00:56:12,270 [Seb Krier]

Right. I don’t think so. Certainly, I would imagine that in the first instance, I think people are still net better off than the counterfactual of not having any bargaining agents at all.

00:56:12,270 --> 00:56:13,590 [Seth]

Well, that’s the Pareto promise, right?

00:56:13,590 --> 00:56:13,610 [Seb Krier]

Yeah.

00:56:13,610 --> 00:56:18,230 [Seth]

First, Fear Theorem tells us we’re going to get a Pareto efficient, allocative efficiency outcome.

00:56:18,230 --> 00:56:18,550 [Seb Krier]

Mm-hmm.

00:56:18,550 --> 00:56:20,370 [Seth]

But, a lot of people-

00:56:20,370 --> 00:56:20,610 [Seb Krier]

Sure

00:56:20,610 --> 00:56:23,110 [Seth]

... trade this efficiency for some equality, right?

00:56:23,110 --> 00:57:04,610 [Seb Krier]

I guess to me, it depends a little bit on the complexity of the problem, and I would guess that over time, you’ll get diminishing returns. I think we’re discussing this, asking Claude Opus for her toothpaste recommendation or whatever. There’s a point to which I would imagine, I guess I don’t know this empirically, and certainly in the project deal and tropics that you had kind of better returns by using the better model, but it’s not clear to me whether over time, actually, this kind of scales linearly in terms of if you’ve got the mega guard model, you necessarily get a better deal. There’s a point to which I think this doesn’t... In the same way as Elon Musk has the same phone as you and I have, and he’s not getting that much more just by virtue of being rich or something. At least, he does in general, just not on the phone part. [chuckles] And

00:57:06,370 --> 00:57:23,230 [Seb Krier]

in any event, I think even then, to the extent that there were to be some sort of inequity, nothing stops, I think, whether it’s the state or philanthropy, whoever, to kind of subsidize access. Maybe you can have vouchers, you have systems that essentially re-equilibrate things. So I’ve always liked the voucher systems, even for private schooling and so on.

00:57:24,250 --> 00:57:26,530 [Seb Krier]

So maybe that’s one way around that.

00:57:27,630 --> 00:57:29,160 [Seb Krier]

And the property-

00:57:29,160 --> 00:57:34,850 [Seth]

Decide whether to use my precious 100,000 tokens to bargain for food or to bargain for noise complaints.

00:57:34,850 --> 00:57:40,590 [Seb Krier]

[laughs] Well, yeah. That’s on you to decide how you want to use it. [laughs] But-

00:57:40,590 --> 00:57:42,100 [Andrey]

Always has been, Seth. Always has been.

00:57:42,100 --> 00:57:42,290 [Seth]

Always has been.

00:57:42,290 --> 00:57:50,370 [Seb Krier]

Yeah. Exactly. That’s not me. And then the other one with the property right assignment is a tricky one, too. And it’s deeply political ultimately as well because-

00:57:50,370 --> 00:57:50,650 [Seth]

Right

00:57:50,650 --> 00:57:51,690 [Seb Krier]

... of course, it’s-

00:57:53,090 --> 00:57:59,050 [Seth]

It reminds me of land redistribution, right? It’s like, “All right, we’re breaking up the plantations. Who gets it?” [laughs]

00:57:59,050 --> 00:58:19,190 [Seb Krier]

Yeah. And I think, I guess what I claim in the essay or something was something like, well, you can just start with what we already have. There’s already a certain kind of distribution and allocation of rights and why do you need to just start anew? Just start with what we have. Then I did agree that there’s a need for them to change. You just then kind of, again, that’s the whole democratic process.

00:58:19,190 --> 00:58:21,110 [Seth]

We’ll start with noise complaints.

00:58:21,110 --> 00:58:21,250 [Seb Krier]

Well-

00:58:21,250 --> 00:58:24,410 [Seth]

Because right now, it’s unclear to me who has the... I have the... Yeah.

00:58:24,410 --> 00:58:43,210 [Seb Krier]

Sorry, I thought of that a little bit. I was thinking that actually, you can just keep the 11:00 PM threshold or curfew. It’s just that this switches the property right allocation. So before 11:00 PM, I have to pay you to be quiet, and after 11:00 PM, you have to pay me to be noisy or something.

00:58:43,210 --> 00:58:43,710 [Seth]

[laughs]

00:58:43,710 --> 00:58:53,540 [Seb Krier]

So I think that still works, but you come up with some sort of agreements as to where these kind of fluids, property rights, or something apply. But for these to be decided, I think you need the

00:58:54,670 --> 00:59:17,890 [Seb Krier]

whole democratic apparatus and this is the other thing I was mentioning earlier, and wanting to make that part of, I don’t know, local democracy, for example, more responsive and effective. So there’s all sorts of kind of decisions being taken every day at the local council or something that I’m completely unaware of and have, unfortunately, very little time to go and engage with. But it probably would be better for me to have-

00:59:17,890 --> 00:59:22,190 [Seth]

Unfortunately, you’re telling me that if I took away an hour of Twitter, that’s what you’d be doing.

00:59:22,190 --> 00:59:23,439 [Seb Krier]

Exactly. I could be online.

00:59:23,439 --> 00:59:26,630 [Andrey]

Wait, you could definitely be on Twitter and go to the council meeting.

00:59:26,630 --> 00:59:26,650 [Seth]

[laughs]

00:59:26,650 --> 00:59:32,150 [Seb Krier]

Yeah, but I have to focus my attention on... [laughs] But I could be consuming slop instead.

00:59:33,430 --> 00:59:34,850 [Seb Krier]

So yeah, I think that

00:59:36,030 --> 00:59:36,690 [Seb Krier]

for that, I wouldn’t

00:59:37,730 --> 00:59:40,070 [Seb Krier]

mind having an agent come back to me and say, “Oh, there’s this

00:59:41,330 --> 00:59:50,850 [Seb Krier]

new road that’s being built, and it would be pretty good for you, given that you take that route a lot and stuff, but it seems to be not going well. Do you want to register some sort of...” But

00:59:52,010 --> 01:00:16,720 [Seb Krier]

yeah. And again, I can see versions of that where this is used for vitocracy, and this is where the whole mechanism design stuff comes into play. But for the property right assignment side of things, I think you just start with what we have today, and then you adjust over time using this mechanism. And again, it’ll probably depend on the setting or where you’re doing that. The curfew one, I guess, would be with your local council, but for all sorts of other things, there’ll be different kind of actors and

01:00:17,970 --> 01:00:20,970 [Seb Krier]

institutions in place for that. You probably need... Yeah.

01:00:20,970 --> 01:00:37,450 [Seth]

Can I go, how about something like stuff that we don’t have property rights around at all right now, or even rules around, but are definitely negative externalities, right? So for example, I have a neighbor in my office who does not shower, who smells really bad, right? So how do we decide-

01:00:37,450 --> 01:00:40,330 [Andrey]

Are you willing to reveal this on a-

01:00:40,330 --> 01:00:40,500 [Seth]

No

01:00:40,500 --> 01:00:42,450 [Andrey]

... public podcast? [laughs]

01:00:42,450 --> 01:00:48,950 [Seth]

So the question is, how do we decide whether he has a right to be smelly, or I have a right to not smell him?

01:00:49,990 --> 01:01:00,549 [Seb Krier]

I guess it, yeah, once again, highly political. [laughs] But I think my bias, of course, is one that’s more, I guess, freedom maximizing a classical liberal or something. So it’s more like-

01:01:00,550 --> 01:01:01,490 [Seth]

The right to be smelly.

01:01:01,490 --> 01:01:01,630 [Seb Krier]

Yeah.

01:01:01,630 --> 01:01:01,990 [Seth]

There we go.

01:01:01,990 --> 01:01:06,630 [Seb Krier]

It’s the harm principle, right? He’s not harming you by being smelly, unfortunately, so you’re going to have to-

01:01:06,630 --> 01:01:08,940 [Andrey]

What do you mean? No, he definitely harms me.

01:01:08,940 --> 01:01:09,910 [Seth]

It really smells bad, dude.

01:01:09,910 --> 01:01:14,540 [Seb Krier]

[laughs] It’s not decreasing your longevity as far as I’m aware. [laughs]

01:01:14,540 --> 01:01:17,550 [Seth]

[laughs] Actually, it makes me want to decrease my longevity.

01:01:17,550 --> 01:01:33,610 [Andrey]

But no, but more seriously, this is actually a great use case for agents because the agent could receive anonymous... A lot of effective altruist rationalist types have this anonymous-

01:01:33,610 --> 01:01:33,910 [Seb Krier]

Oh, yeah

01:01:33,910 --> 01:01:36,710 [Andrey]

... link in their profile, where you can give them anonymous

01:01:38,350 --> 01:01:38,690 [Andrey]

feedback-

01:01:38,690 --> 01:01:39,710 [Seb Krier]

Yeah

01:01:39,710 --> 01:01:43,390 [Andrey]

... about how much they smell. And so with the world of agents then-

01:01:43,390 --> 01:01:43,700 [Seth]

[laughs]

01:01:43,700 --> 01:01:49,150 [Andrey]

... an agent could figure out a way to give anonymous feedback to the person’s agent

01:01:50,350 --> 01:01:52,190 [Andrey]

that the person smells.

01:01:52,190 --> 01:01:52,550 [Seb Krier]

Well-

01:01:52,550 --> 01:01:55,420 [Andrey]

And then that agent could pass it on to the person

01:01:56,206 --> 01:02:14,466 [Seb Krier]

Exactly. To be fair, you can also do that without the agent. You can just put a message on their desk saying, “By the way, please just shower,” or something. [laughing] But I think that in principle, what would be actually more interesting is having the agent solve that problem without the person necessarily being offended by the thing or having the message straight away.

01:02:15,806 --> 01:02:20,186 [Seb Krier]

So having the other agent just... This is getting into nudging territory, right? And this is also controversial in its own right.

01:02:21,426 --> 01:02:43,126 [Seb Krier]

But it actually connects to this other question of do you want your agent to be purely instruction following, or do you want it to help you maybe achieve some higher order goals? And so maybe one of my goals is I want to be not disliked at the office. And so one of the ways is my agent, well, I’m not the smelly guy, but let’s assume [laughing] then the agent will kind of find a subtle way-

01:02:43,126 --> 01:02:43,416 [Seth]

Sure.

01:02:43,416 --> 01:02:46,686 [Seb Krier]

... to be like, “Seb, it’s time to log off Twitter and go and take a shower.”

01:02:47,926 --> 01:02:53,455 [Seb Krier]

[laughing] So yeah, that might be one of them. But yeah, in terms of an initial allocation, I don’t think that he would

01:02:54,986 --> 01:02:58,706 [Seb Krier]

have to pay me to continue being smelly perhaps.

01:02:59,986 --> 01:03:01,185 [Seb Krier]

[chuckles]

01:03:01,186 --> 01:03:02,746 [Seth]

Okay. So

01:03:04,826 --> 01:03:13,406 [Seth]

let me give you some concerns about this vision. Right? So we’ve laid out what could be beautiful about this. We can efficiently allocate externalities, blah, blah, blah.

01:03:14,766 --> 01:03:30,326 [Seth]

One set of concerns I’ve heard from my non-economist friends when I talk about this is this idea that micro bargaining at scale could harm social legitimacy. So if I may give you two exaggerated sci-fi versions of this, and then maybe we can reel it back in into reality.

01:03:30,326 --> 01:03:31,186 [Seb Krier]

Mm-hmm.

01:03:31,186 --> 01:03:37,106 [Seth]

The first version of this comes from the video game Helldivers 2. Are you familiar with that game?

01:03:37,106 --> 01:03:39,606 [Seb Krier]

Yeah, I haven’t played it, but I’m aware broadly of the... Yeah.

01:03:39,606 --> 01:04:29,386 [Seth]

All right. So the premise is kind of like a Starship Troopers sort of universe, where it’s officially a democracy, but really it’s fascism. And their political system is called Managed Democracy. And the way that Managed Democracy works is everyone fills out a preference questionnaire about what they like and they don’t like, and then that goes up into the big algorithm in the sky, and then boop boop boop boop boop, here’s your optimal social policy that we’ve calculated over everyone. Right? So on the one hand, it kind of seems like that is one end limit of what Coasean bargaining at scale would look like. I just fill out a survey, and my AI negotiates with all the other AIs, and I don’t really interact with it beyond that. But you can kind of see how dystopic that looks. It has a lot less social legitimacy than maybe two choice first past the post elections.

01:04:29,386 --> 01:04:39,366 [Seb Krier]

Yeah, I agree. That’s the extreme version of it or something. Seems very undesirable. And so that’s why I was very careful to say, by the way, this is not saying make every single aspect of your life

01:04:41,266 --> 01:04:46,186 [Seb Krier]

a transaction. In fact, I think there’s the whole Michael Sandel stuff, right?

01:04:46,186 --> 01:04:47,666 [Seth]

Oh, Sandel. That’s my next question.

01:04:47,666 --> 01:04:47,906 [Seb Krier]

Right.

01:04:47,906 --> 01:04:49,746 [Seth]

You’re jumping ahead. [chuckles]

01:04:49,746 --> 01:04:51,046 [Seb Krier]

Well, but I think that the

01:04:52,406 --> 01:05:22,676 [Seb Krier]

idea, at least for me, is not for the agent to completely automate your agency away. If anything, ideally, this kind of enhances it in some way. So, I like the idea of the local authority thing because it’s something I wouldn’t be doing in any event. It’s not like I’m having this taken away from me and that, but for the agent, I would be going there or something. So I think there has to be some sort of conscious choice as well from individuals to where do they want to kind of exercise their agency and where do they not. And I think there’s some people who actually do not care as much about participating and

01:05:23,926 --> 01:05:25,386 [Seb Krier]

others who do, and I think that’s fine.

01:05:27,146 --> 01:05:38,386 [Seb Krier]

But I agree that also, there are kind of non-market norms that I think are worth preserving, reciprocity and being a good neighbor and civic participation. And you don’t want to kind of get rid of these entirely either. So,

01:05:39,406 --> 01:06:07,506 [Seb Krier]

in my mind, this will never be the kind of perfect maximalist version of the whole thing. It’ll probably work in some instances. In others, you’re not going to use the system. And in general, I suspect that for many things and trivial things, like actually the smelly coworker, I don’t know, maybe that one is the one where you actually want to use an agent rather than deal with it yourself. But I agree, and it’s a matter of, I guess, developing the right norms around how we would be using there. Where is it acceptable, where is it not?

01:06:09,126 --> 01:06:31,386 [Seb Krier]

But I agree with the bad version of this where no one does anything, and you just have an agent infer everything on your behalf, and now you’re just basically removing the whole participatory element of democracy, which I think is still very important. And so again, here too you will want your agent... In a way it’s not dissimilar to

01:06:33,146 --> 01:06:41,326 [Seb Krier]

these questions of cognitive offloading and so on. Of like, well, if the AI writes on your behalf versus it gives you advice on how you can write the thing better.

01:06:42,606 --> 01:06:46,106 [Seb Krier]

And so now I’m going a bit on a tangent, but I was mentioning this

01:06:47,326 --> 01:06:59,026 [Seb Krier]

positive alignment thing earlier of do you want your agent to be purely instruction and then following, or would you want to specify higher order preferences? I think this is where this comes into play, where I might say, actually, I

01:07:02,126 --> 01:07:06,326 [Seb Krier]

want you to help me achieve this kind of higher order goal or something, which may entail, in fact, asking you to just

01:07:07,346 --> 01:07:46,266 [Seb Krier]

read up on this. And there’s also an element of me actually needing to retain that choice as well and as an individual, right? I will sometimes summarize a paper using an AI, and sometimes I will... And that’s fine because the counterfactual wouldn’t have not been me reading the whole paper because I don’t have... But there are also papers where I would have read the whole paper, and I’m still asking the AI to do it, and that’s kind of also on me to learn and for, I guess, the right norms to rise in terms of, okay, when it’s too much, what is right, what is not. In the same way as, I don’t know, I’ve learned how to do regressions by hand or something one day in the past, and that was helpful in some sense, but then I’ve never done that again, and I’ve started using Stata and R or whatever. So,

01:07:47,406 --> 01:07:47,985 [Seb Krier]

there’s an element of-

01:07:47,986 --> 01:07:48,906 [Seth]

What?

01:07:48,906 --> 01:07:50,786 [Seb Krier]

[chuckles] Actually, yeah.

01:07:50,786 --> 01:07:52,886 [Seth]

Are you telling us you’re not an economist?

01:07:53,966 --> 01:07:55,196 [Seb Krier]

No, I am not an economist.

01:07:55,196 --> 01:07:55,246 [Seth]

You got into Stata and R.

01:07:55,246 --> 01:08:11,046 [Seb Krier]

I was never an economist. [laughing] So yeah, there’s this wider question of where do you want to preserve agency and then also where you want to preserve certain norms on a day-to-day level. And so I think it’s going to be a big messy thing in reality as opposed to the maximalist version of either vision.

01:08:12,142 --> 01:09:27,362 [Seth]

For those of you playing along at home, now is your chance to think about how this conversation has changed your priors. This chance to contemplate your posteriors is sponsored by Revelio Labs. Revelio Labs is a leading provider of labor economics data and data services for companies, academics, and independent researchers. Andrey and I have been working in economics of AI, digitization, and automation for a long time, and we can confirm just how useful Revelio’s data is. Revelio’s team combines comprehensive micro-level data on employee professional profiles, job postings, and employee sentiment with standardizations, mappings, and enrichments available, all to make that data useful without making your modeling decisions for you. The data can be flexibly aggregated to company, market, or industry, and can be used to study questions ranging from career trajectories to occupational transformation, to the returns to skills, and the impact of AI on labor demand for tasks. Can’t imagine anyone who would be interested in that. And Revelio data is available on WRDS. So if you’re an academic with a good library, go see if you have access to their premier data already. And if you don’t, you can reach out to their excellent economics team, and they’ll hook you up. I feel like it’s time for lightning round.

01:09:27,362 --> 01:09:27,682 [Andrey]

All right.

01:09:27,682 --> 01:09:28,342 [Seth]

Let’s do it.

01:09:28,342 --> 01:09:33,702 [Andrey]

Let’s do it. [upbeat music]

01:09:34,802 --> 01:09:35,962 [Andrey]

So we’re-

01:09:35,962 --> 01:09:35,972 [Seth]

Nice

01:09:35,972 --> 01:09:39,372 [Andrey]

... yeah, very excited because you’re a master of the take.

01:09:39,372 --> 01:09:41,252 [Seth]

[laughs]

01:09:41,252 --> 01:09:41,742 [Andrey]

[laughs]

01:09:43,282 --> 01:09:47,702 [Andrey]

So you fairly recently moved from London to New York City.

01:09:47,702 --> 01:09:48,242 [Seth]

Mm-hmm.

01:09:48,242 --> 01:09:49,122 [Andrey]

What are

01:09:50,422 --> 01:09:53,022 [Andrey]

the words that come to mind-

01:09:53,022 --> 01:09:53,312 [Seth]

[laughs]

01:09:53,312 --> 01:09:56,762 [Andrey]

... when you’re thinking about AI in New York City versus AI in London?

01:09:58,042 --> 01:10:05,402 [Seth]

Oh. AI in London feels more academic, theoretic,

01:10:06,562 --> 01:10:53,002 [Seth]

I guess more deep researchy, and more governance-oriented in some way. I think there’s this whole Oxford, Cambridge, UCL connection, the government and UK AI Safety Institute, the fact that government was already dealing with AI quite early. I think that that whole ecosystem is fairly developed here, whereas New York is a bit of a weird, secret third thing of like, well, there’s some interesting people, more like heterodox, random, the edge of culture. Some interesting kind of frontier AI people too, but I wouldn’t really see it as some sort of AI capital or something. It’s just a bit more diverse in terms of the offering. But I kind of like that. That’s the point also. I don’t want to be fully NSF. I kind of like being somewhat distant.

01:10:54,062 --> 01:10:59,551 [Andrey]

What is the reaction of people you meet in Brooklyn or at your-

01:10:59,551 --> 01:11:00,202 [Seth]

[laughs]

01:11:00,202 --> 01:11:04,382 [Andrey]

... or at your EDM shows when you tell them where you work?

01:11:04,382 --> 01:11:07,771 [Seth]

First of all, I do not like EDM at all. But- [laughs]

01:11:07,771 --> 01:11:08,111 [Andrey]

[laughs]

01:11:08,111 --> 01:11:08,122 [Seth]

[laughs]

01:11:08,122 --> 01:11:09,242 [Andrey]

Whoa.

01:11:09,242 --> 01:11:09,322 [Seth]

[laughs]

01:11:09,322 --> 01:11:10,382 [Andrey]

Hottest thing right now.

01:11:10,382 --> 01:11:11,782 [Seth]

Yeah. Going-

01:11:11,782 --> 01:11:13,782 [Andrey]

So how much would you have your DJ-

01:11:13,782 --> 01:11:13,802 [Seth]

Electronic-

01:11:13,802 --> 01:11:14,162 [Andrey]

... mic?

01:11:14,162 --> 01:11:16,912 [Seth]

Fine electronic music. [laughs]

01:11:16,912 --> 01:11:19,672 [Andrey]

[laughs] Oh, yeah, that one’s good. The good-

01:11:19,672 --> 01:11:20,212 [Seth]

[laughs]

01:11:20,212 --> 01:11:20,262 [Andrey]

[laughs]

01:11:20,262 --> 01:11:21,682 [Seth]

Aligned electronic music.

01:11:23,082 --> 01:11:24,232 [Andrey]

Yes. [laughs]

01:11:24,232 --> 01:11:25,732 [Seth]

[laughs] Yeah, I think it

01:11:29,002 --> 01:11:41,262 [Seth]

wouldn’t be the first thing I tell people, perhaps, when I meet someone in Brooklyn. It’s like, “Oh, I work for Vigo AI Lab, by the way.” But I think it depends who and where and what. You get a bit of the stereotypes of, oh, AI bad, evil, blah, blah.

01:11:42,402 --> 01:11:51,922 [Seth]

But you also have some cool communities and interesting creative scenes that actually do engage with technology AI in very interesting ways. And so,

01:11:53,402 --> 01:11:56,412 [Seth]

there’s a bit of everything, but that’s kind of the point also. I’m

01:11:57,522 --> 01:12:01,051 [Seth]

enjoying being here, so I don’t have to talk about AI with every random person I meet. [laughs]

01:12:01,051 --> 01:12:01,122 [Andrey]

[laughs]

01:12:01,122 --> 01:12:11,922 [Seth]

And there’s all these other things that I can kind of explore and discuss and learn about, too. But the attitude really depends on where I end up. There’s a bit of a selection effect going on as well, to be fair. [laughs]

01:12:11,922 --> 01:12:16,022 [Andrey]

The coolest thing in terms of art and AI that you’ve seen out there?

01:12:17,122 --> 01:12:17,402 [Seth]

Hmm.

01:12:18,982 --> 01:12:20,202 [Seth]

In New York specifically, a

01:12:23,142 --> 01:12:27,582 [Seth]

lot of the recent art stuff I’ve been to was not really AI related. Although next week, though, I

01:12:28,862 --> 01:12:31,182 [Seth]

am in fact co-organizing, helping

01:12:32,302 --> 01:12:43,542 [Seth]

some sort of art dinner thing with a bunch of interesting artists coming from London [smacks lips] and showcasing a lot of their work. And so I think they’re called Biangi Systems.

01:12:44,862 --> 01:13:01,742 [Seth]

I’ll try to put a link in the chat or something. But, yeah, shout out Biangi. But so they do interesting stuff, and you get a lot more interesting artifact from their work than, say, a one-shot Suno prompt or something. [laughs]

01:13:01,742 --> 01:13:03,842 [Andrey]

All right. So then natural segue then.

01:13:03,842 --> 01:13:04,902 [Seth]

[laughs]

01:13:04,902 --> 01:13:14,002 [Andrey]

What is your philosophy of electronic music? Or what do you like about electronic music? What do you not like about electronic music? Yeah.

01:13:17,262 --> 01:13:18,002 [Seth]

I don’t think there’s a

01:13:20,022 --> 01:13:24,322 [Seth]

whole philosophy behind it. I think I just kind of like some types of electronic music. I

01:13:26,082 --> 01:13:30,882 [Seth]

like a lot of late ‘80s, early ‘90s electronic music. And they had a particular kind of...

01:13:32,702 --> 01:13:49,742 [Seth]

They’re basically the original futurist in some sense. And a lot of the themes and the music was very much sometimes around, in fact, AI, or sometimes around robots, sometimes about the future, and both utopian and dystopian visions of the future. And I don’t know. I like the kind of

01:13:51,462 --> 01:13:59,982 [Seth]

[smacks lips] audio equivalent of all these stories or something. I read too much, and so [chuckles] it’s nice to have something that’s a bit more like

01:14:01,402 --> 01:14:02,422 [Seth]

that engages in a different way.

01:14:03,822 --> 01:14:07,682 [Seth]

So I don’t know. There’s a lot of early ‘90s techno, acid, trance.

01:14:07,682 --> 01:14:09,822 [Andrey]

What are those for our listeners?

01:14:09,822 --> 01:14:16,582 [Seth]

Well, Drexciya is the kind of canonical one, which is kind of Detroit electro techno from the early ‘90s.

01:14:17,270 --> 01:14:24,430 [Seb Krier]

And they have a whole mythology around the aesthetics, the themes, the kind of

01:14:25,530 --> 01:14:26,430 [Seb Krier]

storyline behind

01:14:27,530 --> 01:14:30,670 [Seb Krier]

the tracks and the EPs and albums they put out.

01:14:31,770 --> 01:14:45,630 [Seb Krier]

So I think Drexciya is a really interesting one. There’s also very cool long articles about them, I think James Stinson and Gerald Donald, and a few others. I think a lot of that Detroit scene at the time was very much into that stuff. So, Ox88, Jeff Mills, Underground Resistance, a lot of the

01:14:46,730 --> 01:14:52,950 [Seb Krier]

classic techno electro stuff from that time. So these would be some examples, but the UK had some really cool stuff too.

01:14:54,770 --> 01:15:02,690 [Seb Krier]

Yeah. And I guess I like the weirder side of music, and when it’s a bit more creative. Which is why I’m going to--

01:15:04,050 --> 01:15:08,110 [Seb Krier]

To the earlier discussion about creativity, I kind of think of slop as

01:15:09,310 --> 01:15:12,330 [Seb Krier]

very much something that’s kind of highly predictable.

01:15:13,910 --> 01:15:18,270 [Seb Krier]

Low creativity, high predictability, low effort.

01:15:21,110 --> 01:15:44,950 [Seb Krier]

One thought I had was that over time, you see this kind of lowering of the barriers to all sorts of art forms. So even this early ‘90s stuff. You had a lot of really good concentrated stuff, but very few people had access to this stuff. And of course, as these synthesizers would be cheaper, as they’re then made kind of digital, as you had tools like Ableton and whatnot, a lot more people were able to create and make music. But as a result of that,

01:15:46,030 --> 01:15:51,810 [Seb Krier]

you had a lot more of the mediocre and average stuff too. But that’s fine. There’s also a lot more of the really good stuff, and if you’re

01:15:53,110 --> 01:15:57,030 [Seb Krier]

good at finding it, or if you can rely on good algorithms, you can filter through the slop.

01:15:58,710 --> 01:16:00,190 [Seb Krier]

Yeah. That’s one thought.

01:16:01,670 --> 01:16:05,730 [Andrey]

I think that the

01:16:07,070 --> 01:16:11,130 [Andrey]

current-- I don’t even know current. Over the past decade, the-

01:16:11,130 --> 01:16:13,170 [Seth]

[background chatter]

01:16:13,170 --> 01:16:13,330 [Andrey]

... the

01:16:14,370 --> 01:16:23,150 [Andrey]

EDM wave. Does it feel very sloppy to you, even somehow before we even had AI-generated music?

01:16:23,150 --> 01:16:27,769 [Seb Krier]

Yeah, definitely. But on the one hand, it’s kind of my taste. I’m sure there’s

01:16:28,990 --> 01:16:34,890 [Seb Krier]

a big level of subjectivity in a lot of this stuff. And then also there’s-- I think you had-

01:16:34,890 --> 01:16:36,650 [Seth]

This podcast is not Yuck Yums.

01:16:36,650 --> 01:16:37,010 [Seb Krier]

[laughs]

01:16:38,790 --> 01:16:40,000 [Seb Krier]

But also, there’s I think all sorts of--

01:16:41,650 --> 01:16:49,370 [Seb Krier]

You had slop equivalent in other fields, in music genres, and domains over time. And I think even in the, I don’t know, was it the ‘80s or something, people would consider,

01:16:51,150 --> 01:17:08,830 [Seb Krier]

I would say smooth jazz as being the equivalent of slop at the time or something, right? [laughing] And I like smooth jazz. I’m really keen on smooth jazz. Even library music or something at the time. So yeah, library music actually was seen as at the time it was just mass-produced, super commercial beast film score music that

01:17:10,030 --> 01:17:28,649 [Seb Krier]

was probably kind of not seen with a lot of, I don’t know, awe and respect or something. Maybe the bye bye. Whereas now it is, of course, because what is slop also changes over time. But yeah, I think just like photography. Once that got democratized, you’ve got trillions of photos every day, and there’s a huge amount of slop in that whole sea of photos that people take.

01:17:28,650 --> 01:17:33,630 [Seth]

And there’s like you say, there’s also a going back and rediscovering. Lovecraft at one point was slop.

01:17:33,630 --> 01:17:33,820 [Seb Krier]

Yeah.

01:17:33,820 --> 01:17:35,130 [Seth]

And now he’s beloved.

01:17:35,130 --> 01:17:36,750 [Seb Krier]

Yeah. Exactly. There’s

01:17:37,870 --> 01:18:21,510 [Seb Krier]

a lot of house music from at some point that was just deemed pretty kind of basic or something, and now it’s coming back a little bit in some communities in some shape or form. So it’s a weird mix of taste and rediscovering old stuff and scarcity because people like being special unique snowflakes and so on. So there’s [chuckles] also the different dynamics at play. But yeah, in my mind, I don’t like EDM or something, but who knows? Maybe in 20 years my tastes will have changed, and maybe I’ll look back and be like, “Oh, actually there’s some pretty good stuff here.” I remember very vividly when I was 18 thinking, “I really hate the sound of acid.” As in the acid techno, these two or three machines. I was like, “I can’t understand how one would like that.” And then literally five years later, six years later, I was just obsessed with acid. And I was like, “How did that happen?”

01:18:22,870 --> 01:18:26,380 [Seth]

It’s because the opposite of love isn’t hate, the opposite of love is indifference.

01:18:26,380 --> 01:18:26,490 [Seb Krier]

[laughs]

01:18:26,490 --> 01:18:30,370 [Seth]

The fact that you hated it so much was a sign.

01:18:30,370 --> 01:18:37,149 [Seb Krier]

Well, I didn’t hate it that much. I just didn’t find it particularly interesting or pleasurable, whatever, at the time. But so yeah.

01:18:38,490 --> 01:18:38,700 [Seb Krier]

Who knows?

01:18:38,700 --> 01:18:41,090 [Seth]

You mentioned reading. You mentioned reading a lot.

01:18:41,090 --> 01:18:41,290 [Seb Krier]

Uh-huh.

01:18:41,290 --> 01:18:43,470 [Seth]

What’s the book that’s been the most influential on you?

01:18:47,170 --> 01:18:53,470 [Seb Krier]

I don’t know. I definitely don’t have an immediate answer to that question because I think particularly in the last decade, my

01:18:54,910 --> 01:18:58,510 [Seb Krier]

consumption habits have been a lot more chaotic.

01:18:58,510 --> 01:18:58,950 [Seth]

[chuckles]

01:18:58,950 --> 01:19:04,630 [Seb Krier]

So there’s been a bit of-- I don’t know. If you asked me the same thing 10 years ago, I would’ve said something like, oh, “On Liberty” by John Stuart Mill or something.

01:19:05,830 --> 01:19:11,150 [Andrey]

See, that’s why-- Isn’t that Tyler’s favorite philosopher?

01:19:11,150 --> 01:19:12,160 [Seb Krier]

I don’t know if it is.

01:19:12,160 --> 01:19:13,030 [Andrey]

Sorry. Economist, excuse me.

01:19:13,030 --> 01:19:13,050 [Seth]

No.

01:19:13,050 --> 01:19:14,270 [Andrey]

His favorite economist. Yeah.

01:19:14,270 --> 01:19:16,750 [Seb Krier]

Right. Well, once again, great minds. What can I say?

01:19:17,890 --> 01:19:21,190 [Seb Krier]

[laughing] But no, I do really like John Stuart Mill. But I

01:19:22,550 --> 01:19:30,570 [Seb Krier]

wouldn’t-- Right now, I think it’s kind of a weird patchwork of loads of different kind of text sources.

01:19:30,570 --> 01:19:32,590 [Seth]

You mentioned Bostrom’s “Superintelligence” during the conversation.

01:19:32,590 --> 01:19:44,470 [Seb Krier]

Yeah. I mean, that one was clearly super influential. Because I think that was-- He published in 2014, I think, and in 2016, ‘17 was when I was starting to read it. And that got me to actually kind of what is part of the motivation to leave my job as a lawyer at the time.

01:19:45,870 --> 01:19:54,570 [Seb Krier]

And see that AI was obviously going to be a big thing, and you had a need for non-technical work as well in that whole thing. So,

01:19:55,690 --> 01:19:59,790 [Seb Krier]

that was fairly influential, I think. Yeah, Bostrom’s “Superintelligence” clearly played a role.

01:20:01,918 --> 01:20:04,138 [Seth]

Mentioned being loved by Tyler

01:20:04,138 --> 01:20:06,547 [Seb Krier]

I didn’t mention that, you did. [laughs]

01:20:06,547 --> 01:20:09,838 [Seth]

Question, why does he love you so much? Do you have kompromat on him?

01:20:09,838 --> 01:20:14,848 [Seb Krier]

[laughs] No, I don’t know. I was really happy that he kind of enjoyed my

01:20:16,458 --> 01:20:38,538 [Seb Krier]

hot takes and rambles on Twitter. And I’ve messaged him to say the same. And funnily enough, I’ve been a kind of a Marginal Revolution reader since the last 15 years or so, or for a very long time. So it’s actually pretty weird for me to be like, “Oh, cool. My name’s popping up on the blog,” and stuff. A lot of my friends, in here at least, knew about Marginal Revolution, so they’re like, “What are you on about? Who cares?” And to me-

01:20:38,538 --> 01:20:39,447 [Andrey]

[laughing]

01:20:39,447 --> 01:20:46,158 [Seb Krier]

... that was pretty cool. Like, yeah, Seb Krier, whatever. And like, all right. But, no, I don’t know. But I think I

01:20:48,038 --> 01:20:52,378 [Seb Krier]

like Tyler and his work a lot. I think Tabarrok as well. I think the whole,

01:20:54,598 --> 01:20:55,298 [Seb Krier]

what’s it called,

01:20:57,618 --> 01:21:14,238 [Seb Krier]

that cluster of blogosphere online economics, GMU, and so on, has been fairly influential, I think as well for me. Even 15 years ago, when I was kind of switching from my Francophone world and upbringing to the kind of more Anglophone where discovering the Adam Smith Institute in London or something.

01:21:15,278 --> 01:21:17,598 [Seb Krier]

All these different influences, I think, definitely had a

01:21:18,698 --> 01:21:20,618 [Seb Krier]

lasting effect in some sense.

01:21:20,618 --> 01:21:24,058 [Andrey]

Pretty good time to wrap up. Is there anything else you want

01:21:25,538 --> 01:21:31,078 [Andrey]

to direct our audience to? Any underappreciated people, ideas,

01:21:32,138 --> 01:21:32,738 [Andrey]

anything else?

01:21:34,077 --> 01:21:34,217 [Seb Krier]

[laughing]

01:21:35,358 --> 01:21:35,798 [Seth]

[laughing]

01:21:35,798 --> 01:21:41,178 [Seb Krier]

That’s a wide question. I don’t know. There’s all sorts of interesting people and ideas. I

01:21:43,518 --> 01:21:46,418 [Seb Krier]

don’t know. Cosmos Institute has been doing some interesting work recently.

01:21:46,418 --> 01:21:47,358 [Andrey]

Mm-hmm.

01:21:47,358 --> 01:22:02,158 [Seb Krier]

I think, as well as Fire, actually. They’ve kind of done some stuff together, which I find very valuable. Yeah, I’d have to think about that one more. I’d prefer preparing a nicely curated list of underrated-

01:22:02,158 --> 01:22:02,168 [Andrey]

Yeah. [laughs]

01:22:02,168 --> 01:22:04,378 [Seb Krier]

... blogs, people, and ideas rather than-

01:22:04,378 --> 01:22:06,718 [Andrey]

Well, you should tweet that. That could be your next-

01:22:06,718 --> 01:22:08,348 [Seth]

There we go. Yeah, I’ll retweet it

01:22:08,348 --> 01:22:09,598 [Andrey]

... super tweet. Yeah.

01:22:09,598 --> 01:22:13,898 [Seb Krier]

Yeah, no, for sure. I’m sure, that’d be a good idea, actually, because I think there

01:22:15,278 --> 01:22:31,458 [Seb Krier]

are a lot of interesting people in the AI world right now doing cool work that isn’t maybe as discussed or something. There’s Calcifer Computing did some interesting work recently. Yeah, there’s a bunch of people here and there, but I’ll have to think about it to give you a more interesting answer.

01:22:32,998 --> 01:22:35,147 [Seth]

Great. We’ll put them in the show notes if you think about it.

01:22:35,147 --> 01:22:35,318 [Seb Krier]

Sounds good.

01:22:36,778 --> 01:22:44,998 [Seth]

Seb, it’s been such an honor having you on. You were an awesome participant and had a lot of patience with our silly questions. [laughs]

01:22:44,998 --> 01:22:52,738 [Seb Krier]

No, it’s been a pleasure. I’ve been a big fan of the show. So again, just like Marginal Revolution, now I can say I’ve got the Justified Posterior’s take. [chuckles]

01:22:52,738 --> 01:22:53,238 [Seth]

There we go.

01:22:53,238 --> 01:22:54,638 [Andrey]

Nice. Thank you.

01:22:54,638 --> 01:22:54,958 [Seb Krier]

Thank you.

01:22:54,958 --> 01:22:56,198 [Andrey]

All right.

01:22:56,278 --> 01:22:56,558 [Seb Krier]

Cool.

01:22:56,558 --> 01:22:58,978 [Andrey]

Keep your posteriors justified. And-

01:22:58,978 --> 01:23:00,338 [Seb Krier]

[laughs]

01:23:00,338 --> 01:23:04,318 [Andrey]

... like, follow, comment, subscribe, et cetera.

01:23:04,318 --> 01:23:04,958 [Seb Krier]

And retweet.

01:23:04,958 --> 01:23:05,598 [Andrey]

Thank you.

01:23:05,598 --> 01:23:09,508 [Seb Krier]

And retweet. [laughs]

01:23:09,508 --> 01:23:17,198 [Andrey]

Of course. [outro music]

To do that well, we’re investing in every aspect of our production. That means more editing help, better equipment in our home studios, and occasionally traveling to record with guests in person. We have ambitions for JPP to sound closer to Odd Lots than to two professors on laptops, and getting there costs money.

It’s essential to our mission that our content remains accessible unconditionally, even as we update our priors. (And yes, we’ll keep making Bayesian puns.)

For those of you with the means, who think this work is well Justified, we’re opening up two ways to support the show on Substack:

• Subscriber ($10 a month, $99 a year): a quarterly virtual hangout with us and other paid subscribers.

• Founding member ($499 a year suggested): the ability to request a topic or guest (which we’ll honor subject to editorial discretion), a WhatsApp chat with us and other founding members, a shoutout on the podcast if you’d like one, and the quarterly Zoom hangout.
  

We’re also on the lookout for grants and other forms of institutional sponsorship. So if you know of any organization interested in supporting this type of work, please let us know.

Keep your posteriors justified,

Andrey and Seth

The conversation starts with Avi’s seminal book, Prediction Machines, and the idea that AI is best understood as a drop in the cost of prediction, which is a complement to judgement. We ask what that book got right and what it got wrong. From there, we interrogate Avi on the murky boundary between prediction and judgment. We had investigated the idea that maybe judgment and prediction were not as separable as economists like to believe in our episode with Alex Imas.

We also ask whether, if AI gets better at predicting human judgment, whether judgment disappears, or do humans simply “move up the stack”? And what is taste exactly? Avi says that sometimes judgment becomes predictable, but humans still matter because goals, values, organizational politics, and “what matters” are often implicit, unstable, and hard to codify. Avi shoots down Seth’s galaxy-brain suggestion that correct ontology choice — i.e., deciding what sort of natural kind a thing is, or understanding when a problem is out of context — is a uniquely separate skill (taste?), calling it just another prediction error. But he does concede that deciding how much to prepare for ‘Black Swan’ events may be an enduring role for judgment.

We then revisit the O-ring theory of production and what it means for automation. We had covered Kremer’s article in a recent episode (see here) and asked Avi about his new paper, riffing on the idea at the worker level. Avi says that if tasks inside jobs are complements rather than substitutes, then automating one task may make the remaining human tasks more valuable, not less. Avi explains why workers may reallocate attention toward the tasks machines cannot yet perform (shooting down Seth’s suggestion that this is actually difficult in most jobs).

The discussion also covers whether AI will augment or replace workers, whether governments should try to steer AI toward human-complementing technologies, and why that distinction may be much harder to define in practice than it sounds. Avi agrees with Andrey and Seth’s pushback on “augmentation good, automation bad” framings (e.g. friend of the show Erik Brynjolfsson’s “Turing Trap”).

Then we get into forecasts: how fast AI capabilities might advance by 2030, what that means for GDP growth by 2050, whether GDP is still the right thing to forecast, and why even very powerful AI may run into bottlenecks in the real economy. We use the paper Forecasting the Economic Effects of AI to ground the discussion.

We close with lightning-round topics including AI’s impact on centralization, privacy/de-anonymization, peer review, and whether academic journals still serve the function they once did.

Papers, books, and ideas mentioned

• Avi Goldfarb’s seminal book with Ajay Agrawal, and Joshua Gans — Prediction Machines

• A black swan is the occurrence of a wildly unpredictable event, which Nassim Taleb argues, in his book by the same name, is more common than we like to think

• A New Riddle of Induction — by Nelson Goodman — is the source of Seth’s thought experiment about “bleen”, a color which is green until 2029 and blue after, and green

• Michael Kremer — “The O-Ring Theory of Economic Development”, covered in this episode of the pod:

• Daron Acemoglu and Pascual Restrepo’s task-based models of automation, especially “The Race Between Man and Machine.”

• Avi mentions David Autor and Ben Thompson on automation and skill scarcity when Seth comments that you may not be able to reallocate effort between tasks as a worker, including their paper “Expertise”

• Erik Brynjolfsson in the “Turing Trap” argues that automation technologies are less good than augmenting technology

• Eric Topol’s book on AI in medicine — Deep Medicine

• John Markoff — Machines of Loving Grace — The source of a title for an influential essay of the same name by Dario of Anthropic. Both draw from an earlier poem about a Sci Fi utopia: https://allpoetry.com/All-Watched-Over-By-Machines-Of-Loving-Grace

• Korinek and Stiglitz on AI, capital, and taxation; Lockwood and Korinek on optimal taxation and automation — We covered these topics at the end of our episode with Basil Halperin in the context of “Tax Policy at the End of History” around the 1:19:00 mark

• We talk about de-anonymization, and Avi references this provocative paper from Florian Ederer

• Avi brings up Bob Gordon, and his argument, famously in the book The Rise and Fall of American Growth, that the early 20th century was incredibly important for increases in US living standards, which digital technologies have not lived up to

• Digital Hermits, by Jeanine Miklós-Thal, Avi Goldfarb, Avery M. Haviv & Catherine Tucker, is a paper by Avi thinking about how information spillovers, now from AI, drive some people to be more private than they would otherwise be. In our conversation, we speculate AI will make these hermits even more “hermetic”

• We discuss this paper on new forecasts of AI and its impact on economic growth: Forecasting the Economic Effects of AI

• Refine and AI-assisted peer review are discussed in this pod. For more, see our episode with Ben Golub, founder of Refine.

This episode is sponsored by Revelio Labs — a great source of labor economics data for academics and firms. Now available on WRDS.

Join our Discord community at this link: https://discord.gg/w3GSapx2d

Transcript

Introduction [00:00]

Seth: Welcome to the Justified Posteriors podcast, the podcast that updates beliefs about the economics of AI and technology. I’m Seth Benzell, your loyal non-fiction machine, coming to you from Chapman University in sunny Southern California.

Andrey: And I’m Andrey Fradkin, coming to you from San Francisco, California. And we are very happy that Justified Posteriors is sponsored by the fine folks at Revelio Labs. And we’re very delighted to have Avi Goldfarb, who is a leading thinker in the field of AI economics and has also been a personal mentor on the show. We’re very excited to hear his thoughts on a variety of topics. Welcome, Avi.

Avi: Thanks so much and thanks for having me on the show and looking forward to it.

Andrey: All right, let’s get started. I have in front of me this book that you might remember writing at some point.

Seth: Gaze into the soul of the man in the bookstore.

What Did Prediction Machines Get Wrong? [01:12]

Andrey: Now, I just think it’s a good cover. And I had to check: when was it released? It was released in 2018. And as I was skimming through it, you know, a lot of interesting points made there are still things that we’re talking about today, almost 10 years after it was released. So let me start off with the following question. And then maybe we can work backwards more into the ideas in the book. But what do you think prediction machines got wrong?

Avi: I think prediction may... I’ll start with a hard question.

Seth: No softballs on Justified Posteriors.

Avi: So on the specifics of which industries and when, to the extent we tried, at least I did not anticipate how quickly language and coding would become prediction problems. And when we talk about disruption and industry disruption, a lot of the examples are things like driving, and we talk about radiology. And we still have plenty of radiologists around. Self-driving cars and trucks. seem like they’re now imminent, but it certainly took a lot longer than we expected back in 2018.

Andrey: So is it a fair assessment to say that the large language models, even in 2018, weren’t on your radar? I guess they weren’t on many people’s radar.

The Three Ideas of Prediction Machines [02:45]

Avi: Not really. We have some discussion of machine translation. So that’s in there as a huge potential use case, but the arrival of ChatGPT and how it sort of changed how we interact with machines and how we think about AI was not really there. Another way to put it is prediction machines had three ideas. So idea number one is AI can be framed as a drop in the cost of prediction. So prediction. As in filling in missing information, statistical prediction is getting better, faster and cheaper. Idea number two is that when something gets cheap, you start using it for unanticipated uses. So when arithmetic got cheap, it wasn’t just that we use computers for accounting. We started to use computers for all sorts of things that we never used to think of as arithmetic problems like imaging and mail and music. And then idea number three is what are the complements to machine prediction? And we talked about data and judgment. The book, and certainly our attention to the book in the first three or four years after it was published, was on idea number one and idea number three. So identify prediction problems in your organization, and then think about what data you need to make those predictions better, and try to understand what matters to you in terms of judgment. And that second point kind of got lost. But in the last four years, it’s become clear to me is that that second point was maybe the biggest one, which is this tool, which still under the hood is computational statistics, enables us to find all sorts of applications for computational stats that we didn’t really imagine before. Judgment and data are still gonna be useful, but that phase one, that step one, that first idea of identifying prediction problems, that’s not really how we think about using AI today. And in some sense, that... was a missing emphasis throughout the book and throughout how we thought about that book, or at least how I thought about that book for the first few years.

Does Proprietary Data Still Matter? [04:59]

Andrey: Very interesting. You mentioned one kind of underlying idea there, whereas you should identify the data that’s going to make your predictions better. Do you think to what extent is that now true, given that your foundation models seemingly can be very smart without having any proprietary data?

Avi: Data is still central to the use of AI, the building of the models. In building a foundation model that, at least in the pre-training stage, that data is essentially interchangeable. You just need more. It doesn’t really matter what. To build a structure of language, and then you can move from there. On later stages of using that model, at least the AI companies seem to think data is valuable to the model companies. And then in terms of use cases within organizations, that’s more a matter of whether you want to delegate sort of the judgment of how to use the model and what the model should output to the vendor or whether it’s something that you need to build in-house. And depending on the organization, some of them are very happy to delegate to the foundation model provider and some of them think they need to fine tune in-house.

Andrey: Well, so there are kind of two little sub ideas in there. One is you have choice. You can fine tune a worse model with your own data. And maybe that will outperform as a frontier model. I think for many cases so far, that’s been a bad bet. But there’s a different idea here. Use whatever model you want, but you design the evaluation. And then you optimize via the prompting strategy or scaffolding towards that. that benchmark for your own use case. Is designing a benchmark proprietary? Should we think of that as a proprietary data that an organization has?

Seth: Is that the judgment part in the judgment prediction distinction?

Vendor Choice as Delegated Judgment [07:01]

Avi: Yeah, I think there’s a bunch of judgment. there’s judgment number one: which which vendor do you use? Because you’re delegating a lot of values as in like, knowing what matters to the maker of the model. And then there is judgment in how heavy-handed do you want to be to make the outputs fit your needs? And then there’s judgment on, okay, you’ve decided to be heavy-handed. What exactly does that mean? And is it, guardrails or is it really making sure that the output from the prompts every time fits your organization’s values or what matters to you?

Andrey: Have you had an opportunity to kind of advise companies on this judgment decision? Like what has your experience been in these situations?

Avi: At a high level, yes. I don’t want to exaggerate my experience, but the things I emphasize and the things that seem to resonate are, one, what I just said, which is recognizing when you choose a vendor, you are delegating your understanding of what matters to that vendor. And then two, that means before you start thinking about choosing a vendor, you need to know what matters to you. So think through, you know, before you go talk to somebody, you should know what your KPIs are and what outcomes you want to see. Because otherwise, once you talk to them, they’ll convince you that their outcomes are the ones you want to see. and so it’s this, I talked to, someone who is running an AI at a... Let’s call it a big healthcare organization. And his job used to be, like five years ago, his job was building tools. He’s like, my job isn’t building tools anymore. There are all sorts of vendors building AI tools for healthcare. Okay. And what my job is now is every week, 20 or more people come in and say, I have a solution for you. And he chooses one or two of them.

Seth: Kind of seems like a good job for an AI.

Avi: Well, maybe, maybe not. But he understands the individuals, the people, guess, in theory that could happen, but the individuals in his organization, what they’re willing to accept, what they don’t. Which decisions they like to have control over, which ones they’re comfortable delegating. For the ones they like to have control over, he has a sense of what might be negotiable and what might not be. He knows where the power structures are and what things might change. Therefore face resistance from people who have the power to resist. He knows those things that might not face resistance from people because the people don’t have power to resist, but they’re going to be really, really unhappy about it. It’s going to bad for the organization. And so there’s all these things that I guess in principle an AI could do, but we’re a long way away, I think, from that.

Can Prediction Eat Judgment? [10:16]

Seth: So let me let me just push down that line a little bit longer is the way to think about this sort of prediction and judgment distinction is is that like as the models get better the Prediction is like eating more and more of the stack right? You know we give the information about our organizational structure to the AI and then maybe it can make a couple more of these decisions for us And you could either imagine that asymptoting to, you know, in 20 years, AI does everything, or you could imagine there are higher and higher levels of judgment that humans keep on getting promoted to. Are one of those two ways the way that you think about it?

Avi: Yes, Andrea Pratt has a note in our first Economics of AI volume that covers that exact idea. I think actually it’s a comment on our paper or the model behind the Prediction Machines book. it’s, well, in principle, with enough data, you can learn to predict judgment. And so you move up the stack. So absolutely. There are some limits to that. There’s limits on you may never get enough data. on that kind of judgment. Judgment can change over time. To the extent that ultimately you’re trying to predict your tastes, then they can change over time. And there’s some limits on causal inference and the impossibility of seeing the counterfactual, which creates a need for a model.

Andrey: But humans have that problem too.

Avi: Yeah, yeah, yeah, no, I agree. But in the need for a model. So then the question is, well, how come LLMs and some of these models seem to be pretty good at doing that? And in the process of prediction, I suspect -- though I don’t know rigorous work on this, so I’m being cautious --

Seth: That’s what this podcast is for.

Avi: this is building some kind of model of the world that is embedded in the training data, like the language.

Taste, Values, and Human Wants [12:16]

Seth: So let’s go back to the one of the examples you gave, which is this idea of taste, right? Because I’ve had so many conversations with other economists about this idea that, well, taste will save us as a scientist, right? Because the AI won’t have taste. I have some ideas about what taste might mean, but can you be a little bit more precise about what you think taste means and why it’s something worth saving?

Avi: So, okay, let’s operate under the assumption that whatever we want to call the machines, their goals are to help humans. Okay, not all humans. And we can debate about which humans, but like ultimately.

Seth: Well, the Anthropic Constitution says, you know, safety first, the idealized anthropic researcher, then the guy that then then like virtue and then like the customer in some order like that.

Avi: I’m gonna, all that matters for the point I’m about to make is that it’s not about the machine’s needs. So in that case, at the very limit, humans have wants and needs and those wants and needs, the machines need us, our judgment to know what our wants and needs are.

Seth: So taste literally as in, this tastes good to me, I want more of this food.

Avi: That would be one specific example of it. Absolutely. Okay. Now, I think we’re a long way from that limit, but that’s what I would argue the limit is.

Seth: That’s the Bailey, right? So now let’s go out to the motte.

Avi: So then it’s more like, okay, what matters to a set of humans, a group, an organization? What can we codify? If you can codify it and say, like, this is your goal, you’re not quite at that limit, but pretty close to it, then the machines can try to optimize on a goal. Goals have so much that are implicit. And so the machine would have to be able to infer the implicit part. Maybe it can, maybe it can’t, I don’t know. And then you can sort of ratchet back all the way to where we are now, which is you still need to tell your agent what you want. You still need to check on it every once in a while and guide it in the right direction. Prompting still has a role.

Ontology, Umbrellas, and Context Shifts [14:45]

Seth: Here’s another way of thinking about taste. And I’m curious whether you think this is in one of the categories you already listed or a new idea or you wouldn’t call this taste, which has to do something like with the idea of your ontology that is kind of built into the system, right? It’s your way of sort of dividing the world up into parts and maybe a good tastemaker or a good judger might have a more refined or more adaptable ontology. than the prediction machine. So I’ll give you an example of what I mean. have a couple of examples in mind, but one example I have is, you know, historically in the data, it’s always been the case that if lots of people show up with umbrellas, it means that you can predict that it’s raining. But then we have these Hong Kong protests and in the Hong Kong protests, they’re the umbrella protests and people bring umbrellas to show that they’re protesting, right? And it seems like a human would do better at adapting to like the completely new context for why you would need umbrellas than, you know, a pre-trained system that was only on historical data. So you can say that that’s like a context switch problem. Is that one of your ideas of taste or is that more of a judgment that’s not a taste?

Avi: Honestly, that seems like a prediction failure to me.

Seth: Right. That’s just we don’t have data on the context that we’ve moved to. The job is to understand when the context has changed, maybe.

Avi: The judgment, I would say the judgment is like, what’s the consequential decision that’s going to be a function of, look outside and I see a lot of people in umbrellas. Yeah. What am going to do? And.

Seth: You know, I should water my plants. Should I water my plants?

Avi: No, I water my plants. Okay. So I look outside, a lot of people are carrying umbrellas and I think, no, I don’t need to water my plants. Okay. And then it turns out it’s a protest. It’s a little bit of weird context, but going with your example.

Seth: It’s gotta be a weird context. That’s the reason that the AI is going to make the wrong decision because it’s out of context.

Avi: the, the automated sprinkler doesn’t go on and, my plants die. Right. Okay. So, the judgment is, is it then worth it for me to invest more either in my prediction technology or to actually go outside and look and to see if there’s rain, to overcome that downside. So what you described as an error in prediction, there’s ways to reduce that error in prediction. The judgment is whether it’s worth the bother to reduce that error in prediction or to create some kind of insurance system where you would say, you know what, I’m gonna water the sprinklers. I’m just gonna run the sprinklers anyway. That’s how I think about judgment. It’s sort of what goes wrong when your prediction fails or it’s one important aspect of judgment.

Seth: Sorry, can I give you an even more abstract?

Andrey: Wait, wait, wait. No. I actually disagree with the premise of the example in many ways. I think a reasoning model would be able to handle the situation, especially with internet access, substantially better than many humans already, because you can call an API to get the weather forecast if you’re unsure. You can read the news. You can use reasoning traces. There’s this kind of implicit assumption in your question that like, we’re just using a raw pre-trained model and like asking it to like, if you, like, if you had a gun to your head, what would you do? You know, and not use any reasoning.

Seth: Okay, but I can tell you a story, right? The weather API was always reliable in the data, but now there’s been a government takeover and I don’t trust the new government and you shouldn’t trust the API weather data anymore, right?

Avi: So Andrey, I actually agree with, like, that seems unrealistic, but I think the idea is what you’re describing is how many resources you wanna put toward making it right, and I would view that as judgment.

Andrey: But I guess the model has that judgment, maybe. Already. Already. Yeah, that’s kind of goes out like the stack of when judgment problems become prediction problems, I guess.

Avi: But then there’s going to be... well, there’s going to be some places where the model is imperfect. Okay. Yes. Still a prediction tool. It might be better than human. Actually, it doesn’t matter if it’s better than human. But to the extent the model is imperfect, how do you want to behave? Like, let’s say the model is right 99.99 % of the time. Does your behavior change at that versus 99.9999 % of the time, even if the human benchmark is 50? And that ultimately is going to is going to be essential to judgment. We do this with self-driving cars. The models aren’t perfect, but they’re better than human. And yet, I still drove to work today, partly because that’s the law in Canada.

Andrey: Do you think there’s hope? I mean, maybe this is kind of too much in the weeds versus the abstract idea, but sometimes people implicitly assume that they’re anchoring on the current technology where there’s an instance of an LMM that does something. But we might be able to design systems of LLMs that are interacting with each other to cover some of these. shortcomings that we can think of. I mean, at a conceptual level, maybe it’s the same thing anyway...

Avi: So maybe another way to think through these trade-offs is to talk about whose judgment, okay? Which is Seth’s example was about, or my example was about my judgment, know, the individual’s judgment and should they listen or not. Andre, I think what you’re describing is the model builder’s judgment on which things is it worth investing in making the model better and when is it okay not? Like they have choices on sort of rate and direction. And those require some understanding of what they think is going to matter in terms of the use cases, the model. And on that, yes, there is a limit where a small number of players have extraordinary power because AI scales their judgment because they embedded into the models. But I do think. then there is still a human or set of humans responsible. It’s not like, the AI did it. It’s humans making those kinds of decisions. And I understand, like, at the limit, that actually gets quite nuanced, especially once we have models with continuous learning. But that’s how I think about that problem.

Grue, Bleen, and Black Swans [21:41]

Seth: All right Andre, can I ask my riddle of induction question?

Andrey: Do you need me to induce it?

Seth: You already know where I’m going with this. I’m curious if Avi knows where I’m going with this, but this goes back to the question of maybe where taste comes in is having a better or a more human ontology than the machine. All right. Have you ever heard of grue and bleen, Avi? These are colors that are different than blue and green. No? Okay, awesome. So briefly, we have this conceptual category, which is a thing that’s green. And a thing that’s green, we think that if you don’t do anything to it, it should be green indefinitely, right?

Avi: Okay, yeah.

Seth: All right. There’s this other thing that’s called bleen and things that are bleen are green until the year 2029. And after 2029, they turn blue. Right. Here’s the issue is that bleen and green things are observationally identical until 2029. Right. Yeah. So an inhuman, bad at forming natural kinds, ontology of an AI might decide that something is bleen instead of thinking it’s green. Right? And a human’s role might be to say, no, that’s a bad definition of a natural kind. That’s a bad ontology. And that would be a role of either taste or judgment. Do you buy that? Is this way too abstract?

Avi: I think what you’re describing is a failure of prediction. I don’t think that’s taste or judgment. The taste or judgment is if you or a machine aren’t sure if something is bleen or green, do you care?

Seth: Okay. Well here’s the thing, you didn’t even have the concept of bleen until I told you about bleen, right?

Avi: So this is just the difference, I think, between known unknowns and unknown unknowns. So in Prediction Machines, we have a whole chapter framed on Rumsfeld and his discussion of known unknowns and unknown unknowns. Look, sometimes you don’t have a prior on it, and it’s an unknown unknown. That doesn’t mean that it’s not a prediction failure. It was just off the support of your data, and you didn’t know what to do about it. And I think that happens all the time.

Seth: Sometimes you find a black swan.

Avi: Yes, exactly. And so like, there might be places where humans are better at that kind of prediction than machines. There might be places where both humans and machines are really awful at that kind of prediction. And if that’s the case, then you want to have robust systems to anticipate those kinds of things. And that’s where judgment comes in. Like, if you’re wrong about the existence of a black swan, you know, does that change anybody’s behavior? I think the answer is no, because black swans and white swans aren’t actually that different from each other. But if there were other examples, like financial crises, where he uses the metaphor of the black swan, then absolutely there are meaningful differences. And you should

Andrey: Financial crises.

Seth: All right, so you’re saying that jobs that will survive TAI number 7 should be Black Swan, anticipator.