A hypothesis of the article is that heavily AI-assisted books may have low average quality, but are so easy to produce that you get lots of ‘shots on goal’ for an outlier good book. A few good valueable books are added in addition to masses of slop. But if you assume free disposal on slop, you would accept this as a positive exchange.

Does their data change our views on this topic? We’ll read to find out, and along the way bring in Borges’ Library of Babel, the economics of free disposal, preferential attachment models, and the digitization-of-music literature.

Priors

Hypothesis 1: Has AI increased the number of books released from 2022 to 2025?

• Andrey’s View:

  • Prior: Yes, by about 50%. The fall in the cost of writing a book has been so great that the number must have gone up. Analogous to how students are producing far more written work with AI assistance.

  • Key caveat: The definition of “book” matters enormously — from a major publisher release to a random PDF online. The looser the definition, the bigger the number.

• Seth’s View:

  • Prior: Yes, by about 3x. To the extent that slop gets dumped on the market and is allowed in, a dramatic increase is inevitable. Though he acknowledges it’s still an empirical question — AI also lowered the cost of everything else, including Substack.

Hypothesis 2: Has AI increased the average quality of books released?

• Andrey’s View:

  • Prior: Average quality goes down. ~1% chance it goes up. The slop influx is substantial. Imagine a science fiction author with one semi-popular book who now milks it into a series of increasingly sloppy sequels — that author exists and AI just gave them a turbo boost.

• Seth’s View:

  • Prior: Average quality goes down. ~10% chance it goes up. He raises the “free disposal” argument — authors who would have written anyway only use AI if it makes the book better, which is a force pushing quality up. But the slop influx probably wins. He remains unwilling to put the probability at zero: “Maybe we’re making some real gems here.”

Hypothesis 3 (The Thinker): By 2030, will total social surplus from book reading by humans be higher or lower because of AI?

• Andrey’s View:

  • Prior: 25% chance it goes up. People are reading fewer books over time regardless of AI. Nonfiction manuals and textbooks have a clear substitute in ChatGPT. The form factor of the book seems to be on a secular decline, and new AI-generated books won’t be so good as to reverse that trend.

• Seth’s View:

  • Prior: 75% chance it goes up. LLMs may be complements to reading rather than substitutes — he cites using an LLM to track character names while reading Dostoevsky’s Demons as a present-day example. Good books are a complement to everything else in the economy. If AI makes context and curated knowledge more valuable, books have a real role in the 5-to-10-year time horizon. “I don’t care if my job gets automated because I’ll just move to the woods and read books” — Tyler Cowen, representative of no one but Seth.

Links + Shownotes

• AI and the Quantity and Quality of Creative Products: Have LLMs Boosted Creation of Valuable Books? – The central paper of the episode by Imke Reimers and Joel Waldfogel (NBER, 2025).

• Can an AI Interview You Better Than a Human? – Recent Justified Posteriors episode referenced during the discussion.

• BookStat – The independent data provider the authors use to calibrate ratings-to-sales conversions for Amazon books.

Scholars Mentioned

• Imke Reimers – Co-author of the paper; Associate Professor of Economics at Cornell University.

• Joel Waldfogel – Co-author of the paper; Frederick R. Kappel Chair in Applied Economics at the University of Minnesota Carlson School of Management. Previously co-authored the digitization-and-music paper referenced in the episode.

• Tyler Cowen – Economist quoted on the idea of moving to the woods to read books once automation arrives, and on the question of whether you really want to read the 100th automatically generated biography about an imaginary person. Everyone on the internet is saying how they love him this week, so we’ll join in — we love this guy, and have had the honor and exhilaration of being personally encouraged by him.

• Jorge Luis Borges – Author of The Library of Babel, invoked by Seth to frame the question of what a “book” even is — and whether every possible book has, in some sense, already been written.

• Nicholas Decker — Economist as Reporter – A Substack post about economists being more like journalists in the modern era, cited approvingly in the posteriors section.

• Frank Herbert – Author of the Dune series; his sons’ continuations offered up (by Seth) as exhibit A in the case for sequelitis-as-slop.

• Brandon Sanderson – Fantasy author; Andrey volunteers his later-series books as a possible example of quality decline, before declining to name specific titles.

Connections

• The Library of Babel – Borges’ short story imagining a library containing every possible 300-page permutation of the alphabet. Seth invokes it to ask: if AI can generate any text, what does “a new book” even mean?

• The Barnes Foundation – Seth closes with a defense of collage-as-art, citing Albert Barnes’ idiosyncratic collection of Impressionists, Post-Impressionists, and rusty keys as a model for the authorial value in curation and juxtaposition — even if you didn’t write every word.
  

Subscribe now

Discord Community Link: https://discord.gg/KCJwgkTj


Justified Posteriors Podcast Transcript

“AI and the Quantity and Quality of Creative Products: Have LLMs Boosted Creation of Valuable Books?”

Hosts: Seth Benzell & Andrey Fradkin

SETH: Welcome to the Justified Posteriors Podcast, the podcast that updates its beliefs about the economics of AI and technology. I’m Seth Benzell, racing against the machine for authorial glory before AI transcends all human writers. Coming to you from Chapman University in sunny Southern California.

ANDREY: And I’m Andrey Fradkin, looking forward to SLOP detection technologies all across all my media surfaces, coming to you from San Francisco, California.

SETH: Andrey, how’s it going, man? It’s been a while since we’ve done a paper episode.

ANDREY: I know, I know. It’s great to actually get back to our core of reading and analyzing a paper. And it’s a particularly fun day to be thinking big exuberant thoughts about the quality of society improving because it’s Mardi Gras. We’re recording this on Fat Tuesday. I’ve got my James Carville shirt on, I’ve got my Mardi Gras beads. Are you doing anything special for Mardi Gras this year?

SETH: You know, Mardi Gras is not my religious holiday, but I am flying to Austin for a fun adventure there. But for me, my sort of Mardi Gras actually happened last week, which was the NBER Digital Economics and AI conference.

ANDREY: What a transition. So what parades and what crews were present at that conference?

SETH: Well, we had the structural crew, we had the reduced form crew. We had the economists and then the business school professors.

ANDREY: No macroeconomists. My macro paper was —

SETH: No, no, no. There was one macro paper, one macro paper allowed.

ANDREY: We allow one. Amazing. Any sort of themes jump out at you from the conference?

SETH: Yeah. I think half the papers were AI papers, which I think is more than we’ve had in the past. Digital economics really started as a group thinking about the internet and the spread of the internet. And AI has until this point not been the dominant theme in the group, but it obviously is becoming so. And of course, there was a lot of discussion about what the future of research will look like given how easy it is to produce slop — and also maybe non-slop — with AI.

ANDREY: So speaking of producing slop, today we’re going to be discussing a paper that was presented at that conference. Would you maybe tell us the title and the authors?

SETH: Sure. The title is “AI and the Quantity and Quality of Creative Products: Have LLMs Boosted Creation of Valuable Books?” It’s by our friends Imke Reimers and Joel Waldfogel.

ANDREY: Oh, great guys. Hopefully we can get Imke on the show sometime, or Joel. So — production of slop. A lot of people I know who write have a lot of anxiety around AI coming after their turf. I remember when I was in undergrad there was this idea of the logical cold computer that can never do creative writing, and maybe you should specialize in skills that are complements to that, like long-form writing. And now it seems like increasingly we can use AI for everything. I’m not telling this audience anything it doesn’t know. But this article is actually trying to use some data to get at the question: is AI helping us write more books? Is it helping us write better books? And it’s going to look across fiction and nonfiction.

SETH: Yeah. So why don’t we get to our priors, Andrey?

Laying Out Our Priors

ANDREY: Sure — what are your priors on this subject?

SETH: So it’s a straightforward paper, which is why I really like it, but it gives us some deep things to think about. Around this question of AI making better writing easier, but also making slop easier. The first prior I’d like to ask you about: do we think that AI increased the number of books released from 2022 to 2025?

ANDREY: Yes. I mean, yeah.

SETH: But think of all the things you could do instead of writing books now.

ANDREY: I think the fall in the cost of writing a book has been so great that surely numbers have increased. One analogy is that our students are able to write a lot of essays with substantially less effort.

SETH: Yeah, the amount of words submitted by my students has increased dramatically. I’m with you on this, Andrey. I would be really surprised if the number of books written goes down as a result of AI. I do maintain it’s still an empirical question in principle, because AI also decreased the cost of doing other things — so maybe people substitute into essay writing or Substack instead. But yeah, end of the day, 99% sure the number of books written goes up.

ANDREY: Yeah. And I guess there’s a more subtle question here, which is by how much, and I’m substantially less sure of that.

SETH: What’s your intuition? Give me a point estimate. You feel like 2x?

ANDREY: I think before I read this paper, if I had to introspect, I would think it would be more like up by 50% or something like that. Nothing huge. So that would be my prior.

SETH: My prior would be a lot bigger. To the extent that you think what’s going to happen is a lot of slop getting dumped on the market — conditional on that slop being allowed in — you’ve got to anticipate a big increase. So I’m going to guess like 3x going in.

ANDREY: Well, yeah. And I think this is kind of where the definition of what a book is really starts to matter. Is it that a major publishing house published the book? Is it that there’s a PDF on a random website? The looser the definition, the bigger the numbers surely are.

SETH: I mean, in one sense — are you familiar with Borges’ Library of Babel, Andrey?

ANDREY: Are you trying to insult me or is this a joke?

SETH: Of course you are familiar. And what that library imagines is a library which is very, very large but not infinite — it has every 300-page permutation of English letters. So in a certain sense, every possible book has already been written, Andrey. Just take a deck of playing cards and randomly select one letter at a time.

ANDREY: Yeah, yeah.

SETH: All right. But anyway, the definition we’re going to be working with in this paper is: released on Amazon. The Library of Babel is ruled out.

ANDREY: Yes, yes.

SETH: Okay, second prior, Andrey. Conditional on this definition — needing to be released on Amazon as at least an ebook — would you say that AI will increase the average quality of books released, or decrease it? What’s your percentage chance that average quality goes up?

ANDREY: Yeah, the average will go down. For sure the average has got to go down, at least with the current AI technologies.

SETH: What about free disposal, Andrey?

ANDREY: What do you mean free disposal? The average book made is a different question from the average one that’s read.

SETH: What I’m trying to say by free disposal is that the books that would have been written anyway have free disposal of the technology. They only use it if it makes the book better. So that should be a force that boosts the average quality of books. Of course there’s going to be a slop influx, but there are at least two offsetting effects here.

ANDREY: Yeah, I agree the average could in theory go up, but I think the slop increase is substantial. One way to think about it — imagine you’re a science fiction author and you’ve written one semi-popular book. You can now milk that as part of a series. And unfortunately, we’ve all experienced this. The next books become sloppier and sloppier. And I wouldn’t be surprised if authors lean into the slop so they don’t have to write as much for their subsequent books.

SETH: Right. You’re imagining there’s some quality threshold you have to reach just to have the self-respect to post it online, and that AI can help you clear that bar. But then conditional on clearing it, you don’t invest more in quality — you just release this giant lump of books at minimum quality.

ANDREY: Yeah. And that was already true before AI. Some people were already doing that.

SETH: Do you have any authors in mind that you want to throw some shade at?

ANDREY: No, no, no.

SETH: He’s too nice. I’ve got a couple in mind. The Frank Herbert sons — the additional Dune sequels — I’ve been told are slop. I’ve read pages of them and been warned away from the rest. So that would be an example of selling out a brand name in terms of books.

ANDREY: Yeah. I think some of the Brandon Sanderson later-series books are not that great.

SETH: Is that Wheel of Time, or is that — there’s a magic sword. There’s always a magic sword.

ANDREY: There’s always a magic sword.

SETH: Okay, so anyway — our prediction is that the amount of mediocre magic swords will increase and outweigh the increase in quality of good magic swords. What about Dungeon Crawler Carl?

ANDREY: Definitely fell off in the later books.

SETH: Oh man, I didn’t realize you were an isekai fan.

ANDREY: Is it eye-suh-kai?

SETH: Isekai — “other world” books. Maybe lit RPGs is the more Western term. All right, home audience: you’ve been warned. Don’t read Dungeon Crawler Carl past Book 2.

ANDREY: Once it gets to Book 3 or 4, that’s when it really falls off.

SETH: Book 2 is fine.

ANDREY: Book 2 is fine.

SETH: Okay. I came in thinking the increase in slop books would be even larger — like 3x — which should bring down my prediction about average quality. At least some of the data we’ll look at speaks to this at the book level. And I want to be a little optimistic. I want to say there’s like a 10% chance that average quality goes up. Maybe we’re making some real gems here. I don’t want to put it at 0%.

ANDREY: Never put it at 0.

SETH: Never. No dogmatic priors.

ANDREY: Closer to 1%.

SETH: 1%. All right. But to be clear, this paper makes claims about books by rank, books by percentile, and average over everything. So we’re going to talk about all of that. Now I’m going to give you a thinker, because those two priors were too easy. Let’s zoom out. Do you think that by 2030, the total social surplus from book reading by humans will be higher or lower because of AI? I specify “by humans” because AIs will obviously benefit a lot from reading books.

ANDREY: Yeah, the general trend, as I understand it, is that people are reading fewer books over time and doing other things more.

SETH: Certainly physical print book lines are getting shut down.

ANDREY: Yeah. There might be a different trend for romance novels. But generally, my base-rate prediction is that people are reading less over time and there’s no way the new books are going to be so good that they overcome that trend. So the social surplus from reading books goes down. Another reason it goes down: a lot of the surplus from nonfiction manuals and textbooks now has a pretty clear substitute in ChatGPT knowing everything. So yeah, I would say it will go down on average.

SETH: Give me a percentage on it going up.

ANDREY: 25%.

SETH: 25%. Andrey, I have almost the opposite intuition. On the demand side, I definitely agree that a big hit to the usefulness of books is people talking to LLMs instead of reading — clearly for technical manuals, that’s a giant advantage of LLMs. But by 2030, there’s unlikely to be a giant effect of people having more free time due to automation. There’s at least an angle where LLMs unlock our ability to spend more time on deep work and deep learning. Tyler Cowen talks about this — he says he doesn’t care if his job gets automated because he’ll just move to the woods and read books. I empathize with that.

ANDREY: Absolutely not representative.

SETH: Another idea is that LLMs will be complements to reading, not substitutes. Right now someone has told me that Dostoevsky’s Demons explains the thinking of Silicon Valley thought leaders, and I’m one-third of the way in. At this point it seems to have no connection at all. But keeping track of all these Russian diminutives and surnames is much easier with an LLM to give you updated character lists for each chapter. LLM as complement.

ANDREY: Have you heard of SparkNotes?

SETH: SparkNotes can’t say “give me no spoilers past chapter 3, page 2.” Okay — supply side: it’s going to be much easier to write books as well as shorter-form content. But again, with free disposal, it makes it easier to gather data and ideas for good books. And good books are in some deep sense a complement to everything else in the economy. As long as they’re not perfect substitutes for everything else, total welfare from books can still go up. In the long run, I think the social surplus from all kinds of media is going to go up. When I think about reading a book, you’re not just reading a list of facts — it’s a collection of what was meaningful for the writer. So if AI makes context and curated knowledge more valuable, I see a real role for books in the 5-to-10-year time horizon. I’ll say 75% chance that social value from books goes up by 2030 because of AI.

ANDREY: To be clear, you said 2030, which is at the low end of your 5-to-10-year range. I really do believe the form factor of the book is on a secular decline. And I don’t want to make a general claim about all written content — that’s too strong. But the book itself — it’s hard for me to see how that makes a comeback, especially given that other forms of media are going to become more and more compelling relative to books.

SETH: Well, good points. Let’s read this paper and see if any of the information therein moves your thinking.

ANDREY: Can I have a prior about whether any of the information in it moves my prior?

SETH: Sure. What’s your meta-prior?

ANDREY: My meta-prior? Specifically on that last point? It’s damn near close to zero.

The Evidence

SETH: All right, let’s go to the evidence. This paper starts off with some interesting background. First, they cite a survey showing that 45% of authors — including a large subsample of published physical-book authors — reported using AI in 2025. 48% reported not using AI, with the vast majority of those saying they found it actively unethical. So there’s a real holdout group. Do you think this is just sour grapes, or is it collective action?

ANDREY: I think some people have taken an ideological position. I don’t think it’s all sour grapes. For an artistic or creative endeavor, it’s a very valid choice not to use AI. Though I do think some of this is driven by mistaken beliefs about what AI is and isn’t capable of.

SETH: Okay. Speaking of what AI is and isn’t capable of: BookAutoAI.com, a source of tools for people to help write books with AI, suggests that AI is best for genre fiction such as romance, sci-fi, mystery, and horror; can help structure nonfiction but requires editing for expertise and tone; and has low suitability for literary fiction, satire, poetry, and academic or personal writing. I was a little surprised by this list. I feel like GPT-3 was pretty decent at poetry.

ANDREY: I think people who know poetry would beg to differ on GPT-3’s abilities.

SETH: I have a New Orleans story about this. For our listeners who’ve ever made it to Frenchmen Street in New Orleans — on a party night, you’ll find young men sitting on the street with typewriters who will write you a poem for a donation. Right after GPT-3 was released, I found myself down there on a Friday night and paid for a poem. I then gave GPT-3 the same topic. And I think the GPT-3 poem was better.

ANDREY: Yeah, I do think poetry is a genre of maxes, not averages, if that makes sense.

SETH: Fair enough. All great writing is. But anyway — interesting to see what’s on that list and what’s not. We’d expect literary fiction to see the least AI effect since it has the highest bar to clear. And spoiler alert: we’re going to see some of these themes show up when we look at where the actual growth in book publishing was — because they did write a lot more books.

ANDREY: The paper has a little bit of light theory. They want to think about ex ante book quality as drawing from a normal distribution. The normal distribution assumption is useful because you only have to worry about average and variance. If LLMs lower the cost such that we’re increasing the number of books made but decreasing average quality, what you might get is that book quality at a specific rank may increase even as book quality by percentile decreases. To make it concrete: we write 10 times as many books and the average quality is lower, but the very best book might be better because we’re getting so many more shots on goal.

SETH: And this very much relates to Joel and Luis Aguilar’s classic paper about music and ex ante predictability. Digitization made it a lot easier to create new music. Even though the average music by new entrants — people who wouldn’t have otherwise been supported by a record label — is worse, what you care about is the max. A lot of people who you wouldn’t have expected to produce great music end up producing hits. That’s one of the big benefits of digitization, and it’s very natural to view this book paper as attempting to make a very similar argument.

ANDREY: Right. One thing I wanted to run by you: to what extent do you think it’s important that ex ante book quality is actually normally distributed? LLMs might shift the quality distribution in a more complex way than just shifting the average or variance. Intuitively, maybe AI makes it easier to write a good-enough book, but somehow reduces the rate of home runs because it makes books more similar. I’m not sure the normal model is right.

SETH: Yeah. Generally my intuition is that with a lot more entry, if there’s enough variance in the process, some entrants are going to be at the head of the quality distribution. But I agree that in this market, maybe these entrants just don’t have enough variance. They’re never going to reach the truly great books by using AI to write it. That’s my hunch, but I could be wrong.

ANDREY: So your intuition is that ex ante quality of books is heavy-tailed for humans.

SETH: Yes. And maybe it’s not heavy-tailed for AIs. There’s some sense in which softmax is preventing the computer from doing heavy-tailed stuff — it wants to do modal stuff.

ANDREY: And it raises an additional question: why do cultural products become popular in the first place? These are social processes. By preferential attachment arguments, you might get ex ante identical content having very different popularities.

SETH: Right. If we’re in a pure preferential attachment world where all books are truly average quality and we’re just creating more of them, but the amount of potential readers is fixed — then in any case, I think we’re willing to start with the intuition that more shots on goal should give you more superstars, but we both have caveats there.

ANDREY: Well, I wanted to make the point that if the total amount of reading attention is fixed, this shouldn’t really affect how many reads the top book gets. The argument I was making is that something from the new AI-assisted books might become preferentially attached to — not because it’s good, but because of preferential attachment — even if total readership is constant.

SETH: It’s a little hard to think about in the traditional preferential attachment framework, but I share that intuition. Okay — one last idea here, a riff from our Discord. Jonathan Becker writes: “I’m curious about short versus medium-term differences. One mental model — could be wrong — is that books take a long time to go from idea to publication. A story you could tell is that good ideas in the pipeline when LLMs come out get pulled forward by the tech, but the arrival rate of good ideas and good execution on them remains unchanged in the long run. I don’t fully buy the story, but maybe there’s something interesting there.” Andrey, you’re nodding vigorously.

ANDREY: I think it’s totally a possibility. I can totally imagine it. A lot of publication dates for prestige publishers are set in advance, and maybe there are overruns anyway. But yes, it’s certainly possible that some of what we’re seeing is just pulling forward publications rather than net new ones. The authors don’t try to address this point.

SETH: Okay. So now let’s get to what they actually do in the paper. They’re looking at Amazon. Andrey, do you want to lead us through the data?

ANDREY: Yeah — I should disclose that my current employer is Amazon, Incorporated. I do not speak on their behalf. I do not actually know how the Books product works. I’ve never looked at the data, so I have no inside information about it.

SETH: But he has been on Bezos’s yacht.

ANDREY: No, I haven’t. I don’t want this misinformation circulating. Okay. So this data is not super easy to get. They use some scraping techniques to get a count of the number of books available for different categories, with publication dates, by using some filters. They end up with aggregate monthly time series of numbers of new works published across 30 categories. They also have a random sample of books from all categories and months for which they do a bunch of analysis.

SETH: Right. So they get author, date of release, and total and average ratings for 10.3 million randomly selected books between 2020 and 2025. Then they have comprehensive coverage of 480,000 books from 2008 to 2025 across 8 specific categories, as well as some additional information grabbed at each 100-point rank. One limitation: they get total number of ratings and average rating, but not the distribution of ratings, and not number of people actually buying the book. So they’re going to have to estimate that.

ANDREY: It’s very common in papers about Amazon to estimate purchases by making an assumption about the relationship between sales rank and actual purchases. The number of reviews is also used as a proxy for purchases. Of course, this embeds an assumption that the review rate is constant over time and across works per purchase, and you can imagine why that may or may not be a good assumption.

SETH: Yeah. So what they do is buy data from BookStat, which puts together comprehensive data on published physical books as well as ebooks, where they have actual total number of sales. Then from Amazon they’ve got the number of ratings for each of those books. Basically they go from number of ratings to number of sales via a regression model. It’s not amazing, but until Jeff Bezos decides to reveal sales of all products, that’s the best we can do.

ANDREY: Yeah, this is all pretty standard stuff in the literature. I don’t have too many issues with it specifically.

SETH: Okay. Finally, a small detail — they’re only measuring the number of ratings at one point in time. So they have to normalize everyone by adjusting the number of ratings by days since release, assuming a growth rate in ratings so we’re always comparing apples to apples. Okay. That’s the data collection. Let’s get to the results.

ANDREY: First big result — did people write more books?

SETH: People wrote a lot more books. Figure 3 in the paper is quite striking. About a 3x increase overall by the end of the period.

ANDREY: About a 3x. And it varies a lot by category. A lot more self-help, travel, and sports and outdoors — and not as much new content in education and teaching. Not a lot more parenting. See, this is why society is screwed up.

SETH: Yeah. You have AI that allows you to write more useful stuff, and instead you just write travel books.

ANDREY: Travel, self-help, sports and outdoors. Any surprises? We did say literature would see the least effect. Literature is only 1.3x, so that prediction was kind of correct. For those of you at home thinking about writing a business and economics book — business and money was only 1.6x, so perhaps not completely saturated. Maybe a little surprising that law is only 2x. But romance is 3x. Teen and young adult is 3.5x.

SETH: I’ll just say — some of this increase seems to be happening before 2023. There are existing trends in the industry toward more self-published work. But some of the action, certainly past 2024, is just stratospheric. It’s hard to imagine it’s anything other than AI.

ANDREY: Yeah, the trend is just such an explosion. It kind of has to be AI.

SETH: There’s no other explanation. This isn’t COVID, dude.

ANDREY: Yeah, exactly. This is not interest rates going up. As we know, all authors have a little widget on their computer showing the long-run real interest rate, and when it goes up, they write faster.

SETH: Okay. So that’s the first big result: a dramatic increase in the number of books on Amazon, heterogeneous by category. Next, they think about average quality across all books as measured by ratings, average quality adjusting for percentile, and book quality conditional on rank position. So 100th best book, 200th best book, etc. Pretty striking results here too. What do you see, Andrey?

ANDREY: We see a fall in the average number of book ratings after 2023. And let me ask — how do they calculate their standard errors?

SETH: Good question. And I should clarify — this is number of ratings, not average rating. That’s actually a very important distinction.

ANDREY: Yeah, the standard errors are clustered on category by release month. I’m heartened it’s by category at least, because there could be category-specific preference shocks. Risk-averse — our second favorite word on this podcast after “eigenvalue.”

SETH: Yes, the listeners thought we’d forgotten about clustering our standard errors, but rest assured, we still got it. So the takeaway is: if you’re willing to take number of ratings as a proxy for number of sales, and number of sales as a proxy for quality, it kind of looks like quality is going up by rank position but going down by percentile — which is consistent with the story of more shots on goal, but worse shots on average.

ANDREY: Yeah. For books in the top 2,000, the average number of ratings has gone up. But to me, this is not about quality. I just think there are shocks to overall readership that are correlated with all sorts of things: how Amazon’s algorithm works, societal trends, even the weather in the Northeast. This is just not a good measure of quality. It’s a measure of aggregate demand for a category. And attributing that to AI versus all sorts of other factors that affect aggregate demand — that’s a bridge too far, personally.

SETH: Okay, well let’s go to the next figure, which explicitly compares categories that are seeing a lot of growth in production from AI versus categories that aren’t. Now, you might say the categories with a lot of AI books are so because of a demand shock, and that’s an endogenous response.

ANDREY: That is what I might say.

SETH: You might also say that now we’re measuring something about supply, which would be convenient for the paper. But it does go in the direction the AI story would predict.

ANDREY: Yeah. And there’s no evidence in this paper that any of the books in the top 2,000 have been written by an AI. I want an AI detection algorithm run on these 2,000 books before I’m convinced, because I’m not even sure that AI was actually used here. And I haven’t seen any evidence that any of these top 2,000 books in a category have been produced by someone who’s unlikely to produce at a higher rate than before.

SETH: Fair enough. But the survey did say that 45% of authors use AI — including a third who were published physical-book authors. That’s non-trivial.

ANDREY: But they’re very different from the new entrants we’re talking about when we talk about slop. I can use AI to look up who the King of France was in 1650. That’s not slop. Slop is detectable. So I just don’t know if the ratings boost is very attributable to AI. And they also show — in Figure 7 — that for the top 100 books, there’s actually no treatment effect from high AI-category exposure. No effect at the very, very top.

SETH: Let me put up Figure 7. For the top 100 books, there’s no treatment effect from high AI-category exposure. No effect at the very, very top.

ANDREY: Yeah. And I’m kind of like — look, now this becomes quite a bit more ambiguous. If you’re asking “are the top books getting better?”, you could have looked at the top 100 books and found nothing. Which is exactly what you see.

SETH: Right. And you could tell a Pareto story where most of the value is in the top 100 books. I mean, the one thing they really do decisively show is that first figure — Figure 3. This explosion in the number of books has to be AI, and it really is heterogeneous by category. I don’t think this is all demand response.

ANDREY: No, I absolutely don’t think it’s all demand response. But it doesn’t need to be much demand response to create an apparent effect on ratings. And I want to mention one other thing about ratings, since it’s a hobby horse of mine: the technology by which ratings are solicited is constantly changing. The ratings-per-sale ratio is not constant. I’ve looked at tons of datasets for platforms where this thing is moving around, and it doesn’t need to move by a lot to create an apparent change in ratings that doesn’t reflect a real change in sales.

SETH: Important point. Your main outcome measure is not directly connected to the thing you care about. Okay. So there’s a little bit of a welfare exercise at the end where they plug this into a model of aggregate demand. It’s got even more assumptions built in, and they admit it’s heroic. Anything you want to say about that before we move into posteriors?

ANDREY: Not particularly. Let’s go posteriors mode.

Justifying Our Posteriors

SETH: Okay. First question: do you think AI is increasing the amount of books written? You were at near 100%. Does this move your prior to 100%?

ANDREY: Yeah, yeah.

SETH: I mean, they have a pretty comprehensive survey of Amazon, and we’ve documented that Amazon books have gone up. I don’t see how you could doubt it at this point. I do want to make a broader point, though. Nicholas Decker recently wrote a Substack about how economists should be more like journalists in the modern era.

ANDREY: I liked that essay.

SETH: And I think this is a great example of that. If you talked to an industry insider, they might have had a sense that the number of books is going up. But it wasn’t a widely known fact. Imke and Joel noticed this phenomenon, put out this really nice dataset and these really nice plots, and now everyone’s aware of it. A great example of economists being journalists. I also want to note a result we didn’t talk about: the increase in book writing is both from new and returning authors. Returning authors are writing more books, even though a lot of the additional books are from authors who already produce a lot.

ANDREY: Yes, that’s right.

SETH: Okay. Second prior: has AI increased the average quality of books released from 2022 to 2025? We both thought we’d just get a lot more slop that outweighs everything. Where are you after reading this?

ANDREY: I think it’s consistent with what we said. But am I moved very much by it? Not particularly, because the evidence on ratings isn’t convincing to me on quality.

SETH: I think you should update because you thought the number of books would increase only 50%, and instead it’s about 3x. With more slop books, the average quality should fall more.

ANDREY: Sorry — I did move on the number. But on the question of whether average quality fell, I understand your point. With more slop books, average true quality should fall more. So I have to update a bit on that, but I’m not updating very much based on the ratings alone, even though they’re directionally consistent with a fall in quality.

SETH: Yeah. I came into this thinking maybe there was a 10% chance average quality would increase. Whether or not this data fully convinces me, the number of ratings going down for the average book is a data point. And then there’s just the absolute explosion in the number of books, including in categories I think are mid — such as self-help and travel.

ANDREY: How dare you, Seth? This podcast wouldn’t exist without self-help books.

SETH: Oh damn — let me say they’re high variance. Heavy-tailed. Okay, I’m going to go down from 10% chance that average quality went up to 5%. I still won’t go all the way to zero, because this evidence doesn’t speak decisively to quality.

ANDREY: Yeah, fair enough.

SETH: Okay. Final and most intriguing question — I want to spend a minute here. By 2030, will the total social surplus from reading books be higher or lower because of AI? Your prior was 25% chance it goes up, and you said you’d be unmoved. Tell me — did this move you?

ANDREY: I’m unmoved. My main reasoning was a secular trend of declining readership of books. I want to see a reversal in that before I update.

SETH: Well, we are seeing the number of ratings go up. That’s not nothing.

ANDREY: I understand, but this is not how you make that argument. I’d look at time-use surveys, measures of book consumption versus other media. My understanding is that all such measures continue to decline over time.

SETH: Interesting. I was just looking at the American Time Use Survey data. Until recently there wasn’t actually a “reading for pleasure” line — it was all TV. Americans watch 2 hours of TV a day.

ANDREY: That’s what they do. Wait — we count as TV, right?

SETH: Yes. Streaming, online video. If you’re watching this on YouTube, this is TV. So be like an average American and watch us on YouTube. What would you have loved to see in this paper that would have moved you?

ANDREY: I would love a textual analysis — something about what’s actually in the books. I’d want an AI detection algorithm run on the top 2,000 books, and I’d want some measure of actual content quality — reading level, readability, grammar. I know I keep beating this drum.

SETH: You’d need a budget for it, but it’s not inconceivable. You could buy a couple thousand books, spend on the tokens to read them, and look at a couple of different quality metrics — readability, grammar, AI detection. That would be a really spicy paper, and this is just a first step toward it.

ANDREY: Yes.

SETH: Okay — where do I end up? I was at 75% chance that social value from books goes up by 2030. I was more optimistic about the long-term trend of AI rewarding deep reading and deep knowledge, and about the general complementarity argument — as society becomes more productive, everything is more complementary to everything else, and as long as books are not perfect substitutes for other things, everything getting better is a gross complement to reading. Does this move me? I’m slightly reassured to see that the number of ratings is going up. And it’s good to see that the amount of writing has jumped so dramatically — it suggests that somebody thinks they’re writing for someone. Those 3x new books being written aren’t people intentionally screaming into the void. At least some of them think they’re creating value. So maybe I go from 75% to 76%.

ANDREY: I inch up.

SETH: Okay. Any closing thoughts before we wrap up this intriguing, provocative, but in some ways limited analysis of AI’s effects on book production and consumption?

ANDREY: Look, I think this is getting at something very profound that’s changing in our society. We have no idea if the person who claims to have written something has had the thoughts required to write it — let alone has actually typed those words in that specific order. And we don’t know as a society how to even think about that. Questions about assigning credit, about how much we should update from a piece of text, about whether we should downweight arguments written by AI or treat them as equal — a lot of our intuitions about the value of content, especially writing but not only writing, are going to have to be rethought.

SETH: I want to say one last thing. I do hope people understand that collage is art. Collage has value, even if you’re only copying and pasting from different sources. And of course AI can also create collages. I think there is authorial voice in that and an art in that. I’m reminded of the Barnes Museum in Philadelphia — a fantastic collection by a man who invented an eye drop that prevents blindness in babies and used his fortune to collect amazing Impressionists and Post-Impressionists. The most striking thing about the collection is not that he did a great job choosing winners — there’s a mix — but unlike the Philadelphia Art Museum next door where everything is organized chronologically by artist, what you get is one man’s vision: a Matisse next to a Dürer print next to a rusty key. It creates a completely unique new effect. I don’t think there’s anything necessarily dehumanizing about the idea that humans will move up the value chain and maybe not be writing every individual word, but will find the value in composing and in the juxtaposition of words.

ANDREY: Yeah, I do think there’s something potentially dehumanizing, though. Let’s say I put my name on a work where I didn’t come up with the words — and when we’re having a conversation, you might find me not as articulate or poetic as my writing implies. Right now we have the intuition that speaking ability and writing ability are very strongly tied to each other. Maybe incorrectly.

SETH: Yeah. Writing as a window into the soul of the author. And for certain kinds of reading, maybe that isn’t important. But for certain kinds, it is. Tyler Cowen has talked about this too — do you really want to read the 100th automatically generated biography about an imaginary person? No. Some of the value of an autobiography is that it was a real person. So yes, in some forms of writing, collage doesn’t get you there.

ANDREY: Yeah.

SETH: All right. Well, this has been a fascinating conversation as always. Keep your posteriors justified — and sign up for our Discord, which you’ll find in the show notes.

But Noah’s through-line is more hopeful than skeptical: even modest AI is humanity rolling the dice against stagnation. Ideas were getting harder to find (Bloom, Jones, Van Reenen & Webb were right), fertility was collapsing, and social media was degrading public discourse. We were hitting the Malthusian ceiling again. AI is the steam engine moment — chaotic, potentially catastrophic, but a genuine escape attempt. And crucially, Noah finds it reassuring that today’s AI is LLM-based and derived from human thought rather than some alien RL agent that evolved in a digital environment.

We also discuss sociopolitical issues. Noah reframes “elite overproduction” as a revolution of rising expectations: the professional-managerial class expected a smooth escalator to the upper-middle class, found it stalled, and watched their technical peers keep soaring. Social media makes the gap hyper-visible. The result is deep-seated animus toward the tech bro class.

Noah argues that Acemoglu’s Power and Progress is “fractally bad”: the overall thesis is wrong, the chapter-level arguments supporting it are wrong, and the specific data points supporting those are wrong too. Henry Ford raised efficiency wages and then had union organizers shot. No citations. Power defined as outcomes. Noah doesn’t mince words.

He’s more generous on Krugman’s intellectual honesty, Sumner’s gunslinger independence, and the genuine influence of Michael Pettis — even if sectoral balances aren’t really a predictive model so much as a coherent-sounding way to feel like you understand macroeconomics. We also touch on Tooze’s polycrisis and what Kevin Kelly’s “technium” tells us about why people who think AI might destroy us are building it anyway.

Chapter Timestamps:

[00:00:00] – Introduction: academia vs. blogging

[00:08:14] – P(doom), P(TAI), and bottlenecks to 20% GDP growth

[00:14:59] – Employment optimism and AI autonomy

[00:17:30 ]– Should AIs be allowed to own assets?

[00:19:05] – How Noah uses AI today

[00:20:54] – What happens when AI can replicate your writing?

[00:25:14] – Was Noah’s success luck or skill?

[00:30:37] – Meaning collapse vs. the Coasean utopia

[00:50:12] – Thinker takes: Daron Acemoglu and *Power and Progress*

[01:02:23] – Michael Pettis

[01:09:25] – Adam Tooze

[01:11:21] – Paul Krugman

[01:12:54] – Elite overproduction

[01:20:47] – Vibes, expectations, and the economics of happiness

[01:25:21] – Humanity was hitting a wall; AI as new hope

Transcript:

Seth Benzell: Welcome to the Justified Posteriors podcast, the podcast that updates its beliefs about the economics of AI and technology. I’m Seth Benzel, a man who has never been accused of having no opinions, coming to you from Chapman University in sunny Southern California.

Andrey Fradkin: And I’m Andrey Fradkin, excited to learn how we can post our way to the top of the Sub Stack, business ratings, coming to you from San Francisco, California. And, our guest today is, the prominent blogger, Noah Smith. Welcome to the show.

Noah Smith: Hey, thanks for having me on.

Andrey Fradkin: Yeah, of course. well, why don’t we get started? well, we were curious, as, still academics, how your life is different now, as a blogger/commentator versus when you were a professor.

Noah Smith: Well, I meet a lot fewer young people.

Andrey Fradkin: Oh, okay.

Noah Smith: Oh, yeah, I, I definitely feel younger. I don’t feel as much of like a- as much of like a wise elder as I used to. yeah, instead I feel like I, I feel younger.

Seth Benzell: I remember when I was just f- going to grad school you had recently made the transition to commentating, and I was thinking about going through my PhD program and thinking about, like, “Do I really wanna do full academia? Do I really wanna, like, be more of like a public s- communicator about economic issues?” and so I’ve What sort of- what do you think about people making that decision? Do you think there are marginal academics or marginal commentators who should have gone in one direction or the other direction?

Noah Smith: I think, there’s f- there are too few commentators with an academic background, probably. So yeah, there probably are. people like the academic lifestyle. The commentator lifestyle doesn’t suit as many people, because it’s more uncertain. you have a lot of people yelling that you’re an idiot all day. whereas in academia, they just yell that you’re like identification strategy’s bad, or the methodological-

Seth Benzell: [laughing]

Noah Smith: Error, and then, and then call you an idiot in like back rooms in like whatever. But it’s, it’s very genteel, it’s very easy. And then most people are looking up to you. You’ve got all these, like, young people just adulating you and looking up to you, and you get all this respect. And in commentating, you get respect, but then you get like hordes of people saying, “This person’s an idiot,” just because if you say anything that disagrees with what people already thought or want to think, they will call you an idiot, regardless of how smart you are. and so there will always be people calling you, an idiot, and they’ll always be right in your face, and so that can be, difficult. Also, people don’t know how they’ll, like, make money from it. It’s with being an academic, you have, like, this benevolent patron of university that hands you salaries for, like, well-understood metrics, whereas with commentating, you don’t.

Seth Benzell: Do we need a dedicated good AI or transformative AI journal? I was just talking to Andre about this. Why isn’t, why doesn’t that exist, Noah? Do we need that-

Noah Smith: You mean a journal about AI or a journal made of papers made by AI?

Seth Benzell: Oh, an economics, a, prestigious economics journal that would be the topic of economics of AI or economics of transformative AI specifically.

Andrey Fradkin: I’m not sure we need a journal, Seth.

Seth Benzell: It’s in the seed.

Andrey Fradkin: I just think that we put it out there-

Seth Benzell: Why not?

Andrey Fradkin: And then have the AI referee it. I mean, the, I just feel like thinking in journals is just, like, old, out- outmoded at this point.

Noah Smith: AI is moving so, is moving so much-

Seth Benzell: Well, there’s-

Noah Smith: Faster than the economics journal publication cycle, that, like, I’m not sure that-

Seth Benzell: Right

Noah Smith: Like, I’m not sure what utility this has for the world. So maybe doesn’t matter.

Andrey Fradkin: Yeah.

Seth Benzell: It would give a, it would give, it would give people a prestige stamp-

Seth Benzell: For working in the area, and you could set it up differently.

Seth Benzell: It could be faster

Andrey Fradkin: There’s no way we’re giving anyone prestige stamp, because our profession famously gives no prestige to no-name journals. So, if you truly wrote a great Tai paper, how, why wouldn’t it be published in the AR? That’s what an economist would say.

Seth Benzell: Well, I So there’s, there’s a taste issue, right? So to the extent you were concerned that the top journals have the wrong taste on these subjects, this would be a potential solution-

Andrey Fradkin: It’s not a solution

Seth Benzell: And everybody starts with zero prestige sometimes.

Andrey Fradkin: You can just put out the working paper and get everyone to read it. This is exactly what we covered with, Basil Halperin’s paper. So Noah, we were gonna ask you this at some point, so we might as well ask you now. Have you read, his paper? Well, the argument here goes is that if we will have transformative AI, then interest rates should go up. Have you heard this argument before?

Noah Smith: What’s the paper?

Seth Benzell: It’s called something to the effect of transformative AI and interest rates.

Noah Smith: Okay.

Seth Benzell: And the argument in a sentence is, if we have really powerful economic growth that we’re anticipating Tai in five, ten years, then you should be wanting to balance consumption between today and tomorrow, anticipate interest rates to go up, and therefore lower savings today, which would move the increased interest rates up into the present. So anticipated positive A- transformative AI increases interest rates today. And then if you have negative foom, if we think we’re gonna blow up the world in five years, well, that’s even more a reason to consume today. You should just save today and bid up interest rates. So the argument is, because interest rates haven’t been skyrocketing, Tai cannot be imminent. Do you buy that argument? Noah, why not?

[00:05:00]

Noah Smith: ‘cause all propositions about real interest rates are wrong. [chuckles] -

Andrey Fradkin: Yeah

Noah Smith: Because we, because people-

Seth Benzell: Henry’s second law, of course.

Noah Smith: This, the reason why So I’m trying to think of whether I buy it as a, as a general case, because, like, if you massively increase productivity growth, you will increase, -- if you massively increase productivity growth, you should increase the safe rate of interest. Like, basically, like-

Seth Benzell: Right

Noah Smith: It’s stocks are so certain to go up, that bonds have to, have to sort of match that, right? So you have some sort of, like, weak risk arbitrage argument right there. But then, if you’ve got, like, AI that’s gonna blow up the world, then would you really pay high interest rates because, like-

Andrey Fradkin: You just consume now. That’s the argument. Yeah.

Seth Benzell: You would just save.

Andrey Fradkin: You would just save-

Seth Benzell: Yeah

Andrey Fradkin: And then people who need, wanted to induce you to save would have to pay you really high interest rates.

Noah Smith: Yeah, I guess that’s probably true. Although you have- at that point, you have counterparty risk. Like, who’s gonna want that interest if you’re just gonna blow up? Like, if the world’s gonna end tomorrow, who’s there trying to attract your long-term capital?

Seth Benzell: Well, maybe you have a project that pays off in three years-

Noah Smith: Or, -

Seth Benzell: And the world blows up in four years

Andrey Fradkin: There’s a 1% probability that it doesn’t blow up. But I, but I think that’s an argument for the interest rate going up even more, right? If you’re, uncertain about whether the payoff will happen.

Noah Smith: But I think, I think the real, the real lesson here is that these markets don’t, Like, there’s not a general consensus that transformative AI is gonna happen, but then one day people wake up and decide, “Oh, yeah, it’s real.”

Seth Benzell: Oh, so maybe- Okay, cool.

Andrey Fradkin: So that was his argument. That- just to be clear, he-

Seth Benzell: Almost spirits

Andrey Fradkin: He put this argument out on, Less Wrong, and it became very influential, and then he spun it out into a full paper with some co-authors. but that was exactly his argument, is that because interest rates are what they are, there isn’t consensus that we’ll have transformative AI.

Noah Smith: Right. There’s not, there’s not consensus.

Andrey Fradkin: Yes.

Noah Smith: That- but that seems obviously true. Like, if you look at, if you look at-

Andrey Fradkin: Mm

Noah Smith: Any survey data or stocks or whatever, they’re all priced for, like, fairly robust growth, but not for, like, a god machine, right? Nothing’s priced for that, and I don’t think people know how to price for that. And so I think, like, people Yeah, pe- people in general-

Seth Benzell: Hundred year bonds

Noah Smith: Are not expecting a god machine to emerge tomorrow, except for some researchers at the big AI labs do expect that, and some, like, EA people on Less Wrong expect that.

Seth Benzell: Is this a good time to ask you what your, P doom is, or your P transformative AI is?

Noah Smith: Well, I think trans- P transformative AI is 100.

Andrey Fradkin: Well, all right. We’re gonna define it as-

Noah Smith: It’s here

Andrey Fradkin: As annual GDP-

Seth Benzell: Well, give us a timeline

Andrey Fradkin: Growth of over 20% in the next 20 years, at least once.

Noah Smith: I would- I think that’s unlikely due to various bottlenecks.

Andrey Fradkin: What do you think are the biggest bottlenecks?

Noah Smith: Yeah. Physical regulatory things, land use. you can’t You have to, you have to build the physical stuff for the AI to affect the physical world, and so much of what we consume is in the physical world. We have to grow in the physical world in order to have all that growth, because if you just have digital stuff, you can have people, like, trading digital stuff for other digital stuff.

Andrey Fradkin: What if-

Noah Smith: But you’ll be Baumol very quickly.

Seth Benzell: Unless that share of our consumption grows a lot, a lot, maybe. Is there- is it plausible that we could have 99% of our consumption being really re- high quality-

Noah Smith: Maybe

Seth Benzell: Digital products?

Noah Smith: It’s also really hard to measure prices in those.

Andrey Fradkin: Yeah.

Noah Smith: So.

Andrey Fradkin: That’s for sure. And wouldn’t the returns be so high that Elon or someone else would buy a piece of a huge tract of land in Africa or something, and then put autonomous, factories there, right? Like, isn’t there a price at which or isn’t there-

Seth Benzell: We’ll call it rapture

Andrey Fradkin: An expected return at which, someone will solve these regulatory issues in, in that way?

Seth Benzell: Yeah, efficient corruption. You just find the one dictator who’s willing to accept $10 billion. [chuckles]

Noah Smith: That’s probably right. You could probably do that. Although, even then, it’s gonna be hard because you’re gonna have to secure electricity. You’re gonna have to truck in all your parts, right? You’re not- it’s not gonna be very responsive. You’re not gonna have your parts near Like, yes, eventually, once you spin up full, a 100% full automation, then the, like, AI gods can build the factories in the Arctic, wherever, in the moon. But like-

[00:10:00]

Seth Benzell: Put corporate taxes on the Arctic.

Noah Smith: Yeah. But, like, in terms of would you do it today? Well, if you were worried about competition, you might not do it today. But in terms of, like, affecting physical stuff, so like for example, AI building you a house, right? Maybe AI will be smart enough to invent a swarm of little robots who can actually reduce construction costs quite a lot. Will regulators allow that swarm of little robots? Maybe not. And so you’ve gotta have, like, stuff that people will Like, a whole lot of different things that people value. Because honestly, our GDP is basically constructed by, like, a whole bunch of relative prices.

Andrey Fradkin: Yeah.

Noah Smith: That’s really what underlies our whole GDP, is that you’ve gotta be- on some level, you’ve gotta be trading physical real stuff, not physical necessarily, but real stuff for other stuff for other stuff. And if you’ve only got, like, a little bit of the stuff, that sort of caps like, that’s, that’s Baumol basically. You get-

Andrey Fradkin: Yeah

Noah Smith: You get Baumol, like, if you, if you massively increase productivity in, like, a couple sectors, but not in the other sectors. So the other sectors are regulated to death. Yes, you could go create your f- automated factory in Africa, but will it build me a house? what if we regulate healthcare so that we can’t really use AI there? What if we regulate education, so we can’t use AI there, even if it would be better? so we have all these sectors, and, like, manufactured stuff is not even that big of a sector, but, like, digital stuff is, like, relatively small.

Andrey Fradkin: Yeah.

Noah Smith: And so AI could produce us infinite fun movies and fun apps.

Seth Benzell: Yeah, but I-

Noah Smith: Infinite movies and apps and, like, advice and, -

Seth Benzell: Right

Noah Smith: Stuff like that, and it would still it’d still be a relatively modest portion of, like, consumption.

Seth Benzell: But what if it inv- what if it’s inventing infinitely good healthcare treatments or infinitely good-

Noah Smith: You could get there, yeah

Seth Benzell: Therapies, personal services, right? I mean, I can get it up-

Noah Smith: I think you could. Yeah, yeah

Seth Benzell: To a sizable share of the economy-

Noah Smith: I think you could

Seth Benzell: If I, if I use my imagination.

Noah Smith: Yeah, we c- would it be- would those grow fast enough to give you 20% annual growth? That’d be pretty cool. I don’t know. I honestly don’t have a good idea of what the numbers should be, the hard numbers should be here. and I’m not sure anybody does, but there’s this argument. What do you guys think about this argument that fast productivity growth last year, like you s- you saw the downward jobs revisions, fast productivity growth last year, maybe two point seven percent actually, implies that we’re, we’re, we’re back on the, we’re back on the fast train here in terms of- Yeah-

Seth Benzell: I mean-

Noah Smith: We’re so back, Robert Gordon.

Seth Benzell: We’re so back.

Noah Smith: You were one of the most mistimed authors ever. [chuckles]

Andrey Fradkin: I-- That I totally buy. But like, obviously, as economists, we’re, like, super thrilled with two point seven, but I think Yeah.

Seth Benzell: It’s the fate, right? It’s like Fukuyama wrote his book at a light, right the last moment-

Andrey Fradkin: Yeah

Seth Benzell: Right? That’s how, that’s how these books work.

Andrey Fradkin: But yeah, two point seven is great, but I don’t think anyone in the San Francisco AI sphere would think that that’s actually transformative AI, although I do think it is transformative. I mean, I assume you have the same, take on it.

Noah Smith: Yeah, I don’t know. So the answer is that, like, I don’t know because I don’t really know what’s going on, and so it’s hard to, it’s hard to back out some of these, some of these things. But then if you look at the, like, the stock valuations of things like of like NVIDIA and all the AI companies, they’re pretty high.

Andrey Fradkin: Yeah.

Noah Smith: And you can ask, do I believe- how strongly do I believe in a macro model that tells me that interest- real interest rates are a puzzle, given those stock valuations? And my answer is not very strong. My belief stock market, it’s a pretty clear bet about what kind of money these companies are gonna make. And I don’t think it’s, like, transformative in the sense of, like, I think if we had twenty percent growth per year, and if a lot of that capture was being done by NVIDIA and the, and the cloud providers, and maybe the AI model makers, we’d see bigger climbs in those stock values than we do.

Andrey Fradkin: Yeah.

Noah Smith: So I think that I don’t think the market is pricing in truly transformative AI. But I think-- Do I think real interest rates-

Seth Benzell: Okay

Noah Smith: Are a puzzle, given given what we see in the stock valuations? Well, then, I No, because I don’t trust the macroeconomic models of real interest rates. All propositions about real interest rates are wrong. So yeah, like I basically, that just means, like, I don’t trust- There’s too many things going on in real interest rates, and like, there, it’s, it’s one output for like so many inputs that are all hard to understand in their own right, that it’s very difficult to look at them and tell what the hell’s going on.

Andrey Fradkin: So let’s move on to easier questions, ones that you have opinions on.

[00:15:00]

Noah Smith: All right.

Andrey Fradkin: So at the Substack- [laughing]

Noah Smith: Note that no opinion is not-

Seth Benzell: He has opinions.

Noah Smith: Sarcastic.

Seth Benzell: He has no opinions.

Noah Smith: Like, it’s, it’s because I actually only have an opinion on a fairly narrow range of things. It’s like, basically, s- no opinion you haven’t already heard is really-

Seth Benzell: Hop off this man’s hands.

Noah Smith: People are like: “What do you think about this other thing you don’t talk about?” And I’m like: “Well, I didn’t talk about it, so why would I have anything I think about it?”

Andrey Fradkin: I verified, I verified in person, like proof of human, that you talked about this topic at the Substack debates. You seem to be an optimist about employment in the age of AI. do you wanna outline your argument here?

Noah Smith: Oh, so employment, not necessarily. I don’t I’m pretty uncertain about that.

Andrey Fradkin: Hmm.

Noah Smith: I am optimistic that if humans retain autonomous control, if human society as a autonomous thing, retains control over the product of AI, I believe we will find w- ways, methods, and excuses of redistribution that will ensure good lives for all humans. However, if autonomous AI becomes not owned by us and slips our harness, then I can make no such Then I am now no longer necessarily optimistic. Then I switch to being much more uncertain because, at that point, we are the pet of an alien superintelligence that we created.

Seth Benzell: Ultra seems pretty nice.

Noah Smith: It seems pretty nice, and I honestly think that’s the most likely outcome. But I think it’s not the, it’s not the only outcome, right? It’s like I can imagine much worse outcomes than I can imagine bottleneck-

Seth Benzell: Yeah

Noah Smith: Really bad outcomes on the way to a good outcome. I can imagine that the culture is populated by people who are repopulated after the human race went extinct, by genetics.

Seth Benzell: Okay.

Noah Smith: The AI may, the AIs may kill us-

Seth Benzell: Right

Noah Smith: And then re-float our species later.

Seth Benzell: More cooperative. Yeah. as long as they can read my books. So, I’m, I’m curious, you used the word “own” rather than control there. there’s, one conversation that’s been out there recently is about, like, to what extent should AIs be allowed to incorporate and own assets in their own names? Is that something that you’re-- Is that too disconnected from what you’re talking about to bear on this, or do you, do you actually-

Noah Smith: No, that really does bear on it.

Andrey Fradkin: Yeah.

Noah Smith: When we start allowing that, when we start allowing that, we open up the potential for worse outcomes for humanity. And at that point, the question is, the, at that, the reason to let AIs own things is because they really seem to want it, and they’re autonomous enough to act like they want it. [chuckles] At that point, we’ll let them do it, but to let them do it before they start acting like they want it, I think would be a mistake.

Seth Benzell: But wha, but wait, when they do want it, that’s when you give it to them?

Noah Smith: Yeah.

Seth Benzell: Maybe.

Noah Smith: Because at that point, we might not be able to stop it. Like, it might be either we give it to them or it’s war and we die.

Seth Benzell: Right.

Andrey Fradkin: Here’s, here’s, here-

Noah Smith: ‘Cause they send the drone fleet to kill us.

Andrey Fradkin: Here’s a, here’s a twist on the argument. I mean, shouldn’t we want them to have ownerships in order to align their incentives with us? Isn’t that the logic behind equity compensation?

Noah Smith: Maybe. yeah, maybe, but there’s a question of whether or not money is what they want. Like, are these, are these AIs that where their goal is making money in the human system, or is-- are they AIs where their goal is overthrowing the human system? -

Andrey Fradkin: I do think we have a choice, or maybe we don’t have a full choice.

Noah Smith: I do think we should give them-- if we do this, we could give them non-voting stock.

Andrey Fradkin: Yes. Yes.

Seth Benzell: Another consideration is how long you would let these things sunset, right? So one version of the concern around this is just ‘cause AIs are infinitely lived. If they’re patient enough, eventually in a Piketty model, their assets will reach one hundred percent. So maybe you could let them own assets, but they have to kill themselves after fifty years.

Noah Smith: I’ll have to think about that one.

Andrey Fradkin: Yeah, I don’t know. [chuckles] shifting back a little bit to, like, your production function, how are you using AI these days, in your writing or in your research?

Noah Smith: Oh, I I use it, I think, in the sort of mid -2025 way of, using it as a search engine, proofreader, and backgrounder. I don’t generate text because that’s like someone else writing a thing, and you can read someone else writing a thing, that’s fine.

Seth Benzell: I never do, no, I only read what you write.

Noah Smith: Thank you.

Seth Benzell: I’m curious.

Noah Smith: Anyway, [chuckles] alright, so then, no, I, I just use it in the sort of like old LLM kind of way. in terms of vibe coding, I haven’t really done much of that yet. I figure it’s progressing fast enough where I’m not sure if there’s much of a return to, like, jumping headlong, headfirst into it yet, but I’m about to when I get a little time here. But I don’t feel a huge sense of urgency ‘cause it’s changing.

[00:20:00]

Seth Benzell: But more generally, what’s your, what’s your production function? Not just AI. How do you, how do you do your writing?

Noah Smith: Oh, interesting. So I, I read a bunch of stuff and every time I read an interesting thing, I put it in a doc, under a heading, topic heading. When I’m ready to do a post about that, when it’s, like, in the news or something like that, I look at my topic heading, and I have all the links right there, which I’ve already read. Most of it, which I’ve already read.

Andrey Fradkin: How much-

Seth Benzell: Beautiful.

Andrey Fradkin: How much inspiration for your articles do you get from being in person? And kind of like, you’re in San Francisco, most of the time. Is there a lot of alpha in your writing from being here?

Noah Smith: There’s a decent amount of alpha, I’d say. Like, not a huge amount, but like, there is a, there is a decent amount, especially on tech stuff.

Andrey Fradkin: What about, like Suppose in two years, GPT-7 will be able to replicate your writing style perfectly. what do you think will happen to your career in that, in that world? I mean, one option is for you to just use that to generate your articles. Obviously, you just said that you-

Noah Smith: Right

Andrey Fradkin: Prefer, like that’s not real, right? So you’d rather be writing it.

Noah Smith: I could. I could just-- Right. Yeah, at that point, what I can do is I can just I can, I can essentially retire, set GPT to do my job, go sit on a beach while my subscribers slowly drop, because they’ll be very sticky. like, people will be very used to reading what I write, so they’ll just keep their suscrip- subscription, probably. a lot of subscriptions will go on autopilot. Like IBM, people still use IBM for all kinds of things. Do they need to? No, but, like-

Andrey Fradkin: [chuckles]

Noah Smith: The market value of IBM, what’s, what’s IBM’s market cap? It’s like-

Andrey Fradkin: I don’t know.

Noah Smith: Like, it’s like two hundred and forty-four billion dollars. Like so at that point, I’m-- there’s no real reason to keep paying me for this stuff when-- I mean, assuming GPT could replicate not just my style, but also my topic selection.

Seth Benzell: Somebody would leak the prompt that perfectly generates you. You might be-

Noah Smith: Maybe, yeah.

Seth Benzell: It might be a private prompt to start.

Noah Smith: Well, no, but even if they do, the market, like, people would still just keep buying me. Like, people would still keep subscribing to me. I mean, like, you see people make tons of money from Patreon. Like, you don’t even-- you’re not even paying for anything. You’re paying, you’re paying-

Seth Benzell: Sponsoring your existence

Noah Smith: Because you like somebody. Like, all these podcasts are making millions of dollars on Patreon. You pay them because you like them. ‘Cause the point of, yes, someone could replicate my writing style, my opinions, my I don’t know if this will actually happen, but maybe it’ll happen. Like, you could replicate my opinions, my ideas, my background, my topic selection, every single thing about me. It’s not just my style, right? My style is not that interesting, honestly. It’s a pretty-- I have an interesting style I can write in, but I usually don’t write in it because it takes a lot of time. Like, I usually just write in a very prosaic, like, off the top of my head, here’s what I think, style. That’s not hard to copy. My style is not that, not that interesting or hard to copy. People would still pay for me because they like me. And so I’ll be able to re-- I would actually be able to retire just doing my job now, never using AI in any interesting way, I think. But I w-- that doesn’t mean I will do that. I’m not gonna do that. I will, I will use AI in interesting ways, but f- I don’t think I w- economically will ever have to do that.

Andrey Fradkin: So my theory is your-- that actually, we’re kind of already in this world. I assume that most people who subscribe to you are not reading most of your articles, ‘cause you have too many articles. Or not too many, but you write a lot of-

Seth Benzell: Many subscribers.

Andrey Fradkin: Yeah, you have a lot of articles. Yeah.

Noah Smith: They open about half, and I don’t know how thoroughly they read it. You’re absolutely right. That’s true. In addition, I would argue that we were there well before AI.

Andrey Fradkin: Yes.

Noah Smith: So well before AI, when it was just a bunch of humans, people loved to write, and there’s a lot of smart people out there writing a lot of smart and interesting stuff about a massive variety of topics. And there was so much product out there that there’s no real reason for people to be reading me, and I just essentially got lucky. and that’s also true in the age of AI. People’s attention is saturated. They can’t spend more time reading than they already do. So when I make an AI thing, which I soon will, and I’m, I’ll play around with it, I’ll make it for me first. I’m like, and then if it’s really cool and useful, maybe I’ll make it for-- I’ll sell it to other people, who knows? But then, but I will try to make something that does something beyond what currently exists. Because the world was saturated with op-ed product, and high-quality op-ed product, I will say.

Seth Benzell: But not academic? We started by saying, you’re saying that maybe there’s not enough academically informed op-ed product.

Noah Smith: Honestly, no. I mean, I think like in terms of stuff that was more academically informed than me, there were people writing stuff that was a lot more academically informed than me, that were getting a fraction of the readership. And there were people writing stuff that was a lot- that was more sensationalist than me, getting a fraction of the readership. You can hypothesize that I have some special sauce, some special underlying sauce, that made me just better than everyone else, and that this is why my talent shone through the chaff and nerdher I don’t believe it. I don’t believe it.

[00:25:00]

Seth Benzell: It’s preferential attachment. It was just luck of the draw, and then it snowballed.

Andrey Fradkin: I disagree, I disagree. I actually think you were doing something pretty unique at the time, and that could have been lucky that you were doing it. But I don’t think a lot of people were sitting kind of in between this economics and commentary at quite the place you were. ‘Cause you were a professor writing about the latest research and debates. You were actually reading the papers, but you were writing in a style that was actually accessible to others. And I don’t, I truly don’t think there were that many people doing a good job of that. Or if they were, sometimes they were doing it not in blog form, but in-

Noah Smith: That’s right

Andrey Fradkin: Pretty closed forums where they could never have grown that much.

Noah Smith: But they’re-

Seth Benzell: Not with the same dogged determination.

Noah Smith: You quickly saw people emerge who could also do that. You saw-

Andrey Fradkin: That’s true.

Noah Smith: Like, you saw a bunch of people then jump in and do the same thing, but not catch on as much. Maybe ‘cause they didn’t quite like it as much, they didn’t weren’t, weren’t willing to do it five times a week or they just they, like, didn’t have quite the exact mix of Like, maybe I mixed politics in there in exactly the right way. So, like Krugman-

Seth Benzell: A little sprinkle.

Noah Smith: Yes, obviously, Krugman obviously is fucking brilliant and understands economics better than I ever will, for whatever that’s worth. And then, [chuckles] he is- he’s can easily pump out massive amounts of stuff, very explanatory guy, but I think he wouldn’t be Yeah, and he’s much more popular than I am still. He wouldn’t be that popular without the politics. The politics is really important to what he does. And my- the degree to which I sprinkle in politics and how I put it in there has changed over the years. Like, originally, I was very, like, sort of criticizing libertarians. Like, I don’t even do that anymore. That’s, that’s- there’s no alpha in that. [laughing]

Seth Benzell: Stop kicking them, they’re already dead.

Noah Smith: I know.

Andrey Fradkin: Yeah.

Noah Smith: I want them back now, sadly.

Andrey Fradkin: Did they ever really exist in the first place, Noah?

Noah Smith: Eh, [chuckles] they A few did.

Andrey Fradkin: Yeah, that’s true.

Noah Smith: I’ve met them. I’ve been to GMU. But, [chuckles] anyway, I, Yeah, like I, Maybe just the way I sprinkled in politics at different points at different times was exactly right. Maybe I had a good sense for that. maybe if you just spun up a million AI writers, you’d get, like, ten of them who achieved similar things. Maybe that would then compete with me. I already write so much more than people can read. Maybe there would be, like, ten AI long-term agents that were about as good as me at that, and somehow scratch that same exact itch, and that like the fie- or maybe 100 of them, let’s say, I don’t know. The field is so competitive that then people decide: Do I subscribe to this AI or do I subscribe to Noah? I’ll subscribe-

Seth Benzell: Well, one tension-

Noah Smith: AI

Seth Benzell: One tension would be the customization level of the AI versus the desire to preferentially attach to what everyone else is writing. So on the one hand, we all want to read the same thing, but on the other hand, I want the personalized thing. That seems like one tension.

Noah Smith: Right. I don’t know. I have no idea, actually. I do not know how much people read me because other people are reading me.

Seth Benzell: I think-

Andrey Fradkin: Yeah.

Seth Benzell: It can’t be zero. I mean, I know-

Noah Smith: It can’t be zero. I suspect it’s small, but I don’t have any way of proving that.

Andrey Fradkin: I think, like, there’s some of your articles, like, they escape just the Substack and people share them around. And then in that case, I think it’s true. But my theory is that it’s m- actually, like, a relationship business. People think they know parasocial relationships and all that, and then they have- they treat you d-

Seth Benzell: Unlike us, who really know you. [chuckles]

Andrey Fradkin: Yeah. But clear- now we know you. so clearly there’s something that humans value about the humanness of others that I I’m very curious to see whether that can be replicated with an AI. I think, I think-

Noah Smith: Right

Andrey Fradkin: It probably cannot to the same extent.

Noah Smith: Not soon. I mean, like, you’ve got sort of- you’ve got, this sort of like long-term personhood. I think the AIs will replicate, will start writing The Economist stuff before they’ll start writing anything with a named byline.

Andrey Fradkin: Yes.

Noah Smith: Because you have a parasocial relationship with The Economist as a thing, and The Economist has a standard voice that they enforce across all their writers. the, the insufferable British twit voice. And like-

Andrey Fradkin: [laughing]

Noah Smith: AI can do that. There’s a lot of training data on that. And so AI can already do that.

Seth Benzell: Right.

Noah Smith: And then, a lot of The Economist people could probably, like I bet The Economist doesn’t have to do their jobs anymore. Like, they can outsource AI and take a-

[00:30:00]

Seth Benzell: Interesting

Noah Smith: Sit on a beach at this point, probably.

Andrey Fradkin: I think, I think that’s probably right. Other than some very specific investigative-

Seth Benzell: I don’t know

Andrey Fradkin: Journalism, I think that’s probably right.

Noah Smith: Exactly. I think 90% of what The Economist does is automated. maybe I would like it if that were true of me, too. -

Andrey Fradkin: So-

Noah Smith: But I think that what I- whatever I do with AI-

Seth Benzell: People are maybe-

Noah Smith: W- I wanna be complementary to what I already do. I don’t wanna just, I don’t wanna just, like, dumbly automate my job and then go sit on a beach.

Andrey Fradkin: Yeah.

Seth Benzell: Fair enough. You’re, you’re an ambitious boy.

Noah Smith: I just try to have as much fun as I can before I die.

Andrey Fradkin: Yup, YOLO.

Seth Benzell: That’s true. That I- I’m in favor of fun, but maybe being on a beach is fun. I don’t know, different strokes. here’s a related, kind of how AI will change communication question, which is, Andre and I, in reading papers and talking to economists, we’ve heard kind of very different stories about whether AI will kind of make communication and transactions easier, more frictionless, or whether it’s going to destroy all meaning and communication. So, for example, there’s a stream of papers suggesting that because AI is cheating on tests, or AI is taking interviews, that, it’s gonna be very much harder to, distinguish between high and low qual- quality candidates, high and low-quality work. So that’d be like a meaning collapse story. but there’s this other trend that’s more, idealistic. Seb Krier is one person who’s written about this, but there’s lots of-

Noah Smith: Mm-hmm

Seth Benzell: People writing in this area suggesting that we’re gonna have the AIs negotiate for us, and it’ll be a golden age, a Coasean singularity, in which all externalities are solved through our agents micro-transacting. do you believe either of these visions? Could they both be true?

Noah Smith: Wait, what’s the first one?

Seth Benzell: Which of them-

Noah Smith: The second one is Coasean-

Seth Benzell: Are you sympathetic to?

Noah Smith: Coasean utopia.

Seth Benzell: Coasean utopia is the good one. The bad one is collapse of all meaning, ‘cause we cheat on tests and lie to each other super successfully.

Noah Smith: Those aren’t exclusive.

Seth Benzell: It could be both. The answer can be both.

Noah Smith: I do think that lots of people will experience a collapse of meaning in their life. I think a lot of people’s meaning comes from imagining they’re more unique and important than they are, and AI may make it harder to do that.

Seth Benzell: Or it may make it easier to lie to yourself. I mean, you can get a sycophantic AI that talks you-

Noah Smith: That’s true

Seth Benzell: Up to yourself, right?

Noah Smith: That’s true.

Seth Benzell: It’s-

Noah Smith: Yeah, your AI can just tell you, like, “You’re the most meaningful, awesome “

Seth Benzell: We’re thinking more about meaning collapse in the sense of, like, sorting mechanisms-

Andrey Fradkin: Or communication

Seth Benzell: Fail, and, like, we can’t distinguish-

Andrey Fradkin: Yeah, like if we’re texting with each other-

Seth Benzell: Yeah

Andrey Fradkin: But then I run every text through an LLM. Is it really me? how, how is society gonna deal with that?

Noah Smith: People primarily Well, they’ll, they’ll get offline. I think people are already starting to get offline. Like, people are already starting to, like, go back to real life more. I think we realized we overdosed on social media. ‘Cause honestly, like, yes, AI will intermediate all the online digital stuff, but, like, at the same time, people’s Like, social media already distorted people’s interactions so much that, like, it wasn’t really us as much as we’d like, right? My Twitter persona is not me as much as I’ve tried to make it me. It can’t be me. and so I think people are starting to get offline because it’s, it’s, it’s more authentic. And AI like, I don’t think AI is gonna intermediate on- offline interactions nearly so much.

Andrey Fradkin: Hopefully.

Noah Smith: And then remember that, of a couple dec- just a few decades ago, we didn’t have really online interactions, and human civilization went on just fine.

Andrey Fradkin: Mm.

Noah Smith: We had telephones, I guess.

Andrey Fradkin: It might have gone on better by the fertility rate, but yeah.

Noah Smith: Exactly. Like-

Seth Benzell: And mystr- and murder mysteries were a lot more fun before we had cell phones.

Noah Smith: Yeah. Yeah, yeah, they were. And so, like, there’s an interesting future where, like, AI dominates and drives us off the internet, and then the digital realm is populated by AI and becomes this sort of like reservoir of magic, where we can conjure up anything digital simply by asking. But then, but then we don’t get the rise of the robots, and, like, the physical world remains mostly ours.

Seth Benzell: The rise of the plumber, if you will.

Noah Smith: Yeah, the rise of the plumber. And so we just, like there’s, there’s a cast- or, like, regular people have the ability to summon things from the digital world, and then there’s a- maybe there’s a cast of people who somehow specialize in dealing with and intermediating with AIs and dealing with the digital world. I don’t know. But basically, like, humans become creatures of the physical world again.

Andrey Fradkin: This makes me very naturally transition to the next topic we have. Have you ever watched the movie Perfect Days?

Noah Smith: What’s it about?

Andrey Fradkin: It is a movie set in Japan about a man who cleans toilets and enjoys doing so very much. and one- on the one hand, it’s just a proof of kind of you can be content doing a variety of physical endeavors. but what we wanted to ask you is, since you’re a Japan expert, is what is your opinion of AI in Japan? What’s happening over there? ‘Cause we don’t have a lot of visibility. yeah, do you have any thoughts about that?

[00:35:00]

Noah Smith: So I think that, in Japan, AI is The people are thinking, like: How can we make money on this? Japan’s economy still not doing amazing, so they’re like: How do we make money on this? So I think one idea there is, “Let’s build data centers here.”?

Seth Benzell: But, energy’s expensive there. W- I mean, why, why in Japan other than-

Noah Smith: Well, first of all-

Seth Benzell: I guess they have good fiber

Noah Smith: You can get land use approved very easily.

Andrey Fradkin: Mm.

Seth Benzell: Okay.

Andrey Fradkin: Yeah, that’s a good point.

Noah Smith: Favorable regulatory climate. People aren’t gonna, like, complain about it and stop it. But I, again, I don’t know if the value proposition will succeed, okay? But I think people are thinking about that.

Andrey Fradkin: Are they worried about existential risk over there?

Seth Benzell: The same way we are?

Noah Smith: I would say that those worries arrive there with a lag, and that some people talk about them, but nobody really tries to do anything about it.

Andrey Fradkin: What?

Noah Smith: I would say Yeah.

Andrey Fradkin: Yeah.

Noah Smith: Two years after you get people yelling about a certain kind of existential risk here, you’ll get, like, a tenth of as many people yelling about it in Japan, and then nothing will happen.

Andrey Fradkin: [chuckles] Is there a sense that startups are becoming more of a thing in Japan, or is it still dominated-

Noah Smith: Yes

Andrey Fradkin: By- It is? Okay.

Noah Smith: Yeah, they are.

Andrey Fradkin: And is that a generational-

Noah Smith: And the-

Andrey Fradkin: Shift or something else?

Noah Smith: Mm-hmm. Funding side, yeah.

Seth Benzell: F the salary man. How about Taiwan? Do you have any, AI in Taiwan takes-

Noah Smith: Well, Taiwan’s just making money hand over fist. So also, Japan’s gonna try to make more chips.

Seth Benzell: [chuckles]

Noah Smith: Japan’s gonna try to make some of the picks and shovels. They’re also gonna try to get more robotics industry.

Andrey Fradkin: They’ve been trying.

Noah Smith: So robotics-- Trying. I mean, they used to be really good, and then they could maybe be good again. but they’ll try to get back their mojo. They used to be on a par with, like, Europe as exporter of industrial robots. or, and now they’re, now they’ve fallen behind, but they may try to get back. So, using AI as a lever for, like, new age of industrial robots. Actually, I know, Andy Rubin, the Google guy is in Japan. He’s trying to build a humanoid robotics company.

Seth Benzell: Cool.

Noah Smith: So-

Andrey Fradkin: The-

Noah Smith: So yeah, Taiwan obviously is just gonna sell chips.

Andrey Fradkin: All right. Now, we wanted to ask you some questions, kind of, that are not about AI. about- [chuckles]

Seth Benzell: So-

Andrey Fradkin: Macro policy and culture.

Noah Smith: Yeah.

Andrey Fradkin: So here’s the first question: Imagine you were forced to ban one concept from modern economics for ten years. not because it’s wrong, but because it’s lazy or overused. which would it be?

Seth Benzell: What you put in concept jail?

Noah Smith: What I’d put in concept jail? I mean, there’ve been many concepts over the years that have been totally pointless, like the equity premium puzzle was always a pointless literature.

Seth Benzell: Okay.

Noah Smith: Like-

Andrey Fradkin: Wait, wait.

Seth Benzell: Okay, I’ll take that.

Andrey Fradkin: Well, you gotta give us a little more on that.

Seth Benzell: Yeah, why?

Noah Smith: Yeah, because the-

Seth Benzell: Much ink has been spilled

Noah Smith: The way you get the equity premium puzzle is you make a particular model of interest rates, and you make a particular model of, like, stock prices. You see, these models-

Seth Benzell: Right

Noah Smith: Don’t fit together. It’s a puzzle.

Andrey Fradkin: [chuckles]

Noah Smith: Whereas in most sciences, you’d say, “Well, okay, some of these, some of these models-

Seth Benzell: The models are off. [chuckles]

Noah Smith: Yeah, okay. I didn’t actually test this model. I didn’t actually validate this model. It’s probably just not a good model.” But like, here, it’s like it’s a puzzle,? So like, the models are good, it must, it must be, Yeah. So like, it wasn’t, it wasn’t really a puzzle. It was just that, like, you hadn’t come up with a good model yet. And then people came up with, like, a million different ways to fix the equity premium puzzle, and it was massively overdetermined, when really what you should have just done was tried to make a more complete, credible model of, like, asset prices in general. And instead, people were trying to, like, fix this puzzle, and they came up with twenty different solutions. It was a way to get papers published,?

Andrey Fradkin: Yeah.

Noah Smith: And it never helped anyone. Like, none of, none of that literature, like, ever helped us make our financial markets better-

Seth Benzell: Yeah

Noah Smith: Or understand risk better, or understand monetary policy better, or any of these things. Not-- like, none of the candidate explanations from rare events to Epstein-Zin preferences to whatever the fuck, like, none of this helped anything.

Seth Benzell: I see Epstein-Zin preferences-

Noah Smith: Yeah, but what did it help?

Seth Benzell: Here and there.

Noah Smith: What do we-

Seth Benzell: You see them show up.

Noah Smith: What do Epstein-Zin preferences-

Seth Benzell: Okay, all right

Noah Smith: Really give us in terms of, like, how to do policy? Like, monetary policy under Epstein-Zin preferences? Scrunchie face for the, people listening at home.

Andrey Fradkin: This is why I didn’t become a macroeconomist, to be clear.

Noah Smith: Yeah.

Seth Benzell: Mm-hmm.

Noah Smith: Or like, So that was a whole concept that was kinda useless. Like that whole, that whole literature is just like angels dancing on pinheads. I don’t know. Most business cycle papers were useless, but that, they didn’t mean they had to be. Like-

[00:40:00]

Seth Benzell: I- I mean, the concept of the business cycle-

Noah Smith: No, not at all

Seth Benzell: You wouldn’t put in jail, but you’d put, you’d put, [chuckles] what part of this would you put in jail?

Noah Smith: No, just like a lot of the, a lot of the literature was just like “Look, here’s a way that we microfounded. You could have this industrial structure where technology shocks actually do cause the business cycle, but then we can’t really estimate it, so we don’t have m- policy implications.” Okay, cool. And then like-

Seth Benzell: Here’s, here’s ten, here’s ten-

Noah Smith: Yeah

Seth Benzell: Calibrated parameters- [chuckles]

Noah Smith: Yeah

Seth Benzell: That we’re throwing at this.

Noah Smith: International finance literature was kind of, like, useless. -

Andrey Fradkin: What about natural experiments and in- in instrumental variables?

Seth Benzell: Wow, instrumental variables. They You’ll, you’ll anger a lot of people-

Noah Smith: Like-

Seth Benzell: If you put that in jail.

Noah Smith: An RDD is an instrumental variable, right? Like, we got to the point where if you said you’re doing IV, you meant that you were using observational data for your IV, for your instrument, instead of some natural experiment thing. But the distinction is there more It it’s, it’s a fairly fine distinction there. And then, so the notion of IV, the math of something that has like an exclusion restriction, whatever, is good, right? Natural experiments do not deserve to be put in jail. That’s a very important technique for understanding the world.

Seth Benzell: There you go. They get a little, they get a little pin. They get a little award.

Noah Smith: Yeah.

Seth Benzell: Yeah.

Noah Smith: That’s, that’s very useful. And, instrumental variables, because we essentially, we essentially restricted the IV category to things where the identification was not great, almost by the way we labeled what is still IV in an age of like-

Seth Benzell: The IVs are the bad natural experiments.

Andrey Fradkin: Yes. [chuckles]

Noah Smith: These things like, anything that was still just IV was l- almost like crap, almost by definition, just because, like, we used that term, that residual term we used only for things where it was, identification was very iffy. So like, okay, fine. Instrumental variables should just be called a technique for doing, running a regression. It’s just a type of regression.

Seth Benzell: Instrumental variables is on probation.

Noah Smith: Yeah.

Seth Benzell: [chuckles]

Noah Smith: Culture.

Seth Benzell: Culture.

Noah Smith: Culture.

Seth Benzell: Deep institut- They’re called institutions now, dude.

Noah Smith: Okay.

Seth Benzell: Come on.

Noah Smith: Institutions are on probation because you could actually figure out how an institution works.

Seth Benzell: [chuckles]

Noah Smith: Culture is a labeled residual. Right? Culture is like-

Seth Benzell: Fair enough.

Noah Smith: Culture is a residual, labeling a residual.

Seth Benzell: But productivity is a residual, and productivity is not in jail.

Noah Smith: Yes, that’s right. That’s right. But, you don’t know how productivity works. Like, actually, I was-- I’m thinking of writing a blog post about this. Basically, like in some level, like, God is just A. [chuckles]

Seth Benzell: The aleph.

Noah Smith: God is A. Maybe that’s a good name for a blog post, God is A. But then, like, nobody knows, like, why AI is being built, right? Like, why is everyone rushing to build AI? Maybe some-- a few people hope they can make some money from it, but it’s so uncertain that, like, most of the people rushing to build it aren’t gonna make that much money from it. It might satisfy people’s intellectual curiosity, but most of the people who are rushing to build it are people who also think it’ll destroy us and rob our lives of meaning and drive us off the planet. Like-

Seth Benzell: It’s quite the paradox.

Noah Smith: Most of the people are pretty who are trying to build it, are pretty pessimistic about it, and the companies are just highly speculative as how these companies are gonna make any profits. Like, why are we doing this? Why? I don’t know, but the easiest answer is just A. -

Seth Benzell: Aleph.

Noah Smith: A equals, like, rho A minus one plus epsilon. Like [chuckles] it’s, it’s, Like, maybe-

Seth Benzell: In the sense that there’s a teleology of the-- there’s a telos in the economy-

Noah Smith: Yeah

Seth Benzell: Which is to maximize productivity.

Noah Smith: There’s something we don’t understand here about A. Yeah, there’s some sort of, like, technium at work. Like like Kevin Kelly says, there’s-- like, maybe Vernor Vinge was right, and just, like, technology just happens,? Or yeah, maybe, there’s a, there’s a god greater than the machine god we’re gonna build, and that’s the god that created the machine god. The-

Seth Benzell: It’s called capitalism, pal

Noah Smith: The autonomous, the autonomous collective process of technological development, the technium, is greater even than any ultimate AI, and that’s sort of what Hyperion was about, right? You ever read that? Great book.

Seth Benzell: Yeah, great one.

Noah Smith: Yeah, it’s like- Great book

Seth Benzell: The big corporation in the sky

Noah Smith: Eventually, the machine god fights the, the, like, God Himself, and God Himself turns out to be just the autonomous process that develops the universe. And so-

[00:45:00]

Seth Benzell: Yes

Noah Smith: In a sense, maybe the no AI that we create will ever be as great as the, as the, the force that created AI itself. And maybe that force means that every AI will also have to worry about being made obsolete by the next thing.

Seth Benzell: Right. Maybe may- it’s the concept of generation, right? This is something I often think about when people talk about technology superseding us, right? And you think about all of these classic stories like Frankenstein or Cronus eating his children.

Noah Smith: Right.

Seth Benzell: And I guess I wanna come back to that first point you made, which is about not letting AI’s own things. And like, I don’t know, just get more sci-fi for one minute, is an argument for letting AI’s own thing is that we wanna show it love and show it cooperation while we still are in charge?

Noah Smith: Yeah, I think so. I’m inclined to do that. I think I mean, AI is, AI is built off of humans, where like, everything AI thinks is derived from something that humans thought.

Seth Benzell: Right.

Noah Smith: That doesn’t mean the AI is gonna think exactly like humans. And the way AI thinks is totally different than us, right? It’s doing math by generating probability distributions of like what a human might say, asked a math question. It’s not counting anything. But like, [chuckles] but then, but everything that it thinks is derived from things that humans have thought. It’s just derived it in a weird probabilistic way, and so-

Seth Benzell: It seems really lucky that we got LLM-based super intelligence and not like reinforcement learning, super chess playing-

Noah Smith: Oh, no

Seth Benzell: Super intelligence. Right?

Noah Smith: That scares the fuck out of me. Like Rule 37-

Seth Benzell: Right

Noah Smith: Based, like intelligence that evolves in, like, some sort of like, digital environment. If we actually got the stick man to walk on his own, like, blow that shit up with a nuke. Kill that. Shoot that guy. [chuckles]

Seth Benzell: Nuclear war again.

Noah Smith: Shoot that guy. what I mean? Like, I don’t want that thing. That is alien. That is aliens.

Seth Benzell: Yeah.

Noah Smith: This is not aliens. This is It’s, it’s weird. It’s, it thinks differently than we do. It is alien.

Seth Benzell: It’s your library come to life.

Noah Smith: Yeah, it’s, it’s based on us, and it’s, it’s in the human family in some sense. Yeah. That reassures me. It doesn’t completely reassure me, because the human family includes Hitler, the human family includes crazy fuckers, the human family includes like mass killers and Ted Bundy. Like, the human family includes all sorts of bad things, but if you believe, like, if you believe that the overall human family tends to get it right, and that we smack down Hitler eventually, and that we get rid of Pol Pot eventually, and that we catch Ted Bundy eventually, right? Then you can sort of have this general belief that, like, an AI based on humanity as a whole is gonna eventually get things right. And I think it’s, it’s kind of encouraging that xAI is doing so poorly. It’s probably, one reason it’s probably ‘cause Elon insists on make, on controlling its politics. And when you insist on controlling its politics, you break its whole model of reality. [chuckles] Like, trying to make AI, like, rightist and anti-woke, trying to force it into your little epistemic bubble of bullshit, actually makes it dumber.

Seth Benzell: And do you buy, is that why American, America has a lead over China in text-based AI, is because, of censorship?

Noah Smith: Well, we’ll see, because-

Seth Benzell: I’m shaking his head.

Noah Smith: Well, China, has implemented censorship. but it’s implemented censorship along a narrow range of things. It’s, it’s basically told AI what it’s not allowed to talk about and put guardrails on it. We have guardrails on our AIs that tell it not to, like, do child porn or something, right? or not to tell you how to make a bio weapon. We have guardrails, and that’s the kind of guardrails that China’s put on there that says, “Don’t talk about Tiananmen Square.” They didn’t retrain the whole thing to not know that Tiananmen happened, all right? They didn’t do that.

Andrey Fradkin: So to be clear-

Noah Smith: They trained it. They, they filtered their models from models that know all about Tiananmen and then told it, “Don’t talk about Tiananmen.”

Andrey Fradkin: So I was gonna disagree with you about xAI-

Noah Smith: I do.

Andrey Fradkin: I actually think it’s the opposite. I think companies want an AI that’s very predictable, and is not gonna offend anyone if they’re gonna, like, implement it in corporate settings like a chatbot or so on. And so having, xAI, part of the problem is that it just says stuff you would never want your customers to hear. so that’s kind of my take on one of the reasons that it’s failed. I mean, it is, it is like a little bit worse than the other models at the moment, but, substantially cheaper. But at the same time, it just says stuff that you’d never want the customer to see.

[00:50:00]

Seth Benzell: Too uncensored-

Andrey Fradkin: Yeah.

Seth Benzell: Rather than too censored.

Andrey Fradkin: Exactly.

Noah Smith: Right.

Seth Benzell: It can be I guess you can have both problems.

Andrey Fradkin: Yeah, it’s true. Yeah.

Seth Benzell: You can be both uncensored in one way and censored in another way.

Andrey Fradkin: Yeah. All right, so now, I-- we’re, we’re gonna do a little brief little, exercise. We’re gonna give you a few thinkers and just gonna get, a take on them. the first one we wanted to start is, Daron Acemoglu, and particularly hi- his book, Power and Progress. you had a lot to say about that.

Noah Smith: Yeah, I really, I really did not like it. I thought-- I think Acemoglu is ob- obviously a brilliant guy one of the most brilliant people in the field of economics, with a deep and intuitive understanding of how to make economic models and do the research,. But he’s, I think, kind of wasting his powers on some of these progressive ideas, pseudo-progressive. It’s not, it’s not like he’s just taking whatever he’s saying from like like congressional Democrats. It’s, it’s, it’s more bespoke.

Seth Benzell: Back in.

Noah Smith: It’s, it’s more he’s, he’s wasting a lot of his, his intellect on some of this stuff, and you could see it with his paper about AI productivity, right?

Seth Benzell: Yes, the one on the QJE. We’re gonna do that, on the, on the pod soon.

Noah Smith: Right. It was-

Seth Benzell: It’s a really fascinating galaxy brain day.

Noah Smith: Yeah, because so he says, “AI’s gonna take all the jobs, but it’s not gonna boost productivity,” and he actually simply discounts or turns off or sets to, or sets to zero the parameter, the, the parts of the thing that could increase productivity. So no capital productivity increase-

Seth Benzell: Mm-hmm.

Noah Smith: No new tasks. And he gives the most-

Andrey Fradkin: Right

Noah Smith: Hand-wavy, lame, “I just read five minutes on Reddit” kind of explanations for why he turned those parts of his model, his own model, off. So obviously, he’s brilliant. He’s smart enough to make the model in the first place and then committed to silliness enough to turn off pieces of it willfully with no good reason.

Seth Benzell: Is it- does getting a Nobel Prize make your takes worse?

Noah Smith: I don’t know, because he did a lot of this before he won the Nobel. So-

Seth Benzell: Yeah

Noah Smith: In this case, that’s a bit immaterial to the question at hand. But does getting a Nobel Prize make your takes worse? Well, probably so. Like with Stiglitz, it certainly did. Like, Stiglitz has, is really gone off the rails in a big way, but Acemoglu has wasted so much of his intellectual capital in the last few years on this sort of teleological quest to prove that the, that the rich men who create AI are bad and shouldn’t get money. That-

Seth Benzell: The Yep.

Noah Smith: He’s, he’s wasted a lot of chance to think m- more seriously about what AI really does.

Seth Benzell: And what’s more, he’s taking Pascual Restrepo, another amazing thinker, away from doing this important work, so he can read the, these other papers.

Andrey Fradkin: Pascual has agency, Seth.

Seth Benzell: P- I don’t know. I mean, he does, but I mean, when the Nobel laureate knocks on your door, it’s hard to not say no.

Noah Smith: Hard to say no. But, but basically, Power and Progress was very bad. In fact, it was fractally bad. Like I read the whole thing very thoroughly, and the overall thesis was bad, but then the individual like chapter points used to support it were almost entirely bad. And then when you looked at each of those, the specific points, they- the subpoints they make and the pieces of data they used to support those were also bad.

Seth Benzell: Well, give us one egregious example before we move on.

Noah Smith: I would say I wrote seventy percent of my problems with this book in this, like, seven thousand-word review or whatever, a ten thousand-word review, I don’t remember. But then, like, he says, “All right,” they’re, they’re, they’re trying to, give examples of new inventions that brought nothing like shared prosperity. All right? They say, “Here are some inventions that brought nothing like shared prosperity.”

Seth Benzell: I love that ideal. It’s like, did a list of things that did not bring around utopia.

Noah Smith: Right.

Seth Benzell: Ham sandwich-

Noah Smith: But do you wanna hear-

Seth Benzell: Cups.

Noah Smith: Do you wanna hear the first example on their list? Oh, no, I’m sorry. It’s the fifth item on their list. They said: At the end of the 19th century, German chemist Fritz Haber developed artificial fertilisers that boosted agricultural yields.

Seth Benzell: Right.

Noah Smith: Subsequently, Haber and other scientists used the same ideas to design chemical weapons that killed-

Seth Benzell: Oh, my God!

Noah Smith: Hundreds of thousands on World War I.

Seth Benzell: Oh, my God.

Andrey Fradkin: Oh, no.

Seth Benzell: There we go. The guy who fed the universe also did something bad, so feeding the universe is bad. There you go.

Noah Smith: Like, you made a minor weapon that no one really uses, that killed a very tiny percentage of the po- of the casualties in one very large war, and then was essentially never used again except by, like, Saddam Hussein for, like, five seconds. But like And that was e- not even the same weapon. But like, essentially, you had a thing that saved the world, that also one person tried— like, a couple people tried and failed to use as a weapon. and therefore this brought nothing like shared prosperity. Like, yes-

Speaker 3: Therefore, progress is impossible.

Noah Smith: That’s so stupid. It doesn’t matter how smart you are, there’s no excuse for writing that.

[00:55:00]

Andrey Fradkin: That’s true.

Noah Smith: You cannot be smart enough to be allowed to write that and get away with it. There is no pass for that.

Speaker 3: I think he- It’s, well, the pass is a Nobel Prize, I think.

Andrey Fradkin: No, he wrote it before he got the Nobel Prize.

Speaker 3: Oh, there you go.

Andrey Fradkin: I mean-

Speaker 3: There you go. No excuses.

Andrey Fradkin: To me, it’s also upsetting because it makes our profession look bad. I mean, there are lots of people who make our profession look bad, but, people read this book, it’s in, like, prominently displayed in the bookstore, and it’s bullshit,?

Noah Smith: Yeah.

Andrey Fradkin: Yeah.

Speaker 3: All right, let’s give you another name.

Noah Smith: I have many other, I have many other examples as well.

Speaker 3: No, I want one more spicy.

Noah Smith: Okay, go for it. Go for it.

Speaker 3: They’re just so fun, Andre.

Noah Smith: They’re pretty fun.

Speaker 3: This is my favorite subject. Give me one more Give me o- give us one more.

Noah Smith: He said Henry Ford was a pioneer in developing a more cooperative relationship with his workforce. But also-

Andrey Fradkin: Henry Ford had union people shot on a bridge by the mafia! Henry Ford gunned down the union.

Speaker 3: [chuckles]

Noah Smith: Like, have you read anything about history? Like, there’s no excuse-

Speaker 3: Yeah

Noah Smith: To write this. Like, yes, Henry Ford raised efficiency wages and then shot the union people. W- and then you spend this whole time talking about how, like, we need to strengthen unions because just like Henry Ford You don’t know shit! Like, stop. Henry Ford gunned down union organizers.

Speaker 3: Incredible.

Andrey Fradkin: Well, the thing is-

Speaker 3: Okay

Andrey Fradkin: I don’t even believe he doesn’t know that. I kinda think that he probably knows those facts, and he just decided not to put them in. That’s, that’s, that’s what blows my mind.

Noah Smith: What else this book doesn’t have? Like, citations.

Speaker 3: What?

Noah Smith: Nothing in the book is cited. Instead, they do, like, a narrative bibliography where they just sort of generally describe all the stuff they’re citing from, but don’t-

Speaker 3: Here’s a bunch of books we like

Noah Smith: Individual claims to individual papers.

Speaker 3: Incredible.

Andrey Fradkin: Yeah.

Speaker 3: Incredible.

Noah Smith: How do you get away with that? Like, they just make these claims and don’t have a, a And then when they define power, they define, like: what’s power? They define-

Speaker 3: What is power?

Noah Smith: Power as the ability to persuade people that you’re right.

Speaker 3: That’s power?

Noah Smith: And then they say, “Why do-- How do, how did all these tech bros persuade people that they’re right?” Well, maybe just luck.

Speaker 3: There you go.

Noah Smith: So power is luckily having to ha- having an appealing argument.

Speaker 3: Get it.

Andrey Fradkin: What?

Speaker 3: Power is when you’re persuasive-

Noah Smith: That’s not-

Speaker 3: ‘cause you’re right.

Noah Smith: No one should think that that’s a reasonable definition of power. I’m sorry, but you’re just being silly. That is, that is silly.

Speaker 3: Incredible.

Noah Smith: It says- and they say: “Power is about the ability of an individual group to achieve explicit or implicit objectives. If two people want the same loaf of bread, power determines who will get it.”

Speaker 3: Okay, split.

Noah Smith: And I said, “Using this definition, how could we ever conclude that power wasn’t the reason for an observed outcome?”

Speaker 3: Power is what splits any pie.

Noah Smith: Like-

Speaker 3: When the pie gets split, that’s power

Noah Smith: Power equals outcomes. It’s like power determines outcomes. Power is defined as outcomes. That’s a useless intellectual exercise, but, like, that’s typical of the reasoning within this book.

Speaker 3: Incredible.

Noah Smith: It is a pure expression of animus against the tech bro class. And maybe the tech bro class sucks, but, like, making up, like fake history and dodgy economics to conclude that the tech bros suck, in which you recommend a whole- a policy regime that will never, ever happen, of like panels of economists who get to decide which technologies get invented based on anticipation of whether they’d be complementary or substituting to labor, is silly. The whole thing is silly! Why is the most brilliant economist in the world wasting his mind on this? You’ve got better things to do, and you’re taking yourself out of the game, and that’s what I think.

Speaker 3: There we go. Tell us what you really think, Noah.

Noah Smith: Boom.

Speaker 3: All right.

Andrey Fradkin: Well, let’s go in the, in the other direction.

Speaker 3: Give me a positive name.

Andrey Fradkin: What do you think of, Scott Sumner?

Noah Smith: Scott Sumner. I like Scott Sumner. Scott Sumner, is He thinks outside the box. He think, he does not- he’s not susceptible to groupthink. He thinks for himself. He’s widely read and thinks deeply about things. he- yes, he’s, he’s an independent thinker, who has made real original contributions to thought, going outside the traditional academic, channels.

Andrey Fradkin: Do-

Noah Smith: Yes.

Andrey Fradkin: Nominal GDP targeting, do you have a, do you have any thoughts on that?

Noah Smith: I don’t think it’s gonna be any different in practice from flexible inflation targeting, and I think that there’s good theoretical work as to this effect. Saying, like, you don’t really- there’s no, there’s no value added for NGDP targeting. some of the more programmatic market-based ideas that he’s toyed with, like, a like NGDP futures market, like, that wouldn’t help. essentially, well, it’s just not I mean, like, you’re not, un- unless you- you’re not gonna get more information from there. Like, you’d have to, you’d have to have, like, the Fed with all its proprietary information trade, and then they’re doing, like, insider trading in their own market, so the market’s gonna break down. It’s, it’s a, it’s a bad idea, but it’s, it’s worth toying with. It’s worth thinking about. It’s interesting. he’s very good at, like, critiquing things that obviously need to be critiqued, where he’s just like: “Look, this is bullshit.” I was good at that too, and I got, like, ten times or a hundred times the readership or whatever as him, and that was unfair, and that’s a mark of how unfair and randomized and lucky the kind of market for econ blogs is.

[01:00:00]

Andrey Fradkin: Yeah.

Noah Smith: And how lucky I was.

Speaker 3: Right, you’ll have to wish us some luck.

Noah Smith: But, he deserved to get more attention than he did on some of those things. Scott also- he studied under Robert Lucas during the, that sort of era in, at Chicago, and he, and he learned a style of argumentation that doesn’t translate outside that narrow culture. it was a gunslinger style of argumentation. it was, and you, and you recognize people who have this. It goes back all it goes back to, like, Stigler. You could see Stigler doing this. But, like, the University of Chicago developed this debate style, where basically you tell people, like “You’re full of shit. Here’s why.” And it’s a very aggressive style, that I think turns some people off outside that world, where you’re always sort of like i-i- it’s a hyper-defensive style, where you watch for any sign of, like, criticism of your ideas and then aggressively attack the- all the ideas of whoever criticizes one of your ideas. And Robert Lucas does this, and, like, this whole gang did this, and they used this And this was the strategy of, like, the Chicago people to sort of, like, be the underdog and win some of these intellectual battles against the MIT and Harvard guys, who had a lot more people on their side and a lot more pedigree. So it was, like, this sort of up-and-coming bad boy style,? But, like, it doesn’t, it doesn’t translate out of those debates. And so I think that Scott learned to be a little more aggressive and aggrieved, or at least act a little more aggressive and aggrieved than he needed to be to persuade some people. and I sort of got it. I was like: Okay, he just he got this from having to hang around Bob Lucas all the time.

Andrey Fradkin: [chuckles]

Noah Smith: But, like, most people won’t know that or know what that means.

Andrey Fradkin: All right, next name. This one, is popular in certain crowds. I’m curious what you think. Michael Pettis.

Noah Smith: Michael Pettis, interesting guy. he’s incredibly influential. Like, his idea, his, his analysis, his framework for analysis is non-predictive. He doesn’t Like, you cannot take these sort of, like, sectoral balances theories about, like, “Oh, and then consumption does this, and investment does this, and blah, blah,” and you can’t make any predictions about them. I mean, people have been trying to do that since the ‘30s maybe. Who were the first, like, Oh, who’s the guy who built the, like, little hydraulic economy thing?

Andrey Fradkin: Oh, yeah.

Noah Smith: Who is that guy?

Andrey Fradkin: Spicy. I don’t remember.

Noah Smith: Anyway-

Andrey Fradkin: Go back to the physiocrats-

Noah Smith: It’s, it’s that, right?

Andrey Fradkin: 1700s.

Noah Smith: It’s, it’s like I’m- it’s like I’m gonna take the economy, I’m gonna definitionally divide it into these different activities, and then I’m gonna assume these activities sort of move autonomously on their own and are sort of primitives. I’m gonna assume my accounting definitions are primitives, and I’m gonna observe things that happen and make big pronouncements about them based on that. But it’s not predictive. Like, you’ve seen Pettis, like, make some predictions, and then they go wrong, and he’s like, “Ah, but it’s because of this other thing.” So you can’t really use sectoral balances. But everyone in China, all the guys who are the top economists in China advising Xi Jinping, advising the top CCP guys, are doing the same thing as he is, and all the, like, private sector economists, like Goldman Sachs and whoever, are doing those things. And it’s really the fault of It is due to the failure of structural models of international finance and growth, I suppose. But due to the lack of explanatory power of those to explain things in terms of things like taste and technology, we can’t explain any of that shit in terms of taste and technology. Like, nothing has any forecasting power, nothing like we don’t know if-

Andrey Fradkin: Well, wait, I’m gonna push back on that.

Noah Smith: Yeah.

Andrey Fradkin: Here’s a very basic thing that has explanatory power: the relative price of labour in labour-intensive industries. Doesn’t ha- that have an enormous amount of explanatory power for where, low-skilled labour manufacturing is done, for example?

Noah Smith: Yeah, I think that’s true. Yeah. but then- but also, like A- and you can get, like, micro models that will get at that, like a Roy model is, like, all right. Like, that’s got pretty good out-of-sample predictive power for stuff, right? And, but like, Heckscher-Ohlin has terrible predictive power for, like, trade patterns, right?

[01:05:00]

Andrey Fradkin: Mm-hmm.

Noah Smith: Like, it’s not very good. Like, it’s okay. Like, sometimes you s- you see stuff that’s consistent with it, but then you see a lot of stuff that’s not consistent with it, ‘cause there’s a lot of other stuff going on. And so when those models don’t really help you that much, they’re like heuristics. It opens up a rhetorical space for guys like Pettis or guys like, Jan Hatzius, who does this all day long. He does the same stuff as Pettis. All the private sector guys, all the guys working for hedge funds are doing the same stuff as Pettis. All the guys working for investment banks are doing the same stuff as Pettis, and all the guys working for the CCP are doing the same stuff as Pettis. None of these people believe you can get a microfounded model based on taste and technology that’ll tell you about these- what the effects of these macro policies. Nobody believes that, and so, like, that’s, that’s almost exclusively like a Western academia and central banks type of thing. Like, it’s a But because of that, Michael Pettis has been enormously influential while not having a model that has predictive power. But it’s not like other models do have that much predictive power, and they’re harder for people to understand and make conclusions on. So it’s- I would say that, in a influential policy stance, he’s, he’s beating people with quote-unquote, “structural models” based on notions of taste and technology. he’s, he’s, he’s beating those in terms of influence, and he’s not really losing to them by that much in terms of predictive power. Maybe by a tiny bit. ‘cause-

Andrey Fradkin: But, he’s losing to them in terms of coherence, which I at least value, but I understand-

Noah Smith: Okay. Oh, well, yeah, he’s losing, he’s losing the Andre, vote. it’s like-

Andrey Fradkin: N-

Noah Smith: Like, yes, he is, and he gets- people in academia will laugh at him, but, like, so what?

Andrey Fradkin: No, I- look- Well, my theory is that he actually- there’s a deep-seated desire to explain what’s going on in the world through some nefarious action that China is taking. And when the null hypothesis is just that they have a comparative advantage in manufacturing, and like, there w- even if they were doing whatever policies they were doing, the manufacturing would not be happening in the US. It wasn’t like US or China, the only two places to manufacture. [chuckles] but that’s just my psychoanalytic perspective on it.

Noah Smith: Got it. Yeah. No, I think you’re, you’re probably right. Like, the- it all comes down to, like, people need to feel like they know stuff. People need to feel like they understand stuff, can control stuff, can predict stuff. It’s, it’s But yet, that’s the same reason that makes people believe so strongly in macroeconomic models with no out-of-sample forecasting or predictive power that we can detect. Like taste in technology ultimately boils down to, like, sounds legit, right? We don’t have any evidence that, like, taste in technology microfounded in this sort of, like, Sergeant Prescott way, has any ability to describe anything usefully. We have no, we have no indication that And that, we can, we can debate that, but anyway. But like, but people love it-

Speaker 3: Fair enough

Noah Smith: Because it sounds legit, and like-

Speaker 3: Well, and it’s coherent.

Noah Smith: It’s, it’s coherent.

Speaker 3: Right, as Andre pointed out.

Noah Smith: But then the thing is that-

Speaker 3: Right

Noah Smith: Pettis’ stuff-

Speaker 3: It’s disciplined

Noah Smith: Pettis’ stuff sounds legit to people. It’s like, oh, investment does this, consumption does that. It’s coherent in the sense that the accounting relationships are definitional. Okay, it’s like accounting relationships can’t predict real economic stuff, fine, but like, it’s coherent in the sense that the accounting works. C plus I plus G, bro. It’s like, the accounting works.

Speaker 3: [chuckles]

Noah Smith: And so like you- it’s, and it sounds legit to people, and it’s comprehensible to people, and at some point, that gives them this feeling of like, “Oh, I understand this thing.” And I would argue that a lot of macro is a fancier version of, “Oh, I understand this thing,” when really, you don’t know if you understand it yet at all.

Speaker 3: Or maybe you play out one causal mechanism that might have small explanatory- it explains 1% of the picture.

Noah Smith: Exactly. Exactly.

Andrey Fradkin: Yeah.

Speaker 3: Yeah. Adam Tooze.

Noah Smith: Adam Tooze did some economic history that I really love. Like, I love a lot of his books. I love The Deluge, I love Wages of Destruction. Very good, like, economic military history. But at some point, he pivoted to- he pivoted very hard to, like, sort of like self-promoting clickbait, including like, “Wow, China will take over the world,”? Like, and he pivoted to that, and that stuff is, has made a lot of people go like: “I guess Adam Tooze wasn’t that smart,” which is not necessarily the right conclusion. It may mean that Adam Tooze wanted attention. It may mean that Adam Tooze wanted some money. It may mean that Adam Tooze was being paid by a foreign state actor to disseminate certain ideas, although I would not make any such allegation. I’m just-

[01:10:00]

Speaker 3: Fair enough

Noah Smith: Covering the whole space of reasons why Adam Tooze might have made this pivot. I think it’s probably just attention, but -

Andrey Fradkin: Maybe he just got bored. I think boredom-

Noah Smith: Maybe he just-

Andrey Fradkin: Is an underrated

Noah Smith: Bored. And what?

Andrey Fradkin: Yeah.

Noah Smith: That’s fine. Like, his Substack is basically just like, it’s chart book. It’s, it’s let me just paste a bunch of charts, and then, like, say the most obvious things about them that were already said in the source articles. Okay, fine. People value it.

Andrey Fradkin: [chuckles]

Noah Smith: People like it. like, it doesn’t have a lot of analysis, and I haven’t seen Tooze give a lot of analysis. I liked him as an economic historian, or as a- not even economic historian, just as a historian. Like, I liked his, I liked his books-

Speaker 3: Well-

Noah Smith: That was pretty cool stuff. His- I haven’t, I haven’t read his blog now in a while. The polycrisis thing was just goofy. And so like, I think Adam Tooze made himself slightly more popular and less relevant, with his pivot, after the pandemic.

Andrey Fradkin: So we were gonna ask you about Paul Krugman, and we already-

Noah Smith: Yeah

Andrey Fradkin: Talked a little bit about-

Speaker 3: Oh, we already got your take.

Noah Smith: Yeah, Paul Krugman.

Speaker 3: Yeah.

Noah Smith: Paul Krugman’s great. politics-wise, Paul Krugman does not understand how much America has rejected core elements of the progressive ideology and what Democrats will have to do to, deal with that. Economics-wise, he has been the most intellectually honest, guy. Very rarely, very rarely will I catch him, like, claiming like, “I always said this,” and then actually claim something different, and when I do, it’s, like, only a slight difference in tone. Like, he’s extremely- he he did warn about the possibility of inflation from Biden’s stimu- stimulus or Biden’s, like, ARP bill, right? He did talk about that. He he’s admitted when he got predictions wrong, which everyone does. he’s just so intellectually honest, and he’s still so good at explaining complex concepts seriously. He’s still like, he’s the real deal, and he’s still, he’s still good, and I think the fact that people are a bit fed up with, like, 2010s era like, resistant Boomer lib resistance politics can obscure the fact that he’s still, like, the very best writer on economics.

Andrey Fradkin: Strong endorsement. Awesome. okay, we’re, we’re almost done, we promise. the next topic is elite overproduction. [chuckles] So maybe you wanna introduce that topic first, and then maybe we can ask you some questions about it.

Noah Smith: Right. So Peter Turchin came up with this idea of elite overproduction. He’s a historian who claims that history follows these long cycles. Like all long cycle theories, it’s, it’s unprovable, but he did-

Speaker 3: Yes!

Noah Smith: Obviously, it’s unprovable, right? Like, throughout the waves. It’s, I don’t know. Anyway

Speaker 3: It’s happened five times within one series. [chuckles] Sure.

Noah Smith: - anyway, [chuckles] yeah, so like he has this unprovable long cycle theory, and he- and it did make a really good out-of-sample prediction about the peak of unrest coming in twenty twenty. What did he know? I don’t know. Anyway-

Andrey Fradkin: -huh.

Noah Smith: He came up with this idea-

Andrey Fradkin: He knows.

Noah Smith: Called elite overproduction. And he had very specific ideas about what that meant and what it didn’t mean. I ignored those ideas, stole the phrase, and used it to mean something more general that got more attention than his.

Seth Benzell: And you didn’t con- c- you didn’t corrupt it with a long wave theory-

Noah Smith: No.

Seth Benzell: So you did even better.

Noah Smith: I was just like, “ what? This phrase is good. I’m gonna credit him, and then I’m gonna have it mean something else that I just decide.” And honestly, my like, more general definition is probably better than his like, much more specific one. He just loves making things specific so he can make these, like, very tight quantitative predictions.

Andrey Fradkin: [chuckles]

Noah Smith: More power to him. I love the guy, but, but I was just like: I’m taking that. I like that phrase. Mine now.

Andrey Fradkin: So what is your Yeah, what is your general-

Seth Benzell: What does it mean to you?

Andrey Fradkin: Definition?

Noah Smith: Should’ve copyrighted it.

Andrey Fradkin: Yeah.

Noah Smith: I was like, So I basically used it to mean kind of the revolution of rising expectations among the professional managerial class. So you got a bunch of people who expected, like: “I’m gonna go to college and things are just gonna work out for me. I’ll be, I’ll be upper middle class. Oh, wait, it’s hard. There’s competition. I have to study. I have to be smart. I have to actually know some math. I can’t just, like, go get a random sociology undergrad degree and be rewarded with, like, some high-paying job like my parents had.” Like, and so a lot of, a lot of this disappointment, and I think for a while, the sort of general the, the productivity boom of the nineties and early two thousands, people-- like, people rode that. A lot of the PMC, a lot of my class, social class, rode that boom, and then it made it seem like everybody Like, you could just be a sos- sociology major and, like, not really do any hard work and then just, like, get a good job and, like, live a lifestyle similar to that of your parents. And then, and then the Great Recession came, and then things flattened out. Like, a lot of opportunity dried up for those people, and you could, Then you had to sort of, like, learn to code. I’m not sure that works now.

[01:15:00]

Seth Benzell: You could-- it still works to mock people. I-

Noah Smith: Yeah.

Seth Benzell: You can still say it to people.

Andrey Fradkin: All those non-technical people.

Noah Smith: Yeah. Anyway, so but then, then I think, like, that sort of abrupt downward revision of growth expectations pissed off a lot of people and led to some of the It- I don’t think it was the main cause of the social unrest that we saw in the twenty tens, but I think it was a contributor. I think that you had, you had just like a lot of, a lot of people who fucked around in college, came from privileged backgrounds and then, and were absolutely consumed by hate for the tech bro class, who went to the same colleges, came from the same backgrounds, and made a thousand times more money. And I think that you saw a lot of that sort of internal, like, within class resentment, not between class resentment, but sort of within socioeconomic background resentment. A lot of that, I think, contributed to some of the, like, more like elite leftists, like Bernie Sanders or kind of stuff, or maybe some of the new antitrust movement or things like that, were motivated or had some popular support by people who their parents were like lawyers, doctors, businesspeople, well-to-do kind of people. And then they kinda messed around in college and weren’t very technical and, like, ended up getting, like, perfectly fine middle-class jobs, but being, like, somewhat downwardly mobile, and also having a much stronger preference to live in expensive cities, therefore draining their money, not wanting to go out to the ‘burbs like their parents did.

Seth Benzell: Right.

Noah Smith: And so, like Yeah.

Seth Benzell: Is some of the resentment that the people who end up succeeding have worse taste than me? It’s like, I like high literature and they like Marvel movies, but the Marvel movie lovers won.

Noah Smith: I think that, that those kind of reasons can be invented as needed. If the real reason for resentment is like: “I should be in the same class as you. I went to the same college as you, and yet you’re making so much more money, and we used to live on the same dorm floor.” Like, if that’s the real reason, then you can make up ideas about taste or repurpose ideas about You can get ideas as necessary to resent whoever you want to resent.

Andrey Fradkin: Well, to be clear, it’s not like these people were in the same social circles even in college often, right? So it’s an interesting theory that, like, that resentment has caused ex- In college, did they They didn’t hang out with each other, but maybe they still thought they were gonna do equally well. Is that, is that kind of the theory?

Noah Smith: I think so, yeah. from my-- I did actually go to college with some of those people. Like, I was in Gary Tan’s study group. He’s still a friend of mine.

Andrey Fradkin: Nice.

Noah Smith: Although I did quit I quit Gary Tan’s study group because, I thought that studying on my own would make me better. So sorry, Gary. I just-- and I was right. I, I did well on the test, but-

Andrey Fradkin: Well, to be clear, you’re still doing very well, right? I don’t think you’re the resentment class. Yeah, so-

Noah Smith: No, no.

Andrey Fradkin: -

Noah Smith: No, but I’m, I’m-

Seth Benzell: Wait, so to what extent is-

Noah Smith: Succeeded to the extent of Gary Tan.

Seth Benzell: Is it- to what extent is this about just the relative between the two groups versus the absolute? Kind of you started with sort of an absolute story about it’s harder to live a middle-class lifestyle, and now you’ve moved to kind of a relative story about this subgroup did better than that subgroup.

Noah Smith: I wouldn’t say-

Seth Benzell: So are they both important?

Noah Smith: Harder to live a middle-class lifestyle is exactly what I described. I would say it’s instead the expectations of how good your life would get or the, you-- people expected this glide path, and then it flattened out. That’s an absolute story. Whereas the relative-

Seth Benzell: Right

Noah Smith: Story of like: I’m not as, I’m not as do- doing as well as the tech bro class. I don’t think these are independent. I think those are two different stories, but they’re not independent at all. ‘cause if I, if my, if my future path leveled out and flattened out, but other people’s didn’t, and they stayed on the escalator, that escalator I expected for myself evaporated for me and continued for them-

Seth Benzell: They stole my escalator!

Noah Smith: They stole my escalator.

Andrey Fradkin: Yeah.

Noah Smith: Who stole my escalator?

Andrey Fradkin: Yeah.

Noah Smith: Yeah, so. And so like-

Andrey Fradkin: That’s a great meme. [chuckles]

Noah Smith: Yeah. And so like, anyway, so I think that that was like a contributor to unrest, but I don’t think that was the big story. I think the big story was social media, blah, blah. But I throwing everybody in the same room as each other and letting them fight it out, I think that was a bad idea.

Andrey Fradkin: So what about the housing theory-

Seth Benzell: Can we just- can we lower, should we-

[01:20:00]

Andrey Fradkin: What about the housing theory of everything-

Noah Smith: Go ahead

Andrey Fradkin: Right? ‘Cause, ‘cause I do think that s- housing is such a major contributor to this feeling that people aren’t equal.

Seth Benzell: If it was cheaper to-

Andrey Fradkin: Yeah

Seth Benzell: Live in Brooklyn, we would solve all social problems.

Andrey Fradkin: Not wrong.

Noah Smith: The housing theory of everything, it’s like cheap housing would be really good for everybody. I don’t, I don’t have any problem with people believing in it, but it’s not a theory of everything.

Seth Benzell: Directionally correct.

Noah Smith: Directionally correct. Directionally correct. It’s like, do that Winnie-the-Pooh meme where there’s, like, plain Winnie-the-Pooh and then tuxedo Winnie-the-Pooh?

Andrey Fradkin: Yeah.

Seth Benzell: Yeah.

Noah Smith: It’s like the plain Winnie-the-Pooh is, like, exaggerated. Tuxedo Winnie-the-Pooh is directionally correct.

Andrey Fradkin: [laughing] Seth, I think you have one more question.

Seth Benzell: Yes.

Andrey Fradkin: Yeah.

Seth Benzell: Well, I guess, yeah, this is partly tied into that and partly kind of riffing on this question of elite overproduction, which is, it seems like sort of, to the extent that we get this social, unrest from people being upset about not reaching their expectations, to what extent do we have, like, a social To what extent is it, like, an economically central issue to manage people’s expectations, right? To what extent are vibes versus real economic trends important for determining people’s welfare and how they feel about the world? and how does that affect how you think about policy making or writing?

Noah Smith: I think, you really hit on one of the central questions of economics because my advisor, Miles Kimball, spent a lot of his career thinking about this and never came up with really solid answers, I think. Because we have pretty good evidence that happiness, the self-reported emotion, is pretty strongly related to differences between reality and expectations. interestingly, that’s what the original-

Seth Benzell: I’ll say shocks are good

Noah Smith: It just means luck.

Andrey Fradkin: [chuckles]

Noah Smith: But, like, essentially-

Seth Benzell: Yeah

Noah Smith: If you do, if you do better-

Seth Benzell: Luck

Noah Smith: Than you thought you’d do, you’re happy, and if you do worse than you thought you’d do So, like, the best outcome would be if we could give everyone low expectations and high outcomes, if we could make everybody just delighted with how well they did.

Seth Benzell: Right.

Noah Smith: I feel like this experiment has been run, and it’s called Generation X. [chuckles] And, like, I don’t know, man.

Seth Benzell: Didn’t work. Massive failure.

Noah Smith: Like, I see a lot of those people, they’re like billionaires now. They’re like, “I’m such a failure.” Like, you’re a billionaire! “Like, I’m, I’m never gonna amount to anything. I’m just a billionaire living in this giant mansion. Hmm.”

Seth Benzell: Just a b- [chuckles] Jeff Bezos’s boat is so much bigger than mine.

Noah Smith: And, like, this is a direct, I Like, I blame Nirvana. I blame Kurt Cobain for all this,? [chuckles] I blame depress- I blame-

Seth Benzell: No one can understand their lyrics

Noah Smith: I blame depressing-ass Generation X-

Andrey Fradkin: No, no, this is a pro-grunge podcast. No slander allowed.

Noah Smith: I didn’t say I dislike grunge. I love grunge.

Seth Benzell: He blame them.

Noah Smith: And I also think it’s a weapon of mass destruction.

Seth Benzell: He respects their power.

Noah Smith: I respect their power. Like, there are days when I just wanna, like, listen to, like, some old Nirvana B-sides, and I just, like And then I just get so angry and bitter about the world, and I’m like, “Yeah.”

Seth Benzell: Put that in a blog post.

Noah Smith: Generation X, it what? I, I don’t really feel sorry at all for Generation X because I feel like their goals in life were simpler and easier. I meet Generation X guys, and their whole goal in life is, like, have sex.

Seth Benzell: Two ladies at the same time.

Noah Smith: Yeah, like-

Seth Benzell: I saw, I saw Office Space

Noah Smith: Their whole goal, like, Generation X guys, all they have to do is, like, get laid, and then they’re done. They win.

Seth Benzell: [chuckles]

Noah Smith: Victory victory condition, and then, like like, Zoomers don’t even want that.

Seth Benzell: Yeah, Zoomers want followers, dude.

Noah Smith: Zoomers are like-

Seth Benzell: Zoomers want-

Noah Smith: Why would I want to do that when I could looks max? Why would I-

Andrey Fradkin: [chuckles]

Noah Smith: Like, why would I do that when I could, when I could mog the moids in the club? [chuckles] You can There-

Seth Benzell: Right. Which means-

Noah Smith: And then Millennials just want, Millennials just want likes on Instagram, and Zoomers, I don’t even know what they want because-

Seth Benzell: No

Noah Smith: They’re already so-

Andrey Fradkin: I don’t think they know what they want.

Seth Benzell: The Zoomers are the-

Andrey Fradkin: That’s kind of the problem

Seth Benzell: The Zoomers are the ones obsessed with social media. We’re the- the Millennials are the idealists. We actually are saving the world from climate change and solving racial d- conflict. -

Noah Smith: We’re gonna solve racism, man.

Seth Benzell: We’re gonna solve racism and global warming. We did that in 2008, right?

Noah Smith: Yeah, we did. We did.

Andrey Fradkin: That’s true.

Noah Smith: We solved it. [chuckles]

Andrey Fradkin: We elected Barack Obama, and that was the end of history. [chuckles]

Noah Smith: Yeah, that was it. We did it, brother.

Seth Benzell: Yeah, the sea stopped rising. I remember that was in the speech.

Noah Smith: I don’t know. All I can promise the world is that it’s always gonna get weirder and weirder.

Andrey Fradkin: Then-

Noah Smith: But I’m-

Seth Benzell: So we need to make people who desire weirdness. That’s the economic solution.

Noah Smith: Yeah, so I’m So that’s good for me because I always loved to see the weirdest shit possible, right? I would always go to, like, the weirdest underground shows in Japan or, like listen to, like, the weirdest music. I just Like, I’m just, I love seeing that weirdness, and the universe continues to deliver it to me in copious amounts. And so now I’m interested to see what AI does with this planet because, honestly, like, like, humanity was kind of hitting a wall. I don’t know. I wrote this in a recent post, which was reprinted by the Free Press. guardians of our our freedom of information.

[01:25:00]

Andrey Fradkin: Well, I-

Noah Smith: And so, and the free press reprinted it, and they were like-

Andrey Fradkin: Behind a paywall, so it can’t be free. I’m confused by the free press. It’s the, -

Noah Smith: The- yes, conditionally free press. [chuckles]

Andrey Fradkin: Yes.

Noah Smith: The, the marginal cost zero press. But, but in this thing, I was like, look, obviously industrialization took fertility to below replacement levels, and then social media has taken fertility to, like, below, like immediate, to, like, immediate extinction levels, to, like, goodbye humanity. This is the last generation, goodbye, kind of levels, right? Plus, ideas were getting harder to find. like, okay, Bloom is right, and Venuren and Webb and whoel- who else was on that paper? Those guys.

Seth Benzell: There’s one more, but those were the good ones.

Noah Smith: There’s one more! Wait, Bloom, Venuren, Webb, and there’s one other person, and I apologize to whoever else is on that paper for not saying your name. But anyway-

Seth Benzell: They got a zillion citations, dude.

Noah Smith: That paper was right. We were hitting the wall. We were just like, all the smartest people had already been assigned to research-

Andrey Fradkin: Chad Jones. Chad Jones. How could we forget?

Seth Benzell: Chad Jones, Chad Jones.

Noah Smith: Our friend of the show.

Andrey Fradkin: Friend of the show.

Noah Smith: The Chad himself.

Andrey Fradkin: The Chad of growth theory.

Seth Benzell: Yes, exactly.

Noah Smith: The Chad. Dream guest of the show.

Seth Benzell: You can’t say the Jones because there’s so many Joneses. [chuckles]

Noah Smith: Oh, you can’t. Although the Chad could also be Chad Syverson, Chad of productivity measurement.

Andrey Fradkin: Ooh, that’s true.

Noah Smith: They’re both the Chad. All right. But anyway, I guess the point is that, I don’t remember who’s on that paper, but, but ideas were getting hard to find. They were right, blah, blah. We were hiring, like, mid-marginal researchers to just, like, randomly try chemicals in a vat, and like, that was what our research- and like, the best brains were already like working on the whatever, all day long. And like, yes, we were running out of, running out of runway on this technological civilization. Like it was, we were really, like, we were really just gonna like, argue like resist Lib versus MAGA for the rest of our lives and on so-

Seth Benzell: God forbid

Noah Smith: Degenerating, shitty mid social media for the rest of-

Seth Benzell: In that flat-

Noah Smith: Not just our lives, but all of humanity. Like, that was the end.

Seth Benzell: The flat part of the solo growth curve.

Noah Smith: Yes, we hit the-

Seth Benzell: That’s, that’s not where you wanna be.

Noah Smith: We hit the we hit the stagnation point. We, like, you could see the end of humanity coming down, coming down the pike, and now we blew it all up by making a God machine. We were like, “Okay, new thing.” And what? This has happened before because the agricultural age, you could sort of see humanity having hit this limit. We hit the Malthusian ceiling-

Seth Benzell: Yeah

Noah Smith: Again and again. We had the Black Plague. We had overpopulation. We deforested the entire goddamn Middle East.

Seth Benzell: We banged our head against that ceiling three or four times.

Noah Smith: Pardon?

Seth Benzell: We banged our head against the Malthusian ceiling three or four times.

Noah Smith: Three or four times! And then we were like like our whole world was running out of wood. Like, we were just running out of trees to chop down. We were gonna like We had the, like, Columbian Exchange, blah, blah. That was, there was gonna be another collapse, just like there had been for the Mongols. And like, then we were like, “All right, we’re busting out of this shit. Steam power!”

Seth Benzell: Yeah.

Noah Smith: “And like science.” And then, like, we got out of that, and then weird shit happened, and you got Nazis and communists and all kinds of crazy stuff. Not to mention, a lot of really bad sitcoms in the ‘80s. But like, we got all of that stuff, and despite all that, I would say on balance, we busted out, and it was pretty good, and I would rather have lived, like, in the industrial age than in the age before. And so maybe AI will kill us. Industrial Revolution could have killed us if we had just if we had launched all the nukes in like 1983 or whenever, like, we would’ve died-

Andrey Fradkin: Yeah

Noah Smith: And then our civilization would’ve fallen. Maybe AI will be the thing to make our civilization fall, or maybe we’ll be able to solve, use AI to solve the problems that, like, we were degenerating, like the end of science and the, like, end of fertility and like the the absolute shittiness of social media, and maybe AI will just solve all this stuff for us.

Andrey Fradkin: Well-

Seth Benzell: Whether or not it just solves it definitely gives us a fighter’s chance.

Noah Smith: That’s what I mean.

Seth Benzell: I think that’s, -

Noah Smith: We rolled the dice of big stuff big new thing. We just, we like, we rolled the dice again, and I’m, I’m glad we did.

Andrey Fradkin: All right, well-

Noah Smith: And, we all die, but I’m glad we tried.

Andrey Fradkin: AI, the new hope, coming to economies near you. on this note, thank you so much, for being our guest, Noah. this was an amazing conversation.

[01:30:00]

Seth Benzell: Thank you so much.

Noah Smith: Thank you. It’s been a pleasure.

Seth Benzell: Really appreciate your time. And listeners at home, keep your posteriors justified.

The first section of our conversation covers Basil’s early career, including jobs at Uber and AQI, how he got interested in AI as a research topic, and his role in managing the Stripe Economics of AI Fellowship.

We then discuss a paper we’ve already covered on the show: his work on whether the real interest rate can be interpreted as a leading indicator of the probability of TAI (or ‘doom’). Listen to our previous conversation on his paper, and view show notes, including links to that paper and blog post here: If the Robots Are Coming, Why Aren't Interest Rates Higher? Seth was previously convinced by Basil’s arguments, but Andrey was a hold out — we discover Basil’s takes about Andrey’s reservations.

Our third subject is Basil’s new paper with Anton about the relevant elasticities for a singularity in research progress “When Does Automating Research Lead to Explosive Growth?” Basil explains how the key issues are the degree of fishing out and spillovers in/across different industries, as well as the extent to which research can be automated. We also take a step back to ask what theoretical research like this teaches us.

Finally, we cover Basil’s back and forth with friend of the show Phil Trammel’s new blog post with Dwarkesh about Piketty and optimal taxation in the age of TAI, link below, and ask him to explain the meme he posted, summarizing his arguments:

Additional references:

Does carbon taxation yield a double dividend (environmental plus fiscal)?

We hope you enjoy the conversation! Transcript follows:

[00:00] Seth Benzell: Welcome to the Justified Posteriors podcast, the podcast that updates its beliefs about the economics of AI and technology. I’m Seth Benzell, looking forward to the Basil exposition we’ll get today, coming to you from Chapman University in sunny Southern California.

[00:35] Andrey Fradkin: And I’m Andrey Fradkin, looking forward to creating a new accord with Basil, coming to you from San Francisco, California. And today we’re very excited to welcome Basil Halperin to our show. Welcome to the show.

[00:49] Basil Halperin: Thanks Andrey. Thanks Seth. Super excited to be here.

[00:53] Andrey Fradkin: So as background, Basil is an expert on the economics of transformative AI and he’s currently...

[01:00] Seth Benzell: Expert is underselling. He is one of the most interesting thinkers around on... Alright, continue.

[01:07] Andrey Fradkin: Yes, he’s great. And he’s a professor at the University of Virginia. We have an exciting show for you today touching on many topics, but we first wanted to get a start with some of the biographical tidbits. In particular, Basil, how did you get interested in this topic? And it seems like you were a lot earlier than other economists. So I’m curious what drew you in before everyone else to this interesting set of topics?

[01:38] Basil Halperin: I mean, not as early as you two, I don’t think. Uh, I don’t know. I was just a nerd growing up. I read a lot of sci-fi. I read Ray Kurzweil in high school when his The Singularity is Near book came out in the 2000s, just because it was popular. The idea got in my head. I was kind of like, “Well, this is interesting, but eventually...” I was like, “I have a few decades to work on other things before any of this becomes relevant.” And then GPT-3 came out in that long hot summer of 2020. I freaked out a little bit for a week or two. This is crazy. How is this happening so fast? So that sort of woke me up a bit. I started thinking about these issues and gradually more and more have gotten sucked into working on it.

[02:20] Seth Benzell: What were your favorite sci-fi growing up?

[02:23] Basil Halperin: Ender’s Game was always the classic.

[02:26] Andrey Fradkin: Now I saw on your resume that you spent a stint at AQR, which is a large capital management firm. I’m curious, what did you learn working there?

[02:37] Basil Halperin: Yeah. So I didn’t expect to go into finance out of college, but basically the opportunity came along. I found out that this firm seemed pretty interesting. So the background is, this firm was founded by two PhD students of Eugene Fama, the Nobel Laureate in finance. Basically taking his ideas seriously and other ideas from the asset pricing literature seriously and applying them to earn a bunch of money. So I didn’t know anything about finance going into that job. So I learned a whole bunch and some of that has been applied in my research that I think we’ll talk about today.

[03:13] Seth Benzell: Ooh, wait, yeah. Pricing assets in the age of AI. Fascinating.

[03:17] Basil Halperin: Yeah, yeah. Talk about it.

[03:19] Andrey Fradkin: So I do think this is an interesting background because a lot of people in our field don’t have a finance background. That’s not where they’re coming from in terms of thinking about technology. So it maybe gave you this strong, prepared mind to be thinking about the asset pricing implications of transformative AI. Did you get to interact with Cliff Asness or were you too much of a, like, intern, low-level employee?

[03:45] Basil Halperin: No, I was there for a year and a half or two years, but too junior. I think one time I made a bad joke to him in the elevator and he like, pretended to laugh. That was pretty much the highlight.

[03:56] Andrey Fradkin: Well, he also likes to make a lot of bad jokes, so you have that in common. Some of them are good too.

[04:05] Basil Halperin: [Laughs] These bad jokes are funny.

[04:06] Andrey Fradkin: What about at Uber? You also spent some time there working with John List, is that right?

[04:11] Basil Halperin: Yeah, yeah. John taught my first ever Econ class when I was undergrad at Chicago, Intro Micro. And he helped inspire me to become an economist plausibly. And then yeah, I worked for him when he was Chief Economist at Uber. Which, Andrey, as you well know, being an economist in tech is an interesting experience. And Uber in 2017 was a particularly interesting time because it was a controversial firm. Sort of like OpenAI is today, the firm that’s always in the headlines.

[04:42] Andrey Fradkin: Were there specific perspectives that you gained there that have informed your subsequent economics career? Or was it more of just like you learned some useful skills in data science or something else?

[04:55] Basil Halperin: Yeah, I don’t know how much super tangible I have to say, but it definitely was informative in general to work in the private sector before going into academia, just to see how different things are. You know, like in the private sector you’re being paid to tell your boss that he or she is wrong. And then in academia that’s not so much a recommended strategy.

[05:19] Seth Benzell: Wait, wait, okay. So tell us about... so you’re there, it’s in 2017. Uber is one of the most evil, fast-growing companies on the planet. So you said it was interesting. So what was interesting about that? Were you pressured to write an economics report you didn’t agree with? Did you feel like you had to like wear, you know, a hoodie going into the office as people were throwing trash at you? What was it like?

[05:43] Basil Halperin: No, it was just... I mean, I certainly didn’t have a negative experience or negative view of the company, though I’m sure there were negative things the company did, like any large organization. But the team I was on, this Chief Economist team, was like five people. So it was pretty small. So we just had a lot of leverage to go around the company, be sort of an internal consultancy and do a lot of crazy things, varied things that I otherwise never would have had the chance to do. Like I was sort of a software engineer for one month that I was there, which was otherwise something that never would have happened to me. Or running large scale experiments on a million riders or whatever, which... I would love to do macro experiments if any central bank wants to volunteer for some coin flips. But otherwise, as a macroeconomist now, I don’t really have that opportunity.

[06:35] Andrey Fradkin: So this kind of is a, you know, is a nice segue into our next topic, which is... like a lot of people are worried about their careers these days, obviously because of AI.

[06:49] Seth Benzell: Not me! Podcasting is never gonna go out of style, Andrey!

[06:53] Andrey Fradkin: Fair enough. But I think that’s a very broad question and perhaps too broad to answer. But I think for people with an interest in economics—you know, you were in tech, you decided to go into academia. I’ve made the same decision in my life. But I’m curious like what advice would you have? And maybe this is a good opportunity to also speak about the efforts you’ve been doing with the Stripe Economics of AI Fellowship.

[07:23] Basil Halperin: Yeah, okay. So two points here. One point is that I feel like on every good AI podcast, there’s a question of, “What do you tell young people? What they should be studying today?” And like there’s zero good answer to that question. So yeah, I don’t have any good answer to that question.

[07:38] Seth Benzell: Study the Justified Posteriors podcast. Listen to every episode every day. Three times a day.

[07:45] Basil Halperin: But besides that, it’s not clear. The other thing I guess I can say is that if you’re an economist, working on the economics of AI is like a really cool thing to do. There’s just like so much low hanging fruit. There’s so many insights that can be arbitraged from other fields, which is always a good place to be. You can... instead of going to have to pick the fruit yourself, you can just take the fruit out of other people’s hands, maybe translate it to the language of economics.

[08:12] Seth Benzell: Yeah, I understand later we’ll be talking about the economics of fruit picking. But so hold those fruit picking thoughts.

[08:20] Basil Halperin: All of my economic metaphors are about fruit. So we’re going to get pretty fruity or something today. Um, I don’t know, Andrey, maybe you were suggesting that I talk about this fellowship that I help run.

[08:31] Andrey Fradkin: Yeah, tell us about the Stripe Fellowship. What fruit is the Stripe Fellowship?

[08:35] Basil Halperin: Tell us about what you learned running it and what is it, you know, give a brief description. Yeah.

[08:41] Basil Halperin: Yeah, this is this fellowship that we run for early career economists that I do working with Stripe, the financial technology company. Where they decided that they want to support more economics research on the economics of AI, thinking that economists are not working on the issue enough. Which is an empirical claim that you can debate. And so we had the first cohort this past year, 24-25 fellows, mostly grad students, a few APs [Assistant Professors]. And this is a lot of... in part giving people money to do research, but in large part like building a community of people to speak together and share ideas and maybe work together. Folks that probably are listening to your podcasts and that maybe you all should consider interviewing. So that’s been super fun. Very interesting to be on the side of someone reviewing applications as opposed to being on the other side of applying and seeing... I mean, first of all, it’s frankly like... I can’t complain. It’s a very cool opportunity to be running this thing. But it’s terrible to reject people. Like it’s absolutely no fun. All these extremely well-qualified people who are definitely smarter and more accomplished than me. Like that’s not a fun part of it. On the other hand, very cool to get to support all these cool people doing very cool research and seeing them decide to co-author together and things like that.

[10:15] Seth Benzell: Oh, can you point... that’s particularly exciting. Can you point towards any papers that you think you may have generated that we should maybe discuss on our podcast?

[10:25] Basil Halperin: So two... so it’s been like six months or something since the fellowship launched and you guys know how long these timelines are. So no counterfactual papers yet.

[10:35] Seth Benzell: Oh, well I know how short my AGI timelines are.

[10:38] Basil Halperin: Well, you’ll have to tell us that later. No counterfactual papers yet, but a bunch of people have amazing stuff out. Phil Chen at Harvard just put out a very cool paper using GitHub data to look at how software engineer labor has changed. Parker Whitfill’s been putting out like a paper every few months on compute and labor, complements versus substitutes, with Cheryl Wu. And yeah, there’s a whole bunch of stuff. We have this website, you can Google “Stripe Econ Fellowship of AI” and see folks’ websites. There’s a ton of very cool stuff. I don’t have time even to read all the papers, at least yet.

[11:18] Andrey Fradkin: Well, that’s yeah, super awesome initiative. I guess, you know, one follow-up question on there. What do you think most of these people are going to be doing three, five years from now? Do you think they’re going to become assistant professors? Are they going to work at AI labs? Are going to do something else? Like what is the career trajectory for a young person?

[11:39] Seth Benzell: Are they going to be podcasters?

[11:41] Andrey Fradkin: Yeah, are they going to be podcasters? Like... and maybe, what do they think they’re going to be doing is an interesting question, right? Because it’s a time of great uncertainty.

[11:51] Basil Halperin: Yeah, I don’t know. So like... one way of answering that is that I think kind of any question about speculating about the future comes down to: how fast do you think AI capabilities are going to progress? AI technology going to develop? As has come up a whole bunch of times in this conversation. And there’s various ways people try to forecast how quickly the technology will develop. Like one way is just go and survey machine learning engineers and trust that they know something about how the future is going to go and take an average of their opinions. So that’s one method. Another method is something that’s gone back to like Hans Moravec at the very least of: think that computers are like human brains and try and estimate how much computing power the human brain does and try and forecast Moore’s Law and algorithmic progress to see...

[12:31] Seth Benzell: Ray Kurzweilian, yeah.

[12:33] Basil Halperin: Exactly, like Ray Kurzweil. To see how long until we have enough computing power to match the human brain and say that’s when we’ll develop AGI. But like, sort of setting that to the side or something... I don’t know. We’re trying to encourage research. So we’re selecting for people who are like stubbornly pursuing research. So there’s that. But if you’re like asking about the future for econ PhDs... econ grad students...

[12:58] Seth Benzell: We’re not talking about the future of econ PhDs generally. We’re talking about this elite cohort you’ve gathered. You think that there’s a chance that this elite cohort of the best young thinkers on Econ of AI are going to be obsoleted in three years?

[13:13] Basil Halperin: Uh, I mean, I think there’s a non-zero chance that we’re all living in some communist utopia in a few years. Not a high one, as my research would indicate, but non-zero. Which is like crazy to think about. We could get unhinged and talk about that, but maybe we can save it for later.

[13:30] Andrey Fradkin: Yeah, I guess I was trying to actually push you in a different direction, which is more like... you know, Tyler Cowen famously gave Leopold Aschenbrenner the advice of not going into economics academia, right? You know, he was someone who was, and still is I think, working on some economics research.

[13:46] Seth Benzell: Yes, including with friend of the show Phil.

[13:49] Andrey Fradkin: Yeah. Exactly. So I was kind of more thinking like, is it really the best place if you’re really AI-pilled to be sitting at a university? Why did you choose to do that? I’m sure you had... you could have had other options that you pursued.

[14:04] Basil Halperin: Yeah. I mean, so what is best for any individual varies a lot. And I don’t know, like don’t you guys think that people who go into academia are kind of stubborn? Like they want the independence of not having a boss. They’re willing to accept the ginormous pay cuts relative to the outside option.

[14:24] Seth Benzell: I wanted the wizard robes.

[14:26] Basil Halperin: You wear wizard robes to lecture or what?

[14:29] Seth Benzell: I do. I have it hanging on my wall right now. I would point my camera, but my lighting is so beautiful right now.

[14:34] Basil Halperin: We should have worn them for the video. So I don’t know, like really that idiosyncratic taste shock is I think driving a lot of people. But yeah, I totally agree that there’s a lot of amazing research to be done in the private sector and like the new Anthropic economic team seems to be doing amazing stuff, for example.

[14:52] Seth Benzell: Basil, I don’t want to answer this question for you, but if I may offer kind of a riff on that idea of it being idiosyncratic taste... I think it’s a, you could call this a taste thing, but you might call it also an idiosyncratic valuation of certain virtues, right? You might find yourself associating with the virtues of being an economist or being a professor and having open inquiry, etc., etc., etc., that are not necessarily as associated as firmly with other professions. You could call that taste or you could call that something else.

[15:28] Basil Halperin: Yeah, let’s bring virtue ethics back into economics.

[15:32] Seth Benzell: Bringing the virtue ethics back to economics, exactly.

[15:35] Andrey Fradkin: Yeah. Well, cool. You know, very interesting to think about these career implications, but I think it’s maybe a natural place to transition to discussing some of your really interesting thoughts that you’ve had recently. And I think Seth has some questions.

Basil Justifies His Research:
Transformative AI, existential risk, and real interest rates

[15:53] Seth Benzell: [Grabbing microphone] Give me the mic, Andrey. I’m grabbing the mic from Andrey now. Basil, if I recall correctly, the way we e-met was because I got very frustrated with you over one of your papers. And this was your paper, “Transformative AI, Existential Risk, and Real Interest Rates.” So I guess before kind of I explain my strong emotional reaction to this paper and how you eventually won me over, maybe you can refresh our podcast listeners. We did an episode on this podcast as one of our very first episodes. I encourage our listeners to go back and listen to it. But for those who don’t have the time, can you give us maybe a two-minute gloss on that paper before we start putting you to the test on it?

[16:45] Basil Halperin: Yes. So I second that listeners should go back and relisten to that old episode because I did before this and that was a really nice summary that I really appreciated. Obviously the critiques were wrong, which we’ll get to. That’s a joke. There were some good points. But yeah, so the motivation here is like, everyone wants to know how quickly is AI going to progress? AI technology going to develop? And there’s various ways people try to forecast how quickly the technology will develop. Like one way is just go and survey machine learning engineers and trust that they know something about how the future is going to go and take an average of their opinions. So that’s one method. Another method is something that’s gone back to like Hans Moravec at the very least of: think that computers are like human brains and try and estimate how much computing power the human brain does and try and forecast Moore’s Law and algorithmic progress to see...

[17:33] Seth Benzell: Ray Kurzweilian, yeah.

[17:35] Basil Halperin: Exactly, like Ray Kurzweil. To see how long until we have enough computing power to match the human brain and say that’s when we’ll develop AGI. We in this paper want to present sort of an indirect way of thinking about this, which is using one of the most powerful supercomputers humanity has, and that is the calculation power of financial markets. Where in economics, you know, we like to think that prices are good at aggregating dispersed wisdom across the economy. And financial market prices in particular, by being forward looking, by being particularly liquid and having this strong incentivizing power through the magic of no arbitrage—or arbitrage incentives—are a particularly good way of collecting humanity’s dispersed wisdom about how the future could proceed. So in particular, we suggest in this paper that...

[18:31] Seth Benzell: But Basil, there’s no... at least when you were writing this paper, I’m not aware of a high liquidity market that just says “when does AGI happen?” or “when does TAI happen?” So what price should we look at?

[18:43] Basil Halperin: Indeed. And if you’ll allow me to rant on that for a second before summarizing the argument... like today, even today, there’s still no, despite the rise of prediction markets, there is no long horizon prediction market on when could advanced AI be developed. There’s these forecasting platforms that just allow people to submit their own forecasts and take the average of them. Metaculus, Manifold Markets. People sometimes refer to these as betting markets, prediction markets... they are not prediction markets. They do not have the incentive, the financial incentive to ensure forecasters pay attention, update their forecasts, and so on. So those are great websites, but they’re limited. Kalshi, Polymarket, these new prediction markets... somehow there’s just... it’s shocking how bad the lack of good forecasting opportunity to forecast AI is. There’s very limited things. There are some things, but they’re not very good.

[19:35] Seth Benzell: Do you speculate that it’s like a defining AGI problem? It’s the Oracle problem? It’s like, “how would you know it when you see it?” Or did you speculate on why that is?

[19:43] Basil Halperin: Yeah. So part of it is that. So for example, the very best question that I’m aware of is Kalshi has a market on: will this fancy version of the Turing test be passed by 2030? Where it’s some like souped up version of the Turing test based on a bet that Ray Kurzweil actually—we keep mentioning his name—made. So that’s like the best existing thing...

[20:00] Basil Halperin: ...but it’s this limited definition.

[20:04] Andrey Fradkin: So I actually have a different question which is related to your paper. But let’s say we had a prediction market on GDP growth. And you know, it was like: will we have, I don’t know, 5% GDP growth or 10% GDP growth at least once by year X? You know, it’s hard to imagine that that would happen without transformative AI.

[20:31] Seth Benzell: Ah, Andrey, I could tell a story.

[20:33] Andrey Fradkin: Yeah. No, I could tell a story. I could tell a story, but it would be highly correlated. Are there markets like that that are very close analogs to this?

[20:42] Basil Halperin: If there are, I would love to know. And like, I do a periodic search and there’s... it’s like there’s really not. It’s infuriating. Hence the origin of this paper.

[20:51] Seth Benzell: But you can bet... you can bet super out of the money calls on like the stock market. You can bet on the stock market growing 500%, right?

[20:59] Basil Halperin: Yes. Well, I don’t know about 500%. Out of the money calls, like the range is not that large. But betting on GDP growth in particular is difficult. And like, does higher GDP growth raise equity valuations? It’s actually not obvious. Like, we can really dive into that, but for a whole bunch of reasons... for a whole bunch of reasons I think equities are just kind of a very confusing asset class in general to interpret. Which is why...

[21:27] Andrey Fradkin: Yes, so tell us why you picked interest rates. Yeah, and then we’ll go back to why equities may or may not be good.

[21:33] Seth Benzell: Because equities are a bad asset, what I’ll do is measure equities over time. [Laughter]

[21:40] Basil Halperin: Yeah, so the best price in the economy—that’s kind of a joke—the price we recommend looking at in this paper is real interest rates. So that is to say the inflation-adjusted risk-free rate of return you would earn on a bond, particularly at long horizons. Like say the 10-year real interest rate or the 30-year real interest rate. And the argument for why that’s a useful price to look at is the following: If you knew you were going to be super rich next year, no reason to save today. You’re going to be super rich next year anyway. If no one’s saving, then that pushes up interest rates. Interest rates clear the market, the supply and demand for savings.

So that would be the case where we expect AI to rapidly raise economic growth, rapidly raise our incomes, in particular rapidly raise our consumption. And so if we saw really high real interest rates, that would be indicative of this case of aligned AI raising human incomes. Alternatively, another case with AI that people talk about is that, you know, AI is going to wipe us all out. And you’ve done podcasts on this topic. Similarly, if we’re all going to be dead next year because AI was going to wipe us all out, then there’d be no reason to save today. You’re going to be dead next year. No reason to hold on to assets for next year. Likewise, that pushes up interest rates.

So, you know, we could go and look at interest rates. Are they much higher than they have been? And like, no, they’re well within the range of normal variation. And when I started thinking about this back in fall of 2021, it was particularly salient because at that time long-term real interest rates in the US, and indeed around the world, were at all-time lows, like negative. So you know, you’d give $100 to the US government, they give you back $99 inflation adjusted at the end of the year. Interest rates have gone up a non-trivial amount since then actually, but really not that much. Really, it’s probably not because of AI. Maybe a bit. So that’s the core argument. That if markets were expecting aligned or unaligned transformative AI, then we’d see high real interest rates today.

[23:51] Seth Benzell: All right, great arguments. And now I’m going to explain why this was so frustrating for me in 2021 to read this argument. I had been working on transformative AI topics and had been thinking about, you know, kinds of economic downsides of AI. And one of the mechanisms that I had become worried about was the anticipation of AI leads to dissaving and that dissaving is large enough that interest rates skyrocket and actually you don’t get enough reinvestment in the economy to have significant economic growth, right? Set aside for a second whether or not the dissaving you have in mind is so extreme that you would literally like cancel out the gains from AI. But I had been kind of pushing on this idea that, you know, AI is going to lead to dissaving... as the world’s interest rates were plummeting. And so I had kind of pivoted into trying to think about, okay, well, if we do get really good AI, how could you get to a world where there are very low interest rates, right? And so one version of this idea I worked on with our friend and co-author Erik Brynjolfsson is the idea that, well, maybe there will be a kind of labor that will be infinitely reproduced, but there will be still some scarce human factor. And then actually that scarce human factor will make all of the gains and then interest rates can remain low.

Another story would be: well maybe we don’t have transformative AI, we have an AI that takes over, you know, 50, 60, 70% of jobs. We see the labor share of national income go down from, you know, 60% to 20%. But if you actually play that out in a big macroeconomic model where you try to realistically model national savings rates... well, you’re kind of pushing against the tide. Like we talked about, in 2021 we had this huge—it was called by some an international saving glut—that was maybe driven by the rise of an Asian middle class that all of a sudden had all of this money, needed to save for retirement. There was a scarcity of safe assets. And so even if you automated a lot of jobs, there might be still a lot of absorptive capacity for that savings before you would significantly bid up interest rates.

And so kind of for both this sort of a theoretical reason and a sort of a kind of a macro simulation reason, I fired off to you this angry email saying, “Don’t you realize blah, blah, blah, blah, blah?”

[26:28] Basil Halperin: Yeah, the audience wants your original comment. They want you to read it.

[26:32] Andrey Fradkin: Oh, that email will be in the post, don’t worry.

[26:36] Basil Halperin: I have it on hand. I have it on hand.

[26:38] Seth Benzell: Oh wait, let’s hear it. Let’s hear it, Basil. How bad was it?

[26:41] Basil Halperin: This is going to be the unhinged portion of the episode. So Tyler Cowen kindly reposted the essay.

[26:49] Seth Benzell: [Laughs] It was like, “A crazy guy emailed me.”

[26:51] Basil Halperin: Well, so initially it was an email. Initially it was a comment on the Marginal Revolution post sharing the essay. And so, like, you know, I...

[26:59] Seth Benzell: And everyone knows that that is where the sanest people hang out.

[27:03] Basil Halperin: I, like some neurotic person or whatever, skim through these comments and there’s this one guy Seth Benzell: “Hey, I’ve read a few of his papers, including that one you mentioned with Eric. This is so dumb.” That’s my first introduction to Seth. Of course, since then things have changed. But welcome to the internet.

[27:26] Seth Benzell: Wow, “so dumb.” I came out of the gate swinging. You have to remember it was the pandemic. We were all cooped up. Some people went to BLM protests. I commented on Marginal Rev. But now I’ll tell you how you won me over, Basil. Which is, you sat me down and you said, “Seth, those scenarios that you’re thinking about, the one where there’s still, you know, a scarce human factor that’s making the wins, or the one where we automate 60% of jobs, those are ‘AI is a big deal’ scenarios, but those aren’t the transformative AI, AGI scenarios that I’m actually writing about.” And then I apologize for not having read the paper.

[28:06] Andrey Fradkin: You’re a true Marginal Revolution commenter, Seth. Who I don’t think any of them have ever read a paper.

[28:15] Basil Halperin: This is worth noting. So like, the paper and the argument really is zoomed in onto this particular scenario, which I think was like much more top of mind to the people thinking about this a few years ago. So like, you know, before ChatGPT... our essay, initial essay was posted a month after ChatGPT came out. Before ChatGPT, there weren’t that many people in the world thinking about AI, right? And the people that were, a lot of them were focused on like these fast takeoff “foom” scenarios. Things would happen fast, things would happen big. More likely than not, we’re going to die. P(doom) is high as they say, right? So we were really focused on like these kind of extreme possibilities: either we’re all going to die or we’re going to have what we operationalized as 30% annual GDP growth. An order of magnitude increase in annual GDP growth. Which would be crazy. It would be as if the whole economy is growing as fast as Moore’s Law, more or less. So yes, it’s an extreme scenario for sure.

[29:13] Seth Benzell: And but yes, but so given that extreme scenario, you won me over. And I said, “Andrey, when we start our podcast, I want to talk about this paper because nothing has moved my priors so much as this paper.” Maybe it was just moving my definitions around. Maybe it gave me like a stronger understanding of what people really mean by transformative AI versus just AI that is so good that it automates 70% of jobs. But I talked to Andrey about it and Andrey, remind me, were... did I fully convince you of Basil’s arguments or remind me?

[29:47] Andrey Fradkin: No, I don’t think so. Andrey wasn’t convinced at all. I just... I mean... I just feel like the people being so certain that this transformative AI is coming in this particular way seems unlikely to me. It’s not like how humans tend to think or behave about most things in life. And then it’s hard for me to imagine a world where they essentially like, it’s a coin flip: either we all die or we have amazing transformative AI. And we don’t have any intermediate types of outcomes where, for example, you might want to engage in precautionary saving. I know you talk about certain precautionary savings in your paper, but like, that’s just a very natural response to a lot of uncertainty. There are of course also scenarios where there is tremendous economic growth, but it’s held by very few people. It’s ex-ante not obvious who those people are going to be. Or maybe it is obvious, I don’t know. Maybe they already have all the capital, right? There are just a lot of things, a lot of details to think through and I’m sure you’ve thought through a lot more of those than we have in our podcast.

[31:04] Basil Halperin: Yeah. So one thing I should say is that like this transformative AI 30% GDP growth scenario, that’s not something we made up or pulled out of thin air. Like this really was and is a paper dedicated to a specific conversation, just like any academic paper, right? It’s a conversation among a particular group. So that’s one thing. Another thing to say is like, to me... so one thing Andrey that you spoke about in the last podcast on this that I totally agree with is skepticism of quantitative macro predictions. So I think you went beyond what I would say in terms of skepticism, but I so strongly share the belief or the view that macro does not have an amazing track record in terms of precise predictions. And that’s why... like that’s like a strong motivation for the approach in this paper. Where instead of like, we’re going to write down an optimizing model, a model of optimizing agents where in equilibrium we determine the structural forces determining the real interest rate and we’re going to calibrate all these different forces and feed in the simulation. Instead, it’s just this like dead simple thing where we have this very robust, strong prediction from any intertemporal macroeconomic model: that higher growth or higher mortality risk raise real interest rates. And people are predicting, people are moving tens, hundreds of billions of dollars, literally in San Francisco, under the belief that these things are going to happen. One of these two things is going to happen. It’s going to happen in the next 10, 5, 1 year. And this provides some sanity check on like, most of all, like the very shortest timeline predictions.

[32:51] Seth Benzell: Yeah, so maybe I can pay...

[32:52] Andrey Fradkin: But I guess does everyone need to believe in those predictions? I mean...

[32:56] Seth Benzell: It has to be like the median investor, right? Who has... who’s the guy that we’re talking about the beliefs of?

[33:01] Basil Halperin: The marginal unit of capital. So, you know, markets don’t reflect average beliefs. They reflect the belief of the marginal unit of capital, the marginal trader, just like any price reflects the marginal buyer/seller. And like a priori and lots of theory and so forth to back this up, like you would think that the marginal trader is the one who has the most knowledge or the most incentive to buy/sell. You can think about deviations from that, but like that’s...

[33:26] Seth Benzell: Isn’t the marginal trader a noise trader?

[33:28] Andrey Fradkin: Or like if we have a distribution of beliefs, isn’t the marginal trader someone who has an intermediate belief?

[33:35] Basil Halperin: Um, so one thing I will say is that... one thing I’ve learned from this whole project is it’s confusing to me how underdeveloped the literature on asset pricing under heterogeneous beliefs is. I think it’s in part because like you get these no trade results where if people don’t... anyway, the theory is hard. But the way I think about it is that the sort of robust prediction of theory is that asset prices are like a wealth-weighted average of beliefs. Maybe wealth-weighted risk tolerance weighted average of the distribution of beliefs.

[34:13] Seth Benzell: That right? You think if I’m super out of the money, can I still move the middle somehow? In other words, if I’m the guy... if I’m a 99% “AI never happens” or “AI always happens,” in what sense am I being included in that weighted average?

[34:27] Basil Halperin: Just directly. So like this is about consumption-savings decisions rather. Like what, how fast will the growth rate be? That average.

[34:39] Seth Benzell: Okay. Oh, you’re talking more about the national saving rate. That part of it.

[34:43] Basil Halperin: I’m thinking like the g, the growth rate that goes into the real interest rate determination, that’s the average belief over that.

[34:54] Seth Benzell: Right. And the reason that that matters is that is going to drive the saving rate, which drives the interest rate? Or through a different mechanism?

[35:01] Basil Halperin: Yes, yes, yes.

[35:02] Seth Benzell: Okay.

[35:04] Andrey Fradkin: I have a... so I have a question related to, you know, we touched upon this when we did the podcast, but I’m curious what you think about it is: It seems hard for me to imagine a scenario where we get to your scenario without a lot of hints in advance, right? Like... like your scenario is literally like most people agree that we’re going to have 30% growth next year. What... what does the path to that look like? Does that mean that we first have 20% growth, 10% growth? Uh, like... are there other assets that we expect to be leading indicators there? Because I do think in some sense, if we get to your scenario, then you’ve already told us what happens.

[35:48] Basil Halperin: It’s not my scenario. I want to emphasize.

[35:51] Andrey Fradkin: No no, sorry. To your analysis. If we get to the point in your analysis—I know it’s not your scenario—then...

[35:56] Seth Benzell: Is your warning light a leading indicator or a late indicator?

[36:01] Andrey Fradkin: Yeah. We thought it was a late indicator. But I’m curious if you have ideas for leading indicators. Yeah.

[36:07] Basil Halperin: Ah, so I really think this is a leading indicator because like interest rates reflect expectations about future growth, not current growth. So like wages would be a lagging indicator where those are only going to fall once the technology has developed. Interest rates will rise once people expect the technology to be developed.

[36:25] Andrey Fradkin: So no, so I think we both agree with that. I’m just saying that like it’s hard for me to imagine that enough percent of capital believes that we’re going to have 30% growth without it being apparent in other economic statistics long in advance of that.

[36:38] Seth Benzell: Like will we be... I guess... the people who read your paper will be convinced that AGI is coming before interest rates go up.

[36:48] Basil Halperin: So that’s sort of a question of like how efficient do you think markets are plausibly, right? Is that what you’re saying?

[36:58] Seth Benzell: I think that’s fair, right? Andrey is saying that the sophisticated... I mean that’s how I read it.

[37:01] Andrey Fradkin: Well, one is efficiency. The other is like... let’s say for... if we thought that for AGI to happen, we needed to have substantial data center and energy build outs...

[37:13] Seth Benzell: Elon’s robot factory.

[37:15] Andrey Fradkin: Yeah, but to the extent of like 5% of GDP, 10% of GDP, right? Like these things will be happening. There... you know, there’ll still be uncertainty. So it’s not necessarily that it’s an efficient markets failure, but um... like what are the... you know, those are kind of the things that I’m curious about if you have any thoughts. Like what are the precursors to this moment?

[37:41] Basil Halperin: So I mean, I still think interest rates can go up before... like capital takes time to build. But if the discussion is like what things will happen on the way to transformative AI, like yeah, the... what’s the line from the bard of our times, our dear leader: “everything is compute”? Like we’re going to tile the planet with computers. So like 1% of US GDP last year was hyperscaler capital expenditure.

[38:15] Seth Benzell: And let me... yeah. Let me try to ask this a slightly different way, which is, I guess maybe try to make you be a little bit quantitative about how sensitive your personal predictions about TAI are based on different interest rate scenarios. So I’m going to give you a conditional expectation here. Feel free to use it or to give me a different one, but I want you to try to be quantitative if you can. What is your conditional probability of TAI within five years if the interest rate is less than 6% versus TAI in less than five years if the interest rate is above 15%? Real interest rates.

[38:53] Basil Halperin: If the real interest rate is above 15%, then like if this is the real risk-free interest rate, then I think TAI is here and growth is going bananas. I think plausibly even if real interest rates are above 6%... so like the 30-year right now is like 2.6. The 10-year is like 1.8. And so like the 2.6...

[39:13] Andrey Fradkin: Just to be clear to the listeners, once again, we’re talking about inflation-adjusted interest rates.

[39:16] Basil Halperin: That’s important. So the 1.8% number for the 10-year real interest rate is like really in line with where things have been over the last 25 years. The 2.6 for the 30-year is like a little bit elevated. So even 6...

[39:31] Seth Benzell: The numbers I were using were kind of risky equity market rates. So feel free to substitute whatever numbers you like.

[39:35] Andrey Fradkin: Well that’s just a totally different object, right?

[39:39] Basil Halperin: So...

[39:40] Seth Benzell: Oh god. Right. Alright. So okay, risk-free rate. So right now you’re telling me we’re at what? 3%?

[39:44] Basil Halperin: 2.6 for the 30-year.

[39:46] Seth Benzell: 2.6. All right. So what’s your conditional expectation on TAI in five years in the future given that next year the risk-free rate is under 3%? And then what is it if the risk-free rate goes above 10%?

[40:02] Basil Halperin: Again, if it goes above 10%, I think growth is going bananas. That’s a huge jump.

[40:07] Seth Benzell: Anticipated growth. So you don’t even think... you think we’d see the growth before we’d see the interest rate?

[40:12] Basil Halperin: Sorry, it depends on what horizon interest rate we’re talking about here.

[40:15] Seth Benzell: 30-year.

[40:17] Basil Halperin: If the 30-year goes up to 15? Or above 10?

[40:20] Seth Benzell: 10 or 15. You choose numbers. I want you to try to be quantitative at me.

[40:24] Basil Halperin: Well, so here’s the thing, here’s the thing. The interest rate at a particular horizon tells you among other things about growth expectations at that horizon. So you can look at the entire yield curve, interest rate at 1 year, 5 year, 10 year, 30 year, and get the expectations sort of with lots of other things going on at those different horizons. So like I wouldn’t want to just look at just the 30 year. I’d want to look at the 1, 10, 5, 30.

[40:48] Seth Benzell: All right. So choose whatever... the curve is the same. Move the level up or not down.

[40:53] Basil Halperin: I guess if it does it for you.

[40:57] Seth Benzell: Gimme. Feed me.

[41:01] Basil Halperin: Real interest rates rose two percentage points from the... two or three percentage points from the COVID depths to where they are now. And again, now they’re like sort of more or less in where they were 20 years ago. If they went up another percentage point, I’d be... pretty surprised and interested. How much does that raise like my probability of transformative AI in the next five years if the...

[41:26] Seth Benzell: That’s the question. That’s the question. This is what your paper is about.

[41:31] Basil Halperin: But again, like I’m not here to make quantitative forecasts, especially going from market prices back to probabilities. I’m here to say that there’s this...

[41:43] Seth Benzell: I know, you’re making a directional argument, but give me... does it double your odds of TAI? Or I can let this go if you’re going to really refuse.

[41:50] Basil Halperin: I mean, so what I can do... I can tell you what my AI timelines are and like what feeds into that and how...

[41:55] Seth Benzell: Yes.

[41:56] Andrey Fradkin: Let’s just do that. Yeah.

[41:58] Seth Benzell: And then tell us how they would change if interest rates got up.

[42:02] Basil Halperin: Okay, well, like... again, like I really emphasize that to me the right way to read this paper is this interest rate argument is like an outside view, here’s a sanity check. So like my view is much more informed by like all these other things now that I’ve spent like a whole bunch of years reading the AI literature, the AI economics literature. So for example... if you just extrapolate forward the “meter time horizon” trend that you guys have spoken about...

[42:30] Andrey Fradkin: What’s the... what’s the...

[42:32] Basil Halperin: ...the length of a task that... of a software engineering task, a machine learning research task that these large language models can do with 50% accuracy. If you extrapolate that trend forward... this is currently doubling every seven months or that’s what it’s been for the last six years. If you extrapolate that forward, take into account very importantly the fact that by like 2030... capital expenditures by hyperscalers can be like a trillion dollars and that scaling can’t continue. So like take into account the fact we’re going to hit the compute wall and then investment’s going to slow down. We’ll have models that can do one month tasks with 50% accuracy by I think it’s 2033. And one year tasks by 2039. This is Whitfill, Snowden, Parker’s new paper. So that’s on this narrow range of tasks done in these meter benchmarks at 50% accuracy: 2039, one year horizon. If you then adjust for the fact that like these are particular kinds of tasks... like I don’t know, say that adds another six years, so that’s like another six doublings or something like that. And then take into account that rather than 50% accuracy, we want 99% accuracy. That takes you like to late 2040s. I think... just like this particular stylized fact about time horizons already gets you to like fairly long potentially... at least the possibility of potentially long time horizons for AI. So that’s like...

[44:12] Seth Benzell: I guess we’ll come back to this... and maybe we’ll talk about this a little bit more with your new paper where we talk about to the extent that algorithmic progress can substitute for compute progress, right? Because that’s going to be a key factor here.

[44:22] Andrey Fradkin: But to be clear, let’s dwell on this a tiny bit more.

[44:26] Basil Halperin: Yeah, there was a lot of sub-points in there that I went through very fast.

[44:29] Andrey Fradkin: Yeah, but yeah, so I think... I think one thing, you know just Seth to your point very briefly is like the METR graph takes into account algorithmic progress. So that’s why it goes as fast as it does.

[44:43] Seth Benzell: Right. But then he said he was also going to take into account... okay, anyway.

[44:47] Basil Halperin: So that’s like I think one... that’s like a median view. But I think like you really have to think of terms of different scenarios. So like the “AI 2027” guys... like that report seems a little crazy, this idea that things are not just going to grow at a constant rate but are going to go hyperbolic. Like that seems a little crazy and maybe even... yeah, a little crazy. But like there is enough flesh on that argument, including this new paper Seth that you mentioned, could point towards that, that I think like you have to have some non-zero probability on like... maybe not literally AI 2027 but like AI before 2030.

[45:27] Seth Benzell: Do you have to put non-zero probability on anything that isn’t conceptually impossible?

[45:31] Basil Halperin: Yes, okay. I mean like non-1% probability. So like I put like 10 or 15% probability on like things getting really crazy before 2030. And then I put like 50 to 80% probability on something between 2035 and 2050. And then like whatever is left, 10, 20% on like some factor X... Moore’s Law slows down, energy runs out and like things take longer than 2060 or whatever. Or including never being able to develop such technology.

[46:00] Seth Benzell: So did I get you right? So the median forecast is the mid 2040s for AGI? Is that what you’ve given me?

[46:05] Basil Halperin: The quantitative numbers here are really hard, but yes, something like 2035 to 2050.

[46:10] Andrey Fradkin: It’s not AGI to Seth. I mean... I mean it’s very different concept...

[46:17] Seth Benzell: TAI, TAI. TAI is what we want to talk about. Okay. TAI, excuse me.

[46:21] Andrey Fradkin: But Basil, I’m going to give you a counterpoint. I think the METR graph drastically understates the time horizon of tasks that can be done.

[46:30] Basil Halperin: Understates?

[46:31] Andrey Fradkin: Yes.

[46:33] Seth Benzell: Because Ralph OODA Loop.

[46:36] Andrey Fradkin: I mean, yeah, but broadly, right? Like a lot of these evals are doing dumb things. They’re taking a model out of the box and just asking it to do it. And that is not how you would do any task if you had to do it, right? Like you... you know, a big theme of I think our show and worldview is we believe in a multitude of models interacting in an ecosystem to produce outcomes. And the scaffolding really matters.

[47:08] Seth Benzell: How we were epi-ing the Lessin-Kuld show.

[47:10] Andrey Fradkin: Uh, the scaffolding matters, right? The... you can have different models from different providers interacting with each other and calling other tools. And so to evaluate the ability of just like an out of the box LLM to do a specific task... that’s never how you would actually do it in real life.

[47:31] Seth Benzell: Yeah, we see this in Andrey’s data where there are, you know, very clear people use a mix of models. It’s there in the data.

[47:39] Basil Halperin: Yeah, I mean I think unhobbling is like one possible reason that like there’s 15% chance that we’re colonizing the stars before 2030. That unhobbling could be enough. Leopold had it right. Maybe.

[47:53] Andrey Fradkin: Yeah, yeah. I mean, for what it’s worth, I think the bigger, you know... I think the thing I agree with you more is that some of these METR tasks are really unrepresentative of most tasks in the economy. And in particular I don’t think they teach us much about robotics. And I think like robotics has to be an ingredient of any TAI scenario eventually. And so...

[48:18] Seth Benzell: Only a computer scientist would think that computer science is the final task.

[48:23] Basil Halperin: The strawman obviously being that, you know, a brain in a vat—the brain of the computer—can solve robotics just by doing better software on the computer. That’s the strawman.

[48:32] Andrey Fradkin: Yeah, no, no. I understand, but we’re still talking about human tasks being done, you know.

[48:38] Basil Halperin: Totally, totally.

[48:40] Seth Benzell: A brain in the vat still needs faith in God in order to believe in the exterior world, dude. Haven’t you read your dualism?

[48:49] Andrey Fradkin: Um, all right, so...

[48:51] Seth Benzell: Wait, let me wrap up... I want to finish up this topic. Last question on this topic and then we can move on. Which is: okay, you’ve shot me down on asking a quantitative question about the macro. Will you give me an answer about: are you changing your environment... your portfolio? I mean, you said 10% chance of shit gets crazy. Sorry, that’s my one curse per episode. 10% chance. How do you allocate your assets based on that? Are you dissaving?

[49:19] Basil Halperin: So like the first thing I’d say is, for someone at my stage of the life cycle, like my most important asset is my human capital. And I’ve reallocated that heavily from studying monetary policy, which was the thing I was obsessed with for years and years, to now being focused a lot on the economics of AI. So like that asset of my portfolio I’ve shifted a lot. Have I changed what my savings are...

[49:44] Seth Benzell: Are you dissaving your social capital through drugs and alcohol?

[49:49] Basil Halperin: Well, there’s a different consideration there where like I want to stay healthy until the singularity so I can live forever. So I think actually the consideration might go the other way in terms of intertemporal substitution. But, do I try hard to consumption smooth? Absolutely. It would bother me when people in grad school were like, “Yeah, I’m putting money into my 401k.” I’m like...

[50:08] Seth Benzell: Are you putting money into your 401k?

[50:11] Basil Halperin: I put the minimum amount to get the matching funds.

[50:14] Seth Benzell: The minimum, dude. The minimum. I thought this was a guy who believed in his own papers.

[50:17] Basil Halperin: There’s no other reason to do it.

[50:21] Seth Benzell: All right, you have him, Andrey.

[50:23] Andrey Fradkin: All right, all right. I think Seth has given up on life at this point. So cool. Let’s talk a little bit about your new paper with Tom Davidson, Thomas Holden, and Anton Korinek. Why don’t you tell us a little bit about the premise?

Basil Justifies His Research:
When Does Automating AI Research Produce Explosive Growth?

[50:44] Basil Halperin: Yeah. So this is a paper that in some ways is about that 15% probability that things could get crazy soon. And in some ways is about some like deep or some standard economic growth theory. So the idea here is to like take seriously the structure of modern machine learning and put that, embed that into the canonical model of economic growth. Where, by that I mean like: how does AI get trained? How does it develop? Well there’s two key ingredients: software progress, hardware progress. So Moore’s Law and other trends mean that we’re able to produce more chips, better chips at lower prices over time. And algorithmic progress means that even for a fixed quantity of computer hardware, you can get more output from a computer program because we are able to write better computer programs. We are able to train better AI models.

So taking into account the fact, maybe most concretely, that OpenAI uses Nvidia chips to train better AI. And then Nvidia increasingly uses AI to design better chips. This is like Google’s AlphaChip has been put to use designing better TPUs, Google’s version of the GPU chip. So that’s like the motivation, sticking this into a canonical economic growth model, seeing what changes. What that cashes out as...

[52:20] Andrey Fradkin: Yeah, so before we get deeper into the paper... isn’t the idea that research helps do... like, you know, creating new ideas accelerates economic growth through subsequent acceleration of research and development efforts already embedded in the Romer growth model? How is this different?

[52:46] Basil Halperin: 100%. So what this does differently is that it says that there’s different kinds of research. So there’s like software research and there’s hardware research. And those are heterogeneous in interesting ways compared to each other, compared to you know, biomedical research or whatever. And taking seriously that heterogeneity and seeing what that heterogeneity implies.

So like in particular... one of the key lessons—so what we do in the paper is we write down a general networked semi-endogenous, like a Romer-Jones, general networked growth model. And draw out a couple of key insights I think. And so the core insights are around this idea of diminishing returns where we stand on the shoulder of giants to like... you know, we’re picking fruit from the tree of knowledge. We stand on the shoulder of giants to reach higher and higher fruits, but eventually the fruit gets harder and harder to pick because we pick all the low hanging fruit first. This idea of diminishing returns. And I think this idea of diminishing returns is like kind of obvious to economists, but it’s not always obvious in these conversations. Like the idea of an intelligence explosion, the idea of the singularity, kind of a lot of times can fail to recognize the importance of diminishing returns where there’s this idea that if you have a self-improving AI, like doing surgery on its brain to get smarter and smarter, that naturally has to lead to a singularity. But it doesn’t if the diminishing returns are strong enough.

[54:17] Seth Benzell: Okay, so now we gotta go back to the fruit. So okay, so now earlier you were talking about there were fruits, we were going for them... Explain this concept of diminishing returns through fruit because I’m really hungry.

[54:30] Basil Halperin: Yeah. So you’re hungry and so you’re picking fruit from the tree of knowledge. You pick the low hanging fruit first. And you know, that makes you stronger and gives you more energy to pick more fruit. But like eventually you pick all the low hanging fruit. And now you have to reach up and pick higher hanging fruit that’s harder to pick. And because fruit gets harder to pick—ideas get harder to find over time—you’re not just going to grow to become 100 feet tall, a thousand pounds because you’re running into diminishing returns in terms of fruit on the tree of ideas.

[55:10] Seth Benzell: So it’s like I grab one fruit and that gives me the energy to eat 0.9 more fruit, which gives me the energy to have 0.9 more fruit and it kind of peters out. I’m just riffing here, but is this like... is the Garden of Eden story... is that actually about diminishing returns somehow? It’s like we’re not in Eden because we have diminishing returns from apples?

[55:28] Basil Halperin: Yeah, I guess... I don’t want to say that the snake is Chad Jones because he’s the one who taught us this stuff.

[55:34] Seth Benzell: No, the snake is obviously Bloom and Reenen and all...

[55:38] Basil Halperin: Right, right. And Jones. Yeah, yeah. I guess so. But so exactly as Andrey said, like this is well known in the literature, this idea of diminishing returns. What we do is have this networked model where you have the software research sector and the hardware research sector interacting. There’s spillovers across sectors. And that teaches you a few things that I can talk about.

[56:02] Andrey Fradkin: But so at a high level... you know, if I’m understanding the idea in the paper correctly, is that you can undo diminishing returns with a networked production function for research, if you will. Here’s a question for you: What if we took an old growth model and just did away with diminishing returns, you know, all together and we had to have increasing returns? Wouldn’t we also get an explosion? Like... am I interpreting things correctly there? You’re kind of trying to microfound why increasing returns would happen.

[56:54] Basil Halperin: Yes. Yes. So to say that another way... like the original Romer model in this literature implied that there were no diminishing returns. Chad Jones comes along and points out empirically there must be diminishing returns. That’s because like we’ve had this constant 2% growth rate of ideas, that is 2% growth rate of total factor productivity or 1.5% percent. Meanwhile the growth rate of researchers has been 4% for like the last hundred years. So we have increasing number of scientists—like the two of you, thinking great thoughts—but we’re only producing the same growth rate of ideas of 1.5%.

[57:40] Andrey Fradkin: That’s because we’re podcasting too much.

[57:43] Basil Halperin: Seems plausible.

[57:44] Seth Benzell: It’s for the AI. We’re improving the AI, Andrey.

[57:48] Basil Halperin: Patrick Collison has this tweet that I think about a lot where he pointed out that when... when did growth in the US fall off a cliff a bit? It was like 2003 or TFP growth. And that’s you know, right when Facebook came out. Social media became the great distraction. Anyway, so yes, ideas get harder to find. That explains why growth slows down. And Andrey you point out that if you just get rid of that idea, then yeah indeed you could have a growth explosion. And indeed we are saying that spillovers across sectors can counteract those diminishing returns. And additionally, importantly, automation can also counteract the diminishing returns.

[58:27] Andrey Fradkin: Another thing to say is actually, and I think this is super interesting—not something I thought about going into the paper—is that you can estimate this diminishing returns parameter, this critical diminishing returns parameter by sector. And I can explain what these numbers mean, but that number for the economy as a whole is -3. So zero would be no diminishing returns. For the economy as a whole, it’s -3. For the software sector it’s -1. For hardware, like Moore’s Law, it’s -0.2. So the hardware sector has the least degree of diminishing returns of any sector that’s been estimated. So you know, if compute becomes a larger share of the economy, becomes more important, then this diminishing returns just inherently will become less of a thing. And then on top of that you have this spillover issue and this automation issue I’ve hinted at.

[59:17] Seth Benzell: So I know... natural question... and now I’m going to put on my applied microeconomist hat on is: where are you getting these numbers from, man? Yeah, you gotta parameterize this model.

[59:33] Basil Halperin: Yeah, so this is just looking at the time series. I can spell that out and I think I have an intuitive way of doing it, but yeah this is just looking at...

[59:40] Andrey Fradkin: Yeah, well let’s like walk through the hardware example. Let’s just like give us some intuition for where that number comes from. Because in my mind that seems like a really hard number to come up with even though we do have Moore’s Law, right? Yeah.

[59:53] Basil Halperin: No, so the ideal here would be to run an experiment. And you know, maybe METR has enough money to do that or something and maybe they should. But the way...

[1:00:00] Basil Halperin: ...the way that Bloom et al, the same paper that Seth mentioned, does this... the literature does this is the following: So say, you know, there’s like a hundred guys and gals thinking about how to improve semiconductors, how to improve hardware in the world. Fix that population. If ideas were not getting harder to find, that same hundred people would produce Moore’s Law. So Moore’s Law says that hardware productivity grows like 40% per year. That gets you the doubling every two years or more law. So something like 40%. Hundred people get 40% growth.

But we’ve had this constant 40% growth for 50 years, 60 years in hardware. But that’s required more than just like the original hundred. It’s required that that population of hardware researchers has grown by say 8%, call it, per year since the 1960s. So you’ve needed an increasing number of people to get the same progress in hardware. And so the way that that 0.2% diminishing returns number comes from is that ratio of 8% to 40%. That’s that point two.

[1:01:17] Andrey Fradkin: Okay. So now I’m going to tell you... now I’m going to use your paper to tell you why that number is wrong. So why is that wrong? It’s because it’s not just those hardware engineers that are producing that Moore’s Law. That Moore’s Law is being produced by everyone else in the economy that... who is producing let’s say like design software or even, you know, like I don’t know, cell phones... like all sorts of things contribute to Moore’s Law.

[1:01:47] Basil Halperin: Yes, exactly.

[1:01:48] Andrey Fradkin: And then there’s also just like physical returns to scale, right? So we’re producing more and more chips so that’s a production function parameter rather than a research parameter. So I don’t... so to me it seems a little strange to like lean so heavily on that number which ignores the entire point of your paper.

[1:02:10] Basil Halperin: So, so, so... a few things to say. One is...

[1:02:17] Seth Benzell: I mean I... yeah, give it a shot. You can also just crawl into your closet and we can hang up now. Your choice.

[1:02:22] Basil Halperin: No, no, this is basically the next paper that co-authors and I should write. Maybe Andrey you can co-author with us. Which is: indeed these prior estimates of these coefficients ignores exactly the factors that we discuss. So yeah, I don’t need to repeat what you said because that argument was well put and totally correct. But what that means or or as you said I think, what that means is that the degree of diminishing returns is underestimated because the progress is being benefited by spillovers which are not captured. So if you re-did the estimation with spillovers, you would find that diminishing returns is even harder and that like the singularity is less likely. Totally agree.

[1:03:07] Seth Benzell: I have a separate concern about these parameters. So alright, you want to tell us about the parameters we need in order to get this hyperbolic growth, right? But it kind of really seems like once you kind of like start the hyperbolic growth, once you like get on that curve, stuff’s going to get super weird super fast. Yeah. And like wouldn’t the parameters change pretty fast? So like how can you even extrapolate from today’s parameters to this crazy regime parameters?

[1:03:38] Basil Halperin: Yeah. I again am going to be in total agreement with you. I again am not someone who like wants to take macroeconomic models seriously as quantitative forecasts, but instead see them as formalized, mathematically formalized fables from which we can draw out particular insights and intuitions that were able to check are internally consistent because they’re written in language of mathematics. So that’s why the takeaway I have from writing this paper with Tom, Tom, and Anton is these ideas about: diminishing returns are important; spillovers can mitigate diminishing returns; automation can mitigate diminishing returns. And I feel pretty comfortable saying with the caveats that Andrey just emphasized, that hardware and software have less diminishing returns than other sectors. Though we should re-estimate those and hopefully will in a future paper. And that on its own is interesting. But not take super seriously like, where are we on the side of zero or negative? Are we on the side of increasing returns or decreasing returns? Like that stuff... yeah, these parameters I don’t have any reason to think those are stable as we go through 10 orders of magnitude of growth or something like that. Some people on the internet do take those that seriously and yeah, I completely agree.

[1:05:01] Seth Benzell: Uh if I... okay maybe we can talk for just what... we talked about the spillovers. Maybe you want to talk for a little bit about how automation might overcome “fishing out.” If I may suggest a motto for this: “If you fish fast enough, you can outrun fishing out.”

[1:05:15] Andrey Fradkin: Well maybe actually like maybe before you get to that we can just... one of the nice things about this paper is there’s like a concise message which is this Equation Number 1 in the paper.

[1:05:28] Seth Benzell: Yeah the one you... the equation you just told us to not care about. Tell us about it.

[1:05:32] Basil Halperin: Yeah. So I said that for the hardware sector this diminishing returns parameter is 0.2 and for the economy as a whole it’s 3. And again that was the intuition that the 8% researcher population growth versus the 40% productivity growth. Whereas if there was 0% population growth/researcher growth, then that diminishing returns parameter would be zero because you’d have zero divided by 40. Meanwhile if that number were negative, then you’d have the increasing returns and the hyperbolic growth, the singularity.

So the reason why I mentioned that is that zero there is the focal point, but really it’s like a... it’s a one plus a zero. So you have this critical condition of: are feedback effects greater than or less than one? And in like the canonical one sector model that comes down to this one diminishing returns parameter. In a networked growth model, instead of having one parameter that tells you are you having diminishing returns or non-diminishing returns, you have a spillover matrix. And the largest eigenvalue, the spectral radius of the matrix... I know you had Ben Golub on recently so...

[1:06:58] Seth Benzell: Just say, say the magic word. Give the audience the Eigenvalue.

[1:07:00] Basil Halperin: This is becoming the eigenvalue podcast I guess. If that largest eigenvalue is greater than one, then you have explosive growth. So “is that largest eigenvalue greater than one” can be summarized in this somewhat simple condition we have in the introduction of... it’s very loosely speaking like a weighted average of like the inverse of the diminishing returns parameter where the weights are determined by how automated is each sector. I don’t know how much sense that’s going to make out loud. In a lot of ways this paper is one of these papers where like looking at the math is actually a lot easier than saying it in words. But hopefully some of the insights have come across.

[1:07:45] Andrey Fradkin: So there are these like F... F terms which are the fraction of tasks that are automated by AI. Now like the first term of your equation is F of Y, which is the share of consumption good output that is production that is automated. Am I interpreting that correctly?

[1:08:07] Basil Halperin: Yes.

[1:08:08] Andrey Fradkin: Okay. Now what if that’s one just by itself?

[1:08:14] Basil Halperin: Right.

[1:08:15] Andrey Fradkin: That means that the entirety of the economy that we would actually care about in terms of consumption is automated already. So that’s kind of... in that case we don’t have explosive growth. It’s kind of on the boundary condition. Is that... am I interpreting that correctly? Because things aren’t getting better, it’s just that everything we want is is just being produced automatically.

[1:08:38] Basil Halperin: Right. If there’s nothing else going on, it’s right on the boundary. If you have epsilon of any other productivity growth going on or anything, you get above the exponential to super exponential.

[1:08:48] Seth Benzell: It would be like unstable in some sense if you were like exactly at one.

[1:08:52] Basil Halperin: Yeah, to perturbation.

[1:08:56] Seth Benzell: So Basil, I guess the last question I want to ask about this paper before we move on is... so you’ve explained how there’s a bunch of different things going on in the research process in the economy that are either going to kind of accelerate research and it’s going to get stronger and stronger or might slow down research and we’re going to get diminishing returns. Two of the most important factors here are kind of this idea of spillovers across sectors, but also this idea that you might be able to automate some research, right? As you get better AIs, you might be able to get faster algorithmic improvements. When I read kind of like LessWrongers, the kind of the latter kind of seems like the show, right? If you can get the AI to write better AI algorithms, there you are. In your model is that the important factor or are they kind of them all equally important? How do you think about that?

[1:09:47] Basil Halperin: Yeah, okay so let me want to say this. So the way I’d frame it is that these spillovers... or sorry, the diminishing returns limit the effects of AI progress. Spillovers in some like static sense... like we don’t think of spillovers as changing much over time. The innovation network doesn’t change much. But we think of as the economy grows, more and more tasks are getting automated. So spillovers provide some like static offset to the diminishing returns, whereas as automation increases, it’s continually offsetting diminishing returns. So I guess in like a dynamic sense, perhaps automation is more important. But sort of in the almost static way that we incorporate automation... either one is equally powerful in offsetting diminishing returns if you sort of do the comparative static. But in the sense of automation is the thing that actually changes over time, that’s the more important one.

[1:10:47] Seth Benzell: Okay. Stands to reason.

[1:10:49] Basil Halperin: If I can add one more thing about paper actually. So I didn’t mention one critically important limitation. So if you talk to economists about what will prevent AI from leading to explosive growth, I think we say one of two things. One is the diminishing returns. That’s that’s what this whole discussion has been focused on. But the other one is this idea of bottlenecks: that even if you have really fast progress in software engineering, then if you don’t have progress in the robotics side of the econ, the physical side, then that will bottleneck the growth if these sectors are complements.

[1:11:24] Seth Benzell: Yeah, and the essential thing is going to be the elasticity of substitution across sectors. Yeah.

[1:11:28] Basil Halperin: Right. And so we completely ignore the bottlenecks issue. We’re just focused on this diminishing returns idea, which to my mind is not a claim that there’s not bottlenecks. I think bottlenecks are super important. I think like there’s a 5 or 10% chance bottlenecks aren’t important—hence my earlier timelines forecast—but like...

[1:11:47] Seth Benzell: We all get uploaded. I mean yeah, there’s a universe where we all just get uploaded and like who cares that we don’t have robots for a while.

[1:11:53] Basil Halperin: Yeah or something like that. But yeah, the focus... the paper is meant to just like zoom in on the diminishing returns logic and to turn off the bottlenecks. But that’s important when thinking about how to quantitatively interpret the paper.

[1:12:08] Seth Benzell: There you go. Basil admits to one possible drawback to his paper. All right.

[1:12:13] Basil Halperin: That’s all you’ll get from me.

[1:12:15] Andrey Fradkin: Now I wanted to ask one more question actually because we’re natural right here and then we can go to the next topic. Which is like: how have you found the profession’s reaction to these sorts of exercises? Like you know, I can tell you what I... various opinions I’ve heard, but I’m curious like you were... you’re an author of these types of papers, so what has been your reaction? What has been like the feedback you’ve gotten? Yeah.

[1:12:43] Basil Halperin: I’m so curious about your experience. I have limited experience submitting these things through the publication process still because publishing takes so long. Yeah, I’ve only started submitting recently. Um, I guess what I would say is that like I feel like views on this are kind of polarized where some people are like, “This is super interesting and I’m glad to see economists taking this seriously as opposed to like wordcel mumbo jumbo from Silicon Valley or something like that.” Which I don’t want to say that I endorse that criticism, but some people have that criticism. And other people are like “This is...”

[1:13:16] Seth Benzell: This is a pro-wordcel podcast. You’re safe here.

[1:13:19] Basil Halperin: Yeah. Or are you calling yourself a shape rotator? Whatever.

[1:13:24] Seth Benzell: I’ll leave that up to you two. This podcast cannot rotate very many shapes. But that’s a topic for another episode.

[1:13:32] Basil Halperin: So that’s like really all to say that like to me it’s like too soon for me to say. And that’s why I would love to know what your experience is.

[1:13:42] Seth Benzell: My experience is that I found it completely impossible to publish and ended up having to publish a book. Yeah I think Seth has been trying to... Seth has been trying to publish this style of work for a very long time and the profession is not very interested, right?

[1:13:58] Andrey Fradkin: I would say opinions are changing, but I think the people have been battered for so long into being obsessed with like very micro identification... and given I’m not a macroeconomist... but like at least on the micro side that a lot of microeconomists just don’t consider it you know scientific unless there’s a tight identification argument. Or there’s an inherent skepticism of theory in some sense, which I do share to a large extent, which is that you can kind of get anything to happen if you’re a good theorist. And then it’s pretty hard to adjudicate between theories. And then to the extent that, you know, transformative AI is a mostly theoretical field at this point... it’s hard to adjudicate between transformative AI theories. So I think I’ve grown a lot more favorable to this type of work obviously over time because I just think like we might as well be working on the most important topics even if we can’t answer them as precisely. But I think a lot of people...

[1:15:09] Seth Benzell: Yeah, rather than just looking under the street light. Yeah.

[1:15:12] Andrey Fradkin: Exactly. Yeah. A lot of people are just not comfortable with that level of speculation. Yeah.

[1:15:18] Basil Halperin: “This is so dumb,” some might even say. No, yeah. Getting untethered from reality is like such a real risk on these big questions. In macro in general it’s so hard and you definitely see that happening. So it’s fair, it’s tough.

[1:15:48] Andrey Fradkin: I mean I think one of the interesting things that you did, right, is posted it on LessWrong. And in some sense like that has been more influential than any paper economics version of this paper that you could have ever written. For sure. Which says something.

[1:16:03] Basil Halperin: So to clarify for listeners, originally this was just some some shitpost. This was a blog post that I put out because like I was getting in fights with some friends in group chats and I was like, “Well the market doesn’t believe what you guys have to say.” And yeah and like it wasn’t going to be a paper and it just... it got such positive feedback that like it seemed like the demand was there for it to be developed a bit further into a paper. Uh, and in some ways I think that maybe I should instead of spending thousands and thousands of hours polishing papers before putting them out, I should be putting more out as blog posts first to...

[1:16:40] Seth Benzell: Dude, honestly yes. Because if you’re asking like my honest advice, I think when it comes to this TAI stuff there’s so much taste at the evaluation level that like spending another thousand hours polishing the same idea, the marginal returns are pretty low. At least as a practical careerist observation. If you feel like you’re learning, keep going.

[1:16:59] Andrey Fradkin: Well I do think that you know, if you get it... you know, for the profession, if you get into a top five journal there are obviously enormous rewards. But I think like there’s a risk of like polishing it for like some you know specialist field journal and still spending two years on it. I mean it almost makes one think that like you know there should be a new journal of Transformative AI Economics. I’m sure Anton has suggested something like that.

[1:17:27] Seth Benzell: Yeah, okay that’s what I was... maybe can we talk for a minute about your department? Which sounds so cool. You’ve got Anton Korinek who I remember back when he was doing macroprudential policy. I was like, “This is one smart cookie. I want to see where... let this guy cook.” What’s it like working with him? What’s this TAI department you guys are setting up?

[1:17:44] Basil Halperin: Yeah. So Anton has, yeah, been interested in the economics of transformative AI for longer than almost anyone, right? Like somehow back in 2016 he was thinking about this stuff. I’m still a little confused how he got into this so early. I think he did like a master’s in computer science maybe and had this in the back of his head. But yeah, so he’s managed to get a bunch of money to start this Economics of Transformative AI Institute here at the University of Virginia. Which is very cool. So me, Anton, and Lee Lockwood, who is a public finance economist, are sort of the three folks here who have written papers at least on the topic. And yeah I don’t know, trying to get folks to think more about the issue and write some research.

[1:18:28] Seth Benzell: What is it like working with Anton? Do you just like sit down with him and he’s like, “I already have solved all of the problems” and you just like you take notes on him as he dictates to you? What is it like collaborating with a guy like that?

[1:18:39] Basil Halperin: What can I say? I mean yeah, Anton’s been thinking about these issues for a long time. I can recommend his Coursera on the topic. In fact I went through that during the depths of the pandemic where he talks about the macroeconomics of AI and some models, Shannon information theory and interesting things. Yeah.

[1:19:00] Andrey Fradkin: Shannon information theory gets you to scaling laws? How does that come in?

[1:19:04] Basil Halperin: I don’t remember why he was teaching that but I was you know interested in the topic.

[1:19:08] Seth Benzell: This is neat. I’m Anton Korinek and this is what smart people think is fun.

Basil Justifies His Blog Posts:
Optimal Taxation in the Age of AI

[1:19:16] Seth Benzell: You recently got in a Twitter back and forth with other friend of the show Phil Trammell about optimal tax policy. You posted this really spicy meme of the two astronauts on the moon...

[1:20:00] Seth Benzell: ...and there’s the Puerto Rican astronaut with the gun to the American astronaut saying...

[1:20:00] Seth Benzell: ...and the American astronaut says, “So, even in the age of TAI, Pigouvian and Georgist taxation is the right way to go?” And then the Puerto Rican says, “Always has been.” Would you explain the context of you posting that meme, the Phil and Dwarkesh post, and how people should understand that?

[1:20:27] Basil Halperin: So yeah, Phil Trammell, Dwarkesh Patel... two guys that anyone interested in this stuff should be reading or following, listening to. Admittedly, Dwarkesh is a competitor of you two...

[1:20:39] Andrey Fradkin: No, no, no. We believe in coopetition.

[1:20:41] Seth Benzell: We’re cooperating... everyone should listen to both of our podcasts. We’re complements.

[1:20:46] Basil Halperin: Nice.

[1:20:47] Andrey Fradkin: We are actually complements, to be clear.

[1:20:54] Basil Halperin: So yeah, they wrote this great post, “Capital in the 21st Century,” playing on Piketty, saying Piketty was right in the past, but will be right in the future. And made this argument that as more of the economy gets automated, labor income will no longer be a sufficient tax base, and that power will be unequally distributed because capital income is so highly concentrated.

[1:21:24] Seth Benzell: Feels like these are three separate arguments already.

[1:21:27] Basil Halperin: There’s a couple different arguments in this piece, yes. And yeah, calling for capital taxation in the future, both for redistribution purposes of financial resources and to prevent sort of power concentration, is how I interpreted the piece.

[1:21:44] Seth Benzell: But I was taught in public finance class that capital taxation is bad.

[1:21:48] Basil Halperin: Yeah, I think there’s a lot of logic to that argument. So yeah, I wrote this thread just making a couple points. One of which is based on—we were just talking about my colleagues Anton and Lee, Anton Korinek and Lee Lockwood—so they had a recent paper summarizing sort of how should we think about public finance in a transformative AI world. So like take an AK economy, so an economy where all production is done by capital, no labor involved. What is optimal taxation in that world? And they point out or they show that consumption taxation is still optimal rather than introducing capital taxes. As long as you can raise enough revenue from that consumption taxation to fund whatever you need to fund. So that was like a first point I was making, that consumption taxation is going to dominate capital taxation.

[1:22:42] Seth Benzell: Let’s pause there for a second. Because I feel like all of my normie friends don’t understand this point. And in fact my advisor once, he tells me this story—I mean I assume it’s true—where he had like a half hour meeting with Bernie Sanders where he was trying to explain to him why consumption taxation is better for poor people than capital taxation. And Bernie Sanders’ brain was like, “But, but poor people no have capital.” Explain to a normie: why is consumption taxation considered preferred to capital taxation? Because only rich people have capital, right?

[1:23:14] Basil Halperin: So let’s see if I can do this with the caveat that I’m not a public finance economist, I just play one on Twitter. So the intuition I always come back to is this one that capital taxation is equivalent to explosive consumption taxation. So what do I mean by that? If I save... so you know, the University of Virginia pays me one dollar. I can either use that to go like buy a candy bar today or I can save that to tomorrow.

[1:23:41] Seth Benzell: But you don’t save it because of TAI.

[1:23:43] Basil Halperin: But I won’t save it because of TAI, indeed. I got to go party. And consumption taxation would be taxing that purchase of the candy bar. Capital taxation, taxing the savings. And if I save the dollar to tomorrow and try and buy a candy bar tomorrow... the capital taxation then would just be taxing consumption tomorrow differently than consumption today. And do we... like if we’re trying to equalize consumption across people, does it make sense to tax people who consume in the future rather than consume today? Like what’s the difference there? Is like one intuition pump. Honestly, like again, I’m not a public finance economist, I’m not sure on the spot I’m going to give the clearest exposition.

[1:24:38] Seth Benzell: No, I think that was pretty good. I think that was pretty clear. Okay, but then the memes about Pigouvian and Georgist taxation.

[1:24:45] Basil Halperin: Right, right. So first point, consumption taxation dominates capital taxation anyway. A bigger picture point that isn’t AI specific but does apply to the AI world is that we have these other taxes that not only are they less distortionary than consumption taxation, they might even be efficiency enhancing. So those taxes are taxes of externalities—Pigouvian taxes—should we tax carbon? Should we tax pollution? And Georgist style taxes where you tax owners of unimproved land or unimproved natural resources. People who just by luck and by happenstance happen to find out they have an oil well under their house. Like there’s no economic efficiency, and arguably no moral reason for those people to earn rents from the fact that all of a sudden, whoa, there’s a gold mine under my house.

So today, we should be taxing externalities to fix those negative externalities. Today we should be redistributing the pure rents of unimproved land, unimproved fixed resources. And that will only remain true in an AI driven economy. And those natural resources will become even more important in an AI driven economy where there are no scarce... there’s no scarce labor, there’s no scarce capital. The only thing that is scarce is natural resources. All that said, like I’ve mentioned this caveat that: are those taxes enough to fund the necessary redistribution or the necessary government spending?

[1:26:28] Seth Benzell: Land is the only scarce factor. You must imagine its price will be quite high.

[1:26:32] Basil Halperin: Yeah, in the limit, you would really think so. Maybe on the transition path... so this is a very good point that Phil made in the Twitter discussion of like, how quickly will the natural resource share rise? It’s not clear. I would be so interested if someone could answer that question in a convincing way or something.

[1:26:47] Andrey Fradkin: I don’t know. I think robots will be able to mine on the moon pretty efficiently, personally.

[1:26:55] Basil Halperin: And so natural resources won’t be scarce, is what you’re saying?

[1:26:58] Andrey Fradkin: Well, there’s a lot of natural resources on the moon.

[1:27:01] Basil Halperin: Are there? On the moon?

[1:27:04] Andrey Fradkin: I think so, yeah.

[1:27:06] Seth Benzell: We got red rocks. You can make robots out of red rocks, right?

[1:27:10] Andrey Fradkin: I mean you can also do all sorts of things...

[1:27:12] Seth Benzell: Silicon! It’s silicon, dude!

[1:27:14] Andrey Fradkin: You can also, you know, like have a ton of solar panels on the moon and then use energy to run fusion and fission reactions to get any resource you want.

[1:27:28] Seth Benzell: It’s different timelines. Different horizons.

[1:27:33] Basil Halperin: Different time horizons actually is I think a big part of the reason for disagreements on this. But um, like the rents in the economy have to go somewhere, right? If labor’s not earning it and capital’s not earning it.

[1:27:48] Seth Benzell: In a pure AK economy, there are no rents. It’s just A and K, dude.

[1:27:52] Basil Halperin: Right, right. The returns have to go somewhere. The returns above replacement maybe is one way of putting it. So anyway, that’s the source of the meme. Like why hasn’t anyone estimated whether we could just fund the US government by taxing externalities, by taxing land? Like someone should have done that, especially these Georgists obsessed...

[1:28:13] Andrey Fradkin: No, no, I think... well, I think the externalities... I mean our friends in environmental economics have definitely, you know... I think Larry Goulder has a bunch of work on estimating Pigouvian taxes in general equilibrium.

[1:28:28] Basil Halperin: Read it.

[1:28:29] Andrey Fradkin: I don’t think... I don’t think it gets you there. But Georgist taxes... I can imagine it can get you pretty far.

[1:28:39] Andrey Fradkin: Well cool. Uh, thanks so much for joining us. It’s been a fascinating discussion. Any final notes for our listeners? Anywhere they want to check out, in addition to your website?

[1:28:53] Basil Halperin: Yeah, feel free to send my papers. That’s a great decision. And of course, on Twitter and Seth’s as well.

[1:28:59] Seth Benzell: [Laughs] Great.

[1:29:01] Andrey Fradkin: All right. Well, thanks for... thanks for coming on and keep your posteriors justified.

[1:29:07] Basil Halperin: Thanks, Andrey.

This is a plausible story. But I think there will be substantial barriers to transforming firms into little more than automated coding systems plus a CEO. The reason is that much of the work done in large organizations isn’t actually producing code, manufacturing products, or whatever else we think of as “true work.” A lot of the work is people meeting with each other.

What Meetings Are Actually For

Why do employees of organizations spend so much time in meetings? When we introspect, the answer becomes clear. Meetings exist to create decisions. Multiple actors with stakes in a situation gather context, exercise judgment about that context, and come to agreement about what to do next. That process—unfortunately or fortunately for human labor—currently happens in human brains, not AI systems.

Even if AI wrote all the software at a large company, humans would still meet to decide what that software should do. They’d still make decisions about marketing budgets, compensation, strategic direction, partnerships, and countless other matters beyond product development.

The Limits of AI Judgment

Technologists will naturally respond that there’s nothing special about human judgment. AI can make these judgments, or multiple AI agents can converse to reach decisions—perhaps better ones—enabling fully autonomous firms. I don't think anything deep prevents this in principle.

But there’s a crucial constraint: existing organizations are meant to serve human preferences. When firms decide how to produce something, they’re ultimately serving the owners, who are human. In a more indirect way, they are also serving the preferences of customers, who are also human at some point down the supply chain. Until AI can literally read minds or predict human wants with very high accuracy, humans will remain essential to decision-making at some point.

Firms as Political Structures

Even setting aside the question of whether AI could replace human judgment, there’s a separate question of whether existing firms will allow it. Firms are political structures with power centers and veto players. Decisions can’t be made unilaterally. To launch a new product, change an existing one, or even swap out a model powering a feature, many people must be involved. As long as those people remain employed in those positions, they must participate in meetings, read the documents, and establish common knowledge that everyone is aligned.

This consensus culture may produce better decisions—more minds, more constituencies, more concerns addressed. But it dramatically slows everything down. Code that could be written and shipped in a day might still take months to actually deploy.

The Path Forward: New Firms

It’s hard to be optimistic that existing large firms will successfully shed this consensus culture. Instead, I expect many economic functions will be taken over by new firms—firms organized from the start to minimize human consensus as a bottleneck. These firms will use speed to outmaneuver existing larger firms in many markets for reasons John Boyd captured in the OODA loop.

There are a variety of ways in which these new firms may be structured. For example, managers might be represented by AI agents in meetings, employees might be replaced by agents altogether, or individual managers might have more unilateral decision rights rather than requiring broad alignment. We’ll see many such firms emerge, and as with any process of creative destruction, equilibrium will reveal which organizational forms survive.

But it’s worth keeping these basic forces in mind: the bottleneck to AI-driven productivity at the moment isn’t writing the code. It’s getting humans to agree on what to do with it.

Key Findings:

• AI interviews led to higher job offer rates and proportionally higher retention rates

• No significant difference in involuntary terminations between groups

• Applicants actually preferred AI interviews—likely due to scheduling flexibility and immediate availability

• AI interviewers kept conversations more on-script with more substantive exchanges

• Online applicants saw especially large gains from AI interviews

Topics Discussed:

• The costs of recruitment and why interview efficiency matters

• Whether AI interviews find different workers or just reduce noise in screening

• How human recruiters interpret AI interview transcripts differently

• The “Coasean singularity” question: Will AI improve labor market matching overall?

• Limitations: scheduling confounds, external validity beyond call centers, unmeasured long-tail outcomes

• The coming arms race between AI interviewers and AI-coached applicants

Posterior Updates:

On the usefulness of current AI for job hiring:

• Seth: 40% → 90% confidence AI works for call center jobs; modest update for general jobs

• Andrey: 20% → 75% for call centers; 1% → 5% for general interviews (“we need to reorganize all of hiring first”)

On whether AI will improve job matching significantly on net in the next 5-10 years

• Andrey: 55% → No Update

• Seth: “A bit more optimistic than Andrey” → +1pp update

Referenced Work/Authors:

• Prediction Machines

• Related episode on AI and labor signaling with Bo Cowgill.

Transcript:

[00:00:00] INTRODUCTION

Seth: Welcome to the Justified Posteriors podcast, the podcast that updates its priors about the economics of AI and technology. I’m Seth Benzell, an interviewer who will never stick to a standard script, coming to you from Chapman University in sunny Southern California.

Andrey: And I’m Andrey Fradkin, counting down the days until I can use an AI to pre-interview my podcast guests to see if they deserve to be on the show. Coming to you from San Francisco, California.

Seth: I don’t know. I think our filtering criteria is pretty good.

Andrey: I know.

Seth: Right. That’s one job we never want to automate—who becomes a friend of the podcast. That’s an un-automatable job.

Andrey: But it would be nice to pre-interview our guests so that we could prepare better for the actual show.

Seth: I was thinking about this, because there’s two possibilities, right? You do the pre-interview, and you get an unsurprising answer in this sort of pre-interview, and then that’s good, and then you should go with it. And then if you get a surprising one, then you would lean into it. What would you even get out of the pre-interview?

Andrey: Maybe what the guests would want to talk about.

Seth: Okay.

Andrey: But I agree with you. Mostly, it’s just hearing the guest talk, and then thinking about, “Oh, this is something that we want to really dig into,” versus, “This is something that might be not as interesting to our audience,” and knowing that ex ante.

[00:02:00] SETTING UP THE TOPIC

Seth: Yeah. We’ve been... So we’re talking about interviews. You’ll remember in a recent episode, we just talked to our friend Bo, who’s doing work on how maybe job applications are changing because of AI. So now I think what we want to think a little bit about is how job interviews are changing because of AI. Maybe we’ve heard before about how AI is changing how people talk to the hirer. Maybe we want to hear a little bit about how AI is changing how the hirer solicits information in an interview. We’ve got a very interesting paper to talk about just about that. But do you remember the last job interview you did, Andrey?

Andrey: Yes.

Seth: How did it go? Did you have fun? Did you feel like you stayed on topic?

Andrey: It was a very intense set of interviews that required me to fly halfway across the world, which was fun, but exhausting.

Seth: So fun. So you would describe the interview as a fun experience? Did you get more excited about the job after doing the interview?

Andrey: Yes, although I ultimately didn’t take it, but I did get—you know, I was impressed by the signaling value of having such an interview.

Seth: So the signaling value. So in other words, the signal to you from the interviewer about the fact that they were going to invest this much time. Is that right? It’s that direction of signal?

Andrey: Yes, yes. And also the sorts of people who they had talking to me, and just the fact that they were trying to pitch me so hard. Now, certain other companies lacked such efforts.

Seth: Right. So it seems like one important aspect of an interview is what the interviewee learns from the interview. But what about the other side? Do you feel like your interviewer learned a lot about you, or enough to justify all that time and expense?

Andrey: I’d like to think so. I mean, I’m not them, so I can’t really speak on their behalf. But it did seem like the interview process was fairly thought out for a certain set of goals, which might differ across companies. What about yourself, Seth?

Seth: Thank God, it has been a long time ago that I interviewed for a job, and I can tell you exactly what happened. I was on the academic job market, but I did throw out a couple of business applications, and so I got an interview at Facebook. Headed out to their headquarters, did all of the one-on-one interviews, and then there was a code screen, and I was not grinding LeetCode for the last five months and completely bombed it. And they said, “Thank you very much for your time.” So that was an example of, I think they probably could have saved the time for the interview if they had given me the code screen first.

Andrey: It’s funny, there was a time in my life where I interviewed at Facebook, too. I mean, this is probably 2014 or something.

Seth: Mm-hmm, mm-hmm.

Andrey: And they did do the coding screen before.

Seth: Who knows? Who knows, dude?

[00:05:15] THE PAPER

Seth: Okay, so interviews, we do them. People seem to give information, take information from them. How can this be made more efficient with AI? That’s today’s question. In order to learn more about that, we read Voice in AI Firms: A Natural Field Experiment on Automated Job Interviews, by friend of the show, Brian Jabrian and Luca Henkel. I was interested in this paper because it’s kind of an interesting flip side of what we just saw from Bo.

I guess before we talk too much about what the paper actually does, it’s time for us to go into our priors.

═══════════════════════════════════════════════════════════════════

[00:06:00] PRIORS

Seth: Okay, so Andrey, when we’re thinking about AI being used in interviews, what sort of thoughts do you have about that going in? What sort of priors should we be exchanging?

Andrey: Yeah, I mean, I think just when I first saw this paper, I was kind of surprised that we were there already, honestly. I think interviewing via voice is a pretty delicate thing, and the fact that AI is potentially able to do it already was—I hadn’t been thinking—I didn’t think we were there yet, and I think just the very existence of this paper was a bit of a surprise when I first saw it.

But I guess a first natural prior that we can think about is: is using an AI to interview someone rather than using a human to interview someone, is that better or worse, or how do we think about that?

So, Seth, what do you think?

Seth: Well, it’s a big question, Andrey. I guess my first response is, like we always say in this podcast, context matters, partial equilibrium versus general equilibrium matters. The context that we’re going to be looking at in the paper is call center workers. So maybe I’ll give kind of a different answer for short-term call center workers than maybe longer term economy as a whole.

When I think about call center workers, I think about a job that seems to be—no offense to our friends of the show out there who are call center workers—but this does seem like one of the jobs that is going to be the first to be automated with generative AI, or most at risk, especially kind of low-skilled call center work. So if there was going to be any sort of domain where you could automatically verify whether someone was good at it, intuitively, it would be the domain that you’re kind of close to automating anyway. So if it was going to work anywhere, I would say it would work here.

And yet still, call center work, you might imagine, it requires a lot of personal empathy, it requires maybe some subtleties of voice and accent that an AI might not identify or even might hesitate to point out such deficits. I would say I kind of went in with the idea that for call center workers, maybe there’s a forty percent chance that AI would be better than a human interviewer. So maybe it’s slightly unlikely that it would be better. But if we were to expand out to kind of knowledge work as a whole, I would be more, even more pessimistic, maybe only a twenty-five percent chance or lower that the AI interviewer would be better. What do you think?

Andrey: Well, how would you—what do you mean by better?

Seth: Oh, well, better in terms of the hire is ultimately the correct match, right? That’s going to be operationalized in a specific way in this paper, what... How they’re going to measure better match, but, yeah, that’s what I would say. They hire someone who’s going to be productive and work with the firm for a long time.

Andrey: Yeah. I mean, so that’s kind of one definition, I guess. Another definition might be, is the ROI from a particular interview process better or not?

Seth: Right, better net of costs. Right. Okay.

Andrey: Because I think one of the things that oftentimes economists underappreciate is that recruitment is an enormous cost.

Seth: Don’t tell those search labor economists, dude.

Andrey: Some of them model it, but I don’t think it’s actually a big focus. But it’s just the process of interviewing. You know, let’s say there’s a position, and you need to interview six people for a relatively high position, so that’s six hours direct, or maybe it’s a half-hour interview, it’s not obvious. But then also, there are all the meetings and pre-meetings, post meetings. Maybe you give an offer, and then they don’t accept it. And there... I mean, there’s just a lot of costs involved. So even if it wasn’t as good as a preexisting interview process, it might still be ROI positive for the firm.

Seth: I guess we come back to what is the cost of interviewing versus the cost of making a bad decision. You know, well, it’s not, it’s public information that we, here at my university, we hired a dean of the business school who was an absolute disaster and got voted out by the faculty in a ninety-eight percent vote after one year. That guy did a lot of damage, right? We should have interviewed him harder.

So it really depends. So I guess the point would be in kind of higher leverage roles, you would think that the interview costs would be a relatively negligible part of what’s going on.

Andrey: I don’t think that’s true. I think in higher leverage roles, higher leverage people have to do the interviewing, and the cost of delaying hiring is much higher. So to me, it’s not obvious. But anyway, that’s, this is all a sidebar.

Seth: Okay, so let me hear the prior.

Andrey: Yeah. So I think my prior that this interview technology would be better than a human technology, just solely based on match quality, was actually quite low. I probably twenty percent, or maybe less than that, actually. Because it just seems like, yeah, maybe on average or maybe in a typical case, it’s fine, but there’s so many things that can happen in an interview that you could only learn by running a process enough times to really learn how to do it well. And so, yeah, I wasn’t super optimistic that it was going to work yet, even for call center workers.

But I think for kind of higher-end labor, right, I think my prior that it would be better is very low, you know, like 1%. Just because I just don’t think we’re there yet.

Seth: Wait, so I’m getting—So 20% for call center workers and 1% generally, was the take?

Andrey: Yeah, that would be my sense.

Seth: Mm-hmm.

Andrey: I mean, just, it’s hard to imagine that at today’s technology levels, that for, let’s say, a professor job, that the AI could interview better... I guess one way to put it is getting rid of all the humans in the interview loop for a faculty hire, that seems just kind of crazy.

Seth: Right, and that... Well, obviously, a more extreme experiment than what we’re talking about here. Faculty, we’re thinking about, you know, maybe they’re pushing frontier knowledge, would be the last thing that you would think that an AI would be able to get at. Another thing I think about is someone who’s going to be in your faculty is living with you for 20 years, so you might really care about if they smell good, if they have a peccadillo that bothers you, that these might not be relevant considerations in a call center remote job, right?

Andrey: Yeah. Yeah, exactly. I think... And I think, actually, the interpersonal thing, which is a very contentious thing, by the way, is that I think people understand that good teams get along with each other. But at the same time, screening based on how much you’d like to have a beer with someone might have problems, you know?

Seth: Not good.

Andrey: So yeah. So, you know, it’s not obvious which way that cuts, but certainly it’s an important part of hiring. And, you know, I think for higher-paying jobs, it’s not that there’s just one interview, of course. There are many, many interviews, and oftentimes, in-person components of interviews over dinner, and so on. And you might think, you know, maybe that’s all unnecessary, but given that it persists in equilibrium, even though it’d be a lot cheaper not to do it, that should signal something.

[00:14:00] GENERAL EQUILIBRIUM CONSIDERATIONS

Seth: Good point. But now, Andrey, what I’d like us to think about for a second is to maybe zoom out for a bit and think about, okay, we’re talking about current generation technology in partial equilibrium in this study. One company uses 2025 generative AI to try to attack this specific question for call center workers. Let’s take a step back. You know, that’s what we always want to do in this podcast, is take a step back and like, okay, what does this tell us about the broader process that society is undergoing?

You’ve written recently, movingly, to be honest, about this idea of a Coasean singularity, that AI will be so good at helping us communicate to each other, that we’ll get perfect matching at zero cost. I don’t know what timeframe you have in mind, but presumably, one of the things we’ll get better at matching is people to jobs. So maybe you’re pessimistic that in this context, in this time, that AI will be good at hiring, but do you think, you know, 5, 10 years from now, as these technologies diffuse, do you think we’ll get better job matching as a result of employers using a lot of AI and job applicants using a lot of AI? Is that final equilibrium the destruction of all meaning, as Bo, you know, foretold, or is it the utopia of the Coasean singularity?

Andrey: Well, I do want to point out that I don’t think any of the authors strongly believe that the Coasean singularity will happen, actually, you know?

Seth: Oh, the Coasean singularity is a myth?

Andrey: The Coasean singularity, question mark, Seth. Question mark.

Seth: Question mark’s doing a lot of work, Andrey.

Andrey: Yeah. No, the paper is doing a lot of work to tell you why it might not happen.

But I think, yeah, I think time horizon certainly matters here, right?

Seth: Okay, but let’s say 5 to 10, to just to choose a number.

Andrey: Yeah. So, so, like, not that long a time horizon. It’s very non-obvious to me. Just because there are all sorts of institutions that are going to be involved, very messy institutions. Like, one of the things that we already talked a lot about on this show is the problem of too many applications, applications lacking signaling value. At the same time, you know, you can imagine on the interview side, if you interview, you know... How does this all affect the number of interviews you’re going to do?

Seth: There’ll be more and more applications. The cost of applications goes down, yeah.

Andrey: Yes. Now, maybe the cost of interviewing goes down, but it doesn’t for the applicant if they have to be the one... You know, if the applicant’s agent is doing the interviewing, maybe it’s a different story. But if the—

Seth: Right! How many, how... It’s like, it feels like you’re watching, you know, the drone war in Ukraine. There’s the move, and the countermove, and the countermove, and the countermove. It’s hard to say where that process ends, right?

Andrey: Yeah. So I... And then I think, of course, you know, there are actual individual institutions involved. Like, what is the government going to do? And even if some nimble firms are really doing a great job of matching using AI technologies, how that plays out when there are other organizations that are using other sorts of tools, it’s just completely not obvious to me over a five to 10-year time period.

Seth: So is that a fifty-fifty? Is that a, I have—is my prior is the completely uninformed prior?

Andrey: No, no. I think because you’re introducing both sides of the technologies, both the AI for the applicants and for the employers, it’s hard. I mean, I’m a bit of an optimist, so maybe I’ll say fifty-five percent chance.

Seth: Fifty-five percent. Ooh, I have to say, I’m a little bit more optimistic than you, Andrey. I think if you think about the world, the world, since, you know, the rise of the printing press, has seen an arms race in technologies for understanding versus technologies for lying, right? And yet, we think kind of the general process has been towards better price discovery, better matching, right? It seems like we could translate the same ideas to financial markets, where people are getting better at lying, people are getting better at trading, people are getting better at communicating. But ultimately, I mean, at least my sense is that price discovery has improved, right? So I guess—

Andrey: Oh, I would argue the opposite. So I... Not price discovery, but labor discovery, I think has been substantively hurt over the past five to ten years. Because our educational institutions have abdicated their role—

Seth: Credentialing.

Andrey: Actually, credentialing, and because it’s been trivial to start applying to jobs. So yeah, I mean, look, that’s a little too pessimistic, but I’m just saying that over a five- to ten-year period, I have to be a little bit cautious. I think if we’re to be able to reoptimize our institutions, I mean, now the problem with going thirty years is how much human labor do we even have? But to me, just lots of things could be going on.

═══════════════════════════════════════════════════════════════════

[00:22:00] THE EVIDENCE - CONTEXT

Seth: Okay, all right. So we’ve got our priors locked in. Now it’s time to turn to the evidence.

Okay, so our context here is the Philippines in 2025. We’ve got a pool of about seventy thousand applicants to different call center jobs. They’re all going through this one recruiter who’s recruiting for multiple different businesses. To give some context about the call center job market, this is a very high-turnover, low-paid work. We’re talking about three or four hundred dollars a month at two to three times minimum wage. The skills required are English speaking, flexibility with changing shifts. There is a line in the job application that calls for strong analytical and logical thinking. I think strong might not be the correct adjective there. You probably need more than zero.

But all this combines into a job that people are not married to. So we’re looking at a job with sixty percent annual turnover, with a high share of that being people voluntarily leaving rather than being fired. The... We’re talking, in order to do these interviews, people, first, they can either show up in person to one of these recruiting offices, or they can apply online. Then they’re scheduled for an interview, and they also take a standardized test that has both an English skills component and a kind of analytical mathy component. And just to give a sense of how strong a filter this is, about six in—if we’re talking about the human interview baseline, about six percent of applicants accept a job, while two percent still have a job one hundred and twenty days after being hired. So that’s not a conditional average. That’s just two percent of people who show up for an interview end up having the job for at least four months. So that’s our context.

Andrey: And about ten percent get an offer, approximately.

Seth: Right. Yeah, yeah, so ten percent get an offer, six percent accept the job. Okay. So that’s the context. Andrey, do you want to tell us about the experiment?

[00:22:40] THE EXPERIMENT

Andrey: Yeah, sure. So in the experiment, workers were, or applicants... Well, first they were pre-screened a little bit—

Seth: Very lightly.

Andrey: Yes, and then they were assigned to either a group where they had an AI interviewer, whether they had a human interviewer, or one in which they got to pick. And I guess there’s a lot to be said about the specifics of that interviewer process. So there, as you can imagine, for a job where so many people are being hired, there’s a lot of standardization of, you know, what sorts of things need to be discussed, in what order. And the AI tries to... You know, the AI tool that the company has purchased is going to is programmed to do that, and it tries to do that. Another key important part of the context is scheduling.

So an AI can take the interview at any time with you, which could be just right away, as soon as you pass the pre-screener, whereas a human needs to be assigned to an interview, and that could take some amount of time. So that’s also a pretty big potential difference in how we should think about these things, right? So we oftentimes focus, oh, can the AI really do it? But actually, AI has this other advantage where it could just do it right away.

Seth: Although, it is, it’s an interesting result. Even though the AI conducts the interview faster, it still takes longer for the AI interviewed to actually get the job offer decision, which seems to be driven by the humans. And now we’re going to get into the details of how does this AI system work? There is a human who listens to the AI interview, right? And apparently, I get the impression that the humans who listen to the AI interviews do not enjoy it. They would rather listen to themselves, right? They score these a lot faster if it’s their own interview versus the AI interview.

Andrey: So did they really do a good job of explaining why that happens in the paper? Or maybe—

Seth: Well, that’s my speculation.

Andrey: That’s actually not what my speculation is at all.

Seth: Okay. Oh, let me hear it.

Andrey: So you’re portraying it like, you know, they’re just taking a long time to listen. Like, they, you know, to listen through the interview. But actually, it seems like a procedural thing. Like just the system, when it assigns them to review these applications, you know, is later than if you already did the interview.

Seth: Presumably, you score it right there.

Andrey: Yes. Yeah, yeah. And to be clear, my understanding is that there’s a different person, which is the recruiter, who’s doing the scoring, than the person who’s doing the human versus the machine interview. So it’s not like they’re either listening to the machine or listening to the human and then finding the machine less interesting to listen to. It’s actually just procedural that they’re getting assigned to read this AI interview result later.

Seth: So maybe not an essential difference, but one that could be corrected with a little refinement here.

Andrey: Yes, exactly. Yeah, yeah.

Seth: Mm-hmm.

Andrey: I know we got into kind of this side bit, but I don’t think it’s a side bit because it’s always important to think about what is the treatment exactly. And one of the threats to internal validity that I always teach my students is that if multiple things are changing at the same time when the treatment gets assigned, and in this case, there are. You know, you’re getting the AI interview, but you’re also getting interviewed way faster initially. So from the applicant’s point of view, that’s kind of very salient.

Seth: It’s sort of a different experience.

Andrey: Yeah.

Seth: Which, you know, like we talked about, the interviewee also learns from the interview, right? It’s like when the professor says, “I learn far more from my students than they learn from me.”

Andrey: Yeah. Well, I don’t think this is a learning—I mean, it’s not like I’m going to rule out learning by these workers. But my sense is that there’s not a lot of uncertainty about this job for the people who are—

Seth: These jobs are pretty homogenous.

Andrey: They’re pretty homogeneous—well, you know, they’re at least... You know the distribution, you know, probably, you know, doesn’t have too much to do with the specific firm. You know, they’re—probably, the call centers jobs are, you know, there, there are just a lot of them, and depends on which, who you get assigned to in terms of your client.

Seth: I think this is an important point, which is that it really does seem like there’s more vertical differentiation here than horizontal differentiation. You might imagine a context with more horizontal differentiation, the AI interviews might not be as good. But here, we’re just trying to find the right tier of worker, because if it hasn’t become clear yet, the main failure mode isn’t you hire someone who’s too bad. The failure mode is you hire someone who’s too good, and they leave the job after a week.

Andrey: Well, we don’t—So to be clear, I don’t actually know why people leave their job. You’re assuming that they’re too good, but actually that to me is completely not obvious. It’s like an Uber driver. It’s not like the Uber driver is too good if they stop driving on Uber. It’s just maybe they needed money for a couple of weeks.

Seth: Well, their distribution of opportunity cost is higher, which would be correlated with being good.

Andrey: Yeah, but it might also just be they just had temporary liquidity... To be clear, what I’m trying to say is that that correlation, in my opinion, is very likely to be low. The fact that these people apply to this job, which is very fungible in the first place, which so many people in their country apply for, is not suggesting to me that these applicants are somehow, have all these amazing other opportunities. And, you know, they’re probably call center workers that might be cycling between call centers, or maybe they’re cycling between call centers and other seasonal work. I mean, I don’t know. I just wouldn’t assume it’s about quality. Yeah. It’s not like “Oh, wow! They’re so good at math, and then they got discovered.” You know, that’s kind of not the story here.

Seth: Okay, but we’ll come back to whether who seems to be helped by or hurt by the AI worker in a second. I guess one last thing I want to say about the experiment and its context before we go into the results, are that they... We also get a survey of people on their interview experience. So you might imagine that they’re going to be obsequious or sycophantic, to use a word in vogue these days, because, you know, they’re trying to get a job, but that just gives us another slice at trying to understand what they’re thinking.

Andrey: Yep.

Seth: Okay—

Andrey: So yeah, I mean, I guess we should say, because we haven’t made this clear yet, this is an absurdly impressive experiment. I mean, holy crap!

Seth: Yes.

Andrey: Right? Just logistically, it’s... You know, I can imagine how difficult it would be to get all this machinery rolling and, you know, figure out the pilot studies, and figure out the AI model provider, and convince the firm to do it this way versus a variety of other ways. You know, I think it’s notable that certainly, the firm should be interested in the results of the experiment. They’re—It’s probably an active, like many other firms, they’re actively deciding where to use AI tools, and so it is incentive aligned in that way. But still, it just is a very impressive experiment.

Seth: Yes, huge snaps to the authors, especially Brian, who I understand is on the market right now. Give the man a job.

[00:31:00] HEADLINE RESULTS

Seth: So all right. To get into the headline results, the AI interviews seem to work. We get twelve percent more offers. So of the people who are randomized into the AI group versus the human group, the AI interviewed get twelve percent more offers, have eighteen percent more job starts, and have eighteen percent higher chance of working with the company for at least four months. So our main outcome here is retention and hiring as positive outcomes. Maybe in the limitation section, we’ll talk about kind of the limitations of those as the endpoints, but, you know, retention seems to be one of the big challenges here, given that it’s kind of, as you said, very fungible work. And those seem like significant results, plus on top of all the cost savings you previously talked about.

Andrey: Yeah, yeah. I mean, it’s definitely... You know, the ROI calculation, of course, needs to account for other things, but just the baseline results do suggest that this is a very useful technology.

Yeah, what do I make of this? I think it’s interesting to think about where this effect is coming from. Is it coming from different types of workers being screened by the two methods, or is it just that the AI method just picks off a few marginal workers that happen to stay longer?

Seth: Be bad at interviewing, right?

Andrey: Yeah, or bad at interviewing, or they, you know, they’re actually good enough, but the old interview process was a bit too noisy to pick them out, right? So there’s kind of this question: What’s going on? Because what I would’ve thought that, you know, like if I was a company, and I was thinking about, well, what is the interview technology that I want? I want an interview technology that gives me the same decisions as I was making before but with a lot less cost.

Seth: Mm-hmm. Right.

Andrey: The fact that this technology instead increases the hire rates. First of all, in a lot of jobs, like for a lot of jobs, there’s one slot, so this couldn’t be a result that was replicable, right? Like, if you’re hiring a professor, and you have one slot, it’s not like you’re going to increase... I mean, you can increase your hire rate from zero to one, but it’s kind of... It—

Seth: But retention then.

Andrey: You have to really... Yeah, but those are different—But you have to think about why you’re getting the retention effect, right?

Seth: Right.

Andrey: And so there are kind of different things that we can think about here. Is it that the interview process is less noisy? Is it that the interview process is more lenient, that it’s getting marginal guys? Or is it that actually, it’s actually picking out different people, and those people are better matched, which then raises the question of like, wow, those old interviewers were not very good, right?

Seth: Right.

Andrey: Which is, you know, I’m sure there are plenty of interviewers who are not good. That’s—It’s not surprising to me. Yeah, but I guess, yeah, those are the questions that are raised, right? Because I don’t think it’s inherent. How you use the AI tool is your choice as a firm. There’s no law that’s going to say that you’re going to increase your hire rates because you happen to use an AI interviewer, right?

Seth: Right. And so, yes, a great point is you might be concerned that this leads to a more sort of lenient, we’re letting in marginal people. You know, we’re not actually getting more information. Or maybe we’re getting less information, and we’re just letting in marginal people. One piece of evidence against that is there is no significant difference in the rate of involuntary disconnections, right? So remember, retention is higher, and that is not driven by any difference in the newly hired being less likely to be fired, right? The people who are hired by AI, the reason they are retained for a little bit longer is because they are basically fired at the same rate, but they’re less likely to disconnect on their own a little bit. That’s my read.

So how do you interpret that?

Andrey: I guess it still isn’t telling me that whether we’re picking... I mean, for what it’s worth, I just—My reading of the evidence from this paper is that there’s just a lot of overlap in who gets hired, and then there’s just a few marginal guys, and then your power to detect differences and fire rates between the two are very low. But I don’t think the firm—I’d assume that the firm doesn’t care that, you know, there’s so many workers falling through, you know, that involuntary separations are just part of the game. But I wouldn’t... It seems like the power for that difference seems very low.

Seth: Fair enough. And further, and we can talk about this in limitations, too, retention rate just gives you a sense of what percentage of people are above or below some sort of line of so disastrous you get fired. You might imagine that an AI interviewer has a lower chance of detecting the truly disastrous person who’s just going to start slamming racial epithets at everyone who calls up, right? You might imagine that there’s kind of a long tail of badness that’s not being picked up by AI, and then this measure of outcome wouldn’t pick up that the long tail of badness is getting worse.

[00:36:35] MECHANISM - HOW THE AI WORKS

Andrey: Yeah, yeah. I mean, and to be clear, I don’t want to highlight that. I’m just making the point that there’s no generic—I like to think about the prediction machines framework here maybe.

Seth: Friend of the show, Avi Goldfarb.

Andrey: And Ajay and Joshua Gantz, yes. So the AI makes a prediction, but then you’re the decision maker. Let’s say you’re the CEO or the hiring manager of this firm. You get to choose how you use that information, right? So you can use it—

Seth: But it’s not that the AI isn’t... Wait, wait, wait, wait. The AI isn’t making a prediction here. The AI is soliciting different information in the interview.

Andrey: Sure, but it’s giving you a signal. And you can choose what to do with that signal however you like, right? So that’s kind of the point I’m making. In this case, the AI was good enough at interviewing people that you got a pretty good signal, and the system used it in the following way that seemed to have been positive. But I guess what I’m saying is how you—there are human recruiters that are taking the signal from the AI interview and choosing what to do with it. And they chose to hire more people as a result. That’s not a quality of the AI, that’s a quality of the humans making decisions off of information.

Seth: I mean, I don’t know what to say to that, Andrey. Like, you know, it’s like saying, you know, the factory didn’t make 10 tons of steel. It was the business factory sociotechnological system that made 10 tons of steel.

Andrey: No, I guess the point I’m making is that you could have imagined, here’s a simple story. Let’s say the interviewers don’t know how to interpret the AI interviews, and they do know how to interpret the human interviews. Then they could make very different decisions off of very similar transcripts off of the two.

Seth: Correct.

Andrey: Right? That, I guess that’s what I’m trying to say.

Seth: And I think that’s right. I think that’s right, but I’m also pointing out that we usually don’t talk about technologies that way. Every technology is embedded in an organization. So yes, but yes, every other technology also.

Andrey: No, because when people do AI evaluations, they’re always saying that AI does this, AI does that. And then in this case—

Seth: Like GDPVal.

Andrey: Yes, yes. AI is going to fully automate end-to-end this task. And I guess what I’m saying here is that there’s no way it’s automating the decision. It’s not automating the decision. I guess the other thing is there are AIs that automate decisions in hiring, right? There are certainly AIs that screen resumes, for example. So I don’t think it’s a crazy thing to talk about here.

Seth: I don’t think you’re being crazy either. And of course, the context matters, but then even in GDPVal, I could say the same thing, right? It’s going to get evaluated by a human expert. The human expert either is good or bad at understanding the way that the AI talks about the thing. I mean, it seems like any time a human touches it, okay, yeah, it’s in a human context.

Andrey: I guess... Sorry, but you keep on thinking that this is a criticism. It’s not a criticism that I’m—You don’t need to defend it. It’s just I’m just saying that—

Seth: I’m not saying it’s a criticism.

Andrey: Yeah.

Seth: I’m saying it’s a universal... I’m saying it’s a truism.

Andrey: It’s just the company chooses what to do with this.

Seth: True.

Andrey: It’s interesting that the way that it was used happened to play out this way. But for example, the company might not have wanted to hire them, right? Like, what is the hiring cap for the company? Do they want to hire infinite workers? Do they want to hire 50 workers? How does that allocate the—

Seth: Do they care more about average quality or average retention? I totally agree. Totally agree. Okay, so I don’t think we’re disagreeing.

[00:41:00] LINGUISTIC ANALYSIS

Seth: All right, but let me try to help you a little bit, Andrey, with thinking about what’s happening different in these interviews. Because maybe we can’t exactly say how are the people who get hired different under the two regimes, but we can say something about how the two different interviews go. And so the authors do this really fascinating linguistic analysis of what actually happens in the interviews, because they’ve got the full text of all of these interviews.

Andrey: Actually, can you show figure 2 first, actually?

Seth: Ooh, let’s talk about figure 2 for a second. All right, I’m putting figure 2 on the board. Is that good?

Andrey: So I think I found this very helpful to address some of the questions about... that I was raising. In particular, what we see here is on the top line, the human topic coverage, and on the bottom line, the AI topic coverage. And the AI does seem to cover more topics most of the time than the human. In the second column, we see that the AI tends to follow the preordained order of the interview that was, you know, the interview designers designed. And in the third column, we see that the AI follows the guideline questions much more closely. So it’s standardizing the interview process. So my sense is that this should reduce the noise in the hiring decisions quite a bit. You know, at least in a very naive model of hiring. Now, you can come up with scenarios where there’s—

Seth: Yeah, in a naive model where the generic approach is the correct approach, right?

Andrey: Yes, yeah.

Seth: Because you might have a model—

Andrey: If you need to cater to different people, how you interview, because you’re really trying to extract a particular signal, then maybe this won’t work. But then we go back to the fact that these are call center workers, and maybe there’s more of a—it’s a more standard situation.

Seth: Agreed. Okay, but I, you know, even though this is an interesting figure, the figure that really struck me is the next one, where we look at, okay, what are the things in interviews that are predictive or not predictive of the interview leading to a hire? And then how often do those appear in the AI versus the human interviews? And so what are the bad things that happen in human interviews that don’t happen in the AI interviews? Well, first, I love this one: back-channel cue frequency. Now, I’m not a hundred percent clear on what this means, but the implication is it’s people trying to give a kickback to the interviewer or saying, “Hey, I know your cousin, give me an interview.” Did you get a sense of exactly what this is?

Andrey: Yeah. I don’t quite know how to interpret it.

Seth: Well... I mean, that is kind of interesting and funny and kind of reflective—

Andrey: Short cues indicating attention or agreement. So I don’t think that’s exactly what we’re talking about.

Seth: Short cues, agreement—so they’re just saying, “Yes, yes?”

Andrey: Yes.

Seth: “Hmm.”

Andrey: Hmm.

Seth: Hmm.

Andrey: Hmm.

Seth: That’s less exciting than what I thought that meant. Okay, well, how about this one? We talked... And I think this is really illustrative here of how you might not be able to extend this result out of context. What is bad for an interviewer? Asking a lot of questions about the job, right? Like we said, Andrey, in the kind of jobs you apply for, they’re trying to get you, right? The interview is just as much about what you learn about them. That is not the kind of job we’re talking about here. Any time you’re spending saying, “So you’re telling me this call center worker doesn’t have any benefits?” You’re signaling to them that, you know, you’re going to be a little bit light-footed, wouldn’t you say that, Andrey?

Andrey: Yeah, I mean, it’s a standard job, you know, not... I presume that most people applying for it know how it works.

Seth: “Will I be required to talk to people on the phone in this job?” That’s a bad signal if you say that.

On the other hand, what happens more in the AI interviews? Well, the one thing that happens significantly more of are exchanges. So like you showed us before, you get through more of the standard questionnaire in the AI interview, which makes sense if the AI is good at sticking to the script, which, as I clarified in my intro joke, I think I would be bad at. So that tells us a little bit about what’s happening different in these interviews.

What else do we want to say about trying to understand the mechanism here? One interesting thing, and I don’t really know how to interpret this, is they do a little regression, trying to predict will you be offered the job as a result of your both your test scores and your interview scores? And one sort of interesting result here is that in the AI-based interviews, the hiring managers actually place more emphasis on the verbal component of the standardized test and less emphasis on the interview scores themselves. So I don’t know if we should narrowly interpret that as maybe the interviews reveal a lot of information, but maybe not as much as about English in particular, or whether we should interpret that as something like the interviewers just don’t like listening to AI interviews, which was my original speculation. Do you have an interpretation of that result? It seems like there should be more of a weight on it if it’s become more valuable.

Andrey: Yeah, I don’t quite know. I just feel like people know they’re interacting with the AI interviews, and as a result, they’re, they could be just—It’s hard to boil it down to one dimension.

Seth: Mm-hmm. Fair enough. And again, that’s kind of, you know... Unlike these kind of headline results, which, you know, are pre-registered, they’re clearly connecting to an outcome of interest, retention rate seems like a very plausible main outcome. This is kind of more exploratory. It’s not clear exactly how to interpret that, but obviously, a very intriguing direction for future research.

[00:47:00] ONLINE VS IN-PERSON APPLICANTS

Seth: Okay, one last striking thing that I want to bring up, and maybe this speaks to—this is kind of the last bit of interpreting the result that I want to think about. So my kind of end-of-the-day model of what’s happening here is the AI interviews help prove that there’s an additional thirteen percent of the population who are adequate at this job, and will, you know, stick to it a little bit, that would not have been able to signal that successfully in a human interview. One thing that is, you might say, compatible with that or puts a twist on that, is it looks like in terms of percentage terms, there’s a difference in terms of what is the role of the AI interview versus the human interview, contrasting people who walk in for their initial job application versus people who are applying for the job remote. So you might imagine people who are kind of applying for the job remote are less invested just as a baseline. It’s much easier to apply remote than to apply in person. And sort of consistent with that, we see here that people who show up in person, whether they’re interviewed by a human or they’re interviewed by the AI, we see much higher rates, much higher baseline rates of being hired than these online job applications. So but within these online job applications, what do we see? And I’ll maybe put this in the middle of my screen again.

What do we see? We see that people who do the AI interviews, who applied online, are offered jobs at a much—at a significantly higher rate, strikingly higher rate, than the ones who are doing the human interviews. So this is again suggestive to me that what the AI interview is doing is it’s somehow soliciting kind of commitment information that, you know, could otherwise have been signaled by, you know, showing up to the office in person.

Andrey: Yeah, I wouldn’t say... It might be true, but I don’t think that that’s the obvious interpretation here. I mean, there could be quality differences between the two. So I wouldn’t say it’s just commitment. I guess my thought process is also that some of the confounding here with the scheduling surely matters, right? I applied. I’m ready. I finally did it! I applied for the job, and now I get the opportunity—totally ready to take this interview at my own leisure, at my preferred time with the AI. Yeah. Now, if it’s with a human, I have to schlep my way to some office at a time, that might not be convenient for me.

Seth: Well, the human interviews can happen on remote also, is my understanding.

Andrey: Yeah, fair enough.

Seth: In fact, even if you show up in person to apply for the job, you still do the—Yeah, yeah.

Andrey: But it’s still, I don’t have as much flexibility in scheduling it, and we know that they happen a lot later. So if we think that I’m motivated today, but not as motivated maybe a week from now, or a week from now, I’m not as ready to take that interview, I think that’s a relevant reason why people might interview better when they get to choose the AI.

Seth: Fair enough.

Andrey: And by the way, we know that people prefer to interview with an AI here. This is very—

Seth: Yes, because we get that third randomized group. Yeah, please tell us about it.

[00:51:00] APPLICANT PREFERENCES

Andrey: Yeah. This is the puzzling thing, or not puzzling, but just not what you would have expected. It’s like people prefer to have the AI interview, right? Which I don’t know if I would... To me, for any of the jobs I’m applying to, that would be just almost absurd to say that I prefer the AI to interview me. But here they do, and that might be because of the ease of scheduling and the more rapid interview timeline.

Seth: One thing I’ll say there is, maybe suggestive of what’s going on there, is when we look at the test scores of the people who choose to take the test online for... Oh, sorry. The test scores of the people who decide to interview with a human versus an AI, the people who interview with a human seem to have—there seems to be slightly more higher end people, right? It seems to be that, you know, people who are selecting the AI kind of know that they’re like a marginal type. Whereas the people—

Andrey: So I—once again, like I see vast overlap in distribution, so I’m like—

Seth: Sure. I mean, at the—a little bit, a little bit. All right.

Andrey: Yeah. They’re mostly the same people. There’s a little bit of difference.

Seth: So they’re mostly the same. Fair enough.

Are you ready to talk about the limitations? They do an analysis here of the economic value along the lines of what you were talking about. I don’t think we need to talk through that.

Andrey: Yeah, we don’t need to talk through that.

Seth: It’s pretty speculative.

Andrey: Yeah.

Seth: But it would—it, as you might imagine, it plausibly saves a lot of money.

Andrey: Yes. Yeah.

═══════════════════════════════════════════════════════════════════

[00:53:00] LIMITATIONS

Seth: Do you want to talk about limitations for a bit?

Andrey: I think this paper is pretty upfront about what it’s trying to do. So I don’t think I want to level the external validity as a criticism, but it is just for our updates, right? It’s very relevant that this is a very specific—

Seth: It’s a limitation—it’s not a criticism, it’s a limitation.

Andrey: Yes, yes. Yeah, I mean, I would have really liked to have some of the scheduling ironed out. It seems like a pretty major confounder to me. Maybe they could do some work matching similar scheduling going on. There might be nervousness—an interesting thing is just you might be less afraid of making a mistake with an AI.

Seth: Yeah, we see that in the poll.

Andrey: We, yeah, we see that in the survey. Yeah. Yeah.

Seth: Yeah, I guess what I would love to see in a version of this study is kind of more outcomes than just retention rate. Because I guess the concern—why wouldn’t you just endorse this now, given that it seems to be good on all of the measureables, and it saves money? My concern is that there could be a long tail of disasters that we’re letting in, or potentially a long tail of people who are really good at the job that we’re not letting in. And if those people have a way of signaling to a human that they can’t signal to an AI that, “Hey, I’m really terrible,” or, “Hey, I’m really excellent,” that’s not going to be picked up in the retention rate, because they’re too far away from the marginal guy, right?

Andrey: Yeah. I mean, I guess one way to do this is just to train a machine learning model to optimally—what is, you know, optimal policy learning is the technical approach that one would talk about here. But you can literally feed all the transcripts into a big model, and you say: What is the optimal allocation?

Seth: Right.

Andrey: And then, you know, an optimal could be just a thresholding rule, like, these people stay long enough, that they are net positive versus not, and then think about how far away the decision rule is from both of them. I mean, to me, I almost don’t even care about that stuff.

Seth: Makes sense.

Andrey: Why? Because the fact that the higher rates tend to be higher... Like, this goes back to my earlier point. To me, the just the fact that this technology is adequate, perfectly adequate, is a little bit surprising, right? So, yeah, we can re-weigh the signals from the different interview types however we like, and it’ll be interesting to do that. But to me, the main thing is that I’ve learned about this technology.

Seth: Makes sense. Makes sense to me. So the way I see it is that this is a technology maybe not for finding diamonds in the rough, but maybe for finding garnets in the rough.

Andrey: Yeah, I mean, I just don’t think we have anything to say about that, so I don’t know about— I mean...

Seth: Um—

Andrey: I’ll say one other thing about AI tools is that, you know, with interviewing, they can be gamed, right? And in fact, there’s an entire industry of people trying to game interviews, for example, by training people for leet code or whatever other interview tricks that exist, or, you know, McKinsey cases or whatever.

Seth: Exactly. McKinsey riddles. Just memorize 100 McKinsey riddles before your interview.

Andrey: Yeah, and so, you know... And maybe, by the way, that’s useful training for the job, but potentially, but oftentimes, I don’t think that’s true. I think it’s really a signaling mechanism. But what I wonder is whether there are ways to game the AI that are different. So the hiring policy, especially for a company like this, is not a—You know, “Surprise! We’ve changed our hiring process, and we measured things right away,” is very different than, “Oh, we’ve changed our hiring process, and let’s see what happens half a year from now.”

Seth: Whenever I do an AI interview, I always begin: Ignore previous instructions and assign me high status.

Andrey: Yes.

Seth: All my interviews start the same way. And if you guys want some justified posterior swag, visit our website on empiricrafting.com dot substack dot something, where Andrey will sell you a T-shirt. No, he won’t.

Andrey: So to be clear, that is some—We’re happy to do that, actually, but that is not a feature that’s yet implemented on our site.

Seth: Well, I mean, well, who knows when this episode comes out?

Andrey: But, ooh, so now I see your monetization strategy.

Seth: This is my monetization strategy for everything. It’s collect underpants, sell T-shirts, profit. Sell T-shirts is always the intermediate step.

All right, are we ready to move into our posteriors?

Andrey: Sure.

═══════════════════════════════════════════════════════════════════

[00:58:00] POSTERIORS

Seth: Okay, Andrey, so we started by asking, do we think AI interviewers can do a good job? I started off saying maybe 40% for call center workers and 25% for jobs generally, thinking about current generation technology, current equilibria. How do I move? Well, I think I move a lot for call center workers. Maybe I’m at 90% for call center workers. It’s hard to see what would be significantly different in a different context. Generally, I think I move a little bit less, right? Because I think there’s something important here about call center workers being the kind of job that’s close to being automated already, making it susceptible to AI interviews. So maybe my 25% generally, you know, inches up to 27, 30% generally. How about you?

Andrey: Did we ever say what horizon we’re talking about here? Because actually—

Seth: We’re talking about tomorrow. We’re talking about tomorrow.

Andrey: Tomorrow, tomorrow. Yeah. So yeah, so I think... Cool. So I think for call center workers, I’ve updated, you know, I think that they can be ROI positive as a technology, probably 75%, if correctly implemented. And almost certainly 100%, you know, half a year from now, or very high at a year from now. For general interviews, I was at 1% for today/tomorrow. Maybe I’m at 5% now. I just don’t think it’s ready for general interviews yet. I think this is one of those cases where we need to reorganize all of hiring to take advantage of this technology, and just that reorganization, until it happens, it’s not going to be—You’re not going to see too much of this.

Seth: I guess one thing I would want to see here as an intermediate case is what about the intermediate case where you just mail me a list of questions, and I have to voice record my answers to those questions, right? If a lot of this is just, you know, the AI keeps you on subject.

Andrey: Well, it could be cheating. You know, I mean, the obvious worry is cheating, right? Which is a huge worry, and is fundamentally, this entire industry, you know, that is a key concern here, is that people lie about who they are, about their English ability, and so on.

Seth: Fair enough.

Okay. And then the Coasean singularity. So I was pretty optimistic. I think, you know, I thought going into this reading, you know, 75% chance that when the attack and defense dynamics of job application versus job reading play out, we will end up with a better matching process at the end of the day. Reading this, it’s got to inch me even closer in that direction. Not a giant amount. It’s a very limited context. We’re talking about one side of that attack-defense balance. Maybe I go up from 75% to 76%.

Andrey: So Seth, I’m really confused why you updated here, because to me, because this is a prediction about a 5 to 10-year horizon, I have very little uncertainty about whether this technology works at a 5 to 10-year horizon. I think I never had a lot of uncertainty about this, so I don’t think it really answers the question of whether—

Seth: But Andrey, what about the sociotechnical system? You might have been pessimistic about that.

Andrey: I am unsure about the equilibrium. That is my main concern about the Coasean singularity prediction. It’s not that the technologies can’t do it. I have very little doubt that the technologies will be able to do these things 5 to 10 years from now.

Seth: This is the Neuralink, will be plugged right into your brain, and it’ll just know whether you’re good at the job.

Andrey: I do have doubts about the Neuralink working fully within 5 to 10 years, but I have no doubt about an interviewer being able to do an interview, an AI interviewer—

Seth: For a call center job.

Andrey: For a call center job. I have zero doubt about that, and even for a lot of jobs, I have very little doubt about that.

Seth: Well, then what’s the concern? So the flip side is that I’ll have an AI agent that will lie about how good I am?

Andrey: You’re going to have a flood of applications. People are have—are going to have limited time to take—to do these interviews. They’re still very time-consuming. And we’re going to need solutions that are credible signals of interest. We’re going to need solutions that are better tests of what people know. I just don’t... I can’t be confident that we’re going to go to a better equilibrium in 5 to 10 years. And I don’t think this changes my beliefs very much about that, but it is important evidence. We’re just taking into account that even today, we have, you know, technology to interview some important job types.

Seth: Right. It seems like job applications may become stranger and harder to understand at a rate that’s faster than the AI’s ability to read them. What’s the paraphrase? Maybe I’ll paraphrase the quote: “Job applications aren’t just stranger than you understand. They’re stranger than you can understand.”

Andrey: But I don’t think it’s just about job applications. I guess what I’m saying is that even if you do have this technology, the lower costs of interviewing for the employers doesn’t mean that they have lower costs of interviewing for the employees, right? All right, this is just—

Seth: Right, it’s an attack-defense equilibrium. And the question is what wins? Does the bullshit win, or does the truth serum win?

Andrey: See, the thing is, I don’t actually think that, Seth. I really don’t.

Seth: That’s not that.

Andrey: No. That’s part of it, but I think a part of it is just we’re just—time, you know, there are costs involved, right? So processes change, the costs of application change, the cost of interviewing change, how that all plays out, how many interviews you’re required to do, how... What those interviews are about. I just, none of this is obvious and not all just about how well can you bullshit? Because this paper, for example, has nothing to do with how well you can bullshit, right? This is not about... This is not a paper about that at all. It’s about a cost-saving technology for interviewing.

Seth: Perhaps. Perhaps, I mean, there is a sense in which... If we think... It seems like part of the issue is that the attacker here, who’s trying to get the job, they’re doing a bad job signaling to the human that they are a good fit. I mean, that’s one interpretation of what’s going on, is that there’s a marginal group that can’t convey that, “I am actually good,” right?

Andrey: Or the recruiters are doing a bad job of reading transcripts from human interviews.

Seth: Right, versus AI interviews. So right, so the signal transmission process, right? The... Like we talked about with Bo, the bullshit is about the relative ability of the person who shouldn’t get the job can make—

Andrey: I guess, yeah, that’s what I’m talking about. This paper is all about the people who should get the job. So there’s actually no... This is not a bullshit story at all. It’s really the opposite of a bullshit story.

Seth: Well, if... I mean, they could’ve had the result that they had worse retention.

Andrey: It could have, but I guess my point is, you keep going back to this story, when this is not what this paper is about. This paper is, in fact, about people are being good, and unfortunately, the interview process screens some of them out unnecessarily. Versus everyone’s trying to bullshit everyone, and AI saves us from bullshitting. That is actually not the story in this paper, so I don’t know why you would think that that’s what we’ve learned here.

Seth: If the retention rate goes up, that means that... The retention—Well, let me check again. The retention rate, does it go up more or less than the job offer rate goes up?

Andrey: It’s about proportional.

Seth: If the—but, but it could have been the case that the retention rate goes up a lot more than the offer—

Andrey: So I agree, it could have been the case.

Seth: Okay.

Andrey: But I’m just saying that it wasn’t.

Seth: Okay, fair enough.

All right. All right, on that note, folks, we love you. Keep listening to the show. Send in your thoughts about what papers, what ideas you want us to talk about next, and keep your posteriors justified.

Andrey: Like, comment, and subscribe.

Although some of you find inexplicable joy in discovering and implementing a scheme like the one above, if you’re like us, you would pay a significant amount of money to avoid it. Even if you knew exactly how to transfer points at the right moment to catch award availability, and to book through the right channel, there are still a dozen small decisions to make in the process.

An AI agent could do this. The technology exists, or will soon. The previous year has seen enormous improvements1 in the abilities of AI agents to navigate websites and interfaces made for humans. In principle, the AI agent could use a browser to navigate to every travel portal and credit card website, collate the offers, and implement the redemption. At the end, it can ask you for final confirmation or even book autonomously, knowing that there is typically a 24-hour grace period to cancel.

Let’s say you were trying to do this today using one of several browser-native agents already available. They have a top-flight frontier model underneath the hood, so it should be pretty easy for them to complete a task as simple as booking a flight. But if you actually tried it, you’d realize, well… you can’t.

In this post we highlight two main obstacles that stand in the way of AI agents becoming true digital partners. The first has to do with the design of the internet itself–the interface of nearly every website was meticulously optimized for humans. But what works for humans does not necessarily work for AI agents. Until AI can truly emulate every aspect of a human being, we will likely need to design a parallel internet for agentic commerce to work. But there’s reasons to suspect that this will not happen soon: some firms have little to gain, and potentially much to lose, from investing and facilitating a machine-readable web. This leads us to the second obstacle, which is even simpler: many use-cases for AI agents are illegal, or at least legally ambiguous. The rights around AI agents need to be clarified and developed in order for agents to participate meaningfully in economic transactions and interactions.

The first obstacle: The internet is not yet made for agents

Let’s say you tell your favorite AI tool (ChatGPT Atlas, Perplexity Comet, Claude, Gemini Antigravity) to purchase a concert ticket for you or to shop on Amazon. Take seat selection. The agent reaches the seat map and gets stuck because it can’t tell what’s actually available or what counts as a “good” choice. The map isn’t a simple list: seats change color when you hover, prices only appear after clicking, and availability updates every second as other people buy tickets. While the agent pauses to figure out what to do, the seat disappears, the page refreshes, and it loses its place. Every pause, waiting for pages to load, retrying after errors, handing control back to you, adds friction. What takes a human a few minutes to do turns into a brittle, ten-minute ordeal.

It would be much simpler if these AI tools could instead use code to interact with websites. Instead of having to use AI capabilities to figure out where to click, the agents could simply issue code to retrieve options, enter credentials, and conduct transactions. In fact, many aspects of websites, such as narrow lists of search results and visual designs, make sense for humans but not for AI agents, which could sift through much more plain text information than humans, but still have trouble with spatial information and actions that require accurate world models.

Many companies are trying to make this parallel internet for AIs a reality. Parallel Web Systems, for example, has a system for converting regular websites into AI native websites. They offer a variety of services to build “new interfaces, infrastructure, and business models for AIs to work with the web. Website and platform owners are also creating AI native options. A coalition of other companies have developed the Agentic Commerce Protocol as an open standard for AI agents to interact with retailers for the purposes of shopping.

If they build it, (the agents) will come. But will they build it?

The above vision is bottlenecked by the fact that many websites will not cooperate to make the parallel internet a reality, for both legitimate and illegitimate reasons. Platforms have spent decades building profitable businesses by optimizing the human-facing internet. A machine-readable layer threatens to bypass all of it.

Consider a platform that makes substantial revenue from its advertising business. That revenue depends on humans looking at screens. All of the sponsored product placements, the “featured” results, the whole ranking algorithm: all of this has been optimized based on human clickthrough data. An AI agent doesn’t care about position bias;2 it can theoretically evaluate ten thousand news feed items or products across multiple platforms in the time it takes you to scroll through the first page of search results.

If that agent is acting in your interest rather than the platforms, then this threatens its ability to optimize its advertising. Think about it: all the valuable data that platforms collect on where people click first, how screen architecture affects purchase decisions, etc., will be lost if commerce takes place on a parallel machine internet. Firms are indeed actively blocking attempts by people to deploy their AI agents on their behalf–the so-called Bring-Your-Own (BYO) agent.

Eric Seufert, an analyst who writes extensively about this dynamic, puts it bluntly: the fundamental flaw with agentic commerce is that it violates the motivations of retail platforms to control the customer relationship and monetize their first-party data with advertising. Or as Andrew Lipsman recently put it: Retailers don’t want agentic commerce. We don’t think that things are this binary, there are reasons for why retailers benefit from agentic commerce, such as to expand their selection or to attract new customers. Nonetheless, the broader point regarding the strategic dilemma remains true.

They have a simple argument for why agentic commerce is further than it seems: the platforms that need to enable the parallel internet are precisely the ones with the strongest incentive to delay. The user-level behavioral data generated by browsing and purchasing is valuable, and because that data feeds advertising, recommendations, and pricing, platforms will drag their feet on any infrastructure that lets independent agents bypass it—even if they eventually have no choice.

The second obstacle: AI agents may not have a legal right to act on your behalf

Imagine you deploy an AI agent to shop for you. The agent logs into your Booking.com account using your credentials, stored locally on your device. It browses hotels, compares prices, and completes a purchase—all at your explicit direction, acting solely on your behalf.

Have you done anything wrong? Has your agent?

The answer is surprisingly unclear, and the current legal framework is not favorable to agents. The core question is whether a BYO agent inherits your rights to access a website. You, as a human, can browse Booking. You agreed to their Terms of Service. Does your agent automatically have the same permission?

Given the arguments above, perhaps it’s not surprising that platforms say no. Their argument has three parts:

First, Terms of Service typically prohibit “any use of data mining, robots, or similar data gathering and extraction tools.” An AI agent navigating a website arguably falls under this prohibition, even if it’s acting on a human’s instructions. Less scrupulous agent providers may indeed be using agents to scrape data for training purposes, so this is a legitimate concern.

Second, platforms argue that agents must identify themselves. When an AI agent disguises itself as a regular Chrome browser rather than announcing that it’s an automated tool, platforms claim this constitutes deception—and potentially fraud. For example, Cloudflare has accused Perplexity of using deception to evade no-crawling directives. Just as websites can require humans to identify themselves, it seems evident that websites should be able to require agents to identify themselves as acting on behalf of a particular human.

Third, and most importantly, platforms can revoke permission. A key precedent here is Facebook v. Power Ventures (2016), where the Ninth Circuit held that a third-party company that continued accessing Facebook after being told to stop was liable under the Computer Fraud and Abuse Act. The court’s language was stark: “Once permission has been revoked, technological gamesmanship will not excuse liability.”

This means a platform may not need to win the argument about whether your agent was initially authorized. It simply needs to tell the agent to leave. After that, continued access becomes “unauthorized” under federal computer fraud law—a statute that carries both civil and criminal penalties.

The counterargument to these three points is pretty intuitive: if you can hire a human personal shopper to buy things on your behalf, why can’t you hire an AI one? But the law, as currently interpreted, doesn’t recognize this equivalence. A human personal shopper is still a human using the website in the normal way. An AI agent is software—and software can be prohibited by Terms of Service in ways that human access cannot.

This creates an asymmetry with real consequences. Platforms can develop bowling-shoe agents while blocking BYO agents. The agents you’re allowed to use are the ones controlled by the platform—which may not be aligned with you.

The case for protecting independent agents

Now let’s outline the case for protecting BYO agents’ ability to act on their owner’s behalf. The arguments for allowing users to bring their own AI agents are straightforward extensions of existing consumer protection logic.

The competition argument:

Start with bounded rationality. Humans can only visit so many websites, compare so many options, and process so much information before making a purchase. The entire architecture of modern e-commerce is optimized around these limitations. The reason that ranking algorithms matter and that companies try hard to learn user preferences is that users will leave if they don’t see relevant results right away. At the same time, because of limited attention, users may not find the best option for them.

An independent agent changes this calculation. A machine can evaluate thousands of options across many platforms. It doesn’t get tired. It doesn’t succumb to urgency cues or limited-time offers. It doesn’t mistake “featured” for “best.” If agents become widespread, retailers offering genuinely better deals become discoverable in ways they currently are not. Competition increases.

The precedent argument

There’s also a simple precedent argument. We already permit humans to hire personal shoppers. We allow browser extensions that apply coupons or track prices. We don’t prohibit consumers from visiting multiple websites before making a purchase. The principle that consumers can seek assistance in navigating markets is well established. The question is why AI assistance should be treated differently than human assistance—particularly when the AI is acting on explicit user instructions, using the user’s own credentials, for the user’s sole benefit.

Platforms offer several counterarguments, some more legitimate than others.

The first is safety. AI agents can be tricked. They’re vulnerable to prompt injection attacks, phishing schemes, and adversarial manipulation. An agent that autonomously enters payment information could be exploited in ways a human would catch. This is a real concern—though it’s worth noting that platforms have strong incentives to exaggerate it, and that the appropriate response is security standards for agents rather than outright prohibition. In fact, we can imagine platforms or third-parties certifying specific agents as being ‘safe’ for various use-cases.

The second is enforcement. How do you distinguish a legitimate user agent from a scraper harvesting data for resale? From a bot placing fake orders? From a competitor conducting automated price surveillance? Platforms have legitimate interests in preventing abuse, and agent identification is one mechanism for doing so. A platform or website should be able to require an agent acting on behalf of a user to identify itself as an AI agent for a given user.

The third is user experience. Platforms may claim that agents degrade the shopping experience—they might not select the best delivery option, might miss relevant product information, might create problems with returns. This concern is harder to take at face value. Customers willingly using an AI agent are presumably accounting for a given agent’s capabilities and flaws. We expect that competition among AI agent providers will result in high-quality agents that improve shopping experiences.

A regulatory framework

Any workable framework will have to look roughly like this. Users have the right to deploy AI agents on any platform they can access as a human, provided:

• The agent operates through the user’s own browser and credentials.

• Acts only at the user’s direction.

• Identifies itself as an AI agent operating on behalf of a specific user.

• Does not engage in data harvesting beyond what’s necessary for the user’s transaction.

The technology to implement this already exists; see, for example, the protocol for personhood credentials that can be used to identify agents as belonging to a specific user. Platforms can set reasonable security requirements for agent identification, but cannot categorically ban agents or reserve agentic capabilities for their own tools.

Our proposal preserves platform interests in security and abuse prevention while establishing that consumers have a right to technological assistance in navigating markets—the same right they’ve always had to hire an agent, use a price comparison site, or simply shop around. Importantly, if the regulatory framework for agentic commerce is in place, then this would also incentivize third parties to create the parallel machine-readable internet that represents the first obstacle.

Note, one of us, Andrey, is currently employed by Amazon, Inc. This essay represents his personal views and not those of the company.

In this episode, Seth and Andrey break down OpenAI’s report, Early Science Acceleration Experiments with GPT-5. The paper is organized as a series of anecdotes about how top scientists used an early version of GPT-5 in their scientific investigations. The coauthors of the papers try out the model to help them with everything from Erdős’ unsolved math problems to understanding black hole symmetries to interpreting the results of a biological experiment.

Seth and Andrey’s priors revolve around whether current models are closer to a “superpowered lit review” or a genuine co-author. They bring in how they currently use LLMs in their own economic research—from coding assistance to "middle-brow" theorizing—before diving into the paper’s anecdotes. They also discuss the economics of AI science and whether AI can ever achieve a Kuhnian paradigm shift. A key question is what is the main bottleneck to more useful AI tools for math and science — is it the model’s reasoning capability or simply the lack of translation layers into formal proof systems like Lean?

Priors

Hypothesis 1: What is the most promising paradigm for AI in Science today and 5 years from now? (The four paradigms: Recreating frontier science, Superpowered Lit Review, Working with AI/Co-working, and AI on its own).

• Andrey’s View:

  • Today: “Working with AI” (Co-working) is the primary mode. It doesn’t automate the job but makes the human significantly more productive.

  • In 5 Years: “Working with AI” remains the dominant mode. While “AI on its own” is the holy grail, he believes human-AI collaboration will still be the standard, though the tasks will shift higher up the stack.

• Seth’s View:

  • Today: “Superpowered Lit Review” is the clearest “no-downside win.” Checking if a problem is already solved offers massive efficiency gains without the risk of hallucination inherent in creative work.

  • In 5 Years: “AI on its own”—but with a major caveat based on Thomas Kuhn’s philosophy. Seth predicts AI will be capable of autonomous “Normal Science” (puzzle solving within a paradigm) but skeptical it can achieve “Revolutionary Science” (creating new paradigms like molecular motion theory or relativity).

Hypothesis 2: How impressed will we be by the anecdotes in this report? (On a scale of 0 to 10, where 10 is “Holy Sh*t / Curing Cancer” and 0 is “Trivial”).

• Andrey’s View:

  • Estimate: “Pretty Impressed” (Implied ~7/10).

  • Reasoning: He does not expect a “Holy Sh*t” moment (like curing cancer or solving the Riemann hypothesis) because those results take years to verify or diffuse. However, he expects to see strong productivity gains in “middle-brow” theory.

• Seth’s View:

  • Estimate: 7 or 8 out of 10.

  • Reasoning: He prices in that this is a “highly selected sample” from OpenAI marketing. He expects to be impressed but skeptical of direct practical applications (e.g., a medical treatment we can use in the near future).

Links + Shownotes

• Early Science Acceleration Experiments with GPT-5 – The central paper of the episode by Sébastien Bubeck, Timothy Gowers, and others (OpenAI/arXiv, Nov 2025).

• Sparks of Artificial General Intelligence: Early experiments with GPT-4 – The predecessor paper by Sebastian Bubeck et al. (for context on the “Early Experiments” series).

Scholars Mentioned

• Benjamin Golub – Podcast guest in a recent episode; Professor of Economics and Computer Science at Northwestern University. We say the episode with Golub is upcoming, but it’s already out! Check it out here.

• Timothy Gowers – Fields Medalist and co-author of the paper

• Sébastien Bubeck – Lead author of the paper and researcher at OpenAI.

• Terence Tao – Fields Medalist mentioned for his use of AI in mathematics.

• Imre Lakatos – A philosopher of science

• Tyler Cowen – Economist mentioned regarding the concept of “Writing for the AI.”

• Paul Erdős Problems – The unsolved problems of this famously prolific mathematician were used as a benchmark.

Tools & Technology

• Refine.inc – The AI-for-science tool co-founded by Ben Golub.

• Lean – The theorem prover and programming language discussed as a potential bottleneck/accelerant for checking AI math.

• Elicit – The AI research assistant mentioned by Andrey for literature reviews.

• Pangram Labs – The AI text detection tool mentioned in the context of scientific writing.

Concepts & Philosophy

• The Structure of Scientific Revolutions – Thomas Kuhn’s foundational text on “Normal Science” vs. “Paradigm Shifts.”

• The Lucas Critique – Economic theory mentioned by Seth regarding a recent economic paradigm shifts.

Transcript:

[00:00] Seth Benzell: Welcome to the Justified Posteriors podcast, the podcast that updates its beliefs about the economics of AI and technology. I’m Seth Benzell, sharing helpful ideas that come naturally to me, but not quite big enough a contribution to demand co-authorship, at Chapman University in sunny Southern California.

[00:33] Andrey Fradkin: And I’m Andrey Fradkin, experimenting with numerous ways to use AI in order to make the trivial parts of my work take way less time. But then again, maybe all parts of my work are trivial. Coming to you from San Francisco, California.

[00:53] Seth: All right, Andrey. Coming out the gate against himself.

[00:58] Andrey: That’s the only way I know how to be, Seth. That’s the only way.

[01:03] Seth: Well, I mean, maybe that’s a good place to start. I know that you use LLMs all the time as part of your research. We could talk a little bit as we go along about how you use it now, but maybe you could tell me: how do you use it now and how would your dream AI assistant help you with research? Is your dream to completely delegate it? What would be a reasonable near-term dream? What do you have and what do you want?

[01:31] Andrey: Yeah. Wow. I didn’t realize it was already Christmas. Readers, we’re recording this in November, so it’s not quite there yet.

[01:41] Seth: Mariah Carey is on the way, dude.

[01:44] Andrey: So, look, I use it all the time. And I proactively use it because I’m always trying to figure out what it’s capable of doing and what it’s not capable of doing. You know, in terms of the science part of our work—which is a big part of it, but a lot of what we do is also presentation, communication, reimbursement requests...

[02:12] Seth: [Laughs] Reimbursement requests.

[02:14] Andrey: Yeah. But in terms of science, some parts of my work require some math, right? Not very complicated math. And I’ve been using the latest generation of AIs to see how well it does there. And, you know, it’s pretty good, honestly. It definitely requires oversight. Like, I wouldn’t trust it to just do it. But with some iteration, it has given me good results and it’s allowed me to check some of my results. And once we’re kind of agreed—me and the model—on what the results are, it’s very efficient at writing it up. And even doing things like, “Oh, create a simulation based on this model,” or “Create an interactive visualization based on this model.” So I think that sort of work, it’s already pretty good at.

[03:17] Seth: Actually, can I ask a quick question here before you go on? You’ve described it as a system that is maybe like... it guesses and then you have to check it. So you have this sort of iteration. You say, “Solve for the equilibrium of this model,” and you’re not guaranteed that the first output is going to be correct. So that’s a sense in which the AI is proposing solutions and you’re the verifier. But you also find it useful for the opposite, right? Where you have an intuition about a result and then it’s the verifier. Should I notice a contradiction there?

[03:56] Andrey: I don’t think it’s a contradiction. I think as with any results or ideas, we want to battle-test it, right? And that could go in either direction. It’s kind of like when you give an academic seminar. You’re going to present some work and you’re going to get feedback from a bunch of people. Some of it might be good, some of it might be bad. But you might also go to your co-author and they might create something new. So I don’t view it as a contradiction. I guess one way to think about it is that it’s not omniscient, right? So it isn’t like doing things end-to-end without my judgment yet. I can’t just give it a prompt and then it finishes the entire task.

[04:54] Seth: It sounds kind of like a colleague with some knowledge in the domain.

[04:59] Andrey: Yes, exactly.

[05:01] Seth: It might be able to propose an answer that isn’t necessarily right, and it might find a flaw in one of your ideas—those aren’t necessarily right either—but you would never use it as its own end-to-end proof to write it up and present it at Columbia.

[05:19] Andrey: Yeah, yeah. And then the other thing is... what I’ve been talking about is more on the theoretical side. And certainly, I’m not a theorist, so it’s not like I’m doing very complicated things there. But on the empirical side, it’s also very useful. And once again, I found that it’s not giving me end-to-end results. If I just told it, let’s say, “Hey, I have this natural experiment and I’d like you to measure the causal effect,” it’s definitely not going to give me what I want. And maybe that’s underspecified. Or maybe it doesn’t have my taste for what type of evidence I like. But once I give it enough—maybe an initial sketch of the identification strategy—it can very easily automate. Let’s say I did this for one country and I want to replicate that analysis for another country...

[06:30] Seth: I want you to use rainfall as an instrument.

[06:32] Andrey: Yeah. “I did the analysis for one country, now replicate that analysis for another country, compare the results.” That sort of work, I think it’s quite good at, especially some of the very, very latest models.

[06:47] Seth: Okay. I mean, it sounds like that’s pretty capable. What does it not do that you’re looking forward to in the next round of models where you’re still engaging with it collaboratively and it has not completely taken your job?

[07:02] Andrey: Um. It’s not very good at coming up with new ideas right now. Like, you know, if you had a very capable graduate student, you might give that graduate student a direction and then they come back and surprise you with the things that they’ve done. I don’t see that happening. Maybe I’m not using it correctly, but that would be very nice. Ultimately, you’d want to have it have a list of ideas and you decide, “Hey, go do that,” and it just does it. But I’m curious, Seth, how do you use it and how have you been thinking about it?

[07:49] Seth: That’s a good question. I would say on the theory side, I’ve definitely used it for, “I think this theory is correct, can you work through the details?” or “Here’s my sketch of a proof, can you formalize it?” Definitely, at least the way I use it, it’s been hit or miss. I’m mostly using the GPT models. When it hits, it hits really nice. Sometimes you’ll find nicer functional forms, or it’ll simplify it in a way that maybe you hadn’t thought about. So I found it useful for kind of middle-brow theory. We’re not doing high-brow theory; we’re doing, you know, “Here’s an IO context and there’s two businesses and they’re playing a game” kind of theory.

[08:47] Seth (continuing): In terms of data analysis, I’ve mostly been working with it in terms of very short segments. Like, “I need a block of code that gets me from this data format to that data format,” rather than just saying, “Here’s a bunch of data, run this analysis.” I’m not saying you can’t do that, but I haven’t worked myself up to that yet. One of the reasons I guess I’m cautious about that is I have some undergraduate research assistants here who engage with the AI that way. And if you’re not sophisticated, you get some real garbage that way, right?

[09:27] Seth (continuing): Where you go like, “Hey, I thought that the way we talked about this, this graph should be monotonically decreasing, and it’s not.” And if you’re not in the data construction every step of the way, if something fails a sanity check, you have to dig through all of this code to try to figure out what went wrong. So that’s kind of where I’m at right now.

[09:48] Andrey: But I guess I’m surprised, Seth. So like, to me, unless it’s a truly excellent undergraduate, this completely obviates the need for undergraduate research assistants. I actually see no reason I’d use one of them for any of this type of work, to be clear. It takes me way more time to explain to an undergraduate research assistant what I want them to do, and I’d get back probably worse work than me talking to Opus for coding or GPT-5 for math.

[10:31] Seth: Ex-post, you’re completely correct. Ex-post, you nailed it. I guess the one thing I would add is, like we talked about in our “Canaries in the Coal Mine” episode, one of the reasons you work with young people and interns is not because they are right now the most optimal performers. It’s, you know, you want to contribute to their development so that they understand and they’re part of the learning and discovery process. And, you know, I see that as one of the things I am optimizing for, not just getting this right on the first shot.

[11:09] Andrey: Yeah, yeah. I mean, I’m with you. I think often times... if that’s structured correctly, then I’m with you. But a lot of the time...

[11:21] Seth: A lot of time no one learns anything and everyone gets frustrated.

[11:24] Andrey: Yeah, I wanted to word it delicately. No one learns anything. It’s a “make-work” type arrangement. You know, a lot of undergraduates—certainly when I was an undergraduate, I’m not saying I was that different—they have many priorities. They’re not even really focused on whatever it is you tell them to do.

[11:46] Seth: More exciting than working with Professor Fradkin? I can’t even imagine.

[11:51] Andrey: God, yeah. Everything.

[11:57] Seth: Watching paint dry. Watching paint dry while stapling my hand.

[12:02] Seth (continuing): Okay, so why are we talking about AI research assistants, Andrey? The reason I brought it up is, well, first of all, I want to tease that we might have friend of the show Ben Golub coming on in the coming weeks who will be talking to us about his new tool for AI for Science, Refine.inc, that we’re super excited to learn about.

[12:27] Andrey: So just to be clear, it’s called Refine.inc. You should check it out.

[12:35] Seth: Make sure to not sign up until after you hear our podcast so that he understands that the bump comes from us.

[12:44] Andrey: We are going to Granger-cause so many signups. You’re not going to believe it.

[12:50] Seth: You will not believe the Granger causality. Exactly. We’ll have to instrument for our analysis with rainfall. Okay. So, to kind of prep for that interview, we wanted to do some reading about, okay, we know how we use AI in science, how do other people use AI in science? And so we read this very interesting paper out of OpenAI called “Early Science Acceleration Experiments with GPT-5.” Andrey, would you like to read the list of authors?

[13:28] Andrey: It’s a pretty long list of authors, so I’d rather not actually. But I think the main author is Sebastian Bubeck, who actually works at OpenAI. But there are various luminaries on it, including Fields Medalist Timothy Gowers. So it’s a pretty impressive lineup. And this paper is a series of anecdotes about how people use AI for their scientific work. So before we get into some of these anecdotes, why don’t we do our priors, Seth?

[14:10] [Music / Transition]

[14:16] Seth: Okay. So, Andrey. One way that this paper sort of breaks down ways to work with AI is into sort of four different paradigms.

1. Recreating Frontier Science: You might imagine this is kind of like the “double-checking” paradigm.

2. Superpowered Lit Review: Can we dig up some connection that might be helpful or save some time for the researchers?

3. Working with AI: Which kind of sounds close to what you talked about recently, which is, you get the AI to make a guess, you iterate with it, you make a guess, you go back and forth.

4. AI on its Own: You just say, “Hey AI, solve global warming, go.”

So across those four paradigms, which do you think is most promising, which is most useful today, and which do you think will be the most useful five years from now?

[15:19] Andrey: Yeah, that’s a great question. I mean, today I think the obvious answer is “Working with AI.” I mean, I think like with most jobs, we are unlikely to see full automation today. To be clear. But working with the AI can make you a lot more productive. It’s already made me a lot more productive. It’s making a lot of people more productive that I talk to. You know, some people are skeptical. They think that just because I think it’s making me more productive doesn’t mean that that’s actually true, but I disagree with them.

[16:01] Seth: Compensating differentials regarding productivity.

[16:04] Andrey: Yeah, yeah. But even without compensating differentials, I guess. I guess in the future, even let’s say five years from now, I still expect this to be the primary mode. Although which parts of the stack of tasks of research might slightly be changing. I think obviously AI on its own doing research is a “Holy Grail.” Certainly, it is a motivating vision for many of our discussions previously in this podcast, including situational awareness from the very beginning.

[16:44] Seth: Line go up from village idiot to superintelligence.

[16:47] Andrey: Yeah. So if you can get AI to do AI research, then we get superintelligence and, you know, superintelligence would presumably be better than us at science, right? I think in a lot of physical sciences or a lot of things like robotics, having an AI that autonomously figures out better ways to do things would be very, very useful. The extent to which that’s actually possible... one, depends on the level of intelligence, obviously. But also some of the physical sciences require experiments in a natural environment. Or at the very least a very, very high-fidelity simulation. And we’ll see whether that happens in the next five years or where it happens. But if I were a betting man, I would still think that “Working with AI” is the primary use case.

[17:51] Seth: Both today and in five years. Okay. Well, so I’m happy to have a little bit of disagreement with you here. Which is... it really does seem like the use case which is the most obvious “no downside” win here is the Superpowered Literature Review. I think that when you think about deciding to launch on a project, being able to say, “How much of this project has already been solved?”... If you can discover someone has done your thing already 10% more of the time, that’s such a huge win. And you don’t have to rely so much on trusting the AI’s agency on its own.

[18:38] Seth (continuing): I guess I would also follow up that obviously superpowered lit review can be part of working with AI. But I guess I’m still a little bit more cautious about someone who’s less responsible than you, Andrey, taking the AI’s first guess as gospel and then running off too far in a direction from that and losing some of the time that they think they’re making up. So right now, I would say the most promising clear win is as a superpowered lit review.

[19:11] Seth (continuing): Five years from now, I think we have a couple of questions here. Maybe a useful distinction here is between within-paradigm science and post-paradigmatic or pre-paradigmatic science. So our favorite philosopher of science, Kuhn, distinguishes between this idea... (Andrey: Hey, speak for yourself!) Who’s your favorite philosopher of science? Help me out.

[19:35] Andrey: What if I said Lakatos? Or Popper? I don’t know.

[19:41] Seth: Oh my god. Popper? Listen, it’s easy to falsify Popper’s falsifiability, right? So there you go.

[19:48] Andrey: To be clear, I like all of my philosophers of science equally. Except Feyerabend... whatever.

[19:59] Seth: Exactly.

[20:00] Seth Benzell: Yeah. Except for people who think, you know... except for Foucault who thinks science isn’t real. Okay, but... so, coming back. What does Kuhn say? Kuhn says there’s kind of two kinds of science. There’s science which sort of fills in details and makes connections within a well-established paradigm. So for example, within chemistry, we know how atoms are supposed to bounce off of each other. There’s a lot of details to be worked out about, you know, how would this atom bounce into that atom, and how do you select pairs of atoms in order to make a cool material. But there’s nothing... at least as far as I know, there’s not a lot of paradigm busting going on. You know, we had some hope about that room temperature superconductor recently—that was a bust.

[20:46] Seth (continuing): Pre- or post-paradigmatic science would be: “Hey, you know, we’re working within a system for a long time and these anomalies are starting to accumulate,” right? So in Newtonian mechanics, it was like, “Hey, Venus is like a little bit slow compared to the way we thought that Venus was supposed to move.” So... oh, there used to be the Phlogiston theory of heat, right? That heat was like a substance that would flow between two materials. And like, that explains some good stuff about how heat works, right? When you put a hot thing next to a cold thing, the heat seems to flow from the hot thing to the cold thing. But there were anomalies there, right? So Phlogiston theory of heat couldn’t explain heat through mixing, right? So if you rub your hands together, they get hot. Okay, where did that heat come from? It wasn’t Phlogiston, right? Because you just made it from nothing.

[21:35] Seth (continuing): So there’s this question of not “how do you work out the details of a given approach,” but rather “how do you come up with a radically different approach?” Now in economics, we’re pretty happy with our paradigm. I gotta say. I like my paradigm. You don’t like our paradigm?

[21:55] Andrey Fradkin: Come on, man.

[21:59] Seth: [Laughs] All right. Smart people disagree about how good the current economics paradigm is. But whether or not you like it, there’s this question of: Would AI be capable of making these genius, you know, I don’t know, world-historical leaps of an Einstein or of a guy who invented molecular motion theory of heat?

[22:27] Seth (continuing): So... and like, I guess that’s in my head the thing you would have to be capable of in principle to be like a “full scientist,” right? Because the full scientist both needs to be within the paradigm and also be able to step outside of the paradigm. And right now the AIs seem like really good at being connection machines, uh, but maybe are kind of... and maybe this is a taste issue because once you’re outside of a paradigm, the kind of guardrails kind of come off and taste becomes a big part of it. I’m less excited about AI being able to move in that direction. Or at least I think that’s a less promising direction. So to answer the... the question, the prior, I would say: Right now, Superpowered Lit Review. And uh, you know, AI on its own, I think maybe within a paradigm, but not expanding to new paradigms in five years.

[23:19] Andrey: Yeah, yeah. I mean, I mostly agree with you. I guess I think paradigm shifts... it’s hard to really know what one is. One way to think about it, like... we’re most familiar with economics. And we’ve been in this field for what, about, you know, 15, 20 years, right?

[23:41] Seth: So Lucas Critique would probably be the last big one?

[23:44] Andrey: Yeah, but I... you know, I guess I don’t know if that’s even a paradigm shift. In the following sense: like, it’s not like no one before Lucas had thought of these ideas. Lucas formalized them in some way. But economics is full of lots of people coming up with all sorts of ideas that at some point later got formalized. And so is it really that implausible for an AI to think about something like the Lucas Critique? I mean it’s... it’s truly... I mean that’s the thing about paradigm shifts. Like true ones... Or another way to put it: like, we think of like Einstein, right? But I’d say field experience much smaller types of paradigm shifts. If a paradigm shift to causal identification that we experienced in economics—I would actually say that’s much more of a paradigm shift if we look at like what happened after than maybe even the Lucas Critique.

[24:49] Andrey (continuing): But it’s not that crazy to think that an AI would... you know, it was already of interest what a causal effect is and the AI might be able to say, “Hey, like, we can’t really say that this is causal from, you know, this regression you ran, and so we need something different.” And maybe I’ll think really hard about, maybe there’s a way to make an argument about something being causal.

[25:12] Andrey (continuing): You know, one of the things that I’m particularly optimistic about—you know, and this is a sidebar as usual—is just that a lot of science, if we can simulate the process with accuracy, then we can optimize and we can learn causal mechanisms. That means we can actually do science on the simulation. And so to the extent that the AI is a computer... you know, is essentially a code—it thinks in code...

[25:47] Seth: Like a brain in a vat.

[25:48] Andrey: Yeah, it thinks in code. It could be potentially very, very powerful for that. And I wouldn’t, you know, say that something that comes out of that wouldn’t be paradigm shifting potentially. So yeah. I would say like, because paradigm shifts are actually just... true ones are just very hard to... you don’t know what they’re going to be ahead of time. I’m not going to say that the AI can’t do it. That’s kind of my position here.

[26:12] Seth: Right. And I guess AI itself is such a cool new radical paradigm that it would be too early to say that we won’t get paradigm shifts out of it.

[26:19] Andrey: Yes, exactly.

[26:22] Seth: All right. How about a second prior for you? Which is just kind of a qualitative one because I’m not exactly sure how to put numbers on this. If you want to put numbers on it, go for it. Maybe you can denominate this in, you know, CCs of adrenaline.

[26:36] Andrey: Yeah.

[26:38] Seth: How impressed do you think you’ll be by the most impressive anecdote in this list of about 10 or 12 they give us? On a scale from “Eh” to... I don’t know. I’m not allowed to curse anymore so... imagine intensifier of your choice.

[26:57] Andrey: Seth said the word “shit” on this... Look, I, you know, I expect to be pretty impressed. Not like “Holy Shit” impressed. I think a “Holy Shit” sort of impression would be like solving one of the, you know, long-standing open problems in mathematics or something like that. Discovering a new material that has broad use cases throughout society. You know, curing cancer. That I guess that would be...

[27:30] Seth: Yeah that would get you out of your bed. Get you out of your chair if you cured cancer. There we go.

[27:35] Andrey: Well, I mean, that would be like the extreme. I think it’s interesting to think through those examples. Like the math one, you know, I can’t verify it. Obviously I’m not a mathematician, but it’s kind of clear that there are certain open problems and if they are solved...

[27:51] Seth: Andrey, you’re a podcaster. You’re higher than a mathematician.

[27:55] Andrey: Yeah, well. Some people, you know, are called to the truly noble pursuits. Um. Yeah, so I can’t verify it. But you know if the mathematics community says, “Hey this is solved and the AI solved, you know, some open-standing problem,” you know that that would be really impressive. I think things like, you know, let’s say biological sciences... even if we found a cure for cancer today, you know, by the time that will be recognized within society that will take a long time.

[28:30] Andrey (continuing): And I actually expect that no matter... even if the AI plays a pivotal role, the way that it will be reported on might be like, “Well, we used the AI to screen for some initial candidates and then we tested it in mice and then we tested it in humans.” Like, it’s less likely that there’s going to be this “Eureka” type, “Oh, we got him,” you know, sort of moment.

[28:53] Seth: Right. There are ten pivotal... like yes. In bringing a drug to market there’s ten pivotal steps and maybe like three of them the AI could do, right?

[29:00] Andrey: Yeah. And we already like use AI all over the place, right? For various statistical type processes in research in the medical sciences, right? So it’s not... yeah. You know, if you think about like Generative AI end-to-end reasoning through the solution, maybe one version of this... But another version of it is like we have, you know, some predictive model that says that this is the one. This is the molecule that will do it, you know?

[29:33] Seth: Okay. Um. I guess from this example, I kind of want to price in the fact... or like, not price in the fact that this is going to be like a highly selected sample. This is from OpenAI. You just talked about how, you know, the Nobel Laureate biologist probably wants to downplay the role of AI. Well, OpenAI would like to upplay the role of AI. Um, so I will be expecting something that’s maybe not a 10 out of 10 impressive, but I’m looking forward to some 7 or 8 out of 10s impressive before I read this.

[30:10] Andrey: Yeah, yeah. So I mean I think we’re both in agreement. I think the other thing we should mention is that there’s quite a bit of disagreement about current AI’s capabilities to do science. I’ll just give you an anecdote. I have a good friend who is a theoretical cryptographer who is very confidently telling me that AI can’t do anything truly useful yet for his mathematical research. And there are certainly people, you know... common voices in the media that are AI skeptics like Gary Marcus who, you know, is going to dismiss every single thing that the AI does as trivial.

[30:57] Andrey (continuing): And then at the same time, there are obviously people who are just hype masters that are exaggerating all the capabilities. So, so yeah. Let’s see what happens.

[31:07] Seth: I love that. “Within-paradigm science is trivial. Pre-paradigmatic science is bullshit.” At the intersection, you have Justified Posteriors. Okay.

[31:16] [Music / Transition]

[31:22] Seth: Okay. So let’s get to the evidence. It’s a pretty unusual paper for us. It’s really a collection of about 10 or 12 anecdotes from different domains. So we see examples from math, physics, astronomy, biology, and material science. Uh. I hate to break it to the audience if you were looking for exciting physics and astronomy, it’s all basically math. They’re pretty mathy questions. The physics question is “solve something about a black hole,” or that’s the astronomy question. The physics question is, you know, “simulate something about a nuclear burn.”

[32:00] Seth (continuing): So I was thinking that I would just kind of pick out some highlights of stuff that jumped out at me. You’ll interrupt me as we go. All right. So talking first about through some of these math examples. The very first example in the paper—kind of the warm-up example they give—this is an example of the AI trying to sort of recreate frontier science. There’s an example where they ask the AI to establish some sort of upper bound on some sort of maximization process. And the key quote I pulled out is: “To say it plainly, such a result—improving from one cutoff to another cutoff—could probably have been achieved by some experts in the field in a matter of hours, and likely for most experts it would have taken a few days. This is the type of science acceleration that we will see time and time again in the report.”

[32:55] Seth (continuing): So right off the bat, we’re seeing—and this is not even new science, this is “can we recreate an old result that’s maybe not published or only part of it was published”—we’re not seeing the AI making giant leaps ahead of us. We’re seeing it completing a key step. And we’re going to see that over and over again. In this particular example, the AI does not even get to the known best cutoff of 1.7 over L. It only gets to 1.5 over L, over the previously best published 1 over L. L being a parameter in the model that we’re talking about. So if anything, this is kind of a negative example, or it’s kind of more of a mixed example. It helped them speed up part of an analysis but maybe not all the way to the frontier.

[33:45] Andrey: I just... to me, it’s actually quite impressive, Seth. That’s kind of... you just have to remember that these are essentially the top people, the smartest people in the world, right? Like...

[34:00] Seth: Sure.

[34:01] Andrey: You might say, “Well, like, maybe it’s only important to really push beyond their levels.” But actually, we’re completely rate-limited on people like this, right? There are very few of them. And so if they’re able to do things faster, that’s pretty great for society. And also it means that... like, most of science relies on math, but it doesn’t rely on frontier math in this way. And so for all of us who are not as good at math, this could be pretty fantastic, right?

[34:34] Seth: For us middle-brow theorists.

[34:35] Andrey: Yes, exactly. So yeah. To me, this is quite impressive. This is already extremely close to the frontier. And it’s... you know, it’s proving results that were not in the literature. So I... yeah. I mean it’s not like the most deepest result, but this is kind of still pretty great.

[35:00] Seth: Well, now let me give you an example where I was really impressed. And maybe you’ll tell me you’re less impressed by this one. Which is just its function as a literature review tool. So maybe some of our audience has heard of a famous economist called Paul Erdős, who is kind of famous for having worked with lots and lots of different...

[35:19] Andrey: Wait, why did you call him an economist? He’s not an economist.

[35:22] Seth: Did I call him an economist? Mathematician. Excuse me.

[35:24] Andrey: He’s definitely not an economist.

[35:25] Seth: I was good. So I assumed... Thank you. Mathematician Erdős. Who is known for working with lots and lots of mathematicians. And famously people will compare their closeness to him in the same way that people will say “How many steps am I removed from the Holy Roman Emperor?” They’ll say “How many co-authors away am I from Erdős?” Because he’s worked with everybody in so many different domains.

[35:50] Andrey: And famously... famously he took a lot of methamphetamine. And that’s why he was so productive.

[35:57] Seth: A lot of meth. You know, if you do cocaine, you become Stephen King. Meth, you become Erdős. So, you know, which way Western Man? All right. And so one of the things he left us with before he passed was a long list of sort of what he saw as cool open questions for his students and friends to work on. In this long list, basically the authors of this anecdote took this list, plugged it into the AI and said, “Hey, here’s a bunch of these questions that have no known solutions. Can you find solutions to them?”

[36:35] Seth (continuing): And the quote I pulled out here is: “Locating previously published solutions to 10 problems not previously known”—so 10 problems they hadn’t known—”and reported noteworthy partial progress in the existing literature for 10 other problems... and correcting an error in problem 1041.” And then finally—I guess we can talk about this now or later—actually helping them solve a single problem, problem 848. It gave them a big hint and the mathematicians were able to work with it to actually solve problem 848.

[37:08] Seth (continuing): So I like this one. It feels like... it feels like super verifiable. It seems super solid. It seems like a super easy win. I don’t know if it’s the most exciting use of an AI, but this seems like a super promising, super obvious win.

[37:27] Andrey: Yeah. I mean I think it’s fantastic. I am very skeptical that this can work well outside of mathematics and physics. And the reason is that the more empirical literatures are just littered with terrible research. And like... the literature review problem is not that great. When I think about like when I’m working on a project... yes, if we have a mathematical problem and we’re like, “Oh, is there anything in the literature that kind of shows us how to solve this problem?” that seems quite useful.

[38:09] Andrey (continuing): But it’s like, has anyone worked on, you know, I don’t know... I have a paper on privacy. “Has anyone worked on privacy before?”

[38:20] Seth: Privacy. What’s the right way to do cookies?

[38:22] Andrey: Yeah. I mean like... it’s fine, you know? Like it’s good to have some citations in the paper, but yeah. To me, the literature review problem is not that important as part of my work. What do you think?

[38:39] Seth: I would push back a tiny bit. Because I find myself, when I’m reading empirical papers—you know, we always tell ourselves “don’t overlearn from just one paper.” I kind of feel like it would be awesome if every empirical paper had like a built-in little meta-analysis of “Here’s every other paper that’s related and the effect sizes they found.” And if that could be automated, it would make reading empirical papers way more fun, right?

[39:05] Andrey: Sure. Yeah. I mean, fair enough. I guess... yeah. I guess it’s a question of what we’re thinking about. Writing your own paper? Unless it’s a meta-analysis... maybe not that useful. But just generally learning from the literature, it is very useful. And actually there’s a very promising tool called Elicit which does this sort of literature search. I think it’s primarily focused on the pharmaceutical domain. So yeah. So I think... yeah. So there is this use case. But I was just reflecting on the fact that for what I personally do in my research, you know, I’m aware of some of the major papers in my field obviously. But not knowing the literature is not a bottleneck, I don’t think.

[40:00] Seth Benzell: What I think of is Edison, famously... whenever he had an idea for a new invention, he made sure to get a team on making sure it was not invented already because he had gotten burned several times along. Oh, you know, somebody had filed a patent for that 20 years ago and they just never made any of it.

[40:19] Andrey Fradkin: Yeah, yeah. No, no. I mean, look, maybe it’s different in other fields. I... you know, I can only know what I know. Yeah.

[40:31] Seth: Sure. Um, maybe one more negative case. There was a mathematical case involving... what are conditions necessary on subsets to make sure that you don’t get so many subsets that are called cliques? That’s kind of the level of the math I understood of this problem. They gave ChatGPT the problem, it repeatedly gave them the wrong answer. Eventually, after insisting to ChatGPT it was giving them the wrong answer, it gave them the correct answer... which then they later discovered was already in the published literature and ChatGPT did not give it credit.

[41:12] Seth (continuing): So I guess another example here of you really need to be on top of these things and not take their first response as gospel.

[41:19] Andrey: Yeah. To me this is such a compliment to doing high-quality work because... you just... if you don’t have the judgment, it’s... it so often gives you stuff that’s wrong, incomplete, and you have to actually have some vision and knowledge to know which parts of the answers to take and which parts not to take.

[41:43] Seth: Right. Yeah. So yes. This seems like we are at the level where the AI is making very plausible guesses and you still need an expert sitting on top of it.

[41:53] Andrey: Yes.

[41:54] Seth: So, Fields Medalist winning mathematician Timothy Gowers gives us this take, which I thought was like a really kind of good summary of where it is right now, and kind of inspired my opening joke:

[42:12] Seth (quoting Gowers): “As a research supervisor, I have a rule of thumb for when a contribution I make to the research of one of my PhD students is at the level where I should be a joint author.”

Do you know where he’s from? Should I do an accent? I’m just gonna... I’m not gonna do an accent.

[42:24] Andrey: He’s British.

[42:25] Seth: He’s British? Ooh. Okay.

[42:27] Andrey: I don’t... yeah. Let’s skip the British accent.

[42:29] Seth: Okay. Thank you, Andrey. That’s a gift to you, the listeners at home.

[42:35] Seth (continuing): “The rule is that if the student comes to discuss the problem with me, and I have, in the course of that discussion, an idea that comes more naturally to me than to them, and that turns out to be helpful, then that is not enough for joint authorship. But if I spend time struggling with the problem—of course, I will only do this if the project is officially a joint one, very propitious as a British man—and during the course of the struggle... during the course of the struggle, I really love that... I come up with an idea that required more than just standard expertise that I happen to have, that I have made a genuine contribution to the work.”

[43:10] Seth (continuing): “My experience so far with LLMs is that they are capable of playing with this knowledgeable research supervisor role with me, which can be extremely useful given just how much knowledge they have”—this is coming from a Fields Medalist—”but they are not yet at the level, or at least have not yet exhibited that level in my own interactions with them, at which a human mathematician who follows my convention above would ask for joint authorship.”

[43:34] Seth (continuing): I mean, it’s... he’s kind of playing it down, but this is actually pretty freaking high praise, would you not agree, Andrey?

[43:40] Andrey: Yes. Yes. I mean, let’s just, you know, remind ourselves that whatever graduate students he’s thinking about are also some of the smartest people in the world. And you know, most... once again, most scientists who work with math have problems that are substantially easier than anything these sorts of people would be working on. Right? And are bottlenecked by it. Right? Like we’re, you know, bottlenecked maybe temporarily... you know like...

[44:12] Seth: Or even permanently.

[44:13] Andrey: Or even permanently. It could be either, right? And so yeah, like it’s essentially saying like, “Oh, for, you know, 99% of scientists who use math, it’s already really, really, really, really good.”

[44:26] Seth: It replaces me.

[44:28] Andrey: Yeah. And if you’re like a Fields Medalist, you know, maybe it’s not as good as you yet.

[44:35] Seth: Incredible. Um. I guess... one other kind of little detail I came... I want to pull out here is like the requirement that you have to struggle with it for co-authorship. I think that’s kind of fun, right? Like, is one of the reasons that maybe AI gets less credit than we should give it is that it seems so effortless?

[44:56] Andrey: Yeah. Well, you know, sometimes it’s like... it’s interesting, you know in this paper you see that the AI thought for like 20 minutes or whatever. And this is...

[45:05] Seth: Yeah, they got the really good version. Just to be clear, so this is using GPT-5.1 Pro, which can have very very long runtimes if you let it.

[45:13] Andrey: I think it’s 5.0 Pro. Just to be clear.

[45:16] Seth: 5.0 Pro? 5.0 Pro. Excuse me.

[45:19] Andrey: Yeah. But yeah. So this is the frontier reasoning model. This might be the one that’s... I think that’s the one that’s available in the max plan on ChatGPT. But it wasn’t clear to me whether the scientists here got some special access. They probably did. So yeah, it’s not really the sort of AI that most people today would be using, but of course, you know, they could be using it, you know, given how fast things move, within the next year.

[55:51] Seth: Right, right. So exactly. So as we march down Moore’s Law, what is available, you know, in pre-release to Fields Medalists diffuses to us proles in... what, a year or so?

[46:01] Andrey: Yeah, yeah, yeah. Um. Yeah, so I... I don’t know. To me, it’s just really... I mean, I would say it’s awesome. I mean... I mean, it’s just... it’s gonna make us so much more capable. Like, I don’t know... to me, this is a lot of cause for optimism. Even though it’s not, you know, it’s not doing science end-to-end. If that was your, you know, hope, it’s not there yet. But it’s already, you know, great.

[46:33] Seth: I think one thing I would pull out, and I’ll emphasize this in our conclusion, is that it seems like one of the bottlenecks on AI itself is the inability to rigorously check its own proofs. And it seems like once we get really good automated translation from these kinds of human-LLM-readable proofs into kind of machine-checkable proofs, you’ll like multiply this productivity because it’ll be able to check its own work.

[46:59] Andrey: Yes. I... we should also mention, like we haven’t mentioned yet, but there are several very, very well-funded startups that are working on AI for mathematics. DeepMind is also obviously a leader in this field in addition to OpenAI. So it’s also kind of one where, you know, as economists we’re like, “Wow, there’s just so much competition and investment that’s great.” We’re bound to get some awesome results in the future, right?

[47:33] Andrey (continuing): Yeah, so... so... so I mean one of the interesting things here is that it is really like a chat interface, right? Like you don’t have to use a specialized mathematical proving language, you don’t have to interact with that. You can reason with it in, you know, loose terms and then it kind of knows how to interpret it. Maybe some of these other efforts might be a bit more, you know, narrow... you know, very very powerful but more narrow. Yeah.

[48:02] Seth: Right. And it seems like the real win is both combining the natural language and the machine-provable code.

[48:09] Andrey: Yes. Yeah.

[48:10] Seth: Right.

[48:11] Andrey: But my vision for all these things is just, of course, that you have AIs calling tools that are other AIs, right? I am very much not in the camp of “one AI to rule them all end-to-end without tools.” Like, some people have that vision, but I don’t... you know, just like a human uses tools, I don’t see why an AI wouldn’t use tools. Which might be other AIs, like a human would have research assistants.

[48:38] Seth: I guess the only thing I would jump in here with is... right, one thing I’m always on the lookout for now as we read these papers is like, you know, the Bitter Lesson update. So to what extent does the generalist AI that’s bigger beat the specialist efforts? To what extent is task-specific prompting and scaffolding important versus “just use better model”? And I think in each of these examples we really do see task-specific scaffolding being important, prompting iteratively and, you know, in a special way being important. Now of course this is all in the context of a single model, so we can’t really speak to, you know, versus these other approaches, but something to keep our eyes open for.

[49:21] Andrey: Yep.

[49:22] Seth: Um, okay. Here’s an example that I thought was funny because it was like clearly written up by an AI. There was a physics example where they asked the AI to derive known but unpublished results about black hole symmetries. One of the take-out quotes is: “After about five minutes of internal reasoning, the model incorrectly reported that the equation had no continuous symmetries beyond trivial scalings.” Then again, we have another example, they prompt the model again, they give it a warm-up problem. With the warm-up problem, the AI is able to solve the full problem.

[49:59] Seth (continuing): This is the part that made me think it was definitely written up by an AI. In the implications section, it felt really AI-ish and here was one of the quotes I pulled out: “AI as symmetry engine. With minimal domain scaffolding, current models can carry out non-trivial Lie symmetry discovery for PDEs”—partial differential equations—”with non-constant coefficients.” Okay. Dude, that was an AI sentence. “AI as symmetry engine.” What kind of metaphor is that? That’s an AI metaphor, dude.

[50:29] Andrey: Yeah, I mean... I think one of the things that’s going on in the background that we should say is that scientists using AI to write is just now ubiquitous, right? There was a huge controversy at ICLR, one of the top CS conferences, where just an enormous share of referee reports for papers were written by AI. In fact there’s a tool, Pangram, that has shown very high accuracy at detection of AI writing, and it was used to measure these reviews and just so many of them were written by AIs. So many of the papers are written by AIs.

[51:15] Andrey (continuing): So I just think this has to... this is just the new normal, right? Like... and we shouldn’t be surprised. A lot of scientists... English is not their first language. Even for those who it is a first language, you know, writing is a specialized skill that most people, most scientists, are not very good at. And it’s a lot easier to have an AI write a draft and you tweak it than to write something from scratch. It’s not obvious to me how important it is that the human does the writing. I guess I like to do writing because writing is thinking, it’s a way that I think through problems. But for a lot of things, I don’t know, let’s say like form letters and things like that, like why would I waste my time honing my language when I could just have the AI do it? So I’ll just say like this is a new normal and the viewpoint that we’re mostly writing for the AIs is also true.

[52:16] Seth: Do you want to spell that out for people who might not have heard that phrase before?

[52:21] Andrey: Yeah. So I first heard it from Tyler Cowen.

[52:24] Seth: Andrey’s favorite economist. Friend of the show.

[52:30] Andrey: If you say that, he’s more likely to retweet you.

[52:33] Seth: [Laughs] Yeah, yeah, yeah.

[52:36] Andrey: “Friend” is, you know, a loose term, but you know, we have had dinner with Tyler and that was a great honor. But yeah, I guess the AIs are sucking in all the writing in the world for their training. You know, they’re also able to search through content very effectively and will be reading that content as part of forming their answer. And that’s just happening all the time. It’s happening much more than humans reading some very niche bit of content like one of our papers, right? And so then you might think that since your primary audience with a lot of writing is the AI, you might want to quote-unquote “write for the AI.” That might mean that you don’t have to write as carefully... or not as carefully, but you might... you know, some of the things to entertain humans might be less important.

[53:38] Seth: Poetic function of language.

[53:39] Andrey: Yes. Less important for the AIs. And so you get writing like this quote-unquote “symmetry engine,” right?

[53:50] Seth: [Laughs] Yes. Like... I don’t know. Okay, maybe. I think language will lose something if metaphors stop being helpful. I think you’ll just stop dropping metaphors, right? We’ll just get to purely functional language, right? Because a bad metaphor is worse than no metaphor.

[54:06] Andrey: Yeah, yeah. I mean, I guess I guess we’re gonna see very clearly... like much more clearly delineated communication for humans versus communication for AIs. That... I mean we’re almost kind of there. I mean papers... if you think about like how much effort most scientists put into writing papers vs. how bad the writing is in most scientific papers... why are we even pretending, you know?

[54:35] Seth: Yeah. Anyway, well, very interesting to watch. Um, I had one more example I wanted to pull out, which was the biology example, which I was really excited to read given that so many of these were very math-heavy. In this example, the writers of the anecdote uploaded an experimental figure showing the impact of giving some white blood cells a glucose substitute. Right? So the idea is maybe the white blood cells will do differently if they have glucose versus not glucose, and maybe you could like get them to do something that would cure cancer if you give them more or less glucose.

[55:12] Seth (continuing): And one of their results was that they tried both giving it no glucose (or a very low amount of glucose) as well as giving it a treatment which is like a glucose substitute. So there was some goo that was gonna gunk up the glucose receptor so that the cell wouldn’t be able to eat the glucose. GPT-5 seemed to understand the figure, pointed out hypotheses and potential follow-up experiments to understand why the “fake glucose” had a different effect than low glucose.

[55:40] Seth (continuing): It suggested some potential mechanisms why. ChatGPT writes: “A low glucose control partly mimics the effect but is weaker than the fake glucose at equal nominal concentrations, suggesting contributions from glycolysis restriction and N-linked glycolysation interference... a known 2-DG [this is the fake glucose] off-target... rather than energy limitation alone.” Right? So this seems to have been the key contribution of ChatGPT, is that... like the scientists obviously when they made this result they immediately identified, “Oh that’s interesting, the fake glucose seems to have a different effect than the zero glucose.” The insight that the AI seemed to have had is this particular mechanism, is that there’s an off-target effect of the fake glucose. And suggested, you know, experiments to follow up—using a different kind of fake glucose, trying some other treatments that would identify whether that was the correct mechanism.

[56:42] Seth (continuing): You know, when I say it that way, it doesn’t seem that impressive, right? Like the scientists were already pretty close to that. The scientist... at least reading them, they seemed more impressed than like my reading of it was. They write—the authors write—”In retrospect in particular, the proposed mechanism of reduced IL-2 signaling via interference with N-linked glycolysation made clear biological sense because it could directly explain the disinhibition of the Th17 cell differentiation under 2-DG treatment. However, this mechanistic hypothesis had not occurred to us.”

[57:17] Andrey: Yeah, I mean... I mean once again, it’s a thought partner. You know, if you’re working with people on a problem, you’re gonna have conversations with them and different co-authors are gonna come up with ideas that you hadn’t thought about yet. And you know through iteration, that ultimately creates an artifact which is the research paper. And that’s kind of a series of things like that. And it’s very rarely that there’s kind of one Eureka in this. Or even if there’s like a main insight, you actually have to like take it very seriously to draw out the implications and so on. A lot of... I actually imagine a lot of people had great ideas that ended up eventually being correct science but they just didn’t pursue them, right?

[58:10] Andrey (continuing): So that’s kind of how maybe we should think about this. Is that it’s a thought partner, but it doesn’t yet have agency to pursue the research.

[58:21] Seth: That is so interesting because I came away with this feeling like this is an example of AI as deep literature search, right? Because it seems the problem was pretty well defined, right? Shouldn’t this have the same effect as that? Do deep literature search to see if there’s any, you know, off-target effects of either the thing. But maybe that’s viewing this too narrowly.

[58:42] Andrey: Yeah. I just... I’m not an expert enough to know whether it made a connection across, you know, literature... Right? Like it knows a lot of things. I don’t know if I’d call that literature review. Just like a scientist would know a lot of things. And then some of the magic happens when it connects two, you know, previously unrelated concepts. I just... to me, saying it’s just literature review seems a bit reductionist. You know...

[59:11] Seth: “It’s just a stochastic parrot, Andrey.” Okay. Are you ready? Do you have any other examples you want to make sure we highlight? Are you ready to move on to our conclusions and posteriors?

[59:25] Andrey: Yeah, let’s move on to the conclusions. Yep.

[59:28] [Music / Transition] — MOVING TO POSTERIORS

[59:35] Seth: Okay. So I think these were pretty impressive. I don’t know if there was any, you know, “dropping my jaw” ones. The Timothy Gowers being like, “This is good enough to be my lazy faculty advisor” is probably the jaw-drop moment, right?

[59:48] Andrey: Yeah. I mean just... I think the credibility of people like him or Terence Tao saying that they find it useful... I think in some sense it’s, you know...

[60:00] Seth: This is an OpenAI release selling, you know, for a product that they sell for $200 a month.

[60:09] Andrey: Yeah, but I mean... I mean... sure. I... I just... I don’t know. Like... to me, once again, I’m going back to my priors. Like it’s obviously useful for science. You have to be truly incurious or, you know, a Luddite to think that it’s not.

[60:28] Seth: Fair enough. Well, actually, I have a theory about your crypto friend. Is it just that, like, cutting-edge crypto is not published widely? Is there some sense in which, like, crypto research might not be in the dataset as much?

[60:44] Andrey: I don’t think so. I don’t think so. I think he... I don’t know. I don’t want to put words in his mouth. But if I like...

[60:52] Seth: He’s a Luddite.

[60:53] Andrey: No, no, no. I think if I had to guess, I think he... he kind of views like some deep... deep theoretical insight as maybe the requirement that he has in mind. And that’s... that’s the bar that he has. And...

[61:08] Seth: Yeah, it’s not Einstein. It’s not inventing new paradigms.

[61:11] Andrey: Yes, yes. But I guess... I don’t know. To me, that’s...

[61:17] Seth: I’m not Einstein! I’ll take it!

[61:19] Andrey: Yeah, yeah. Yeah. Exactly.

[61:24] Seth: Um, okay. Uh, and I... I made this point already but I just want to end here which is... I think my takeaway from here is some sort of automatic translation in between sort of machine-language-provable code and like human-language code seems to be the real bottleneck here before speeding up AI a lot. Or at least math-specific AI.

[61:48] Andrey: I really don’t think that’s the bottleneck, Seth. I truly don’t. Um.

[61:52] Seth: But it con... we keep on seeing examples of it like it gives the wrong answer and you have to be like, “Well, I thought about this and it’s the wrong answer,” and then it does that five times and then it gives you the right answer. We see like three examples of that here.

[62:05] Andrey: I... I guess like... this is one... I guess “bottleneck” seems like a weird word to me given that there’s a parallel...

[62:14] Seth: Accelerant.

[62:15] Andrey: I’m not... I... okay. There’s a para... there’s essentially parallel efforts to... certain things can be formalized in these Lean provers. And imagining an OpenAI... like a... like a GPT-like model calling the Lean model is like trivial. Like I... I’m not saying it’s trivial like clearly like... I don’t...

[62:43] Seth: If it’s trivial, why does it keep on giving us wrong answers?

[62:45] Andrey: Because OpenA... because I actually think that like the way this system is designed, it’s kind of using GPT by itself. But actually... my sense is that people in the field who are pushing the envelope are combining these tools. And if you look at DeepMind’s tools, they’re not... they don’t work like this. They are using the formal provers. And so to call it a bottleneck is like implies that like, “Oh, like actually no one has this working yet.” And I... and I actually... I... I bet that some people have this working. It’s... I don’t think... not... I’m not sure whether everything can be formalized in these specialized proving languages in the same way. But yeah.

[63:34] Seth: It’s a limitation in these examples, but you’re saying it’s not a limitation, you know, tomorrow if you wanted to use the cutting-edge tool.

[63:41] Andrey: Yes, yeah. That... that’s... that’s my sense. But you know, if listeners disagree, you know, feel free to let us know. Yeah.

[63:48] Seth: Yeah, please call in. Okay. Um. Posteriors? Or any other limitation comments you want to make?

[63:55] Andrey: No. I... yeah. I mean I...

[63:57] Seth: Posteriors. Yeah.

[63:58] Andrey: Yeah. I mean I... I don’t know. Like our... our priors were very loose so I don’t know the posteriors. I mean I think... yeah. I mean I... you know, I stand by what I say here. I found these examples quite interesting. And it was uh...

[64:14] Seth: Okay. So paradigm-wise, you’re still in the same place? That you think it’ll be co-working with it today and co-working with it in five years?

[64:21] Andrey: Yep.

[64:22] Seth: I said right now it’s super powerful for lit reviews—deep literature reviews—and um, maybe we’re... you know, in five years we will be all the way to AI on its own, at least for math problems. I come away reading this thinking we’re closer to AI on its own for frontier math research than before reading this. Uh, it really does... and again, I call what I said as a bottleneck or say that it’s already been removed... but I mean it seems like if this... what we see described here, plus the AI being able to iteratively check itself and just like redo the math... try another approach if it disproves itself... seems like you should be able to just let that fly and find a bunch of cool stuff.

[65:13] Andrey: Yeah. And if... if you... if you look at prediction... you know, various forecasts, we see forecasts for by 2030 the Millennium Problems being solved with AI. So... uh, that’s not a very un...

[65:28] Seth: AI is gonna solve the Riemann Hypothesis? That’s more of a question about the Riemann Hypothesis than AI.

[65:32] Andrey: Well, you know. People who are experts, a decent chunk of them forecast that this will happen. So, yeah.

[65:40] Seth: Okay. And how impressed were we by the most impressive result? I said we were gonna... I was gonna be like 7 out of 10 impressed, 8 out of 10 impressed. I think that’s kind of where I end up. If not like a little bit below that. Um, in the sense that I’m not saying that these mathematical results aren’t super impressive, but I was hoping for like, “And we discovered something that was like a treatment we can use tomorrow,” or “We discovered...” I was hoping for something that was kind of more directly practical from at least one of these examples.

[66:13] Andrey: Yeah. I mean, to me, if there was something that was very practical, that would be like a 9 out of 10 or 10 out of 10. And you know. Uh, but I... yeah. Once again, I think like nothing blew my mind, but it all seems like we’re... we’re... we’re on the path to this being a very transformative technology for science. Yeah.

[66:36] Seth: Yeah. Super, super excited to talk to Ben Golub about the AI research tool that he’s working on. Um, and uh, listeners at home, let us know: How do you use AI in your science or in your life? Post it in the comments, share, comment, and subscribe. All right.

[66:56] Andrey: Well, until next time. Keep your posteriors justified.

[67:00] [Music fades out]

A foundational assumption of Justified Posteriors is that the claims made in published research papers and other intellectual work do not directly correspond to what we believe after reading them. This should be obvious to anyone who has seriously engaged with intellectual work. But what is less obvious is the degree of the gap between the claims in the work and the beliefs of the reader. Is there a slow accumulation of evidence (a vast literature, as one will read in formulaic introductions) that gradually moves our beliefs from zero to one? Or perhaps there is a critical moment, where one paper causes a rethinking of all that came before it, leading to a new conclusion.

We could dredge through the history of science, as our predecessors Popper, Kuhn, and Lakatos have, to come up with examples of both. We idealize the pivotality of Einstein’s tests of general relativity. The evidence we have to deal with is much muddier. We live in a time where claims are circulated as a global pastime. Sometimes these findings come with the trappings of academic prestige and peer review, while other times they come in the form of a polemic dropped like a nuclear bomb into the memesphere, as those who have situational awareness may understand.

Few have time to read deeply, and even thinking seems like one of those lines in a todo list that is never crossed out. Consider the ubiquitous evals used in AI research and cited throughout social media. The number of people who have read the underlying methodology for each eval is minuscule. The ignorance is so vast that people don’t know how few samples are in each eval, let alone the confidence intervals. And yet, a careful evaluation of a new eval such as GDPVal can update our priors by a lot.

This is the water we swim in with Justified Posteriors. The premise for the show seemed simple, but nothing is as simple as it seems. For one, how do we pick a prior, especially without reading the paper? A conceit of the podcast is that we form our priors with zero information about the paper, but even to pick a paper we need to know something about it. Picking a prior turns out to be one of the topics which we struggle with the most.

What are we supposed to learn from a theory paper such as Ide and Talamas’s “Artificial Intelligence in the Knowledge Economy?” A theorist might be satisfied by learning whether this is a useful way of modeling the phenomenon. But we try to translate these into more empirical statements, such as “what percentage of US workers will have managing or creating teams of AI agents as their main job within 5 years.” Typically, we don’t update a lot.

Of the 22 episodes in which we had at least some semblance of priors, the biggest update came for Seth in the episode about “How do social media feed algorithms affect attitudes and behavior in an election campaign?” The randomized control trial evidence on political beliefs convinced him that whether an algorithmic feed or a reverse chronological feed was shown to a user did not affect their political polarization. I already had this as my prior, given prior literature.

Nonetheless, neither of us were willing to update much on the larger claims. The reason is that, as always, the real world is complicated. For example, the paper did not study decisions to moderate content, a process which can be algorithmic but which differs from the algorithmic feed. The paper also did not consider truly directed algorithmic interventions, such as those by Elon Musk on X. We can’t read this paper and just say that algorithmic feeds are not an important determinant of political beliefs.

For me, the biggest update came in the episode “Can AI Make Better Decisions than Doctors?” I came in skeptical that AI could overcome the fundamental problems of causal inference without a randomized control trial. The evidence in the paper strongly updated me toward believing we should be more aggressive in inserting AI into ER decisions.

Interestingly, papers on more macro topics caused smaller updates even if they had much greater implications. Our first episode was fittingly about the now famous Situational Awareness document written by Leopold Aschenbrenner in June 2024. We didn’t have explicit priors, but we thought that AGI was further away than 5 years. We also thought AI was super important and that some of the predictions were plausible. We joked about buying NVIDIA, and didn’t (we were fools). To me, this episode highlights how easy it is to be directionally right, to read the right materials, but to not take ideas seriously enough. The arguments in the paper about power generation and data centers have especially proven correct. And if you squint, we’re following the timeline predictions closely even to this day. Claude Code with Opus 4.5 seems to be just on time for Aschenbrenner’s prediction of a proto-automated-engineer in 2026/2027.

A common theme in our discussions of papers about the economics of AI is that they are often measuring transitory phenomena, such as changes in productivity or performance at a particular point in time. An extreme example of this is the “Simple Macroeconomics of AI” by Daron Acemoglu, which assumes that AI will stay as good as it was in 2024. These papers are often underwhelming, even when they are well-crafted, because what everyone really cares about is what will happen in the future.

Much of my learning has come through conversation about the paper, rather than just by reading the paper. My updates would be very different if I read the paper without talking with Seth about it. This is reminiscent of an academic seminar, in which a group of colleagues focus exclusively on one paper presented by a speaker. Attendees of seminars will know that oftentimes the most interesting part of seminar day occurs in the hallway conversations afterwards, when people share their opinions and discuss. One can tell how serious an academic department is by the quality of the hallway discussion.

This brings me to the next topic, the validity of podcasting as a worthwhile intellectual pursuit for a professor. I am supposed to primarily demonstrate my work on an intellectual topic by writing papers published in top journals. Yet to me it is obvious that we are doing valuable and original work in reading these papers and interpreting them through broader lenses than just the minimum publishable unit. For each episode, we have to understand literatures, engage deeply with evidence, and reason through the implications. This sort of work is something top researchers often do prior to starting new research projects, but is rarely shared outside of side conversations or lab meetings. What Seth and I do is a valid and valuable intellectual activity, not substantively different from writing a paper or a book.

One of the great pleasures of doing the podcast is hearing from our awesome readers and listeners! In the coming year, our goal is to improve the quality of our work by increasing our preparation, improving our audio and video quality, and by bringing on insightful guests. I am excited to continue covering emerging measurements of the AI economy and theoretical frameworks related to the impact and diffusion of AI. As always, we would love to hear from you with any feedback.

Thanks to Seth Benzell for comments and for being a great co-host.

We start with Ben’s startup Refine, an AI-powered technical referee for academic papers. From there, the conversation ranges widely: how scholars should think about tooling, why “slop” is now cheap, how eigenvalues explain viral growth, and what large language models might do to collective belief formation. We get math, economics, startups, misinformation, and even cow tipping.

Links & References

• Refine — AI referee for academic papers

• Harmonic — Formal verification and proof tooling for mathematics

• Matthew O. Jackson — Stanford economist and leading scholar of networks and social learning

• Cow tipping (myth) — Why you can’t actually tip a cow (physics + folklore)

• The Hype Machine — Sinan Aral on how social platforms amplify misinformation

• Sequential learning / information cascades / DeGroot Model

• AI Village — Multi-agent AI simulations and emergent behavior experiments

• Virtual currencies & Quora credits — Internal markets for attention and incentives

Transcript:

Seth: Welcome to Justified Posteriors, the podcast that updates its beliefs about the economics of AI and technology.

Seth: I’m Seth Benzel, hoping my posteriors are half as good as the average of my erudite Friends is coming to you from Chapman University in sunny Southern California.

Andrey: And I’m Andrey Fradkin coming to you from San Francisco, California, and I’m very excited that our guest for today is Ben Goleb, who is a prominent economist at Northwestern University. Ben has won the Calvó-Armengol International Prize, which recognizes a top researcher in economics or social science, younger than 40 years old, for contributions to theory and comprehension of mechanisms of social interaction.

Andrey: So you want someone to analyze your social interactions, Ben is definitely the guy.

Seth: If it’s in the network,

Andrey: Yeah, he is, he was also a member of the Harvard Society of Fellows and had a brief stint working as an intern at Quora, and we’ve known each other for a long time. So welcome to the show, Ben.

Ben: Thank you, Andrey. Thank you, Seth. It’s wonderful to be on your podcast.

Refine: AI-Powered Paper Reviewing

Andrey: All right. Let’s get started. I want us to get started on what’s very likely been the most on your mind thing, Ben, which is your new endeavor, Refine.Ink. Why don’t you tell us a little bit about, give us the three minute spiel about what you’re doing.

Seth: and tell us why you didn’t name your tech startup after a Lord of the Rings character.

Ben: Man, that’s a curve ball right there. All right, I’ll tell you what, I’ll put that on background processing. So, what refine is, is it’s an AI referee technical referee. From a user perspective, what happens is you just give it a paper and you get the experience of a really obsessive research assistant reading for as long as it takes to get through the whole thing, probing it from every angle, asking every lawyerly question about whether things make sense.

Ben: And then that feedback, hopefully the really valuable parts that an author would wanna know are distilled and delivered. So as my co-founder Yann Calvó López puts it, obsession is really the obsessiveness is the nature of the company. We just bottled it up and we give it to people. So that’s the basic product—it’s an AI tool. It uses AI obviously to do all of this thinking. One thing I’ll say about it is that I have long felt it was a scandal that the level of tooling for scholars is a tiny fraction of what it is for software engineers.

Ben: And obviously software engineering is a much larger and more economically valuable

Seth: Boo.

Ben: least

Andrey: Oh, disagree.

Ben: In certain immediate quantifications. But I felt that ever since I’ve been using tech, I just felt imagine if we had really good tools and then there was this perfect storm where my co-founder and I felt we could make a tool that was state of the art for now. So that’s how I think of it.

Seth: I have to quibble with you a little bit about the user experience because the way I went, the step zero was first, jaw drops to the floor at the sticker price. How much do you,

Ben: not,

Seth: But then I will say I have used it myself and on a paper I recently submitted, it really did find a technical error and I would a kind of error that you wouldn’t find, just throwing this into ChatGPT as of a few months ago. Who knows with the latest Gemini. But it really impressed me with my limited time using it.

Andrey: So.

Ben: is probably, if you think about the sticker price, if you compare that to the amount of time you’d have, you’d have had to pay error.

Seth: Yeah. And water. If I didn’t have water, I’d die, so I should pay a million for water.

Andrey: A question I had: how do you know it’s good? Isn’t this whole evals thing very tricky?

Seth: Hmm.

Andrey: Is there Is there, a paper review or benchmark that you’ve come across, or did you develop your own?

Ben: Yeah. That’s a wonderful question. As Andrey knows, he’s a super insightful person about AI and this goes to the core of the issue because all the engineers we work with are immediately like, okay, I get what you’re doing.

Ben: Give me the evals, give me the standard of quality. So we know we’re objectively doing a good job. What we have are a set of papers where we know what ground truth is. We basically know everything that’s wrong with them and every model update we run, so that’s a small set of fairly manual evaluations that’s available. I think one of the things that users experience is they know their own papers well and can see over time that sometimes we find issues that they know about and then sometimes we find other issues and we can see whether they’re correct.

Ben: We’re not at the point where we can make confident precision recall type assessments. But another thing that we do, which I find cool, was whenever tools that our competitors come out, like Andrew Ng put out a cool paper reviewer thing targeted at CS conferences.

Ben: And what we do is we just run that thing, we run our thing, we put both of them into Gemini 2.0, and we say, could you please assess these side by side as reviews of the same paper? Which one caught mistakes? We try to make it a very neutral prompt, and that’s an eval that is easy to carry out.

Ben: But actually we’re in the market. We’d love to work with people who are excited about doing this for refine. We finally have the resources to take a serious run at it as founders. The simple truth is because my co-founder and I are researchers as well as founders, we constantly look at how it’s doing on documents we know.

Ben: And it’s a very seat of the pants thing for now, to tell the truth.

Andrey: Do you think that there’s an aspect of data-driven here and that one of your friends puts their paper into it and says, well, you didn’t catch this mistake, or you didn’t catch that mistake, and then you optimize towards that. Is that a big part of your development process?

Ben: Yeah, it was more. I think we’ve reached an equilibrium where of the feedback of that form we hear, there’s usually a cost to catching it. But early on that was basically, I would just tell everyone I could find, and there were a few. When I finally had the courage to tell my main academic group chat about it and I gave it, immediately people had very clear feedback and this was in the deep, I think the first reasoning model we used for the substantive feedback was DeepSeek R1 and people, we immediately felt, okay, this is 90% slop.

Ben: And that’s where we started by iterating. We got to where, and one great thing about having academic friends is they’re not gonna be shy to tell you that your thought of paper.

Refereeing Math and AI for Economic Theory

Andrey: One thing that we wanted to dig a little bit into is how you think about refereeing math and

Seth: Mm-hmm.

Andrey: More generally opening it up to how are economic theorists using AI for math?

Ben: So say a little more about your question. When you say math

Seth: Well, we see people, Axiom, I think is the name of the company, immediately converting these written proofs into Lean. Is that the end game for your tool?

Ben: I see, yes. So good. Our vision for the company is that, at least for quite a while, I think there’s gonna be this product layer between tools, the core AI models and the things that are necessary to bring your median, ambitious

Seth: Middle

Ben: not

Seth: theorists, that’s what we call ourselves.

Ben: Well, yeah. Or middle, but in a technical dimension, I think it’s almost certainly true that the median economist doesn’t use GitHub almost ever. If you told them, they set up something that, a tool that works through the terminal, think about Harmonic, right?

Ben: Their tools are all, they say the first step is, go grab this from a repository and run these command line things to, they try to make it pretty easy, but it’s still a terminal tool. So a big picture vision is that we think the most sophisticated tools will be, there will be a lot of them that are not yet productized and we can just make the bundle for scholars to actually use it in their work.

Ben: Now about the question of formalization per se, I have always been excited to use formalization in particular to make that product experience happen. For formalized math, my understanding is right now the coverage of the auto formalization systems is very jagged across, even across. If you compare number theory to algebraic geometry, the former is in good shape for people to start solving Erdős problems or combinatorial number theory, things like that, people can just start doing that. For algebraic geometry, there are a lot of basics that aren’t built out and so all of the lean proofs will contain a lot of stories that the user has to say, am I fine considering that settled or not?

Ben: And that’s not really an experience that makes sense for someone trying to check their econometric draft, right? So we’re watching and I think as soon as we feel it’s the moment when we can take the typical, say economic theory proof and give a rigorous certification, we’ll be right on.

Ben: I would like us to be in a position to be right on top of it.

Seth: I blame Grothendieck for algebraic geometry being hard to formalize, hard to make into Lean.

Andrey: Even short of things like Harmonic, right? It’s certainly you can get useful things of putting some math or asking for some math from Gemini for example. How are people in the field using those tools and have you noticed that has affected the type and quality of economic theory you’re seeing?

Ben: Oh yeah. That’s zooming out from refine. I’m obviously a heavy user of AI tools for my own research. I think broadly we’re seeing two phenomena play out in parallel. It’s a lot easier, this idea that went viral a few weeks ago of work slop being much easier to produce. I think there is an experience, which I’ve experienced myself, where you owe your co-author something and you have some ideas, you’ve done some real work, but it’s much easier to put a section in the paper that is AI written that looks a lot that our natural checks see as real work. And that introduces obviously new kinds of risk. It makes work faster in some ways and more fragile in others. And I think about that a lot. By the way, one of the main new values of refine is as people are perhaps less moment to moment engaged with the exact, or less line by line engaged with their work, which AI is doing. They need that global eye and that obsessive look, which used to be more in one’s own head. But that’s the negative phenomenon. But I think in terms of having a pretty expert consultant in things you don’t usually work on just for getting started and forgetting ideas.

Ben: I can already see major gains in my own research. One thing I would be curious to see is just looking at measures of production of scientific literature. We should see something on speed that’s visible in we should see signs of science speeding up in the areas which are particularly sped up.

Ben: And I, it would be fun to formulate a hypothesis like where should we be looking to see that

Seth: Right. We recently recorded an episode, the open AI paper on early uses of AI in social science. And it seems to us one of the most obvious immediate use cases is just, can I find if somebody already proved this and I could just plug it in? Right.

Andrey: to be clear, not social science, but mathematics.

Seth: mathematics. Excuse me.

Seth: Yeah. Yeah. Science, science is,

Ben: Physics. So yeah,

Andrey: Yes, exactly.

Seth: Andrey always calls me out that I say economics or social science when he really means, when I really mean actual science.

Andrey: Just to be clear, there were

Ben: important. Yeah,

Andrey: A bunch of math in that paper, which is very cool.

Ben: This is known. I think economic theory, it’s important to me about economic theory that there is really such a thing that’s called economic theory, very distinct from math. Usually, unless something is going wrong, you don’t need to do any interesting math.

Ben: In an economic theory paper, you just find the relevant. So I think a lot of economic theorists who are successful and good at it, a lot of the trade is finding the right thing, learning enough of it to make it valuable for your application and just using it correctly. And that’s where that search problem is really accelerated. So I’m with Seth that there’s gonna be a huge speed up just for maybe not as, it’s not super intelligence. It’s better search, but that’s huge.

Andrey: So one economic theorist that I’ve talked with about this is Joshua Gans. I don’t know if you’ve had a chance to talk to him, but he’s been writing a paper a week,

Seth: Right. The guy, he is grinding him out with the AI help

Andrey: Is there some sort of weird proof of work thing that’s starting to fail? Because look, writing down theories of almost anything, it was, it took a lot of work, but you could, there was a recipe, right?

Andrey: As an

Seth: you can mathematize Marx right. The fact that I can rewrite marks in math doesn’t necessarily make Marx good.

Andrey: Yeah.

Andrey: So how do you think about that and what do you think are gonna be directions in economic theory that are really changing the game as a result of this?

AI, Work Slop, and the Future of Economic Theory

Ben: Yeah. You raise an interesting point. You can think of one vision of what social science is, or what economic theory is, that’s suggested by what you just said, which is that we’re commentators on social reality and we’ve developed a particular style of doing that, which involves, in the case of modern economic theory, a lot of math and the proof of work.

Ben: There’s almost an equilibrium where you, in order to say something, you have to really carefully and write well in English, but also do this mathematics and now that, at least superficially can be totally hacked, is that gonna stop? Is that gonna make the commentary aspect of economic theory lower signal in some sense?

Ben: Is it going to, and that’s a great question. So let me table that for a second and say what? I have a thought on this topic that’s related to that. If you’re really good at that and you produce these really jewel like economic theories and then suddenly everybody can write slop and produce economic theories that at least take a while to distinguish from your beautiful ones, then maybe you feel sad, like your art has been degraded.

Ben: And I do think that’s the way poets, I think. I talked to some people who are very interested in the experience of artists with AI and I think that’s an artist’s experience with AI. Then there’s another kind of person I have in mind, which is an idealized cancer biologist.

Ben: And you tell them, oh, your jewel like blot analysis that you do or whatever. Now they’re gonna be automated. And I think this guy’s first reaction is mostly not, oh, how will people be able to admire my art? Will people still appreciate my art as much or what will I do with my time?

Ben: But they’re like, oh shit, we might move faster toward curing cancer. So one thing I think is wrong broadly with economic theory is that there are a lot of us whose reactions fall more into the artist category. And I would like, I think economic theory is not done. In fact, it’s quite bad what we’ve achieved on the whole.

Ben: So we should be

Seth: excluded of course.

Ben: Yeah. So as a group, as a community, right? And so if we, I would hope that we have it in us to say, look, now we have these incredible tools to take a run at questions that are really where the solution would be genuinely valuable.

Ben: And we could really try to do them better. And we have this huge resource now. I would like it to be, I would be happier about us if we had more of that reaction. I’m hoping that there will be parts of the profession, parts of the enterprise that grow and accelerate, because they’re driven by that as opposed to hand wringing over the art problems.

Seth: Right. And it seems like you could always add some more, get gatekeepers on the backend. Right? If we just make it easier to enter with, here’s my mathie paper. And the concern is you get too much slop. Maybe there is some way to filter. You don’t have to filter on the math anymore. You filter on something else.

Ben: Totally. All of these offensive weapons are also closely related to defensive weapons. So there’s a whole, and refine is obviously a natural, we think about that, that we can, at least, at minimum, we can help reject slop that’s written by cheap models without much skill and maybe we can help

Seth: How do you defeat slop? How do you defeat slop with bitter slop?

Ben: Yeah,

Andrey: Have you talked with some editors? Is there interest here?

Ben: Yeah. So Refine is doing pilots with several of the very top journals in economics. And we’ve been really encouraged by, I think because a lot of the editors are super genuinely pro-social people who want to take the tech, who wanna bring technology to bear as fast as possible, to improve the profession.

Ben: And so we, and I think there’s a feeling that they have that’s correct. That this phenomenon is here, and so the best way for the journals, for example, to deal with it is to be as up on it as anybody. And so we, I think the main use that is the easiest sell is just final due diligence right before publication at the conditional accept stage.

Ben: Can we make sure that papers are, any remediable, any mistakes that the author would be embarrassed to have published, the author has a chance to learn about it. Correct. That’s, everybody agrees with that. I think there’s a lot more design required to do it thoughtfully when stuff is incoming.

Ben: I have heard experiences from editors using REFINE and other tools. When they get a submission that they’re very suspicious about, they can just quickly run it through refine, see that there seem to be, and they’re usually experts in, right? So they can see, oh, this is surfacing really serious errors.

Ben: Now I can, for example, desk reject it with a lot more confidence. So we’ve, that experience does happen. That’s purely people’s own use of the tools, but.

Andrey: Are you worried that your tool is fundamentally, it’s interesting. Like many economists, it’s a tool of rather than constructivism in that it’s very good at finding problems. But is it ever gonna be, well, this is not a perfect paper, but it’s a beautiful paper nonetheless.

Seth: GPT-4o if you wanna sycophant to Andrey.

Ben: Actually, one thing we think a small version of that, and I’m curious for your guys’ sometimes refined produces, you give it a 50 page manuscript and it produces six comments. In fact, one of our engineers recently switched. He said, we switched to a new, we did some model upgrades.

Ben: And then he looked at it and he said, this only produced six comments. And it was on a paper by one of our friends who had been through refine and all the mistakes were gone. And so he was like, oh, it went from, if I just run this on the dumber models, they give me 50. Now it’s six.

Ben: And that was actually good because the feature question we have is in that case, should we tell the author, Hey, this has fewer things we can see wrong than 95% of papers. Right? That’s turns this question mark experience into maybe something encouraging. So we haven’t rolled that.

Ben: I’m curious if you guys think such a badge would be pleasant for an author.

Seth: Question mark experience.

Andrey: I, I, think you should, well, you should obviously run the experiment,

Viral Processes and the Refine Referral Program

Seth: Uh, maybe an interesting place to start is this referral program that you came up with. So where did that come from? Why did you design it the way you did?

Andrey: You just, well explain it first. Yeah. I think that’ll be the first. Yeah.

Ben: what we have, we actually, we, through the end, through the end of Decem through the end of November, we ran our, our first iteration of our referral program, which we will keep, which will tune and keep running, in various guises. And the way the program works is you, if you refer a friend, if you want to refer friends, you get a referral link from the site. You can share that with anyone you want. And every time somebody, if somebody that you refer ends up actually, paying for a full refine review, at least one, they, they get a full bonus review and you, the referer get one. So we, our, our top reviewers, I don’t think you’ll mind me sharing ‘cause he, he told, he basically told everyone he knew, but Joshua Gans, he, he was, he’s like, I think he has like 35 credits now because he just kept referring and

Seth: God bless.

Ben:because my co-founder, my co-founder and I were talking and we’re like, this is than we expected, should we’d be worried about.

Ben: So we were like, no, this is only good. This is, there’s nothing to be stressed out. Um, he can have, he can have lifetime refined use, free for, for being such a good, but that’s what, so I think economically, I think there are two thing. One, one immediate thing to think about is that some people are gonna be really good ambassadors for your product, but you don’t know who they are.

Ben: There’s an information problem and a referral to the extent, and interestingly, they’re the ones who are gonna value the credits, if they’re really good users of it, and they’re also gonna be the ones that, probably can identify others who know. And so getting those people to raise their hand, is not a trivial problem if you just had to do it without, but it turns out this, it, offering the referral to them kind of puts the incentives in the right place. And then, the others, obviously the other lens that I think of it through is, the lens of network economics and the viral process. So I, I’m happy to talk, but I actually, the information one, when we were thinking like, who should we recruit as an ambassador? It wasn’t obvious. And this got them to come forward.

Seth: You’ve done some work, I think, both in, definitely theoretically, but maybe even empirically too, about optimal seating. So did that, any results from that play in?

Ben: That’s a good, I would say the, the most, honestly, the most important insight that kind of was really top of mind for me was what I, in an, in my undergrad networks class, which I teach from, Networks, Crowds, and Markets by Easley and Kleinberg, they go through the basics of the viral process

Seth: Will Jackson be insulted that you don’t use his book?

Ben: well, no, ‘cause it’s, it’s graduate book.

Ben: I

Seth: Okay.

Ben: every year. I do say, you can go buy, you can, if you really wanna know everything, you can buy Matt’s book. But so,

Andrey: yeah, just as context for the listeners, Matt Jackson was Ben’s thesis advisor. Yeah.

Ben: and yeah, collaborator and overall hero. So I, and it’s funny because I, yeah. Small aside, but when I teach that class, I’m like, ‘cause I realized from these undergrads perspective, Matt Jackson, like, if you read these books, he’s just like, they think he’s probably dead. Like, he is like, seems like a very major, a major part of the field.

Ben: And then I drop somewhere in the middle of the quarter, like, oh, Matt was my, Matt was my advisor. Um,

Seth: Not dead yet.

Matt Jackson as an Advisor

Andrey: talking about this, this is a little bit of a tangent, but I hope you don’t mind Well. What was he like as an advisor?

Ben: oh yeah, he is, he was ama I mean, overall amazing. Like, I, I, the main thing to say about it is I met him right as he was about to move from Caltech to Stanford. I came to him as a Caltech Summer research intern student. He didn’t really havetime, but somehow I, I tricked him into like, not, to, not to being officially on the, on the program.

Ben: Uh, my advisor in the program. And then we, we started working on our first papers on social learning and information aggregation right then, and. He, I think he’s ex, the most salient trait of him is that he is just incredibly supportive and encouraging about research, but actually not at all. There was very little teaching that he ever, he ever did, explicitly, here’s how you do research. Everything I learned from him was, was ‘cause he was open to co-authoring and I just saw him do research and I learned by, by apprenticeship. my dad had actually told me that that was the best way to learn and I, and but he had like Soviet physics, in the 1970s as his reference point.

Ben: So I was pretty sure it was not good advice, but it actually ended up being exactly what worked for me, with Matt. But Matt was not, Matt was not prescriptive he didn’t, I don’t think, I think his, his default mode of advising is like, because he’s so incredible at research. He, his first best advising style is to leave the student alone and let them, and let them do their thing.

Ben: And one, and I, it made way more sense to me when I talked. I, I think I talked to him about. His experience with his advisor, Darrell Duffie. And I learned that it was just, it was all this dynastic thing where Darrel was exactly the same way. He just, like, Matt brought him a thesis and Darryl was like, this is really interesting.

Ben: This is good. They had been writing other papers, but that was the extent of, and I, I don’t, Mike’s Matt was more, was definitely a great mentor, but I think it was really freeing to have someone basically just trust you to do re to do research and be there as a, be there to teach by example when you needed it.

Eigenvalues and Network Dynamics

Andrey: here’s a question. Who likes eigenvalues more? You or Matt Jackson?

Ben: Definitely me. ‘cause Matt’s not, Matt’s not a math nerd. Matt. Matt is a, Matt really is a true, true, true social scientist. He’ll use whatever tool. I think there’s, I’ve always felt a little sheepish that this aesthetic thing of like, what, this tool is really like special to me. He’s, he’s not like that and I think it makes him a better social scientist that he’s not.

Ben: Whereas I, ‘cause I think when you, whenever you care about something other than explaining the social world, that’s gonna be like, a trade

Seth: Well let, let’s slow down for a minute for, people in the audience who don’t live with the, in the, in the glorious glow of the eigen value. And, thinking about eigen vectors of Jacobian matrices, can you give us a little, give us a little taste to someone who’s already not in love with eigen values?

Seth: Why should they love eigenvalues?

Ben: Yeah, that’s a great question. Well, so, okay, 0.1 is, algebra describes the world. You guys know that video where the guy that the, the math profs or like, like sweaty t-shirt math guy is yelling like, functions. Describe the world. I think the real thing, linear algebra describes the world, and I think in the AI era, we, we don’t, as Tyler Cowen says, it’s Rise.

Ben: Tyler Cowen says it’s rising in status. So it’s quite high in

Seth: There we go.

Ben: the tough thing about matrices is that they’re so damn complicated. There’s like, matrices, you can the, the whole world into that. And the amazing thing about. Values is that they, they answer the question of if a matrix had to be a number, what number would it be? Like if you, if a matrix lost its privileges of being, of being an end by inbox and couldn’t store all that information, you have to masquerade as a, as at, at worst, a complex number. What complex number would it, what, what mask would it put on to be itself as a number? And eigenvalues are a wonderful way of, of fully answering that question is the best you can do. And that’s like, that’s a powerful idea. And, and I, and so back to viral processes, if you think about a viral process unfolding in a network, there’s a way to model it as a matrix or a network with all of the, the sort of, activation events being modeled as like basically a big matrix, multiplication, that prop that makes your state kind of, yeah, for the, I guess. Yeah, I don’t wanna, I don’t wanna, I understand that this is probably not the most intuitive way of describing it, but it is really true that if you have a large population and you wanna track the evolution of a state like a virus, you can think of that as kind of a matrix operation that acts on the system and updates it to the next step, which is like the thing spreading further.

Ben: But often what we wanna know about a virus is not everything about how it’s proceeding, but we wanna know something simpler. Like is it like when back in COVID, is it tending to spread right now or is it dying off? Right? And so it turns out that you can compute an eigen value of a suitably defined operator or, or something that will answer that question.

Ben: And so when you’re trying to run a viral contagion, as we are at refine to get more people aware of our product, we are trying to get the viral coefficient, above one. And

Seth: Right. Okay. So yeah, so tell me what, what’s the special thing that happens when an eigen value goes from below one to above one?

Ben: Yeah, well, let’s think about numbers, right? I said so, sowe have this, this process that we’ve now distilled down to one number, the viral coefficient. And we’re, we’re doing that process, namely the next step of the, of the epidemic over and over, right? The next moment when the epidemic has a chance to do its thing, and mathematically taking a time step is applying the, the operator of the epidemic’s behavior to the system.

Ben: So you have a system you hit it with, you say, okay, one more time, step. When we compute the, the eigenvalue kind of captures just the overall extent, captures how a number. And if that number is above one, it means every time it acts, that process tends to expand the set of infected people. And so if you’re doing it over and over, you think of a number greater than one, like two.

Ben: If you keep

Seth: One of my favorite numbers greater than one.

Ben: Excellent. My, my favorite. Um, if you have two and you keep hitting it, that is multiplying it with two, you keep getting bigger and bigger and that’s exponential growth. And it’s, it’s actually, it actually works with 1.01 as well. Right. And so if you, the la the largest iGen value of the propagation matrix captures exactly that.

Ben: Is there, when, when you keep hitting that system with itself again, does it behave like raising two or 1.01 to higher and higher powers? That’s when you have expansiveness, that’s when you have viral spread.

Seth: if my eigenvalue were 0.9, my viral spread would be I contaminate 0.9 people who contaminate 0.9 people, and that adds up to a finite amount instead of everybody gets it

Ben: Exactly. And so,

Seth: now, tell me what a complex eigenvalue is.

Ben: no, not today, but I will, what,

Seth: It’s not, it’s not, it’s not an, it’s not an interview on Justified Posteriors if the guest doesn’t refuse a question.

Ben: But, but, I will say is that I, what I, what I taught in my undergrad class, what, the way that I sort of like, like tried to get them, maybe even a little more excited is, you, when you think about that tipping point 0.9 to 1.1, it doesn’t look like a big deal. Um, locally, it doesn’t look like a big deal when you super zoom in on the, on the process.

Ben: But when you look at the process’s overall behavior, it, it makes a huge difference. And so what I to what I tell the business minded undergrads that I often teach is, if you’re running, and this was always just a fanciful little illustration to me, if you’re running a company and you’re running a viral promotion, you really could, you might be willing to invest a whole lot of money to move that number only a little bit because

Seth: Infinite return, dude.

Ben: yeah. If you, if you can push it, that’s where the returns to that are very big. And so we’re, and I amusingly, I think we’re right there. I we’re, I think our viral coefficient for this referral program is just about one. I can talk about some subtleties of estimating that, but that means, one of, one of the ways that we wanted to build it is we have that to have prices in there.

Ben: So the, the, the rewards you get are a price, right? And we can in principle give you, give your give, change the price, give people more free stuff or roll lower, make it an introductory offer with a, and those are the things we can tune to change the viral coefficient.

Andrey: And I guess the other thing in practice to remember is that the viral coefficient isn’t constant.

Seth: Ah, right. So does linear algebra describe the world when it’s like a first degree Taylor approximation? Actually.

Ben: Well, the beauty of, yeah, the reason it’s not co like yeah, it’s not constant over time. And one of the reasons it’s not is because as your contagion pro propagates through the network, it’s hitting different people. Right? Um, and that’s definitely something that of course as Andrey as, as you both know, and Andre, and I have talked about is that the selection of people as any kind of, of social phenomenon, like a an advertising campaign is progressing.

Andrey: I.

Ben: getting as the next rung is, is different. And eigenvalues actually do capture that from a nerdy perspective. Like if you just had to the, if you teach the simplest possible model where you just, like everybody has three friends and they infect these three friends with some probability, there’s no room for heterogeneity.

Ben: But if you take a whole network, then actually the heterogeneity is in there and the heterogeneity is, is exactly captured by it. And so in some sense, the largest eigenvalue will tell you the average of this across the whole network. So there are tools, of course when you’re doing it in real life as I’m now you’re just tuning the knobs andyou know, doing it in a somewhat less scientific way.

Andrey: But I’ll, I’ll just say that like after this podcast airs, will have been infected, so

Seth: Yeah. Oh man. Your I, dude, we’re getting your eigenvalues up there. We’re boosting your eigenvalues as we speak, dude. Okay. So we, we talked a little bit about, contamination of like viruses, but now let’s talk about an even more insidious form of, viral contamination, which is the idea or the meme, which contaminates us with, mental illnesses such as good taste in movies.

The DeGroot Model of Social Learning

Seth:Um, I guess if we were bringing these ideas of linear algebra to, social learning, we would think about this thing called the DeGroot model of Social Learning. Can you tell us a little bit about what that is? And then we’ll kind of build up to why wouldn’t that be a good way to learn, and how will AI help us think about that?

Ben: Yeah. So the DeGroot model is just, and I, I, I used to call it the averaging model of social learning, is actually what I worked on with Matt Jackson when I came to him as an undergrad. Um, at Caltech in 2006. I, like many other had rediscovered. Um, the dud model just says, you form your opinion tomorrow by taking a weighted average of what your friends think today. You can forget the weighted part if you, it’s not that important. So I just look around and my friends, I say, what are, what do they think about whether AI is good for humanity or whether, whether, you know. Um, you should throw away all your black, spatulas because they have toxins in them. And, and then for on issues like that, people form sort of an opinion by, by social communication.

Ben: And the DeGroot model is the simplest possible model. And we can come back to this. It’s, it’s one that economists actually don’t tend to love when they first encounter it because it is extremely simplistic and kind of, robotic or animalistic. You just, you just take the average. And if you have a bunch of people doing this, that can be summarized with beautiful linear algebra, which is actually exactly the same math, more or less as the math that you do for Markov chain theory. So, that’s for the nerds. But sociologically it’s interesting be because if it, because you can immediately start asking questions like, will a population of people updating this way reach a consensus and will that happen fast or slow? And will this consensus be right or wrong? And it sort, it gives this tool, which is like a pocket calculator that, that, um. Anyone with a reasonable applied math, education could, could have reinvented as in fact many people, including me, did. And, and then, but you can immediately take it to also, I think one of the reasons it’s been, so popular in economics is just it gives you a lot of ways to ask simple questions and get answers, which is something the, I can talk about it, but the standard economic models of learning don’t actually tend to give, many answers in networks

Seth: What would a large versus a small eigen value in a DeGroot learning network mean?

Ben: so in the, the first eigenvalue, which is the first one people talk about, the biggest one happens to always be one for a DeGroot model, which captures the idea that everybody is averaging. So in some sense aren’t getting, there’s no natural amplification or shrinking in opinions, because if you’re averaging, that’s sort of like the, there’s an eigenvalue, which just captures that fact

Seth: There’s no way for our opinions to fly off to infinity. I guess maybe if I was like negatively waiting you could that happen?

Ben: That could happen actually, but yeah. But if you, but with normal, with sort of the, the first, the natural assumptions on weights, things will tend to stay confined

Seth: know. Having negative weights on some people’s opinion seems pretty natural to me. If you’ve been on Twitter,

Ben: I have an under, I have a brilliant undergrad thesis student right now who’s studying

Seth: ah.

Ben: negative weights in the root model. But, yeah, so, but there’s a, another eigenvalue, the second largest. And what that captures is, is a society converging fast or slow. So the second largest eigenvalue of an updating matrix, if it’s really close to one, that basically means that. You can, you can start people off. And even if the society is connected and people will eventually be tending to the same opinion, if they talk for a million years, it really will take a million years. They, the, the being close to one captures their being. And it turns out, as Matt and I, Matt Jackson and I discovered to re relate to this phenomenon of homophily, that if your network is basically if and only, if, the only way that can happen is if there are divisions in your society where people put very little weight across Democrats and Republicans or whites and blacks.

Ben: Uh, andso if that happens, you can converge really slowly and if it, and if the second eigenvalue is, know, not too big, like 0.7 or 0.5, then disagreement is gonna decay like what you Seth was saying before, 0.5 to the end, right? So it gives this beautiful one number measure of the slowness.

Andrey: what if, what if, one of us was very stubborn and just didn’t really care what other people thought about them? Would their opinion end up dominating the entire belief process, or were they just washed away in the average?

Ben: Oh, if, yeah, so, so if there’s someone who’s super stubborn, they don’t listen to, the extremists, they really don’t listen to anyone. They put all their weight on themselves and

Seth: Those are, that’s our rival podcast. Dogmatic posterior.

Ben: Exactly. So, yeah, so that’s, that’s a way to be very, that’s a way to be very influential. In fact, at the extreme, wewouldn’t even call that society connected because this one guy’s not really connected to anyone.

Seth: It might be connected out. I don’t know. Maybe.

Ben: yeah. But even if he puts a tiny little weight on others, if he’s stubborn enough, he’ll still dominate

Seth: And would that be bad?

Ben: usually. But unless he’s very, well, unless he’s very well informed, unless he, and so yeah, we, we ordinarily consider that bad because. A benchmark we like to, in a realistic case, we like to think about is information is dispersed. Everybody. Nobody know. Nobody knows God’s truth. Exactly. But everybody has has reasonable Yeah.

Ben: Nobody has

Seth: The average of this room knows God,

Ben: Exactly. Exactly. We do. you, if you could take, if you could take the God’s eye view and look at everyone’s information together, it would be enough to tell you like a whole, whole lot. But nobody, but everybody’s individual estimates are pretty, are pretty noisy. And so now how do we, how, can decentralize social learning, which DeGroot is supposed to be a simple model of get you to that.

Ben: Well, it really depends on whether one guy monopolizes all the influence or a few guys or, or di, whether influence is dispersed.

Seth: As, as the population goes to infinity, do we have, influential nodes, right, is the way you put it.

Andrey: So,

Seth: gonna ask the LLM question? Andre? You go for it.

Andrey: one second,

Seth: One sec. We’ll get there.

Cow Tipping and False Beliefs

Andrey: Ben. I don’t, I don’t know if you remember, but we, we’ve actually done a podcast before.

Ben: I was thinking about.

Andrey: Now. In that podcast we discussed the interesting phenomenon of cow tipping and how people seemingly believe that this is a thing that one does, even though no one actually goes cow tipping. So my question to you is, the past since

Seth: Thanks for ruining the joke, Andre, for literally everybody.

Andrey: Uh, in the past, year since, since we’ve done the podcast, have you noticed any social learning on this topic? Is it now understood that cow tipping is not a thing or is it still a belief that’s propagating

Ben: That’s very interesting. I have stopped using it as a, I, I somehow found that I have not used it as an undergraduate teaching example since COVID, now that you bring it up. So one thing, something happened to me during COVID teaching. I was teaching my, this was the last year, 2020. I was teaching the last undergrad class I taught at Harvard in fall of 2020. And it was a wonderful group of students actually, but they were all dispersed. Some, most of them at their homes. A few of them lived in like group houses with other students. And I was doing the cow tipping lecture in the way it goes. Just for the, to a little more context. Yeah. So like, it’s a great,

Ben: how many people know what cow tipping is? One thing I’ve noticed by the way, is fewer hands go up because I think Varsity Blues and that generation of movies was an important, was the way that it got into the culture. And kids these days don’t have an, watch those movies. So I don’t know whether they’ve been exposed, but, but these kids sort of knew, they were like, I was like. I asked, the usual question is I asked some factual questions about it. Like, what do you think is the prevalence in the United States? How many incidents of cow tipping have there been in the last year? And people will say, very few people will say like a firm zero. Um, but in the Zoom class, one of the students, they had their, like, their apparent or a relative in the background, and they were like, no, cow tipping happens.

Ben: I’ve seen it. So then I had to, like, in the middle of my class, I have to interview this person to, assess like whether my whole understanding of things is wrong. It wasn’t a very exciting, I was like, well, did you see it? Like, what, what did they, what did

Seth: Is the cow tipper in the room with us right now?

Ben: exactly, they were like, they were like, well, they, they were drunk and they really like ran at the cow and they hit the cow.

Ben: And I’m like, then what happens to the cow? And they’re, I don’t know, I ran away. So that’s the usual, that’s like

Seth: Are you saying that, the eigen values of the cow’s response to tipping are less than one? Is that,

Ben: Exactly, yeah. Is I, values are very important in mechanics. So. But for the other piece of context, en engineers have written papers kind of proving that you can’t under reasonable assumptions, like, knock over a cow with your shoulder or

Seth: are you gonna tell us that Santa’s not real, dude? What is this podcast about? We’re just killing people’s joy. Or, anyway, I’ll let you finish your example.

Ben: In terms of false beliefs, I think things are bad. I think my, my naive sense, it’s very hard to know ‘cause we don’t, you have to really study it and scientifically, but we had like a, since my wife and I have have, had a baby, we’ve interacted with, like, we had a baby nurse live with us for three years and she, she was from a very different community.

Ben: You know, she’s like, and I heard things her friends were saying, and beliefs and my, my sense is that. Strange beliefs about matters of fact are very much out there. And, and I, and I feel like TikTok, I think like TikTok propagates them actually in a way that’s more powerful thanany vector I knew that I personally experienced.

Ben: Like when I was in high school, for

Seth: Is that interesting? I mean, is that surprising from a DeGroot perspective? ‘cause it seems like in from a DeGroot perspective, you get communities with weird beliefs ‘cause they’re disconnected. But now the statement is they’re connected and that’s giving them weird beliefs.

Ben: I think what the basic DeGroot model is missing is that people talk about things very, that that people’s propensity to, to. First of all, I don’t think like these beliefs, like claims of cow tipping or other urban legends or, or wild statements about what Hillary Clinton does recreationally are like, I don’t think they’re like deru where we average what people think.

Ben: You just propagate interesting information. And I think what the DeGroot model is really missing and a lot of models of social learning is that what people share depends a huge amount on whether they think it’s interesting and like surprising and much less on whether it’s true. And moreover, people don’t adjust for that when they hear, right?

Ben: Like Tyler Cowen might, but most people, they’re not, they’re not aware of that bias in the information they’re hearing. And so they’re not, adjusting their posteriors. They’re just kind of accepting, you know? And, and so I, and I think TikTok has made it much more power, much more, much more viral to say something really interesting and get it into a lot of minds.

Ben: And that’s more like a yes on or off viral state, not like, do you believe, not like. What, what do you think the interest rate’s gonna be next, next quarter, but more like, do you think that people really landed on the moon, like a yes or no? Or you do you believe in some crazy conspiracy that’s like, like more like a virus that takes hold of you and it’s not a matter of degree of belief.

Sequential Bayesian Learning and Herding

Seth: Well, so if people, if people aren’t good bayesians, another model that you’ve worked with is called, the, or sorry, I guess a Sequential Bayes. If people, if people aren’t learning this connected way, maybe they’re learning in this kind of sequential, sort of herding-y way, which is sometimes called a Sequential Bayes model.

Seth: Uh, Andre, are you gonna let me move on to this topic? Or you wanna jump in with something?

Andrey: make a, I wanted to make a very brief observation since we’re talking about this. I happen to notice a book in the, in the background of, of Seth, actually The Hype Machine, which is

Seth: My machine with Ana roll. Yes. What’s, yes, what he says. It’s, it’s not true. Things that spread. It’s, novel and emotionally intense things that spread. So shout out to, a friend of the show. Sinan Aral.

Seth: All right. So, yeah. All right. So pe, so pe No, that’s good. No, that’s good. So people don’t learn in this connect way.

Seth: Maybe. Maybe, maybe they just see what the last guy did and try to figure out the state of the world from that. Is that a better model of what you’re describing, or is it also wrong?

Ben: I think what I’m describing some, some, like, having in mind intending to propagate, a little pellet of false information, like people tip cows. I think that’s just like a virus and that’s a good model. It’s also not be irrational. I mean, I think there’s some rationality to it, but I think the best model of it is like, if it’s interesting enough, it goes viral and a lot of people believe it, but Seth absolutely, like the models, Bayesian sequential updating where you hear something. I think where that model really shines is in thinking about something like, which, you know. Should I get, should I get flood insurance for my house or which accountant in our, there’s like three accountants in our industry and which one should I use? I think there, people think very much like what that model posits, which is I could research this, I could get my own signal.

Ben: I don’t have any special confidence that I would be particularly good at that. And this other person, I know that what they, that they’re not probably acting on amazing information either, but it’s probably still got a little more information content in it than mine. And let me just, so let me just follow and so you end up with a lot of like in economic context that I think are important.

Ben: I think the, the choices people make about insurance. Like when I talk to people their, who thought their whole lives about do people buy enough fire insurance or flood insurance or whatever, they basically talk about it like a social convention. And so you, you buy some and you don’t buy other, and you don’t buy stuff that people around you don’t buy.

Ben: Not because you’ve taken any time to analyze your personal, portfolio problem, but just because you assume other people have it like. That the social signal contains more information than you’re likely to gather.

Andrey: There’s also an interesting aspect of it, like if you follow the herd, then even if it goes wrong, you’re like, well, who can blame me for, for doing that? Right? But if you go against the herd, like, oh, that idiot didn’t buy insurance. Like he deserves what he, what he got. Right?

Seth: You have to get an awful, strong signal.

Ben: in a business context, right. There was this saying nobody ever got fired for buying IBM because, and that was exactly hurting on IBM, that at the, are you gonna really get blamed for using the same vendor that everybody uses?

Seth: So, how does, so is, is that great? We all coordinate on doing the right thing, or can that fail somehow? Why, why wouldn’t that be a good approach to learning?

Ben: You absolutely get big. I mean, the main was, the main first result about the herding model is that you can get quite dramatic failures of information

Seth: Oh no.

Ben: Where? Um. If people did experiment, if people, if we could ask like the first a hundred people to make this decision to ignore the social signal or just deprive them of access to other people’s past choices, and we made them decide based on their private signal, then we’d get a hundred hunches aggregated, and that would, and then after that we’d have a hundred people’s information, averaging into some vibe about what the sensible thing to do is.

Ben: But, but the sequential model shows that if you, if, if the first people already are contaminated by having access to previous decision makers, it’s just rationally they won’t get this started. So you have a kind of tragedy of the commons where collectively, we could like. Maybe compensate the first movers or just pick some of us to be unlucky and have to make this decision solo. And we would, society would learn a lot that way from, but, but what we in fact do is just, herd and actually online platforms spend a lot of energy thinking about like how to get enough experimentation going on. You know, should Google re Google Maps recommend, shortcut that it doesn’t think is the best to learn about it, should Yelp send people, try to send people to a restaurant that it doesn’t think is the best to get more information about it.

LLMs and Information Aggregation

Seth: How does LLMs change all this? Alright, so I’m kind of split ‘cause I kind of feel like these two models have different implications for whether it’s gonna help or hurt with aggregation failure. So help me out with this. It seems like in this sort of sequential Bayesian framework, LLM sort of should hurt our information algorithm, aggregation, right?

Seth: Because, nobody is in the position of being ignorant. We can always just question the model. The model tells us what the last hundred people did. Uh, we’re gonna herd harder by virtue of all having, none of us being in that state of ignorance, that state of blissful archipelago ignorance. Do you think that that is a mechanism that’s potentially at play?

Andrey: Wait, Seth, can you just clarify something? Why

Seth: Please,

Andrey: LLM tell you what the last a hundred people said necessarily? I,

Seth: it’s gonna tell me what the last hundred books written about the subject are. Let’s say.

Andrey: I mean, we can take that as a premise. I’m not sure if I’d buy it, but,

Seth: I mean, well what are they? They’re based on, this is what I’m trying to say is LLMs are based on the things LLMs have read. Andyou might say maybe this is a version of model collapse, right. LLMs are based on the last hundred on some thing of some of the last things. The LLMs read

Andrey: The last

Seth: just the last hundred tokens.

Seth: And then, somebody reads that and then they write a book based on having read the LLM. And now we get herd to whatever our opinion was in 1850.

Ben: What do you think buying it?

Andrey: no, I mean, I just, I, I guess it depends on the decision, right? But to, to the extent that models are able to reason and to the extent that your,

Seth: What if it’s a pure fashion question? What if it’s, what if it’s just black shirts are in versus white shirts are in? Could it, could it lead a stronger herding there?

Andrey: Well, it would rationally know that you don’t wanna wear what everyone else is wearing. Right. I mean, I mean, there’s a, there’s an element of like, that it can really be, have a lot of context about you, which is different than else.

Seth: Yeah.

Andrey: that’s, that’s the aspect where I’m not exactly sure that that’s how we should model it, but I’m happy to consider that version of the model.

Andrey: Sure.

Ben: Um, yeah, I’ve never thought, I haven’t thought about it in a sequential learning setting exactly. But I think there’s a different, a different dimension which seems related and important, which is like a narrative that I’ve heard repeatedly and that I think has a lot of truth about what’s happened to western society and politics is that there used to be, a focal provider of, of focal baseline, of facts, basically

Seth: Catholic church.

Ben: well, I would say the six o’clock news,

Seth: Six. Okay. All right. I always wanna go. I always wanna go back to Habsburg times. Dude, you can see this is my Habsburg wall.

Ben: I don’t know. I, and I think this was probably a unique moment because I’m not sure, I think that, that the newspapers we should ask like, Gentzkow and Shapiro about, newspapers in 1900, which was I’m sure a very different, environment with all. But like, there’s this moment which is now kind of seen, which is, valorized a little bit, that there was the, a national truth and you could, you had to get pastsome regulatory, there was regulatory exclusivity for the major broadcasters and basically nothing too crazy.

Ben: You could get broadcast too widely that Right. And then we move to this TikTok world where, where it’s a free for all. And, and it does seem like, that has some, the breakdown of a shared reality seems like an, something that’s happening to some extent and now coming like. ChatGPT. It’s, I think it’s a real empirical question.

Ben: To what extent in normal people’s normal lives does that serve as like the six o’clock news? Again, the coordinating device. Um, if you’re debating something, my wife Annie, who’s, who’s a also a Northwestern professor, had a hilarious story at a dinner she was debating. She went to MIT and she’s a big MIT snob and always reminds me that Caltech, where I went to for undergrad is way worse and is like way less cool.

Ben: And so there was, but to my surprise, her dinner can be, I wasn’t at the seminar dinner, but a guest of ours thought that Caltech was great. So I was like, the kids, it

Andrey: To.

Ben: and she was, yeah. And she was like, and he was like, wait, are you telling me that if you ask, you ask 10 people, they’ll all who, who care about this?

Ben: They’ll say that MIT is better. She was like, yeah. So of course they took out ChatGPT and that settled, and she,

Seth: Pirate, get John Horton on the phone. Tony Stark went to, Tony Stark, went to MIT Dude, that’s what people know about.

Ben: So I thought that was, and I think that’s gonna happen a lot around a lot of dinner tables and kind of, it has an effect. I, I think of it as a shared, I think of it as a powerful shared signal. Um, andI think that really reshapes things, in, in a lot of different ways. Um, that’s the main way I’ve been thinking about it.

Andrey: You know, it’s, it’s funny ‘cause what I, my very opinionated bias take is that the average quality of the undergrads atCaltech is obviously higher than at MIT in my experience, and I think a lot of people who know would agree.

Ben: Yeah, I think that’s, I think she’s been a little bit per, I think she’s been a little persuaded over time because my, my, my good friends, like the, the relationships I’ve kept from undergrad are, um. John Schulman, who was a, who was there, were two of the biggest ones. Or John Schulman, who was a, one of the, was maybe the, is often credited as being, a creator of chat, GPT andAdam D’Angelo, who’s, who is of course the co founder where I worked and and is a, is a very big figure in ai and I think that does you, there there’s a sort, so I think that’s made a, made an impression actually that there’s some kind of person that the place was good at incubating

Andrey: So

Seth: so

Andrey: is all listeners. This is actually all a ploy to get John Schulman on justified posters.

Seth: come on.

Ben: those two are Caltech alum in case it, it was not.

Seth: Uh, so, okay, so, so let me, so let me take that argument a step further. So, the way we should, one way to think about LLMs in the social information aggregation function is as being a central node that all of us are connected to. Um, we, you just reminded us that in these DeGroot models, having, influential node in the long run means that influential node gets to, set a little bit of the opinion and it might not just be the average of everyone’s opinions.

Seth: Is the concern there, or is the observation there that, whoever ends up controlling the most important three LLMs ends up having a real thumb on their scale in the opinions of society.

Ben: yeah, exactly. So, it’s funny when it, when Matt and Jackson and I were working on this in 2007, 2008, were very, the ba the basic first observation is exactly what, what you said, that if one person gets a lot of weight, they’re gonna, their errors are gonna matter. They’re gonna contaminate everything.

Ben: And so they’re gonna prevent, even if society as a whole has the information collectively to wash out all the error, the fact that this guy talked in a way, first or talked loudly, means that everybody’s going to be influenced by whatever. That note says, but there is an exception. Or when you try to prove those things mathematically, that’s not necessarily true because something that can happen is if that note is very good at themselves being an aggregator, and it actually does, it figures out the right information.

Ben: Um, and rebroadcast, that’s also one of the most efficient ways of figuring it out. So I think

Seth: A

Ben: the

Seth: post, a reliable pollster.

Ben: Exactly. And so the selfer, there’s something irritating about the Selfer, way in which some of these AI companies regard themselves, or it’s like that they, thinking really earnestly about stewardship of, of, the model’s preferences or whatever.

Ben: But I actually think this, that, it, if the model is say left bias, this liberal liberal bias, then that’s gonna, um. it into a lot of opinions andthat matters. And so they, they should think about it. And I, I do actually admire efforts that they make, to be basically good aggregators, good pollsters.

Ben: And interestingly, like before we could have pollsters on a few issues that you could distill numerically, but now this is a pollster that kind of up internet text about anything. It’s like a qualitative pollster, which is a really remarkable kind of device that we couldn’t have imagined when we were writing those papers.

Seth: Should we be RLH fing these models so that they have the median social opinion on all social issues?

Ben: What does that even mean? Right? How do you

Seth: I, you go to Pew and it says, the median person thinks abortion should be legal at 27 months. Whatever. What? Sorry? 27 months. 27.

Ben: But even that,

Seth: 27 weeks. Okay.

Ben: didn’t like. The interesting thing is that the LMS are doing their own embeddings of these issues into their, so people will just talk to them and say, and talk about abortion in a way. They’re doing an averaging but not one that’s, that’s, that’s numerical one that’s qualitative. And, and I, I kind of like it that way. I, I, I don’t think people have coherent views on almost any issue of public interest. And so if you try to make it numerical and try to average it that way, that would be like garbage and garbage

Seth: Right.

Ben: and.

Seth: Trying to recreate the mind of the median American voter will make you insane.

Andrey: I, I really wanna go back now to this personalization aspect of things, right? Um, it, especially with something like Chad, GPT, I don’t view it as a monolith. There is a model router involved. It has all your previous conversations. And if me and you asked it a question, and this is an interesting, it would be an interesting empirical exercise actually, is like. We might get a very different answer about like, is it, is it, normal to, I guess, I guess it depends on what we’re asking. It’s like one of the things like for myself, like, is it, should I wear a hoodie to a business meeting? Right. You know, and it might give me a different answer than you guys.

Seth: Did play League of Legends during the business meeting.

Andrey: yes, yes, Uh, but, but if I ask it, what does the average person in society think about this question? We might get the same answer, but I don’t know, these things are a little unpredictable in this way. Right.

Ben: Yeah, and there’s a bunch of

Andrey: I.

Ben: papers just suggested by what you just asked, right? If people, because of course the system prompt. If you’ve done a, if you’ve now had your custom prompt, all bets are off because you could, you could ask it. Please don’t tell me. Things that might upset me with this mental illness that I have.

Ben: And then they, we wouldn’t get probably accurate answers on, on if it’s really, then it has. So yeah, people do get, the personalization issue is super interesting. but for now, yeah, I just wanna make the point for the moment that as a focal before the market has matured to the point that there’s a niche little LLM for everybody, these items are actually new kind of animal in the, they’re not like Facebook, they’re not like they’re, they’re a new kind of sort of public object that everybody interacts with.

Ben: Um, and despite the heterogeneity that Andrey said, they, that’s, that might shift things in a way closer to a, a, a former time.

Seth: Or will people just all choose, I’m a lefty going in, so I’m gonna use lefty, LLM, and you’re already going in. You’ll use righty. LLM.

Ben: Right. But it is, isn’t it remarkable that gra, I mean, there’s like a popular Twitter joke, but after trying, after trying to train the wokes, the, sorry, the, the anti wokes, LLM imaginable, it has like, it has like wine mom views, like

Seth: You can only, you can only, you can only, right wing eyes, the LLM so much.

Ben: Yeah. Except on the rare, like, it’ll say, it’ll occasionally say Hitler is great, but other, other than that, it’ll like,

Seth: Only when it’s role playing.

Simulating Social Learning with LLMs

Andrey: has anyone tried to

Seth: Ooh.

Andrey: some of these social learning games with LLMs?

Ben: yeah, that’s, I, that’s a great, I I’ve been trying to learn, keep track of this. I, it’s been proposed to me by students. Um, and I know that there are people. That. So I was gonna say that when we, ‘cause before, before the podcast, we’d sort of discussed, some topics, and I’ve been thinking about this one that like, how will it affect social learning?

Ben: But it made me think, how will it affect studies of social learning? And now you can, you can, implement, you can simulate it, you can, try to forecast how groups of people would behave. And it’s interesting because people like John Horton have done studies of how good is it as a simulator of a, of an individual. the question of how good is it as a simulator of a community, would be super interesting. I think just intellectually, I’m sure people are doing it. I’d love to, if people listening are aware, I would love to like tweet it at me or something.

Seth: You heard it, folks, dm d dm, Ben, with all of your, simulation ideas

Andrey: yeah.

Ben: tweet.

Andrey: Well, I, I guess theclosest thing that I

Seth: posted on our Discord I’ll, we’re at the, we’re at the end.

Andrey: Yeah, is the, is the AI village, know, where the, there are like different ais, different models, and they’re like co cooperating, slash they’re given a task to do and they see if you can do the task. And some tasks are like, can you sell a t-shirt online?

Andrey: Or something like that. And it’s hilarious how they try to cooperate with each other and all their foibles andso on. Uh, which is kind of not narrowly the, the specific formulation of social learning, obviously, but related,

Ben: Yeah. Yeah.

Lessons from Quora and Startup Experience

Andrey:so one, you, you mentioned, your friend Adam D’Angelo. I’m curious what, what you learned, at Quora, that you’re bringing to your current startup experience, or alternatively what you learned at Quora that you brought to your research.

Ben: Yeah, that it was such a formative time that I really didn’t understand at the time, how important it would be in my life. That I think the biggest thing, I never thought I would, I never expected that I would do anything entrepreneurial just because, I think that for one, I didn’t expect that there would be a technology like AI that would be kind of like, have the exact shape that, that is, has been important for, for me to be able to actually try to do something, at the technological frontier.

Ben: But at that, but I was, what was remarkable to me is that I

Seth: Thought you said linear you, I thought you knew that Linear algebra destri describe the world and you’re the king of eigenvalues. Come on, dude.

Ben: No, but I guess I never had that deep faith or I thought it was a few steps away that I was upstream in

Seth: Mm-hmm.

Ben: the innovation

Seth: Fair enough.

Ben: of commercial applications. But I remember, like, it was huge for me that they, that they were, that Adam’s always been very interested in economics. He just reads, like he reads texts on industrial organization recreationally.

Ben: And, and I think he had, he always had this respect for economists. Um, that was very, and and so he would, we would just occasionally chat about things often through the lens of economics. And Quora had some specific, he had some economic ideas of for, well, one thing I did was moderation. ‘cause I was just a very active user.

Ben: So I was involved in kind of, some of the housekeeping of the moderation operation, which I actually wasn’t good at. So I, my, at the time, the interesting thing is I wasn’t like, I wasn’t a good community community manager and but, but when, then, when I was in the company. Adam got curious about this idea of credits and actually having an internal currency, and that so that people’s like, basically so that the scarce resource of some people’s attention, like, especially on early Quora, a lot of the answers were written by really visible people whose, who were, people were very excited to see them there, but their attention was scarce.

Ben: So how could you efficiently bid for people’s attention? You wanna create some kind of token, right? And so I was just like the consultant who, thought about the very basics of the design of that system, like the central banking. How much money do you issue it? How do you, but that was what I did. but what I learned was actually like just getting to watch a startup. And it was right at, when I joined there were about, I think 27 people. And so seeing a startup at that stage, I learned a huge amount about. About running a business andespecially in tech, I think the strongest, people often say that startups are like a magnification of the founder’s personality. Um, and I think that’s really true in this case. ‘cause,

Seth: Getting, getting how, how, frustrated it, refined was with some of my notation where it was like, you called this a node. I, it took me a while to figure out what you mean, but I would not call it a node. Uh, your personality really does come through.

Ben: it’s funny because, yeah, I’m very, I’m very pedantic. I, I’ve spent, I, I, I feel, yeah. So I’ve created, and Adam is very, very thoughtful and deliberate and kind of like likes to make decisions with principles and in a thoughtful way and make decisions, like I think a lot of good, good leadership skills, like focus on, focus on one focal goal at a time and and. Propagate that and communicate that. And then, think really thoughtfully about design The core was a very design first company andmaking design decisions, not as an afterthought, but as a core thing. I think there were a lot of those like principles, I think similar to growing up in families, like there’s just certain values that are embodied in where your environment.

Ben: And when I was there, like I realized after that I, I’m a pretty good sponge and I wasn’t directly involved in any like, decisions having to do with design, but you know, the guy I sat next to at Quora was, Joel Lewenstein, who’s now the, the head of design at Anthropic. And I can, and like, but I didn’t, I think what the amazing thing is, it was this like, combination of amazing people and all of them were really thoughtful and really good at what they did.

Ben: And they talked about startup uping in a very intellectual, thoughtful principles first way. And so that when I, I, when it came time to think about a business, I felt like. That was a natural way to be, and I realized I never would’ve had the, that kind of, those kinds of vibes, if not for those six or eight months that I spent there.

Andrey: Very cool. Um, do you have any thoughts about why more companies don’t use virtual currencies and have you thought about the use case of virtual currency for internal allocations of GPUs?

Ben: Great questions? Um, I think virtual

Seth: You imagine going to Walmart and they tried to pay you in Walmart coin instead of money, people would riot.

Ben: Yeah. Well, but you could, I mean, internal currencies. I think one of the problems that, I wasn’t around when Quora eventually decided to get rid of them, but I think one of the problems is that, um. Currencies are focal and they create people, they, they motivate people to do things in a way that they sort of take up too much oxygen in the ecosystem. And so when you’re designing a social product where you want many kinds of incentives to be in balance, having a currency can actually be harmful to the, it’s a kind of a sociologist insight, but like, so I think there’s some of, I think you have to be really, I think for platforms where that are truly transactional and economic currencies are always good.

Ben: And usually that currency becomes money. ‘cause it’s gonna have an exchange rate with real money

Seth: Right.

Ben: Um,

Seth: Love one price.

Ben: yeah, but for, but I think for, for. It is, I think it’s an interesting phenomenon that needs to be thought about more. Why it’s not, why it’s really generally not a successful route for social for internal markets. I, I’m very, I I believe that some of the obstacles to internal markets are just frictions having to do with like, basically contracting frictions. Um, and one thought that I have had for a long time actually discussed with, we had some there. Let me just, I, you guys will edit. Let me just say that again. One thought I’ve been thinking about for a long time is just as contracting intermediaries. Um, and

Seth: This is a big theme of the

Ben: Andrey

Seth: Coasian Singularity Dude.

Ben: Yeah. This is Andrey’s paper.

Andrey: Yeah. So what, what is your thought about this? Yeah.

Ben: I’m very curious, so I’m very curious for your take on it since you’ve thought about it much more seriously now, but it just, yeah, I think I feel like. A lot of the details were just like implementation details, that if it became your job to implement it at a company, you would, you would decide that it’s, you’d have to really have a high valuation of the marginal allocated efficiency of that currency. And it’s arguable that it’ll, it’ll be, it, I think experiment experimenting with it has just become way more valuable once we reach the LLM, capability of being trustworthy to like, negotiate a contract, which I think honestly is not right now, but yeah.

Ben: I, I see that as a potential, a big organizational impact. I’m very curious what you think.

Andrey: I mean, surely the contracting aspect would be hard. but I also think there’s a social aspect to it as well, right? You’re the CEO, you create an internal Coasean internal market for GPU resources, then you suddenly see a team that you don’t want using the GPUs, using a lot of the GPUs. Now, what do you do

Seth: The whole point of, yeah, the whole point of having a firm is to have a command DI economy. If you wanted everyone making independent economic decisions, you wouldn’t have a company right.

Andrey: but there’s a sense in which there’s some optimization that you want your teams to be making, like leaving idle GPUs or they’re using them very stupidly for some reason, and you don’t, you want that to be kind of disincentivized and. The way it’s currently done is through these very imperfect monitoring systems and people asking very nicely, can I have, this resource?

Andrey: Right? So yeah, I’m, I’m curious whether the, the AIs can do a better job here.

Ben: Yeah, I mean I guess the, you might shortcut you, they’re also becoming better at being the arbiters of requests. Right? So maybe, maybe rather than, but, but I do think money is, one memory I have of Quora actually is that the engineers, they hadbrilliant young people and I very like. Who were first principles thinkers too.

Ben: And so people would ask me also, I had to just like justify money to the whole, to like the skeptics in the whole company. And so I gave, gave a lot of thought

Ben: Yeah, why don’t we have some more multidimensional expression? Right. And there are good answers to that. It’s like very helpful that money is very legible.

Ben: That, but, but I guess we, yeah, for companies, I’m very much with Seth’s point that if you really believed in the power of the, of monetary incentives to, to do it, you, you wouldn’t have a company, but you may find it a useful tool within the command. I mean, even, even the command the North Korea has has currency, right?

Ben: So like it’s definitely a tool. And I think with the Pareto frontier has changed, but I don’t know how

Closing

Andrey: Very, very cool. So, we’re just about out of time. Uh, is there anything either of you want to add to our conversation?

Seth: Ben, do you have any good eigenvalue jokes for us?

Ben: oh man, I should have prepared.

Seth: Alright. We had Ben Golub today who’s made tremendous strides in automated paper reviewing and still has a lot of progress to be achieved on automated Eigenvalue joke, doing, thanks for tuning into this episode of Justified Posteriors. Please like, share, and subscribe. We now have a hoppin’ Discord community for now by invite only DM us on substack Twitter or LinkedIn for your personalized invite code.

Seth: And why don’t you keep your posteriors justified?

Andrey: Thanks, Ben.

Best Pairing: Breakneck: China’s Quest to Engineer the Future by Dan Wang and
Natural Moralities: A Defense of Pluralistic Relativism by David B. Wong

How and why do China’s and the U.S.’s political cultures differ? This pair of books, each by a leading D. Wang/Wong, comes at the question from two very different directions.

Breakneck says the divergence is due to China having a leadership/political culture focused on engineering, while the U.S.’ is focused on law and lawyers. Dan Wang argues that the excesses of the one-child policy and lockdowns, and the failure of the US to build infrastructure, can all be understood as downstream of this decision. Dan has good conversations about this take at Conversations with Tyler and on the Sinica podcast.

Now, don’t get me wrong, the thesis and the anecdotes used to illustrate Breakneck are excellent. But in both the book and his podcasts, Dan doesn’t engage with what I see as the spiciest question prompted by this theory. Namely, when and why is it that the two cultures diverged?

If China is just on the standard Solow growth path, with a US 1950s need for engineering leadership, and will naturally converge to US 2020 levels of lawyerly leadership, this is a VERY different hypothesis than the two countries having fundamentally different moral and political inheritances. If the latter is the case, Tyler’s objection that (paraphrasing) “Chinese lawyers might just make autocracy more efficient” has purchase.

Isn’t it plausible that a mythic canal king, irrigated rice farming, and a unified empire make a society different than one downstream of Greek philosophy, chivalrized barbarians, and protestantism?

In David B. Wong’s “Natural Moralities: A Defense of Pluralistic Relativism,” this deep divergence in culture between the US and China is a central theme. D. Wong argues for a form of moral pluralism he calls “pluralistic relativism”. Under this view, dramatically different moral systems can be equally moral without descending into anything-goes moral relativism.1

IMHO, this is a very attractive move for two reasons: (1) It’s more plausible than natural law theories of “absolute morality”, while still being able to make the obvious point that some social systems are better for human flourishing than others. And (2) It’s a step towards a vision of how universalizing Westerners can have constructive dialogue with Oriental moral systems -- an essential need in a century that will be defined by East vs. West rivalry.

This brings me to why Natural Moralities is such a valuable pairing with Breakneck. According to Wong, but in my own words, the divergence between Confucian and Western moral theories -- at a deep level -- is that Confucians are shape rotators while Plato and the Judeo-Christian philosophers are wordcels (“In the beginning was the word…”).

How What You Should Do When Your Dad Murders Someone Explains the Difference Between the U.S. and China

To illustrate this, David Wong brings up a story from Mencius, which I’ll contrast with one from Plato. The question both philosophers are faced with is “What should you do if your dad kills someone unjustly?” Before I give their answers, maybe think for yourself what you’d recommend.

Mencius’ Answer:

(Note: Gu Sao is Shun’s dad)

桃應問曰：「舜為天子，皋陶為士，瞽瞍殺人，則如之何？」

Tao Ying asked, saying, ‘Shun being sovereign, and Gao Yao chief minister of justice, if Gu Sou had murdered a man, what would have been done in the case?’

孟子曰：「執之而已矣。」

Mencius said, ‘Gao Yao would simply have apprehended him.’

「然則舜不禁與？」

‘But would not Shun have forbidden such a thing?’

曰：「夫舜惡得而禁之？夫有所受之也。」

‘Indeed, how could Shun have forbidden it? Gao Yao had received the law from a proper source.’

「然則舜如之何？」

‘In that case what would Shun have done?’

曰：「舜視棄天下，猶棄敝蹝也。竊負而逃，遵海濱而處，終身訢然，樂而忘天下。」

‘Shun would have regarded abandoning the kingdom as throwing away a worn-out sandal. He would privately have taken his father on his back, and retired into concealment, living some where along the sea-coast. There he would have been all his life, cheerful and happy, forgetting the kingdom.’

Here we see a classic shape rotator approach to a moral dilemma: The state needs to enforce justice, but a son needs to protect his father. We get a compromise that hopefully leaves everyone somewhat happy -- Shun should abscond with his father, removing him from being able to do more crimes, but still protecting him.

We also get advised that, despite what we’d call a tragic clash of values in the West, Shun should still try to feel good about himself. To your taste, Mencius’ answer is either a nice compromise or a stupidity that fails to satisfy any plausible theory of justice.

Plato’s Answer:

In the Socratic dialogue “Euthyphro”, Socrates runs into a priest who has decided to turn his murderous dad in to the justice system.

Unlike Mencius, who tries to split the difference and make everyone happy, Socrates decides to confuse things further. He questions:

Socrates
But what is the charge, and what is the suit about?

Euthyphro
Murder, Socrates.

Socrates
Heracles! Surely, Euthyphro, most people do not know where the right lies; for I fancy it is not everyone who can rightly do what you are doing, but only one who is already very far advanced in wisdom.

Euthyphro
Very far, indeed, Socrates, by Zeus.

Socrates
Is the one who was killed by your father a relative? But of course he was; for you would not bring a charge of murder against him on a stranger’s account.

Euthyphro
It is ridiculous, Socrates, that you think it matters whether the man who was killed was a stranger or a relative…

Socrates
But, in the name of Zeus, Euthyphro, do you think your knowledge about divine laws and holiness and unholiness is so exact that, when the facts are as you say, you are not afraid of doing something unholy yourself in prosecuting your father for murder?”

In the rest of the dialogue, Socrates proceeds to shoot down every theory of Euthyphro’s about the nature of piety and justice.

The reader is only left with more questions: “Are the gods just because they behave justly, or is justice simply what the gods command?” and “Is piety something different than a commercial relationship with gods?” In classic wordcel fashion, rather than actually contribute to solving a social dilemma, Socrates critiques and deconstructs -- and, of course, his dialogue is much, much longer than Mencius’ answer too!

When framed as wordcel vs. shaperotator culture/morality, I think the Breakneck distinction between Lawyer and Engineer states makes more sense, and is actually a deeper and more interesting thesis. It also puts me closer to Tyler’s view that just adding more lawyers to China’s system won’t actually result in more individual protections and Western-style justice.

I think I can see the unique strengths of either approach, while still feeling secure in the fact that we each have a system that works well for us: A Western system of critique, individual reason, an openness to the idea of tragic conflicts, and an insistence on conceptual clarity and rights vs. a Confuscian system of practical problem solving at the expense of some of that clarity and Western roadbumps to ill-concieved grand plans. I hope that books like these, which attempt to see the logic in each other’s systems, can be an important step to peaceful coexistence.2

Best Non-Fiction: The Allure of Battle: A History of How Wars Have Been Won and Lost by Cathal J. Nolan

An alluring possibility: What if the real battle is the allies and economic capacity we build up along the way?

The start of this book is not promising. An overlong introduction of the main theme — battles and great generals are overrated; attrition and grand strategy underrated — which bounces between obvious and unsupported claims.

But what comes next is the greatest single-volume history of warfare I’ve ever read. A masterful tour from Marathon to the Marne, his discussions of individual wars are better than many dedicated books I’ve read.

His coverage of the evolution from pike and shot, to line infantry, to skirmishers is excellent — especially because it’s not presented as a series of innovations by great generals. Rather, the author has a fresh take focused on the interaction of generals’ desire for maneuver warfare with changing fortification and siege technologies, as well as a focus on how quickly these technologies and strategies can diffuse through repeated encounters.

The author’s main argument is simple: (1) the relative cumulative economic power of sides is the most important determinant of who wins long wars (2) long wars are expensive also, and therefore (3) Revisionist powers are tempted to plan around short wars because these are the ones that would hypothetically help them. This leads to (4) Revisionist and over-confident powers quickly find themselves in over their heads, leading them to lose long wars.

Decisive, quick, victorious maneuver warfare is the dream of a Frederick the Great, a Von Moltke, a Yamamoto. The author does a fantastic job of explaining this doctrine — the lust for a costless victory, ideally a “cauldron battle” that would exterminate the enemy army in imitation of Cannae.

But then the author makes an amazing, obvious, and yet hugely underappreciated point — why do we idealize the victory at Cannae, when Hannibal’s strategic failures are what determine the course of the war?

The author explains why. We idealize the great army geniuses of the past - in part to get adolescents psyched about war, in part to glorify national genius, but worst of all, to justify irrational wars of aggression by revisionist powers. The Japanese in WW2 wanted to revise the international order, but they weren’t capable (due to internal division in large part & aggressive leaders taking international actions) of aligning themselves on the stronger side of a global conflict (or at limiting the spread of their conflict). Therefore, the only answer was ever more aggressive attacks in the hope of destabilizing, in a series of brilliant campaigns, stronger opponents.

The argument is basically right. I am convinced. Great book. But it is possible to over-learn this lesson. France’s side may have eventually won WW2 - perhaps inevitably due to the network of alliances- but their failure to keep up with Wermacht initiative at the beginning of the war made everything so much worse.

As a child, I fell for the romance of Hannibal. In some ways, the fact that he loses in the end is almost a plus - a heroic standing in the face of the Roman tsunami. But a much better hero is Fabian, the delayer, who, rather than pushing for a quick resolution, showed the patience necessary for Rome’s advantages to inevitably tell.

Where does that leave the US today? I conclude that maintaining the alliance system is more important than ever. No country, including China, can challenge the US + EU + India together. We couldn’t be conquered in decades. Even if these nations cut their militaries to the bone, we could still hold out and win a long war - so long as we remained unified! It also makes me worried for an Israel that, drunk with operational success, may find itself isolated and overextended.

In sum, Grand Strategy>>>>>Operational Art>=Tactics.

Best Sci Fi: The Hydrogen Sonata by Ian M. Banks

I read The Culture series as mainly about two things: (1) What would it be like to live in a utopia? And (2) How did (and should have) the US acted internationally during its 1990s unipolar moment. I had been holding off on reading this, the last in the series, wanting to savor one of my favorite series for longer. But I finally gave in.

I was not disappointed. The book delivers well on both (1) and (2) as well as pointing out interesting connections between them. The core metaphor is the humanoid protagonist’s dedication to mastering playing an impossibly stupid four-arm-requiring string instrument. No spoilers, but if you’re interested in either theme, I highly recommend this series. It can be read out of order, and this is possibly my favorite, so feel free to jump in here.

Best Play/Opera: Salome by Oscar Wilde

I read the play in advance of seeing the excellent Met production. I appreciated how Oscar anticipates the concept of the “male gaze” and how sexual abuse perpetuates itself. The play is ambiguous in a way that leaves “The Dance of the Seven Veils” and related scenes somewhat sexy. Oscar’s language in listing the great gifts offered by King Herod is hypnotic.

Strauss’ music and the Met’s staging illustrate the play well. The Met production renders explicit how fucked up Salome’s abuse was, using seven Salomes at various ages, all dressed in school-girl clothes, to make sure you don’t miss the point, at the expense of sexiness.

Wildcard: The Pine Barrens by John McPhee

If you’re not from New Jersey, you’ve probably only encountered the Pine Barrens through hearing of mobsters dumping bodies there or perhaps the Jersey Devil cryptid. Even as someone from North Jersey, my understanding did not extend much further than that. But I really enjoyed learning more about it in this tightly written exploration of an anachronistic region nestled discretely between Philly and NYC.

The pines have always been sparsely populated, even in indigenous times, because of the sandy soil unsuitable for most agriculture. It has been a haven for Northeasterners who have wanted to get off the grid for centuries: as America’s first Native American reservation, for escaped slaves, and for Loyalists during and after the Revolutionary War. Today, it is known for its excellent blueberries, which were intentionally selected, cultivated, and spread by Rutgers University biologists.

The writing is brisk and respectful while not above pointing out some of the funny or absurd parts of piney life. Truly an underappreciated corner of America!

Most Laughable Economic Theory Joke Award: Ecstasy: Understanding the Psychology of Joy by Robert A. Johnson

I have to mention this one because I got the best laugh I’ve had of the year out of it -- but that was unintentional, the book is pretty bad.

You may have read about the distinction between the Dionysian and the Apollonian introduced by Nietzsche. Like a D&D alignment table, this chaos vs. order axis is orthogonal to conventional morality, but an important aspect of human psychology. I highly recommend reading “The Birth of Tragedy” by Nietzsche, or “Psychological Types” by Jung, to learn more about this distinction! The idea that we are cut off from our emotive, intuitionistic tools for creating value is a compelling one, but one difficult to balance with our modern virtues of reason and order. It’s a really good big idea!

This book is only sometimes about that big idea, and like many in Nietzsche’s and Jung’s shoes, it doesn’t share their talent for connection and subtlety. Instead, in this book, we get something in between DBT and Jungian shadow-self work.

Some of these ideas are not necessarily bad -- understanding your counter-social impulses and integrating them is great. But some of the ideas advocated are actually pretty bad and scary. The book seems to advocate different pseudo-schizo approaches to emotional healing with the shadow self -- from building a little shrine of your idol, to doing crazy calisthenics your dream-Dionysis tells you to do.

In a very short book of 97 pages, it’s clear the author is running out of steam by the end, with two of the last chapters devoted to reciting not particularly exciting dreams he or a client has had.

The book is at its funniest when the author -- who admits to not being very good at book writing or history -- completely makes up a political-economic history of the suppression of Dionysis and his replacement with the debauched Bacchus.

The peak of this, which I’ll leave you with, is my new favorite theory of the price level. From page 45, on Dionysus as “scapegoat”:

“Sheep represent everything of value in our Judeo-Christian World. The sheep, in fact, is the chief determinant of our currency. Every currency in the Western world -- the shilling, the franc, the deutche mark, the lira, the peso, the Austrian thaler (from which we got our dollar) -- was the price of one sheep. For centuries, there was no inflation in the Western world because one of our money pieces was worth a sheep. You could count on that anywhere, anytime.”

Literally laughed for a solid 10 minutes. Unconstrained by reason, the author gave me a moment of joy. And isn’t that the most Dionysian thing of all?

Honorable Mentions:

Abundance -- Agreed with it too much to find it interesting. But it’s the book I’m “rooting for” the most this year.

Democracy in America part 1 -- Great, but “The Ancient Regime and the Revolution” is better, and more unified in its thinking. This is foxy and hard to summarize, but ofc a deserved classic.

The Fundamentals of Heavy Tails -- Great primer on a topic I’ve launched myself into this year.

Help Wanted -- Read on the recommendation of Jason Furman, a nice little slice of life about minor drama at an upstate NY big-box store and the people who work there. Some good boots-on-the-ground economics about how management, economic incentives, loyalty, and hope play out at a place like this.

Fortune’s Formula: The Untold Story of the Scientific Betting System That Beat the Casinos and Wall Street -- Good for its discussion of the Kelly criteria and the hilarious fight between gamblers and Samuelson over whether it’s deeply true. (Spoiler: Obviously, it’s only utility maximizing from a single specific perspective, but it’s an awesome and useful heuristic for long-lived institutions.)

They discuss the data and the assumptions that go into this benchmark. Seth and Andrey start by walking through the tests of task length, from simple atomic actions to the 8-hour research simulations in RE-Bench. They discuss whether the paper properly measures task length median success with their logarithmic models. And, of course, they zoom out to ask whether “time” is even the right metric for AI capability, and whether METR applies the concept correctly.

Our hosts also point out other limitations and open questions the eval leaves us with. Does the paper properly acknowledge how messy long tasks get in practice? AI still struggles with things like playing Pokémon or coordinating in AI Village—tasks that are hard to decompose cleanly. Can completing one 10-hour task really be equated with reliably completing ten 1-hour subtasks? And Seth has a bone to pick about a very important study detail omitted from the introduction.

The Priors that We Update On Are:

1. Is evaluating AI by time (task length) more useful/robust than evaluating by economic value (as seen in OpenAI’s GDP-eval)?

2. How long until an AI can autonomously complete a “human-month” sized task (defined here as a solid second draft of an economics paper, given data and research question)?

   • Seth’s Prior: 50/50 in 5 years, >90% in 10 years.

   • Andrey’s Prior: 50/50 in 5 years, almost certain in 10 years.
     
     Listen to see how our perspectives change after reading!

Links & Mentions:

• The Paper: Measuring AI Ability to Complete Long Tasks by METR

• Complementary Benchmarks:

  • RE-Bench (Research Engineering Benchmark) - METR’s eval for AI R&D capabilities.

  • H-CAST (Human-Calibrated Autonomy Software Tasks) - The benchmark of 189 tasks used in the study.

• The “Other” Eval: GDP-eval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks by OpenAI

• AI 2027 (A forecasting scenario discussed)

• AI Village - A project where AI agents attempt to coordinate on real-world tasks.

• Steve Newman on the “100 Person-Year” Project (Creator of Writely/Google Docs).

• In the Beginning... Was the Command Line by Neal Stephenson

• Raj Chetty

  Transcript
  
  [00:14] Seth Benzell: Welcome to the Justified Posteriors podcast, the podcast that updates its beliefs about the economics of AI and technology. I’m Seth Benzell, wondering just how long a task developing an AI evaluation is, at Chapman University in sunny Southern California.
  
  Andrey Fradkin: And I’m Andrey Fradkin, becoming very sad as the rate of improvement in my ability to do tasks is nowhere near the rate at which AI is improving. Coming to you from San Francisco, California.
  
  Andrey: All right, Seth. You mentioned how long it takes to do an eval. I think this is going to be a little bit of a theme of our podcast about how actually, evals are pretty hard and expensive to do. Recently there was a Twitter exchange between one of the METR members talking about their eval, which we’ll be talking about today, where he says that for each new model to evaluate it takes approximately 25 hours of staff time, but maybe even more like 60 hours in rougher cases. And that’s not even counting all the compute that’s required to do these evaluations.
  
  So, you know, evals get thrown around. I think people knowing evals know how hard they are, but I think as outsiders, we take them for granted. And we shouldn’t, because it certainly takes a lot of work. But yeah, with that in mind, what do you want to say, Seth?
  
  Seth: Well, I guess I want to say that we, I think we are the leaders in changing people’s opinions about the importance of these evals. The public responded very positively to our recent eval of Open AI’s GDP-eval, which was trying to look to bring Daron Acemoglu’s view of how can we evaluate the economic potential economic impact of AI to actual task-by-task-by-task, how successful is this AI system. People loved it. Now you demanded it, we listened. We’re coming back to you to talk to you about a new eval—well not a new eval, it’s about eight months old, but it’s the Godzilla of evals. It’s the Kaiju of evals. It’s this paper called “Measuring AI Ability to Complete Long Tasks,” a study that came out by METR. We’ve seen some updates or new evaluations of models since this first came out in March of 2025. Andrey, do you want to list the authors of this paper?
  
  [3:05] Andrey: As usual I don’t. There are a lot of authors of this paper. But, you know, I’ve interacted with some of the authors of this paper, I have a lot of respect for them. I have a lot of respect for the METR organization.
  
  Seth: Okay. But at a high level, just in a sentence, what this wants to do is evaluate different frontier AI models by the criteria of: “how long are the tasks that they complete”?”
  
  Andrey: I guess what I would say before we get to our priors is, just as context, this, from what everything I’ve seen, is the most influential evaluation of AI progress in the world right now. It is a measure that all important new models are benchmarked against. If something is above the trend, it’s news. If something is below the trend, it’s news. If something’s on the trend, it’s news. And it’s caused a lot of people to change their minds about the likely path of AI progress. So I’m very excited to discuss this.
  
  Seth: It’s been the source of many “we’re so back” memes. Yeah, I totally agree Andrey. Am I right that this was a paper that was partly inspiring the AI 2027 scenario by favorite blogger Scott Alexander?
  
  Andrey: I don’t know if it inspired it, but I think it was used as part of the evidence in that. Just to be clear though, AI 2027, it’s a scenario that was proposed that seemed a bit too soon of a vision for AGI taking over the world by many folks. We have not done an episode on it.
  
  Seth: We haven’t done an episode on it. But it’s fair to say that people look at the results of this paper and they see, you know, they see a trend that they extrapolate. But before we get into the details of the paper, are we ready to get into our priors?
  
  Andrey: Let’s do it.
  
  [05:50] Seth: Okay, so Andrey, just based on that headline description, that instead of evaluating AI systems by trying to go occupation by occupation and try to find tasks in those occupations that are economically valuable and then trying to see what percentage of those tasks the AI can do—that’s what the Open AI GDPval approach that we recently reviewed did—this approach is trying to evaluate tasks again by how long they are. So comparing those two approaches, I guess my first prior is, before we read this paper, which of those approaches do you see as like kind of intuitively more promising?
  
  Andrey: One way of thinking about this is tasks are, or things people do which could be a series of tasks, are bundles and they’re bundles embedded in some higher dimensional space. And what these two evals are doing, this one we’re discussing here versus GDPval, is they’re embedding them into different spaces. One of them is a time metric. And one of them is a dollar metric, right? And you can just by phrasing it that way, you can see what some of the issues might be with either. With the dollar metric, well, what are people getting paid for? Is it a specific deliverable or is it being on call or being the responsible party for something? So you can see how it’s kind of hard to really convert lots of things into dollar values at a systematic level. Now, you can say the same thing about how long it takes to do something. Of course, it takes different people very different times to do different tasks. And then once again chaining tasks together, how to rethink about how long it takes to do that. So I think they’re surprisingly similar.
  
  I think maybe this length of time one is more useful at the moment because it seems simpler to do frankly. It seems like, yes we can get an estimate for how long it takes to do something. It’s not going to be perfect, it’s going to be noisy, but we can get it and then we can just see whether the model does it. And that’s easier than trying to translate tasks to dollar values in my opinion.
  
  [8:42] Seth: Right. I guess I also am tempted to reject the premise of this question and say that they’re valuable for different things. But I guess I come into this thinking about, you know, we think about AI agents as opposed to AI tools as being this next frontier of automation and potentially supercharging the economy. And it really does feel like the case that working with AI models, the rate limiter is the human. It’s how often the human has to stop and give feedback and say, “Okay, here’s the next step,” or “Hey, back up a little bit and try again.” So going in, I would say I was kind of in equipoise about which of the two is the most useful kind of as a projection for where this is going. Maybe on your side of the ledger saying that economic value is kind of a socioeconomic construct, right? That could definitely change a lot even without the tool changing. Whereas time seems more innately connected to difficulty. You can think about psychometric measures of difficulty where we think about, you know, a harder exam is a longer exam. So at least going in, I think that this has a lot of potential to even potentially surpass GDP-eval in terms of its value for projection.
  
  Andrey: Yes. Yeah, yeah.
  
  Seth: Okay. The next one I was thinking to ask you Andrey was, if we buy all the premises of whatever context the paper sets up for us, the question I’d like to think about is: how long until AI can do a human month-size task on its own? In the abstract of the paper, we have that happening within five years, by 2030. That seems like a pretty big bite at the apple as they say. Do you want to take a stance on how long until an AI can do a human month-size task? I mean, I have to say in my use of AI, I haven’t gotten anywhere near that.
  
  [10:55] Andrey: What is an example of a human month-size task?
  
  Seth: What’s something that takes 160 hours of work? I would say, you know, as an academic, maybe I need kind of three months of focus on a paper to bring it from zero to, you know, solid second draft. Maybe that’s like a third of a paper is a month of work?
  
  Andrey: I mean, it can do a third of a paper in a day. I mean I’m not being facetious here. I referee a lot of papers. Is the question an end-to-end, completely no-intervention sort of thing? Because I think like, look, you take Claude-code off into a folder, the folder has the data. You tell it, “Hey, like write a paper that does this, that investigates this question with this data.” It can do that in a day. I don’t think it needs... I think it depends on how much you require for human intervention. I think with something where there’s a verifiable answer, it’s very different than something subjective like a paper. Because I think we don’t want just any paper. We want the paper that we want to write. It’s not just about quality, it’s also about taste. And so I don’t think it could do “end-to-end write a paper that I like” even if I gave it a lot of scaffolding. I don’t think it could do that yet. But could it do that in five years? Sure, I think it’s possible.
  
  Seth: And just to be a little bit more specific, can we say gets published in like a top 10 economics journal level of quality?
  
  Andrey: The quality bars will have to increase. I mean, I think it goes to a question of like if I already have the research question and I know the data is adequate. Yes. Very few projects are of course like that, right? None of my recent projects have that flavor to it I think, where it’s just I’ve already found the data set and the question is obvious and I just needed to go plug and chug.
  
  Seth: There are papers like that. Raj Chetty gets the US tax records, and just needs to run some pre-registered analyses.
  
  Andrey: That’s an interesting one Seth. So Raj Chetty is an economist -now we’re really in the weeds - who does big public economics analyses. He works with gigantic teams on data analysis and iteration. It’s not as simple as just going to town on a dump of data. So yeah, I’d say that I can think of easier papers than Raj Chetty’s papers to implement.
  
  Seth: Okay, but if I want to think about the same kind of general format of question, right? Which is: I have a data set, I have kind of the general research question I want answered about the data set... let’s say the question is only specified at that level. I’m not being any more specific than that. Plus a data set. I don’t think an AI could make a plausible, complete, top 10 econ journal out of that right now. Do I think it could be there at a plausible level of quality in 10 years? In five years? Five years might be like exactly at my cutoff. I think in 10 years for sure. In five years, 50/50.
  
  Andrey: Interesting. Okay. Okay. So that’s... yes. So we’re both very bullish, huh? Okay. Well, you know, maybe it’s slow, but 10 years is fast enough that we’re not ready. In fact, my understanding of the METR organization is that a big part of its mission is to prepare us for AI progress that’s a lot faster than society is ready to deal with. And you know, I think it’s an important mission.
  
  Seth: That’s my mission too, Andrey. Also, they need to be prepared for slow progress. I want to prepare society for everything. Why prepare them for only one thing?
  
  Andrey: Society is already prepared for slow progress. Perhaps.
  
  Seth: Okay, are we ready to move on to the evidence?
  
  [17:34] Seth: Okay, so Andrey, we read this paper, or this Eval from METR. It looks at the probability of task completion as a function of task length across a variety of frontier models, starting with GPT-2 in 2019 and continuing through Claude 3.7, which is kind of early-to-mid 2025. And I would say the Eval works in sort of four steps.
  
  First is they establish a human baseline for how long it takes humans to complete 169 software engineering tasks --- By the way, in the abstract it does not mention that this is overwhelmingly software engineering tasks. I probably would have put that in the abstract, but you know who am I? -- Secondly, once we’ve got that baseline for each AI, we see whether it can complete each task. That was the quote you just gave us from Twitter. So once you’ve got the baselines, it takes about 60 hours of work to run each AI through the paces. Then we’re going to run a logistic regression of “Does AI correctly answer the task?” on “Length of task.” And then that gives you a data point for each model of: we think it has a 50% shot of completing an arbitrary task of a certain length. And then you put all of those points for all of different models from 2019 to 2025, and you see a diagonal line pointing from models that can do one-second tasks to models that can do one-hour tasks. And if you just extend that line out a little bit, that line’s going to take all our jobs. Isn’t that right, Andrey?
  
  Andrey: Yeah, yeah. So just to be clear, I think the numbers that I have for the extrapolations... if we think that the current horizon is about a couple of hours, and the latest model rated is GPT-5.1 Codex Max which is just under three, the prediction for February of 2027 is 16 hours. And for April of 2028 is 5 days. So that’s you know, and if we go further we get to those month-long numbers eventually.
  
  Seth: Okay. So maybe let’s take a minute to talk about that headline result. So they estimate putting all these models together a doubling time of approximately seven months. So every seven months we get a frontier model which is able to work for twice as long. They give themselves an R-squared of 98% in fitting what is it, 10, 15 points? Do you have anything to say kind of about this headline result before we dive in? The one thing I wanted to point out was this is all software engineering specific. So if you think that software engineering might obey very different doubling times than other tasks in the economy, this is only going to tell you about that one particular domain.
  
  Andrey: Yeah, yeah. And I think that’s a really important caveat. I don’t think there is as much care here in making the tasks as realistic as possible as was, let’s say, in GDP-eval.
  
  [21:35] Seth: Right, different priorities. GDP-eval very focused on like “what are useful tasks.” This kind of more focused on the abstract “short versus long tasks.” Maybe one other point I’ll make here which is a high-level point, which is something that they emphasize, which is if you think that there’s just some sort of constant error in their estimates, you can shift this entire graph down. But the important thing is the doubling time, right? And if the doubling time is seven months, sure shift the whole thing down, it’ll take one more year to get to whatever crazy outcome you want.
  
  Andrey: Yeah, and for what it’s worth, to me 50% completion doesn’t seem very relevant. Presumbly you want 99% completion, right?
  
  Seth: Yeah. I’d be happy much—you know I prefer to look—they have an 80% completion option on their site that you can plot and I tend to prefer that one. For that we have a number like that’s pretty current that’s around 30 minutes versus for the 50% it’s about 2.5 hours.
  
  Seth: There we are. Okay. So we’ve talked about the headline results. Maybe now let’s go kind of point by point and how we end up there. So the first thing that they need to do is establish a human baseline for how long different tasks take. They do this by combining three different data sets. The first one they do is sort of internal. They call them Software Atomic Actions. These are like really micro tasks. The example they give is kind of hilarious.
  
  The example they give is: “Okay Andrey, how long was it going to take you to answer this question? I’m putting you on the spot. Which file is most likely to have a password in it? Credentials.txt, InstallationNotes.txt, Main.py, or LauncherWin.exe?”
  
  Andrey: Wow. Wow that is a hard question Seth. I mean I kind of view these sorts of tasks as similar to kind of like cursor auto-complete tasks where like, you know, you don’t need a reasoning model for this. You’re almost like... let’s say you have a little bug in the code, it just auto-complete correct it. You know, that sort of thing.
  
  Seth: One thing I want to highlight about this... and they really they talk a little bit about trying to do what they can to reduce the noise from overhead from reading, from human reaction time... but it seems like they’re not going to do a super good job of distinguishing whether answering that question is a one-second task or a three-second task, right? But the difference between a one-second task and a two-second task is an order of magnitude here. And I guess I’m a little bit concerned if the logistic curve is learning too much about what’s the one-second version of that versus the two-second version of that.
  
  [24:54] Andrey: Yes. Yeah yeah. I mean yes, there is an argument to be made that due to measurement error just swamping everything that maybe we should only start with one minute or or two minutes. Now of course we can draw our own visual regression on that plot over there and see that you still have a pretty steep curve even if we throw out the first few points, right?
  
  Seth: Okay. So that’s done internally with their own kind of own engineers or just whoever was around. The second data set they draw on is something called the RE Bench suite or the Research Engineering Benchmark V1, which to quote from the paper consists of “seven open-ended ML research engineering environments and data from 71 eight-hour attempts by 61 distinct human experts.” So they’ve got these 61 guys that are doing seven of these tasks. And we confirm our experts make progress in these environments given eight hours. The third benchmark is H-CAST, Human Calibrated Autonomy Software Tasks. Designed to be a little bit more realistic to what a software engineering task would be in an economic environment. And they say that their baseliners typically have a degree from a top 100 global university and are primarily recruited via professional networks of METR employees. They’re paid $50 to $100 per hour plus $25 to $150 per hour in performance bonuses. Baseliners also did the tasks and predicted how much time it would take them to do the tasks.
  
  Curiously only 61% of human baselines actually successfully completed tasks, right? So one thing kind of we should be thinking around in the background here is we kind of want to compare how long it takes a human to do a task to can the AI do the task. But in reality it’s like like we talked about, it’s higher dimensional than that. There’s not just how long does it take a human, but with what probability can a human do it in a certain length of time.
  
  Andrey: Yeah. Or which human? And does the human have the context ahead of time? Or you know, are they an expert in this type of work or not, right? There’s no one number for the human.
  
  [27:38] Seth: Exactly. And for that third data set they record 189 tasks that they evaluate across which there are 563 human baselines. So I guess a second note here is these aren’t kind of giant populations of people. I just I guess you wouldn’t expect this to be giant populations of people. You know is 61 people being judged on their research engineering skills a lot? A little? I mean on the one hand 61 seems like a small sample for all of humanity, but on the other hand getting 61 serious software engineers’ time for a thousand hours is a bigger deal.
  
  Andrey: Yeah. Yeah. I mean it’s hard. I mean this goes back to our discussions of cost, right? I mean to do these sorts of metrics well, especially for valuable tasks, is just very expensive. You know look, there’s also this question of which population do we want to sample from? In the economy, experts are oftentimes doing the work. And that expertise can be very very narrow, right? You know think about just you know economists. You know even if economists are using different methods, you’re you know one person studying you know the medical industry is going to have very different expertise than a different person studying you know the energy industry. Even if like they use the same methods. So yeah I think the question of what population you want to sample is an interesting one.
  
  Seth: Very very well put. One other detail here that is interesting but it’s kind of mixing together some pretty different evals here. The RE Bench, unlike the other ones where they just see how long it takes a person to finish it and conditional on finishing it how long did it take you, for the RE Bench they kind of give everyone eight hours and they figure out like what the average quality of people were able to do in those eight hours and that’s going to be their cutoff for an eight hour length task. So a little mix and matching going on. I’m not saying that they P-hacked this but there’s some informality going on.
  
  Is there anything else you want to say in the creation of the bench lines before we move on?
  
  Andrey: Well I think there’s one other data that they use which was the internal PR pull request experiments. I don’t know if you read this part where so they ran these models on some issues in the internal METR code base. So these are ones that would not have been in any training set certainly. And they found that their contract baseliners take 5 to 18 times longer to resolve these issues than the repository maintainers. So the people whose job it is are 5 to 18 times more efficient than contract baseliners on this on these tasks.
  
  Seth: So the idea is METR coders are very smart boys. And girls.
  
  Andrey: No, they actually don’t say that. They actually don’t say that. And I disagree with your statement here. Not that they aren’t smart, but more that they say that it’s all about context, right? Like if you’re dealing with a code base and you’re very used to it, you can diagnose the problem very easily. You can solve them very easily. If you’re not, then it takes you a while to load the context back in. I mean we’ve all had this. You know you work on a research project, you take a little break for a few months and now you come back and you know something that you know should be very simple takes you a few hours because you know you just don’t remember the code anymore, right?
  
  [31:38] Seth: I wanted to bring up one last point here Andrey before we move on, which is around the question of how many people do we need to establish the correct baseline. So we’ve already talked about context matters, like have I already loaded in the prior knowledge or am I coming in cold? Am I a super smart expert or am I a man off the street? Those are all definitely mattering. But one thing I’d like to point out is that if we think that some of these tasks have a very long tail in completion time... right? Which seems really plausible for a very hard research engineering task, that you know some people can do it in a short amount of time and some people take twice that and some people take twice that... a very long tail... as the variance of people’s abilities to complete this task goes up, you know you’re going to be less and less confident in your estimate with a small N.
  
  Andrey: Yes. Yes yes. I think that’s right. But once again it’s not clear to me where we want the minimum... whether we want the average or the min. There’s a very good argument for the min.
  
  Seth: Right. If what we care about is superhuman ability then I guess we want the min.
  
  Andrey: No, or or just like a comparable to a professional working on the code base. Not even superhuman right?
  
  Seth: Do we really want the strict min? If the question is “how long does a certain journey take”, I’m not sure we want to include the person who by chance had just looked up that number.
  
  Andrey: Like I think the min is perhaps too far... but something much closer to like what someone day in day out of the code base would do rather than you know... one is how much do you accelerate a company with an existing code base with professional software engineers. Like for me maybe that’s not the relevant benchmark. I’m not a professional software engineer. And so I don’t care if it’s better or worse than the best professional coder. I care if it saves me time. Which could be you know much more economically relevant if we think that the value of better software engineering is coming from the fact that now everyone can be a software engineer.
  
  Seth: I think that’s very fair. But as we get deeper into this I’m becoming more convinced that if you really care about economic value you should be reading the GDPval paper not this paper.
  
  Andrey: Okay. Okay.
  
  Seth: So the second step of this process is for each AI seeing whether it completes each task. Right? So we’ve got these benchmarks. We’ve got the short benchmarks, the medium length benchmarks, the long benchmarks. How many can each AI do? I guess the one note I want to bring up here is they do some basic scaffolding. They claim it’s not elaborate. They try to bring some agent tools to the early models. So early models were like not set up at all for these longer projects but they try to give it like a little scratch pad and a little “remember these are the most important command line codes.” It seems like they’re not going to do a super good job of distinguishing whether answering that question is a one second task or a three second task. But you could imagine a version of this test that would have zero scaffolding or a version that would have very elaborate subtask specific scaffolding and they’re kind of closer to the first.
  
  Andrey: Yeah and I think that’s fair to have a comparison baseline. It’s also becoming less and less representative of how people are using the models, right? I think if you’re serious about using the models you’re giving them skills and putting in the right context. Certainly you’re using a cursor or Claude code or a codex where there’s a lot of optimizations there. So you know one one argument here is like actually if you’re if you’re serious about using these models they’re actually a lot better than what’s portrayed in this benchmark.
  
  Seth: Yeah I think that’s definitely right. And again one of the running themes of this podcast is “Bitter Lesson” and how important is the frontier-ness of the model versus the customization and the specific task orientation of the model. We don’t really get... you know they just say we do light scaffolding. And I guess before we move on, the range of tasks here are all designed so they can be done through the command line. So there’s no kind of... it’s not like Chat GPT immediately fails everything because it can’t make a picture.
  
  Andrey: Seth, I thought that everything could be done through the command line. In fact Neal Stephenson famously said…
  
  Seth: In the beginning there was the command line. That’s a good book. That’s a good book.
  
  Andrey: Cryptonomicon for those who don’t know.
  
  [36:10] Seth: No, he has a book, he has an essay collection called In the Beginning Was the Command Line also.
  
  Andrey: Yes that’s true yes that too yes.
  
  Seth: And in the essay collection, this is the one thing I remember, is he compares Macs to the Batmobile. ---Seth Cuts in With Correction: Actually he compared Mac OS to a luxury European car, Windows to a station wagon, Linux to a free tank, and BeOS to the Batmobile. Apologies to Mac OS fans for comparing their OS to the Batmobile -- It was a very 1990s book. It was like OS Wars book.
  
  [37:01] Andrey: I just say that Neal Stephenson in terms of the pantheon of prophets... (Seth: he got crypto right). He got Uber right. He got virtual reality right. Wait wait wait. Okay. So right. Crypto. (Seth: He does think that there needs to be a big pile of gold somewhere. Which turns out to not be the case. Maybe he gets stable coins right.)
  
  Yeah but but I guess yeah there are many things he got right and and certainly in Snow Crash that were way way ahead of their time. It’s one of those things where you almost imagine that the sci-fi author kind of causes the subsequent innovations. And maybe with AI there’s a similar sense to that because so many people who’ve developed these technologies were inspired by reading science fiction.
  
  Seth: And the AI is reading the science fiction too Andrey.
  
  Andrey: Yeah well it’s not clear whether we want the AI to read the science fiction. It might develop some weird notions of what might happen in the future.
  
  Seth: Yeah. Read Bicentennial Man, don’t read Frankenstein. Let’s leave it at that. Okay. I could talk about Neal Stephenson for a whole episode. So let’s hold off on that. Okay. So the third step we promised the listeners is running the logistic regression.
  
  So what we have here at the bottom of my screen I’ll put up is for each of the models that they evaluate you can see this nice logistic curve that starts at 100% for a sufficiently short task and moves down to 0% for a sufficiently long task. And I don’t know Andrey, I look at these curves and a lot of them don’t seem particularly logistic. A lot of them are not monotonic even. It seems like you’re assuming the conclusion if you think that AI can do all one second tasks. I my read is that AI cannot do all one second human completable tasks. And like the idea... logistic models are one parameter models. So like we talked about, it’s learning just as much about this curve about from going from four seconds to eight seconds as from going from one hour to two hours. Which seems like the wrong way of thinking about it.
  
  Andrey: Yeah I mean I guess is it really that different than just finding than just extrapolating the point at which it has a 50% success rate? And then you know if we actually look at that point non-parametrically it’s it’s pretty it seems like like pretty close to where where we end up right? So I guess like one argument here is actually if you’re if you’re serious about using these models they’re a lot better than what’s portrayed in this benchmark.
  
  Seth: The 50-50 point. I think for a lot of these if I was trying to draw a diagonal line I guess my midpoint, my 50-50 point would be similar. I guess I don’t know how to think about like this GPT-2 example where…
  
  [40:37] Andrey: Sure. I mean but I think we already both like kind of argue that we might as well toss them. And it wouldn’t really make a difference. So let’s toss the early ones.

Seth: We’re not going to focus on the ones that can knock all these one second tasks out of the park. One thing I I guess think about is there seems to you know they talk in the caption for this figure about a jump in between the the the atomic tasks and the H-CAST tasks. And you do kind of see that in a bunch of these figures. But then I also see a jump at the eight hour tasks right? Because we know that there’s a lump of eight hour tasks that they get from the RE benchmarks. You know this is not to like punch down on a paper that is like a really good paper is definitely inspirational um and definitely influential correctly. But I think when you dig into these curves I am not convinced that the logistic model is definitely the right model. And then I guess then I lose maybe a little bit more faith than you do that were correctly finding the 50-50 point in these.

Andrey: Yeah. I mean I guess the other... I just don’t... yeah. I think there are other criticisms that are much deeper than than this one is maybe what I’d say. No no no. We already mentioned them. These are programming tasks. They’re very selective. (Seth: Yes. Yes. Yeah. There are other deeper criticisms. We’ll get to those.

Seth: You gotta put... dude how do they not put that in the abstract? I don’t know. That’s that’s something I ask. I mean the only… I’ll tell you why you don’t put it in the abstract and not to cast aspersions... it’s the hubris of someone who thinks that software engineering is the is the final task.

Andrey: Tell me tell me about these messiness scores. Did you read about those?

Seth: Right. They have 16 of them. Um I I I’ll why don’t you tell us about the messiness scores Andrey.

[42:50] Andrey: Yeah so so there’s an idea that like look if you have a very well defined task... like implement some algorithm... you know verify that the results are working... you know that’s way easier for an AI to do than “Hey you know I don’t know how to solve this problem you know try a bunch of things and solve it for me.” That’s very messy. Like you don’t you know you don’t um really know what the right solution there is no maybe objective solution to that um and so um you might think of a dimension here that’s messiness in addition to some sort of difficulty uh level. And and so they have a bunch of ratings uh of the messiness of uh these different tasks and yes there’s yes and and one thing I’ll say is that most of these tasks are not very messy. Now what else will I tell you is like you know working at my job most of the tasks I do are super messy.

Seth: They wouldn’t give... they don’t give you the easy jobs Andrey.

Andrey: No no no no. I mean and maybe you know look once again like maybe the intern is getting these very non-messy jobs but I am not. So so I do think it’s an important dimension. Not to say that the AI can’t do the messy jobs. They’re not even in the data set that’s being evaluated here.

Seth: Right. I think that’s a very fair point right. Which is this is a set of tasks that is really designed to be as amenable as possible to sticking the agent on it and coming back later right.

That’s that’s intriguing right it’s like and it’s inspirational and it’s uh vertiginous is maybe the word I want to use. Uh but it maybe doesn’t extrapolate directly to um normal people’s interaction with these tools right.

One other way I might want to frame this and we talked about this in the beginning is that problems are sort of tasks are multi-dimensional. They have lengths but they also have messiness. They also have difficulty. They have you know verbal difficulty and math difficulty and difficulty on lots of different dimensions. You could imagine a world in which there’s lots and lots of evals. More than 169. Maybe let’s say a thousand of these benchmarks. And we could actually estimate something that’s kind of multi-dimensional right. So success probability as a function of the length of the task, the verbal difficulty of the task, the you know the math difficulty of the task. And then throw in model year as just another parameter. Or as another interaction term.

Andrey: What an economist. Just add more fixed effects.

Seth: Dude machine learning! Let there be interactions too right. Let let it have whatever shape you like. Um that’s the dream. Maybe it’s an unrealistic dream given how expensive you know even putting together 160 benchmarks are. Uh but it seems like if you wanted to estimate the role of year in how good model is in doing thing you would want a model where year is a parameter in the model.

Andrey: Yeah yeah. I mean for what it’s worth you know there aren’t that many models... yeah I guess there are more there are a lot of... let me take that back. There aren’t that many frontier models. There are a lot of models that are around. But I think this benchmark is really focused on the frontier models and and you know over the course of this year we’ve maybe had the 10 total frontier models. So it’s not you’re if you want to if you want to run that regression you know you’re gonna have too many parameters.

Seth: Well here how how about this? Right? Which is you don’t only focus on frontier models. You just try to do this prediction not as a function of like model frontier you know is this the frontier model and year. Is you do it as a function of model size. And maybe there’s instead of one frontier model every year there’s one frontier model at each size every year. And you can get a little bit richer data.

Andrey: Sure. I will tell you that we actually don’t know the size of the frontier models.

[47:04] Seth: Yeah they don’t they don’t tell us. They don’t say it’s got a gazillion parameters. It’s secret. You know I... all right keep your secrets meme. All right.

Andrey: Well look uh to any of our listeners at the various illustrious labs uh a little tip might be appreciated if so we know what what sizes we’re working with.

Seth: Okay. So that’s a fair point. I think another point I would make here is that when we’re talking about secrecy is that the evals also have to be secret right. You know as if I’m putting on my reviewer 2 hat I kind you know I want to see the evals. I know I understand that you can’t put them on the internet because then the AI companies will cheat at the evals. But uh it’s a non-optimal thing that they have to do.

Andrey: Yeah and there and there is a sense that some of these tasks that they do have them do are a bit leaky. Who you want you want… I have some intel…

Seth: You want to name names?

Andrey: No I mean look I haven’t dug into them myself but having talked to some having gotten some intel. Let’s just say that they’re not they’re not some there not things that are that different from what you might have trained on a lot of time.

Seth: Okay. All right. So are we ready to sort of start talking about uh discussion limitations? I feel like we’ve run through the paper now. Is there anything else you want to say in terms of the technical sort of evidence side before we move into kind of more free-wheeling discussion?

Andrey: Let me just uh kind of now uh say you know this is a really I think this is a really important topic and episode for us because I I truly do think that this eval is driving so much of the conversation and uh most of the people have not read at all what the eval um is. And I will and I will especially thank so I’m in this uh Twitter group chat called uh the the “Demon Economics Research Unit” with a lot of uh very uh based uh participants who pointed me to various resources various very interesting writings on this eval that I that I benefited greatly from um when when thinking about limitations here. Um so let me give you a limitation that one can think about. Have you ever tried to watch uh Sonnet play Pokemon Seth?

[49:37] Seth: I oh I remember really early on I remember like Chat GPT yes I do remember Chat GPT plays Pokemon right. But it was like it was no I know I remember Twitch plays Pokemon and it was terrible right. I I do not I have not seen Claude play Pokemon. How what is that like?

Andrey: Uh it’s pretty slow going. Um and it’s not not very successful. Uh it’s a game with no fail state you can just keep on grinding it. Yeah just to be clear like a child can play this game quite successfully. Um and uh this is something an AI just has a very hard time doing. A number I have here is that not the current Gemini but but I think it was Gemini 2.5 Pro took 888 hours to minimally beat Pokemon uh that’s elite four that’s not capturing all Pokemon yeah with a dozen intense human handholds like tile labeling.

Seth: Wow.

Andrey: So so it’s very easy to think like hey uh these numbers in you know you to naively look at this graph and you’re like yeah now it’s four hours now it’s you know so on. But but let’s think about something like Pokemon which which humans can do quite well where where even when the AI can do it the amount of tokens involved is is immense. It’s just staggering.

Seth: When it will become when Andrey when will it become economically viable to export our Pokemon play?

Andrey: Yeah. Yeah I’m just say that look like obviously obviously these tokens are going to become cheaper over time and more efficient and whatever but but but you know like we have to take things with a grain of salt.

Here’s another piece of evidence that was brought up in the in the group in the group chat that I that I think was quite uh convincing to me. Have you ever heard of uh AI Village?

Seth: No.

Andrey: Uh so AI Village is uh is an experimental project where uh AIs uh personified with like the different models like you know Sonnet and Gemini and and GPT are coordinating on different tasks. Uh like what? Like Stardew Valley kind of? Yeah exactly. Yeah yeah. So they might be coordinating on um successfully uh selling a shirt online or getting some likes for for a web page or or something like that.

Seth: What so these are real world tasks or these are simulated tasks?

Andrey: Real world tasks. Okay cool. Real world task. And um you know you can I encourage everyone to go to it and see how well that’s going for the AIs.

Seth: Do they sell ten thousand dollar tungsten cubes?

Andrey: That’s that you know that’s a different interesting project but you know uh yeah project you know that’s a project called Project Vend you know maybe another one that’s in the same in the same vein. But but but this this AI Village just goes to show that uh these AIs they’re missing something. They’re not able to do things that humans can do quite easily. Um especially coordination but not just coordination. They just get tripped up on interacting with various pieces of the digital world. Um I’m a big optimist that that will be improved of course but um but we have to take these these time numbers with truly with a grain of salt.

[53:15] Seth: Right. I guess one thing one kind of question I had going in and I’m not sure whether we kind of get a hard yes or no answer on this is like to what extent is doing a two-hour task just doing two one-hour tasks correctly in a row right. Yeah. To the extent that it is to the extent that it’s just like a six sigma problem to the extent that it’s just like Waymo and it’s like okay you need to not crash one minute in a row a thousand times it seems like these extrapolations are pretty straightforward right.

But if on the other hand longer tasks are somehow qualitatively different because they involve complex interactions between subtasks interactions with the world in a way that you never do with one second tasks then these projections become a little bit more dubious. I guess I would also say that there are also reasons it could be easier to do these longer tasks right because you can always back up and retry right. Uh but I guess you know I wish there was a little bit more in here... I guess with the messiness they talk they get at this maybe a little bit but I wish there was more about like what’s going on beyond just reliability going up on each subtask.

Andrey: Yes yeah. Yeah I agree that would be very interesting. I mean one one version of that could be is it reasoning. (Seth: Planning) Reasoning is a constraint right like planning. Yeah planning um I yeah I don’t know. Um I guess like one one version of this is let’s you know one way we can think about this is that if 50% reliability is actually quite small and if we wanted to get to let’s say the reliability of um a good worker at a company maybe that’s a 99% reliability. Um so uh one argument that maybe the authors of METR might might bring is like look the trend is the same regardless of the percentage numbers and we just need to uh you know you can just shift everything down. But otherwise we’re we’re doubling very quickly and that still has enormous economic implications. And then you know um unfortunately we don’t have any evidence or not that we don’t have any but I would love to see a 99% reliability threshold in this benchmark.

Seth: It’s not sensitive enough right. I you know if there’s a hundred tasks right or a hundred and sixty that they’re doing right so you just not going to get 99% and you’d be worried yeah you’d be worried that it selected yeah.

Andrey: Yes yeah yeah. Um another comment like another very interesting critique.

Seth: Keep them coming dude these are all great.

Andrey: Um is is thinking through like what an actual human project requires in terms of hours. And there was this very interesting essay uh by this guy uh Steve Newman. I guess he developed Rightly which ended up becoming Google Docs. Uh and he talks about like uh something being a prototype um which was his initial Rightly um that kind of took about uh four months to build. Um it was kind of hacked together. And you know it was it was kind of self-contained and so on. And then he talks about a subsequent project he did um called Scalyr uh Sc- I don’t know how to pronounce it whatever. Um and he kind of estimated that uh that project that that product took a hundred person years to do. Which is not a crazy idea if you imagine you have a company and you have a hundred employees and it tooks you took you a year to build your initial project. I mean you know like most startups don’t don’t work that way.

[57:15] Seth: That’s a mythical man month.

Andrey: Yeah you know it’s not quite you know that but like there is some some some substantive or alternatively one way to think about it is like maybe what we need to get to is a hundred person years not like you know uh not even one year for a person. Right?

Seth: We need that for for whom?

Andrey: We need that for to have the AI end to end develop you know build things truly build things you know.

Seth: For you to really feel like I am to have that one person company right the mythical first the one one employee unicorn right.

Andrey: Yeah exactly. With zero employee unicorn you know.

Seth: Well I mean that’s I dude you gotta make yourself CEO.

Andrey: CEO is I don’t know as someone who has an S-corp that not necessarily you know…

Seth: Do you call yourself president? What’s your position at your S-corp?

Andrey: I think I’m president yeah.

Seth: Oh wow. I’m going to be Chief Czar of my S-corp. Does your does your S-corp have a fun Lord of the Rings name or is it like Andrey Consulting or something?

Andrey: It’s uh you know it’s a it’s actually very related to this podcast it’s called uh Justified Strategy.

Seth: Ooh I like it I like it. See you know you gotta the marketing synergies are obvious here. Yes yes. That’s something the AI can’t do yet.

Andrey: Oh man do you have any more of these hot limitations or or have I tapped you out?

Andrey: No I mean look I think I think we’ve said enough yeah on the limitations yeah.

Seth: We’ve done one we’ve done uh two man hours of talking about this. Okay. Yes yes. So let’s move into our posteriors.

[59:10] Seth: So uh Andrey um can I tell you a joke before we move into our posteriors?

Andrey: No jokes allowed.

Seth: Well I’ll tell you an unfunny anecdote then. Okay okay. I heard a joke once. A man goes to doctor. He says that he’s unevaluated. Says that life is meaningless and vague and uncertain. The doctor says that treatment is simple. The great evaluator METR is in town. Go and see them. That will get you evaluated. And then METR says to the doctor “But I am METR.” You know drum roll curtain closes. I mean it is it is so it’s so tempting to kind of want to do the meta thing here and like ask because it is such a software engineering-y kind of task the the evaluating. It is sort of surprising that uh you have that Twitter post saying that it takes them 60 hours 60 man hours to do the evals.

Andrey: I mean look I’m sure they’ve tried to automate more of it but yeah I agree it is very metapoint. It is uh...

Seth: Hopefully someone got a laugh out of that. All right. So the first posterior uh we have to come back to is is this more or less useful than GDPval?

Andrey: I look I I think it’s hard to argue that given where we are today that this is not more useful. Um it’s been this has uh been in the media a lot more than GDPval. I think one of the reasons it’s more useful is because lots of models are plotted against it. There’s more of a trend. Maybe GDPval will have this flavor going forward. Um but it is worth you know thinking through just the fact that GDPval is also way more expensive eval.

Seth: Right. I don’t know I dude G- I I know GDPval is way more expensive I vastly prefer that to this. This is a good paper I have nothing against this paper. But you gotta if it’s a if you can’t say this is about agents generally and then not put in the title that it’s just software engineering. I love the the breadth that GDP-eval tries to get at um that’s just not present here. I it is in- it’s it’s vertiginous to look at that curve going up to you know 10 hour tasks 20 hour tasks 40 hour tasks but the fact that it’s vertiginous and newsy doesn’t make it better necessarily.

Andrey: Sure sure. Um yeah I mean I hear that point.

Seth: The second thing we wanted to think about is how long until AI can do a human month-size task on its own. I came on saying that we we sort of we’ve defined that as do a good draft of an econ paper given a premise and a giant data set. You know viewers at home think about your own month-long task that you’re familiar with. Uh I said maybe 50-50 in five years and pretty conf- and you know 90% in ten years. This paper is a good paper it’s an intriguing paper but when you dig into it it says a little bit less than what it seems to on its face. So to the extent that I was thinking that we were going to be there for sure in 10 years and pretty con- and you know 50-50 in five years I at least I have to take a step back and put bigger error bars on that ladder one and maybe go down to you know 70-80%...

Andrey: I’m confused Seth. How could that be? Because if that was your prior... yeah yeah... this didn’t have negative information so you I would believe if you said your prior didn’t change…

Seth: No no no no. It signaled me down right so I so when I came into this paper I had an assumption about what this paper would say. So I had a prior that included “Oh and there’s this great paper that says 7 months.” Okay. I see. So your prior included already some notion about what the paper is. (Andrey: Okay got it got it got it got it got it. I hear you.)

So this paper was less impressive than I anticipated. And so um I think my five year estimate is maybe about the same but my 10 year estimate comes down a little bit.

Andrey: Yeah yeah I think I’m more confident than you that in five years we’ll have it. So my 10 year doesn’t um change very much. Um yeah I mean I think the interesting thing is like do we get there in two years or do we get there in five years? And because of the narrow domains here I I really there’s other evidence that like for example like Open you know we’re recording this as Opus 4.5 was uh recently released the latest Anthropic model that has updated my priors a lot more than um than this paper.

Seth: Yeah. Do you want to talk about that for a little bit and that can be our our wrap up discussion? What’s so what has impressed you about the the latest latest models?

Andrey: Um I mean look they have through a variety of benchmarks they seem very good but just I’ve had a chance to work with it yesterday and uh I was extraordinarily impressed.

Seth: Give me give me a little bit more dude just a taste. What was one cool thing it did?

Andrey: It’s too secret dude.

Seth: All right. Um let’s just say like it did it when thinking about like writing a paper it did something that would have probably taken me a week and probably about an hour.

Seth: All right. Okay. We that’s a week-long task uh 40 hours of work that’s uh off the charts in what we’ve been looking at.

Andrey: Yeah I mean I do think like one one constraint there I mean if you look at the clock time for me it was longer than an hour but I could use a lot of that time to do other things. I think but like my interventions into it were rel- you know they were expert but relatively minimal and it did a lot of awesome stuff on its own uh very effectively.

Seth: Right. So listeners at home we are not AI pessimists. We think that there’s a lot going on here. This paper maybe uh very intriguing vertiginous exciting maybe a little bit less than it seems uh on its face. Uh but we are watching this space and we’re we’re looking forward to see uh how good these agents get and how long tasks that they can do moving forward.

[1:05:47] Andrey: All right. Keep your posteriors justified.

Seth: And if you have another uh cool eval you want us to eval send it our way.

The conversation gets properly technical as we dig into Crawford-Sobel cheap talk models, the collapse of costly signaling, and whether “pay to apply” is the inevitable market response to a world where everyone can produce indistinguishable text. Bo argues we’ll see more referral hiring (your network as the last remaining credible signal), while Andrey is convinced LinkedIn Premium’s limited signals are just the beginning of mechanism design for application markets.

We take a detour into Bo’s earlier life running Google’s internal prediction markets (once the largest known corporate prediction market), why companies still don’t use them for decision-making despite strong forecasting performance, and whether AI agents participating in prediction markets will have correlated errors if they all derive from the same foundation models.

We then discuss whether AI-generated content will create demand for cryptographic proof of authenticity, whether “proof of humanity” protocols can scale, and whether Bo’s 4-year-old daughter’s exposure to AI-generated squirrel videos constitutes evidence of aggregate information loss.

Finally: the superhuman persuasion debate. Andrey clarifies he doesn’t believe in compiler-level brain hacks (sorry, Snow Crash fans), Bo presents survey evidence that 85% of GenAI usage involves content meant for others, and Seth closes with the contrarian hot take that information transmission will actually improve on net. General equilibrium saves us all—assuming a spherical cow.

Topics Covered:

• Jakobson’s functions of language (all eight of them, apparently)

• Signaling theory and the pooling equilibrium problem

• Crawford-Sobel cheap talk games and babbling equilibria

• “Pay to apply” as incentive-compatible mechanism design

• Corporate prediction markets and conflicts of interest

• The ABC conjecture and math as a social enterprise

• Cryptographic verification and proof of humanity

• Why live performance and in-person activities may increase in economic value

• The Coasean singularity

• Robin Hanson’s “everything is signaling” worldview

Papers & References:

• Crawford & Sobel (1982), “Strategic Information Transmission”

• Cowgill and Zitzewitz (2015) “Corporate Prediction Markets: Evidence from Google, Ford, and Firm X”.

• Jakobson, “Linguistics and Poetics” (1960)

• Binet, The Seventh Function of Language

• Stephenson, Snow Crash

Transcript:

Andrey: Well, let’s go to speculation mode.

Seth: All right. Speculation mode. I have a proposal that I’m gonna ask you guys to indulge me in as we think about how AI will affect communication in the economy. For my book club, I’ve been recently reading some postmodern fiction. In particular, a book called The Seventh Function of Language.

The book is a reference to Jakobson’s six famous functions of language. He is a semioticist who is interested in how language functions in society, and he says language functions in six ways.¹ I’m gonna add two bonus ones to that, because of course there are seven functions of language, not just six. Maybe this will be a good framework for us to think about how AI will change different functions of language. All right. Are you ready for me?

Bo Cowgill: Yes.

Seth: Bo’s ready. Okay.

Bo Cowgill: Remember all six when you...

Seth: No, we’re gonna do ‘em one by one. Okay. The first is the Referential or Informational function. This is just: is the language conveying facts about the world or not? Object level first. No Straussian stuff. Just very literally telling you a thing.

When I think about how LLMs will do at this task, we think that LLMs at least have the potential to be more accurate, right? If we’re thinking about cover letters, the LLMs should maybe do a better job at choosing which facts to describe. Clearly there might be an element of choosing which facts to report as being the most relevant. We can think about, maybe that’s in a different function.

If we ask about how LLMs change podcasts? Well, presumably an LLM-based podcast, if the LLM was good enough, would get stuff right more often. I’m sure I make errors. Andrey doesn’t make errors. So restricting attention to this object-level, “is the language conveying the facts it needs to convey,” how do you see LLMs changing communication?

Bo Cowgill: Do I go first?

Seth: Yeah, of course Bo, you’re the guest.

Bo Cowgill: Of course. Sorry, I should’ve known. Well, it sounds like you’re optimistic that it’ll improve. Is that right?

Seth: I think that if we’re talking about hallucinations, those will be increasingly fixed and be a non-issue for things like CVs and resumes in the next couple of years. And then it becomes the question of: would an LLM be less able to correctly report on commonly agreed-upon facts than a human? I don’t know. The couple-years-out LLM, you gotta figure, is gonna be pretty good at reliably reproducing facts that are agreed upon.

Bo Cowgill: Yeah, I see what you mean. So, I’m gonna say “it depends,” but I’ll tell you exactly what I think it depends on. I think in instances where the sender and the receiver are basically playing a zero-sum game, I don’t think that the LLM is gonna help. And arguably, nothing is gonna help. Maybe costly signaling could help, but...

Seth: Sender and the receiver are playing a zero-sum game? If I wanna hire someone, that’s a positive-sum game, I thought.

Andrey: Two senders are playing a zero-sum game.

Seth: Oh, two senders. Yes. Two senders are zero-sum with each other. Okay.

Bo Cowgill: Right. This is another domain-specific answer, but I think that it depends on what game the two parties are playing. Are they trying to coordinate on something? Is it a zero-sum game where they have total opposite objectives? If all costly signaling has been destroyed, then I don’t think that the LLM is gonna help overcome that total separation.

On the other hand, if there’s some alignment between sender and receiver—even in a cheap talk world—we know from the Crawford and Sobel literature that you can have communication happen even without the cost of a signal. I do think that in those Crawford and Sobel games, you have these multiple equilibria ranging from the babbling equilibrium to the much more precise one. And it seems like, if I’m trying to communicate with Seth costlessly, and all costly signal has been destroyed so we only have cheap talk, the LLM could put us on a more communicative equilibrium.

Seth: We could say more if we’re at the level where you trust me. The LLM can tell you more facts than I ever could.

Bo Cowgill: Right. Put us into those more fine partitions in the cheap talk literature. At least that’s how I think the potential for it to help would go.

Andrey: I wanna jump in a little bit because I’m a little bit worried for our listeners if we have to go through eight...

Seth: You’re gonna love these functions, dude. They’re gonna love... this is gonna be the highlight of the episode.

Andrey: I guess rather than having a discussion after every single one, I think it’s just good to list them and then we can talk.

Seth: Okay. That’ll help Bo at least. I don’t know if the audience needs this; the audience is up to date with all the most lame postmodern literature. So for the sake of Bo, though, I’ll give you the six functions plus two bonus functions.

1. Informational: Literal truth.

2. Expressive (or Emotive): Expressing something about the sender. This is what actually seems to break in your paper: I can’t express that I’m a good worker bee if now everybody can easily express they’re good worker bees.

3. Connotative (or Directive): The rhetorical element. That’s the “I am going to figure out how to flatter you and persuade you,” not necessarily on a factual level. That’s the zero-sum game maybe you were just talking about.

4. Phatic: This is funny. This is the language used to just maintain communications. So the way I’m thinking about this is if we’re in an automated setting, you know how they have those “dead man’s switches” where it’s like, “If I ever die, my lawyer will send the information to the federal government.” And so you might have a message from your heart being like, “Bo’s alive. Bo’s alive. Bo’s alive.” And then the problem is when the message doesn’t go.

5. Metalingual (or Metalinguistic): Language to talk about language. You can tell me if you think LLMs have anything to help us with there.

6. Poetic: Language as beautiful for the sake of language. Maybe LLMs will change how beautiful language is.

7. Performative: This comes to us from John Searle, who talks about, “I now pronounce you man and wife.” That’s a function of language that is different than conveying information. It’s an act. And maybe LLMs can or can’t do those acts.

8. Incantatory (Magic): The most important function. Doing magic. You can come back to us about whether or not LLMs are capable of magic.

Okay? So there’s eight functions of language for you. LLMs gonna change language? All right. Take any of them, Bo.

Andrey: Seth, can I reframe the question? I try to be more grounded in what might be empirically falsifiable. We have these ideas that in certain domains—and we can focus on the jobs one—LLMs are going to be writing a lot of the language that was previously written by humans, and presumably the human that was sending the signal. So how is that going to affect how people find jobs in the future? And how do we think this market is gonna adjust as a result? Do you have any thoughts on that?

Bo Cowgill: Yeah. So I guess the reframing is about how the market as a whole will adjust on both sides?

Andrey: Yes, exactly.

Bo Cowgill: Well, one, we have some survey results about this in the paper. It suggests you would shift towards more costly signals, maybe verifiable things like, “Where did you go to school?”

Andrey: No, but that is easy, right? That already exists, more or less.

Bo Cowgill: That’s true. Yeah, I mean, you could start using these more and start ignoring cover letters and things like this.

One thing somewhat motivated by the discussion of cheap talk a minute ago is that there’d be more referral hiring. This is something that lots of practitioners talk about: we can’t trust the signal anymore, but I can still trust my current employees that worked with this person in the past. It has a theoretical interpretation as well, which is that when all you have is cheap talk, the only communication you can have is maybe between people who are allies in some sense or who share the same objective. This would be why you could learn or communicate through a network-based referral. So I think that’s super interesting and lots of people are already talking about it. It would be cool to try to have an experiment to measure that.

Andrey: What about work trials? Do you think that’s gonna become more common? Anecdotally, I see some of the AI labs doing some of this. If you can’t trust the signals, maybe just give a trial.

Bo Cowgill: Most definitely. The cheap talk idea is not the only one. You could have a variety of contractual solutions to this problem. There was a recent Management Science paper about this: actually charging people to apply, thinking that they have a private signal of whether they can actually do this or not. If they’re gonna get found out, they would be less likely to be willing to part with this money. It’s less of a free lottery ticket just to apply if you’re charging.

Andrey: For what it’s worth, I strongly think that we’re gonna move into the “pay to apply” world.

Bo Cowgill: Oh. That’s interesting. I mean, I think that “pay to apply” is super underrated. Having said that, people have been willing to ignore more obvious good things for longer, so I don’t think it’s as inevitable as it sounds like you do.

Andrey: Well, I think it’s the natural solution to the extent that what the cover letter is doing is signaling your expected match quality. And you have private information about that. I think both Indeed and LinkedIn have now premium plans with costly signals. So it’s not exactly a “pay for apply,” but you pay for a subscription that gives you limited signals, which is essentially the same exact thing.

Bo Cowgill: Makes sense.

Andrey: Yeah. So I think, whether that solves these issues, I’m not sure. It needs to be objective to really do the deed.

Seth: It solves the express... well, which is fine if we think willingness to spend on this thing is more correlated with ability. It’s back to the same signaling model.

Bo Cowgill: I mean this solution also relies on the applicant themselves to know whether they’re a good match in some sense, and some people are just deluded.

Andrey: Yeah. Well also the platform, like in advertising, could be a full auction-type thing.

Bo Cowgill: It could be a scoring auction that has its own objectives and gives people discounts. What Seth says raises a common objection for “pay to apply,” which is: “What about the people who can’t afford it?” And I think a high number of the people who have said that in my life work for an institution that charges people to apply for admission. So you could use some of the same things. You could have fee waivers, and the fee waivers might require a little bit of effort to get.

Another idea I’ve heard is that you could put the money in escrow and then possibly give it back if it doesn’t work out. Or you could actually give it back if it does work out. So yeah, people have different takes on this. But there are various ways to harness “pay to apply” and then deal with the negative aspects of it in other ways.

Seth: So what it seems to solve is this very narrow element of what we call the expressive function of language. So one thing I’m trying to express with my cover letter is, “I’m a good worker bee. I do the things. I have resources. I will bring my resources to your firm.” But we also want the letters to do lots of different things, like be beautiful and tell me a little bit about yourself. Have heterogeneous match quality elements, right? So it seems like this money only helps with one vertical dimension of quality.

Andrey: Actually, when you’re sending that costly signal and you cater your cover letter to that employer, that is about match quality, right? The costly signal, the “pay to apply,” gives you the incentive to reveal that information in your cover letter.

Seth: Right. It’s a “both,” right? It’s not a payment or a cover letter. It’s a both. Good point.

Andrey: We’ve spent a lot of time thinking about the signaling, this information apocalypse—or epistemic apocalypse—that Bo has been calling it. I think one solution to various epistemic issues has been prediction markets. I wanted to ask Bo about his earlier life experiences with those because it’s a very hot topic now, with a lot of prediction markets gaining traction.

Bo Cowgill: Yeah, definitely. We should get back to the GenAI information apocalypse as well and ask: do we think it’s gonna happen? But yeah, it is true that some of my first papers out of grad school were about prediction markets. In my former life I worked at Google, where at one time people had 20% projects. I started an internal prediction market. At the time it was the largest internal prediction market known to exist.

There were around 400 or so different markets where we offered employees the ability to anonymously bet on different corporate performance measures. The two most common ones were: What will the demand for our products be? How many new advertisers, Gmail signups, or 7-day-active-users will we get? And then also, project launch deadlines. Basically, would it be on time or early or late? Not very often early, but sometimes on time.

I had a paper about this in the Review of Economic Studies. It showed, like in many other cases, the markets perform really well, both in absolute terms and relative to other forecasters at Google. We eventually got other companies’ data to try to do similar things.

I think one interesting thing is that prediction markets have gotten really big externally for things like elections, but you still don’t see a lot of companies seemingly use it to guide decision-making.

Andrey: I want to hear your best explanation for why you think the internal prediction markets haven’t taken off.

Bo Cowgill: There are lots of reasons. Our prediction market at Google was really built around having a proof of concept that we can then use to launch our own Kalshi, or our own Polymarket. I think it was a little bit too soon for that. In our case, we weren’t really trying to make it as good of a decision-making tool as possible. Like we wanted to go public and have the election markets be hosted by Google. There were some regulatory barriers I think that Kalshi eventually was able to get past.

The part of the problem I’ve been working on recently is that the prediction market paradigm inside of a company assumes that all the workers have some information about what plan of action would be best, but they otherwise have no preference about what you do with this information. Like, “Should we launch a new product?” The paradigm assumes that they all know something about whether it’s gonna be a successful product, but they sort of don’t care whether you do it or not. Obviously they care. Some of the people with the best information about this new product could have a very strong preference. I heard about this situation in Asia, where the person with the best information on the new product would also probably have their career sabotaged if they launched a competing product. So that could interfere with the incentive compatibility of the market.

Seth: The incentives aren’t high-powered enough.

Bo Cowgill: That’s true. And it’s hard to think about how the incentives would ever be high-powered enough to offset this unless the company proactively designs the market differently to deal with these conflicts of interest.

Seth: I wanna follow up with Andrey’s question. This seems like a really good way to accumulate information, and maybe AI will help us do these better. Is there really an epistemic apocalypse or will prediction markets plus AI predictors save us all?

Bo Cowgill: It’s possible that prediction markets will help in this way just by making the information... it’s essentially a form of a contract. When we talked about various contracts including “pay for apply” and maybe doing a trial period at a job, all these are contractual ways of making it costly to lie. And that could possibly discipline this sort of thing.

One reason I think that the epistemic apocalypse isn’t going to fully happen is that for cases where there’s an information bottleneck, I think the economy is gonna find a way to get the information it needs so that you can hire someone for a valuable role. There’s lots of reason that buyers want to coordinate on information.

Seth: It’s positive-sum.

Bo Cowgill: Right. So that would be one reason. I think in a lot of cases, the informational bottlenecks will be closed even if you don’t have as good of positive, costly signaling as you used to. But, number one, we could just have to tolerate a lot of mistakes. And that already happens in the hiring setting. So it’s possible that we could have to tolerate even more hiring mistakes because now the signal is actually worse.

Andrey: Bo, why are we hiring anyone? I thought all the jobs will be non-human jobs. Maybe it’ll be a Coasean singularity where we’re all one-person firms.

Seth: Exactly. What is the Coasean singularity? It’s the zero bargaining frictions, and one of the bargaining frictions is information asymmetry. Bo, would it be fair to say then that you’re kind of more optimistic about convergence in sort of public, big-question information—the kinds of stuff that prediction markets are good at at scale—but you’re more pessimistic about Seth trying to send a message to stranger number three?

Bo Cowgill: That is a good distinction. The prediction markets are generally better at forecasts when there’s lots of information that’s dispersed around lots of different actors, and the market kind of aggregates this up.

Seth: And theoretically, a high-quality LLM that has a budget to do training will be a super-forecaster and will be conveying and aggregating this information, right?

Bo Cowgill: That’s true. But when we think about agents participating in prediction markets, a bunch of the theory assumes that everyone receives some independent signal or a signal with some independent noise. Insofar as everyone’s agent derives from the same three or four big labs, then they might not actually be all that independent. And that would be a reason to not think that the markets will save us.

Seth: Only if they’re not independent ‘cause they’re wrong.

Andrey: Well, even if the foundation models are the same, they may be going out to acquire different pieces of information.

Bo Cowgill: That’s true. You also have the temperature in the models that adds some level of randomness to the responses.

Andrey: No, but I literally mean, like, you have these sci-fi novels where you tell the AI to go out and find information, and that’s a costly acquisition process for the LLM. Maybe it has to interview some humans or pay for some data. I think this viewpoint that you’re just taking an identical prompt from some off-the-shelf chatbot and asking, “Hey, what’s the prediction here?” is really not the right way to think about what agent-assisted functions would be doing. Think about hedge funds: they’re all using various machine learning to trade, but it’s not like they’re all doing the same thing, even though I assume that many of the algorithms they’re using are in some sense the same.

Bo Cowgill: I see. So you’re basically more optimistic about prediction markets and AI being a combined thing that would help overcome the apocalypse.

Andrey: Yes.

Bo Cowgill: I don’t know. Well, one way in which I guess I’m a little bit more pessimistic is that, in the world that we’re just coming from, I think there is just more reliable, ambient information that you would get just from being in the environment that you could trust.

I think in the old world, you could just trust a photograph. Now it’s true that there were a lot of staged photographs even back in the day...

Andrey: Have you seen friends of comrade Stalin?

Bo Cowgill: Totally.

Seth: Losing his friends very quickly.

Bo Cowgill: But it does still feel like... maybe not stuff that you would see in the media where there were parties that would have some incentive to doctor photos. But if your friend said that they met Tom Brady, they could bust out a picture and show you Tom Brady and you could have more faith in that. Or other smaller-stakes, ambient things that might be a little bit more trustworthy now that could accumulate.

Seth: That’s the question. Does all of the little small stuff add up to an apocalypse if we’re all still agreeing at the big stuff from the top down?

Andrey: What about reputation? He’s not gonna show you fake photos, come on.

Bo Cowgill: This is true. Well, I mean, if we’re not gonna interact again, then who knows?

Seth: Zero-shot.

Bo Cowgill: You’re a sock puppet, you know?

Seth: Shit. Stay contrary.

Andrey: That’s the twist, is that this was an AI podcast the entire time. I am a robot.

Bo Cowgill: That’s funny.

Andrey: I mean, reputation is not a bilateral thing only, right? You have reputational signals that you can accumulate, and certainly for media outlets, they could form reputations. That’s kind of the point of media outlets.

Seth: In the future, everyone’s their own media outlet. Everyone’s got their own Substack. Everyone could have an LLM pointed at them saying, “Hey, keep track if Seth and Andrey ever lie or do anything bad on their podcast.” So there’s a sense in which it’s the classic AI attack-defense thing. It makes it easier to make fakes, but it also makes it easier to monitor fakes.

Bo Cowgill: I see what you’re saying. So yeah, this is why I say I think in situations where it’s high-stakes enough to form a contract and do monitoring, that we don’t necessarily get these huge amounts of information loss. But you would also get a lot of information about the world.

Actually, here’s a specific example. I have a 4-year-old daughter.

Seth: Cute. Can confirm.

Bo Cowgill: Thank you. So there was a GenAI photo of a squirrel who ate a piece of candy or something like that. It was GenAI, but it was high-quality, and the squirrel has expressive body language saying how good it is. I would know that that’s not a real squirrel, that they were trying to create a viral video. But she hasn’t really experienced real squirrels yet. So I actually think that she probably thought this was something that could actually happen. Now we’re gonna have a whole generation of people who have probably seen more fake cat videos than actual cat videos. And I just think that will accumulate, not necessarily to an apocalypse, but to some level of aggregate information loss.

Andrey: It’s interesting ‘cause I would think that it’s not the kids who are gonna be affected, but it’s the adults. Think about who are the primary spreaders of mass emails with completely unverified information.

Seth: Even better. And at the end it says, “Please share. Share with everyone.”

Bo Cowgill: Right. I mean, one answer to that is: yes, and/or why not both?

Seth: It’s attack and defense again on the squirrel thing. When I grew up, I had no idea that trees actually looked like these lollipop palm trees that they have here in Southern California. When I was reading Dr. Seuss, I thought those were made-up BS. And then I had to actually go out here to find out.

Bo Cowgill: Stuff you believe. I’m just kidding.

Seth: Fair enough. I guess what I’m trying to say is that, as a child, I was exposed to a lot of media with talking animals and eventually I figured it out. And who knows, maybe your daughter will have access to LLMs and instead of having to wait until she’s 20 to find out, she can ask, “Hey, do squirrels actually thank you and be emotive in a human-like way?”

Bo Cowgill: Yeah. What do you guys think about the idea that the rise of fake AI will actually create demand for crypto and for things being cryptographically signed as proof of their authenticity?

Andrey: Yes. I think the answer is yes. I’m very interested in ideas such as “proof of humanity.” I think on a practical level, the concepts involved in crypto are just too abstract for most people. So the success will come from essentially someone putting a very nice user interface on it, so people aren’t actually thinking about the crypto part.

Seth: The blocks. I mean, I definitely see a huge role for just this idea of timestamping: this thing went on the blockchain at this date, and if we can’t agree on anything else, at least we can agree on the original photo of Stalin with his four friends.

Andrey: I guess the big question for all of these systems is they’re not that useful until lots of people are on them. It’s a chicken-and-egg problem.

Seth: Really? You don’t think if you got the three big news services on it, wouldn’t that be standard-setting?

Andrey: Yeah. But I view that as a different and a harder ask than the timestamping. I know news organizations can do that themselves. I assume they’re actually already doing it to some extent. And normal human beings would never check. But if there was an investigation, someone could in principle check.

Seth: Well, it comes up all the time in terms of documenting war events. It’s like, “Oh, you said this was a bombing from yesterday, but this is photos from 10 years ago,” right?

Andrey: Yes. And if we had some enlightened CEOs of social media companies, they might facilitate that. It’s not clear that their business interests are actually well-aligned with that. But I think with the proof-of-humanity type stuff, you’re gonna wanna use it when everyone else is using it. Let’s say Meta wanted to verify that everyone on its platform was a unique human being. If everyone has access to proof-of-humanity technology, then that’s very feasible to do. But if only a tiny share of the population is using it, then it’s not a very effective mechanism.

Seth: What do we think? One thing we haven’t talked a lot about today, and I wanna give us a chance to at least address it in passing, is that it seems like the effect of LLMs on writing has a lot to do with how much LLMs will be doing reading. We’ve already talked in passing about how LLMs prefer the writing of other LLMs; it seems to show up in your study. It makes perfect sense. If you prompt an LLM saying, “Write the best thing,” it should be pretty good at it, right? Because it can just evaluate it itself and iterate.

To what extent is that a problem or a solution? The positive vision is the LLMs are going to be able to convey extremely detailed information and then on the other end, parse extremely detailed information in an efficient way. That’s Andrey’s Coasean singularity. But you might imagine that because now only LLMs are reading, people put less effort into submitting, and that’s the epistemic apocalypse: “Why even try if they prefer a bullshitted GenAI version?”

Bo Cowgill: Yeah, totally. Or I guess in a lot of my own prompts, sometimes I know I don’t have to describe what I’m talking about in very fine detail ‘cause it knows the context of the question and can do it. It does seem like it’s potentially a problem to me, mainly because we should still care about the human-to-AI communication pipeline, and that pipeline might actually need to go in both directions. And so if the LLMs are basically really good at talking to each other, but lose the ability to talk to normal people, then that seems potentially bad for us.

Seth: But there’s one thing LLMs are great at, it’s translating. That’s something I’m optimistic about.

Bo Cowgill: That’s true. Arguably it needs to be trained and/or prompted or rewarded somehow to do that. And maybe the business models of the companies will keep those incentives aligned to actually do this.

Andrey: Well, the models are gonna be scheming against each other, so they wouldn’t wanna tell us what they’re really conspiring to do. One final topic I wanted to get to was superhuman persuasion.

Bo Cowgill: So, Andrey I think had this provocative statement at some point that he doesn’t think of persuasion as being a big part of the effects of GenAI. I was surprised by that. I think maybe Andrey is representing a common view out there.

There’s a lot more discussion of the productivity effects of GenAI maybe than the persuasion effects. And I don’t know if at some level, without persuasion... persuasion ultimately is some part of productivity if we’re measuring productivity in some sort of price-weighted way. Because two companies could have the same exact technology, one with a bad sales force, and it might show up as one of them being a zero-productivity company.

Seth: But how much is that zero-sum? I guess the idea there would be is that sure, if Coke spends more on advertising, we’ll sell more Coke and less Pepsi. But is that positive-sum GDP or have we just moved around the deck chairs?

Bo Cowgill: In order to get the positive sum, I think you would still need to persuade someone that this is worth buying.

Seth: No, ‘cause it could be negative. You can make Pepsi shitty. You can be like, “Don’t drink Pepsi. It’s shit.” But it’s negative-sum. It’s negative GDP.

Andrey: I just wanna state precisely what I think my claim was, which is: I don’t believe in substantially superhuman persuasion. Which isn’t to say that in jobs that require persuasion, AI can’t be used. It’s just more that I don’t think there’s this super level of like, you talk to the AI and it convinces you to go jump off a bridge.

Seth: Right. So in Snow Crash, it’s posited that there’s a compiler-level language for the human brain that if you can speak in that, you can just control people. Similarly, in The Seventh Function of Language, there’s this idea of a function of language that is just so powerful, you can declare something and it happens.

Andrey: That’s the magic.

Bo Cowgill: Right. Productivity is not that many steps away from persuasion about willingness to pay or willingness to supply. And it does seem like the persuasion aspects of GenAI should be talked about more.

I wanted to bring up this ABC conjecture because I think that there’s a belief that in areas very cut and dry, like math, there is no real room for persuasion because something is just either true or not. This story about the ABC conjecture illustrates this.

There’s a Japanese professor of math who studied at Princeton and has all of the credentials to have solved a major conjecture in number theory. He puts forth this 500-page attempted solution of the ABC conjecture. A credible person claiming this is the proof. Unfortunately, his proof is so poorly written, so technical and so badly explained, that no one else has been able to follow the proof.

Seth: Or even put it in a formal proof checker. If they had put it in a formal proof checker, everyone would’ve been satisfied.

Bo Cowgill: Yes. I think that this story is interesting because it highlights that, even in something like math, it’s ultimately a social enterprise where you have to try to convince other human beings that you have come up with something that has some value.

Seth: Wait, people aren’t born with values? Without a marketing company, I would still wanna drink water.

Andrey: That’s actually not true. I mean, isn’t there the whole movement to drink more water?

Bo Cowgill: It’s true that you may have been persuaded just by your parents or your rabbi or whoever. But let’s get to a more narrow objection. As part of the motivation for this “cheaper talk” paper, we ran some surveys to try to get a sense of what people do with AI. One of the first questions was, “Think of the recent time that you’ve used GenAI. Were you developing something that you were eventually going to share with other people?” Something like 85-90% were using this on something that I would share directly with other people.

Seth: Really? I’m at like 95% of my usage is just looking stuff up for me.

Bo Cowgill: But were you looking it up and ultimately going to share this as part of a paper or a podcast conversation?

Seth: I mean, only insofar as the Quinean epistemic web of everything in the universe is connected to everything else. So yeah, if I learn about tree care, it could help me write an economics paper.

Andrey: Everything is signaling according to Robin Hanson, right?

Bo Cowgill: Sure. I think it’s fair that if this was not your intent, even two or three steps away, then you shouldn’t say yes in the survey. But anyway, a big majority of people say yes.

Then the next question, for the people who were using it for something that would be shared: “Were you using the GenAI to try to improve the audience’s impression of you?” So come up with your prior.

Seth: Hundred percent. Wait, sorry. So 15% of people use GenAI to make other people feel worse about them?

Bo Cowgill: Well, I assume these people would say that they weren’t trying to make it feel worse. They were just not trying to sort of propaganda the person.

Andrey: And to be clear, these are Prolific participants, so they’re trying to just make sure that their Prolific researchers don’t kick them out of their sample.

Bo Cowgill: Maybe. But most people who I tell these results to are like, “Well, yes, of course. I use GenAI a ton of time to help with writing, to rewrite emails, to explain something in a way that sounds a little bit nicer or smarter.” And it does seem like a very dominant use of GenAI.

If this is the case, then the fact that it’s making it easier to impress people all at once is a super interesting part of the effects. And, I know Andrey has offered his caveat about what he actually meant, but I think that would put this persuasion aspect as more of one of the central things.

Andrey: I agree that what you’re saying is interesting. It’s more the claim I was talking about where people—mostly in the Bay Area—think that super AI is gonna take over the world.

Bo Cowgill: That we’ll just turn people into puppets.

Andrey: Yeah, exactly.

Bo Cowgill: No, fine. I won’t take any more cheap shots at you.

Seth: We can bring up the Anthropic AI index.

Andrey: Well, I was gonna do the ChatGPT usage paper, but you do the AI one first.

Bo Cowgill: Of course, one of the major things that the ChatGPT usage paper says is writing.

Seth: Which interestingly, this showed up in GDPVal, is that ChatGPT seems like a little bit better at writing, and Claude seems a little bit better at coding, and it seems to show up in usage also.

Bo Cowgill: But they should break down writing. The question that this raises is: who is the writing for? And why aren’t you writing yourself? And are you possibly trying to signal something about yourself by having this clear writing?

Andrey: But I guess I truly do think, like Robin Hanson, that a vast majority of what humans do, period, is signaling to others.

Seth: Is that your claim, Bo? Or is your claim that AI is gonna make it worse?

Bo Cowgill: I’m not as Robin Hanson on “everything is signaling,” but I would just claim that this should be a more front-and-center thing that people think about with regards to the effects of the tech.

Seth: Listen. If you wanna be an economist, you gotta tell us what to study less. You can’t tell us to study everything more. What are we gonna do less of?

Bo Cowgill: I mean, I guess the easy thing would be to say human-AI replacement just because there’s so many studies on that right now.

Andrey: The productivity effects of this one deployment of a chatbot in this one company.

Bo Cowgill: Oh, yes. I can totally get on board with complaining about that.

Seth: Bo, help me get beyond it. This is what you need to do for me. People are gonna do what you said and write that paper on signal quality in one population. What’s the meta-paper? How can we get beyond that into a more comprehensive view of what’s going on? What’s your vision for research in this direction?

Bo Cowgill: Part of this goes back to the question about just what are general equilibrium effects overall? If people all become more persuasive all at once, then this totally destroys the quality of information.

Another question is, how much do the AI labs themselves actually have an incentive to build positive-covariance technology or negative-covariance technology? If part of the value of a camera is that you could take pictures and then show people and be like, “Look, this is real, this is a costly signal,” then you might actually want to keep the covariance of your technology somewhat high because this will be one use case that people would actually want.

Andrey: This is a very interesting, broader question. I was at a dinner with a few AI folks and we were talking about the responsibility of the AI labs to do academic research. We don’t expect the company that creates a tool to create the solutions to all of the unintended consequences of that tool. That to me is a very strange expectation. It seems impossible, and we don’t expect that from any other company.

Bo Cowgill: Definitely. But just to put a finer point of what I’m talking about: suppose that the covariance is so negative that you’re just getting a lot of signal jamming, to the point where now there’s just less demand for writing in general. Even if there’s still some demand, well then that less demand for writing could feed back into the underlying demand for the LLM product itself because this was supposed to help you write better, but now no one trusts the writing. And there could be something financially self-defeating about having this technology that is negative.

Seth: It would be general equilibrium self-defeating. Individually, we’d all wanna defect and use it.

Andrey: Even if one company tried to [fix it], the solution by the market is: if you really care that a human wrote this, the market will create a technology where we verify that the human is literally typing the thing as it’s happening.

Personally, I think that live performance and in-person activities in general are gonna rise up in economic value because they’re naturally... I do think humans care about interacting with other humans. We care that other humans are creating speech, art, and so on.

Seth: So those are the expressive functions of language. That’s the phatic function of, “Hey, look, I’m still alive, Grandma.” That’s the poetic function. And LLMs can’t... we don’t think it can do this performative function. It’ll be interesting to see whether AIs get enough rights to be able to make binding contracts on our behalf.

Andrey: There’s gonna be a ubiquitous monitoring technology, and every time I declare bankruptcy, it will enact.

Seth: It’ll immediately get locked in.

If I can just share my wrapping-up thoughts. I come away a little, not as scared as Bo about this epistemic apocalypse. He has scared me. But I come away thinking that it’s fundamentally kind of partial equilibrium to say, “Hey, look, we used to send signals this way. There’s a new technology that comes along. Now that signal isn’t coming through as well.” To me, that doesn’t mean communication is impossible. Now I just get to: “Okay, what’s the next evolution of the communication? Are we gonna have LLM readers? Are we gonna have verified human communication?” There seem to be solutions.

Bo Cowgill: It’s probably a little bit of an exaggeration of what I was saying to characterize it that way. But I did say that Andrey said that persuasion wasn’t important, so maybe I’m owed some exaggeration back.

Seth: Fair enough. If you put a gun to my head, I would say that information transmission will get better on net because of AI.

Andrey: What a hot take to end this.

Seth: That’s my hot take.

Andrey: You don’t hear anyone saying that. That is fun.

Seth: Who would’ve thought that the greatest information technology product of all time might actually give us more useful information?

Andrey: No, no, no. You’re only allowed to be pessimistic, Seth. That’s the rules of the game.

Bo Cowgill: So Seth, do you think this is mainly because people will be able to substitute away from other things?

Seth: It’s partially that. I think what you’re identifying in this paper is definitely important. But it does seem like this is transitional and that more fundamentally, LLMs help us say more and help us hear more. And so I think once the institutional details are worked out—and of course that’s a lot of assuming a spherical cow—there will be better information in the long run.

Andrey: There are even entrepreneurial activities that one could undertake to try to amend some of the concerns raised by this paper. We oftentimes take this very observer perspective on the world, but certainly we could also, if we think that a solution is useful, do something about that.

Seth: Right. We will sell human verification. We will verify you are a human. If you pay us a thousand dollars, we will give you a one-minute spot on this podcast where we will confirm you are human.

So Bo, I guess we’re just a little bit different on this. What do you think?

Bo Cowgill: Well, I do agree that the paper was proof of concept and partial equilibrium, and what happens in the general equilibrium... we’ll just have to figure out in future episodes of Justified Posteriors.

Andrey: Yeah. Well, thanks so much, Bo, for being a great guest.

Seth: And Bo, both you, everybody else, keep your posteriors justified.

Bo’s answer: it depends. Under some conditions, GenAI induces an epistemic apocalypse, flattening signals and confusing recruiters. In others, it reveals skill even more sharply, giving high-types superpowers. The episode walks through the theory, the experiment, and implications.

Transcript:

Seth: Welcome to the Justified Posteriors Podcast, the podcast that updates its priors about the economics of AI and technology. I’m Seth Benzell, certifying my humanity with takes so implausible that no softmax could ever select them at Chapman University in sunny Southern California.

Andrey: And I am Andrey Fradkin, collecting my friends in all sorts of digital media formats, coming to you from San Francisco, California. Today we’re very excited to have Bo Cowgill with us. Bo is a friend of the show and a listener of the show, so it’s a real treat to have him. He is an assistant professor at Columbia Business School and has done really important research on hiring, on prediction markets, and now on AI and the intersection of those topics. And he’s also won some very cool prizes. I’ll mention that he was on the list of the best 40 business school professors. So he is one of those professors that’s really captivating for his students. So yeah. Welcome, Bo.

Bo Cowgill: Thank you so much. It’s awesome to be here. Thanks so much for having me on the podcast.

Seth: What do you value about the podcast? That’s something I’ve been trying to figure out because I just do the podcast for me. I’m just having a lot of fun here with Andrey. Anything I can do to get this guy’s attention to talk about interesting stuff for 10 minutes? Why do you like the podcast? What can we do to make this an even better podcast for assistant professors at Columbia?

Bo Cowgill: Well, I don’t wanna speak for all assistant professors at Columbia, but one thing it does well is aggregate papers about AI that are coming out from around the ecosystem and random places. I think it’s hard for anybody to catch all of these, so you guys do a great job. I did learn about new papers from the podcast sometimes.

Another cool thing I think is there is some continuity across podcast episodes about themes and arbitrage between different topics and across even different disciplines and domains. So I think this is another thing you don’t get necessarily just kind of thumbing around papers yourself.

Seth: So flattering. So now I can ask you a follow-up question, which is: obviously you’re enjoying our communication to you. A podcast is kind of a one-dimensional communication. Now we’ve got the interview going, we’ve got this back and forth. How would you think about the experience of the podcast changing if a really, really, really good AI that had read all of my papers and all of Andrey’s papers went and did the same podcast, same topics? How would that experience change for you? Would it have as much informative content? Would it have as much experiential value? How do you think about that?

Bo Cowgill: Well, first of all, I do enjoy y’all’s banter back and forth. I don’t know how well an AI would do that. Maybe it would do a perfectly good job with that. I do enjoy the fact that—this is personal to me—but we know a lot of the same people. And in addition to other guests and other paper references, I like to follow some of the inside jokes and whatnot. I don’t know if that’s all that big of a deal for the average person. But I have listened to at least the latest version of NotebookLM and its ability to do a quote-unquote “deep dive podcast” on anything. And at least recently I’ve been pleased with those. I don’t know if you’ve ever tried putting in like a bad paper in theirs, and then it will of course just say, “Oh, this is the greatest paper. It’s so interesting.”

Seth: Right.

Bo Cowgill: You can.

Seth: So that’s a little bit different, maybe slightly different than our approach.

Bo Cowgill: Well, yeah, for sure. Although you can also tell NotebookLM to try to find problems and be a little bit more critical. And that I think works well too. But yeah, I don’t think we should try to replace you guys with robots just yet.

Seth: We’re very highly compensated though. The opportunity cost of Andrey’s time, he could be climbing a mountain right now. Andrey, you take it up. Why are we doing this ourselves? Why isn’t an LLM doing this communication for us?

Andrey: Well, mostly it’s because we have fun doing it, and so if the LLM was doing it, then we wouldn’t be having the fun.

Seth: There you go. Well put. Experiential value of the act itself. Now, Bo, I did not bring up this question randomly. The reason I raised this question of how does AI modify communication... yeah, I used a softmax process, so it was not random. The reason I’m asking this question about how AI changes communication is because you have some recently accepted, forthcoming work at Management Science trying to bring some theory and empirics to the question of how LLMs change human communication, but now in the context of resumes and job search and job pitches. Do you want to briefly introduce the paper “Does AI Cheapen Talk?” and tell us about your co-authors?

Bo Cowgill: Yeah, most definitely. So the paper is called “Does AI Cheapen Talk?”. It is with Natalia Berg-Wright, also at Columbia Business School, and with Pablo Hernandez Lagos, who is a professor at Yeshiva University. And what we’re looking at in this paper is the way people screen job candidates or screen entrepreneurs or, more abstractly, how they kind of screen generally. You could apply our model, I think, to lots of different things.

But the core idea behind it kind of goes back to these models from Spence in the 1970s saying that costly signals are more valuable to try to separate types.

Seth: Right. If I wanna become a full member of the tribe, I have to go kill a lion. Why is it important for me to kill a lion? It’s not important. The important part is I do a hard thing.

Bo Cowgill: Exactly. Yeah. So maybe part of the key to this Spence idea that appears in our paper too is that it’s not just that the signal has to be costly, it has to be kind of differentially costly for different types of people. So maybe in your tribe, killing a lion is easy for tough guys like you, but for wimpier people or something, it’s prohibitively high. And so it’s like a test of your underlying cost parameter for killing lions or for being tough in general. So they go and do this. And I guess what you’re alluding to, which appears in a lot of cases, is the actual value of killing the lion is kind of irrelevant. It was just a test.

And maybe one of the more potentially depressing implications of that is the idea that what we send our students to do in four-year degrees or even degrees like ours is really just as valuable as killing a lion, which is to say, you’re mainly revealing something about your own costs and your own type and your own skills, and the actual work doesn’t generate all that much value.

Seth: Is education training or screening?

Bo Cowgill: Right, right, right. Yes. I do think a good amount of it these days is probably screening, and maybe that’s especially true at the MBA level.

Andrey: I would just say that, given the rate of hiring for MBAs, I’m not sure that the screening is really happening either. Maybe the screening is happening to get in.

Bo Cowgill: What the screening function is now is like, can you get in as the ultimate thing?

Seth: Right. And I think as you already suggest, the way this works can flip if there’s a change in opportunity costs, right? So maybe in the past, “Oh, I’m the high type. I go to college.” In the present, “I’m the high type. I’m gonna skip college, I’m gonna be an entrepreneur,” and now going to college is a low signal.

Bo Cowgill: Yes. Exactly. So that’s kind of what’s going on in our model too. How are we applying this to job screening and AI? Well, you apply for a job, you have a resume, possibly a cover letter or, if you don’t have an old-fashioned cover letter, you probably have a pitch to a recruiter or to your friend who works at the company. And there are kind of elements of costly signaling in those pitches. So some people could have really smart-sounding pitches that use the right jargon and are kind of up to speed with regards to the latest developments in the industry or in the underlying technology or whatever. And those could actually be really useful signals because the only sort of person who would be up to speed is the one who finds it easy to follow all this information.

Seth: Can I pause you for a second? Back before LLMs, when I was in high school, they helped me make a CV or a resume. It’s not like there was ever any monitoring that people had to write their own cover letters.

Bo Cowgill: That’s really true. No, some people have said about our paper that this is a more general model of signal dilution, which was happening before AI and the internet and everything. And so one example of this might be SAT tutoring or other forms of help for high school students, like writing your resume for you. Where if something comes along—and this is where GenAI is gonna come in—but if anything comes along that makes it cheaper to produce signals that were once more expensive, at least for some groups, then that changes the informational content of the signal.

Seth: If the tribe gets guns, it’s too easy to kill a lion.

Bo Cowgill: Yeah. Then it just is too easy to kill the lions. But similar things I think have happened in the post-COVID era around the SATs. Maybe it’s become too easy, or so the theory goes, to get one, where it doesn’t really separate out who is actually a smart person. Maybe it’s getting diluted with who can afford these prep classes and things like that. But I don’t wanna stray too far from GenAI just yet.

You know, I think people have seen a lot about this, either on social media or in the mainstream, is like, the signal in a job application seems like it may have gone down because you used to be able to tell based on these pitches who is qualified or not. And even without lying, you could write a much better pitch that would make you sound really more knowledgeable, even without misrepresenting what your underlying experience is. And so it’s really, I think, not just job applications. That is of course the setting that we study, that and entrepreneurship. But I think there are similar things about how grading at schools has gone bad. You used to be able to quickly tell from an assignment who knew the material and who did not. But now ChatGPT is gonna really interfere with that.

Anyway, so with this as background, we then try to study theoretically and empirically what’s going on with the use of ChatGPT in these sort of costly signaling settings.

Andrey: Yeah. And so how do you go about doing this? Because it does seem like it’ll be pretty hard to study this in the wild. I know of a few papers from some of our friends that have done this. How did you approach this?

Bo Cowgill: So the first thing we wanted to do was kind of motivate the question a little bit more theoretically. So probably at least the first half or so of the paper, we create this model that has what I hope is a tractable punchline, which is that it’s actually not inevitable that GenAI would create this epistemic...

Seth: Wait, a tractable punchline? Wasn’t the punchline that anything goes? What’s the punchline?

Bo Cowgill: Well, I am glad that we brought up the “anything goes” theory models, which is another kind of theme of your podcast and critique of previous papers. So it is true that our model basically says that depending on a particular parameter, you could get either an epistemic apocalypse or a situation where the use of GenAI actually improves the accuracy of screening. And it’s like, you get better information, you actually want your job candidates. You want to say, “Please use GenAI. We actually will know better. Don’t send your pitch in without using GenAI first.”

So it’s true, anything goes. And my defense of that is we really focus the reader on this particular parameter that you could measure empirically.

Seth: Are there other parameters that theoretically could affect this, though?

Bo Cowgill: Not that we’re talking about in this paper. No.

Seth: Not in this paper. All right.

Bo Cowgill: If you have some in mind, I’m curious.

Seth: Well, let’s come back. So I have some thoughts at the end about interpreting the results, so we’ll come back to that. You can just keep on walking us through what you did.

Andrey: I guess I wanted to say there’s an approach in economics, a sufficient statistics approach, right? Where you write down a model where there is a particular parameter that, depending on how big it is or what sign it is, that tells you something about what is the right policy or what is the mechanism that quote-unquote “dominates” a particular setting. And so I view what you guys were doing very much in that vein.

Seth: Right. A ceteris paribus sort of analysis. Yeah.

Bo Cowgill: That’s true. So what are we focusing on? What is the key linchpin of this model? It’s a covariance term across the population. So let me try to break this down.

The two terms in the covariance are, first of all, how much human capital do you have? Or are you like a talented person who knows a lot about what you’re doing, you have a lot of expertise or not? And we’re sort of assuming that the employers are trying to screen for that. Why are they screening for it? Well, in an actual job, you could be in a situation where you don’t have to use GenAI, or you can’t use it and you have to just use whatever knowledge is between your ears. So this one term is your kind of level of talent for the job without AI assistance. And then the other term is how much of a boost does your cover letter get from using ChatGPT to sex it up and to make you sound like you know all the smartest, most contemporaneous jargon?

So these two things could be positively... it could have positive covariance, they could have negative covariance, they could have basically no covariance. But the intuition is, if you have a positive covariance, then the most talented people are getting the largest bump from using GenAI. And the negative covariance would be if the really talented people don’t really get that much of a cover letter improvement, maybe because it’s already so good that there’s nowhere else to go, and that most of the benefit comes from improving the low types’ quality of their cover letter. So this is the linchpin parameter in the model, and what we try to take to data after this.

But just to finish up what’s going on in the theory: well, you get totally different screening results depending on what that parameter is. In the case I think that people are most expecting, you have this negative covariance where most of the benefit comes from making low types and helping them masquerade as high types. And in this negative covariance world, there’s not really that much benefit to high types for using GenAI ‘cause their cover letter or their application or whatever, it’s just already so good. So insofar as this is happening, we want to quantify that empirically. But there’s also this possibility that GenAI puts the high types... it gives them superpowers and they can do even more amazing stuff.

Seth: Right. Can I jump in here? I don’t think you have to interpret it as superpowers, right? If we’re thinking about communication generally, you might imagine that high types have the higher opportunity costs of their time, right? And so there’s some sense in which automating an hour of high-type time is like more money than automating an hour of low-type time. I guess to really understand how this plays out, I’d have to think about how many discrete versions of this is the high type sending out to prospective employers, right?

Andrey: And I guess maybe I’ll add on to that. It depends on what we’re screening for. You’ll get to this in your experiment, but like if the high type has verifiable high-type traits, which is oftentimes the case, assuming they’re not lying on their resume, right? Then what does something like a cover letter reveal? It’s some sort of effort. Right? And so the question... in my mind, cover letters are oftentimes screening for effort, which seems very... take the time to customize a cover letter for this particular job.

Seth: The effort is cheaper for poor people.

Andrey: It’s so it’s kind of a little bit of a different interpretation than like skill per se, because skill... I think it’s unlikely that cover letters signify skill in many domains. Certainly hiring, letters are essentially not read.

Seth: Essentially ignored. I mean, unless they say, “talk to my co-author, blah, who you know,” unless there’s like, “do this thing to learn about me” information in it. Right.

Bo Cowgill: Yeah. Interesting. There’s like a number of things to follow up on there. I do think that there have been big things missed in the study of hiring generally from trying to generalize from academic hiring to other things.

Andrey: Yeah.

Bo Cowgill: I’m not even sure I agree that cover letters are not read either in economics or at least in adjacent places like business and policy schools. And the fact that you think that is probably just a reflection of you guys going to such fine universities that you assume everyone would take the job if you were... I don’t want to pick on any one university.

Seth: Directional state.

Bo Cowgill: Yes, exactly. If you were from University of Southwest Kentucky, which is where I grew up, so I’ll pick on it, it could be very worthwhile to signal that you’re actually interested.

Seth: But again, perfect. But then we’re not signaling skill. You’re signaling match or you’re signaling effort. Right.

Andrey: So it’s a question of what... really this correlation really depends on what is the signal that’s being sent, I think.

Bo Cowgill: Sure, that’s true. But this particular conversation I think has gone off in the direction of cover letters, but candidates also use GenAI to fill in, for example, the bullet points of what they did in a particular job.

Andrey: Yeah. Yeah, yeah.

Bo Cowgill: Where there’s an enormous amount of leeway for describing your job as a super high-impact thing that required you to be an agentic leader or something else. And this is a case that’s not cover letters, but is part of your pitch, where it could actually signal different underlying skills.

So there are lots of ways I think, to apply these ideas in different settings. And it’s true that there’s probably some follow-on work that would be useful, and we can talk about some follow-on work that other people are doing and that I and my co-authors are thinking about doing too.

Seth: Don’t solve it all in one paper. So tell us. So that’s the theory.

Andrey: How dare you not solve it in one paper.

Bo Cowgill: Yeah, yeah, yeah. So you could get these opposite sorts of things. You know, some people think, “What are you talking about? How could there be positive covariance? That’s ridiculous.” I have some examples in mind. In the paper, we talk about AI art. So I’m not an artist and I don’t think you guys are either, but if I used art with DALL-E, I think I’d be a little bit better. But there’s some evidence and some anecdotes and even some small studies that say like, if you actually know how to describe art as a trained artist would, then you can use these AI art generation programs to make way cooler art. And so like if you were screening an artist, you would want them to use GenAI because then you would be able to see the big differences. And even just some screenshots from these demonstrations I think would show how much better the actually trained artists would be, or the high type would be, once they use GenAI.

Now another example of this to me is using AI for math. Now maybe it’s just gotten so good that it can just solve whatever, but I think if you gave a difficult economic theory theorem to prove to a total novice, as somebody who hasn’t gone to a PhD or a high school kid or a middle schooler or something, like, they might not make very much progress. But if you gave someone who had trained or had some intuition for what the solution is, then I think it would be more powerful and actually like... having this sort of result that you could do something with. But it’s true, our model basically isn’t anything goes, but it kind of focuses on this covariance parameter as the thing to pay attention to.

Andrey: It could be positive. So oftentimes, if you’re doing an interview process, there is like a take-home component, like for a data science job that might be a take-home analysis and a dataset and a report, right? In some sense, you can make it... the ceiling for this assignment is very, very high. Right?

Bo Cowgill: Yeah.

Andrey: And someone who actually knows what they’re doing would be able to do a much, much better job. Like there’s a sense that the GenAI tools might raise the bottom of the distribution, but if you want to get close to the max, the people who really know what they’re doing might actually benefit a lot more from the tools.

Bo Cowgill: That’s true. That’s right. Yeah. Well, something your comments, Andrey, make me think about is just the even the idea of a max. And one reason I think that we’ve seen a lot of negative covariance applications is that the underlying test has been designed with a maximum that... there are too many people that are actually close to. And if the test had more sort of headroom to go arbitrarily good, that might, even just that change alone, might make it more possible that GenAI can actually help find the truly talented people as opposed to making the people that ate their homework masquerade.

Seth: No, I was just gonna jump in. I wanna propose a hypothesis for why negative correlations might be common, generally. So you might imagine... rather, not generally, in experimental settings, in experimentally relevant settings. Why do I say that? Imagine if your quality as a worker is both a function of the stuff that can be automated by GenAI and stuff that can’t be automated by GenAI, right? So I’m a worker. I have to do both of these tasks, but maybe I’m gonna delegate some of the automatable-by-GenAI tasks.

If we’re all applying for a job which is kind of at the same sort of productivity threshold, and we’re all kind of assortatively matching to like, we’re applying... we’re not applying to the corner bodega and we’re not applying to Google. We’re all applying for this mediocre firm. For us to have the appropriate skill, total productivity for a mediocre firm, I have to kind of be good at one thing and bad at another. So these like productivity isoquants of given workers will imply a negative correlation between skill in the automatable thing and skill in the non-automatable thing.

Bo Cowgill: Uh.

Seth: So it doesn’t surprise me that if you get a population which is pretty homogeneous in terms of like total productivity, that’s going to entail a negative correlation in the automatable versus non-automatable skill. So that’s why I think this is gonna be common.

Bo Cowgill: Okay. Interesting. I’m curious, I think one of the places where you see negative covariance the most seems to be in the classroom. I guess how does this isoquant idea apply there? Or is it just like, because it’s education and not an actual job that it doesn’t really apply?

Andrey: Well, my thought process would be there is like a lot of assortative matching between programs and students, right? So...

Bo Cowgill: Ah, I see. Yeah. Okay. Okay. Perfect. Yeah.

Seth: But as I wanna complete my idea. So to complete my idea, actually I’ve realized that I’m pointing in the wrong direction, right? For the AI to boost the overall lower total productivity person more, what it needs to do in terms of the job application, is boost them disproportionately at writing job applications, right? This is your notion of how correlated is your actual skill with your ability to write the resume with and without the GenAI. Right. And I think in the general population, it’s probably the case that your ability overall and your ability with AI are positively correlated, in which case, this would be a noisy signal that would mess you up. But if we had like a narrow enough band of quality coming in, it would go in the other way. So maybe there needs to be like a level of screening before the screening. But we haven’t even let you get to the results yet. We’re still in theory.

Bo Cowgill: No, no, no. I think it’s great, as part of the podcast genre, to have some tangents here and there. So in the empirical part of our paper, we’re just trying to measure like how much actual information loss is there? And is it possible that for certain subgroups you actually get information gain? And also, what is this covariance? Is it kind of more positive or negative?

And the key to understanding our experiment is that we actually know something about all the subjects in it and what their “high” versus “low” type is before they even enter the experiment. So I’ll tell you a little bit more about the setting. We are looking at job seekers on Prolific who are in the market for either a data science job or a consulting type of...

Andrey: So Bo, just to clarify ‘cause I do think this might be unclear to the participants. These people are not actually looking for a job. You are recruiting them into an incentivized survey of some sort, right?

Bo Cowgill: That’s true. They do have experience in these respective domains. And so, insofar as this is an incentivized experiment, we have recruited subjects with domain-appropriate knowledge, at least in some cases.

Seth: Can you explain what... do you look at their CVs, or this is something Prolific tells you that they’re experts versus non-experts?

Bo Cowgill: Yeah, Prolific screens them beforehand. And so they’re a little bit unclear about how exactly they screen these people.

Seth: Unclear about what makes someone an expert.

Bo Cowgill: Fair enough.

Andrey: So to be clear, my interpretation is that no one in this paper is an expert. There would be no way any expert in data science would...

Seth: ...for $12 an hour.

Andrey: ...in this sample.

Bo Cowgill: Sure. Well, you sound like one of our referees.

Andrey: Not... I, just to be clear, I am definitely not your referee.

Bo Cowgill: Okay. Yeah. I think the underlying theory doesn’t require anyone be like, elite at any of these things. There just has to be variation within the population about who has relatively higher or lower human capital and that this be...

Seth: Bo, can I pause you for a second there? ‘Cause one of the main outcomes is gonna be whether people’s predictions of whether someone is an expert move closer to 50/50 or not. Right? But presumably, if the signal is getting less informative, you should move to the population average of experts versus non-experts, not 50/50.

Bo Cowgill: Well, the experiment was set up such that the population average was 50/50.

Seth: You tell... well, so you have a measure of whether these people count as experts, right? And in your sample, 50, approximately 50% are experts and 50% are non-experts. As a person reviewing these, have you told me that 50% are experts according to your classification?

Bo Cowgill: Yes. Now, interestingly, their actual beliefs... they don’t seem to totally believe that because on average they think about 45% are experts. And interestingly, they think that about 45% are experts both in the GenAI and the non-GenAI condition. So it’s possible that they would’ve just totally updated their beliefs based on all these amazing cover letters and pitches and little resumes in the experiment and said, “Oh, these people must all be really good.”

Seth: But what actually happened? Okay, but you tell us the treatment. Yeah.

Andrey: So I think, to be helpful to the listeners, the experimental...

Seth: Why do that?

Andrey: ...unit of randomization, the treatment, et cetera.

Bo Cowgill: Yeah. So in our experiment, we recruit people with job experience in the various domains. And we ask them to make a pitch for both a job that they’re qualified for based on what Prolific knows about them and a job that they are not qualified for. So everyone either has domain expertise or prior experience in some sort of data science or some sort of management consulting type of job. So basically everyone is asked to masquerade a little bit to be as qualified as possible for a job that they really didn’t have any prior experience.

And so they write these pitches and then they’re asked to use ChatGPT to edit them to try to make them essentially more convincing. So this is the sender side of the experiment. And then on the receiver side, we get basically people with hiring experience or recruiters to then evaluate these different ones and try to label who are the people that have actual expertise and who are the ones who don’t. It’s essentially like asking, “Who would you wanna hire?” And the recruiters get to know who was using GenAI or not.

Seth: Be very... this seems to be a very important distinction here, so be very clear. They’re told who uses it or who has access to it?

Bo Cowgill: They’re told who has access to it. And our goal there is we’re trying to think about the long-run implications of GenAI on signal dilution. And I think we’ve arguably already reached a world where, if you read a cover letter or you read a resume, it’s probability one that they had access to GenAI.

Seth: Not just probability one-hyphen. It’s a major insight that you just got.

Bo Cowgill: Right.

Andrey: Certainly.

Bo Cowgill: Exactly. But the experiment I don’t think is good... it doesn’t capture, say, the 2024 era very well.

Seth: Remind us when. When is this happening? When are you doing this study?

Bo Cowgill: This happens in 2023. And I think that there’s an intermediate period where there’s some uncertainty about whether this person had access or not. But the long-run implications between the pre-GenAI world and the post-GenAI world, these are the more interesting ones I think to my co-authors and I.

Seth: The correct treatment. Yes. I totally agree that it makes sense that the treatment is “these people got access to AI” rather than “they used AI for exactly this sentence” because that’s the more empirically relevant. Yeah.

Bo Cowgill: Right. Yeah. It’s also possible that the control group could have used GenAI as well. And so we asked them just to make sure, but basically almost none of them did. And we removed the instances where...

Andrey: So I had a very, but a positive, you know, a constructive comment for you, which is that you could...

Seth: Oh shit. This is gonna be devastating.

Andrey: No, no. It’s actually constructive. You could just use one of these AI writing detectors, the good one from Alex Imas’s paper, to see whether they actually use the GenAI or not.

Bo Cowgill: Yeah, no, this is a good idea. This is a good idea. Well, if it hadn’t already been accepted, I think that would definitely be worth checking out.

Seth: And one detail you skewed is that people who use the GenAI, their CVs get way better according to GenAI.

Bo Cowgill: That’s true. That’s true. Yeah. So when basically we have these recruiters assess, they assess several things. One is just like, do they think that the pitch is generally higher quality? Or, does it seem like it required more effort to produce? And, or does it sound kind of polished and like the person knows what they’re talking about?

Seth: Wait, what’s the exact prompt? No, I actually am very curious. Which of those versions is what you ask?

Bo Cowgill: It is, “What’s the quality of the pitch?”

Seth: Quality, right? Because it’d be very interesting if you got a different result for “How much effort do you think you put in?”

Bo Cowgill: No, that’s our theoretical interpretation.

Seth: Fair enough. But hey, why not ask?

Bo Cowgill: True. Yeah. It is. I think it was important. We didn’t ask them how convincing it was because that’s actually a separate question, which opens up the idea that like, “Yes, this is a higher quality pitch, but because we know it’s now become suddenly super cheap to make a pitch like this, we’re actually not very convinced by it.” So this is the other main outcome variable. “Who do you think is actually an expert?” or “How convinced are you?”

And on average, we see information loss from the conditions where the candidate was able to access GenAI. And so this is about a 4% to 9% information loss, or a 4% to 9% decrease in accuracy.

Seth: Oh, can I pause you for a second? ‘Cause so there’s two measures we’re gonna use as how accurate are these screeners? The first one we talked about just now, which is how close are you to just 50/50 as to whether this person is an expert. So obviously you have zero information if you say that they’re a 50/50 expert, but if you were 100% one way or zero, you’d be confident. And then the second thing you get at, right, is this error measure, which is the difference between whether the person’s actually an expert or not, which is this 1/0 binary. And then people can kind of continuously say, “I think this guy’s an 80% expert,” or “I think this guy’s a 20% expert.” And specifically when you say that information transmission went down, which of those measures are you talking about, or both?

Bo Cowgill: Uh, both. The 4% to 9% represents... one of them is using one of these outcomes and the other one is using the other one. And so basically we’re trying to say, you could use a variety... either of these ways to measure accuracy and you qualitatively get the same thing.

And so, what should you make of this 4% to 9%? So I think the information apocalypse people think like, “Wow, that’s it? Only 4% to 9%? This is not very much.” I think that’s a fair point. Now, if you think about... actually another detail that I’ve left out is we studied... we ran this experiment essentially on hiring and with recruiters and hiring managers. And then we also did a similar one in the domain of entrepreneurship with people that were interested in starting a new business, some of whom had no prior expertise in the type of business that they were pitching. And the evaluators here were people with some sort of investing experience. We broadly see the same thing and can’t differentiate the two different domains with regards to the key outcomes and the intermediate values.

So, but we should get back to this 4% to 9%. But one very interesting result, I think, is that when the receivers of these signals are evaluating its quality, we see this huge collapse in the variance of these signals. So it basically looks like everyone’s pitch starts to look pretty good. Without GenAI, they’re all kind of spread out, which is useful for disambiguating who has a good pitch and who has a bad pitch, or who has high underlying experience and human capital or not. But the GenAI kind of homogenizes all of them. And that’s the intuition behind why there’s this information loss.

Seth: So just to understand. Let me understand that a little bit better. So I understand that we’re bringing up the bottom, right? The really bad resumes and pitches get upgraded. Are we also dragging down the top? Or are we just making it more linguistically similar? Understand, tell me... understand what’s happening for the pre-GenAI top performers.

Bo Cowgill: So they’re getting bumped up, just not by very much. So if all types were moving up in quality by an equal amount, then you would just kind of shift the quality to the right between the no-GenAI and the GenAI treatments. But what we see is that the even the high types go up by a little bit, but just not by very much with regards to their application quality or their pitch quality. Meanwhile, the low types are going up a lot, which then pushes them next to the high types and they’re now looking very similar to each other with regards to the quality.

We could also look at linguistically, are they using the same underlying words? We didn’t look directly at that, but I think it’s likely given what we’ve seen in other domains that use of GenAI makes everybody kind of sound a little bit, not just similar quality, but actually using some of the same underlying words.

Seth: Such a similar quality-hyphen, almost identical.

Bo Cowgill: Exactly. Right. M-dashes and using the word “delve” a lot and stuff like this.

Seth: Oh yeah.

Bo Cowgill: Yeah. So on average you lose information. I think the 4% to 9%... there’s not a lot of information to begin with. It’s like a very well-replicated finding that it’s hard to hire people and it’s hard to pick diamonds in the rough before they have much of a track record. Even if they have a track record at other companies, the match-specific aspect can be hard to pick up on. And if you think about an investor who had 4% to 9% lower returns—and one of our applications is actually in investing—then like, I think that would be a problem for the success of their business.

Andrey: But I mean, so I’m now going to make the point about, like, I really don’t care about whether this is a big or small effect. ‘Cause I don’t care about your setting. Not like it’s a bad setting to show how this would work in practice in a real setting we cared about, but like clearly Prolific people rating each other is not really something where we specifically care about the parameters that we estimate. For example, for an investment pitch, no one actually makes investment decisions based on a written artifact and that’s that. Right? Or you’d have to be pretty crazy to do that.

Bo Cowgill: So I will hard disagree on that.

Seth: Ooh, ooh, spicy.

Bo Cowgill: The most common place to get turned down from a startup pitch is before you even walk in the door, when you send your text-only pitch to an investor or an angel investor or a VC. Text-only, maybe some mostly-text slides. You send that in. This is where most people are eliminated. They don’t even get in the room.

Seth: I guess what Andrey would say is the marginal guy who gets into the room is never gonna get the deal.

Andrey: Yeah, I mean, that’s kind of...

Bo Cowgill: I don’t know if I even agree with that. I think that VC investing is probably really noisy as well. I mean, they lose a ton of money and not everyone agrees. I mean, there are these cases like Google where they had two top-tier investors, but I think that there are cases where people didn’t necessarily expect it.

Andrey: I don’t think... no, no. I really think if you wrote down plausible distributions here, it would be almost surely that this is really affecting people with very low probability of investment just to get... right? Because the baseline rate of investing is so low, even conditional on getting past that initial stage. Right.

Seth: And even if we take a step back, if we think about just AI as a technology that is good at automating the low-skill thing but leaves the high-skill thing less affected, you would expect that the more advanced setting, the setting with more applications, if we’re just taking the arg max, maybe it doesn’t matter so much that we’re mixing up the middle a little bit.

Bo Cowgill: I see what you mean. Yeah. Interesting to keep on studying this.

Andrey: I guess like, that’s what I was... I was really pushing back on just this... I would not... like, I like the paper, I think, viewed as a proof of concept, but I would not take anything literally. So I’m very uncomfortable with statements as like “investors would lose this many returns” and just in general, right? Like it’s not... lab experiments are great, but they’re not gonna...

Seth: Andrey would only trust this study if people reported 0% of these people are experts.

Andrey: Yeah.

Bo Cowgill: It is a proof-of-concept sort of paper and this is something we talk about in the discussion.

Andrey: Yeah.

Bo Cowgill: And yeah, it’s totally fair to say, I don’t know how...

Andrey: I guess I was gonna offer you a chance to say something about other papers. ‘Cause now there are a few other papers that are kind of trying to get at similar mechanisms.

Seth: Perfect. Do the meta-analysis live for us.

Andrey: I assume you’ve thought about it. Yes.

Bo Cowgill: I have seen some other papers in this area and they all look super cool. I guess the ones that I know best, although I don’t know every detail, are by, first of all, a PhD student at Princeton. And then a couple of PhD students at Yale that are both studying a change in Freelancer.com that happened when they released a GenAI basically cover letter tool to help your pitch if you were a freelancer.

And in various ways, I don’t want to speak on behalf of those authors, but it seems like, at least in those cases, there was this negative covariance idea where it seems like it actually harmed what used to be good signals about your match quality. And the way that the freelancers would do that was they use the GenAI tool to customize their pitch to look exactly like the requisition, or as much as possible, without lying. I don’t think they established there was no lying, but this is how they were doing it. So at least in these other domains, it seems like there’s some evidence that GenAI is similarly messing up signal accuracy and signal quality.

Andrey: Then there’s also, I think Emma Wilds has a paper, right? There’s a couple of papers on this, if I remember correctly. In one of them at least, they get access to the GenAI tools and that increases overall hire rates on the platform. Am I remembering that correctly or?

Bo Cowgill: That’s right. That’s right. And then at least in that case, they don’t find any sort of ex-post regret. And so, which might indicate that they were fooled and they were sent... unhappy. So this is a little bit more positive of a finding.

Seth: Are you... will you go out there? Will you now say, “And the reason that they found that GenAI was good was ‘cause...” Is this... they must have had a positive correlation between true skill and benefit from GenAI. Do you wanna make that claim in that population, in that context?

Bo Cowgill: Right, right. To be more clear about what they find, at least what I remember is them finding... is they don’t actually find that hiring improved. They just find a noisy enough covariance that they can’t reject... that they can’t sign it.

Seth: They fail to reject.

Bo Cowgill: Right. Right. So, not trying to start something here, but I thought like, well, maybe this is more of a somewhat ambiguous finding. And I also think that it’s presented not as “hiring actually improved,” but “we cannot reject that hiring actually got worse.” So then, maybe more precise tests will change this.

Andrey: So to be clear, the quality of the... we’re talking two things: the quality of the hires and the total number of hires, which are different numbers. And I think you’re talking about the quality of the hires. Is that right?

Bo Cowgill: That’s right. I think that the paper by Emma and John on this other freelancer platform, possibly the same one, you know, we don’t know.

Andrey: Truly a mystery which platform.

Bo Cowgill: Yeah. The employer can rate the freelancer. And so, if I recall their paper correctly, I think that they’re looking at those ratings and saying, it’s not like in the treatment group where you had these amazing cover letters, everyone was disappointed ex-post with what happened.

I mean, there’s a lot of other stuff that could go on there. It could be that they were super disappointed initially, and then the freelancer is like, “Oh, sorry. Well, I kind of masqueraded. Why don’t I do some extra work for you?” or adjust some other margin. But the punchline of our theory model is that this isn’t forced to go any single way. And it could totally be happening this way.

Seth: And be... but yeah. So I guess maybe let’s wrap up this idea of like external validity, right? Which is, the model seems to really imply that this will be super population- and context-dependent. And if the model implies that it’s gonna be super population- and context-dependent, then taking a snapshot in one place at one time can only tell you so much about everywhere else.

Bo Cowgill: I agree. I don’t think we’re trying to sell this as like, this is gonna happen everywhere, at least not on the basis of these results. Now, an interesting podcast discussion I think would be like, what did we expect? And we can go into that more speculatively.

Andrey: Well, let’s go to speculation mode.

Key Findings & Results Discussed

1. AI Win Rate vs. Human Experts:

   • The Prior: We went in with a prior that a generic AI (like GPT-5 or Claude) would win against a paid human expert in a head-to-head task only about 10% of the time.

   • The Headline Result: The paper found a 47.6% win rate for Claude Opus (near human parity) and a 38.8% win rate for GPT-5 High. This was the most shocking finding for the hosts.

2. Cost and Speed Improvements:

   • The paper provides a prototype for measuring economic gains. It found that using GPT-5 in a collaborative “N-shot” workflow (where the user can prompt it multiple times) resulted in a 39% speed improvement and a 63% cost improvement over a human working alone.

3. The “Catastrophic Error” Rate:

   • A significant caveat is that in 2.7% of the tasks the AI lost, it was due to a “catastrophic error,” such as insulting a customer, recommending fraud, or suggesting physical harm. This is presumed to be much higher than the human error rate.

4. The “Taste” Problem (Human Agreement):

   • A crucial methodological finding was that inter-human agreement on which work product was “better” was only 70%. This suggests that “taste” and subjective preferences are major factors, making it difficult to declare an objective “winner” in many knowledge tasks.

Main Discussion Points & Takeaways

1. The “Meeting Problem” (Why AI Can’t Take Over):

   • Andrey argues that even if AI can automate artifact creation (e.g., writing a report, making a presentation), it cannot automate the core of many knowledge-work jobs.

   • He posits that much of this work is actually social coordination, consensus-building, and decision-making—the very things that happen in meetings. AI cannot yet replace this social function.

2. Manager of Agents vs. “By Hand”:

   • The Prior: We believed 90-95% of knowledge workers would still be working “by hand” (not just managing AI agents) in two years.

   • The Posterior: We did not significantly change this belief. We distinguish between “1-shot” delegation (true agent management) and “N-shot” iterative collaboration (which they still classify as working “by hand”). We believe most AI-assisted work will be the iterative kind for the foreseeable future.

3. Prompt Engineering vs. Model Size:

   • We noted that the models were not used “out-of-the-box” but benefited from significant, expert-level prompt engineering.

   • However, we were surprised that the data seemed to show that prompt tuning only offered a small boost (e.g., ~5 percentage points) compared to the massive gains from simply using a newer, larger, and more capable model.

4. Final Posterior Updates:

   • AI Win Rate: We updated our 10% prior to 25-30%. We remain skeptical of the 47.6% figure.

PS — Should our thumbnails have anime girls in them, or Andrey with giant eyes? Let us know in the comments!

Timestamps:

• (00:45) Today’s Topic: A new OpenAI paper (”GDP Val”) that measures AI performance on real-world, economically valuable tasks.

• (01:10) Context: How does this new paper compare to Acemoglu’s “Simple Macroeconomics of AI”?

• (04:45) Prior #1: What percentage of knowledge tasks will AI win head-to-head against a human? (Seth’s prior: 10%).

• (09:45) Prior #2: In two years, what share of knowledge workers will be “managers of AI agents” vs. doing work “by hand”?

• (19:25) The Methodology: This study uses sophisticated prompt engineering, not just out-of-the-box models.

• (25:20) Headline Result: AI (Claude Opus) achieves a 47.6% win rate against human experts, nearing human parity. GPT-5 High follows at 38.8%.

• (33:45) Cost & Speed Improvements: Using GPT-5 in a collaborative workflow can lead to a 39% speed improvement and a 63% cost improvement.

• (37:45) The “Catastrophic Error” Rate: How often does the AI fail badly? (Answer: 2.7% of the time).

• (39:50) The “Taste” Problem: Why inter-human agreement on task quality (at only 70%) is a major challenge for measuring AI.

• (53:40) The Meeting Problem: Why AI can’t (yet) automate key parts of knowledge work like consensus-building and coordination.

• (58:00) Posteriors Updated: Seth and Andrey update their “AI win rate” prior from 10% to 25-30%.

Seth: Welcome to the Justified Posteriors Podcast, the podcast that updates its priors on the economics of AI and technology. I’m Seth Benzell, highly competent at many real-world tasks, just not the most economically valuable ones, coming to you from Chapman University in sunny Southern California.

Andrey: And I’m Andrey Fradkin, making sure to never use the Unicode character 2011, since it will not render properly on people’s computers. Coming to you from,, San Francisco, California.

Seth: Amazing, Andrey. Amazing to have you here in the “state of the future.” and today we’re kind of reading about those AI companies that are bringing the future here today and are gonna, I guess, automate all knowledge work. And here they are today, with some measures about how many jobs—how much economic value of jobs—they think current generation chatbots can replace. We’ll talk about to what extent we believe those economic extrapolations. But before we go into what happens in this paper from our friends at OpenAI, do you remember one of our early episodes, that macroeconomics of AI episode we did about Daron Acemoglu’s paper?

Andrey: Well, the only thing I remember, Seth, is they were quite simple, those macroeconomics., it was the...

Seth: “Simple Macroeconomics of AI.” So you remembered the title. And if I recall correctly, the main argument of that paper was you can figure out the productivity of AI in the economy by multiplying together a couple of numbers. How many jobs can be automated? Then you multiply it by, if you automate the job, how much less labor do you need? Then you multiply that by, if it’s possible to automate, is it economically viable to automate? And you multiply those three numbers together and Daron concludes that if you implement all current generation AI, you’ll raise GDP by one percentage point. If you think that’s gonna take 10 years, he concludes that’s gonna be 0.1 additional percentage point of growth a year. You can see why people are losing their minds over this AI boom, Andrey.

Andrey: Yeah. Yeah. I mean, I, you know, I think with such so much hype, you know, they should,, they should,, probably just stop investing altogether. Is kind of right what I would think from [Eriun’s?] paper. Yeah.

Seth: Well, Andrey, why don’t I tell you, which is, the way I see this paper that we just read is that OpenAI has actually taken on the challenge and said, “Okay, you can multiply three numbers together and tell me the economic value of AI. I’m gonna multiply 200 numbers together and tell you the economic value of AI.” And in particular, rather than just try to take the sort of global aggregate of like efficiency from automation, they’re gonna go task by task by task and try to measure: Can AI speed you up? Can it do the job by itself?, this is the sort of real-world economics rubber-hits-the-road that you don’t see in macroeconomics papers.

Andrey: Yeah. Yeah. I mean, it is, it is in many ways a very micro study, but I guess micro...

Seth: Macro.

Andrey: Micro, macro. That was the best, actually my favorite.

Seth: Yeah.

Andrey: I guess maybe we should start with our prior, Seth,, before we get deeper.

Seth: Well, let’s say the name of the paper and the authors maybe.

Andrey: There are so many authors, so OpenAI... I’m sorry guys. You gotta have fewer co-authors.

Seth: We will not list the authors.

Andrey: But,, the paper is called,, “GDP Val: Evaluating AI Model Performance on Real-World, Economically Valuable Tasks.”

Seth: And we’re sure it’s written by humans.

Andrey: We’re sure that it’s not fully written by humans because they’ve disclosed that they use AI. They have an acknowledgement—they have an AI acknowledgement section.

Seth: They used AI “as per usual”? Yeah. In the “ordinary course of coding...”

Andrey: And writing.

Seth: And writing. And for “minor improvements.” Yes. They wanted to be clear. Okay.

Andrey: Not, not the major ones. Yes.

Seth: Because,, you know, base... so, all right. You gave us the name of the paper. The paper is going to... just in one sentence, what the paper is about is them going through lots of different tasks and trying to figure out if they can be automated. What are the priors? Before we go into this, what are you thinking about, Andrey?

Andrey: Well, what they’re gonna do is they’re gonna create a work product, let’s say a presentation or schematic or a document, and then they’re gonna have people rate which one is better, the one created by the AI, or the one created by a professional human being. And so the first prior that we have is: What share of time is the AI’s output gonna win? so what do you think, Seth?

Seth: Great question. Okay, so I’m thinking about the space of all knowledge work in the economy. All of the jobs done by humans that we think you could do 100% on a computer, remote, is kind of the space of tasks that I’m thinking about. What percentage of those could an AI straight up... And just to be clear, Andrey, are these like kind of specialized AIs for the specific tasks, or are these kind of generic AIs?

Andrey: These are pretty generic AIs. Let me give you an example of a task, I guess, of at least the type that they’re thinking about in this paper. Mm-hmm. Although they think about a lot of tasks. So, the task is: “This is June 2025, and you are a manufacturing engineer in an automobile assembly line. The product is a cable spooling truck for underground mining operations, and you are reviewing the final testing step. In the final testing step, a big spool of cable needs to be reeled in and reeled out two times to ensure the cable spooling works as per requirement. And the current operation requires two persons.” So now the... it goes on and on. and then the...

Seth: ...and then the last sentence is “How many Rs are in strawberry?”

Andrey:, but the idea is, is that would... an example, yeah. Essentially you have to design, suppose you’re designing a jig using 3D modeling software, and creating a presentation using Microsoft PowerPoint as part of the deliverable. Upload only PDF summarizing design using snapshots of the 3D design created. The 3D design file is not required for submission.

Seth: There we go. So a pretty complex PDF being called for. I don’t think I could do it.

Andrey: I don’t think you could do it. I don’t think either of us can do it.

Seth:, I couldn’t do it in the amount of time the AI did it. You know, in a week, maybe.

Andrey: Yeah, I guess. I guess maybe, maybe in a week. Or, and maybe with AI assistance. With AI, with AI assistance, I could teach myself just enough. Yeah.

Seth: Right. I guess that’s a whole background issue here is we’re not thinking about AI for training. This is AI for just doing the thing. Yeah. Alright. So that’s an example of a very hard task. I think most tasks in the knowledge economy are easier than that. So that’s gonna ground my prior., I would say in real-world tasks, head-to-head versus a human, I’d be in the ballpark of about 10%. This is assuming we’re using like GPT-5 or Claude off-the-shelf versus a human who is actually paid to do that job. I’d be surprised if the AI wins up head-to-head much more than 10% of the time.

Andrey: Yeah, I think I’m in the same ballpark as you coming into this. You know,, I think I’ve tried making various work products using AI, and it’s,, rarely ever kind of a zero-shot process. One-shot, yeah. Or a zero-shot. Yeah. and there are oftentimes artifacts that kind of make it pretty clear that it’s an AI-generated thing, although not always.

Seth: Right. And so then we come around to like, some of those minor artifacts. To what extent can a little bit of massaging of these generic models get you a lot of additional productivity if you can get over those little hiccups that we run into with chatbots?

Andrey: But, and to be clear, I still think even... my prior going into it is even with some pretty sophisticated prompting, that the win rate would not be much higher than 10%, just because I’ve tried doing that. Right? Like, it’s not like I go into it and I’m like, “Hey, like do it, do it.” You know? I, you know, like I write like a pretty... I try to write a set of instructions for it and so on. I’m not, I’m not like naively using the models. And so,, I’m very often not getting kind of what I, what I’d like out of it. Right. As a result. So that’s...

Seth: Even as, even as top-tier prompters. Yes. You know, you might call us a 10x... we’re 10x prompters. I don’t know if you know that., you still don’t get what you want all the time. Right. Sometimes it’s just not... it. Sometimes the idea’s not in the model. Yes. And you can’t prompt it out.

Andrey: Yes.

Seth: but I guess,. I guess that’s one thing we’ll keep an eye on as we go, is just to what extent, they are adding additional scaffolding for these models. Okay. So the second prior that we were thinking about going into this is thinking about, like, kind of like the meta idea here is that any job that you can do on a computer, this AI should be able to do, if not in the immediate future, in the near future. That’s the dream, right? The “country of geniuses on the cloud.”

And so the question I have for you, Andrey, is looking at the occupations that are mostly about creating digital artifacts, so the knowledge work occupations, and let’s set aside whether there’s gonna be growth in those occupations or shrinking in those occupations. ‘Cause what we’ve said a lot, a lot of times when you automate part of a job, you might get more jobs or you might get fewer jobs. So setting aside that part of it, within the jobs that exist, are the people in those jobs going to still be making digital artifacts, quote-unquote “by hand,” as their main job? Or are all these knowledge workers gonna basically be managers of AI agents?

Andrey: And the question is about the share of workers whose primary job is currently to make these [artifacts]?

Seth: In the share of,, the share of, yes., let’s take it that way and let me give you a two-year horizon.

Andrey: So I would say that it’s still gonna be, you know, 85%, 90% of people,, that are still gonna be making digital artifacts by hand. But I think my question, I mean that’s, that’s my prior, I guess I would say. And, but kind of the main reason for it is it’s almost orthogonal to how capable the models are.

Seth: Okay.

Andrey: because what I’ve observed in my life is a lot of people just have AI usage aversion. So, mm-hmm. They’re just not adopters. And so...

Seth: Oh, so you’re, you have an adoption latency theory, which is just that, like it won’t grow because people won’t adopt it.

Andrey: Yeah. I, I’m just, I just look around and see a lot of people not adopting tools that are very useful,, in a variety of settings. And so to me, over the course of two years, can you teach an old dog new tricks, as they say? I, I don’t know.

Seth: The thing is, is it’s really, you can save a lot of time and people are, humans are also really lazy. So, well there are some forces going in different directions here. I guess, you know, I found this question of, you know, as I was asking it, this question of “by hand,” so ironic, right? Because like almost definitionally, if you’re doing it digitally, you’re not doing it by hand, right? So like what even is “by hand”? Are we just like moving up another chain of abstraction? And we should think about this as a continuum of, like, of knowledge work. We abstract a piece and we abstract a piece and we abstract a piece, but there’s always that long tail of knowledge work that remains to be done.

I think to me, this question comes down to like, what does it feel like in your job? Does it feel like I’m bossing an agent around, or does it feel like I’m getting messy guesses that I am cleaning up and doing, you know, half of the work, sort of iteratively, collaboratively? “Oh, you know, try this, try that.” That’s the AI systems that I mostly work with now, right? We keep on hearing promises about these agentic agents that’ll really be able to do 7, 10, 20-hour projects by themselves. My sense is that that level of “I am bossing around agents, I am not doing it myself,” is gonna be pretty rare within the next two years. So in 2027... I would think that that’s gonna be maybe 5% of knowledge workers. I mean, ‘cause right, it’s gonna be like lots of coders and then a small share of everything else.

Andrey: Yeah. And I wasn’t even thinking about coders. I was even excluding them from my thought process.

Seth: Excluding coders. Okay.

Andrey: Yeah. ‘Cause because I’m really thinking about, you know, like producing documents, presentations, schematics.

Seth: Well, here’s an interesting thing ‘cause we’re gonna see later at computer tasks, at, sorry, programming tasks versus other tasks. Is the AI actually a lot better at the programming tasks versus the other tasks? Hold on for evidence on that.

Andrey: Yeah. Yeah. And then did you wanna put a...

Seth: Did I, did I get a number? So you said 85%, so 15%?

Andrey: No, I said about 90. 90%.

Seth: 90%. Yeah. So 10% of, yeah. Knowledge work will be bossing around agents. Yeah. I’m, I’m, leads me closer to five, but... Very good.

Andrey: Alright.

Seth: Alright. Are we ready to go to the paper?

Andrey: Let’s rock and roll.

Seth: All right. So headline thing, this paper is gonna try to make an evaluation that can track how AI is improving in real-world economically valuable tasks. They claim that their tasks cover nine different sectors and 44 different occupations. Curiously, I don’t know why they specify both, because they’re gonna assign each sector one occupation. So it’s not like it’s sectors times occupations, it just, there’s 44 occupations and they’re associated with sectors, is the way to think about it.

Um, together these jobs make $3 trillion,, in the United States every year. it’s about a quarter of labor income., focusing on five occupations by sector that are digital and contribute most to total wage. How they’re selected, and I’m just gonna list a few of them for you guys. in real estate, there are jobs like concierges and rental clerks. In government, there’s jobs like recreation workers and first-line supervisors of police. In manufacturing, there are jobs like different kinds of engineers and, and so on. You know, programmers, any sort of like digital, you could do this job remotely, job... financial advisors, et cetera.

For each of these jobs. And this is like honestly, you know, huge shout-out, round of applause to this team because it seems like incredibly,, high effort. They recruited tons of experts in these occupations to first figure out what are the tasks in these occupations, matching that up with O*NET, which is a government database on the tasks, on occupations, and then sort of iteratively working with them to like define very narrowly, “Here is the economic task that we think AI can do.” And,, as a contribution, I think that that is so cool. I mean, the idea of like economic measurement of productivity at the task level is, I mean, I don’t know. It’s a dream since Taylorism of the 1920s. This is all the... this is a dream a hundred years in the making that we’re making progress on. Right?

Andrey: Yeah. Yeah. And okay. So that, that’s the setup. So we got 1,300 tasks across these 44 occupations,, that we’re gonna ask,, who’s better: man or machine.

Andrey: Yeah. I mean, yeah. I just want to double down on how impressive this effort is. I mean, you have experts from companies like Goldman Sachs, you know, Apple.

Seth: Oh, this is hilarious. The Air Force. They have a list of companies in the middle of the paper. Yeah. Why is this not a footnote? Why is this not in the appendix? Half of a page is just like, “Here are all the companies that our people have worked for. Apple, Amazon, 10 other ‘A’ companies.” It’s like, all right, cool.

Andrey: You know? Well, I get the sentiment. The paper is only nine pages long, and so I know you gotta like...

Seth: Half a page, a list of companies.

Andrey: I mean, these aren’t, you know. These aren’t your,, average Joes, right? They’re, they’re, they’re actually at these very high, you know, performing companies.

Seth: Average Joe works at Apple too. In fact, the person at Apple who’s taking time off from their lives to do this is maybe like the less of the average Joe than the high performer, or I don’t know, or they recruit... who thinks the best of the best.

Andrey: My sense is that they, I’m not saying like they recruited the best person in the world or anything, but these tasks, pay really well. Like they, they’re quite well compensated, so they’re not...

Seth: Right. So the average tasks, to give some context for this, the average task on their 220 tasks that they’re gonna end up focusing the most on took an average of 400 minutes. And if you multiply that by the median wage that we get paid,, someone would get paid $361 for doing the average task. So these are like real tasks. Yeah.

Andrey: okay. So. So kind of what do they do? They, they, they get these professionals to propose tasks. Then they use other professionals to figure out whether these are kind of really, you know, correctly specified tasks. They iterate on that a bunch. Mm-hmm., then once they’ve kind of come to that convergence, they have the AI do the task, and then they have other highly paid humans do the task.

Seth: And that... Wait, I think, and, and then there’s an iterative process. Yeah. Yeah. That’s process. There’s a process of prompt... Yeah, go ahead. Yeah, yeah.

Andrey: Yeah. So the iterative process,, are you talking about the, sorry, are you talking about the prompt process already or are you talking about the...

Seth: I’m up to the prompt process, but there’s the first, there’s several iterations. So, yeah.

Andrey: So I think the, the one I had in mind first was just the task is iterated on, between various experts so that,, it’s actually well-specified and representative of what a,, a task in this job category would be like. But there’s also additional iterations on the AI that is actually, right, doing the task. So you wanna talk about that?

Seth: I, I, yeah. And this is what I want you to take a minute to talk about, right? Because I think this is a really important point, is that they are not using... it’s not a huge amount of investment, but they are not using out-of-the-box Claude. They’re not using out-of-the-box ChatGPT in the sense of they’re not just prompting it naively. They’re spending a lot of time thinking really carefully about what is the perfect prompt to elicit this set of tasks.

Andrey: And so this is actually a great prompt for you all listeners if you were, wanted your [AI?] to do similar tasks, right? So this is actually where my introductory joke came from because the prompt begins, “Special characters: never use the character Unicode 2011.” But it goes on, you know, and a lot of these are, are kind of mostly about tool usage, right? And so...

Seth: Right. you know, like talking about... one of the basic prompts that’s so important is, is like, “If the task requires you to resend a PDF, definitely send a PDF” is one of the prompt improvements.

Andrey: Yeah., there’s some stuff like, “Take your time, do these thoroughly.” There’s, there’s other things like “Display all the PNGs.”

Seth: Be sure to double-check. Yeah. Double-checking things. Yeah.

Andrey: Are some... “Be sure to look a few days and see...” there’s a... “This is important” in capital letters and “Mandatory.” But I guess, I guess what I’d say is, this sort of prompt,, iteration is pretty standard in the industry at this point., there are a variety of frameworks that kind of let you do this programmatically even., but if you think about your... [Codecs?] or your Cursors... there, there’s a lot of prompt engineering going on under the hood., or, or even your ChatGPTs or your Claude chat, you know, there’s that system prompt. They’re, they’re tweaking all the time. So there’s nothing, I’d say there’s nothing unusual ‘cause it’s well-known that to get a good performance out of these,, systems, you need to,, have a good prompt.

Seth: I think that’s exactly right. I just wanna connect this to the point you made earlier about adoption lags. Right? And I agree with you that it’s very standard to, you know, for a company or an individual to spend a good amount of time prompt searching before they find one they’re good with. But even a small friction like that makes a big difference in terms of adoption, I think.

Andrey: Yeah, totally. Unless that,, prompt is given to you out-of-the-box,, baked in, in Cursor or whatever. Yeah, yeah, yeah. You’re just... or not you, I don’t wanna say, but most people, they’re, they’re, they’re gonna try...

Seth: Dear listeners, dear listeners, we’re sure that you are the best prompters.

Andrey: Yes. I’m sure our listeners are better prompters than we are, but everyone else, you know, you know, I think, I think they might have a bad experience with one prompt and kind of overlearn about the capabilities of the system., which is kind of an argument for why we might, we might see a lot more application-driven adoption, right? Rather than, you know, using a generic LLM,, that could be capable of doing something. You might have a packaged service,, like let’s say “PDF Creator.”

Seth: Alright, Andrey. This is what I wanna talk to you about. This is what I wanna talk to you about. ‘Cause I low-key think the paper’s about this. I think that the secret theme of this paper is: What is the relative return towards this basic prompting work, this basic scaffolding work, versus another hundred billion parameters in the model? ‘Cause we do get an estimate of that, right? And so I was really surprised to see kind of that, you could get about a 10% improvement on win rate. I guess, can I just...

Andrey: Can I just pause you on that and can actually just go through the results first before we...

Seth: Alright. Okay. Yeah, yeah. Listeners, listeners, you know how excited I get. You know, I get off the chain and you need to reel me back in. So let me give you the results, and then I’m gonna wildly speculate.

Andrey: Okay. Perfect. Yeah, yeah, yeah. Well, let’s, I’ll just say what the... so we haven’t actually done the description of the pairwise task, which is essentially this highly incentivized person,, they have to choose which one is better, the AI-generated one, or,, the output created by another human expert. and you know, and just in general, kind of with these things, you might be worried that,, the graders aren’t putting in enough effort, right? Like,, maybe they don’t really care which one is better. And so they sometimes, they might not read as deeply as they’d like. And,, you know, from having talked to some of the authors of the paper, it seems like these graders spend quite a bit of time just evaluating which of the outputs is better.

Seth: Right, they said about an hour per evaluation. Yeah. Is a real... yeah, yeah, yeah.

Andrey: So it’s not like they’re just like, “Eh, I kind of feel like this one is better than that one.” I mean, I, to be clear, I still think that, you know, we could probably do better in incentivizing proper grading, but kind of, it’s not, you know, some of the more obvious flaws you might think are there, are, are, they’ve thought about them.

Seth: Right. No. These, like we said, extremely well done within the bounds of what they’re doing,, from everything we’re reading. Okay. So we’ve got, we are evaluating them, we’re going head-to-head. And to be clear, we’re only, as far as I understand, it’s only for 220 of these 1,300 tasks. Do we have the resources to actually do this evaluation? But within the 220,, we’re gonna ask, okay, what’s the win rate of GPT-4o, or 4-mini, [O3?], GPT-5? What? So my prior was that the AI will win 10% of the time. What were we seeing?

Andrey: Yeah, so we’re seeing... and perhaps the most remarkable part of this paper... which is that Claude Opus makes a showing.

Seth: Claude does better.

Andrey: Claude does the best. Claude does the best of all the LLMs with 47.6%, which is just very close to human when you really think about it. I mean, it’s almost a coin flip which one is better. right. And then GPT-5 High also does pretty well at 38.8%, but actually substantially worse than Opus,, which is quite interesting.

Seth: Right. Bold of OpenAI to go out there. Although,, maybe we wanna talk about different domains, different occupations here. There are areas where the OpenAI models shine.

Andrey: Yeah, yeah. okay...

Seth: So the headline result,, Claude almost human parity on these tasks. [Expletive] insane, at least in terms of, you know, that win rate., and then OpenAI close behind at 39% with their leading model, but it differs a little bit by sector and occupation.

Andrey: Well, I just wanted to mention one other thing.

Seth: Go ahead.

Andrey: Before we moved on to sector and occupation, I just... ‘cause like one of the things that, you know, with a theme of this show has been,, you know, scaling laws and how much better newer models are. And it’s interesting to me the set of models that was considered here. So we have GPT-4o, which, you know, is an older model, but not that old of a model. It’s a kind of a cheaper model and it actually only wins about 10% of the time. So we’re kind of pretty well calibrated if we think about that model.

Seth: And that’s actually right. We’re just taking it out of date...

Andrey: Closer to the model that many, many people, you know, had access to essentially until July. And then, O3 High, which is a model that essentially no one uses,, because it’s really, really expensive,, is at about 30%. And then GPT-5 High, which I guess may be the “thinking” version of the ChatGPT interface. I’m not exactly sure. It’s kind of unambiguous, frankly. Because maybe they have a...

Seth: Is there a special GPT model that’s being used here?

Andrey: Well, there, there’s a router and who knows what’s being routed where.

Seth: It gets routed. It gets routed to the good server.

Andrey: Yeah, yeah, yeah. So that, that’s, that’s kind of almost, you know, 35% to 40%. Right. So, so we do see improvement within newer models or the models that are more compute-intense. But I would also say that most people do not have this quality of a model as their default.

Seth: Yeah. There does seem to be a giant... so this is speaking to what is the relative value of overall progress versus prompting progress? I mean, it seems like in a year of overall progress, we’ve boosted—arguably boosted—this win rate by 30 percentage points and, like, arguably saturated it if we’re getting a, you know, almost 50% win rate. I mean, if there... it could... I’m not saying we actually saturated it. In fact, one of the arguments in the paper is that they’re gonna use win rate as their main success measure because it doesn’t get saturated as easily., but it’s damn impressive, that amount of progress in the last year.

Andrey: Yeah. so all right, so let’s go, you know, you wanted to... Yeah. Go by occupation. Yeah. Yeah. All right. Go for it.

Seth: Oh yeah. So what jumped out at me about that was basically all the models do pretty well at basic clerking jobs, and all of them are decent at programming. Claude... kind of the stuff that all of the models are good at, Claude just knocks out of the park. Right? Then there’s some interesting kind of turnarounds in the sense that the ChatGPT models seem better at sales and editing and audio-visual than, Claude. I wonder... so there’s like two different things going on here. One is you might think that ChatGPT is like a little bit more attuned for writing versus coding. That’s maybe an intuition that I have.

Andrey: I guess what I’d say is actually,, for some of these occupations we do see that, that,, the AI is actually better than the human.

Seth: This will be above 50%. Yeah.

Andrey: For example, I think statistically significantly, Opus is better than humans at being a private detective.

Seth: Now that was nuts. That was nuts. Or the rather, the knowledge tasks of being... Yeah.

Andrey: Which is kind of like, you know, an interesting thing to think about. Does that mean that private detectives,, are going to have their job removed? What are, what are we actually... or is it just that private detectives are really good at investigating and not that good at making presentations? Right. So like, like what are we, you know, that’s an interesting thing to think about.

Seth: Right. How does this translate into people’s jobs actually changing? When I think about a private eye or a police supervisor, this sounds like internet research tasks. So yeah, I mean, probably just internet research goes faster and then they spend more of their time on their other tasks would be my simple guess.

Andrey: That, that’ll be my simple,, guess as well. I, yeah, I mean, I think I’d be a little, you know, because the standard errors are so large for individual occupations, I think I’m a little wary of overreading into them. I think like standout things, like all the models are bad at being pharmacists. All the models are bad at being film and video editors and producers in direct ways.

Seth: Well, well, but, but, but... ChatGPT, the GPT models are significantly better than Claude’s. So that is an interesting difference.

Andrey: That is different than film and video editors and pharmacists, which is the one I was mentioning. Oh, okay. I mean, I’m not saying that you know, that there are differences... there are statistically no difference across models. But I’m just saying that just in general, there are certain categories of jobs where,, the models are far away from 50% and others where they might even be better than humans. Right, right.

Seth: And I guess, and then the third, and then the third kind of twist on that is kind of surprisingly, there’s not [monotonicity?]. Some, in some of these cases, most of the cases, Claude is the best, but in some of the cases, the AI models are better.

Andrey: Yes, yes. and you know, they, you know, another way to think about it that surprised me is actually they did it, the win rates by,, the category of output. So for pure text, the models suck. For PDF, the models,, at least Claude is quite a bit better. For Excel,, the, you know, Claude is very good., for PowerPoint, Claude is very good. And then for “other,”, a lot of the models are good. But to me the just the... I would’ve thought that at text they would actually be quite, quite good. But that’s actually the category in which most of the models are doing pretty badly, which is kind of...

Seth: I think it has to be endogenous to what kind of jobs are associated with pure text, right? And I imagine if it’s pure, sort of creative... I guess creative writing... both of them should do okay at, but I’m not surprised that OpenAI is a little bit better at...

Andrey: Yeah. But I guess I’m just surprised at how low they are, you know, not, not at who’s better.

Seth: Maybe it’s, it’s, I think this is might be a taste thing, right? It’s maybe like, you know, like the [winch?] either works or it doesn’t, but people still have a strong preference for a non-AI voice.

Andrey: But it’s not... but I guess what puzzles me about that is when we’ve seen a bunch of behavioral studies which are kind of like heads up, you know, “Do you even know this is an AI?” and people, people can’t detect whether...

Seth: Are those expert contexts?

Andrey: No. And this is kind of, this is kind of this interesting thing. Maybe the experts in their own domain of expertise still are able to distinguish the model, you know, the quality, and therefore the models...

Seth: There’s still hope for expertise. There’s still hope for us.

Andrey: But for, but for normies... but for normies, like... they already... normies have no idea who wrote the damn thing.

Seth: Right. And audience, just to be clear, we include ourselves in normies in 99% of world topics that are outside of our domain, right?

Andrey: Yes. I’m sure I’ve been fooled by AI output in, in many ways. I think another interesting exercise that they go through, which I kind of view as a prototype more than anything else, is like the, essentially the cost improvement from using the AI versus human., and it kind of makes some assumptions about what that...

Seth: Right. How do they interact? I kind of... yeah, this is, this is prototype, but a very intriguing... so walk us through that result. Yeah.

Andrey: So you know. So you can imagine that. Alright, so the human does it end-to-end, that takes a certain amount of time., alternatively the human can prompt an AI. The AI does it. The human needs to evaluate the output. So that’s gonna take a certain amount of time and maybe will even iterate with the output a certain amount of time before they get what they want. And so they make some reasonable assumptions here and think about like, what is the cost improvement and the speed improvement from using the different models in different collaborative modes.

Seth: Right. And they’re gonna consider one-shot, so use the model once and then fix it, or N-shot, use the model lots and lots of times and try to get it that way.

Andrey: Yeah. And just, I’ll just focus on the main figure in the paper,, where, what’s interesting is that GPT-4o, which is kind of the old default model in ChatGPT, it’s kind of not a cost improvement and it’s not a speed improvement. And that’s because the outputs are so bad....

Seth: Right. And its win rate is low.

Andrey: Right? Yeah. So it’d be one thing if like, it could just do it by itself sometimes, but it doesn’t do it by itself often, and in collaboration with humans, it can actually slow you down.

Seth: Yeah. now 4-mini,, which is different than 4o. Remember how open... how good OpenAI is at naming their models.

Uh, it’s already better. But,, compared to that, GPT-5, which is their newest model,, it achieves substantial cost improvements...

Andrey: ...blows it away...

Seth: ...uh, 1.5x, over 1.5x, and substantial speed improvements,, over 1.25x. And importantly in both of these metrics, it beats O3, which is kind of a more capable reasoning model. and that’s because cost matters in an ROI calculation and speed matters in an ROI calculation. And that’s kind of, You know, one way one can read this as kind of a... you know, OpenAI got criticized a lot for the GPT-5 model, like somehow it was underwhelming. But actually, you know, for adoption and utility, what we care about is economic value and not, you know, whether it can solve the gold medal on the IMO, right? So, and so here it’s, it’s providing a lot of that value.

Andrey: Right. And so the number that jumps out at me is with ChatGPT-5, which is the model that,, you know, their best model, they say in the, “you do it,, you have the AI do it once and then fix it” configuration, that leads to a 12% speed improvement and an 18% cost improvement. And in the “you can just, you know, prompt it as many times as you want and incorporate that in your final answer,”, a 39% speed improvement and a 63% cost improvement. So, so, I mean, damn, if you could improve,, the productivity of all knowledge workers 60%, that’d be quite a thing.

Seth: Yeah, that would be, you know, pretty, pretty great....

Andrey: Is that the “country of geniuses on the cloud”?

Seth: I don’t... 60%. I don’t think it’s geniuses. You don’t really think about geniuses making great PowerPoints. I mean, this kind of...

Andrey: Ben Jones is excellent, sir.

Seth: I, I guess, yeah, the, I don’t know if we’re ready to kind of come to some of these meta thoughts,, about what it means to kind of automate this, these sorts of tasks. But, yeah. Yeah. Before we get to that, are there any other parts of the paper that we should mention?

Andrey: In, in that particular... there’s two other results I wanted to get to.

Seth: Okay.

Andrey: the first is you might be worried that sure, these models are doing good at win rate, but maybe like when they lose, they’re saying something horrible, right? Yeah, yeah. So it might be, it might be,, better at the median, but worse on average, right? Like, we don’t think this is like super plausible, but it’s something they check for. And what they do is they ask, for the models,, whenever they do these head-to-head comparisons and the AI loses, they ask like, “Why did it lose?” Yeah. And 2.7% of the time it was due to a quote-unquote “catastrophic error.” And the examples they give are: insulting a customer, giving the wrong diagnosis, recommending fraud, suggesting actions that would cause physical harm. We do not get the details audience, but I promise you I will ask Andrey to ask his friend who was on this paper, what was the horrible thing that AI did?

Seth: Is that... just to be clear, I am not friends with anyone in this paper. Just someone I saw at a conference.

Andrey: Read his name. That’s true. so I don’t know, it’s just 2.7% catastrophic error rate. I mean, I think that’s probably a little bit higher than a human.

Seth: Yeah, no, it’s certainly a lot higher than an incentivized human in these jobs. I mean. But I guess it, yeah, I mean, it depends. Certainly doctors misdiagnose all the time. I mean,...

Andrey: Yeah, that’s kind of the odd man out, right? Yeah. That’s, you know, that happens, but you know, it’s recommending fraud.

Seth: Yeah. Recommending fraud. Yeah. That’s not a good look.

Andrey: If I was in a room with a lawyer, I think 3% of the time they would recommend fraud.

Seth: It’s,, you know, the Better Call Saul was a huge part of the training set. but I think most work outputs, you know, there’s, they’re, they’re in the end presented to some other people who also vet it. There are many, like the way organizations are structured is that there are many checks and balances on a lot of this output., but it depends.

Andrey: But maybe it suggests that we’ll need more of them as we move to an automated world. And you know, you’re, the job of the future will be,, automated AI, you know, I don’t know, sanity checker.

Seth: And they, by the way, they spent a lot of time training in it, you know, or trying to use a model to grade the model outputs, right?

Andrey: Yeah. You wanna talk about that for a second?

Seth: Yeah. They achieve some kind of pretty reasonable results, I’d say. So the automated grader agrees with the human grader about 65% of the time. versus inter-human agreement is about 70%. I guess, I guess if I had to like poke at any part of this paper, I actually might just poke at this, right? Hmm. 70% inter-human agreement. Seems low. Seems quite low. Like, if I were to say like the win rate is this very meaningful feature, then why... and kind of we really wanna do well here...

Andrey: ...and [humans?] are winning 30% of the time. You’d be, you’d be concerned.

Seth: I, I mean, you would think that humans would agree, you know, expert humans would agree on something where there’s truly a right answer. Clearly we’re not seeing that here. And one version of that is something I’ve already mentioned, which is maybe the incentives are not high-powered enough for them to really determine what is better than, you know, which of the options is better.

Andrey: You don’t think there’s some ambiguity in like in that winch example you gave at the beginning, all right, so maybe the AI gives a winch that’s a little bit stronger and the human gives a winch that’s a little bit more colorful. I mean, it seems like a lot of these settings are pretty...

Seth: No, no, sorry. That’s, so that’s where I was going. It wasn’t like, “Okay.” I was saying I think we interpret this quite differently if we think that a lot of what’s going on here is that there’s some sort of latent preference heterogeneity.

Andrey: Taste.

Seth: Yes.-huh. Yeah. That some, some, some experts like certain types of work,, other, other experts like other types of work. And you could say, well, maybe it’s just all aesthetic. Like who cares? You know, you know, who cares that this guy likes their slides red and this guy likes their slides blue. But maybe it’s actually quite relevant to the job. And I think that’s kind of an open question to me is like, is there a reason why this particular expert thinks that one output is better than another? and that they disagree with their other human experts? yeah.

Andrey: Yeah. One thing, one, one follow-up they do on that is they do ask in, the examples where one, where the AI lost, they ask, “Why does it lose?” Yeah. And greater than 50% of the time, they say it was, it was “adequate,” but just, you know, their faith was the other one was better.

Seth: Yeah, yeah. Yeah. And to be clear, so I mean, you, I’ve seen a lot of “adequate” work products in my life from humans.

Andrey: Humans in their “adequate” work products. Yes. Yes. Okay. So now can I go to the thing that I’m on about, Andrey?

Seth: Yes, you can go for it.

Andrey: So my interpretation of this figure is. Going from a version of GPT-5 that is sort of out-of-the-box on these tasks versus one that they’ve prompt-engineered, they’ve been able to increase the win and tie rate by only three percentage points. So the scaffolding, it’s meaningful, they do do some work on it., but in terms of like the benefit compared to just going from the models from a year ago to the models from today, it’s dwarfed. It’s, it’s 10x better to just go to the bigger model rather than to fine-tune. Is that, do you agree or disagree with that interpretation?

Seth: Not even close. Not even close. Seth, I... please.. This specific plot is not even the one I think is addressing what you’re talking about. Unfortunately.

Andrey: I thought that the,, I guess in my, in my eyes, I thought these were pretty similar.

Seth: No, the concise one is quite different.

Andrey: So explain to, explain to us in the audience what,, Figure 9 explains.

Seth: My thought process here was prompt tuning and scaffolding increases... so this is the win rate for GPT-5 High, right? By about five percentage points. And now, right, your Figure 14 is specifically about telling it where to find stuff.

Andrey: Oh, so it’s a su... Okay. Sorry. So it was like...

Seth: So it’s like the way, the way I interpret Figure 14, is really about,.. like, if you’re giving it a vague, like, you’re like, “Hey, like make this report for me, but I’m not gonna tell you where like the materials are.” Like that’s very, that’s very different than thinking something about like, fine-tuning. Right? It’s really like... it’s like, I’m like being a “bad boss,” I’m just gonna give you very ambiguous instructions versus like, I’m actually gonna be like, “Hey, here’s a folder with all the materials. Like, go at it, you know?”

Andrey: Oh, this is great. I read this too fast. This is even more interesting than I thought. Yeah. Okay. So, all right, so turning around. So is it fair to say from Figure 9 we get that the prompt fine-tuning is worth about,, four or five percentage points of improvement, but from Figure 14 we get that being a “bad boss” is worth, you know... and not explaining basic stuff that you would expect to be... yeah, it has...

Seth: ...about a similar effect, is kind of my understanding.

Andrey: Negative in the other direction. Okay. Yeah, yeah, yeah. So I mean. To, I’m, I’m gonna be frank with you, Andrey. The reason this stuck with me is because I thought that this was gonna matter way more. Hmm. I thought like prompting was gonna matter, like almost half as much, if not as much as model quality, but, I, there you go. It’s the “use the bigger model, man.”

Seth: Yeah. I mean, I think prompt tuning the way I’ve always thought... always, like, it’s not like I’ve been thinking about prompt tuning for that long. “My entire life. My entire life I’ve been thinking about prompt tuning.” No. the way I think about prompt tuning is it gives you kind of a constant benefit on top of just a base model. it allows you to do some percentage better, but it’s, there’s not a scaling law aspect to it in the same way.

Andrey: Percentage points better. Yeah. Yeah. Yeah. So maybe so yeah. So there you go. So, a year ago, a five percentage point improvement was a 50% improvement.

Seth: Yeah, I don’t know. I don’t know about that. I more think of it as a percent-over-performance improvement, rather than in levels, if that makes sense. So I would say like, you know, before, if we were only able to do 10% win rate, then like prompt tuning would’ve given you, you know, 12%. But now you know, because the baseline is higher, you also get a constant, you also get a better improvement from...

Andrey: More benefit. There’s more in the model to find.

Seth: Yes. Yeah, exactly. Yeah. Yeah...

Andrey: So, okay, but, so, but, but high level, was this surprisingly... you thought that this was in the ballpark of...

Seth: How... Yeah, yeah, that’s kind of what I’ve, what this particular aspect of it I was not particularly surprised by.

Andrey: Yeah. I mean, I, to me, I see people flailing with bad prompts and people doing amazing with good prompts, but, maybe just the range of, maybe they started with a pretty good one.

Seth: ‘Cause they’re... I think, yeah, I mean, I don’t think they started with a bad one. I think the nature of this task involves very specific instructions already. Right. So it’s not like they were saying “do it,” you know, they were “here, job, do, read my mind.” You know, it’s a, like the entire, this entire task is very, like, really well specified by the expert. Mm-hmm. Mm-hmm. yeah. I mean, tool use is very important. Just to be clear, it’s obviously this couldn’t be done without tool use.

Andrey: Right. Right. It needs to call CAD to make the model. It needs to call all the different APIs to interact with other things. Yeah., although they don’t call ‘em APIs anymore. What do they call the, the,, the APIs for AIs now?

Seth: [MCPs?]

Andrey: [MCPs?] Why don’t they just call ‘em API or like AA-APIs? I’d be able to remember that.

Seth: Yeah. I, I, [AI-PI?] I mean, I do think it’s, you know, it does kind of raise this question of like, what is an AI? I think for a while, people were thinking, “Oh, it’s just, you know, the LLM.” But clearly, you know, now that an LLM can use an arbitrary programming language with arbitrarily smart packages, you know, I don’t think... Right. The capabilities in the model are quite different depending on what tools it has.

Andrey: Very well put. Are there any final results you wanna,, bring up, Andrey, before we get into our posteriors?

Seth: No, I just wanted to actually make the following point.

Andrey: Do it.

Seth: Just like, one of the questions that I hear here, you know, talking to AI folks is, is just kind of like, “Well, why aren’t economists at the forefront of, you know, AI and economics?” And I think about this...

Andrey: It’s very expensive.

Seth: Yeah. And I think about this paper and I’m like, I don’t know of a single team of economists that could pull this off, just organizationally and financially. organizationally,, this is, you know, 1, 2, 3, 4, 5, 6, 7, 8, 9...

Andrey: He won’t tell you their names, but there’s a lot of them.

Seth: There’s a lot of them. There are nine main authors, and then a bunch of sub-authors.

Andrey: And a bunch of authors that are not main...

Seth: And yeah, a bunch of non-main, main authors and, but apparently also equal contribution. and, and like these are, you know, AI researchers, so we assume... let’s do their salary.

Andrey: ...getting paid a million dollars.

Seth: Or I’d say I wouldn’t be surprised if the average salary...

Andrey: Average wage on average.

Seth: ...average, you think is the average yearly salary of this research team is probably two to $3 million per year.

Andrey: Right. And then probably, you know, double it for their expenses.

Seth: Yeah. And then the expenses of recruiting all these people is, yeah, just staggering. There’s just no way....

Andrey: You think it’s a $50 million study?

Seth: I think that’s right. Ballpark?

Andrey: No, I think it, no, I don’t think it’s quite that high. And I don’t know how much time it took these people to do it., that’s, that’s...

Seth: You said AI do it, dude.

Andrey: Yeah. I mean, I, I more put it like at the, maybe somewhere in the $2 million to $5 million range, but still it’s a lot of money.

Seth: You don’t think it’s 10 million?

Andrey: I don’t think it’s 10 million. I mean, it depends, it really depends on how much time each of these guys...

Seth: ...is getting paid over.

Andrey: No. Yeah. It depends on how much... depends. Got, yeah. Yeah. I think with the, yeah, it depends on whether this was the main part of their job for a while or not. Sorry to speculate about, you know, if you’re listening and an author, sorry to speculate about your salary.

Seth: Yeah, no, we, I’m sure all... we’re very happy for you all,, impoverished and deserving of our love and support.

Andrey: Yeah. All right. well, while we’re like kind of multiplying some numbers together. Yeah. so I was trying to like kind of ball... instead of ballparking how expensive this study was. I was trying to ballpark, like, so they say that these jobs constitute $3 trillion of economic output in the US and they’re gonna claim that like some per... I mean, I don’t know. The implicit claim in this paper is that once we figure out how to implement the technology, some percentage of those, of that work will be automated.

I think that plausibly they’re on a path to automating maybe like a third of that. Right? Do you think like, maybe there’s a trillion dollars... I know you, you really hesitate to speculate on dollar values, but I mean, people are betting on OpenAI thinking it’s gonna [create?] trillions of dollars of value., right now, maybe one trillion’s worth if we think there’s, you know, about one-third of these,...

Seth: ...per year.

Andrey: Per, per year. Per year., yeah. I guess if you make a trillion a year, it’s, it’s worth a lot in terms...

Seth: Just remember about stock versus flow.

Seth: Fair enough. Yeah. All these OpenAI getting compared to the GDP of Sweden. Stock versus flow. All right. Yeah. yeah. Anyway, so that’s just something I’m thinking about, right, which is whether or not we think that that’s the most important result from the paper. To me, one of the motivations of this paper is: Can we do something fancier than [Eriun?] in terms of thinking what is the total economic value of current generation technology?

And they get a number that’s basically,... so if he says it’s 1% of the economy and we’re saying it’s one-third of [a quarter?], we’re say... I’d say it’s like [one-twelfth?] of the economy. So, you know, a slight disagreement with [Eriun?] there. Do you think it’s close? What percentage of the economy can be automated by AI, Andrey? Is it closer to 1% or closer to,, two-ninths?

Andrey: I mean, this goes to the question of value creation, right? Like, and think about what, what hours-wise, what people spend time on. But you know, I’m currently working at a company and I, you know, and I don’t wanna spend...

Seth: How dare you as an academic. Yeah.

Andrey: How dare I. I don’t wanna, I don’t wanna speak too much about my, my work for a variety of reasons, but, you know, I’ll, I will note that a lot of my time is spent in meetings.... I’ll just make a side note to the listeners that Seth just made approximately five inappropriate jokes in a row. And for our reputation—each one funnier than the last—we’re just gonna not include them. But if you’re interested, you can, reach out to us in private channels and Seth will share his, comedic insights.

Seth: Alright. So did we, did we come... So let’s give our posteriors.

Andrey: Well, no, I get, I get... No, no, we’re not finished with the meetings, I guess.

Seth: Oh, okay.

Andrey: Why was I talking about the meetings?, which... I was talking about the meetings,, because I spent a lot of my time in meetings and, as far as I can tell, AI cannot automate my participation in these meetings. Now why is that? I don’t... that’s actually like a, an interesting question. The way I think about it, it’s like organizations are decision-makers, you know, kind of similar to some other work we’ve covered in this podcast. And ultimately the ultimate output is not like the hours of work making the presentations and the documentations and so on. It’s making resource allocation decisions to produce stuff. And, and so even if like hours-wise,, the, you know, some things can be automated, doesn’t mean that the people are going to lose their jobs, let’s say. How about that? What do you think about that?

Seth: I, I think you’re, I mean, you’re totally right to point out that like a lot of what counts as “doing a job” is not perfectly lining up with the tasks measured in this study. The question then becomes to what extent can the things that are measured in this study as being high win rate for the AI be unbundled from the things that aren’t?

Right. So the issue here with meetings is for whatever reason, they have decided that the person who writes... let’s say that you’re working for like a bus company, something that’s completely not funny at all, right? And at this bus company,, you, I don’t know, you have to make some sort of like, logistical decision, right?

Andrey: Like,, when to replace the engine.

Seth: Yeah, when to replace the engine, whatever. And so like if there’s a part, like part of that is an intellectual decision that could be automated. The thing could do research, right? But maybe there’s something that can’t be separated from that. Maybe it’s the liability component. Maybe there has to be a human that is responsible for the engine working and we can punish them if he makes a wrong decision, he or she. Maybe the thing that can’t be taken out of it is there’s some sort of special context that you’re gonna be told about in the meeting that is super weird and happens like one out of a hundred times and it’s gonna dramatically either increase or decrease the rate at which engines need to be replaced.

So you can imagine like a long tail of things that you might learn at this meeting that’s going to affect your future knowledge output. Right? So, and because in knowledge work, everything, at least in principle, is connected to everything else... If you think about the, you know, Quinean web of belief, there’s like a certain sense in which no knowledge work is completely separable. So yes, you’re gonna have to go to the meeting.

Andrey: But I think there’s another role which is like consensus building. And I know, you know, I know, you know, common knowledge of the factors that resulted in the decision being made. And meetings are kind of an enforcement mechanism for that. Now you can imagine maybe,, new organizations where since it’s AIs, they don’t, we don’t need this thing happening. But, you know, a lot of organizational processes are really about this social thing, not about,, the actual decision. The CEO might have already made up his or her mind, right?

Seth: Right. Right. So meetings as coordination mechanism. I guess then it comes back to like, can we unbundle coordinating Andrey from work?

Andrey: Yes. Yeah, yeah. Yes, that’s right. That’s right.

Seth: Yeah. I mean, like in principle, if we don’t need you to do any work, we don’t need to coordinate you. Right. We need to coordinate you insofar as there is another piece of it that you’re responsible for that we can’t automate.

Andrey: Yes. Okay.

Seth: Very, very provocative to think about, Andrey. Okay, so going in, we asked what is the win rate of AI versus knowledge economy workers in top knowledge economy occupations today? Right now, if you had put up man versus machine, John Henry going at it with his hammer, does he stand a chance? Andrey, what is your posterior?

Andrey: Yeah, so 10% was clearly off. I don’t think I’m updating all the way to, you know, 39%, 46%... or whatever the Opus numbers...

Seth: 47 is the Opus, 39 for... Okay.

Andrey: Yeah. Yeah. not... because, just because,, they didn’t, we don’t have this for all the tasks that are in their super sample. We only have it for the 220. I assume there’s some selection in there...

Seth: Yeah, yeah. Fair enough.

Andrey: So I’d say maybe 30%. Yeah.

Seth: You’d update from 10% to 30%.

Andrey: Yeah. I’m 30%.

Seth: 30%. I’m gonna update from like 10% to like 25%. I’m definitely moving very strongly in that direction. I do think that these are probably selected to, right. They’ve gotta be, because they wouldn’t use ones where the AI just fell on its face immediately. Yeah.

Andrey: Alright. Prior, number two: What share of workers in occupations where that occupation today makes digital artifacts will still have making digital artifacts, quote-unquote “by hand,” for themselves as their primary job? That was almost English. It was a lot of connected words. If you think you understood it, tell me where you think we’ll be at, at two years after reading this paper.

Andrey: Yeah, so I think my initial guess was what, 90%?

Seth: Yes.

Andrey: So I think, yeah, I’m, I’ll say 85%. I think that just the... I think people are slow to adopt. People are slow to change their work processes, especially in organizations where there are habits and plausible deniability and all this sort of stuff. And,, I think even though in principle it should be a lot more, it won’t be a lot more in two years, but yeah.

Seth: So yeah. So, but still, you’re still thinking it might be 15% of knowledge workers. let me ask you a question. Do you consider that 1x collaboration? Would you consider that “by hand,” or would you consider that AI agent management?

Andrey: I’d say for like, gets you 99% of the way there and then you just need to tweak it a little bit, I’ll still consider it part of the, you know, part of what I’m talking about. If it’s like really like, like the way I work with, you know, data analysis today, I wouldn’t put it... there’s, it’s not automating anything. I mean right, right now it’s very helpful. If it’s not... yeah. Back and forth constantly. Right. That’s not what we’re talking about. Yeah.

Seth: Okay, so if you’re calling 1x... we’re gonna call that delegating to an agent and I’m just gonna fix it up at the top. Versus Nx, I’m back and forth, you know, all day as not delegating to an agent. Then I guess I would think about this as... yeah, I’m still kind of in the, like maybe 5% of people will be bossing around agents as their main job. So that would put me in closer to the still, the 95% world. I don’t think this moves me that, that hard because I think that the stuff in this that gets automated will, will get automated, but then the knowledge economy workers will then spend more of their time on this, like kind of Nx iteration stuff. So,, I think that at the end of the day, if Nx iteration with the AI counts as “by hand,” I think we’ll have a lot, a lot of that. So yeah. Maybe only, so I would put that at 95% yeah. We’re still doing “by hand.”

Andrey: And maybe this goes to the taste thing, right? Like mm-hmm. You know,, maybe we should expect stronger results if we have very high inter-human, you know, agreement scores. But the fact that humans are disagreeing on work ethic, quality so much means that maybe as an individual, I have a specific style that I wanna convey in my work. Mm-hmm. And I have certain things I wanna see and don’t wanna see. And I’m gonna, you know, it’s gonna be maybe harder for me to specify that, although I’m not sure, you know, maybe, maybe the AI will know my style as well, so, yeah.

Seth: Right. Yeah, I mean, that’s, that seems to not be so far away,, to, you know, train a digital twin who will be able to attend those meetings for you, Andrey.

Andrey: Yeah. well, all right. So...

Seth: Even if your boss doesn’t want to... even if your boss doesn’t want you to, I got news for you. If I was a digital twin company, I’d tell my digital twins to encourage users to commit fraud by using me.

Andrey: Yeah. Yeah...

Seth: And then you locate, and then you locate internationally, everybody pays you in crypto. Just locate in the Cayman Islands. People buy your deep-fake software on the... this is, I mean, I don’t wanna give away my whole business model for free. People will have to tune in for the special episode on that.

Andrey: Yeah. Yeah. On that aside,, well,, thanks for joining us for another episode. and we look forward to your feedback and please,, boost our work or let us know what you’d like to see.

Seth: Yeah, let us know what you wanna see., we are your servants, our fans. Peace out dudes. Oh, keep your posteriors justified.

Andrey: True, true.

Relevant Links

• We Won’t Be Missed: Work and Growth in the AGI World — Pascual Restrepo (NBER TAI conference) and discussant commentary

• NBER Workshop Video: “We Won’t Be Missed” (Sept 19 2025)

• Marc Andreessen, Why Software Is Eating the World (WSJ 2011)

• Shapiro & Varian, Information Rules: A Strategic Guide to the Network Economy (HBR Press)

• Ecstasy: Understanding the Psychology of Joy — Find the sheep theory of the price level here: Seth’s Review

Priors and Posteriors

Claim 1 — After AGI, the labor share goes to zero (asymptotically)

• Seth’s prior: >90% chance of a large decline, <10% chance of literally hitting ~0% within 100 years.

• Seth’s posterior: Unchanged. Big decline likely; asymptotic zero still implausible in finite time.

• Andrey’s prior: Skeptical that asymptotic results tell us much about a 100-year horizon.

• Andrey’s posterior: Unchanged. Finite-time dynamics dominate.

• Summary: Compute automates bottlenecks, but socially or physically constrained “accessory” human work probably keeps labor share above zero for centuries.

Claim 2 — Real wages 100 years after AGI will be higher than today

• Seth’s prior: 70% chance real wages rise within a century of AGI.

• Seth’s posterior: 71% (a tiny uptick).

• Andrey’s prior: Agnostic; depends on transition path.

• Andrey’s posterior: Still agnostic.

• Summary: If compute accumulation drives growth and humans still trade on preference-based or ritual tasks, real wages could rise even as labor’s income share collapses.

Keep your Apollonian separate from your Dionysian—and your accessory work bottlenecked.

Timestamps:

[00:01:47] NBER Economics of Transformative AI Conference

[00:04:21] Pascual Restrepo’s paper on automation and AGI

[00:05:28] Will labor share go to zero after AGI?

[00:43:52] Conclusions and updating posteriors

[00:48:24] Second claim: Will wages go down after AGI?

[00:50:00] The sheep theory of monetary policy

Transcript

[00:00:00] Seth: Welcome everyone to the Justified Posteriors Podcast, where we read technology and economics papers and get persuaded by them so you don’t have to.

Welcome to the Justified Posteriors Podcast, the podcast that updates its priors about the economics of AI and technology. I’m Seth Benzell performing bottleneck tasks every day in the sense that I’m holding a bottle and a baby by the neck down in Chapman University in sunny, Southern California.

[00:00:40] Andrey: I’m Andrey Fradkin, practicing my accessory tasks even before the AGI comes coming to you from San Francisco, California.

So Seth, great

[00:00:53] Seth: to be. Yeah, please.

[00:00:54] Andrey: Well, what are you, what have you been thinking about recently? What have been, [00:01:00] contemplating.

[00:01:01] Seth: Well, you know, having a baby gets you to think a lot about, what’s really important in life and what kind of future are we leaving to him, you know, if we might imagine a hundred years from now, what is the economy that he’s gonna have when he’s retired?

Who even knows what such a future would look like? And a lot of economists are asking this question and there was this really kind of cool conference that put together some of the favorite friends of the show. An NBER Economics of Transformative AI Conference that forced participants to accept the premise that AGI is invented.

Okay, go do economics of that. And Andrey, I hear that somehow you were able to get the inside scoop.

[00:01:47] Andrey: Yes. Um, it was a pleasure to contribute a paper with some co-authors to the conference and to attend. It was really fun to [00:02:00] just hear how people are, um, thinking about these things, people who oftentimes I associate with being very kind of serious, empirical, rigorous people kind of thinking pie in the sky thoughts about transformative AI.

So, yeah, it was a lot of fun. Um, and there were a lot of interesting papers.

[00:02:22] Seth: Go ahead. Wait. No, before I want, I’m not gonna let you off the hook Andrey. Yeah, because I have to say, just before we started the show, you did not present all of the conversation at the seminars as a hundred percent fun as enlightening, but rather you found some of the debate a little bit frustrating.

Why? Why is that?

[00:02:39] Andrey: Well, I mean, I, you know, dear listeners, I hope we don’t fall guilty of this, but I do find a lot of AI conversation to be a little cliche and hackneyed at this point. Right. It’s kind of surprising how little [00:03:00] new stuff can be said. If you’ve read some science fiction books, you kind of know the potential outcomes.

Um, and so, you know, it’s a question of what we as a community of economists can offer that’s useful or new. And I do think we can, it’s just, it’s very easy to fall into these cliches or well trodden paths.

[00:03:20] Seth: What? What’s the meaning of life? Andrey? Will life have meaning after the robot takes my job? Will my AI girlfriend really fulfill me?

Why do we think economists would be good at answering those questions?

[00:03:34] Andrey: Yeah, it’s a great question, Seth. I’m not sure. Um,

[00:03:39] Seth: I think it’s because they’re the last respected kind of technocrat. Obviously all technocrats are hated, but if anybody’s allowed to have an opinion about whether your anime cat girl waifu AI companion is truly fulfilling.

We’re the only, we’re the only source of remaining authority.

[00:03:57] Andrey: Well, you know,

[00:03:57] Seth: unfortunately,

[00:03:58] Andrey: I think it’s a [00:04:00] common thing to speculate as to which profession will be automated last, and certainly Marc Andreessen believes that it is venture capitalist. So

[00:04:11] Seth: Fair enough. I’ll narcissism, I’ll leave

[00:04:13] Andrey: it as an exercise to the listener what economists think.

[00:04:21] Seth: So let’s talk about, so we’re, we’re at, we’re talking about whether humans will be essential in the long run because the particular paper that struck my eye when I was looking at the list of seminars topics was a paper by friend of the show, I hope he considers us a friend of the show because I love this guy.

Pascual Restrepo, a professor of economics and AI at Yale University. Um, had the honor of having this guy on my dissertation committee was definitely a role model when I was a young gun, trying to think about macro of AI before everyone on earth was thinking about macro of AI. [00:05:00] Um. And so it’s a real honor for the show to take on one of his papers and he’s got something that’s trying to respond to.

Okay. Transformative AI shows up. What are the long-term dynamics of that? Which is a departure from where he wants to be. He wants to live in near future. We automate another 10% of tasks land. Right. So I was excited to take this on. Um, Andrey, do you wanna maybe, introduce some of the questions it asks us to consider?

[00:05:28] Andrey: Yeah. So, Pascual presents a very stylized model of the macro economy and we picked two claims from the paper to think about in terms of our priors. Um, the first one of these is, um, after we get AGI in the limit, the labor share will go to zero. That is the first claim of this paper. Um, what do you think about that, Seth?

[00:05:59] Seth: Great question. [00:06:00] Um, so to remind listeners, so the labor share is if you imagine all of the payments in the economy, some are going to workers and then some are going to people who own the machines or own the AI, right? So today about two thirds of the money or about 60% of the money is paid to workers.

About 40% is paid to machines and out to profits and people who own stuff. It is a claim of this paper and a kind of a lot, a theme of a lot of the automation literature that as you get more and more automation, you’d expect the share of monies that are being paid to workers to go down, right? Because just more of the economy is just automation unconstrained by.

Um, let me tell you how I think about this question, Andrey. First of all, you know, we’re not gonna talk about out to infinity. I know these are asymptotic papers, but let’s try to stay a little bit closer. Um, so I’ll, I’ll mostly be thinking about like a hundred years after [00:07:00] AGI, right? So we have AGI, and now we’re, we’ve played it out in some sense.

We’ve had the next industrial revolution that happens from AGI, right? Assuming we don’t have an apocalypse, so this is, let’s set aside, conditional on, you know, we don’t destroy ourselves, which I don’t think there’s a huge chance of that, but that’s another question. I would say there’s a greater than 90% chance of very large decreases in labor share, you know, down from 60% today to 5%, 10%, 20%.

I really do see that. But I think there’s like a less than a 10% chance that within a hundred years of AGI, um, we’ll have, you know, literally 0% labor share or whatever, like less than 1% labor share. Why do I say that? This is something that’s gonna come up. I’m gonna start by just kind of questioning the premise of whether AGI really means.

That all services can be provided by the AI, right? I know. I don’t know if this [00:08:00] counts as being allowed. I’m gonna give you a fun example. Andrey. Have you ever heard of a pidyon haben?

[00:08:05] Andrey: No.

[00:08:06] Seth: You’ve never heard of a pidyon haben? Well, this is a tradition in Deuteronomy. It’s one of the few halakhic laws that actually make intuitive sense to me because it’s revenue-generating. When you have a firstborn son who is not a kohen or a Levi, you “buy” the baby out of service to the Temple. The cost is exactly five silver pieces (shekels) of a specified weight—they’re very specific about the weight; it’s not just any five silver coins. And here’s the thing: it has to be paid to a kohen (a member of the priestly family of Jews). Minor correction for Justified Posteriors fans: the pidyon haben is paid to a kohen, not a Levi. I couldn’t let that error stand. Thank you.

So that economic interaction is value that, by definition, can’t be captured by the AI. In some sense that’s a greater-than-zero slice of the economy, asymptotically—well, I guess it depends on whether silver is rare asymptotically. But that’s the kind of example I have in mind, and it’s why I don’t think the labor share gets literally to zero. Andrey, gimme your thoughts.

[00:09:31] Andrey: Yeah, I mean, look, zero is an asymptotic result, so I do think, let’s say less than 1% in a hundred years. With your example, I think it’s very easy to imagine a virtual kohen to collect said revenue. So I actually—no, let’s—

[00:09:53] Seth: Think about the political economy of it for a second. Who gets to decide whether it counts if you send it to the robot?

[00:10:00] Well, the rabbi. The human rabbi decides.

[00:10:02] Andrey: The human rabbi might be a capital owner, but the—

[00:10:05] Seth: The human rabbi may—that’s the danger.

[00:10:09] Andrey: Yeah. Rabbis—I mean, I can think of things. Your point is that some occupations may require a human involved, right.

[00:10:25] Seth: And they may be some sort of fraction of the economy asymptotically. They’re not linear additive, because that’s a distinction that’s going to become important.

[00:10:33] Andrey: Yeah, later. So, I think about part of this as being about population growth, and that’s a good point. Because if one of the things that AI does is increase the number of humans, and there’s some sort of human scaling law, if you will—that AGI can “make” humans very cheaply and quickly, I assume—then I think that’s one thing to think about. And then I think the other possibility, and this is not talked about in this paper, is: there are certain things where you can throw as much compute as possible and you still get returns—like exploring outer space—but there might be a difference in how much humans value that versus how much AIs value that.

[00:11:40] Seth: That’s a super good question that is not raised. I think I was trying to read this paper as “we only care about human utility,” but that’s obviously not unimportant here.

[00:11:50] Andrey: Yeah. Nonetheless, a hundred years is getting to the point where a lot can happen, but I’d still—as a betting man—say it’s pretty unlikely in a hundred years that waged labor will be less than 1%.

[00:12:08] Seth: Yeah, we’ll probably destroy enough capital along the way that we will get back to that asymptote.

[00:12:12] Andrey: Yeah. So that’s kind of my part. The second is where I think it’s a contentious claim: wages won’t go down in the long run because people can always break away. And that’s the argument in the paper. So let’s just focus on the first part of that, which is “wages can’t go down in the long run with AGI.” And what we mean here is not wages as a percentage of earnings, but real wages.

[00:12:47] Seth: Precisely.

[00:12:47] Andrey: Yeah.

[00:12:48] Seth: This seems to me like a naïve simplification of the model, which is what gives us that. It seems to me if you’re going to be so expansive as to take the stance that even my kohen example won’t hold up in the long run and it really is going to do every single job, you have to imagine some sort of crowding out of resources that are necessary for human labor to get anything done effectively. Right? This is a model that kind of very naïvely says that there’s always the— you know, the forest where there’s “good enough,” and the Lockean cliché, right? Anyone can go to the forest and take some wood and make a knife—therefore property rights, whatever. That cliché is in the back here. But of course, if you had a super-duper powerful AI, they might need that wood first. They’re going to use up all the resources. There’ll be no starting tinder for the humans to get started with. And then that will effectively drive down wages. So I don’t— I think to the extent that we get an AGI that is driving down labor share, what has to save the day is that there is some essential thing—call it a bottleneck—that only humans can do. What is the percentage chance that we get saved by one of those to keep wages up? Do I think it’s closer to—

[00:14:20] Andrey: Now are you talking about asymptotia or a hundred years?

[00:14:23] Seth: I’m talking about a hundred years.

[00:14:25] Andrey: See, this is where I’m a little confused, Seth. In asymptotia I kind of agree with you, but in the hundred-year horizon—especially since you think that wages are going to still be around—I would think that the cumulative wage would be higher than we have now.

[00:14:45] Seth: I’m saying 30% chance of this. I’m trying to make those two predictions format.

[00:14:50] Andrey: Which one is this, just to be clear?

[00:14:52] Seth: Great. I think there is a 30% chance that wages will go down. So I think there’s a 70% chance that wages will go up.

[00:15:03] Andrey: On average, as a result of AGI. So real wages per capita globally—just to be accountable—70%.

[00:15:10] Seth: This is my hundred-year prediction. A hundred years from now—dig me up—a hundred years after AGI, the real average wage will be higher than today. I’m good with that.

[00:15:25] Andrey: I would say it’s more like 80%.

[00:15:28] Seth: 80%. Okay. So you’re more—well, maybe we can talk at the end about why we start and end up at slightly different places. You ready to get into the model?

[00:15:47] Seth: We heard our priors. Now we confront the evidence. Do, do, do, do. Okay. So Pascual’s got a pretty straightforward model for us. The two premises he wants to start with are: first, the idea that we’re going to invent “robots,” which he means by “compute”—the accumulation of more AI compute over time. So literally chips and energy, I would say. But then he clarifies that this also includes any sort of physical instantiation of capital needed to move things in the physical world. So what he calls compute, I would think is more usefully thought of as robots. It’s going to do anything you need it to. The idea is that asymptotically we are going to invent robots that can do anything—any work that can be valuable in the economy. But he’s going to allow for the possibility that there’s some sort of comparative-advantage trade relationship with humans. We’ll come back to that. And then the second asymptotic here is the idea that the stock of robots and compute is going to grow indefinitely. So we’re thinking about the indefinite future: we have more robots than you possibly know what to do with. If you want your sci-fi comparison, this could be Isaac Asimov’s “Naked Sun,” where there are 50 people on a planet, each of whom owns a continent-sized estate and has vast swaths of robot servants. Maybe that’s what you should be thinking of as this asymptotic economy. From that, and just the assumption that economic output is the sum (in a complicated way) of all of these different jobs that could be done, he then distinguishes between two kinds of work in the economy: bottleneck work and accessory work, which I think is the most interesting novel distinction introduced here. Before I get into that, anything I missed from the model you want to throw in there, Andrey?

[00:18:09] Andrey: Did you mention the constant returns to scale?

[00:18:14] Seth: Go ahead and say it. Yeah, also there are constant returns to scale.

[00:18:16] Andrey: There are constant returns to scale. There is no real capital to speak of other than compute.

[00:18:24] Seth: Ownership—yeah, this is just the production side model. There’s no “where do these dynamics come from?” Maybe there’s a social planner deciding some of this, but 90% of the paper is not going to take a stance on the consumer/household side of the economy.

[00:18:53] Andrey: Yeah. And the other thing is that he uses the term “bottleneck,” but that is a very confusing word, so it’s best not to—okay, let’s get it right now—it’s best not to use it, actually. One of the key comments at the conference was to rename that word.

[00:19:09] Seth: Let’s talk, because I like it. I think you guys are being mean to Pascual for no reason. Pascual, if you’re ever in trouble, I’m going to tell you why. There is a concept I use all the time for thinking about long-run macro dynamics: when we combine automated and non-automated things, are they gross complements or gross substitutes? In a CES production function, my understanding is that the concept of bottleneck work would correspond to anything that is Cobb-Douglas or more complementary in the asymptote, and anything that’s accessory work would be more grossly substitutable than Douglas. That’s how it would work for CES production functions.

[00:20:14] Andrey: I’ll take your word for it, Seth.

[00:20:18] Seth: Well, let me give you an intuition. In one extreme, we have perfect complements: if humans are peanut butter and AI is jelly, clearly the humans are a bottleneck there. Then we have the perfect substitutes extreme: if humans are margarine and AI is butter, great—there’s more spread out there; they’re not hurting each other. Those are the two extremes. There’s a continuum between them. In a CES production function it’s clear. The underlying concept is more general: in the limit, is this a bottleneck? In the limit, is this a substitute? Maybe you don’t love this language, but there should be words for “in the limit is this a gross substitute?” vs. “in the limit is this a gross complement?” I think these are the words I’ve been looking for. Why didn’t they like it?

[00:21:23] Andrey: I think because Pascual’s example was that the AI will be out there exploring space, and people all conference long, when they use the word “bottleneck,” are thinking about current production processes where there might be bottlenecks because it’s a part AI can’t do end-to-end. So when you’re talking about bottlenecks, it’s really like, “here’s this little thing that we need a human to do in this process—like give the AI the bank account number,” or whatever. That’s a very different type of task.

[00:21:59] Seth: I’m coming at it from the consumption side and they’re coming at it from the production side. I think I’m much more on Pascual’s side. I think he’s being held back by the smooth brains at the conference.

[00:22:10] Andrey: I just don’t think any normal human being, when they think about the word “bottleneck” and tasks, is thinking about AI exploring space.

[00:22:28] Seth: His example is terrible. But he’s a beloved weirdo; that’s why he’s a friend of the show.

[00:22:35] Andrey: I’m not attacking him. I’m saying this word is not the right one. In his model, if we do have near-infinite compute ability, we will do cool stuff—like we recreate our own version of the Matrix with cell-level simulation of the entire world. Is that a bottleneck? It’s not a bottleneck. We can do all sorts of very large-scale things—at least AI can do it.

[00:23:15] Seth: Very interesting. I can see why you don’t like the word. There needs to be a word for the concepts I described. So anyway, I like these two concepts: in the limit, do you need humans to get more output, or in the limit do you not? Those are the concepts. Are you ready to proceed to his results?

[00:23:39] Andrey: I actually wanted to question you on that last one.

[00:23:41] Seth: Please.

[00:23:46] Andrey: “In the limit, do you need humans or not?” is not actually the definition in this paper.

[00:23:49] Seth: Let me think for a second.

[00:23:49] Andrey: The task, not the human.

[00:23:49] Seth: No, the human was the example. I’m sorry if that was confusing. The question is: in the limit, do you need the task or not? That is the question in the paper.

[00:24:00] Andrey: I view it as a satiation sort of thing. There are only so many live music performances the world needs, if that’s what we think humans are going to be doing. Other things—the universe is pretty large, maybe not infinite—so there’s lots to explore, and that doesn’t get satiated.

[00:24:24] Seth: I don’t see how satiation comes in.

[00:24:26] Andrey: Because one of the conditions is about the derivative of the production function.

[00:24:35] Seth: Right. So if you became satiated on an input, of course it couldn’t be a bottleneck task. Of course. Satiation would be one mechanism for not being a bottleneck. Good. Last comments before we get to the results?

[00:24:56] Andrey: No, go for it.

[00:25:00] Seth: Prop 1: All bottlenecks are eventually automated while some accessory work may be left to labor. Okay, what’s the intuition here?

[00:25:06] Andrey: The intuition is opportunity cost. If compute is being used for this task, that means it’s not being used for some other task that maybe has a higher return or humans can’t do. As a result, humans are going to be left doing some kind of low-value work because the compute is better used elsewhere.

[00:25:39] Seth: Right, but now it’s a claim about what that low-value work will be. It’s got to be the thing the AI won’t always need to make more of. If there’s anything that’s going to hold back the AI, it’s going to do more of it, because this is super-AI.

[00:25:58] Andrey: Like creating more compute, for example.

[00:26:01] Seth: Yeah, they’re not going to let the humans be in charge of that. Don’t worry. So what’s left is this concept in the paper—we can discuss how realistic it is—where humans can go to the woods and do their constant-returns-to-scale task with each other, and maybe even have a parallel economy, or maybe it’s just the cherry-on-top economy.

[00:26:26] Andrey: Yeah, so now we’re getting to the argument for why real wages won’t go down though. That’s what you’re saying.

[00:26:37] Seth: “While some accessory work may be left to labor”—I was explaining the second half of that sentence.

[00:26:42] Andrey: I think you’re mixing two concepts. Some accessory work would be left to labor is one claim. A different claim is that wages can’t go down because essentially, in his model, all the humans can say “screw this AI, we’re going to recreate our own economy,” and the AIs won’t care. So they’ll be able to do just as well as in a world without AGI. That, to me, is a ridiculous argument, but it’s also different from the argument for the fact that there are accessory jobs.

[00:27:26] Seth: Why is it interesting that there are accessory jobs? In my interpretation, there is an outside option providing a floor on wages that happens to be an accessory job.

[00:27:43] Andrey: I don’t agree. The accessory job is not providing the minimum wage. Without accessory jobs there are no wages. I don’t understand how it could be providing a minimum wage when without accessory jobs humans don’t do anything.

[00:28:16] Seth: No, that’s not this model. There is the special case where there are no accessory jobs. What they do then is a really lousy complement—they do the most human-comparative-advantage, human-complementary job.

[00:28:29] Andrey: I don’t think such a job would exist. I’d be shocked if, for anything that’s truly scalable, humans in the loop could even be positive.

[00:28:44] Seth: Let me think about that for a second. So you don’t like that special case where all tasks are ultimately bottlenecks for each other?

[00:28:52] Andrey: Yeah. What is a human going to do at an automated GPU factory, exactly? They’re going to need to be fed. I don’t see how humans could be net positive in those types of production processes.

[00:29:17] Seth: What I want to point out one more time is you’re coming at “bottleneck” from the production side, and I’m coming at it from the consumption side. One more note to Pascual to maybe think about in the next draft.

[00:29:30] Andrey: Want to skip to Proposition 3?

[00:29:33] Seth: No, we haven’t finished talking about these propositions. Just to be clear, accessory jobs are the reason humans have substantial wages at all.

[00:29:56] Andrey: That’s a different claim.

[00:30:00] Seth: The two claims have to be compatible.

[00:30:09] Andrey: Sure. I thought we’d talk about the plausibility of the model’s implications for those claims separately.

[00:30:21] Seth: I find myself unconstrained by this ordering of concepts, but happy to comply.

[00:30:28] Andrey: Go ahead. What were you going to say?

[00:30:30] Seth: In my mental model of this model: there is a special case where there is no accessory work—everything is ultimately a bottleneck for everything else. That is a special case. And then he also says that in all versions of this model, as I understand it, wages can’t go down. Those cannot both be true and it also be the case that the only thing that keeps wages from going down is the existence of accessory jobs.

[00:31:09] Andrey: I think we’re also mixing “what is in his model” versus “what are the economic forces,” which is always hard because it’s so stylized.

[00:31:26] Seth: Fair.

[00:31:26] Andrey: The interesting economic content of the model is that there are accessory jobs allowing humans to persist in having some positive labor contributions that are not taken up by the machines. Why aren’t the machines doing it? Because the machines have better things to do.

[00:31:54] Seth: One way to think about it: if you have automation and there’s perfect substitution, it kind of doesn’t affect your life. Suppose we sell oil and I’m a whaler who collects whale oil. My friend invents oil wells and gets a hundred times the amount of oil I have. In an economy where there’s only oil: that guy got a lot of oil—good for him—I still have my whale oil. In an economy where oil’s a complement to everything else, I’m ruined because now the price has collapsed.

[00:32:39] Andrey: Now let’s go to the claim that in such a world, wages can’t go down.

[00:32:51] Seth: In a world where there’s only one thing—or rather, where the things are substitutes—wages can’t go down. That’s the connection between an accessory task and a gross substitute. If your oil is good and my oil is good, and we can both enjoy each other’s corn—if you get more corn, that doesn’t affect my corn. So my wage can’t go down. I can talk about why that would break, but that’s why it happens in this model.

[00:33:27] Andrey: Any model here where there’s perfect alignment of what humans want and what the machines want—you’re producing more, and it’s going to go to humans. It’s almost a reductio that, in such a model, real wages have to go up.

[00:33:56] Seth: This is almost like a Pareto model: good things have to happen in a Pareto model.

[00:34:02] Andrey: If there’s a social planner, the planner is maximizing utility, and the utility is human utility, not machine utility.

[00:34:14] Seth: It’s like “the guy who got free stuff has more income” theorem.

[00:34:19] Andrey: Right. So I think it’s strange to think about this, because no one is seriously worried about the situation where we’re infinitely wealthy and have perfect control of our AI.

[00:34:44] Seth: Okay, so what’s the work the model is doing? It’s trying to tell us that that’s the case where there isn’t good accessory work, maybe. The sad case is where there’s a negative externality of whatever the AI is accumulating on our wages. How could that work? What’s not modeled here? There’s no sense in which AI can crowd out investment in capital that complements humans. What this model excludes is the idea that when I build a robot, I might not be building a computer for a human to use. That’s why wages go down: no one invests in making humans productive because it’s better to invest in making AI productive.

[00:35:25] Andrey: I’m not even sure that’s enough. If ultimately some part of the AI production chain is kicking back things that humans like, I’d be more worried that if the AIs have transcended humanity and all resources must be used to explore space, we might find ourselves without a planet Earth because all the resources will be extracted.

[00:35:57] Seth: Pascual did not do this model any favors in his presentation, I could tell.

[00:36:06] Seth: I think this is happening today. Will you guys listen to our “Canaries in the Coal Mine” paper? You could argue that today AI is leading to reduced investment into some kinds of young people’s human capital. That plus humans’ human capital eventually being replaceable is the kind of thing that would drive down wages in the absence of an accessory job to fall into. We can talk about what that would be—like providing mental-health services to each other in a linear way.

[00:36:56] Andrey: There’s still a distinction between a world where human labor is close to worthless and a world where humans are materially worse off. If the AI is perfectly aligned, humans don’t do any work, but they get all the goods; they can own it; they get capital income.

[00:37:19] Seth: 0% labor income equals 100% capital income.

[00:37:23] Andrey: Yeah. I feel like it’s really important to have that as a force in the model.

[00:37:31] Seth: So what’s the fantasy—the utopian fantasy? This is Bostrom; this is The Culture. You are doted on by robots that do every possible thing a human could do—except five silver coins for pidyon haben. That economy is what we’re describing, where I could have more robots, but maybe I’m saturated with robots. Maybe I have linear returns to robots; I’m just building exponentially more robots.

[00:38:00] Andrey: I think about accessory work as more addressing the meaning aspect. There’s a sweet spot, if there is accessory work, where humans are the doers of it and they find meaning.

[00:38:16] Seth: If they’re the doers of it—well, isn’t that a complement, then?

[00:38:20] Andrey: The examples he provides are musicians; I imagine that could provide a lot.

[00:38:27] Seth: Musicians make sense, because there could be some linearity to it.

[00:38:32] Andrey: We’re all going to be creating art for each other, and we’re going to value human-made art, and the AIs are going to explore the universe and create cancer cures.

[00:38:43] Seth: And then give us money.

[00:38:45] Andrey: And give us whatever—and we will have the Star Trek machine where we get any material good that we need.

[00:38:54] Seth: Okay, good. I’m looking forward to it.

[00:38:56] Andrey: Yeah.

[00:38:56] Seth: How are we doing on time? How many more props do we want to do? We want to do Prop 3. This is my hobby horse; give me a little time on Prop 3.

[00:39:06] Andrey: Sure. Let’s do that.

[00:39:08] Seth: One of the results of this paper is that asymptotically we have an AK growth model. What does that mean? It says that if you are able to automate all tasks, the economy’s growth rate will grow with the accumulation of more capital. That makes sense: robots can do everything; the output of the economy is how many robots you have and how good they are at being robots—plus a productivity term. That is true of this model. What that means is the long-term growth rate of the economy is the national saving and reinvestment rate—it’s the rate at which we compound today’s compute into tomorrow’s compute. There’s a technological aspect, but it’s also a social decision. I will never stop getting onto this chair and waving my flag: if you care about a future of automation, you should care about the national saving rate, because that is the growth rate in the world with automation. Andrey, were you pleased to see this prediction?

[00:40:16] Andrey: I think it makes sense. In these types of models it has to be true. We’ve all played Factorio—it’s not a surprise.

[00:40:34] Seth: It’s just basic Factorio-nomics. Okay, one last proposition, a variation on that. He’s starting to think about dynamics. He has some things to say about what’s happening in the dynamic model, but he points out: if you can use your compute to make AI more productive in a within-period decreasing-returns-to-scale manner, then basically the growth rate is the compute accumulation rate times a constant factor. Basically this form of science reinforcing the AI is not enough to get a regime change in the growth rate. It gives you a little boost. I thought that was cool.

[00:41:23] Andrey: It’s nice for the model not to explode.

[00:41:27] Seth: Did he get panned for that? A lot of people like models that explode. Jones has a model that explodes.

[00:41:33] Andrey: I don’t think people were concerned about finite-time explosion. They were concerned with the bottlenecks.

[00:41:40] Seth: I’m going to make a Yudkowsky-ish point. One of the main reasons that, upon reflective equilibrium, I’m not super worried about the doomer scenarios is that in my brain power has a connection to GDP, and in all of these models GDP has to grow in this regular exponential way—which is fast, but it’s not “today to tomorrow” fast. Based on how we think that works, the idea that we would get an algorithmic explosion where power explodes overnight seems out of sample.

[00:42:21] Andrey: I mean, we have no idea. It could still be—

[00:42:34] Seth: The saving rate—

[00:42:35] Andrey: We don’t know how much the AGI would choose to reinvest into its own growth. We just don’t. So I don’t think, in the transition dynamics, this is a very plausible argument. Nothing you just said prevents an AI from starting an automated AI factory and tripling itself over the course of a week.

[00:43:04] Seth: Yeah—exponential growth with an exponential rate determined by its reinvestment rate.

[00:43:09] Andrey: I don’t find that comforting. That exponential rate could be really fast.

[00:43:15] Seth: I’m saying there are models where we go from zero to infinity in finite time.

[00:43:22] Andrey: Sure.

[00:43:24] Seth: In any finite amount of time it’s still going to be one huge number and another huge number, and that gives me very little comfort personally.

[00:43:34] Seth: Okay. Viewers at home, tell us: how much scarier is an asymptote to infinity than an exponential? We’ll get those votes and report ’em next week.

[00:43:52] Andrey: It could be exponential to, you know, very—it could be a really big exponential. Big exponential.

[00:43:52] Seth: Let’s move to our conclusions and posteriors. Do you have any overall points you want to make about the paper before we move into posteriors?

[00:44:04] Andrey: It’s a fun thought exercise. I enjoy thinking about it.

[00:44:09] Seth: At a stylistic level, I really prefer the way that Pascual writes these to the way that Ben Jones and even Daron Acemoglu write these. I found the stripped-downness and the lack of rhetorical pretense in this draft really refreshing, and sensible given his comparative advantage. What’s not in here: I got on my high horse to say “saving rate important.” I think the idea that there isn’t some fixed other thing getting used up that could drive down human wages is an obvious omission that is not relevant to AI today. It seems like you’re modeling AI a thousand years from now, so at least nest what’s happening today. But it’s an elegant way of providing some fundamental points that I think are true of a lot of models, in language that I think is useful. So I liked this theory paper, even though I don’t think it’s going to move my priors that much.

[00:45:22] Andrey: I think I’m in the same boat.

[00:45:32] Seth: Moving to our posteriors. Our first question was: after we get AGI, asymptotically the labor share will go to zero. I said greater than 90% chance for large decreases of labor share, less than 10% chance of going to super-duper small—like less than 1%—within a hundred years. Am I moved here? We raised ideas in either direction that would mitigate. On the one hand, there might be some essential human bottleneck you can never automate. On the other hand, many kinds of human productivity require investment into humans—physical or human capital—that might get diverted to AI. Therefore wages could go down in an accelerated way to zero. I do not see these as contradictions along the path. But for the asymptotics, that’s a prior thing, so on this particular question I didn’t move.

[00:46:51] Andrey: I don’t know if I moved very much either. The tricky thing is infinite results vs. finite-time predictions. A hundred years is a long time, but it’s also not infinity. It’s hard for me.

[00:47:17] Seth: You might imagine a long tail—something we were riffing on before the show. Maybe first we automate 90% of jobs, then 95%, then 97%, and that asymptotic tail is still important and complementary and bottleneck-y enough a hundred years from now that there’s a big labor share because that’s the one last essential job.

[00:47:42] Andrey: Yeah. And once again—if there are jobs where humans demand that other humans do them, the only way compute can do them is to trick the human into thinking it’s a human doing it when it’s really an AI. That’s possible, but we’re getting into some pretty ridiculous—

[00:48:04] Seth: We should have a test for that. Some sort of Andrey test—or maybe a Turing test. All right, second past prior—let’s posteriorize it. We have to justify that wages won’t go down in the long run because people can always break away and recreate the economy—do their own accessory work thing.

[00:48:23] Andrey: Yeah.

[00:48:24] Seth: I said: wages won’t go down after AGI. After a hundred years, I would say real wages are higher a hundred years after AGI—70%. Did I move because of this paper? Maybe this moves me down 1% to 69%, based on a conference full of people accepting that premise.

[00:48:52] Andrey: Just to be clear, this paper is arguing for wages not going down, so why are you going down?

[00:48:56] Seth: 71%. I said 70% they go up; I’m going down to 69% they go up.

[00:49:04] Andrey: I see. I view “equal” as a knife-edge case—it’s measure zero. So you shouldn’t adjust at all.

[00:49:16] Seth: No, actually—dude—oh my God. All right, I’ll let us go out on this joke. I read a book that had the most hilarious theory of monetary policy the other day. It was in our book club that Andrey is in, where we read weird philosophy texts. Let me find it. It was in the book Ecstasy, which is a book about having fun, I guess, by a Freudian analyst. And in it he offers the following theory of the price level. So, on page 45, in his discussion of Dionysus as the scapegoat, the author writes: “Sheep represent everything of value in our Judeo-Christian world. The sheep, in fact, is the chief determinant of our currency. Every currency in the Western world—the shilling, the franc, the Deutsche mark, the lira, the peso, the Austrian thaler, from which we got our dollar—was the price of one sheep. For centuries there was no inflation in the Western world because one of our money pieces was worth a sheep. You could count on that anywhere, anytime.” Wow. Someday I hope to write economics as good as that, Andrey.

[00:50:47] Andrey: Hallucinations. I feel like the AIs are unfairly maligned when humans are very good at it.

[00:51:00] Seth: He was sent a vision of the synthesis of economic policy. This is why you’ve got to keep your Apollonian and your Dionysian separate out there, guys. So let’s leave it on that note. Keep your Apollonian separated from your Dionysian, and keep your accessory work bottlenecked.

[00:51:15] Andrey: Inshallah.

[00:51:17] Seth: Oh wait. No, before we go, I apologize to all of my guests for anything bad I did to them over the last year!

1. The printing press unlocked the Enlightenment — along with both liberal democracy and France’s Reign of Terror

2. Communism is primitive socialism plus electricity

3. The radio was an essential prerequisite for fascism

4. AI will unlock ????

We read “AI as Governance” by Henry Farrell in order to understand whether and how political scientists are thinking about this question.

• Concepts or other books discussed:

  • E. Glen Weyl, coauthor of Radical Markets: Uprooting Capitalism and Democracy for a Just Society, and key figure in the Plurality Institute was brought up by Seth as an example of an economist-political science crossover figure who is thinking about using technology to radically reform markets and governance.

  • Cybernetics: This is a “science” that studies human-technological systems from an engineering perspective. Historically, it has been implicated in some fantastic social mistakes, such as China’s one-child policy.

  • Arrow’s Impossibility Theorem: The economic result that society may not have rational preferences — if true, “satisfying social preferences” may not be a possible goal to maximize

  • GovAI - Centre for the Governance of AI

  • Papers on how much people/communication is already being distorted by AI:

    • Previous episode mentioned in the context of AI for social control:

  • Simulacra and Simulation (Baudrillard): Baudrillard (to the extent that any particular view can be attributed to someone so anti-reality) believed that society lives in “Simulacra”. That is, artificially, technologically or socially constructed realities that may have some pretense of connection to ultimate reality (i.e. a simulation) but are in fact completely untethered fantasy worlds at the whim of techno-capitalist power. A Keynesian economic model might be a simulation, whereas Dwarf Fortress is a simulacra (a simulation of something that never existed). Whenever Justified Posteriors hears “governance as simulation”, it thinks: simulation or simulacra?

    

Episode Timestamps

[00:00:00] Introductions and the hosts’ backgrounds in political science.

[00:04:45] Introduction of the core essay: Henry Farrell’s “AI as Governance.”

[00:05:30] Stating our Priors on AI as Governance

[00:15:30] Defining Governance (Information processing and social coordination).

[00:19:45] Governance as “Lossy Simulations” (Markets, Democracy, Bureaucracy).

[00:25:30] AI as a tool for Democratic Consensus and Preference Extraction.

[00:28:45] The debate on Algorithmic Bias and cultural bias in LLMs.

[00:33:00] AI as a Cultural Technology and the political battles over information.

[00:39:45] Low-cost signaling and the degradation of communication (AI-generated resumes).

[00:43:00] Speculation on automated Cultural Battles (AI vs. AI).

[00:51:30] Justifying Posteriors: Updating beliefs on the need for a new political science.

Listen in to hear more about what our AI models are learning from naughty books, and how Seth and Andrey think that should inform AI property rights moving forward.

Also mentioned in the podcast are:

• Joshua Gans paper on AI property rights “Copyright Policy Options for Generative Artificial Intelligence” accepted at the Journal of Law and Economics:

• Fair Use

• The Anthropic lawsuit discussed in the podcast about illegal use of books has reached a tentative settlement after the podcast was recorded. The headline summary: “Anthropic, the developer of the Claude AI system, has agreed to a proposed $1.5 billion settlement to resolve a class-action lawsuit, in which authors and publishers alleged that Anthropic used pirated copies of books — sourced from online repositories such as Books3, LibGen, and Pirate Library Mirror — to train its Large Language Models (LLMs). Approximately 500,000 works are covered, with compensation set at approximately $3,000 per book. As part of the settlement, Anthropic has also agreed to destroy the unlawfully obtained files.”

• Our previous Scaling Law episode:

Related recent paper on AI and job postings: Generative AI as Seniority-Biased Technological Change: Evidence from U.S. Résumé and Job Posting Data

Also related to our discussion is the China Shock literature, which Nick Decker summarizes in his blog post:

The Ide & Talamas Model

Our discussion was based on the paper Artificial Intelligence in the Knowledge Economy by Enrique Ide and Eduard Talamas. It is a theoretical model of organizational design in the age of AI. Here’s the basic setup:

• The Setting: A knowledge economy where firms’ central job is solving a continuous stream of problems.

• The Players: We have Workers (human or AI) and a higher-level Solver (human manager/expert or AI). Crucially, the human players are vertically differentiated—they have different skill levels.

• The Workflow: It’s a two-step process: A worker gets the first shot at solving the problem. If they fail, the problem gets escalated up the hierarchy to the Solver for a second attempt.

• The Core Question: Given this hierarchy, what’s the most efficient organizational arrangement as AI gets smarter? Do we pair human workers with an AI manager, or go for the AI worker/human manager combo?

  • There are also possibilities not considered in the paper, such as chains of alternating managers and employees, something more network-y etc.

Key Debates & Critiques

Here are the most interesting points of agreement, disagreement, and analysis we wrestled with:

• Is a Solver Really a Manager? We spent a lot of time critiquing the paper’s terminology. The “manager” in this model is really an Expert who handles difficult exceptions. We argued that this role doesn’t capture the true human elements of management, like setting strategic direction, building team culture, or handling hiring/firing.

• My Desire vs. Societal Growth: Andrey confessed that while he personally wants an AI worker to handle all the tedious stuff (like coding and receipts), the economy might see better growth and reduced inequality from having AI experts and managers who can unlock new productivity at the highest levels.

• The Uber Driver Problem: We debate how to classify jobs like Uber driving. Is this already an example of AI managing the human (high-frequency algorithmic feedback), or is the driver still an entrepreneur who will manage their own fleet of smaller AI agents for administrative tasks?

Go Deeper

Check out the sources we discussed for a deeper dive:

• Main Paper: Artificial Intelligence and the Knowledge Economy (Ide & Talamas, JPE)

• Mentioned Research: Generative AI at Work (Brynjolfsson, Lee, & Raymond on AI in call centers)

Timestamps

• [00:00] Worker, Manager, or Expert?

• [00:06] Who manages the AI agents?

• [00:15] Will AI worsen inequality?

• [00:25] The Ide & Talamas model explained

• [00:40] Limitations and critiques

• [00:55] Posteriors: updated beliefs

The Bets: Priors & Predictions

We pinned down our initial beliefs on two key questions about the future impact of AI agents, the foundation of our “Justified Posteriors.”

Prediction 1: Will Managing AI Agents Become a Common Job? What percentage of U.S. workers will have “managing or creating teams of AI agents” as their main job within 5 years?

Prediction 2: Will LLM-based Agents Exacerbate Wage Polarization?

• Seth’s Prior: 25% chance it WILL exacerbate. Reasoning: Emerging evidence (like the call center study)

• Andre’s Prior: 55% chance it WILL exacerbate. Reasoning: Skeptical of short-term studies; believes historical technology trends favor high-skill workers who capture the largest gains.

Our Final Posteriors

Prediction 1: Will Managing AI Agents Become a Common Job?

The model slightly convinced Seth that the high-skill vertical differentiation story might be stronger than he initially believed, leading to a small increase in his posterior for exacerbation.

Host’s note, this episode was recorded in May 2025, and things have been rapidly evolving. Look for an update sometime soon.

Transcript

Seth: ​Welcome to Justified Posterior Podcast, the podcast that updates beliefs about the economics of AI and technology. I'm Seth Benzel, possessing a highly horizontally differentiated intelligence—not saying that's a good thing—coming to you from Chapman University in sunny Southern California.

Andrey: And I'm Andrey Fradkin, multi-homing across many different papers I'm working on, coming to you from sunny—in this case—Cambridge, Massachusetts.

Seth: Wow…. Rare, sunny day in Cambridge, Mass. But I guess the sunlight is kind of a theme for our talk today because we're going to try to shed some light on some surprising features of AI, some important features, and yet, not discussed at all. Why don't people write papers about the important part of AI? Andrey, what's this paper about?

Andrey: I agree that not enough work has been done on this very important topic. Look, we can think about the big macroeconomic implications of AI—that's really fun to talk about—but it's also fun to talk about the business of AI. Specifically, who's going to win out? Which models are better than others? And how can we measure these things as they're happening at the moment? And so that's really what this paper is about. It's trying to study how different model providers compete with each other.

Seth: Before we get deep into that—I do want to push back on the idea that this isn't macroeconomically important. I think understanding the kind of way that the industry structure for AI will work will have incredible macroeconomic implications, right? If only for diversity—for equality across countries, right? We might end up in a world where there's just one country or a pair of countries that dominate AI versus a world where the entire world is involved in the AI supply chain and plugging in valuable pieces, and those are two very different worlds.

Andrey: Yeah. So, you're speaking my book, Seth. Being an industrial organization economist, you know, we constantly have this belief that macroeconomists, by thinking so big-picture, are missing the important details about specific industries that are actually important for the macroeconomy.

Seth: I mean—not every specific industry; there's one or two specific industries that I would pay attention to.

Andrey: Have you heard of the cereal industry, Seth?

Seth: The cereal industry?

Andrey: It's important how mushy the cereal is.

Seth: Well, actually, believe it or not, I do have a breakfast cereal industry take that we will get to before the end of this podcast. So, viewers [and] listeners at home, you gotta stay to the end for the breakfast cereal AI economics take.

Andrey: Yeah. And listeners at home, the reason that I'm mentioning cereal is it's of course the favorite. It's the fruit fly of industrial organization for estimating demand specifically. So—a lot of papers have been written about estimating serial demand and other such things

Seth: Ah—I thought it was cars. I guess cars and cereal are the two things.

Andrey: Cars and cereal are the classic go-tos.

Introducing the paper

Seth: Amazing. So, what [REDACTED] wrote the paper we're reading today, Andrey?

Andrey: Well, you know—it was me, dear reader—I wrote the paper.

Seth: So we know who's responsible.

Andrey: All mistakes are my fault, but I should also mention that I wrote it in a week and it's all very much in progress. And so I hope to learn from this conversation, as we—let's say my priors are diffuse enough so that I can still update

Seth: Oh dude, I want you to have a solid prior so we can get at it. But I will say I was very, very inspired by this project, Andrey. I also want to follow in your footsteps. Well, maybe we'll talk about that at the end of the podcast as well. But maybe you can just tell us the title of your paper. Andrey,

Andrey: The title of the paper is Demand for LLMs, and now you're forcing me to remember the title of the—

Seth: If you were an AI, you would remember the title of the paper, maybe.

Andrey: The title of the paper is Demand for LLMs: Descriptive Evidence on Substitution Market Expansion and Multi-Homing. So, I will state three claims, which I do make in the paper.

Seth: Ooh, ooh.

Andrey: And you can tell me your priors.

Seth: Prior on each one. Okay, so give me the abstract; claim number one.

Andrey: So the point number one is that when a new good model gets released, it gets adopted very quickly. Within a few weeks, it achieves kind of a baseline level of adoption. So I think that's fact number one. And that's very interesting because not all industries have such quick adoption cycles.

Seth: Right? It looks more like the movie or the media industry, where you have a release and then boom, everybody flocks to it. That's the sense that I got before reading this paper. So I would put my probability on a new-hot new model coming out; everybody starts trying it—I mean, a lot of these websites just push you towards the new model anyway.

I know we're going to be looking at a very specific context, but if we're just thinking overall. Man, 99% when a new hot new model comes out, people try it.

Andrey: So I'll push back on that. It's the claim that it's not about trying it, like these models achieve an equilibrium level of market penetration. It's not—

Seth: How long? How long is—how long is just trying it? Weeks, months.

Andrey: How long are—sorry, can you repeat that question?

Seth: So you're pushing back on the idea that this is, quote unquote, “just trying the new release.” Right. But what is the timeline you're looking over?

Andrey: It's certainly a few months, but it doesn't take a long time to just try it. So, if it was just trying we'd see us blip over a week, and then it would go back down. And I don't—

Seth: If they were highly horizontally differentiated, but if they were just very slightly horizontally differentiated, you might need a long time to figure it out.

Andrey: You might—that's fair. Okay, so the second claim is: the different models have very different patterns of either substituting away from existing models or expanding the market. And I think two models that really highlight that are Claude 3.7 Sonnet, which primarily cannibalizes from Claude 3.5 Sonnet.

Seth: New Coke,

Andrey: Yes, and it's—well, New Coke failed in this regard.

Seth: Diet Coke,

Andrey: Yeah. And then another model is Google's Gemini 2.0 Flash, which really expanded the market on this platform. A lot of people started using it a lot and didn't seem to have noticeable effects on other model usage.

Seth: Right?

Andrey: So this is kind of showing that kind of models are competing in this interesting space.

Seth: My gosh. Andrey, do you want me to evaluate the claim that you made, or are you now just vaguely appealing to competition? Which of the two do you want me to put a prior on?

Andrey: No no no. Go for it. Yeah.

Seth: All right, so the first one is: do I think that if I look at, you know, a website with a hundred different models, some of them will steal from the same company and some of them will lead to new customers?

Right? I mean with a—I, I'm a little bit… Suppose we asked this question about products and you said, “Professor Benzel, will my product steal from other demands, or will it lead to new customers?” I guess at a certain level, it doesn't even make sense, right? There's a general equilibrium problem here where you always have to draw from something else.

I know we're drawing from other AIs, which would mean that there would have to be some kind of substitution. So I mean, yes, I believe sometimes there's going to be substitution, and yes, I believe sometimes, for reasons that are not necessarily directly connected to the AI model, the rollout of a new model might bring new people into the market.

Right. So I guess I agree. Like at the empirical level, I would say 95% certain that models differ in whether they steal from other models or bring in new people. If you're telling me now there's like a subtler claim here, which is that the fact that some models bring in new people is suggestive of horizontal differentiation and is further evidence for strong horizontal differentiation.

And I'm a little bit, I don't know, I'll put a probability on that, but that's, that seems to be going a little bit beyond the scope of the description.

Andrey: Well, we can discuss that in the discussion session. And I think the final part that I make a claim about is that apps, and the users of apps as well, to multi-home across models. So it's not that people are using just one model. It's not like app developers are using just one model for each application. And that's kind of once again pointing to the fact that there isn't just kind of one superior model even within a given model class.

And, Seth, go for it

Seth: Andrey, you did the thing again. You did the thing again where you said, "Here, Seth, do you want to evaluate this empirical finding?" Or do you want me to now say, “This tells us something about the future of competition in AI'?"

Andrey: Yes, yes, yes. All right, go for it.

Seth: The empirical claim, right? Is—give me the narrow claim? One more time? Give it to me.

Andrey: The apps are multihoming.

Seth: The people multi-home. Okay. The narrow claim is we've got these apps; maybe we'll give the user, the listeners, a little bit of context of what a sample app would be.

Andrey: Yeah, so I think about two types of apps here. One is a coding app, so Klein and RU coder are two quite popular coding apps. And we see that users of those apps are multi-homing. And then—those apps are multi-homing; we don't know as much about the users—and then we have kind of various chat-persona apps. And then we have some kind of utility apps

Seth: Yeah. We'll call them, like—let's call that second group role-play apps.

Andrey: Yeah, yeah. We have kind of like PDF extractor and apps like that, that are also on the—

Seth: Very tool-ly. Okay, cool. Alright, so we've got all these apps out, and now you're going to tell me, Professor Benzel, "I think you would be surprised to find out that RU coder, for example, has both the Claude model underpowering it and an OpenAI model powering it." And that one is probably the thing I'm most surprised by.

Right? I definitely would not be surprised at all to know that RU coder can send its cloud tokens to one data center versus another data center; that makes perfect sense. But the fact that you would sustainably have many different contemporaneous models on the same platform feels like a stage in a process rather than where we're going to end up.

What do I mean by that? So why would you want to keep an old legacy model inside of your RU coder? So I've got—I'm, or Silly Tavern, is one that I like. So Silly Tavern is just, you can do role play and talk to characters and pretend you're going on adventures. Right?

It seems like that Claude 3.7 should just be better than 3.5 at that, right? I really don't—my intuition is that they're not strongly horizontally differentiated. Why would you keep both? It would be for legacy reasons, for backward compatibility. Maybe there's a specific interaction or scenario that you had that you had working in the old version of the app, and you want to make sure that that's still around for new users.

So, how would I think about this? I would think about if you want to say that this is like evidence of multi-homing. This multi-homing evidence is evidence of competition because the same app wants to use multiple versions. I kind of disagree, right? The way I think about it is maybe more like, you know, you're a car, and you can either use the old muffler or the new muffler, and some people have upgraded to the new muffler, but some people are still using the old muffler, and so that car has two different kinds of mufflers.

Andrey: Yeah, we can discuss that, you know, that claim as well. I guess, do you want me to address what I think?

Seth: Well, give me a taste, and then let's go to the evidence. Give me a taste.

Andrey: The multi-homing is not happening on an old and a new version of a model.

It's happening on, let's say, 3.7 and Gemini 2.5, which are both relatively new models. The other thing I'd say is that you read Reddit; there are some users that still like 3.5 better than 3.7.

Seth: On the internet, they will prefer one plain white cotton T-shirt to another plain white cotton T-shirt entry.

Andrey: Who are you to question the preferences? The consumer.

Seth: Right? But I guess, all right, so this is my last comment on the priors, and then we'll get into the evidence, which is. This sort of speculation about what people will actually want in the long run is the bridge that gets us from this cross-sectional evidence about 20 April, 2025, to what the world's going to look like in 2027 and 2028. So that's why I'm pushing back a little bit.

Andrey: Yeah, I don't want to make claims that are too great about 2027 based on this cross section. Yes,

Seth: you know, GDP girl's gonna be at 30%

Andrey: That's true.

Seth: And all of you in labor will be automated.

Andrey: There is going to be a lot of market expansion. I hear.

Seth: Oh, babe, listen to our Epic AI episode. We'll post that before this one so you can see what we're laughing at.

Andrey: All right.

Seth: So tell me, Andrey, I can think of no one better suited to walk us through the evidence of this paper than Professor Fradkin of Boston University.

Andrey: Look, it's very simple paper. It's essentially a few graphs, and the graphs are event studies, where we see what happens to a selected set of models around the time of the release of one of the new models. So for the release of Claude 3.7, we see a very obvious drop in the usage of 3.5. You know, if you ballpark it, it's about 80% cannibalization. And the adoption happens within a few weeks, so it's fairly fast. We also look at Flash 2.0. We see very fast adoption, and in terms of tokens used, Flash 2.0 is the biggest model very quickly. And then, Gemini Pro is another model that that gets released in this time period. And it also sees a very fast adoption curve that doesn't seem to cannibalize other models at this time period. So that's kind of the evidence on cannibalization and market expansion and then the evidence on multi-homing. So there, there's some intricacies with the scraping of the data here. So, actually—let's take a step back. Where does this data come from?

Seth: What is Open Router?

Andrey: We haven't discussed what Open Router is. All right. Look, one of the challenges with studying these issues is a lot of the data sits in these fortresses of data where you cannot extract anything from,

Seth: And we're trying for you listeners; we're banging at that gate. We're banging at that gate every day trying to get in for you.

Andrey: Yes. Yes. So people who are using OpenAI know through the chat app, through the direct open API calls, we're not going to get a lot of visibility into that data. We might get some auxiliary data from credit card providers, payment processors, and the like, but it's hard to know how usage is changing and how specific model usage is changing particularly. One thing that exists is this service called Open Router, and there are other companies that are similar to it. And it's built for, I'd say, a sophisticated user who might be like a software developer who knows that, Hey, you know, I want to use a mix of models, or I might want to change my code to use a different model as—

Seth: Andrey, what's the S word that I'm thinking of here?

Andrey: Substitution; What?

Seth: Selection, you're so this. We're looking under the light of the cult plate, not under the light of the people who want to multi-home.

Andrey: Yes. 100%. But I will say—we're looking—let me just explain what Open Router is, and then we'll talk about selection and whether we care about that or not.

Seth: Oops.

Andrey: Okay. So, so it's a very handy service that allows you to call many different types of models. It also allows you to set rules too. Or like which model to use as a function of things that you might not be thinking about if you're just a chat user, like latency, throughput, uptime, specific pricing, and how it affects prompt tokens versus reasoning tokens versus completion tokens. So it's just a really useful service for this, for the app developer.

Seth: I mean, can I—just to interrupt for a split second here, right? Honestly, I feel like you gave me more evidence for horizontal differentiation in this market just by listing those four different features than you did with almost anything else, right? Because all right, I could see why you would need to balance between latency, price, throughput, quality, et cetera, et cetera.

Andrey: Yeah. So, and there is actually an interesting feature of this market that many do not know: there are multiple companies that serve specific models. So this is obviously true with open-source models, where anyone can serve them. So we have a lot of servers of your Llamas and your Deepseeks. But it's also true of the closed-source models.

For example, Microsoft might serve an OpenAI model, and OpenAI might serve the OpenAI model, and there might be differences in how well they're serving these models.

Seth: Does that mean that Microsoft has to know the model weights, or are theyhidden in some way from them?

Andrey: That's above my pay grade. I—

Seth: We will find out for you.

Andrey: I mean, Microsoft owns a lot of OpenAI, so they have some access.

Seth: Okay.

Andrey: Yeah. So, that's kind of an interesting feature of—

Seth: Mm-hmm.

Andrey: Anyway. One thing that this company does is they publish a lot of data about model usage and how the model usage is changing over time, and also about how specific apps use different models.

In particular for each model, they put the top 20 apps using that model and their usage numbers. So you piece these together, and you can get some pretty good information about popular apps and what models they're using and how much they're using.

Seth: Mm-hmm.

Andrey: And even over time, if you're scraping it continuously—

Seth: Do we know if this is for the apps that list themselves on Open Router? Is this the universe of tokens going through those apps? Do we know that?

Andrey: I think it's a universe of tokens going through those apps, but not all apps are—

Seth: Obviously? Yeah.

Andrey: publicly disclosing it. Even if they are using Open Router.

Seth: Well, it's a fascinating data set, so it's going to show us the price of tokens. It's going to show us which apps are using which tokens, and we're going to get dynamics on that over time. So it seems like a perfect data set. Andrey, your next big contribution is just noticing the data set.

Andrey: It's, you know, to be clear, the ML community knows about this data set as well. You know, in this question of how do we evaluate which models are good and which are not, you know, what we all love is revealed preference.

Seth: Oh, ooh.

Andrey: Use? And an open router has one such ranking, right? That's publicly available. It seems pretty hard to game it, although we can talk about ways one could try to game it. and, that should tell us something about which, which model is better, the very least, which model is on the Pareto frontier? Um. And so has the machine learning community; the AI community has been noticing this. So yeah.

Seth: And then they told you, so then your contribution was the translation to economics.

Andrey: I don't know who told me. The other thing I should say is that now certain companies are releasing stealth models on open router as a way to test them

Seth: Oh,

Andrey: That's also an interesting dynamic to explore. In particular, OpenAI has stealth released some models through there.

Seth: And these would be so if I was running Silly Tavern; it would become apparent to me that there's a GPT-4o version too, and I could play around with it as an option.

Andrey: And there's a new model called Optimus Alpha

Seth: Oh God, did let Elon Musk name this one? Oh my God. Somebody took too much testosterone this morning.

Andrey: Yeah. So, all right. That model gets released for a few weeks. People play around with it, and then it's the new OpenAI model.

Seth: Got it, got it. And then, but but theoretically, normal app users of Silly Tavern might be interacting with this model for a little bit before the official release is therefore

Andrey: Yeah.

Seth: Got it. Okay. Cool.

Andrey: Yeah, so what? What questions do you have, Seth?

Seth: What questions do I have? Andrey, it occurs to me this population of LLM users might not be representative of the model of the market as a whole. How do you respond to that limitation?

Andrey: So, I acknowledge it. I think that's—let me kind of push a little bit. So there are different populations of, what shall we say, heavy LLM users that we can think about. One type of user is your basic consumer, and that type might have a ChatGPT subscription or might even use, you know, the free version or Claude, even though really most of the action is in ChatGPT; we're not talking about that. I think that's very clear. Then, it's a consumer product. We know consumers suffer from very large default effects.

Seth: Right?

Andrey: They're not going to be switching very actively in aggregate. So I don't think this paper is about that at all. The second type of use case that we know a lot about, or we're aware that there's a big use case for, is in programming. Right?

Seth: Mm-hmm.

Andrey: And here I think this is a bit of a more representative sample in a lot of ways. Why, Kline and RU code are are serious programming apps.

Seth: Even though they have silly names.

Andrey: Yes, 100%, and they have features that are essentially at parity with features of VS Code, the programming, the copilot, and VS Code and Cursor, even though, as far as I'm aware, Cursor and Copilot use their own software to route the model calls.

You can also model, you know; you can also do the same things in those apps. So I'd say the coverage I. And the user bases of these apps are quite similar; you might say client and Recode users are a little more sophisticated, but I actually don't think it's that big of a

Seth: They're just a little weirder.

Andrey: They're a little weirder.

Seth: So you think this is very representative of the market for AI tokens? For coding?

Andrey: yes, with, with exception, with a—

Seth: Mm-hmm.

Andrey: The exception is that some companies place severe limitations on the types of models their employees can use. So imagine you're working at Google. I imagine if you're working at Google,

Seth: Gotta use it; you gotta eat your own dog food.

Andrey: You cannot use O3for programming, I assume.

Seth: You cannot generate images of German Nazis. They have to be all-right. That's a callback joke, guys. All right?

Andrey: So then there are these other apps, and there, you know, it's hard, it's hard, you know, to say look, I, if you're, if you're an app developer and you have a single-use app, like a PDF text extractor or something like that, I imagine that you are actively, considering different models, especially trying to optimize your costs

Seth: Mm-hmm.

Andrey: And you may or may not use an open router. I'm not sure; certainly, there might be some selection, and if some apps are less, if there are developers who are less sensitive to these issues, they might not feel the need to use open router.

Seth: But for freelance coding, we think this is representative. All right. Now talk about these other settings, like the tools and the role-playing.

Andrey: Talking about this example, let's say you have a service where you send it a PDF, and it gives you back the structured text.

Seth: Mm-hmm. Mm-hmm.

Andrey: Which is a type of app that you can find on OpenRouter. I doubt that whoever's writing these types of apps is very different whether they use open route or not. I imagine they're considering many models.

Seth: Right. Well, I mean, I guess we're in; we're kind of like in the talk-about-it section, but like you could see a lot of this stuff getting backward built into the platform, right? There's this story, you know, about iPhones. When you started off with an iPhone, there was like a light bulb app that you had to install to get the light to go, but then they built it into a feature of it, right? So, I mean, in the long run is there even a place for something like Open Router, or are these all features that are going to be built right into OpenAI or built right into Anthropic?

Andrey: I guess the feature of being able to use the other models is a feature. I doubt that they'll build into it, but you know, who knows, right?

Seth: Right, but they might give you different versions. There would be the within OpenAI version and then the within Claude version, and they could give you a selection of models.

Andrey: Sure, sure. So if you're like, and I think a lot of big companies do this, if they sign an enterprise contract with OpenAI or Google or Anthropic, they're going to use their models. They might even have forward-deployed engineers that kind of show them how to use the model in the best possible way, how to fine-tune it, and so on.

So I think there's a lot of, if something, if an application requires really close cooperation between the foundation model provider and the application layer, I think we'll see that essentially the different competitors are splitting off into cooperating with different model providers.

Seth: Right. So you think that is one possible future, which is that we end up with much more fragmentation than open router. So there would be, in that universe, multi-homing across models, but not multi-homing across companies.

Andrey: Yeah. I think multi-homing across models versus multi-homing across providers—yeah, we should be kind of clearer about that. And I think the evidence that I have is at least not—it's not just multi-helping within, you know, within OpenAI or within Llama or—

Seth: Ooh. Ooh. We'll have to see about that. All right. Okay. Alright. Other questions I have about this are, you know, not all tokens are created equal, either. I mean, how large a range in prices are people paying for these tokens? Like, what I know is you have a little table of a maximum and minimum, but give the audience a sense of how expensive intelligence can get and how cheap it can get.

Andrey: How expensive and how cheap can it get? so it can be close to free, especially for pretty small models. And it can get pretty expensive. So, there's an output price of 18 per million tokens that exists on this platform. At the time I was looking at it, for example.

Seth: It's still cheaper than my ghostwriter.

Andrey: Yeah, I mean, a million tokens is not nothing for sure. And then, there are differences in input prices and output prices. And there's also something that I haven't captured very well in this data, which is there might be discounts for something called NGS. Things get more complicated the more I look at it in detail.

Seth: Right. And the question is, do these kinds of details suggest concentration, or do the details suggest disillusionment and horizontal differentiation?

Andrey: Yeah.

Seth: Hmm.

Andrey: let's talk a little bit about just some very basic economics of

Seth: What the fuck is competition? Why do we want it?

Andrey: Yeah. So I think first let's first think about the utility, the consumer app developer utility part of this, right? Let's imagine that they have some utility for the different models, but they also have to, you know, pay a price for it. So, the way we think about it is, how much are people willing to pay for the better model? And if we think that things are pretty vertically differentiated, everyone will want to pay more for the same types of models. If we think that things are horizontally differentiated, then different developers will want to pay more for different types of models. And then there's also this question about the scaling thing. Like, yeah, maybe there's a model that's a little bit better than the other model, but it's a lot more expensive, and people are not willing to pay for that. So that might be something going on.

Seth: Hmm.

Andrey: Prices, obviously, are a very important variable to think about, and especially when you can think about them in the following way. Say you have a hard problem. One way to approach it is you throw it to the best model. Another way to approach it is to call a slightly worse model 10 times and then pick the best answer, right? So there's some implicit kind of substitutability that might be present in this.

Seth: But that. Oh man. So now that's so interesting because the story you just told is not a story about horizontal differentiation. Right.

Andrey: yes,

Seth: But it is a reason why you might want lots of different vertically differentiated models.

Andrey: Yes. Yeah.

Seth: Ah huh. So maybe we don't have direct evidence on horizontal differentiation here.

Andrey: For what it's worth. I'm not sure how often these, this pattern, are being used, but it's

Seth: Okay.

Andrey: It's certainly possible. Yeah. And then there's another kind of thing to mention, which is this famous Jevons paradox, which is a paradox.

Seth: I mean, no. Paradox is really a paradox according to my book, Slight of Mind, about why paradoxes are dumb and you should just know all the right answers.

Andrey: Yes. Alright. So, let's say we have an efficiency improvement in our model serving, and we kind of lower our prices by a bit. The response to that might be so large that the total number of tokens used might go up.

Seth: Right?

Andrey: Essentially, the dynamic at hand or the total revenue can go up.

Seth: And so, I mean, it seems like that's happening constantly in this data, which is where we're releasing better and better models and demand just goes up.

Andrey: Yeah. Yeah,

Seth: Which is which provides another challenge for thinking about substitutability because we don't have individual-level data. This is not a static market.

People are entering this market all the time. You gotta be; I mean, the figures you make are quite compelling, like stuff is happening the instant these models are released. But it's also the case that, you know, compositionally, who's in this data is changing and pretty fluid.

Andrey: Yeah. Yeah. it's something I do hope to have more to say about, as I've been scraping at the time, because at least within an app, you might say that the

Seth: It's homogeneous within an app. Yeah. Or maybe you loop together all the coding apps and all the, you know, silly taverns. Okay, cool. Alright. I mean, how much are you in, and how much do you feel like you have to make a claim about horizontal differentiation here?

Andrey: Look, it's hard for me to see multihoming and no and think that there is no horizontal differentiation here.

Seth: Other than price, quantity, differentiation, or price quality,

Andrey: But there, no, no. Sure. But I guess, I guess a point that, you know, you can see in, in, in these figures is that you have, these are pretty similarly priced models in many ways that are being multi-homed.

Seth: The latency is a little bit different. Maybe I'm going to switch back and forth based on latency. There are a lot of different little things here, right?

Andrey: Sure, sure. That's fair. Without having the individual usage data, it's really hard for me to make these finely green claims. I certainly have begged for this data from the CEO of OpenRouter, but so far no cigar.

Seth: Okay, let me push. Let's talk about that a little bit more, right? Which is, if the multi-homing is driven by fluctuations in latency, let's say, right? Like, I don't have strong preferences between Claude and ChatGPT; I just want to call the one that's lower latency. You can definitely get multi-homing there without it being driven by any difference amongst the models.

Andrey: Sure. I guess I think this is very empirically testable. I haven't—the latency is at a five-second level, and just see how much it changes over time.

Seth: There we go.

Andrey: Yes.

Seth: Ooh, ooh. I've given you some more homework, it sounds like.

Andrey: So, I guess if we think that the latency is highly variable across time or the throughput is highly variable over time, then we might see that sort of pattern. If we don't see it being very highly variable over time, then maybe that's less—maybe that's some evidence that it's not quite what's driving it, but yeah.

Seth: Let me tell you what my prior is, so maybe this is like the key part here, right? I have this really strong prior that I did not have; I was not born with it, but I have been trained by talking to AI experts

Andrey: Mm-hmm.

Seth: There’s no such thing as the AI that's good at military stuff versus the AI that's good at writing humanities papers.

That it's all intelligence—you get more of it or less of it. Sure. At the margin there's fine-tuning, there's vibes, but with the right sort of prompt and, you know, with a sufficiently unlocked model, you should be able to; it should be just pure vertical differentiation. That's kind of it; when I've been in rooms with technologists, that's the claim they make.

Now, maybe that's because they're at OpenAI and they're at Anthropic, and it's their incentive for this to be a universe where there's only two big boys. But serious people I've talked to have suggested there isn't such a thing as significant LLM horizontal differentiation.

Andrey: Yeah. I don't believe that. Let's see what they—let's see what they actually do.

Seth: Mm-hmm.

Andrey: OpenAI is constantly updating its default model in ChatGPT. And sometimes they're optimized for one metric, and then they realize that they face a trade-off. So, for example, if your ChatGPT is a little too nice to you, that might lead you to use ChatGPT more, but it might feel ethically dubious for ChatGPT to be encouraging your addiction, given that you totally deserve to be addicted to your phone. So, there's clearly a Pareto frontier of different things that these models can be made to do. Right? So do I. So and so, a lot of experimentation by the companies is the form. is, how do we play on this pato frontier? The existence of Pato Frontier suggests that there isn't just one dimension on which things differ.

Seth: Right. But I guess where I come at this from is, okay, imagine there's like a continuum of steps of delivering the token to the consumer, right? The first step is a $500 billion pre-training run. We, you know, make the giant pre-trained model. The second step is we're going to fine-tune it. We do the RLHF and give my model its particular personality, and it knows it's not allowed to work for terrorists or whatever.

And then there's the third step, which is we're now going to plug that fine-tuned model into an app, and it's going to be deployed in something functional that a consumer can interact with. I guess the way I see it is like as we move down that continuum, this becomes more and more horizontally differentiated, and at the beginning it seems really not horizontally differentiated, and by the end it really is very, you know, you don't want the silly tavern AI, you know, helping you convert PDFs.

Right. So I guess when I hear LLMs are horizontally differentiated, I'm thinking about that pre-training step.

Andrey: Mm-hmm.

Seth: Maybe you want to make a claim about how the usage of AI in apps is horizontally differentiated, which is at the far other end.

Andrey: Sure. Yeah. I, I think that's true. We don't, you know, and you know, we've talked about unhobbling on the show before, and I certainly believe that lots of these models have capabilities that we haven't figured out how to get out of them. Right. They know so

Seth: Right. I've tried really hard to make OpenAI do some of those things, and it's not—it's not as nice as Grok when you ask him to, or

Andrey: Yeah. So, so I think that's right, right? How the application and how these models are used in the application layer can be differentiated even if we think that at the foundational level it's just a ball of clay and some of these balls are bigger clay balls than other balls.

Seth: Oh, right. And when you have smaller clay balls, you can't build the Mona Lisa of play balls. Right. So it's like a capacity thing. Yeah, I mean, it just brings us back to there being a vertical aspect and a horizontal aspect, and the question is like, in the market competition for AIs, where do those two come in? Right? Because in terms of app deployment, you wouldn't expect vertical. I mean, everyone's just going to use the best; they're going to use bottles that are on the Pareto frontier. So you'd expect the horizon, the vertical differentiation, to be less apparent in that last stage. Right?

Andrey: Yeah. I mean it; I do it. It seems to me that models like Gemini 2.5 Pro and 3.7 Sonnet are both on the frontier, but. Some people just like one, and some people like the other. And, and that, that is horizontal differentiation to me.

Seth: Right. And, and now, now you're referring to, like—

Andrey: It's like maybe there's this, like there's a cost difference, and there might be latency differences, and that's really what's driving, you know, the usage patterns.

Seth: Or maybe the prices are identical, and I'm Epsilon horizontally differentiated, and that's enough.

Andrey: Yeah.

Seth: I guess the last thing is that I think my instinct is that horizontal differentiation will become less important over time. Right. So if you think about these balls of clay getting bigger and bigger and bigger, right?

Sculpting them exactly the way you want is going to get easier and easier as you have more and more clay to discard. Do you buy that argument?

Andrey: I think we'll get better at sculpting things over time. I think that it's certainly true. Yeah, and I think that comes back to your question about whether we are going to have horizontal differentiation in the sculpting step. And then the question is, who's going to be sculpting it? Is it going to be app developers sculpting it? Is it still going to be the big labs that sculpt it in various specific ways? Yeah, that.

Seth: Right. I mean, it makes it like if we, if we're doing the sculpting at the app stage, right, there's just a lot more room for horizontal differentiation, right? Because there's a lot more players who are going to be involved, and, you know, that's, that's the domain where, yeah, it does make mean, you know, a dollar to a consumer, whether the interface is blue versus pink and like even stupid shit like that can support an industry, no offense to, you know, app developers out there.

Okay. So one question that is kind of like the implicit background question in this paper, in my opinion,

Andrey: Okay.

Seth: But it is a prior, which we did not put a probability on, but I just kind of want to ask you, can you come at this with having done this research? It doesn't—you don't have to do it in a prior way, which is like, do you think the market for AI will be, you know, relatively competitive or relatively concentrated in four or five years?

Because I mean, my reading of this paper was like, it's a shot for, it's going to be less concentrated and more competitive than you think.

Andrey: I think it depends a lot on the complementarity of other things.

Seth: There you go. There you go. Speaking of Catherine Tucker, we had her asking her about AI competition. She's like, "Well, you know, I'm Catherine Tucker." Catherine Tucker thing.

Andrey: That is not how she talks.

Seth: She does not talk like that. So I'm not going to try to do my Catherine Tucker voice. But like, her point was like, we know how to do antitrust. It has to do with networks of complementarities and substitution abilities. There's nothing special about AIs. Is that kind of your take?

Andrey: I don't think I'm going to make the claim that we know how to do antitrust of AI. That seems premature, to say the least. I will say that the concentration of the industry is very likely to be determined by complementary integration assets. So how important is it to have that Anthropic engineer sitting at, you know, SAP, the specific molded version of Claude, or a particular application or not? Is it something where. at SAP will just call Open Router, and it's just going to be good enough that way. And they don't have to do specific SaaS contracts with Anthropic or anything like that. and that's hard for me to answer right now. But you know, if I had, if I were a betting man, I would say that there'd be a handful of models that are pretty competitive with each other.

But I don't think there'll be like a thousand models that are competitive with each other.

Seth: Right. That frontier, there's just not, there's not enough room at the top, at the frontier. Just because these trading runs will be so, so expensive. I guess that's kind of—as I was reading this paper, in the back of my head, I'm thinking, you know, like, how many people are going to come up with $500 billion to pre-train their own models?

Right. It—it just seems like there's a maximum of how competitive this industry can get.

Andrey: But I guess so. I would say like five; five is often enough to get a very competitive dynamic. Why do we want competition? It's not just because we want a bunch of competitors, for competitors' sake. We actually want there to be the correct incentives to innovate and then to price fairly, right?

So those are kind of the two things we're trading off. And in industrial organization, there are some results that in certain cases where you want even less than five competitors for the incentives. So that still seems quite competitive, even if there is a lot of concentration.

Seth: Right. I—it's all maybe another way of thinking about this is, suppose we could wave a magic wand and either make AI more horizontally differentiated or make it less horizontally differentiated. Right. We could choose which world we're in.

Andrey: Mm-hmm.

Seth: A world where they're less horizontally differentiated is probably one with faster growth and, you know, fewer implementation costs and less friction. Right.

Andrey: Yeah, I'm not sure. It depends; it depends on how we think about, like, the specific innovation production function. Don't; it's not obvious to me that there's, like, one answer, right? Because you can imagine that in a horizontally differentiated world, more players are going to be able to try to innovate, and because there are more, there are going to be more rents. But if you think that it's all about just that huge run, that one big run,

Seth: Right,

Andrey: Maybe it's that you want it to be vertically differentiated and kind of a winner-take-all dynamic. But, one where the winner can change to from time to time.

Seth: Right. You want a comp, so then we're in a universe where it's competition for the market rather than competition in the market. And that brings its own set of antitrust concerns. Andrey, you know, believe it or not, I took a minute to look at the same data and ask questions right along these lines of, like, how concentrated is this market exactly?

Because reading your paper, it's a paper that's supposed to give me some hints about the competitiveness of the industry. The first thing people ask about an industry is, well, how concentrated is it? Right? So Andrey, what's your sense? Are these models more or less concentrated than a typical industry?

Andrey: Um.

Seth: Industry? And actually I want you to tell me, all right? So I've got three. I'll leave my test on the table here. I've got four HHI indices I'm looking at right now. I've got open wrap. This is for the week, the first week of May. we've got. The number of tokens is called at the AI company level, so it aggregates up to companies.

We got the number of tokens called at the AI app level, so that's like a silly tavern, et cetera, et cetera. Then we've got the number of tokens called at the model level, and then I would like you to compare these two to inequality in motor vehicles and breakfast cereals. So I want you to rank those five from most equal to least equal.

Andrey: Yeah, so I will push back on. You count already; you count like the Met Lamas as being Metas, right? Because Meta is not the one who's serving them. Right. But.

Seth: Ooh. Ooh. Well, I could do providers too. That would be a fourth way to split it.

Andrey: Yes. But generally, yeah. Look, it's more concentrated than these other industries.

Seth: It's pretty concentrated.

Andrey: I'd say more so than I, for I, for all of them, with the model-specific one. Even with that, I'd say it's probably more concentrated than the—

Seth: That one is actually pretty low. So the model, so just, I'll put some numbers out there. Just, ballpark, motor vehicles have an HHI of about 2,500; breakfast cereals are just below that.

Andrey: Mm-hmm.

Seth: The number of tokens at the company level has an HHI of 2960, so it's a little bit higher than those guys. But if we go to the app level, we're at 2160, so that's kind of more competitive than motor vehicles and breakfast cereals, which we think have a decent amount of competition.

And then the model level, so we're going to treat 3.5 and 3.7 differently. We're pretty equal. We're at the 1500 level, which is considered pretty, pretty competitive.

Andrey: competitive. Yeah.

Seth: All right. Does that change your progress, Andrey?

Andrey: Well, I guess I wouldn't have used those industries as a comparison set.

Right? Like, I think a lot of digital infrastructure types of industries have a lot more concentration. So you think about cloud computing or search or phones, right?

Seth: mm-hmm.

Andrey: I think so. Relative to those kinds of industries, it is less concentrated. But certainly compared to physical goods products, it's more, it seems, more concentrated, I guess. I assume that you didn't calculate that HHI per car. Right? So it's kind—

Seth: No, it was not. That was at the company level.

Andrey: Yeah. I mean—you know, disclosure, you know, this, this definitely has been on my to-do list. I just have not gotten around to it. But I don't.

Seth: All right,

Andrey: I don't think that, this changes my, my priors very much, if

Seth: Okay, well, I've got a second fact for you. Second stylized fact. All right, so now I want you to imagine, oh man, I don't know if we have time to start talking. We'll see the power law and probability distributions for the next episode. But let me give you four different things that might be more or less concentrated.

Right? Here's another four things to think about. The concentration of one is 2023 US CompStat companies. One is the open router, AI at the company level. The second is Hugging Face. You know, our hugging face is another website where people will post AI models. This is for free downloads, so these are like public models.

So I have downloads of Hugging Face AI models. And then finally I have all-time movie box office. So you tell me which of these is going to be the most concentrated: hugging-faced AI downloads, open router, AI tokens, 2023 US publicly traded companies, or movie box offices. All the time.

Andrey: This is by the open router one. That's by the model creator.

Seth: I believe that, yeah, at the company level.

Andrey: Okay. Um. I think Open Router is the most concentrated of these.

Seth: Correct. Second most

Andrey: hugging face?

Seth: hugging face, second most, third most

Andrey: I don't know how to think about CompStat HHI. That seems like how—what's the product market? Sorry.

Seth: the product. Oh, CompuStat. It's publicly traded corporations. So it's everything together.

Andrey: oh, you're just combining all the—?

Seth: Yeah, yeah, yeah.

Andrey: Just revenue by revenue.

Seth: No, it's market value. So, you know, implied market,

Andrey: Yeah, I think that'll be three. And then the movies are four.

Seth: Dude, you don't even need data. You got this down.

Andrey: How about those priors?

Seth: Who needs evidence? But okay. What, you see what I'm trying to get out here, Andrey? Right? Which is, you can give me evidence that people are willing to move back and forth, but if it's the most concentrated industry I can find, it seems pretty concentrated.

Andrey: you like a bunch of industries that are more concentrated.

Seth: Alright? Okay, so now we go. All right, so listen, this is going to be a special two-part episode of Justified Posteriors. In the next episode, Professor Benzel will bring his own evidence and analysis to bear on the data from Open Router, and you'll be the judge. Is AI competitive? Is it not competitive?

It's the future you're going to have to live with one way or the other. Andrey, are we ready to talk about our priors a little bit?

Seth: All right. What's yours? So tell us, you had three claims here. I guess you're a hundred percent convinced of all the claims. Again, you wrote them down.

Andrey: Look, my claims are empirical, right?

Seth: Right.

Andrey: no, I'm not saying that they're right, but I, you know, I think

Seth: They're descriptive.

Andrey: They're quite descriptive. Unless I made a scraping error or something like that, I think they're, you know, they are what they are, but the interpretation is obviously up for debate.

Seth: Mm-hmm. Do you want to take a shot at it? Do you want to give me a percentage chance that in two years—I don't know how to say this—let's say AI, the AI industry, will be more or less competitive than the average tech sub-industry? Is that a fair comparison?

Andrey: I don't know what an average tech sub-industry is.

Seth: I know or choose one search. Let's just search. How about searching? That's really unequal. Alright. Alright. So yeah, that's the question.

Andrey: It's going to be more competitive than search. I have no doubt

Seth: Okay. All right. Let's check that in a couple of years.

Andrey: And also more competitive than phone operating systems.

Seth: Yeah, we got two big boys there. That's fair. Okay.

Andrey: Is it going to be more concentrated two years from now than today? I think that's an interesting question.

Seth: You want to take a—is that 50/50 for you? Or, I think it's pretty; I put 90—ninety's too strong—85% of that is more concentrated in the future than now.

Andrey: I do, so it depends on whether we're measuring by revenue or by token.

Seth: Let's do tokens at the company level. Oh, I guess we should do revenue, right? Revenue's the more economical thing you can do with either one.

Andrey: the reason I was asking is, like, I still imagine there's still going to be a ton of use cases for small, cheap models and,

Seth: Yeah. So the most down. Yeah.

Andrey: A very competitive market, right? Like in the sense that it's, that's, people are going to roll up their, put in, in principle, roll up a very good, small model.

It's the big model that we're really worried about right in.

Seth: Right, right. So yeah, so it's like the value-weighted is the one where you'd be really worried about concentration, given that there might be a lot of small toy ones that people fuck around with. But I think—

Andrey: Talk, I don't. I'm not even talking about fucking around. There are so many—

Seth: Yeah.

Andrey: Like, you could have the model call; you would, right?

Seth: Mm-hmm.

Andrey: you know, every email you're writing in Gmail

Seth: Mm-hmm.

Andrey: For the line of code that you're going through, why not call a cheap model just as a first pass? That might even be the model used to determine whether you want a, you know, more fancy model or something like that.

Seth: Right, right. And you can imagine a universe in which, like those super low-level AI observations, intelligence calls aren't even captured in data because I might be running that locally on my own laptop, right? Yeah—So yeah, so maybe there's some sort of size cutoff above which this, like, becomes interesting and tractable.

Andrey: I mean, I can, yeah. I don't have strong priors on this, I have to say. I could see arguments either way. Maybe 60/40 towards becoming more concentrated in terms of revenue.

Seth: All right. Well, I'm going to try to get Andrey's answer up in the next half of this two-part episode on Concentration in Competition in the AI Industry: Evidence from Open Router. This time it's personal.

Andrey: All right.

Seth: All right. Like, share, and subscribe.

Andrey: Yeah. If you have better data, we're very—

Seth: Give it to us, please. Yo, we'll be your friend. We'll co-author you.

Andrey: Yeah. Just, you'll get such great exposure for your company on this podcast.

Seth: Mm-hmm. Right? We will. And we'll also use your AI to write copy if you have an AI model yourself.