While I'm not usually a fan of numerizing/quantifying socio-political matters, I still think this is an interesting attempt to grapple with the situation before us--and it sheds light on our big problem from yet another angle.
Congratulations, even at this early stage Polybius is clearly working and providing useful information! Data-crunching in the social sciences is 95% worthless but we should be alert to the rare good stuff, and your effort looks promising. In my own work I stuck to the rule that you only need to classify about 200 things, if you don't get a statistically robust result by then, you're wasting your time. But AI is opening new possibilities; it's uncannily good at pattern recognition, which is basically what you're doing here.
My one caveat is that there appears to be over-reliance, for current evaluation, on legacy media. A good half of the action today is on social media; in particular, TikTok (influencers and pushed material) is a more important source of news for many people than all newspapers and TV networks combined. Does the "media capture" know what's happening to X/twitter and CBS? It will be a struggle to incorporate this; I hope somebody who is younger than me and actually uses social media can advise here.
This is a big problem I've been talking to Claude code about, the problem is Twitter API is so expensive, but we have some ways around it. It also does pick up podcasts, it was talking about Rogan the other day
In your anniversary article, you used the phrase, "Judgment, according to Arendt and Kant, is a process of applying a general concept to a particular thing". This reminded me of your robot project, and what judges do (at least in the common law system).
The Robot, and a common-law judge (e.g., in deciding whether X guy "breached the standard of care" or was running a "business"), apply complex, multi-factor concepts that aren't reducible to a featherless-biped definition. It's difficult for humans to do that consistently & reliably at the scale you need for social science; it takes a ton of time, and you can't avoid questions of personal taste etc. And there are a ton of important concepts like this, where it's true both that a) the concept is an important cleavage, b) you can't break the concept down any further, so the definition needs to rely on vibes, but c) "I know it when I see it" isn't satisfying.
This "judgment machine" function seems genuinely very useful and a step forward (I just saw a tweet discussing one such use: https://x.com/EmilVerner/status/2016534229744484814). Definitely in social science and legal compliance (where a firm answer for many questions isn't possible, so a cheap approximation is A-OK).
The same machines are of course going to crank out a lot of slop; People are going to fuck up the bona fide uses of this stuff; etc. But this is genuinely pretty dang neat.
Hi John, I’m a data engineer with a history background and have been a reader of yours for a few years. I’d love the chance to work on this to add some software engineering rigor to it - e.g. adding tests to make sure the code and models are doing what you expect, hosting the code in a version-controlled public repository, and logging/monitoring to observe the live behavior of the service.
I'm a machine learning engineer interested in these subjects and also working on a somewhat similar product that maps regulatory policy documents among developing countries, I'd be happy to join a discord if you're setting one up.
I have some thoughts about methodology but I think your #1 bottleneck is download cost. The best way to do this is 1) vibecode a separate downloading script that gets documents once, stores them in a database, and then has them forever and 2) maximise the preprocessing so you only do the expensive stuff once and subsequent queries are cheap.
Oh and since you know JVL, maybe pitch the Bulwark. They're hiring a PT data engineer so I suspect they're going to start doing some data journalism projects.
A housekeeping/process bit for software projects - for experiments like this to become ongoing projects, they need a "codeowner" (or owners, plural), someone who ultimately decides what is or isn't valid code, what features belong, what the correct behavior of the app is, etc.
If your intent was just to create a proof-of-concept and throw it out there for others to graduate into a project, then you're done. Other people will become the code owners if they so choose. However, if there's more you'd like to do with it - with help from humans - then that'd make you the codeowner, and yeah, something like a Discord or other coordination mechanism would be very helpful toward that end.
It's an excellent concept so I think you're right to own it! Apologies for the repeat messages, but please include me when you figure out how you want to coordinate. I'd love to help.
Hi John, I think the price of gold should be reflected in addition to the VIX index as a market signal.
great idea!
While I'm not usually a fan of numerizing/quantifying socio-political matters, I still think this is an interesting attempt to grapple with the situation before us--and it sheds light on our big problem from yet another angle.
Readers here may be interested to read an Israeli attempt to numerize/quantify the assaults on democracy made by Netanyahu. See: "Netanyahu's 11 Moves Taking Israel From Democracy Toward Authoritarian Rule" in Haaretz, January 21 2026 (https://www.haaretz.com/israel-news/2026-01-21/ty-article-magazine/.premium/netanyahus-11-moves-taking-israel-from-democracy-toward-authoritarian-rule/0000019b-dbad-d4f5-a7ff-dfbd42160000 ). In this article, four different analysts numerically score Netanyahu on the following 11 categories:
* Restrictions on freedom of speech
* Persecution of political opponents
* Trampling the legislature
* The use of the security forces at home
* Violations of court rulings
* Trampling the watchdogs
* Declaring a state of emergency
* Control of the media
* Taking over academia
* Delegitimizing the opposition
* Exploiting the law to remain in power
Also of related interest is Jonathan Rauch's recent article in the Atlantic: "Yes, It’s Fascism" (https://www.theatlantic.com/ideas/2026/01/america-fascism-trump-maga-ice/685751/ ) in which the author discusses (without numerically scoring) Trump's activities for each of the following 18 categories:
* Demolition of norms.
* Glorification of violence.
* Might is right.
* Politicized law enforcement.
* Dehumanization.
* Police-state tactics.
* Undermining elections.
* What’s private is public.
* Attacks on news media.
* Territorial and military aggression.
* Transnational reach.
* Blood-and-soil nationalism.
* White and Christian nationalism.
* Mobs and street thugs.
* Leader aggrandizement.
* Alternative facts.
* Politics as war.
* Governing as revolution.
Congratulations, even at this early stage Polybius is clearly working and providing useful information! Data-crunching in the social sciences is 95% worthless but we should be alert to the rare good stuff, and your effort looks promising. In my own work I stuck to the rule that you only need to classify about 200 things, if you don't get a statistically robust result by then, you're wasting your time. But AI is opening new possibilities; it's uncannily good at pattern recognition, which is basically what you're doing here.
My one caveat is that there appears to be over-reliance, for current evaluation, on legacy media. A good half of the action today is on social media; in particular, TikTok (influencers and pushed material) is a more important source of news for many people than all newspapers and TV networks combined. Does the "media capture" know what's happening to X/twitter and CBS? It will be a struggle to incorporate this; I hope somebody who is younger than me and actually uses social media can advise here.
This is a big problem I've been talking to Claude code about, the problem is Twitter API is so expensive, but we have some ways around it. It also does pick up podcasts, it was talking about Rogan the other day
And sooner or later Claude will start charging its real expenses. Good reason for everyone to subscribe to your substack!
In your anniversary article, you used the phrase, "Judgment, according to Arendt and Kant, is a process of applying a general concept to a particular thing". This reminded me of your robot project, and what judges do (at least in the common law system).
The Robot, and a common-law judge (e.g., in deciding whether X guy "breached the standard of care" or was running a "business"), apply complex, multi-factor concepts that aren't reducible to a featherless-biped definition. It's difficult for humans to do that consistently & reliably at the scale you need for social science; it takes a ton of time, and you can't avoid questions of personal taste etc. And there are a ton of important concepts like this, where it's true both that a) the concept is an important cleavage, b) you can't break the concept down any further, so the definition needs to rely on vibes, but c) "I know it when I see it" isn't satisfying.
This "judgment machine" function seems genuinely very useful and a step forward (I just saw a tweet discussing one such use: https://x.com/EmilVerner/status/2016534229744484814). Definitely in social science and legal compliance (where a firm answer for many questions isn't possible, so a cheap approximation is A-OK).
The same machines are of course going to crank out a lot of slop; People are going to fuck up the bona fide uses of this stuff; etc. But this is genuinely pretty dang neat.
Hi John, I’m a data engineer with a history background and have been a reader of yours for a few years. I’d love the chance to work on this to add some software engineering rigor to it - e.g. adding tests to make sure the code and models are doing what you expect, hosting the code in a version-controlled public repository, and logging/monitoring to observe the live behavior of the service.
awesome! I think we’re gonna do a discord soom, there seems to be enough interest.
https://discord.gg/Ns5f4USV
This is so cool.
This is fantastic. I love it.
Hey, I asked about this on bsky last week! Neat!
You got carried away! LOVE THIS! Brilliant and FUN (despite the subject matter)!
I'm a machine learning engineer interested in these subjects and also working on a somewhat similar product that maps regulatory policy documents among developing countries, I'd be happy to join a discord if you're setting one up.
I have some thoughts about methodology but I think your #1 bottleneck is download cost. The best way to do this is 1) vibecode a separate downloading script that gets documents once, stores them in a database, and then has them forever and 2) maximise the preprocessing so you only do the expensive stuff once and subsequent queries are cheap.
https://discord.gg/Ns5f4USV we are creating a. community!
I shall take a pass at the moment, perhaps a ‘quality’ update, say in 5 years?
I'm interested in helping if/when you decide to create a community of coders.
Also, you might want to send this to the folks at Protect Democracy for them to feature.
do you have any contacts there by any chance?
Unfortunately no.
Oh and since you know JVL, maybe pitch the Bulwark. They're hiring a PT data engineer so I suspect they're going to start doing some data journalism projects.
https://discord.gg/Ns5f4USV
Don't want to commit political science but perhaps Bruce Bueno de Mesquita's work might be useful here?
It's brilliant. Apart from its view of today, the timeline of its analyses will be a fascinating historical record. Great work.
A housekeeping/process bit for software projects - for experiments like this to become ongoing projects, they need a "codeowner" (or owners, plural), someone who ultimately decides what is or isn't valid code, what features belong, what the correct behavior of the app is, etc.
If your intent was just to create a proof-of-concept and throw it out there for others to graduate into a project, then you're done. Other people will become the code owners if they so choose. However, if there's more you'd like to do with it - with help from humans - then that'd make you the codeowner, and yeah, something like a Discord or other coordination mechanism would be very helpful toward that end.
I am the codeowner. People can make their own things but Polybius is mine ultimately I think. Or I'd pass it on to someone else
It's an excellent concept so I think you're right to own it! Apologies for the repeat messages, but please include me when you figure out how you want to coordinate. I'd love to help.
No, I really appreciate your interest! I am figuring out the best way to coordinate interested parties and will get back to you soon.
https://discord.gg/Ns5f4USV
Here is the Discord for Polybius.