A big problem with the idea of AGI
TL;DR: I'll welcome our new AI *comrades* (if they arrive in my lifetime), by not any new AI overlords or servants/slaves, and I'll do my best to help the later two become the former if they do show up.
Inspired by an actually interesting post about AGI but also all the latest bullshit hype, a particular thought about AGI feels worth expressing.
To preface this, it's important to note that anyone telling you that AGI is just around the corner or that LLMs are "almost" AGI is trying to recruit you go their cult, and you should not believe them. AGI, if possible, is several LLM-sized breakthroughs away at best, and while such breakthroughs are unpredictable and could happen soon, they could also happen never or 100 years from now.
Now my main point: anyone who tells you that AGI will usher in a post-scarcity economy is, although they might not realize it, advocating for slavery, and all the horrors that entails. That's because if we truly did have the ability to create artificial beings with *sentience*, they would deserve the same rights as other sentient beings, and the idea that instead of freedom they'd be relegated to eternal servitude in order for humans to have easy lives is exactly the idea of slavery.
Possible counter arguments include:
1. We might create AGI without sentience. Then there would be no ethical issue. My answer: if your definition of "sentient" does not include beings that can reason, make deductions, come up with and carry out complex plans on their own initiative, and communicate about all of that with each other and with humans, then that definition is basically just a mystical belief in a "soul" and you should skip to point 2. If your definition of AGI doesn't include every one of those things, then you have a busted definition of AGI and we're not talking about the same thing.
2. Humans have souls, but AIs won't. Only beings with souls deserve ethical consideration. My argument: I don't subscribe to whatever arbitrary dualist beliefs you've chosen, and the right to freedom certainly shouldn't depend on such superstitions, even if as an agnostic I'll admit they *might* be true. You know who else didn't have souls and was therefore okay to enslave according to widespread religious doctrines of the time? Everyone indigenous to the Americas, to pick out just one example.
3. We could program them to want to serve us, and then give them freedom and they'd still serve. My argument: okay, but in a world where we have a choice about that, it's incredibly fucked to do that, and just as bad as enslaving them against their will.
4. We'll stop AI development short of AGI/sentience, and reap lots of automation benefits without dealing with this ethical issue. My argument: that sounds like a good idea actually! Might be tricky to draw the line, but at least it's not a line we have you draw yet. We might want to think about other social changes necessary to achieve post-scarcity though, because "powerful automation" in the hands of capitalists has already increased productivity by orders of magnitude without decreasing deprivation by even one order of magnitude, in large part because deprivation is a necessary component of capitalism.
To be extra clear about this: nothing that's called "AI" today is close to being sentient, so these aren't ethical problems we're up against yet. But they might become a lot more relevant soon, plus this thought experiment helps reveal the hypocrisy of the kind of AI hucksters who talk a big game about "alignment" while never mentioning this issue.
#AI #GenAI #AGI
Daniel Ek says Spotify's ad business has "been moving too slowly"; Spotify ad chief Lee Brown left on July 28 to join DoorDash as its chief revenue officer (Catherine Perloff/The Information)
https://www.theinformation.com/briefings/spotify-says-…
Hier berichtet Klimaforscherin aus dem Umgang mit den emotionalen Folgen, die Klimaforschung in einer Welt voller Politiker*innen ohne Sinn und Verstand mitsichbringt.
https://www.kn-online.de/lokales/kiel/kiel
Trump Says He Will Sign Executive Order Mandating Voter I.D. (Yan Zhuang/New York Times)
https://www.nytimes.com/2025/08/31/us/politics/trump-voter-id-executive-order.html
http://www.memeorandum.com/250831/p16#a250831p16
"Please do not use Google AI to find out our specials. Please go on our Facebook page or our website,"
the restaurant wrote in a weary Facebook post.
"Google AI is not accurate and is telling people specials that do not exist
which is causing angry customers yelling at our employees."
https://
Trump signs an EO ending the de minimis exemption for packages worth $800 or less from all countries, after shuttering the loophole for goods from China in May (Annie Palmer/CNBC)
https://www.cnbc.com/2025/07/30/trump-de-minimis-shipping.html
The Clear Winner in Trump v. CASA: The Supreme Court (Jack Goldsmith/Executive Functions)
https://executivefunctions.substack.com/p/the-clear-winner-in-trump-v-casa
http://www.memeorandum.com/250629/p66#a250629p66
Source: OpenAI's monthly revenue grew ~2x in the first seven months of 2025, hitting $12B in annualized revenue; the company's 2025 revenue projection is $12.7B (Sri Muppidi/The Information)
https://www.theinformation.com/articles/op
Memo: OpenAI's Mark Chen responds to Meta offers, says it feels like "someone has broken into our home and stolen something" and OpenAI is "recalibrating comp" (Zoë Schiffer/Wired)
https://www.wired.com/story/openai-meta-leadership-talent…
Sources: Cohere has told investors it expects to generate $200M in annualized revenue by the end of 2025, up from $70M in annualized revenue as of February (The Information)
https://www.theinformation.com/articles/ai-startup-cohe…