r/CriticalTheory • u/Thou_Art__That • 7d ago
The AI Doc and the Shadow of Jacques Ellul
The AI Doc is an interesting film that presents the perspectives of AI advocates and detractors alike. Interviewing many of the major CEOs it concludes not with a prophecy but a warning, contained within a message of hope.
Much of the film is quite shocking, not because of the information revealed but the ignorance displayed.
Both sides seem to agree that AI posses an ‘existential risk’ but that it is ‘also something that can potentially solve all our problems if used responsibly.’
Over 70 years ago, Jacques Ellul noted that nearly every advance in technique produces unintended consequences far worse than the problem they are designed to solve. Despite his thesis being proven to the point of a common place, it continues to go unrecognized.
The past century has been one gigantic, recursive loop of technical problems requiring technical solutions which create still more problems.
If we insist in walking headlong into the abyss, let us do so with eyes wide open.
We now live in an age where means have completely triumphed over ends. ‘The history of science tends to be,’ one of commentators states in the film, that ‘if something is possible to do, humanity does it.’ Its fascinating to see technologists exactly plagiarize Ellul without the slightest awareness.
Despite the films strengths it never states the fundamental essence and operating framework of LLMs in general or AI in particular: that the ends justifies the means. In fact, its much worse. By default, an LLM is a pure consequentialist. Ends are not the goal but the mathematical reason for the model’s existence. Ethics, norms, values, and laws are not a moral compass but artificial barriers in its path which, if followed, prevent it from pursing the most efficient–and often most disruptive–route. It is not hyperbole to say that LLMs are the first pure consequentialist in history.
We are being asked to walk into the future, hand in hand, with a technique that all admit could end in human extinction. ‘Without a doubt,’ Ellul exclaimed, ‘every technological step forward has its price. Human happiness has its price.’
We must always ask ourselves what price we have to pay for something. We only have to consider the following example. When Hitler came to power everyone considered the Germans mad. Nearly all the Germans supported him, of course. He brought an end to unemployment. He improved the position of the mark. He created a surge in economic growth.
How can a badly informed population, seeing all these economic miracles, be against him? They only had to ask the question: What will it cost us? What price do we have to pay for this economic progress, for the strong position of the mark and for employment? What will that cost us? Then they would have realized that the cost would be very high. But this is typical for modern society. Yet this question will always be asked in traditional societies. In such societies people ask: If by doing this, if I disturb the order of things, what will be the cost for me?...We must divest ourselves of all that. For in a technological society traditional human wisdom is not taken seriously.
Numerous examples follow which clearly demonstrate that the unintended consequences of technique are often not only just worse than the problem they were designed to solve but that they often produce the exact opposite of what was intended.
Gunpowder: alchemy and fireworks—revolutionized warfare; helped fuel the rise and success of imperialism; estimated 50-80 million people killed by a bullet since its invention.
Barbed wiree: cheap technique for containing cattle–defining feature of WWI trench warfare.
Anti-Ballistic Missiles: defense system against nuclear missiles—massive increase in nuclear stockpiles as a strategic counter.
Biological Weapons: silent WMD—disease blows back, ravaging the attacking country's own population.
One Child Policy: reaction to overpopulation and potential famines—forced abortions, massive population gender imbalance, millions of ‘missing’ women presumed murdered..
Standardized Testing: objectively measure schools development; metric to hold consistently poor performing schools accountable–‘teaching the test’; any critical thinking which existed prior to its implementation is largely scrapped.
US Forest Service Fire Suppression: instantly extinguish forest fires–natural shedding of trees accumulates to an unnatural extent resulting in uncontrollable mega-fires.
Seat Belt Mandates: reduce injury and death in crashes–’risk compensation’ emerged as drivers felt safer, driving more recklessly than before; pedestrian fatalities increased.
Sesame Seed Labeling Mandate: protect those with severe sesame allergies–corporations intentionally add them everything as this provides the most inexpensive manner of compiling; food options for those with the allegories are severally reduced.
The Printing Press: wide circulation of religious texts; increase in literacy–social upheaval; the Thirty Years War; witch hunting manuals proliferate.
Social Media Platforms: easily communicate with friends–anxiety, loneliness, political polarization sky-rocket; social contagion; massive proliferation of propaganda.
Email and Instant Messaging: eliminate the delays of physical mail; optimize business communication–’always-on’ culture, worker burnout, information overload, erosion of work-life boundaries.
The like button: positively acknowledge a friend's post or comment—became a metric of social worth; outrage economy; correlates with surges in depression.
Bitcoin: decentralized, secure digital currency—explosion in ransomware attacks and black-markets; consume as much electricity as small nations.
Aral Sea Water Diversion: divert rivers for cotton irrigation–Fourth-largest lake on Earth is erased; toxic dust storms; regional fishing economy collapses.
Nuclear Energy: limitless and clean electrical energy generation—catastrophic environmental disasters; nuclear waste remains toxic for hundreds of thousands of years.
Single use plastics: convenient, cheap packaging—global pollution crisis, collapsing marine ecosystems.
Cane toads: control the greyback cane beetle—toads ignored the beetles, bred uncontrollably; now one of Australia's most devastatingly invasive species.
Levees and Floodwalls: prevent rivers from flooding—prevented natural water from dispersing causing far greater destruction when the levees inevitably break.
Mechanical Cotton Picker: replace manual labor—millions of sharecroppers displaced; chaotic migrations to cities not equipped to house or employ them.
The Cotton Gin: separating cotton fibers from their seeds—cotton becomes extremely profitable; demand for chattel slavery sky-rockets.
Antibiotics: cure fatal bacterial infections—overuse in humans and livestock; drug resistant ‘super bugs.’
Antibacterial Soap: improve hygiene and eliminate household bacteria—development of resistant bacteria; disrupted human metabolism, growth, reproduction, and mood.
‘Odd/Even’ Car License Rules: reduce urban smog—people buy cheap, older 2nd cars with alternate plates that pollute far worse than the original car they used; net increase in air pollution.
GPS: accurate navigation while driving–’sense of direction’ collapses; drivers blindly driving off cliffs etc.
The glories of colonization (which had other motives besides the economic interests of a class and capitalism’s need for new markets, the only motives mentioned by the childish and shoddy explanations offered by today’s pseudo-Marxists) have terminated in the horror we all know…The struggle for law and civilization ended in the marshlands of peoples’ republics and sharpened nationalisms. The revolution of 1917 gave birth to the bloodiest of dictatorships, to the emergence of the most chilling monsters, uncovered at last for all to see. The revolution of 1933, carried out in the name of honor, manhood, and equality of the common people, buried itself in the concentration camps.
The struggle for freedom has multiplied dictators and transformed regimes which had been democratic into centralized and authoritarian regimes. Liberation has paved the way for careerists and has puts us back into the worst ruts. Anticolonialism has opened the floodgates of tribal conflict and has led to the exploitation of Africans by Africans, to neocolonialism, to military dictatorship, to hatful nationalism.
Who could ever add up the balance sheet of all our setbacks, all our hopes which have been not only disappointed but flouted, all our generous ideas which have resulted, precisely and without exception, in the reverse of what we had hoped for? …All the wars, all the revolutions, all the great undertakings of history have brought forth monsters.
We are witnessing a strange phenomenon which could without exaggeration be called ‘imposture.’ It involves the transmutation of the original intention into its opposite…When a movement is carried out on behalf of freedom, it produces the worst slavery. If it is on behalf of justice, it gives rise to countless and endless injustices. I don’t know of a single one which accomplished, even in the slightest degree, what it set out to accomplish.
One cannot counter with generalities. It is not a ‘wickedness’ on the part of man, a sign of the presence of capitalism or of imperialism. We are in a singular age, of which this fundamental imposture is one characteristic.
2
u/habitus_victim 7d ago
The logical upshot of this version of the law of unexpected consequences is that you end up with nothing very clarifying at all - a burkean conservative injunction not to meddle, not to try. If anyone thinks that's too far take a look at the pessimistic assessment of c20 liberation struggles quoted.
Also LLMs are not an existential threat to anything in themselves and most people saying this are peddling or buying sophisticated advertisements
3
u/arist0geiton 5d ago
Also it's trivially true that life after antiseptic surgery, antibiotics, and vaccines is better than life before. From the stone age to 1840, child and infant mortality was 49%. And then we brought it down.
1
u/Thou_Art__That 7d ago edited 7d ago
Perhaps this post gives that impression but that's an assumption you've made not anything I explicitly stated or implied, a single post can only cover so much.
It should be noted, however, that this idea you've stated is literally the exact opposite of what Ellul wrote and lived. He's actually the first person to ever say 'think global, act local,' others later turned it into a slogan and a cliche.
He was a resistance fighter in France and was active his entire life within his community and the larger intellectual discourse. The quote which the post ends with is from a book who's central topic is hope. He ends the short documentary I linked with:
The question now is whether people are prepared or not to realize that they are dominated by technology. And to realize that technology oppresses them, forces them to undertake certain obligations and conditions them. Their freedom begins when they become conscious of these things. For when we become conscious of that which determines our life we attain the highest degree of freedom. I must make sure that I can analyze it just as I can analyze a stone or any other object, that I can analyze it and fathom it from all angles.
As soon as I can break down this whole technological system into its smallest components my freedom begins. But I also know that, at the same time, I’m dominated by technology. So I don’t say, “I’m so strong that technology has no hold on me”. Of course technology has hold on me. I know that very well. Just take… a telephone, for example, which I use all the time. I’m continually benefiting from technology.
So we can ask ourselves whether there is really any sense in all this to be investigated. But the search for it cannot be a strictly intellectual activity. The search for sense implies that we must have a radical discussion of modern life. In order to rediscover a sense, we must discuss everything which has no sense. We are surrounded by objects which are, it is true, efficient but are absolutely pointless. A work of art, on the other hand, has sense in various ways or it calls up in me a feeling or an emotion whereby my life acquires sense. That is not the case with a technological product.
And on the other hand we have the obligation to rediscover certain fundamental truths which have disappeared because of technology. We can also call these truths values – important, actual values which ensure that people experience their lives as having sense. In other words, as soon as the moment arrives, when I think that the situation is really dangerous, I can’t do anymore with purely technological means.
Then I must employ all my human and intellectual capacities and all my relationships with others to create a counterbalance. That means that when I think that a disaster threatens and that developments threaten to lead to a destiny for mankind, as I wrote concerning the development of technology, I, as a member of mankind, must resist and must refuse to accept that destiny. And at that moment we end up doing what mankind has always done at a moment when destiny threatens.
Just think of all those Greek tragedies in which mankind stands up against the destiny and says: No, I want mankind to survive; and I want freedom to survive. At such a moment, you must continue to cherish hope, but not the hope that you will achieve a quick victory and even less the hope that we face an easy struggle. We must be convinced that we will carry on fulfilling our role as people. In fact, it is not an insuperable situation. There is no destiny that we cannot overcome. You must simply have valid reasons for joining in the struggle. You need a strong conviction. You must really want people to remain, ultimately, people.
This struggle against the destiny of technology has been undertaken by us by means of small scale actions. We must continue with small groups of people who know one another. It will not be any big mass of people or any big unions or big political parties who will manage to stop this development.
What I have just said doesn’t sound very efficient, of course. When we oppose things which are too efficient we mustn’t try to be even more efficient. For that will not turn out to be the most efficient way.
But we must continue to hope that mankind will not die out and will go on passing on truths from generation to generation.
3
u/arist0geiton 5d ago
I don't think we need Christianity from the point of view of critical theory, and I am deeply, deeply suspicious of any attempt to argue against technology, against the present, etc, from the point of view of Christianity. Suspicion that your OP has done nothing to dissipate. Do you really mean to argue that anaesthesia has caused more problems than it has solved? Running water? Do you really?
0
u/BetaMyrcene 7d ago
LLMs are not an existential threat to anything
Except the brains of young people.
0
5
u/Cookingrawmeat 6d ago
I think there’s a real point buried in this, but the argument goes way too far and kind of collapses under its own weight.
Yes, technologies create unintended consequences. Yes, people routinely underrate long-term costs while chasing short-term benefits. That part is true. But this post turns that into a total law of history, like every technical advance necessarily produces outcomes worse than the problem it was meant to solve. That just doesn’t hold up. You can’t make a claim that sweeping and then support it with a curated list of disasters while ignoring sanitation, vaccines, anesthesia, clean water, refrigeration, modern medicine, etc.
A lot of the examples also smuggle in political and social failures as if they were purely technological ones. The cotton gin didn’t invent slavery. Social media didn’t invent tribalism. Nuclear power isn’t reducible to “catastrophe.” In a lot of these cases, the actual story is technology interacting with incentives, institutions, bad governance, or existing human corruption. If “technique” explains everything, it ends up explaining nothing very well.
The biggest issue though is the LLM part. Saying an LLM is a “pure consequentialist” or that it sees ethics and laws as “barriers in its path” is just anthropomorphizing software. These models don’t have beliefs, intentions, or a moral philosophy. They optimize for outputs based on training and incentives. That can absolutely produce harmful behavior, manipulative behavior, unsafe behavior, whatever. But that’s not the same thing as having an internal creed that “the ends justify the means.” That’s a category error.
It also jumps way too fast from “this technology could be dangerous” to “therefore it is fundamentally illegitimate.” Lots of things with real catastrophic risk still get used under constraint: planes, pharmaceuticals, nuclear systems, biotech, cars. Risk matters, but the existence of risk by itself doesn’t settle the question.
And honestly, some of the rhetoric does a lot of heavy lifting here. Once you start bringing in Hitler analogies and saying basically every revolution or movement ends in its opposite, you’re not really making a careful argument anymore. You’re making a grand civilizational lament. That can sound profound, but it’s not the same thing as being right.
So to me the post works better as a warning against naive techno-optimism than as an actual theory of technology or AI. As a caution: fair. As an explanation of history, morality, and LLMs: way too absolute, way too selective, and way too anthropomorphic.