How to Destroy Surveillance Capitalism

- The most surprising thing about the rebirth of flat Earthers in the - 21st century is just how widespread the evidence against them is. - You can understand how, centuries ago, people whoâd never gained a - high-enough vantage point from which to see the Earthâs curvature - might come to the commonsense belief that the flat-seeming Earth - was, indeed, flat. -

- But today, when elementary schools routinely dangle GoPro cameras - from balloons and loft them high enough to photograph the Earthâs - curve â to say nothing of the unexceptional sight of the curved - Earth from an airplane window â it takes a heroic effort to maintain - the belief that the world is flat. -

- Likewise for white nationalism and eugenics: In an age where you can - become a computational genomics datapoint by swabbing your cheek and - mailing it to a gene-sequencing company along with a modest sum of - money, ârace scienceâ has never been easier to refute. -

- We are living through a golden age of both readily available facts - and denial of those facts. Terrible ideas that have lingered on the - fringes for decades or even centuries have gone mainstream seemingly - overnight. -

- When an obscure idea gains currency, there are only two things that - can explain its ascendance: Either the person expressing that idea - has gotten a lot better at stating their case, or the proposition - has become harder to deny in the face of mounting evidence. In other - words, if we want people to take climate change seriously, we can - get a bunch of Greta Thunbergs to make eloquent, passionate - arguments from podiums, winning our hearts and minds, or we can wait - for flood, fire, broiling sun, and pandemics to make the case for - us. In practice, weâll probably have to do some of both: The more - weâre boiling and burning and drowning and wasting away, the easier - it will be for the Greta Thunbergs of the world to convince us. -

- The arguments for ridiculous beliefs in odious conspiracies like - anti-vaccination, climate denial, a flat Earth, and eugenics are no - better than they were a generation ago. Indeed, theyâre worse - because they are being pitched to people who have at least a - background awareness of the refuting facts. -

- Anti-vax has been around since the first vaccines, but the early - anti-vaxxers were pitching people who were less equipped to - understand even the most basic ideas from microbiology, and - moreover, those people had not witnessed the extermination of - mass-murdering diseases like polio, smallpox, and measles. Todayâs - anti-vaxxers are no more eloquent than their forebears, and they - have a much harder job. -

- So can these far-fetched conspiracy theorists really be succeeding - on the basis of superior arguments? -

- Some people think so. Today, there is a widespread belief that - machine learning and commercial surveillance can turn even the most - fumble-tongued conspiracy theorist into a svengali who can warp your - perceptions and win your belief by locating vulnerable people and - then pitching them with A.I.-refined arguments that bypass their - rational faculties and turn everyday people into flat Earthers, - anti-vaxxers, or even Nazis. When the RAND Corporation - blames - Facebook for âradicalizationâ and when Facebookâs role in - spreading coronavirus misinformation is - blamed - on its algorithm, the implicit message is that machine - learning and surveillance are causing the changes in our consensus - about whatâs true. -

- After all, in a world where sprawling and incoherent conspiracy - theories like Pizzagate and its successor, QAnon, have widespread - followings, something must be afoot. -

- But what if thereâs another explanation? What if itâs the material - circumstances, and not the arguments, that are making the difference - for these conspiracy pitchmen? What if the trauma of living through - real conspiracies all around us â conspiracies - among wealthy people, their lobbyists, and lawmakers to bury - inconvenient facts and evidence of wrongdoing (these conspiracies - are commonly known as âcorruptionâ) â is making people vulnerable to - conspiracy theories? -

- If itâs trauma and not contagion â material conditions and not - ideology â that is making the difference today and enabling a rise - of repulsive misinformation in the face of easily observed facts, - that doesnât mean our computer networks are blameless. Theyâre still - doing the heavy work of locating vulnerable people and guiding them - through a series of ever-more-extreme ideas and communities. -

- Belief in conspiracy is a raging fire that has done real damage and - poses real danger to our planet and species, from epidemics - kicked - off by vaccine denial to genocides - kicked - off by racist conspiracies to planetary meltdown caused by - denial-inspired climate inaction. Our world is on fire, and so we - have to put the fires out â to figure out how to help people see the - truth of the world through the conspiracies theyâve been confused - by. -

- But firefighting is reactive. We need fire - prevention. We need to strike at the traumatic - material conditions that make people vulnerable to the contagion of - conspiracy. Here, too, tech has a role to play. -

- Thereâs no shortage of proposals to address this. From the EUâs - Terrorist Content - Regulation, which requires platforms to police and remove - âextremistâ content, to the U.S. proposals to - force - tech companies to spy on their users and hold them liable - for - their usersâ bad speech, thereâs a lot of energy to force - tech companies to solve the problems they created. -

- Thereâs a critical piece missing from the debate, though. All these - solutions assume that tech companies are a fixture, that their - dominance over the internet is a permanent fact. Proposals to - replace Big Tech with a more diffused, pluralistic internet are - nowhere to be found. Worse: The âsolutionsâ on the table today - require Big Tech to stay big because only the - very largest companies can afford to implement the systems these - laws demand. -

- Figuring out what we want our tech to look like is crucial if weâre - going to get out of this mess. Today, weâre at a crossroads where - weâre trying to figure out if we want to fix the Big Tech companies - that dominate our internet or if we want to fix the internet itself - by unshackling it from Big Techâs stranglehold. We canât do both, so - we have to choose. -

- I want us to choose wisely. Taming Big Tech is integral to fixing - the internet, and for that, we need digital rights activism. -

Digital rights activism, a quarter-century on

- Digital rights activism is more than 30 years old now. The - Electronic Frontier Foundation turned 30 this year; the Free - Software Foundation launched in 1985. For most of the history of the - movement, the most prominent criticism leveled against it was that - it was irrelevant: The real activist causes were real-world causes - (think of the skepticism when - Finland - declared broadband a human right in 2010), and real-world - activism was shoe-leather activism (think of Malcolm Gladwellâs - contempt - for âclicktivismâ). But as tech has grown more central to - our daily lives, these accusations of irrelevance have given way - first to accusations of insincerity (âYou only care about tech - because youâre - shilling - for tech companiesâ) to accusations of negligence (âWhy - didnât you foresee that tech could be such a destructive force?â). - But digital rights activism is right where itâs always been: looking - out for the humans in a world where tech is inexorably taking over. -

- The latest version of this critique comes in the form of - âsurveillance capitalism,â a term coined by business professor - Shoshana Zuboff in her long and influential 2019 book, The - Age of Surveillance Capitalism: The Fight for a Human Future at the - New Frontier of Power. Zuboff argues that âsurveillance - capitalismâ is a unique creature of the tech industry and that it is - unlike any other abusive commercial practice in history, one that is - âconstituted by unexpected and often illegible mechanisms of - extraction, commodification, and control that effectively exile - persons from their own behavior while producing new markets of - behavioral prediction and modification. Surveillance capitalism - challenges democratic norms and departs in key ways from the - centuries-long evolution of market capitalism.â It is a new and - deadly form of capitalism, a ârogue capitalism,â and our lack of - understanding of its unique capabilities and dangers represents an - existential, species-wide threat. Sheâs right that capitalism today - threatens our species, and sheâs right that tech poses unique - challenges to our species and civilization, but sheâs really wrong - about how tech is different and why it threatens our species. -

- Whatâs more, I think that her incorrect diagnosis will lead us down - a path that ends up making Big Tech stronger, not weaker. We need to - take down Big Tech, and to do that, we need to start by correctly - identifying the problem. -

Tech exceptionalism, then and now

- Early critics of the digital rights movement â perhaps best - represented by campaigning organizations like the Electronic - Frontier Foundation, the Free Software Foundation, Public Knowledge, - and others that focused on preserving and enhancing basic human - rights in the digital realm â damned activists for practicing âtech - exceptionalism.â Around the turn of the millennium, serious people - ridiculed any claim that tech policy mattered in the âreal world.â - Claims that tech rules had implications for speech, association, - privacy, search and seizure, and fundamental rights and equities - were treated as ridiculous, an elevation of the concerns of sad - nerds arguing about Star Trek on bulletin board - systems above the struggles of the Freedom Riders, Nelson Mandela, - or the Warsaw ghetto uprising. -

- In the decades since, accusations of âtech exceptionalismâ have only - sharpened as techâs role in everyday life has expanded: Now that - tech has infiltrated every corner of our life and our online lives - have been monopolized by a handful of giants, defenders of digital - freedoms are accused of carrying water for Big Tech, providing cover - for its self-interested negligence (or worse, nefarious plots). -

- From my perspective, the digital rights movement has remained - stationary while the rest of the world has moved. From the earliest - days, the movementâs concern was users and the toolsmiths who - provided the code they needed to realize their fundamental rights. - Digital rights activists only cared about companies to the extent - that companies were acting to uphold usersâ rights (or, just as - often, when companies were acting so foolishly that they threatened - to bring down new rules that would also make it harder for good - actors to help users). -

- The âsurveillance capitalismâ critique recasts the digital rights - movement in a new light again: not as alarmists who overestimate the - importance of their shiny toys nor as shills for big tech but as - serene deck-chair rearrangers whose long-standing activism is a - liability because it makes them incapable of perceiving novel - threats as they continue to fight the last centuryâs tech battles. -

- But tech exceptionalism is a sin no matter who practices it. -

Donât believe the hype

- Youâve probably heard that âif youâre not paying for the product, - youâre the product.â As weâll see below, thatâs true, if incomplete. - But what is absolutely true is that ad-driven - Big Techâs customers are advertisers, and what companies like Google - and Facebook sell is their ability to convince - you to buy stuff. Big Techâs product is - persuasion. The services â social media, search engines, maps, - messaging, and more â are delivery systems for persuasion. -

- The fear of surveillance capitalism starts from the (correct) - presumption that everything Big Tech says about itself is probably a - lie. But the surveillance capitalism critique makes an exception for - the claims Big Tech makes in its sales literature â the breathless - hype in the pitches to potential advertisers online and in ad-tech - seminars about the efficacy of its products: It assumes that Big - Tech is as good at influencing us as they claim they are when - theyâre selling influencing products to credulous customers. Thatâs - a mistake because sales literature is not a reliable indicator of a - productâs efficacy. -

- Surveillance capitalism assumes that because advertisers buy a lot - of what Big Tech is selling, Big Tech must be selling something - real. But Big Techâs massive sales could just as easily be the - result of a popular delusion or something even more pernicious: - monopolistic control over our communications and commerce. -

- Being watched changes your behavior, and not for the better. It - creates risks for our social progress. Zuboffâs book features - beautifully wrought explanations of these phenomena. But Zuboff also - claims that surveillance literally robs us of our free will â that - when our personal data is mixed with machine learning, it creates a - system of persuasion so devastating that we are helpless before it. - That is, Facebook uses an algorithm to analyze the data it - nonconsensually extracts from your daily life and uses it to - customize your feed in ways that get you to buy stuff. It is a - mind-control ray out of a 1950s comic book, wielded by mad - scientists whose supercomputers guarantee them perpetual and total - world domination. -

What is persuasion?

- To understand why you shouldnât worry about mind-control rays â but - why you should worry about surveillance - and Big Tech â we must start by unpacking what - we mean by âpersuasion.â -

- Google, Facebook, and other surveillance capitalists promise their - customers (the advertisers) that if they use machine-learning tools - trained on unimaginably large data sets of nonconsensually harvested - personal information, they will be able to uncover ways to bypass - the rational faculties of the public and direct their behavior, - creating a stream of purchases, votes, and other desired outcomes. -

- The impact of dominance far exceeds the impact of manipulation and - should be central to our analysis and any remedies we seek. -

- But thereâs little evidence that this is happening. Instead, the - predictions that surveillance capitalism delivers to its customers - are much less impressive. Rather than finding ways to bypass our - rational faculties, surveillance capitalists like Mark Zuckerberg - mostly do one or more of three things: -

1. Segmenting

- If youâre selling diapers, you have better luck if you pitch them - to people in maternity wards. Not everyone who enters or leaves a - maternity ward just had a baby, and not everyone who just had a - baby is in the market for diapers. But having a baby is a really - reliable correlate of being in the market for diapers, and being - in a maternity ward is highly correlated with having a baby. Hence - diaper ads around maternity wards (and even pitchmen for baby - products, who haunt maternity wards with baskets full of - freebies). -

- Surveillance capitalism is segmenting times a billion. Diaper - vendors can go way beyond people in maternity wards (though they - can do that, too, with things like location-based mobile ads). - They can target you based on whether youâre reading articles about - child-rearing, diapers, or a host of other subjects, and data - mining can suggest unobvious keywords to advertise against. They - can target you based on the articles youâve recently read. They - can target you based on what youâve recently purchased. They can - target you based on whether you receive emails or private messages - about these subjects â or even if you speak aloud about them - (though Facebook and the like convincingly claim thatâs not - happening â yet). -

- This is seriously creepy. -

- But itâs not mind control. -

- It doesnât deprive you of your free will. It doesnât trick you. -

- Think of how surveillance capitalism works in politics. - Surveillance capitalist companies sell political operatives the - power to locate people who might be receptive to their pitch. - Candidates campaigning on finance industry corruption seek people - struggling with debt; candidates campaigning on xenophobia seek - out racists. Political operatives have always targeted their - message whether their intentions were honorable or not: Union - organizers set up pitches at factory gates, and white supremacists - hand out fliers at John Birch Society meetings. -

- But this is an inexact and thus wasteful practice. The union - organizer canât know which worker to approach on the way out of - the factory gates and may waste their time on a covert John Birch - Society member; the white supremacist doesnât know which of the - Birchers are so delusional that making it to a meeting is as much - as they can manage and which ones might be convinced to cross the - country to carry a tiki torch through the streets of - Charlottesville, Virginia. -

- Because targeting improves the yields on political pitches, it can - accelerate the pace of political upheaval by making it possible - for everyone who has secretly wished for the toppling of an - autocrat â or just an 11-term incumbent politician â to find - everyone else who feels the same way at very low cost. This has - been critical to the rapid crystallization of recent political - movements including Black Lives Matter and Occupy Wall Street as - well as less savory players like the far-right white nationalist - movements that marched in Charlottesville. -

- Itâs important to differentiate this kind of political organizing - from influence campaigns; finding people who secretly agree with - you isnât the same as convincing people to agree with you. The - rise of phenomena like nonbinary or otherwise nonconforming gender - identities is often characterized by reactionaries as the result - of online brainwashing campaigns that convince impressionable - people that they have been secretly queer all along. -

- But the personal accounts of those who have come out tell a - different story where people who long harbored a secret about - their gender were emboldened by others coming forward and where - people who knew that they were different but lacked a vocabulary - for discussing that difference learned the right words from these - low-cost means of finding people and learning about their ideas. -

2. Deception

- Lies and fraud are pernicious, and surveillance capitalism - supercharges them through targeting. If you want to sell a - fraudulent payday loan or subprime mortgage, surveillance - capitalism can help you find people who are both desperate and - unsophisticated and thus receptive to your pitch. This accounts - for the rise of many phenomena, like multilevel marketing schemes, - in which deceptive claims about potential earnings and the - efficacy of sales techniques are targeted at desperate people by - advertising against search queries that indicate, for example, - someone struggling with ill-advised loans. -

- Surveillance capitalism also abets fraud by making it easy to - locate other people who have been similarly deceived, forming a - community of people who reinforce one anotherâs false beliefs. - Think of - the - forums where people who are being victimized by multilevel - marketing frauds gather to trade tips on how to improve their luck - in peddling the product. -

- Sometimes, online deception involves replacing someoneâs correct - beliefs with incorrect ones, as it does in the anti-vaccination - movement, whose victims are often people who start out believing - in vaccines but are convinced by seemingly plausible evidence that - leads them into the false belief that vaccines are harmful. -

- But itâs much more common for fraud to succeed when it doesnât - have to displace a true belief. When my daughter contracted head - lice at daycare, one of the daycare workers told me I could get - rid of them by treating her hair and scalp with olive oil. I - didnât know anything about head lice, and I assumed that the - daycare worker did, so I tried it (it didnât work, and it doesnât - work). Itâs easy to end up with false beliefs when you simply - donât know any better and when those beliefs are conveyed by - someone who seems to know what theyâre doing. -

- This is pernicious and difficult â and itâs also the kind of thing - the internet can help guard against by making true information - available, especially in a form that exposes the underlying - deliberations among parties with sharply divergent views, such as - Wikipedia. But itâs not brainwashing; itâs fraud. In the - majority - of cases, the victims of these fraud campaigns have an - informational void filled in the customary way, by consulting a - seemingly reliable source. If I look up the length of the Brooklyn - Bridge and learn that it is 5,800 feet long, but in reality, it is - 5,989 feet long, the underlying deception is a problem, but itâs a - problem with a simple remedy. Itâs a very different problem from - the anti-vax issue in which someoneâs true belief is displaced by - a false one by means of sophisticated persuasion. -

3. Domination

- Surveillance capitalism is the result of monopoly. Monopoly is the - cause, and surveillance capitalism and its negative outcomes are - the effects of monopoly. Iâll get into this in depth later, but - for now, suffice it to say that the tech industry has grown up - with a radical theory of antitrust that has allowed companies to - grow by merging with their rivals, buying up their nascent - competitors, and expanding to control whole market verticals. -

- One example of how monopolism aids in persuasion is through - dominance: Google makes editorial decisions about its algorithms - that determine the sort order of the responses to our queries. If - a cabal of fraudsters have set out to trick the world into - thinking that the Brooklyn Bridge is 5,800 feet long, and if - Google gives a high search rank to this group in response to - queries like âHow long is the Brooklyn Bridge?â then the first - eight or 10 screensâ worth of Google results could be wrong. And - since most people donât go beyond the first couple of results â - let alone the first page of results â - Googleâs choice means that many people will be deceived. -

- Googleâs dominance over search â more than 86% of web searches are - performed through Google â means that the way it orders its search - results has an outsized effect on public beliefs. Ironically, - Google claims this is why it canât afford to have any transparency - in its algorithm design: Googleâs search dominance makes the - results of its sorting too important to risk telling the world how - it arrives at those results lest some bad actor discover a flaw in - the ranking system and exploit it to push its point of view to the - top of the search results. Thereâs an obvious remedy to a company - that is too big to audit: break it up into smaller pieces. -

- Zuboff calls surveillance capitalism a ârogue capitalismâ whose - data-hoarding and machine-learning techniques rob us of our free - will. But influence campaigns that seek to displace existing, - correct beliefs with false ones have an effect that is small and - temporary while monopolistic dominance over informational systems - has massive, enduring effects. Controlling the results to the - worldâs search queries means controlling access both to arguments - and their rebuttals and, thus, control over much of the worldâs - beliefs. If our concern is how corporations are foreclosing on our - ability to make up our own minds and determine our own futures, - the impact of dominance far exceeds the impact of manipulation and - should be central to our analysis and any remedies we seek. -

4. Bypassing our rational faculties

- This is the good stuff: using machine - learning, âdark patterns,â engagement hacking, and other - techniques to get us to do things that run counter to our better - judgment. This is mind control. -

- Some of these techniques have proven devastatingly effective (if - only in the short term). The use of countdown timers on a purchase - completion page can create a sense of urgency that causes you to - ignore the nagging internal voice suggesting that you should shop - around or sleep on your decision. The use of people from your - social graph in ads can provide âsocial proofâ that a purchase is - worth making. Even the auction system pioneered by eBay is - calculated to play on our cognitive blind spots, letting us feel - like we âownâ something because we bid on it, thus encouraging us - to bid again when we are outbid to ensure that âourâ things stay - ours. -

- Games are extraordinarily good at this. âFree to playâ games - manipulate us through many techniques, such as presenting players - with a series of smoothly escalating challenges that create a - sense of mastery and accomplishment but which sharply transition - into a set of challenges that are impossible to overcome without - paid upgrades. Add some social proof to the mix â a stream of - notifications about how well your friends are faring â and before - you know it, youâre buying virtual power-ups to get to the next - level. -

- Companies have risen and fallen on these techniques, and the - âfallenâ part is worth paying attention to. In general, living - things adapt to stimulus: Something that is very compelling or - noteworthy when you first encounter it fades with repetition until - you stop noticing it altogether. Consider the refrigerator hum - that irritates you when it starts up but disappears into the - background so thoroughly that you only notice it when it stops - again. -

- Thatâs why behavioral conditioning uses âintermittent - reinforcement schedules.â Instead of giving you a steady drip of - encouragement or setbacks, games and gamified services scatter - rewards on a randomized schedule â often enough to keep you - interested and random enough that you can never quite find the - pattern that would make it boring. -

- Intermittent reinforcement is a powerful behavioral tool, but it - also represents a collective action problem for surveillance - capitalism. The âengagement techniquesâ invented by the - behaviorists of surveillance capitalist companies are quickly - copied across the whole sector so that what starts as a - mysteriously compelling fillip in the design of a serviceâlike - âpull to refreshâ or alerts when someone likes your posts or side - quests that your characters get invited to while in the midst of - main questsâquickly becomes dully ubiquitous. The - impossible-to-nail-down nonpattern of randomized drips from your - phone becomes a grey-noise wall of sound as every single app and - site starts to make use of whatever seems to be working at the - time. -

- From the surveillance capitalistâs point of view, our adaptive - capacity is like a harmful bacterium that deprives it of its food - source â our attention â and novel techniques for snagging that - attention are like new antibiotics that can be used to breach our - defenses and destroy our self-determination. And there - are techniques like that. Who can forget the - Great Zynga Epidemic, when all of our friends were caught in - FarmVilleâs endless, mindless dopamine loops? - But every new attention-commanding technique is jumped on by the - whole industry and used so indiscriminately that antibiotic - resistance sets in. Given enough repetition, almost all of us - develop immunity to even the most powerful techniques â by 2013, - two years after Zyngaâs peak, its user base had halved. -

- Not everyone, of course. Some people never adapt to stimulus, just - as some people never stop hearing the hum of the refrigerator. - This is why most people who are exposed to slot machines play them - for a while and then move on while a small and tragic minority - liquidate their kidsâ college funds, buy adult diapers, and - position themselves in front of a machine until they collapse. -

- But surveillance capitalismâs margins on behavioral modification - suck. Tripling the rate at which someone buys a widget sounds - great - unless - the base rate is way less than 1% with an improved rate - ofâ¦ still less than 1%. Even penny slot machines pull down pennies - for every spin while surveillance capitalism rakes in - infinitesimal penny fractions. -

- Slot machinesâ high returns mean that they can be profitable just - by draining the fortunes of the small rump of people who are - pathologically vulnerable to them and unable to adapt to their - tricks. But surveillance capitalism canât survive on the - fractional pennies it brings down from that vulnerable sliver â - thatâs why, after the Great Zynga Epidemic had finally burned - itself out, the small number of still-addicted players left behind - couldnât sustain it as a global phenomenon. And new powerful - attention weapons arenât easy to find, as is evidenced by the long - years since the last time Zynga had a hit. Despite the hundreds of - millions of dollars that Zynga has to spend on developing new - tools to blast through our adaptation, it has never managed to - repeat the lucky accident that let it snag so much of our - attention for a brief moment in 2009. Powerhouses like Supercell - have fared a little better, but they are rare and throw away many - failures for every success. -

- The vulnerability of small segments of the population to dramatic, - efficient corporate manipulation is a real concern thatâs worthy - of our attention and energy. But itâs not an existential threat to - society. -

If data is the new oil, then surveillance capitalismâs engine - has a leak

- This adaptation problem offers an explanation for one of - surveillance capitalismâs most alarming traits: its relentless - hunger for data and its endless expansion of data-gathering - capabilities through the spread of sensors, online surveillance, and - acquisition of data streams from third parties. -

- Zuboff observes this phenomenon and concludes that data must be very - valuable if surveillance capitalism is so hungry for it. (In her - words: âJust as industrial capitalism was driven to the continuous - intensification of the means of production, so surveillance - capitalists and their market players are now locked into the - continuous intensification of the means of behavioral modification - and the gathering might of instrumentarian power.â) But what if the - voracious appetite is because data has such a short half-life â - because people become inured so quickly to new, data-driven - persuasion techniques â that the companies are locked in an arms - race with our limbic system? What if itâs all a Red Queenâs race - where they have to run ever faster â collect ever-more data â just - to stay in the same spot? -

- Of course, all of Big Techâs persuasion techniques work in concert - with one another, and collecting data is useful beyond mere - behavioral trickery. -

- If someone wants to recruit you to buy a refrigerator or join a - pogrom, they might use profiling and targeting to send messages to - people they judge to be good sales prospects. The messages - themselves may be deceptive, making claims about things youâre not - very knowledgeable about (food safety and energy efficiency or - eugenics and historical claims about racial superiority). They might - use search engine optimization and/or armies of fake reviewers and - commenters and/or paid placement to dominate the discourse so that - any search for further information takes you back to their messages. - And finally, they may refine the different pitches using machine - learning and other techniques to figure out what kind of pitch works - best on someone like you. -

- Each phase of this process benefits from surveillance: The more data - they have, the more precisely they can profile you and target you - with specific messages. Think of how youâd sell a fridge if you knew - that the warranty on your prospectâs fridge just expired and that - they were expecting a tax rebate in April. -

- Also, the more data they have, the better they can craft deceptive - messages â if I know that youâre into genealogy, I might not try to - feed you pseudoscience about genetic differences between âraces,â - sticking instead to conspiratorial secret histories of âdemographic - replacementâ and the like. -

- Facebook also helps you locate people who have the same odious or - antisocial views as you. It makes it possible to find other people - who want to carry tiki torches through the streets of - Charlottesville in Confederate cosplay. It can help you find other - people who want to join your militia and go to the border to look - for undocumented migrants to terrorize. It can help you find people - who share your belief that vaccines are poison and that the Earth is - flat. -

- There is one way in which targeted advertising uniquely benefits - those advocating for socially unacceptable causes: It is invisible. - Racism is widely geographically dispersed, and there are few places - where racists â and only racists â gather. This is similar to the - problem of selling refrigerators in that potential refrigerator - purchasers are geographically dispersed and there are few places - where you can buy an ad that will be primarily seen by refrigerator - customers. But buying a refrigerator is socially acceptable while - being a Nazi is not, so you can buy a billboard or advertise in the - newspaper sports section for your refrigerator business, and the - only potential downside is that your ad will be seen by a lot of - people who donât want refrigerators, resulting in a lot of wasted - expense. -

- But even if you wanted to advertise your Nazi movement on a - billboard or prime-time TV or the sports section, you would struggle - to find anyone willing to sell you the space for your ad partly - because they disagree with your views and partly because they fear - censure (boycott, reputational damage, etc.) from other people who - disagree with your views. -

- Targeted ads solve this problem: On the internet, every ad unit can - be different for every person, meaning that you can buy ads that are - only shown to people who appear to be Nazis and not to people who - hate Nazis. When thereâs spillover â when someone who hates racism - is shown a racist recruiting ad â there is some fallout; the - platform or publication might get an angry public or private - denunciation. But the nature of the risk assumed by an online ad - buyer is different than the risks to a traditional publisher or - billboard owner who might want to run a Nazi ad. -

- Online ads are placed by algorithms that broker between a diverse - ecosystem of self-serve ad platforms that anyone can buy an ad - through, so the Nazi ad that slips onto your favorite online - publication isnât seen as their moral failing but rather as a - failure in some distant, upstream ad supplier. When a publication - gets a complaint about an offensive ad thatâs appearing in one of - its units, it can take some steps to block that ad, but the Nazi - might buy a slightly different ad from a different broker serving - the same unit. And in any event, internet users increasingly - understand that when they see an ad, itâs likely that the advertiser - did not choose that publication and that the publication has no idea - who its advertisers are. -

- These layers of indirection between advertisers and publishers serve - as moral buffers: Todayâs moral consensus is largely that publishers - shouldnât be held responsible for the ads that appear on their pages - because theyâre not actively choosing to put those ads there. - Because of this, Nazis are able to overcome significant barriers to - organizing their movement. -

- Data has a complex relationship with domination. Being able to spy - on your customers can alert you to their preferences for your rivals - and allow you to head off your rivals at the pass. -

- More importantly, if you can dominate the information space while - also gathering data, then you make other deceptive tactics stronger - because itâs harder to break out of the web of deceit youâre - spinning. Domination â that is, ultimately becoming a monopoly â and - not the data itself is the supercharger that makes every tactic - worth pursuing because monopolistic domination deprives your target - of an escape route. -

- If youâre a Nazi who wants to ensure that your prospects primarily - see deceptive, confirming information when they search for more, you - can improve your odds by seeding the search terms they use through - your initial communications. You donât need to own the top 10 - results for âvoter suppressionâ if you can convince your marks to - confine their search terms to âvoter fraud,â which throws up a very - different set of search results. -

- Surveillance capitalists are like stage mentalists who claim that - their extraordinary insights into human behavior let them guess the - word that you wrote down and folded up in your pocket but who really - use shills, hidden cameras, sleight of hand, and brute-force - memorization to amaze you. -

- Or perhaps theyâre more like pick-up artists, the misogynistic cult - that promises to help awkward men have sex with women by teaching - them âneurolinguistic programmingâ phrases, body language - techniques, and psychological manipulation tactics like âneggingâ â - offering unsolicited negative feedback to women to lower their - self-esteem and prick their interest. -

- Some pick-up artists eventually manage to convince women to go home - with them, but itâs not because these men have figured out how to - bypass womenâs critical faculties. Rather, pick-up artistsâ - âsuccessâ stories are a mix of women who were incapable of giving - consent, women who were coerced, women who were intoxicated, - self-destructive women, and a few women who were sober and in - command of their faculties but who didnât realize straightaway that - they were with terrible men but rectified the error as soon as they - could. -

- Pick-up artists believe they have figured out a - secret back door that bypasses womenâs critical faculties, but they - havenât. Many of the tactics they deploy, like negging, became the - butt of jokes (just like people joke about bad ad targeting), and - thereâs a good chance that anyone they try these tactics on will - immediately recognize them and dismiss the men who use them as - irredeemable losers. -

- Pick-up artists are proof that people can believe they have - developed a system of mind control even when it doesnât - work. Pick-up artists simply exploit the fact that - one-in-a-million chances can come through for you if you make a - million attempts, and then they assume that the other 999,999 times, - they simply performed the technique incorrectly and commit - themselves to doing better next time. Thereâs only one group of - people who find pick-up artist lore reliably convincing: other - would-be pick-up artists whose anxiety and insecurity make them - vulnerable to scammers and delusional men who convince them that if - they pay for tutelage and follow instructions, then they will - someday succeed. Pick-up artists assume they fail to entice women - because they are bad at being pick-up artists, not because pick-up - artistry is bullshit. Pick-up artists are bad at selling themselves - to women, but theyâre much better at selling themselves to men who - pay to learn the secrets of pick-up artistry. -

- Department store pioneer John Wanamaker is said to have lamented, - âHalf the money I spend on advertising is wasted; the trouble is I - donât know which half.â The fact that Wanamaker thought that only - half of his advertising spending was wasted is a tribute to the - persuasiveness of advertising executives, who are - much better at convincing potential clients to - buy their services than they are at convincing the general public to - buy their clientsâ wares. -

What is Facebook?

- Facebook is heralded as the origin of all of our modern plagues, and - itâs not hard to see why. Some tech companies want to lock their - users in but make their money by monopolizing access to the market - for apps for their devices and gouging them on prices rather than by - spying on them (like Apple). Some companies donât care about locking - in users because theyâve figured out how to spy on them no matter - where they are and what theyâre doing and can turn that surveillance - into money (Google). Facebook alone among the Western tech giants - has built a business based on locking in its users - and spying on them all the time. -

- Facebookâs surveillance regime is really without parallel in the - Western world. Though Facebook tries to prevent itself from being - visible on the public web, hiding most of what goes on there from - people unless theyâre logged into Facebook, the company has - nevertheless booby-trapped the entire web with surveillance tools in - the form of Facebook âLikeâ buttons that web publishers include on - their sites to boost their Facebook profiles. Facebook also makes - various libraries and other useful code snippets available to web - publishers that act as surveillance tendrils on the sites where - theyâre used, funneling information about visitors to the site â - newspapers, dating sites, message boards â to Facebook. -

- Big Tech is able to practice surveillance not just because it is - tech but because it is big. -

- Facebook offers similar tools to app developers, so the apps â - games, fart machines, business review services, apps for keeping - abreast of your kidâs schooling â you use will send information - about your activities to Facebook even if you donât have a Facebook - account and even if you donât download or use Facebook apps. On top - of all that, Facebook buys data from third-party brokers on shopping - habits, physical location, use of âloyaltyâ programs, financial - transactions, etc., and cross-references that with the dossiers it - develops on activity on Facebook and with apps and the public web. -

- Though itâs easy to integrate the web with Facebook â linking to - news stories and such â Facebook products are generally not - available to be integrated back into the web itself. You can embed a - tweet in a Facebook post, but if you embed a Facebook post in a - tweet, you just get a link back to Facebook and must log in before - you can see it. Facebook has used extreme technological and legal - countermeasures to prevent rivals from allowing their users to embed - Facebook snippets in competing services or to create alternative - interfaces to Facebook that merge your Facebook inbox with those of - other services that you use. -

- And Facebook is incredibly popular, with 2.3 billion claimed users - (though many believe this figure to be inflated). Facebook has been - used to organize genocidal pogroms, racist riots, anti-vaccination - movements, flat Earth cults, and the political lives of some of the - worldâs ugliest, most brutal autocrats. There are some really - alarming things going on in the world, and Facebook is implicated in - many of them, so itâs easy to conclude that these bad things are the - result of Facebookâs mind-control system, which it rents out to - anyone with a few bucks to spend. -

- To understand what role Facebook plays in the formulation and - mobilization of antisocial movements, we need to understand the dual - nature of Facebook. -

- Because it has a lot of users and a lot of data about those users, - Facebook is a very efficient tool for locating people with - hard-to-find traits, the kinds of traits that are widely diffused in - the population such that advertisers have historically struggled to - find a cost-effective way to reach them. Think back to - refrigerators: Most of us only replace our major appliances a few - times in our entire lives. If youâre a refrigerator manufacturer or - retailer, you have these brief windows in the life of a consumer - during which they are pondering a purchase, and you have to somehow - reach them. Anyone whoâs ever registered a title change after buying - a house can attest that appliance manufacturers are incredibly - desperate to reach anyone who has even the slenderest chance of - being in the market for a new fridge. -

- Facebook makes finding people shopping for refrigerators a - lot easier. It can target ads to people whoâve - registered a new home purchase, to people whoâve searched for - refrigerator buying advice, to people who have complained about - their fridge dying, or any combination thereof. It can even target - people whoâve recently bought other kitchen - appliances on the theory that someone whoâs just replaced their - stove and dishwasher might be in a fridge-buying kind of mood. The - vast majority of people who are reached by these ads will not be in - the market for a new fridge, but â crucially â the percentage of - people who are looking for fridges that these - ads reach is much larger than it is than for - any group that might be subjected to traditional, offline targeted - refrigerator marketing. -

- Facebook also makes it a lot easier to find people who have the same - rare disease as you, which might have been impossible in earlier - eras â the closest fellow sufferer might otherwise be hundreds of - miles away. It makes it easier to find people who went to the same - high school as you even though decades have passed and your former - classmates have all been scattered to the four corners of the Earth. -

- Facebook also makes it much easier to find people who hold the same - rare political beliefs as you. If youâve always harbored a secret - affinity for socialism but never dared utter this aloud lest you be - demonized by your neighbors, Facebook can help you discover other - people who feel the same way (and it might just demonstrate to you - that your affinity is more widespread than you ever suspected). It - can make it easier to find people who share your sexual identity. - And again, it can help you to understand that what you thought was a - shameful secret that affected only you was really a widely shared - trait, giving you both comfort and the courage to come out to the - people in your life. -

- All of this presents a dilemma for Facebook: Targeting makes the - companyâs ads more effective than traditional ads, but it also lets - advertisers see just how effective their ads are. While advertisers - are pleased to learn that Facebook ads are more effective than ads - on systems with less sophisticated targeting, advertisers can also - see that in nearly every case, the people who see their ads ignore - them. Or, at best, the ads work on a subconscious level, creating - nebulous unmeasurables like âbrand recognition.â This means that the - price per ad is very low in nearly every case. -

- To make things worse, many Facebook groups spark precious little - discussion. Your little-league soccer team, the people with the same - rare disease as you, and the people you share a political affinity - with may exchange the odd flurry of messages at critical junctures, - but on a daily basis, thereâs not much to say to your old high - school chums or other hockey-card collectors. -

- With nothing but âorganicâ discussion, Facebook would not generate - enough traffic to sell enough ads to make the money it needs to - continually expand by buying up its competitors while returning - handsome sums to its investors. -

- So Facebook has to gin up traffic by sidetracking its own forums: - Every time Facebookâs algorithm injects controversial materials â - inflammatory political articles, conspiracy theories, outrage - stories â into a group, it can hijack that groupâs nominal purpose - with its desultory discussions and supercharge those discussions by - turning them into bitter, unproductive arguments that drag on and - on. Facebook is optimized for engagement, not happiness, and it - turns out that automated systems are pretty good at figuring out - things that people will get angry about. -

- Facebook can modify our behavior but only in a - couple of trivial ways. First, it can lock in all your friends and - family members so that you check and check and check with Facebook - to find out what they are up to; and second, it can make you angry - and anxious. It can force you to choose between being interrupted - constantly by updates â a process that breaks your concentration and - makes it hard to be introspective â and staying in touch with your - friends. This is a very limited form of mind control, and it can - only really make us miserable, angry, and anxious. -

- This is why Facebookâs targeting systems â both the ones it shows to - advertisers and the ones that let users find people who share their - interests â are so next-gen and smooth and easy to use as well as - why its message boards have a toolset that seems like it hasnât - changed since the mid-2000s. If Facebook delivered an equally - flexible, sophisticated message-reading system to its users, those - users could defend themselves against being nonconsensually - eyeball-fucked with Donald Trump headlines. -

- The more time you spend on Facebook, the more ads it gets to show - you. The solution to Facebookâs ads only working one in a thousand - times is for the company to try to increase how much time you spend - on Facebook by a factor of a thousand. Rather than thinking of - Facebook as a company that has figured out how to show you exactly - the right ad in exactly the right way to get you to do what its - advertisers want, think of it as a company that has figured out how - to make you slog through an endless torrent of arguments even though - they make you miserable, spending so much time on the site that it - eventually shows you at least one ad that you respond to. -

Monopoly and the right to the future tense

- Zuboff and her cohort are particularly alarmed at the extent to - which surveillance allows corporations to influence our decisions, - taking away something she poetically calls âthe right to the future - tenseâ â that is, the right to decide for yourself what you will do - in the future. -

- Itâs true that advertising can tip the scales one way or another: - When youâre thinking of buying a fridge, a timely fridge ad might - end the search on the spot. But Zuboff puts enormous and undue - weight on the persuasive power of surveillance-based influence - techniques. Most of these donât work very well, and the ones that do - wonât work for very long. The makers of these influence tools are - confident they will someday refine them into systems of total - control, but they are hardly unbiased observers, and the risks from - their dreams coming true are very speculative. -

- By contrast, Zuboff is rather sanguine about 40 years of lax - antitrust practice that has allowed a handful of companies to - dominate the internet, ushering in an information age with, - as - one person on Twitter noted, five giant websites each filled - with screenshots of the other four. -

- However, if we are to be alarmed that we might lose the right to - choose for ourselves what our future will hold, then monopolyâs - nonspeculative, concrete, here-and-now harms should be front and - center in our debate over tech policy. -

- Start with âdigital rights management.â In 1998, Bill Clinton signed - the Digital Millennium Copyright Act (DMCA) into law. Itâs a complex - piece of legislation with many controversial clauses but none more - so than Section 1201, the âanti-circumventionâ rule. -

- This is a blanket ban on tampering with systems that restrict access - to copyrighted works. The ban is so thoroughgoing that it prohibits - removing a copyright lock even when no copyright infringement takes - place. This is by design: The activities that the DMCAâs Section - 1201 sets out to ban are not copyright infringements; rather, they - are legal activities that frustrate manufacturersâ commercial plans. -

- For example, Section 1201âs first major application was on DVD - players as a means of enforcing the region coding built into those - devices. DVD-CCA, the body that standardized DVDs and DVD players, - divided the world into six regions and specified that DVD players - must check each disc to determine which regions it was authorized to - be played in. DVD players would have their own corresponding region - (a DVD player bought in the U.S. would be region 1 while one bought - in India would be region 5). If the player and the discâs region - matched, the player would play the disc; otherwise, it would reject - it. -

- However, watching a lawfully produced disc in a country other than - the one where you purchased it is not copyright infringement â itâs - the opposite. Copyright law imposes this duty on customers for a - movie: You must go into a store, find a licensed disc, and pay the - asking price. Do that â and nothing else â and - you and copyright are square with one another. -

- The fact that a movie studio wants to charge Indians less than - Americans or release in Australia later than it releases in the U.K. - has no bearing on copyright law. Once you lawfully acquire a DVD, it - is no copyright infringement to watch it no matter where you happen - to be. -

- So DVD and DVD player manufacturers would not be able to use - accusations of abetting copyright infringement to punish - manufacturers who made noncompliant players that would play discs - from any region or repair shops that modified players to let you - watch out-of-region discs or software programmers who created - programs to let you do this. -

- Thatâs where Section 1201 of the DMCA comes in: By banning tampering - with an âaccess control,â the rule gave manufacturers and rights - holders standing to sue competitors who released superior products - with lawful features that the market demanded (in this case, - region-free players). -

- This is an odious scam against consumers, but as time went by, - Section 1201 grew to encompass a rapidly expanding constellation of - devices and services as canny manufacturers have realized certain - things: -

- Any device with software in it contains a âcopyrighted workâ â - i.e., the software. -
- A device can be designed so that reconfiguring the software - requires bypassing an âaccess control for copyrighted works,â - which is a potential felony under Section 1201. -
- Thus, companies can control their customersâ behavior after they - take home their purchases by designing products so that all - unpermitted uses require modifications that fall afoul of - Section 1201. -

- Section 1201 then becomes a means for manufacturers of all - descriptions to force their customers to arrange their affairs to - benefit the manufacturersâ shareholders instead of themselves. -

- This manifests in many ways: from a new generation of inkjet - printers that use countermeasures to prevent third-party ink that - cannot be bypassed without legal risks to similar systems in - tractors that prevent third-party technicians from swapping in the - manufacturerâs own parts that are not recognized by the tractorâs - control system until it is supplied with a manufacturerâs unlock - code. -

- Closer to home, Appleâs iPhones use these measures to prevent both - third-party service and third-party software installation. This - allows Apple to decide when an iPhone is beyond repair and must be - shredded and landfilled as opposed to the iPhoneâs purchaser. (Apple - is notorious for its environmentally catastrophic policy of - destroying old electronics rather than permitting them to be - cannibalized for parts.) This is a very useful power to wield, - especially in light of CEO Tim Cookâs January 2019 warning to - investors that the companyâs profits are endangered by customers - choosing to hold onto their phones for longer rather than replacing - them. -

- Appleâs use of copyright locks also allows it to establish a - monopoly over how its customers acquire software for their mobile - devices. The App Storeâs commercial terms guarantee Apple a share of - all revenues generated by the apps sold there, meaning that Apple - gets paid when you buy an app from its store and then continues to - get paid every time you buy something using that app. This comes out - of the bottom line of software developers, who must either charge - more or accept lower profits for their products. -

- Crucially, Appleâs use of copyright locks gives it the power to make - editorial decisions about which apps you may and may not install on - your own device. Apple has used this power to - reject - dictionaries for containing obscene words; to - limit - political speech, especially from apps that make sensitive - political commentary such as an app that notifies you every time a - U.S. drone kills someone somewhere in the world; and to - object - to a game that commented on the Israel-Palestine conflict. -

- Apple often justifies monopoly power over software installation in - the name of security, arguing that its vetting of apps for its store - means that it can guard its users against apps that contain - surveillance code. But this cuts both ways. In China, the government - ordered - Apple to prohibit the sale of privacy tools like VPNs with - the exception of VPNs that had deliberately introduced flaws - designed to let the Chinese state eavesdrop on users. Because Apple - uses technological countermeasures â with legal backstops â to block - customers from installing unauthorized apps, Chinese iPhone owners - cannot readily (or legally) acquire VPNs that would protect them - from Chinese state snooping. -

- Zuboff calls surveillance capitalism a ârogue capitalism.â - Theoreticians of capitalism claim that its virtue is that it - aggregates - information in the form of consumersâ decisions, producing - efficient markets. Surveillance capitalismâs supposed power to rob - its victims of their free will through computationally supercharged - influence campaigns means that our markets no longer aggregate - customersâ decisions because we customers no longer decide â we are - given orders by surveillance capitalismâs mind-control rays. -

- If our concern is that markets cease to function when consumers can - no longer make choices, then copyright locks should concern us at - least as much as influence campaigns. An - influence campaign might nudge you to buy a certain brand of phone; - but the copyright locks on that phone absolutely determine where you - get it serviced, which apps can run on it, and when you have to - throw it away rather than fixing it. -

Search order and the right to the future tense

- Markets are posed as a kind of magic: By discovering otherwise - hidden information conveyed by the free choices of consumers, those - consumersâ local knowledge is integrated into a self-correcting - system that makes efficient allocationsâmore efficient than any - computer could calculate. But monopolies are incompatible with that - notion. When you only have one app store, the owner of the store â - not the consumer â decides on the range of choices. As Boss Tweed - once said, âI donât care who does the electing, so long as I get to - do the nominating.â A monopolized market is an election whose - candidates are chosen by the monopolist. -

- This ballot rigging is made more pernicious by the existence of - monopolies over search order. Googleâs search market share is about - 90%. When Googleâs ranking algorithm puts a result for a popular - search term in its top 10, that helps determine the behavior of - millions of people. If Googleâs answer to âAre vaccines dangerous?â - is a page that rebuts anti-vax conspiracy theories, then a sizable - portion of the public will learn that vaccines are safe. If, on the - other hand, Google sends those people to a site affirming the - anti-vax conspiracies, a sizable portion of those millions will come - away convinced that vaccines are dangerous. -

- Googleâs algorithm is often tricked into serving disinformation as a - prominent search result. But in these cases, Google isnât persuading - people to change their minds; itâs just presenting something untrue - as fact when the user has no cause to doubt it. -

- This is true whether the search is for âAre vaccines dangerous?â or - âbest restaurants near me.â Most users will never look past the - first page of search results, and when the overwhelming majority of - people all use the same search engine, the ranking algorithm - deployed by that search engine will determine myriad outcomes - (whether to adopt a child, whether to have cancer surgery, where to - eat dinner, where to move, where to apply for a job) to a degree - that vastly outstrips any behavioral outcomes dictated by - algorithmic persuasion techniques. -

- Many of the questions we ask search engines have no empirically - correct answers: âWhere should I eat dinner?â is not an objective - question. Even questions that do have correct answers (âAre vaccines - dangerous?â) donât have one empirically superior source for that - answer. Many pages affirm the safety of vaccines, so which one goes - first? Under conditions of competition, consumers can choose from - many search engines and stick with the one whose algorithmic - judgment suits them best, but under conditions of monopoly, we all - get our answers from the same place. -

- Googleâs search dominance isnât a matter of pure merit: The company - has leveraged many tactics that would have been prohibited under - classical, pre-Ronald-Reagan antitrust enforcement standards to - attain its dominance. After all, this is a company that has - developed two major products: a really good search engine and a - pretty good Hotmail clone. Every other major success itâs had â - Android, YouTube, Google Maps, etc. â has come through an - acquisition of a nascent competitor. Many of the companyâs key - divisions, such as the advertising technology of DoubleClick, - violate the historical antitrust principle of structural separation, - which forbade firms from owning subsidiaries that competed with - their customers. Railroads, for example, were barred from owning - freight companies that competed with the shippers whose freight they - carried. -

- If weâre worried about giant companies subverting markets by - stripping consumers of their ability to make free choices, then - vigorous antitrust enforcement seems like an excellent remedy. If - weâd denied Google the right to effect its many mergers, we would - also have probably denied it its total search dominance. Without - that dominance, the pet theories, biases, errors (and good judgment, - too) of Google search engineers and product managers would not have - such an outsized effect on consumer choice. -

- This goes for many other companies. Amazon, a classic surveillance - capitalist, is obviously the dominant tool for searching Amazon â - though many people find their way to Amazon through Google searches - and Facebook posts â and obviously, Amazon controls Amazon search. - That means that Amazonâs own self-serving editorial choicesâlike - promoting its own house brands over rival goods from its sellers as - well as its own pet theories, biases, and errorsâ determine much of - what we buy on Amazon. And since Amazon is the dominant e-commerce - retailer outside of China and since it attained that dominance by - buying up both large rivals and nascent competitors in defiance of - historical antitrust rules, we can blame the monopoly for stripping - consumers of their right to the future tense and the ability to - shape markets by making informed choices. -

- Not every monopolist is a surveillance capitalist, but that doesnât - mean theyâre not able to shape consumer choices in wide-ranging - ways. Zuboff lauds Apple for its App Store and iTunes Store, - insisting that adding price tags to the features on its platforms - has been the secret to resisting surveillance and thus creating - markets. But Apple is the only retailer allowed to sell on its - platforms, and itâs the second-largest mobile device vendor in the - world. The independent software vendors that sell through Appleâs - marketplace accuse the company of the same surveillance sins as - Amazon and other big retailers: spying on its customers to find - lucrative new products to launch, effectively using independent - software vendors as free-market researchers, then forcing them out - of any markets they discover. -

- Because of its use of copyright locks, Appleâs mobile customers are - not legally allowed to switch to a rival retailer for its apps if - they want to do so on an iPhone. Apple, obviously, is the only - entity that gets to decide how it ranks the results of search - queries in its stores. These decisions ensure that some apps are - often installed (because they appear on page one) and others are - never installed (because they appear on page one million). Appleâs - search-ranking design decisions have a vastly more significant - effect on consumer behaviors than influence campaigns delivered by - surveillance capitalismâs ad-serving bots. -

Monopolists can afford sleeping pills for watchdogs

- Only the most extreme market ideologues think that markets can - self-regulate without state oversight. Markets need watchdogs â - regulators, lawmakers, and other elements of democratic control â to - keep them honest. When these watchdogs sleep on the job, then - markets cease to aggregate consumer choices because those choices - are constrained by illegitimate and deceptive activities that - companies are able to get away with because no one is holding them - to account. -

- But this kind of regulatory capture doesnât come cheap. In - competitive sectors, where rivals are constantly eroding one - anotherâs margins, individual firms lack the surplus capital to - effectively lobby for laws and regulations that serve their ends. -

- Many of the harms of surveillance capitalism are the result of weak - or nonexistent regulation. Those regulatory vacuums spring from the - power of monopolists to resist stronger regulation and to tailor - what regulation exists to permit their existing businesses. -

- Hereâs an example: When firms over-collect and over-retain our data, - they are at increased risk of suffering a breach â you canât leak - data you never collected, and once you delete all copies of that - data, you can no longer leak it. For more than a decade, weâve lived - through an endless parade of ever-worsening data breaches, each one - uniquely horrible in the scale of data breached and the sensitivity - of that data. -

- But still, firms continue to over-collect and over-retain our data - for three reasons: -

- 1. They are locked in the aforementioned - limbic arms race with our capacity to shore up our attentional - defense systems to resist their new persuasion - techniques. Theyâre also locked in an arms race with - their competitors to find new ways to target people for sales - pitches. As soon as they discover a soft spot in our attentional - defenses (a counterintuitive, unobvious way to target potential - refrigerator buyers), the public begins to wise up to the tactic, - and their competitors leap on it, hastening the day in which all - potential refrigerator buyers have been inured to the pitch. -

- 2. They believe the surveillance capitalism - story. Data is cheap to aggregate and store, and both - proponents and opponents of surveillance capitalism have assured - managers and product designers that if you collect enough data, you - will be able to perform sorcerous acts of mind control, thus - supercharging your sales. Even if you never figure out how to profit - from the data, someone else will eventually offer to buy it from you - to give it a try. This is the hallmark of all economic bubbles: - acquiring an asset on the assumption that someone else will buy it - from you for more than you paid for it, often to sell to someone - else at an even greater price. -

- 3. The penalties for leaking data are - negligible. Most countries limit these penalties to - actual damages, meaning that consumers whoâve had their data - breached have to show actual monetary harms to get a reward. In - 2014, Home Depot disclosed that it had lost credit-card data for 53 - million of its customers, but it settled the matter by paying those - customers about $0.34 each â and a third of that $0.34 wasnât even - paid in cash. It took the form of a credit to procure a largely - ineffectual credit-monitoring service. -

- But the harms from breaches are much more extensive than these - actual-damages rules capture. Identity thieves and fraudsters are - wily and endlessly inventive. All the vast breaches of our century - are being continuously recombined, the data sets merged and mined - for new ways to victimize the people whose data was present in them. - Any reasonable, evidence-based theory of deterrence and compensation - for breaches would not confine damages to actual damages but rather - would allow users to claim these future harms. -

- However, even the most ambitious privacy rules, such as the EU - General Data Protection Regulation, fall far short of capturing the - negative externalities of the platformsâ negligent over-collection - and over-retention, and what penalties they do provide are not - aggressively pursued by regulators. -

- This tolerance of â or indifference to â data over-collection and - over-retention can be ascribed in part to the sheer lobbying muscle - of the platforms. They are so profitable that they can handily - afford to divert gigantic sums to fight any real change â that is, - change that would force them to internalize the costs of their - surveillance activities. -

- And then thereâs state surveillance, which the surveillance - capitalism story dismisses as a relic of another era when the big - worry was being jailed for your dissident speech, not having your - free will stripped away with machine learning. -

- But state surveillance and private surveillance are intimately - related. As we saw when Apple was conscripted by the Chinese - government as a vital collaborator in state surveillance, the only - really affordable and tractable way to conduct mass surveillance on - the scale practiced by modern states â both âfreeâ and autocratic - states â is to suborn commercial services. -

- Whether itâs Google being used as a location tracking tool by local - law enforcement across the U.S. or the use of social media tracking - by the Department of Homeland Security to build dossiers on - participants in protests against Immigration and Customs - Enforcementâs family separation practices, any hard limits on - surveillance capitalism would hamstring the stateâs own surveillance - capability. Without Palantir, Amazon, Google, and other major tech - contractors, U.S. cops would not be able to spy on Black people, ICE - would not be able to manage the caging of children at the U.S. - border, and state welfare systems would not be able to purge their - rolls by dressing up cruelty as empiricism and claiming that poor - and vulnerable people are ineligible for assistance. At least some - of the statesâ unwillingness to take meaningful action to curb - surveillance should be attributed to this symbiotic relationship. - There is no mass state surveillance without mass commercial - surveillance. -

- Monopolism is key to the project of mass state surveillance. Itâs - true that smaller tech firms are apt to be less well-defended than - Big Tech, whose security experts are drawn from the tops of their - field and who are given enormous resources to secure and monitor - their systems against intruders. But smaller firms also have less to - protect: fewer users whose data is more fragmented across more - systems and have to be suborned one at a time by state actors. -

- A concentrated tech sector that works with authorities is a much - more powerful ally in the project of mass state surveillance than a - fragmented one composed of smaller actors. The U.S. tech sector is - small enough that all of its top executives fit around a single - boardroom table in Trump Tower in 2017, shortly after Trumpâs - inauguration. Most of its biggest players bid to win JEDI, the - Pentagonâs $10 billion Joint Enterprise Defense Infrastructure cloud - contract. Like other highly concentrated industries, Big Tech - rotates its key employees in and out of government service, sending - them to serve in the Department of Defense and the White House, then - hiring ex-Pentagon and ex-DOD top staffers and officers to work in - their own government relations departments. -

- They can even make a good case for doing this: After all, when there - are only four or five big companies in an industry, everyone - qualified to regulate those companies has served as an executive in - at least a couple of them â because, likewise, when there are only - five companies in an industry, everyone qualified for a senior role - at any of them is by definition working at one of the other ones. -

- While surveillance doesnât cause monopolies, monopolies certainly - abet surveillance. -

- Industries that are competitive are fragmented â composed of - companies that are at each otherâs throats all the time and eroding - one anotherâs margins in bids to steal their best customers. This - leaves them with much more limited capital to use to lobby for - favorable rules and a much harder job of getting everyone to agree - to pool their resources to benefit the industry as a whole. -

- Surveillance combined with machine learning is supposed to be an - existential crisis, a species-defining moment at which our free will - is just a few more advances in the field from being stripped away. I - am skeptical of this claim, but I do think that - tech poses an existential threat to our society and possibly our - species. -

- But that threat grows out of monopoly. -

- One of the consequences of techâs regulatory capture is that it can - shift liability for poor security decisions onto its customers and - the wider society. It is absolutely normal in tech for companies to - obfuscate the workings of their products, to make them deliberately - hard to understand, and to threaten security researchers who seek to - independently audit those products. -

- IT is the only field in which this is practiced: No one builds a - bridge or a hospital and keeps the composition of the steel or the - equations used to calculate load stresses a secret. It is a frankly - bizarre practice that leads, time and again, to grotesque security - defects on farcical scales, with whole classes of devices being - revealed as vulnerable long after they are deployed in the field and - put into sensitive places. -

- The monopoly power that keeps any meaningful consequences for - breaches at bay means that tech companies continue to build terrible - products that are insecure by design and that end up integrated into - our lives, in possession of our data, and connected to our physical - world. For years, Boeing has struggled with the aftermath of a - series of bad technology decisions that made its 737 fleet a global - pariah, a rare instance in which bad tech decisions have been - seriously punished in the market. -

- These bad security decisions are compounded yet again by the use of - copyright locks to enforce business-model decisions against - consumers. Recall that these locks have become the go-to means for - shaping consumer behavior, making it technically impossible to use - third-party ink, insulin, apps, or service depots in connection with - your lawfully acquired property. -

- Recall also that these copyright locks are backstopped by - legislation (such as Section 1201 of the DMCA or Article 6 of the - 2001 EU Copyright Directive) that ban tampering with - (âcircumventingâ) them, and these statutes have been used to - threaten security researchers who make disclosures about - vulnerabilities without permission from manufacturers. -

- This amounts to a manufacturerâs veto over safety warnings and - criticism. While this is far from the legislative intent of the DMCA - and its sister statutes around the world, Congress has not - intervened to clarify the statute nor will it because to do so would - run counter to the interests of powerful, large firms whose lobbying - muscle is unstoppable. -

- Copyright locks are a double whammy: They create bad security - decisions that canât be freely investigated or discussed. If markets - are supposed to be machines for aggregating information (and if - surveillance capitalismâs notional mind-control rays are what make - it a ârogue capitalismâ because it denies consumers the power to - make decisions), then a program of legally enforced ignorance of the - risks of products makes monopolism even more of a ârogue capitalismâ - than surveillance capitalismâs influence campaigns. -

- And unlike mind-control rays, enforced silence over security is an - immediate, documented problem, and it does - constitute an existential threat to our civilization and possibly - our species. The proliferation of insecure devices â especially - devices that spy on us and especially when those devices also can - manipulate the physical world by, say, steering your car or flipping - a breaker at a power station â is a kind of technology debt. -

- In software design, âtechnology debtâ refers to old, baked-in - decisions that turn out to be bad ones in hindsight. Perhaps a - long-ago developer decided to incorporate a networking protocol made - by a vendor that has since stopped supporting it. But everything in - the product still relies on that superannuated protocol, and so, - with each revision, the product team has to work around this - obsolete core, adding compatibility layers, surrounding it with - security checks that try to shore up its defenses, and so on. These - Band-Aid measures compound the debt because every subsequent - revision has to make allowances for them, too, - like interest mounting on a predatory subprime loan. And like a - subprime loan, the interest mounts faster than you can hope to pay - it off: The product team has to put so much energy into maintaining - this complex, brittle system that they donât have any time left over - to refactor the product from the ground up and âpay off the debtâ - once and for all. -

- Typically, technology debt results in a technological bankruptcy: - The product gets so brittle and unsustainable that it fails - catastrophically. Think of the antiquated COBOL-based banking and - accounting systems that fell over at the start of the pandemic - emergency when confronted with surges of unemployment claims. - Sometimes that ends the product; sometimes it takes the company down - with it. Being caught in the default of a technology debt is scary - and traumatic, just like losing your house due to bankruptcy is - scary and traumatic. -

- But the technology debt created by copyright locks isnât individual - debt; itâs systemic. Everyone in the world is exposed to this - over-leverage, as was the case with the 2008 financial crisis. When - that debt comes due â when we face a cascade of security breaches - that threaten global shipping and logistics, the food supply, - pharmaceutical production pipelines, emergency communications, and - other critical systems that are accumulating technology debt in part - due to the presence of deliberately insecure and deliberately - unauditable copyright locks â it will indeed pose an existential - risk. -

Privacy and monopoly

- Many tech companies are gripped by an orthodoxy that holds that if - they just gather enough data on enough of our activities, everything - else is possible â the mind control and endless profits. This is an - unfalsifiable hypothesis: If data gives a tech company even a tiny - improvement in behavior prediction and modification, the company - declares that it has taken the first step toward global domination - with no end in sight. If a company fails to - attain any improvements from gathering and analyzing data, it - declares success to be just around the corner, attainable once more - data is in hand. -

- Surveillance tech is far from the first industry to embrace a - nonsensical, self-serving belief that harms the rest of the world, - and it is not the first industry to profit handsomely from such a - delusion. Long before hedge-fund managers were claiming (falsely) - that they could beat the S&P 500, there were plenty of other - ârespectableâ industries that have been revealed as quacks in - hindsight. From the makers of radium suppositories (a real thing!) - to the cruel sociopaths who claimed they could âcureâ gay people, - history is littered with the formerly respectable titans of - discredited industries. -

- This is not to say that thereâs nothing wrong with Big Tech and its - ideological addiction to data. While surveillanceâs benefits are - mostly overstated, its harms are, if anything, - understated. -

- Thereâs real irony here. The belief in surveillance capitalism as a - ârogue capitalismâ is driven by the belief that markets wouldnât - tolerate firms that are gripped by false beliefs. An oil company - that has false beliefs about where the oil is will eventually go - broke digging dry wells after all. -

- But monopolists get to do terrible things for a long time before - they pay the price. Think of how concentration in the finance sector - allowed the subprime crisis to fester as bond-rating agencies, - regulators, investors, and critics all fell under the sway of a - false belief that complex mathematics could construct âfully hedgedâ - debt instruments that could not possibly default. A small bank that - engaged in this kind of malfeasance would simply go broke rather - than outrunning the inevitable crisis, perhaps growing so big that - it averted it altogether. But large banks were able to continue to - attract investors, and when they finally did - come a-cropper, the worldâs governments bailed them out. The worst - offenders of the subprime crisis are bigger than they were in 2008, - bringing home more profits and paying their execs even larger sums. -

- Big Tech is able to practice surveillance not just because it is - tech but because it is big. The reason every - web publisher embeds a Facebook âLikeâ button is that Facebook - dominates the internetâs social media referrals â and every one of - those âLikeâ buttons spies on everyone who lands on a page that - contains them (see also: Google Analytics embeds, Twitter buttons, - etc.). -

- The reason the worldâs governments have been slow to create - meaningful penalties for privacy breaches is that Big Techâs - concentration produces huge profits that can be used to lobby - against those penalties â and Big Techâs concentration means that - the companies involved are able to arrive at a unified negotiating - position that supercharges the lobbying. -

- The reason that the smartest engineers in the world want to work for - Big Tech is that Big Tech commands the lionâs share of tech industry - jobs. -

- The reason people who are aghast at Facebookâs and Googleâs and - Amazonâs data-handling practices continue to use these services is - that all their friends are on Facebook; Google dominates search; and - Amazon has put all the local merchants out of business. -

- Competitive markets would weaken the companiesâ lobbying muscle by - reducing their profits and pitting them against each other in - regulatory forums. It would give customers other places to go to get - their online services. It would make the companies small enough to - regulate and pave the way to meaningful penalties for breaches. It - would let engineers with ideas that challenged the surveillance - orthodoxy raise capital to compete with the incumbents. It would - give web publishers multiple ways to reach audiences and make the - case against Facebook and Google and Twitter embeds. -

- In other words, while surveillance doesnât cause monopolies, - monopolies certainly abet surveillance. -

Ronald Reagan, pioneer of tech monopolism

- Technology exceptionalism is a sin, whether itâs practiced by - technologyâs blind proponents or by its critics. Both of these camps - are prone to explaining away monopolistic concentration by citing - some special characteristic of the tech industry, like network - effects or first-mover advantage. The only real difference between - these two groups is that the tech apologists say monopoly is - inevitable so we should just let tech get away with its abuses while - competition regulators in the U.S. and the EU say monopoly is - inevitable so we should punish tech for its abuses but not try to - break up the monopolies. -

- To understand how tech became so monopolistic, itâs useful to look - at the dawn of the consumer tech industry: 1979, the year the Apple - II Plus launched and became the first successful home computer. That - also happens to be the year that Ronald Reagan hit the campaign - trail for the 1980 presidential race â a race he won, leading to a - radical shift in the way that antitrust concerns are handled in - America. Reaganâs cohort of politicians â including Margaret - Thatcher in the U.K., Brian Mulroney in Canada, Helmut Kohl in - Germany, and Augusto Pinochet in Chile â went on to enact similar - reforms that eventually spread around the world. -

- Antitrustâs story began nearly a century before all that with laws - like the Sherman Act, which took aim at monopolists on the grounds - that monopolies were bad in and of themselves â squeezing out - competitors, creating âdiseconomies of scaleâ (when a company is so - big that its constituent parts go awry and it is seemingly helpless - to address the problems), and capturing their regulators to such a - degree that they can get away with a host of evils. -

- Then came a fabulist named Robert Bork, a former solicitor general - who Reagan appointed to the powerful U.S. Court of Appeals for the - D.C. Circuit and who had created an alternate legislative history of - the Sherman Act and its successors out of whole cloth. Bork insisted - that these statutes were never targeted at monopolies (despite a - wealth of evidence to the contrary, including the transcribed - speeches of the actsâ authors) but, rather, that they were intended - to prevent âconsumer harmâ â in the form of higher prices. -

- Bork was a crank, but he was a crank with a theory that rich people - really liked. Monopolies are a great way to make rich people richer - by allowing them to receive âmonopoly rentsâ (that is, bigger - profits) and capture regulators, leading to a weaker, more favorable - regulatory environment with fewer protections for customers, - suppliers, the environment, and workers. -

- Borkâs theories were especially palatable to the same power brokers - who backed Reagan, and Reaganâs Department of Justice and other - agencies began to incorporate Borkâs antitrust doctrine into their - enforcement decisions (Reagan even put Bork up for a Supreme Court - seat, but Bork flunked the Senate confirmation hearing so badly - that, 40 years later, D.C. insiders use the term âborkedâ to refer - to any catastrophically bad political performance). -

- Little by little, Borkâs theories entered the mainstream, and their - backers began to infiltrate the legal education field, even putting - on junkets where members of the judiciary were treated to lavish - meals, fun outdoor activities, and seminars where they were - indoctrinated into the consumer harm theory of antitrust. The more - Borkâs theories took hold, the more money the monopolists were - making â and the more surplus capital they had at their disposal to - lobby for even more Borkian antitrust influence campaigns. -

- The history of Borkâs antitrust theories is a really good example of - the kind of covertly engineered shifts in public opinion that Zuboff - warns us against, where fringe ideas become mainstream orthodoxy. - But Bork didnât change the world overnight. He played a very long - game, for over a generation, and he had a tailwind because the same - forces that backed oligarchic antitrust theories also backed many - other oligarchic shifts in public opinion. For example, the idea - that taxation is theft, that wealth is a sign of virtue, and so on â - all of these theories meshed to form a coherent ideology that - elevated inequality to a virtue. -

- Today, many fear that machine learning allows surveillance - capitalism to sell âBork-as-a-Service,â at internet speeds, so that - you can contract a machine-learning company to engineer - rapid shifts in public sentiment without - needing the capital to sustain a multipronged, multigenerational - project working at the local, state, national, and global levels in - business, law, and philosophy. I do not believe that such a project - is plausible, though I agree that this is basically what the - platforms claim to be selling. Theyâre just lying about it. Big Tech - lies all the time, including in their sales - literature. -

- The idea that tech forms ânatural monopoliesâ (monopolies that are - the inevitable result of the realities of an industry, such as the - monopolies that accrue the first company to run long-haul phone - lines or rail lines) is belied by techâs own history: In the absence - of anti-competitive tactics, Google was able to unseat AltaVista and - Yahoo; Facebook was able to head off Myspace. There are some - advantages to gathering mountains of data, but those mountains of - data also have disadvantages: liability (from leaking), diminishing - returns (from old data), and institutional inertia (big companies, - like science, progress one funeral at a time). -

- Indeed, the birth of the web saw a mass-extinction event for the - existing giant, wildly profitable proprietary technologies that had - capital, network effects, and walls and moats surrounding their - businesses. The web showed that when a new industry is built around - a protocol, rather than a product, the combined might of everyone - who uses the protocol to reach their customers or users or - communities outweighs even the most massive products. CompuServe, - AOL, MSN, and a host of other proprietary walled gardens learned - this lesson the hard way: Each believed it could stay separate from - the web, offering âcurationâ and a guarantee of consistency and - quality instead of the chaos of an open system. Each was wrong and - ended up being absorbed into the public web. -

- Yes, tech is heavily monopolized and is now closely associated with - industry concentration, but this has more to do with a matter of - timing than its intrinsically monopolistic tendencies. Tech was born - at the moment that antitrust enforcement was being dismantled, and - tech fell into exactly the same pathologies that antitrust was - supposed to guard against. To a first approximation, it is - reasonable to assume that techâs monopolies are the result of a lack - of anti-monopoly action and not the much-touted unique - characteristics of tech, such as network effects, first-mover - advantage, and so on. -

- In support of this thesis, I offer the concentration that every - other industry has undergone over the same - period. From professional wrestling to consumer packaged goods to - commercial property leasing to banking to sea freight to oil to - record labels to newspaper ownership to theme parks, - every industry has undergone a massive shift - toward concentration. Thereâs no obvious network effects or - first-mover advantage at play in these industries. However, in every - case, these industries attained their concentrated status through - tactics that were prohibited before Borkâs triumph: merging with - major competitors, buying out innovative new market entrants, - horizontal and vertical integration, and a suite of anti-competitive - tactics that were once illegal but are not any longer. -

- Again: When you change the laws intended to prevent monopolies and - then monopolies form in exactly the way the law was supposed to - prevent, it is reasonable to suppose that these facts are related. - Techâs concentration can be readily explained without recourse to - radical theories of network effects â but only if youâre willing to - indict unregulated markets as tending toward monopoly. Just as a - lifelong smoker can give you a hundred reasons why their smoking - didnât cause their cancer (âIt was the environmental toxinsâ), true - believers in unregulated markets have a whole suite of unconvincing - explanations for monopoly in tech that leave capitalism intact. -

Steering with the windshield wipers

- Itâs been 40 years since Borkâs project to rehabilitate monopolies - achieved liftoff, and that is a generation and a half, which is - plenty of time to take a common idea and make it seem outlandish and - vice versa. Before the 1940s, affluent Americans dressed their baby - boys in pink while baby girls wore blue (a âdelicate and daintyâ - color). While gendered colors are obviously totally arbitrary, many - still greet this news with amazement and find it hard to imagine a - time when pink connoted masculinity. -

- After 40 years of studiously ignoring antitrust analysis and - enforcement, itâs not surprising that weâve all but forgotten that - antitrust exists, that in living memory, growth through mergers and - acquisitions were largely prohibited under law, that - market-cornering strategies like vertical integration could land a - company in court. -

- Antitrust is a market societyâs steering wheel, the control of first - resort to keep would-be masters of the universe in their lanes. But - Bork and his cohort ripped out our steering wheel 40 years ago. The - car is still barreling along, and so weâre yanking as hard as we can - on all the other controls in the car as well as - desperately flapping the doors and rolling the windows up and down - in the hopes that one of these other controls can be repurposed to - let us choose where weâre heading before we careen off a cliff. -

- Itâs like a 1960s science-fiction plot come to life: People stuck in - a âgeneration ship,â plying its way across the stars, a ship once - piloted by their ancestors; and now, after a great cataclysm, the - shipâs crew have forgotten that theyâre in a ship at all and no - longer remember where the control room is. Adrift, the ship is - racing toward its extinction, and unless we can seize the controls - and execute emergency course correction, weâre all headed for a - fiery death in the heart of a sun. -

Surveillance still matters

- None of this is to minimize the problems with surveillance. - Surveillance matters, and Big Techâs use of surveillance - is an existential risk to our species, but - thatâs not because surveillance and machine learning rob us of our - free will. -

- Surveillance has become much more efficient - thanks to Big Tech. In 1989, the Stasi â the East German secret - police â had the whole country under surveillance, a massive - undertaking that recruited one out of every 60 people to serve as an - informant or intelligence operative. -

- Today, we know that the NSA is spying on a significant fraction of - the entire worldâs population, and its ratio of surveillance - operatives to the surveilled is more like 1:10,000 (thatâs probably - on the low side since it assumes that every American with top-secret - clearance is working for the NSA on this project â we donât know how - many of those cleared people are involved in NSA spying, but itâs - definitely not all of them). -

- How did the ratio of surveillable citizens expand from 1:60 to - 1:10,000 in less than 30 years? Itâs thanks to Big Tech. Our devices - and services gather most of the data that the NSA mines for its - surveillance project. We pay for these devices and the services they - connect to, and then we painstakingly perform the data-entry tasks - associated with logging facts about our lives, opinions, and - preferences. This mass surveillance project has been largely useless - for fighting terrorism: The NSA can - only - point to a single minor success story in which it used its - data collection program to foil an attempt by a U.S. resident to - wire a few thousand dollars to an overseas terror group. Itâs - ineffective for much the same reason that commercial surveillance - projects are largely ineffective at targeting advertising: The - people who want to commit acts of terror, like people who want to - buy a refrigerator, are extremely rare. If youâre trying to detect a - phenomenon whose base rate is one in a million with an instrument - whose accuracy is only 99%, then every true positive will come at - the cost of 9,999 false positives. -

- Let me explain that again: If one in a million people is a - terrorist, then there will only be about one terrorist in a random - sample of one million people. If your test for detecting terrorists - is 99% accurate, it will identify 10,000 terrorists in your - million-person sample (1% of one million is 10,000). For every true - positive, youâll get 9,999 false positives. -

- In reality, the accuracy of algorithmic terrorism detection falls - far short of the 99% mark, as does refrigerator ad targeting. The - difference is that being falsely accused of wanting to buy a fridge - is a minor nuisance while being falsely accused of planning a terror - attack can destroy your life and the lives of everyone you love. -

- Mass state surveillance is only feasible because of surveillance - capitalism and its extremely low-yield ad-targeting systems, which - require a constant feed of personal data to remain barely viable. - Surveillance capitalismâs primary failure mode is mistargeted ads - while mass state surveillanceâs primary failure mode is grotesque - human rights abuses, tending toward totalitarianism. -

- State surveillance is no mere parasite on Big Tech, sucking up its - data and giving nothing in return. In truth, the two are symbiotes: - Big Tech sucks up our data for spy agencies, and spy agencies ensure - that governments donât limit Big Techâs activities so severely that - it would no longer serve the spy agenciesâ needs. There is no firm - distinction between state surveillance and surveillance capitalism; - they are dependent on one another. -

- To see this at work today, look no further than Amazonâs home - surveillance device, the Ring doorbell, and its associated app, - Neighbors. Ring â a product that Amazon acquired and did not develop - in house â makes a camera-enabled doorbell that streams footage from - your front door to your mobile device. The Neighbors app allows you - to form a neighborhood-wide surveillance grid with your fellow Ring - owners through which you can share clips of âsuspicious characters.â - If youâre thinking that this sounds like a recipe for letting - curtain-twitching racists supercharge their suspicions of people - with brown skin who walk down their blocks, - youâre - right. Ring has become a de facto, - off-the-books arm of the police without any of the pesky oversight - or rules. -

- In mid-2019, a series of public records requests revealed that - Amazon had struck confidential deals with more than 400 local law - enforcement agencies through which the agencies would promote Ring - and Neighbors and in exchange get access to footage from Ring - cameras. In theory, cops would need to request this footage through - Amazon (and internal documents reveal that Amazon devotes - substantial resources to coaching cops on how to spin a convincing - story when doing so), but in practice, when a Ring customer turns - down a police request, Amazon only requires the agency to formally - request the footage from the company, which it will then produce. -

- Ring and law enforcement have found many ways to intertwine their - activities. Ring strikes secret deals to acquire real-time access to - 911 dispatch and then streams alarming crime reports to Neighbors - users, which serve as convincers for anyone whoâs contemplating a - surveillance doorbell but isnât sure whether their neighborhood is - dangerous enough to warrant it. -

- The more the cops buzz-market the surveillance capitalist Ring, the - more surveillance capability the state gets. Cops who rely on - private entities for law-enforcement roles then brief against any - controls on the deployment of that technology while the companies - return the favor by lobbying against rules requiring public - oversight of police surveillance technology. The more the cops rely - on Ring and Neighbors, the harder it will be to pass laws to curb - them. The fewer laws there are against them, the more the cops will - rely on them. -

Dignity and sanctuary

- But even if we could exercise democratic control over our states and - force them to stop raiding surveillance capitalismâs reservoirs of - behavioral data, surveillance capitalism would still harm us. -

- This is an area where Zuboff shines. Her chapter on âsanctuaryâ â - the feeling of being unobserved â is a beautiful hymn to - introspection, calmness, mindfulness, and tranquility. -

- When you are watched, something changes. Anyone who has ever raised - a child knows this. You might look up from your book (or more - realistically, from your phone) and catch your child in a moment of - profound realization and growth, a moment where they are learning - something that is right at the edge of their abilities, requiring - their entire ferocious concentration. For a moment, youâre - transfixed, watching that rare and beautiful moment of focus playing - out before your eyes, and then your child looks up and sees you - seeing them, and the moment collapses. To grow, you need to be and - expose your authentic self, and in that moment, you are vulnerable - like a hermit crab scuttling from one shell to the next. The tender, - unprotected tissues you expose in that moment are too delicate to - reveal in the presence of another, even someone you trust as - implicitly as a child trusts their parent. -

- In the digital age, our authentic selves are inextricably tied to - our digital lives. Your search history is a running ledger of the - questions youâve pondered. Your location history is a record of the - places youâve sought out and the experiences youâve had there. Your - social graph reveals the different facets of your identity, the - people youâve connected with. -

- To be observed in these activities is to lose the sanctuary of your - authentic self. -

- Thereâs another way in which surveillance capitalism robs us of our - capacity to be our authentic selves: by making us anxious. - Surveillance capitalism isnât really a mind-control ray, but you - donât need a mind-control ray to make someone anxious. After all, - another word for anxiety is agitation, and to make someone - experience agitation, you need merely to agitate them. To poke them - and prod them and beep at them and buzz at them and bombard them on - an intermittent schedule that is just random enough that our limbic - systems never quite become inured to it. -

- Our devices and services are âgeneral purposeâ in that they can - connect anything or anyone to anything or anyone else and that they - can run any program that can be written. This means that the - distraction rectangles in our pockets hold our most precious moments - with our most beloved people and their most urgent or time-sensitive - communications (from ârunning late can you get the kid?â to âdoctor - gave me bad news and I need to talk to you RIGHT NOWâ) as well as - ads for refrigerators and recruiting messages from Nazis. -

- All day and all night, our pockets buzz, shattering our - concentration and tearing apart the fragile webs of connection we - spin as we think through difficult ideas. If you locked someone in a - cell and agitated them like this, weâd call it âsleep deprivation - torture,â and it would be - a war crime - under the Geneva Conventions. -

Afflicting the afflicted

- The effects of surveillance on our ability to be our authentic - selves are not equal for all people. Some of us are lucky enough to - live in a time and place in which all the most important facts of - our lives are widely and roundly socially acceptable and can be - publicly displayed without the risk of social consequence. -

- But for many of us, this is not true. Recall that in living memory, - many of the ways of being that we think of as socially acceptable - today were once cause for dire social sanction or even imprisonment. - If you are 65 years old, you have lived through a time in which - people living in âfree societiesâ could be imprisoned or sanctioned - for engaging in homosexual activity, for falling in love with a - person whose skin was a different color than their own, or for - smoking weed. -

- Today, these activities arenât just decriminalized in much of the - world, theyâre considered normal, and the fallen prohibitions are - viewed as shameful, regrettable relics of the past. -

- How did we get from prohibition to normalization? Through private, - personal activity: People who were secretly gay or secret - pot-smokers or who secretly loved someone with a different skin - color were vulnerable to retaliation if they made their true selves - known and were limited in how much they could advocate for their own - right to exist in the world and be true to themselves. But because - there was a private sphere, these people could form alliances with - their friends and loved ones who did not share their disfavored - traits by having private conversations in which they came out, - disclosing their true selves to the people around them and bringing - them to their cause one conversation at a time. -

- The right to choose the time and manner of these conversations was - key to their success. Itâs one thing to come out to your dad while - youâre on a fishing trip away from the world and another thing - entirely to blurt it out over the Christmas dinner table while your - racist Facebook uncle is there to make a scene. -

- Without a private sphere, thereâs a chance that none of these - changes would have come to pass and that the people who benefited - from these changes would have either faced social sanction for - coming out to a hostile world or would have never been able to - reveal their true selves to the people they love. -

- The corollary is that, unless you think that our society has - attained social perfection â that your grandchildren in 50 years - will ask you to tell them the story of how, in 2020, every injustice - had been righted and no further change had to be made â then you - should expect that right now, at this minute, there are people you - love, whose happiness is key to your own, who have a secret in their - hearts that stops them from ever being their authentic selves with - you. These people are sorrowing and will go to their graves with - that secret sorrow in their hearts, and the source of that sorrow - will be the falsity of their relationship to you. -

- A private realm is necessary for human progress. -

Any data you collect and retain will eventually leak

- The lack of a private life can rob vulnerable people of the chance - to be their authentic selves and constrain our actions by depriving - us of sanctuary, but there is another risk that is borne by - everyone, not just people with a secret: crime. -

- Personally identifying information is of very limited use for the - purpose of controlling peoplesâ minds, but identity theft â really a - catchall term for a whole constellation of terrible criminal - activities that can destroy your finances, compromise your personal - integrity, ruin your reputation, or even expose you to physical - danger â thrives on it. -

- Attackers are not limited to using data from one breached source, - either. Multiple services have suffered breaches that exposed names, - addresses, phone numbers, passwords, sexual tastes, school grades, - work performance, brushes with the criminal justice system, family - details, genetic information, fingerprints and other biometrics, - reading habits, search histories, literary tastes, pseudonymous - identities, and other sensitive information. Attackers can merge - data from these different breaches to build up extremely detailed - dossiers on random subjects and then use different parts of the data - for different criminal purposes. -

- For example, attackers can use leaked username and password - combinations to hijack whole fleets of commercial vehicles that - have - been fitted with anti-theft GPS trackers and immobilizers or - to hijack baby monitors in order to - terrorize - toddlers with the audio tracks from pornography. Attackers - use leaked data to trick phone companies into giving them your phone - number, then they intercept SMS-based two-factor authentication - codes in order to take over your email, bank account, and/or - cryptocurrency wallets. -

- Attackers are endlessly inventive in the pursuit of creative ways to - weaponize leaked data. One common use of leaked data is to penetrate - companies in order to access more data. -

- Like spies, online fraudsters are totally dependent on companies - over-collecting and over-retaining our data. Spy agencies sometimes - pay companies for access to their data or intimidate them into - giving it up, but sometimes they work just like criminals do â by - sneaking - data out of companiesâ databases. -

- The over-collection of data has a host of terrible social - consequences, from the erosion of our authentic selves to the - undermining of social progress, from state surveillance to an - epidemic of online crime. Commercial surveillance is also a boon to - people running influence campaigns, but thatâs the least of our - troubles. -

Critical tech exceptionalism is still tech - exceptionalism

- Big Tech has long practiced technology exceptionalism: the idea that - it should not be subject to the mundane laws and norms of - âmeatspace.â Mottoes like Facebookâs âmove fast and break thingsâ - attracted justifiable scorn of the companiesâ self-serving rhetoric. -

- Tech exceptionalism got us all into a lot of trouble, so itâs ironic - and distressing to see Big Techâs critics committing the same sin. -

- Big Tech is not a ârogue capitalismâ that cannot be cured through - the traditional anti-monopoly remedies of trustbusting (forcing - companies to divest of competitors they have acquired) and bans on - mergers to monopoly and other anti-competitive tactics. Big Tech - does not have the power to use machine learning to influence our - behavior so thoroughly that markets lose the ability to punish bad - actors and reward superior competitors. Big Tech has no rule-writing - mind-control ray that necessitates ditching our old toolbox. -

- The thing is, people have been claiming to have perfected - mind-control rays for centuries, and every time, it turned out to be - a con â though sometimes the con artists were also conning - themselves. -

- For generations, the advertising industry has been steadily - improving its ability to sell advertising services to businesses - while only making marginal gains in selling those businessesâ - products to prospective customers. John Wanamakerâs lament that â50% - of my advertising budget is wasted, I just donât know which 50%â is - a testament to the triumph of ad executives, - who successfully convinced Wanamaker that only half of the money he - spent went to waste. -

- The tech industry has made enormous improvements in the science of - convincing businesses that theyâre good at advertising while their - actual improvements to advertising â as opposed to targeting â have - been pretty ho-hum. The vogue for machine learning â and the - mystical invocation of âartificial intelligenceâ as a synonym for - straightforward statistical inference techniques â has greatly - boosted the efficacy of Big Techâs sales pitch as marketers have - exploited potential customersâ lack of technical sophistication to - get away with breathtaking acts of overpromising and - underdelivering. -

- Itâs tempting to think that if businesses are willing to pour - billions into a venture that the venture must be a good one. Yet - there are plenty of times when this rule of thumb has led us astray. - For example, itâs virtually unheard of for managed investment funds - to outperform simple index funds, and investors who put their money - into the hands of expert money managers overwhelmingly fare worse - than those who entrust their savings to index funds. But managed - funds still account for the majority of the money invested in the - markets, and they are patronized by some of the richest, most - sophisticated investors in the world. Their vote of confidence in an - underperforming sector is a parable about the role of luck in wealth - accumulation, not a sign that managed funds are a good buy. -

- The claims of Big Techâs mind-control system are full of tells that - the enterprise is a con. For example, - the - reliance on the âBig Fiveâ personality traits as a primary - means of influencing people even though the âBig Fiveâ theory is - unsupported by any large-scale, peer-reviewed studies and is - mostly - the realm of marketing hucksters and pop psych. -

- Big Techâs promotional materials also claim that their algorithms - can accurately perform âsentiment analysisâ or detect peoplesâ moods - based on their âmicroexpressions,â but - these - are marketing claims, not scientific ones. These methods are - largely untested by independent scientific experts, and where they - have been tested, theyâve been found sorely wanting. - Microexpressions are particularly suspect as the companies that - specialize in training people to detect them - have - been shown to underperform relative to random chance. -

- Big Tech has been so good at marketing its own supposed superpowers - that itâs easy to believe that they can market everything else with - similar acumen, but itâs a mistake to believe the hype. Any - statement a company makes about the quality of its products is - clearly not impartial. The fact that we distrust all the things that - Big Tech says about its data handling, compliance with privacy laws, - etc., is only reasonable â but why on Earth would we treat Big - Techâs marketing literature as the gospel truth? Big Tech lies about - just about everything, including how well its - machine-learning fueled persuasion systems work. -

- That skepticism should infuse all of our evaluations of Big Tech and - its supposed abilities, including our perusal of its patents. Zuboff - vests these patents with enormous significance, pointing out that - Google claimed extensive new persuasion capabilities in - its - patent filings. These claims are doubly suspect: first, - because they are so self-serving, and second, because the patent - itself is so notoriously an invitation to exaggeration. -

- Patent applications take the form of a series of claims and range - from broad to narrow. A typical patent starts out by claiming that - its authors have invented a method or system for doing every - conceivable thing that anyone might do, ever, with any tool or - device. Then it narrows that claim in successive stages until we get - to the actual âinventionâ that is the true subject of the patent. - The hope is that the patent examiner â who is almost certainly - overworked and underinformed â will miss the fact that some or all - of these claims are ridiculous, or at least suspect, and grant the - patentâs broader claims. Patents for unpatentable things are still - incredibly useful because they can be wielded against competitors - who might license that patent or steer clear of its claims rather - than endure the lengthy, expensive process of contesting it. -

- Whatâs more, software patents are routinely granted even though the - filer doesnât have any evidence that they can do the thing claimed - by the patent. That is, you can patent an âinventionâ that you - havenât actually made and that you donât know how to make. -

- With these considerations in hand, it becomes obvious that the fact - that a Big Tech company has patented what it - says is an effective mind-control ray is - largely irrelevant to whether Big Tech can in fact control our - minds. -

- Big Tech collects our data for many reasons, including the - diminishing returns on existing stores of data. But many tech - companies also collect data out of a mistaken tech exceptionalist - belief in the network effects of data. Network effects occur when - each new user in a system increases its value. The classic example - is fax machines: A single fax machine is of no use, two fax machines - are of limited use, but every new fax machine thatâs put to use - after the first doubles the number of possible fax-to-fax links. -

- Data mined for predictive systems doesnât necessarily produce these - dividends. Think of Netflix: The predictive value of the data mined - from a million English-speaking Netflix viewers is hardly improved - by the addition of one more userâs viewing data. Most of the data - Netflix acquires after that first minimum viable sample duplicates - existing data and produces only minimal gains. Meanwhile, retraining - models with new data gets progressively more expensive as the number - of data points increases, and manual tasks like labeling and - validating data do not get cheaper at scale. -

- Businesses pursue fads to the detriment of their profits all the - time, especially when the businesses and their investors are not - motivated by the prospect of becoming profitable but rather by the - prospect of being acquired by a Big Tech giant or by having an IPO. - For these firms, ticking faddish boxes like âcollects as much data - as possibleâ might realize a bigger return on investment than - âcollects a business-appropriate quantity of data.â -

- This is another harm of tech exceptionalism: The belief that more - data always produces more profits in the form of more insights that - can be translated into better mind-control rays drives firms to - over-collect and over-retain data beyond all rationality. And since - the firms are behaving irrationally, a good number of them will go - out of business and become ghost ships whose cargo holds are stuffed - full of data that can harm people in myriad ways â but which no one - is responsible for antey longer. Even if the companies donât go - under, the data they collect is maintained behind the minimum viable - security â just enough security to keep the company viable while it - waits to get bought out by a tech giant, an amount calculated to - spend not one penny more than is necessary on protecting data. -

How monopolies, not mind control, drive surveillance - capitalism: The Snapchat story

- For the first decade of its existence, Facebook competed with the - social media giants of the day (Myspace, Orkut, etc.) by presenting - itself as the pro-privacy alternative. Indeed, Facebook justified - its walled garden â which let users bring in data from the web but - blocked web services like Google Search from indexing and caching - Facebook pages â as a pro-privacy measure that protected users from - the surveillance-happy winners of the social media wars like - Myspace. -

- Despite frequent promises that it would never collect or analyze its - usersâ data, Facebook periodically created initiatives that did just - that, like the creepy, ham-fisted Beacon tool, which spied on you as - you moved around the web and then added your online activities to - your public timeline, allowing your friends to monitor your browsing - habits. Beacon sparked a user revolt. Every time, Facebook backed - off from its surveillance initiative, but not all the way; - inevitably, the new Facebook would be more surveilling than the old - Facebook, though not quite as surveilling as the intermediate - Facebook following the launch of the new product or service. -

- The pace at which Facebook ramped up its surveillance efforts seems - to have been set by Facebookâs competitive landscape. The more - competitors Facebook had, the better it behaved. Every time a major - competitor foundered, Facebookâs behavior - got - markedly worse. -

- All the while, Facebook was prodigiously acquiring companies, - including a company called Onavo. Nominally, Onavo made a - battery-monitoring mobile app. But the permissions that Onavo - required were so expansive that the app was able to gather - fine-grained telemetry on everything users did with their phones, - including which apps they used and how they were using them. -

- Through Onavo, Facebook discovered that it was losing market share - to Snapchat, an app that â like Facebook a decade before â billed - itself as the pro-privacy alternative to the status quo. Through - Onavo, Facebook was able to mine data from the devices of Snapchat - users, including both current and former Snapchat users. This - spurred Facebook to acquire Instagram â some features of which - competed with Snapchat â and then allowed Facebook to fine-tune - Instagramâs features and sales pitch to erode Snapchatâs gains and - ensure that Facebook would not have to face the kinds of competitive - pressures it had earlier inflicted on Myspace and Orkut. -

- The story of how Facebook crushed Snapchat reveals the relationship - between monopoly and surveillance capitalism. Facebook combined - surveillance with lax antitrust enforcement to spot the competitive - threat of Snapchat on its horizon and then take decisive action - against it. Facebookâs surveillance capitalism let it avert - competitive pressure with anti-competitive tactics. Facebook users - still want privacy â Facebook hasnât used surveillance to brainwash - them out of it â but they canât get it because Facebookâs - surveillance lets it destroy any hope of a rival service emerging - that competes on privacy features. -

A monopoly over your friends

- A decentralization movement has tried to erode the dominance of - Facebook and other Big Tech companies by fielding âindiewebâ - alternatives â Mastodon as a Twitter alternative, Diaspora as a - Facebook alternative, etc. â but these efforts have failed to attain - any kind of liftoff. -

- Fundamentally, each of these services is hamstrung by the same - problem: Every potential user for a Facebook or Twitter alternative - has to convince all their friends to follow them to a decentralized - web alternative in order to continue to realize the benefit of - social media. For many of us, the only reason to have a Facebook - account is that our friends have Facebook accounts, and the reason - they have Facebook accounts is that we have - Facebook accounts. -

- All of this has conspired to make Facebook â and other dominant - platforms â into âkill zonesâ that investors will not fund new - entrants for. -

- And yet, all of todayâs tech giants came into existence despite the - entrenched advantage of the companies that came before them. To - understand how that happened, you have to understand both - interoperability and adversarial interoperability. -

- The hard problem of our species is coordination. -

- âInteroperabilityâ is the ability of two technologies to work with - one another: Anyone can make an LP that will play on any record - player, anyone can make a filter you can install in your stoveâs - extractor fan, anyone can make gasoline for your car, anyone can - make a USB phone charger that fits in your carâs cigarette lighter - receptacle, anyone can make a light bulb that works in your light - socket, anyone can make bread that will toast in your toaster. -

- Interoperability is often a source of innovation and consumer - benefit: Apple made the first commercially successful PC, but - millions of independent software vendors made interoperable programs - that ran on the Apple II Plus. The simple analog antenna inputs on - the back of TVs first allowed cable operators to connect directly to - TVs, then they allowed game console companies and then personal - computer companies to use standard televisions as displays. Standard - RJ-11 telephone jacks allowed for the production of phones from a - variety of vendors in a variety of forms, from the free - football-shaped phone that came with a Sports - Illustrated subscription to business phones with - speakers, hold functions, and so on and then answering machines and - finally modems, paving the way for the internet revolution. -

- âInteroperabilityâ is often used interchangeably with - âstandardization,â which is the process when manufacturers and other - stakeholders hammer out a set of agreed-upon rules for implementing - a technology, such as the electrical plug on your wall, the CAN bus - used by your carâs computer systems, or the HTML instructions that - your browser interprets. -

- But interoperability doesnât require standardization â indeed, - standardization often proceeds from the chaos of ad hoc - interoperability measures. The inventor of the cigarette-lighter USB - charger didnât need to get permission from car manufacturers or even - the manufacturers of the dashboard lighter subcomponent. The - automakers didnât take any countermeasures to prevent the use of - these aftermarket accessories by their customers, but they also - didnât do anything to make life easier for the chargersâ - manufacturers. This is a kind of âneutral interoperability.â -

- Beyond neutral interoperability, there is âadversarial - interoperability.â Thatâs when a manufacturer makes a product that - interoperates with another manufacturerâs product despite - the second manufacturerâs objections and even - if that means bypassing a security system designed to prevent - interoperability. -

- Probably the most familiar form of adversarial interoperability is - third-party printer ink. Printer manufacturers claim that they sell - printers below cost and that the only way they can recoup the losses - they incur is by charging high markups on ink. To prevent the owners - of printers from buying ink elsewhere, the printer companies deploy - a suite of anti-customer security systems that detect and reject - both refilled and third-party cartridges. -

- Owners of printers take the position that HP and Epson and Brother - are not charities and that customers for their wares have no - obligation to help them survive, and so if the companies choose to - sell their products at a loss, thatâs their foolish choice and their - consequences to live with. Likewise, competitors who make ink or - refill kits observe that they donât owe printer companies anything, - and their erosion of printer companiesâ margins are the printer - companiesâ problems, not their competitorsâ. After all, the printer - companies shed no tears when they drive a refiller out of business, - so why should the refillers concern themselves with the economic - fortunes of the printer companies? -

- Adversarial interoperability has played an outsized role in the - history of the tech industry: from the founding of the âalt.*â - Usenet hierarchy (which was started against the wishes of Usenetâs - maintainers and which grew to be bigger than all of Usenet combined) - to the browser wars (when Netscape and Microsoft devoted massive - engineering efforts to making their browsers incompatible with the - otherâs special commands and peccadilloes) to Facebook (whose - success was built in part by helping its new users stay in touch - with friends theyâd left behind on Myspace because Facebook supplied - them with a tool that scraped waiting messages from Myspace and - imported them into Facebook, effectively creating an Facebook-based - Myspace reader). -

- Today, incumbency is seen as an unassailable advantage. Facebook is - where all of your friends are, so no one can start a Facebook - competitor. But adversarial compatibility reverses the competitive - advantage: If you were allowed to compete with Facebook by providing - a tool that imported all your usersâ waiting Facebook messages into - an environment that competed on lines that Facebook couldnât cross, - like eliminating surveillance and ads, then Facebook would be at a - huge disadvantage. It would have assembled all possible ex-Facebook - users into a single, easy-to-find service; it would have educated - them on how a Facebook-like service worked and what its potential - benefits were; and it would have provided an easy means for - disgruntled Facebook users to tell their friends where they might - expect better treatment. -

- Adversarial interoperability was once the norm and a key contributor - to the dynamic, vibrant tech scene, but now it is stuck behind a - thicket of laws and regulations that add legal risks to the - tried-and-true tactics of adversarial interoperability. New rules - and new interpretations of existing rules mean that a would-be - adversarial interoperator needs to steer clear of claims under - copyright, terms of service, trade secrecy, tortious interference, - and patent. -

- In the absence of a competitive market, lawmakers have resorted to - assigning expensive, state-like duties to Big Tech firms, such as - automatically filtering user contributions for copyright - infringement or terrorist and extremist content or detecting and - preventing harassment in real time or controlling access to sexual - material. -

- These measures put a floor under how small we can make Big Tech - because only the very largest companies can afford the humans and - automated filters needed to perform these duties. -

- But thatâs not the only way in which making platforms responsible - for policing their users undermines competition. A platform that is - expected to police its usersâ conduct must prevent many vital - adversarial interoperability techniques lest these subvert its - policing measures. For example, if someone using a Twitter - replacement like Mastodon is able to push messages into Twitter and - read messages out of Twitter, they could avoid being caught by - automated systems that detect and prevent harassment (such as - systems that use the timing of messages or IP-based rules to make - guesses about whether someone is a harasser). -

- To the extent that we are willing to let Big Tech police itself â - rather than making Big Tech small enough that users can leave bad - platforms for better ones and small enough that a regulation that - simply puts a platform out of business will not destroy billions of - usersâ access to their communities and data â we build the case that - Big Tech should be able to block its competitors and make it easier - for Big Tech to demand legal enforcement tools to ban and punish - attempts at adversarial interoperability. -

- Ultimately, we can try to fix Big Tech by making it responsible for - bad acts by its users, or we can try to fix the internet by cutting - Big Tech down to size. But we canât do both. To replace todayâs - giant products with pluralistic protocols, we need to clear the - legal thicket that prevents adversarial interoperability so that - tomorrowâs nimble, personal, small-scale products can federate - themselves with giants like Facebook, allowing the users whoâve left - to continue to communicate with users who havenât left yet, reaching - tendrils over Facebookâs garden wall that Facebookâs trapped users - can use to scale the walls and escape to the global, open web. -

Fake news is an epistemological crisis

- Tech is not the only industry that has undergone massive - concentration since the Reagan era. Virtually every major industry â - from oil to newspapers to meatpacking to sea freight to eyewear to - online pornography â has become a clubby oligarchy that just a few - players dominate. -

- At the same time, every industry has become something of a tech - industry as general-purpose computers and general-purpose networks - and the promise of efficiencies through data-driven analysis infuse - every device, process, and firm with tech. -

- This phenomenon of industrial concentration is part of a wider story - about wealth concentration overall as a smaller and smaller number - of people own more and more of our world. This concentration of both - wealth and industries means that our political outcomes are - increasingly beholden to the parochial interests of the people and - companies with all the money. -

- That means that whenever a regulator asks a question with an - obvious, empirical answer (âAre humans causing climate change?â or - âShould we let companies conduct commercial mass surveillance?â or - âDoes society benefit from allowing network neutrality - violations?â), the answer that comes out is only correct if that - correctness meets with the approval of rich people and the - industries that made them so wealthy. -

- Rich people have always played an outsized role in politics and more - so since the Supreme Courtâs Citizens United - decision eliminated key controls over political spending. Widening - inequality and wealth concentration means that the very richest - people are now a lot richer and can afford to spend a lot more money - on political projects than ever before. Think of the Koch brothers - or George Soros or Bill Gates. -

- But the policy distortions of rich individuals pale in comparison to - the policy distortions that concentrated industries are capable of. - The companies in highly concentrated industries are much more - profitable than companies in competitive industries â no competition - means not having to reduce prices or improve quality to win - customers â leaving them with bigger capital surpluses to spend on - lobbying. -

- Concentrated industries also find it easier to collaborate on policy - objectives than competitive ones. When all the top execs from your - industry can fit around a single boardroom table, they often do. And - when they do, they can forge a consensus - position on regulation. -

- Rising through the ranks in a concentrated industry generally means - working at two or three of the big companies. When there are only - relatively few companies in a given industry, each company has a - more ossified executive rank, leaving ambitious execs with fewer - paths to higher positions unless they are recruited to a rival. This - means that the top execs in concentrated industries are likely to - have been colleagues at some point and socialize in the same circles - â connected through social ties or, say, serving as trustees for - each othersâ estates. These tight social bonds foster a collegial, - rather than competitive, attitude. -

- Highly concentrated industries also present a regulatory conundrum. - When an industry is dominated by just four or five companies, the - only people who are likely to truly understand the industryâs - practices are its veteran executives. This means that top regulators - are often former execs of the companies they are supposed to be - regulating. These turns in government are often tacitly understood - to be leaves of absence from industry, with former employers - welcoming their erstwhile watchdogs back into their executive ranks - once their terms have expired. -

- All this is to say that the tight social bonds, small number of - firms, and regulatory capture of concentrated industries give the - companies that comprise them the power to dictate many, if not all, - of the regulations that bind them. -

- This is increasingly obvious. Whether itâs payday lenders - winning - the right to practice predatory lending or Apple - winning - the right to decide who can fix your phone or Google and - Facebook winning the right to breach your private data without - suffering meaningful consequences or victories for pipeline - companies or impunity for opioid manufacturers or massive tax - subsidies for incredibly profitable dominant businesses, itâs - increasingly apparent that many of our official, evidence-based - truth-seeking processes are, in fact, auctions for sale to the - highest bidder. -

- Itâs really impossible to overstate what a terrifying prospect this - is. We live in an incredibly high-tech society, and none of us could - acquire the expertise to evaluate every technological proposition - that stands between us and our untimely, horrible deaths. You might - devote your life to acquiring the media literacy to distinguish good - scientific journals from corrupt pay-for-play lookalikes and the - statistical literacy to evaluate the quality of the analysis in the - journals as well as the microbiology and epidemiology knowledge to - determine whether you can trust claims about the safety of vaccines - â but that would still leave you unqualified to judge whether the - wiring in your home will give you a lethal shock - and whether your carâs brakesâ software will - cause them to fail unpredictably and whether - the hygiene standards at your butcher are sufficient to keep you - from dying after you finish your dinner. -

- In a world as complex as this one, we have to defer to authorities, - and we keep them honest by making those authorities accountable to - us and binding them with rules to prevent conflicts of interest. We - canât possibly acquire the expertise to adjudicate conflicting - claims about the best way to make the world safe and prosperous, but - we can determine whether the adjudication - process itself is trustworthy. -

- Right now, itâs obviously not. -

- The past 40 years of rising inequality and industry concentration, - together with increasingly weak accountability and transparency for - expert agencies, has created an increasingly urgent sense of - impending doom, the sense that there are vast conspiracies afoot - that operate with tacit official approval despite the likelihood - they are working to better themselves by ruining the rest of us. -

- For example, itâs been decades since Exxonâs own scientists - concluded that its products would render the Earth uninhabitable by - humans. And yet those decades were lost to us, in large part because - Exxon lobbied governments and sowed doubt about the dangers of its - products and did so with the cooperation of many public officials. - When the survival of you and everyone you love is threatened by - conspiracies, itâs not unreasonable to start questioning the things - you think you know in an attempt to determine whether they, too, are - the outcome of another conspiracy. -

- The collapse of the credibility of our systems for divining and - upholding truths has left us in a state of epistemological chaos. - Once, most of us might have assumed that the system was working and - that our regulations reflected our best understanding of the - empirical truths of the world as they were best understood â now we - have to find our own experts to help us sort the true from the - false. -

- If youâre like me, you probably believe that vaccines are safe, but - you (like me) probably also canât explain the microbiology or - statistics. Few of us have the math skills to review the literature - on vaccine safety and describe why their statistical reasoning is - sound. Likewise, few of us can review the stats in the (now - discredited) literature on opioid safety and explain how those stats - were manipulated. Both vaccines and opioids were embraced by medical - authorities, after all, and one is safe while the other could ruin - your life. Youâre left with a kind of inchoate constellation of - rules of thumb about which experts you trust to fact-check - controversial claims and then to explain how all those respectable - doctors with their peer-reviewed research on opioid safety - were an aberration and then how you know that - the doctors writing about vaccine safety are - not an aberration. -

- Iâm 100% certain that vaccinating is safe and effective, but Iâm - also at something of a loss to explain exactly, - precisely, why I believe this, given all the - corruption I know about and the many times the stamp of certainty - has turned out to be a parochial lie told to further enrich the - super rich. -

- Fake news â conspiracy theories, racist ideologies, scientific - denialism â has always been with us. Whatâs changed today is not the - mix of ideas in the public discourse but the popularity of the worst - ideas in that mix. Conspiracy and denial have skyrocketed in - lockstep with the growth of Big Inequality, which has also tracked - the rise of Big Tech and Big Pharma and Big Wrestling and Big Car - and Big Movie Theater and Big Everything Else. -

- No one can say for certain why this has happened, but the two - dominant camps are idealism (the belief that the people who argue - for these conspiracies have gotten better at explaining them, maybe - with the help of machine-learning tools) or materialism (the ideas - have become more attractive because of material conditions in the - world). -

- Iâm a materialist. Iâve been exposed to the arguments of conspiracy - theorists all my life, and I have not experienced any qualitative - leap in the quality of those arguments. -

- The major difference is in the world, not the arguments. In a time - where actual conspiracies are commonplace, conspiracy theories - acquire a ring of plausibility. -

- We have always had disagreements about whatâs true, but today, we - have a disagreement over how we know whether something is true. This - is an epistemological crisis, not a crisis over belief. Itâs a - crisis over the credibility of our truth-seeking exercises, from - scientific journals (in an era where the biggest journal publishers - have been caught producing pay-to-play journals for junk science) to - regulations (in an era where regulators are routinely cycling in and - out of business) to education (in an era where universities are - dependent on corporate donations to keep their lights on). -

- Targeting â surveillance capitalism â makes it easier to find people - who are undergoing this epistemological crisis, but it doesnât - create the crisis. For that, you need to look to corruption. -

- And, conveniently enough, itâs corruption that allows surveillance - capitalism to grow by dismantling monopoly protections, by - permitting reckless collection and retention of personal data, by - allowing ads to be targeted in secret, and by foreclosing on the - possibility of going somewhere else where you might continue to - enjoy your friends without subjecting yourself to commercial - surveillance. -

Tech is different

- I reject both iterations of technological exceptionalism. I reject - the idea that tech is uniquely terrible and led by people who are - greedier or worse than the leaders of other industries, and I reject - the idea that tech is so good â or so intrinsically prone to - concentration â that it canât be blamed for its present-day - monopolistic status. -

- I think tech is just another industry, albeit one that grew up in - the absence of real monopoly constraints. It may have been first, - but it isnât the worst nor will it be the last. -

- But thereâs one way in which I am a tech - exceptionalist. I believe that online tools are the key to - overcoming problems that are much more urgent than tech - monopolization: climate change, inequality, misogyny, and - discrimination on the basis of race, gender identity, and other - factors. The internet is how we will recruit people to fight those - fights, and how we will coordinate their labor. Tech is not a - substitute for democratic accountability, the rule of law, fairness, - or stability â but itâs a means to achieve these things. -

- The hard problem of our species is coordination. Everything from - climate change to social change to running a business to making a - family work can be viewed as a collective action problem. -

- The internet makes it easier than at any time before to find people - who want to work on a project with you â hence the success of free - and open-source software, crowdfunding, and racist terror groups â - and easier than ever to coordinate the work you do. -

- The internet and the computers we connect to it also possess an - exceptional quality: general-purposeness. The internet is designed - to allow any two parties to communicate any data, using any - protocol, without permission from anyone else. The only production - design we have for computers is the general-purpose, âTuring - completeâ computer that can run every program we can express in - symbolic logic. -

- This means that every time someone with a special communications - need invests in infrastructure and techniques to make the internet - faster, cheaper, and more robust, this benefit redounds to everyone - else who is using the internet to communicate. And this also means - that every time someone with a special computing need invests to - make computers faster, cheaper, and more robust, every other - computing application is a potential beneficiary of this work. -

- For these reasons, every type of communication is gradually absorbed - into the internet, and every type of device â from airplanes to - pacemakers â eventually becomes a computer in a fancy case. -

- While these considerations donât preclude regulating networks and - computers, they do call for gravitas and caution when doing so - because changes to regulatory frameworks could ripple out to have - unintended consequences in many, many other domains. -

- The upshot of this is that our best hope of solving the big - coordination problems â climate change, inequality, etc. â is with - free, fair, and open tech. Our best hope of keeping tech free, fair, - and open is to exercise caution in how we regulate tech and to - attend closely to the ways in which interventions to solve one - problem might create problems in other domains. -

Ownership of facts

- Big Tech has a funny relationship with information. When youâre - generating information â anything from the location data streaming - off your mobile device to the private messages you send to friends - on a social network â it claims the rights to make unlimited use of - that data. -

- But when you have the audacity to turn the tables â to use a tool - that blocks ads or slurps your waiting updates out of a social - network and puts them in another app that lets you set your own - priorities and suggestions or crawls their system to allow you to - start a rival business â they claim that youâre stealing from them. -

- The thing is, information is a very bad fit for any kind of private - property regime. Property rights are useful for establishing markets - that can lead to the effective development of fallow assets. These - markets depend on clear titles to ensure that the things being - bought and sold in them can, in fact, be bought and sold. -

- Information rarely has such a clear title. Take phone numbers: - Thereâs clearly something going wrong when Facebook slurps up - millions of usersâ address books and uses the phone numbers it finds - in them to plot out social graphs and fill in missing information - about other users. -

- But the phone numbers Facebook nonconsensually acquires in this - transaction are not the âpropertyâ of the users theyâre taken from - nor do they belong to the people whose phones ring when you dial - those numbers. The numbers are mere integers, 10 digits in the U.S. - and Canada, and they appear in millions of places, including - somewhere deep in pi as well as numerous other contexts. Giving - people ownership titles to integers is an obviously terrible idea. -

- Likewise for the facts that Facebook and other commercial - surveillance operators acquire about us, like that we are the - children of our parents or the parents to our children or that we - had a conversation with someone else or went to a public place. - These data points canât be property in the sense that your house or - your shirt is your property because the title to them is - intrinsically muddy: Does your mom own the fact that she is your - mother? Do you? Do both of you? What about your dad â does he own - this fact too, or does he have to license the fact from you (or your - mom or both of you) in order to use this fact? What about the - hundreds or thousands of other people who know these facts? -

- If you go to a Black Lives Matter demonstration, do the other - demonstrators need your permission to post their photos from the - event? The online fights over - when - and how to post photos from demonstrations reveal a nuanced, - complex issue that cannot be easily hand-waved away by giving one - party a property right that everyone else in the mix has to respect. -

- The fact that information isnât a good fit with property and markets - doesnât mean that itâs not valuable. Babies arenât property, but - theyâre inarguably valuable. In fact, we have a whole set of rules - just for babies as well as a subset of those rules that apply to - humans more generally. Someone who argues that babies wonât be truly - valuable until they can be bought and sold like loaves of bread - would be instantly and rightfully condemned as a monster. -

- Itâs tempting to reach for the property hammer when Big Tech treats - your information like a nail â not least because Big Tech are such - prolific abusers of property hammers when it comes to - their information. But this is a mistake. If we - allow markets to dictate the use of our information, then weâll find - that weâre sellers in a buyersâ market where the Big Tech monopolies - set a price for our data that is so low as to be insignificant or, - more likely, set at a nonnegotiable price of zero in a click-through - agreement that you donât have the opportunity to modify. -

- Meanwhile, establishing property rights over information will create - insurmountable barriers to independent data processing. Imagine that - we require a license to be negotiated when a translated document is - compared with its original, something Google has done and continues - to do billions of times to train its automated language translation - tools. Google can afford this, but independent third parties cannot. - Google can staff a clearances department to negotiate one-time - payments to the likes of the EU (one of the major repositories of - translated documents) while independent watchdogs wanting to verify - that the translations are well-prepared, or to root out bias in - translations, will find themselves needing a staffed-up legal - department and millions for licenses before they can even get - started. -

- The same goes for things like search indexes of the web or photos of - peoplesâ houses, which have become contentious thanks to Googleâs - Street View project. Whatever problems may exist with Googleâs - photographing of street scenes, resolving them by letting people - decide who can take pictures of the facades of their homes from a - public street will surely create even worse ones. Think of how - street photography is important for newsgathering â including - informal newsgathering, like photographing abuses of authority â and - how being able to document housing and street life are important for - contesting eminent domain, advocating for social aid, reporting - planning and zoning violations, documenting discriminatory and - unequal living conditions, and more. -

- The ownership of facts is antithetical to many kinds of human - progress. Itâs hard to imagine a rule that limits Big Techâs - exploitation of our collective labors without inadvertently banning - people from gathering data on online harassment or compiling indexes - of changes in language or simply investigating how the platforms are - shaping our discourse â all of which require scraping data that - other people have created and subjecting it to scrutiny and - analysis. -

Persuasion worksâ¦ slowly

- The platforms may oversell their ability to persuade people, but - obviously, persuasion works sometimes. Whether itâs the private - realm that LGBTQ people used to recruit allies and normalize sexual - diversity or the decadeslong project to convince people that markets - are the only efficient way to solve complicated resource allocation - problems, itâs clear that our societal attitudes - can change. -

- The project of shifting societal attitudes is a game of inches and - years. For centuries, svengalis have purported to be able to - accelerate this process, but even the most brutal forms of - propaganda have struggled to make permanent changes. Joseph Goebbels - was able to subject Germans to daily, mandatory, hourslong radio - broadcasts, to round up and torture and murder dissidents, and to - seize full control over their childrenâs education while banning any - literature, broadcasts, or films that did not comport with his - worldview. -

- Yet, after 12 years of terror, once the war ended, Nazi ideology was - largely discredited in both East and West Germany, and a program of - national truth and reconciliation was put in its place. Racism and - authoritarianism were never fully abolished in Germany, but neither - were the majority of Germans irrevocably convinced of Nazism â and - the rise of racist authoritarianism in Germany today tells us that - the liberal attitudes that replaced Nazism were no more permanent - than Nazism itself. -

- Racism and authoritarianism have also always been with us. Anyone - whoâs reviewed the kind of messages and arguments that racists put - forward today would be hard-pressed to say that they have gotten - better at presenting their ideas. The same pseudoscience, appeals to - fear, and circular logic that racists presented in the 1980s, when - the cause of white supremacy was on the wane, are to be found in the - communications of leading white nationalists today. -

- If racists havenât gotten more convincing in the past decade, then - how is it that more people were convinced to be openly racist at - that time? I believe that the answer lies in the material world, not - the world of ideas. The ideas havenât gotten more convincing, but - people have become more afraid. Afraid that the state canât be - trusted to act as an honest broker in life-or-death decisions, from - those regarding the management of the economy to the regulation of - painkillers to the rules for handling private information. Afraid - that the world has become a game of musical chairs in which the - chairs are being taken away at a never-before-seen rate. Afraid that - justice for others will come at their expense. Monopolism isnât the - cause of these fears, but the inequality and material desperation - and policy malpractice that monopolism contributes to is a - significant contributor to these conditions. Inequality creates the - conditions for both conspiracies and violent racist ideologies, and - then surveillance capitalism lets opportunists target the fearful - and the conspiracy-minded. -

Paying wonât help

- As the old saw goes, âIf youâre not paying for the product, youâre - the product.â -

- Itâs a commonplace belief today that the advent of free, - ad-supported media was the original sin of surveillance capitalism. - The reasoning is that the companies that charged for access couldnât - âcompete with freeâ and so they were driven out of business. Their - ad-supported competitors, meanwhile, declared open season on their - usersâ data in a bid to improve their ad targeting and make more - money and then resorted to the most sensationalist tactics to - generate clicks on those ads. If only weâd pay for media again, weâd - have a better, more responsible, more sober discourse that would be - better for democracy. -

- But the degradation of news products long precedes the advent of - ad-supported online news. Long before newspapers were online, lax - antitrust enforcement had opened the door for unprecedented waves of - consolidation and roll-ups in newsrooms. Rival newspapers were - merged, reporters and ad sales staff were laid off, physical plants - were sold and leased back, leaving the companies loaded up with debt - through leveraged buyouts and subsequent profit-taking by the new - owners. In other words, it wasnât merely shifts in the classified - advertising market, which was long held to be the primary driver in - the decline of the traditional newsroom, that made news companies - unable to adapt to the internet â it was monopolism. -

- Then, as news companies did come online, the ad - revenues they commanded dropped even as the number of internet users - (and thus potential online readers) increased. That shift was a - function of consolidation in the ad sales market, with Google and - Facebook emerging as duopolists who made more money every year from - advertising while paying less and less of it to the publishers whose - work the ads appeared alongside. Monopolism created a buyerâs market - for ad inventory with Facebook and Google acting as gatekeepers. -

- Paid services continue to exist alongside free ones, and often it is - these paid services â anxious to prevent people from bypassing their - paywalls or sharing paid media with freeloaders â that exert the - most control over their customers. Appleâs iTunes and App Stores are - paid services, but to maximize their profitability, Apple has to - lock its platforms so that third parties canât make compatible - software without permission. These locks allow the company to - exercise both editorial control (enabling it to exclude - controversial - political material) and technological control, including - control over who can repair the devices it makes. If weâre worried - that ad-supported products deprive people of their right to - self-determination by using persuasion techniques to nudge their - purchase decisions a few degrees in one direction or the other, then - the near-total control a single company holds over the decision of - who gets to sell you software, parts, and service for your iPhone - should have us very worried indeed. -

- We shouldnât just be concerned about payment and control: The idea - that paying will improve discourse is also dangerously wrong. The - poor success rate of targeted advertising means that the platforms - have to incentivize you to âengageâ with posts at extremely high - levels to generate enough pageviews to safeguard their profits. As - discussed earlier, to increase engagement, platforms like Facebook - use machine learning to guess which messages will be most - inflammatory and make a point of shoving those into your eyeballs at - every turn so that you will hate-click and argue with people. -

- Perhaps paying would fix this, the reasoning goes. If platforms - could be economically viable even if you stopped clicking on them - once your intellectual and social curiosity had been slaked, then - they would have no reason to algorithmically enrage you to get more - clicks out of you, right? -

- There may be something to that argument, but it still ignores the - wider economic and political context of the platforms and the world - that allowed them to grow so dominant. -

- Platforms are world-spanning and all-encompassing because they are - monopolies, and they are monopolies because we have gutted our most - important and reliable anti-monopoly rules. Antitrust was neutered - as a key part of the project to make the wealthy wealthier, and that - project has worked. The vast majority of people on Earth have a - negative net worth, and even the dwindling middle class is in a - precarious state, undersaved for retirement, underinsured for - medical disasters, and undersecured against climate and technology - shocks. -

- In this wildly unequal world, paying doesnât improve the discourse; - it simply prices discourse out of the range of the majority of - people. Paying for the product is dandy, if you can afford it. -

- If you think todayâs filter bubbles are a problem for our discourse, - imagine what theyâd be like if rich people inhabited free-flowing - Athenian marketplaces of ideas where you have to pay for admission - while everyone else lives in online spaces that are subsidized by - wealthy benefactors who relish the chance to establish - conversational spaces where the âhouse rulesâ forbid questioning the - status quo. That is, imagine if the rich seceded from Facebook, and - then, instead of running ads that made money for shareholders, - Facebook became a billionaireâs vanity project that also happened to - ensure that nobody talked about whether it was fair that only - billionaires could afford to hang out in the rarified corners of the - internet. -

- Behind the idea of paying for access is a belief that free markets - will address Big Techâs dysfunction. After all, to the extent that - people have a view of surveillance at all, it is generally an - unfavorable one, and the longer and more thoroughly one is - surveilled, the less one tends to like it. Same goes for lock-in: If - HPâs ink or Appleâs App Store were really obviously fantastic, they - wouldnât need technical measures to prevent users from choosing a - rivalâs product. The only reason these technical countermeasures - exist is that the companies donât believe their customers would - voluntarily submit to their terms, and they - want to deprive them of the choice to take their business elsewhere. -

- Advocates for markets laud their ability to aggregate the diffused - knowledge of buyers and sellers across a whole society through - demand signals, price signals, and so on. The argument for - surveillance capitalism being a ârogue capitalismâ is that - machine-learning-driven persuasion techniques distort - decision-making by consumers, leading to incorrect signals â - consumers donât buy what they prefer, they buy what theyâre tricked - into preferring. It follows that the monopolistic practices of - lock-in, which do far more to constrain consumersâ free choices, are - even more of a ârogue capitalism.â -

- The profitability of any business is constrained by the possibility - that its customers will take their business elsewhere. Both - surveillance and lock-in are anti-features that no customer wants. - But monopolies can capture their regulators, crush their - competitors, insert themselves into their customersâ lives, and - corral people into âchoosingâ their services regardless of whether - they want them â itâs fine to be terrible when there is no - alternative. -

- Ultimately, surveillance and lock-in are both simply business - strategies that monopolists can choose. Surveillance companies like - Google are perfectly capable of deploying lock-in technologies â - just look at the onerous Android licensing terms that require - device-makers to bundle in Googleâs suite of applications. And - lock-in companies like Apple are perfectly capable of subjecting - their users to surveillance if it means keeping the Chinese - government happy and preserving ongoing access to Chinese markets. - Monopolies may be made up of good, ethical people, but as - institutions, they are not your friend â they will do whatever they - can get away with to maximize their profits, and the more - monopolistic they are, the more they can get - away with. -

An âecologyâ moment for trustbusting

- If weâre going to break Big Techâs death grip on our digital lives, - weâre going to have to fight monopolies. That may sound pretty - mundane and old-fashioned, something out of the New Deal era, while - ending the use of automated behavioral modification feels like the - plotline of a really cool cyberpunk novel. -

- Meanwhile, breaking up monopolies is something we seem to have - forgotten how to do. There is a bipartisan, trans-Atlantic consensus - that breaking up companies is a foolâs errand at best â liable to - mire your federal prosecutors in decades of litigation â and - counterproductive at worst, eroding the âconsumer benefitsâ of large - companies with massive efficiencies of scale. -

- But trustbusters once strode the nation, brandishing law books, - terrorizing robber barons, and shattering the illusion of - monopoliesâ all-powerful grip on our society. The trustbusting era - could not begin until we found the political will â until the people - convinced politicians theyâd have their backs when they went up - against the richest, most powerful men in the world. -

- Could we find that political will again? -

- Copyright scholar James Boyle has described how the term âecologyâ - marked a turning point in environmental activism. Prior to the - adoption of this term, people who wanted to preserve whale - populations didnât necessarily see themselves as fighting the same - battle as people who wanted to protect the ozone layer or fight - freshwater pollution or beat back smog or acid rain. -

- But the term âecologyâ welded these disparate causes together into a - single movement, and the members of this movement found solidarity - with one another. The people who cared about smog signed petitions - circulated by the people who wanted to end whaling, and the - anti-whalers marched alongside the people demanding action on acid - rain. This uniting behind a common cause completely changed the - dynamics of environmentalism, setting the stage for todayâs climate - activism and the sense that preserving the habitability of the - planet Earth is a shared duty among all people. -

- I believe we are on the verge of a new âecologyâ moment dedicated to - combating monopolies. After all, tech isnât the only concentrated - industry nor is it even the most concentrated - of industries. -

- You can find partisans for trustbusting in every sector of the - economy. Everywhere you look, you can find people whoâve been - wronged by monopolists whoâve trashed their finances, their health, - their privacy, their educations, and the lives of people they love. - Those people have the same cause as the people who want to break up - Big Tech and the same enemies. When most of the worldâs wealth is in - the hands of a very few, it follows that nearly every large company - will have overlapping shareholders. -

- Thatâs the good news: With a little bit of work and a little bit of - coalition building, we have more than enough political will to break - up Big Tech and every other concentrated industry besides. First we - take Facebook, then we take AT&T/WarnerMedia. -

- But hereâs the bad news: Much of what weâre doing to tame Big Tech - instead of breaking up the big companies also - forecloses on the possibility of breaking them up later. -

- Big Techâs concentration currently means that their inaction on - harassment, for example, leaves users with an impossible choice: - absent themselves from public discourse by, say, quitting Twitter or - endure vile, constant abuse. Big Techâs over-collection and - over-retention of data results in horrific identity theft. And their - inaction on extremist recruitment means that white supremacists who - livestream their shooting rampages can reach an audience of - billions. The combination of tech concentration and media - concentration means that artistsâ incomes are falling even as the - revenue generated by their creations are increasing. -

- Yet governments confronting all of these problems all inevitably - converge on the same solution: deputize the Big Tech giants to - police their users and render them liable for their usersâ bad - actions. The drive to force Big Tech to use automated filters to - block everything from copyright infringement to sex-trafficking to - violent extremism means that tech companies will have to allocate - hundreds of millions to run these compliance systems. -

- These rules â the EUâs new Directive on Copyright, Australiaâs new - terror regulation, Americaâs FOSTA/SESTA sex-trafficking law and - more â are not just death warrants for small, upstart competitors - that might challenge Big Techâs dominance but who lack the deep - pockets of established incumbents to pay for all these automated - systems. Worse still, these rules put a floor under how small we can - hope to make Big Tech. -

- Thatâs because any move to break up Big Tech and cut it down to size - will have to cope with the hard limit of not making these companies - so small that they can no longer afford to perform these duties â - and itâs expensive to invest in those automated - filters and outsource content moderation. Itâs already going to be - hard to unwind these deeply concentrated, chimeric behemoths that - have been welded together in the pursuit of monopoly profits. Doing - so while simultaneously finding some way to fill the regulatory void - that will be left behind if these self-policing rulers were forced - to suddenly abdicate will be much, much harder. -

- Allowing the platforms to grow to their present size has given them - a dominance that is nearly insurmountable â deputizing them with - public duties to redress the pathologies created by their size makes - it virtually impossible to reduce that size. Lather, rinse, repeat: - If the platforms donât get smaller, they will get larger, and as - they get larger, they will create more problems, which will give - rise to more public duties for the companies, which will make them - bigger still. -

- We can work to fix the internet by breaking up Big Tech and - depriving them of monopoly profits, or we can work to fix Big Tech - by making them spend their monopoly profits on governance. But we - canât do both. We have to choose between a vibrant, open internet or - a dominated, monopolized internet commanded by Big Tech giants that - we struggle with constantly to get them to behave themselves. -

Make Big Tech small again

- Trustbusting is hard. Breaking big companies into smaller ones is - expensive and time-consuming. So time-consuming that by the time - youâre done, the world has often moved on and rendered years of - litigation irrelevant. From 1969 to 1982, the U.S. government - pursued an antitrust case against IBM over its dominance of - mainframe computing â but the case collapsed in 1982 because - mainframes were being speedily replaced by PCs. -

- A future U.S. president could simply direct their attorney general - to enforce the law as it was written. -

- Itâs far easier to prevent concentration than to fix it, and - reinstating the traditional contours of U.S. antitrust enforcement - will, at the very least, prevent further concentration. That means - bans on mergers between large companies, on big companies acquiring - nascent competitors, and on platform companies competing directly - with the companies that rely on the platforms. -

- These powers are all in the plain language of U.S. antitrust laws, - so in theory, a future U.S. president could simply direct their - attorney general to enforce the law as it was written. But after - decades of judicial âeducationâ in the benefits of monopolies, after - multiple administrations that have packed the federal courts with - lifetime-appointed monopoly cheerleaders, itâs not clear that mere - administrative action would do the trick. -

- If the courts frustrate the Justice Department and the president, - the next stop would be Congress, which could eliminate any doubt - about how antitrust law should be enforced in the U.S. by passing - new laws that boil down to saying, âKnock it off. We all know what - the Sherman Act says. Robert Bork was a deranged fantasist. For - avoidance of doubt, fuck that guy.â In other - words, the problem with monopolies is - monopolism â the concentration of power into - too few hands, which erodes our right to self-determination. If - there is a monopoly, the law wants it gone, period. Sure, get rid of - monopolies that create âconsumer harmâ in the form of higher prices, - but also, get rid of other monopolies, too. -

- But this only prevents things from getting worse. To help them get - better, we will have to build coalitions with other activists in the - anti-monopoly ecology movement â a pluralism movement or a - self-determination movement â and target existing monopolies in - every industry for breakup and structural separation rules that - prevent, for example, the giant eyewear monopolist Luxottica from - dominating both the sale and the manufacture of spectacles. -

- In an important sense, it doesnât matter which industry the breakups - begin in. Once they start, shareholders in - every industry will start to eye their - investments in monopolists skeptically. As trustbusters ride into - town and start making lives miserable for monopolists, the debate - around every corporate boardroomâs table will shift. People within - corporations whoâve always felt uneasy about monopolism will gain a - powerful new argument to fend off their evil rivals in the corporate - hierarchy: âIf we do it my way, we make less money; if we do it your - way, a judge will fine us billions and expose us to ridicule and - public disapprobation. So even though I get that it would be really - cool to do that merger, lock out that competitor, or buy that little - company and kill it before it can threaten it, we really shouldnât â - not if we donât want to get tied to the DOJâs bumper and get dragged - up and down Trustbuster Road for the next 10 years.â -

20 GOTO 10

- Fixing Big Tech will require a lot of iteration. As cyber lawyer - Lawrence Lessig wrote in his 1999 book, Code and Other - Laws of Cyberspace, our lives are regulated by four - forces: law (whatâs legal), code (whatâs technologically possible), - norms (whatâs socially acceptable), and markets (whatâs profitable). -

- If you could wave a wand and get Congress to pass a law that - re-fanged the Sherman Act tomorrow, you could use the impending - breakups to convince venture capitalists to fund competitors to - Facebook, Google, Twitter, and Apple that would be waiting in the - wings after they were cut down to size. -

- But getting Congress to act will require a massive normative shift, - a mass movement of people who care about monopolies â and pulling - them apart. -

- Getting people to care about monopolies will take technological - interventions that help them to see what a world free from Big Tech - might look like. Imagine if someone could make a beloved (but - unauthorized) third-party Facebook or Twitter client that dampens - the anxiety-producing algorithmic drumbeat and still lets you talk - to your friends without being spied upon â something that made - social media more sociable and less toxic. Now imagine that it gets - shut down in a brutal legal battle. Itâs always easier to convince - people that something must be done to save a thing they love than it - is to excite them about something that doesnât even exist yet. -

- Neither tech nor law nor code nor markets are sufficient to reform - Big Tech. But a profitable competitor to Big Tech could bankroll a - legislative push; legal reform can embolden a toolsmith to make a - better tool; the tool can create customers for a potential business - who value the benefits of the internet but want them delivered - without Big Tech; and that business can get funded and divert some - of its profits to legal reform. 20 GOTO 10 (or lather, rinse, - repeat). Do it again, but this time, get farther! After all, this - time youâre starting with weaker Big Tech adversaries, a - constituency that understands things can be better, Big Tech rivals - whoâll help ensure their own future by bankrolling reform, and code - that other programmers can build on to weaken Big Tech even further. -

- The surveillance capitalism hypothesis â that Big Techâs products - really work as well as they say they do and thatâs why everything is - so screwed up â is way too easy on surveillance and even easier on - capitalism. Companies spy because they believe their own BS, and - companies spy because governments let them, and companies spy - because any advantage from spying is so short-lived and minor that - they have to do more and more of it just to stay in place. -

- As to why things are so screwed up? Capitalism. Specifically, the - monopolism that creates inequality and the inequality that creates - monopolism. Itâs a form of capitalism that rewards sociopaths who - destroy the real economy to inflate the bottom line, and they get - away with it for the same reason companies get away with spying: - because our governments are in thrall to both the ideology that says - monopolies are actually just fine and in thrall to the ideology that - says that in a monopolistic world, youâd better not piss off the - monopolists. -

- Surveillance doesnât make capitalism rogue. Capitalismâs unchecked - rule begets surveillance. Surveillance isnât bad because it lets - people manipulate us. Itâs bad because it crushes our ability to be - our authentic selves â and because it lets the rich and powerful - figure out who might be thinking of building guillotines and what - dirt they can use to discredit those embryonic guillotine-builders - before they can even get to the lumberyard. -

Up and through

- With all the problems of Big Tech, itâs tempting to imagine solving - the problem by returning to a world without tech at all. Resist that - temptation. -

- The only way out of our Big Tech problem is up and through. If our - future is not reliant upon high tech, it will be because - civilization has fallen. Big Tech wired together a planetary, - species-wide nervous system that, with the proper reforms and course - corrections, is capable of seeing us through the existential - challenge of our species and planet. Now itâs up to us to seize the - means of computation, putting that electronic nervous system under - democratic, accountable control. -

- I am, secretly, despite what I have said earlier, a tech - exceptionalist. Not in the sense of thinking that tech should be - given a free pass to monopolize because it has âeconomies of scaleâ - or some other nebulous feature. Iâm a tech exceptionalist because I - believe that getting tech right matters and that getting it wrong - will be an unmitigated catastrophe â and doing it right can give us - the power to work together to save our civilization, our species, - and our planet. -