Summary

  • “In this book, I will explore the business practices of leading Internet and finance companies, focusing on their use of proprietary reputation, search, and finance technologies in our often chaotic information environment” (14). Pasquale’s main thrust is that we have a deleterious information asymmetry b/t corporations and individuals that undermines citizenship and governance; we can battle this asymmetry though legal channels (as it is a legal effect first and foremost, he argues) and work toward a more transparent (again a value) society.
  • “The black box society is animated by the belief that information is useful only to the extent that it is exclusive—that is, secret” (57).

Keywords

  • “black box”: “It can refer to a recording device, like the data-monitoring systems in planes, trains, and cars. Or it can mean a system whose workings are mysterious; we can observe its inputs and outputs, but we cannot tell how one becomes the other” (3).
  • “never events”: “grotesque wastes of resources that should not be possible in a minimally functional financial system” (102)

Notes

1. Introduction

  • “Much of what we can find out about companies, governments, or even one another, is governed by law” (2). Laws “set limits to inquiry” but these laws are not natural, but rather socially constructed.
  • Focus: the privacy asymmetry between corporations and individuals: “The law, so aggressively protective of secrecy in the world of commerce, is increasingly silent when it comes to the privacy of persons” (3).
  • Three strategies by which corps keep black boxes opaque:
    1. real secrecy “establishes a barrier between hidden content and unauthorized access to it”
    2. legal secrecy “obliges those privy to certain information to keep it secret”
    3. obfuscation “involves deliberate attempts at concealment when secrecy has been compromised” (6).
  • The challenge of all of this is that “authority is increasingly expressed algorithmically,” which is to say automatically and through software—though this software is always designed and programmed by humans and expresses their biases (8). Will be useful to compare this w/ Noble.
  • Note how the rise of game theory in MBA programs (“which stressed the importance of gaining an information advantage over rivals”) emerges in similar contexts to the cybernetics and information sciences projects of the 1950’s (13).

2. Reputation

  • Context collapse in algorithmic reputation production: “That’s an uncomfortable reality in a world where credit scores have escaped from their native financial context and established themselves as arbiters of general reliability in other areas, like car insurance (23).
  • I do wonder sometimes the root of all the money in statements like “There’s money to be made from knowing if someone is gay” (25); how do folks profit off information asymmetry? Is our entire economy becoming an extension of the marketing industry?
  • Pasquale notes the “creepiness” of big data; this is of interest to Chun as well, and will be of relevance to my Speculation and Strangeness list. What do we make of the turn to the affect of “creepiness” in contemporary digital culture?
  • I’m interested in the implicit “we” in sentences like: “Medical reputations are being created in processes we can barely understand, let alone control” (28).
  • “Employers invested in these technologies pooh-pooh the ‘creepiness’ objection as a matter of taste or a regrettable lack of the toughness the work world requires. But the creepy feeling is world disclosive; it is an emotional reaction that alerts us to the possibility of real harm” (37–38).
  • Feedback loops: we go looking for data in certain places and then find it, and use its supposed “objectivity” to obscure the fact that we were looking for something specific in the first place (42).
  • Fascinating simulation example: datasharing between the NSA and the DEA justified by “creating criminal investigations in which they retrospectively fabricated alternative grounds for suspecting and investigating the target” (47); “Separate and parallel ‘realities’ are constructed and documented.” A blurred line b/t intelligence (anticipatory) and investigation (post-evidentiary).
  • The question of “cyber-hygiene” pops up on pp. 53—this is a concept I’ve explored in my IoT and EFF security work, and has ramifications for my work on 3D printing and toxicity as well.
  • Once again, transparency is the solution: “This situation comes up reliably enough in the communications context that there is a long-standing solution: require both conduits and content providers to disclose whether they are raising the profile of those who pay them” (71).
  • Pasquale is arguing for the internet as a utility—the precise dream that the FTC under Trump will destroy two years after he wrote this book (91).

4. Finance

  • “From mortgage brokers to CEOs, finance’s many workers rely on opaque complexity to generate profits. But much of the opacity is neither natural nor inevitable. Rather, it prevails in many critical transactions in order to give privileged insiders an advantage over their clients, regulators, and risk managers” (101–02).
  • “Secret algorithms—obscured by a triple layer of technical complexity, secrecy, and ‘economic espionage’ laws that can land would-be whistle-blowers in prison—still prevent us from understanding what is truly going on in many major financial firms” (103).
  • Futures trading emerging in the 90s as a way to smooth over the inherent risks of lending in an effort to maintain constant growth a la a corporation (104).
  • Finance as a fundamental problem of prediction; data belies its own futurity: “One question for a database like this is, How relevant is the past to the future?” (110).
  • “You would think that after repeated crises, from the S&L crimes of the late 1980s to the accounting scandals of the early 2000s, the finance sector would have cleaned house by 2008. But rather than improving internal processes at companies they could not fully understand (let alone control), financiers started insuring against bad outcomes” (113). This is emergence of the credit swap.
  • Interesting citation of Harman to map the “legion” nature of the modern corporation (118).
  • Affects of black boxes: “Sometimes Wall Street workers’ black boxes are so effective they even ‘fool’ their creators. Or, worse, they foster cynicism that render concepts like fairness or professionalism quaint relics, words to be forgotten as soon as they stand in the way of a profit” (123).
  • Digital intensification and the temporality of HFT on pp. 128–29. (The information arms race then has consequences for what kinds of infrastructures get built and where.): “HFT is the ultimate in financial self-reference, where perceptions of value come entirely from signals encoded on trading terminals. Lately, the limiting factor in fast trading the speed of light in fiber-optic cables. Thus firms are paying to construct ultrafast cables between financial centers. Spread Networks spent over $200 million to lay a cable between Chicago and New York-area exchanges, estimating that firms could make $20 billion in a year exploiting price discrepancies (lasting less than a second) between the two cities. Modelers have devised more extreme solutions to the time delay problem. An ‘optimal scheme’ would ‘push trading firms to build new computers [at] the exact, optimal points in between markets’—even if that happened to be in the middle of an ocean” (131).
  • Disconnections from “reality,” whatever that is: “The towering digital edifices of credit erected by advanced finance are increasingly disconnected from actual productivity” (135).
  • “Journalist and anthropologist Gillian Tett has described the eerie ‘social silences’ around exotic derivatives trades with a disarming observation: ‘Once something is labeled boring, it’s the easiest way to hide it in plain sight’” (137).

Archive and Impact

  • Pasquale is a legal scholar at UMD; this book is first and foremost about modifications we can make to the existing legal system in order to reduce the privacy asymmetry between corporations and individuals. As such, its recommendations are not as politically radical as those theorists working closer to the avant-garde like Galloway (or me for that matter). In particular, I wonder about the desire for openness and transparency as given virtues, which seems willfully naive in 2018. (Contrast with a more philosophically nuanced concept like Glissant’s “right to opacity.”)
  • I’m a little shocked that Pasquale cites Hayek approvingly in service of market-based solutions, but then again Pasquale’s perspective here is pretty mainline liberal, so .