I see you
~~ Artifical Intelligence as a metaphor of unchecked capitalism ~~
There was a sponsor message in an NPR podcast this morning that gave me pause. Here’s how it went:
[Company] fits seamlessly into your life, allowing you to easily manage your coverage, pay your bill, and even file a claim with the [Company] mobile app. And they really get to know you.
Pretty normal copy with predictable jargon. My wetware ad blocker was already filtering out most of it. A digitally transformed business is saying “we know you” to smokescreen a different reality: that you are a unique identifier seamlessly connecting a large number of data points, allowing the company to easily sell you more of its services.
But then the message took an unexpected turn.
Thanks to a network of 19,000 agents, you’ll have someone local to talk you through options that fit your personal needs.
When flesh-and-blood humans are invoked, the meaning of “we know you” shifts. [Company] almost started to sound like an interesting business, in spite of its mobile app.
Netflix has a model of my viewing habits that can accurately predict what movies I’ll like next. But so did the clerk in the video rental fifteen years ago. In fact, when much later we bumped into each other at a music festival one drunken morning, he ID’d me based on my rental history.
The thought that my bank “knows” me, in an algorithmic, illicit-activity-preventing, lifestyle-predicting sort of way is creepy and alarming. And it makes me think of Patrick McKenzie’s banker in Japan:[1]
That was the first of a dozen stories which you wouldn’t believe actually happened about that bank. Taro correctly intuited when I started dating a young lady, and when we broke up, solely based on on my spending habits. He considered that part and parcel with looking out for my financial interests.
When someone claims that an IT system “knows you,” that is a disingenuous way of saying you are under surveillance. Language has such a way of shifting our frame of reference that the gaslighting is usually not even intentional anymore.
To be known, you need a knower; to be seen, you need a seer. You need a sentient being across from you, a person who shares with you the capacity for joy and sadness, suffering and redemption.
Names like machine learning or artificial intelligence are representative of today’s unchecked capitalism. The names mislabel the things, which are profit-maximizing computational tools that involve neither learning nor intelligence. There is no person across from you. When a society eliminates the humans from its transactions but hangs on to the data, you are not known and you are not seen. At best, you are being monitored.