The Creepy TikTok Algorithm Doesn’t Know You

It’s a truth universally acknowledged that “the algorithm” knows you better than you know yourself. A computer can supposedly predict whether you’ll quit your job or break up with your partner. With 1,000 words of your writing, it can determine your age within four years. And no algorithm seems closer to omniscience than TikTok’s, which is reportedly helping users discover their sexuality and unpack their childhood trauma. Whereas Facebook asks you to set up a profile, and hand over a treasure trove of personal information in the process, TikTok simply notices—or seems to. The results can feel magical, writes Jess Joho in Mashable, as if TikTok is “reading your soul like some sort of divine digital oracle, prying open layers of your being never before known to your own conscious mind.”

As ever, there’s a dark side. Most users will be sucked into WitchTok and gardening how-to’s, but others will end up in an infinite scroll about ADHD, Tourette’s syndrome, or autism (#mentalhealth, for example, has garnered almost 21 billion views—a fraction of #dance, which has 341 billion views, but nonetheless a significant showing). This information can be extremely liberating, especially for conditions usually shrouded in stigma. But some are concerned about the deference given to the platform’s pseudo-psychiatric content. “Once [the algorithm] puts you on a side, it keeps you there,” says psychology professor Inna Kanevsky, better known on TikTok as the debunking @dr_inna, “and it starts seeming like you’re diagnosed.” That’s a potentially life-altering takeaway proffered up by a for-profit algorithm and content creators of varying reliability.

But the debate over self-diagnosis is just one small part of the ways in which we are positioning TikTok and similar algorithms not as mere prediction-generating machines but insight-producing ones. While recommendation algorithms from Amazon to Netflix are designed to guess what you’d like to see next, TikTok can feel as if it’s showing you who you’ve always been. In the process, we are opening ourselves to the peril and promise of “outsourcing self-awareness to AI.” While this new relationship to computing may offer new opportunities for personal growth, it’s also started to sort us into ever more rigid identities.

The human mind has tried to make sense of itself through technological metaphors for thousands of years, writes artificial intelligence expert George Zarkadakis in his 2016 book In Our Own Image. In the Roman empire, for example, the success of hydraulic engineering gave rise to the understanding of human intelligence as the flow of four “humours.” Now we describe the brain as a computer—just one of so many machines that “stores” and “retrieves” memories and “processes information.” While all of these metaphors remain deeply flawed, the close collaboration of human neuroscience and computer science has revealed some intriguing similarities between our minds and our machines.

Like any algorithm, TikTok’s divinatory properties are just the end result of a series of repeated steps. When someone creates a new account, the algorithm targets them with a variety of popular videos designed to test their response to broad categories of content, from viral dances to home repairs, according to a recent Wall Street Journal investigation. When the newspaper set 100-plus bots loose on TikTok, the platform’s “rabbit holing” keyed in on every bot’s preprogrammed interests in less than two hours.

It’s pretty simple math, but it’s also an uncanny mimic of statistical learning, says John Bargh, a professor of psychology and cognitive science at Yale University. That’s one way the human mind unconsciously acquires new knowledge—simply by noticing patterns in the world around it. Like the TikTok algorithm, people are learning all the time, often without even realizing it.

In other words, it’s a process that plays out mostly without a person’s conscious awareness, and yet is entirely determined by what pulls your attention.

So although TikTok seems to uncover things about users that they didn’t necessarily know about themselves, in reality it’s more accurate to say that TikTok shows you where your attention already goes—or would go, if you were freed from the social norms that keep your curiosity corralled offline. While Joho wrote in Mashable that “algorithms knew I was bi before I did,” she ultimately concluded her breakthrough was less about the algorithm’s ability to reveal her desires and more about the power of heteronormativity to conceal them. In these moments, TikTok is like a hotel vanity mirror, magnifying parts of your reflection you may have otherwise glossed over. Such high resolution can be revealing, but users err in treating it as anything more mystical.

The TikTok algorithm’s near-universal renown has made it all too easy to treat TikTok’s probabilistic functions as diagnostic, or even deterministic. In another context, this might be understood as trust—if the algorithm was right that you’re a lesbian cottagecore Gleek obsessed with French bulldogs, it’s at least worth considering what it means for the same system to serve you ADHD content. But these questions are loaded, as TikTok is designed to relentlessly exploit any curiosity it engenders.

In case it wasn’t clear, TikTok’s real motivation isn’t psychoanalysis, it’s profit. The algorithm “is trying to differentiate you from … the vanilla user” to keep your attention (and keep earning money), says Johannes Eichstaedt, a computational social scientist at the Stanford Institute for Human-Centered AI. “So any deviation you give it, it will go and explore.” In real life, the differences between someone who gets served gardening videos and someone who doesn’t is likely smaller than what those two people might have in common. Yet companies are riding these near-invisible distinctions straight to the bank.

What started out feeling like endless possibility can quickly become calcifying, as digital identities on TikTok and other platforms slowly “entomb [a person] as an ever-more stable image of what I like and why,” writes visual artist Jenny Odell in her bestselling book How To Do Nothing. The more the self is reduced to “a consistent and recognizable pattern of habits, desires, and drives that can be more easily advertised to and appropriated, like units of capital,” Odell wrote, the easier it is to market to you—whoever that is anymore.

The consequences aren’t limited to targeted advertisements (Facebook Watch or Hello Fresh?) or even mental health diagnoses (ADHD or OCD?). At the same time that we have imbued TikTok with mystical qualities, we have discounted the mysteries—unknowable even to an algorithm—within ourselves. We’ve forked over that sense of endless possibility in exchange for a keyword-searchable self-conception, where your Netflix category code (lately, I’m #1476024: Christmas, Children & Family Films, From the 1990s) feels not only as important an identifier as your thoughts on the Green New Deal and as durable an identifier as the place where you actually went to college. It’s the faulty logic of a “generation,” which arbitrarily assumes that you have more in common with people born before a certain year than after, but worse, because every generation is now shorter than a 3-minute video.

There are countless proposed solutions for the problems posed by these algorithms, running the gamut from going deeper into the Metaverse to retreating entirely into the woods. But discarding broken metaphors for the brain—and embracing the true chaos of human experience—is a good place to start.

Sigmund Freud developed the idea of the unconscious as a swirling depth of repressed memories, beliefs, and desires more than a century ago. Like many of Freud’s ideas, the unconscious has been debunked by a subsequent century of scientific research; today, cognitive scientists talk about “implicit” or “automatic” functions, not the id and superego. But “in the popular imagination [Freud has become] a kind of poet of the mind,” Louis Menand wrote in The New Yorker. His ideas still hold sway in literature, marketing, and the way we talk about ourselves and each other, even in the algorithmic era. Revealing the “real you” remains a core tenant of the cult of self-discovery to this day—only we also rely on psilocybin, wellness retreats, and TikTok to get us there, in addition to talk therapy. Now it’s TikTok that has endless depths, while the humans are getting played like slot machines.

The trouble is, this dogged pursuit of total transparence isn’t as noble or even necessary as it seems. Some things are worth interrogating further, including our prejudices and our desires, but much of the universe (up to and including our own interiority) is sure to remain mysterious no matter how hard we try to comprehend it. To truly know a person (or oneself) is less about their favorite Taylor Swift song than their life story, their goals for the future, their defense mechanisms and coping strategies, their skills and weaknesses, how they see themselves, and more, argues the Northwestern psychologist Daniel McAdams. It’s a lot more data to crunch, but instead of flattening ourselves into a series of ones and zeros, our complexity is something to protect.