↓Skip to main content

The List and the Oracle

As far back as you can remember, you’ve been bringing your questions to a small box on a screen. At the beginning it was a search engine, like Google; today it is increasingly an AI tool of some sort. But the results we retrieve from these search tools are fundamentally different. This article explains these differences and why they matter.

Consider how the results from traditional search tools differ formally from those provided by AI assistants. In a traditional search tool, such as a library database (and until recently, web searches), your results appear as a long, ordered list. However, the result from an AI tool takes the form of an answer. While this may seem insignificant, the formal characteristics of these responses carry enormous consequences. They each embody a distinct theory of knowledge and position you in a completely different way with respect to the work of learning. I call these two forms The List and The Oracle. In the following two sections, I examine the formal properties of these two research approaches at closer range.

The list #

When we search using traditional tools, we receive a list:

The list is diverse. #

In the list, you are presented with a great many results. And before you have clicked on a single one, you have been taught something important. The list tells you that your question doesn’t have a single answer; instead, it evokes a tremendous number of responses and admits a great diversity in the angle of approach. It is worth stopping for a moment to understand how unusual this type of response really is. Imagine if you asked your friend what they thought of a movie you watched together and your friend responded by reaching into her bag and producing 728 scholarly articles about the film. As an answer to your question, this is a failure. But as a picture of reality, the response is very accurate. There really are hundreds of unique views on the film. Some of these views are quite similar; others are utterly incommensurable. The plurality and diversity of the list is not the technology being unhelpful or confusing or antiquated. It is the list refusing to pretend that there is only one answer to your question.

The list arrives in an order that you can change. #

When you receive search results, the list is organized in a ranked order—by relevance, date of publication, frequency of citation, or some other value. This order is not rigid; you are free to reorder the list with different values and then see how the list responds. Consider what this fact reveals. Somewhere, someone made a judgment, and the judgment was to emphasize certain criteria over others. But these criteria are not the only or “best” way to rank the results; they are only what someone decided. And you may decide otherwise—by choosing to emphasize different features or criteria that will reorganize the sources according to something you value. The default rankings are not objective or neutral. A Google search, for example, is profoundly shaped by advertisers and algorithmic filtering that is quite opaque to the user. However, the virtue of the list is that its ranking is visible, something that you have under your control.

The list presents a world much larger than your view of it. #

If you spend enough time searching, you will quickly discover that any question worth asking already has a deep history and a vast record. The form of the list reveals that there is a horizon of knowledge out there and you are far, far away from it. Indeed, you might spend months or years or a lifetime and never finish considering it all. There is always another response, a different argument, a new contribution, a return to an old one. If you leave and return later, you will see something new has been offered for consideration. Critically, these conversations are not hidden from you—the whole history is arrayed there for you to investigate. You are free to pick your way through the entire landscape, going this way or that as your curiosity directs. It will remain there when you grow tired and leave, and it will be there when you return.

Everything in the list is signed. #

Someone has taken responsibility for each item in the list. The items are associated with names, credentials, affiliations, institutions. This information is more important and useful than it may first appear. With a name, you may research someone’s biography, their previous writing, their training and expertise, who funds them, etc. Institutional affiliations may say a great deal about a person’s commitments, politics, values, or character. Such research provides a great many clues about whether the text before you may be trusted or what purposes it serves. If the named person is alive, they may be contacted—scholars and authors are often delighted to receive inquiries about their work. When you cite them in your own writing, you may borrow their authority for your own purposes, and you will in turn be judged for having done so. Understanding that people stand behind the items in the list, reveals that they are not facts to memorize or accept; they are views that may be debated, countered, extended. The items are an open invitation to dialogue with people who share an interest in something that matters to you.

The list’s items point to each other. #

Through citations, the items in the list index each other, forming a vast web of connections. Each work in the list acknowledges the prior work upon which it is built and to which it owes a debt. In turn, each new item will be referenced by future works, providing a way to trace a conversation or problem far back into the past. This allows us to understand that knowledge is built, not found—it is assembled from the work of others and it will, in turn, become the material for future work. To view the list is to understand that you have arrived in the middle of things, at a place where you might choose to go to work yourself.

The list’s items are dated. #

The items in the list appeared during a single moment in time, in a particular cultural context, surrounded by a cloud of other conversations and happenings. The date tells us when the item was deemed important, relevant, interesting, contested; however, it also provides an opportunity to contextualize the item by considering how it was influenced by the assumptions or values that were ascendant at the time of its production. While some of these events and influences may be clearly thematized within the item itself, others may tug and pull on it invisibly. Things that are significant at one moment in time will not necessarily always be so; things that are important to our time were insignificant, or unknown, or unthinkable at other times. The item’s date helps us understand how time and social change affect the production and reception of knowledge.

The list can be empty. #

A query can return nothing at all. This is not a failure, but a form of reconnaissance. If you search with a keyword that is not in use, the list will tell you so. But the meaning of such an event is not “there is nothing here,” it is instead “there is nothing here under this word.” This discovery reveals something about how the knowledge within the discipline is organized, and provides intelligence about the shared terminology that is used in the field. An empty list is feedback that tells you that the tool you are using is not equal to the job at hand. The list helps you understand this without pretending otherwise.

None of this should be taken to mean that using the list is always a fun experience. It can be tedious and frustrating to be sure. The point is that the inconvenience of the list reflects the actual difficulty of the world and the challenges of learning and research. It is not a malfunction or a technical failure when the list returns hundreds of contradictory results, it is just accurately revealing the messy state of the question you’ve become interested in. The world is diverse and difficult and contested; the knowledge of it is produced by people who are held accountable for the words and ideas they express; it is much larger than anyone’s modest view of it; and it is unfinished. And that last part is important because it is an invitation to you—a person who lives in the middle of it all—who may judge and respond and contribute something of your own to the conversation.


The oracle #

When we search with an AI tool we receive an answer:

The oracle’s answer is singular. #

When the response from the oracle arrives, there is nothing else but the response. The response does not take the form of hundreds of items, each one ready to complicate what we’ve read before. While nothing in the response claims that there is only one answer to your query, the form it takes powerfully suggests otherwise. It is true that the response may indicate that considerable disagreement exists on a particular question, and proceed to list several points of contention. While that is better than nothing, notice that you are still getting a single voice that reports to you that a disagreement exists. A single voice that summarizes a conversation in its own words in a few brief paragraphs is not the same thing as putting you into the room where the conversation is happening and its history is preserved.

The oracle’s answer conceals its own construction. #

The oracle’s response conceals the sources that were used to generate its response. In fact, the oracle does not actually retain the source materials at all; instead, it compresses and abstracts its training corpus into billions of statistical weights that are subsequently used to generate responses, word after word, through a process of inference in a given context. It generates each answer from the digital traces of sources it no longer possesses. In the list, you may see how knowledge is constructed: there are joints and fissures and clear declarations about its assembly. The sources that the author used to construct the item have a clear provenance and are reviewable, which helps the researcher assess its truthfulness and credibility. We know that the oracle’s training corpora contains writing from field experts who teach at Oxford mixed with Reddit posters named “dogboner,” but the role of these particular texts in the oracle’s response is impossible to recover or test. So even a correct answer provides no means to determine whether it is so or how it was achieved.

The oracle’s answer has no outside. #

Nothing in the oracle’s response hints at what information it used, what it chose to set aside, what it was unable to reach. There are not 10 or 20 further pages of other responses you may examine or further refine. Compare the experience of discovering 3,780 results from a search in a database to the oracle’s brief output. This response tells you that you are looking at a single fragment of something much, much larger. A paragraph of prose from the oracle, however, tells you nothing about how much coverage exists on the topic of your concern. The response gives you no way to estimate whether the response draws on three or three hundred sources, or whether it was drawn from the most respected work in the field or whatever references it could find on the open web.

The oracle’s answer is unsigned. #

The oracle’s response is not made by an author or an institution. No one put their reputation on the line by creating and publishing it. With a name we can find out who is behind the item, what they have written in the past, whether they are reviled or ridiculed or have been forced to retract their work in the past. The oracle’s response provides no such opportunity. Instead of responsibility, we receive the following warning: “AI can make mistakes, please double-check responses.”

The oracle’s answer is undated. #

The oracle’s response is ahistorical, a view from nowhere. Although the response arrives at the particular moment when a question is asked, it is constructed from materials that date from many disparate moments in time, from many distinct cultural contexts, and from a great number of conversations or debates. None of these dense, complicating details, however, are legible within the response. Unlike the items in the list, the oracle’s answer is impossible to locate or historicize. If the model was trained on information where or when a particular framework was the accepted view, that one-sided and limited perspective will be passed on to you in a confident, declarative response.

The oracle cannot be experimented upon. #

In traditional search you are free to alter a search term or keyword and watch as the results respond. When the addition of a new keyword turns 34 results into 215, you have learned something important about the world: the discipline(s) you are searching within prefer a term or concept you didn’t think was important at first. But rephrasing a prompt for the oracle doesn’t yield similar intelligence. Your new prompt will produce different prose, but there is nothing to count afterwards as a measurement and no way to determine what made the difference in the response. Did the new answer result from some term or word you supplied? Did it occur because some term you used is important to the scholarly literature or problem you are investigating? Was it because the AI model simply output a different response because that’s what it does each time it runs? You can’t know because you can’t audit how the oracle works or what information it is drawing from or how its weights and training rewards condition its response. The list is open source and you have control over all the knobs and switches; the oracle is a black box.

The oracle is never silent. #

If you ask the oracle about something where its records are thin or almost nonexistent, it will still produce several confident paragraphs in response, because responding in confident paragraphs is what the machine does. In the list, a null response is information; you’ve learned that there are no records under the term you’ve used in your query. And there are no records under this term because that is not a critical term that organizes the knowledge in the discipline. A magical answer machine exists to provide answers, not silence. It provides answers even when doing so is unhelpful or misleading or partial. While newer tools of this sort will sometimes admit when they lack insight into the subject (a genuine improvement over prior versions), they are incapable of making metacritical assessments about: 1) material where their coverage is thin, 2) material that exists that they were never trained on, 3) material that exists behind a paywall that they cannot access. When faced with these unknowns, the oracle often produces plausible responses and generalities rather than report lacunae. As a result, the oracle’s responses look similar where its record of information is rich as when its record is comparatively poor. The list can show the frontiers and edges of knowledge exist; the oracle cannot.