How the Bases of Knowledge Evolve

Prologue: What does “That’s exactly right” actually tell us?

You read a horoscope or personality profile and think, “That sounds just like me.” But the person next to you reads the same passage and has the same reaction. Whom, exactly, has the profile understood?

Psychologist Bertram Forer once demonstrated the problem in a classroom. After taking a test, 39 students received personality descriptions bearing their own names. Most rated the descriptions highly. Yet every student had received the same text. Forer, 1949 “This description fits me” does not yet establish that it can distinguish me from someone else.

Debates over whether traditional Chinese medicine “works” often collapse several questions into one: Does a particular treatment help? Is its explanation correct? Under what other conditions will it work? Establishing a benefit does not validate an entire explanatory system. NCCIH, 2019 In imperial China, the astronomical bureau both calculated eclipses and interpreted celestial omens. Even an accurate eclipse prediction would not vouch for a prediction of human fortune or disaster. History of the Ming, vol. 74 How much can one success actually support?

The unit of analysis in this essay is a particular inference: What was observed? What was inferred from it? What justifies that step? The complexity of a system should also be assessed in relation to these questions. A grammar can generate countless stories without making them true. The more a system can express, the more carefully we must ask which of its explanations the evidence has ruled out.

Algebra offers a way to separate these issues. Imagine drawing with a few basic directions. Where can you get by adding multiples of them? The set of points you can reach is the space they span. In linear algebra, a collection of vectors is a basis for a specified space if it spans that space and no vector can be written as a linear combination of the others. The second condition is called linear independence. The intuition we will borrow is simple: Which basic distinctions have we chosen, and what can we distinguish with them?

Following individual inferences leads to a larger question: How do people change the ways they distinguish things in the world? This essay proposes a research program:

At its core, the history of human thought is a history of evolving bases: a continuing search for coordinate systems that are more orthogonal (more separable), more complete, and capable of spanning the world of experience at ever finer resolution.

Here, a “basis” means the basic distinctions through which we make sense of the world. “More orthogonal” suggests that judgments can be checked separately; “more complete” means accommodating aspects previously left out; “finer resolution” means making distinctions fine enough to answer the question at hand. A mathematical basis already spans its specified space. The broader use of “more complete” refers instead to an expansion in the phenomena and questions a framework can handle. These are epistemological analogies informed by mathematics.

We will begin with particular judgments: when do we need to separate old associations, introduce new distinctions, or change how information is represented? The resulting tools can help us trace such changes. How much of intellectual history they explain remains a question for comparative historical research.

Abstract and keywords

Abstract

Focusing on particular inferences, this essay asks when local experience supports broader claims, when evidence leaves competing explanations unresolved, when models or records lack needed distinctions, and when a representation is costly to use. “Bundled empirical coordinates” let a single cue call up several associations at once, aiding memory and action while also allowing success in one area to lend unwarranted credibility to others. Knowledge can improve through changes both to basic distinctions and to the rules for using, testing, and revising them. A road-guide thought experiment, algebraic models, and Joseph Black’s experiments illustrate, respectively, possible changes, formal constraints, and practical methods. Together they provide an analytical perspective to be tested through detailed historical research.

Keywords: bundled empirical coordinates; local fit; explanatory ambiguity; representational incompleteness; scientific representation

1. Bundling experience: How one label can carry a whole set of judgments

Imagine a guide who knows a mountain road. “Hard going in the rainy season” is enough to suggest leaving later, choosing another route, or carrying extra supplies. The phrase bundles experience with seasons, standing water, and slippery surfaces. People can make sensible decisions without working out every cause on every trip. Then the road is reinforced and its drainage improved. The rains return, but travel may no longer be difficult. The guide’s past expertise was real; the old label can still mislead today.

Call such a cue a bundled empirical coordinate: a word, category, or schema that calls up several properties, circumstances, and recommendations at once. To analyze it, we must also ask: What change would make this inference worth reconsidering?

We need to separate the distinctions we use to understand the world from the rules for using and revising them. The first is what this essay loosely calls a “basis.” The second determines which inferences are licensed and how to respond to counterexamples. A change in rules may lead us to split or supplement existing distinctions. It may also improve judgments before those distinctions change. Not every improvement in knowledge amounts to a change of basis.

Classical knowledge traditions offer larger examples of bundling. A chapter of the Suwen, a classical Chinese medical text, links directions, climate, tastes, organ categories, and emotions in sequences of correspondences. “Wood” names more than a material: it is associated with east, wind, sourness, the liver, and anger. Source text

Figure 1Enlarge ↗

The chart works like a portable index. But placing things in one column does not establish that one can be inferred from another. Associations may come from observation, symbolism, analogy, or cosmological ordering. What a category is actually being used to infer must be examined one inference at a time.

A small vocabulary need not imply simple knowledge. Labels can call up examples, procedures, and skilled judgments. Classical traditions also made distinctions within their categories. The ancient Greek physician Galen, for example, distinguished the amounts of bodily humors in a mixture from the strength of their effects. Galen’s commentary, pp. 60–66 With limited memory, compression is valuable, and rules that omit details may be well suited to particular tasks. Simon, 1956

We will first examine three limits: evidence drawn from too narrow a range of conditions, explanations the evidence cannot distinguish, and representations that lack a needed distinction. Then we will ask whether one association can be withdrawn when a limit is discovered. Modern models face these problems too.

2. Local fit: Does tracing a curve mean understanding it?

An old Chinese parable tells of a man who drops his sword from a boat. He marks the place on the boat where it fell, planning to search there when the boat stops. The mark remains; the boat has moved. Lüshi Chunqiu, “Cha Jin” The mark preserves a location on the boat, but not its relation to the place where the sword fell. Making the mark more precise would not repair that broken relation.

Why success close to home does not guarantee success farther away

Rules learned from experience also depend on conditions. Traveling one road a thousand times and traveling a thousand different roads teach different things. More experience does not necessarily mean experience of more conditions. When the setting, materials, or procedures change, we need grounds for believing that an old relationship still holds. Peters et al., 2016

Can performance in a familiar region identify a unique rule? Suppose we know only a small range of conditions where $s$ is close to zero. Consider

For $|s|\leq 0.5$, the two differ by no more than $0.025$. Observations that are not fine-grained enough may fail to distinguish them. At $s=2$, however, one gives $2$ and the other $3.6$. Agreement nearby has not settled the answer farther away.

Figure 2Enlarge ↗

This is different from overfitting. A rule may capture a local relationship accurately and still fail in another environment. Overfitting involves learning details peculiar to the sample, so performance may deteriorate even on new cases under similar conditions. A larger model may fit training data better without making more accurate predictions. Goodfellow et al., sec. 5.2

What does flexibility buy us?

Fitting a model is like using an adjustable drawing method to follow observed points. The available strokes and the ways they can be combined determine which shapes are easy to draw. Linear combinations of basis vectors are one such method. Adjustable components, nonlinear operations, and multilayer combinations give us other families of models.

Any finite set of points with distinct horizontal coordinates can be interpolated exactly by a polynomial of sufficiently high degree—even if the points contain noise. A successful fit may show that the drawing method is flexible enough. It does not, by itself, show that it has found the causes of the phenomenon. Model capacity describes the room available for adjustment. Tests under new conditions ask how far the agreement holds.

Mathematical note | What interpolation and additional capacity guarantee

Interpolating finite data.

Given $n$ points $(s_i,q_i)$ with pairwise distinct horizontal coordinates, define

$$\ell_i(s)=\prod_{j\ne i}\frac{s-s_j}{s_i-s_j},\qquad p(s)=\sum_{i=1}^{n}q_i\ell_i(s).$$

Because $\ell_i(s_j)$ is $1$ when $i=j$ and $0$ otherwise, $p(s_i)=q_i$, and $p$ has degree at most $n-1$. This guarantees only that the curve passes through those points. It guarantees neither accuracy between them nor accuracy outside their range. Nor does it establish that any fixed low-dimensional model can achieve the same fit.

Increasing model capacity. If two model families satisfy $\mathcal F\subseteq\mathcal G$, their empirical loss $\widehat L$, defined using the same loss function on the same data, satisfies

$$\inf_{f\in\mathcal G}\widehat L(f) \leq\inf_{f\in\mathcal F}\widehat L(f).$$

The reason is simply that every old candidate remains in the larger set. The inequality compares infima of training loss; it does not require a model that attains the infimum. Actual optimization may not get close to it, and no guarantee of performance on unseen data follows. Adding category names without expanding the set of functions the model can realize does not even meet the premise.

3. Explanatory ambiguity: When two indicators move together, which gets the credit?

One guide thinks rainfall is the main predictor of difficult travel; another thinks standing water matters more. If heavier rain has always meant deeper water, both may make excellent predictions. Would another hundred records of the same kind settle which guide has the better explanation?

Let $u,v$ be two observed quantities measured in specified units, $q$ the predicted outcome, and $a,b$ their weights:

If every sample satisfies $v=u$, with $u\ne 0$ in at least one case, the model reduces to $q=(a+b)u$. Suppose the recorded outcomes are exactly $q=2u$. Weights of $(2,0)$, $(1,1)$, and $(0,2)$ all fit: any choice with $a+b=2$ fits equally well.

More samples satisfying $v=u$ still identify only the sum of the weights. Yet at $u=1,v=0$, those three choices predict $2$, $1$, and $0$. Equally accurate in the past, they disagree in a new situation.

A change of coordinates cannot supply the missing evidence. Rewrite the old observations as “sum” and “difference,” and every difference is still zero. We need a new comparison on which the competing accounts diverge.

Listing the values of one observed quantity across samples gives a vector. Figure 3 rescales each of two such column vectors to length 1 so we can compare their directions:

Figure 3Enlarge ↗

Directions that are not at right angles do not automatically imply nonuniqueness. Exact collinearity causes the nonuniqueness in this example; near collinearity can make estimates unstable. Redundancy can also help with error tolerance and reconstruction: recovering an object and representing it with unique coefficients are different problems. Kovačević and Chebira, 2008

In empirical reasoning, the trouble often comes in the next step: “These occurred together before” is allowed, without further checks, to stand for “These must occur together when conditions change.” Call this overbinding in inference. Insufficient evidence does not prove the association false. The overreach lies in asking it to answer a question the evidence cannot yet settle.

Algebraic note | Why more samples of the same kind do not separate the weights

Write the two observation columns as a matrix $A=[\mathbf u\ \mathbf v]$. When $\mathbf v=\mathbf u\ne 0$, the matrix has rank $1$, not $2$. For any real number $\lambda$,

$$A\begin{pmatrix}a+\lambda\\b-\lambda\end{pmatrix} =A\begin{pmatrix}a\\b\end{pmatrix}.$$

Changing the coefficients in direction $(1,-1)$ leaves the predictions unchanged, so the current data cannot distinguish those coefficients. “Unique coefficients” in Figure 3 means that the linear least-squares solution is unique when the two given columns are linearly independent. If the target lies in their span, its exact representation is unique as well.

4. Incomplete representations: What is missing from the map?

Two stretches of road look equally long on a map, but one is an easy walk and the other leaves you breathless. If the map records horizontal position but not elevation, adding decimal places will not reveal the steep slope. Two kinds of gap need to be distinguished: the model may be unable to express the target, or the record may not preserve the distinction we need.

Start with expressive range. If the only allowed direction is $(1,0)$, adjusting its weight $c$ can take us only to $(c,0)$. The target $y=(2,1)$ also requires a step upward. The closest point the old model can reach is $(2,0)$, leaving a gap of one unit.

Figure 4Enlarge ↗

Limited expressive range does not necessarily mean missing observations. Suppose $x$ has been fully recorded, but we allow only straight lines $a+bx$ to describe $x^2$. With at least three distinct values of $x$, an exact fit is impossible. Adding the feature $x^2$, calculated from the existing record, solves the problem without a new observation. New expressive power and new information can be different things.

Geometric note | How a fixed expressive range sets a floor on error

In a finite-dimensional Euclidean space, write the available directions as the columns of a matrix $\Phi$. Their linear combinations form the subspace $S=\operatorname{span}(\Phi)$. Decompose the target $y$ into its orthogonal projection onto $S$ and a perpendicular residual:

$$y=P_Sy+r,\qquad r\perp S.$$

For any coefficients $c$, the vector $P_Sy-\Phi c$ lies in $S$. The Pythagorean theorem therefore gives

$$\|y-\Phi c\|_2^2 =\|P_Sy-\Phi c\|_2^2+\|r\|_2^2.$$

The first term is nonnegative and can be made zero with suitable coefficients, so

$$\min_c\|y-\Phi c\|_2=\|r\|_2.$$

As long as the representable range stays fixed, parameter adjustment cannot reduce the error below the length of the residual. This is a limit on the model’s form, not a verdict on all the information available. In practice, residuals may also come from noise, measurement bias, or mistakes in procedure; those possibilities require separate investigation.

From an event to a note: How different situations become the same record

Now consider information lost in recording. Someone knocks at the door. An audio recording can preserve the number and rhythm of the knocks while offering no clue to the visitor’s clothing color. A silent video showing the visitor can preserve the color but not the sound. Each selects different aspects of the event.

After listening to the recording, we might write only, “Someone knocked.” Three knocks and five knocks become the same sentence. The count was audible before; it is absent from the note:

Here $x$ is the situation. The observation rule $O$ determines what is recorded as $y$; $E$ encodes that record as $z$; and $g$ uses it to estimate a target $t(x)$, such as the number of knocks, producing $\widehat t$. Failing to observe something and observing it without retaining it are two different stages of information loss.

Figure 5Enlarge ↗

Why can’t we work backward? Because the same note could have come from three knocks or five. Whichever we guess, we cannot guarantee that it is right: the record has not preserved the evidence needed to identify the original situation. If the audio still exists, we can check it. The note alone cannot guarantee recovery.

That is what makes the process lossy: originally distinct situations have been merged. Compression or dimensionality reduction need not be lossy, however. If two columns are known to be identical throughout, storing just one still lets us recover both. The question is whether the content we need can be uniquely determined. Singh, 2015, Lecture 16

A twelve-hour clock provides another intuitive example: $3\bmod12=15\bmod12=3$. Taking the remainder discards the number of full turns and retains the position within a turn. Certain differences are explicitly treated as distinctions we will no longer make. Times separated by whole turns belong to the same equivalence class.

What does common sense add when it helps us guess?

If we know that this visitor usually knocks three times, we can make a better guess. But that support comes from background experience; it does not establish what happened this time. Background knowledge can also impose definite constraints. If it rules out every other candidate, recovery may be certain. As long as candidates with different answers remain, a plausible completion is not guaranteed to reproduce the original fact.

The limit is always relative to all the information available and the task at hand. If we ask only whether someone knocked, the note is sufficient. It is insufficient if we ask how many times or what color the visitor wore. Losing information about the whole situation does not make a record useless for every task.

General formulation | When recovery is possible, and when ambiguity remains

Fix the observation and encoding rules, and let $\mathcal X$ be the set of allowed situations. The full chain is

$$x\in\mathcal X\xrightarrow{\;O\;}y\in\mathcal Y \xrightarrow{\;E\;}z\in\mathcal Z\xrightarrow{\;g\;}\widehat t.$$

Let $R=E\circ O$. For a representation $z$, the situations compatible with it form the preimage

$$R^{-1}(\{z\})=\{x\in\mathcal X:R(x)=z\}.$$

Here $R^{-1}(\{z\})$ denotes a set; it does not assume that an inverse function exists. If this set contains two different situations, $z$ alone cannot uniquely recover the whole of $x$. Exact recovery of $x$ over all allowed situations requires $R$ to be injective. For general mappings, these preimage sets describe the ambiguity. For a linear orthogonal projection, the discarded component lies in the orthogonal complement of the projection subspace. For integers $A,B$ and a positive integer $m$, if $R(A)=A\bmod m$, then $R(A)=R(B)$ precisely when $m$ divides $A-B$. The remainder is unique; the original integer is not.

If we need only the target $t(x)$, a weaker condition suffices. An everywhere-correct rule $g$ satisfying $t=g\circ R$ exists if and only if

$$R(x_1)=R(x_2)\ \Longrightarrow\ t(x_1)=t(x_2) \qquad\text{for all }x_1,x_2\in\mathcal X.$$

Necessity follows because the same input must produce the same answer. For sufficiency, assign to each preimage set the target value shared by all its members. Thus, inability to recover the whole situation need not prevent us from determining a particular target. The claim here concerns deterministic mappings and exact recovery.

If different target values remain compatible with the same representation, prediction error has a lower bound. Let targets lie in a metric space $(\mathcal T,d)$, and suppose

$$R(x_1)=R(x_2)=z,\qquad d\bigl(t(x_1),t(x_2)\bigr)=\Delta>0.$$

The triangle inequality gives

$$\Delta\leq d\bigl(t(x_1),g(z)\bigr)+d\bigl(g(z),t(x_2)\bigr),$$

so at least one error is no smaller than $\Delta/2$:

$$\max_{i\in\{1,2\}}d\bigl(g(z),t(x_i)\bigr)\geq\frac{\Delta}{2}.$$

This is a two-point worst-case error bound for deterministic prediction under fixed information, not an average error rate. Background knowledge already built into the rule does not remove the bound merely by changing the guess. New information, or justified exclusion of one situation, changes the premises. Nearly identical representations, noise, and probabilistic predictions require additional assumptions; this formula cannot be applied to them directly.

Expressive range limits what answers the model can express. Information loss limits what distinctions the evidence supports. Identifying the gap tells us whether to change the model, add observations, or improve the record.

Further reading | Can experience and training change what we see?

What can a different way of seeing reveal?

When explaining color to someone blind from birth, is the aim to convey which things are red, or to convey what seeing red feels like? Different aims call for different kinds of understanding. Research has found that congenitally blind adults possess rich color knowledge and make some color-related inferences much like sighted adults. That does not mean the study has settled questions about first-person visual experience. Kim et al., 2021

The Buddhist Sāmaññaphala Sutta links meditative training to vision through the “divine eye,” including accounts of beings dying and being reborn. DN 2 The teaching places the acquisition of a particular kind of insight at the center. Whether the proposed ability and what it reveals are reliable remains a question to assess.

In the observation–encoding chain, a new observational ability might change $O$; learning to recognize a cue might change $E$. Practice might also change the mental state being observed. To assess whether knowledge has increased, ask: Which situations can now be distinguished that could not be distinguished before? Direct experience provides material about experience; inferences from it to the external world need further grounds. Openness to experience does not, by itself, mean openness to revising a conclusion. If every anomaly can be attributed to insufficient practice, with no way to distinguish inadequate practice from a mistaken claim, new experiences may instead protect an answer already accepted.

5. Making revision possible: Separate the judgments, then reconnect them

Even with the same information, people can respond to failure in different ways. Two guides both know the rule “Hard going in the rainy season.” After the road is improved, one keeps the old records but withdraws the claim that rainy-season travel must be difficult and checks the drainage again. The other keeps adding categories such as “an exceptional rainy season” or “hidden travel difficulty,” making every outcome fit the old account. The key question is whether one association can be reconsidered without discarding the rest of the experience.

If improved drainage can be independently established and leads to new predictions, it is a substantive revision. Revision can also mean narrowing a claim’s scope or lowering confidence in it; one mismatch need not justify abandoning an explanation. But if every possible outcome can only count as another instance of the old account, evidence loses its power to change the judgment. Popper, 1959, sec. 22

How do separate checks become actual procedures? In the eighteenth century, Joseph Black investigated a white powder called magnesia alba. It bubbled on contact with acid and formed salts. After strong heating, the material no longer bubbled. But for each acid in these comparisons, it still formed the same kind of salt as before heating. Two properties that had appeared together could now be separated. The heat treatment is called “calcination.” Here “salt” refers to a class of chemical products, not just table salt. Black, Part I, pp. 14–15

Figure 6Enlarge ↗

Could the lost property be restored? Black’s weight measurements showed that calcination made the material substantially lighter. He dissolved the calcined material in acid, then added an alkali to precipitate a powder. The precipitate bubbled in acid again, and most of the lost weight returned. He explained the changes in terms of “fixed air”—a gas that could be bound within the material and released from it—and inferred that the added alkali supplied it during restoration. Black, pp. 16–19

What role did the alkali play? In a separate experiment, he produced a precipitate using a caustic alkali that had lost its “fixed air.” This precipitate did not bubble in acid. The explanation now had another difference to account for: producing a precipitate did not guarantee the return of bubbling; the properties of the added alkali mattered too. Salt formation, bubbling, weight loss, and restoration could be tracked separately and related anew through these comparisons. Black, pp. 30–31, 35

Call this opportunity operational separability in inquiry: judgments can be checked and revised separately even when they share instruments and background assumptions. It does not guarantee that evidence will always uniquely identify which association to withdraw; shared assumptions may also need reconsideration. Black’s comparisons show how opportunities for testing can multiply: vary treatments and reagents separately, then track which properties change. For the guide, this would mean retaining established records of road conditions while testing exactly which judgments about travel the rainy season can support.

Separate the judgments, then investigate how they are connected. This is the testing opportunity that “more orthogonal” points toward in this essay. Mathematical orthogonality, statistical independence, and causal identification remain distinct concepts. Simply switching to orthogonal coordinates adds no information.

6. Choosing a representation: Why the same information can be easier or harder to use

An answer may already be in the record but be difficult to spot. Can a different way of writing it make it stand out?

The same information, different patterns in view

Two people record points on a circle. One lists horizontal and vertical coordinates; the other records each point’s distance from the center and its angle from a fixed reference direction. With the origin at the center, the same circle satisfies $u^2+v^2=4$ in the first record and simply $r=2$ in the second. The second person can immediately see that every entry in the distance column is the same.

Figure 7Enlarge ↗

Equivalent information need not cost the same to understand and use. If the task concerned the boundary of a rectangle aligned with the coordinate axes, Cartesian coordinates might be more direct.

Choosing a representation means choosing which distinctions stand out and how to extract and combine them. A full change of basis is not itself a lossy projection; discarding some coordinates afterward may erase distinctions. A projection in the strict mathematical sense also requires a specified target subspace and a rule for determining the result.

If both can approximate the target, why is one more economical?

The same target can have very different costs of expression. For the known periodic function $f(s)=\sin s$, the cubic Taylor truncation about zero is

Keeping terms through degree nine gives $T_9$: five nonzero terms that closely approximate the function over the interval shown. In the standard Fourier family, which includes $\sin s$, a single sine term represents the function exactly.

Figure 8Enlarge ↗

Completeness asks whether the representation can ultimately suffice; efficiency asks what it costs to get there. Which representation is more economical depends on the task, the required accuracy, and how cost is measured. Even when both can approximate the target, their costs at finite term counts and precision can differ greatly. This compares the cost of using given components; discovering which components to choose has an intellectual cost of its own.

Mathematical note | Why Taylor polynomials approximate the sine function in this example

A Taylor expansion determines a polynomial from derivatives at one point; it is not the same as fitting arbitrary data. For the sine function here, take a nonnegative integer $m$. On $|s|\le a$, with $a\ge 0$, the Taylor remainder gives

$$\left|\sin s-\sum_{k=0}^{m}\frac{(-1)^k s^{2k+1}}{(2k+1)!}\right| \leq\frac{a^{2m+3}}{(2m+3)!}.$$

This uses the fact that the next even-degree coefficient is zero. For fixed $a$, the right-hand side tends to zero as $m$ increases, so the truncations converge uniformly to sine on every fixed bounded interval. A general smooth function need not equal its Taylor series. “Completeness” for different function families also requires a specified function space and notion of convergence.

The two representations draw attention to different structures. Fourier coefficients highlight frequency components. Taylor coefficients record a function’s value and successive derivatives at a chosen point, highlighting local variation. Whether they recover the whole function depends on the conditions for the expansion.

If the task is to identify periodic behavior, decomposition by frequency is attractive. Once that representation is available, overlapping cycles become easier to detect. Tasks help shape representations; representations make some patterns easier to discover; new questions then prompt further changes. Tasks and representations can shape each other. But the origins of a representation do not, by themselves, establish its suitability for a new task.

What does rearranging actually add?

Shuffling changes the order of the cards; it does not add cards. Rearranging three columns labeled “rainfall,” “standing water,” and “travel conditions,” with their headings, may make relationships easier to see. It adds no observation. Six possible arrangements are not six independent pieces of corroborating evidence. Elaborate transformation rules can organize reliable records or weave untested claims together. Smooth internal operation does not settle whether a system is right about the world.

Algebraic extension | What rearrangement preserves

The six rearrangements of three columns form a permutation group under composition: consecutive rearrangements are still rearrangements; we can leave the columns unchanged; and every rearrangement can be undone. The example draws attention to what remains unchanged through the transformations: the observations in each column and their correspondence across columns. An encoding that discards information is generally not reversible, and an arbitrary collection of inference rules need not form a group.

Naming a direction does not establish an entity with that name

“Hard going in the rainy season” begins as a summary of experience. It can later become a rule that new situations are expected to obey. Names help us remember, but they can also make an explanation in need of testing seem self-evident. The associations behind a name may come from repeated observation, symbolism, or analogy. Those different origins are easy to forget in use.

Representations and names can also come apart in neural networks. A network can learn many features and multilayer nonlinear combinations without first requiring a familiar human label for every step. Its ability depends on the interplay of architecture, data, and training, not on high dimensionality alone. LeCun et al., 2015 One neuron may contribute to several features, and one feature may be distributed across several neurons. Human labels need not map one-to-one onto the internal computation. Elhage et al., 2022 Researchers have identified and intervened on some features and computational pathways, though their coverage remains limited. Anthropic, 2025

Computing effectively, explaining the computation in familiar words, and establishing that it corresponds to a real mechanism are three different achievements.

7. Evolving bases: A better map for which journey?

This perspective is meant to help us choose the next step. The following guide organizes the questions to ask about a judgment. Several difficulties may be present at once.

Try another setting. A gardener’s successful experience comes entirely from a greenhouse. Watering has always coincided with fertilizing, and sunlight was never recorded. Will the method work outdoors? What did water and fertilizer each contribute? Does sunlight matter? These questions call, respectively, for comparisons across conditions, separate variation, and additional observations. If an outdoor trial fails, the greenhouse successes can remain on record while the claim “It works everywhere” is withdrawn and outcomes under different sunlight conditions are examined. Failure does not require erasing the past. Calling it a “special case” does not explain it either.

Return to the map. Adding elevation introduces another aspect. Replacing “high” and “low” with measured height differences increases resolution. No longer letting elevation stand in for road conditions separates judgments. All three may help, depending on the task and what has been checked. To find a train station, a schematic may be more useful than a topographic map. The value or cost of an omission depends on the question. Potochnik, 2017

If a richer record allows the old record to be recovered, it preserves the possibility of reusing old methods. Practical gains still depend on the costs of obtaining, learning, and using the information. Representations such as audio and silent video may preserve different information, making them impossible to rank on a single scale of “more.”

Formal note | Preserving old information also preserves old strategies

If a new representation $R_2$ can recover an old representation $R_1$ through a map $h$, so that $R_1=h\circ R_2$, any old strategy $g_1$ can be reused as $g_2=g_1\circ h$, producing the same output:

$$g_2(R_2(x))=g_1(R_1(x)).$$

With the task and evaluation criteria fixed, reuse of these strategies allowed, and additional information and computation costs excluded, the new strategy set includes every performance achievable by the old strategies. Optimal performance therefore cannot be worse. Limits on learning, measurement error, and costs of use affect the gains in practice.

Human memory, time, and computational ability are limited. This “bounded rationality” helps explain why compression can be useful. Research on scientific representation asks how representations support judgments about their objects. Simon, 1956; Nguyen and Frigg, 2022 This essay connects those questions: how can basic distinctions, inferential links, and revision rules work together so that withdrawing one association does not destroy other well-supported judgments?

To turn this into historical research on evolving bases, we must compare before and after: What can now be distinguished that could not be distinguished before? Which formerly bundled judgments can now be checked separately? For which new questions can we now test possible answers? Black’s experiments provide one such episode. Studying further changes also requires tracing how instruments, materials, training, and institutions made them possible.

The opening research program makes the broader claim that such restructurings play a foundational role in the development of human knowledge. Supporting it requires repeated evidence, across comparable tasks, that major advances depend on improved distinctions, opportunities for testing, or scope of application. If major advances chiefly result from executing the same procedure more skillfully or cheaply, without systematic change in those respects, the program’s explanatory scope should be narrowed. When the tasks themselves change, we must make the standard of comparison explicit before saying what has become “more complete.”

Preserving sound experience and revising theory can therefore proceed together: one success need not underwrite every explanation, and one failure need not erase every observation.

Return to the systems that seem “so accurate.” We can now ask a further question: What did this success actually rule out—and what result would change my judgment?

Experience helps us learn a familiar road. Knowledge that remains open to revision should also tell us why we think the road is passable, when it is not, and what we need to add to the map next.

References

1. Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review, 63(2), 129–138. DOI; full text in the author’s archive.
2. Nguyen, J., & Frigg, R. (2022). Scientific Representation. Elements in the Philosophy of Science. Cambridge University Press. DOI.
3. Potochnik, A. (2017). Idealization and the Aims of Science. University of Chicago Press. Publisher’s page.
4. Popper, K. (1959). The Logic of Scientific Discovery. Hutchinson. Section 22 distinguishes falsifiability from actual falsification. Relevant chapters; later publisher reprint.
5. Kovačević, J., & Chebira, A. (2008). An introduction to frames. Foundations and Trends in Signal Processing, 2(1), 1–94. DOI.
6. Black, J. (1756). Experiments upon magnesia alba, quicklime, and some other alcaline substances. Essays and Observations, Physical and Literary, 2, 157–225. Citations here refer to Alembic Club Reprints, No. 1, 1898 reprint, Part I, especially pp. 14–19, 30–31, and 35. The paper was read in 1755. Digital transcription.
7. Huangdi Neijing, Suwen, “Yin Yang Ying Xiang Da Lun” (《黄帝内经·素问·阴阳应象大论》), especially the passages linking east, wind, wood, sourness, liver, and emotions. Chinese Text Project (Chinese).
8. Galen. Commentary on Hippocrates’ On the Nature of Man. English translation by W. J. Lewis, with assistance from J. A. Beach, UCL Medicina Antiqua digital text. The passages cited correspond to Kühn, vol. 15, pp. 51–66; numbers at the beginnings of paragraphs indicate pages in that edition. Translation.
9. Peters, J., Bühlmann, P., & Meinshausen, N. (2016). Causal inference using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society: Series B, 78(5), 947–1012. Authors’ preprint and journal information.
10. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press. Chapter 5, especially sec. 5.2, “Capacity, Overfitting and Underfitting.” Authors’ website.
11. Singh, A. (2015). Information Processing and Learning, Lecture 16: Sufficient statistics, information bottleneck, lossy compression. Carnegie Mellon University. Lecturer and scribe: Aarti Singh; co-scribe: Danai Koutra. Sections 16.1–16.3 inform the distinction between preserving task-relevant information and lossy compression. Lecture notes.
12. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436–444. Article and abstract.
13. Elhage, N., et al. (2022). Toy Models of Superposition. Transformer Circuits Thread. Model studies of features, directions, polysemantic neurons, and superposition. Full report.
14. Anthropic (2025). Tracing the thoughts of a large language model. March 27, 2025. A report on selected computational pathways in Claude 3.5 Haiku and the method’s limitations, with links to the original papers. Research report.
15. National Center for Complementary and Integrative Health (2019). Traditional Chinese Medicine: What You Need To Know. Updated April 2019. Evidence organized by specific practices and health conditions. Agency overview.
16. Zhang Tingyu et al. History of the Ming (《明史》), vol. 74, Treatise 50, “Official Posts III,” entry on the imperial astronomical bureau (钦天监). The cited passages describe advance eclipse calculations, reports based on divination manuals, and the omen-reading duties of the baozhang zheng official. Digital transcription (Chinese).
17. Sāmaññaphala Sutta (《沙门果经》), Dīgha Nikāya, DN 2, especially “The Passing Away & Re-appearance of Beings,” on vision through the “divine eye” following meditative concentration. Dhammatalks translation.
18. Kim, J. S., Aheimer, B., Montané Manrara, V., & Bedny, M. (2021). Shared understanding of color among sighted and blind adults. Proceedings of the National Academy of Sciences, 118(33), e2020192118. DOI; full text. A comparison of color knowledge and inference in congenitally blind and sighted adults.
19. Forer, B. R. (1949). The fallacy of personal validation: A classroom demonstration of gullibility. Journal of Abnormal and Social Psychology, 44(1), 118–123. See pp. 120–121 for procedure and results. DOI; scan of the original article.
20. Lüshi Chunqiu (《吕氏春秋》), “Shen Da Lan,” “Cha Jin,” the passage about marking a boat to find a lost sword. The source uses the parable to discuss changing circumstances and the adaptation of laws. Chinese Text Project (Chinese).