1. A somewhat hard story: tensors
This piece opens with a story that has no easy way in. Tensor is a word used widely, in physics, engineering and now AI, but only a tiny fraction of everyone on Earth can truly understand it and explain it. Even in specialist work, the number of axes of a dimension and the rank number of a tensor get mixed up, and in deep learning papers and tool docs the single word dimension takes turns meaning both. Mathematics, physics and deep learning each handle the problem to their own taste, and there is still no integrating discipline to sort out, in one frame, how their usages disagree and how far each goes wrong. So it is fine if the first section doesn't read cleanly the first time. You can reach where this piece is going without understanding tensors all the way. This section only marks what a tensor is, Section 2 where the two kinds of tensor part ways, and from Section 3 on the piece turns to what that split changes in how we read LLMs and JSON.
Tensors are layered by how many times coordinate values are laid out, and that count is called the rank. A scalar, rank 0, marks a pure quantity in itself, with no axis coordinate system assumed. However many axes you set up and whichever way you turn them, its value stays the same, so it has nothing to do with the axis count. A temperature or a mass is a scalar. A vector, rank 1, is one set of coordinate values with an axis coordinate system assumed. A matrix, rank 2, is those sets laid out in a row again. Each further layer of laying-out raises the rank by one. The axis count only sets how many values go into a set; it does not change the rank. Whether a vector sits in a 3-axis or a 768-axis coordinate system, its rank is 1. Scalar, vector and matrix are nicknames given to the rank-0, rank-1 and rank-2 tensor. Tensor is the general name for all of them, whatever the rank.
2. Physical tensors and deep learning tensors part ways
Most of the tensors mentioned on YouTube and the like these days are LLM tensors. Yet physics students also have to learn the tensor concept to work through quantum mechanics, such as Heisenberg's matrix mechanics. In quantum mechanics the vector that holds a state, a rank-1 tensor, is never observed directly; each observable quantity is represented by a matrix, a rank-2 tensor, acting on that vector, and a measurement returns one of the values that matrix allows. And the prototype of the tensor that physics uses is in turn handled by mathematics.
This section sets the physics tensor against the deep learning tensor. The shared name makes them sound like one thing, but they do different jobs. A tensor in physics is an arrangement built so that it points to the same thing no matter which direction you look from. A tensor in deep learning is a table of numbers stacked in rows, columns, and layers. If one spreadsheet is a rank-2 tensor, a stack of those sheets is a rank-3 tensor.
Things like this are often called three-dimensional tensors, but strictly speaking that is an improper expression. There are rank-3 tensors; there is no such thing as a three-dimensional tensor. And what separates the physics tensor from the deep learning tensor is neither rank nor axis count. It is whether there is a promise that, when the coordinates change, the coordinate values change together by a fixed rule. A physics tensor has to keep that promise to count as a tensor at all; a deep learning tensor makes no such promise.
In the physical tensors most commonly used, every layer uses the same axis coordinate system. A rank-2 stress tensor in three-axis space uses the same three axes for both layers, so it has 3×3 coordinate values. Rotate the axes and every value changes, but all layers change together by the same rule, so what they point to stays the same. That promise is the transformation rule, and it is why textbooks define a physical tensor as a multilinear quantity that follows specific rules under coordinate transformation. Because the rule is part of the definition, a physical tensor is not tied to any one coordinate system. Stress tensors and curvature tensors are examples, used in the formal domain where fixed axes, an assumed substance, and a demand for identical computation all hold — the formal-level transformation of <Cognitive Axioms 04_2>.
A deep learning tensor is a container that holds numbers in several layers. Often each layer has its own axis count, as with batch size, token count and feature count, and sometimes the counts match, as in a 768×768 matrix. Either way, it promises no rule to follow when the coordinates change. It is usually called a multidimensional array; it shares only the name with the physical tensor, and its purpose and mathematical structure are different. What LLMs use is the deep learning tensor.
3. What an LLM tensor is
Seen from my axis coordinate system, an LLM's tensor can also be understood as something like a modified, simplified version built to hold the relational components between tokens — position, arrangement, layer, and so on.
Three of the four elements of an axis coordinate system survive intact in the LLM tensor. There are many axes, and coordinates are measured as real numbers. Units aren't dropped either. A unit isn't something agreed on and handed down from outside; it's built into the act of measurement called scaling. The scaling done in an LLM tensor is itself the act of setting units, so the units are already in place.
The only thing dropped is the axis names. Nothing inside an LLM records what each axis points to; the names live at the level of input, output, and interpretation. In this sense the LLM tensor is an axis coordinate system with only the names removed, and it is distinct from the physical tensor, whose definition includes a transformation rule.
4. JSON converges
Many data formats have been tried, but JSON — a plain-text delivery system that pairs axis names with coordinates in the simplest possible form — has settled in as today's most popular general-purpose format for passing data around. JSON started from the goal of defining data and handing it over, and defining data calls for an axis coordinate system. JSON's two structures, objects and arrays, match the two ways an axis coordinate system points at things: by name and by order.
JSON carries no axis transformation formula, no assumed substance, no coordinate independence. The demand for definition, the match with the two ways of pointing, and the absence of axis transformation and substance: these three made JSON converge on the axis coordinate frame.
5. JSON's address
JSON becoming the general-purpose data format is an event on a separate track from the axis coordinate system. But the structure the format came with pointed at the axis coordinate frame from the start, and since that frame is the underlying structure of human cognitive computation, JSON's structure converges on it regardless of how it came to be widely used.
In the axis coordinate systems used in cognitive computation, the only thing in focus is the object's coordinates. The number of axes, the axis names, and the units are set up as preconditions for deriving them. For recording, preserving, and consulting things as data, JSON has been converging on the form of a container — a vehicle — that pairs coordinates with axis names and carries them. That is JSON's address as seen from the axis coordinate frame.