An embedding can look substantially different after an orthogonal change of basis, even though its cosine similarities have not changed. I built a small browser instrument that makes that mismatch visible: the points move, the axes change, and every phrase keeps the same five cosine neighbours.
The measured numerical drift stays below 2e-15
across 180 short phrases embedded into 384 dimensions with MiniLM. That is the narrow result. It is not a claim that every transformation preserves an embedding, or that individual axes can never be useful.
The working embedding-tours instrument is on billiem.uk. It is not embedded here. Open it to cycle through raw coordinate pairs, PCA pairs and a seeded grand tour, then use the basis control in raw-coordinate mode to switch between the original and orthogonally transformed coordinates.
The raw-coordinate view is where the central comparison is easiest to see. In the original basis, the instrument moves through coordinate pairs: dimensions 1 and 2, then 2 and 3, and onwards. Switching the basis applies the same orthogonal transformation to every vector. The resulting coordinate tour can look quite different because the individual coordinates have changed.
The cosine-neighbour structure has not changed. A shared orthogonal change of basis preserves dot products, lengths, distances and cosines. In this experiment, all five cosine neighbours are retained for every phrase.
That makes the visual difference useful precisely because it is a little unsettling. The raw coordinates supplied by the model do not have a unique claim on the neighbour geometry. Another basis can produce a different-looking tour while representing the same cosine relationships.
This is also the limited point behind work on the interpretability of embedding coordinates. It does not follow that axes are always meaningless. It follows that cosine structure alone does not privilege the particular raw axes I happened to receive.
The instrument has two datasets: a deterministic eight-dimensional synthetic dataset and the 384-dimensional phrase embeddings. Both can be viewed in three modes.
These are not three labels over the same picture. They produce meaningfully different views of the same dataset. The selection controls also let you follow one point and inspect its original-space neighbours while the projection changes.
The distinction between the first and third modes already has established names. The tourr project describes cycling through axis-aligned views as a little tour. Daniel Asimov's
I found that history after starting with the simpler coordinate-pair animation. I had not invented a new visualisation technique. I had independently reached the entrance to an existing family of ideas, which was probably the coolest part of the experiment.
There is much more complete work in this area. Distill used the grand tour to inspect neural-network activations, and the recent dtour project provides a broader browser interface for steering through high-dimensional data and embeddings. My instrument is not a replacement for either.
Every frame remains a two-dimensional projection. Animating the projection does not recover everything lost when hundreds of dimensions are placed on a flat screen. It gives more partial views.
That limitation matters when watching clusters appear, separate or overlap. The tour can show how structure changes across projection planes, but it does not turn the screen into a faithful view of the complete high-dimensional space.
The basis switch is the sharpest version of that warning. A projection can change dramatically while the measured cosine-neighbour geometry stays fixed. The picture moved; the neighbour relationships did not.
The experiment stayed deliberately small: two datasets, three projection modes, a basis comparison, selection and playback controls. I do not expect a researcher to find a new technique in it. It may help another developer who is still learning and finds embeddings confusing, or it may just be satisfying to watch the structure move. I think the narrow result is worth seeing either way.
This article was adapted with AI assistance from an original article on billiem.uk. The original article was reviewed before publication.