Embeddings

How AI converts meaning into math — and why similar things end up close together.

// The Concept

An embedding is a dense vector representation of text, images, or other data in a continuous high-dimensional space. The defining property is deceptively simple: semantically similar items produce vectors that are close together. "King" ends up near "Queen." "Python programming" lands close to "coding in Python" but far from "python snake." "Machine learning consultant" clusters near "AI strategy advisor" even though the phrases share no words at all.

This is fundamentally different from how computers traditionally represented text. In older systems, every word was an isolated symbol with no inherent relationship to any other word. "Dog" and "canine" were as unrelated as "dog" and "quantum." Embeddings changed that by projecting words — and eventually sentences, paragraphs, and entire documents — into a geometric space where distance corresponds to meaning.

The idea traces back to distributional semantics: the hypothesis that words appearing in similar contexts have similar meanings. Word2Vec (2013) operationalized this by training a shallow neural network to predict context words, producing 300-dimensional vectors where arithmetic worked on meaning. The famous example: vector("King") - vector("Man") + vector("Woman") produces a vector closest to "Queen." That result demonstrated that embeddings don't just capture similarity — they encode structured semantic relationships.

Modern embedding models are far more powerful. They process entire passages, not just words. OpenAI's text-embedding-3-large produces 3,072-dimensional vectors. Cohere's embed-v3 supports multilingual input. Google's Gecko powers Vertex AI search. These models are trained on massive corpora with sophisticated objectives that capture nuance, context, and domain-specific meaning. Every piece of content you publish gets embedded by these systems — and its position in that high-dimensional space determines what it gets associated with, retrieved alongside, and recommended next to.

// How It Works

An embedding model takes a variable-length input (a word, sentence, or document) and maps it to a fixed-length vector. Training teaches the model to place semantically similar inputs close together, measured by cosine similarity — the angle between two vectors in the high-dimensional space.

The training process is what makes embeddings powerful. Modern models use contrastive learning: they're shown pairs of similar texts (positive pairs) and dissimilar texts (negative pairs), and they learn to push positive pairs together and negative pairs apart in vector space. The model processes text through transformer layers, and the final representation — often the mean of all token embeddings, or a special [CLS] token — becomes the document's embedding.

Dimensionality matters. More dimensions allow the model to capture finer-grained distinctions, but at computational cost. A 3,072-dimensional vector can represent far more nuance than a 300-dimensional one. The cosine similarity between two 3,072-dimensional vectors can distinguish between "AI strategy consultant in Denver" and "AI strategy consultant in Boston" — a level of specificity that lower-dimensional embeddings simply cannot encode.

One critical property: embeddings are dense, not sparse. Every dimension carries information for every input. This is the opposite of traditional bag-of-words representations, where most dimensions are zero. Dense representations allow embeddings to capture relationships that keyword matching entirely misses — synonymy, paraphrase, conceptual analogy, even cross-lingual meaning.

// Why It Matters for Search

Semantic search, RAG retrieval, and entity resolution all operate in embedding space. When an AI system decides whether your page is relevant to a query, it is computing the cosine similarity between the query's embedding vector and your document's embedding vector. This is not keyword matching. This is geometric proximity in a space where meaning has been compressed into coordinates.

When Google serves an AI Overview, or Perplexity retrieves sources for an answer, or ChatGPT with browsing selects pages to cite — in every case, the retrieval step involves embedding similarity. Your content's position in embedding space determines what queries it gets matched to, what other documents it clusters with, and ultimately whether it gets selected as a source.

This is why entity SEO is more powerful than keyword SEO in the AI era. Keywords live in string space — exact matches or nothing. Entities live in embedding space — semantic neighborhoods where related concepts, credentials, and authority signals cluster together. A well-constructed entity presence creates a dense, coherent region in embedding space that captures queries from multiple angles, even queries that use completely different language than your page.

Your content's embedding is arguably more important than your keyword usage. Two pages about "entity SEO" with identical keyword density will occupy different positions in embedding space based on the depth of their content, the specificity of their examples, the authority signals in their schema, and the semantic coherence of their surrounding text. The one with richer, more specific, more authoritative content will occupy a better neighborhood — closer to the queries that matter.

// In Practice

Use consistent entity terminology across all pages and platforms. When AI systems embed your entity name, they place it into a region of vector space. Inconsistent naming fragments this signal. If you use "Green" on one page, "Guerin Green" on another, and "NovCog" on a third without connecting them, you create three separate, weak points in embedding space instead of one strong cluster. Consistent naming with explicit connections (schema sameAs, cross-references, co-occurrence) creates a tight, reinforced embedding neighborhood.

This is precisely why the Distributed Authority Network strategy works at the embedding level. When your entity name appears consistently across your personal site, your agency site, your GitHub profile, your community pages, and your schema markup — all using the same @id reference, the same name string, the same credential descriptions — the embedding models encounter this consistent signal across multiple sources. The result is a robust, high-confidence embedding that resists dilution and surfaces reliably across diverse queries.

Write content with clear topical focus. Each page should embed into a specific, well-defined region of vector space. A page that tries to cover five topics creates a muddled embedding that sits between all of them, close to none. A page that goes deep on one topic creates a sharp, precise embedding that matches strongly on relevant queries. Structure your site so that each page owns a specific conceptual territory in embedding space, and use internal linking and schema to connect those territories into a coherent map of your expertise.

Cross-platform consistency is your multiplier. The same entity description on your website, your Google Business Profile, your LinkedIn, your GitHub — each appearance reinforces the same point in embedding space. Embedding models are trained on crawled web data. When they see the same entity described consistently across multiple authoritative sources, the resulting embedding is stronger and more stable than one built from a single source.