Embedding based semantic search product manager

Semantic search is the most quietly product-shaped problem in AI. The engineering is tractable (embed, index, retrieve), but whether results feel right is decided by a chain of product judgments: what "similar" should mean in your domain, what gets embedded versus filtered, how relevance is scored against business goals, and what the user sees when confidence is low. Teams that treat search as a backend ticket ship search that technically works and practically disappoints.

The system I own in production

WisOwl AI's core is an embedding-based semantic matching engine I designed and implemented on FAISS and Supabase pgvector, matching candidates and roles in the Indian hiring market in real time. Recruiting is an unforgiving domain for semantic search: a "similar candidate" who's similar in the wrong dimension (same words, wrong seniority; same title, wrong stack) costs the recruiter time and the platform trust. Getting it right enough to hold 10+ recruitment-firm and startup customers and 10,000+ jobseekers meant learning what actually moves relevance:

• Representation beats model choice. What text you embed, and how you structure it, matters more than which embedding model you pick. We rebuilt our candidate representation twice; each rebuild moved relevance more than any model swap.
• Hybrid is usually the answer. Embeddings for concepts, lexical for exact terms (names, skills, codes), and a reranker on top. Pure-vector search fails on precisely the queries where users most expect precision.
• Relevance needs a labeled truth. A golden set of queries with human-judged results, measured with recall and NDCG, turns "search feels off" into a tractable engineering conversation. Building that set is product work, because someone must decide what right means.
• Low-confidence UX is part of search. Showing weak matches confidently erodes trust faster than showing fewer results honestly.

Working together on semantic search

Common engagements: a relevance audit (two weeks: golden set construction, retrieval metrics, representation review, ranked fixes), or fractional ownership of your search/matching product while it becomes the asset your business needs it to be. Backed by a decade of PM work, including eight years of growth product at CaaStle, so I measure relevance improvements in retention as well as NDCG.

Frequently asked questions

Users say our search "doesn't get it." Where do you start?

With a labeled golden set: fifty real queries, human-judged results, and retrieval metrics. That turns a vibe into a diagnosis, usually split between representation problems (wrong text embedded) and missing hybrid/lexical handling, in that order.

Which embedding model is best?

The one that wins on your golden set at your price point. It's genuinely domain-dependent, and cheaper models win more often than the leaderboards suggest. Your eval set makes the choice empirical and re-decidable every time models improve.

Semantic search versus keeping our existing keyword search?

Keep both. That's what hybrid means. Keyword search is excellent at what users type exactly; embeddings rescue what they mean. Replacing lexical search wholesale is the most common regression I get called in to fix.

How do you measure search success at the business level?

Search-to-outcome conversion (found → acted), abandonment rates, and repeat usage of search-dependent workflows. At WisOwl the number that matters is whether recruiters come back and run tomorrow's roles through us. Yours has an equivalent, and we'll define it in week one.

Related pages