Why does the assistant sometimes answer without showing products?

By design. Greetings and general questions don't trigger a catalog search, and product cards only appear after the agent actually calls a catalog tool. That keeps it from attaching random products to small talk.

How does photo search work?

The shopper uploads a photo. A vision model describes it, and the agent decides whether that description should become a catalog search. Results are then ranked the same way as a typed search.

What happens when the catalog grows beyond the pilot?

The catalog loads through one interface, so the sample file can be swapped for a database, a vector store or the store's product API without changing the agent. Embeddings are cached in memory today; a full catalog would store them persistently.

Catalog assistant with photo search

Catalog assistant with photo search for a resin manufacturer

By Sagar DavaraPublished 23 September 2026

Pilot

In short

Mirai Minds built a pilot catalog assistant for a resin manufacturer in India. Shoppers ask in plain language or upload a photo; an OpenAI Responses API agent decides whether to search the catalog, ranks products with hybrid semantic and keyword search, and shows product cards only from real results. The pilot runs on a 15-product sample of the catalog.

The problem

The manufacturer sells resin-art supplies, such as pigments, through its online store. It wanted an assistant that answers product questions in plain language and can find items from a photo a shopper uploads, without inventing products, prices or links.

What we built

An agent on the OpenAI Responses API. Every message goes to one agent, which either replies conversationally or calls a catalog tool for text or image search.
Hybrid retrieval. Semantic similarity and keyword matching are combined, and each result carries a label saying how it matched. If embeddings fail, keyword search still returns results.
Photo search. Uploads of up to 5 MB in JPG, PNG, WEBP or GIF are described by a vision model, which also decides whether a catalog search makes sense.
Product cards. Cards show the catalog's title, description, price and link, with images fetched from the store's public product pages and a placeholder when an image can't be resolved.
Tests. Automated tests cover agent decisions, tool calls, conversation history, image handling and failure cases, and run before every build.

How it works in production

The pilot runs on Cloudflare Workers with the catalog loaded from a 15-product sample of title, description, price and link. Categories the system infers help ranking, but they are never shown to shoppers as catalog facts: everything a shopper sees comes from the catalog itself. The whole flow runs through one chat endpoint, which keeps the pilot easy to embed in the store. The same grounding pattern is described under custom LLM and document AI, and photo search under computer vision.

What we'd change

Connect the live catalog before measuring anything. Fifteen products are enough to test behavior, not retrieval quality.
Store embeddings persistently. Rebuilding them after each restart is fine for 15 products and wasteful for a full catalog.
Turn pilot traffic into a test set. Real shopper questions and photos, with the products a person would pick, are the benchmark the next phase needs.