# Catalog assistant with photo search for a resin manufacturer

> Mirai Minds built a pilot catalog assistant for a resin manufacturer in India. Shoppers ask in plain language or upload a photo; an OpenAI Responses API agent decides whether to search the catalog, ranks products with hybrid semantic and keyword search, and shows product cards only from real results. The pilot runs on a 15-product sample of the catalog.

Client: Resin manufacturer (India) (Resin art supplies) · Status: pilot · Year: 2026–present · Written by Sagar Davara

## Key facts

- **Agent:** OpenAI Responses API with catalog search tools
- **Search:** Hybrid semantic and keyword ranking with match labels
- **Photo search:** JPG, PNG, WEBP or GIF up to 5 MB, described by a vision model
- **Grounding:** Product cards appear only after a catalog tool call
- **Hosting:** Next.js on Cloudflare Workers
- **Pilot data:** 15-product catalog sample


## The problem

The manufacturer sells resin-art supplies, such as pigments, through its online store. It wanted an assistant that answers product questions in plain language and can find items from a photo a shopper uploads, without inventing products, prices or links.

## What we built

- **An agent on the OpenAI Responses API.** Every message goes to one agent, which either replies conversationally or calls a catalog tool for text or image search.
- **Hybrid retrieval.** Semantic similarity and keyword matching are combined, and each result carries a label saying how it matched. If embeddings fail, keyword search still returns results.
- **Photo search.** Uploads of up to 5 MB in JPG, PNG, WEBP or GIF are described by a vision model, which also decides whether a catalog search makes sense.
- **Product cards.** Cards show the catalog's title, description, price and link, with images fetched from the store's public product pages and a placeholder when an image can't be resolved.
- **Tests.** Automated tests cover agent decisions, tool calls, conversation history, image handling and failure cases, and run before every build.

## How it works in production

The pilot runs on Cloudflare Workers with the catalog loaded from a 15-product sample of title, description, price and link. Categories the system infers help ranking, but they are never shown to shoppers as catalog facts: everything a shopper sees comes from the catalog itself. The whole flow runs through one chat endpoint, which keeps the pilot easy to embed in the store. The same grounding pattern is described under [custom LLM and document AI](/services/custom-llm-development), and photo search under [computer vision](/services/computer-vision).

## What we'd change

- **Connect the live catalog before measuring anything.** Fifteen products are enough to test behavior, not retrieval quality.
- **Store embeddings persistently.** Rebuilding them after each restart is fine for 15 products and wasteful for a full catalog.
- **Turn pilot traffic into a test set.** Real shopper questions and photos, with the products a person would pick, are the benchmark the next phase needs.

## Stack

- **AI:** OpenAI Responses API, OpenAI embeddings, OpenAI vision model
- **App:** Next.js 16, React 19, Tailwind CSS 4
- **Hosting:** Cloudflare Workers, Wrangler
- **Testing:** Node test runner: agent and retrieval tests

## Frequently asked questions

### Why does the assistant sometimes answer without showing products?

By design. Greetings and general questions don't trigger a catalog search, and product cards only appear after the agent actually calls a catalog tool. That keeps it from attaching random products to small talk.

### How does photo search work?

The shopper uploads a photo. A vision model describes it, and the agent decides whether that description should become a catalog search. Results are then ranked the same way as a typed search.

### What happens when the catalog grows beyond the pilot?

The catalog loads through one interface, so the sample file can be swapped for a database, a vector store or the store's product API without changing the agent. Embeddings are cached in memory today; a full catalog would store them persistently.


---

Canonical: https://www.miraiminds.co/work/resin-catalogue-assistant
Last updated: 2026-09-23
Publisher: Mirai Minds LLP, 906 Sarthana Business Hub, Nana Varachha, Surat, Gujarat 395013, India. hello@miraiminds.co