# Visual product search and cart-recovery calls for an ethnic-wear brand

> Mirai Minds built two AI add-ons for an ethnic-wear e-commerce brand in India. Staff photograph a product, or a marketing reel playing on a screen, and get matching catalog products back from an image-embedding index. A separate service checks abandoned checkouts every 5 minutes and triggers a recovery call from our Voice Agents platform.

Client: Ethnic-wear e-commerce brand (India) (Fashion e-commerce) · Status: production · Year: 2026 · Written by Sagar Davara

## Key facts

- **Visual search input:** A staff photo of a product or of a reel on screen, up to 300 KB
- **Index:** Catalog photos plus frames from marketing reels
- **Results:** Top 10 products, de-duplicated across photo and reel matches
- **Isolation:** Search runs apart from the production database
- **Cart recovery:** Abandoned checkouts checked every 5 minutes
- **Calls:** Placed by our Voice Agents platform, with the cart as context


## The problem

The brand sells ethnic wear online, and its staff use an in-house app. Two gaps stood out. Staff needed a fast way to find a product from what a customer shows them, which is often a photo or a reel rather than a product name. And abandoned checkouts were going without any follow-up.

## What we built

- **Visual product search.** A staff endpoint takes a photo and sends it to a separate search service that holds the image-embedding index. The index scores two kinds of material with the same query: catalog product photos and frames from the brand's marketing reels. Each product keeps its best score across both, results are de-duplicated, and the top 10 come back with direct product links. Every failure, from missing credentials to an unreachable index, returns one clear "visual search unavailable" message instead of a raw error.
- **Reel indexing.** Reels are processed incrementally on AWS Fargate, frames are indexed once, and reel matches map to the products tagged in that reel.
- **Cart recovery.** A small FastAPI service polls the store's abandoned checkouts every 5 minutes through the Shopify Admin GraphQL API, skips carts it has already handled, adds the cash-on-delivery charge and asks our [Voice Agents platform](/work/voice-agents-platform) to place a call with the cart as context. Phone numbers are masked in its logs.

## How it works in production

Visual search went live for staff in July 2026. It is connected privately to the production API but runs apart from the production database, so search load stays off the store's database. Uploads are capped at 300 KB, which keeps the round trip short from a phone on a shop floor. Cart recovery runs on its 5-minute schedule; the voice agent handles the conversation, and call outcomes land in the platform's call history. The underlying work is described under [computer vision](/services/computer-vision) and [voice AI agents](/services/voice-ai-agents).

## What we'd change

- **Measure recovery with a holdout group from the first week.** Without one, the effect of a call can't be separated from customers who would have come back anyway, so we don't publish a recovery rate.
- **Use webhooks where the checkout supports them.** A 5-minute poll is simple and dependable, but a webhook would let the call go out sooner after the cart is abandoned.

## Stack

- **Visual search:** Image-embedding index, AWS Lambda, AWS Fargate for reel processing
- **Store:** Shopify Admin GraphQL API, FastAPI, SQLAlchemy
- **Voice:** Mirai Minds Voice Agents platform
- **Scheduling:** APScheduler

## Frequently asked questions

### Why index video reels as well as product photos?

Staff often see a product in a marketing reel rather than on a product page. Frames from each reel are indexed, and a reel match returns the products currently tagged in that reel, so a photo of a screen still finds the right item.

### What does the recovery call know about the cart?

The service reads the abandoned checkout from the store: items and variants, totals, discounts and taxes, plus a cash-on-delivery charge it calculates. That goes to the voice agent as call context, so the agent talks about the actual cart.

### How do you know the calls recover revenue?

We don't have a clean number yet, which is why none is published here. The honest way to measure it is a holdout group of abandoned carts that get no call, compared over the same weeks.


---

Canonical: https://www.miraiminds.co/work/ecommerce-visual-search-and-cart-recovery
Last updated: 2026-09-23
Publisher: Mirai Minds LLP, 906 Sarthana Business Hub, Nana Varachha, Surat, Gujarat 395013, India. hello@miraiminds.co