CallAuditAI: rubric scoring for counseling and sales calls

The problem

Sales and counseling teams record their calls, but managers can listen to only a few. The admissions counseling company wanted every recorded call scored the same way, against a fixed rubric, so coaching could start from the calls that most needed it.

What we built

A small service with one job: take a recording and the counselor's name, and return a scorecard.

• Scoring. Gemini listens to the audio and scores each criterion of the rubric. The rubric has four categories: opening and rapport building (30%), needs discovery and information gathering (30%), value proposition and objection handling (20%), and closing (20%). Each criterion carries its own weight inside its category.
• The scorecard. An Excel sheet with section headers, the weight of every criterion, category totals and an overall score. Any criterion scored zero is highlighted as a red flag. The same result is available as JSON for teams that want to load it elsewhere.
• Transcripts and costs on request. A diarized transcript, labeling the company representative and the customer, comes from a separate, lighter Gemini call. Model cost per call is returned only when the request carries a secret key, so cost details stay out of everyday responses.
• A log and an upload page. Each processed call and its duration are logged, and a simple web page lets staff upload recordings without touching the API.

How it works in production

The client's team uploads recordings, one per request, and downloads the scorecards. Scoring and transcription are separate calls, so the transcript can be skipped when only the score is needed. The service was deployed for the client in January 2026; in September 2026 the product gained JSON output, transcripts and cost reporting. For the broader pattern, see custom LLM and document AI; for the calls themselves, voice AI agents.

What we'd change

• Calibrate against human scores. Before trusting totals, we'd have managers score a sample of calls blind and compare with the model criterion by criterion. Subjective criteria, such as warmth and tone, need that check most.
• Batch uploads and trends. One file per request keeps the service simple. Teams with higher call volumes want batch uploads and per-counselor trends over time.