AI phone agent pilot
How to run an AI phone agent pilot in your service department.
A pilot has one job: show whether the AI books real appointments and hands off cleanly when a person is needed, on your calls, at your store. Here is a 30-day plan: where to start, what to measure each week, which test calls to place, and what Cleo shows you along the way.
The plan
A 30-day pilot in four steps.
Before day 1: write down your baseline
From your own phone system, write down last month’s calls by hour and day, the calls that went unanswered or to voicemail, and the calls after closing. These are your numbers, not Cleo’s, and they are what you compare against on day 30.
Before day 1: set the rules and the handoffs
Decide which services callers can book and the booking rules your scheduler supports. Set who takes sales, service and parts calls and each department’s hours, and name the person who returns callbacks, and how fast.
Who answers at each step, and what to check
Before day 1
Set the baseline
All hoursYour teamWrite down last month's calls from your own phone system. Set booking rules, the transfer list and who returns callbacks.
Days 1 to 14
After hours only
OpenYour teamClosedCleoYour team answers as today. After closing, Cleo answers. Check every overnight booking in your scheduler.
Days 15 to 30
Add overflow
OpenYour team→CleoClosedCleoYour team still gets every call first. Cleo picks up when nobody answers within the rings you set, up to six.
Day 30
Decide
KeepTuneWidenStopKeep after hours only, tune the rings, widen what Cleo books, or stop.
A pilot plan, not a result. Cleo call log tags shown as they appear in the dashboard.
Why pilot first
Bookings are easy to see. Handoffs are where it breaks.
Pied Piper's 2025 Service Telephone Effectiveness Study looked at dealerships using AI on service calls. WardsAuto reported the results on July 21, 2025.
Public study, not Cleo results91%
handled successfully without relying on an employee, at dealerships using AI on service calls
9%
of the time, the AI could not handle the caller's request
So test the calls the AI cannot finish, not just the ones it books. A good pilot measures where every handoff ended.
Source: WardsAuto, "Better call handling boosts service loyalty, study finds", July 21, 2025, reporting Pied Piper's 2025 Service Telephone Effectiveness Study. These are study figures, not Cleo results. Read the article
What to measure
Seven measures, and where Cleo shows each one.
| Measure | Where to find it in Cleo | How to read it |
|---|---|---|
| Calls Cleo answered | Dashboard: Calls answered | Inbound calls handled by Cleo. Not every call to your store. |
| Appointments booked | Dashboard: Appointments booked, split into booked on calls and by text | Open a few in your scheduler each week. Cleo tells a caller they are booked only after the scheduler accepts the booking. |
| Handoffs that reached a person | Call log tag: Transfer | The transfer went through. |
| Handoffs that did not go through | Call log tag: Transfer Attempted | Cleo tried to transfer, the transfer did not go through (for example, nobody picked up), and it took a message instead. This is the number your team controls. |
| Bookings that did not finish | Call log tag: Appt Incomplete | The booking could not be completed. Cleo does not retry a failed booking; it creates a callback task for your staff. |
| Follow-ups returned | Tasks in Cleo, and the email staff get when a call leaves a task type they subscribe to | Check every one was returned, and how fast. |
| What was said | Each call: summary, outcome tags, transcript and recording. The call list exports to CSV (caller name, phone number, tags, sentiment, duration, date and time), filtered by tag, sentiment, duration or date | Listen to a sample every week, including every Transfer Attempted call. |
An answered call is not a booking, and a booked appointment is not a completed visit or revenue.
Before real callers do
Six test calls to place yourself.
Use a test vehicle, call from a number you control, and cancel any test bookings in your scheduler afterwards.
Book a routine service at a time you know is open.
Check Cleo reads back the day and time, and that the appointment is in your scheduler.
Ask for a time that is full.
Cleo should offer the nearest open times, and offer a callback from your service team when nothing fits.
Call back to move that appointment.
On a reschedule, Cleo books the new time first, so a failed move leaves the original appointment in place (Xtime workflow; confirm the steps for your scheduler).
Ask what a brake job costs.
With Cleo's service-quote workflow, Cleo never quotes a price, estimate or range: it transfers to service, or takes a callback if service cannot take the call.
Ask for parts after the parts counter closes.
Cleo should take a message and save it as a task, and the staff who subscribe to that task type get an email.
Ask for a person while a department is open, and let it ring.
Cleo should take a message when nobody picks up. Find the call afterwards under Transfer Attempted.
The study figures above are public and are not Cleo results. An answered call is not a booking, and a booked appointment is not a completed visit or revenue.
Real dealership results
A full deployment at El Cerrito Honda.
633
Calls answered at El Cerrito Honda in 30 days
112
Appointments booked in the same 30 days
34
Of those 112 appointments booked while the dealership was closed
El Cerrito Honda runs Cleo the way this plan does: after hours first, then a three-ring overflow across sales, service and parts. Cleo dashboard, rolling 30 day view captured September 26, 2026. Bookings are a separate reported figure, not a breakdown of the 633 answered calls, and a booked appointment is not a completed visit or revenue. These are one dealership’s results, not a promise of yours.
If a call is not answered within three rings in any department, Cleo steps in. This ensures no customer is left waiting or forced to hang up.
Sheriff Demitri, General Manager, El Cerrito Honda
Day 30
Decide with the same scorecard.
01 · Keep after hours only
If your team already answers daytime calls well, stop there. Cleo keeps answering after closing, and your team keeps every daytime call.
02 · Keep overflow, tune the rings
Fewer rings means Cleo answers sooner; more rings gives your team longer. Your team still gets every call first.
03 · Widen what Cleo books
Add services or relax a booking rule once the scorecard shows bookings that hold up in your scheduler.
Or stop. If bookings did not hold up in your scheduler, or handoffs kept failing, the scorecard shows you why. Count outcomes the same way for every option you are weighing.
FAQ
Still have questions?
How long should an AI phone agent pilot run at a dealership?
Long enough to cover every day of your week and the calls your team cannot answer. This plan uses 30 days: two weeks with Cleo answering after hours only, then two weeks with overflow added, where your team gets each call first and Cleo answers when nobody picks up within the number of rings you set.
What should we measure during an AI phone agent pilot?
How do we know the appointments the AI books are real?
What happens when the AI cannot help a caller?
Contact
Plan your pilot with Cleo.
Book a 30-minute demo and bring last month’s call numbers. We will walk through where Cleo would answer first and how you would score it.
