How We Messed Up the AI SDR Hypothesis: Spent 1.5 Months and $35,000

Dashly Solution team is dedicated to creating solutions that aim to double the metrics of the inbound funnel. We explored various ways to develop our product, and during a discovery phase, we identified a problem: when marketers pass leads to sales, managers often start processing them too late.

Our funnel audits show that the SLA (Service Level Agreement) is often over 12 hours when it should be 15 minutes. Maintaining such a short response time is challenging but essential to convert a lead into a meeting.

To achieve this, companies hire SDRs (Sales Development Representatives). Their tasks are simple: ask a few qualifying questions, schedule a meeting, and ensure the lead attends. This process is quite standardized, requiring little expertise and deep involvement, making it automatable. Additionally, SDRs are cheaper than sales reps (account executives), require less training, and can be onboarded quickly.

This is one of the reasons we developed the concept and vision of a 7-step Customer Journey Map (CJM):

1. Lead capturing.
2. Qualifying quiz with a few key questions to prioritize leads.
3. Based on quiz data: ABCD segmentation to identify the hottest leads needing immediate attention.
4. Booking a meeting with priority leads immediately after the quiz without involving a manager.
5. Establishing contact through a reliable channel.
6. Nurturing leads to the meeting: answering their questions in the chosen channel, sharing useful materials, and conveying the meeting’s value — all through personalized messages. 
7. Reviving the lead database for those who didn’t attend the meeting. 

Imagine the headache we can relieve for the team! Our solution would handle a huge chunk of routine work, making the lead’s CJM seamless and fast.

We had defined our goal; now, we just needed to figure out how to achieve it. So, we began a series of experiments to find the best approach.

One of Our Experiments: AI SDR

After lead submission and ABCD segmentation, we decided to implement an AI bot. It was supposed to answer the lead’s questions (whether the product suits them, what to expect from the meeting, etc.), ask clarifying questions, and pass the context to the sales team.

Our hypothesis was that the bot would keep the lead engaged on the site, preventing them from moving to competitors, and the dialogue content would help sales personalize meetings. Sounds solid, right?

For our bot to talk to leads effectively:

• We obtained the client’s sales kit, detailing how managers communicate and including FAQs.
• We transcribed manager recordings ourselves and fed them to ChatGPT.
• After launching the bot, we monitored its interactions and fine-tuned it to respond correctly.

The work was massive. The experiment took us several sprints.

After implementation, we saw good results in high-level metrics: 50% of people engaged with the bot after qualification, and a third had fairly deep dialogues.

However, this didn’t impact meeting scheduling at all…

We assumed the AI bot would keep the lead engaged until a manager contacted them. But in reality, the bot interrupted the lead’s natural flow, causing them to drop off.

We realized we needed to focus on reducing the time it takes to schedule a meeting rather than holding the lead’s attention. This was a resource-intensive but crucial insight.

The second significant expectation from the experiment was that the bot’s dialogue content should aid sales in their follow-up with leads. Our vision was:

• The AI bot would further qualify the person, revealing their interests.
• Sales would receive a summary of the dialogue in the CRM, making the meeting more personalized.

Once again, things didn’t go as planned…

Feedback showed that managers weren’t using this information at all and approached the meetings with the same questions.

The key takeaway here is: it’s a headache for managers to learn and adapt. Developing this scenario should have simplified their work. Instead, we provided information they didn’t use.

We decided we weren’t ready to fight this battle or train managers to work differently when they weren’t interested.

The bot hypothesis cost us six weeks of work and around $35,000 but didn’t help increase meeting conversion. It was painful, but that’s how growth works.

We shelved our AI experiment until a stronger hypothesis came along. For now, we’ve decided to focus on scheduling meetings to minimize manager involvement in this process.