How to Evaluate AI Consultants Without Getting Burned: The 7-Question Framework

You know the feeling.

You're sitting across from a consultant (or on a Zoom call), and they're confidently telling you how AI will "transform your business" and "10X your efficiency." They've got the buzzwords down. They're showing you impressive case studies. They're making it sound easy.

And in the back of your mind, there's that nagging voice: "Is this real? Or am I about to waste $50K on someone who's going to ghost me after the first invoice?"

I've seen both sides of this. As an AI consultant for the past 11 years, I've been hired to fix projects that went wrong. I've audited systems built by consultants who clearly had no business building them. I've had conversations with business owners who spent six figures and got... nothing that actually worked in production.

But I've also seen incredible implementations. Consultants who delivered exactly what they promised. Systems that actually saved 20+ hours per week and are still running two years later. Partnerships that transformed businesses.

The difference? The business owner knew the right questions to ask.

This isn't about finding the "perfect" consultant. It's about systematically evaluating whether someone can actually deliver what they're promising, or if you're about to become their next cautionary tale.

Why Most Evaluation Processes Fail

Before we get to the framework, let's talk about why most business owners get this wrong.

The Common Mistake: Evaluating consultants based on confidence and presentation skills.

The consultant who sounds most certain, has the slickest deck, and quotes the biggest numbers often gets the contract. But confidence doesn't equal competence. I've met consultants who could sell ice to penguins but couldn't debug a simple automation to save their lives.

The Real Problem: You're trying to evaluate expertise in a domain where you don't have expertise.

If you're hiring a plumber, you can at least see if water still leaks. But with AI and automation? The work is invisible. The consultant can tell you anything, and you won't know if it's true until six months later when the system either works or doesn't.

This creates an information asymmetry that predatory consultants exploit ruthlessly.

The 7-Question Framework

Here's the framework I give to every business owner before they hire anyone (including me). These seven questions create a systematic evaluation process that separates real expertise from polished sales pitches.

Question 1: "Show me something you built that's still running in production after 12+ months."

Why This Matters: Anyone can build a demo. Most consultants can deliver a first version that looks impressive. But systems that are still running a year later? That's a completely different skill set.

What You're Really Testing:

• Can they build maintainable systems?
• Do their clients actually keep using what they build?
• Have they dealt with the reality of edge cases, breaking changes, and scale?

Red Flags:

• "All my work is under NDA." (Convenient excuse for having nothing to show.)
• They only show screenshots or demos, never real production systems.
• The examples are all from within the last 3 months.
• They can't articulate what broke and how they fixed it.

Green Flags:

• They walk you through a system that's been running for 18+ months.
• They explain what's broken over time and how they've maintained it.
• They can show you metrics from before and after implementation.
• The client is still actively using it (and they can introduce you).

How to Ask It: "I'd love to see an example of something you built 12-18 months ago that's still being used today. Can you walk me through what you built, what broke along the way, and how you fixed it?"

Question 2: "What would you NOT automate for a business like mine?"

Why This Matters: Any consultant can tell you what they can build. A good consultant knows when NOT to build.

What You're Really Testing:

• Do they have strategic judgment, or are they just hammers looking for nails?
• Will they tell you the truth, or just sell you more work?
• Do they understand the difference between what's technically possible and what's actually valuable?

Red Flags:

• "We can automate everything!" (No. You can't. And you shouldn't.)
• They can't name a single thing they'd advise against.
• Every problem has an AI solution in their world.
• They pivot back to selling without answering.

Green Flags:

• They immediately name 2-3 things that aren't worth automating.
• They explain why (usually: "Too complex for the ROI" or "You'd spend more maintaining it than doing it manually").
• They reference times they've talked clients OUT of projects.
• They prioritize based on impact, not technical impressiveness.

How to Ask It: "Based on what you know about my business, what's something I probably think I should automate, but actually shouldn't?"

Question 3: "Walk me through a project that failed. What happened and what did you learn?"

Why This Matters: Everyone fails. The question is whether they learned from it or just moved on to the next client.

What You're Really Testing:

• Self-awareness and honesty
• How they handle failure (blame the client or take responsibility?)
• Whether they've actually been in the trenches or just coasted on easy wins
• Growth mindset vs. ego protection

Red Flags:

• "I've never had a project fail." (Liar or inexperienced.)
• They blame the client entirely. ("They didn't give us access" or "They kept changing requirements.")
• Vague answer with no real details.
• Defensive body language or deflection.

Green Flags:

• They answer without hesitation (shows they've thought about this).
• They take ownership of their part. ("I should have...")
• They explain what they'd do differently now.
• The lesson learned shows up in their current process.

How to Ask It: "Tell me about a project that didn't go as planned. What happened, what was your role in it, and what would you do differently today?"

Question 4: "How will you prevent scope creep and keep this on budget?"

Why This Matters: The $20K project that becomes $80K is a tale as old as time. You need to know how they handle the inevitable moment when reality doesn't match the initial plan.

What You're Really Testing:

• Do they have a defined process for managing scope?
• How do they handle change requests?
• Are they transparent about what's included vs. what costs extra?
• Have they thought about this, or are they winging it?

Red Flags:

• Vague answer about "staying flexible."
• No clear definition of what's in scope vs. out of scope.
• "We'll figure it out as we go."
• Fixed price with no clear deliverables.
• Everything is "included" (until it's not).

Green Flags:

• Clear scope document with specific deliverables.
• Defined change request process.
• They explain how they handle discoveries mid-project.
• Transparent about assumptions and risks.
• They've built buffer time for unknowns.

How to Ask It: "What happens when we discover something mid-project that wasn't in the original scope? How do you handle that, and how do we make sure this doesn't balloon to 3X the budget?"

Question 5: "Who will actually be doing the work?"

Why This Matters: You're talking to the senior consultant with 15 years of experience. But is that who's building your automation, or is it their junior hire who just finished a bootcamp?

What You're Really Testing:

• Will you get what you're paying for?
• Is there a bait-and-switch happening?
• What's the actual skill level of the team?

Red Flags:

• "My team will handle it." (Who is your team?)
• Can't or won't introduce you to the people doing the work.
• Obvious skill gap between who you're talking to and who's executing.
• They get defensive when you ask.

Green Flags:

• Clear answer about who's doing what.
• You get to meet or talk to the actual implementers.
• Their experience level matches the complexity of your project.
• They're transparent about any outsourcing or delegation.

How to Ask It: "I appreciate talking with you, but who specifically will be doing the hands-on work? Can I meet them or see examples of systems they've personally built?"

Question 6: "What happens after you deliver? How do you ensure this doesn't break when APIs change or my business scales?"

Why This Matters: The ghost moment. You pay the final invoice, they disappear, and three months later something breaks and you have no idea how to fix it.

What You're Really Testing:

• Have they thought about long-term maintainability?
• Is there a support plan, or are you on your own?
• Do they build systems that your team can maintain, or black boxes only they understand?

Red Flags:

• "It won't break." (Everything breaks.)
• No mention of documentation or training.
• Ongoing support is suspiciously expensive (vendor lock-in).
• They avoid the question.

Green Flags:

• Clear documentation deliverable.
• Training for your team is included.
• Reasonable support options (not mandatory, but available).
• They explain their approach to maintainability.
• They can hand it off to your internal team or another consultant.

How to Ask It: "What happens in six months when an API changes or something breaks? How do you ensure my team can maintain this, or do I need to keep you on retainer forever?"

Question 7: "Can you connect me with a client who had a similar project? Not your best client, just someone similar."

Why This Matters: References they choose are curated. But a similar client? That's where you get honest feedback.

What You're Really Testing:

• Are they confident enough in their work to let you talk to real clients?
• Do they have relevant experience with businesses like yours?
• Can you hear the unfiltered truth?

Red Flags:

• Won't provide references. ("Everything's under NDA.")
• Only provides cherry-picked "best" clients.
• No clients similar to your size, industry, or use case.
• References feel coached or overly rehearsed.

Green Flags:

• Provides 2-3 references without hesitation.
• At least one is similar to your business.
• They give you space to ask whatever you want.
• The reference is honest about both wins and challenges.

What to Ask the Reference:

• "What was the biggest challenge during the project?"
• "Did it come in on time and on budget?"
• "Is the system still running today?"
• "What would you do differently if you could start over?"
• "Would you hire them again?"

How to Score Your Evaluation

Here's how I weight these questions:

Must-Pass (Automatic Disqualification if Failed):

• Question 1 (Production examples)
• Question 5 (Who's doing the work)
• Question 7 (References)

If they fail any of these three, walk away. No exceptions.

High Weight (Major Concerns if Failed):

• Question 3 (Failed project)
• Question 6 (Post-delivery support)

If they fail both of these, seriously reconsider.

Medium Weight (Concerns but Not Dealbreakers):

• Question 2 (What not to automate)
• Question 4 (Scope management)

If they fail one of these, dig deeper. If they fail both, probably walk away.

The Emotional Side: Overcoming Fear of Making the Wrong Choice

Here's what nobody tells you: Even with a perfect framework, you'll still feel uncertain.

That's not a bug. That's a feature of making big decisions.

The goal isn't to eliminate uncertainty. The goal is to make a confident decision despite uncertainty.

What I've learned after 11 years and hundreds of implementations is this: The difference between business owners who scale with confidence and those who stay stuck isn't that they never make mistakes. It's that they have a system for making decisions, and they trust that system.

This framework gives you that system.

You won't be 100% certain. But you'll be confident that you did your due diligence. You'll know you asked the hard questions. You'll have evidence, not just gut feeling.

And when you move forward, you'll do it with clarity and conviction instead of second-guessing yourself at 2 AM.

That's what confidence actually feels like. Not certainty. Conviction.

Red Flags Checklist: Walk Away If...

Beyond the seven questions, here are instant disqualifiers I've learned from seeing projects go wrong:

❌ They're overly focused on tools, not outcomes. "We'll build you a custom GPT!" Cool. But will it actually save me 20 hours per week?

❌ They guarantee specific results without knowing your business. "We'll 10X your efficiency." How do you know? You met me 20 minutes ago.

❌ They can't explain technical concepts in plain English. If they can't explain it simply, they either don't understand it or they're hiding behind jargon.

❌ They push for full payment upfront. Milestone-based payments align incentives. Full payment upfront removes theirs.

❌ They don't ask probing questions about your business. If they're not trying to understand your context, they're selling a cookie-cutter solution.

❌ They make the complex sound trivial. "Yeah, we can build that in a weekend." Building? Maybe. Building it RIGHT? Months.

❌ They're weirdly secretive about their process. Transparency in approach builds trust. Secrecy is a manipulation tactic.

What Good Looks Like: Green Flags to Look For

On the flip side, here's what a trustworthy consultant actually does:

✅ Asks more questions than they answer in the first conversation. They're trying to understand your business, not pitch their services.

✅ Tells you what you DON'T need. "You don't need AI for that. Just hire a VA." (Instantly earns trust.)

✅ Explains things in plain English without dumbing it down. Treats you like an intelligent person who doesn't happen to be a technical expert.

✅ Shares frameworks and teaches you how to think. You leave conversations smarter, not more confused.

✅ Has a defined process with clear deliverables. You know exactly what you're getting and when.

✅ References include both wins and challenges. Honesty about what went wrong shows self-awareness.

✅ Proposes a pilot or phased approach for large projects. "Let's prove this works before you commit $50K."

The Confidence You're Actually Buying

Here's what most business owners miss: When you hire an AI consultant, you're not just buying automations or systems.

You're buying confidence.

Confidence that you're making the right investment. Confidence that someone competent is handling the technical complexity. Confidence that you can scale without working 80-hour weeks. Confidence that you can take a vacation without everything falling apart.

That's the real product.

The automations are just the delivery mechanism.

And that's why getting the hiring decision right is so critical. A bad consultant doesn't just waste your money. They destroy your confidence. They make you second-guess yourself. They make you afraid to make bold moves.

A good consultant does the opposite. They give you clarity. They reduce your anxiety. They make you feel empowered to make the decisions that scale your business.

Use this framework. Ask the hard questions. Trust the process.

And when you find someone who passes these tests, move forward with conviction.

You're not eliminating risk. You're making a confident, informed decision despite risk.

That's what scaling with confidence actually means.

Your Next Step

If you're evaluating consultants right now, print this framework. Literally. Use it in your next conversation.

Ask all seven questions. See how they respond. Score them using the weights above.

And if you want a second opinion on a consultant you're considering, or you want to see how I answer these questions myself, book a free 15-minute consultation.

I'll give you my honest assessment—including whether I think you even need a consultant at all.

Get Your Confidence Roadmap: Book Your Free 15-Min Consultation

No pitch. No obligation. Just honest guidance to help you make the right decision for your business.

Because the cost of hiring the wrong consultant isn't just money. It's time, momentum, and confidence.

And you can't afford to waste any of those.

---

Ready to make a confident decision about AI automation? Let's talk through your specific situation and build a roadmap you can trust. Book your free consultation here.