Too many ecommerce marketing agencies today exclusively subscribe to the teachings of modern day philosopher and Vogue queen extraordinaire Madonna:

Much like Rita Hayworth, they know how to “give good face.”

But they don’t know how to do much else.

They can throw around words like growth, efficiency, scale, attribution, testing, optimization, and performance with the confidence of people who have absolutely heard these words at least three times before in their lives… usually while a CFO or VP of Sales is interrogating them about why all that fancy “brand awareness” doesn’t seem to be increasing revenue. The case studies are polished. The screenshots are cropped attractively. Everyone sounds highly capable.

Of course, a step up from there, you may find agencies who also have a few polished case studies up their sleeves, so you feel confident enough to say yes to a retainer.

But then… six months later… you’re inches from banging your head against a wall, because you’re in yet another recurring meeting with fancy decks telling you that your campaigns are “trending in the right direction,” while your bottom line is screaming at you that it’s malnourished.

And any time you push for answers, the picture somehow only gets more vague and unclear the more your agency talks.

You know everyone is busy (and they should be, for what you’re paying them), but keep wondering quietly: “What the hell is actually going on?”

And you know what sucks?

Too many of you reading this right now are nodding along, because this is exactly what you’re dealing with. Why? Because this situation is all too common. Way more common than it should be.

The issue is rarely whether an agency can say the right things. The issue is whether they understand ecommerce well enough to make better decisions inside your business, and whether they can prove that before the contract starts. To figure that out, you need to ask an agency the right questions that pushes them out of their shiny pitches and into a conversational space that resembles reality.

Below are 10 questions worth asking before you hire an ecommerce marketing agency, plus what those questions are really testing and what to pay attention to in the answers.

1. How do you define success for a brand like ours?

This is a good opening question because it forces the agency to show what lens they use before they start talking about channels.

A strong answer should sound grounded in business outcomes: revenue, contribution, customer acquisition efficiency, repeat purchase behavior, margin realities, growth stage, and channel role should all be somewhere in the neighborhood.

Of course, you shouldn’t expect an agency to know your business better than you do in that first discovery call. They should, however, know enough to avoid giving you rambling “it depends” answers built entirely around clicks, impressions, and “scaling spend.”

Put another way, if you can pick up whatever answer they gave you and drop it into a conversation with a brand that is in a completely different industry with completely different buyers and variables, you should probably move along.

What you are really testing here is whether they think like operators or platform managers. That means if they ask you clarifying questions, that’s a good thing. Because if the answer sounds like a standard paid media KPI stack with no real curiosity about your economics, that should make you uneasy.

2. What do you need to understand about our business before you can make smart recommendations?

OK, I mentioned this in the last section, but agencies that act like they already have all of the information they need to speak fluently to your business and their recommendations are a massive red flag.

A good agency should have questions about your needs, budget, and internal capabilities. (That means you should come prepared to speak to those things.)

A strong agency will want to know things like:

What you are really looking for is curiosity with judgment behind it.

If the agency is ready to prescribe the entire solution before it has asked anything meaningful about your business, that is usually a bad sign. Fast certainty is often just recycled thinking wearing nice shoes.

3. What kinds of ecommerce brands do you work best with, and where do you usually struggle?

I love this question for two reasons:

A strong answer should include specifics: product types, business models, growth stages, acquisition complexity, average budgets, or operational realities they know well.

You’re listening for fluency and precision.

If the agency claims it works equally well with everyone from fast-growth DTC brands to local service providers to B2B SaaS to enterprise retail and somehow has no real weak spots, that is less impressive than they probably think it sounds.

It’s kind of like a job interview, right?

No one is getting points here for being “too detailed-oriented” or “caring too much.”

A strong ecommerce marketing agency will not only point to where they excel, but also who is not the right fit for them.

4. Walk me through how you would evaluate our paid performance today before changing anything.

Now you want to dig into how an agency thinks.

A strong agency should be able to talk through a sensible diagnostic process. They should mention things like channel mix, campaign structure, audience quality, creative fatigue, branded versus non-branded performance, new customer versus returning customer dynamics, landing page friction, attribution limits, and where the current numbers may be flattering reality.

What you are testing is not whether they can produce instant brilliance from limited information. You are testing whether their first instinct is strategic diagnosis or immediate tinkering.

If the answer is mostly “we would launch tests and optimize spend” without much explanation of what they would study first and why, that suggests they may be more comfortable in motion than in analysis.

5. How do you report performance in a way that actually helps clients make decisions?

If there’s one thing every agency knows how to do, it’s build a report.

No one is questioning that.

What you’re really looking for here is how they translate performance into decisions:

Reporting quality shapes how useful your agency relationship becomes over time, so pay attention.

You should be wary of answers that focus only on dashboards, frequency, or tool names. The issue is not whether they can display numbers. The issue is whether they can interpret them in a way that makes your team sharper.

6. How do you think about branded demand, remarketing, and net-new customer acquisition?

This is one of the best questions in the whole list because it gets straight at whether the agency knows how performance can paint a pretty, but deeply inaccurate picture of what’s actually happening inside of your campaigns.

A strong agency should understand that branded search and remarketing can be valuable, efficient, and commercially useful while still telling a very different story than prospecting or net-new customer acquisition. They should be able to talk about channel role, incrementality limits, audience temperature, and where the “pretty” results might be doing less hard work than they appear to be doing.

What you are listening for is honesty and nuance.

If the agency talks about all performance like it lives in one giant bucket of interchangeable success, they are going to be harder to trust once the numbers start getting complicated.

7. How do creative, landing pages, and channel performance connect in your process?

A lot of agencies still talk like performance lives entirely inside the media account.

That is too small a view for ecommerce.

A good agency should be able to explain how creative testing, landing page feedback, offer structure, and channel strategy work together. They do not need to own every one of those functions internally to speak intelligently about the connections. They do need to understand that weak pages, stale creative, and muddy offers can do more damage to performance than bid adjustments ever will.

This question is testing whether they see the system or only the ad account.

If the answer sounds like channel optimization in a vacuum, that usually means they are going to push spend around while the bigger conversion issues keep sitting there unbothered.

8. What will my team actually need to do to make this relationship work?

This is one of the most underrated questions to ask.

Agencies are often happy to talk about what they do for you. You also need clarity on what working with them will require from your side. Shopify’s current hiring guidance explicitly tells brands to consider how much time they have to invest in working with an agency and what their internal team is already good at.

A strong answer here should be practical. You want to know:

You are trying to avoid a very common agency disappointment, which is discovering that you have hired a team to reduce friction and somehow wound up with a second team you now have to manage into usefulness.

9. Can you show me how you think, not just what you’ve done?

Case studies are fine. Every agency has them. They also tend to be highly curated little fairy tales where the agency did the smart thing, the market behaved beautifully, and the graph went up in a morally satisfying manner.

This question forces them to go deeper.

Ask them to walk you through a decision:

What you are testing here is judgment.

You do not just want proof that they have worked on accounts. You want proof that they can reason their way through growth problems.

10. How are you leveraging new technologies like AI to optimize strategy and performance?

A smart agency should be able to answer this without sounding like it duct-taped ChatGPT to its workflow and called it innovation.

We are an AI-native agency ourselves, we know firsthand how AI can absolutely help agencies move faster, spot patterns sooner, scale testing, accelerate analysis, and reduce time spent on repetitive work. What you want to hear, though, is how an agency is using AI to support stronger thinking rather than replace it.

If the answer sounds like they are firehosing AI across creative, reporting, strategy, and execution without much human judgment behind it, that should make you nervous. Strong agencies know the real value isn’t using AI everywhere. It is knowing where AI helps them scale what humans still need to direct: prioritization, interpretation, creative judgment, business context, and strategy.

Red flags to watch for from an agency

A few things should make you nervous quickly.

One is vague confidence. If the agency speaks in smooth performance nouns without giving you much actual logic, be careful.

Another is no meaningful ecommerce specificity. If they do not seem curious about margins, repeat purchase behavior, merchandising pressure, or customer acquisition dynamics, they may know the platforms better than they know your business.

A third is reporting talk without decision talk. If they can tell you how often they report and what tools they use but not how they turn that into sharper decisions, that matters.

You should also pay attention to overpromising. Fast certainty, universal confidence, and suspiciously clean answers usually age poorly.

Remember your real goal in these conversations

Your goal is not to find the agency that says the most impressive things.It’s to find the one that asks smart questions, understands ecommerce realities, shows real judgment, and can make your business stronger once the pitch deck goes away.

Before you hire anyone, pressure-test them in the places that matter:

A strong ecommerce marketing agency should leave you feeling clearer, not dazzled.

Clearer about your goals, clearer about the tradeoffs, clearer about what good performance actually looks like, and clearer about whether this team can help you get there. That is a much better hiring standard than “they seemed smart on the call.”