Too big to personalise by hand, too small to personalise with AI

When reaching out to close a sale, common wisdom tells you to learn what you can about the person on the other side of the table – their pain points, workflows, budget, favourite football team, star sign, and whether they take pineapple on their pizza. For organisations with relatively few prospects, this works wonderfully and there is arguably no better investment than a team of skilful account managers. Perhaps unsurprisingly, the data support this common wisdom, with McKinsey reporting that 71% of customers expect personalised outreach, and 76% being frustrated by companies' failure to deliver it.

While this method proves effective for businesses with few leads, it is seldom possible when managing over 50,000 leads. The world's largest companies resolved this problem by deploying AI. Uber's machine-learning models schedule promotions around each user's order history and engagement. Netflix uses AI to recommend shows based on prior viewing and surface new content. This capability rests on two things the largest companies have (and increasingly defend): oceans of behavioural data; and the budget for teams of highly skilled data scientists who turn it into live decisions.

This leaves a large cohort in an 'anti-Goldilocks' zone. Too big to personalise by hand, too small to personalise with AI. The cost of being stranded in this zone and getting it 'just wrong' is now well quantified. According to BCG, leaders in personalisation grow revenue 10 percentage points faster than laggards. McKinsey concurs, but opts to cushion this figure in a 5-15% range. The academic literature is similarly telling. Adding just the recipient's name to an email subject line raised open rates by 20% and sales leads by 31%.

The anti-Goldilocks zone cuts across numerous sectors. Universities process tens of thousands of applicants and, at most, hundreds of thousands of leads. Recruitment firms and in-house recruitment teams juggle candidate databases in the hundreds of thousands. Fintech and consumer lenders hold application and repayment histories on similar numbers of customers, while insurance agencies sit on years of quote and renewal data. Direct-to-consumer brands hold purchase, subscription and retention histories for hundreds of thousands of customers. Estate agents, car dealerships and a long tail of mid-sized consumer companies do the same with enquiry data they do not fully utilise.

The way out of the value trap of generic outreach, delivered to leads that forgot they filled an enquiry form, is best understood as a ladder. Each rung answers a different question and demands a different, more sophisticated, technology.

While the Netflixes and Amazons of the world have the capability to climb to the very top and deliver causal AI to its customers, most firms in the anti-Goldilocks zone do Level 1 and Level 2. Some outsource Level 3 or utilise their CRM's native correlational AI, where the data is available. Almost none reach Level 4, not for lack of want, but for the lack of data, data scientists and dearth of vendors offering a hand to pull them up.

Correlational AI is valuable, and often seen as a significant step up from Level 2. It is. It also harbours an expensive flaw. It cannot tell between the customer who converts because you contacted them with a personalised email and one who would have converted anyway. In fact, the academics Rößler & Schoder found that contacting the bottom 30% of leads actually destroys value. Think of the gym membership you'd forgotten about, until a promotional email reminded you to cancel it. Or the university applicant who receives an email that dissuades them, say, advertising city-campus nightlife to a mature applicant with a family, and pushing them towards a rival's offer.

Causal AI makes exactly this distinction, sorting every lead into four archetypes.

The persuadables convert only because you reached them; they alone generate incremental return. The sure things convert regardless, so spend on them is waste dressed as success. The lost causes never convert. And the sleeping dogs are actively turned away by contact; reaching them destroys value. Methodologically, this is only possible by advanced machine learning that is measured against a randomised control group – a multidimensional A/B test of sorts. If your conversion rate is not measured with this, you cannot know which of your spend is working.

The commercial implication of getting this right is drastic, and is a level of personalisation and technical depth that is deeper than what BCG and McKinsey claimed earlier. In an oft-cited retail bank case, targeting customers based on causal uplift increased total sales by ~10%, reduced mail volumes by ~60% and more than doubled the campaign's profitability. Applying this to, say higher education where financial pressures are acute, is potentially the difference between an open university and a closed university.

Two emerging trends are making the top rung more accessible to the universities, recruitment agencies, lenders, insurers and other mid-sized organisations stranded in the anti-Goldilocks zone.

• Advancements in AI and machine learning techniques are unlocking this technology in increasingly data-thin environments; and
• The forward-deployed-engineer business model allows for the rapid deployment of software and models built bespoke for each individual client.

The middle market is heading towards a halfway house between full outsourcing and full in-house. Expert AI/ML engineers build causal AI into bespoke software for each client's data-thin environment, and the client pays an annual licence, much as they already do for Salesforce or HubSpot.

• Locate yourself on the ladder honestly. Is your organisation on Level 1, Level 2, Level 3 or Level 4? Are you sending the same marketing and outreach emails to everybody in your CRM? What are the reasons you are stuck on your level – is it data availability, talent availability, strategic focus, caution about AI?
• Chart a clear AI implementation roadmap. What are your next steps to getting to the next rung in the ladder? What support do you need to get there?
• Buy the capability rather than building the team. The defining feature of the anti-Goldilocks zone is that an in-house data-science function rarely pays back at these volumes. The data scientists with the expertise to build causal AI typically cost $200-$300k per year in the US or £70-95k in the UK.

The capabilities that gave the consumer giants their edge can now be democratised. These techniques are no longer gated by scale or budget. The universities, recruitment agencies, lenders, insurers, estate agencies, law firms, gyms (etc.) of the anti-Goldilocks zone can plausibly reach for the same capabilities that were once the preserve of the giants alone.