LLM Sales-Email Intent Scoring for Inbound Lead Prioritization

A practical LLM-based intent-scoring design can do exactly one thing: make a single call to a language model, read a few floating-point scores, and fall back to a keyword heuristic if the model fails. No multi-agent orchestration. No fine-tuned BERT. No LightGBM ensemble. And according to the 2026 literature, an LLM semantic scorer outperforms keyword-based intent detection (Sanjei et al., 2026). The useful insight is that an effective design for sales-email intent scoring can also be one of the simplest — a bounded, schema-constrained LLM step embedded inside an existing dataflow graph, designed to fail open rather than cascade errors downstream. This article unpacks why that design is attractive, what the research actually says, and how to build it without over-engineering.

---

Why Email Text Is an Under-Used Signal

Traditional lead scoring relies on explicit signals: job title, company size, industry vertical, past purchase history. These work for routing inbound forms, but they ignore the email body itself. "Manual lead qualification is time-consuming, inconsistent, and prone to human error," note Sanjei et al. (2026) in their Lead Sense AI paper. An email that says "I need pricing for your enterprise plan by Friday, and I have a board meeting to approve the budget" contains far stronger intent signals than a form field labeled "company size: 500+ employees." Yet most scoring pipelines discard the natural-language payload.

LLMs change this by extracting semantic intent from unstructured text directly. Sanjei et al. propose a pipeline that takes raw email text, extracts three semantic features — purchase intent, urgency, and sentiment — and combines them into a single lead score on a 0–1 scale. Their experimental evaluation reports that LLM semantic understanding dramatically outperforms keyword-based detection. This is the central empirical claim we ground the implementation on.

---

The Deployment-Realistic Paradigm

Before reaching for a multi-agent framework or a fine-tuned transformer, consider the landscape. Vayadande et al. (2026) published a survey of four paradigms for buying-intent detection:

Paradigm	Survey Verdict
Deep-learning (transformers)	Most powerful given large labelled data, but little interpretability and weak under domain shift. Requires training infrastructure.
Knowledge-graph reasoning	Adds interpretability via structured relations, but high construction/maintenance cost. No KG asset exists for most teams.
Reinforcement learning	Adaptive in sequential settings, but sensitive to reward design and sample efficiency.
Agent-based (LLM)	Autonomy and multi-step coordination, but introduces reliability, latency, and operational overhead.

The survey's caveat on agent-based systems is the important one: full multi-agent autonomy adds latency and overhead. The deployment-realistic answer is to take the minimal agentic form — a single, schema-validated, fail-open LLM step. This sidesteps the survey's reliability caveats while still using the LLM's semantic strength. In LangGraph terms, that maps to one ainvoke_json_validated call per inbound email, with a keyword heuristic ready as a fallback. No tool loops, no sub-agents, no parallel calls.

---

How the Lead Sense AI Pipeline Maps to a LangGraph Node

Sanjei et al. (2026) present a pipeline that combines an LLM semantic scorer with a LightGBM classifier over semantic embeddings. The LLM stage extracts the three intent dimensions from raw email text. The LightGBM then fuses those with embedding features to produce the final lead score. Only the LLM stage is a LangGraph artifact. The LightGBM and embedding ensemble are offline ML: they require a training loop, labelled conversion data, and embedding infrastructure. This design explicitly defers that to a future roadmap and does not attempt to reproduce the paper's hybrid system or its reported results.

What this design implements is the LLM intent-scoring node directly. The schema is a Pydantic model:

class EmailIntentScore(BaseModel):
    purchase_intent: float = Field(ge=0.0, le=1.0)  # Lead Sense AI dimension
    urgency: float         = Field(ge=0.0, le=1.0)
    sentiment: float       = Field(ge=0.0, le=1.0)  # 0=negative .. 1=positive
    lead_score: float      = Field(ge=0.0, le=1.0)  # combined score
    confidence: float      = Field(ge=0.0, le=1.0)
    reason: str            = Field(default="", max_length=500)
    evidence: list[str]    = Field(default_factory=list)  # verbatim cues, each <=200 chars

Each score must be in [0.0, 1.0], enforced by Pydantic validation. The LLM returns this JSON; ainvoke_json_validated retries on ValidationError and raises only after all retries fail — at which point the node falls back to the keyword scorer. The node maps these fields to namespaced state keys to avoid clobbering existing routing telemetry: intent_confidence, intent_reason, intent_evidence, plus intent_score_source set to "llm" or "keyword_fallback". The keyword fallback mirrors the existing classify node's exception handling: it catches LlmDisabledError, StructuredOutputError, LlmTimeoutError, or any other LLM exception and gracefully degrades without crashing the graph.

This is a research-grounded design, not a deployed system with measured outcomes.

---

Privacy and Security: The Email Body Is Adversarial

Inbound email is an attacker-controlled surface. Uddin et al. (2026) studied an LLM/transformer (fine-tuned RoBERTa) classifier over adversarial email text and emphasized explainability — exactly why we want reason and evidence provenance fields. An email can contain injection attacks, phishing payloads, or crafted content designed to manipulate the scoring model. The OWASP A03 (Injection) risk is real: an attacker could embed instructions to leak scores or bias the output.

The mitigation is wrap_untrusted, a function from backend/llm/prompt_safety.py that neutralizes control characters, zero-width Unicode, and bidi overrides; defangs fence-spoof runs (<<</>>>); scans for injection markers; and wraps the body in a data-not-instructions preamble. It is the same mechanism used by the existing classify and extract_scheduling_handoff nodes. The body is labeled "INBOUND EMAIL BODY" and clamped to a bounded character limit. Per A09 (Logging), only the scores and provenance source are logged to OTel/LangSmith — never the original body or the evidence text.

---

What the LLM Step Buys

What happens if we remove the LLM stage and rely only on the keyword fallback? Sanjei et al. (2026) provide an implicit comparison: their evaluation compares the full LLM-powered pipeline against a keyword-based baseline, and the paper's primary claim is that LLM semantic understanding "dramatically outperforms" keywords. Without the LLM, the nuanced detection of purchase intent, urgency, and sentiment embedded in natural language is lost. A sentence like "We're evaluating vendors and would like to schedule a technical review" might trigger zero keywords in a buying-intent catalog, while an LLM can infer purchase stage from context.

Vayadande et al. (2026) offer a complementary view by comparing all four paradigms. Dropping to deep-learning (transformer) models alone gains raw power but loses interpretability and incurs training-data requirements. The four-paradigm taxonomy makes clear that the LLM/agentic path is the one with immediate deployability without labelled data. This design thus makes a deliberate trade-off: it does not use the full hybrid (LLM + LightGBM), and so does not claim the paper's hybrid accuracy, but it is deployable today.

---

The Supporting Research Landscape

Sanjei et al. (2026) is the primary grounding. A small set of additional verified 2026 papers reinforces specific technical decisions:

• Vayadande et al. (2026) — the four-paradigm buying-intent survey that legitimizes a single bounded LLM step as the deployment-realistic choice.

• Uddin et al. (2026) — an explainable transformer/LLM (fine-tuned RoBERTa) approach to phishing-email detection over adversarial email text. Its explainability emphasis supports the reason + evidence provenance fields, and it is a reminder that inbound email is an attacker-controlled surface.

• Putri et al. (2026) — Click2Buy Time, a Transformer-LSTM hybrid that models purchase intent with an explicit temporal signal. This maps to a future calibration: a single-email score ignores cadence and recency, and a later version could fold in time-since-last-touch as an additional input. That extension is offline ML, not this node.

• Biabani Mahalli et al. (2026) (DOI 10.1007/s11227-026-08432-6) — a knowledge-distilled transformer for intent detection and slot filling. Knowledge distillation as a cost-reduction lever parallels a multi-model-routing approach (cheap tier first, escalate on low confidence) for keeping per-email cost down.

---

Practical Takeaways for Implementation

1. Start with a single LLM call, not a multi-agent system. The Vayadande survey warns that full autonomy adds latency and operational overhead. A bounded step with schema validation and a fail-open fallback is the pragmatic choice.

2. Enforce numeric boundaries. All scores must stay in [0.0, 1.0]. Without a validator, an LLM may return an out-of-range value, breaking downstream routing. Pydantic Field(ge=0.0, le=1.0) rejects this and triggers a retry.

3. Fail open, not closed. If the LLM is unavailable or returns malformed output, do not crash the graph. Catch the exception and emit a keyword-based fallback score with intent_score_source="keyword_fallback". This mirrors the graceful-degradation pattern already used in the classify node.

4. Namespace your state keys. Do not reuse generic keys like confidence and reasoning if they already carry the label-classifier's values. Introduce namespaced keys (intent_confidence, intent_reason) to avoid clobbering routing telemetry.

5. Log only the scores, not the email body. OTel/LangSmith logs should contain the scores and source, but never the raw text or evidence snippets.

---

A Decision Framework for When to Trust the LLM Score

Condition	Action	Cost per email
LLM call succeeds, high confidence	Use LLM lead_score as-is	1 API call
LLM call succeeds, low confidence	Accept LLM score but flag for human review	1 API call + human latency
LLM fails (exception of any kind)	Fall back to keyword score; set source="keyword_fallback"	0 API calls
Keyword fallback also degrades	Emit a conservative default and continue; do not crash	0 API calls

This framework keeps the graph resilient. A future calibration against real reply/conversion outcomes would let a team tune the confidence threshold. For now, the keyword fallback is a safety net, never the primary path.

---

Limitations and the Road Forward

The most important limitation is that this design implements only the LLM semantic stage of the Lead Sense AI pipeline. The LightGBM/embedding ensemble is deferred because it requires labelled outcome data at volume and embedding infrastructure that has been decommissioned from the codebase. Sanjei et al. (2026) describe a hybrid that uses both stages, so this design intentionally does not match the paper's hybrid system — it trades that for deployability today. A future roadmap task could calibrate lead_score against real conversion data using offline ML.

Second, the cost model is simple: one DeepSeek ainvoke_json_validated call per inbound email, at standard-tier pricing. Future work could follow a knowledge-distillation path (Biabani Mahalli et al., 2026): route standard-tier emails through a smaller, distilled model and escalate low-confidence cases to the full LLM, to reduce inference cost.

Finally, human oversight is not optional. The system outputs a score, but a human should review leads with borderline scores to avoid false negatives or false positives. The node is a prioritization tool, not a replacement for sales judgment.

---

What to Watch For

Over-optimism about full automation. It is tempting to pitch LLM scoring as a hands-off replacement for human qualification. The design here acknowledges that human judgment remains necessary for borderline cases by emitting confidence and expecting downstream logic to route low-confidence scores for human review.

Data minimisation. Treat the inbound body as untrusted and log only scores and provenance — an approach aligned with data-minimisation principles.

Equating intent with purchase readiness. An email expressing strong interest does not guarantee buying power or timing. The score is one prioritization signal among several, not a verdict.