Patent · Voice Assistants

Memory-Network NLU for Tmall Genie

Fusing rules and neural memory for domain routing — first inventor on patent CN112002313B.

Google Patents page for CN112002313B — Interaction method and device for voice interaction — Granted patent CN112002313B (first inventor) — semantic matching for voice interaction.

Problem

Every voice command on Tmall Genie had to be routed to the right skill domain (music, shopping, IoT, …). Early rule-and-template systems topped out around ~65% routing accuracy and failed opaquely on open-set or noisy ASR input. Pure neural classifiers were harder to constrain with product rules and harder to explain when wrong.

Approach

I designed a memory-network NLU architecture that keeps rule knowledge as addressable memory while learning soft representations of the utterance. The fusion raised production domain-classification accuracy to about ~85%, with open-set handling and acoustic “unclear expression” signals so failure modes stayed diagnosable.

Domain classification architecture fusing DC model semantic features with rule memory via multi-head attention — Domain classification: fusing a DC model with rule memory and bias through multi-head attention.

This layer, together with the ASR stack, formed one of the two foundational AI paths every Tmall Genie command passed through.

Domain classification accuracy rising from 65% to 85% with MemNet and unclear-expression model milestones — Full-sample DC accuracy: ~65% → ~85%; when ASR is correct, up to 98.89%.

Impact

Granted Chinese patent CN112002313B (first inventor; filed 2019, granted 2023)
Production routing for a platform that reached tens of millions of households
Later cited as prior art by organizations outside Alibaba (including insurance and university filings)

Memory-Network NLU for Tmall Genie

Problem

Approach

Impact

Links