How Named Entity Extractor (NEX) Helps Chefling Create Shoppable Orders

Introducing NEX

“Hey, Alexa! Can you add Nachos to my shopping list?”

“Nachos added to your shopping list!”

This is how a usual conversation would sound between the user and Alexa. Add more details to the command and you come across some discrepancies.

Hey, Alexa! Can you add 15 oz of Doritos Nachos?”

“Doritos Nachos added to your shopping list!”

And that’s where the differences become apparent. The basic skill on Alexa struggles to comprehend between different entities and their attributes, especially when given a command to add items with quantity, product and brand name. We recognized this as a limitation early on and realized that there was much potential in bringing a complete 100% hands-free operation.

We’ve been working hard to make this easier for our users with NEX. If you’ve heard about the machine learning algorithmic model — Named Entity Recognition (NER), you might instantly feel at home with NEX (Named Entity Extractor). NEX powers our exclusive skill on Chefling — My Chef. NEX is designed to assist shopping via voice assistants like Amazon’s Alexa and Google Home by facilitating recognition and precise information extraction based on specific item attributes.

How NEX Works

The powerhouse of the NEX algorithm resides in deep learning and NLP (Natural language processing) that helps in learning from huge data sets to find relevant items (eg. grocery, food items) based on user’s voice command. NEX uses a Bi-LSTM Sequence Tagging to extract names by capturing the context in both directions and actively converting input.

How NEX processes voice command to create a shoppable order

For instance, “Alexa, add 15 oz Doritos Nacho chips, 12 oz green apples and 6 packets of Tetley green tea to my shopping list”. For the above voice command, NEX will extract the items as per:

{‘Brand’: ‘Doritos’, ‘Item’: ‘Nachos chips’, ‘Quantity’: ’15 oz’},

{‘Brand’: ‘’, ‘Item’: ‘Green apples’, ‘Quantity’: ’12 oz’},

{‘Brand’: ‘Tetley’, ‘Item’: ‘Green tea’, ‘Quantity’: ‘6 packets’}

Thus, the output of NEX is effectively populated and ready to be consumed in the shopping cart.

Objective of NEX

The primary objective of NEX is to build an order that can be directly used for shopping. An order generally consists of the Brand Name + Product Name + Quantity. NEX recognizes the entities separately as item name, item property/specification, and item quantity. In case, Alexa or Google home is unable to match the product with command, NEX uses the food and grocery dataset based on word2vec to recommend the closest match for the given user input.

Robustness of NEX

The robustness of NEX is what makes it different from other algorithms. NEX is capable of extracting multiple items from a single command, making it extremely robust for actions that require chunking and recognition of products and items in long commands. NEX is also currently trained on ~12K brands and ~450K products.

Demo of how NEX works

Further scope for NEX

Because of the versatility of NEX, the application of NEX can be applied in different scenarios like customer issue resolution, sentiment analysis of person/company on social media and e-commerce search.

NEX can be built as a coherent algorithm that is adaptive and responsive. For instance, to find all ingredients from recipes or developing a universal NEX enabled search. The future of NEX can be extended further to support different queries and requirements.

Source: Deep Learning on Medium