UK-based Volume releases tool for marketers to train NLP engines
QBox helps marketers figure out any category confusion in the models used by leading Natural Language Processing engines.
Natural Language Processing engines are becoming common tools in marketing applications like chatbots, but they require training.
UK-based marketing agency Volume, which specializes in conversational solutions, is now out with what it describes as the “first Training and Testing-as-a-Service platform for natural language data models.”
The platform — called QBox — helps marketers test whether the training data used to fine-tune a NLP engine is working as it should.
QBox, CEO Chris Sykes said in a statement, “allows natural-language data model developers to look into the black box they are typically working with, to easily understand any impact of a change on their training data.”
For example, CTO Benoit Alvarez told me, a bank may be setting up a chatbot on its website to direct users to information, online forms or live help. The chatbot needs to categorize queries depending on whether the user is interested in mortgages, personal loans, car loans, checking accounts or other bank products.
The NLP engines from the providers Volume works with — IBM Watson Assistant, Microsoft LUIS, Google’s Dialogflow or Facebook’s wit.ai — will categorize an inquiry into the available classes or categories, he said, after the engine has been trained by sample inquiries.
But it’s difficult to know when the categorization of the training data is mistaken, he said, since the NLP engines are essentially black boxes.
As a self-service platform, QBox lets the marketer see which inquiries about personal loans, for instance, are mistakenly categorized as mortgages. The platform knows the correct classes into which the training data should be categorized. It shows where the confusion is, and possible reasons for the mistakes, such as whether concepts like loans versus mortgages are too similar for clean categorization without a tweaking of the categories.
The results are presented visually so the training data can be modified, which Volume says generally takes about 10 minutes and doesn’t require a data scientist’s involvement. QBox was originally developed as an internal tool at Volume, Alvarez said.
The result, he said, is generally a 30 percent improvement in how accurately the queries get categorized.