Source: Deep Learning on Medium
Why is Industrial Context Important?
Technology has been the fundamental driver of economics in human history.
Artificial Intelligent, one representative of the general-purpose technology, has catalyzed waves of modern-day innovation and opportunities, fueling the fourth industrial revolution. It started re-shaping the business landscape, everyday life, and relationships between different entities.
To me, Artificial Intelligent was one of society’s most exciting and rewarding journeys into the unknown.
The reality was, in the industry, there were more people who talk about AI, compared to those doing AI. The applicability of AI solution was still quite narrow, and most of the AI initiatives failed due to a lack of core value that can justify the investment and engineering effort.
The missing piece in the puzzle was not just about the technological content, but the appropriate INDUSTRIAL CONTEXT. But before we delve into the actual example, let first take a look at Natural Language Processing.
Natural Language Processing
Natural Language Processing was a tract of Artificial Intelligence and Linguistics, concerned itself with the design and implementation of computational machinery that communicated with a human using natural language.
At its most ambitious, scientists and researchers aimed to design a general framework and system for the machine that has the capability of using language as fluently and flexible as humans do.
Although Natural Language Processing itself did not equal to Artificial General Intelligent, it was still one of the preliminary if you considering the embodiment learning was essential. In a simpler term, a truly intelligent body needed to learn from the outside world, which included reading, seeing, communicating, experiencing, etc.
According to Sebastian Ruder, a research scientist at DeepMind, the 15 years’ worth of NLP work could be condensed into eight milestones, which also triggered the industry reactions and merging applications.
Voice Assistant and ChatBot
Virtual Assistant(or Chatbot) was an industry implementation of Natural Language Processing technology. It can be seen as an automated system trained on general-purpose large corpus or service transcripts that can use AI to recognize and respond to customer requests. When we talked about Chabot, the first thing that came into your mind was Siri or Google Assistant (you probably have even interacted with them today). Indeed, tech giants were in the race for releasing consumer voice products as shown on the graph below,
Voice Assistant can be designed against horizontal functionality, for instance, conversation agent, Q&A bot, survey bots, and virtual agent. Or it can be aligned to the vertical industry, including Banking & Finance Services, Consumer, Public Service, Government, Health Care, etc. A simple voice assistant should be able to interact with people, provide relevant information, and even complete basic functions like filling a form or booking and scheduling a call-back.
Generally speaking, a successful large scale implementation of voice assistant was tied up to three premises,
- An immediate problem to solve: A company was facing a challenge to scale up even after spending huge money on customer service.
- Content to train an ML Engine: Thousand of transcripts or dialogs generated by the user that could enable the training of a deep learning model.
- Human AI collaboration Model: virtual assistant should be designed to enhance the human activity, not replacing them. That will allow virtual and human agents working together.
But how did a successful NLP solution looked like in the real world, or even more specific, how should I design a virtual assistant system to solve an existing problem. In the following content, I articulated the design thinking behind a general-purpose bot solution — a smart knowledge assistant.
How to Build A Knowledge Assistant
Human generates more data over the past two years than in the previous 5,000 years of humanity. And around 80 percent of those data will be unstructured in five years. Companies across the globe were facing the same challenge, how they could better manage, search, and analyze those unstructured data.
Let’s look at a scenario. An audit team was working on a project to assess the financial operations and financial records of a multi-national organization. They needed to check compliance against regulations from different counties, which would involve lots of time spent on manually processing document, then search and compare the terms and conditions.
Even if there were a content management system for those regulation policies, the standard search engine would not be sufficient enough to help them get what they want because,
- A typical search engine could only accept key works and could not understand natural language, or abstract keywords
- The search result rarely brought up the target document
- A search engine usually returned a long list of result and required extensive manual reading to locate the right answer
- The search result was highly correlated to the terminology users used
- No customization
The biggest problem here was, the existing system did not have the cognitive understanding capability. To bridge the gap, we need to incorporate the latest horsepower NLP to augment the search solution.
A virtual knowledge assistant came handy in solving the above challenge. The goal for a properly scoped designed knowledge assistant could offer,
- High speed to retrieving information functionality, which allowed the user to a query a large corpus
- High accuracy that only provided the top results (less than 3 based on confidence score) to the end-user
- Integrated to internal files system to build a self-managed knowledge base
- The ability to offer a granular answer to the end-user
For example, I asked the virtual knowledge assistant: “How can I become a Registered Company Auditors (RCAs) ?” then the assistant scanned through ten of thousand policies and told you “To become an RCA, you need to show the Australian Securities & Investments Commission (ASIC) that you meet the requirements of section 1280 of the Corporations Act 2001 “
This was not a dream with the newest breakthrough in deep learning and Natural Language processing.
Solution Architecture and Design
For the front end user experience design, the team brought the feedback and human factors into the design thinking. Below was the solution UI snapshot. See how we evolved the solution experience from a standard SharePoint search UI to a fully integrated interactive single-page application. (The mockup data was used in the screenshot below)
Behind the interactive front end was the Artificial Intelligent Engine that serves as the brain. To achieved this goal, the data science team has done several R&D, accessing the applicability and suitability of a large number of different NLP techniques in the industry. In the end, few stand out to be considered as the critical components for Virtual Knowledge Assistant.
The engineering team needs to decouple the main functionalities to ensure future scalability of the solution. Hence each component would inherently serve as a microservice container or cloud-native serverless function in the next iteration.
The secret sauce of the solution would be the fine-tuned BERT model.
BERT stands for Bidirectional Encoder Representations from Transformer, it was the newest language representation model developed by Google AI Language, and it took the NLP community by storm and throned most of the NLP competition leaderboard. It was also becoming an industrial standard for transfer learning method in NLP.
The data science team was using the original BERT and re-train on the following steps,
- First, fine-turn pre-trained model on the client provided corpus
- Second, fine-turn the last layer on the tailed downstream tasks.
The training process was on the Google Cloud Platform using GPU NVIDIA® Tesla® V100, 8 GPUs128 GB HBM2. The whole experiment took 4–5 days, and we managed to improve the result from 67% (via the BiDAF model from AllenNLP) to over 89% (fine-tune BERT).