Source: Deep Learning on Medium
Teaching bots to understand and speak African languages
A couple of days ago, i was going on one of my regular 5km runs and i had this really random urge to speak to my phone in my native language to perform some task but i was like it’s probably not going to understand me and then it hit me, THERE IS NO CHAT BOT OR VOICE ASSISTANT THAT SPEAKS OUR NATIVE LANGUAGES and yes i was screaming at all my fellow techies and researchers that hadn’t thought of or done it yet.
So i started doing some research about it and i couldn’t find any bots or assistants online so I decided to build one myself and trust me, it wasn’t among the top 10 easiest things i’d ever done in my life and here’s why.
I started off my little adventure by looking for speech and text datasets on Nigerian and African languages in general and I was gravely disappointed about how little research is going on in Africa like there’s little or no resource on NLP for our languages. So I did some further research(trust, i won’t give up so easily) and i was lucky enough to find Dr Ikechukwu E. Onyenwe’s PhD research thesis on the Igbo NLP project, a couple of well written research papers that spoke on preparing a corpus for the igbo language using the new world translation bible of the Jehovah witness church, tokenization based on white spaces, pos tagset among others.
It was a really good read and just like I was, he was gravely disappointed that there weren’t any resources on our languages for use for natural language processing and he is totally committed to seeing it happen which i really admired. So after reading, it gave me some ideas to build my chatbot even though it was something simple, i felt i really needed to do it as it’s a step in the right direction.
So i prepared my dataset and built a simple contextual chat bot using a small sequential deep learning model on tensorflow that ran for about 678 epochs to get the lowest possible validation loss and highest validation accuracy and deployed it on Facebook messenger using flask and heroku server to prove that it a actually works and it did(i was a little too excited but i tried to play it cool). The bot uses the Igbo language to order a type of jeans Igbo traders call stock jeans and ask’s for payment methods when it’s done.
There is still a lot of work needed to be done and i’d like to see the African community come together and create more resources for NLP as it can open a lot of doors by bringing artificial intelligence and natural language processing to places that need it the most, we all need to come together and make it happen, there are about 2000 languages in Africa so it’s certainly not a one man job. Thank you