Our Progress on the NLP(Natural Language Processing) aspect of the Project

By -
Since the beginning of our project, as a team, we have been doing background studies on the concepts involved in our project namely, Digital Signal Processing, Speech Recognition and Natural language processing (NLP). I have been handling the NLP aspect of our project for the past months and we have arrived at some conclusions relevant to our scope. 

As per Wikipedia, 

NLP is a subfield of linguistics, computer science, information engineering, and artificial intelligence concerned with the interactions between computers and human languages, in particular how to program computers to process and analyze large amounts of natural language data.
According to the methodology mentioned above, following the audio signal analysis and speech to text conversion, the NLP module detects the offensive profanity using Machine Learning. So far, we have implemented the machine learning model for English with around 96% using an opensource training dataset which consists of twitter feeds.

 As for Sinhala and Tamil datasets, we are currently scraping the social media for feeds and posts which has offensive words in them using keywords to track them down. For the Sinhala language, we have so far obtained around 10,000 entries which are under our manual classification process to train it. As for Tamil, we are in the process of getting the data.

 Firstly, we approached the English language, and we created the ML model, after trying out and analyzing several algorithms and their error rates. Since our model has a few errors dealing with nuances, we are planning on trying Deep Learning to tackle the problem.

 Thus far this is the progress on the NLP aspect of our project and we are confident that we can attain an accuracy well above 60%(as mentioned in SDS and SRS) using our system.
Keep it high!

Comments

Post a Comment

Popular posts from this blog

The proposed method and technology

By -
Image
We are proposing a method that can easily find and replace the objectionable content in an audio file using an artificial intelligence model. This method will be useful for the media editors, since they can easily find and replace the objectionable content in the audio file with one click. He can input the audio file and select the objectionable content type. After that our system will automatically find and replace the objectionable content with the predefined words or phrases. The replaced words are intended to match with the voice and the modulation of the original input.  When the program starts to run, the user will have the ability to input an audio file(wav or mp3) to be analyzed. Thereafter the audio clip will be analyzed and filtered out the noise using several filters (eg: Low pass, High Pass, etc.). Following that, the user can choose what specific objectionable audio content should be identified and replaced. The program will provide four options namely...
Read more

System Evaluation

By -
Image
After the models were finalized, the hyperparameters of the classifiers and vectorizers were tuned for optimum accuracy using the pipelining method in Scikit-learn. The classification reports of final chosen models for Sinhala, Tamil and English are depicted in TABLE XV. The performance of the finalized binary classification models was evaluated using the Receiver Operating Characteristic (ROC) Curve in Fig 3, Fig 4. and Fig 5, which display the graphical relationships among the metrics shown in TABLE XIV. With the use of ‘Area Under the Curve’ measure, from Fig 1, Fig 2. and Fig 3, it can be observed that the performance of the classifiers is greater than the classifiers with no power. Fig. 1. Receiver Operating Characteristic Curve of the model for Sinhala. Fig. 2. Receiver Operating Characteristic Curve of the model for Tamil.                               Fig. 3.  Receiver Operating Characteristic Curve ...
Read more

Conclusion | Automatic Audio Replacement of Objectionable Content for Sri Lankan Locale

By -
Image
When considering Sri Lanka, the government had to temporarily ban a few social media platforms several times from 2018 to 2019 to control the violence between the communities which was accelerating via hate speech and racist fake news spreading through social media. Yet, there is no proper mechanism to specifically filter and replace objectionable content in audio for Sri Lankan locale. In addressing this issue, we proposed a system for Sri Lankan locale, which automatically detects and replaces objectionable content in audio. In this proposed system, to selectively filter out the potentially objectionable audio content, the input audio is first preprocessed and converted into text format. The presence of racist, cursing, and sexist objectionable content are detected along with their corresponding locations and timestamps through the filtering mechanism. Afterwards, the detected objectionable content is seamlessly replaced with predetermined audio input. The model was tested against a ...
Read more