High level implementation components of the system.

By -


The main workflow of our system is maintained through four inter-dependent sub-modules namely, Digital Signal Processing (DSP) module, Speech Recognition Module, Natural Language Processing (NLP) module, and the Audio Replacing Module. 

A. DSP Module

The article about the full description and the progress of the Digital Signal processing of our system can be viewed through this link. 

At a glance, the DSP module of the system accepts the user input audio, converts it into a .WAV (Windows Wave) file if it is in some other audio format, and then performs the noise reduction process. To accomplish these tasks, High Pass Filter and the Low Pass Filter are used, with their cut-off frequency ranges set considering the fundamental human voice frequency range and the results obtained through testing out various voice samples with varying cut off frequency ranges. After the noise reduction process, the cleaned audio samples are amplified through the DSP module as its final task.

B. Speech Recognition Module

Since we are classifying the objectionable content in textual form, the use of speech recognition is required to convert the audio into text format. We used the Google Speech Recognizer to complete this task accurately and efficiently for all the three languages, Sinhala, Tamil, and English.

C. NLP Module

In order to validate the contextual offensive nature, a preliminary filtering model was created which takes the converted sentences as input and classifies whether they are simply offensive or non-offensive through a binary classification. If the text is classified as offensive, then secondary filtering is carried out with a separate multi-class text classification model which classifies each word in the sentence into sexist, racist, cursing, and non-offensive categories. The models in preliminary filtering involve the Term Frequency–Inverse Document Frequency (TF-IDF) vectorizer and Support vector Machine (SVM) algorithm with varying hyperparameters. This conclusion for the binary model was arrived at considering that SVM performs well at text classification.

If the text is classified as offensive, then secondary filtering is carried out with a separate multi-class text classification model which classifies each word in the sentence into sexist, racist, cursing, and non-offensive categories. The secondary filtering models were trained after several comparisons with Naive Bayes, Decision Tree, and other classification algorithms along with Countvectorizer and TF-IDF vectorizer. Considering the results with the pipelined comparisons, different classifiers were chosen accordingly.

D. Audio Replacing Module

This process is started with the first identified objectionable word. The component gets the start time of that specific word and splits the audio into two parts. Next, the component gets the end time of the first identified objectionable word and accordingly splits the second part of the audio. Then it stores the tail part of the 2nd portion of the audio. Likewise, the same method applies to all identified objectionable words and finally, the system gets a series of audio clips stored in a folder. At last, the split audio clips are merged based on the user requirements, using a beep, silence or recorded audio.



Comments

Post a Comment

Popular posts from this blog

Conclusion | Automatic Audio Replacement of Objectionable Content for Sri Lankan Locale

By -
Image
When considering Sri Lanka, the government had to temporarily ban a few social media platforms several times from 2018 to 2019 to control the violence between the communities which was accelerating via hate speech and racist fake news spreading through social media. Yet, there is no proper mechanism to specifically filter and replace objectionable content in audio for Sri Lankan locale. In addressing this issue, we proposed a system for Sri Lankan locale, which automatically detects and replaces objectionable content in audio. In this proposed system, to selectively filter out the potentially objectionable audio content, the input audio is first preprocessed and converted into text format. The presence of racist, cursing, and sexist objectionable content are detected along with their corresponding locations and timestamps through the filtering mechanism. Afterwards, the detected objectionable content is seamlessly replaced with predetermined audio input. The model was tested against a ...
Read more

User Interface design

By -
Image
    User Interface gives the users the first impression of the system.  Hence it was developed with a proper understanding of the user goals, skills, preferences, and tendencies. Our Software system will mainly target the editors in the media industry, yet this can be used by whoever wants to automatically replace objectionable content from an audio clip. When we are addressing software that aims the general public,we considered that this software is accessible for almost every one of them. People of different age groups, with different interests, will be using this software.     We created simple UI, along with a decent set of formatting. Our GUIs only contain the most necessary elements. The language used within the UI is simple, and full of clarity, in order to improve the ease of use of the users. When catering to the needs of a greater community of users, it is important to have a good communication system between the user and the software. All the ...
Read more

System Evaluation

By -
Image
After the models were finalized, the hyperparameters of the classifiers and vectorizers were tuned for optimum accuracy using the pipelining method in Scikit-learn. The classification reports of final chosen models for Sinhala, Tamil and English are depicted in TABLE XV. The performance of the finalized binary classification models was evaluated using the Receiver Operating Characteristic (ROC) Curve in Fig 3, Fig 4. and Fig 5, which display the graphical relationships among the metrics shown in TABLE XIV. With the use of ‘Area Under the Curve’ measure, from Fig 1, Fig 2. and Fig 3, it can be observed that the performance of the classifiers is greater than the classifiers with no power. Fig. 1. Receiver Operating Characteristic Curve of the model for Sinhala. Fig. 2. Receiver Operating Characteristic Curve of the model for Tamil.                               Fig. 3.  Receiver Operating Characteristic Curve ...
Read more