System Work Flow

By -


Our system is a software component as a whole, which can be further divided into four sub-components namely, Digital Signal Processing Component, Speech Recognition Component, Natural Language Processing Component and Audio Replacing component. All these high-level components of our system work together in order to achieve the final objective of our system, which is to automatically detect and replace objectionable content in audio clips of local languages. This major process can be further divided into several sub-processes. Those sub-processes can be identified through our system's main workflow, which is depicted in Fig. 1.



Figure 1:  Main workflow of the system

a) Conversion of input audio files of other audio formats into .WAV (Windows Wave) format

- In order to proceed ahead with our system, the input audio files should be in .WAV format. Yet, the inputs that the users need to process might be in various formats such as .MP3 (Moving Picture Expert Group Layer-3 Audio), .MPA, .WMA, .MP4 etc. Considering all these situations, and making it easier for the users to use the application, our system is developed in a way that it accepts all kinds of audio formats. Our system has the ability to convert any of those audio formats into .WAV format. Furthermore, our system even has the ability to accept .MP4 files, separate its audio portion and then convert into a .WAV format to proceed ahead.

b) Pre-processing of audio files

- The audio inputs of users might be recorded under various environmental conditions and may be in different qualities. But in order to proceed ahead through the system and to get accurate results, the audios should be clean. Hence the input audios are first filtered through the system to make it noise-free and then amplified.

c) Clipping of audio

- The noise-free audio is then clipped. Through this process, the words are separated and saved as separate audio files.

d) Speech to Text Conversion

- The objectionable content detection in our system is mainly based on text. Hence, all audio inputs of all three languages, Sinhala, Tamil and English, are converted into text. For this purpose, Google Speech Recognizer is used.

e) Profanity Detection

- Profanity detection is performed using the converted text under three categories, namely, racist, sexist, and cursing for all the three languages, Sinhala, Tamil, and English.

f) Replacements for profanity detected

- The detected objectionable content of all three languages is replaced with a predetermined audio clip through the audio replacement component of the software.

The main workflow of our system is maintained through four inter-dependent sub-modules namely, Digital Signal Processing (DSP) module, Speech Recognition Module, Natural Language Processing (NLP) module, and the Audio Replacing Module. The article that covers the about these modules can be viewed through this link. 

Comments

Post a Comment

Popular posts from this blog

Conclusion | Automatic Audio Replacement of Objectionable Content for Sri Lankan Locale

By -
Image
When considering Sri Lanka, the government had to temporarily ban a few social media platforms several times from 2018 to 2019 to control the violence between the communities which was accelerating via hate speech and racist fake news spreading through social media. Yet, there is no proper mechanism to specifically filter and replace objectionable content in audio for Sri Lankan locale. In addressing this issue, we proposed a system for Sri Lankan locale, which automatically detects and replaces objectionable content in audio. In this proposed system, to selectively filter out the potentially objectionable audio content, the input audio is first preprocessed and converted into text format. The presence of racist, cursing, and sexist objectionable content are detected along with their corresponding locations and timestamps through the filtering mechanism. Afterwards, the detected objectionable content is seamlessly replaced with predetermined audio input. The model was tested against a ...
Read more

User Interface design

By -
Image
    User Interface gives the users the first impression of the system.  Hence it was developed with a proper understanding of the user goals, skills, preferences, and tendencies. Our Software system will mainly target the editors in the media industry, yet this can be used by whoever wants to automatically replace objectionable content from an audio clip. When we are addressing software that aims the general public,we considered that this software is accessible for almost every one of them. People of different age groups, with different interests, will be using this software.     We created simple UI, along with a decent set of formatting. Our GUIs only contain the most necessary elements. The language used within the UI is simple, and full of clarity, in order to improve the ease of use of the users. When catering to the needs of a greater community of users, it is important to have a good communication system between the user and the software. All the ...
Read more

System Evaluation

By -
Image
After the models were finalized, the hyperparameters of the classifiers and vectorizers were tuned for optimum accuracy using the pipelining method in Scikit-learn. The classification reports of final chosen models for Sinhala, Tamil and English are depicted in TABLE XV. The performance of the finalized binary classification models was evaluated using the Receiver Operating Characteristic (ROC) Curve in Fig 3, Fig 4. and Fig 5, which display the graphical relationships among the metrics shown in TABLE XIV. With the use of ‘Area Under the Curve’ measure, from Fig 1, Fig 2. and Fig 3, it can be observed that the performance of the classifiers is greater than the classifiers with no power. Fig. 1. Receiver Operating Characteristic Curve of the model for Sinhala. Fig. 2. Receiver Operating Characteristic Curve of the model for Tamil.                               Fig. 3.  Receiver Operating Characteristic Curve ...
Read more