System Evaluation

By -



After the models were finalized, the hyperparameters of the classifiers and vectorizers were tuned for optimum accuracy using the pipelining method in Scikit-learn. The classification reports of final chosen models for Sinhala, Tamil and English are depicted in TABLE XV.




The performance of the finalized binary classification models was evaluated using the Receiver Operating Characteristic (ROC) Curve in Fig 3, Fig 4. and Fig 5, which display the graphical relationships among the metrics shown in TABLE XIV. With the use of ‘Area Under the Curve’ measure, from Fig 1, Fig 2. and Fig 3, it can be observed that the performance of the classifiers is greater than the classifiers with no power.
Fig. 1. Receiver Operating Characteristic Curve of the model for Sinhala.


Fig. 2. Receiver Operating Characteristic Curve of the model for Tamil.


                              Fig. 3.  Receiver Operating Characteristic Curve of the model for English. 

The Precision vs Recall curve can be used to measure the quality of the output generated by the classifiers. Using the AUC measure, it can be concluded that the classifier is returning accurate results and majority results are positive. It can be clearly seen from Fig 4, Fig 5 and Fig 6.

Fig. 4. Precision vs Recall Curve of the model for Sinhala.


Fig. 5. Precision vs Recall Curve of the model for Tamil.Add caption


Fig. 6. Precision vs Recall Curve of the model for English

Classification reports of final chosen multi-class models for Sinhala, Tamil and English is shown in TABLE XVI




Normalized confusion matrix for the finalized Multi-class model for Sinhala is shown in TABLE XVII


Normalized confusion matrix for the finalized Multi-class model for Tamil is shown in TABLE XVIII




Normalized confusion matrix for the finalized Multi-class model for English is shown in TABLE XIX.






The testing of the system was done with a set of a few people of age within 25 to 30 and found an overall positive response on their experience with the system.





Comments

Post a Comment

Popular posts from this blog

Conclusion | Automatic Audio Replacement of Objectionable Content for Sri Lankan Locale

By -
Image
When considering Sri Lanka, the government had to temporarily ban a few social media platforms several times from 2018 to 2019 to control the violence between the communities which was accelerating via hate speech and racist fake news spreading through social media. Yet, there is no proper mechanism to specifically filter and replace objectionable content in audio for Sri Lankan locale. In addressing this issue, we proposed a system for Sri Lankan locale, which automatically detects and replaces objectionable content in audio. In this proposed system, to selectively filter out the potentially objectionable audio content, the input audio is first preprocessed and converted into text format. The presence of racist, cursing, and sexist objectionable content are detected along with their corresponding locations and timestamps through the filtering mechanism. Afterwards, the detected objectionable content is seamlessly replaced with predetermined audio input. The model was tested against a
Read more

High level implementation components of the system.

By -
Image
The main workflow of our system is maintained through four inter-dependent sub-modules namely, Digital Signal Processing (DSP) module, Speech Recognition Module, Natural Language Processing (NLP) module, and the Audio Replacing Module.  A. DSP Module The article about the full description and the progress of the Digital Signal processing of our system can be viewed through this link .  At a glance, the DSP module of the system accepts the user input audio, converts it into a .WAV (Windows Wave) file if it is in some other audio format, and then performs the noise reduction process. To accomplish these tasks, High Pass Filter and the Low Pass Filter are used, with their cut-off frequency ranges set considering the fundamental human voice frequency range and the results obtained through testing out various voice samples with varying cut off frequency ranges. After the noise reduction process, the cleaned audio samples are amplified through the DSP module as its final task.
Read more