Addressing Abuse, Trolls, and Roasts on an Online Platform

Social media is a platform where anyone and everyone can share their views and thoughts. It helps in the development of society with everyone contributing and sharing their ideas and impressions. But it has some drawbacks as well. Some people can use this platform in inappropriate ways thus it can become harmful as well for the people and society at large. There are many examples of online harassment of internet users. Due to this, even educational institutions are apprehensive about students’ misbehaving on their online platforms.

Hence we need to come up with an approach where people can share their experiences, views, and thoughts without fear of being trolled or harassed.  Many platforms are devising ways of taking safety measures to improve the safety of their users and protect them from cyber troubles. But the measures that are in place right now are not real-time measures. Some platform refines their content on the basis of upvoters and downvoters from crowdsourcing, manual analysis of content, and turning off comments to mitigate inappropriate comments. But these approaches are not real-time, are inefficient, and non-scalable. Therefore there is a need for a solution that is real-time and can automatically detect inappropriate content without human intervention.

One solution to solve this issue is the “Perspective API” – the API which helps to analyze the comments and content in real-time. The Perspective API is built to make the world a safer place where it’s easier to host conversations.  Perspective was created by Jigsaw and Google’s Counter Abuse Technology team in a collaborative research initiative called Conversation-AI. The objective of the Perspective API is to build interest, quality, and compassion of online conversation at scale. Developers and publishers can utilize Perspective API  to recognize and filter text that can be harmful and offensive which can include potentially offensive text, including threats, insults, profanity, and toxic language.  The API is based on machine learning models that give a score of impact a comment might have on a conversation.  This score gives real-time feedback to commenters, or help moderators do their job, or allow readers to find relevant information.

Perspective API can address 85-90% of the problems and it will evolve further. Perspective can help make moderation more efficient by sorting and flagging comments automatically, but there should always be a human in the loop for moderation decisions.

How to use Google Perspective API? (For detailed reference)

You should have a Google account, that gives you access to Google Cloud Console. Use Google Cloud to authenticate your API requests. Go to the Google Cloud console and use an existing project or follow these steps to create a new one:

a. Sign in with your Google account.

b. On the Console page, click the “Select a Project” drop-down and then click New Project.

c. Name the project.

d. Click “Select a Project” again. Your project is now included in the “Project” drop-down.

The Most Important step to use google Perspective API is that you need to whitelist the email account.

After you have created a Google Cloud project, please fill out  the form to request access to the API. Upon submission, you will receive an email confirmation and be able to view and enable the API.

Generate an API key to authenticate your requests.

Go to the API credentials page, click Create credentials, and choose “API Key”. (Make sure you’re in the same project you have created in start).

Now you are all set to use this API with API key in respective language project.

Here is Command Prompt Shell CURL Request you can fire in cmd shell.

curl -H "Content-Type: application/json" -X POST https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=YOUR_API_KEY -d "{\"comment\": {\"text\": \"hello\"},languages: ['en'],requestedAttributes: {TOXICITY:{},INSULT:{},FLIRTATION:{}} }"

Attach screenshots of some examples

Here is sample PHP CURL Request you can integrate in your PHP project.

    	$ch = curl_init();
	$cmtTEXT=”HELLO”;
        curl_setopt($ch, CURLOPT_URL, 'https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=YOUR_API_KEY);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_POST, 1);
        curl_setopt($ch, CURLOPT_POSTFIELDS, "{comment: {text: '".$cmtTEXT."'},languages: ['en'],requestedAttributes: {TOXICITY:{},INSULT:{},FLIRTATION:{}} }");
        
        $headers = array();
        $headers[] = 'Content-Type: application/json';
        $headers[] = 'Accept: application/json';

        curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
        
        $res = curl_exec($ch);
        if (curl_errno($ch)) {
            echo 'Error:' . curl_error($ch);
        }
        curl_close($ch);
        $dt=$res;

        print_r($dt);
        exit();