{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Overview","metadata":{}},{"cell_type":"markdown","source":"**As we know there is no training dataset provided for this competition, this notebook shows you how to use previous Jigsaw competitions datasets to create a dataset for this competition. Don't just use the output file since what I've done at the end is just a basic idea. One must understand the idea so that you can create a dataset on your own depending on how many times particular comment can be repeated.**","metadata":{}},{"cell_type":"markdown","source":"**Training data**\n\nNote, there is no training data for this competition. You can refer to previous Jigsaw competitions for data that might be useful to train models. But note that the task of previous competitions has been to predict the probability that a comment was toxic, rather than the degree or severity of a comment's toxicity.\n\n* Toxic Comment Classification Challenge\n* Jigsaw Unintended Bias in Toxicity Classification\n* Jigsaw Multilingual Toxic Comment Classification\n\nWhile we don't include training data, we do provide a set of paired toxicity rankings that can be used to validate models.","metadata":{}},{"cell_type":"markdown","source":"**Evaluation**\n\nSubmissions are evaluated on Average Agreement with Annotators. For the ground truth, annotators were shown two comments and asked to identify which of the two was more toxic. Pairs of comments can be, and often are, rated by more than one annotator, and may have been ordered differently by different annotators.\n\nFor each of the approximately 200,000 pair ratings in the ground truth test data, we use your predicted toxicity score to rank the comment pair. The pair receives a 1 if this ranking matches the annotator ranking, or 0 if it does not match.\n\nThe final score is the average across all the pair evaluations.","metadata":{}},{"cell_type":"markdown","source":"**Objective**\n\nIn this competition you will be ranking comments in order of severity of toxicity. You are given a list of comments, and each comment should be scored according to their relative toxicity. Comments with a higher degree of toxicity should receive a higher numerical value compared to comments with a lower degree of toxicity.","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport random","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"comp1=pd.read_csv('../input/jigsaw-multilingual-toxic-comment-classification/jigsaw-toxic-comment-train.csv')","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:07:26.476904Z","iopub.execute_input":"2021-11-10T10:07:26.477707Z","iopub.status.idle":"2021-11-10T10:07:52.66536Z","shell.execute_reply.started":"2021-11-10T10:07:26.477668Z","shell.execute_reply":"2021-11-10T10:07:52.664701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"comp1[['id','comment_text','toxic','severe_toxic']].head()","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:42:55.999127Z","iopub.execute_input":"2021-11-10T10:42:55.999448Z","iopub.status.idle":"2021-11-10T10:42:56.024696Z","shell.execute_reply.started":"2021-11-10T10:42:55.99941Z","shell.execute_reply":"2021-11-10T10:42:56.023687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"comp2=pd.read_csv('../input/jigsaw-multilingual-toxic-comment-classification/jigsaw-unintended-bias-train.csv')","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:40:36.004227Z","iopub.execute_input":"2021-11-10T10:40:36.005088Z","iopub.status.idle":"2021-11-10T10:40:58.678576Z","shell.execute_reply.started":"2021-11-10T10:40:36.005046Z","shell.execute_reply":"2021-11-10T10:40:58.677695Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"comp2[['id','comment_text','toxic','severe_toxicity']].head()","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:42:41.262597Z","iopub.execute_input":"2021-11-10T10:42:41.262905Z","iopub.status.idle":"2021-11-10T10:42:41.4357Z","shell.execute_reply.started":"2021-11-10T10:42:41.262869Z","shell.execute_reply":"2021-11-10T10:42:41.434843Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(comp1.shape)\nprint(comp2.shape)","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:09:23.565866Z","iopub.execute_input":"2021-11-10T10:09:23.566669Z","iopub.status.idle":"2021-11-10T10:09:23.574639Z","shell.execute_reply.started":"2021-11-10T10:09:23.566606Z","shell.execute_reply":"2021-11-10T10:09:23.573686Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* For now let's just consider the dataset from the very first Jigsaw competition and see how we can use that to create a train dataset for the current competition.","metadata":{}},{"cell_type":"code","source":"comp1.toxic.value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:09:41.380012Z","iopub.execute_input":"2021-11-10T10:09:41.380329Z","iopub.status.idle":"2021-11-10T10:09:41.395594Z","shell.execute_reply.started":"2021-11-10T10:09:41.380298Z","shell.execute_reply":"2021-11-10T10:09:41.394764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"comp1.severe_toxic.value_counts()","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:12:23.061317Z","iopub.execute_input":"2021-11-10T10:12:23.06162Z","iopub.status.idle":"2021-11-10T10:12:23.070262Z","shell.execute_reply.started":"2021-11-10T10:12:23.061591Z","shell.execute_reply":"2021-11-10T10:12:23.069469Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Idea","metadata":{}},{"cell_type":"markdown","source":"* We can create a dataset to train our models in this competition in the format of validation_data.csv\n* There are many ways one can create a training dataset by considering the non toxic comments i.e. toxicity==0 as less toxic ones and toxic comments i.e. toxicity==1 as more toxic ones. \n* And we can also use the severe_toxic feature to compare comments whose toxicity==1 and divide the ones whose severe_toxicity==0 as less_toxic and ones whose severe_toxity==1 as more_toxic. All of this only for the dataset from the first Jigsaw competition. We have the 2nd competition dataset which is almost 10 times that of first.\n* If we consider all the possibilities we can create a dataset of size ****(idk, you calculate).\n* So for now I've created it this way. For each non toxic comment(202165 of those) I've chosen a toxic comment(from 21384 comments) randomly. The length of train.csv will be 202165\n","metadata":{}},{"cell_type":"code","source":"less_toxic=[]\nmore_toxic=[]\nnon_toxic=comp1[comp1.toxic==0].reset_index(drop=True)\ntoxic=comp1[comp1.toxic==1].reset_index(drop=True)\nfor i in range(len(non_toxic)):\n    less_toxic.append(non_toxic.loc[i,'comment_text'])\n    j=random.randint(0,len(toxic)-1)\n    more_toxic.append(toxic.loc[j,'comment_text'])","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:38:07.449121Z","iopub.execute_input":"2021-11-10T10:38:07.449632Z","iopub.status.idle":"2021-11-10T10:38:11.392656Z","shell.execute_reply.started":"2021-11-10T10:38:07.449585Z","shell.execute_reply":"2021-11-10T10:38:11.391879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train=pd.DataFrame()\ntrain['less_toxic']=less_toxic\ntrain['more_toxic']=more_toxic","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:39:23.094939Z","iopub.execute_input":"2021-11-10T10:39:23.095214Z","iopub.status.idle":"2021-11-10T10:39:23.200273Z","shell.execute_reply.started":"2021-11-10T10:39:23.095184Z","shell.execute_reply":"2021-11-10T10:39:23.199465Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head()","metadata":{"execution":{"iopub.status.busy":"2021-11-10T10:39:28.773337Z","iopub.execute_input":"2021-11-10T10:39:28.773635Z","iopub.status.idle":"2021-11-10T10:39:28.783193Z","shell.execute_reply.started":"2021-11-10T10:39:28.773604Z","shell.execute_reply":"2021-11-10T10:39:28.782376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.to_csv('train.csv',index=False)","metadata":{},"execution_count":null,"outputs":[]}]}