{"metadata":{"colab":{"provenance":[],"gpuType":"T4"},"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"accelerator":"GPU","kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":10737,"databundleVersionId":290346,"sourceType":"competition"},{"sourceId":5930,"sourceType":"modelInstanceVersion","modelInstanceId":4703}],"dockerImageVersionId":30626,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Overview\n\nKerasNLP makes it very easy to create simple model pipelines at a very fast rate. In this guide we create a simple text classification pipeline from scratch including augmentation, model building etc.","metadata":{"id":"pcanbuwJ7PUX"}},{"cell_type":"markdown","source":"## Imports & setup\n\nThis tutorial requires you to have KerasNLP installed:\n\n```shell\npip install keras-nlp\n```\n\nWe begin by importing all required packages:","metadata":{"id":"DmC_kCnI7VPq"}},{"cell_type":"code","source":"!pip install keras-nlp wandb\n","metadata":{"execution":{"iopub.status.busy":"2024-03-08T05:47:33.824275Z","iopub.execute_input":"2024-03-08T05:47:33.825205Z","iopub.status.idle":"2024-03-08T05:47:47.027523Z","shell.execute_reply.started":"2024-03-08T05:47:33.825168Z","shell.execute_reply":"2024-03-08T05:47:47.026226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport skimage.io as io\nimport random\nimport os\nimport cv2\nimport pandas as pd\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom glob import glob\nfrom scipy.io import loadmat\nimport matplotlib.pyplot as plt\nimport keras_nlp\nimport tensorflow as tf\nimport keras\nfrom keras.layers import *\nfrom keras import Sequential\nimport matplotlib.pyplot as plt","metadata":{"id":"5WBE3tLKUU94","execution":{"iopub.status.busy":"2024-03-08T05:47:47.029802Z","iopub.execute_input":"2024-03-08T05:47:47.030170Z","iopub.status.idle":"2024-03-08T05:48:00.161623Z","shell.execute_reply.started":"2024-03-08T05:47:47.030128Z","shell.execute_reply":"2024-03-08T05:48:00.160656Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data loading\n\nThis guide uses the\n[Quora Insincere Questions Classification Dataset](https://www.kaggle.com/competitions/quora-insincere-questions-classification/data)\nfor demonstration purposes.\n\nTo get started, we first load the dataset:\n","metadata":{"id":"N75_5WJeok06"}},{"cell_type":"code","source":"df = pd.read_csv('/kaggle/input/quora-insincere-questions-classification/train.csv')\ndf","metadata":{"id":"HbS3wKtXUcxV","outputId":"4b458f43-226f-4f43-deba-0052db9a137a","execution":{"iopub.status.busy":"2024-03-08T05:48:00.162998Z","iopub.execute_input":"2024-03-08T05:48:00.163803Z","iopub.status.idle":"2024-03-08T05:48:04.334866Z","shell.execute_reply.started":"2024-03-08T05:48:00.163764Z","shell.execute_reply":"2024-03-08T05:48:04.333871Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"text = df['question_text'].tolist()\ntarget = df['target'].tolist()","metadata":{"id":"g1hVyigCUn7O","execution":{"iopub.status.busy":"2024-03-08T05:48:04.337085Z","iopub.execute_input":"2024-03-08T05:48:04.337432Z","iopub.status.idle":"2024-03-08T05:48:04.400593Z","shell.execute_reply.started":"2024-03-08T05:48:04.337399Z","shell.execute_reply":"2024-03-08T05:48:04.399556Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from kaggle_secrets import UserSecretsClient\nimport wandb\nuser_secrets = UserSecretsClient()\nsecret_value_0 = user_secrets.get_secret(\"api_key\")\nwandb.login(key = secret_value_0)","metadata":{"execution":{"iopub.status.busy":"2024-03-08T05:48:04.401688Z","iopub.execute_input":"2024-03-08T05:48:04.402035Z","iopub.status.idle":"2024-03-08T05:48:08.569906Z","shell.execute_reply.started":"2024-03-08T05:48:04.402010Z","shell.execute_reply":"2024-03-08T05:48:08.568984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"run = wandb.init(project=\"quora\")\ntable = wandb.Table(data=df)\nrun.log({'data':table})\nrun.finish()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model Building\n\nWe use the pretrained `Roberta Classifier` from the KerasNLP to build a simple text classifier.","metadata":{"id":"N0EqjUZTCofa"}},{"cell_type":"code","source":"from wandb.keras import WandbMetricsLogger\nrun = wandb.init(project=\"quora\",name = 'model_training')\n\nclassifier = keras_nlp.models.RobertaClassifier.from_preset(\n    \"roberta_base_en\",\n    num_classes=2,\n)\nclassifier.backbone.trainable = False\n\nhistory = classifier.fit(x=text[:5000], y=target[:5000], verbose =1, epochs=1,batch_size=16,callbacks=[WandbMetricsLogger()])","metadata":{"id":"ww1txzj_YCjh","outputId":"eb83a9e9-0861-4eac-8147-a8f7aaa0e916","execution":{"iopub.status.busy":"2024-03-08T05:48:15.851643Z","iopub.execute_input":"2024-03-08T05:48:15.852664Z","iopub.status.idle":"2024-03-08T05:56:13.761829Z","shell.execute_reply.started":"2024-03-08T05:48:15.852629Z","shell.execute_reply":"2024-03-08T05:56:13.760537Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"run.finish()","metadata":{"id":"byM0Z84SksVk","outputId":"4efed16f-7b00-4d54-99ec-1955afc7d44d","execution":{"iopub.status.busy":"2024-03-08T05:56:13.764497Z","iopub.execute_input":"2024-03-08T05:56:13.764940Z","iopub.status.idle":"2024-03-08T05:56:19.103057Z","shell.execute_reply.started":"2024-03-08T05:56:13.764899Z","shell.execute_reply":"2024-03-08T05:56:19.102348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"classifier.predict([text[0]])","metadata":{"id":"vMuEl49mkcES","outputId":"9e551d45-2cd4-41d3-9298-21c7b27a570b","execution":{"iopub.status.busy":"2023-12-24T02:54:16.403845Z","iopub.execute_input":"2023-12-24T02:54:16.404503Z","iopub.status.idle":"2023-12-24T02:54:37.596785Z","shell.execute_reply.started":"2023-12-24T02:54:16.404451Z","shell.execute_reply":"2023-12-24T02:54:37.595579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"id":"4aECUazjnBH2"},"execution_count":null,"outputs":[]}]}