{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Acknowledgements:\n- https://www.kaggle.com/tarunpaparaju/jigsaw-multilingual-toxicity-eda-models#Introduction\n- https://github.com/dipanjanS/deep_transfer_learning_nlp_dhs2019\n\nRun it on [Kaggle Kernels](https://www.kaggle.com/spsayakpaul/jigsaw-multilingual-toxic-comment-classification). "},{"metadata":{"id":"N0PbF_7bca25"},"cell_type":"markdown","source":"In this notebook, I am going to build a baseline model based on [DistilBERT](https://medium.com/huggingface/distilbert-8cf3380435b5) for the Jigsaw Multilingual Toxic Comment Classification (Kaggle challenge [link](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification)). \n\n**What am I predicting?** (comes from the challenge homepage)\n\nYou are predicting the probability that a comment is toxic. A toxic comment would receive a 1.0. A benign, non-toxic comment would receive a 0.0. In the test set, all comments are classified as either a 1.0 or a 0.0."},{"metadata":{"id":"9npbakvTcSg8","outputId":"11d91e22-b482-4c32-82e2-15c6fa4d1629","trusted":true},"cell_type":"code","source":"import tensorflow as tf\nprint(tf.__version__)","execution_count":null,"outputs":[]},{"metadata":{"id":"YKrrMbWAd_fX"},"cell_type":"markdown","source":"An amazing EDA on the dataset in available here: https://www.kaggle.com/tarunpaparaju/jigsaw-multilingual-toxicity-eda-models. "},{"metadata":{"id":"ZzPnhW-ed9Ja"},"cell_type":"markdown","source":"## Load and prepare data"},{"metadata":{"id":"JDmH46e-c4Yz","outputId":"5907bd56-f994-49c8-f4f5-bee810b1bd83","trusted":true},"cell_type":"code","source":"!ls /kaggle/input/jigsaw-multilingual-toxic-comment-classification/","execution_count":null,"outputs":[]},{"metadata":{"id":"WOIVOHMxdlGO"},"cell_type":"markdown","source":"Data description is available [here](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/data). "},{"metadata":{"id":"gR66Hwq1dei3","trusted":true},"cell_type":"code","source":"# Load datasets\nimport pandas as pd\nimport os\n\nDATA_PATH = \"/kaggle/input/jigsaw-multilingual-toxic-comment-classification/\"\n\nTEST_PATH = os.path.join(DATA_PATH, \"test.csv\")\nVAL_PATH = os.path.join(DATA_PATH, \"validation.csv\")\nTRAIN_PATH = os.path.join(DATA_PATH, \"jigsaw-toxic-comment-train.csv\")\n\nval_data = pd.read_csv(VAL_PATH)\ntest_data = pd.read_csv(TEST_PATH)\ntrain_data = pd.read_csv(TRAIN_PATH)","execution_count":null,"outputs":[]},{"metadata":{"id":"id1prBeqeVJM","outputId":"2acc9df9-f0c9-4623-e461-754859db4191","trusted":true},"cell_type":"code","source":"# Preview train set\ntrain_data.sample(5)","execution_count":null,"outputs":[]},{"metadata":{"id":"-yyaGcAhehZ6"},"cell_type":"markdown","source":"Columns (comes from [here](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/data)): \n- id - identifier within each file.\n- comment_text - the text of the comment to be classified.\n- toxic:identity_hate - whether or not the comment is classified as toxic. "},{"metadata":{"id":"Ya4qc1NrgI6s","outputId":"c52b6859-70fd-4d9a-9e98-eaf26ed4a6c8","trusted":true},"cell_type":"code","source":"val_data.sample(5)","execution_count":null,"outputs":[]},{"metadata":{"id":"zbkaRDiCgLD9","outputId":"f00bc7e5-a4eb-4ad8-89ed-d7dcd5e2f4f9","trusted":true},"cell_type":"code","source":"test_data.sample(5)","execution_count":null,"outputs":[]},{"metadata":{"id":"ErgoMBGIe-Vz"},"cell_type":"markdown","source":"It's a multilingual dataset as you can see. \n\nI am going to borrow the helper functions as shown here: https://www.kaggle.com/tarunpaparaju/jigsaw-multilingual-toxicity-eda-models. "},{"metadata":{"id":"80ouy6L9eZMX","trusted":true},"cell_type":"code","source":"# Remove usernames and links\nimport re\n\nval = val_data\ntrain = train_data\n\ndef clean(text):\n    # fill the missing entries and convert them to lower case\n    text = text.fillna(\"fillna\").str.lower()\n    # replace the newline characters with space \n    text = text.map(lambda x: re.sub('\\\\n',' ',str(x)))\n    text = text.map(lambda x: re.sub(\"\\[\\[User.*\",'',str(x)))\n    # remove usernames and links\n    text = text.map(lambda x: re.sub(\"\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\",'',str(x)))\n    text = text.map(lambda x: re.sub(\"\\(http://.*?\\s\\(http://.*\\)\",'',str(x)))\n    return text\n\nval[\"comment_text\"] = clean(val[\"comment_text\"])\ntest_data[\"content\"] = clean(test_data[\"content\"])\ntrain[\"comment_text\"] = clean(train[\"comment_text\"])","execution_count":null,"outputs":[]},{"metadata":{"id":"OJatJaA2rEoW","outputId":"0f54b8c6-8289-4848-f2b5-981be52296c2","trusted":true},"cell_type":"code","source":"# Load DistilBERT tokenizer\nimport transformers\n\ntokenizer = transformers.DistilBertTokenizer.from_pretrained('distilbert-base-multilingual-cased')","execution_count":null,"outputs":[]},{"metadata":{"id":"v-7uiZJprjaO"},"cell_type":"markdown","source":"The following function comes from [here](https://github.com/dipanjanS/deep_transfer_learning_nlp_dhs2019/blob/master/notebooks/6%20-%20Transformers%20-%20DistilBERT.ipynb)."},{"metadata":{"id":"CRbtTaUvrXxF","trusted":true},"cell_type":"code","source":"import numpy as np\nimport tqdm\n\ndef create_bert_input_features(tokenizer, docs, max_seq_length):\n    \n    all_ids, all_masks = [], []\n    for doc in tqdm.tqdm(docs, desc=\"Converting docs to features\"):\n        tokens = tokenizer.tokenize(doc)\n        if len(tokens) > max_seq_length-2:\n            tokens = tokens[0 : (max_seq_length-2)]\n        tokens = ['[CLS]'] + tokens + ['[SEP]']\n        ids = tokenizer.convert_tokens_to_ids(tokens)\n        masks = [1] * len(ids)\n        # Zero-pad up to the sequence length.\n        while len(ids) < max_seq_length:\n            ids.append(0)\n            masks.append(0)\n        all_ids.append(ids)\n        all_masks.append(masks)\n    encoded = np.array([all_ids, all_masks])\n    return encoded","execution_count":null,"outputs":[]},{"metadata":{"id":"DZh_s_Ecrn-c","trusted":true},"cell_type":"code","source":"# Segregate the comments and their labels (not applicable for test set)\ntrain_comments = train.comment_text.astype(str).values\nval_comments = val_data.comment_text.astype(str).values\ntest_comments = test_data.content.astype(str).values\n\ny_valid = val.toxic.values\ny_train = train.toxic.values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import gc\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"id":"DDCRnH-AsLeZ","outputId":"c7d8e136-a3c6-43c1-8da6-72efdd056522","trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"# Encode the comments\nMAX_SEQ_LENGTH = 500\n\ntrain_features_ids, train_features_masks = create_bert_input_features(tokenizer, train_comments, \n                                                                      max_seq_length=MAX_SEQ_LENGTH)\nval_features_ids, val_features_masks = create_bert_input_features(tokenizer, val_comments, \n                                                                  max_seq_length=MAX_SEQ_LENGTH)\n# test_features = create_bert_input_features(tokenizer, test_comments, \n#                                            max_seq_length=MAX_SEQ_LENGTH)","execution_count":null,"outputs":[]},{"metadata":{"id":"6lm5Yy88xN2L","outputId":"8934568d-b538-4559-974c-78a6c42855dc","trusted":true},"cell_type":"code","source":"# Verify the shapes\nprint(train_features_ids.shape, train_features_masks.shape, y_train.shape)\nprint(val_features_ids.shape, val_features_masks.shape, y_valid.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Configure TPU\nfrom kaggle_datasets import KaggleDatasets\n\ntpu = tf.distribute.cluster_resolver.TPUClusterResolver()\ntf.config.experimental_connect_to_cluster(tpu)\ntf.tpu.experimental.initialize_tpu_system(tpu)\nstrategy = tf.distribute.experimental.TPUStrategy(tpu)\n\nGCS_DS_PATH = KaggleDatasets().get_gcs_path('jigsaw-multilingual-toxic-comment-classification')\n\nEPOCHS = 2\nBATCH_SIZE = 32 * strategy.num_replicas_in_sync","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create TensorFlow datasets for better performance\ntrain_ds = (\n    tf.data.Dataset\n    .from_tensor_slices(((train_features_ids, train_features_masks), y_train))\n    .repeat()\n    .shuffle(2048)\n    .batch(BATCH_SIZE)\n    .prefetch(tf.data.experimental.AUTOTUNE)\n)\n    \nvalid_ds = (\n    tf.data.Dataset\n    .from_tensor_slices(((val_features_ids, val_features_masks), y_valid))\n    .repeat()\n    .batch(BATCH_SIZE)\n    .prefetch(tf.data.experimental.AUTOTUNE)\n)","execution_count":null,"outputs":[]},{"metadata":{"id":"mMSzIXvAyR27"},"cell_type":"markdown","source":"## Model building and training"},{"metadata":{"id":"mk-WornLuqSx","trusted":true},"cell_type":"code","source":"# Create utility function to get a training ready model on demand\ndef get_training_model():\n    inp_id = tf.keras.layers.Input(shape=(MAX_SEQ_LENGTH,), dtype=tf.int64, name=\"bert_input_ids\")\n    inp_mask = tf.keras.layers.Input(shape=(MAX_SEQ_LENGTH,), dtype=tf.int64, name=\"bert_input_masks\")\n    inputs = [inp_id, inp_mask]\n\n    hidden_state = transformers.TFDistilBertModel.from_pretrained('distilbert-base-multilingual-cased')(inputs)[0]\n    pooled_output = hidden_state[:, 0]    \n    dense1 = tf.keras.layers.Dense(128, activation='relu')(pooled_output)\n    output = tf.keras.layers.Dense(1, activation='sigmoid')(dense1)\n\n    model = tf.keras.Model(inputs=inputs, outputs=output)\n    model.compile(optimizer=tf.optimizers.Adam(learning_rate=2e-5, \n                                            epsilon=1e-08), \n                loss='binary_crossentropy', metrics=['accuracy'])\n\n    return model","execution_count":null,"outputs":[]},{"metadata":{"id":"a1ED6Cyxxpd2","trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"# Authorize wandb\nimport wandb\n\nwandb.login()\nfrom wandb.keras import WandbCallback","execution_count":null,"outputs":[]},{"metadata":{"id":"NTniAlOc2_fj","outputId":"df3f2bf2-ee30-4472-dfe3-658eb848e979","trusted":true},"cell_type":"code","source":"# Initialize wandb\nwandb.init(project=\"jigsaw-toxic\", id=\"distilbert-tpu-kaggle-weighted\")","execution_count":null,"outputs":[]},{"metadata":{"id":"Z2b4mh00zPQq","outputId":"911a2410-23d1-4889-e533-f2195ca9d4fe","trusted":true},"cell_type":"code","source":"# Create 32 random indices from the English only test comments\nRANDOM_INDICES = np.random.choice(test_comments.shape[0], 32)\nRANDOM_INDICES","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We will be logging some sample predictions on the test dataset to see how our model is doing as it is getting trained. Now, as this is a mulitlingual dataset, we may need to convert a given comment to a language of our choice to make sense of the model's prediction. We will be using the `googletrans` library. "},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install -q googletrans","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Demo examples of translations\nfrom googletrans import Translator\n\nsample_comment = test_comments[48649]\nprint(\"Original comment:\", sample_comment)\ntranslated_comment = Translator().translate(sample_comment)\nprint(\"\\n\")\nprint(\"Translated comment:\", translated_comment.text)","execution_count":null,"outputs":[]},{"metadata":{"id":"ydbxVOtV0by0","trusted":true},"cell_type":"code","source":"# Create a sample prediction logger\n# A custom callback to view predictions on the above samples in real-time\nclass TextLogger(tf.keras.callbacks.Callback):\n    def __init__(self):\n        super(TextLogger, self).__init__()\n\n    def on_epoch_end(self, logs, epoch):\n        samples = []\n        for index in RANDOM_INDICES:\n            # Grab the comment and translate it\n            comment = test_comments[index]\n            translated_comment = Translator().translate(comment).text\n            # Create BERT features\n            comment_feature_ids, comment_features_masks = create_bert_input_features(tokenizer,  \n                                    comment, max_seq_length=MAX_SEQ_LENGTH)\n            # Employ the model to get the prediction and parse it\n            predicted_label = self.model.predict([comment_feature_ids, comment_features_masks])\n            predicted_label = np.argmax(predicted_label[0])\n            if predicted_label==0: predicted_label=\"Non-Toxic\"\n            else: predicted_label=\"Toxic\"\n            \n            sample = [comment, translated_comment, predicted_label]\n            \n            samples.append(sample)\n        wandb.log({\"text\": wandb.Table(data=samples, \n                                       columns=[\"Comment\", \"Translated Comment\", \"Predicted Label\"])})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Garbage collection\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Account for the class imbalance\nfrom sklearn.utils import class_weight\n\nclass_weights = class_weight.compute_class_weight('balanced',\n                                                 np.unique(y_train),\n                                                 y_train)\nclass_weights","execution_count":null,"outputs":[]},{"metadata":{"id":"MedenR-oxzhi","outputId":"9bf330f7-4a77-4da3-a42c-b4c17679677f","trusted":true},"cell_type":"code","source":"# Train the model\nimport time\n\nstart = time.time()\n\n# Compile the model with TPU Strategy\nwith strategy.scope():\n    model = get_training_model()\n    \nmodel.fit(train_ds, \n          steps_per_epoch=train_data.shape[0] // BATCH_SIZE,\n          validation_data=valid_ds,\n          validation_steps=val_data.shape[0] // BATCH_SIZE,\n          epochs=EPOCHS,\n          class_weight=class_weights,\n          callbacks=[WandbCallback(), TextLogger()],\n          verbose=1)\nend = time.time() - start\nprint(\"Time taken \",end)\nwandb.log({\"training_time\":end})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**As I am logging some demo predictions in between this training time should not be used for any benchmarks. **\n\nLet's try a CNN (with 1D convolutions) now. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create utility function to get a training ready model on demand\ndef get_training_model_cnn():\n    inp_id = tf.keras.layers.Input(shape=(MAX_SEQ_LENGTH,), dtype=tf.int64, name=\"bert_input_ids\")\n    inp_mask = tf.keras.layers.Input(shape=(MAX_SEQ_LENGTH,), dtype=tf.int64, name=\"bert_input_masks\")\n    inputs = [inp_id, inp_mask]\n\n    hidden_state = transformers.TFDistilBertModel.from_pretrained('distilbert-base-multilingual-cased')(inputs)[0]\n    pooled_output = hidden_state[:, 0]    \n    reshaped_pooled = tf.keras.layers.Reshape((768,1), input_shape=(768,))(pooled_output)\n    conv_1 = tf.keras.layers.Conv1D(64, 2, activation='relu')(reshaped_pooled)\n    pooled_2 = tf.keras.layers.GlobalAveragePooling1D()(conv_1)\n    dense_1 = tf.keras.layers.Dense(128, activation='relu')(pooled_2)\n    output = tf.keras.layers.Dense(1, activation='sigmoid')(dense_1)\n\n    model = tf.keras.Model(inputs=inputs, outputs=output)\n    model.compile(optimizer=tf.optimizers.Adam(learning_rate=2e-5, \n                                            epsilon=1e-08), \n                loss='binary_crossentropy', metrics=['accuracy'])\n\n    return model","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Garbage collection\ngc.collect()\n\n# Reinitialize wandb\nwandb.init(project=\"jigsaw-toxic\", id=\"distilbert-tpu-kaggle-weighted-cnn\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Train the CNN-based model\nstart = time.time()\n\n# Compile the model with TPU Strategy\nwith strategy.scope():\n    model = get_training_model_cnn()\n    \nmodel.fit(train_ds, \n          steps_per_epoch=train_data.shape[0] // BATCH_SIZE,\n          validation_data=valid_ds,\n          validation_steps=val_data.shape[0] // BATCH_SIZE,\n          epochs=EPOCHS,\n          class_weight=class_weights,\n          callbacks=[WandbCallback(), TextLogger()],\n          verbose=1)\nend = time.time() - start\nprint(\"Time taken \",end)\nwandb.log({\"training_time\":end})","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The model generalizes better. "}],"metadata":{"colab":{"name":"Jigsaw Multilingual Toxic Comment Classification.ipynb","provenance":[],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU","widgets":{"application/vnd.jupyter.widget-state+json":{"fbfda2c54a4e4c39a20b308ec4c1dc41":{"model_module":"@jupyter-widgets/controls","model_name":"HBoxModel","state":{"_view_name":"HBoxView","_dom_classes":[],"_model_name":"HBoxModel","_view_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_view_count":null,"_view_module_version":"1.5.0","box_style":"","layout":"IPY_MODEL_da4b537602124ec09c5626b24f32f276","_model_module":"@jupyter-widgets/controls","children":["IPY_MODEL_9720b4c309104fe8affaf46308cac913","IPY_MODEL_401bf9db41354d69a7c795335d9735ef"]}},"da4b537602124ec09c5626b24f32f276":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","state":{"_view_name":"LayoutView","grid_template_rows":null,"right":null,"justify_content":null,"_view_module":"@jupyter-widgets/base","overflow":null,"_model_module_version":"1.2.0","_view_count":null,"flex_flow":null,"width":null,"min_width":null,"border":null,"align_items":null,"bottom":null,"_model_module":"@jupyter-widgets/base","top":null,"grid_column":null,"overflow_y":null,"overflow_x":null,"grid_auto_flow":null,"grid_area":null,"grid_template_columns":null,"flex":null,"_model_name":"LayoutModel","justify_items":null,"grid_row":null,"max_height":null,"align_content":null,"visibility":null,"align_self":null,"height":null,"min_height":null,"padding":null,"grid_auto_rows":null,"grid_gap":null,"max_width":null,"order":null,"_view_module_version":"1.2.0","grid_template_areas":null,"object_position":null,"object_fit":null,"grid_auto_columns":null,"margin":null,"display":null,"left":null}},"9720b4c309104fe8affaf46308cac913":{"model_module":"@jupyter-widgets/controls","model_name":"IntProgressModel","state":{"_view_name":"ProgressView","style":"IPY_MODEL_f75d44d8b8dd45c290d8480426b42cc3","_dom_classes":[],"description":"Downloading: 100%","_model_name":"IntProgressModel","bar_style":"success","max":995526,"_view_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","value":995526,"_view_count":null,"_view_module_version":"1.5.0","orientation":"horizontal","min":0,"description_tooltip":null,"_model_module":"@jupyter-widgets/controls","layout":"IPY_MODEL_949c793594f149d1b903da1e1a025998"}},"401bf9db41354d69a7c795335d9735ef":{"model_module":"@jupyter-widgets/controls","model_name":"HTMLModel","state":{"_view_name":"HTMLView","style":"IPY_MODEL_c79d073616014765b3a59e9b0babe27e","_dom_classes":[],"description":"","_model_name":"HTMLModel","placeholder":"​","_view_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","value":" 996k/996k [00:00&lt;00:00, 2.05MB/s]","_view_count":null,"_view_module_version":"1.5.0","description_tooltip":null,"_model_module":"@jupyter-widgets/controls","layout":"IPY_MODEL_63e7f4dd67e44fd2906c117522aa76cf"}},"f75d44d8b8dd45c290d8480426b42cc3":{"model_module":"@jupyter-widgets/controls","model_name":"ProgressStyleModel","state":{"_view_name":"StyleView","_model_name":"ProgressStyleModel","description_width":"initial","_view_module":"@jupyter-widgets/base","_model_module_version":"1.5.0","_view_count":null,"_view_module_version":"1.2.0","bar_color":null,"_model_module":"@jupyter-widgets/controls"}},"949c793594f149d1b903da1e1a025998":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","state":{"_view_name":"LayoutView","grid_template_rows":null,"right":null,"justify_content":null,"_view_module":"@jupyter-widgets/base","overflow":null,"_model_module_version":"1.2.0","_view_count":null,"flex_flow":null,"width":null,"min_width":null,"border":null,"align_items":null,"bottom":null,"_model_module":"@jupyter-widgets/base","top":null,"grid_column":null,"overflow_y":null,"overflow_x":null,"grid_auto_flow":null,"grid_area":null,"grid_template_columns":null,"flex":null,"_model_name":"LayoutModel","justify_items":null,"grid_row":null,"max_height":null,"align_content":null,"visibility":null,"align_self":null,"height":null,"min_height":null,"padding":null,"grid_auto_rows":null,"grid_gap":null,"max_width":null,"order":null,"_view_module_version":"1.2.0","grid_template_areas":null,"object_position":null,"object_fit":null,"grid_auto_columns":null,"margin":null,"display":null,"left":null}},"c79d073616014765b3a59e9b0babe27e":{"model_module":"@jupyter-widgets/controls","model_name":"DescriptionStyleModel","state":{"_view_name":"StyleView","_model_name":"DescriptionStyleModel","description_width":"","_view_module":"@jupyter-widgets/base","_model_module_version":"1.5.0","_view_count":null,"_view_module_version":"1.2.0","_model_module":"@jupyter-widgets/controls"}},"63e7f4dd67e44fd2906c117522aa76cf":{"model_module":"@jupyter-widgets/base","model_name":"LayoutModel","state":{"_view_name":"LayoutView","grid_template_rows":null,"right":null,"justify_content":null,"_view_module":"@jupyter-widgets/base","overflow":null,"_model_module_version":"1.2.0","_view_count":null,"flex_flow":null,"width":null,"min_width":null,"border":null,"align_items":null,"bottom":null,"_model_module":"@jupyter-widgets/base","top":null,"grid_column":null,"overflow_y":null,"overflow_x":null,"grid_auto_flow":null,"grid_area":null,"grid_template_columns":null,"flex":null,"_model_name":"LayoutModel","justify_items":null,"grid_row":null,"max_height":null,"align_content":null,"visibility":null,"align_self":null,"height":null,"min_height":null,"padding":null,"grid_auto_rows":null,"grid_gap":null,"max_width":null,"order":null,"_view_module_version":"1.2.0","grid_template_areas":null,"object_position":null,"object_fit":null,"grid_auto_columns":null,"margin":null,"display":null,"left":null}}}}},"nbformat":4,"nbformat_minor":4}