{
  "id": 462838,
  "title": "403rd Place Solution for the Stanford Ribonanza RNA Folding Competition",
  "url": "/competitions/stanford-ribonanza-rna-folding/discussion/462838",
  "author_name": "",
  "post_date": "2023-12-21T22:15:38.308000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Thank you to the organizers and Kaggle for hosting such a great competition.<br>\nI value everyone's sharing at Kaggle. <br>\nThe public notebook provided by SWORDSMAN is amazing and I am thankful for it <a href=\"https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\" target=\"_blank\">https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739</a>. 感谢您创建笔记本.</p>\n<h1>Context</h1>\n<p>Business context: <a href=\"https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/overview\" target=\"_blank\">https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/overview</a><br>\nData context: <a href=\"https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/data\" target=\"_blank\">https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/data</a></p>\n<h1>Overview of the approach</h1>\n<p>The SN_filter is set to 1 the signal_to_noise&gt;1.0 and reads &gt; 100.<br>\nTwo files for training experiments with DMS_MaP and 2A3_MaP.<br>\nRepeat dataset 5 time.</p>\n<h1>Described the models or algorithms used</h1>\n<p>Tensorflow keras RNN with Attention.</p>\n<table>\n<thead>\n<tr>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.15803</td>\n<td>0.24428</td>\n</tr>\n</tbody>\n</table>\n<p>Attention 8 heads.<br>\n6 layers.</p>\n<p>GaussianNoise 0.01.<br>\nDense 400 relu.<br>\nGaussianNoise 0.01.<br>\nDense 40 relu.<br>\nDense 2.</p>\n<h4>RNN Model:</h4>\n<table>\n<thead>\n<tr>\n<th>Layer  type</th>\n<th>Output Shape</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>PositionalEmbedding</td>\n<td>990</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>Dropout</td>\n<td>0</td>\n</tr>\n<tr>\n<td>Dense</td>\n<td>79600</td>\n</tr>\n<tr>\n<td>Dense</td>\n<td>16040</td>\n</tr>\n<tr>\n<td>Dense</td>\n<td>82</td>\n</tr>\n<tr>\n<td>GaussianNoise</td>\n<td>0</td>\n</tr>\n<tr>\n<td>GaussianNoise</td>\n<td>0</td>\n</tr>\n</tbody>\n</table>\n<p>Params: 8,850,520</p>\n<h1>Described the data preprocessing, feature engineering, and/or feature selection strategy</h1>\n<p>Encoding sin and cos.<br>\nEmbedding 2048.</p>\n<h1>Details of the submission</h1>\n<p>What work:<br>\n  A Custom Loss Function hat take two arguments: target value and predicted value  Sum 0 / Sum 1,0.<br>\n  Epoch 0 Lr 0.0005. Epoch 1-60 Lr is calculated by multiplying Lr by 0.904.</p>\n<h1>Preventing overfitting  and Validation Strategy</h1>\n<p>KFold.</p>\n<p>Values outside the interval  0 1 are clipped.</p>\n<h1>Code samples</h1>\n<p>Source public notebook <a href=\"https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\" target=\"_blank\">https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739</a></p>\n<h4>Code samples prepare dataset</h4>\n<pre><code> n  ():\n    df = df_c.get_chunk()\n    df = df[df[] &gt; ]\n    df_DMS_MaP = df[df[] == ]\n    df_2A3_MaP = df[df[] == ]    \n    delete_list = []\n     k  df.columns:\n           k     k:\n            delete_list.append(k)\n    df_DMS_MaP = df_DMS_MaP.drop(delete_list, axis=)\n    df_2A3_MaP = df_2A3_MaP.drop(delete_list, axis=)\n     n ==  :\n        df_DMS_MaP.to_csv(+train_file, header=df_DMS_MaP.keys() ,index=)\n        df_2A3_MaP.to_csv(+train_file, header=df_2A3_MaP.keys() ,index=)\n    :\n        df_DMS_MaP.to_csv(+train_file, mode=, header=,index=)        \n        df_2A3_MaP.to_csv(+train_file, mode=, header=,index=)\n</code></pre>\n<h4>Code samples demonstrating model training</h4>\n<pre><code>     GlobalSelfAttention(\n        num_heads=num_heads,\n        key_dim=d_model,\n        dropout=dropout_rate)\n</code></pre>\n<pre><code>    x = self.pos_embedding(x)  \n    x = tf.keras.layers.Dropout(dropout_rate)(x)\n    \n        x = GlobalSelfAttention( num_heads=num_heads, key_dim=d_model, dropout=dropout_rate)(x)\n        x = FeedForward(d_model, dff)(x)    \n    x = tf.keras.layers.GaussianNoise(stddev=)(x)\n    x1 = tf.keras.layers.Dense(, activation=)(x)\n    x1 = tf.keras.layers.GaussianNoise(stddev=)(x1)\n    x1 = tf.keras.layers.Dense(,activation=)(x1)\n    o1 = tf.keras.layers.Dense()(x1) \n</code></pre>\n<h4>Code samples model inference</h4>\n<p>Predict two chemical modifiers DMS Y[:,1]  and 2A3 Y[:,0].</p>\n<pre><code>  Y = model.predict( np.array(test_df.loc[span*g_i:span*(g_i+)-,].to_list()).astype() )\n  np.clip(Y,,)\n</code></pre>\n<h1>Sources</h1>\n<p><a href=\"https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\" target=\"_blank\">https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739</a><br>\n<a href=\"https://www.kaggle.com/code/misakimatsutomo/stanford-rrf-tensorflow-tpu/edit/run/153311838\" target=\"_blank\">https://www.kaggle.com/code/misakimatsutomo/stanford-rrf-tensorflow-tpu/edit/run/153311838</a></p>\n<p>Thank you for taking the time to read the writeup.</p>",
  "messages": [
    {
      "id": 2570097,
      "postDate": "2023-12-21T22:15:38.310Z",
      "content": "<p>Thank you to the organizers and Kaggle for hosting such a great competition.<br>\nI value everyone's sharing at Kaggle. <br>\nThe public notebook provided by SWORDSMAN is amazing and I am thankful for it <a href=\"https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\" target=\"_blank\">https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739</a>. 感谢您创建笔记本.</p>\n<h1>Context</h1>\n<p>Business context: <a href=\"https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/overview\" target=\"_blank\">https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/overview</a><br>\nData context: <a href=\"https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/data\" target=\"_blank\">https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/data</a></p>\n<h1>Overview of the approach</h1>\n<p>The SN_filter is set to 1 the signal_to_noise&gt;1.0 and reads &gt; 100.<br>\nTwo files for training experiments with DMS_MaP and 2A3_MaP.<br>\nRepeat dataset 5 time.</p>\n<h1>Described the models or algorithms used</h1>\n<p>Tensorflow keras RNN with Attention.</p>\n<table>\n<thead>\n<tr>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>0.15803</td>\n<td>0.24428</td>\n</tr>\n</tbody>\n</table>\n<p>Attention 8 heads.<br>\n6 layers.</p>\n<p>GaussianNoise 0.01.<br>\nDense 400 relu.<br>\nGaussianNoise 0.01.<br>\nDense 40 relu.<br>\nDense 2.</p>\n<h4>RNN Model:</h4>\n<table>\n<thead>\n<tr>\n<th>Layer  type</th>\n<th>Output Shape</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>PositionalEmbedding</td>\n<td>990</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>EncoderLayer</td>\n<td>1458968</td>\n</tr>\n<tr>\n<td>Dropout</td>\n<td>0</td>\n</tr>\n<tr>\n<td>Dense</td>\n<td>79600</td>\n</tr>\n<tr>\n<td>Dense</td>\n<td>16040</td>\n</tr>\n<tr>\n<td>Dense</td>\n<td>82</td>\n</tr>\n<tr>\n<td>GaussianNoise</td>\n<td>0</td>\n</tr>\n<tr>\n<td>GaussianNoise</td>\n<td>0</td>\n</tr>\n</tbody>\n</table>\n<p>Params: 8,850,520</p>\n<h1>Described the data preprocessing, feature engineering, and/or feature selection strategy</h1>\n<p>Encoding sin and cos.<br>\nEmbedding 2048.</p>\n<h1>Details of the submission</h1>\n<p>What work:<br>\n  A Custom Loss Function hat take two arguments: target value and predicted value  Sum 0 / Sum 1,0.<br>\n  Epoch 0 Lr 0.0005. Epoch 1-60 Lr is calculated by multiplying Lr by 0.904.</p>\n<h1>Preventing overfitting  and Validation Strategy</h1>\n<p>KFold.</p>\n<p>Values outside the interval  0 1 are clipped.</p>\n<h1>Code samples</h1>\n<p>Source public notebook <a href=\"https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\" target=\"_blank\">https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739</a></p>\n<h4>Code samples prepare dataset</h4>\n<pre><code> n  ():\n    df = df_c.get_chunk()\n    df = df[df[] &gt; ]\n    df_DMS_MaP = df[df[] == ]\n    df_2A3_MaP = df[df[] == ]    \n    delete_list = []\n     k  df.columns:\n           k     k:\n            delete_list.append(k)\n    df_DMS_MaP = df_DMS_MaP.drop(delete_list, axis=)\n    df_2A3_MaP = df_2A3_MaP.drop(delete_list, axis=)\n     n ==  :\n        df_DMS_MaP.to_csv(+train_file, header=df_DMS_MaP.keys() ,index=)\n        df_2A3_MaP.to_csv(+train_file, header=df_2A3_MaP.keys() ,index=)\n    :\n        df_DMS_MaP.to_csv(+train_file, mode=, header=,index=)        \n        df_2A3_MaP.to_csv(+train_file, mode=, header=,index=)\n</code></pre>\n<h4>Code samples demonstrating model training</h4>\n<pre><code>     GlobalSelfAttention(\n        num_heads=num_heads,\n        key_dim=d_model,\n        dropout=dropout_rate)\n</code></pre>\n<pre><code>    x = self.pos_embedding(x)  \n    x = tf.keras.layers.Dropout(dropout_rate)(x)\n    \n        x = GlobalSelfAttention( num_heads=num_heads, key_dim=d_model, dropout=dropout_rate)(x)\n        x = FeedForward(d_model, dff)(x)    \n    x = tf.keras.layers.GaussianNoise(stddev=)(x)\n    x1 = tf.keras.layers.Dense(, activation=)(x)\n    x1 = tf.keras.layers.GaussianNoise(stddev=)(x1)\n    x1 = tf.keras.layers.Dense(,activation=)(x1)\n    o1 = tf.keras.layers.Dense()(x1) \n</code></pre>\n<h4>Code samples model inference</h4>\n<p>Predict two chemical modifiers DMS Y[:,1]  and 2A3 Y[:,0].</p>\n<pre><code>  Y = model.predict( np.array(test_df.loc[span*g_i:span*(g_i+)-,].to_list()).astype() )\n  np.clip(Y,,)\n</code></pre>\n<h1>Sources</h1>\n<p><a href=\"https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\" target=\"_blank\">https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739</a><br>\n<a href=\"https://www.kaggle.com/code/misakimatsutomo/stanford-rrf-tensorflow-tpu/edit/run/153311838\" target=\"_blank\">https://www.kaggle.com/code/misakimatsutomo/stanford-rrf-tensorflow-tpu/edit/run/153311838</a></p>\n<p>Thank you for taking the time to read the writeup.</p>",
      "rawMarkdown": "Thank you to the organizers and Kaggle for hosting such a great competition.\nI value everyone's sharing at Kaggle. \nThe public notebook provided by SWORDSMAN is amazing and I am thankful for it https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739. 感谢您创建笔记本.\n\n# Context\nBusiness context: https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/overview\nData context: https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/data\n\n# Overview of the approach\nThe SN_filter is set to 1 the signal_to_noise>1.0 and reads > 100.\nTwo files for training experiments with DMS_MaP and 2A3_MaP.\nRepeat dataset 5 time.\n\n# Described the models or algorithms used\n\nTensorflow keras RNN with Attention.\n|Public LB|Private LB|  \n| --- | --- |  \n|0.15803|0.24428|\n\nAttention 8 heads.\n6 layers.\n\nGaussianNoise 0.01.\nDense 400 relu.\nGaussianNoise 0.01.\nDense 40 relu.\nDense 2.\n\n#### RNN Model:\n\n|Layer  type |                Output Shape |      \n| --- | --- |    \n|PositionalEmbedding|                  990  |                                                                \n|EncoderLayer |                 1458968    |                                                             \n|EncoderLayer|                  1458968    |                                                             \n|EncoderLayer |                 1458968   |                                                            \n|EncoderLayer|                  1458968   |\n|EncoderLayer|                  1458968   |\n|EncoderLayer |                 1458968  |                                                             \n|Dropout|                          0          |                                                        \n|Dense  |                           79600  |                                                              \n|Dense  |                           16040  |                                                           \n|Dense  |                         82      |                                                        \n|GaussianNoise |                   0   |                                                                  \n|GaussianNoise  |                  0      |   \n                                                     \nParams: 8,850,520\n \n# Described the data preprocessing, feature engineering, and/or feature selection strategy\n\nEncoding sin and cos.\nEmbedding 2048.\n\n# Details of the submission\nWhat work:\n  A Custom Loss Function hat take two arguments: target value and predicted value  Sum 0 / Sum 1,0.\n  Epoch 0 Lr 0.0005. Epoch 1-60 Lr is calculated by multiplying Lr by 0.904.\n\n# Preventing overfitting  and Validation Strategy\n\nKFold.\n\nValues outside the interval  0 1 are clipped.\n\n# Code samples\nSource public notebook https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\n\n#### Code samples prepare dataset\n\n```python\nfor n in range(4):\n    df = df_c.get_chunk()\n    df = df[df[\"SN_filter\"] >0 ]\n    df_DMS_MaP = df[df[\"experiment_type\"] == \"DMS_MaP\"]\n    df_2A3_MaP = df[df[\"experiment_type\"] == \"2A3_MaP\"]    \n    delete_list = []\n    for k in df.columns:\n        if 'reactivity' in k and 'error'  in k:\n            delete_list.append(k)\n    df_DMS_MaP = df_DMS_MaP.drop(delete_list, axis=1)\n    df_2A3_MaP = df_2A3_MaP.drop(delete_list, axis=1)\n    if n == 0 :\n        df_DMS_MaP.to_csv(\"DMS_MaP_\"+train_file, header=df_DMS_MaP.keys() ,index=False)\n        df_2A3_MaP.to_csv(\"2A3_MaP_\"+train_file, header=df_2A3_MaP.keys() ,index=False)\n    else:\n        df_DMS_MaP.to_csv(\"DMS_MaP_\"+train_file, mode='a', header=False,index=False)        \n        df_2A3_MaP.to_csv(\"2A3_MaP_\"+train_file, mode='a', header=False,index=False)\n```\n\n#### Code samples demonstrating model training \n ```python\n     GlobalSelfAttention(\n        num_heads=num_heads,\n        key_dim=d_model,\n        dropout=dropout_rate)\n```\n\n```python\n    x = self.pos_embedding(x)  \n    x = tf.keras.layers.Dropout(dropout_rate)(x)\n    # Repeat 6 times.\n        x = GlobalSelfAttention( num_heads=num_heads, key_dim=d_model, dropout=dropout_rate)(x)\n        x = FeedForward(d_model, dff)(x)    \n    x = tf.keras.layers.GaussianNoise(stddev=0.01)(x)\n    x1 = tf.keras.layers.Dense(400, activation='relu')(x)\n    x1 = tf.keras.layers.GaussianNoise(stddev=0.01)(x1)\n    x1 = tf.keras.layers.Dense(40,activation='relu')(x1)\n    o1 = tf.keras.layers.Dense(2)(x1) \n```\n\n#### Code samples model inference \nPredict two chemical modifiers DMS Y[:,1]  and 2A3 Y[:,0].\n```python\n  Y = model.predict( np.array(test_df.loc[span*g_i:span*(g_i+1)-1,\"sequence\"].to_list()).astype(int) )\n  np.clip(Y,0,1)\n```\n\n# Sources\nhttps://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\nhttps://www.kaggle.com/code/misakimatsutomo/stanford-rrf-tensorflow-tpu/edit/run/153311838\n\nThank you for taking the time to read the writeup.",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2570097": "Thank you to the organizers and Kaggle for hosting such a great competition.\nI value everyone's sharing at Kaggle. \nThe public notebook provided by SWORDSMAN is amazing and I am thankful for it https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739. 感谢您创建笔记本.\n\n# Context\nBusiness context: https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/overview\nData context: https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding/data\n\n# Overview of the approach\nThe SN_filter is set to 1 the signal_to_noise>1.0 and reads > 100.\nTwo files for training experiments with DMS_MaP and 2A3_MaP.\nRepeat dataset 5 time.\n\n# Described the models or algorithms used\n\nTensorflow keras RNN with Attention.\n|Public LB|Private LB|  \n| --- | --- |  \n|0.15803|0.24428|\n\nAttention 8 heads.\n6 layers.\n\nGaussianNoise 0.01.\nDense 400 relu.\nGaussianNoise 0.01.\nDense 40 relu.\nDense 2.\n\n#### RNN Model:\n\n|Layer  type |                Output Shape |      \n| --- | --- |    \n|PositionalEmbedding|                  990  |                                                                \n|EncoderLayer |                 1458968    |                                                             \n|EncoderLayer|                  1458968    |                                                             \n|EncoderLayer |                 1458968   |                                                            \n|EncoderLayer|                  1458968   |\n|EncoderLayer|                  1458968   |\n|EncoderLayer |                 1458968  |                                                             \n|Dropout|                          0          |                                                        \n|Dense  |                           79600  |                                                              \n|Dense  |                           16040  |                                                           \n|Dense  |                         82      |                                                        \n|GaussianNoise |                   0   |                                                                  \n|GaussianNoise  |                  0      |   \n                                                     \nParams: 8,850,520\n \n# Described the data preprocessing, feature engineering, and/or feature selection strategy\n\nEncoding sin and cos.\nEmbedding 2048.\n\n# Details of the submission\nWhat work:\n  A Custom Loss Function hat take two arguments: target value and predicted value  Sum 0 / Sum 1,0.\n  Epoch 0 Lr 0.0005. Epoch 1-60 Lr is calculated by multiplying Lr by 0.904.\n\n# Preventing overfitting  and Validation Strategy\n\nKFold.\n\nValues outside the interval  0 1 are clipped.\n\n# Code samples\nSource public notebook https://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\n\n#### Code samples prepare dataset\n\n```python\nfor n in range(4):\n    df = df_c.get_chunk()\n    df = df[df[\"SN_filter\"] >0 ]\n    df_DMS_MaP = df[df[\"experiment_type\"] == \"DMS_MaP\"]\n    df_2A3_MaP = df[df[\"experiment_type\"] == \"2A3_MaP\"]    \n    delete_list = []\n    for k in df.columns:\n        if 'reactivity' in k and 'error'  in k:\n            delete_list.append(k)\n    df_DMS_MaP = df_DMS_MaP.drop(delete_list, axis=1)\n    df_2A3_MaP = df_2A3_MaP.drop(delete_list, axis=1)\n    if n == 0 :\n        df_DMS_MaP.to_csv(\"DMS_MaP_\"+train_file, header=df_DMS_MaP.keys() ,index=False)\n        df_2A3_MaP.to_csv(\"2A3_MaP_\"+train_file, header=df_2A3_MaP.keys() ,index=False)\n    else:\n        df_DMS_MaP.to_csv(\"DMS_MaP_\"+train_file, mode='a', header=False,index=False)        \n        df_2A3_MaP.to_csv(\"2A3_MaP_\"+train_file, mode='a', header=False,index=False)\n```\n\n#### Code samples demonstrating model training \n ```python\n     GlobalSelfAttention(\n        num_heads=num_heads,\n        key_dim=d_model,\n        dropout=dropout_rate)\n```\n\n```python\n    x = self.pos_embedding(x)  \n    x = tf.keras.layers.Dropout(dropout_rate)(x)\n    # Repeat 6 times.\n        x = GlobalSelfAttention( num_heads=num_heads, key_dim=d_model, dropout=dropout_rate)(x)\n        x = FeedForward(d_model, dff)(x)    \n    x = tf.keras.layers.GaussianNoise(stddev=0.01)(x)\n    x1 = tf.keras.layers.Dense(400, activation='relu')(x)\n    x1 = tf.keras.layers.GaussianNoise(stddev=0.01)(x1)\n    x1 = tf.keras.layers.Dense(40,activation='relu')(x1)\n    o1 = tf.keras.layers.Dense(2)(x1) \n```\n\n#### Code samples model inference \nPredict two chemical modifiers DMS Y[:,1]  and 2A3 Y[:,0].\n```python\n  Y = model.predict( np.array(test_df.loc[span*g_i:span*(g_i+1)-1,\"sequence\"].to_list()).astype(int) )\n  np.clip(Y,0,1)\n```\n\n# Sources\nhttps://www.kaggle.com/code/liuyanfeng/stanford-rrf-tensorflow-tpu?scriptVersionId=150578739\nhttps://www.kaggle.com/code/misakimatsutomo/stanford-rrf-tensorflow-tpu/edit/run/153311838\n\nThank you for taking the time to read the writeup."
  }
}