{
  "id": 121394,
  "title": "Key to success in this competition",
  "url": "/competitions/tensorflow2-question-answering/discussion/121394",
  "author_name": "",
  "post_date": "2019-12-13T01:11:57.395045Z",
  "votes": 4,
  "comment_count": 8,
  "views": 0,
  "content": "<p>A typical workflow for any competition for me includes:\n1. Exploratory Data Analysis and Data Visualization\n2. Feature Engineering and Data pre-processing\n3. Model selection and Model Architecture\n4. Model Training and Evaluation (inc. train/val/test split, hyper-parameter optimization, data augmentation, etc.)\n5. Post-processing outputs</p>\n\n<p>In this competition, at least for me:\n1. Exploratory Data Analysis and Data Visualization: Limited due to textual data\n2. Feature Engineering and Data pre-processing: Part of standard BERT flow\n3. Model selection and Model Architecture: Using default BERT parameters\n4. Model Training and Evaluation (inc. train/val/test split, hyper-parameter optimization, data augmentation, etc.): Using pretrained BERT\n5. Post-processing outputs: Again limited scope</p>\n\n<p>Or am I missing out something very obvious? </p>\n\n<p>This is my first NLP competition on kaggle.</p>",
  "messages": [
    {
      "id": "693941",
      "postDate": "12/13/2019 01:11:57",
      "content": "<p>A typical workflow for any competition for me includes:\n1. Exploratory Data Analysis and Data Visualization\n2. Feature Engineering and Data pre-processing\n3. Model selection and Model Architecture\n4. Model Training and Evaluation (inc. train/val/test split, hyper-parameter optimization, data augmentation, etc.)\n5. Post-processing outputs</p>\n\n<p>In this competition, at least for me:\n1. Exploratory Data Analysis and Data Visualization: Limited due to textual data\n2. Feature Engineering and Data pre-processing: Part of standard BERT flow\n3. Model selection and Model Architecture: Using default BERT parameters\n4. Model Training and Evaluation (inc. train/val/test split, hyper-parameter optimization, data augmentation, etc.): Using pretrained BERT\n5. Post-processing outputs: Again limited scope</p>\n\n<p>Or am I missing out something very obvious? </p>\n\n<p>This is my first NLP competition on kaggle.</p>",
      "rawMarkdown": "A typical workflow for any competition for me includes:\n1. Exploratory Data Analysis and Data Visualization\n2. Feature Engineering and Data pre-processing\n3. Model selection and Model Architecture\n4. Model Training and Evaluation (inc. train/val/test split, hyper-parameter optimization, data augmentation, etc.)\n5. Post-processing outputs\n\nIn this competition, at least for me:\n1. Exploratory Data Analysis and Data Visualization: Limited due to textual data\n2. Feature Engineering and Data pre-processing: Part of standard BERT flow\n3. Model selection and Model Architecture: Using default BERT parameters\n4. Model Training and Evaluation (inc. train/val/test split, hyper-parameter optimization, data augmentation, etc.): Using pretrained BERT\n5. Post-processing outputs: Again limited scope\n\nOr am I missing out something very obvious? \n\nThis is my first NLP competition on kaggle.",
      "votes": null
    },
    {
      "id": "693994",
      "postDate": "12/13/2019 02:47:41",
      "content": "<p>So-Inserting BERT into Steps 1-5? 😉 </p>",
      "rawMarkdown": "So-Inserting BERT into Steps 1-5? 😉",
      "votes": null
    },
    {
      "id": "694089",
      "postDate": "12/13/2019 06:42:23",
      "content": "<p>Some things you might want to work on:\n- target design\n- loss function design\n- how to split long documents into chunks which fit into model input size limit\n- sampling during training\n- how to do inference quickly and fit many different models within the time limit</p>",
      "rawMarkdown": "Some things you might want to work on:\n- target design\n- loss function design\n- how to split long documents into chunks which fit into model input size limit\n- sampling during training\n- how to do inference quickly and fit many different models within the time limit",
      "votes": null
    },
    {
      "id": "694122",
      "postDate": "12/13/2019 08:08:35",
      "content": "<p>Thanks a ton <a href=\"/lopuhin\">@lopuhin</a> , I'll look into these</p>",
      "rawMarkdown": "Thanks a ton @lopuhin , I'll look into these",
      "votes": null
    },
    {
      "id": "694123",
      "postDate": "12/13/2019 08:09:10",
      "content": "<p>It's the SOTA, after-all.</p>\n\n<p>What does your workflow look like in this competition <a href=\"/init27\">@init27</a> ?</p>",
      "rawMarkdown": "It's the SOTA, after-all.\n\nWhat does your workflow look like in this competition @init27 ?",
      "votes": null
    },
    {
      "id": "694231",
      "postDate": "12/13/2019 10:21:45",
      "content": "<p>I'm using Linear Regression 🤓 </p>",
      "rawMarkdown": "I'm using Linear Regression 🤓",
      "votes": null
    },
    {
      "id": "696072",
      "postDate": "12/16/2019 04:55:15",
      "content": "<p><a href=\"/lopuhin\">@lopuhin</a> I have a question about the third row, \"how to split long documents into chunks which fit into model input size limit\". Isn't is just 512 tokens with 128 stride following google's official bert-joint? Have u tried other cases?</p>",
      "rawMarkdown": "lopuhin I have a question about the third row, \"how to split long documents into chunks which fit into model input size limit\". Isn't is just 512 tokens with 128 stride following google's official bert-joint? Have u tried other cases?",
      "votes": null
    },
    {
      "id": "696096",
      "postDate": "12/16/2019 05:38:40",
      "content": "<blockquote>\n  <p>Isn't is just 512 tokens with 128 stride following google's official bert-joint?</p>\n</blockquote>\n\n<p>I'm feeding top-level long answer candidates separately.</p>",
      "rawMarkdown": "&gt; Isn't is just 512 tokens with 128 stride following google's official bert-joint?\n\nI'm feeding top-level long answer candidates separately.",
      "votes": null
    },
    {
      "id": "696297",
      "postDate": "12/16/2019 12:20:52",
      "content": "<blockquote>\n  <p>Isn't is just 512 tokens with 128 stride following google's official bert-joint?</p>\n</blockquote>\n\n<p>If all you do is to reproduce exactly what others have done then indeed there isn't much room for improvement.</p>",
      "rawMarkdown": "&gt;  Isn't is just 512 tokens with 128 stride following google's official bert-joint?\n\nIf all you do is to reproduce exactly what others have done then indeed there isn't much room for improvement.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 693994,
      "author_name": "init27",
      "author_url": "",
      "post_date": "12/13/2019 02:47:41",
      "content": "<p>So-Inserting BERT into Steps 1-5? 😉 </p>",
      "votes": null,
      "replies": [
        {
          "id": 694123,
          "author_name": "rohitagarwal",
          "author_url": "",
          "post_date": "12/13/2019 08:09:10",
          "content": "<p>It's the SOTA, after-all.</p>\n\n<p>What does your workflow look like in this competition <a href=\"/init27\">@init27</a> ?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 694231,
          "author_name": "init27",
          "author_url": "",
          "post_date": "12/13/2019 10:21:45",
          "content": "<p>I'm using Linear Regression 🤓 </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 694089,
      "author_name": "lopuhin",
      "author_url": "",
      "post_date": "12/13/2019 06:42:23",
      "content": "<p>Some things you might want to work on:\n- target design\n- loss function design\n- how to split long documents into chunks which fit into model input size limit\n- sampling during training\n- how to do inference quickly and fit many different models within the time limit</p>",
      "votes": null,
      "replies": [
        {
          "id": 694122,
          "author_name": "rohitagarwal",
          "author_url": "",
          "post_date": "12/13/2019 08:08:35",
          "content": "<p>Thanks a ton <a href=\"/lopuhin\">@lopuhin</a> , I'll look into these</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 696072,
          "author_name": "seriousran",
          "author_url": "",
          "post_date": "12/16/2019 04:55:15",
          "content": "<p><a href=\"/lopuhin\">@lopuhin</a> I have a question about the third row, \"how to split long documents into chunks which fit into model input size limit\". Isn't is just 512 tokens with 128 stride following google's official bert-joint? Have u tried other cases?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 696096,
          "author_name": "lopuhin",
          "author_url": "",
          "post_date": "12/16/2019 05:38:40",
          "content": "<blockquote>\n  <p>Isn't is just 512 tokens with 128 stride following google's official bert-joint?</p>\n</blockquote>\n\n<p>I'm feeding top-level long answer candidates separately.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 696297,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "12/16/2019 12:20:52",
          "content": "<blockquote>\n  <p>Isn't is just 512 tokens with 128 stride following google's official bert-joint?</p>\n</blockquote>\n\n<p>If all you do is to reproduce exactly what others have done then indeed there isn't much room for improvement.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "693941": "A typical workflow for any competition for me includes:\n1. Exploratory Data Analysis and Data Visualization\n2. Feature Engineering and Data pre-processing\n3. Model selection and Model Architecture\n4. Model Training and Evaluation (inc. train/val/test split, hyper-parameter optimization, data augmentation, etc.)\n5. Post-processing outputs\n\nIn this competition, at least for me:\n1. Exploratory Data Analysis and Data Visualization: Limited due to textual data\n2. Feature Engineering and Data pre-processing: Part of standard BERT flow\n3. Model selection and Model Architecture: Using default BERT parameters\n4. Model Training and Evaluation (inc. train/val/test split, hyper-parameter optimization, data augmentation, etc.): Using pretrained BERT\n5. Post-processing outputs: Again limited scope\n\nOr am I missing out something very obvious? \n\nThis is my first NLP competition on kaggle.",
    "693994": "So-Inserting BERT into Steps 1-5? 😉",
    "694089": "Some things you might want to work on:\n- target design\n- loss function design\n- how to split long documents into chunks which fit into model input size limit\n- sampling during training\n- how to do inference quickly and fit many different models within the time limit",
    "694122": "Thanks a ton @lopuhin , I'll look into these",
    "694123": "It's the SOTA, after-all.\n\nWhat does your workflow look like in this competition @init27 ?",
    "694231": "I'm using Linear Regression 🤓",
    "696072": "lopuhin I have a question about the third row, \"how to split long documents into chunks which fit into model input size limit\". Isn't is just 512 tokens with 128 stride following google's official bert-joint? Have u tried other cases?",
    "696096": "&gt; Isn't is just 512 tokens with 128 stride following google's official bert-joint?\n\nI'm feeding top-level long answer candidates separately.",
    "696297": "&gt;  Isn't is just 512 tokens with 128 stride following google's official bert-joint?\n\nIf all you do is to reproduce exactly what others have done then indeed there isn't much room for improvement."
  },
  "source": "meta"
}