{
  "id": 195527,
  "title": "New competition for Transformer?",
  "url": "/competitions/riiid-test-answer-prediction/discussion/195527",
  "author_name": "",
  "post_date": "2020-11-05T23:42:33.472918600Z",
  "votes": 17,
  "comment_count": 3,
  "views": 0,
  "content": "<p>mamas is saying \"Is Attention All You Need?\"  and organizer paper also used transformer.</p>",
  "messages": [
    {
      "id": "1070592",
      "postDate": "11/05/2020 23:42:33",
      "content": "<p>mamas is saying \"Is Attention All You Need?\"  and organizer paper also used transformer.</p>",
      "rawMarkdown": "mamas is saying \"Is Attention All You Need?\"  and organizer paper also used transformer.",
      "votes": null
    },
    {
      "id": "1071321",
      "postDate": "11/06/2020 18:18:52",
      "content": "<p>It looks like it is. Many KT models implementations <a href=\"https://paperswithcode.com/task/knowledge-tracing\" target=\"_blank\">here</a>, including some with Transformer but no SAINT implementation (the ones used by organizers). I'm trying to implement SAINT from scratch with PyTorch but the paper is not so detailed.</p>",
      "rawMarkdown": "It looks like it is. Many KT models implementations [here](https://paperswithcode.com/task/knowledge-tracing), including some with Transformer but no SAINT implementation (the ones used by organizers). I'm trying to implement SAINT from scratch with PyTorch but the paper is not so detailed.",
      "votes": null
    },
    {
      "id": "1071365",
      "postDate": "11/06/2020 19:21:38",
      "content": "<p>Please remember I have never told I was using Transformer. (I love LightGBM and CatBoost) </p>",
      "rawMarkdown": "Please remember I have never told I was using Transformer. (I love LightGBM and CatBoost)",
      "votes": null
    },
    {
      "id": "1071455",
      "postDate": "11/06/2020 22:50:01",
      "content": "<p>I kind of read that team name as suggesting that mamas might not be using a transformer at all, but who knows! Abhimanyu (2nd place) has talked about using a transformer though, so I think it's safe to assume it works really well. My best guess would be that the top positions are actually a mix of transformers and gradient boosting models (there's plenty of solid traditional feature engineering to be done here), but it's possible I'm off base and it's transformers all the way down. </p>",
      "rawMarkdown": "I kind of read that team name as suggesting that mamas might not be using a transformer at all, but who knows! Abhimanyu (2nd place) has talked about using a transformer though, so I think it's safe to assume it works really well. My best guess would be that the top positions are actually a mix of transformers and gradient boosting models (there's plenty of solid traditional feature engineering to be done here), but it's possible I'm off base and it's transformers all the way down.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1071321,
      "author_name": "mpware",
      "author_url": "",
      "post_date": "11/06/2020 18:18:52",
      "content": "<p>It looks like it is. Many KT models implementations <a href=\"https://paperswithcode.com/task/knowledge-tracing\" target=\"_blank\">here</a>, including some with Transformer but no SAINT implementation (the ones used by organizers). I'm trying to implement SAINT from scratch with PyTorch but the paper is not so detailed.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1071365,
      "author_name": "mamasinkgs",
      "author_url": "",
      "post_date": "11/06/2020 19:21:38",
      "content": "<p>Please remember I have never told I was using Transformer. (I love LightGBM and CatBoost) </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1071455,
      "author_name": "aquatic",
      "author_url": "",
      "post_date": "11/06/2020 22:50:01",
      "content": "<p>I kind of read that team name as suggesting that mamas might not be using a transformer at all, but who knows! Abhimanyu (2nd place) has talked about using a transformer though, so I think it's safe to assume it works really well. My best guess would be that the top positions are actually a mix of transformers and gradient boosting models (there's plenty of solid traditional feature engineering to be done here), but it's possible I'm off base and it's transformers all the way down. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1070592": "mamas is saying \"Is Attention All You Need?\"  and organizer paper also used transformer.",
    "1071321": "It looks like it is. Many KT models implementations [here](https://paperswithcode.com/task/knowledge-tracing), including some with Transformer but no SAINT implementation (the ones used by organizers). I'm trying to implement SAINT from scratch with PyTorch but the paper is not so detailed.",
    "1071365": "Please remember I have never told I was using Transformer. (I love LightGBM and CatBoost)",
    "1071455": "I kind of read that team name as suggesting that mamas might not be using a transformer at all, but who knows! Abhimanyu (2nd place) has talked about using a transformer though, so I think it's safe to assume it works really well. My best guess would be that the top positions are actually a mix of transformers and gradient boosting models (there's plenty of solid traditional feature engineering to be done here), but it's possible I'm off base and it's transformers all the way down."
  },
  "source": "meta"
}