{
  "id": 416190,
  "title": "Execution Time Fluctuations vs Efficiency Leaderboard",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/416190",
  "author_name": "Oleksiy Kononenko",
  "post_date": "2023-06-10T04:37:07.165000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I’ve been playing with a simple logistic model, that <a href=\"https://www.kaggle.com/code/philculliton/student-performance-efficiency-leaderboard/notebook\" target=\"_blank\">looks promising</a> from the efficiency point of view (if interested, my notebook is available <a href=\"https://www.kaggle.com/code/kononenko/datatable-linearmodel-0-676lb-in-6-seconds\" target=\"_blank\">here</a>), and found that execution time may change dramatically from run to run. From 10-20% up to 100% for exactly the same code.</p>\n<p>Though it is not critical for lightweight models, I imagine it could be an issue for more complex ones. Also, I am not sure how the Efficiency Leaderboard is functioning, when execution time fluctuates so much.</p>\n<p>Anyone else experiencing similar issues? What’s your overall impression on stability of the computational environment?</p>",
  "messages": [
    {
      "id": 2294433,
      "postDate": "2023-06-10T04:37:07.167Z",
      "content": "<p>I’ve been playing with a simple logistic model, that <a href=\"https://www.kaggle.com/code/philculliton/student-performance-efficiency-leaderboard/notebook\" target=\"_blank\">looks promising</a> from the efficiency point of view (if interested, my notebook is available <a href=\"https://www.kaggle.com/code/kononenko/datatable-linearmodel-0-676lb-in-6-seconds\" target=\"_blank\">here</a>), and found that execution time may change dramatically from run to run. From 10-20% up to 100% for exactly the same code.</p>\n<p>Though it is not critical for lightweight models, I imagine it could be an issue for more complex ones. Also, I am not sure how the Efficiency Leaderboard is functioning, when execution time fluctuates so much.</p>\n<p>Anyone else experiencing similar issues? What’s your overall impression on stability of the computational environment?</p>",
      "rawMarkdown": "I’ve been playing with a simple logistic model, that [looks promising](https://www.kaggle.com/code/philculliton/student-performance-efficiency-leaderboard/notebook) from the efficiency point of view (if interested, my notebook is available [here](https://www.kaggle.com/code/kononenko/datatable-linearmodel-0-676lb-in-6-seconds)), and found that execution time may change dramatically from run to run. From 10-20% up to 100% for exactly the same code.\n\nThough it is not critical for lightweight models, I imagine it could be an issue for more complex ones. Also, I am not sure how the Efficiency Leaderboard is functioning, when execution time fluctuates so much.\n\nAnyone else experiencing similar issues? What’s your overall impression on stability of the computational environment?"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2294433": "I’ve been playing with a simple logistic model, that [looks promising](https://www.kaggle.com/code/philculliton/student-performance-efficiency-leaderboard/notebook) from the efficiency point of view (if interested, my notebook is available [here](https://www.kaggle.com/code/kononenko/datatable-linearmodel-0-676lb-in-6-seconds)), and found that execution time may change dramatically from run to run. From 10-20% up to 100% for exactly the same code.\n\nThough it is not critical for lightweight models, I imagine it could be an issue for more complex ones. Also, I am not sure how the Efficiency Leaderboard is functioning, when execution time fluctuates so much.\n\nAnyone else experiencing similar issues? What’s your overall impression on stability of the computational environment?"
  }
}