{
  "id": 411246,
  "title": "Persistent problems with making valid submissions",
  "url": "/competitions/asl-fingerspelling/discussion/411246",
  "author_name": "",
  "post_date": "2023-05-18T10:18:38.972870200Z",
  "votes": 11,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Dear Organisers ( <a href=\"https://www.kaggle.com/thadstarner\" target=\"_blank\">@thadstarner</a> , <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> ),</p>\n<p>We are very enthousiastic about this new competition. And I assume you want us to contribute to the progress in the field and focus on models and data problems.</p>\n<p>However, more than a week after the start of the competition, no-one has managed to make a successful submission, not even high-scoring participants from the previous competition (who are familiar with the data format and making TfLite submissions), and not even with a dummy model that makes random or constant predictions.</p>\n<p>This is very frustrating and a waste of time. Also, clearly, this has nothing to do with the time and memory constraints that are part of the challenge in this competition, but rather with a lack of information with respect to the assumptions that are made by the scoring code. For this reason I would like to insist that the organisers share code that can make a dummy submission (e.g. constant or random predictions) and clarify the requirements to the model output format in order to make a successful submission.</p>",
  "messages": [
    {
      "id": "2264314",
      "postDate": "05/18/2023 10:18:38",
      "content": "<p>Dear Organisers ( <a href=\"https://www.kaggle.com/thadstarner\" target=\"_blank\">@thadstarner</a> , <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> ),</p>\n<p>We are very enthousiastic about this new competition. And I assume you want us to contribute to the progress in the field and focus on models and data problems.</p>\n<p>However, more than a week after the start of the competition, no-one has managed to make a successful submission, not even high-scoring participants from the previous competition (who are familiar with the data format and making TfLite submissions), and not even with a dummy model that makes random or constant predictions.</p>\n<p>This is very frustrating and a waste of time. Also, clearly, this has nothing to do with the time and memory constraints that are part of the challenge in this competition, but rather with a lack of information with respect to the assumptions that are made by the scoring code. For this reason I would like to insist that the organisers share code that can make a dummy submission (e.g. constant or random predictions) and clarify the requirements to the model output format in order to make a successful submission.</p>",
      "rawMarkdown": "Dear Organisers ( @thadstarner , @sohier ),\n\nWe are very enthousiastic about this new competition. And I assume you want us to contribute to the progress in the field and focus on models and data problems.\n\nHowever, more than a week after the start of the competition, no-one has managed to make a successful submission, not even high-scoring participants from the previous competition (who are familiar with the data format and making TfLite submissions), and not even with a dummy model that makes random or constant predictions.\n\nThis is very frustrating and a waste of time. Also, clearly, this has nothing to do with the time and memory constraints that are part of the challenge in this competition, but rather with a lack of information with respect to the assumptions that are made by the scoring code. For this reason I would like to insist that the organisers share code that can make a dummy submission (e.g. constant or random predictions) and clarify the requirements to the model output format in order to make a successful submission.",
      "votes": null
    },
    {
      "id": "2264630",
      "postDate": "05/18/2023 15:34:24",
      "content": "<p>last competition took as input a numpy array, seems from their eval description, this time the input is a dataframe?</p>\n<p>def load_relevant_data_subset(pq_path):<br>\n    return pd.read_parquet(pq_path, columns=selected_columns)</p>",
      "rawMarkdown": "last competition took as input a numpy array, seems from their eval description, this time the input is a dataframe?\n\ndef load_relevant_data_subset(pq_path):\n    return pd.read_parquet(pq_path, columns=selected_columns)",
      "votes": null
    },
    {
      "id": "2278769",
      "postDate": "05/29/2023 00:34:56",
      "content": "<p>Also struggling with this… Managed to make a model which works great using the provided evaluation code, but can't get it to work on submission.</p>",
      "rawMarkdown": "Also struggling with this... Managed to make a model which works great using the provided evaluation code, but can't get it to work on submission.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2264630,
      "author_name": "jonathanplante",
      "author_url": "",
      "post_date": "05/18/2023 15:34:24",
      "content": "<p>last competition took as input a numpy array, seems from their eval description, this time the input is a dataframe?</p>\n<p>def load_relevant_data_subset(pq_path):<br>\n    return pd.read_parquet(pq_path, columns=selected_columns)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2278769,
      "author_name": "anokas",
      "author_url": "",
      "post_date": "05/29/2023 00:34:56",
      "content": "<p>Also struggling with this… Managed to make a model which works great using the provided evaluation code, but can't get it to work on submission.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2264314": "Dear Organisers ( @thadstarner , @sohier ),\n\nWe are very enthousiastic about this new competition. And I assume you want us to contribute to the progress in the field and focus on models and data problems.\n\nHowever, more than a week after the start of the competition, no-one has managed to make a successful submission, not even high-scoring participants from the previous competition (who are familiar with the data format and making TfLite submissions), and not even with a dummy model that makes random or constant predictions.\n\nThis is very frustrating and a waste of time. Also, clearly, this has nothing to do with the time and memory constraints that are part of the challenge in this competition, but rather with a lack of information with respect to the assumptions that are made by the scoring code. For this reason I would like to insist that the organisers share code that can make a dummy submission (e.g. constant or random predictions) and clarify the requirements to the model output format in order to make a successful submission.",
    "2264630": "last competition took as input a numpy array, seems from their eval description, this time the input is a dataframe?\n\ndef load_relevant_data_subset(pq_path):\n    return pd.read_parquet(pq_path, columns=selected_columns)",
    "2278769": "Also struggling with this... Managed to make a model which works great using the provided evaluation code, but can't get it to work on submission."
  },
  "source": "meta"
}