{
  "id": 455173,
  "title": "How to boost the prediction ?",
  "url": "/competitions/stanford-ribonanza-rna-folding/discussion/455173",
  "author_name": "",
  "post_date": "2023-11-13T15:54:34.405158300Z",
  "votes": 2,
  "comment_count": 12,
  "views": 0,
  "content": "<p>I need 2 days to do my prediction. I need more optimisation and computational power ?</p>",
  "messages": [
    {
      "id": "2523636",
      "postDate": "11/13/2023 15:54:34",
      "content": "<p>I need 2 days to do my prediction. I need more optimisation and computational power ?</p>",
      "rawMarkdown": "I need 2 days to do my prediction. I need more optimisation and computational power ?",
      "votes": null
    },
    {
      "id": "2523820",
      "postDate": "11/13/2023 19:24:05",
      "content": "<p>Without seeing the code I guess is more about the algorithm itself. I mean, for example are you predicting in batches or sequence to sequence?<br>\n1<br>\nfor sequence in test:<br>\n     prectictions += [model(sequence)]<br>\n2<br>\nBATCH_SIZE = 1024<br>\nfor batch in test_batches:<br>\n     prectictions += [model(batch)]</p>",
      "rawMarkdown": "Without seeing the code I guess is more about the algorithm itself. I mean, for example are you predicting in batches or sequence to sequence?\n1\nfor sequence in test:\n     prectictions += [model(sequence)]\n2\nBATCH_SIZE = 1024\nfor batch in test_batches:\n     prectictions += [model(batch)]",
      "votes": null
    },
    {
      "id": "2523864",
      "postDate": "11/13/2023 20:22:14",
      "content": "<p>I predict sequence after sequence of 1823484 line in test_sequences.csv.<br>\nI have 2 models one for DMS and one for 2a3.</p>",
      "rawMarkdown": "I predict sequence after sequence of 1823484 line in test_sequences.csv.\nI have 2 models one for DMS and one for 2a3.",
      "votes": null
    },
    {
      "id": "2523870",
      "postDate": "11/13/2023 20:28:27",
      "content": "<p>Even with a small Neural networks it's 2 days for the prediction. A small NN like 457x256x457 is a lot of computation for the prediction. </p>",
      "rawMarkdown": "Even with a small Neural networks it's 2 days for the prediction. A small NN like 457x256x457 is a lot of computation for the prediction.",
      "votes": null
    },
    {
      "id": "2523878",
      "postDate": "11/13/2023 20:35:55",
      "content": "<p>That's it. Load test in batches as larger you can and predict all the batch at once esch step.</p>",
      "rawMarkdown": "That's it. Load test in batches as larger you can and predict all the batch at once esch step.",
      "votes": null
    },
    {
      "id": "2524022",
      "postDate": "11/13/2023 23:47:12",
      "content": "<p>for submitting , you can choose the submission.csv file in your notebook</p>",
      "rawMarkdown": "for submitting , you can choose the submission.csv file in your notebook",
      "votes": null
    },
    {
      "id": "2524057",
      "postDate": "11/14/2023 00:45:52",
      "content": "<p>Yes, but will take a while to upload. Is a heavy file.</p>",
      "rawMarkdown": "Yes, but will take a while to upload. Is a heavy file.",
      "votes": null
    },
    {
      "id": "2524063",
      "postDate": "11/14/2023 00:57:02",
      "content": "<p>i predict 7giga bytes. It's in creation now.</p>",
      "rawMarkdown": "i predict 7giga bytes. It's in creation now.",
      "votes": null
    },
    {
      "id": "2524449",
      "postDate": "11/14/2023 08:35:36",
      "content": "<p>Is i tpossible to let run a notebook alone without be connected</p>",
      "rawMarkdown": "Is i tpossible to let run a notebook alone without be connected",
      "votes": null
    },
    {
      "id": "2524478",
      "postDate": "11/14/2023 09:15:51",
      "content": "<p>I'm not sure. But I think not unless is a submission noteboock.</p>",
      "rawMarkdown": "I'm not sure. But I think not unless is a submission noteboock.",
      "votes": null
    },
    {
      "id": "2524487",
      "postDate": "11/14/2023 09:26:56",
      "content": "<p>What is the size of your submission.csv ?</p>",
      "rawMarkdown": "What is the size of your submission.csv ?",
      "votes": null
    },
    {
      "id": "2529202",
      "postDate": "11/18/2023 03:33:36",
      "content": "<p>Has anyone tried dask? I wonder if you could use that and cloud</p>",
      "rawMarkdown": "Has anyone tried dask? I wonder if you could use that and cloud",
      "votes": null
    },
    {
      "id": "2529488",
      "postDate": "11/18/2023 09:48:40",
      "content": "<p>I do now:<br>\n<code>predictions= model.predict(current_sequences, batch_size=1000, verbose=0)</code><br>\nIt's 20minutes to 40 now !!!!!</p>",
      "rawMarkdown": "I do now:\n`predictions= model.predict(current_sequences, batch_size=1000, verbose=0)`\nIt's 20minutes to 40 now !!!!!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2523820,
      "author_name": "sacuscreed",
      "author_url": "",
      "post_date": "11/13/2023 19:24:05",
      "content": "<p>Without seeing the code I guess is more about the algorithm itself. I mean, for example are you predicting in batches or sequence to sequence?<br>\n1<br>\nfor sequence in test:<br>\n     prectictions += [model(sequence)]<br>\n2<br>\nBATCH_SIZE = 1024<br>\nfor batch in test_batches:<br>\n     prectictions += [model(batch)]</p>",
      "votes": null,
      "replies": [
        {
          "id": 2523864,
          "author_name": "rgismeyssonnier",
          "author_url": "",
          "post_date": "11/13/2023 20:22:14",
          "content": "<p>I predict sequence after sequence of 1823484 line in test_sequences.csv.<br>\nI have 2 models one for DMS and one for 2a3.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2523870,
              "author_name": "rgismeyssonnier",
              "author_url": "",
              "post_date": "11/13/2023 20:28:27",
              "content": "<p>Even with a small Neural networks it's 2 days for the prediction. A small NN like 457x256x457 is a lot of computation for the prediction. </p>",
              "votes": null,
              "replies": [
                {
                  "id": 2529488,
                  "author_name": "rgismeyssonnier",
                  "author_url": "",
                  "post_date": "11/18/2023 09:48:40",
                  "content": "<p>I do now:<br>\n<code>predictions= model.predict(current_sequences, batch_size=1000, verbose=0)</code><br>\nIt's 20minutes to 40 now !!!!!</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            },
            {
              "id": 2523878,
              "author_name": "sacuscreed",
              "author_url": "",
              "post_date": "11/13/2023 20:35:55",
              "content": "<p>That's it. Load test in batches as larger you can and predict all the batch at once esch step.</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2524022,
                  "author_name": "rgismeyssonnier",
                  "author_url": "",
                  "post_date": "11/13/2023 23:47:12",
                  "content": "<p>for submitting , you can choose the submission.csv file in your notebook</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2524057,
                      "author_name": "sacuscreed",
                      "author_url": "",
                      "post_date": "11/14/2023 00:45:52",
                      "content": "<p>Yes, but will take a while to upload. Is a heavy file.</p>",
                      "votes": null,
                      "replies": [
                        {
                          "id": 2524063,
                          "author_name": "rgismeyssonnier",
                          "author_url": "",
                          "post_date": "11/14/2023 00:57:02",
                          "content": "<p>i predict 7giga bytes. It's in creation now.</p>",
                          "votes": null,
                          "replies": [
                            {
                              "id": 2524449,
                              "author_name": "rgismeyssonnier",
                              "author_url": "",
                              "post_date": "11/14/2023 08:35:36",
                              "content": "<p>Is i tpossible to let run a notebook alone without be connected</p>",
                              "votes": null,
                              "replies": [
                                {
                                  "id": 2524478,
                                  "author_name": "sacuscreed",
                                  "author_url": "",
                                  "post_date": "11/14/2023 09:15:51",
                                  "content": "<p>I'm not sure. But I think not unless is a submission noteboock.</p>",
                                  "votes": null,
                                  "replies": [
                                    {
                                      "id": 2524487,
                                      "author_name": "rgismeyssonnier",
                                      "author_url": "",
                                      "post_date": "11/14/2023 09:26:56",
                                      "content": "<p>What is the size of your submission.csv ?</p>",
                                      "votes": null,
                                      "replies": []
                                    }
                                  ]
                                }
                              ]
                            }
                          ]
                        }
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2529202,
      "author_name": "tuttlen",
      "author_url": "",
      "post_date": "11/18/2023 03:33:36",
      "content": "<p>Has anyone tried dask? I wonder if you could use that and cloud</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2523636": "I need 2 days to do my prediction. I need more optimisation and computational power ?",
    "2523820": "Without seeing the code I guess is more about the algorithm itself. I mean, for example are you predicting in batches or sequence to sequence?\n1\nfor sequence in test:\n     prectictions += [model(sequence)]\n2\nBATCH_SIZE = 1024\nfor batch in test_batches:\n     prectictions += [model(batch)]",
    "2523864": "I predict sequence after sequence of 1823484 line in test_sequences.csv.\nI have 2 models one for DMS and one for 2a3.",
    "2523870": "Even with a small Neural networks it's 2 days for the prediction. A small NN like 457x256x457 is a lot of computation for the prediction.",
    "2523878": "That's it. Load test in batches as larger you can and predict all the batch at once esch step.",
    "2524022": "for submitting , you can choose the submission.csv file in your notebook",
    "2524057": "Yes, but will take a while to upload. Is a heavy file.",
    "2524063": "i predict 7giga bytes. It's in creation now.",
    "2524449": "Is i tpossible to let run a notebook alone without be connected",
    "2524478": "I'm not sure. But I think not unless is a submission noteboock.",
    "2524487": "What is the size of your submission.csv ?",
    "2529202": "Has anyone tried dask? I wonder if you could use that and cloud",
    "2529488": "I do now:\n`predictions= model.predict(current_sequences, batch_size=1000, verbose=0)`\nIt's 20minutes to 40 now !!!!!"
  },
  "source": "meta"
}