Within the same graph, we sometimes have as many as 100040 configs. But sometimes, many of those configs have exactly the same value in node_config_feat, while the config_runtime is different.
\nIs this intentional? Does this imply that node_config_feat didn't capture all the differences among the configurations? I looked up the feature extraction code, but noticed that the \"message.runs\" structure is always empty.
", "messages": [ { "id": "2478148", "postDate": "10/11/2023 19:25:54", "content": "Within the same graph, we sometimes have as many as 100040 configs. But sometimes, many of those configs have exactly the same value in node_config_feat, while the config_runtime is different.
\nIs this intentional? Does this imply that node_config_feat didn't capture all the differences among the configurations? I looked up the feature extraction code, but noticed that the \"message.runs\" structure is always empty.
", "rawMarkdown": "Within the same graph, we sometimes have as many as 100040 configs. But sometimes, many of those configs have exactly the same value in node_config_feat, while the config_runtime is different.\n\nIs this intentional? Does this imply that node_config_feat didn't capture all the differences among the configurations? I looked up the feature extraction code, but noticed that the \"message.runs\" structure is always empty.", "votes": null }, { "id": "2479514", "postDate": "10/12/2023 16:57:47", "content": "Interesting finding. Related to my own question about the data collection process here: https://www.kaggle.com/competitions/predict-ai-model-runtime/discussion/438819 . The organizers claim the variation be less than 1%. Which files are you looking at? I could have a look to double check.
", "rawMarkdown": "Interesting finding. Related to my own question about the data collection process here: https://www.kaggle.com/competitions/predict-ai-model-runtime/discussion/438819 . The organizers claim the variation be less than 1%. Which files are you looking at? I could have a look to double check.", "votes": null }, { "id": "2479994", "postDate": "10/13/2023 04:04:51", "content": "I forgot to add that the same layout config may appear multiple times because we use random search (for random collection) and genetic search (for default collection) to generate layout configs. Therefore, the same config maybe explored and measured multiple times. The runtimes from different runs won't be exactly the same due to noises (like when you run the same program on your laptop multiple times). However, the runtime variation should be <%1.
", "rawMarkdown": "I forgot to add that the same layout config may appear multiple times because we use random search (for random collection) and genetic search (for default collection) to generate layout configs. Therefore, the same config maybe explored and measured multiple times. The runtimes from different runs won't be exactly the same due to noises (like when you run the same program on your laptop multiple times). However, the runtime variation should be <%1.", "votes": null } ], "comments": [ { "id": 2479514, "author_name": "carldehlin", "author_url": "", "post_date": "10/12/2023 16:57:47", "content": "Interesting finding. Related to my own question about the data collection process here: https://www.kaggle.com/competitions/predict-ai-model-runtime/discussion/438819 . The organizers claim the variation be less than 1%. Which files are you looking at? I could have a look to double check.
", "votes": null, "replies": [ { "id": 2479994, "author_name": "mangpophothilimthana", "author_url": "", "post_date": "10/13/2023 04:04:51", "content": "I forgot to add that the same layout config may appear multiple times because we use random search (for random collection) and genetic search (for default collection) to generate layout configs. Therefore, the same config maybe explored and measured multiple times. The runtimes from different runs won't be exactly the same due to noises (like when you run the same program on your laptop multiple times). However, the runtime variation should be <%1.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2478148": "Within the same graph, we sometimes have as many as 100040 configs. But sometimes, many of those configs have exactly the same value in node_config_feat, while the config_runtime is different.\n\nIs this intentional? Does this imply that node_config_feat didn't capture all the differences among the configurations? I looked up the feature extraction code, but noticed that the \"message.runs\" structure is always empty.", "2479514": "Interesting finding. Related to my own question about the data collection process here: https://www.kaggle.com/competitions/predict-ai-model-runtime/discussion/438819 . The organizers claim the variation be less than 1%. Which files are you looking at? I could have a look to double check.", "2479994": "I forgot to add that the same layout config may appear multiple times because we use random search (for random collection) and genetic search (for default collection) to generate layout configs. Therefore, the same config maybe explored and measured multiple times. The runtimes from different runs won't be exactly the same due to noises (like when you run the same program on your laptop multiple times). However, the runtime variation should be <%1." }, "source": "meta" }