There is always someone sad just about any action: too early sharing -- bad, many guys will crowd LB and push self working people down, too late sharing -- bad, those who monitor forum and have time will get advantage, too much GPU power -- bad, too little -- bad, the list goes on and on. One in common: you almost will never hear such complaints from hardworking top scoring guys ;)
", "rawMarkdown": "There is always someone sad just about any action: too early sharing -- bad, many guys will crowd LB and push self working people down, too late sharing -- bad, those who monitor forum and have time will get advantage, too much GPU power -- bad, too little -- bad, the list goes on and on. One in common: you almost will never hear such complaints from hardworking top scoring guys ;)", "votes": null }, { "id": "279810", "postDate": "02/08/2018 18:53:46", "content": "\n\n\nyou almost will never hear such complaints from hardworking top scoring guys
\n
That doesn't mean the problem doesn't exist and we shouldn't work towards solving it :)
", "rawMarkdown": "> you almost will never hear such complaints from hardworking top scoring guys\n\nThat doesn't mean the problem doesn't exist and we shouldn't work towards solving it :)", "votes": null }, { "id": "279813", "postDate": "02/08/2018 18:58:46", "content": "It means that better spend your time for brainstorming and trying your best than for complaining
\n\nand yes, there are rules with the answers for each of your points..except probably the last one, sorry
", "rawMarkdown": "It means that better spend your time for brainstorming and trying your best than for complaining\n\nand yes, there are rules with the answers for each of your points..except probably the last one, sorry", "votes": null }, { "id": "279822", "postDate": "02/08/2018 19:28:38", "content": "Ah, just noticed 3rd point. ods.ai is not an organisation. It is a community of friends and we don't have any sharing outside of kaggle forums. More over, in kaggle we are competing against each other even more then against general public, so sharing outside teams isn't in our best interests. What we get from community is mostly bravado and incentive to compete hard. For example, even though it isn't prohibited to complain about anything between us, it is highly discouraged and laughed at (like friends do all the time). So nobody is handheld in our community and get equal \"fight or die\" attitude.
\n\nAnd you may see fruits of this on top of the leaderboard.
", "rawMarkdown": "Ah, just noticed 3rd point. ods.ai is not an organisation. It is a community of friends and we don't have any sharing outside of kaggle forums. More over, in kaggle we are competing against each other even more then against general public, so sharing outside teams isn't in our best interests. What we get from community is mostly bravado and incentive to compete hard. For example, even though it isn't prohibited to complain about anything between us, it is highly discouraged and laughed at (like friends do all the time). So nobody is handheld in our community and get equal \"fight or die\" attitude. \n\nAnd you may see fruits of this on top of the leaderboard.", "votes": null }, { "id": "279827", "postDate": "02/08/2018 19:34:28", "content": "Hi Manish -
\n\nI'd like to respond to your 3rd point (as the others have been discussed previously). I, too, had concerns that there might be coordination between ods.ai teams. Let me share some thoughts.
\n\nThis is something we take very seriously. If anyone ever has specific information that helps us keep the integrity of the leader board, we encourage them to pass it along.
", "rawMarkdown": "Hi Manish -\n\nI'd like to respond to your 3rd point (as the others have been discussed previously). I, too, had concerns that there might be coordination between ods.ai teams. Let me share some thoughts.\n\n - We have extensive anti-cheating algorithms. Any team that is flagged (ods or otherwise) will be thoroughly reviewed and the appropriate action taken.\n - Even though there is an application form to join the ods.ai slack, it appears to be open to anyone who is interested (and is able to benefit from majority-Russian discussion). I applied and was granted access in less than 20 minutes.\n - There are hundreds of people participating on the channel, and only a small number in the top teams, and there is nothing in the discussions that would indicate anyone was breaking the rules by sharing code or data. The channel is very much in the spirit of the KaggleNoobs slack channel, i.e., sharing resources, ideas, helping people get unstuck, etc.\n\nThis is something we take very seriously. If anyone ever has specific information that helps us keep the integrity of the leader board, we encourage them to pass it along.", "votes": null }, { "id": "279885", "postDate": "02/08/2018 22:08:59", "content": "Sir!! with the way this submissions are generated (Predicting actual phone label) rather than a probability and the high accuracy in this competition. I don't think your anti-cheating algorithms will be effective in this one. potato - potatoe sort of thing you know, subjective.
", "rawMarkdown": "Sir!! with the way this submissions are generated (Predicting actual phone label) rather than a probability and the high accuracy in this competition. I don't think your anti-cheating algorithms will be effective in this one. potato - potatoe sort of thing you know, subjective.", "votes": null }, { "id": "279892", "postDate": "02/08/2018 22:20:21", "content": "One would have to know how the algorithms worked before being able to comment on their effectiveness. But still, to your point, it is impossible to be able to catch every form of cheating. We do, though, constantly improve, and try to make it so the amount of work required to cheat successfully is high enough that it just doesn't make sense to do it.
", "rawMarkdown": "One would have to know how the algorithms worked before being able to comment on their effectiveness. But still, to your point, it is impossible to be able to catch every form of cheating. We do, though, constantly improve, and try to make it so the amount of work required to cheat successfully is high enough that it just doesn't make sense to do it.", "votes": null }, { "id": "279903", "postDate": "02/08/2018 22:49:31", "content": "So you are telling me if I gave my best submission to my buddy and instructed him to flip 1 or 2 entries you will detect this. His submission and mine will be identical in everything but 2 entries and you have no way of telling we cheated.. I don't think it is rocket science, even if you detected this at the current accuracy we a talking maybe 20 - 30 images will be different in TOP teams. What are the chances multiple teams get the same items wrong. I think its pretty high.
", "rawMarkdown": "So you are telling me if I gave my best submission to my buddy and instructed him to flip 1 or 2 entries you will detect this. His submission and mine will be identical in everything but 2 entries and you have no way of telling we cheated.. I don't think it is rocket science, even if you detected this at the current accuracy we a talking maybe 20 - 30 images will be different in TOP teams. What are the chances multiple teams get the same items wrong. I think its pretty high.", "votes": null }, { "id": "279904", "postDate": "02/08/2018 22:53:30", "content": "David,
\n\nKaggle's policy is to not discuss the methodology of our anti-cheating algorithm.
\n\nBest,\nAddison
", "rawMarkdown": "David,\n\nKaggle's policy is to not discuss the methodology of our anti-cheating algorithm. \n\nBest,\nAddison", "votes": null }, { "id": "279909", "postDate": "02/08/2018 23:20:36", "content": "Hi Manish,
\n\n\n\n\nIs it fair to share a high scoring code (LB .95) via github a week\n before end of the contest when people have worked hard to reach there \n over past month?
\n
I pondered about this when sharing my code. My first iteration was ~0.70 LB and the reason I started sharing it then (before it got to .95) was to contribute a baseline (same way kernels do) and get feedback, not just for me, but all community. I got great feedback (e.g. from Chun Ming Lee, Ivan Romanov, Matt Kleinsmith, kirk, Alberto, etc.) which I'm sure helped others too. Thanks to the feedback I fixed a few things and got ~0.95... at that point I stopped sharing further improvements b/c I thought it was too high of a baseline to share.
\n\nSo it is fair? I guess for you it is not if you're just after LB score; but sharing openly levels the playing field and you can take ideas of my code. Furthermo Kaggle allows open sharing and in fact is one reason I like Kaggle vs. other sites, b/c I can learn from others WHILE doing the competition which is when motivation is at its high. I learned a lot from Her CherKeng on a few competitions so I wanted to contribute too.
\n\n\n\n\nIs it fair to allow all external data just 2 days before the end of\n the contest?
\n
External data was allowed all the way through (read the rules). I decided to share my training set (which I just finished just then) in the spirit of previous Kaggle competitions.
\n\n\n\n\nIs it fair to allow people from one organisation to pile up and hog\n the leaderboard? They can share external data without the need to\n download again, they can share ideas among themselves(and not via this\n forum) which can give them a significant boost.
\n
I too just joined ods.ai and was given instant access.
\n\n\n\n\nBut hey, is life fair? Ofcourse not :)
\n
I too have my complaints which I stated: massive group ensembles.
\n\nAt the end of the day Kaggle needs to balance the interest of 3 parties: competitors, competition promoters and Kaggle itself. There's always compromises.
", "rawMarkdown": "Hi Manish,\n\n> Is it fair to share a high scoring code (LB .95) via github a week\n> before end of the contest when people have worked hard to reach there \n> over past month?\n\nI pondered about this when sharing my code. My first iteration was ~0.70 LB and the reason I started sharing it then (before it got to .95) was to contribute a baseline (same way kernels do) and get feedback, not just for me, but all community. I got great feedback (e.g. from Chun Ming Lee, Ivan Romanov, Matt Kleinsmith, kirk, Alberto, etc.) which I'm sure helped others too. Thanks to the feedback I fixed a few things and got ~0.95... at that point I stopped sharing further improvements b/c I thought it was too high of a baseline to share. \n\nSo it is fair? I guess for you it is not if you're just after LB score; but sharing openly levels the playing field and you can take ideas of my code. Furthermo Kaggle allows open sharing and in fact is one reason I like Kaggle vs. other sites, b/c I can learn from others WHILE doing the competition which is when motivation is at its high. I learned a lot from Her CherKeng on a few competitions so I wanted to contribute too. \n\n> Is it fair to allow all external data just 2 days before the end of\n> the contest?\n\nExternal data was allowed all the way through (read the rules). I decided to share my training set (which I just finished just then) in the spirit of previous Kaggle competitions. \n\n> Is it fair to allow people from one organisation to pile up and hog\n> the leaderboard? They can share external data without the need to\n> download again, they can share ideas among themselves(and not via this\n> forum) which can give them a significant boost.\n\nI too just joined ods.ai and was given instant access.\n\n> But hey, is life fair? Ofcourse not :)\n\nI too have my complaints which I stated: massive group ensembles. \n\nAt the end of the day Kaggle needs to balance the interest of 3 parties: competitors, competition promoters and Kaggle itself. There's always compromises.", "votes": null }, { "id": "279937", "postDate": "02/09/2018 01:14:18", "content": "For your question 1, I have something to say, my understanding about the share spirit. Similar question looks like : Some company work hard on MachineLearning platform, some team work very hard on cnn-algorithm. so, why some company share the tensorflow, why some team share the fast-rcnn algorithm? Does these sharing things(tensorflow/keras/sklearn/cnn) ruined your interest about data science/data competition?
\n\nIt is not about fair, sharing make community better! Their sharing benefit others, and others benefit the whole community more. We should thank those sharing very much!
", "rawMarkdown": "For your question 1, I have something to say, my understanding about the share spirit. Similar question looks like : Some company work hard on MachineLearning platform, some team work very hard on cnn-algorithm. so, why some company share the tensorflow, why some team share the fast-rcnn algorithm? Does these sharing things(tensorflow/keras/sklearn/cnn) ruined your interest about data science/data competition?\n\nIt is not about fair, sharing make community better! Their sharing benefit others, and others benefit the whole community more. We should thank those sharing very much!", "votes": null } ], "comments": [ { "id": 279808, "author_name": "ceperaang", "author_url": "", "post_date": "02/08/2018 18:46:44", "content": "There is always someone sad just about any action: too early sharing -- bad, many guys will crowd LB and push self working people down, too late sharing -- bad, those who monitor forum and have time will get advantage, too much GPU power -- bad, too little -- bad, the list goes on and on. One in common: you almost will never hear such complaints from hardworking top scoring guys ;)
", "votes": null, "replies": [ { "id": 279810, "author_name": "", "author_url": "", "post_date": "02/08/2018 18:53:46", "content": "\n\n\nyou almost will never hear such complaints from hardworking top scoring guys
\n
That doesn't mean the problem doesn't exist and we shouldn't work towards solving it :)
", "votes": null, "replies": [] }, { "id": 279813, "author_name": "vladimirgroza", "author_url": "", "post_date": "02/08/2018 18:58:46", "content": "It means that better spend your time for brainstorming and trying your best than for complaining
\n\nand yes, there are rules with the answers for each of your points..except probably the last one, sorry
", "votes": null, "replies": [] } ] }, { "id": 279822, "author_name": "ceperaang", "author_url": "", "post_date": "02/08/2018 19:28:38", "content": "Ah, just noticed 3rd point. ods.ai is not an organisation. It is a community of friends and we don't have any sharing outside of kaggle forums. More over, in kaggle we are competing against each other even more then against general public, so sharing outside teams isn't in our best interests. What we get from community is mostly bravado and incentive to compete hard. For example, even though it isn't prohibited to complain about anything between us, it is highly discouraged and laughed at (like friends do all the time). So nobody is handheld in our community and get equal \"fight or die\" attitude.
\n\nAnd you may see fruits of this on top of the leaderboard.
", "votes": null, "replies": [] }, { "id": 279827, "author_name": "resolut", "author_url": "", "post_date": "02/08/2018 19:34:28", "content": "Hi Manish -
\n\nI'd like to respond to your 3rd point (as the others have been discussed previously). I, too, had concerns that there might be coordination between ods.ai teams. Let me share some thoughts.
\n\nThis is something we take very seriously. If anyone ever has specific information that helps us keep the integrity of the leader board, we encourage them to pass it along.
", "votes": null, "replies": [ { "id": 279885, "author_name": "godaibo", "author_url": "", "post_date": "02/08/2018 22:08:59", "content": "Sir!! with the way this submissions are generated (Predicting actual phone label) rather than a probability and the high accuracy in this competition. I don't think your anti-cheating algorithms will be effective in this one. potato - potatoe sort of thing you know, subjective.
", "votes": null, "replies": [] }, { "id": 279892, "author_name": "inversion", "author_url": "", "post_date": "02/08/2018 22:20:21", "content": "One would have to know how the algorithms worked before being able to comment on their effectiveness. But still, to your point, it is impossible to be able to catch every form of cheating. We do, though, constantly improve, and try to make it so the amount of work required to cheat successfully is high enough that it just doesn't make sense to do it.
", "votes": null, "replies": [] }, { "id": 279903, "author_name": "godaibo", "author_url": "", "post_date": "02/08/2018 22:49:31", "content": "So you are telling me if I gave my best submission to my buddy and instructed him to flip 1 or 2 entries you will detect this. His submission and mine will be identical in everything but 2 entries and you have no way of telling we cheated.. I don't think it is rocket science, even if you detected this at the current accuracy we a talking maybe 20 - 30 images will be different in TOP teams. What are the chances multiple teams get the same items wrong. I think its pretty high.
", "votes": null, "replies": [] }, { "id": 279904, "author_name": "addisonhoward", "author_url": "", "post_date": "02/08/2018 22:53:30", "content": "David,
\n\nKaggle's policy is to not discuss the methodology of our anti-cheating algorithm.
\n\nBest,\nAddison
", "votes": null, "replies": [] } ] }, { "id": 279909, "author_name": "antorsae", "author_url": "", "post_date": "02/08/2018 23:20:36", "content": "Hi Manish,
\n\n\n\n\nIs it fair to share a high scoring code (LB .95) via github a week\n before end of the contest when people have worked hard to reach there \n over past month?
\n
I pondered about this when sharing my code. My first iteration was ~0.70 LB and the reason I started sharing it then (before it got to .95) was to contribute a baseline (same way kernels do) and get feedback, not just for me, but all community. I got great feedback (e.g. from Chun Ming Lee, Ivan Romanov, Matt Kleinsmith, kirk, Alberto, etc.) which I'm sure helped others too. Thanks to the feedback I fixed a few things and got ~0.95... at that point I stopped sharing further improvements b/c I thought it was too high of a baseline to share.
\n\nSo it is fair? I guess for you it is not if you're just after LB score; but sharing openly levels the playing field and you can take ideas of my code. Furthermo Kaggle allows open sharing and in fact is one reason I like Kaggle vs. other sites, b/c I can learn from others WHILE doing the competition which is when motivation is at its high. I learned a lot from Her CherKeng on a few competitions so I wanted to contribute too.
\n\n\n\n\nIs it fair to allow all external data just 2 days before the end of\n the contest?
\n
External data was allowed all the way through (read the rules). I decided to share my training set (which I just finished just then) in the spirit of previous Kaggle competitions.
\n\n\n\n\nIs it fair to allow people from one organisation to pile up and hog\n the leaderboard? They can share external data without the need to\n download again, they can share ideas among themselves(and not via this\n forum) which can give them a significant boost.
\n
I too just joined ods.ai and was given instant access.
\n\n\n\n\nBut hey, is life fair? Ofcourse not :)
\n
I too have my complaints which I stated: massive group ensembles.
\n\nAt the end of the day Kaggle needs to balance the interest of 3 parties: competitors, competition promoters and Kaggle itself. There's always compromises.
", "votes": null, "replies": [] }, { "id": 279937, "author_name": "yyqing", "author_url": "", "post_date": "02/09/2018 01:14:18", "content": "For your question 1, I have something to say, my understanding about the share spirit. Similar question looks like : Some company work hard on MachineLearning platform, some team work very hard on cnn-algorithm. so, why some company share the tensorflow, why some team share the fast-rcnn algorithm? Does these sharing things(tensorflow/keras/sklearn/cnn) ruined your interest about data science/data competition?
\n\nIt is not about fair, sharing make community better! Their sharing benefit others, and others benefit the whole community more. We should thank those sharing very much!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "279804": "1. Is it fair to share a high scoring code (LB .95) via github a week before end of the contest when people have worked hard to reach there over past month?\n 2. Is it fair to allow all external data just 2 days before the end of the contest?\n 3. Is it fair to allow people from one organisation to pile up and hog the leaderboard? They can share external data without the need to download again, they can share ideas among themselves(and not via this forum) which can give them a significant boost.\n 4. But hey, is life fair? Ofcourse not :)", "279808": "There is always someone sad just about any action: too early sharing -- bad, many guys will crowd LB and push self working people down, too late sharing -- bad, those who monitor forum and have time will get advantage, too much GPU power -- bad, too little -- bad, the list goes on and on. One in common: you almost will never hear such complaints from hardworking top scoring guys ;)", "279810": "> you almost will never hear such complaints from hardworking top scoring guys\n\nThat doesn't mean the problem doesn't exist and we shouldn't work towards solving it :)", "279813": "It means that better spend your time for brainstorming and trying your best than for complaining\n\nand yes, there are rules with the answers for each of your points..except probably the last one, sorry", "279822": "Ah, just noticed 3rd point. ods.ai is not an organisation. It is a community of friends and we don't have any sharing outside of kaggle forums. More over, in kaggle we are competing against each other even more then against general public, so sharing outside teams isn't in our best interests. What we get from community is mostly bravado and incentive to compete hard. For example, even though it isn't prohibited to complain about anything between us, it is highly discouraged and laughed at (like friends do all the time). So nobody is handheld in our community and get equal \"fight or die\" attitude. \n\nAnd you may see fruits of this on top of the leaderboard.", "279827": "1. Yes. And it's the coolest part actually. If you work hard you can incorporate the new code in your result and get a nice boost on LB, if not you can learn from it. I think Kaggle is mostly about learning. \n2. Yes. Check rules for competition. \n3. Ideas it's just ideas. Hard work pays off. \n4. It's up to the individual to decide whether expect something from life or take it.", "279846": "Hi Manish -\n\nI'd like to respond to your 3rd point (as the others have been discussed previously). I, too, had concerns that there might be coordination between ods.ai teams. Let me share some thoughts.\n\n - We have extensive anti-cheating algorithms. Any team that is flagged (ods or otherwise) will be thoroughly reviewed and the appropriate action taken.\n - Even though there is an application form to join the ods.ai slack, it appears to be open to anyone who is interested (and is able to benefit from majority-Russian discussion). I applied and was granted access in less than 20 minutes.\n - There are hundreds of people participating on the channel, and only a small number in the top teams, and there is nothing in the discussions that would indicate anyone was breaking the rules by sharing code or data. The channel is very much in the spirit of the KaggleNoobs slack channel, i.e., sharing resources, ideas, helping people get unstuck, etc.\n\nThis is something we take very seriously. If anyone ever has specific information that helps us keep the integrity of the leader board, we encourage them to pass it along.", "279885": "Sir!! with the way this submissions are generated (Predicting actual phone label) rather than a probability and the high accuracy in this competition. I don't think your anti-cheating algorithms will be effective in this one. potato - potatoe sort of thing you know, subjective.", "279892": "One would have to know how the algorithms worked before being able to comment on their effectiveness. But still, to your point, it is impossible to be able to catch every form of cheating. We do, though, constantly improve, and try to make it so the amount of work required to cheat successfully is high enough that it just doesn't make sense to do it.", "279903": "So you are telling me if I gave my best submission to my buddy and instructed him to flip 1 or 2 entries you will detect this. His submission and mine will be identical in everything but 2 entries and you have no way of telling we cheated.. I don't think it is rocket science, even if you detected this at the current accuracy we a talking maybe 20 - 30 images will be different in TOP teams. What are the chances multiple teams get the same items wrong. I think its pretty high.", "279904": "David,\n\nKaggle's policy is to not discuss the methodology of our anti-cheating algorithm. \n\nBest,\nAddison", "279909": "Hi Manish,\n\n> Is it fair to share a high scoring code (LB .95) via github a week\n> before end of the contest when people have worked hard to reach there \n> over past month?\n\nI pondered about this when sharing my code. My first iteration was ~0.70 LB and the reason I started sharing it then (before it got to .95) was to contribute a baseline (same way kernels do) and get feedback, not just for me, but all community. I got great feedback (e.g. from Chun Ming Lee, Ivan Romanov, Matt Kleinsmith, kirk, Alberto, etc.) which I'm sure helped others too. Thanks to the feedback I fixed a few things and got ~0.95... at that point I stopped sharing further improvements b/c I thought it was too high of a baseline to share. \n\nSo it is fair? I guess for you it is not if you're just after LB score; but sharing openly levels the playing field and you can take ideas of my code. Furthermo Kaggle allows open sharing and in fact is one reason I like Kaggle vs. other sites, b/c I can learn from others WHILE doing the competition which is when motivation is at its high. I learned a lot from Her CherKeng on a few competitions so I wanted to contribute too. \n\n> Is it fair to allow all external data just 2 days before the end of\n> the contest?\n\nExternal data was allowed all the way through (read the rules). I decided to share my training set (which I just finished just then) in the spirit of previous Kaggle competitions. \n\n> Is it fair to allow people from one organisation to pile up and hog\n> the leaderboard? They can share external data without the need to\n> download again, they can share ideas among themselves(and not via this\n> forum) which can give them a significant boost.\n\nI too just joined ods.ai and was given instant access.\n\n> But hey, is life fair? Ofcourse not :)\n\nI too have my complaints which I stated: massive group ensembles. \n\nAt the end of the day Kaggle needs to balance the interest of 3 parties: competitors, competition promoters and Kaggle itself. There's always compromises.", "279937": "For your question 1, I have something to say, my understanding about the share spirit. Similar question looks like : Some company work hard on MachineLearning platform, some team work very hard on cnn-algorithm. so, why some company share the tensorflow, why some team share the fast-rcnn algorithm? Does these sharing things(tensorflow/keras/sklearn/cnn) ruined your interest about data science/data competition?\n\nIt is not about fair, sharing make community better! Their sharing benefit others, and others benefit the whole community more. We should thank those sharing very much!" }, "source": "meta" }