{ "id": 277884, "title": "Team of 9 allowed?", "url": "/competitions/landmark-recognition-2021/discussion/277884", "author_name": "CPMP", "post_date": "2021-10-11T18:21:05.299000", "votes": 57, "comment_count": 21, "views": 0, "content": "

I was surprised to see 9 authors on the 3rd place paper: https://arxiv.org/pdf/2110.02794.pdf

\n

Does it mean that it was a team of 9 competing? I thought the team limit was 5.

\n

@andrefaraujo @maggiemd

\n

Edit. It is actually 9, not 8, authors.

", "messages": [ { "id": 1541674, "postDate": "2021-10-11T18:21:05.300Z", "content": "

I was surprised to see 9 authors on the 3rd place paper: https://arxiv.org/pdf/2110.02794.pdf

\n

Does it mean that it was a team of 9 competing? I thought the team limit was 5.

\n

@andrefaraujo @maggiemd

\n

Edit. It is actually 9, not 8, authors.

", "rawMarkdown": "I was surprised to see ~~ 8~~ 9 authors on the 3rd place paper: https://arxiv.org/pdf/2110.02794.pdf\n\nDoes it mean that it was a team of 9 competing? I thought the team limit was 5.\n\n@andrefaraujo @maggiemd \n\nEdit. It is actually 9, not 8, authors.", "votes": 57 }, { "id": 1542409, "postDate": "2021-10-12T13:44:17.340Z", "content": "

In the ICCV Instance-Level Recognition Workshop yesterday, when asked about why the team had 9 members on the paper and the slides, and was everyone part of the team, ByteDance teams answered yes. Everyone worked on the competition by training the models, but only 5 people submitted.

\n

I think it's clearly a violation of Rule B.6.B \"Team membership may not exceed the Maximum Team Size stated on the Competition Website.\"

\n

All the hosts were in the workshop, and the session was recorded.

", "rawMarkdown": "In the ICCV Instance-Level Recognition Workshop yesterday, when asked about why the team had 9 members on the paper and the slides, and was everyone part of the team, ByteDance teams answered yes. Everyone worked on the competition by training the models, but only 5 people submitted.\n\nI think it's clearly a violation of Rule B.6.B \"Team membership may not exceed the Maximum Team Size stated on the Competition Website.\"\n\nAll the hosts were in the workshop, and the session was recorded.", "votes": 27 }, { "id": 1543060, "postDate": "2021-10-13T05:41:20.043Z", "content": "

Interesting, some people downvote, meaning they find it OK to have a team larger than allowed. I will always be surprised by how flexible ethics is for some.

", "rawMarkdown": "Interesting, some people downvote, meaning they find it OK to have a team larger than allowed. I will always be surprised by how flexible ethics is for some.", "votes": 22 }, { "id": 1543785, "postDate": "2021-10-13T20:07:10.587Z", "content": "

Hey folks, we are investigating what happened in this situation. Thanks for your patience while we gather the details.

", "rawMarkdown": "Hey folks, we are investigating what happened in this situation. Thanks for your patience while we gather the details.", "votes": 17, "replies": [ { "id": 1544931, "postDate": "2021-10-14T19:20:21.483Z", "content": "

@wcukierski There is a similar situation in the Landmark Retrieval competition https://www.kaggle.com/c/landmark-retrieval-2021/discussion/278036

", "rawMarkdown": "@wcukierski There is a similar situation in the Landmark Retrieval competition https://www.kaggle.com/c/landmark-retrieval-2021/discussion/278036", "votes": 9 } ] }, { "id": 1543335, "postDate": "2021-10-13T11:27:24.090Z", "content": "

@maggiemd @inversion

\n

We expect Kaggle to take a view on this topic.

\n

Thanks @cpmpml for raising this

", "rawMarkdown": "@maggiemd @inversion \n\nWe expect Kaggle to take a view on this topic. \n\nThanks @cpmpml for raising this", "votes": 9 }, { "id": 1544183, "postDate": "2021-10-14T06:09:45.823Z", "content": "

Thank you for all your feedback and comments! We totally understood the concerns, and I would like to address to all your questions on behalf of the team.

\n
    \n

  1. Both my colleague and I are non-English speakers, and when my colleague said during the ICCV workshop that the other teammates \"helped with training and discussions\", what he actually meant was two of the coauthors from engineering side helped prepare, allocate and approve the GPU resources and quotas to make sure we have them available during the last few weeks of the competitions. None of them were involved in any part of the actual model training code or pipeline. To appreciate what they have done for us, we think it is reasonable to put their names on the final workshop paper.

    2. \n

  2. In a big team of a company, it is hard to not let anyone else know that we are doing a competition like this, and this is especially true when 5 of us even set the competition results as our work targets, and to review them during weekly team discussions. Another two coauthors are our managers, and they granted us the chance to take part in the competitions as part of work, and before the ICCV conference, they helped review and make some corrections/suggestions on the final draft before publishing it. So we think it is also reasonable to put both their names on the final paper.

    3. \n

  3. During the whole process of the competition, only 5 of us wrote the actual training code, came up and designed the final solutions, and we merged team as you can see on the LB.

    4. \n
\n

We once again thank you for the feedback, and will definitely work on the improvement of our communications in English.

\n

We are happy to work with Kaggle to provide evidences to showcase the engineering positions of our coauthors Tianling Bian, Yanyu Yan, as well as the manager positions of Qi She, Cheng Yang, should Kaggle requests for them. cc, @wcukierski

\n

Thanks,

", "rawMarkdown": "Thank you for all your feedback and comments! We totally understood the concerns, and I would like to address to all your questions on behalf of the team. \n\n1. Both my colleague and I are non-English speakers, and when my colleague said during the ICCV workshop that the other teammates \"helped with training and discussions\", what he actually meant was two of the coauthors from engineering side helped prepare, allocate and approve the GPU resources and quotas to make sure we have them available during the last few weeks of the competitions. None of them were involved in any part of the actual model training code or pipeline. To appreciate what they have done for us, we think it is reasonable to put their names on the final workshop paper. \n\n2. In a big team of a company, it is hard to not let anyone else know that we are doing a competition like this, and this is especially true when 5 of us even set the competition results as our work targets, and to review them during weekly team discussions. Another two coauthors are our managers, and they granted us the chance to take part in the competitions as part of work, and before the ICCV conference, they helped review and make some corrections/suggestions on the final draft before publishing it. So we think it is also reasonable to put both their names on the final paper. \n\n3. During the whole process of the competition, only 5 of us wrote the actual training code, came up and designed the final solutions, and we merged team as you can see on the LB. \n\nWe once again thank you for the feedback, and will definitely work on the improvement of our communications in English. \n\nWe are happy to work with Kaggle to provide evidences to showcase the engineering positions of our coauthors Tianling Bian, Yanyu Yan, as well as the manager positions of Qi She, Cheng Yang, should Kaggle requests for them. cc, @wcukierski \n\nThanks,", "votes": -16, "replies": [ { "id": 1544347, "postDate": "2021-10-14T09:10:47.693Z", "content": "
\n

Both my colleague and I are non-English speakers

\n
\n

So are all kagglers who entered this discussion. The only native speaker I see here is @wcukierski

", "rawMarkdown": "> Both my colleague and I are non-English speakers\n\nSo are all kagglers who entered this discussion. The only native speaker I see here is @wcukierski ", "votes": 9 }, { "id": 1544762, "postDate": "2021-10-14T16:03:13.273Z", "content": "

To appreciate what they have done for us, we think it is reasonable to put their names on the final workshop paper.

\n

The common practice (and good?) is to include: grants, organizations, funds etc that helped the research, in the Acknowledgements section.

\n

If they are coauthors, following the research ethics (and at a top venue like ICCV this is important), they have contributed to the paper, however, it does not mean they have contributed to the competition neccesarily.
\nUnless there is a rule for companies participating here, I think is impossible to prove who really participated/contributed in the competition. In any case I think is quite ugly/sad the bad practice of 5 guys from the same company doing submissions separately, but sharing managers/resources, it is just my opinion of course.

\n

Let's see what kaggle does because it might create a precedent for teams representing companies.

", "rawMarkdown": "```To appreciate what they have done for us, we think it is reasonable to put their names on the final workshop paper.```\n\nThe common practice (and good?) is to include: grants, organizations, funds etc that helped the research, in the Acknowledgements section.\n\nIf they are coauthors, following the research ethics (and at a top venue like ICCV this is important), they have contributed to the paper, however, it does not mean they have contributed to the competition neccesarily.\nUnless there is a rule for companies participating here, I think is impossible to prove who really participated/contributed in the competition. In any case I think is quite ugly/sad the bad practice of 5 guys from the same company doing submissions separately, but sharing managers/resources, it is just my opinion of course.\n\nLet's see what kaggle does because it might create a precedent for teams representing companies.", "votes": 9 } ] }, { "id": 1564252, "postDate": "2021-10-29T02:49:14.950Z", "content": "

Hello participants,

\n

Kaggle has reviewed evidence regarding two teams who have published academic papers with an author list containing names who were not members of their respective Kaggle teams. Specifically, this inquiry focused on whether this constituted a violation of rule 6B:

\n
\n

Each Team member must be a single individual with a separate Kaggle account. You must register individually for the Competition before joining a Team. You must confirm your Team membership to make it official by responding to the Team notification message sent to your Kaggle account. Team membership may not exceed the Maximum Team Size stated on the Competition Website.

\n
\n

In both cases, we found the nature and extent of contributions from the named paper authors did not violate the competition rules. We arrived at this decision by requesting clarification of the specific contributions of each author.

\n

Guidance for future, large teams:

\n

Kaggle's research competitions often coincide with opportunities for publishing via preprints, conferences, or journals. We recognize it is common in academic publishing for author lists to contain co-authors who were loosely involved in the work, or involved only by organizational affiliation. At the same time, it is imperative that affiliated research groups do not have an unfair advantage by exceeding Kaggle’s team size limits.

\n

Teams who publish work resulting from competitions should expect to document their individual contributions and may receive additional scrutiny pertaining to their publication. Below we provide representative (but not exhaustive) guidance of what Kaggle considers permissible vs non-permissible contributions.

\n

Permissible:

\n\n

Non permissible:

\n\n

Thank you for bringing this to our attention.

", "rawMarkdown": "Hello participants,\n\nKaggle has reviewed evidence regarding two teams who have published academic papers with an author list containing names who were not members of their respective Kaggle teams. Specifically, this inquiry focused on whether this constituted a violation of rule 6B:\n\n> Each Team member must be a single individual with a separate Kaggle account. You must register individually for the Competition before joining a Team. You must confirm your Team membership to make it official by responding to the Team notification message sent to your Kaggle account. Team membership may not exceed the Maximum Team Size stated on the Competition Website.\n\nIn both cases, we found the nature and extent of contributions from the named paper authors did not violate the competition rules. We arrived at this decision by requesting clarification of the specific contributions of each author.\n\n**Guidance for future, large teams:**\n\nKaggle's research competitions often coincide with opportunities for publishing via preprints, conferences, or journals. We recognize it is common in academic publishing for author lists to contain co-authors who were loosely involved in the work, or involved only by organizational affiliation. At the same time, it is imperative that affiliated research groups do not have an unfair advantage by exceeding Kaggle’s team size limits.\n\nTeams who publish work resulting from competitions should expect to document their individual contributions and may receive additional scrutiny pertaining to their publication. Below we provide representative (but not exhaustive) guidance of what Kaggle considers permissible vs non-permissible contributions.\n\nPermissible:\n - Provisioning hardware resources or providing other generic IT support\n - Providing generic ML guidance/teaching (unrelated to the specific problem)\n - Authoring existing generalized tooling/libraries (unrelated to the specific problem, still subject to licensing requirements)\n\nNon permissible:\n - Contributing to the code or ideas used in the submission\n - Providing specific scientific guidance (related to the specific problem)\n - Running experiments on behalf of the team\n\nThank you for bringing this to our attention.\n", "votes": -8, "replies": [ { "id": 1564437, "postDate": "2021-10-29T08:30:02.723Z", "content": "

Well, didn't the presenter for one of these teams admitted that all team members contributed to training models? This has been recorded hence it is easy to check.

\n
\n

We arrived at this decision by requesting clarification of the specific contributions of each author.

\n
\n

Well, if I had cheated then I would not say it.

\n

This is a permit to cheat: have more team members than allowed, then say they were just doing one of these things:

\n\n

If people were doing this for a team I am part of, then I would not include them as paper author as they did not directly contribute. I would thank them in the acknowledgement section of the paper.

", "rawMarkdown": "Well, didn't the presenter for one of these teams admitted that all team members contributed to training models? This has been recorded hence it is easy to check.\n\n> We arrived at this decision by requesting clarification of the specific contributions of each author.\n\nWell, if I had cheated then I would not say it.\n\nThis is a permit to cheat: have more team members than allowed, then say they were just doing one of these things:\n\n- Provisioning hardware resources or providing other generic IT support\n- Providing generic ML guidance/teaching (unrelated to the specific problem)\n- Authoring existing generalized tooling/libraries (unrelated to the specific problem, still subject to licensing requirements)\n\nIf people were doing this for a team I am part of, then I would not include them as paper author as they did not directly contribute. I would thank them in the acknowledgement section of the paper.", "votes": 7 }, { "id": 1564520, "postDate": "2021-10-29T10:03:23.527Z", "content": "

I am sorry I might be a little confused, @wcukierski you listed in Non permissible everything the team did and admitted lol, I assume is \"Non permissible from now on?\"
\nI honestly did not expect an answer, thank you @wcukierski !

", "rawMarkdown": "I am sorry I might be a little confused, @wcukierski you listed in `Non permissible` everything the team did and admitted lol, I assume is \"`Non permissible` **from now on**?\"\nI honestly did not expect an answer, thank you @wcukierski !", "votes": 2 } ] }, { "id": 1542239, "postDate": "2021-10-12T09:15:07.410Z", "content": "

I think for resource a lot of people use this technique, take help from some org for resource later give them credit (ex: Z by HP or jarvislabs.ai). But I didn't know one could take advice from other people too ….

", "rawMarkdown": "I think for resource a lot of people use this technique, take help from some org for resource later give them credit (ex: Z by HP or jarvislabs.ai). But I didn't know one could take advice from other people too ....", "votes": -10, "replies": [ { "id": 1542270, "postDate": "2021-10-12T10:13:09.460Z", "content": "

Give credit to org/people who provide compute resource is one thing. Working at more than 5 is a different thing. The latter is against competition rules.

", "rawMarkdown": "Give credit to org/people who provide compute resource is one thing. Working at more than 5 is a different thing. The latter is against competition rules.", "votes": 8 } ] }, { "id": 1541875, "postDate": "2021-10-12T00:52:48.087Z", "content": "

The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources.

", "rawMarkdown": "The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources. ", "votes": -20, "replies": [ { "id": 1542136, "postDate": "2021-10-12T06:55:31.660Z", "content": "

@weimin

\n
\n

The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources.

\n
\n

Thanks, you confirmed they are part of the team. What I don't get is how you interpret team size limit of 5. From the competition rules:

\n
\n

Team Limits

\n

The maximum team size is 5.

\n
\n

You are a seasoned Kaggler, you can't pretend you don't understand this.

", "rawMarkdown": "@weimin\n\n> The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources. \n\nThanks, you confirmed they are part of the team. What I don't get is how you interpret team size limit of 5. From the competition rules:\n\n> Team Limits\n> \n> The maximum team size is 5.\n\nYou are a seasoned Kaggler, you can't pretend you don't understand this.", "votes": 15 }, { "id": 1542140, "postDate": "2021-10-12T07:02:38.860Z", "content": "
\n

The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources.

\n
\n

So their contribution was significant enough for publishing the paper but not significant for being included in a Kaggle team?

", "rawMarkdown": "> The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources.\n\nSo their contribution was significant enough for publishing the paper but not significant for being included in a Kaggle team?", "votes": 12 }, { "id": 1542144, "postDate": "2021-10-12T07:06:12.960Z", "content": "

@weimin advise in a competition, which is largely driven by ideas, looks like a significant team contribution. Imagine I would not have participated, but instead \"offered advise\" to a team. You would perceive this as fair?

", "rawMarkdown": "@weimin advise in a competition, which is largely driven by ideas, looks like a significant team contribution. Imagine I would not have participated, but instead \"offered advise\" to a team. You would perceive this as fair?", "votes": 23 }, { "id": 1542145, "postDate": "2021-10-12T07:11:34.100Z", "rawMarkdown": "", "votes": 12, "isDeleted": true }, { "id": 1542857, "postDate": "2021-10-12T21:22:56.507Z", "content": "

if I hire 50 people to work for me in a competition that maximun team size is 5 is ok? I'm sure I will build models that best fit the metric, but I don't think it would be fair with other teams and definitely not ok with some Kaggle rules for any competition.

", "rawMarkdown": "if I hire 50 people to work for me in a competition that maximun team size is 5 is ok? I'm sure I will build models that best fit the metric, but I don't think it would be fair with other teams and definitely not ok with some Kaggle rules for any competition.", "votes": 17 }, { "id": 1543062, "postDate": "2021-10-13T05:45:16.497Z", "content": "
\n

but still given that they are from the same company, it would be better to merge right away.

\n
\n

This is a slightly different topic: is it OK to not start working as a team but team later? I know many people who think you should team as soon as you exchange information. My take is that as long as you end up teaming it does not matter.

\n

The issue raised by people is that when you enter late, staying separate gives you more submissions than if you team right away. However, whatever the way you go, all teams are bound by the same limit, usually 5 times the number of days in the competition.

", "rawMarkdown": "> but still given that they are from the same company, it would be better to merge right away.\n\nThis is a slightly different topic: is it OK to not start working as a team but team later? I know many people who think you should team as soon as you exchange information. My take is that as long as you end up teaming it does not matter.\n\nThe issue raised by people is that when you enter late, staying separate gives you more submissions than if you team right away. However, whatever the way you go, all teams are bound by the same limit, usually 5 times the number of days in the competition.", "votes": 10 } ] }, { "id": 1861470, "postDate": "2022-07-19T03:53:34.580Z", "content": "

Regardless of all the details, I think publishing papers is already outside the scope of kaggle competition.

", "rawMarkdown": "Regardless of all the details, I think publishing papers is already outside the scope of kaggle competition." } ], "comments": [ { "id": 1542409, "author_name": "Bo", "author_url": "", "post_date": "2021-10-12T13:44:17.340000", "content": "

In the ICCV Instance-Level Recognition Workshop yesterday, when asked about why the team had 9 members on the paper and the slides, and was everyone part of the team, ByteDance teams answered yes. Everyone worked on the competition by training the models, but only 5 people submitted.

\n

I think it's clearly a violation of Rule B.6.B \"Team membership may not exceed the Maximum Team Size stated on the Competition Website.\"

\n

All the hosts were in the workshop, and the session was recorded.

", "votes": 27, "replies": [] }, { "id": 1543060, "author_name": "CPMP", "author_url": "", "post_date": "2021-10-13T05:41:20.043000", "content": "

Interesting, some people downvote, meaning they find it OK to have a team larger than allowed. I will always be surprised by how flexible ethics is for some.

", "votes": 22, "replies": [] }, { "id": 1543785, "author_name": "Will Cukierski", "author_url": "", "post_date": "2021-10-13T20:07:10.587000", "content": "

Hey folks, we are investigating what happened in this situation. Thanks for your patience while we gather the details.

", "votes": 17, "replies": [ { "id": 1544931, "author_name": "CPMP", "author_url": "", "post_date": "2021-10-14T19:20:21.483000", "content": "

@wcukierski There is a similar situation in the Landmark Retrieval competition https://www.kaggle.com/c/landmark-retrieval-2021/discussion/278036

", "votes": 9, "replies": [] } ] }, { "id": 1543335, "author_name": "narsil (jobs-in-data.com)", "author_url": "", "post_date": "2021-10-13T11:27:24.090000", "content": "

@maggiemd @inversion

\n

We expect Kaggle to take a view on this topic.

\n

Thanks @cpmpml for raising this

", "votes": 9, "replies": [] }, { "id": 1544183, "author_name": "Weimin Wang", "author_url": "", "post_date": "2021-10-14T06:09:45.823000", "content": "

Thank you for all your feedback and comments! We totally understood the concerns, and I would like to address to all your questions on behalf of the team.

\n
    \n

  1. Both my colleague and I are non-English speakers, and when my colleague said during the ICCV workshop that the other teammates \"helped with training and discussions\", what he actually meant was two of the coauthors from engineering side helped prepare, allocate and approve the GPU resources and quotas to make sure we have them available during the last few weeks of the competitions. None of them were involved in any part of the actual model training code or pipeline. To appreciate what they have done for us, we think it is reasonable to put their names on the final workshop paper.

    2. \n

  2. In a big team of a company, it is hard to not let anyone else know that we are doing a competition like this, and this is especially true when 5 of us even set the competition results as our work targets, and to review them during weekly team discussions. Another two coauthors are our managers, and they granted us the chance to take part in the competitions as part of work, and before the ICCV conference, they helped review and make some corrections/suggestions on the final draft before publishing it. So we think it is also reasonable to put both their names on the final paper.

    3. \n

  3. During the whole process of the competition, only 5 of us wrote the actual training code, came up and designed the final solutions, and we merged team as you can see on the LB.

    4. \n
\n

We once again thank you for the feedback, and will definitely work on the improvement of our communications in English.

\n

We are happy to work with Kaggle to provide evidences to showcase the engineering positions of our coauthors Tianling Bian, Yanyu Yan, as well as the manager positions of Qi She, Cheng Yang, should Kaggle requests for them. cc, @wcukierski

\n

Thanks,

", "votes": -16, "replies": [ { "id": 1544347, "author_name": "CPMP", "author_url": "", "post_date": "2021-10-14T09:10:47.693000", "content": "
\n

Both my colleague and I are non-English speakers

\n
\n

So are all kagglers who entered this discussion. The only native speaker I see here is @wcukierski

", "votes": 9, "replies": [] }, { "id": 1544762, "author_name": "Nanashi", "author_url": "", "post_date": "2021-10-14T16:03:13.273000", "content": "

To appreciate what they have done for us, we think it is reasonable to put their names on the final workshop paper.

\n

The common practice (and good?) is to include: grants, organizations, funds etc that helped the research, in the Acknowledgements section.

\n

If they are coauthors, following the research ethics (and at a top venue like ICCV this is important), they have contributed to the paper, however, it does not mean they have contributed to the competition neccesarily.
\nUnless there is a rule for companies participating here, I think is impossible to prove who really participated/contributed in the competition. In any case I think is quite ugly/sad the bad practice of 5 guys from the same company doing submissions separately, but sharing managers/resources, it is just my opinion of course.

\n

Let's see what kaggle does because it might create a precedent for teams representing companies.

", "votes": 9, "replies": [] } ] }, { "id": 1564252, "author_name": "Will Cukierski", "author_url": "", "post_date": "2021-10-29T02:49:14.950000", "content": "

Hello participants,

\n

Kaggle has reviewed evidence regarding two teams who have published academic papers with an author list containing names who were not members of their respective Kaggle teams. Specifically, this inquiry focused on whether this constituted a violation of rule 6B:

\n
\n

Each Team member must be a single individual with a separate Kaggle account. You must register individually for the Competition before joining a Team. You must confirm your Team membership to make it official by responding to the Team notification message sent to your Kaggle account. Team membership may not exceed the Maximum Team Size stated on the Competition Website.

\n
\n

In both cases, we found the nature and extent of contributions from the named paper authors did not violate the competition rules. We arrived at this decision by requesting clarification of the specific contributions of each author.

\n

Guidance for future, large teams:

\n

Kaggle's research competitions often coincide with opportunities for publishing via preprints, conferences, or journals. We recognize it is common in academic publishing for author lists to contain co-authors who were loosely involved in the work, or involved only by organizational affiliation. At the same time, it is imperative that affiliated research groups do not have an unfair advantage by exceeding Kaggle’s team size limits.

\n

Teams who publish work resulting from competitions should expect to document their individual contributions and may receive additional scrutiny pertaining to their publication. Below we provide representative (but not exhaustive) guidance of what Kaggle considers permissible vs non-permissible contributions.

\n

Permissible:

\n\n

Non permissible:

\n\n

Thank you for bringing this to our attention.

", "votes": -8, "replies": [ { "id": 1564437, "author_name": "CPMP", "author_url": "", "post_date": "2021-10-29T08:30:02.723000", "content": "

Well, didn't the presenter for one of these teams admitted that all team members contributed to training models? This has been recorded hence it is easy to check.

\n
\n

We arrived at this decision by requesting clarification of the specific contributions of each author.

\n
\n

Well, if I had cheated then I would not say it.

\n

This is a permit to cheat: have more team members than allowed, then say they were just doing one of these things:

\n\n

If people were doing this for a team I am part of, then I would not include them as paper author as they did not directly contribute. I would thank them in the acknowledgement section of the paper.

", "votes": 7, "replies": [] }, { "id": 1564520, "author_name": "Nanashi", "author_url": "", "post_date": "2021-10-29T10:03:23.527000", "content": "

I am sorry I might be a little confused, @wcukierski you listed in Non permissible everything the team did and admitted lol, I assume is \"Non permissible from now on?\"
\nI honestly did not expect an answer, thank you @wcukierski !

", "votes": 2, "replies": [] } ] }, { "id": 1542239, "author_name": "Awsaf", "author_url": "", "post_date": "2021-10-12T09:15:07.410000", "content": "

I think for resource a lot of people use this technique, take help from some org for resource later give them credit (ex: Z by HP or jarvislabs.ai). But I didn't know one could take advice from other people too ….

", "votes": -10, "replies": [ { "id": 1542270, "author_name": "CPMP", "author_url": "", "post_date": "2021-10-12T10:13:09.460000", "content": "

Give credit to org/people who provide compute resource is one thing. Working at more than 5 is a different thing. The latter is against competition rules.

", "votes": 8, "replies": [] } ] }, { "id": 1541875, "author_name": "Weimin Wang", "author_url": "", "post_date": "2021-10-12T00:52:48.087000", "content": "

The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources.

", "votes": -20, "replies": [ { "id": 1542136, "author_name": "CPMP", "author_url": "", "post_date": "2021-10-12T06:55:31.660000", "content": "

@weimin

\n
\n

The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources.

\n
\n

Thanks, you confirmed they are part of the team. What I don't get is how you interpret team size limit of 5. From the competition rules:

\n
\n

Team Limits

\n

The maximum team size is 5.

\n
\n

You are a seasoned Kaggler, you can't pretend you don't understand this.

", "votes": 15, "replies": [] }, { "id": 1542140, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2021-10-12T07:02:38.860000", "content": "
\n

The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources.

\n
\n

So their contribution was significant enough for publishing the paper but not significant for being included in a Kaggle team?

", "votes": 12, "replies": [] }, { "id": 1542144, "author_name": "Dieter", "author_url": "", "post_date": "2021-10-12T07:06:12.960000", "content": "

@weimin advise in a competition, which is largely driven by ideas, looks like a significant team contribution. Imagine I would not have participated, but instead \"offered advise\" to a team. You would perceive this as fair?

", "votes": 23, "replies": [] }, { "id": 1542145, "author_name": "", "author_url": "", "post_date": "2021-10-12T07:11:34.100000", "content": "", "votes": 12, "replies": [] }, { "id": 1542857, "author_name": "Giba", "author_url": "", "post_date": "2021-10-12T21:22:56.507000", "content": "

if I hire 50 people to work for me in a competition that maximun team size is 5 is ok? I'm sure I will build models that best fit the metric, but I don't think it would be fair with other teams and definitely not ok with some Kaggle rules for any competition.

", "votes": 17, "replies": [] }, { "id": 1543062, "author_name": "CPMP", "author_url": "", "post_date": "2021-10-13T05:45:16.497000", "content": "
\n

but still given that they are from the same company, it would be better to merge right away.

\n
\n

This is a slightly different topic: is it OK to not start working as a team but team later? I know many people who think you should team as soon as you exchange information. My take is that as long as you end up teaming it does not matter.

\n

The issue raised by people is that when you enter late, staying separate gives you more submissions than if you team right away. However, whatever the way you go, all teams are bound by the same limit, usually 5 times the number of days in the competition.

", "votes": 10, "replies": [] } ] }, { "id": 1861470, "author_name": "Mingzheng Shi", "author_url": "", "post_date": "2022-07-19T03:53:34.580000", "content": "

Regardless of all the details, I think publishing papers is already outside the scope of kaggle competition.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1541674": "I was surprised to see ~~ 8~~ 9 authors on the 3rd place paper: https://arxiv.org/pdf/2110.02794.pdf\n\nDoes it mean that it was a team of 9 competing? I thought the team limit was 5.\n\n@andrefaraujo @maggiemd \n\nEdit. It is actually 9, not 8, authors.", "1542409": "In the ICCV Instance-Level Recognition Workshop yesterday, when asked about why the team had 9 members on the paper and the slides, and was everyone part of the team, ByteDance teams answered yes. Everyone worked on the competition by training the models, but only 5 people submitted.\n\nI think it's clearly a violation of Rule B.6.B \"Team membership may not exceed the Maximum Team Size stated on the Competition Website.\"\n\nAll the hosts were in the workshop, and the session was recorded.", "1543060": "Interesting, some people downvote, meaning they find it OK to have a team larger than allowed. I will always be surprised by how flexible ethics is for some.", "1543785": "Hey folks, we are investigating what happened in this situation. Thanks for your patience while we gather the details.", "1543335": "@maggiemd @inversion \n\nWe expect Kaggle to take a view on this topic. \n\nThanks @cpmpml for raising this", "1544183": "Thank you for all your feedback and comments! We totally understood the concerns, and I would like to address to all your questions on behalf of the team. \n\n1. Both my colleague and I are non-English speakers, and when my colleague said during the ICCV workshop that the other teammates \"helped with training and discussions\", what he actually meant was two of the coauthors from engineering side helped prepare, allocate and approve the GPU resources and quotas to make sure we have them available during the last few weeks of the competitions. None of them were involved in any part of the actual model training code or pipeline. To appreciate what they have done for us, we think it is reasonable to put their names on the final workshop paper. \n\n2. In a big team of a company, it is hard to not let anyone else know that we are doing a competition like this, and this is especially true when 5 of us even set the competition results as our work targets, and to review them during weekly team discussions. Another two coauthors are our managers, and they granted us the chance to take part in the competitions as part of work, and before the ICCV conference, they helped review and make some corrections/suggestions on the final draft before publishing it. So we think it is also reasonable to put both their names on the final paper. \n\n3. During the whole process of the competition, only 5 of us wrote the actual training code, came up and designed the final solutions, and we merged team as you can see on the LB. \n\nWe once again thank you for the feedback, and will definitely work on the improvement of our communications in English. \n\nWe are happy to work with Kaggle to provide evidences to showcase the engineering positions of our coauthors Tianling Bian, Yanyu Yan, as well as the manager positions of Qi She, Cheng Yang, should Kaggle requests for them. cc, @wcukierski \n\nThanks,", "1564252": "Hello participants,\n\nKaggle has reviewed evidence regarding two teams who have published academic papers with an author list containing names who were not members of their respective Kaggle teams. Specifically, this inquiry focused on whether this constituted a violation of rule 6B:\n\n> Each Team member must be a single individual with a separate Kaggle account. You must register individually for the Competition before joining a Team. You must confirm your Team membership to make it official by responding to the Team notification message sent to your Kaggle account. Team membership may not exceed the Maximum Team Size stated on the Competition Website.\n\nIn both cases, we found the nature and extent of contributions from the named paper authors did not violate the competition rules. We arrived at this decision by requesting clarification of the specific contributions of each author.\n\n**Guidance for future, large teams:**\n\nKaggle's research competitions often coincide with opportunities for publishing via preprints, conferences, or journals. We recognize it is common in academic publishing for author lists to contain co-authors who were loosely involved in the work, or involved only by organizational affiliation. At the same time, it is imperative that affiliated research groups do not have an unfair advantage by exceeding Kaggle’s team size limits.\n\nTeams who publish work resulting from competitions should expect to document their individual contributions and may receive additional scrutiny pertaining to their publication. Below we provide representative (but not exhaustive) guidance of what Kaggle considers permissible vs non-permissible contributions.\n\nPermissible:\n - Provisioning hardware resources or providing other generic IT support\n - Providing generic ML guidance/teaching (unrelated to the specific problem)\n - Authoring existing generalized tooling/libraries (unrelated to the specific problem, still subject to licensing requirements)\n\nNon permissible:\n - Contributing to the code or ideas used in the submission\n - Providing specific scientific guidance (related to the specific problem)\n - Running experiments on behalf of the team\n\nThank you for bringing this to our attention.\n", "1542239": "I think for resource a lot of people use this technique, take help from some org for resource later give them credit (ex: Z by HP or jarvislabs.ai). But I didn't know one could take advice from other people too ....", "1541875": "The other coauthors are too busy to compete or to make any submission, they either offered advice, or simply helped to approve the computing resources. ", "1861470": "Regardless of all the details, I think publishing papers is already outside the scope of kaggle competition." } }