I've got little to no experience with/knowledge of machine learning, so please bear with me if this seems like a stupid or irrelevant question.
\r\nBasically what I'm wondering is how much freedom we have in terms of the approach used to solve this challenge. Are machine learning algorithms the only admissible solutions, or will any approach which meets the requirements of the challenge be considered?\r\n
\r\nAs I understand it, the requirements of this competition are to produce some programmatic solution which accepts as input a stackoverflow question and some metadata about it and produces as output an estimated probability of how likely it is that the question\r\n will be closed (and for what reason); we may use the training data as a reference while building out this solution.
\r\nThis requirement would not seem, in and of itself, to limit solutions strictly to machine learning based approaches. But does it? And if it doesn't, is there any point at which a solution becomes too far out there to be elligible. For instance, I could\r\n imagine a simple solution that just farms the work out to something like Mechanical Turk, but I think that would probably *not* be considered a valid approach (because 1) it requires human input to make the determination of how likely a question is to be banned\r\n and 2) incurs a fee for each question that is evaluated).
\r\nSo if a line in the sand exists, where is it? Is machine learning the only \r\nvalid elligible approach to solving the problem, or is any algorithmic approach that does not rely upon human input/intervention (such as Mechanical Turk) considered valid as well?
", "messages": [ { "id": "13537", "postDate": "08/27/2012 23:29:08", "content": "I've got little to no experience with/knowledge of machine learning, so please bear with me if this seems like a stupid or irrelevant question.
\r\nBasically what I'm wondering is how much freedom we have in terms of the approach used to solve this challenge. Are machine learning algorithms the only admissible solutions, or will any approach which meets the requirements of the challenge be considered?\r\n
\r\nAs I understand it, the requirements of this competition are to produce some programmatic solution which accepts as input a stackoverflow question and some metadata about it and produces as output an estimated probability of how likely it is that the question\r\n will be closed (and for what reason); we may use the training data as a reference while building out this solution.
\r\nThis requirement would not seem, in and of itself, to limit solutions strictly to machine learning based approaches. But does it? And if it doesn't, is there any point at which a solution becomes too far out there to be elligible. For instance, I could\r\n imagine a simple solution that just farms the work out to something like Mechanical Turk, but I think that would probably *not* be considered a valid approach (because 1) it requires human input to make the determination of how likely a question is to be banned\r\n and 2) incurs a fee for each question that is evaluated).
\r\nSo if a line in the sand exists, where is it? Is machine learning the only \r\nvalid elligible approach to solving the problem, or is any algorithmic approach that does not rely upon human input/intervention (such as Mechanical Turk) considered valid as well?
", "rawMarkdown": "", "votes": null }, { "id": "13538", "postDate": "08/27/2012 23:31:52", "content": "[quote=Adam Roth;13537]
\r\nI've got little to no experience with/knowledge of machine learning, so please bear with me if this seems like a stupid or irrelevant question.
\r\nBasically what I'm wondering is how much freedom we have in terms of the approach used to solve this challenge. Are machine learning algorithms the only admissible solutions, or will any approach which meets the requirements of the challenge be considered?\r\n
\r\nAs I understand it, the requirements of this competition are to produce some programmatic solution which accepts as input a stackoverflow question and some metadata about it and produces as output an estimated probability of how likely it is that the question\r\n will be closed (and for what reason); we may use the training data as a reference while building out this solution.
\r\nThis requirement would not seem, in and of itself, to limit solutions strictly to machine learning based approaches. But does it? And if it doesn't, is there any point at which a solution becomes too far out there to be elligible. For instance, I could\r\n imagine a simple solution that just farms the work out to something like Mechanical Turk, but I think that would probably *not* be considered a valid approach (because 1) it requires human input to make the determination of how likely a question is to be banned\r\n and 2) incurs a fee for each question that is evaluated).
\r\nSo if a line in the sand exists, where is it? Is machine learning the only \r\nvalid elligible approach to solving the problem, or is any algorithmic approach that does not rely upon human input/intervention (such as Mechanical Turk) considered valid as well?
\r\n[/quote]The solution needs to be entirely automated (so no Mechanical Turk). Additionally, it should not make API calls to external services (such as Mechanical Turk).
", "rawMarkdown": "", "votes": null }, { "id": "13540", "postDate": "08/27/2012 23:41:47", "content": "I was fairly confident that something like Mechanical Turk would be out of bounds. That's why I used it as an example.
\r\nAnyhow, what you're saying is that barring external services, any self-contained, automated/programmatic solution is considered acceptable?
", "rawMarkdown": "", "votes": null }, { "id": "13541", "postDate": "08/27/2012 23:46:16", "content": "[quote=Adam Roth;13540]
\r\nI was fairly confident that something like Mechanical Turk would be out of bounds. That's why I used it as an example.
\r\nAnyhow, what you're saying is that barring external services, any self-contained, automated/programmatic solution is considered acceptable?
\r\n[/quote]Refer to the terms and conditions and\r\n\r\nrules for more precise information. In general, black-box solutions are unacceptable, as well as any code that would require additional licenses for Stack Exchange to use. If you have any specific questions where you believe the terms / rules are not clear,\r\n ask them here.
", "rawMarkdown": "", "votes": null }, { "id": "13543", "postDate": "08/28/2012 01:00:14", "content": "Thanks, it was mostly the usage of the term "Model" in the contest description that made it seem like perhaps a specific approach was being advocated. But the official definition of the term clears that confusion up nicely:
\r\nany code, text, algorithm or series of algorithms, equation or series of equations, material, software, designs, documents, descriptions or specifications which is used, in whole or in part, directly or indirectly, in calculating, drafting, building,\r\n devising, calibrating, testing, evaluating, analyzing or generating an Entry, or which itself constitutes the whole or part of an Entry.\r\n
So a "Model" can be essentially anything that solves the problem, and the term is not meant to exclusively connote predictive/probabilistic models of the sort specifically discussed in the context of machine learning, or any other kind of singular technology.\r\n That works for me.
\r\nEdit: How come by blockquote doesn't render like a quote/blockquote?
", "rawMarkdown": "", "votes": null }, { "id": "13577", "postDate": "08/28/2012 21:40:00", "content": "[quote=Ben Hamner;13541]
\r\n[quote=Adam Roth;13540]
\r\nI was fairly confident that something like Mechanical Turk would be out of bounds. That's why I used it as an example.
\r\nAnyhow, what you're saying is that barring external services, any self-contained, automated/programmatic solution is considered acceptable?
\r\n[/quote]Refer to the terms and conditions and\r\n\r\nrules for more precise information. In general, black-box solutions are unacceptable, as well as any code that would require additional licenses for Stack Exchange to use. If you have any specific questions where you believe the terms / rules are not clear,\r\n ask them here.
\r\n[/quote]
\r\n\r\n
Is third-party MIT licenced software okay to use? I'm currently using \r\nFast CSV Reader for parsing in my solution. If so, would I have to provide its source code in a submission, or would just the binary (and licence) file be okay?
", "rawMarkdown": "", "votes": null }, { "id": "13578", "postDate": "08/28/2012 21:40:41", "content": "[quote=DPenner;13577]
\r\n[quote=Ben Hamner;13541]
\r\n[quote=Adam Roth;13540]
\r\nI was fairly confident that something like Mechanical Turk would be out of bounds. That's why I used it as an example.
\r\nAnyhow, what you're saying is that barring external services, any self-contained, automated/programmatic solution is considered acceptable?
\r\n[/quote]Refer to the terms and conditions and\r\n\r\nrules for more precise information. In general, black-box solutions are unacceptable, as well as any code that would require additional licenses for Stack Exchange to use. If you have any specific questions where you believe the terms / rules are not clear,\r\n ask them here.
\r\n[/quote]
\r\n\r\n
Is third-party MIT licenced software okay to use? I'm currently using \r\nFast CSV Reader for parsing in my solution. If so, would I have to provide its source code in a submission, or would just the binary (and licence) file be okay?
\r\n[/quote]Yes, provide the source code as well.
", "rawMarkdown": "", "votes": null }, { "id": "13586", "postDate": "08/28/2012 23:59:23", "content": "Related to DPenner's question, what if I'm using a dependency management system like Maven to handle third-party libraries used by my project? Do I need to submit the actual third-party libraries/code, or is it sufficient to submit my 'pom.xml' file (Maven\r\n uses the 'pom.xml' file to automatically download any missing dependencies and add them to the project classpath) and some brief instructions on how to build/run the project using Maven?
", "rawMarkdown": "", "votes": null }, { "id": "13589", "postDate": "08/29/2012 00:30:42", "content": "[quote=Adam Roth;13586]
\r\nRelated to DPenner's question, what if I'm using a dependency management system like Maven to handle third-party libraries used by my project? Do I need to submit the actual third-party libraries/code, or is it sufficient to submit my 'pom.xml' file (Maven\r\n uses the 'pom.xml' file to automatically download any missing dependencies and add them to the project classpath) and some brief instructions on how to build/run the project using Maven?
\r\n[/quote]That's fine, so long as the third party libraries use licenses that would enable Stack Overflow to build off them & none of them use black box models.
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 13538, "author_name": "benhamner", "author_url": "", "post_date": "08/27/2012 23:31:52", "content": "[quote=Adam Roth;13537]
\r\nI've got little to no experience with/knowledge of machine learning, so please bear with me if this seems like a stupid or irrelevant question.
\r\nBasically what I'm wondering is how much freedom we have in terms of the approach used to solve this challenge. Are machine learning algorithms the only admissible solutions, or will any approach which meets the requirements of the challenge be considered?\r\n
\r\nAs I understand it, the requirements of this competition are to produce some programmatic solution which accepts as input a stackoverflow question and some metadata about it and produces as output an estimated probability of how likely it is that the question\r\n will be closed (and for what reason); we may use the training data as a reference while building out this solution.
\r\nThis requirement would not seem, in and of itself, to limit solutions strictly to machine learning based approaches. But does it? And if it doesn't, is there any point at which a solution becomes too far out there to be elligible. For instance, I could\r\n imagine a simple solution that just farms the work out to something like Mechanical Turk, but I think that would probably *not* be considered a valid approach (because 1) it requires human input to make the determination of how likely a question is to be banned\r\n and 2) incurs a fee for each question that is evaluated).
\r\nSo if a line in the sand exists, where is it? Is machine learning the only \r\nvalid elligible approach to solving the problem, or is any algorithmic approach that does not rely upon human input/intervention (such as Mechanical Turk) considered valid as well?
\r\n[/quote]The solution needs to be entirely automated (so no Mechanical Turk). Additionally, it should not make API calls to external services (such as Mechanical Turk).
", "votes": null, "replies": [] }, { "id": 13540, "author_name": "adamroth", "author_url": "", "post_date": "08/27/2012 23:41:47", "content": "I was fairly confident that something like Mechanical Turk would be out of bounds. That's why I used it as an example.
\r\nAnyhow, what you're saying is that barring external services, any self-contained, automated/programmatic solution is considered acceptable?
", "votes": null, "replies": [] }, { "id": 13541, "author_name": "benhamner", "author_url": "", "post_date": "08/27/2012 23:46:16", "content": "[quote=Adam Roth;13540]
\r\nI was fairly confident that something like Mechanical Turk would be out of bounds. That's why I used it as an example.
\r\nAnyhow, what you're saying is that barring external services, any self-contained, automated/programmatic solution is considered acceptable?
\r\n[/quote]Refer to the terms and conditions and\r\n\r\nrules for more precise information. In general, black-box solutions are unacceptable, as well as any code that would require additional licenses for Stack Exchange to use. If you have any specific questions where you believe the terms / rules are not clear,\r\n ask them here.
", "votes": null, "replies": [] }, { "id": 13543, "author_name": "adamroth", "author_url": "", "post_date": "08/28/2012 01:00:14", "content": "Thanks, it was mostly the usage of the term "Model" in the contest description that made it seem like perhaps a specific approach was being advocated. But the official definition of the term clears that confusion up nicely:
\r\nany code, text, algorithm or series of algorithms, equation or series of equations, material, software, designs, documents, descriptions or specifications which is used, in whole or in part, directly or indirectly, in calculating, drafting, building,\r\n devising, calibrating, testing, evaluating, analyzing or generating an Entry, or which itself constitutes the whole or part of an Entry.\r\n
So a "Model" can be essentially anything that solves the problem, and the term is not meant to exclusively connote predictive/probabilistic models of the sort specifically discussed in the context of machine learning, or any other kind of singular technology.\r\n That works for me.
\r\nEdit: How come by blockquote doesn't render like a quote/blockquote?
", "votes": null, "replies": [] }, { "id": 13577, "author_name": "dpenner", "author_url": "", "post_date": "08/28/2012 21:40:00", "content": "[quote=Ben Hamner;13541]
\r\n[quote=Adam Roth;13540]
\r\nI was fairly confident that something like Mechanical Turk would be out of bounds. That's why I used it as an example.
\r\nAnyhow, what you're saying is that barring external services, any self-contained, automated/programmatic solution is considered acceptable?
\r\n[/quote]Refer to the terms and conditions and\r\n\r\nrules for more precise information. In general, black-box solutions are unacceptable, as well as any code that would require additional licenses for Stack Exchange to use. If you have any specific questions where you believe the terms / rules are not clear,\r\n ask them here.
\r\n[/quote]
\r\n\r\n
Is third-party MIT licenced software okay to use? I'm currently using \r\nFast CSV Reader for parsing in my solution. If so, would I have to provide its source code in a submission, or would just the binary (and licence) file be okay?
", "votes": null, "replies": [] }, { "id": 13578, "author_name": "benhamner", "author_url": "", "post_date": "08/28/2012 21:40:41", "content": "[quote=DPenner;13577]
\r\n[quote=Ben Hamner;13541]
\r\n[quote=Adam Roth;13540]
\r\nI was fairly confident that something like Mechanical Turk would be out of bounds. That's why I used it as an example.
\r\nAnyhow, what you're saying is that barring external services, any self-contained, automated/programmatic solution is considered acceptable?
\r\n[/quote]Refer to the terms and conditions and\r\n\r\nrules for more precise information. In general, black-box solutions are unacceptable, as well as any code that would require additional licenses for Stack Exchange to use. If you have any specific questions where you believe the terms / rules are not clear,\r\n ask them here.
\r\n[/quote]
\r\n\r\n
Is third-party MIT licenced software okay to use? I'm currently using \r\nFast CSV Reader for parsing in my solution. If so, would I have to provide its source code in a submission, or would just the binary (and licence) file be okay?
\r\n[/quote]Yes, provide the source code as well.
", "votes": null, "replies": [] }, { "id": 13586, "author_name": "adamroth", "author_url": "", "post_date": "08/28/2012 23:59:23", "content": "Related to DPenner's question, what if I'm using a dependency management system like Maven to handle third-party libraries used by my project? Do I need to submit the actual third-party libraries/code, or is it sufficient to submit my 'pom.xml' file (Maven\r\n uses the 'pom.xml' file to automatically download any missing dependencies and add them to the project classpath) and some brief instructions on how to build/run the project using Maven?
", "votes": null, "replies": [] }, { "id": 13589, "author_name": "benhamner", "author_url": "", "post_date": "08/29/2012 00:30:42", "content": "[quote=Adam Roth;13586]
\r\nRelated to DPenner's question, what if I'm using a dependency management system like Maven to handle third-party libraries used by my project? Do I need to submit the actual third-party libraries/code, or is it sufficient to submit my 'pom.xml' file (Maven\r\n uses the 'pom.xml' file to automatically download any missing dependencies and add them to the project classpath) and some brief instructions on how to build/run the project using Maven?
\r\n[/quote]That's fine, so long as the third party libraries use licenses that would enable Stack Overflow to build off them & none of them use black box models.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "13537": "", "13538": "", "13540": "", "13541": "", "13543": "", "13577": "", "13578": "", "13586": "", "13589": "" }, "source": "meta" }