{ "id": 6373, "title": "test set question", "url": "/competitions/yandex-personalized-web-search-challenge/discussion/6373", "author_name": "", "post_date": "2013-11-23T05:43:33.577Z", "votes": null, "comment_count": 4, "views": 1372, "content": "

On page: https://www.kaggle.com/c/yandex-personalized-web-search-challenge/data

\n

"From this set of
queries we filter out all queries with two or more clicks performed at
the same unit of time. "

\n

Does this mean that each TEST query has at most 2 urls can be labeled relevant, and they can NOT be the same label?

\n

 

\n

"Finally, from the resulting set of queries we
uniformly sample only one query and consider it to be a test query. "

\n

Can we get more details about the "uniformly sample" algorithm?

\n

 

\n

Thank you.

\n

 

", "messages": [ { "id": "35077", "postDate": "11/23/2013 05:43:33", "content": "

On page: https://www.kaggle.com/c/yandex-personalized-web-search-challenge/data

\n

"From this set of
queries we filter out all queries with two or more clicks performed at
the same unit of time. "

\n

Does this mean that each TEST query has at most 2 urls can be labeled relevant, and they can NOT be the same label?

\n

 

\n

"Finally, from the resulting set of queries we
uniformly sample only one query and consider it to be a test query. "

\n

Can we get more details about the "uniformly sample" algorithm?

\n

 

\n

Thank you.

\n

 

", "rawMarkdown": "", "votes": null }, { "id": "35452", "postDate": "11/28/2013 16:34:22", "content": "

Here's my understanding:

\n

1) It should be OK that a given TEST query has more than 2 relevant urls, and their labels can be different. The original statement just says that a query cannot click on 2 or more urls at the same time.

\n

2) There should be various tools or packages for random sampling, depending on your developing environment.

", "rawMarkdown": "", "votes": null }, { "id": "35479", "postDate": "11/29/2013 01:11:12", "content": "

Thank you for the reply. :)

\n

But in my opinion, (user, query, dwell time) defines a label, rather than (query, dwell time).

\n

Maybe that's why we have different understanding?

", "rawMarkdown": "", "votes": null }, { "id": "35481", "postDate": "11/29/2013 02:36:36", "content": "

see https://www.kaggle.com/c/yandex-personalized-web-search-challenge/forums/t/6310/clicks-at-the-same-unit-of-time

\n

If there are two clicks in the session at the same "TimePassed"-Value (and, at least as it reads, same SERPID), the query is disregarded as a test query, since you wouldn't know which click is really the later one (or just by order in the file, which shouldn't be trusted).

\n

As an example, look at Session 565, two clicks with different URLID at TimePassed = 365, so relevance is uncertain.

", "rawMarkdown": "", "votes": null }, { "id": "35773", "postDate": "12/03/2013 19:12:38", "content": "

You are right. The user factor is included in estimating a label. I think we share the same understanding.

\n

 

\n

[quote=vbs 0711;35479]

\n

Thank you for the reply. :)

\n

But in my opinion, (user, query, dwell time) defines a label, rather than (query, dwell time).

\n

Maybe that's why we have different understanding?

\n

[/quote]

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 35452, "author_name": "sunnyuanovo", "author_url": "", "post_date": "11/28/2013 16:34:22", "content": "

Here's my understanding:

\n

1) It should be OK that a given TEST query has more than 2 relevant urls, and their labels can be different. The original statement just says that a query cannot click on 2 or more urls at the same time.

\n

2) There should be various tools or packages for random sampling, depending on your developing environment.

", "votes": null, "replies": [] }, { "id": 35479, "author_name": "vbs0711", "author_url": "", "post_date": "11/29/2013 01:11:12", "content": "

Thank you for the reply. :)

\n

But in my opinion, (user, query, dwell time) defines a label, rather than (query, dwell time).

\n

Maybe that's why we have different understanding?

", "votes": null, "replies": [] }, { "id": 35481, "author_name": "corrision", "author_url": "", "post_date": "11/29/2013 02:36:36", "content": "

see https://www.kaggle.com/c/yandex-personalized-web-search-challenge/forums/t/6310/clicks-at-the-same-unit-of-time

\n

If there are two clicks in the session at the same "TimePassed"-Value (and, at least as it reads, same SERPID), the query is disregarded as a test query, since you wouldn't know which click is really the later one (or just by order in the file, which shouldn't be trusted).

\n

As an example, look at Session 565, two clicks with different URLID at TimePassed = 365, so relevance is uncertain.

", "votes": null, "replies": [] }, { "id": 35773, "author_name": "sunnyuanovo", "author_url": "", "post_date": "12/03/2013 19:12:38", "content": "

You are right. The user factor is included in estimating a label. I think we share the same understanding.

\n

 

\n

[quote=vbs 0711;35479]

\n

Thank you for the reply. :)

\n

But in my opinion, (user, query, dwell time) defines a label, rather than (query, dwell time).

\n

Maybe that's why we have different understanding?

\n

[/quote]

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "35077": "", "35452": "", "35479": "", "35481": "", "35773": "" }, "source": "meta" }