{ "id": 1656, "title": "Data aggregation and primary key", "url": "/competitions/kddcup2012-track2/discussion/1656", "author_name": "", "post_date": "2012-03-30T16:35:42.363Z", "votes": null, "comment_count": 2, "views": 2549, "content": "

Hi,

\r\n

As I understand from:

\r\n
\r\n

We divide each session into multiple instances, where each instance describes an impressed ad under a certain setting  (i.e., with certain depth and position values).  We aggregate instances with\r\n the same user id, ad id, query, and setting in order to reduce the dataset size.

\r\n
\r\n

in "training.txt" userid+adid+queryid+depth+position is a primary key. But that is not true, aproximately 1% of the instances are duplicated in that sense. For example, the following one is duplicated

\r\n
$ cat training.txt | grep 21258213 | grep 356694
0\t1\t1658343530815135762\t21258213\t2298\t3\t2\t7246\t4307\t7183\t16230\t356694
0\t1\t1658343530815135762\t21258213\t2298\t3\t3\t7246\t4307\t7183\t16137\t356694
0\t1\t1658343530815135762\t21258213\t2298\t3\t3\t7246\t4307\t7183\t16230\t356694
\r\n
 
\r\n
It's a mistake or I am misunderstanding the problem statement?
\r\n
 
\r\n
Thank you.
\r\n

 

", "messages": [ { "id": "9857", "postDate": "03/30/2012 16:35:42", "content": "

Hi,

\r\n

As I understand from:

\r\n
\r\n

We divide each session into multiple instances, where each instance describes an impressed ad under a certain setting  (i.e., with certain depth and position values).  We aggregate instances with\r\n the same user id, ad id, query, and setting in order to reduce the dataset size.

\r\n
\r\n

in "training.txt" userid+adid+queryid+depth+position is a primary key. But that is not true, aproximately 1% of the instances are duplicated in that sense. For example, the following one is duplicated

\r\n
$ cat training.txt | grep 21258213 | grep 356694
0\t1\t1658343530815135762\t21258213\t2298\t3\t2\t7246\t4307\t7183\t16230\t356694
0\t1\t1658343530815135762\t21258213\t2298\t3\t3\t7246\t4307\t7183\t16137\t356694
0\t1\t1658343530815135762\t21258213\t2298\t3\t3\t7246\t4307\t7183\t16230\t356694
\r\n
 
\r\n
It's a mistake or I am misunderstanding the problem statement?
\r\n
 
\r\n
Thank you.
\r\n

 

", "rawMarkdown": "", "votes": null }, { "id": "9873", "postDate": "03/31/2012 06:14:49", "content": "

Hi, LARCA,

\r\n

Thank you for the question. My colleague, Liubin Wang, helped checking the data. We think there is no mistake in the data set.\r\n

\r\n

The 1st and the 2nd instances cannot be aggregated due to different position (on the 7th field).
\r\nThe 1st and the 3rd instances cannot be aggregated due to different position.
\r\nThe 2nd and the 3rd cannot be aggregated due to different description id (on the 11th field).

\r\n

As you quoted:

\r\n

We aggregate instances with the same user id, ad id, query, and setting in order to reduce the dataset size.

\r\n

The "position" and "description id" are parts of "setting".

", "rawMarkdown": "", "votes": null }, { "id": "9880", "postDate": "03/31/2012 18:49:16", "content": "

Thnk you Yi Wang for the reply.

\r\n

So what makes unique each instance is userid+adid+queryid+depth+position+descriptionid. Any other feature to consider as part of "setting"?

\r\n

Thanks again!

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 9873, "author_name": "yiwang0", "author_url": "", "post_date": "03/31/2012 06:14:49", "content": "

Hi, LARCA,

\r\n

Thank you for the question. My colleague, Liubin Wang, helped checking the data. We think there is no mistake in the data set.\r\n

\r\n

The 1st and the 2nd instances cannot be aggregated due to different position (on the 7th field).
\r\nThe 1st and the 3rd instances cannot be aggregated due to different position.
\r\nThe 2nd and the 3rd cannot be aggregated due to different description id (on the 11th field).

\r\n

As you quoted:

\r\n

We aggregate instances with the same user id, ad id, query, and setting in order to reduce the dataset size.

\r\n

The "position" and "description id" are parts of "setting".

", "votes": null, "replies": [] }, { "id": 9880, "author_name": "mingot", "author_url": "", "post_date": "03/31/2012 18:49:16", "content": "

Thnk you Yi Wang for the reply.

\r\n

So what makes unique each instance is userid+adid+queryid+depth+position+descriptionid. Any other feature to consider as part of "setting"?

\r\n

Thanks again!

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "9857": "", "9873": "", "9880": "" }, "source": "meta" }