{
  "id": 14922,
  "title": "SearchInfo Table issues - SQLite",
  "url": "/competitions/avito-context-ad-clicks/discussion/14922",
  "author_name": "",
  "post_date": "2015-06-28T14:35:26.347Z",
  "votes": null,
  "comment_count": 1,
  "views": 487,
  "content": "<p>Hi,</p>\n<p>I'm having issues with the SearchInfo Table (from the SQLite file). First there seems to be an issue with the Private Key because SearchID is not unique:</p>\n<p>&quot;SearchID INTEGER, --not a primary key b/c: TSV_no_header/SearchInfo.tsv:1735958: INSERT failed: UNIQUE constraint failed: SearchInfo.SearchID&quot;&nbsp;</p>\n\n<p>Then if I try a simple query on SearchID it doesn't work:</p>\n<p>SELECT * from SearchInfo WHERE SearchID = 4;</p>\n<p>This doesn't return anything but I know there is a row with SearchID=4 from a ROWID query:</p>\n<p>SELECT * from SearchInfo WHERE ROWID = 4;</p>\n<p>I created a new column (TestCol) and copy the SearchID column to it, that seems to fix the query issue. So I can count the duplicates:</p>\n<p>SELECT TestCol, COUNT(*) c FROM SearchInfo GROUP BY TestCol HAVING c &gt; 1;</p>\n<p>|105564817</p>\n<p>That seems to indicates that 105564817 out of 112159462&nbsp;rows have no SearchID??&nbsp;</p>\n<p>I'm downloading the SearchInfo.tsv file now to see if it's a bug.&nbsp;Providing the Checksum of the file would be useful given the size of the files.</p>\n<p>Anyone else facing the same issue?</p>",
  "messages": [
    {
      "id": "82862",
      "postDate": "06/28/2015 14:35:26",
      "content": "<p>Hi,</p>\n<p>I'm having issues with the SearchInfo Table (from the SQLite file). First there seems to be an issue with the Private Key because SearchID is not unique:</p>\n<p>&quot;SearchID INTEGER, --not a primary key b/c: TSV_no_header/SearchInfo.tsv:1735958: INSERT failed: UNIQUE constraint failed: SearchInfo.SearchID&quot;&nbsp;</p>\n\n<p>Then if I try a simple query on SearchID it doesn't work:</p>\n<p>SELECT * from SearchInfo WHERE SearchID = 4;</p>\n<p>This doesn't return anything but I know there is a row with SearchID=4 from a ROWID query:</p>\n<p>SELECT * from SearchInfo WHERE ROWID = 4;</p>\n<p>I created a new column (TestCol) and copy the SearchID column to it, that seems to fix the query issue. So I can count the duplicates:</p>\n<p>SELECT TestCol, COUNT(*) c FROM SearchInfo GROUP BY TestCol HAVING c &gt; 1;</p>\n<p>|105564817</p>\n<p>That seems to indicates that 105564817 out of 112159462&nbsp;rows have no SearchID??&nbsp;</p>\n<p>I'm downloading the SearchInfo.tsv file now to see if it's a bug.&nbsp;Providing the Checksum of the file would be useful given the size of the files.</p>\n<p>Anyone else facing the same issue?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "83215",
      "postDate": "07/02/2015 11:57:56",
      "content": "<p>In case that helps someone else, my bugs were from the <strong>indexing</strong> that was crashing halfway through - hence the query failed.</p>\n<p>The fix was to move the sqlite temp folder on a bigger hard drive, I was running out of space on the system drive. Then re-index the SearchInfo column.</p>\n<p>I didn't find any duplicate SearchID after all.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 83215,
      "author_name": "npetitclerc",
      "author_url": "",
      "post_date": "07/02/2015 11:57:56",
      "content": "<p>In case that helps someone else, my bugs were from the <strong>indexing</strong> that was crashing halfway through - hence the query failed.</p>\n<p>The fix was to move the sqlite temp folder on a bigger hard drive, I was running out of space on the system drive. Then re-index the SearchInfo column.</p>\n<p>I didn't find any duplicate SearchID after all.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "82862": "",
    "83215": ""
  },
  "source": "meta"
}