A lot of succcessful kernels have used nq-test.tfrecords as one of the input files. Where do they source it from and what does it contain?
https://www.kaggle.com/prokaj/bert-joint-baseline#nq-test.tfrecords\nhttps://www.kaggle.com/prokaj/bert-joint-baseline-notebook/data\nhttps://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers/data\nhttps://www.kaggle.com/andrewgao/tensorflow-2-0-bert-yes-no-answers/data\nhttps://www.kaggle.com/prokaj/bert-preprocess-notebook/data\nhttps://www.kaggle.com/ymcdull/tensorflow-2-0-edited/data\nhttps://www.kaggle.com/jonathandickson/my-submission-code/data\nhttps://www.kaggle.com/kerneler/starter-bert-joint-baseline-9e284fc6-f/data
\n\n@prokaj @mmmarchetti @andrewgao @ymcdull @jonathandickson @kerneler
\n\nOthers - ??
", "messages": [ { "id": "696739", "postDate": "12/17/2019 01:54:42", "content": "A lot of succcessful kernels have used nq-test.tfrecords as one of the input files. Where do they source it from and what does it contain?
https://www.kaggle.com/prokaj/bert-joint-baseline#nq-test.tfrecords\nhttps://www.kaggle.com/prokaj/bert-joint-baseline-notebook/data\nhttps://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers/data\nhttps://www.kaggle.com/andrewgao/tensorflow-2-0-bert-yes-no-answers/data\nhttps://www.kaggle.com/prokaj/bert-preprocess-notebook/data\nhttps://www.kaggle.com/ymcdull/tensorflow-2-0-edited/data\nhttps://www.kaggle.com/jonathandickson/my-submission-code/data\nhttps://www.kaggle.com/kerneler/starter-bert-joint-baseline-9e284fc6-f/data
\n\n@prokaj @mmmarchetti @andrewgao @ymcdull @jonathandickson @kerneler
\n\nOthers - ??
", "rawMarkdown": "A lot of succcessful kernels have used `nq-test.tfrecords` as one of the input files. Where do they source it from and what does it contain?\n\nhttps://www.kaggle.com/prokaj/bert-joint-baseline#nq-test.tfrecords\nhttps://www.kaggle.com/prokaj/bert-joint-baseline-notebook/data\nhttps://www.kaggle.com/mmmarchetti/tensorflow-2-0-bert-yes-no-answers/data\nhttps://www.kaggle.com/andrewgao/tensorflow-2-0-bert-yes-no-answers/data\nhttps://www.kaggle.com/prokaj/bert-preprocess-notebook/data\nhttps://www.kaggle.com/ymcdull/tensorflow-2-0-edited/data\nhttps://www.kaggle.com/jonathandickson/my-submission-code/data\nhttps://www.kaggle.com/kerneler/starter-bert-joint-baseline-9e284fc6-f/data\n\n@prokaj @mmmarchetti @andrewgao @ymcdull @jonathandickson @kerneler \n\nOthers - ??", "votes": null }, { "id": "696800", "postDate": "12/17/2019 04:42:28", "content": "It's from the Google Natural Questions Bert Joint baseline available at https://github.com/google-research/language/tree/master/language/question_answering/bert_joint (those file are from the Google cloud bucket linked in the README).\nit contains pre-computed inputs for the test set, the result of tokenizing and chunking the test documents. As the test set for this competition differs from the NQ dataset I think it would either not be used or would have been re-generated for the Kaggle test set using the provided scripts. Though I think in order to run on the private test set most kernels would not be using it.
", "rawMarkdown": "It's from the Google Natural Questions Bert Joint baseline available at https://github.com/google-research/language/tree/master/language/question_answering/bert_joint (those file are from the Google cloud bucket linked in the README).\nit contains pre-computed inputs for the test set, the result of tokenizing and chunking the test documents. As the test set for this competition differs from the NQ dataset I think it would either not be used or would have been re-generated for the Kaggle test set using the provided scripts. Though I think in order to run on the private test set most kernels would not be using it.", "votes": null }, { "id": "697041", "postDate": "12/17/2019 12:09:14", "content": "Thanks a lot for your response @thomasbrandon .
\n\nHowever, I couldn't find any Google cloud bucket linked in the README.\nAlso, why do you think it doesn't make sense to regenerate this for the private test set.
\n\nApologies if my questions are very basic since I'm new to this.
", "rawMarkdown": "Thanks a lot for your response @thomasbrandon . \n\nHowever, I couldn't find any Google cloud bucket linked in the README.\nAlso, why do you think it doesn't make sense to regenerate this for the private test set.\n\nApologies if my questions are very basic since I'm new to this.", "votes": null }, { "id": "697312", "postDate": "12/17/2019 17:44:10", "content": "The cloud bucket is downloaded with:\ngsutil cp -R gs://bert-nq/bert-joint-baseline .
Though actually it only contains the training set (nq-train.tfrecords) and model weights, The test records would be the result of running the language.question_answering.bert_joint.prepare_nq_data script on the Kaggle test JSON.
\n\n\nwhy do you think it doesn't make sense to regenerate this for the private test set.
\n
Sorry, I wasn't clear there. I meant that you could either create the nq-test inputs for the public test set and store them in a dataset to access from your kernel. Or, as your kernel has to run on the private test set which you can't store in a dataset (as you don't have access), you could just do your prediction straight off the JSON for both the public and private test set.
", "rawMarkdown": "The cloud bucket is downloaded with:\n```gsutil cp -R gs://bert-nq/bert-joint-baseline .```\n\nThough actually it only contains the training set (`nq-train.tfrecords`) and model weights, The test records would be the result of running the `language.question_answering.bert_joint.prepare_nq_data` script on the Kaggle test JSON.\n\n> why do you think it doesn't make sense to regenerate this for the private test set.\n\nSorry, I wasn't clear there. I meant that you could either create the nq-test inputs for the public test set and store them in a dataset to access from your kernel. Or, as your kernel has to run on the private test set which you can't store in a dataset (as you don't have access), you could just do your prediction straight off the JSON for both the public and private test set.", "votes": null }, { "id": "704412", "postDate": "12/27/2019 12:56:46", "content": "below script in the kernel generate the file
\n\neval_records='nq-test.tfrecords'\nif not os.path.exists(eval_records):\n # tf2baseline.FLAGS.max_seq_length = 512\n eval_writer = bert_utils.FeatureWriter(\n filename=os.path.join(eval_records),\n is_training=False)
\n\ntokenizer = tokenization.FullTokenizer(vocab_file=path+'bert-joint-baseline/vocab-nq.txt', \n do_lower_case=True)\n\nfeatures = []\nconvert = bert_utils.ConvertExamples2Features(tokenizer=tokenizer,\n is_training=False,\n output_fn=eval_writer.process_feature,\n collect_stat=False)\n\nn_examples = 0\ntqdm_notebook= tqdm.tqdm_notebook if not on_kaggle_server else None\nfor examples in bert_utils.nq_examples_iter(input_file=nq_test_file, \n is_training=False,\n tqdm=tqdm_notebook):\n for example in examples:\n n_examples += convert(example)\n\neval_writer.close()\nprint('number of test examples: %d, written to file: %d' % (n_examples,eval_writer.num_features))\nIt's from the Google Natural Questions Bert Joint baseline available at https://github.com/google-research/language/tree/master/language/question_answering/bert_joint (those file are from the Google cloud bucket linked in the README).\nit contains pre-computed inputs for the test set, the result of tokenizing and chunking the test documents. As the test set for this competition differs from the NQ dataset I think it would either not be used or would have been re-generated for the Kaggle test set using the provided scripts. Though I think in order to run on the private test set most kernels would not be using it.
", "votes": null, "replies": [ { "id": 697041, "author_name": "rohitagarwal", "author_url": "", "post_date": "12/17/2019 12:09:14", "content": "Thanks a lot for your response @thomasbrandon .
\n\nHowever, I couldn't find any Google cloud bucket linked in the README.\nAlso, why do you think it doesn't make sense to regenerate this for the private test set.
\n\nApologies if my questions are very basic since I'm new to this.
", "votes": null, "replies": [] }, { "id": 697312, "author_name": "thomasbrandon", "author_url": "", "post_date": "12/17/2019 17:44:10", "content": "The cloud bucket is downloaded with:\ngsutil cp -R gs://bert-nq/bert-joint-baseline .
Though actually it only contains the training set (nq-train.tfrecords) and model weights, The test records would be the result of running the language.question_answering.bert_joint.prepare_nq_data script on the Kaggle test JSON.
\n\n\nwhy do you think it doesn't make sense to regenerate this for the private test set.
\n
Sorry, I wasn't clear there. I meant that you could either create the nq-test inputs for the public test set and store them in a dataset to access from your kernel. Or, as your kernel has to run on the private test set which you can't store in a dataset (as you don't have access), you could just do your prediction straight off the JSON for both the public and private test set.
", "votes": null, "replies": [] } ] }, { "id": 704412, "author_name": "wentixiaogege", "author_url": "", "post_date": "12/27/2019 12:56:46", "content": "below script in the kernel generate the file
\n\neval_records='nq-test.tfrecords'\nif not os.path.exists(eval_records):\n # tf2baseline.FLAGS.max_seq_length = 512\n eval_writer = bert_utils.FeatureWriter(\n filename=os.path.join(eval_records),\n is_training=False)
\n\ntokenizer = tokenization.FullTokenizer(vocab_file=path+'bert-joint-baseline/vocab-nq.txt', \n do_lower_case=True)\n\nfeatures = []\nconvert = bert_utils.ConvertExamples2Features(tokenizer=tokenizer,\n is_training=False,\n output_fn=eval_writer.process_feature,\n collect_stat=False)\n\nn_examples = 0\ntqdm_notebook= tqdm.tqdm_notebook if not on_kaggle_server else None\nfor examples in bert_utils.nq_examples_iter(input_file=nq_test_file, \n is_training=False,\n tqdm=tqdm_notebook):\n for example in examples:\n n_examples += convert(example)\n\neval_writer.close()\nprint('number of test examples: %d, written to file: %d' % (n_examples,eval_writer.num_features))\n