{
  "id": 121304,
  "title": "Any good way to restore tf1 checkpoint?",
  "url": "/competitions/tensorflow2-question-answering/discussion/121304",
  "author_name": "",
  "post_date": "2019-12-12T10:07:52.726389500Z",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>It seems the baseline \"<a href=\"https://www.kaggle.com/philculliton/using-tensorflow-2-0-w-bert-on-nq\">Using Tensorflow 2.0 w/ Bert on NQ</a>\" kernel doesn't initialize from any checkpoint.\nThe line \"<em>checkpoint_tf.restore(init_checkpoint)</em>\" in its script doesn't restore anything because the initial checkpoint it uses is tf1 checkpoint, but this function can only restore tf2 checkpoint, and that's why everyone got different score running that kernel.</p>\n\n<p>Any idea for initializing the model from tf1 checkpoint? or simply make it possible to use the original \"modeling.py\" script from original BERT</p>",
  "messages": [
    {
      "id": "693373",
      "postDate": "12/12/2019 10:07:52",
      "content": "<p>It seems the baseline \"<a href=\"https://www.kaggle.com/philculliton/using-tensorflow-2-0-w-bert-on-nq\">Using Tensorflow 2.0 w/ Bert on NQ</a>\" kernel doesn't initialize from any checkpoint.\nThe line \"<em>checkpoint_tf.restore(init_checkpoint)</em>\" in its script doesn't restore anything because the initial checkpoint it uses is tf1 checkpoint, but this function can only restore tf2 checkpoint, and that's why everyone got different score running that kernel.</p>\n\n<p>Any idea for initializing the model from tf1 checkpoint? or simply make it possible to use the original \"modeling.py\" script from original BERT</p>",
      "rawMarkdown": "It seems the baseline \"[Using Tensorflow 2.0 w/ Bert on NQ](https://www.kaggle.com/philculliton/using-tensorflow-2-0-w-bert-on-nq)\" kernel doesn't initialize from any checkpoint.\nThe line \"*checkpoint\\_tf.restore(init\\_checkpoint)*\" in its script doesn't restore anything because the initial checkpoint it uses is tf1 checkpoint, but this function can only restore tf2 checkpoint, and that's why everyone got different score running that kernel.\n\nAny idea for initializing the model from tf1 checkpoint? or simply make it possible to use the original \"modeling.py\" script from original BERT",
      "votes": null
    },
    {
      "id": "693411",
      "postDate": "12/12/2019 10:42:40",
      "content": "",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 693411,
      "author_name": "mosicr",
      "author_url": "",
      "post_date": "12/12/2019 10:42:40",
      "content": "",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "693373": "It seems the baseline \"[Using Tensorflow 2.0 w/ Bert on NQ](https://www.kaggle.com/philculliton/using-tensorflow-2-0-w-bert-on-nq)\" kernel doesn't initialize from any checkpoint.\nThe line \"*checkpoint\\_tf.restore(init\\_checkpoint)*\" in its script doesn't restore anything because the initial checkpoint it uses is tf1 checkpoint, but this function can only restore tf2 checkpoint, and that's why everyone got different score running that kernel.\n\nAny idea for initializing the model from tf1 checkpoint? or simply make it possible to use the original \"modeling.py\" script from original BERT",
    "693411": ""
  },
  "source": "meta"
}