{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"# Offline Predictions with AutoML models for Bengali Handwritten graphemes"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"We have seen how to import data and train models in Google AutoML and export the models into the `saved_model` format of Tensorflow - among others (see [this Notebook](https://www.kaggle.com/wardenga/bengali-handwritten-graphemes-with-automl). In this Notebook we are going to import the `saved_model.pb` produced by AutoMl and make predictions. "},{"metadata":{"trusted":true},"cell_type":"code","source":"import tensorflow.compat.v1 as tf #modyficatin for tensorflow 2.1 might follow soon\ntf.disable_v2_behavior()\nimport pandas as pd\nimport numpy as np\nimport io\nfrom matplotlib.image import imsave\nimport csv\nimport os\nimport time\nimport gc","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Load the model with `tf.saved_model.loader.load()` inside a `tf.Session`. Then we transform the data to an image (as in [this Notebook](https://www.kaggle.com/wardenga/bengali-handwritten-graphemes-with-automl)) since images are what we fed AutoML with. \n\nNote that the path fed to the loader has to be to the DIRECTORY that the `saved_model.pb` is contained in, not the file."},{"metadata":{"trusted":true},"cell_type":"code","source":"def make_predict_batch(img,export_path):\n    \"\"\"\n    INPUT\n        -`img` list of bytes representing the images to be classified\n        \n    OUTPUT\n        -dataframe containing the probabilities of the labels and the la\n        els as columnames\n    \"\"\"\n    \n    \n    with tf.Session(graph=tf.Graph()) as sess:\n        tf.saved_model.loader.load(sess, ['serve'], export_path)\n        graph = tf.get_default_graph()\n        \n        feed_dict={'Placeholder:0':img}\n        y_pred=sess.run(['Softmax:0','Tile:0'],feed_dict=feed_dict)\n        \n        if len(img)==1:\n            labels=[label.decode() for label in y_pred[1]]\n        else:\n            labels=[label.decode() for label in y_pred[1][0]]\n        \n    return pd.DataFrame(data=y_pred[0],columns=labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The actual prediction is made in the following Part of the above function (inside the `tf.Session`.\n\n`\nfeed_dict={'Placeholder:0':[imageBytearray.getvalue()]}\ny_pred=sess.run(['Softmax:0','Tile:0'],feed_dict=feed_dict)\n`\n\nTo understand How to adopt this for your pre-trained model we have to dive a bit into the structure of the model (see [this Blogpost](https://heartbeat.fritz.ai/automl-vision-edge-exporting-and-loading-tensorflow-saved-models-with-python-f4e8ce1b943a)). In fact we have to identify the input (here: 'Placeholder:0') and output nodes of the graph. Some trial and error can be involved here, especially since the last nodes in this example are not giving the actual prediction but the order of the labels, while the 'Softmax'-node actually gives the probabilities (You can look at the structure of the graph with the webapp [Netron](https://lutzroeder.github.io/netron/)). Lets look at an example prediction"},{"metadata":{"trusted":true},"cell_type":"code","source":"i=0 \nname=f'train_image_data_{i}.parquet'\ntest_img = pd.read_parquet('../input/bengaliai-cv19/'+name)[0:10]\ntest_img.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"height=137\nwidth=236\n#we need the directory of the saved model\ndir_path='../input/trained-models/Trained_Models/tf_saved_model-Bengaliai_vowel-2020-01-27T205839579Z'\n\nimages=test_img.iloc[:, 1:].values.reshape(-1, height, width)\nimage_id=test_img.image_id\nimagebytes=[]\nfor i in range(test_img.shape[0]):\n    imageBytearray=io.BytesIO()\n    imsave(imageBytearray,images[i],format='png')\n    imagebytes.append(imageBytearray.getvalue())\n\nres=make_predict_batch(imagebytes,dir_path)\nres['image_id']=image_id\nres.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To obtain the label run:"},{"metadata":{"trusted":true},"cell_type":"code","source":"res.drop(['image_id'],axis=1).idxmax(axis=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The following Function takes this into account and also formats a submission file following the requirements of the Bengali.Ai competition."},{"metadata":{"trusted":true},"cell_type":"code","source":"#walk the working directory to find the names of the directories\nimport os \ninputFolder = '../input/' \nfor root, directories, filenames in os.walk(inputFolder): \n    for filename in filenames: print(os.path.join(root,filename))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def make_submit(images,height=137,width=236):\n    \"\"\"\n    \n    \"\"\"\n    consonant_path='../input/trained-models/Trained_Models/tf_saved_model-Bengaliai_consonant-2020-01-27T205840376Z'\n    root_path='../input/trained-models/Trained_Models/tf_saved_model-Bengaliai_root-2020-01-27T205838805Z'\n    vowel_path='../input/trained-models/Trained_Models/tf_saved_model-Bengaliai_vowel-2020-01-27T205839579Z'\n    num=images.shape[0]\n    #transform the images from a dataframe to a list of images and then bytes\n    image_id=images.image_id\n    images=images.iloc[:, 1:].values.reshape(-1, height, width)\n    imagebytes=[]\n    for i in range(num):\n        imageBytearray=io.BytesIO()\n        imsave(imageBytearray,images[i],format='png')\n        imagebytes.append(imageBytearray.getvalue())\n    \n    #get the predictions from the three models - passing the bytes_list\n    start_pred=time.time()\n    prediction_root=make_predict_batch(imagebytes,export_path=root_path)\n    prediction_consonant=make_predict_batch(imagebytes,export_path=consonant_path)\n    prediction_vowel=make_predict_batch(imagebytes,export_path=vowel_path)\n    end_pred=time.time()\n    print('Prediction took {} seconds.'.format(end_pred-start_pred))\n    \n    start_sub=time.time()\n    p0=prediction_root.idxmax(axis=1)\n    p1=prediction_vowel.idxmax(axis=1)\n    p2=prediction_consonant.idxmax(axis=1)\n        \n    row_id = []\n    target = []\n    for i in range(len(image_id)):\n        row_id += [image_id.iloc[i]+'_grapheme_root', image_id.iloc[i]+'_vowel_diacritic',image_id.iloc[i]+'_consonant_diacritic']\n        target += [p0[i], p1[i], p2[i]]\n        \n    submission_df = pd.DataFrame({'row_id': row_id, 'target': target})\n    #submission_df.to_csv(name, index=False)\n        \n    end_sub=time.time()\n    print('Writing the submission_df took {} seconds'.format(end_sub-start_sub))\n    return submission_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Finally we can make the submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"with open('submission.csv','w') as sub:\n    writer=csv.writer(sub)\n    writer.writerow(['row_id','target'])\n\nbatchsize=1000\n\nstart = time.time()\nfor i in range(4):\n    start1 = time.time()\n    name=f'test_image_data_{i}.parquet'\n    print('start with '+name+'...')\n    test_img = pd.read_parquet('../input/bengaliai-cv19/'+name)\n    \n    print('starting prediction')\n    start1 = time.time()\n    #split into smaler filesl\n    for r in range(np.ceil(test_img.shape[0]/batchsize).astype(int)):\n            \n        df=make_submit(test_img[r*batchsize:np.minimum((r+1)*batchsize,test_img.shape[0]+1)])\n        df.to_csv('submission.csv',mode='a',index=False,header=False)\n    \n    end1 = time.time()\n    print(end1 - start1)\n    del test_img\n\nend = time.time()\nprint(end - start)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df.head()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}