{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Herbarium2022\n\nThe Herbarium 2022: Flora of North America is a part of a project of the New York Botanical Garden funded by the National Science Foundation to build tools to identify novel plant species around the world. The dataset strives to represent all known vascular plant taxa in North America, using images gathered from 60 different botanical institutions around the world.\n\n\n\n## **Today! I'm trying tensorflow framework**\n\nI'm impressive see this notebook! full credit\nThen refer herbarium 2020 competition: [https://www.kaggle.com/seraphwedd18/herbarium-consolidating-the-details#Submission](http://)\nI love it!\n\n### **Enjoy! Tensorflow**","metadata":{}},{"cell_type":"markdown","source":"## **Path**","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n if filename.endswith('.jpg'):\n break\n print(os.path.join(dirname, filename))","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:01:16.881004Z","iopub.execute_input":"2022-02-17T04:01:16.881277Z","iopub.status.idle":"2022-02-17T04:06:02.647596Z","shell.execute_reply.started":"2022-02-17T04:01:16.881248Z","shell.execute_reply":"2022-02-17T04:06:02.646812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_sub = pd.read_csv('../input/herbarium-2022-fgvc9/sample_submission.csv')\ndisplay(sample_sub)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:02.6493Z","iopub.execute_input":"2022-02-17T04:06:02.649749Z","iopub.status.idle":"2022-02-17T04:06:02.741876Z","shell.execute_reply.started":"2022-02-17T04:06:02.649711Z","shell.execute_reply":"2022-02-17T04:06:02.741158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Load Data ---> json**","metadata":{}},{"cell_type":"code","source":"import json, codecs\n\nwith codecs.open(\"../input/herbarium-2022-fgvc9/train_metadata.json\", 'r',\n encoding='utf-8', errors='ignore') as f:\n train_meta = json.load(f)\n \nwith codecs.open(\"../input/herbarium-2022-fgvc9/test_metadata.json\", 'r',\n encoding='utf-8', errors='ignore') as f:\n test_meta = json.load(f)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:02.744223Z","iopub.execute_input":"2022-02-17T04:06:02.744492Z","iopub.status.idle":"2022-02-17T04:06:14.625466Z","shell.execute_reply.started":"2022-02-17T04:06:02.744456Z","shell.execute_reply":"2022-02-17T04:06:14.624704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(train_meta.keys())","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:14.627274Z","iopub.execute_input":"2022-02-17T04:06:14.627526Z","iopub.status.idle":"2022-02-17T04:06:14.633867Z","shell.execute_reply.started":"2022-02-17T04:06:14.627493Z","shell.execute_reply":"2022-02-17T04:06:14.632985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Analysis the data**\n## **Train_meta ---> Dataframe**","metadata":{}},{"cell_type":"code","source":"train_df = pd.DataFrame(train_meta['annotations'])\ndisplay(train_df)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:14.63507Z","iopub.execute_input":"2022-02-17T04:06:14.63839Z","iopub.status.idle":"2022-02-17T04:06:15.989842Z","shell.execute_reply.started":"2022-02-17T04:06:14.638346Z","shell.execute_reply":"2022-02-17T04:06:15.989179Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_cat = pd.DataFrame(train_meta['categories'])\n#train_cat.columns = [ 'category_id', 'scientificName','family', 'genus']\ndisplay(train_cat)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:15.991182Z","iopub.execute_input":"2022-02-17T04:06:15.991456Z","iopub.status.idle":"2022-02-17T04:06:16.028927Z","shell.execute_reply.started":"2022-02-17T04:06:15.99142Z","shell.execute_reply":"2022-02-17T04:06:16.028301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_img = pd.DataFrame(train_meta['images'])\ntrain_img.columns = ['image_id','file_name', 'license']\ndisplay(train_img)\n","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:16.030113Z","iopub.execute_input":"2022-02-17T04:06:16.031458Z","iopub.status.idle":"2022-02-17T04:06:17.100226Z","shell.execute_reply.started":"2022-02-17T04:06:16.031416Z","shell.execute_reply":"2022-02-17T04:06:17.099561Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_meta.keys()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:17.101528Z","iopub.execute_input":"2022-02-17T04:06:17.1018Z","iopub.status.idle":"2022-02-17T04:06:17.106664Z","shell.execute_reply.started":"2022-02-17T04:06:17.101765Z","shell.execute_reply":"2022-02-17T04:06:17.105948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_gen = pd.DataFrame(train_meta['genera'])\ntrain_gen.columns = ['genus_id', 'genus']\ndisplay(train_gen)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:17.108014Z","iopub.execute_input":"2022-02-17T04:06:17.10846Z","iopub.status.idle":"2022-02-17T04:06:17.127068Z","shell.execute_reply.started":"2022-02-17T04:06:17.108425Z","shell.execute_reply":"2022-02-17T04:06:17.126368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Merge_Important_data's**","metadata":{}},{"cell_type":"code","source":"train_df = train_df.merge(train_cat, on='category_id', how='outer')\ntrain_df = train_df.merge(train_img, on='image_id', how='outer')\ntrain_df = train_df.merge(train_gen, on='genus_id', how='outer')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:17.130791Z","iopub.execute_input":"2022-02-17T04:06:17.130995Z","iopub.status.idle":"2022-02-17T04:06:18.280616Z","shell.execute_reply.started":"2022-02-17T04:06:17.130971Z","shell.execute_reply":"2022-02-17T04:06:18.279848Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.columns\n","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:18.282042Z","iopub.execute_input":"2022-02-17T04:06:18.282285Z","iopub.status.idle":"2022-02-17T04:06:18.28773Z","shell.execute_reply.started":"2022-02-17T04:06:18.282251Z","shell.execute_reply":"2022-02-17T04:06:18.28708Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(train_df.info())\ndisplay(train_df)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:18.289112Z","iopub.execute_input":"2022-02-17T04:06:18.289638Z","iopub.status.idle":"2022-02-17T04:06:19.055211Z","shell.execute_reply.started":"2022-02-17T04:06:18.289552Z","shell.execute_reply":"2022-02-17T04:06:19.054562Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Identify_to_remove_NullData**","metadata":{}},{"cell_type":"code","source":"na = train_df.file_name.isna()\nkeep = [x for x in range(train_df.shape[0]) if not na[x]]\ntrain_df = train_df.iloc[keep]","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:19.058896Z","iopub.execute_input":"2022-02-17T04:06:19.060774Z","iopub.status.idle":"2022-02-17T04:06:23.69625Z","shell.execute_reply.started":"2022-02-17T04:06:19.060734Z","shell.execute_reply":"2022-02-17T04:06:23.695514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.info()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:23.697615Z","iopub.execute_input":"2022-02-17T04:06:23.697863Z","iopub.status.idle":"2022-02-17T04:06:24.325029Z","shell.execute_reply.started":"2022-02-17T04:06:23.69783Z","shell.execute_reply":"2022-02-17T04:06:24.324329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Test_meta ---> DataFrame**","metadata":{}},{"cell_type":"code","source":"test_df = pd.DataFrame(test_meta)\ntest_df.columns = ['file_name', 'image_id', 'license']\nprint(test_df.info())\ndisplay(test_df)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:24.326587Z","iopub.execute_input":"2022-02-17T04:06:24.327076Z","iopub.status.idle":"2022-02-17T04:06:24.596528Z","shell.execute_reply.started":"2022-02-17T04:06:24.327036Z","shell.execute_reply":"2022-02-17T04:06:24.59588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Generate_CSV----> Important_Data**","metadata":{}},{"cell_type":"code","source":"train_df.to_csv('train_data.csv', index=False)\ntest_df.to_csv('test_data.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:24.597699Z","iopub.execute_input":"2022-02-17T04:06:24.598018Z","iopub.status.idle":"2022-02-17T04:06:30.880443Z","shell.execute_reply.started":"2022-02-17T04:06:24.597978Z","shell.execute_reply":"2022-02-17T04:06:30.879736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **EDA ---> NewData**","metadata":{}},{"cell_type":"code","source":"#dataexploration\nprint(\"Total Unique Values for each columns:\")\nprint(\"{0:10s} \\t {1:10d}\".format('train_df', len(train_df)))\nfor col in train_df.columns:\n print(\"{0:10s} \\t {1:10d}\".format(col, len(train_df[col].unique())))\n","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:30.881773Z","iopub.execute_input":"2022-02-17T04:06:30.882039Z","iopub.status.idle":"2022-02-17T04:06:31.570902Z","shell.execute_reply.started":"2022-02-17T04:06:30.882002Z","shell.execute_reply":"2022-02-17T04:06:31.570157Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"family = train_df[['family', 'genus_id', 'scientificName']].groupby(['family','genus_id']).count()\ndisplay(family.describe())","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:31.572043Z","iopub.execute_input":"2022-02-17T04:06:31.573578Z","iopub.status.idle":"2022-02-17T04:06:31.773525Z","shell.execute_reply.started":"2022-02-17T04:06:31.573536Z","shell.execute_reply":"2022-02-17T04:06:31.77272Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Build the Model_TensorflowFramework**","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.models import Model\nfrom tensorflow.keras.layers import Dense, Dropout, Conv2D, MaxPool2D, Flatten, BatchNormalization, Input, concatenate\nfrom tensorflow.keras.optimizers import Adam\nfrom tensorflow.keras.utils import plot_model\nfrom sklearn.model_selection import train_test_split as tts\n\nin_out_size = (120*120) + 3 #We will resize the image to 120*120 and we have 3 outputs\ndef xavier(shape, dtype=None):\n return np.random.rand(*shape)*np.sqrt(1/in_out_size)\n\ndef fg_model(shape, lr=0.001):\n '''Family-Genus model receives an image and outputs two integers indicating both the family and genus index.'''\n i = Input(shape)\n \n x = Conv2D(3, (3, 3), activation='relu', padding='same', kernel_initializer=xavier)(i)\n x = Conv2D(3, (5, 5), activation='relu', padding='same', kernel_initializer=xavier)(x)\n x = MaxPool2D(pool_size=(3, 3), strides=(3,3))(x)\n x = BatchNormalization()(x)\n x = Dropout(0.5)(x)\n x = Conv2D(16, (5, 5), activation='relu', padding='same', kernel_initializer=xavier)(x)\n #x = Conv2D(16, (5, 5), activation='relu', padding='same', kernel_initializer=xavier)(x)\n x = MaxPool2D(pool_size=(5, 5), strides=(5,5))(x)\n x = BatchNormalization()(x)\n x = Dropout(0.5)(x)\n x = Flatten()(x)\n \n o1 = Dense(310, activation='softmax', name='family', kernel_initializer=xavier)(x)\n \n o2 = concatenate([o1, x])\n o2 = Dense(3678, activation='softmax', name='genus_id', kernel_initializer=xavier)(o2)\n \n o3 = concatenate([o1, o2, x])\n o3 = Dense(32094, activation='softmax', name='category_id', kernel_initializer=xavier)(o3)\n \n x = Model(inputs=i, outputs=[o1, o2, o3])\n \n opt = Adam(lr=lr, amsgrad=True)\n x.compile(optimizer=opt, loss=['sparse_categorical_crossentropy', \n 'sparse_categorical_crossentropy', \n 'sparse_categorical_crossentropy'],\n metrics=['accuracy'])\n return x\n\nmodel = fg_model((120, 120, 3))\nmodel.summary()\nplot_model(model, to_file='full_model_plot.png', show_shapes=True, show_layer_names=True)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:31.775045Z","iopub.execute_input":"2022-02-17T04:06:31.775301Z","iopub.status.idle":"2022-02-17T04:06:43.785801Z","shell.execute_reply.started":"2022-02-17T04:06:31.775266Z","shell.execute_reply":"2022-02-17T04:06:43.785005Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Augmentation of Data_Image**","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n\ntrain_datagen = ImageDataGenerator(featurewise_center=False,\n featurewise_std_normalization=False,\n rotation_range=180,\n width_shift_range=0.1,\n height_shift_range=0.1,\n zoom_range=0.2)\n","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:43.78723Z","iopub.execute_input":"2022-02-17T04:06:43.788122Z","iopub.status.idle":"2022-02-17T04:06:43.795411Z","shell.execute_reply.started":"2022-02-17T04:06:43.788079Z","shell.execute_reply":"2022-02-17T04:06:43.794736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"m = train_df[['file_name', 'family', 'genus_id', 'category_id']]\nfam = m.family.unique().tolist()\nm.family = m.family.map(lambda x: fam.index(x))\ngen = m.genus_id.unique().tolist()\nm.genus_id = m.genus_id.map(lambda x: gen.index(x))\ndisplay(m)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:43.796946Z","iopub.execute_input":"2022-02-17T04:06:43.797588Z","iopub.status.idle":"2022-02-17T04:06:58.686914Z","shell.execute_reply.started":"2022-02-17T04:06:43.797552Z","shell.execute_reply":"2022-02-17T04:06:58.686175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Train_data_TensorflowModel_epoch_2**","metadata":{}},{"cell_type":"code","source":"train, verif = tts(m, test_size=0.2, shuffle=True, random_state=17)\ntrain = train[:40000]\nverif = verif[:10000]\nshape = (120, 120, 3)\nepochs = 2\nbatch_size = 32\n\nmodel = fg_model(shape, 0.007)\n\n#Disable the last two output layers for training the Family\nfor layers in model.layers:\n if layers.name == 'genus_id' or layers.name=='category_id':\n layers.trainable = False\n\n#Train Family for 2 epochs\nmodel.fit_generator(train_datagen.flow_from_dataframe(dataframe=train,\n directory='../input/herbarium-2022-fgvc9/train_images',\n x_col=\"file_name\",\n y_col=[\"family\", \"genus_id\", \"category_id\"],\n target_size=(120, 120),\n batch_size=batch_size,\n class_mode='multi_output'),\n validation_data=train_datagen.flow_from_dataframe(\n dataframe=verif,\n directory='../input/herbarium-2022-fgvc9/train_images',\n x_col=\"file_name\",\n y_col=[\"family\", \"genus_id\", \"category_id\"],\n target_size=(120, 120),\n batch_size=batch_size,\n class_mode='multi_output'),\n epochs=epochs,\n steps_per_epoch=len(train)//batch_size,\n validation_steps=len(verif)//batch_size,\n verbose=1,\n workers=8,\n use_multiprocessing=False)\n\n#Reshuffle the inputs\ntrain, verif = tts(m, test_size=0.2, shuffle=True, random_state=17)\ntrain = train[:40000]\nverif = verif[:10000]\n\n#Make the Genus layer Trainable\nfor layers in model.layers:\n if layers.name == 'genus_id':\n layers.trainable = True\n \n#Train Family and Genus for 2 epochs\nmodel.fit_generator(train_datagen.flow_from_dataframe(dataframe=train,\n directory='../input/herbarium-2022-fgvc9/train_images',\n x_col=\"file_name\",\n y_col=[\"family\", \"genus_id\", \"category_id\"],\n target_size=(120, 120),\n batch_size=batch_size,\n class_mode='multi_output'),\n validation_data=train_datagen.flow_from_dataframe(\n dataframe=verif,\n directory='../input/herbarium-2022-fgvc9/train_images',\n x_col=\"file_name\",\n y_col=[\"family\", \"genus_id\", \"category_id\"],\n target_size=(120, 120),\n batch_size=batch_size,\n class_mode='multi_output'),\n epochs=epochs,\n steps_per_epoch=len(train)//batch_size,\n validation_steps=len(verif)//batch_size,\n verbose=1,\n workers=8,\n use_multiprocessing=False)\n\n#Reshuffle the inputs\ntrain, verif = tts(m, test_size=0.2, shuffle=True, random_state=17)\ntrain = train[:40000]\nverif = verif[:10000]\n\n#Make the category_id layer Trainable\nfor layers in model.layers:\n if layers.name == 'category_id':\n layers.trainable = True\n \n#Train them all for 2 epochs\nmodel.fit_generator(train_datagen.flow_from_dataframe(dataframe=train,\n directory='../input/herbarium-2022-fgvc9/train_images',\n x_col=\"file_name\",\n y_col=[\"family\", \"genus_id\", \"category_id\"],\n target_size=(120, 120),\n batch_size=batch_size,\n class_mode='multi_output'),\n validation_data=train_datagen.flow_from_dataframe(\n dataframe=verif,\n directory='../input/herbarium-2022-fgvc9/train_images',\n x_col=\"file_name\",\n y_col=[\"family\", \"genus_id\", \"category_id\"],\n target_size=(120, 120),\n batch_size=batch_size,\n class_mode='multi_output'),\n epochs=epochs,\n steps_per_epoch=len(train)//batch_size,\n validation_steps=len(verif)//batch_size,\n verbose=1,\n workers=8,\n use_multiprocessing=False)\n\n'''\nfor i in range(epochs):\n n = 1\n for X, Y in train_datagen.flow_from_dataframe(dataframe=train,\n directory='../input/herbarium-2020-fgvc7/nybg2020/train/',\n x_col=\"file_name\",\n y_col=[\"family\", \"genus\", \"category_id\"],\n target_size=(120, 120),\n batch_size=batch_size,\n class_mode='multi_output'):\n model.train_on_batch(X, Y, reset_metrics=False)\n loss, fam_loss, gen_loss, cat_loss, fam_acc, gen_acc, cat_acc = model.evaluate(X, Y, verbose=False)\n if n%10==0:\n print(f\"For epoch {i} batch {n}: {loss}, {fam_loss}, {gen_loss}, {cat_loss}, {fam_acc}, {gen_acc}, {cat_acc}\")\n for layers in model.layers:\n if layers.name == 'family' and fam_acc>0.90:\n layers.trainable=False\n elif layers.name == 'genus':\n if fam_acc>0.75:\n layers.trainable=True\n else:\n layers.trainable=False\n elif layers.name == 'category_id':\n if fam_acc>0.75 and gen_acc>0.5:\n layers.trainable=True\n else:\n layers.trainable=False\n n += 1\n'''","metadata":{"execution":{"iopub.status.busy":"2022-02-17T04:06:58.688325Z","iopub.execute_input":"2022-02-17T04:06:58.688566Z","iopub.status.idle":"2022-02-17T05:04:09.193958Z","shell.execute_reply.started":"2022-02-17T04:06:58.688533Z","shell.execute_reply":"2022-02-17T05:04:09.191465Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.save('fg_model.h5')","metadata":{"execution":{"iopub.status.busy":"2022-02-17T05:04:09.196042Z","iopub.execute_input":"2022-02-17T05:04:09.19697Z","iopub.status.idle":"2022-02-17T05:04:24.269425Z","shell.execute_reply.started":"2022-02-17T05:04:09.196894Z","shell.execute_reply":"2022-02-17T05:04:24.268551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Test_data**","metadata":{}},{"cell_type":"code","source":"test_df.info()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T05:04:24.275838Z","iopub.execute_input":"2022-02-17T05:04:24.27704Z","iopub.status.idle":"2022-02-17T05:04:24.401081Z","shell.execute_reply.started":"2022-02-17T05:04:24.276995Z","shell.execute_reply":"2022-02-17T05:04:24.400355Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df.iloc[:10000]","metadata":{"execution":{"iopub.status.busy":"2022-02-17T05:04:24.402608Z","iopub.execute_input":"2022-02-17T05:04:24.402866Z","iopub.status.idle":"2022-02-17T05:04:24.422171Z","shell.execute_reply.started":"2022-02-17T05:04:24.402829Z","shell.execute_reply":"2022-02-17T05:04:24.421359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"batch_size = 32\ntest_datagen = ImageDataGenerator(featurewise_center=False,\n featurewise_std_normalization=False)\ntest = test_df.iloc[:10000]\ngenerator = test_datagen.flow_from_dataframe(\n dataframe = test, #Limiting the test to the first 10,000 items\n directory = '../input/herbarium-2022-fgvc9/test_images',\n x_col = 'image_id',\n target_size=(120, 120),\n batch_size=batch_size,\n class_mode=None, # only data, no labels\n shuffle=False\n)\n\nfamily,genus_id,category_id = model.predict(generator, verbose=1)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T05:04:24.42388Z","iopub.execute_input":"2022-02-17T05:04:24.424146Z","iopub.status.idle":"2022-02-17T05:07:24.560834Z","shell.execute_reply.started":"2022-02-17T05:04:24.424111Z","shell.execute_reply":"2022-02-17T05:07:24.559995Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Submission_f**","metadata":{}},{"cell_type":"code","source":"sub = pd.DataFrame()\nsub['Id'] = test_df.file_name\nsub['Id'] = sub['Id'].astype('int64')\nsub['Predicted'] = np.concatenate([np.argmax(category_id, axis=1), 23718*np.ones((len(test_df.image_id)-len(category_id)))], axis=0)\nsub['Predicted'] = sub['Predicted'].astype('int32')\ndisplay(sub)\nsub.to_csv('category_submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T05:17:44.922153Z","iopub.execute_input":"2022-02-17T05:17:44.922473Z","iopub.status.idle":"2022-02-17T05:17:45.700176Z","shell.execute_reply.started":"2022-02-17T05:17:44.92244Z","shell.execute_reply":"2022-02-17T05:17:45.699423Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub['Predicted'] = np.concatenate([np.argmax(family, axis=1), np.zeros((len(test_df.image_id)-len(family)))], axis=0)\nsub['Predicted'] = sub['Predicted'].astype('int32')\ndisplay(sub)\nsub.to_csv('family_submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T05:18:06.431238Z","iopub.execute_input":"2022-02-17T05:18:06.431775Z","iopub.status.idle":"2022-02-17T05:18:06.733943Z","shell.execute_reply.started":"2022-02-17T05:18:06.431738Z","shell.execute_reply":"2022-02-17T05:18:06.733274Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub['Predicted'] = np.concatenate([np.argmax(genus_id, axis=1), np.zeros((len(test_df.image_id)-len(genus_id)))], axis=0)\nsub['Predicted'] = sub['Predicted'].astype('int32')\ndisplay(sub)\nsub.to_csv('genus_submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T05:18:50.945927Z","iopub.execute_input":"2022-02-17T05:18:50.946473Z","iopub.status.idle":"2022-02-17T05:18:51.298613Z","shell.execute_reply.started":"2022-02-17T05:18:50.946434Z","shell.execute_reply":"2022-02-17T05:18:51.297881Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## ***Thankyou for visiting guys! feel free to use---->starter***\n\nReference:\n1. [https://www.kaggle.com/seraphwedd18/herbarium-consolidating](http://) ---> **Full_credit**\n2. [https://www.kaggle.com/venkatkumar001/herbarium-22-fgvc9-baseline](http://)","metadata":{}}]}