{"cells":[{"metadata":{},"cell_type":"markdown","source":"# **CNN HyperNetwork**"},{"metadata":{},"cell_type":"markdown","source":"#### The idea behind this CNN HyperNetwork is to take the data classification given in this competition for each species and create a set of \"Upper Neural Networks\". There will be 6 upper networks, each of which will predict the  pictures class, family, genus, kingdom, order, phylum. Using this classification, there will then be a set of \"Lower Neural Networks\",where the predictions of the upper networks will feed into the group predicted by the network in order to specify the species in the group. Essentialy this method is taking a picture and making the classification simplier for the entire network to get a better accuracy.\n\nThis is a design layout of the HyperNetwork:\n![Hyper Neural Network Design](https://scontent-sjc3-1.xx.fbcdn.net/v/t1.15752-9/61281631_395800384479998_6138768229000019968_n.png?_nc_cat=101&_nc_oc=AQnU4YB-zVRHBTVfZDc7UthRTfCH9P8Tu0H5wkt1vVR-Dl_PIuICVYepBdmcPuCF1SwlU1zQrrWMqyJK23FNzuIu&_nc_ht=scontent-sjc3-1.xx&oh=56f83dd4005f141c121f4520a6726421&oe=5D9D316F)\n\nThis is how the HyperNetwork will traing on all these diffrent models:\n![Theory of HyperNetwork](https://scontent-sjc3-1.xx.fbcdn.net/v/t1.15752-9/60771839_643704266054313_6017675504444768256_n.png?_nc_cat=104&_nc_oc=AQl_3C0S_UcB_OisLMNymFV137wd65OMNsaQJgUe-S5E1Lz7UQFWhT1TyvQGSc7cIwayDfpTOkh8bGsaP2HCTEAx&_nc_ht=scontent-sjc3-1.xx&oh=ff9a5e5826b217a01dbfcc4ed5dba12c&oe=5D820A61)\n"},{"metadata":{},"cell_type":"markdown","source":"These are all the libraries that will be used throught this Kernel:"},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"from __future__ import absolute_import, division, print_function\nfrom skimage import transform \nfrom skimage.color import rgb2gray\nfrom PIL import Image\nfrom resizeimage import resizeimage\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.preprocessing import LabelEncoder\n\nimport os\nimport pickle\nimport json\nimport tensorflow as tf\nimport numpy as np\nimport pandas as pd","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Step 1: Organize Data\n#### The initial step is to take all the data that is currently given through this competition, in the JSON files, and organize all the data that has been given."},{"metadata":{},"cell_type":"markdown","source":"First, taking the values of all the Json files and loading the important parts of the files into Data Frames to be used to create Data Set with all the species classification. Then each column in this data set is encoded to be used for training part of the Upper Network."},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"# open open category to id file\nwith open('val2019.json') as f:\n    values = json.load(f)\n\n#open id to images file\nwith open('train2019.json') as f:\n    images = json.load(f)\n\nannotation_images = pd.DataFrame.from_dict(images[\"annotations\"])\ncatagory_images = pd.DataFrame.from_dict(images[\"categories\"])\nimages_id = pd.DataFrame.from_dict(images[\"images\"])\n\ntrain_data = pd.DataFrame({'ImageId':None,'Class':None,'Family':None,'Genus':None,'Kingdom':None,'Name':None,'Order':None,'Phylum':None}, index = [0])\n\nscaled_features = catagory_images.copy()\ncol_names = ['class', 'family','kingdom','order','phylum']\nfeatures = catagory_images[col_names]\nfor i in range(0,7):\n    labelencoder = LabelEncoder()\n    features.values[:,i] = labelencoder.fit_transform(features.values[:,i])\n\nscaled_features[col_names] = features\n\nindex = 0;\nfor t in annotation_images.category_id:\n    current_info = pd.DataFrame(scaled_features.loc[t]).T\n    \n    train_data.loc[index] = [index, current_info[\"class\"].values[0], current_info[\"family\"].values[0], current_info[\"genus\"].values[0], current_info[\"kingdom\"].values[0], current_info[\"name\"].values[0], current_info[\"order\"].values[0], current_info[\"phylum\"].values[0]]\n    print(index)\n    index += 1\n\ntrain_data = train_data.assign(File_Name=images_id['file_name'].values)\n\ntrain_data.to_csv(r'/upperNN.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next, the Lower training set is created to have the data sorted by per speices in order to feed the Lower Network the training data."},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"lower_train_data = pd.DataFrame({'FileName': images_id['file_name'],'ImageId': annotation_images['id'], 'CatagoryID':annotation_images['category_id']})\n\nlower_train_data = lower_train_data.sort_values(by=['CatagoryID'])\n\nlower_train_data.set_index(keys=['CatagoryID'], drop=False,inplace=True)\n\n# get a list of names\nnumbers=lower_train_data['CatagoryID'].unique().tolist()\n\nfor number in numbers:\n    data = lower_train_data.loc[lower_train_data['CatagoryID']==number]\n\n    data.to_csv( str(number) + '.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Last, in order to create all of the lower models there needs to be grouping for all the species in the data set based on the Class, Family, Genus, Kingdom, Order, Phylum. Each of these groupings will create an individual model for the overall Hypernetwork."},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"# now we can perform a lookup on a 'view' of the dataframe\n\nordered_features = scaled_features.sort_values(['class', 'family', 'genus', 'kingdom', 'order', 'phylum'])\n\nfeatures = ['class', 'family', 'genus', 'kingdom', 'order', 'phylum']\n\nclasst = scaled_features['class'].unique().tolist()\nfamily = scaled_features['family'].unique().tolist()\ngenus = scaled_features['genus'].unique().tolist()\nkingdom = scaled_features['kingdom'].unique().tolist()\norder = scaled_features['order'].unique().tolist()\nphylum = scaled_features['phylum'].unique().tolist()\n\nfor a in kingdom:\n    for b in phylum:\n        for c in classt:\n            for d in order:\n                for e in family:\n                        current = ordered_features['id'].loc[(ordered_features['kingdom'] == a) & (ordered_features['phylum'] == b) & (ordered_features['class'] == c) & (ordered_features['order'] == d) & (ordered_features['family'] == e)]\n                        if len(current) != 0:\n                            print(len(current))\n                            current.to_csv(str(a) + '_' + str(b) + '_' + str(c) + '_' + str(d) + '_' + str(e) + '.csv', index=False)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Step 2: Converting Images\n#### The next step in the Hypernetwork is to convert all the Images given into a format that can be used to train the models in pixel values"},{"metadata":{},"cell_type":"markdown","source":"The code below takes all the files Names through the upperNN.csv file and resizing the image to 75 x 75 and grayscaling the images. Because the data is so large required the data to be split into 11 diffrent files because of how large this was."},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#this is how we can create a data file with all the images in a 1,75,75 shape \n#modify the for loop in order to use it and the allImages are all the images file names\n#update the .open method to the directory of the train images then the + im\n# the Images variable will contain all the picures each resized to 75 by 75\n\nupperNN = pd.read_csv('upperNN .csv')\n\nfile_names = upperNN['File_Name']\n\nImages = np.empty([1,75, 75])\ni = 0\na = 0\nfor im in file_names:\n        if i >= 5000:\n            Images = Images[1:]\n            output = open(str(a)+'.pkl', 'wb')\n            pickle.dump(Images, output)\n            output.close()            \n            Images = np.empty([1,75, 75])\n            a += 1\n            i = 0\n        img = Image.open(im, 'r').convert('LA')\n        cover = resizeimage.resize_cover(img, [75, 75], validate=False)\n        np_im = np.array(cover)\n    \n        pix_val_flat = np.array([x for sets in np_im for x in sets])\n        train_data = pix_val_flat[:,0].astype('float64') /255\n        train_data = np.resize(train_data, (1,75,75))\n        \n        Images = np.concatenate((Images,train_data))\n        i += 1\n        print(i)\n        \nImages = Images[1:]\noutput = open(str(a)+'.pkl', 'wb')\npickle.dump(Images, output)\noutput.close()\n        \nImages = np.empty([1,75, 75])\ni = 0\na = 54\n\nfor num in range(54,65):\n    if i == 5:\n        Images = Images[1:]\n        output = open(str(a)+'.pkl', 'wb')\n        pickle.dump(Images, output)\n        output.close()            \n        Images = np.empty([1,75, 75])\n        i = 0\n        a += 1\n    pkl_file = open(str(num) + '.pkl', 'rb')\n\n    data1 = pickle.load(pkl_file)\n\n    Images = np.concatenate((Images,data1))\n    \n    pkl_file.close()\n    print(i)\n    i += 1\nImages = Images[1:]\noutput = open(str(a)+'.pkl', 'wb')\npickle.dump(Images, output)\noutput.close()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next, After all the image data has been converted to pixel values, I sorted all the pictures based on their picture ID for the Lower Network Train data as each picture will be sorted by the specie ID."},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"Images = np.empty([1,75, 75])\ni = 0\nfor num in range(0,1010):\n    current = pd.read_csv('Lower_NN_Data/' + str(num) + '.csv')\n    for id in current['ImageId']:\n        file_num = int(id / 25000)\n        index = id - (file_num * 25000)\n        if file_num == 0:\n            im = data1[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 1:\n            im = data2[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 2:\n            im = data3[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 3:\n            im = data4[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 4:\n            im = data5[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 5:\n            im = data6[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 6:\n            im = data7[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 7:\n            im = data8[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 8:\n            im = data9[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 9:\n            im = data10[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n        if file_num == 10:\n            im = data11[index].reshape(1,75,75)\n            Images = np.concatenate((Images,im))\n    Images = Images[1:]\n    output = open(str(i) + '.pkl', 'wb')\n    pickle.dump(Images, output)\n    output.close()\n    Images = np.empty([1,75, 75])\n    print(i)\n    i += 1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Step 3: Tensorflow Models\n#### This part of the proccess is the Tensorflow models for the lower and the upper network models"},{"metadata":{},"cell_type":"markdown","source":"The way this code is written is so that creating a model is automated by calling on the CNN function with the arguments of train_data, train_val, eval_data, eval_labels, output_nodes_number, model_name, and model_type (lower or upper). After this function finishes training on 2000 steps it will save the model in the local directory with the model_name as its title."},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"tf.logging.set_verbosity(tf.logging.INFO)\n\ndef CNN(train_data, train_labels, eval_data, eval_labels, output_nodes_number, model_name, model_type):\n      \n    if model_type == \"lower\":\n        #tensorflow model function for lowerNN\n        def cnn_model(features, labels, mode):\n            \n            input_layer = tf.reshape(features[\"x\"], [-1, 75, 75,1])\n            \n            # Convolutional Layer #1\n            conv1 = tf.layers.conv2d(\n                  inputs=input_layer,\n                  filters=32,\n                  kernel_size=[5, 5],\n                  padding=\"same\",\n                  activation=tf.nn.relu)\n            \n              # Pooling Layer #1\n            pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2)\n            \n              # Convolutional Layer #2 and Pooling Layer #2\n            conv2 = tf.layers.conv2d(\n                  inputs=pool1,\n                  filters=64,\n                  kernel_size=[5, 5],\n                  padding=\"same\",\n                  activation=tf.nn.relu)\n            pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2)\n            \n              # Dense Layer\n            pool2_flat = tf.reshape(pool2, [-1,  18 * 18 * 64])\n            dense = tf.layers.dense(inputs=pool2_flat, units=1024, activation=tf.nn.relu)\n            dropout = tf.layers.dropout(\n                  inputs=dense, rate=0.4, training=mode == tf.estimator.ModeKeys.TRAIN)\n            \n              # Logits Layer\n            logits = tf.layers.dense(inputs=dropout, units=output_nodes_number)\n            \n            predicted_classes =tf.argmax(input=logits, axis=1)\n            predictions = {\n                        'class_ids': predicted_classes[:, tf.newaxis],\n                        'probabilities': tf.nn.softmax(logits, name=\"softmax_tensor\"),\n                        'logits': logits,\n                    }\n            export_outputs = {\n              'prediction': tf.estimator.export.PredictOutput(predictions)\n              }\n            if mode == tf.estimator.ModeKeys.PREDICT:  \n                return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions, export_outputs=export_outputs)\n            \n              # Calculate Loss (for both TRAIN and EVAL modes)\n            loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)            \n              # Configure the Training Op (for TRAIN mode)\n            if mode == tf.estimator.ModeKeys.TRAIN:\n                optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.0001)\n                train_op = optimizer.minimize(\n                    loss=loss,\n                    global_step=tf.train.get_global_step())\n                return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op)\n            \n              # Add evaluation metrics (for EVAL mode)\n            eval_metric_ops = {\n                  \"accuracy\": tf.metrics.accuracy(\n                      labels=labels, predictions=predictions[\"class_ids\"])\n            }\n            return tf.estimator.EstimatorSpec(\n                  mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)\n    \n    if model_type == \"upper\":\n        #tensorflow model function for upperNN\n        def cnn_model(features, labels, mode):\n            \n            input_layer = tf.reshape(features[\"x\"], [-1, 75, 75, 1])\n            \n            # Convolutional Layer #1\n            conv1 = tf.layers.conv2d(\n                  inputs=input_layer,\n                  filters=32,\n                  kernel_size=[5, 5],\n                  padding=\"same\",\n                  activation=tf.nn.relu)\n            \n              # Pooling Layer #1\n            pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2)\n            \n              # Convolutional Layer #2 and Pooling Layer #2\n            conv2 = tf.layers.conv2d(\n                  inputs=pool1,\n                  filters=64,\n                  kernel_size=[5, 5],\n                  padding=\"same\",\n                  activation=tf.nn.relu)\n            pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2)\n            \n              # Dense Layer\n            pool2_flat = tf.reshape(pool2, [-1, 18 * 18 * 64])\n            dense = tf.layers.dense(inputs=pool2_flat, units=1024, activation=tf.nn.relu)\n            dropout = tf.layers.dropout(\n                  inputs=dense, rate=0.4, training=mode == tf.estimator.ModeKeys.TRAIN)\n            \n              # Logits Layer\n            logits = tf.layers.dense(inputs=dropout, units=output_nodes_number)\n            \n            predicted_classes =tf.argmax(input=logits, axis=1)\n            predictions = {\n                        'class_ids': predicted_classes[:, tf.newaxis],\n                        'probabilities': tf.nn.softmax(logits, name=\"softmax_tensor\"),\n                        'logits': logits,\n                    }\n            export_outputs = {\n              'prediction': tf.estimator.export.PredictOutput(predictions)\n              }\n            if mode == tf.estimator.ModeKeys.PREDICT:  \n                return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions, export_outputs=export_outputs)\n            \n              # Calculate Loss (for both TRAIN and EVAL modes)\n            loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)            \n            \n              # Configure the Training Op (for TRAIN mode)\n            if mode == tf.estimator.ModeKeys.TRAIN:\n                optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.0001)\n                train_op = optimizer.minimize(\n                    loss=loss,\n                    global_step=tf.train.get_global_step())\n                return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op)\n            \n              # Add evaluation metrics (for EVAL mode)\n            eval_metric_ops = {\n                  \"accuracy\": tf.metrics.accuracy(\n                      labels=labels, predictions=predictions[\"class_ids\"])\n            }\n            return tf.estimator.EstimatorSpec(\n                  mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)\n    #This is where we need to load up the data for each group\n\n    ModelDir = model_name\n    # Create the Estimator\n    \n    run_config = tf.contrib.learn.RunConfig(\n    model_dir=ModelDir,\n    keep_checkpoint_max=1)\n    \n    cnn_classifier = tf.estimator.Estimator(\n        model_fn=cnn_model, config=run_config)\n\n    # Set up logging for predictions\n    tensors_to_log = {\"probabilities\": \"softmax_tensor\"}\n\n    logging_hook = tf.train.LoggingTensorHook(\n        tensors=tensors_to_log, every_n_iter=50)\n\n    # Train the model\n    train_input_fn = tf.estimator.inputs.numpy_input_fn(\n        x={\"x\": train_data},\n        y=train_labels,\n        batch_size=100,\n        num_epochs=None,\n        shuffle=True)\n\n    #change steps to 20000\n    cnn_classifier.train(input_fn=train_input_fn, steps=2000, hooks=[logging_hook])\n\n    # Evaluation of the neural network\n    eval_input_fn = tf.estimator.inputs.numpy_input_fn(\n        x={\"x\": eval_data},\n        y=eval_labels,\n        num_epochs=1,\n        shuffle=False)\n\n    eval_results = cnn_classifier.evaluate(input_fn=eval_input_fn)\n    print(eval_results)\n    \n        #instead of predicting on a test data set we will save the model\n    #model_dir = cnn_classifier.export_savedmodel(\n       # model_name,\n        #serving_input_receiver_fn=serving_input_receiver_fn)\n\n    return  ModelDir","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def cnn_model_test(features, labels, mode):\n            \n            input_layer = tf.reshape(features[\"x\"], [-1, 75, 75,1])\n            \n            # Convolutional Layer #1\n            conv1 = tf.layers.conv2d(\n                  inputs=input_layer,\n                  filters=32,\n                  kernel_size=[5, 5],\n                  padding=\"same\",\n                  activation=tf.nn.relu)\n            \n              # Pooling Layer #1\n            pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2)\n            \n              # Convolutional Layer #2 and Pooling Layer #2\n            conv2 = tf.layers.conv2d(\n                  inputs=pool1,\n                  filters=64,\n                  kernel_size=[5, 5],\n                  padding=\"same\",\n                  activation=tf.nn.relu)\n            pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2)\n            \n              # Dense Layer\n            pool2_flat = tf.reshape(pool2, [-1,  18 * 18 * 64])\n            dense = tf.layers.dense(inputs=pool2_flat, units=1024, activation=tf.nn.relu)\n            dropout = tf.layers.dropout(\n                  inputs=dense, rate=0.4, training=mode == tf.estimator.ModeKeys.TRAIN)\n            \n              # Logits Layer\n            logits = tf.layers.dense(inputs=dropout, units=1)\n            \n            predicted_classes =tf.argmax(input=logits, axis=1)\n            predictions = {\n                        'class_ids': predicted_classes[:, tf.newaxis],\n                        'probabilities': tf.nn.softmax(logits, name=\"softmax_tensor\"),\n                        'logits': logits,\n                    }\n            export_outputs = {\n              'prediction': tf.estimator.export.PredictOutput(predictions)\n              }\n            if mode == tf.estimator.ModeKeys.PREDICT:  \n                return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions, export_outputs=export_outputs)\n            \n              # Calculate Loss (for both TRAIN and EVAL modes)\n            loss = tf.losses.sigmoid_cross_entropy(multi_class_labels=labels, logits = logits)\n            \n              # Configure the Training Op (for TRAIN mode)\n            if mode == tf.estimator.ModeKeys.TRAIN:\n                optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.0001)\n                train_op = optimizer.minimize(\n                    loss=loss,\n                    global_step=tf.train.get_global_step())\n                return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op)\n            \n              # Add evaluation metrics (for EVAL mode)\n            eval_metric_ops = {\n                  \"accuracy\": tf.metrics.accuracy(\n                      labels=labels, predictions=predictions[\"class_ids\"])\n            }\n            return tf.estimator.EstimatorSpec(\n                  mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Step 4: Training Model\n#### This step is required to take all the sorted data and being able to train the models on the train data and labels data in order to create this hypernetwork."},{"metadata":{},"cell_type":"markdown","source":"The code below is written as an automation to create all the Upper Neural Network. A model is created for every one of the col_names and is trained and evaluated on all the training data provided."},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#just a start on how to automate creating all the models for the upper NN\n\n#creating a basic model to start the model training\n# this code chunk will auto run and create a model trained on all the data for the species classification\n# this is all automated and may take a long time to create al 5 models as it has to go through\n#25000 images 11 times as the data is split up inot 11 files for each model\n# bellow is the code for Upper NN to give to ur dad to run\ncol_names = ['Class', 'Family','Kingdom','Order','Phylum']\n#, 'Family','Kingdom','Order','Phylum'\na = 0\nb = 1\noutput_nodes = 0\nfor cat in col_names:\n    \n    pkl_file = open('Data/UpperNN_data/64.pkl', 'rb')\n    data1 = pickle.load(pkl_file)\n    \n    upperNN = pd.read_csv('Data/upperNN.csv')\n    Labels = upperNN[cat][249999:265214]\n    output_nodes = upperNN[cat].unique().size\n    \n    \n    #this splits the data into training and val data for the model and also reshapes the label data\n    X_train, X_val, y_train, y_val = train_test_split(data1, Labels, test_size = 0.05, random_state = 0)\n    y_train = np.asarray(y_train).reshape((-1,1))\n    y_val = np.asarray(y_val).reshape((-1,1))\n    \n    model_location = CNN(X_train,y_train,X_val,y_val,output_nodes, str(cat), \"upper\")\n\n    ##need prediction values\n\n    for i in range(54,64):\n        \n        pkl_file = open('Data/UpperNN_data/' + str(i) + '.pkl', 'rb')\n        data1 = pickle.load(pkl_file)\n    \n        upperNN = pd.read_csv('Data/upperNN.csv')\n        Labels = upperNN[cat][a * 25000:b * 25000]\n        a += 1\n        b += 1\n    \n        #this splits the data into training and val data for the model and also reshapes the label data\n        X_train, X_val, y_train, y_val = train_test_split(data1, Labels, test_size = 0.05, random_state = 0)\n        y_train = np.asarray(y_train).reshape((-1,1))\n        y_val = np.asarray(y_val).reshape((-1,1))\n                    \n        #this session will open up any saved model created in directory and will run prediction on that\n        # you can also train with it using the training lines\n        with tf.Session() as sess:\n          # Restore variables from disk.\n            currentCheckpoint = tf.train.latest_checkpoint(cat)\n            saver = tf.train.import_meta_graph(currentCheckpoint + \".meta\")\n            saver.restore(sess, currentCheckpoint)\n            print(\"Model restored.\")\n                  \n            sunspot_classifier = tf.estimator.Estimator(\n            model_fn=cnn_model_test, model_dir=cat)\n                \n                # Set up logging for predictions\n                # Log the values in the \"Softmax\" tensor with label \"probabilities\"\n            tensors_to_log = {\"probabilities\": \"softmax_tensor\"}\n            logging_hook = tf.train.LoggingTensorHook(\n                tensors=tensors_to_log, every_n_iter=50)\n                  \n                  ## train here\n            train_input_fn = tf.estimator.inputs.numpy_input_fn(\n                           x={\"x\": X_train},\n                           y=y_train,\n                           batch_size=100,\n                           num_epochs=None,\n                           shuffle=True)\n        \n                #change steps to 20000\n            sunspot_classifier.train(input_fn=train_input_fn, steps=2000)\n            \n                # Evaluation of the neural network\n            eval_input_fn = tf.estimator.inputs.numpy_input_fn(\n                    x={\"x\": X_val},\n                    y=y_val,\n                    num_epochs=1,\n                    shuffle=False)\n            \n            eval_results = sunspot_classifier.evaluate(input_fn=eval_input_fn)\n            print(eval_results)\n    if b == 11:\n        break","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next, the code below is written to automatically train a model based on all the species grouping in the Sorted_species file, creating a set of training and eval data for the model to be created and saved in the current directory to be used for predicitons."},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#this will be the lower NN code to run\n#this code is very basic where it will look into the Data/Sorted_species files one by one as these\n# files contains the groups for all the species and it will train one model for each of the groupings\n# once read it will look at all the species ids and then look into the Data/Train_data file to get all \n# the picture data and store itinto train_data for each species\n# the code will also take the labels from the Data/Lower_NN_Data files to create the Labels for the training\n# then after the code splits the data created into training and validation data to train the model\noutput_nodes_number = 0\n\nfor filename in os.listdir(\"Data/Sorted_species\"):\n    if filename.endswith(\".csv\"):\n        data = pd.read_csv(\"Data/Sorted_species/\" + filename, names=['a'])\n        output_nodes_number = data.size \n        name = os.path.splitext(filename)[0]\n        train_data = np.empty([1,75, 75])\n        Labels = np.empty([1])\n        for species in data['a']:\n                \n            pkl_file = open(\"Data/Train_data/\" + str(species) + '.pkl', 'rb')\n            data1 = pickle.load(pkl_file)\n            train_data = np.concatenate((train_data,data1))\n            pkl_file.close()\n             \n            current_labels = pd.read_csv(\"Data/Lower_NN_Data/\" + str(species) + '.csv')[\"CatagoryID\"]\n            Labels = np.concatenate((Labels,current_labels.values))\n        \n        Labels = Labels[1:]\n        train_data = train_data[1:]\n         \n        labelencoder = LabelEncoder()\n        Labels[0:] = labelencoder.fit_transform(Labels[0:])\n\n        X_train, X_val, y_train, y_val = train_test_split(train_data, Labels, test_size = 0.20, random_state = 0)\n        y_train = np.asarray(y_train).astype('int32').reshape((-1,1))\n        y_val = np.asarray(y_val).astype('int32').reshape((-1,1))\n        \n        model_location = CNN(X_train,y_train,X_val,y_val,output_nodes_number, name, \"lower\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Part 5: Predictions\n#### Now that all the models have been created the last step is to predict all the test data using all of these models"},{"metadata":{},"cell_type":"markdown","source":"First, convert all the image files to the same style of the Test data."},{"metadata":{"_kg_hide-output":true,"trusted":true,"collapsed":true},"cell_type":"code","source":"# open open category to id file\nwith open('test2019.json') as f:\n    test = json.load(f)\n    \nTest_Data = pd.DataFrame.from_dict(test[\"file_name\",\"id\"])\n\nfile_names = Test_data['file_name']\n\nImages = np.empty([1,75, 75])\ni = 0\nfor im in file_names:\n        img = Image.open(im, 'r').convert('LA')\n        cover = resizeimage.resize_cover(img, [75, 75], validate=False)\n        np_im = np.array(cover)\n    \n        pix_val_flat = np.array([x for sets in np_im for x in sets])\n        train_data = pix_val_flat[:,0].astype('float64') /255\n        train_data = np.resize(train_data, (1,75,75))\n        \n        Images = np.concatenate((Images,train_data))\n        i += 1\n        print(i)\n        \nTestData = Images[1:]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"col_names = ['Class', 'Family','Kingdom','Order','Phylum']\npred = np.empty([1])\n\nwith tf.Session() as sess:\n    test_catagory = pd.DataFrame({'Class':None,'Family':None,'Kingdom':None,'Order':None,'Phylum':None}, index = [0])\n    \n    for cat in col_names:\n        # Restore variables from disk.\n        currentCheckpoint = tf.train.latest_checkpoint(cat)\n        saver = tf.train.import_meta_graph(currentCheckpoint + \".meta\")\n        saver.restore(sess, currentCheckpoint)\n        print(\"Model restored.\")         \n        sunspot_classifier = tf.estimator.Estimator(\n            model_fn=cnn_model_test, model_dir=cat)\n        # predict with the model and print results\n        pred_input_fn = tf.estimator.inputs.numpy_input_fn(\n                x={\"x\": TestData},shuffle=False)\n        pred_results = sunspot_classifier.predict(input_fn=pred_input_fn)\n\n        pred_val = np.array([p['class_ids'] for p in pred_results]).squeeze()\n        \n        test_catagory[cat] = pred_val\n\n    test_catagory = test_catagory[1:]\n    \n    for pred_val in range(0,len(test_catagory.index)):\n        # Restore variables from disk.\n        currentCheckpoint = tf.train.latest_checkpoint(test_catagory['Kingdom'][pred_val] + '_' + test_catagory['Phylum'][pred_val] \n                                                       + '_' + test_catagory['Class'][pred_val] + '_' + test_catagory['Order'][pred_val] + '_' \n                                                       + test_catagory['Family'][pred_val] + '.csv')\n        saver = tf.train.import_meta_graph(currentCheckpoint + \".meta\")\n        saver.restore(sess, currentCheckpoint)\n        print(\"Model restored.\")         \n        sunspot_classifier = tf.estimator.Estimator(\n            model_fn=cnn_model_test, model_dir=cat)\n        # predict with the model and print results\n        pred_input_fn = tf.estimator.inputs.numpy_input_fn(\n                x={\"x\": TestData[pred_val]},shuffle=False)\n        pred_results = sunspot_classifier.predict(input_fn=pred_input_fn)\n\n        pred_value = np.array([p['class_ids'] for p in pred_results]).squeeze()\n        \n        np.concatenate((pred,pred_value))\n        \npred = pred[1:]\n\noutput1 = pd.DataFrame({'Id':Test_Data['id'],'Prediction':pred})\n","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}