{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# EDA on Flower Classification with TPU Competition Dataset","metadata":{"papermill":{"duration":0.007254,"end_time":"2022-02-11T14:05:25.454217","exception":false,"start_time":"2022-02-11T14:05:25.446963","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# I. Goals\n\nClassify images of flowers in 104 different classes. Classical image classification problem. Distinguish flowers which might be very similar in forms and colors. Images are from 5 public datasets\n\n\n* There appears to be a label hirarchy (flower type hirarchy. Some classes are very narrow, containing only a particular sub-type of flower (e.g. pink primroses) while other classes contain many sub-types (e.g. wild roses).\n\n* Metrics: Macro-F1 score does not take class-imbalance into account\n* Performance on public test set, there is no hidden set. Careful with overfitting","metadata":{"papermill":{"duration":0.005917,"end_time":"2022-02-11T14:05:25.466723","exception":false,"start_time":"2022-02-11T14:05:25.460806","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# II. Data Extraction\n\n* Data is available local in Kaggle but also in a GC bucket. See below.\n* n TFRecord format. \n\n* same data in different resolution?\n\n","metadata":{"papermill":{"duration":0.005928,"end_time":"2022-02-11T14:05:25.479065","exception":false,"start_time":"2022-02-11T14:05:25.473137","status":"completed"},"tags":[]}},{"cell_type":"code","source":"! ls ../input/tpu-getting-started\n","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:45.551170Z","iopub.execute_input":"2022-03-02T02:19:45.551497Z","iopub.status.idle":"2022-03-02T02:19:46.358658Z","shell.execute_reply.started":"2022-03-02T02:19:45.551402Z","shell.execute_reply":"2022-03-02T02:19:46.357973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import tensorflow as tf\nprint(tf.__version__)\nimport pandas as pd\nimport seaborn as sns","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:46.360435Z","iopub.execute_input":"2022-03-02T02:19:46.360666Z","iopub.status.idle":"2022-03-02T02:19:51.530335Z","shell.execute_reply.started":"2022-03-02T02:19:46.360638Z","shell.execute_reply":"2022-03-02T02:19:51.529512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from kaggle_datasets import KaggleDatasets\n\nGCS_DS_PATH = KaggleDatasets().get_gcs_path('tpu-getting-started')\nprint(GCS_DS_PATH) # what do gcs paths look like?","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:51.531631Z","iopub.execute_input":"2022-03-02T02:19:51.531876Z","iopub.status.idle":"2022-03-02T02:19:52.391342Z","shell.execute_reply.started":"2022-03-02T02:19:51.531849Z","shell.execute_reply":"2022-03-02T02:19:52.390406Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"names from https://www.kaggle.com/ryanholbrook/create-your-first-submission","metadata":{}},{"cell_type":"code","source":"class_names = ['pink primrose',    'hard-leaved pocket orchid', 'canterbury bells', 'sweet pea',     'wild geranium',     'tiger lily',           'moon orchid',              'bird of paradise', 'monkshood',        'globe thistle',         # 00 - 09\n           'snapdragon',       \"colt's foot\",               'king protea',      'spear thistle', 'yellow iris',       'globe-flower',         'purple coneflower',        'peruvian lily',    'balloon flower',   'giant white arum lily', # 10 - 19\n           'fire lily',        'pincushion flower',         'fritillary',       'red ginger',    'grape hyacinth',    'corn poppy',           'prince of wales feathers', 'stemless gentian', 'artichoke',        'sweet william',         # 20 - 29\n           'carnation',        'garden phlox',              'love in the mist', 'cosmos',        'alpine sea holly',  'ruby-lipped cattleya', 'cape flower',              'great masterwort', 'siam tulip',       'lenten rose',           # 30 - 39\n           'barberton daisy',  'daffodil',                  'sword lily',       'poinsettia',    'bolero deep blue',  'wallflower',           'marigold',                 'buttercup',        'daisy',            'common dandelion',      # 40 - 49\n           'petunia',          'wild pansy',                'primula',          'sunflower',     'lilac hibiscus',    'bishop of llandaff',   'gaura',                    'geranium',         'orange dahlia',    'pink-yellow dahlia',    # 50 - 59\n           'cautleya spicata', 'japanese anemone',          'black-eyed susan', 'silverbush',    'californian poppy', 'osteospermum',         'spring crocus',            'iris',             'windflower',       'tree poppy',            # 60 - 69\n           'gazania',          'azalea',                    'water lily',       'rose',          'thorn apple',       'morning glory',        'passion flower',           'lotus',            'toad lily',        'anthurium',             # 70 - 79\n           'frangipani',       'clematis',                  'hibiscus',         'columbine',     'desert-rose',       'tree mallow',          'magnolia',                 'cyclamen ',        'watercress',       'canna lily',            # 80 - 89\n           'hippeastrum ',     'bee balm',                  'pink quill',       'foxglove',      'bougainvillea',     'camellia',             'mallow',                   'mexican petunia',  'bromelia',         'blanket flower',        # 90 - 99\n           'trumpet creeper',  'blackberry lily',           'common tulip',     'wild rose']                                                                                                                                               # 100 - 102\nlen(class_names)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:52.393217Z","iopub.execute_input":"2022-03-02T02:19:52.393429Z","iopub.status.idle":"2022-03-02T02:19:52.405461Z","shell.execute_reply.started":"2022-03-02T02:19:52.393402Z","shell.execute_reply":"2022-03-02T02:19:52.404743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# III. Meet & Greet Data\n\nFor purpose of EDA focus partly on 512x512","metadata":{"papermill":{"duration":0.005853,"end_time":"2022-02-11T14:05:25.491155","exception":false,"start_time":"2022-02-11T14:05:25.485302","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## Categories of Flowers","metadata":{}},{"cell_type":"code","source":"class_names","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:52.406594Z","iopub.execute_input":"2022-03-02T02:19:52.407171Z","iopub.status.idle":"2022-03-02T02:19:52.422417Z","shell.execute_reply.started":"2022-03-02T02:19:52.407133Z","shell.execute_reply":"2022-03-02T02:19:52.421896Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Flowers exist in groups","metadata":{}},{"cell_type":"code","source":"categories = ['lily', 'rose', 'iris', 'tulip', 'daisy', 'poppy']","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:52.423270Z","iopub.execute_input":"2022-03-02T02:19:52.423826Z","iopub.status.idle":"2022-03-02T02:19:52.431682Z","shell.execute_reply.started":"2022-03-02T02:19:52.423782Z","shell.execute_reply":"2022-03-02T02:19:52.430949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"category_id_map = {name:i for i, name in enumerate(categories)}\nid_count = max(category_id_map.values())\nids = []\nfor name in class_names:\n    for cat in categories:\n        if cat in name.split():\n            ids.append(category_id_map[cat])\n            break\n    else:\n        id_count +=1\n        ids.append(id_count)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:52.432819Z","iopub.execute_input":"2022-03-02T02:19:52.433224Z","iopub.status.idle":"2022-03-02T02:19:52.443260Z","shell.execute_reply.started":"2022-03-02T02:19:52.433198Z","shell.execute_reply":"2022-03-02T02:19:52.442402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class_groups = pd.DataFrame(zip(class_names, ids), columns=['names', 'id'])\nclass_groups.groupby('id')['names'].apply(list).head(len(categories)).values","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:52.444534Z","iopub.execute_input":"2022-03-02T02:19:52.444877Z","iopub.status.idle":"2022-03-02T02:19:52.469751Z","shell.execute_reply.started":"2022-03-02T02:19:52.444843Z","shell.execute_reply":"2022-03-02T02:19:52.469158Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> Some flowers are of the same type/category and hence expect classification errors among them.","metadata":{}},{"cell_type":"code","source":"class_name_mapping = {i:name for i, name in enumerate(class_names)}\n","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:52.470630Z","iopub.execute_input":"2022-03-02T02:19:52.471035Z","iopub.status.idle":"2022-03-02T02:19:52.479168Z","shell.execute_reply.started":"2022-03-02T02:19:52.470977Z","shell.execute_reply":"2022-03-02T02:19:52.478240Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Images","metadata":{}},{"cell_type":"code","source":"! ls ../input/tpu-getting-started\n","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:52.481454Z","iopub.execute_input":"2022-03-02T02:19:52.481972Z","iopub.status.idle":"2022-03-02T02:19:53.268605Z","shell.execute_reply.started":"2022-03-02T02:19:52.481936Z","shell.execute_reply":"2022-03-02T02:19:53.267893Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IMAGE_SIZE = [512, 512]\n\ndata_root = \"../input/tpu-getting-started\"\n#data_root = GCS_DS_PATH\n\ndata_path = data_root + '/tfrecords-jpeg-512x512'\n\n\ntrain_512 = tf.io.gfile.glob(data_path + '/train/*.tfrec')\nval_512 = tf.io.gfile.glob(data_path + '/val/*.tfrec')\ntest_512 = tf.io.gfile.glob(data_path + '/test/*.tfrec') \nall_512 = [train_512, val_512, test_512]","metadata":{"papermill":{"duration":0.005897,"end_time":"2022-02-11T14:05:25.503234","exception":false,"start_time":"2022-02-11T14:05:25.497337","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-03-02T02:19:53.269927Z","iopub.execute_input":"2022-03-02T02:19:53.270825Z","iopub.status.idle":"2022-03-02T02:19:53.323253Z","shell.execute_reply.started":"2022-03-02T02:19:53.270790Z","shell.execute_reply":"2022-03-02T02:19:53.322624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"16 files per set","metadata":{}},{"cell_type":"code","source":"[len(dset) for dset in all_512]","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:53.324045Z","iopub.execute_input":"2022-03-02T02:19:53.324261Z","iopub.status.idle":"2022-03-02T02:19:53.329293Z","shell.execute_reply.started":"2022-03-02T02:19:53.324235Z","shell.execute_reply":"2022-03-02T02:19:53.328750Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_512","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:53.330288Z","iopub.execute_input":"2022-03-02T02:19:53.330721Z","iopub.status.idle":"2022-03-02T02:19:53.339346Z","shell.execute_reply.started":"2022-03-02T02:19:53.330677Z","shell.execute_reply":"2022-03-02T02:19:53.338754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# IV. Univariate Analysis","metadata":{}},{"cell_type":"code","source":"len(class_names)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:53.340437Z","iopub.execute_input":"2022-03-02T02:19:53.340873Z","iopub.status.idle":"2022-03-02T02:19:53.349140Z","shell.execute_reply.started":"2022-03-02T02:19:53.340833Z","shell.execute_reply":"2022-03-02T02:19:53.348588Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tensorflow.data import Dataset, TFRecordDataset","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:53.350162Z","iopub.execute_input":"2022-03-02T02:19:53.350845Z","iopub.status.idle":"2022-03-02T02:19:53.361239Z","shell.execute_reply.started":"2022-03-02T02:19:53.350809Z","shell.execute_reply":"2022-03-02T02:19:53.360579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"record_sample = TFRecordDataset(train_512)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:53.362225Z","iopub.execute_input":"2022-03-02T02:19:53.362834Z","iopub.status.idle":"2022-03-02T02:19:53.443974Z","shell.execute_reply.started":"2022-03-02T02:19:53.362763Z","shell.execute_reply":"2022-03-02T02:19:53.443357Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_elements = 0\nfor element in record_sample:\n    num_elements += 1\nnum_elements","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:19:53.445100Z","iopub.execute_input":"2022-03-02T02:19:53.445470Z","iopub.status.idle":"2022-03-02T02:20:34.447220Z","shell.execute_reply.started":"2022-03-02T02:19:53.445430Z","shell.execute_reply":"2022-03-02T02:20:34.445675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Image loading pipeline. References\n\n* https://www.tensorflow.org/api_docs/python/tf/data/Dataset#shuffle\n* https://www.kaggle.com/ryanholbrook/create-your-first-submission","metadata":{}},{"cell_type":"code","source":"\ndef decode_image(image_data):\n    # images are encoded as jpg\n    image = tf.image.decode_jpeg(image_data, channels=3)\n    image = tf.cast(image, tf.float32) / 255.0  # convert image to floats in [0, 1] range\n    #image = tf.reshape(image, [*IMAGE_SIZE, 3]) # explicit size needed for TPU\n    return image\n\ndef read_labeled_tfrecord(example):\n    LABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n        \"class\": tf.io.FixedLenFeature([], tf.int64),  # shape [] means single element\n    }\n    example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    label = tf.cast(example['class'], tf.int32)\n    depth = tf.constant(104)\n    #one_hot_encoded = tf.one_hot(indices=label, depth=depth)\n    \n    return image, label # returns a dataset of (image, label) pairs\n\ndef read_unlabeled_tfrecord(example):\n    UNLABELED_TFREC_FORMAT = {\n        \"image\": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring\n        \"id\": tf.io.FixedLenFeature([], tf.string),  # shape [] means single element\n        # class is missing, this competitions's challenge is to predict flower classes for the test dataset\n    }\n    example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n    image = decode_image(example['image'])\n    idnum = example['id']\n    return image, idnum # returns a dataset of image(s)\n\ndef load_dataset(filenames, labeled=True, ordered=False):\n    # Read from TFRecords. For optimal performance, reading from multiple files at once and\n    # disregarding data order. Order does not matter since we will be shuffling the data anyway.\n\n    ignore_order = tf.data.Options()\n    if not ordered:\n        ignore_order.experimental_deterministic = True # False # disable order, increase speed\n\n    AUTO = tf.data.experimental.AUTOTUNE\n    dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTO) # automatically interleaves reads from multiple files\n    dataset = dataset.with_options(ignore_order) # uses data as soon as it streams in, rather than in its original order\n    dataset = dataset.map(read_labeled_tfrecord if labeled else read_unlabeled_tfrecord, num_parallel_calls=AUTO)\n    # returns a dataset of (image, label) pairs if labeled=True or (image, id) pairs if labeled=False\n    return dataset","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:20:34.448831Z","iopub.execute_input":"2022-03-02T02:20:34.449229Z","iopub.status.idle":"2022-03-02T02:20:34.459866Z","shell.execute_reply.started":"2022-03-02T02:20:34.449185Z","shell.execute_reply":"2022-03-02T02:20:34.459080Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_train_512 = load_dataset(train_512, labeled=True)\nds_val_512 = load_dataset(val_512, labeled=True)\nds_test_512 = load_dataset(test_512, labeled=False)\n","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:20:34.461068Z","iopub.execute_input":"2022-03-02T02:20:34.461275Z","iopub.status.idle":"2022-03-02T02:20:34.672240Z","shell.execute_reply.started":"2022-03-02T02:20:34.461250Z","shell.execute_reply":"2022-03-02T02:20:34.671672Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for b, l in ds_train_512:\n    break","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:20:34.673329Z","iopub.execute_input":"2022-03-02T02:20:34.673570Z","iopub.status.idle":"2022-03-02T02:20:34.740869Z","shell.execute_reply.started":"2022-03-02T02:20:34.673541Z","shell.execute_reply":"2022-03-02T02:20:34.740012Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"l","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:20:34.742214Z","iopub.execute_input":"2022-03-02T02:20:34.743179Z","iopub.status.idle":"2022-03-02T02:20:34.751158Z","shell.execute_reply.started":"2022-03-02T02:20:34.743146Z","shell.execute_reply":"2022-03-02T02:20:34.750292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_ds_size(dataset, dtype='train'):\n    num_elements = 0\n    labels = []\n    for img, label in dataset:\n        num_elements += 1\n        labels.append(label.numpy())\n    print(f\"{dtype}: number of images: {num_elements}\")\n    if dtype != 'test':\n        return pd.Series([class_name_mapping[label] for label in labels])\n    \nds_train_512_labels = get_ds_size(ds_train_512, dtype='train'), \nds_val_512_labels = get_ds_size(ds_val_512, dtype='val')\nget_ds_size(ds_test_512, dtype='test')","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:20:34.752588Z","iopub.execute_input":"2022-03-02T02:20:34.753075Z","iopub.status.idle":"2022-03-02T02:21:16.685049Z","shell.execute_reply.started":"2022-03-02T02:20:34.753049Z","shell.execute_reply":"2022-03-02T02:21:16.683977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"total = 12753 + 3712 + 7382\n12753/total, 3712/total, 7382/total","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:16.686317Z","iopub.execute_input":"2022-03-02T02:21:16.686652Z","iopub.status.idle":"2022-03-02T02:21:16.693557Z","shell.execute_reply.started":"2022-03-02T02:21:16.686622Z","shell.execute_reply":"2022-03-02T02:21:16.692608Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> Test set is 2x the validation set in size","metadata":{}},{"cell_type":"markdown","source":"## Class Distribution","metadata":{}},{"cell_type":"code","source":"def get_class_distr(ds_labels):\n    ds_dist = pd.concat([ds_labels.value_counts(), \n               ds_labels.value_counts(normalize=True)], axis=1)\n    ds_dist.columns = ['counts', 'fraction']\n    return ds_dist\nds_train_512_labeldist = get_class_distr(ds_train_512_labels[0])\nds_val_512_labeldist = get_class_distr(ds_val_512_labels)","metadata":{"papermill":{"duration":0.007105,"end_time":"2022-02-11T14:05:25.640057","exception":false,"start_time":"2022-02-11T14:05:25.632952","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2022-03-02T02:21:16.695371Z","iopub.execute_input":"2022-03-02T02:21:16.695911Z","iopub.status.idle":"2022-03-02T02:21:16.711124Z","shell.execute_reply.started":"2022-03-02T02:21:16.695775Z","shell.execute_reply":"2022-03-02T02:21:16.710319Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_train_512_labeldist","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:16.712431Z","iopub.execute_input":"2022-03-02T02:21:16.712790Z","iopub.status.idle":"2022-03-02T02:21:16.737748Z","shell.execute_reply.started":"2022-03-02T02:21:16.712750Z","shell.execute_reply":"2022-03-02T02:21:16.736980Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_train_512_labeldist.head(20)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:16.738723Z","iopub.execute_input":"2022-03-02T02:21:16.739007Z","iopub.status.idle":"2022-03-02T02:21:16.749999Z","shell.execute_reply.started":"2022-03-02T02:21:16.738980Z","shell.execute_reply":"2022-03-02T02:21:16.748991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Problem Classes: 27 classes with less than 10 images in the training set!","metadata":{}},{"cell_type":"code","source":"ds_val_512_labeldist.tail(28)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:34:39.085593Z","iopub.execute_input":"2022-03-02T02:34:39.086431Z","iopub.status.idle":"2022-03-02T02:34:39.097757Z","shell.execute_reply.started":"2022-03-02T02:34:39.086387Z","shell.execute_reply":"2022-03-02T02:34:39.096686Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"problem_classes = ds_val_512_labeldist.tail(27).index\nproblem_classes","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:36:13.382327Z","iopub.execute_input":"2022-03-02T02:36:13.382637Z","iopub.status.idle":"2022-03-02T02:36:13.389040Z","shell.execute_reply.started":"2022-03-02T02:36:13.382599Z","shell.execute_reply":"2022-03-02T02:36:13.388261Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:16.766772Z","iopub.execute_input":"2022-03-02T02:21:16.767000Z","iopub.status.idle":"2022-03-02T02:21:16.770433Z","shell.execute_reply.started":"2022-03-02T02:21:16.766971Z","shell.execute_reply":"2022-03-02T02:21:16.769874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for key in ds_val_512_labeldist.to_dict()['fraction'].keys():\n    if not np.isclose(ds_val_512_labeldist.to_dict()['fraction'][key], \n                      ds_val_512_labeldist.to_dict()['fraction'][key]):\n        print(f\"{key} not close\")\n    \n","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:16.771558Z","iopub.execute_input":"2022-03-02T02:21:16.772210Z","iopub.status.idle":"2022-03-02T02:21:16.813824Z","shell.execute_reply.started":"2022-03-02T02:21:16.772172Z","shell.execute_reply":"2022-03-02T02:21:16.813014Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> * Classes are highly imbalanced, in fact some have only 18 images in train, and 5 images in valid set!\n> * Majority class makes up only 6% of the whole data.\n> * class distribution in train and valid is the same, as it should be.","metadata":{}},{"cell_type":"code","source":"ds_train_512_labeldist['counts'].plot(kind='hist')","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:16.815015Z","iopub.execute_input":"2022-03-02T02:21:16.815592Z","iopub.status.idle":"2022-03-02T02:21:17.082088Z","shell.execute_reply.started":"2022-03-02T02:21:16.815560Z","shell.execute_reply":"2022-03-02T02:21:17.081192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.boxplot(x=ds_train_512_labeldist['counts'])","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:17.083202Z","iopub.execute_input":"2022-03-02T02:21:17.083405Z","iopub.status.idle":"2022-03-02T02:21:17.250894Z","shell.execute_reply.started":"2022-03-02T02:21:17.083381Z","shell.execute_reply":"2022-03-02T02:21:17.250266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_train_512_labeldist['counts'].median()","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:17.251818Z","iopub.execute_input":"2022-03-02T02:21:17.252178Z","iopub.status.idle":"2022-03-02T02:21:17.257560Z","shell.execute_reply.started":"2022-03-02T02:21:17.252143Z","shell.execute_reply":"2022-03-02T02:21:17.256879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"> 9 classes have large number of images (outliers above, above ~280) while the median is 88 images per class","metadata":{}},{"cell_type":"markdown","source":"## Images Visual Analysis","metadata":{}},{"cell_type":"code","source":"from matplotlib import pyplot as plt\nimport math\n\ndef batch_to_numpy_images_and_labels(data):\n    images, labels = data\n    numpy_images = images.numpy()\n    numpy_labels = labels.numpy()\n    if numpy_labels.dtype == object: # binary string in this case,\n                                     # these are image ID strings\n        numpy_labels = [None for _ in enumerate(numpy_images)]\n    # If no labels, only image IDs, return None for labels (this is\n    # the case for test data)\n    return numpy_images, numpy_labels\n\ndef title_from_label_and_target(label, correct_label):\n    if correct_label is None:\n        return CLASSES[label], True\n    correct = (label == correct_label)\n    return \"{} [{}{}{}]\".format(class_names[label], 'OK' if correct else 'NO', u\"\\u2192\" if not correct else '',\n                                class_names[correct_label] if not correct else ''), correct\n\ndef display_one_flower(image, title, subplot, red=False, titlesize=16):\n    plt.subplot(*subplot)\n    plt.axis('off')\n    plt.imshow(image)\n    if len(title) > 0:\n        plt.title(title, fontsize=int(titlesize) if not red else int(titlesize/1.2), color='red' if red else 'black', fontdict={'verticalalignment':'center'}, pad=int(titlesize/1.5))\n    return (subplot[0], subplot[1], subplot[2]+1)\n    \ndef display_batch_of_images(databatch, predictions=None, FIGSIZE=13):\n    \"\"\"This will work with:\n    display_batch_of_images(images)\n    display_batch_of_images(images, predictions)\n    display_batch_of_images((images, labels))\n    display_batch_of_images((images, labels), predictions)\n    \"\"\"\n    # data\n    images, labels = batch_to_numpy_images_and_labels(databatch)\n    if labels is None:\n        labels = [None for _ in enumerate(images)]\n        \n    # auto-squaring: this will drop data that does not fit into square\n    # or square-ish rectangle\n    rows = int(math.sqrt(len(images)))\n    cols = len(images)//rows + 1\n        \n    # size and spacing\n    #FIGSIZE = 13.0\n    SPACING = 0.1\n    subplot=(rows,cols,1)\n    if rows < cols:\n        plt.figure(figsize=(FIGSIZE,FIGSIZE/cols*rows))\n    else:\n        plt.figure(figsize=(FIGSIZE/rows*cols,FIGSIZE))\n    \n    # display\n    for i, (image, label) in enumerate(zip(images[:rows*cols], labels[:rows*cols])):\n        title = '' if label is None else class_names[label]\n        correct = True\n        if predictions is not None:\n            title, correct = title_from_label_and_target(predictions[i], label)\n        dynamic_titlesize = FIGSIZE*SPACING/max(rows,cols)*40+3 # magic formula tested to work from 1x1 to 10x10 images\n        subplot = display_one_flower(image, title, subplot, not correct, titlesize=dynamic_titlesize)\n    \n    #layout\n    plt.tight_layout()\n    if label is None and predictions is None:\n        plt.subplots_adjust(wspace=0, hspace=0)\n    else:\n        plt.subplots_adjust(wspace=SPACING, hspace=SPACING)\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:38:03.925310Z","iopub.execute_input":"2022-03-02T02:38:03.925722Z","iopub.status.idle":"2022-03-02T02:38:03.947338Z","shell.execute_reply.started":"2022-03-02T02:38:03.925659Z","shell.execute_reply":"2022-03-02T02:38:03.946228Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds_train_512","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:17.276276Z","iopub.execute_input":"2022-03-02T02:21:17.276965Z","iopub.status.idle":"2022-03-02T02:21:17.291986Z","shell.execute_reply.started":"2022-03-02T02:21:17.276929Z","shell.execute_reply":"2022-03-02T02:21:17.291218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Random sample of flowers","metadata":{"execution":{"iopub.status.busy":"2022-02-25T18:46:30.804842Z","iopub.execute_input":"2022-02-25T18:46:30.805153Z","iopub.status.idle":"2022-02-25T18:46:30.874943Z","shell.execute_reply.started":"2022-02-25T18:46:30.805107Z","shell.execute_reply":"2022-02-25T18:46:30.874036Z"}}},{"cell_type":"code","source":"ds_train_512 = load_dataset(train_512, labeled=True)\n\nds_train_512 = ds_train_512.batch(10)\nbs = next(iter(ds_train_512))\n\ndisplay_batch_of_images(bs)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:17.293273Z","iopub.execute_input":"2022-03-02T02:21:17.293712Z","iopub.status.idle":"2022-03-02T02:21:18.707002Z","shell.execute_reply.started":"2022-03-02T02:21:17.293660Z","shell.execute_reply":"2022-03-02T02:21:18.705840Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Findings\n\n* details of background visible\n* some images have slighly blurry flowers\n* Images appear to stem from the outside, garden, nature etc.\n* sometime its one flower, sometimes multiple\n* picture angle on plant(s) seem fairly arbitray\n","metadata":{}},{"cell_type":"code","source":"ds_train_512 = load_dataset(train_512, labeled=True)\n\nds_train_512 = ds_train_512.batch(40)\nbs = next(iter(ds_train_512))\n\ndisplay_batch_of_images(bs)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:18.708498Z","iopub.execute_input":"2022-03-02T02:21:18.708815Z","iopub.status.idle":"2022-03-02T02:21:22.568740Z","shell.execute_reply.started":"2022-03-02T02:21:18.708780Z","shell.execute_reply":"2022-03-02T02:21:22.567475Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Analysis:\n* Indoor plants are also possible!\n* close up shots showing only part of the flower exist\n* Not clear if flowers are in different blooming stages\n* insects on flowers\n* flowers within category can differ signiifcantly: geranium vs wild geranium","metadata":{}},{"cell_type":"markdown","source":"## Flowers by Class","metadata":{}},{"cell_type":"code","source":"#class_name_mapping","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:22.570107Z","iopub.execute_input":"2022-03-02T02:21:22.570329Z","iopub.status.idle":"2022-03-02T02:21:22.573732Z","shell.execute_reply.started":"2022-03-02T02:21:22.570303Z","shell.execute_reply":"2022-03-02T02:21:22.572971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"inverse_class_name_mapping = {class_name_mapping[i]: i for i in class_name_mapping}","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:22.575055Z","iopub.execute_input":"2022-03-02T02:21:22.575268Z","iopub.status.idle":"2022-03-02T02:21:22.588473Z","shell.execute_reply.started":"2022-03-02T02:21:22.575243Z","shell.execute_reply":"2022-03-02T02:21:22.587712Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tqdm import tqdm\nfrom numpy.random import default_rng\n","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:22.589750Z","iopub.execute_input":"2022-03-02T02:21:22.590013Z","iopub.status.idle":"2022-03-02T02:21:22.597392Z","shell.execute_reply.started":"2022-03-02T02:21:22.589982Z","shell.execute_reply":"2022-03-02T02:21:22.596552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_batch_by_class(files, name = 'iris', top_n= 10, FIGSIZE=13):\n    \n    class_idx = inverse_class_name_mapping[name]\n    print(class_idx)\n    \n    max_imgs_per_class = ds_val_512_labeldist.loc[name,'counts']\n    \n    if top_n > max_imgs_per_class:\n        top_n = max_imgs_per_class\n        print(f\"warning, class has only {max_imgs_per_class} images. Show all images for class\")\n        \n    \n    # get position of class images in dataset\n    sample_idx = []\n    \n    ds = load_dataset(files, labeled=True)\n    ds = ds.batch(1)\n    for i, (img, label) in tqdm(enumerate(ds)):\n        if label.numpy()[0] == class_idx:\n            sample_idx.append(i)\n            \n    # choose randomly top_n images\n    rng = default_rng(42)\n    sample_idx_shuffled = sample_idx.copy()\n    rng.shuffle(sample_idx_shuffled)\n    top_n_sample = sample_idx_shuffled[:top_n]\n\n    ds = load_dataset(files, labeled=True)\n    ds = ds.batch(1)\n    # get thte images for each data point\n    images_class = []\n    tmp = []\n    for i, (img, label) in tqdm(enumerate(ds)):\n        if i in top_n_sample:\n            images_class.append(img)\n            tmp.append(label)\n\n    batch = tf.stack([tf.squeeze(img) for img in images_class]), tf.stack([class_idx for i in range(len(images_class))])\n    \n    display_batch_of_images(batch, FIGSIZE=FIGSIZE)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:38:27.420974Z","iopub.execute_input":"2022-03-02T02:38:27.421434Z","iopub.status.idle":"2022-03-02T02:38:27.431090Z","shell.execute_reply.started":"2022-03-02T02:38:27.421392Z","shell.execute_reply":"2022-03-02T02:38:27.430487Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Most Common Class: Iris","metadata":{}},{"cell_type":"code","source":"display_batch_by_class(train_512, name = 'iris', top_n= 10)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:21:22.609987Z","iopub.execute_input":"2022-03-02T02:21:22.610188Z","iopub.status.idle":"2022-03-02T02:22:02.955463Z","shell.execute_reply.started":"2022-03-02T02:21:22.610164Z","shell.execute_reply":"2022-03-02T02:22:02.954878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Problem Classes","metadata":{}},{"cell_type":"markdown","source":"#### Siam Tulip - one of the least common classes with only 5 images","metadata":{}},{"cell_type":"code","source":"display_batch_by_class(train_512, name = 'siam tulip', top_n= 20)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:22:02.956423Z","iopub.execute_input":"2022-03-02T02:22:02.956820Z","iopub.status.idle":"2022-03-02T02:22:43.133570Z","shell.execute_reply.started":"2022-03-02T02:22:02.956784Z","shell.execute_reply":"2022-03-02T02:22:43.132509Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* **Danger**: Is there something common in their background which could mislead the algorithm to use wrong features for identification? This class is especially prone due to the low number of images","metadata":{}},{"cell_type":"code","source":"display_batch_by_class(train_512, name = 'moon orchid', top_n= 20)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:22:43.134796Z","iopub.execute_input":"2022-03-02T02:22:43.135126Z","iopub.status.idle":"2022-03-02T02:23:23.043847Z","shell.execute_reply.started":"2022-03-02T02:22:43.134997Z","shell.execute_reply":"2022-03-02T02:23:23.043244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Look at all problem classes","metadata":{}},{"cell_type":"code","source":"problem_classes","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:36:56.957647Z","iopub.execute_input":"2022-03-02T02:36:56.958011Z","iopub.status.idle":"2022-03-02T02:36:56.963659Z","shell.execute_reply.started":"2022-03-02T02:36:56.957976Z","shell.execute_reply":"2022-03-02T02:36:56.963130Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for class_name in problem_classes:\n    display_batch_by_class(train_512, name = class_name, top_n= 10, FIGSIZE=6)","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:46:18.495101Z","iopub.execute_input":"2022-03-02T02:46:18.495775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Analysis for problem classes:\n* same shot, from front, one plant only: hard-leaved pocket\n* there can be still large variation in color and shape for each image per class.\n* some classes have few images with similar background","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Impact of Resolution\n\n* How does the image attributes change when decreasing the resolution?\n* Which features are not visible anymore?","metadata":{}},{"cell_type":"code","source":"files_all_res = [\n    tf.io.gfile.glob(data_root + '/tfrecords-jpeg-512x512' + '/train/*.tfrec'),\n    tf.io.gfile.glob(data_root + '/tfrecords-jpeg-331x331' + '/train/*.tfrec'),\n    tf.io.gfile.glob(data_root + '/tfrecords-jpeg-224x224' + '/train/*.tfrec'),\n    tf.io.gfile.glob(data_root + '/tfrecords-jpeg-192x192' + '/train/*.tfrec')\n]\nresolutions = [512, 331, 224, 192]\n","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:23:23.044822Z","iopub.execute_input":"2022-03-02T02:23:23.045616Z","iopub.status.idle":"2022-03-02T02:23:23.101876Z","shell.execute_reply.started":"2022-03-02T02:23:23.045562Z","shell.execute_reply":"2022-03-02T02:23:23.101225Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Compare impact of resolution on images by comparing the same image of flowers.\nUnfortunately the images are not in the same order for different resolutions\nand no unique flower id exists to link the images of different resolution.\nHence I pick one class for which I can plot all flower images.","metadata":{}},{"cell_type":"code","source":"for res, files in zip(resolutions, files_all_res):\n    #ds = load_dataset(files, labeled=True)\n    print(f\"Image Resolution: {res}\")\n    display_batch_by_class(files, name = 'moon orchid', top_n= 20)\n    #ds = ds.batch(1)\n    #batch = next(iter(ds))\n    #print(res)\n    #display_batch_of_images(batch)\n","metadata":{"execution":{"iopub.status.busy":"2022-03-02T02:23:23.102801Z","iopub.execute_input":"2022-03-02T02:23:23.103086Z","iopub.status.idle":"2022-03-02T02:25:06.057323Z","shell.execute_reply.started":"2022-03-02T02:23:23.103049Z","shell.execute_reply":"2022-03-02T02:25:06.056660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Augmentation Strategy\n\n* Images appear blurry: blurr\n* Images are zoomed in and out: zoom in/out\n* Images are brighter and darker","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}