{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **Introduction**","metadata":{}},{"cell_type":"markdown","source":"L'objectif du notebook est la création d'un modèle de classification d'image à l'aide d'une TPU (Tensor processing Unit) pour identifier les fleurs de 104 differentes éspéces. On explore les performances de plusieurs modèles convolutionnels et le potentiel du Vision Transformer et de la technique d'ensemble..\nSources:\n\nhttps://medium.com/swlh/understanding-inception-simplifying-the-network-architecture-54cd31d38949\nhttps://towardsdatascience.com/review-xception-with-depthwise-separable-convolution-better-than-inception-v3-image-dc967dd42568\nhttps://medium.com/machine-intelligence-and-deep-learning-lab/vit-vision-transformer-cc56c8071a20#:~:text=The%20Multi%2DHead%20Attention%20in,are%20semantically%20relevant%20for%20classification.>\nhttps://heartbeat.comet.ml/reviewing-efficientnet-increasing-the-accuracy-and-robustness-of-cnns-6aaf411fc81d>\n \nGithub:>\n https://github.com/FrancescoFran/Projet8---Petals-to-the-metal-competition.git\n ","metadata":{}},{"cell_type":"code","source":"pip install --quiet vit-keras","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:42:50.724476Z","iopub.execute_input":"2023-02-15T06:42:50.724760Z","iopub.status.idle":"2023-02-15T06:42:57.412042Z","shell.execute_reply.started":"2023-02-15T06:42:50.724721Z","shell.execute_reply":"2023-02-15T06:42:57.411260Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pip install -U -q efficientnet","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:43:01.191984Z","iopub.execute_input":"2023-02-15T06:43:01.192307Z","iopub.status.idle":"2023-02-15T06:43:08.261626Z","shell.execute_reply.started":"2023-02-15T06:43:01.192274Z","shell.execute_reply":"2023-02-15T06:43:08.260101Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport glob\nimport warnings\nimport re\nimport math\nimport random\nimport tensorflow_addons as tfa\nimport timeit\nimport tensorflow as tf\nimport tensorflow.keras.backend as K\nimport tensorflow_hub as hub\nimport warnings\nfrom vit_keras import vit\nimport os\nimport seaborn as sns\nfrom sklearn.metrics import classification_report\nfrom sklearn.model_selection import KFold\nfrom sklearn.utils import class_weight\nfrom matplotlib import pyplot as plt\n\nfrom sklearn.metrics import f1_score, precision_score, recall_score,\\\n confusion_matrix\n\nfrom tensorflow.data import Iterator\nfrom kaggle_datasets import KaggleDatasets\nprint('Tensorflow version ' + tf.__version__)\n\n\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n print(os.path.join(dirname, filename))\n \nwarnings.filterwarnings(\"ignore\")","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-output":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.status.busy":"2023-02-15T06:43:10.206280Z","iopub.execute_input":"2023-02-15T06:43:10.206622Z","iopub.status.idle":"2023-02-15T06:43:19.066782Z","shell.execute_reply.started":"2023-02-15T06:43:10.206589Z","shell.execute_reply":"2023-02-15T06:43:19.065930Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Detect accelerator**","metadata":{}},{"cell_type":"markdown","source":"Une TPU est utilisée pour l'entrainement des modèles. Les TPU sont des accélérateurs connectés au réseau spécialisés dans les tâches deep learning et la première étape est de les localiser sur le réseau. C’est ce que fait TPUClusterResolver.connect(). Ensuite TPUStrategy doit être instanciée. TPUStrategy est un objet contenant le code d'entrainement distribué qui fonctionnera sur les TPU avec leurs 8 cores de calcul.","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"Chaque core TPU possède une VPU (Vector Processing Unit) traditionnelle \n ainsi que une partie HW dédiée à la multiplication de matrices capable de traiter des matrices 128x128. C’est la partie qui accélère spécifiquement le procès d'entrainement des modèles. Enfin, TPUStrategy est utilisé en instanciant le modèle dans le cadre de la stratégie. Cela crée le modèle sur le TPU et Tensorflow distribuera l'entrainement entre les huit TPU cores en créant huit répliques différentes du modèle, une pour chaque core.","metadata":{}},{"cell_type":"code","source":"# Détecte l'HW, retourne la stratégie de distribution appropriée\ntry:\n # connecte à une TPU et instancie une stratégie de distribution\n tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n # détection TPU.\n # Aucun paramètre nécessaire si la variable d’environnement TPU_NAME est définie\n # Pour Kaggle c'est toujours le cas.\n print('Running on TPU ', tpu.master())\nexcept ValueError:\n tpu = None\n\nif tpu:\n tf.config.experimental_connect_to_cluster(tpu)\n tf.tpu.experimental.initialize_tpu_system(tpu)\n strategy = tf.distribute.experimental.TPUStrategy(tpu)\nelse:\n # stratégie de distribution par défaut dans Tensorflow. Fonctionne sur CPU et GPU unique.\n strategy = tf.distribute.get_strategy()\n\nprint(\"REPLICAS: \", strategy.num_replicas_in_sync)","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:43:25.786914Z","iopub.execute_input":"2023-02-15T06:43:25.787228Z","iopub.status.idle":"2023-02-15T06:43:31.039810Z","shell.execute_reply.started":"2023-02-15T06:43:25.787197Z","shell.execute_reply":"2023-02-15T06:43:31.039089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Chargement des données**","metadata":{}},{"cell_type":"markdown","source":"## **Chemin des données**","metadata":{}},{"cell_type":"markdown","source":"Lorsqu’ils sont utilisés avec des TPU, les jeux de données doivent être stockés dans un Google Cloud Storage bucket. Il est possible d’utiliser les données de n’importe quel CGS bucket en donnant son chemin. Les TPUs ont huit cores qui agissent de façon independante. On peut envoyer des données à chaque core plus efficacement en divisant l’ensemble de données en plusieurs fichiers or shards. Le type de fichier le plus pratique à utiliser pour le partage dans TensorFlow est un TFRecord. Un TFRecord est un fichier binaire qui contient des séquences de byte-strings. Les données doivent être sérialisées (encodées sous forme de byte-strings) avant d’être écrites dans un TFRecord. Les TPU sont équipés de 128 Go de mémoire haute vitesse permettant de plus grands batches, de plus grands modèles et aussi de plus grandes taille de données. On peut facilement gérer donc les images de taille 512x512 px fournies dans le jeu de données (dimensions d’image disponibles: 192, 224, 331, 512).","metadata":{}},{"cell_type":"code","source":"IMAGE_SIZE = [512, 512]\n# Data access\nGCS_DS_PATH = KaggleDatasets().get_gcs_path('tpu-getting-started')\n# num_parallel_reads=AUTO instruit l'API\n# à lire de plusieurs fichiers si disponibles.\n# Il calcule le nombre de fichiers automatiquement.\nAUTO = tf.data.experimental.AUTOTUNE\n\n\nGCS_PATH_SELECT = { # taille d'images disponibles\n 192: GCS_DS_PATH + '/tfrecords-jpeg-192x192',\n 224: GCS_DS_PATH + '/tfrecords-jpeg-224x224',\n 331: GCS_DS_PATH + '/tfrecords-jpeg-331x331',\n 512: GCS_DS_PATH + '/tfrecords-jpeg-512x512'\n}\nGCS_PATH = GCS_PATH_SELECT[IMAGE_SIZE[0]]\n\n\n# list des files de training dans GCS\nTRAINING_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/train/*.tfrec')\n# list des files de validation dans GCS\nVALIDATION_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/val/*.tfrec')\n# list des files de test files dans GCS\nTEST_FILENAMES = tf.io.gfile.glob(GCS_PATH + '/test/*.tfrec')\n\nprint(TRAINING_FILENAMES)","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:43:34.337473Z","iopub.execute_input":"2023-02-15T06:43:34.338138Z","iopub.status.idle":"2023-02-15T06:43:34.999668Z","shell.execute_reply.started":"2023-02-15T06:43:34.338094Z","shell.execute_reply":"2023-02-15T06:43:34.998121Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Import du jeu de données**","metadata":{}},{"cell_type":"markdown","source":"On définit des fonctions pour lire tfrecords et charger le jeu de données. Ensuite, on définit les noms des classes (104) et on crée des fonctions pour charger le jeu de données de training, le jeu de données de validation et le jeu de données de test. On définit également la taille du batch. Pour aller vite sur une TPU, la taille du batch doit être augmentée. La règle de base est d’utiliser des batches de 128 éléments par core (ex : taille de batch de 128*8=1024 pour une TPU de 8 cores). À cette taille, le potentiel de l'HW des multiplicateurs de matrices 128x128 du TPU est bien exploité.","metadata":{}},{"cell_type":"code","source":"def decode_image(image_data):\n image = tf.image.decode_jpeg(image_data, channels=3)\n # converte image en floats dans la plage [0, 1]\n image = tf.cast(image, tf.float32) / 255.0\n # taille explicite demandée par le TPU\n image = tf.reshape(image, [*IMAGE_SIZE, 3])\n return image\n\n\ndef read_labeled_tfrecord(example):\n LABELED_TFREC_FORMAT = {\n # tf.string signifie bytestring\n \"image\": tf.io.FixedLenFeature([], tf.string),\n # shape [] signifie single element\n \"class\": tf.io.FixedLenFeature([], tf.int64),\n }\n example = tf.io.parse_single_example(example, LABELED_TFREC_FORMAT)\n image = decode_image(example['image'])\n label = tf.cast(example['class'], tf.int32)\n return image, label # returns a dataset of (image, label) pairs\n\n\ndef read_unlabeled_tfrecord(example):\n UNLABELED_TFREC_FORMAT = {\n # tf.string signifie bytestring\n \"image\": tf.io.FixedLenFeature([], tf.string),\n # shape [] signifie single element\n \"id\": tf.io.FixedLenFeature([], tf.string),\n # pas de classe pour le jeu de test,\n # car la competition consiste à predire les classes des fleurs\n # pour le jeu de test\n }\n example = tf.io.parse_single_example(example, UNLABELED_TFREC_FORMAT)\n image = decode_image(example['image'])\n idnum = example['id']\n return image, idnum # returns a dataset of image(s)\n\n\ndef load_dataset(filenames, labeled=True, ordered=False):\n # Lire depuis TFRecords. Pour des performances optimales,\n # on lit depuis plusieurs fichiers à la fois\n # et on respecte pas l'ordre.\n # L’ordre n’a pas d’importance puisque nous allons appliquer le shuffling aux données\n ignore_order = tf.data.Options()\n if not ordered:\n # désactiver l’ordre, augmenter la vitesse\n ignore_order.experimental_deterministic = False\n\n # lecture entrelacée à partir de plusieurs fichiers\n dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=AUTO)\n # on utilise les données dès qu’elles sont entrées, plutôt que dans leur ordre initial\n dataset = dataset.with_options(ignore_order)\n dataset = dataset.map(read_labeled_tfrecord if labeled\n else read_unlabeled_tfrecord,\n num_parallel_calls=AUTO)\n # on obtien un jeu de données de paires (image, label)\n # si labeled=True ou (image, id) paires si labeled=False\n return dataset\n\n\nCLASSES = [\n 'pink primrose', 'hard-leaved pocket orchid', 'canterbury bells',\n 'sweet pea', 'wild geranium', 'tiger lily',\n 'moon orchid', 'bird of paradise', 'monkshood',\n 'globe thistle', 'snapdragon', \"colt's foot\",\n 'king protea', 'spear thistle', 'yellow iris',\n 'globe-flower', 'purple coneflower', 'peruvian lily',\n 'balloon flower', 'giant white arum lily', 'fire lily',\n 'pincushion flower', 'fritillary', 'red ginger',\n 'grape hyacinth', 'corn poppy', 'prince of wales feathers',\n 'stemless gentian', 'artichoke', 'sweet william',\n 'carnation', 'garden phlox', 'love in the mist',\n 'cosmos', 'alpine sea holly', 'ruby-lipped cattleya',\n 'cape flower', 'great masterwort', 'siam tulip',\n 'lenten rose', 'barberton daisy', 'daffodil',\n 'sword lily', 'poinsettia', 'bolero deep blue',\n 'wallflower', 'marigold', 'buttercup',\n 'daisy', 'common dandelion', 'petunia',\n 'wild pansy', 'primula', 'sunflower',\n 'lilac hibiscus', 'bishop of llandaff', 'gaura',\n 'geranium', 'orange dahlia', 'pink-yellow dahlia',\n 'cautleya spicata', 'japanese anemone', 'black-eyed susan',\n 'silverbush', 'californian poppy', 'osteospermum',\n 'spring crocus', 'iris', 'windflower',\n 'tree poppy', 'gazania', 'azalea',\n 'water lily', 'rose', 'thorn apple',\n 'morning glory', 'passion flower', 'lotus',\n 'toad lily', 'anthurium', 'frangipani',\n 'clematis', 'hibiscus', 'columbine',\n 'desert-rose', 'tree mallow', 'magnolia',\n 'cyclamen ', 'watercress', 'canna lily',\n 'hippeastrum ', 'bee balm', 'pink quill',\n 'foxglove', 'bougainvillea', 'camellia',\n 'mallow', 'mexican petunia', 'bromelia',\n 'blanket flower', 'trumpet creeper', 'blackberry lily',\n 'common tulip', 'wild rose']\n\n\ndef get_training_dataset(filenames, augmentation=False):\n dataset = load_dataset(filenames, labeled=True)\n if augmentation:\n # map la fonction data_augment pour les images du jeu de connées\n dataset = dataset.map(data_augment,\n num_parallel_calls=AUTO).concatenate(dataset)\n # le jeu de training est appelé pendant plusieurs epochs\n dataset = dataset.repeat()\n dataset = dataset.shuffle(2048)\n dataset = dataset.batch(BATCH_SIZE)\n # prefetch du prochain batch pendant l'entrainement (autotune prefetch buffer size)\n dataset = dataset.prefetch(AUTO)\n return dataset\n\n\ndef get_validation_dataset(ordered=False):\n dataset = load_dataset(VALIDATION_FILENAMES, labeled=True, ordered=ordered)\n dataset = dataset.batch(BATCH_SIZE)\n dataset = dataset.cache()\n dataset = dataset.prefetch(AUTO)\n return dataset\n\n\ndef get_test_dataset(filenames,ordered=False):\n dataset = load_dataset(filenames, labeled=False, ordered=ordered)\n dataset = dataset.batch(BATCH_SIZE)\n dataset = dataset.prefetch(AUTO)\n return dataset\n\n\ndef count_data_items(filenames):\n # le nombre d’éléments de données est écrit dans le nom des fichiers . tfrec,\n # i.e. flowers00-230.tfrec = 230 éléments\n n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1))\n for filename in filenames]\n return np.sum(n)\n\n\nNUM_TRAINING_IMAGES = int(count_data_items(TRAINING_FILENAMES))\nNUM_VALIDATION_IMAGES = int(count_data_items(VALIDATION_FILENAMES))\nNUM_TEST_IMAGES = int(count_data_items(TEST_FILENAMES))\n\n# On définit la dimension du batch aui sera 16 avec TPU off\n# and 128 (=16*8) avec TPU on\nBATCH_SIZE = 16 * strategy.num_replicas_in_sync\n\nprint('Dataset: {} training images, {} validation images, \\\n{} test images'.format(NUM_TRAINING_IMAGES, NUM_VALIDATION_IMAGES,\n NUM_TEST_IMAGES))","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:43:58.110098Z","iopub.execute_input":"2023-02-15T06:43:58.110547Z","iopub.status.idle":"2023-02-15T06:43:58.137356Z","shell.execute_reply.started":"2023-02-15T06:43:58.110512Z","shell.execute_reply":"2023-02-15T06:43:58.136135Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Un jeu de données dans TensorFlow est un flux data records. Les jeu de training et de validation sont des flux de paires (image, étiquette). Le jeu de test est un flux de paires (image, idnum).","metadata":{}},{"cell_type":"code","source":"np.set_printoptions(threshold=15, linewidth=80)\ntraining_dataset = get_training_dataset(TRAINING_FILENAMES)\nvalidation_dataset = get_validation_dataset()\ntest_dataset = get_test_dataset(TEST_FILENAMES)\n\nprint(\"\\nTraining data shapes:\")\nfor image, label in training_dataset.take(3):\n print(image.numpy().shape, label.numpy().shape)\nprint(\"Training data label examples:\", label.numpy())\nprint(\"\\nValidation data shapes:\")\nfor image, label in validation_dataset.take(3):\n print(image.numpy().shape, label.numpy().shape)\nprint(\"Validation data label examples:\", label.numpy())\nprint(\"\\nTest data shapes:\")\nfor image, idnum in test_dataset.take(3):\n print(image.numpy().shape, idnum.numpy().shape)\nprint(\"Test data IDs:\", idnum.numpy().astype('U')) # U=unicode string","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:44:05.933568Z","iopub.execute_input":"2023-02-15T06:44:05.933865Z","iopub.status.idle":"2023-02-15T06:44:16.509149Z","shell.execute_reply.started":"2023-02-15T06:44:05.933834Z","shell.execute_reply":"2023-02-15T06:44:16.507903Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Analyse exploratoire des données**","metadata":{}},{"cell_type":"markdown","source":"On réalise une analyse exploratoire des données afin de visualiser la distribution des images par classe.\n","metadata":{}},{"cell_type":"code","source":"train_dataset = get_training_dataset(TRAINING_FILENAMES, augmentation=False)\nvalidation_dataset = get_validation_dataset()\ntrain_labels_ds = train_dataset.map(lambda image, label: label).unbatch()\nvalidation_labels_ds = validation_dataset.map(lambda image,\n label: label).unbatch()\ntrain_labels = next(iter(train_labels_ds.batch(NUM_TRAINING_IMAGES))).numpy()\nvalidation_labels = next(iter(\n validation_labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\ntrain_labels_string = [CLASSES[i] for i in train_labels]\nvalidation_labels_string = [CLASSES[i] for i in validation_labels]\nprint('Number of labels in training dataset:',len(train_labels_string))\nprint('Number of labels in validation dataset:',len(validation_labels_string))","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:44:21.341856Z","iopub.execute_input":"2023-02-15T06:44:21.342735Z","iopub.status.idle":"2023-02-15T06:44:36.512775Z","shell.execute_reply.started":"2023-02-15T06:44:21.342698Z","shell.execute_reply":"2023-02-15T06:44:36.510991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_df = pd.DataFrame(train_labels_string, columns=['class'])\nlabels_count_df = pd.DataFrame(\n labels_df.value_counts().sort_values(ascending=False),\n columns=['train_labels_count']).reset_index()\n\nval_labels_df = pd.DataFrame(validation_labels_string,columns=['class'])\nval_labels_count_df = pd.DataFrame(\n val_labels_df.value_counts().sort_values(\n ascending =False), columns=['val_labels_count']).reset_index()\n\nlabels_count_df['val_labels_count'] = val_labels_count_df['val_labels_count']\nlabels_count_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:44:39.735304Z","iopub.execute_input":"2023-02-15T06:44:39.735661Z","iopub.status.idle":"2023-02-15T06:44:39.809285Z","shell.execute_reply.started":"2023-02-15T06:44:39.735618Z","shell.execute_reply":"2023-02-15T06:44:39.808254Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"matplotlib.rc_file_defaults()\nfig, ax = plt.subplots(figsize=(20, 14))\nax = sns.barplot(x=labels_count_df['class'],\n y=labels_count_df['train_labels_count'],\n color='b', label='Train')\nax = sns.barplot(x=val_labels_count_df['class'],\n y=val_labels_count_df['val_labels_count'],\n color='r', label='Val')\nplt.xticks(rotation='vertical')\nplt.legend(loc='upper right')\nplt.xlabel('class')\nplt.title('Distribution of number of images per class',fontsize=16)","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:44:45.556766Z","iopub.execute_input":"2023-02-15T06:44:45.557165Z","iopub.status.idle":"2023-02-15T06:44:48.146023Z","shell.execute_reply.started":"2023-02-15T06:44:45.557124Z","shell.execute_reply":"2023-02-15T06:44:48.145169Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Le jeu de données est donc déséquilibré car plusieurs classes ont petites proportions par rapport à d’autres classes. Pour éviter que les modèles aient mauvaises performances par rapport aux classes minoritaire on peut adopter deux techniques: \n* ‘Sampling methods’ qui consistent à créer des nouveaux échantillons pour les classes minoritaires ou supprimer des échantillons des classes majoritaires\n* 2.\t‘Cost-sensitive methods’ qui consistent à augmenter le poids des échantillons des classes minoritaires par rapport aux classes majoritaires afin d’obtenir des poids équilibrés par rapport aux nombre relatif d’échantillons entre les classes.","metadata":{}},{"cell_type":"markdown","source":"

","metadata":{}},{"cell_type":"markdown","source":" On adopte la deuxième approche avec l’utilisation de la fonction class_weight de la librairie sklearn.","metadata":{}},{"cell_type":"code","source":"class_weights = class_weight.compute_class_weight('balanced',\n classes=np.unique(\n train_labels),\n y=train_labels)\ndict_weights = dict(enumerate(class_weights))\nprint(class_weights)\nclass_weights_labels_string = [CLASSES[i] for i in np.unique(train_labels)]\n\nlabels_2_df = pd.DataFrame(class_weights_labels_string, columns=['class'])\nlabels_2_df['weights'] = class_weights\nlabels_2_df = labels_2_df.set_index('class').sort_values('weights',\n ascending=False)\nlabels_2_df = labels_2_df.sort_values('weights', ascending=True)\nlabels_count_df = labels_count_df.set_index('class')","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:44:53.487301Z","iopub.execute_input":"2023-02-15T06:44:53.487789Z","iopub.status.idle":"2023-02-15T06:44:53.508305Z","shell.execute_reply.started":"2023-02-15T06:44:53.487749Z","shell.execute_reply":"2023-02-15T06:44:53.507266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_count_df['weights'] = labels_2_df['weights']\nlabels_count_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:44:57.487761Z","iopub.execute_input":"2023-02-15T06:44:57.489141Z","iopub.status.idle":"2023-02-15T06:44:57.502863Z","shell.execute_reply.started":"2023-02-15T06:44:57.489078Z","shell.execute_reply":"2023-02-15T06:44:57.502253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels_count_df=labels_count_df.reset_index()","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:45:03.807354Z","iopub.execute_input":"2023-02-15T06:45:03.807696Z","iopub.status.idle":"2023-02-15T06:45:03.815926Z","shell.execute_reply.started":"2023-02-15T06:45:03.807661Z","shell.execute_reply":"2023-02-15T06:45:03.815185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"matplotlib.rc_file_defaults()\nax1 = sns.set_style(style=None, rc=None)\nfig, ax1 = plt.subplots(figsize=(18, 10))\nsns.barplot(data=labels_count_df, x='class', y='train_labels_count', alpha=0.8, ax=ax1)\nplt.xticks(rotation='vertical', fontsize=8)\nax2 = ax1.twinx()\nsns.lineplot(data=labels_count_df['weights'], marker='o', sort=False, ax=ax2)\nplt.title('Distribution of number of images and weights per class',fontsize=16)","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:45:06.645605Z","iopub.execute_input":"2023-02-15T06:45:06.645943Z","iopub.status.idle":"2023-02-15T06:45:09.023884Z","shell.execute_reply.started":"2023-02-15T06:45:06.645905Z","shell.execute_reply":"2023-02-15T06:45:09.023206Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Examples d'images du jeu de données**","metadata":{}},{"cell_type":"markdown","source":"On définit des fonctions pour visualiser des images du jeu de données avec leurs étiquettes.","metadata":{}},{"cell_type":"code","source":"def batch_to_numpy_images_and_labels(data):\n images, labels = data\n numpy_images = images.numpy()\n numpy_labels = labels.numpy()\n # binary string si image ID strings\n if numpy_labels.dtype == object:\n numpy_labels = [None for _ in enumerate(numpy_images)]\n # Si pas d'étiquette, seulement image IDs,\n # return None pour les étiquettes (pour les données de test)\n return numpy_images, numpy_labels\n\n\ndef display_one_flower(image, title, subplot, red=False, titlesize=16):\n plt.subplot(*subplot)\n plt.axis('off')\n plt.imshow(image)\n if len(title) > 0:\n plt.title(title, fontsize=int(titlesize)\n if not red else int(titlesize/1.2),\n color='red' if red else 'black',\n fontdict={'verticalalignment': 'center'},\n pad=int(titlesize/1.5))\n return (subplot[0], subplot[1], subplot[2]+1)\n\n\ndef title_from_label_and_target(label, correct_label):\n if correct_label is None:\n return CLASSES[label], True\n correct = (label == correct_label)\n return \"{} [{}{}{}]\".format(CLASSES[label], 'OK' if correct else 'NO', u\"\\u2192\" if not correct else '',\n CLASSES[correct_label] if not correct else ''), correct\n\ndef display_one_flower(image, title, subplot, red=False, titlesize=16):\n plt.subplot(*subplot)\n plt.axis('off')\n plt.imshow(image)\n if len(title) > 0:\n plt.title(title, fontsize=int(titlesize) if not red else int(titlesize/1.2), color='red' if red else 'black', fontdict={'verticalalignment':'center'}, pad=int(titlesize/1.5))\n return (subplot[0], subplot[1], subplot[2]+1)\n\n\ndef display_batch_of_images(databatch, predictions=None,\n display_mismatches_only=False):\n \"\"\"Fonction valable pour:\n display_batch_of_images(images)\n display_batch_of_images(images, predictions)\n display_batch_of_images((images, labels))\n display_batch_of_images((images, labels), predictions)\n \"\"\"\n # données\n images, labels = batch_to_numpy_images_and_labels(databatch)\n if labels is None:\n labels = [None for _ in enumerate(images)]\n\n # auto-squaring: cela supprime les données qui ne rentrent pas\n # dans le rectangle carré ou carré\n rows = int(math.sqrt(len(images)))\n cols = len(images)//rows\n\n FIGSIZE = 10.0\n SPACING = 0.1\n subplot = (rows, cols, 1)\n if rows < cols:\n plt.figure(figsize=(FIGSIZE, FIGSIZE/cols*rows))\n else:\n plt.figure(figsize=(FIGSIZE/rows*cols, FIGSIZE))\n\n # display\n for i, (image, label) in enumerate(zip(images[:rows*cols],\n labels[:rows*cols])):\n title = '' if label is None else CLASSES[label]\n correct = True\n if predictions is not None:\n title, correct = title_from_label_and_target(predictions[i],\n label)\n \n dynamic_titlesize = FIGSIZE*SPACING/max(rows, cols)*40+3\n if display_mismatches_only:\n if predictions[i] != label:\n subplot = display_one_flower(image, title, subplot,\n not correct,\n titlesize=dynamic_titlesize)\n else:\n subplot = display_one_flower(image, title, subplot, not correct,\n titlesize=dynamic_titlesize)\n\n # layout\n plt.tight_layout()\n if label is None and predictions is None:\n plt.subplots_adjust(wspace=0, hspace=0)\n else:\n plt.subplots_adjust(wspace=SPACING, hspace=SPACING)\n plt.show()\n\n\n# On itère sur les batches du jeu de données\nds_iter = iter(training_dataset.unbatch().batch(6))\n# Sélection d'un batch\none_batch = next(ds_iter)\n# Visualization du batch\ndisplay_batch_of_images(one_batch)","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:45:17.018130Z","iopub.execute_input":"2023-02-15T06:45:17.018731Z","iopub.status.idle":"2023-02-15T06:45:19.143761Z","shell.execute_reply.started":"2023-02-15T06:45:17.018691Z","shell.execute_reply":"2023-02-15T06:45:19.141327Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Data augmentation**","metadata":{}},{"cell_type":"markdown","source":"Le processus de data augmentation des données dérive de nouvelles images de celles existantes en appliquant des transformations géométriques aléatoires afin d’incorporer un niveau de variation dans le jeu de training qui permet au modèle de mieux généraliser sur des données inconnues. L’augmentation des données du jeu de training est extrêmement puissante en termes d’augmentation de la précision de réduction de l’overfitting.\n Les transformations appliquées au jeu de training sont :\n* Random blockout - sélection au hasard d’une région rectangle dans une image et remplacement de ses pixels par des pixels noirs.\n* Random_flip_left_right - retournement de l’image au hasard horizontalement (de gauche à droite).\n* Random_saturation – ajustement de la saturation de l’images RVB par un facteur aléatoire.\n* Random_contrast - ajustement du contraste de l’images par un facteur aléatoire.\n* Rotation aléatoire - rotation de l’image dans le sens antihoraire par un angle aléatoire.","metadata":{}},{"cell_type":"code","source":"def random_blockout(img, sl=0.1, sh=0.2, rl=0.4):\n p = np.random.random()\n if p >= 0.25:\n w, h, c = IMAGE_SIZE[0], IMAGE_SIZE[1], 3\n origin_area = tf.cast(h*w, tf.float32)\n\n e_size_l = tf.cast(tf.round(tf.sqrt(origin_area * sl * rl)), tf.int32)\n e_size_h = tf.cast(tf.round(tf.sqrt(origin_area * sh / rl)), tf.int32)\n\n e_height_h = tf.minimum(e_size_h, h)\n e_width_h = tf.minimum(e_size_h, w)\n\n erase_height = tf.random.uniform(shape=[],\n minval=e_size_l,\n maxval=e_height_h,\n dtype=tf.int32)\n erase_width = tf.random.uniform(shape=[],\n minval=e_size_l,\n maxval=e_width_h,\n dtype=tf.int32)\n\n erase_area = tf.zeros(shape=[erase_height,\n erase_width, c])\n erase_area = tf.cast(erase_area, tf.uint8)\n\n pad_h = h - erase_height\n pad_top = tf.random.uniform(shape=[],\n minval=0,\n maxval=pad_h,\n dtype=tf.int32)\n pad_bottom = pad_h - pad_top\n\n pad_w = w - erase_width\n pad_left = tf.random.uniform(shape=[], minval=0,\n maxval=pad_w, dtype=tf.int32)\n pad_right = pad_w - pad_left\n\n erase_mask = tf.pad([erase_area], [[0, 0], [pad_top, pad_bottom],\n [pad_left, pad_right],\n [0, 0]], constant_values=1)\n erase_mask = tf.squeeze(erase_mask, axis=0)\n erased_img = tf.multiply(tf.cast(img, tf.float32),\n tf.cast(erase_mask, tf.float32))\n\n return tf.cast(erased_img, img.dtype)\n else:\n return tf.cast(img, img.dtype)\n\n\ndef data_augment(image, label):\n\n num = random.randint(1, 5)\n if num == 1:\n augmentation_type = 'random_blockout'\n image = random_blockout(image, sl=0.1, sh=0.2, rl=0.4)\n elif num == 2:\n augmentation_type = 'random_flip_left_right'\n image = tf.image.random_flip_left_right(image)\n elif num == 3:\n augmentation_type = 'random_saturation'\n image = tf.image.random_saturation(image, 1, 3)\n elif num == 4:\n augmentation_type = 'random_contrast'\n image = tf.image.random_contrast(image, 0.4, 0.6)\n elif num == 5:\n augmentation_type = 'image_rotate'\n image = tfa.image.rotate(image, angles=random.randint(1, 359))\n return image, label\n\ntraining_dataset = get_training_dataset(TRAINING_FILENAMES, augmentation=True)","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:45:29.363542Z","iopub.execute_input":"2023-02-15T06:45:29.363859Z","iopub.status.idle":"2023-02-15T06:45:30.440240Z","shell.execute_reply.started":"2023-02-15T06:45:29.363825Z","shell.execute_reply":"2023-02-15T06:45:30.439178Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Examples de augmented images**","metadata":{}},{"cell_type":"code","source":"# On itère sur le batch suivant\nno_aug_train_set = get_training_dataset(TRAINING_FILENAMES, augmentation=False)\nno_aug_train_batch = (next(iter(no_aug_train_set.unbatch().batch(6))))\nimages, _ = batch_to_numpy_images_and_labels(no_aug_train_batch)\n\ni = 1\nrow = 6\ncol = 6\nplt.figure(figsize=(18, 18))\nfontsize = 10\n\nfor image in images:\n plt.subplot(6, col+1, i)\n plt.imshow(image)\n plt.axis('off')\n plt.title('Initial image', fontsize=fontsize)\n plt.subplot(6, col+1, i+1)\n plt.imshow(random_blockout(image, sl=0.1, sh=0.2, rl=0.4))\n plt.axis('off')\n plt.title('Random_blockout', fontsize=fontsize)\n plt.subplot(6, col+1, i+2)\n plt.axis('off')\n plt.title('Initial image', fontsize=fontsize)\n plt.imshow(tf.image.random_flip_left_right(image))\n plt.axis('off')\n plt.title('Random_flip_left_right', fontsize=fontsize)\n plt.subplot(6, col+1, i+3)\n plt.imshow(tf.image.random_saturation(image, 3, 5))\n plt.axis('off')\n plt.title('Random_saturation', fontsize=fontsize)\n plt.subplot(6, col+1, i+4)\n plt.imshow(tf.image.random_contrast(image, 0.8, 3))\n plt.axis('off')\n plt.title('Random_contrast', fontsize=fontsize)\n plt.subplot(6, col+1, i+5)\n plt.imshow(tfa.image.rotate(image, angles=random.randint(0, 364)))\n plt.axis('off')\n plt.title('Random_rotate', fontsize=fontsize)\n i += col+1","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:45:38.915750Z","iopub.execute_input":"2023-02-15T06:45:38.916947Z","iopub.status.idle":"2023-02-15T06:45:45.067518Z","shell.execute_reply.started":"2023-02-15T06:45:38.916889Z","shell.execute_reply":"2023-02-15T06:45:45.064338Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Modèles de classification et entrainement**","metadata":{}},{"cell_type":"markdown","source":"## **Custom Learning Rate scheduler**","metadata":{}},{"cell_type":"markdown","source":"On définit un Learning Rate schedule pour le Fine Tuning du modèle pre-entrainé avec une rampe LR, car commencer avec un LR élevé pourrait reduir l'éfficacité des poids du modèle pre-entrainé.","metadata":{}},{"cell_type":"code","source":"EPOCHS = 100\ndef exponential_lr(epoch,\n start_lr = 0.00001, min_lr = 0.00001, max_lr = 0.00005,\n rampup_epochs = 5, sustain_epochs = 0,\n exp_decay = 0.8):\n\n def lr(epoch, start_lr, min_lr, max_lr, rampup_epochs, sustain_epochs, exp_decay):\n # augmentation lineaire de start jusqu'à rampup_epochs\n if epoch < rampup_epochs:\n lr = ((max_lr - start_lr) /\n rampup_epochs * epoch + start_lr)\n # constante max_lr pendant sustain_epochs\n elif epoch < rampup_epochs + sustain_epochs:\n lr = max_lr\n # réduction éxponentiale jusqu'à min_lr\n else:\n lr = ((max_lr - min_lr) *\n exp_decay**(epoch - rampup_epochs - sustain_epochs) +\n min_lr)\n return lr\n return lr(epoch,\n start_lr,\n min_lr,\n max_lr,\n rampup_epochs,\n sustain_epochs,\n exp_decay)\n\nlr_callback = tf.keras.callbacks.LearningRateScheduler(exponential_lr, verbose=True)\n\nrng = [i for i in range(EPOCHS)]\ny = [exponential_lr(x) for x in rng]\nplt.plot(rng, y)\nplt.title('Learning rate schedule')\nprint(\"Learning rate schedule: {:.3g} to {:.3g} to {:.3g}\".format(y[0], max(y), y[-1]))","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:45:58.158902Z","iopub.execute_input":"2023-02-15T06:45:58.159268Z","iopub.status.idle":"2023-02-15T06:45:58.379503Z","shell.execute_reply.started":"2023-02-15T06:45:58.159233Z","shell.execute_reply":"2023-02-15T06:45:58.378291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Chargement du modèle et transfer learning**","metadata":{}},{"cell_type":"markdown","source":"On définit une fonction pour charger des modèles pré-entrainé et appliquer la technique de transfer learning avec Total Fine tuning.","metadata":{}},{"cell_type":"code","source":"def define_model(model_name,optimizer='adam',weights='imagenet',include_top=False, trainable=True):\n K.clear_session()\n with strategy.scope():\n print('Loading and compiling model..')\n pretrained_model = model_name(\n weights=weights,\n include_top=False,\n input_shape=[*IMAGE_SIZE, 3]\n )\n pretrained_model.trainable = True\n \n model = tf.keras.Sequential([\n # A une base pre-entrainée pour extraire les features des images...\n pretrained_model,\n # ... on ajoute une nouvelle tête comme classifieur.\n tf.keras.layers.GlobalAveragePooling2D(),\n tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n ])\n model.compile(\n optimizer=optimizer,\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy']\n )\n \n model.summary() \n \n return model","metadata":{"execution":{"iopub.status.busy":"2023-02-15T06:46:05.120856Z","iopub.execute_input":"2023-02-15T06:46:05.121250Z","iopub.status.idle":"2023-02-15T06:46:05.131656Z","shell.execute_reply.started":"2023-02-15T06:46:05.121211Z","shell.execute_reply":"2023-02-15T06:46:05.130095Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Modèle ResNet50**","metadata":{}},{"cell_type":"markdown","source":" L’architecture ResNet (Residual Network) a été introduit afin de faciliter l’entrainement des réseaux profonds en éliminant le problème de dégradation liée à la diminution de performance de l’architecture CNN classique avec l’augmentation de la profondeur du réseau. Le modèle ResNet50, caractérisé par une architecture simple avec ‘Identity blocks’ et ‘Convolutional blocks’, a été donc utilisé comme modèle baseline. ","metadata":{}},{"cell_type":"markdown","source":"\n\n\n
","metadata":{}},{"cell_type":"markdown","source":"

","metadata":{}},{"cell_type":"markdown","source":"## **Chargement du modèle**","metadata":{}},{"cell_type":"code","source":"ResNet50_model = define_model(tf.keras.applications.resnet50.ResNet50)\nparam_count = []\n# on sauvegarde l'info du nombre de paramètres du modèle\nparam_count.append(ResNet50_model.count_params())","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:21:51.739382Z","iopub.status.busy":"2023-02-05T09:21:51.738562Z","iopub.status.idle":"2023-02-05T09:22:03.514010Z","shell.execute_reply":"2023-02-05T09:22:03.513129Z","shell.execute_reply.started":"2023-02-05T09:21:51.739346Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# schéma du modèle\ntf.keras.utils.plot_model(ResNet50_model, show_shapes=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2023-02-05T09:22:05.515695Z","iopub.status.busy":"2023-02-05T09:22:05.515322Z","iopub.status.idle":"2023-02-05T09:22:05.703042Z","shell.execute_reply":"2023-02-05T09:22:05.702012Z","shell.execute_reply.started":"2023-02-05T09:22:05.515664Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Entrainement du modèle**","metadata":{}},{"cell_type":"code","source":"BEST_MODEL_PATH = \"/kaggle/working/ResNet50_model_best.h5\"\nFILE_DIR = os.path.dirname(BEST_MODEL_PATH)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:22:08.690637Z","iopub.status.busy":"2023-02-05T09:22:08.689980Z","iopub.status.idle":"2023-02-05T09:22:08.696641Z","shell.execute_reply":"2023-02-05T09:22:08.695658Z","shell.execute_reply.started":"2023-02-05T09:22:08.690583Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"On définit les callbacks function:\n* earlystopping afin d’arrêter la formation lorsque la métrique 'val_loss' cesse de s’améliorer\n* lr_callback pour définir le profil du taux learning rate\n* checkpointer afin de sauvegarder les paramètres du meilleur modèle pendant l'entrainement\net ensuite on demarre l'entrainement","metadata":{}},{"cell_type":"code","source":"earlystopping = tf.keras.callbacks.EarlyStopping(\n monitor='val_loss',\n min_delta=1e-4,\n patience=5,\n mode='auto',\n restore_best_weights=False,\n verbose=1)\n\ncheckpointer = tf.keras.callbacks.ModelCheckpoint(\n filepath=BEST_MODEL_PATH,\n options=None,\n monitor='val_loss',\n verbose=1,\n save_best_only=True,\n save_weights_only=True,\n mode='min')\n \ncallbacks = [earlystopping, lr_callback, checkpointer]\n\nEPOCHS = 100\nSTEPS_PER_EPOCH = NUM_TRAINING_IMAGES // BATCH_SIZE\n \nstart_time = timeit.default_timer()\n \nResNet50_history = ResNet50_model.fit(\n training_dataset,\n validation_data=validation_dataset,\n epochs=EPOCHS,\n steps_per_epoch=STEPS_PER_EPOCH,\n callbacks=callbacks,\n class_weight=dict_weights\n)\n\nResNet50_model_elapsed = timeit.default_timer() - start_time\ntrain_time = []\ntrain_time.append(ResNet50_model_elapsed)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:22:10.808259Z","iopub.status.busy":"2023-02-05T09:22:10.807393Z","iopub.status.idle":"2023-02-05T09:46:45.326492Z","shell.execute_reply":"2023-02-05T09:46:45.325804Z","shell.execute_reply.started":"2023-02-05T09:22:10.808218Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on sauvegarde l'historique d'entrainement du modèle\nnp.save('ResNet50_history.npy',ResNet50_history.history)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:46:49.469901Z","iopub.status.busy":"2023-02-05T09:46:49.469125Z","iopub.status.idle":"2023-02-05T09:46:49.475765Z","shell.execute_reply":"2023-02-05T09:46:49.474568Z","shell.execute_reply.started":"2023-02-05T09:46:49.469841Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Performance du modèle**","metadata":{}},{"cell_type":"code","source":"# on visualise les courbes de loss et accuracy sur le jeu de training et\n# de validation pendant l'entrainement\nfig = plt.figure(figsize=[14, 7], dpi=80)\ni = 0\nfor metric in ['loss', 'sparse_categorical_accuracy']:\n i += 1\n fig.add_subplot(1, 2, i)\n ax = plt.plot(ResNet50_history.history[metric])\n plt.plot(ResNet50_history.history['val_'+metric])\n plt.title('ResNet50 ' + metric,fontsize=16)\n plt.legend(['Training ' + metric, 'Validation '+metric], loc='upper left')\n plt.ylabel(metric,fontsize=14)\n plt.xlabel('Epoch',fontsize=14)\nplt.tight_layout()\nplt.show","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:46:56.574741Z","iopub.status.busy":"2023-02-05T09:46:56.574456Z","iopub.status.idle":"2023-02-05T09:46:57.204946Z","shell.execute_reply":"2023-02-05T09:46:57.204251Z","shell.execute_reply.started":"2023-02-05T09:46:56.574711Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Matrice de confusion et performances**","metadata":{}},{"cell_type":"code","source":"#on charge le modèle avec les meilleurs paramètres obtenus lors de l'entrainement\nResNet50_model.load_weights(BEST_MODEL_PATH)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:47:04.820200Z","iopub.status.busy":"2023-02-05T09:47:04.819350Z","iopub.status.idle":"2023-02-05T09:47:07.960887Z","shell.execute_reply":"2023-02-05T09:47:07.960113Z","shell.execute_reply.started":"2023-02-05T09:47:04.820160Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on représente la matrice de confusion sur le jeu de validation à partir\n# des classes vraies et prédictes\ncmdataset = get_validation_dataset(ordered=True)\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\n\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\ncm_probabilities = ResNet50_model.predict(images_ds)\ncm_predictions = np.argmax(cm_probabilities, axis=-1)\n\nlabels = range(len(CLASSES))\ncmat = confusion_matrix(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n)\ncmat = (cmat.T / cmat.sum(axis=1)).T # normalize\n\n\n# on visualize la matrice de confusion\nfig = plt.figure(figsize=(18, 14))\nsns.heatmap(cmat,\n xticklabels=CLASSES, yticklabels=CLASSES, annot=False, cmap='inferno')\nplt.xlabel('True label')\nplt.ylabel('Predicted label')\nplt.xticks(fontsize=8)\nplt.yticks(fontsize=8)\nplt.title('Confusion matrix with ResNet50', fontsize=16)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:47:23.810798Z","iopub.status.busy":"2023-02-05T09:47:23.810134Z","iopub.status.idle":"2023-02-05T09:47:42.565310Z","shell.execute_reply":"2023-02-05T09:47:42.564159Z","shell.execute_reply.started":"2023-02-05T09:47:23.810762Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on calcule les métriques F1-score, precision et recall\n# et on les sauvegarde avec les paramètres du modèle\n# et les infos liées à l'entrainement\nResNet50_f1_score = f1_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nResNet50_precision = precision_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nResNet50_recall = recall_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\n\nmax_train_accuracy = []\nmax_val_accuracy = []\nmin_val_loss = []\nepochs = []\nF1_score = []\nprecision = []\nrecall = []\n\nResNet50_history = np.load('/kaggle/working/ResNet50_history.npy', allow_pickle=True).item()\n\nmax_train_accuracy.append(max(ResNet50_history['sparse_categorical_accuracy']))\nmax_train_accuracy = [round(num,3) for num in max_train_accuracy]\nmax_val_accuracy.append(max(ResNet50_history['val_sparse_categorical_accuracy']))\nmax_val_accuracy = [round(num,3) for num in max_val_accuracy]\nmin_val_loss.append(min(ResNet50_history['val_loss']))\nmin_val_loss = [round(num,3) for num in min_val_loss]\nepochs.append(len(ResNet50_history['val_loss']))\nF1_score.append(ResNet50_f1_score)\nF1_score = [round(num,3) for num in F1_score]\nprecision.append(ResNet50_precision)\nprecision = [round(num,3) for num in precision]\nrecall.append(ResNet50_recall)\nrecall = [round(num,3) for num in recall]\n\nscore = {'parameters': param_count, 'epochs': epochs, 'min_loss': min_val_loss, \n 'max_train_accuracy': max_train_accuracy, 'max_val_accuracy': max_val_accuracy, \n 'F1_score': F1_score, 'precision': precision, 'recall': recall}\nscore_df = pd.DataFrame(score,index=['ResNet50'])\nscore_df.to_csv('score_df.csv',index=False), \nscore_df","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:47:47.129051Z","iopub.status.busy":"2023-02-05T09:47:47.127812Z","iopub.status.idle":"2023-02-05T09:47:47.174359Z","shell.execute_reply":"2023-02-05T09:47:47.173367Z","shell.execute_reply.started":"2023-02-05T09:47:47.128975Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Modèle InceptionV3**","metadata":{}},{"cell_type":"markdown","source":"L’architecture Inception introduit le bloc Inception qui contient plusieurs couches de convolution et de pooling superposées, pour obtenir de meilleurs résultats et réduire le nombre de paramètres et donc les ressources de calcul par rapport à un réseau CNN standard.","metadata":{}},{"cell_type":"markdown","source":"\n\n\n
\n

","metadata":{}},{"cell_type":"markdown","source":"## **Chargement du modèle**","metadata":{}},{"cell_type":"code","source":"Inception_model = define_model(tf.keras.applications.InceptionV3)\nscore_df = pd.read_csv('/kaggle/working/score_df.csv')","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:47:53.202630Z","iopub.status.busy":"2023-02-05T09:47:53.201979Z","iopub.status.idle":"2023-02-05T09:48:11.463782Z","shell.execute_reply":"2023-02-05T09:48:11.462889Z","shell.execute_reply.started":"2023-02-05T09:47:53.202576Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on sauvegarde l'info du nombre de paramètres du modèle\nparam_count = list(score_df['parameters'])\nparam_count.append(Inception_model.count_params())","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:48:13.993695Z","iopub.status.busy":"2023-02-05T09:48:13.993186Z","iopub.status.idle":"2023-02-05T09:48:14.008057Z","shell.execute_reply":"2023-02-05T09:48:14.007384Z","shell.execute_reply.started":"2023-02-05T09:48:13.993643Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# schéma du modèle\ntf.keras.utils.plot_model(Inception_model, show_shapes=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2023-02-05T09:48:16.461773Z","iopub.status.busy":"2023-02-05T09:48:16.461479Z","iopub.status.idle":"2023-02-05T09:48:16.732947Z","shell.execute_reply":"2023-02-05T09:48:16.732149Z","shell.execute_reply.started":"2023-02-05T09:48:16.461744Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Entrainement du modèle**","metadata":{}},{"cell_type":"code","source":"BEST_MODEL_PATH = \"/kaggle/working/Inception_model_best.h5\"\nFILE_DIR = os.path.dirname(BEST_MODEL_PATH)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:48:23.039954Z","iopub.status.busy":"2023-02-05T09:48:23.038862Z","iopub.status.idle":"2023-02-05T09:48:23.044644Z","shell.execute_reply":"2023-02-05T09:48:23.043561Z","shell.execute_reply.started":"2023-02-05T09:48:23.039905Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"earlystopping = tf.keras.callbacks.EarlyStopping(\n monitor='val_loss',\n min_delta=1e-4,\n patience=5,\n mode='auto',\n restore_best_weights=False,\n verbose=1)\n\ncheckpointer = tf.keras.callbacks.ModelCheckpoint(\n filepath=BEST_MODEL_PATH,\n options=None,\n monitor='val_loss',\n verbose=1,\n save_best_only=True,\n save_weights_only=True,\n mode='min')\n \ncallbacks = [earlystopping, lr_callback, checkpointer]\n\nEPOCHS = 100\nSTEPS_PER_EPOCH = NUM_TRAINING_IMAGES // BATCH_SIZE\n \nstart_time = timeit.default_timer()\n \nInception_history = Inception_model.fit(\n training_dataset,\n validation_data=validation_dataset,\n epochs=EPOCHS,\n steps_per_epoch=STEPS_PER_EPOCH,\n callbacks=callbacks,\n class_weight=dict_weights\n)\n\nInception_model_elapsed = timeit.default_timer() - start_time\ntrain_time.append(Inception_model_elapsed)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T09:48:30.963285Z","iopub.status.busy":"2023-02-05T09:48:30.962493Z","iopub.status.idle":"2023-02-05T10:34:41.898674Z","shell.execute_reply":"2023-02-05T10:34:41.897435Z","shell.execute_reply.started":"2023-02-05T09:48:30.963244Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on sauvegarde l'historique d'entrainement du modèle\nnp.save('Inception_history.npy',Inception_history.history)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T10:34:44.511383Z","iopub.status.busy":"2023-02-05T10:34:44.511045Z","iopub.status.idle":"2023-02-05T10:34:44.517268Z","shell.execute_reply":"2023-02-05T10:34:44.516300Z","shell.execute_reply.started":"2023-02-05T10:34:44.511348Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Performance du modèle**","metadata":{}},{"cell_type":"code","source":"# on visualise les courbes de loss et accuracy sur le jeu de training et\n# de validation pendant l'entrainement\nfig = plt.figure(figsize=[14, 7], dpi=80)\ni = 0\nfor metric in ['loss', 'sparse_categorical_accuracy']:\n i += 1\n fig.add_subplot(1, 2, i)\n ax = plt.plot(Inception_history.history[metric])\n plt.plot(Inception_history.history['val_' + metric])\n plt.title('InceptionV3 ' + metric,fontsize=16)\n plt.legend(['Training ' + metric, 'Validation ' + metric], loc='upper left')\n plt.ylabel(metric,fontsize=14)\n plt.xlabel('Epoch',fontsize=14)\nplt.tight_layout()\nplt.show","metadata":{"execution":{"iopub.execute_input":"2023-02-05T10:34:49.447269Z","iopub.status.busy":"2023-02-05T10:34:49.446884Z","iopub.status.idle":"2023-02-05T10:34:50.090863Z","shell.execute_reply":"2023-02-05T10:34:50.090135Z","shell.execute_reply.started":"2023-02-05T10:34:49.447230Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Matrice de confusion et performances** ","metadata":{}},{"cell_type":"code","source":"# on charge le modèle avec les meilleurs paramètres obtenus lors de l'entrainement\nInception_model.load_weights(BEST_MODEL_PATH)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T10:35:57.363258Z","iopub.status.busy":"2023-02-05T10:35:57.362495Z","iopub.status.idle":"2023-02-05T10:36:00.933506Z","shell.execute_reply":"2023-02-05T10:36:00.932515Z","shell.execute_reply.started":"2023-02-05T10:35:57.363213Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on represente la matrice de confusion sur le jeu de validation à partir\n# des classes vraies et prédictes\n\ncmdataset = get_validation_dataset(ordered=True)\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\n\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\ncm_probabilities = Inception_model.predict(images_ds)\ncm_predictions = np.argmax(cm_probabilities, axis=-1)\n\nlabels = range(len(CLASSES))\ncmat = confusion_matrix(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n)\ncmat = (cmat.T / cmat.sum(axis=1)).T # normalize\n\n\n# on visualize la matrice de confusion\nfig = plt.figure(figsize=(18, 14))\nsns.heatmap(cmat,\n xticklabels=CLASSES, yticklabels=CLASSES, annot=False, cmap='inferno')\nplt.xlabel('True label')\nplt.ylabel('Predicted label')\nplt.xticks(fontsize=8)\nplt.yticks(fontsize=8)\nplt.title('Confusion matrix with InceptionV3', fontsize=16)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T10:36:11.074115Z","iopub.status.busy":"2023-02-05T10:36:11.073753Z","iopub.status.idle":"2023-02-05T10:36:38.430306Z","shell.execute_reply":"2023-02-05T10:36:38.429159Z","shell.execute_reply.started":"2023-02-05T10:36:11.074078Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on calcule les métriques F1-score, precision et recall\n# et on les sauvegarde avec les paramètres du modèle\n# et les infos liées à l'entrainement\n\nInception_f1_score = f1_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nInception_precision = precision_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nInception_recall = recall_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n) \n\nInception_history = np.load('/kaggle/working/Inception_history.npy', allow_pickle=True).item()\n\nmax_train_accuracy = list(score_df['max_train_accuracy'])\nmax_val_accuracy = list(score_df['max_val_accuracy'])\nmin_val_loss = list(score_df['min_loss'])\nepochs = list(score_df['epochs'])\nF1_score = list(score_df['F1_score'])\nprecision = list(score_df['precision'])\nrecall = list(score_df['recall'])\n\nmax_train_accuracy.append(max(Inception_history['sparse_categorical_accuracy']))\nmax_train_accuracy = [round(num,3) for num in max_train_accuracy]\nmax_val_accuracy.append(max(Inception_history['val_sparse_categorical_accuracy']))\nmax_val_accuracy = [round(num,3) for num in max_val_accuracy]\nmin_val_loss.append(min(Inception_history['val_loss']))\nmin_val_loss = [round(num,3) for num in min_val_loss]\nepochs.append(len(Inception_history['val_loss']))\nF1_score.append(Inception_f1_score)\nF1_score = [round(num,3) for num in F1_score]\nprecision.append(Inception_precision)\nprecision = [round(num,3) for num in precision]\nrecall.append(Inception_recall)\nrecall = [round(num,3) for num in recall]\n\nscore = {'parameters': param_count, 'epochs': epochs, 'min_loss': min_val_loss, \n 'max_train_accuracy': max_train_accuracy, 'max_val_accuracy': max_val_accuracy, \n 'F1_score': F1_score, 'precision': precision, 'recall': recall}\nscore_df = pd.DataFrame(score,index=['ResNet50', 'InceptionV3'])\nscore_df.to_csv('score_df.csv',index=False), \nscore_df","metadata":{"execution":{"iopub.execute_input":"2023-02-05T10:36:43.418263Z","iopub.status.busy":"2023-02-05T10:36:43.417869Z","iopub.status.idle":"2023-02-05T10:36:43.462597Z","shell.execute_reply":"2023-02-05T10:36:43.461715Z","shell.execute_reply.started":"2023-02-05T10:36:43.418222Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Modèle Xception**","metadata":{}},{"cell_type":"markdown","source":" Xception est une extension de l’architecture Inception qui remplace le bloc Inception avec le bloc Modified Depthwise Separable Convolutions constitué d’une Pointwise Convolution suivie d’une Depthwise Convolution","metadata":{}},{"cell_type":"markdown","source":"\n\n\n
","metadata":{}},{"cell_type":"markdown","source":"Par rapport à la convolution conventionnelle, nous n’avons pas besoin d’effectuer la convolution sur tous les canaux. Cela signifie que le nombre de connexions est moindre et que le modèle est plus léger.","metadata":{}},{"cell_type":"markdown","source":"## **Chargement du modèle**","metadata":{}},{"cell_type":"code","source":"Xception_model = define_model(tf.keras.applications.Xception)\nscore_df = pd.read_csv('/kaggle/working/score_df.csv')","metadata":{"execution":{"iopub.execute_input":"2023-02-05T10:36:51.384658Z","iopub.status.busy":"2023-02-05T10:36:51.384212Z","iopub.status.idle":"2023-02-05T10:37:01.033423Z","shell.execute_reply":"2023-02-05T10:37:01.032491Z","shell.execute_reply.started":"2023-02-05T10:36:51.384619Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on sauvegarde l'info du nombre de paramètres du modèle\nparam_count = list(score_df['parameters'])\nparam_count.append(Inception_model.count_params())","metadata":{"execution":{"iopub.execute_input":"2023-02-05T10:37:02.685372Z","iopub.status.busy":"2023-02-05T10:37:02.685073Z","iopub.status.idle":"2023-02-05T10:37:02.700591Z","shell.execute_reply":"2023-02-05T10:37:02.699501Z","shell.execute_reply.started":"2023-02-05T10:37:02.685342Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# schéma du modèle\ntf.keras.utils.plot_model(Xception_model, show_shapes=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2023-02-05T10:37:06.021297Z","iopub.status.busy":"2023-02-05T10:37:06.020998Z","iopub.status.idle":"2023-02-05T10:37:06.247874Z","shell.execute_reply":"2023-02-05T10:37:06.246670Z","shell.execute_reply.started":"2023-02-05T10:37:06.021269Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Entrainement du modèle**","metadata":{}},{"cell_type":"code","source":"BEST_MODEL_PATH = \"/kaggle/working/Xception_model_best.h5\"\nFILE_DIR = os.path.dirname(BEST_MODEL_PATH)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T10:37:11.228965Z","iopub.status.busy":"2023-02-05T10:37:11.228622Z","iopub.status.idle":"2023-02-05T10:37:11.234521Z","shell.execute_reply":"2023-02-05T10:37:11.233324Z","shell.execute_reply.started":"2023-02-05T10:37:11.228917Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"earlystopping = tf.keras.callbacks.EarlyStopping(\n monitor='val_loss',\n min_delta=1e-4,\n patience=5,\n mode='auto',\n restore_best_weights=False,\n verbose=1)\n\ncheckpointer = tf.keras.callbacks.ModelCheckpoint(\n filepath=BEST_MODEL_PATH,\n options=None,\n monitor='val_loss',\n verbose=1,\n save_best_only=True,\n save_weights_only=True,\n mode='min')\n \ncallbacks = [earlystopping, lr_callback, checkpointer]\n\nEPOCHS = 100\nSTEPS_PER_EPOCH = NUM_TRAINING_IMAGES // BATCH_SIZE\n \nstart_time = timeit.default_timer()\n \nXception_history = Xception_model.fit(\n training_dataset,\n validation_data=validation_dataset,\n epochs=EPOCHS,\n steps_per_epoch=STEPS_PER_EPOCH,\n callbacks=callbacks,\n class_weight=dict_weights\n)\n\nXception_model_elapsed = timeit.default_timer() - start_time\ntrain_time.append(Xception_model_elapsed)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T10:37:15.553995Z","iopub.status.busy":"2023-02-05T10:37:15.553641Z","iopub.status.idle":"2023-02-05T11:02:14.761684Z","shell.execute_reply":"2023-02-05T11:02:14.760484Z","shell.execute_reply.started":"2023-02-05T10:37:15.553958Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on sauvegarde l'historique d'entrainement du modèle\nnp.save('Xception_history.npy',Xception_history.history)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T11:02:18.363487Z","iopub.status.busy":"2023-02-05T11:02:18.362917Z","iopub.status.idle":"2023-02-05T11:02:18.370248Z","shell.execute_reply":"2023-02-05T11:02:18.369151Z","shell.execute_reply.started":"2023-02-05T11:02:18.363431Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Performance du modèle**","metadata":{}},{"cell_type":"code","source":"# on visualise les courbes de loss et accuracy sur le jeu de training et\n# de validation pendant l'entrainement\nfig = plt.figure(figsize=[14, 7], dpi=80)\ni = 0\nfor metric in ['loss', 'sparse_categorical_accuracy']:\n i += 1\n fig.add_subplot(1, 2, i)\n ax = plt.plot(Xception_history.history[metric])\n plt.plot(Xception_history.history['val_' + metric])\n plt.title('Xception ' + metric,fontsize=16)\n plt.legend(['Training ' + metric, 'Validation ' + metric], loc='upper left')\n plt.ylabel(metric,fontsize=14)\n plt.xlabel('Epoch',fontsize=14)\nplt.tight_layout()\nplt.show","metadata":{"execution":{"iopub.execute_input":"2023-02-05T11:02:21.927220Z","iopub.status.busy":"2023-02-05T11:02:21.926903Z","iopub.status.idle":"2023-02-05T11:02:22.551614Z","shell.execute_reply":"2023-02-05T11:02:22.550888Z","shell.execute_reply.started":"2023-02-05T11:02:21.927191Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Matrice de confusion et performances**","metadata":{}},{"cell_type":"code","source":"# on charge le modèle avec les meilleurs paramètres obtenus lors de l'entrainement\nXception_model.load_weights(BEST_MODEL_PATH)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T11:02:28.812540Z","iopub.status.busy":"2023-02-05T11:02:28.811586Z","iopub.status.idle":"2023-02-05T11:02:31.137252Z","shell.execute_reply":"2023-02-05T11:02:31.136547Z","shell.execute_reply.started":"2023-02-05T11:02:28.812495Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on représente la matrice de confusion sur le jeu de validation à partir\n# des classes vraies et prédites\n\ncmdataset = get_validation_dataset(ordered=True)\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\n\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\ncm_probabilities = Xception_model.predict(images_ds)\ncm_predictions = np.argmax(cm_probabilities, axis=-1)\n\nlabels = range(len(CLASSES))\ncmat = confusion_matrix(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n)\ncmat = (cmat.T / cmat.sum(axis=1)).T # normalize\n\n\n# # on visualise la matrice de confusion\nfig = plt.figure(figsize=(18, 14))\nsns.heatmap(cmat,\n xticklabels=CLASSES, yticklabels=CLASSES, annot=False, cmap='inferno')\nplt.xlabel('True label')\nplt.ylabel('Predicted label')\nplt.xticks(fontsize=8)\nplt.yticks(fontsize=8)\nplt.title('Confusion matrix with Xception', fontsize=16)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T11:02:34.113157Z","iopub.status.busy":"2023-02-05T11:02:34.112803Z","iopub.status.idle":"2023-02-05T11:02:50.455223Z","shell.execute_reply":"2023-02-05T11:02:50.454188Z","shell.execute_reply.started":"2023-02-05T11:02:34.113119Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on calcule les métriques F1-score, precision et recall\n# et on les sauvegarde avec les paramètres du modèle\n# et les infos liées à l'entrainement\nXception_f1_score = f1_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nXception_precision = precision_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nXception_recall = recall_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\n\nXception_history = np.load('/kaggle/working/Xception_history.npy', allow_pickle=True).item()\n\nmax_train_accuracy = list(score_df['max_train_accuracy'])\nmax_val_accuracy = list(score_df['max_val_accuracy'])\nmin_val_loss = list(score_df['min_loss'])\nepochs = list(score_df['epochs'])\nF1_score = list(score_df['F1_score'])\nprecision = list(score_df['precision'])\nrecall = list(score_df['recall'])\n\nmax_train_accuracy.append(max(Xception_history['sparse_categorical_accuracy']))\nmax_train_accuracy = [round(num,3) for num in max_train_accuracy]\nmax_val_accuracy.append(max(Xception_history['val_sparse_categorical_accuracy']))\nmax_val_accuracy = [round(num,3) for num in max_val_accuracy]\nmin_val_loss.append(min(Xception_history['val_loss']))\nmin_val_loss = [round(num,3) for num in min_val_loss]\nepochs.append(len(Xception_history['val_loss']))\nF1_score.append(Xception_f1_score)\nF1_score = [round(num,3) for num in F1_score]\nprecision.append(Xception_precision)\nprecision = [round(num,3) for num in precision]\nrecall.append(Xception_recall)\nrecall = [round(num,3) for num in recall]\n\nscore = {'parameters': param_count, 'epochs': epochs, 'min_loss': min_val_loss, \n 'max_train_accuracy': max_train_accuracy, 'max_val_accuracy': max_val_accuracy, \n 'F1_score': F1_score, 'precision': precision, 'recall': recall}\nscore_df = pd.DataFrame(score,index=['ResNet50', 'InceptionV3', 'Xception'])\nscore_df.to_csv('score_df.csv', index=False), \nscore_df","metadata":{"execution":{"iopub.execute_input":"2023-02-05T11:03:00.814538Z","iopub.status.busy":"2023-02-05T11:03:00.814237Z","iopub.status.idle":"2023-02-05T11:03:00.862957Z","shell.execute_reply":"2023-02-05T11:03:00.861991Z","shell.execute_reply.started":"2023-02-05T11:03:00.814508Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Modèle Vision Transformer (ViT)**","metadata":{}},{"cell_type":"markdown","source":"Vision Transformer est un modèle de classification d’image récent (Oct 2020) qui remplace le réseau CNN (Convolutional Neural Network) avec une architecture transformer-based.\nLes Transformers ont représenté la base pour le développement d’algorithmes NLP très performants comme BERT, GPT-3 et ViT est la première implémentation des transformers qui a depassé le SOTA avec plusieurs benchmarks.\nL’architecture de ViT consiste des blocs suivants :","metadata":{}},{"cell_type":"markdown","source":"\n\n\n
","metadata":{}},{"cell_type":"markdown","source":"ViT fait un split de l’image avec sections de 16x16 pixels, non superposés. Les sections sont 'flattened' et convertis en embedded vectors. Une embedding classe trainable supplémentaire est ajoutée pour la classification. Une positional embedding est ensuite ajoutée aux embedded vectors de l'image pour garder une trace de la séquence. \nLe Transformer encoder est composé de plusieurs stacks de blocs identiques. Chaque bloc est doté d’une couche Attention Multi-Head suivie d’une couche Feed-Forward. Il y a une connexion résiduelle autour de chacune des deux sous-couches, suivie d’une couche de normalisation. Le Multi-Head Attention in Vision Transformers l’aide à ne prêter attention qu’à la partie pertinente de l’image. Toutes les sous-couches ainsi que les couches d'embedding dans le modèle produisent une sortie de dimension embedded D. Le vecteur en sortie de l’étape précédente d'embedding est passé par l’architecture du Transformer Encoder pour obtenir enfin le vecteur de contexte. ","metadata":{}},{"cell_type":"markdown","source":"\n\n\n
","metadata":{}},{"cell_type":"markdown","source":"Le vecteur c0 du context vector est envoyè à la tête MLP pour nous donner le vecteur de probabilité final pour la prédiction de la classe. \nLe modèle ViT n’a pas les mêmes propriétés des modèles CNN et donc demande un entrainement plus profond avec plus de data pour apprendre les mêmes caractéristiques d’un modèle CNN. \nEn revanche le modèle CNN est capable dans les premières layers de détecter exclusivement des caractéristiques locales de l’image et devient capables de détecter des caractéristiques globales sur l’image seulement dans les layers plus profonds différemment du modèle ViT qui peut extraire des caractéristiques globales du premier layer.\n","metadata":{}},{"cell_type":"markdown","source":"## **Chargement du modèle**","metadata":{}},{"cell_type":"code","source":"IMAGE_SIZE = 512\n\nK.clear_session()\nwith strategy.scope():\n pretrained_model = vit.vit_b16(\n image_size=IMAGE_SIZE,\n activation='softmax',\n pretrained=True,\n include_top=False,\n pretrained_top=False,\n classes=104)\n\n ViT_model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.Flatten(),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(11, activation=tfa.activations.gelu),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(104, 'softmax')], name='vision_transformer')\n\nlearning_rate = 1e-4\n\noptimizer = tfa.optimizers.RectifiedAdam(learning_rate=learning_rate)\n\nViT_model.compile(optimizer=optimizer,\n loss=tf.keras.losses.SparseCategoricalCrossentropy(),\n metrics=['sparse_categorical_accuracy'])\n\nViT_model.summary()\n\nscore_df = pd.read_csv('/kaggle/input/resnet50-inception-xception/score_df.csv')\nscore_df.at[2, 'parameters'] = 21074576\nscore_df","metadata":{"execution":{"iopub.execute_input":"2023-02-05T15:39:18.544128Z","iopub.status.busy":"2023-02-05T15:39:18.543365Z","iopub.status.idle":"2023-02-05T15:39:36.984944Z","shell.execute_reply":"2023-02-05T15:39:36.983857Z","shell.execute_reply.started":"2023-02-05T15:39:18.544077Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on sauvegarde l'info du nombre de paramètres du modèle\nparam_count = list(score_df['parameters'])\nparam_count.append(ViT_model.count_params())","metadata":{"execution":{"iopub.execute_input":"2023-02-05T15:39:40.502830Z","iopub.status.busy":"2023-02-05T15:39:40.502445Z","iopub.status.idle":"2023-02-05T15:39:40.514986Z","shell.execute_reply":"2023-02-05T15:39:40.513730Z","shell.execute_reply.started":"2023-02-05T15:39:40.502786Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# schéma du modèle\ntf.keras.utils.plot_model(ViT_model, show_shapes=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2023-02-05T15:39:42.954508Z","iopub.status.busy":"2023-02-05T15:39:42.954163Z","iopub.status.idle":"2023-02-05T15:39:44.348752Z","shell.execute_reply":"2023-02-05T15:39:44.347855Z","shell.execute_reply.started":"2023-02-05T15:39:42.954460Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Entrainement du modèle**","metadata":{}},{"cell_type":"code","source":"earlystopping = tf.keras.callbacks.EarlyStopping(\n monitor='val_loss',\n min_delta=1e-4,\n patience=10,\n mode='auto',\n restore_best_weights=False,\n verbose=1)\n\n\nBEST_MODEL_PATH = \"/kaggle/working/ViT_model_best.h5\"\nFILE_DIR = os.path.dirname(BEST_MODEL_PATH)\n\ncheckpointer = tf.keras.callbacks.ModelCheckpoint(\n filepath=BEST_MODEL_PATH,\n options=None,\n monitor='val_loss',\n verbose=1,\n save_best_only=True,\n save_weights_only=True,\n mode='min')\n\ncallbacks = [earlystopping, checkpointer]","metadata":{"execution":{"iopub.execute_input":"2023-02-05T15:39:46.807859Z","iopub.status.busy":"2023-02-05T15:39:46.807267Z","iopub.status.idle":"2023-02-05T15:39:46.815815Z","shell.execute_reply":"2023-02-05T15:39:46.814803Z","shell.execute_reply.started":"2023-02-05T15:39:46.807818Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"EPOCHS = 150\n\nSTEPS_PER_EPOCH = NUM_TRAINING_IMAGES // BATCH_SIZE\n\nViT_history = ViT_model.fit(\n training_dataset,\n validation_data=validation_dataset,\n epochs=EPOCHS,\n steps_per_epoch=STEPS_PER_EPOCH,\n callbacks=callbacks\n )","metadata":{"execution":{"iopub.execute_input":"2023-02-05T15:39:51.859411Z","iopub.status.busy":"2023-02-05T15:39:51.858960Z","iopub.status.idle":"2023-02-05T18:27:28.102213Z","shell.execute_reply":"2023-02-05T18:27:28.100974Z","shell.execute_reply.started":"2023-02-05T15:39:51.859379Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on sauvegarde l'historique d’entrainement du modèle\nnp.save('ViT_history.npy',ViT_history.history)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T18:27:32.029275Z","iopub.status.busy":"2023-02-05T18:27:32.028941Z","iopub.status.idle":"2023-02-05T18:27:32.037330Z","shell.execute_reply":"2023-02-05T18:27:32.035889Z","shell.execute_reply.started":"2023-02-05T18:27:32.029236Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Performance du modèle**","metadata":{}},{"cell_type":"code","source":"# on visualise les courbes de loss et accuracy sur le jeu de training et\n# de validation pendant l'entrainement\nfig = plt.figure(figsize=[14, 7], dpi=80)\ni = 0\nfor metric in ['loss', 'sparse_categorical_accuracy']:\n i += 1\n fig.add_subplot(1, 2, i)\n ax = plt.plot(ViT_history.history[metric])\n plt.plot(ViT_history.history['val_' + metric])\n plt.title('ViT ' + metric, fontsize=16)\n plt.legend(['Training ' + metric, 'Validation ' + metric],\n loc='upper left')\n plt.ylabel(metric, fontsize=14)\n plt.xlabel('Epoch', fontsize=14)\nplt.tight_layout()\nplt.show","metadata":{"execution":{"iopub.execute_input":"2023-02-05T18:27:39.866738Z","iopub.status.busy":"2023-02-05T18:27:39.866121Z","iopub.status.idle":"2023-02-05T18:27:40.448712Z","shell.execute_reply":"2023-02-05T18:27:40.447609Z","shell.execute_reply.started":"2023-02-05T18:27:39.866697Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Visuation des Attention Maps**","metadata":{}},{"cell_type":"markdown","source":"On visualise les Attention Maps sur l’image d’entrée à l’aide de la fonction « visualize.attention_map », qui calcule la moyenne des « Attention Weights » entre toutes les têtes du Multi-Head Self-Attention de chaque Encoder. ","metadata":{}},{"cell_type":"code","source":"from vit_keras import vit, utils, visualize\nimport PIL\nfrom PIL import Image\nimport cv2\n\nimage_size=512\nno_aug_train_batch = (next(iter(no_aug_train_set.unbatch().take(1))))\n\npretrained_model = vit.vit_b16(\n image_size=image_size,\n activation='softmax',\n pretrained=True,\n include_top=False,\n pretrained_top=False,\n classes=104)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T12:33:48.496774Z","iopub.status.busy":"2023-02-05T12:33:48.495637Z","iopub.status.idle":"2023-02-05T12:33:59.669150Z","shell.execute_reply":"2023-02-05T12:33:59.668105Z","shell.execute_reply.started":"2023-02-05T12:33:48.496730Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(18, 18))\nt=0\nfor i in range(2):\n image,label=next(iter(no_aug_train_set.unbatch().take(1)))\n numpy_image = image.numpy()\n plt.subplot(2, 4, 4*t + 1)\n plt.imshow(numpy_image)\n plt.axis('off')\n plt.title('Original image')\n img = PIL.Image.fromarray((numpy_image * 255).astype(np.uint8))\n img = img.convert('RGB')\n img.save(\"/kaggle/working/attention_map.jpg\")\n\n\n image = utils.read(\"/kaggle/working/attention_map.jpg\", image_size)\n attention_map = visualize.attention_map(model=pretrained_model, image=image)\n plt.subplot(2, 4, 4*t + 2)\n plt.imshow(attention_map)\n plt.axis('off')\n plt.title('Attention Map')\n \n image,label=next(iter(no_aug_train_set.unbatch().take(1)))\n numpy_image = image.numpy()\n plt.subplot(2, 4, 4*t + 3)\n plt.imshow(numpy_image)\n plt.axis('off')\n plt.title('Original image')\n img = PIL.Image.fromarray((numpy_image * 255).astype(np.uint8))\n img = img.convert('RGB')\n img.save(\"/kaggle/working/attention_map.jpg\")\n\n\n image = utils.read(\"/kaggle/working/attention_map.jpg\", image_size)\n attention_map = visualize.attention_map(model=pretrained_model, image=image)\n plt.subplot(2, 4, 4*t + 4)\n plt.imshow(attention_map)\n plt.axis('off')\n plt.title('Attention Map')\n t+=1\n ","metadata":{"execution":{"iopub.execute_input":"2023-02-05T12:34:05.819069Z","iopub.status.busy":"2023-02-05T12:34:05.818654Z","iopub.status.idle":"2023-02-05T12:34:56.982272Z","shell.execute_reply":"2023-02-05T12:34:56.981334Z","shell.execute_reply.started":"2023-02-05T12:34:05.819036Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Matrice de confusion et performances**","metadata":{}},{"cell_type":"code","source":"# on charge le modèle avec les meilleurs paramètres obtenus lors de l'entrainement\nViT_model.load_weights(BEST_MODEL_PATH)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T18:30:58.061315Z","iopub.status.busy":"2023-02-05T18:30:58.060986Z","iopub.status.idle":"2023-02-05T18:31:02.387910Z","shell.execute_reply":"2023-02-05T18:31:02.385660Z","shell.execute_reply.started":"2023-02-05T18:30:58.061282Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on représente la matrice de confusion sur le jeu de validation à partir\n# des classes vraies et prédites\ncmdataset = get_validation_dataset(ordered=True)\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\n\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\ncm_probabilities = ViT_model.predict(images_ds)\ncm_predictions = np.argmax(cm_probabilities, axis=-1)\n\nlabels = range(len(CLASSES))\ncmat = confusion_matrix(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n)\ncmat = (cmat.T / cmat.sum(axis=1)).T # normalize\n\n\n# on visualise la matrice de confusion\nfig = plt.figure(figsize=(18, 14))\nsns.heatmap(cmat,\n xticklabels=CLASSES, yticklabels=CLASSES, annot=False, cmap='inferno')\nplt.xlabel('True label')\nplt.ylabel('Predicted label')\nplt.xticks(fontsize=8)\nplt.yticks(fontsize=8)\nplt.title('Confusion matrix with ViT', fontsize=16)","metadata":{"execution":{"iopub.execute_input":"2023-02-05T18:31:04.708409Z","iopub.status.busy":"2023-02-05T18:31:04.708091Z","iopub.status.idle":"2023-02-05T18:31:27.864306Z","shell.execute_reply":"2023-02-05T18:31:27.863310Z","shell.execute_reply.started":"2023-02-05T18:31:04.708376Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on calcule les métriques F1-score, precision et recall\n# et on les sauvegarde avec les paramètres du modèle\n# et les infos liées à l'entrainement\nViT_f1_score = f1_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nViT_precision = precision_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nViT_recall = recall_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\n\nViT_history = np.load('/kaggle/working/ViT_history.npy', allow_pickle=True).item()\n\nmax_train_accuracy = list(score_df['max_train_accuracy'])\nmax_val_accuracy = list(score_df['max_val_accuracy'])\nmin_val_loss = list(score_df['min_loss'])\nepochs = list(score_df['epochs'])\nF1_score = list(score_df['F1_score'])\nprecision = list(score_df['precision'])\nrecall = list(score_df['recall'])\n\nmax_train_accuracy.append(max(ViT_history['sparse_categorical_accuracy']))\nmax_train_accuracy = [round(num,3) for num in max_train_accuracy]\nmax_val_accuracy.append(max(ViT_history['val_sparse_categorical_accuracy']))\nmax_val_accuracy = [round(num,3) for num in max_val_accuracy]\nmin_val_loss.append(min(ViT_history['val_loss']))\nmin_val_loss = [round(num,3) for num in min_val_loss]\nepochs.append(len(ViT_history['val_loss']))\nF1_score.append(ViT_f1_score)\nF1_score = [round(num,3) for num in F1_score]\nprecision.append(ViT_precision)\nprecision = [round(num,3) for num in precision]\nrecall.append(ViT_recall)\nrecall = [round(num,3) for num in recall]\nscore = {'parameters': param_count, 'epochs': epochs, 'min_loss': min_val_loss, \n 'max_train_accuracy': max_train_accuracy, \n 'max_val_accuracy': max_val_accuracy, \n 'F1_score': F1_score, \n 'precision': precision, \n 'recall': recall}\nscore_df = pd.DataFrame(score,index=['ResNet50','InceptionV3','Xception','ViT'])\nscore_df.to_csv('score_df.csv',index=False), \nscore_df","metadata":{"execution":{"iopub.execute_input":"2023-02-05T18:31:40.561365Z","iopub.status.busy":"2023-02-05T18:31:40.561064Z","iopub.status.idle":"2023-02-05T18:31:40.610346Z","shell.execute_reply":"2023-02-05T18:31:40.609415Z","shell.execute_reply.started":"2023-02-05T18:31:40.561328Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Modèle EfficientNetB7**","metadata":{}},{"cell_type":"markdown","source":"EfficientNet est un groupe de modèles de réseau convolutionnel qui a permis d’atteindre très bonnes performances avec la base de données Imagenet avec très peu de paramètres par rapport aux autres modèles. Le groupe de modèles EfficientNet comprend 8 modèles, de B0-B7, où chaque modèle renvoie chronologiquement à une plus grande précision et à un plus grand nombre de paramètres. \nEfficientNet définit une architecture de base très simple EfficientNet-B0 et une méthode efficace, le ‘Compound Scaling’, pour augmenter la taille du modèle afin d’obtenir une amélioration des performances.\nMBConv est le bloc principal du modèle EfficientNet. MBConv est constitué d’une ‘Shortcut Connection’ entre le début et la fin d’un bloc convolutif.","metadata":{}},{"cell_type":"markdown","source":"\n\n\n
\n

\n\n

","metadata":{}},{"cell_type":"markdown","source":"## **Chargement du modèle**","metadata":{}},{"cell_type":"code","source":"with strategy.scope():\n load_locally = tf.saved_model.LoadOptions(experimental_io_device='/job:localhost')\n pretrained_model = hub.KerasLayer(\n 'https://tfhub.dev/tensorflow/efficientnet/b7/feature-vector/1', \n trainable=True, input_shape=[*IMAGE_SIZE, 3], load_options=load_locally)\n EfficientNetB7_model = tf.keras.Sequential([\n # le format demandé pour les images pour les modèles TFHub est float32 dans [0,1]\n tf.keras.layers.Lambda(lambda data: tf.image.convert_image_dtype(data, tf.float32),\n input_shape=[*IMAGE_SIZE, 3]),\n pretrained_model, \n tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n ])\nEfficientNetB7_model.compile(\n optimizer='adam',\n loss = 'sparse_categorical_crossentropy',\n metrics=['sparse_categorical_accuracy'],\n)\n\nEfficientNetB7_model.summary()\n\nscore_df = pd.read_csv('/kaggle/input/vision-transformer/score_df.csv')\nscore_df","metadata":{"_kg_hide-output":true,"execution":{"iopub.execute_input":"2023-02-07T14:46:40.113222Z","iopub.status.busy":"2023-02-07T14:46:40.112744Z","iopub.status.idle":"2023-02-07T14:47:40.489566Z","shell.execute_reply":"2023-02-07T14:47:40.488406Z","shell.execute_reply.started":"2023-02-07T14:46:40.113187Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on sauvegarde l'info du nombre de paramètres du modèle\nparam_count = list(score_df['parameters'])\nparam_count.append(EfficientNetB7_model.count_params())","metadata":{"execution":{"iopub.execute_input":"2023-02-07T14:47:50.291181Z","iopub.status.busy":"2023-02-07T14:47:50.290773Z","iopub.status.idle":"2023-02-07T14:47:50.314765Z","shell.execute_reply":"2023-02-07T14:47:50.313288Z","shell.execute_reply.started":"2023-02-07T14:47:50.291142Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# schéma du modèle\ntf.keras.utils.plot_model(EfficientNetB7_model, show_shapes=True)","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2023-02-07T14:48:03.936616Z","iopub.status.busy":"2023-02-07T14:48:03.936302Z","iopub.status.idle":"2023-02-07T14:48:05.380442Z","shell.execute_reply":"2023-02-07T14:48:05.379466Z","shell.execute_reply.started":"2023-02-07T14:48:03.936585Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Entrainement du modèle**","metadata":{}},{"cell_type":"code","source":"earlystopping = tf.keras.callbacks.EarlyStopping(\n monitor='val_loss',\n min_delta=1e-4,\n patience=5,\n mode='auto',\n restore_best_weights=False,\n verbose=1)\n\n\nBEST_MODEL_PATH = \"/kaggle/working/EfficientNetB7_model_best.h5\"\nFILE_DIR = os.path.dirname(BEST_MODEL_PATH)\n\ncheckpointer = tf.keras.callbacks.ModelCheckpoint(\n filepath=BEST_MODEL_PATH,\n options=None,\n monitor='val_loss',\n verbose=1,\n save_best_only=True,\n save_weights_only=True,\n mode='min')\n\ncallbacks = [earlystopping, lr_callback, checkpointer]","metadata":{"execution":{"iopub.execute_input":"2023-02-07T14:48:09.422930Z","iopub.status.busy":"2023-02-07T14:48:09.422323Z","iopub.status.idle":"2023-02-07T14:48:09.431141Z","shell.execute_reply":"2023-02-07T14:48:09.429674Z","shell.execute_reply.started":"2023-02-07T14:48:09.422848Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"EPOCHS = 150\n\nSTEPS_PER_EPOCH = NUM_TRAINING_IMAGES // BATCH_SIZE\n\nEfficientNetB7_history = EfficientNetB7_model.fit(\n training_dataset,\n validation_data=validation_dataset,\n epochs=EPOCHS,\n steps_per_epoch=STEPS_PER_EPOCH,\n callbacks=callbacks,\n class_weight=dict_weights\n)","metadata":{"execution":{"iopub.execute_input":"2023-02-07T14:48:15.323073Z","iopub.status.busy":"2023-02-07T14:48:15.322707Z","iopub.status.idle":"2023-02-07T16:14:36.341572Z","shell.execute_reply":"2023-02-07T16:14:36.340304Z","shell.execute_reply.started":"2023-02-07T14:48:15.323037Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on sauvegarde l'historique d’entrainement du modèle\nnp.save('EfficientNetB7_history.npy',EfficientNetB7_history.history)","metadata":{"execution":{"iopub.execute_input":"2023-02-07T16:18:36.899343Z","iopub.status.busy":"2023-02-07T16:18:36.898692Z","iopub.status.idle":"2023-02-07T16:18:36.907731Z","shell.execute_reply":"2023-02-07T16:18:36.906655Z","shell.execute_reply.started":"2023-02-07T16:18:36.899298Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Performance du modèle**","metadata":{}},{"cell_type":"code","source":"# on visualise les courbes de loss et accuracy sur le jeu de training et\n# de validation pendant l'entrainement\nfig = plt.figure(figsize=[14, 7], dpi=80)\ni = 0\nfor metric in ['loss', 'sparse_categorical_accuracy']:\n i += 1\n fig.add_subplot(1, 2, i)\n ax = plt.plot(EfficientNetB7_history.history[metric])\n plt.plot(EfficientNetB7_history.history['val_' + metric])\n plt.title('EfficientNetB7 ' + metric, fontsize=16)\n plt.legend(['Training '+ metric, 'Validation ' + metric],\n loc='upper left')\n plt.ylabel(metric, fontsize=14)\n plt.xlabel('Epoch', fontsize=14)\nplt.tight_layout()\nplt.show","metadata":{"execution":{"iopub.execute_input":"2023-02-07T16:18:40.600851Z","iopub.status.busy":"2023-02-07T16:18:40.600342Z","iopub.status.idle":"2023-02-07T16:18:41.284995Z","shell.execute_reply":"2023-02-07T16:18:41.284136Z","shell.execute_reply.started":"2023-02-07T16:18:40.600812Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Matrice de confusion et performances**","metadata":{}},{"cell_type":"code","source":"# on charge le modèle avec les meilleurs paramètres obtenus lors de l'entrainement\nEfficientNetB7_model.load_weights(BEST_MODEL_PATH)","metadata":{"execution":{"iopub.execute_input":"2023-02-07T16:19:00.249550Z","iopub.status.busy":"2023-02-07T16:19:00.248511Z","iopub.status.idle":"2023-02-07T16:19:11.467590Z","shell.execute_reply":"2023-02-07T16:19:11.466353Z","shell.execute_reply.started":"2023-02-07T16:19:00.249479Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on représente la matrice de confusion sur le jeu de validation à partir\n# des classes vraies et prédites\ncmdataset = get_validation_dataset(ordered=True)\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\n\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\ncm_probabilities = EfficientNetB7_model.predict(images_ds)\ncm_predictions = np.argmax(cm_probabilities, axis=-1)\n\nlabels = range(len(CLASSES))\ncmat = confusion_matrix(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n)\ncmat = (cmat.T / cmat.sum(axis=1)).T # normalize\n\n\n# on visualise la matrice de confusion\nfig = plt.figure(figsize=(18, 14))\nsns.heatmap(cmat,\n xticklabels=CLASSES, yticklabels=CLASSES, annot=False, cmap='inferno')\nplt.xlabel('True label')\nplt.ylabel('Predicted label')\nplt.xticks(fontsize=8)\nplt.yticks(fontsize=8)\nplt.title('Confusion matrix with EfficientNetB7', fontsize=16)","metadata":{"execution":{"iopub.execute_input":"2023-02-07T16:19:55.131602Z","iopub.status.busy":"2023-02-07T16:19:55.130955Z","iopub.status.idle":"2023-02-07T16:20:08.466571Z","shell.execute_reply":"2023-02-07T16:20:08.465594Z","shell.execute_reply.started":"2023-02-07T16:19:55.131561Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on calcule les métriques F1-score, precision et recall\n# et on les sauvegarde avec les paramètres du modèle\n# et les infos liées à l'entrainement\nEfficientNetB7_f1_score = f1_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nEfficientNetB7_precision = precision_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nEfficientNetB7_recall = recall_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\n\nEfficientNetB7_history = np.load('/kaggle/working/EfficientNetB7_history.npy', \n allow_pickle=True).item()\n\nmax_train_accuracy = list(score_df['max_train_accuracy'])\nmax_val_accuracy = list(score_df['max_val_accuracy'])\nmin_val_loss = list(score_df['min_loss'])\nepochs = list(score_df['epochs'])\nF1_score = list(score_df['F1_score'])\nprecision = list(score_df['precision'])\nrecall = list(score_df['recall'])\n\nmax_train_accuracy.append(max(EfficientNetB7_history['sparse_categorical_accuracy']))\nmax_train_accuracy = [round(num,3) for num in max_train_accuracy]\nmax_val_accuracy.append(max(EfficientNetB7_history['val_sparse_categorical_accuracy']))\nmax_val_accuracy = [round(num,3) for num in max_val_accuracy]\nmin_val_loss.append(min(EfficientNetB7_history['val_loss']))\nmin_val_loss = [round(num,3) for num in min_val_loss]\nepochs.append(len(EfficientNetB7_history['val_loss']))\nF1_score.append(EfficientNetB7_f1_score)\nF1_score = [round(num,3) for num in F1_score]\nprecision.append(EfficientNetB7_precision)\nprecision = [round(num,3) for num in precision]\nrecall.append(EfficientNetB7_recall)\nrecall = [round(num,3) for num in recall]\nscore = {'parameters': param_count, 'epochs': epochs, 'min_loss': min_val_loss, \n 'max_train_accuracy': max_train_accuracy, \n 'max_val_accuracy': max_val_accuracy, \n 'F1_score': F1_score, \n 'precision': precision, \n 'recall': recall}\nscore_df = pd.DataFrame(score,index=['ResNet50','InceptionV3','Xception','ViT','EfficientNetB7'])\nscore_df.to_csv('score_df.csv',index=False), \nscore_df","metadata":{"execution":{"iopub.execute_input":"2023-02-07T16:20:16.093936Z","iopub.status.busy":"2023-02-07T16:20:16.093564Z","iopub.status.idle":"2023-02-07T16:20:16.150211Z","shell.execute_reply":"2023-02-07T16:20:16.149104Z","shell.execute_reply.started":"2023-02-07T16:20:16.093884Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Evaluation des performances des modèles**","metadata":{}},{"cell_type":"markdown","source":"On compare les performances des differents modèles. ","metadata":{}},{"cell_type":"code","source":"fontsize1 = 18\nfontsize2 = 16\nfontsize3 = 14\nfontsize4 = 12\n\nprint(score.keys())\nwidth = 0.2\n\nfig, axes = plt.subplots(nrows=1, ncols=2, figsize=(20,7))\n\nbarchart1 = score_df.plot.bar(ax=axes[0], y='parameters', width=width*2, rot=0, \n fontsize=fontsize2)\nbarchart1.bar_label(axes[0].containers[0], label_type='edge', padding=15)\naxes[0].set_ylim(top=10e+07)\naxes[0].set_ylabel('count', fontsize=fontsize3)\naxes[0].set_title('Number of parameters', fontsize=fontsize1)\naxes[0].set_xticklabels(['ResNet50', 'InceptionV3', 'Xception', 'ViT', 'EfficientNetB7'], \n fontsize=fontsize2)\naxes[0].legend(fontsize=fontsize4)\n \nbarchart2 = score_df.plot.bar(ax=axes[1], y='epochs', ylabel='count', \n width=width*2, rot=0, fontsize=fontsize2)\nbarchart2.bar_label(axes[1].containers[0], label_type='edge', padding=15, fontsize = fontsize4)\naxes[1].set_ylim(top=130)\naxes[1].set_ylabel('count', fontsize=fontsize3)\naxes[1].set_title('Epochs', fontsize=fontsize1)\naxes[1].set_xticklabels(['ResNet50', 'InceptionV3', 'Xception', 'ViT', 'EfficientNetB7'], \n fontsize=fontsize2)\naxes[1].legend(fontsize=fontsize4)\n\nfig, axes = plt.subplots(nrows=2, ncols=1, figsize=(20,15))\n\nbarchart3 = score_df.plot.bar(ax=axes[0], y=['min_loss', 'max_train_accuracy', 'max_val_accuracy'], \n width=width*3, rot=0, fontsize=fontsize2)\nbarchart3.bar_label(axes[0].containers[0], label_type='edge', padding=15, fontsize = fontsize4)\nbarchart3.bar_label(axes[0].containers[1], label_type='edge', padding=15, fontsize = fontsize4)\nbarchart3.bar_label(axes[0].containers[2], label_type='edge', padding=15, fontsize = fontsize4)\naxes[0].set_ylim(top=1.6)\naxes[0].set_title('Loss vs Accuracy', fontsize=fontsize1)\naxes[0].set_xticklabels(['ResNet50','InceptionV3','Xception','ViT', 'EfficientNetB7'], \n fontsize=fontsize2)\naxes[0].legend(fontsize=fontsize4)\n\nbarchart4 = score_df.plot.bar(ax=axes[1], y=['precision', 'recall', 'F1_score'], \n width=width*3, rot=0, fontsize=fontsize2)\nbarchart4.bar_label(axes[1].containers[0], label_type='edge', padding=15, fontsize = fontsize4)\nbarchart4.bar_label(axes[1].containers[1], label_type='edge', padding=15, fontsize = fontsize4)\nbarchart4.bar_label(axes[1].containers[2], label_type='edge', padding=15, fontsize = fontsize4)\naxes[1].set_ylim(bottom=0.5, top=1.1)\naxes[1].set_title('Precision, Recall, F1-Score', fontsize=fontsize1)\naxes[1].set_xticklabels(['ResNet50', 'InceptionV3', 'Xception', 'ViT', 'EfficientNetB7'], \n fontsize=fontsize2)\naxes[1].legend(fontsize=fontsize4)","metadata":{"execution":{"iopub.execute_input":"2023-02-07T16:28:24.191745Z","iopub.status.busy":"2023-02-07T16:28:24.191410Z","iopub.status.idle":"2023-02-07T16:28:25.852943Z","shell.execute_reply":"2023-02-07T16:28:25.851678Z","shell.execute_reply.started":"2023-02-07T16:28:24.191709Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Le modèle de classification plus performant est donc Vision Transformer suivi par EfficientNetB7 et Xception. Vision Transformer est le modèle plus complexe en termes de nombre de paramètres et celui qui a été entrainé plus longtemps. ","metadata":{}},{"cell_type":"markdown","source":"# **Modèle Ensemble: Xception et Vision Transformer**","metadata":{}},{"cell_type":"markdown","source":"Afin de maximiser la performance sur le jeu de test on utilise les Ensembles Models qui sont des modèles prédictifs qui combinent les prévisions de deux ou plusieurs autres modèles. Les modèles qui contribuent à l’ensemble, peuvent être du même type ou de types différents et peuvent ou non être formés sur les mêmes données de formation. Les prédictions faites par les membres de l’ensemble peuvent être combinées à l’aide de statistiques, comme le mode ou la moyenne. \nIl y a deux raisons principales d’utiliser un ensemble plutôt qu’un seul modèle, et elles sont liées : \n* Performance : un ensemble peut faire de meilleures prévisions et obtenir de meilleures performances que n’importe quel modèle contributif.\n* Robustesse : un ensemble réduit la dispersion des prédictions et de la performance du modèle.","metadata":{}},{"cell_type":"markdown","source":"

","metadata":{}},{"cell_type":"markdown","source":"Les modèles Ensemble sont divisé en quatre catégories :\n* Sequential methods – Méthodes avec ‘base learners’ qui utilisent données d’entrée qui dépendent du ‘base learner’ précèdent (BOOSTING)\n* Parallel methods - Méthodes qui utilisent données d’entrée indépendants (RANDOM FOREST)\n* Homogeneous Ensemble” : combinaison de la même typologie de classifieur\n* Heterogeneous Ensemble” : combinaison de classifieurs de typologies différentes","metadata":{}},{"cell_type":"markdown","source":"## **Technique Weighted Average**","metadata":{}},{"cell_type":"markdown","source":"On a utilisé une méthode parallèle avec un Heterogeneous ensemble car on a combiné avec une moyenne pesée les résultats des classifieurs Xception et Vision Transformer et EfficientNetB7 et Vision Transformer, tous entrainées sur le même jeu de données.\nLes poids de la moyenne pesées des résultats ont été calculés sur la base des prédictions sur le jeu de validation des deux modèles afin de maximiser le F1-score.","metadata":{}},{"cell_type":"code","source":"# on charge les modèles Xception et ViT avec le meilleur jeu de paramètres\nXception_model = define_model(tf.keras.applications.Xception)\nXception_model.load_weights('/kaggle/input/resnet50-inception-xception/Xception_model_best.h5')\n\nIMAGE_SIZE = 512\nwith strategy.scope():\n pretrained_model = vit.vit_b16(\n image_size=IMAGE_SIZE,\n activation='softmax',\n pretrained=True,\n include_top=False,\n pretrained_top=False,\n classes=104)\n\n ViT_model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.Flatten(),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(11, activation=tfa.activations.gelu),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(104, 'softmax')], name='vision_transformer')\n \nViT_model.summary()\n\nViT_model.load_weights('/kaggle/input/vision-transformer/ViT_model_best.h5')\n\nscore_df = pd.read_csv('/kaggle/input/efficientnetb7/score_df.csv')\nscore_df","metadata":{"_kg_hide-output":true,"execution":{"iopub.execute_input":"2023-02-09T11:27:32.583846Z","iopub.status.busy":"2023-02-09T11:27:32.583547Z","iopub.status.idle":"2023-02-09T11:28:19.188911Z","shell.execute_reply":"2023-02-09T11:28:19.188150Z","shell.execute_reply.started":"2023-02-09T11:27:32.583812Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on définit le meilleur paramètre alpha pour la moyenne pesée des prédictions des deux modèles\nIMAGE_SIZE = [512,512]\ncmdataset = get_validation_dataset(ordered=True)\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\nprint('Predicting with ViT_best_model')\nm1 = ViT_model.predict(images_ds)\nprint('Predicting with Xception_best_model')\nm2 = Xception_model.predict(images_ds)\n\nscores = []\nfor alpha in np.linspace(0,1,100):\n cm_probabilities = alpha*m1 + (1-alpha)*m2\n cm_predictions = np.argmax(cm_probabilities, axis=-1)\n scores.append(f1_score(cm_correct_labels, cm_predictions, \n labels=range(len(CLASSES)), \n average='macro'))\n\nbest_alpha = np.argmax(scores) / 100\n# on calcule les prédictions des deux modèles\ncm_probabilities = best_alpha*m1 + (1-best_alpha)*m2\ncm_predictions = np.argmax(cm_probabilities, axis=-1)\n\nprint(best_alpha, max(scores))","metadata":{"execution":{"iopub.execute_input":"2023-02-09T11:28:19.192002Z","iopub.status.busy":"2023-02-09T11:28:19.191703Z","iopub.status.idle":"2023-02-09T11:28:46.873267Z","shell.execute_reply":"2023-02-09T11:28:46.871656Z","shell.execute_reply.started":"2023-02-09T11:28:19.191961Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on visualise la paerformance du modèle Ensemble Xception-ViT\n# avec la variation de alpha\nprint(\"Correct labels: \", cm_correct_labels.shape, cm_correct_labels)\nprint(\"Predicted labels: \", cm_predictions.shape, cm_predictions)\nplt.plot(scores)\nplt.axvline(x=best_alpha*100, color='red')\nplt.xlabel('alpha')\nplt.ylabel('F1-Score')\nplt.title('Best alpha of ensemble model Xception-ViT')","metadata":{"execution":{"iopub.execute_input":"2023-02-09T11:28:58.504928Z","iopub.status.busy":"2023-02-09T11:28:58.504333Z","iopub.status.idle":"2023-02-09T11:28:58.685492Z","shell.execute_reply":"2023-02-09T11:28:58.683788Z","shell.execute_reply.started":"2023-02-09T11:28:58.504896Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Matrice de confusion et performance**","metadata":{}},{"cell_type":"code","source":"# on représente la matrice de confusion sur le jeu de validation à partir\n# des classes vraies et prédites\nimport seaborn as sns\nfrom sklearn.metrics import classification_report\n\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\ncm_probabilities = best_alpha*m1+(1-best_alpha)*m2\ncm_predictions = np.argmax(cm_probabilities, axis=-1)\n\n\nlabels = range(len(CLASSES))\ncmat = confusion_matrix(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n)\ncmat = (cmat.T / cmat.sum(axis=1)).T\n\n\n# on visualise la matrice de confusion\nfig = plt.figure(figsize=(18, 14))\nsns.heatmap(cmat,\n xticklabels=CLASSES, yticklabels=CLASSES, annot=False, cmap='inferno')\nplt.xlabel('True label')\nplt.ylabel('Predicted label')\nplt.xticks(fontsize=8)\nplt.yticks(fontsize=8)\nplt.title('Confusion matrix with Ensemble model Xception-Vision transformer', \n fontsize=16)","metadata":{"execution":{"iopub.execute_input":"2023-02-09T11:29:06.544389Z","iopub.status.busy":"2023-02-09T11:29:06.544118Z","iopub.status.idle":"2023-02-09T11:29:10.489327Z","shell.execute_reply":"2023-02-09T11:29:10.488393Z","shell.execute_reply.started":"2023-02-09T11:29:06.544361Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on calcule les métriques F1-score, precision et recall\n# et on les sauvegarde\nXception_ViT_f1_score = f1_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nXception_ViT_precision = precision_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nXception_ViT_recall = recall_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\n\nF1_score = list(score_df['F1_score'])\nprecision = list(score_df['precision'])\nrecall = list(score_df['recall'])\n\nF1_score.append(Xception_ViT_f1_score)\nF1_score = [round(num,3) for num in F1_score]\nprecision.append(Xception_ViT_precision)\nprecision = [round(num,3) for num in precision]\nrecall.append(Xception_ViT_recall)\nrecall = [round(num,3) for num in recall]\nscore_Xception_ViT = {'F1_score': F1_score, \n 'precision': precision, \n 'recall': recall}\nscore_Xception_ViT_df = pd.DataFrame(score_Xception_ViT,index=['ResNet50','InceptionV3','Xception','ViT', \n 'EfficientNetB7','Ensemble Xception_ViT'])\nscore_Xception_ViT_df.to_csv('score_Xception_ViT_df.csv',index=False), \nscore_Xception_ViT_df\n\n# on visualise les performances des modèles\nfontsize1 = 18\nfontsize2 = 16\nfontsize3 = 14\nfontsize4 = 12\n\nwidth = 0.2\n\nfig, axes = plt.subplots(nrows=1, ncols=1, figsize=(20,7))\n\nbarchart = score_Xception_ViT_df.plot.bar(ax = axes, y=['precision', 'recall', 'F1_score'], \n width=width*3, \n rot=0, \n fontsize=fontsize2)\nbarchart.bar_label(axes.containers[0], \n label_type='edge', \n padding=15, \n fontsize = fontsize4)\nbarchart.bar_label(axes.containers[1], \n label_type='edge', \n padding=15, \n fontsize = fontsize4)\nbarchart.bar_label(axes.containers[2], \n label_type='edge', \n padding=15, \n fontsize=fontsize4)\naxes.set_ylim(bottom=0.5, \n top=1.1)\naxes.set_title('Precision, Recall, F1-Score', fontsize=fontsize1)\naxes.set_xticklabels(['ResNet50','InceptionV3','Xception','ViT', 'EfficientNetB7', 'Ensemble Xception_ViT'], \n fontsize=fontsize2)\naxes.legend(fontsize=fontsize4)","metadata":{"execution":{"iopub.execute_input":"2023-02-09T11:29:38.908981Z","iopub.status.busy":"2023-02-09T11:29:38.907101Z","iopub.status.idle":"2023-02-09T11:29:39.350443Z","shell.execute_reply":"2023-02-09T11:29:39.349217Z","shell.execute_reply.started":"2023-02-09T11:29:38.908883Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"score_Xception_ViT_df = pd.read_csv('/kaggle/input/ensemble-xcept-vit/score_Xception_ViT_df.csv')\nscore_Xception_ViT_df.rename(index={0:'ResNet50',1:'InceptionV3',2:'Xception',3:'ViT',4:'EfficientNetB7', 5:'Xception_ViT'})","metadata":{"execution":{"iopub.execute_input":"2023-02-09T12:14:06.527041Z","iopub.status.busy":"2023-02-09T12:14:06.526723Z","iopub.status.idle":"2023-02-09T12:14:06.549490Z","shell.execute_reply":"2023-02-09T12:14:06.548561Z","shell.execute_reply.started":"2023-02-09T12:14:06.527004Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Modèle Ensemble: EfficientNetB7 et Vision Transformer**","metadata":{}},{"cell_type":"code","source":"# on charge les modèles EfficientNetB7 et ViT avec le meilleur jeu de paramètres\nwith strategy.scope():\n IMAGE_SIZE = [512,512]\n load_locally = tf.saved_model.LoadOptions(experimental_io_device='/job:localhost')\n pretrained_model = hub.KerasLayer(\n 'https://tfhub.dev/tensorflow/efficientnet/b7/feature-vector/1', \n trainable=True, input_shape=[*IMAGE_SIZE, 3], load_options=load_locally)\n EfficientNetB7_model = tf.keras.Sequential([\n # the expected image format for all TFHub image models is float32 in [0,1) range\n tf.keras.layers.Lambda(lambda data: tf.image.convert_image_dtype(data, tf.float32), \n input_shape=[*IMAGE_SIZE, 3]),\n pretrained_model, \n tf.keras.layers.Dense(len(CLASSES), activation='softmax')\n ])\n \n IMAGE_SIZE = 512\n pretrained_model = vit.vit_b16(\n image_size=IMAGE_SIZE,\n activation='softmax',\n pretrained=True,\n include_top=False,\n pretrained_top=False,\n classes=104)\n\n ViT_model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.Flatten(),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(11, activation=tfa.activations.gelu),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(104, 'softmax')], name='vision_transformer')\n \nEfficientNetB7_model.summary() \nViT_model.summary()\n\nEfficientNetB7_model.load_weights('/kaggle/input/efficientnetb7/EfficientNetB7_model_best.h5')\nViT_model.load_weights('/kaggle/input/vision-transformer/ViT_model_best.h5')\n\nscore_df = pd.read_csv('/kaggle/input/ensemble-xcept-vit/score_Xception_ViT_df.csv')\nscore_df","metadata":{"execution":{"iopub.execute_input":"2023-02-09T12:14:11.394530Z","iopub.status.busy":"2023-02-09T12:14:11.394228Z","iopub.status.idle":"2023-02-09T12:15:48.682116Z","shell.execute_reply":"2023-02-09T12:15:48.680186Z","shell.execute_reply.started":"2023-02-09T12:14:11.394499Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on définit le meilleur paramètre alpha pour la moyenne pesée des prédictions des deux modèles\nIMAGE_SIZE = [512,512]\ncmdataset = get_validation_dataset(ordered=True)\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\nprint('Predicting with ViT_best_model')\nm1 = ViT_model.predict(images_ds)\nprint('Predicting with EfficientNetB7_best_model')\nm2 = EfficientNetB7_model.predict(images_ds)\n\nscores = []\nfor alpha in np.linspace(0,1,100):\n cm_probabilities = alpha*m1 + (1-alpha)*m2\n cm_predictions = np.argmax(cm_probabilities, axis=-1)\n scores.append(f1_score(cm_correct_labels, cm_predictions, \n labels=range(len(CLASSES)), \n average='macro'))\n\nbest_alpha = np.argmax(scores) / 100\ncm_probabilities = best_alpha*m1 + (1-best_alpha)*m2\ncm_predictions = np.argmax(cm_probabilities, axis=-1)\n\nprint(best_alpha, max(scores))","metadata":{"execution":{"iopub.execute_input":"2023-02-09T12:15:48.685152Z","iopub.status.busy":"2023-02-09T12:15:48.684556Z","iopub.status.idle":"2023-02-09T12:16:28.630435Z","shell.execute_reply":"2023-02-09T12:16:28.629435Z","shell.execute_reply.started":"2023-02-09T12:15:48.685116Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on visualise la performance du modèle Ensemble EfficientNetB7-ViT\n# avec la variation de alpha\nprint(\"Correct labels: \", cm_correct_labels.shape, cm_correct_labels)\nprint(\"Predicted labels: \", cm_predictions.shape, cm_predictions)\nplt.plot(scores)\nplt.axvline(x=best_alpha*100, color='red')\nplt.xlabel('alpha')\nplt.ylabel('F1-Score')\nplt.title('Best alpha of ensemble model EfficientNetB7-ViT')","metadata":{"execution":{"iopub.execute_input":"2023-02-07T18:33:06.133013Z","iopub.status.busy":"2023-02-07T18:33:06.132412Z","iopub.status.idle":"2023-02-07T18:33:06.421447Z","shell.execute_reply":"2023-02-07T18:33:06.420540Z","shell.execute_reply.started":"2023-02-07T18:33:06.132976Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Matrice de confusion et performance**","metadata":{}},{"cell_type":"code","source":"# on représente la matrice de confusion sur le jeu de validation à partir\n# des classes vraies et prédites\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\ncm_probabilities = best_alpha*m1+(1-best_alpha)*m2\ncm_predictions = np.argmax(cm_probabilities, axis=-1)\n\n\nlabels = range(len(CLASSES))\ncmat = confusion_matrix(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n)\ncmat = (cmat.T / cmat.sum(axis=1)).T\n\n\n# on visualise la matrice de confusion\nfig = plt.figure(figsize=(18, 14))\nsns.heatmap(cmat,\n xticklabels=CLASSES, yticklabels=CLASSES, annot=False, cmap='inferno')\nplt.xlabel('True label')\nplt.ylabel('Predicted label')\nplt.xticks(fontsize=8)\nplt.yticks(fontsize=8)\nplt.title('Confusion matrix with Ensemble model EfficientNetB7-Vision transformer', \n fontsize=16)","metadata":{"execution":{"iopub.execute_input":"2023-02-09T12:16:38.017367Z","iopub.status.busy":"2023-02-09T12:16:38.016756Z","iopub.status.idle":"2023-02-09T12:16:43.016971Z","shell.execute_reply":"2023-02-09T12:16:43.016057Z","shell.execute_reply.started":"2023-02-09T12:16:38.017306Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on calcule les métriques F1-score, precision et recall\n# et on les sauvegarde\nEfficientNetB7_ViT_f1_score = f1_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nEfficientNetB7_ViT_precision = precision_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\nEfficientNetB7_ViT_recall = recall_score(\n cm_correct_labels,\n cm_predictions,\n labels=labels,\n average='macro',\n)\n\nF1_score = list(score_df['F1_score'])\nprecision = list(score_df['precision'])\nrecall = list(score_df['recall'])\n\nF1_score.append(EfficientNetB7_ViT_f1_score)\nF1_score = [round(num,3) for num in F1_score]\nprecision.append(EfficientNetB7_ViT_precision)\nprecision = [round(num,3) for num in precision]\nrecall.append(EfficientNetB7_ViT_recall)\nrecall = [round(num,3) for num in recall]\nscore_EfficientNetB7_ViT = { 'F1_score': F1_score, \n 'precision': precision, \n 'recall': recall}\nscore_EfficientNetB7_ViT_df = pd.DataFrame(score_EfficientNetB7_ViT,\n index=['ResNet50', 'InceptionV3', 'Xception', 'ViT', \n 'EfficientNetB7', 'Ensemble Xception_ViT', \n 'Ensemble EfficientNetB7_ViT'])\nscore_EfficientNetB7_ViT_df.to_csv('score_EfficientNetB7_ViT_df.csv',index=False), \nscore_EfficientNetB7_ViT_df\n\n# on visualise les performances des modèles\nfontsize1 = 18\nfontsize2 = 16\nfontsize3 = 14\nfontsize4 = 12\n\nwidth = 0.25\n\nfig, axes = plt.subplots(nrows=1, ncols=1, figsize=(20,7))\n\nbarchart = score_EfficientNetB7_ViT_df.plot.bar(ax=axes, y=['precision', 'recall', 'F1_score'], \n width=width*3, \n rot=0, \n fontsize=fontsize2)\nbarchart.bar_label(axes.containers[0], \n label_type='edge', \n padding=15, \n fontsize = fontsize4)\nbarchart.bar_label(axes.containers[1], \n label_type='edge', \n padding=15, \n fontsize = fontsize4)\nbarchart.bar_label(axes.containers[2], \n label_type='edge', \n padding=15, \n fontsize=fontsize4)\naxes.set_ylim(bottom=0.5, \n top=1.1)\naxes.set_title('Precision, Recall, F1-Score', fontsize=fontsize1)\naxes.set_xticklabels(['ResNet50','InceptionV3','Xception','ViT', 'EfficientNetB7', \n 'Ensemble Xception_ViT','Ensemble EfficientNetB7_ViT'], \n fontsize=fontsize2,\n rotation=45)\naxes.legend(fontsize=fontsize4)","metadata":{"execution":{"iopub.execute_input":"2023-02-09T12:17:57.206449Z","iopub.status.busy":"2023-02-09T12:17:57.205788Z","iopub.status.idle":"2023-02-09T12:17:57.784335Z","shell.execute_reply":"2023-02-09T12:17:57.783231Z","shell.execute_reply.started":"2023-02-09T12:17:57.206391Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Le modèle plus performent est donc l’Ensemble Xception-Vision Transformer qui est donc sélectionné comme modèle final.","metadata":{}},{"cell_type":"code","source":"score_EfficientNetB7_ViT_df = pd.read_csv(\n '/kaggle/input/ensemble-efficientnetb7-vit/score_EfficientNetB7 _ViT_df.csv')\nscore_EfficientNetB7_ViT_df.rename(index={0:'ResNet50',1:'InceptionV3',2:'Xception',3:'ViT',\n 4:'EfficientNetB7',5:'Xception_ViT', 6:'EfficientNetB7_ViT'})","metadata":{"execution":{"iopub.execute_input":"2023-02-09T12:23:38.320426Z","iopub.status.busy":"2023-02-09T12:23:38.318921Z","iopub.status.idle":"2023-02-09T12:23:38.351028Z","shell.execute_reply":"2023-02-09T12:23:38.350090Z","shell.execute_reply.started":"2023-02-09T12:23:38.320375Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Performance modèle final sur le jeu de validation**","metadata":{}},{"cell_type":"code","source":"Xception_model = define_model(tf.keras.applications.Xception)\nXception_model.load_weights('/kaggle/input/resnet50-inception-xception/Xception_model_best.h5')\n\nIMAGE_SIZE = 512\nwith strategy.scope():\n pretrained_model = vit.vit_b16(\n image_size=IMAGE_SIZE,\n activation='softmax',\n pretrained=True,\n include_top=False,\n pretrained_top=False,\n classes=104)\n\n ViT_model = tf.keras.Sequential([\n pretrained_model,\n tf.keras.layers.Flatten(),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(11, activation=tfa.activations.gelu),\n tf.keras.layers.BatchNormalization(),\n tf.keras.layers.Dense(104, 'softmax')], name='vision_transformer')\n \nViT_model.summary()\n\nViT_model.load_weights('/kaggle/input/vision-transformer/ViT_model_best.h5')","metadata":{"execution":{"iopub.execute_input":"2023-02-07T19:15:03.876545Z","iopub.status.busy":"2023-02-07T19:15:03.875692Z","iopub.status.idle":"2023-02-07T19:15:54.224613Z","shell.execute_reply":"2023-02-07T19:15:54.222888Z","shell.execute_reply.started":"2023-02-07T19:15:03.876481Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IMAGE_SIZE = [512,512]\ncmdataset = get_validation_dataset(ordered=True)\nimages_ds = cmdataset.map(lambda image, label: image)\nlabels_ds = cmdataset.map(lambda image, label: label).unbatch()\ncm_correct_labels = next(iter(labels_ds.batch(NUM_VALIDATION_IMAGES))).numpy()\nprint('Predicting with ViT_best_model')\nm1 = ViT_model.predict(images_ds)\nprint('Predicting with Xception_best_model')\nm2 = Xception_model.predict(images_ds)\n\nscores = []\nfor alpha in np.linspace(0,1,100):\n cm_probabilities = alpha*m1 + (1-alpha)*m2\n cm_predictions = np.argmax(cm_probabilities, axis=-1)\n scores.append(f1_score(cm_correct_labels, cm_predictions, \n labels=range(len(CLASSES)), \n average='macro'))\n\nbest_alpha = np.argmax(scores) / 100\ncm_probabilities = best_alpha*m1 + (1-best_alpha)*m2\ncm_predictions = np.argmax(cm_probabilities, axis=-1)","metadata":{"execution":{"iopub.execute_input":"2023-02-07T19:15:58.336013Z","iopub.status.busy":"2023-02-07T19:15:58.335711Z","iopub.status.idle":"2023-02-07T19:16:25.441486Z","shell.execute_reply":"2023-02-07T19:16:25.440668Z","shell.execute_reply.started":"2023-02-07T19:15:58.335984Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# validation visuelle pour un batch de 20 images\ndataset = get_validation_dataset()\ndataset = dataset.unbatch().batch(20)\nbatch = iter(dataset)\nimages, labels = next(batch)\nprobabilities = alpha*ViT_model.predict(images) + (1-alpha)*Xception_model.predict(images)\npredictions = np.argmax(probabilities, axis=-1)\ndisplay_batch_of_images((images, labels), predictions)","metadata":{"execution":{"iopub.execute_input":"2023-02-07T19:16:29.387011Z","iopub.status.busy":"2023-02-07T19:16:29.386172Z","iopub.status.idle":"2023-02-07T19:16:47.074396Z","shell.execute_reply":"2023-02-07T19:16:47.073416Z","shell.execute_reply.started":"2023-02-07T19:16:29.386970Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# on calcule le nombre d'erreurs pour les prédictions sur le jeu de validation\nmismatches = sum(cm_predictions!=cm_correct_labels)\nprint('Number of mismatches on validation data: {} out of {} or ({:.2%})'\n .format(mismatches, NUM_VALIDATION_IMAGES, mismatches/NUM_VALIDATION_IMAGES))","metadata":{"execution":{"iopub.execute_input":"2023-02-07T19:17:31.891529Z","iopub.status.busy":"2023-02-07T19:17:31.891056Z","iopub.status.idle":"2023-02-07T19:17:31.907679Z","shell.execute_reply":"2023-02-07T19:17:31.906696Z","shell.execute_reply.started":"2023-02-07T19:17:31.891488Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Evaluations prédictions sur le jeu de test**","metadata":{}},{"cell_type":"code","source":"# prédictions des classes du jeu de test avec le meilleur modèle Xception-ViT\ntest_ds = get_test_dataset(TEST_FILENAMES, ordered=True)\n\nprint('Computing predictions...')\ntest_images_ds = test_ds.map(lambda image, idnum: image)\nprint('Predicting with ViT_best_model')\nm1 = ViT_model.predict(test_images_ds)\nprint('Predicting with EfficientNetB7_best_model')\nm2 = Xception_model.predict(test_images_ds)\nprobabilities = best_alpha*m1 + (1-best_alpha)*m2\npredictions = np.argmax(probabilities, axis=-1)\n\nprint(predictions)","metadata":{"execution":{"iopub.execute_input":"2023-02-07T19:18:25.520981Z","iopub.status.busy":"2023-02-07T19:18:25.520603Z","iopub.status.idle":"2023-02-07T19:18:55.370694Z","shell.execute_reply":"2023-02-07T19:18:55.369807Z","shell.execute_reply.started":"2023-02-07T19:18:25.520942Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Generating submission.csv file...')\n\n# on obtient les image ids du jeu de test\ntest_ids_ds = test_ds.map(lambda image, idnum: idnum).unbatch()\ntest_ids = next(iter(test_ids_ds.batch(NUM_TEST_IMAGES))).numpy().astype('U')\n\n# on sauvegarde le fichier pour la participation à la competition\nnp.savetxt(\n 'submission.csv',\n np.rec.fromarrays([test_ids, predictions]),\n fmt=['%s', '%d'],\n delimiter=',',\n header='id,label',\n comments='',\n)\n\n# on visualise les premières prédictions\n!head submission.csv","metadata":{"execution":{"iopub.execute_input":"2023-02-07T19:19:04.716142Z","iopub.status.busy":"2023-02-07T19:19:04.715810Z","iopub.status.idle":"2023-02-07T19:19:08.745856Z","shell.execute_reply":"2023-02-07T19:19:08.744790Z","shell.execute_reply.started":"2023-02-07T19:19:04.716107Z"}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" ","metadata":{}}]}