{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 1. Introduction + Set-up\n\nMachine learning has a phenomenal range of applications, including in health and diagnostics. This tutorial will explain the complete pipeline from loading data to predicting results, and it will explain how to build an Modulation image classification model from scratch","metadata":{}},{"cell_type":"code","source":"import re\nimport os\nimport numpy as np\nimport pandas as pd\nimport tensorflow as tf\nfrom kaggle_datasets import KaggleDatasets\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split\n# import efficientnet.tfkeras as efn\n# from tensorflow.keras import backend as K\n# import tensorflow_addons as tfa\n\ntry:\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()#tpu='4'\n    print('Device:', tpu.master())\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    strategy = tf.distribute.experimental.TPUStrategy(tpu)#tpu\nexcept:\n    strategy = tf.distribute.get_strategy()\nprint('Number of replicas:', strategy.num_replicas_in_sync)\n    \nprint(tf.__version__)\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-07-14T11:51:35.478110Z","iopub.execute_input":"2021-07-14T11:51:35.478552Z","iopub.status.idle":"2021-07-14T11:51:43.240579Z","shell.execute_reply.started":"2021-07-14T11:51:35.478490Z","shell.execute_reply":"2021-07-14T11:51:43.239606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We need a Google Cloud link to our data to load the data using a TPU. While we're at it, we instantiate constant variables. It is generally better practice to use constant variables instead of hard-coding numbers.","metadata":{}},{"cell_type":"code","source":"##make sure your dataset is public\nAUTOTUNE = tf.data.experimental.AUTOTUNE\nGCS_PATH = KaggleDatasets().get_gcs_path('iot-malware')#'dataset-cnn-train-ht-new'\nBATCH_SIZE = 16 * strategy.num_replicas_in_sync#16\nIMAGE_SIZE = [256, 256]#\nEPOCHS = 50","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:43.245513Z","iopub.execute_input":"2021-07-14T11:51:43.245849Z","iopub.status.idle":"2021-07-14T11:51:43.649195Z","shell.execute_reply.started":"2021-07-14T11:51:43.245807Z","shell.execute_reply":"2021-07-14T11:51:43.648220Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"GCS_PATH","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:43.651746Z","iopub.execute_input":"2021-07-14T11:51:43.652158Z","iopub.status.idle":"2021-07-14T11:51:43.661020Z","shell.execute_reply.started":"2021-07-14T11:51:43.652115Z","shell.execute_reply":"2021-07-14T11:51:43.659914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!gsutil ls $GCS_PATH","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:43.662607Z","iopub.execute_input":"2021-07-14T11:51:43.663030Z","iopub.status.idle":"2021-07-14T11:51:46.566201Z","shell.execute_reply.started":"2021-07-14T11:51:43.662987Z","shell.execute_reply":"2021-07-14T11:51:46.565330Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Load the data\n\nThe Chest X-ray data we are using from [*Cell*](https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia) divides the data into train, val, and test files. There are only 16 files in the validation folder, and we would prefer to have a less extreme division between the training and the validation set. We will append the validation files and create a new split that resembes the standard 80:20 division instead.","metadata":{}},{"cell_type":"code","source":"# link = '/dataset_cnn_train_HT_new'\n# link = '/dataset_cnn_modulation/dataset_cnn_train_HT_new'\nlink = '/IOT_Malware_dataset'","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:46.567690Z","iopub.execute_input":"2021-07-14T11:51:46.568095Z","iopub.status.idle":"2021-07-14T11:51:46.573453Z","shell.execute_reply.started":"2021-07-14T11:51:46.568049Z","shell.execute_reply":"2021-07-14T11:51:46.572408Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"filenames = tf.io.gfile.glob(str(GCS_PATH +link+ '/Benign/*'))#/chest_xray/train/*/*\nfilenames.extend(tf.io.gfile.glob(str(GCS_PATH +link+ '/Malware/*')))\n# filenames.extend(tf.io.gfile.glob(str(GCS_PATH + link+'/2PSK/*')))\n# filenames.extend(tf.io.gfile.glob(str(GCS_PATH + link+'/32QAM/*')))\n# filenames.extend(tf.io.gfile.glob(str(GCS_PATH + link+'/64QAM/*')))\n# filenames.extend(tf.io.gfile.glob(str(GCS_PATH + link+'/8PSK/*')))\n# filenames.extend(tf.io.gfile.glob(str(GCS_PATH + link+'/8QAM/*')))\n# filenames.extend(tf.io.gfile.glob(str(GCS_PATH + link+'/QPSK/*')))\n\n\ntrain_filenames, val_filenames = train_test_split(filenames, test_size=0.2)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:46.575072Z","iopub.execute_input":"2021-07-14T11:51:46.575515Z","iopub.status.idle":"2021-07-14T11:51:48.277577Z","shell.execute_reply.started":"2021-07-14T11:51:46.575472Z","shell.execute_reply":"2021-07-14T11:51:48.276435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Run the following cell to see how many healthy/normal chest X-rays we have and how many pneumonia chest X-rays we have.","metadata":{}},{"cell_type":"code","source":"COUNT_16PSK = len([filename for filename in train_filenames if \"Benign\" in filename])\nprint(\"Benign images count in training set: \" + str(COUNT_16PSK))\n\nCOUNT_16QAM = len([filename for filename in train_filenames if \"Malware\" in filename])\nprint(\"Malware images count in training set: \" + str(COUNT_16QAM))\n\n# COUNT_2PSK = len([filename for filename in train_filenames if \"2PSK\" in filename])\n# print(\"2PSK images count in training set: \" + str(COUNT_2PSK))\n\n# COUNT_32QAM = len([filename for filename in train_filenames if \"32QAM\" in filename])\n# print(\"32QAM images count in training set: \" + str(COUNT_32QAM))\n\n# COUNT_64QAM = len([filename for filename in train_filenames if \"64QAM\" in filename])\n# print(\"64QAM images count in training set: \" + str(COUNT_64QAM))\n\n# COUNT_8PSK = len([filename for filename in train_filenames if \"8PSK\" in filename])\n# print(\"8PSK images count in training set: \" + str(COUNT_8PSK))\n\n# COUNT_8QAM = len([filename for filename in train_filenames if \"8QAM\" in filename])\n# print(\"8QAM images count in training set: \" + str(COUNT_8QAM))\n\n# COUNT_QPSK = len([filename for filename in train_filenames if \"QPSK\" in filename])\n# print(\"QPSK images count in training set: \" + str(COUNT_QPSK))","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:48.279116Z","iopub.execute_input":"2021-07-14T11:51:48.279587Z","iopub.status.idle":"2021-07-14T11:51:48.293962Z","shell.execute_reply.started":"2021-07-14T11:51:48.279540Z","shell.execute_reply":"2021-07-14T11:51:48.292835Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Notice that the there are way more images that are classified as pneumonia than normal. This shows that we have a imbalance in our data. We will correct for this imbalance later on in our notebook.","metadata":{}},{"cell_type":"code","source":"train_list_ds = tf.data.Dataset.from_tensor_slices(train_filenames)\nval_list_ds = tf.data.Dataset.from_tensor_slices(val_filenames)\n\nfor f in train_list_ds.take(10):\n    print(f.numpy())","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:48.297216Z","iopub.execute_input":"2021-07-14T11:51:48.297526Z","iopub.status.idle":"2021-07-14T11:51:48.416336Z","shell.execute_reply.started":"2021-07-14T11:51:48.297498Z","shell.execute_reply":"2021-07-14T11:51:48.415356Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Run the following cell to see how many images we have in our training dataset and how many images we have in our validation set. Verify that the ratio of images is 80:20.","metadata":{}},{"cell_type":"code","source":"TRAIN_IMG_COUNT = tf.data.experimental.cardinality(train_list_ds).numpy()\nprint(\"Training images count: \" + str(TRAIN_IMG_COUNT))\n\nVAL_IMG_COUNT = tf.data.experimental.cardinality(val_list_ds).numpy()\nprint(\"Validating images count: \" + str(VAL_IMG_COUNT))","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:48.418190Z","iopub.execute_input":"2021-07-14T11:51:48.418687Z","iopub.status.idle":"2021-07-14T11:51:48.427758Z","shell.execute_reply.started":"2021-07-14T11:51:48.418624Z","shell.execute_reply":"2021-07-14T11:51:48.426720Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As expected, we have two labels for our images.","metadata":{}},{"cell_type":"code","source":"CLASS_NAMES = np.array([str(tf.strings.split(item, os.path.sep)[-1].numpy())[2:-1]\n                        for item in tf.io.gfile.glob(str(GCS_PATH + \"/IOT_Malware_dataset/*\"))])\nCLASS_NAMES","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:48.429035Z","iopub.execute_input":"2021-07-14T11:51:48.429342Z","iopub.status.idle":"2021-07-14T11:51:50.223644Z","shell.execute_reply.started":"2021-07-14T11:51:48.429293Z","shell.execute_reply":"2021-07-14T11:51:50.221383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Currently our dataset is just a list of filenames. We want to map each filename to the corresponding (image, label) pair. The following methods will help us do that.\n\nAs we only have two labels, we will rewrite the label so that `1` or `True` indicates pneumonia and `0` or `False` indicates normal.","metadata":{}},{"cell_type":"code","source":"file_path =GCS_PATH + '/IOT_Malware_dataset/Benign'","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.225157Z","iopub.execute_input":"2021-07-14T11:51:50.225608Z","iopub.status.idle":"2021-07-14T11:51:50.231681Z","shell.execute_reply.started":"2021-07-14T11:51:50.225561Z","shell.execute_reply":"2021-07-14T11:51:50.230535Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"file_path","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.233303Z","iopub.execute_input":"2021-07-14T11:51:50.234058Z","iopub.status.idle":"2021-07-14T11:51:50.246853Z","shell.execute_reply.started":"2021-07-14T11:51:50.233919Z","shell.execute_reply":"2021-07-14T11:51:50.245620Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.path.sep","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.248414Z","iopub.execute_input":"2021-07-14T11:51:50.248729Z","iopub.status.idle":"2021-07-14T11:51:50.260845Z","shell.execute_reply.started":"2021-07-14T11:51:50.248698Z","shell.execute_reply":"2021-07-14T11:51:50.259744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# parts = tf.strings.split('hello world/anas',os.path.sep)#.numpy()\n# parts\n\nparts = tf.strings.split(file_path, os.path.sep)\nparts","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.262281Z","iopub.execute_input":"2021-07-14T11:51:50.262689Z","iopub.status.idle":"2021-07-14T11:51:50.294221Z","shell.execute_reply.started":"2021-07-14T11:51:50.262658Z","shell.execute_reply":"2021-07-14T11:51:50.293503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"parts[-1]","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.295437Z","iopub.execute_input":"2021-07-14T11:51:50.295910Z","iopub.status.idle":"2021-07-14T11:51:50.305628Z","shell.execute_reply.started":"2021-07-14T11:51:50.295867Z","shell.execute_reply":"2021-07-14T11:51:50.304619Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"a = parts[-1] == \"Benign\"\na","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.307054Z","iopub.execute_input":"2021-07-14T11:51:50.307519Z","iopub.status.idle":"2021-07-14T11:51:50.332807Z","shell.execute_reply.started":"2021-07-14T11:51:50.307462Z","shell.execute_reply":"2021-07-14T11:51:50.331233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"parts = tf.strings.split(file_path, os.path.sep)\nparts","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.334459Z","iopub.execute_input":"2021-07-14T11:51:50.335055Z","iopub.status.idle":"2021-07-14T11:51:50.366875Z","shell.execute_reply.started":"2021-07-14T11:51:50.335003Z","shell.execute_reply":"2021-07-14T11:51:50.365602Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"parts[2] == \"Malware\"","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.368257Z","iopub.execute_input":"2021-07-14T11:51:50.368620Z","iopub.status.idle":"2021-07-14T11:51:50.377154Z","shell.execute_reply.started":"2021-07-14T11:51:50.368587Z","shell.execute_reply":"2021-07-14T11:51:50.376214Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"place = 4\ndef get_label(file_path):\n    # convert the path to a list of path components\n    parts = tf.strings.split(file_path, os.path.sep)\n    # The second to last is the class-directory\n    if parts[place] == \"Benign\":\n        return [1.,0.]\n#     elif parts[place] == \"16QAM\":\n#         return [0.,1.,0.,0.,0.,0.,0.,0.]\n#     elif parts[place] == \"2PSK\":\n#         return [0.,0.,1.,0.,0.,0.,0.,0.]\n#     elif parts[place] == \"32QAM\":\n#         return [0.,0.,0.,1.,0.,0.,0.,0.]\n#     elif parts[place] == \"64QAM\":\n#         return [0.,0.,0.,0.,1.,0.,0.,0.]\n#     elif parts[place] == \"8PSK\":\n#         return [0.,0.,0.,0.,0.,1.,0.,0.]\n#     elif parts[place] == \"8QAM\":\n#         return [0.,0.,0.,0.,0.,0.,1.,0.]\n    else:\n        parts[place] == \"Malware\"\n        return [0.,1.]","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.378482Z","iopub.execute_input":"2021-07-14T11:51:50.378786Z","iopub.status.idle":"2021-07-14T11:51:50.391331Z","shell.execute_reply.started":"2021-07-14T11:51:50.378757Z","shell.execute_reply":"2021-07-14T11:51:50.390086Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"file_path =GCS_PATH + '/IOT_Malware_dataset/Benign'\nget_label(file_path)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.392734Z","iopub.execute_input":"2021-07-14T11:51:50.393089Z","iopub.status.idle":"2021-07-14T11:51:50.429143Z","shell.execute_reply.started":"2021-07-14T11:51:50.393042Z","shell.execute_reply":"2021-07-14T11:51:50.428084Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"file_path =GCS_PATH + '/IOT_Malware_dataset/Malware'\nget_label(file_path)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.430593Z","iopub.execute_input":"2021-07-14T11:51:50.430881Z","iopub.status.idle":"2021-07-14T11:51:50.455952Z","shell.execute_reply.started":"2021-07-14T11:51:50.430852Z","shell.execute_reply":"2021-07-14T11:51:50.455007Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# image = tf.image.resize(image, IMAGE_SIZE,BILINEAR)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.457197Z","iopub.execute_input":"2021-07-14T11:51:50.457507Z","iopub.status.idle":"2021-07-14T11:51:50.461124Z","shell.execute_reply.started":"2021-07-14T11:51:50.457477Z","shell.execute_reply":"2021-07-14T11:51:50.459986Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Function to decode our images (normalize and reshape)\ndef decode_img(image_data):\n    image = tf.image.decode_jpeg(image_data, channels = 3)\n    # Resize image to be aligned with the inference phase\n    image = tf.image.resize(image, IMAGE_SIZE)#BILINEAR\n    # convert image to floats in [0, 1] range\n#     image = tf.cast(image, tf.float32)# / 255.0\n    # explicit size needed for TPU\n    image = tf.reshape(image, [*IMAGE_SIZE, 3])\n    return image","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.462394Z","iopub.execute_input":"2021-07-14T11:51:50.462865Z","iopub.status.idle":"2021-07-14T11:51:50.476142Z","shell.execute_reply.started":"2021-07-14T11:51:50.462833Z","shell.execute_reply":"2021-07-14T11:51:50.475046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The images originally have values that range from [0, 255]. CNNs work better with smaller numbers so we will scale this down.","metadata":{}},{"cell_type":"code","source":"# def decode_img(img):\n#   # convert the compressed string to a 3D uint8 tensor\n#   img = tf.image.decode_jpeg(img, channels=3)\n#   # Use `convert_image_dtype` to convert to floats in the [0,1] range.\n#   img = tf.image.convert_image_dtype(img, tf.float32)\n#   # resize the image to the desired size.\n#   return tf.image.resize(img, IMAGE_SIZE)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.478063Z","iopub.execute_input":"2021-07-14T11:51:50.478718Z","iopub.status.idle":"2021-07-14T11:51:50.488945Z","shell.execute_reply.started":"2021-07-14T11:51:50.478672Z","shell.execute_reply":"2021-07-14T11:51:50.487819Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def process_path(file_path):\n    label = get_label(file_path)\n    # load the raw data from the file as a string\n    img = tf.io.read_file(file_path)\n    img = decode_img(img)\n    return img, label","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.490724Z","iopub.execute_input":"2021-07-14T11:51:50.491492Z","iopub.status.idle":"2021-07-14T11:51:50.505640Z","shell.execute_reply.started":"2021-07-14T11:51:50.491444Z","shell.execute_reply":"2021-07-14T11:51:50.504841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_ds = train_list_ds.map(process_path, num_parallel_calls=AUTOTUNE)\n\nval_ds = val_list_ds.map(process_path, num_parallel_calls=AUTOTUNE)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.511818Z","iopub.execute_input":"2021-07-14T11:51:50.512118Z","iopub.status.idle":"2021-07-14T11:51:50.840167Z","shell.execute_reply.started":"2021-07-14T11:51:50.512091Z","shell.execute_reply":"2021-07-14T11:51:50.839145Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_ds","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.844112Z","iopub.execute_input":"2021-07-14T11:51:50.844511Z","iopub.status.idle":"2021-07-14T11:51:50.851903Z","shell.execute_reply.started":"2021-07-14T11:51:50.844478Z","shell.execute_reply":"2021-07-14T11:51:50.850507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's visualize the shape of an (image, label) pair.","metadata":{}},{"cell_type":"code","source":"train_ds.take(1)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.853531Z","iopub.execute_input":"2021-07-14T11:51:50.853876Z","iopub.status.idle":"2021-07-14T11:51:50.868349Z","shell.execute_reply.started":"2021-07-14T11:51:50.853844Z","shell.execute_reply":"2021-07-14T11:51:50.867470Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for image, label in train_ds.take(5):\n    print(\"Image shape: \", image.numpy().shape)\n    print(\"Label: \", label.numpy())","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:50.869605Z","iopub.execute_input":"2021-07-14T11:51:50.870010Z","iopub.status.idle":"2021-07-14T11:51:52.390531Z","shell.execute_reply.started":"2021-07-14T11:51:50.869980Z","shell.execute_reply":"2021-07-14T11:51:52.389554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Load and format the test data as well.","metadata":{}},{"cell_type":"code","source":"test_list_ds = tf.data.Dataset.list_files(str(GCS_PATH + '/*'))\nTEST_IMAGE_COUNT = tf.data.experimental.cardinality(test_list_ds).numpy()\ntest_ds = test_list_ds.map(process_path, num_parallel_calls=AUTOTUNE)\ntest_ds = test_ds.batch(BATCH_SIZE)\n\nTEST_IMAGE_COUNT","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:52.391810Z","iopub.execute_input":"2021-07-14T11:51:52.392104Z","iopub.status.idle":"2021-07-14T11:51:54.031564Z","shell.execute_reply.started":"2021-07-14T11:51:52.392076Z","shell.execute_reply":"2021-07-14T11:51:54.030593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Visualize the dataset\n\nFirst, let's use buffered prefetching so we can yield data from disk without having I/O become blocking.","metadata":{}},{"cell_type":"code","source":"def prepare_for_training(ds, cache=True, shuffle_buffer_size=1000):\n    # This is a small dataset, only load it once, and keep it in memory.\n    # use `.cache(filename)` to cache preprocessing work for datasets that don't\n    # fit in memory.\n    if cache:\n        if isinstance(cache, str):\n            ds = ds.cache(cache)\n        else:\n            ds = ds.cache()\n\n    ds = ds.shuffle(buffer_size=shuffle_buffer_size)\n\n    # Repeat forever\n    ds = ds.repeat()\n\n    ds = ds.batch(BATCH_SIZE)\n\n    # `prefetch` lets the dataset fetch batches in the background while the model\n    # is training.\n    ds = ds.prefetch(buffer_size=AUTOTUNE)\n\n    return ds","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:54.032766Z","iopub.execute_input":"2021-07-14T11:51:54.033068Z","iopub.status.idle":"2021-07-14T11:51:54.042012Z","shell.execute_reply.started":"2021-07-14T11:51:54.033039Z","shell.execute_reply":"2021-07-14T11:51:54.040947Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Call the next batch iteration of the training data.","metadata":{}},{"cell_type":"code","source":"train_ds = prepare_for_training(train_ds)\nval_ds = prepare_for_training(val_ds)\n\nimage_batch, label_batch = next(iter(train_ds))","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:51:54.043361Z","iopub.execute_input":"2021-07-14T11:51:54.043663Z","iopub.status.idle":"2021-07-14T11:52:22.124979Z","shell.execute_reply.started":"2021-07-14T11:51:54.043634Z","shell.execute_reply":"2021-07-14T11:52:22.123887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"type(val_ds)","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:22.126369Z","iopub.execute_input":"2021-07-14T11:52:22.126689Z","iopub.status.idle":"2021-07-14T11:52:22.132544Z","shell.execute_reply.started":"2021-07-14T11:52:22.126657Z","shell.execute_reply":"2021-07-14T11:52:22.131337Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Define the method to show the images in the batch.","metadata":{}},{"cell_type":"code","source":"label_batch","metadata":{"scrolled":true,"execution":{"iopub.status.busy":"2021-07-14T11:52:22.133751Z","iopub.execute_input":"2021-07-14T11:52:22.134377Z","iopub.status.idle":"2021-07-14T11:52:23.359635Z","shell.execute_reply.started":"2021-07-14T11:52:22.134330Z","shell.execute_reply":"2021-07-14T11:52:23.358657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"label_batch[3] == [1,0]","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:23.361088Z","iopub.execute_input":"2021-07-14T11:52:23.361407Z","iopub.status.idle":"2021-07-14T11:52:23.370153Z","shell.execute_reply.started":"2021-07-14T11:52:23.361378Z","shell.execute_reply":"2021-07-14T11:52:23.369210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def show_batch(image_batch, label_batch):\n    plt.figure(figsize=(10,10))\n    for n in range(8):\n        ax = plt.subplot(2,4,n+1)\n        plt.imshow(image_batch[n])\n        if np.all(label_batch[n]== [1,0]):\n            plt.title(\"Benign\")\n#         elif np.all(label_batch[n]== [0,1,0,0,0,0,0,0]):\n#             plt.title(\"16QAM\")\n#         elif np.all(label_batch[n]== [0,0,1,0,0,0,0,0]):\n#             plt.title(\"2PSK\")\n#         elif np.all(label_batch[n]== [0,0,0,1,0,0,0,0]):\n#             plt.title(\"32QAM\") \n#         elif np.all(label_batch[n]== [0,0,0,0,1,0,0,0]):\n#             plt.title(\"64QAM\")\n#         elif np.all(label_batch[n]== [0,0,0,0,0,1,0,0]):\n#             plt.title(\"8PSK\")\n#         elif np.all(label_batch[n]== [0,0,0,0,0,0,1,0]):\n#             plt.title(\"8QAM\")\n        else:\n            plt.title(\"Malware\")\n        plt.axis(\"off\")","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:23.371298Z","iopub.execute_input":"2021-07-14T11:52:23.371593Z","iopub.status.idle":"2021-07-14T11:52:23.383060Z","shell.execute_reply.started":"2021-07-14T11:52:23.371566Z","shell.execute_reply":"2021-07-14T11:52:23.382177Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As the method takes in numpy arrays as its parameters, call the numpy function on the batches to return the tensor in numpy array form.","metadata":{}},{"cell_type":"code","source":"show_batch(image_batch.numpy(), label_batch.numpy())","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:23.384264Z","iopub.execute_input":"2021-07-14T11:52:23.384696Z","iopub.status.idle":"2021-07-14T11:52:24.411880Z","shell.execute_reply.started":"2021-07-14T11:52:23.384667Z","shell.execute_reply":"2021-07-14T11:52:24.410819Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4. Build the CNN\n\nTo make our model more modular and easier to understand, let's define some blocks. As we're building a convolution neural network, we'll create a convolution block and a dense layer block.\n\nThe architecture for this CNN has been inspired by this [article](https://towardsdatascience.com/deep-learning-for-detecting-pneumonia-from-x-ray-images-fc9a3d9fdba8).","metadata":{}},{"cell_type":"code","source":"# def conv_block(filters):\n#     block = tf.keras.Sequential([\n#         tf.keras.layers.SeparableConv2D(filters, 3, activation='relu', padding='same'),\n#         tf.keras.layers.SeparableConv2D(filters, 3, activation='relu', padding='same'),\n#         tf.keras.layers.BatchNormalization(),\n#         tf.keras.layers.MaxPool2D()\n#     ]\n#     )\n    \n#     return block\n\n# def dense_block(units, dropout_rate):\n#     block = tf.keras.Sequential([\n#         tf.keras.layers.Dense(units, activation='relu'),\n#         tf.keras.layers.BatchNormalization(),\n#         tf.keras.layers.Dropout(dropout_rate)\n#     ])\n    \n#     return block\n\n# def build_model():\n#     model = tf.keras.Sequential([\n#         tf.keras.Input(shape=(*IMAGE_SIZE, 3)),\n        \n#         tf.keras.layers.Conv2D(16, 3, activation='relu', padding='same'),\n#         tf.keras.layers.Conv2D(16, 3, activation='relu', padding='same'),\n#         tf.keras.layers.MaxPool2D(),\n        \n#         conv_block(32),\n#         conv_block(64),\n        \n#         conv_block(128),\n#         tf.keras.layers.Dropout(0.2),\n        \n#         conv_block(256),\n#         tf.keras.layers.Dropout(0.2),\n        \n#         tf.keras.layers.Flatten(),\n#         dense_block(512, 0.7),\n#         dense_block(128, 0.5),\n#         dense_block(64, 0.3),\n        \n#         tf.keras.layers.Dense(8, activation='softmax')#sigmoid\n#     ])\n    \n#     return model","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:24.413231Z","iopub.execute_input":"2021-07-14T11:52:24.413568Z","iopub.status.idle":"2021-07-14T11:52:24.418458Z","shell.execute_reply.started":"2021-07-14T11:52:24.413536Z","shell.execute_reply":"2021-07-14T11:52:24.417266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The following method will define the function to build our model for us. The Dropout layers are important as they \"drop out,\" hence the name, certain nodes to reduce the likelikhood of the model overfitting. We want to end the model with a Dense layer of one node, as this will be the output that determines if an X-ray shows an image of pneumonia.","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras import models, layers\n# from tensorflow.keras.applications import ResNet50, DenseNet121,InceptionResNetV2","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:24.419780Z","iopub.execute_input":"2021-07-14T11:52:24.420065Z","iopub.status.idle":"2021-07-14T11:52:24.440570Z","shell.execute_reply.started":"2021-07-14T11:52:24.420037Z","shell.execute_reply":"2021-07-14T11:52:24.439354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def modelEfficientNetB3():\n    \n#     model = models.Sequential()\n#     model.add(ResNet50(include_top = False, weights = \"imagenet\",\n#                             input_shape=(*IMAGE_SIZE, 3)))\n#     model.add(layers.GlobalAveragePooling2D())\n#     #model.add(layers.Flatten())\n#     model.add(layers.Dense(1024, activation = 'relu'))#512\n#     model.add(layers.Dropout(0.5))\n# #     model.add(layers.Dense(1024, activation = 'relu'))\n# #     model.add(layers.Dropout(0.4))\n# #     model.add(layers.Dense(1024, activation = 'relu'))\n# #     model.add(layers.Dropout(0.23))\n# #     model.add(layers.Dense(1024, activation = 'relu'))\n# #     model.add(layers.Dropout(0.2))\n#     model.add(layers.Dense(8, activation = \"softmax\"))\n    \n#     return model ","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:24.441970Z","iopub.execute_input":"2021-07-14T11:52:24.442381Z","iopub.status.idle":"2021-07-14T11:52:24.454603Z","shell.execute_reply.started":"2021-07-14T11:52:24.442347Z","shell.execute_reply":"2021-07-14T11:52:24.453525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 5. Correct for data imbalance\n\nWe saw earlier in this notebook that the data was imbalanced, with more images classified as pneumonia than normal. We will correct for that in this following section.","metadata":{}},{"cell_type":"code","source":"# initial_bias = np.log([COUNT_Healthy/COUNT_CMD_1])\n# initial_bias","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:24.455887Z","iopub.execute_input":"2021-07-14T11:52:24.456394Z","iopub.status.idle":"2021-07-14T11:52:24.473895Z","shell.execute_reply.started":"2021-07-14T11:52:24.456344Z","shell.execute_reply":"2021-07-14T11:52:24.473128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# weight_for_0 = ((1 / COUNT_16PSK)*(TRAIN_IMG_COUNT)/9.0,0,0,0,0,0,0,0)\n# weight_for_1 = (0, (1 / COUNT_16QAM)*(TRAIN_IMG_COUNT)/9.0,0,0,0,0,0,0)\n# weight_for_2 = (0,0,(1 / COUNT_2PSK)*(TRAIN_IMG_COUNT)/9.0,0,0,0,0,0)\n# weight_for_3 = (0,0,0,(1 / COUNT_32QAM)*(TRAIN_IMG_COUNT)/9.0,0,0,0,0)\n# weight_for_4 = (0,0,0,0,(1 / COUNT_64QAM)*(TRAIN_IMG_COUNT)/9.0,0,0,0)\n# weight_for_5 = (0,0,0,0,0,(1 / COUNT_8PSK)*(TRAIN_IMG_COUNT)/9.0,0,0)\n# weight_for_6 = (0,0,0,0,0,0,(1 / COUNT_8QAM)*(TRAIN_IMG_COUNT)/9.0,0)\n# weight_for_7 = (0,0,0,0,0,0,0,(1 / COUNT_QPSK)*(TRAIN_IMG_COUNT)/9.0)\n\n# class_weight = {(1,0,0,0,0): weight_for_0, (0,1,0,0,0): weight_for_1,(0,0,1,0,0): weight_for_2, (0,0,0,1,0): weight_for_3, (0,0,0,0,1): weight_for_4}\n\n# print('Weight for class 0: ',weight_for_0)\n# print('Weight for class 1: ',weight_for_1)\n# print('Weight for class 0: ',weight_for_2)\n# print('Weight for class 1: ',weight_for_3)\n# print('Weight for class 0: ',weight_for_4)\n","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:24.475022Z","iopub.execute_input":"2021-07-14T11:52:24.475484Z","iopub.status.idle":"2021-07-14T11:52:24.488624Z","shell.execute_reply.started":"2021-07-14T11:52:24.475439Z","shell.execute_reply":"2021-07-14T11:52:24.487791Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"LR = 0.0001\nfrom tensorflow.keras.applications import ResNet50,EfficientNetB7,MobileNetV2\ndef get_model():    \n    inp = tf.keras.layers.Input(shape = (*IMAGE_SIZE, 3))\n\n    x = MobileNetV2(weights = 'imagenet', include_top = False)(inp)\n    x = tf.keras.layers.GlobalAveragePooling2D()(x)\n#     x.layers.trainable = False\n#     x = tf.keras.layers.Flatten()(x)\n#     x = tf.keras.layers.Dense(1024, activation='relu')(x)\n#     x = tf.keras.layers.Dropout(0.4)(x)\n#     x = tf.keras.layers.Dense(1024, activation='relu')(x)\n#     #Add dropout to avoid Overfit\n#     x = tf.keras.layers.Dropout(0.45)(x)\n#     x = tf.keras.layers.Dense(512, activation='relu')(x)\n#     #Add dropout to avoid Overfit\n#     x = tf.keras.layers.Dropout(0.45)(x)\n    x = tf.keras.layers.Dense(256, activation='relu',,kernel_regularizer=tf.keras.regularizers.l2(0.0001))(x)\n    x = tf.keras.layers.Dropout(0.2)(x)\n    output = tf.keras.layers.Dense(2, activation = 'softmax')(x)\n        \n    model = tf.keras.models.Model(inputs = [inp], outputs = [output])\n\n    opt = tf.keras.optimizers.Adam(learning_rate = LR)\n\n    model.compile(\n        optimizer = opt,\n        loss = [tf.keras.losses.CategoricalCrossentropy(label_smoothing = 0.4)],\n        metrics = [tf.keras.metrics.CategoricalAccuracy()])\n\n    return model","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:24.489764Z","iopub.execute_input":"2021-07-14T11:52:24.490226Z","iopub.status.idle":"2021-07-14T11:52:24.501827Z","shell.execute_reply.started":"2021-07-14T11:52:24.490179Z","shell.execute_reply":"2021-07-14T11:52:24.501106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 6. Train the model\n\nSince there are only two possible labels for the image, we will be using the `Categorical_crossentropy` loss. When we fit the model, identify the class weights. Because we are using a TPU, training will be relatively quick.\n\nFor our metrics, we want to include precision and recall as they will provide use with a more informed picture of how good our model is. Accuracy tells us what fractions are the labels are correct. Since our data is not balanced, accuracy might give a skewed sense of a good model\n\nPrecision is the number of true positives (TP) over the sum of TP and false positives (FP). It shows what fraction of labeled positives are actually correct.\n\nRecall is the number of TP over the sum of TP and false negatves (FN). It shows what fraction of actual positives are correct.","metadata":{}},{"cell_type":"code","source":"LR = 0.0001\nwith strategy.scope():\n    \n    model = get_model()#build_model()\n    opt = tf.keras.optimizers.Adam(learning_rate = 0.0001)#learning_rate = LR\n    METRICS = ['categorical_accuracy', tf.keras.metrics.Precision(name='categorical_precision'), tf.keras.metrics.Recall(name='categorical_recall')]\n    \n    model.compile(optimizer=opt,loss=[tf.keras.losses.CategoricalCrossentropy()],metrics=METRICS)#'categorical_crossentropy' label_smoothing = 0.2","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:24.502901Z","iopub.execute_input":"2021-07-14T11:52:24.503361Z","iopub.status.idle":"2021-07-14T11:52:34.156691Z","shell.execute_reply.started":"2021-07-14T11:52:24.503305Z","shell.execute_reply":"2021-07-14T11:52:34.155720Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"From exploring the data and the model, I noticed that the training for the model has a slow start. However, after 25 epochs, the model slowly starts to converge.","metadata":{}},{"cell_type":"code","source":"checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\"xray_model.h5\",\n                                                    save_best_only=True, monitor='categorical_accuracy')\n\nearly_stopping_cb = tf.keras.callbacks.EarlyStopping(monitor='val_categorical_accuracy', restore_best_weights=False, patience=16)\nlr_scheduler = tf.keras.callbacks.ReduceLROnPlateau(monitor='loss', patience=2, verbose=1, factor=0.5,mode=\"min\", min_lr=0.0001)\nlr_scheduler_1 = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', patience=2, verbose=1, factor=0.5,mode=\"min\", min_lr=0.0001)\n","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:34.157984Z","iopub.execute_input":"2021-07-14T11:52:34.158263Z","iopub.status.idle":"2021-07-14T11:52:34.164459Z","shell.execute_reply.started":"2021-07-14T11:52:34.158236Z","shell.execute_reply":"2021-07-14T11:52:34.163591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"EPOCHS =128 #64#64\nhistory = model.fit(\n    train_ds,\n    steps_per_epoch=TRAIN_IMG_COUNT // BATCH_SIZE,\n    epochs=EPOCHS,\n    validation_data=val_ds,#val_ds\n    validation_steps=TRAIN_IMG_COUNT // BATCH_SIZE,#VAL_IMG_COUNT // BATCH_SIZE,\n    #class_weight=class_weight,\n    shuffle=True,\n#     callbacks=[checkpoint_cb, early_stopping_cb, lr_scheduler, lr_scheduler_1]\n)\n\n","metadata":{"execution":{"iopub.status.busy":"2021-07-14T11:52:34.165877Z","iopub.execute_input":"2021-07-14T11:52:34.166472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots(4, 1, figsize=(30, 30))#1, 4\nax = ax.ravel()\n\nfor i, met in enumerate(['categorical_precision', 'categorical_recall', 'categorical_accuracy', 'loss']):\n    ax[i].plot(history.history[met])\n    ax[i].plot(history.history['val_' + met])\n    ax[i].set_title('Model {}'.format(met))\n    ax[i].set_xlabel('epochs')\n    ax[i].set_ylabel(met)\n    ax[i].legend(['train', 'val'])\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss, acc, prec, rec = model.evaluate(val_ds,steps=VAL_IMG_COUNT // BATCH_SIZE)#val_ds  test_ds\n#model.evaluate(steps=)\nloss, acc, prec, rec","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# xxcx","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"val_ds_1 = val_list_ds.map(process_path, num_parallel_calls=AUTOTUNE)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"type(val_ds_1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(val_ds_1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ds = val_ds_1.prefetch(buffer_size=AUTOTUNE)\ntype(ds)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(ds.take(10))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from tqdm import tqdm\ny_pred = []  # store predicted labels\ny_true = []  # store true labels\n\n# iterate over the dataset\nfor image_batch, label_batch in tqdm(val_ds_1):   # use dataset.unbatch() with repeat\n   # append true labels\n#    label_batch_1 = np.array(label_batch)\n    \n   y_true.append(label_batch)\n   # compute predictions\n#    image_batch = np.array(image_batch)\n#    image_batch = image_batch.reshape(1,256,256,3) \n#    preds = model.predict(image_batch)\n   # append predicted labels\n   y_pred.append(image_batch)#np.argmax(preds, axis = - 1)\n\n# convert the true and predicted labels into tensors\ncorrect_labels = tf.concat([item for item in y_true], axis = 0)\npredicted_labels = tf.concat([item for item in y_pred], axis = 0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_batch = np.array(y_pred)\nimage_batch = image_batch.reshape(-1,256,256,3) \npreds = model.predict(image_batch)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# preds","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# y_true","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_true_1 = np.array(y_true)\n# y_true_1","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import sklearn.metrics as metrics\nfrom sklearn.model_selection import cross_val_score\nimport seaborn as sns\nsns.set(style=\"whitegrid\")\nplt.figure()\nax = plt.subplot()\nax.set_title('Confusion Matrix')\n# #pred = model.predict_classes(X_test) #we can't using this function because not founding in Function Model\n# pred = model.predict(X_test)\npred = np.argmax(preds, axis=1)\nY_TEST = np.argmax(y_true_1, axis =1)\nclasses=['16PSK', '16QAM','2PSK','32QAM', '64QAM','8PSK','8QAM','QPSK']\n# Y_TEST = np.argmax(correct_labels, axis =-1)\ncm = metrics.confusion_matrix(Y_TEST,pred)\n# classes=['Benign', 'Malware']\nsns.heatmap(cm, annot=True,fmt='d',xticklabels=classes, yticklabels=classes,cmap='Blues')#YlGnBu_r or Blues or twilight_shifted_r\nprint('Number of images:',Y_TEST.shape)\nprint('Actual image ',Y_TEST[0:21])\nprint('Predic image ',pred[0:21])\nplt.xlabel('Predicted')\nplt.ylabel('Actual')\nplt.show","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ccccccccccccccccc","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nimg = []\nlab = []\nfor image, label in tqdm(val_ds_1.take(10)):\n    img.append(image)\n    lab.append(label)\n#     print(\"Image shape: \", image.numpy().shape)\n#     print(\"Label: \", label.numpy())\n","metadata":{"scrolled":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_1 = np.array(img)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_1 = img_1.reshape(-1,256,256,3)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preds = model.predict(img_1)","metadata":{"scrolled":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import sklearn.metrics as metrics\nfrom sklearn.model_selection import cross_val_score\nimport seaborn as sns\nsns.set(style=\"whitegrid\")\nplt.figure()\nax = plt.subplot()\nax.set_title('Confusion Matrix')\n# #pred = model.predict_classes(X_test) #we can't using this function because not founding in Function Model\n# pred = model.predict(X_test)\n# pred = np.argmax(pred, axis=1)\n# Y_TEST = np.argmax(y_test, axis =1)\nclasses=['16PSK', '16QAM','2PSK','32QAM', '64QAM','8PSK','8QAM','QPSK']\n#Y_TEST = np.argmax(correct_labels, axis =1)\ncm = metrics.confusion_matrix(lab,preds)\n# classes=['Benign', 'Malware']\nsns.heatmap(cm, annot=True,fmt='d',xticklabels=classes, yticklabels=classes,cmap='Blues')#YlGnBu_r or Blues or twilight_shifted_r\nprint('Number of images:',Y_TEST.shape)\nprint('Actual image ',Y_TEST[0:21])\n# print('Predic image ',correct_labels[0:21])\nplt.xlabel('Predicted')\nplt.ylabel('Actual')\nplt.show","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lab_1 = np.array(lab)","metadata":{"scrolled":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"lab_1","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"label = lab_1[0:10]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(img)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img_1= np.array(img[0:10])\nimg_1.shape","metadata":{"scrolled":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":" preds = model.predict(img_1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preds","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import sklearn.metrics as metrics\nfrom sklearn.model_selection import cross_val_score\nimport seaborn as sns\nsns.set(style=\"whitegrid\")\nplt.figure()\nax = plt.subplot()\npre = np.argmax(preds, axis =1)\nY_TEST = np.argmax(label, axis =1)\ncm = metrics.confusion_matrix(Y_TEST,pre)\n# classes=['Benign', 'Malware']\nsns.heatmap(cm, annot=True,fmt='d',cmap='Blues')#YlGnBu_r or Blues or twilight_shifted_r\nprint('Number of images:',Y_TEST.shape)\nprint('Actual image ',Y_TEST[0:21])\n# print('Predic image ',correct_labels[0:21])\nplt.xlabel('Predicted')\nplt.ylabel('Actual')\nplt.show","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# img_1= np.array(img)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# img_1[0].shape","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# preds = model.predict(img_1)","metadata":{"scrolled":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"val=val_ds_1.take(100)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(val)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"val","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\ny_pred = []  # store predicted labels\ny_true = []  # store true labels\n\n# iterate over the dataset\nfor image_batch, label_batch in tqdm(val):   # use dataset.unbatch() with repeat\n   # append true labels\n   y_true.append(np.argmax(label_batch, axis = - 1))\n   # compute predictions\n   image_batch_1 = np.array(image_batch)\n   image_batch_1 = image_batch_1.reshape(1,256,256,3)\n   preds = model.predict(image_batch_1)\n   # append predicted labels\n   y_pred.append(np.argmax(preds, axis = - 1))\n\n# convert the true and predicted labels into tensors\ncorrect_labels = tf.concat([item for item in y_true], axis = 0)\npredicted_labels = tf.concat([item for item in y_pred], axis = 0)","metadata":{"_kg_hide-output":true,"scrolled":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predicted_labels","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import sklearn.metrics as metrics\nfrom sklearn.model_selection import cross_val_score\nimport seaborn as sns\nsns.set(style=\"whitegrid\")\nplt.figure()\nax = plt.subplot()\nax.set_title('Confusion Matrix')\n# #pred = model.predict_classes(X_test) #we can't using this function because not founding in Function Model\n# pred = model.predict(X_test)\n# pred = np.argmax(pred, axis=1)\n# Y_TEST = np.argmax(y_test, axis =1)\nclasses=['16PSK', '16QAM','2PSK','32QAM', '64QAM','8PSK','8QAM','QPSK']\n#Y_TEST = np.argmax(correct_labels, axis =1)\ncm = metrics.confusion_matrix(correct_labels,predicted_labels)\n# classes=['Benign', 'Malware']\nsns.heatmap(cm, annot=True,fmt='d',xticklabels=classes, yticklabels=classes,cmap='Blues')#YlGnBu_r or Blues or twilight_shifted_r\nprint('Number of images:',Y_TEST.shape)\nprint('Actual image ',Y_TEST[0:21])\n# print('Predic image ',correct_labels[0:21])\nplt.xlabel('Predicted')\nplt.ylabel('Actual')\nplt.show","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for image, label in val_ds.take(1):\n    print(\"Image shape: \", image.numpy().shape)\n    print(\"Label: \", label.numpy())","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(1):\n    image_1 = image[i].numpy()\n    #print(label[i])\n    val_label = label.numpy()\n    y_label = np.argmax(val_label[i], axis=0)\n    print(y_label)\n    image_1 = image_1.reshape(1,256,256,3)\n    pred = model.predict(image_1)\n    print(pred)\n    pred_1 = np.argmax(pred, axis=1)\n    print(pred_1)\n    print('===========================================================')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import sklearn.metrics as metrics\nfrom sklearn.model_selection import cross_val_score\nimport seaborn as sns\nsns.set(style=\"whitegrid\")\nplt.figure()\nax = plt.subplot()\nax.set_title('Confusion Matrix')\n#pred = model.predict_classes(X_test) #we can't using this function because not founding in Function Model\nimage_1 = image.numpy()\nimage_1 = image_1.reshape(-1,256,256,3)\npred = model.predict(image_1)\npred = np.argmax(pred, axis=1)\n\nval_label = label.numpy()\ny_label = np.argmax(val_label, axis=1)\nY_TEST = y_label\ncm = metrics.confusion_matrix(Y_TEST,pred)\nclasses=['16PSK', '16QAM','2PSK','32QAM', '64QAM','8PSK','8QAM','QPSK']\nsns.heatmap(cm, annot=True,xticklabels=classes, yticklabels=classes,cmap='Blues')#YlGnBu_r or Blues or twilight_shifted_r\nprint('Number of images:',Y_TEST.shape)\nprint('Actual image ',Y_TEST[0:21])\nprint('Predic image ',pred[0:21])\nplt.xlabel('Predicted')\nplt.ylabel('Actual')\nplt.show","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import classification_report\nprint(classification_report(Y_TEST, pred))\nprint('16PSK', '16QAM','2PSK','32QAM', '64QAM','8PSK','8QAM','QPSK')\n#support : means number each classifier\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import label_binarize\nfrom sklearn.metrics import precision_recall_curve\nfrom tensorflow.keras.utils import to_categorical\n# precision recall curve\nprecision = dict()\nrecall = dict()\nPRED = to_categorical(pred)\ny = Y_TEST\n#Df['label'].values\n# Binarize the output\ny = label_binarize(y, classes=[0,1,2,3,4,5,6,7])\nn_classes = y.shape[1]\n\nfor i in range(n_classes):\n    precision[i], recall[i], _ = precision_recall_curve(y [:, i],\n                                                        PRED[:, i])\n    plt.plot(recall[i], precision[i], lw=2, label='class {}'.format(i))\n\nplt.xlabel(\"recall\")\nplt.ylabel(\"precision\")\nplt.legend(loc=\"best\")\nplt.title(\"precision vs. recall curve\")\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.preprocessing import label_binarize\nfrom sklearn.metrics import roc_curve, auc\n\nPRED = to_categorical(pred)\ny = Y_TEST\n#Df['label'].values\n# Binarize the output\ny = label_binarize(y, classes=[0,1,2,3,4,5,6,7])\nn_classes = y.shape[1]\n\nfpr = dict()\ntpr = dict()\nroc_auc = dict()\nfor i in range(n_classes):\n       fpr[i], tpr[i], _ = roc_curve(y[:,i], PRED[:,i])\n       roc_auc[i] = auc(fpr[i], tpr[i])\ncolors = ['blue', 'red', 'green']\n#cls = {1:'normal', 2:'other pneumonia', 0:'covid'}\ncls = {0:'covid', 1:'normal', 2:'other pneumonia'}\nfor i, color ,c in zip(range(n_classes), colors, cls.values()):\n    plt.plot(fpr[i], tpr[i], color=color, lw=1.5,\n             label='ROC curve of '+c+ '(AUC = {1:0.2f})'\n             ''.format(i, roc_auc[i]))\n    \nplt.plot([0, 1], [0, 1], 'k--',linestyle='--')\nplt.xlim([-0.05, 1.0])\nplt.ylim([0.0, 1.05])\nplt.xlabel('False Positive Rate')\nplt.ylabel('True Positive Rate')\nplt.title('ROC for multi-class data')\nplt.legend(loc=\"lower right\")\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# history = model.fit(train_ds, \n#           validation_data=val_ds,\n                    \n#           epochs=50, batch_size=64, shuffle=True, \n#           callbacks=[checkpoint_cb, early_stopping_cb, lr_scheduler, lr_scheduler_1]\n#          )","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 7. Finetune the model\n\nFinetuning is an art when it comes to Machine Learning, and there are many ways to adjust the model in efforts to improve it. Finetuning is beyond the scope of this notebook, but check out this [article](https://www.pyimagesearch.com/2019/06/03/fine-tuning-with-keras-and-deep-learning/) for more information.\n\nFor our purposes, we'll use Keras callbacks to further finetune our model. The checkpoint callback saves the best weights of the model, so next time we want to use the model, we do not have to spend time training it. The early stopping callback stops the training process when the model starts becoming stagnant, or even worse, when the model starts overfitting. Since we set `restore_best_weights` to `True`, the returned model at the end of the training process will be the model with the best weights (i.e. low loss and high accuracy).","metadata":{}},{"cell_type":"code","source":"checkpoint_cb = tf.keras.callbacks.ModelCheckpoint(\"xray_model.h5\",\n                                                    save_best_only=True)\n\nearly_stopping_cb = tf.keras.callbacks.EarlyStopping(patience=16,\n                                                     restore_best_weights=True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We also want to finetune our learning rate. Too high of a learning rate will cause the model to diverge. Too small of a learning rate will cause the model to be too slow. We implement the exponential learning rate scheduling method below.","metadata":{}},{"cell_type":"code","source":"def exponential_decay(lr0, s):\n    def exponential_decay_fn(epoch):\n        return lr0 * 0.1 **(epoch / s)\n    return exponential_decay_fn\n\nexponential_decay_fn = exponential_decay(0.0001, 20)\n\nlr_scheduler_1 = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', patience=2, verbose=1, factor=0.5,mode=\"min\", min_lr=0.00001)\nlr_scheduler = tf.keras.callbacks.LearningRateScheduler(exponential_decay_fn)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history = model.fit(\n    train_ds,\n    steps_per_epoch=TRAIN_IMG_COUNT // BATCH_SIZE,\n    epochs=50,\n    validation_data=val_ds,\n    validation_steps=VAL_IMG_COUNT // BATCH_SIZE,\n    #class_weight=class_weight,\n    callbacks=[checkpoint_cb, early_stopping_cb, lr_scheduler,lr_scheduler_1]\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 8. Visualizing model performance\n\nLet's plot the model accuracy and loss for the training and the validating set. These plots show the accuracy and loss values for the second round of training. Since we initially trained the model with 30 epochs, these would be epochs 31-45. Note that no random seed is specified for this notebook. For your notebook, there might be slight variance.","metadata":{}},{"cell_type":"code","source":"fig, ax = plt.subplots(2, 2, figsize=(20, 20))\nax = ax.ravel()\nfor i, met in enumerate(['categorical_precision', 'categorical_recall', 'categorical_accuracy', 'loss']):\n    ax[i].plot(history.history[met])\n    ax[i].plot(history.history['val_' + met])\n    ax[i].set_title('Model {}'.format(met))\n    ax[i].set_xlabel('epochs')\n    ax[i].set_ylabel(met)\n    ax[i].legend(['train', 'val'])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We see that the accuracy for our model is around 98%. Finetune the model further to see if we can achieve a higher accuracy.","metadata":{}},{"cell_type":"markdown","source":"# 9. Predict and evaluate results\n\nLet's evaluate the model on our test data!","metadata":{}},{"cell_type":"code","source":"loss, acc, prec, rec = model.evaluate(val_ds,steps=VAL_IMG_COUNT // BATCH_SIZE)#val_ds  test_ds\n#model.evaluate(steps=)\nloss, acc, prec, rec\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"loss, acc, prec, rec","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}