{"cells":[{"metadata":{"_uuid":"29cef868b4828df297d5d4bfeef48a8ef8d81559"},"cell_type":"markdown","source":"This kernel is my first try at making a NN using keras to apply it to the cancer cell competion. Any comments are more than welcome on any topic, as I am a very early beginner in data science :-)"},{"metadata":{"_uuid":"124276c1d04cc102370f3c303e7f325d9a4b835f"},"cell_type":"markdown","source":"**Things that I learned**\nThis is my first deep learning code, so obviously, it can only be a learning experience. Since I've read a few Kernels already, I figured if any other beginner like me stumbles upon my Kernel, maybe this might be helpful. If not, well at least I get to write feedback for myself ;-)\n\n- I started by trying to make a [57k, 96, 96, 3] np.ndarray containing all the arrays of all the images we need to classify. While that did seem to work with smaller set (I tried with 25k, and it worked), at 57k, the Kernel just crashes. After some investigation (*puts Sherlock's hat on*) the issue seems to be memory overload. I mean I'm just turning 57,000+ images into 96x96x3 arrays, what could go wrong? Next step is to try inserting the prediction inside the for loop. Here's the idea: I'm still training my model (Well, not really mine, rather the one used in the week 2 of Course 4 of Andrew Ng's Coursera Deep Learning course) with a small amount of data, just to see if it's working. I'm taking baby steps, I'll gradually add more data as things work (eventually, I hope). I saw a tweet from Andrej Karapathy a fw weeks ago saying that you should try making a small model, with little data, until it overfits, and then move on. This allows you to check that the model is working, and helps find potential (as a beginner, I'd change the word 'potential' by 'numerous', but maybe it's just me) sources of error. Once I have my first model, then the prediciton is done image by image: I take an image, convert it to an array, and then predict it. Repeat ~57,000 times. This means no storing of the arrays, and (hopefully) no memory crash."},{"metadata":{"_uuid":"261a9e07c86f65cb749bd65eca0d5e9ae0dfd05b"},"cell_type":"markdown","source":"**TO DO**\nProblem at the moment: memory overload.\n- Train model with about 2000 train ex, 500 test ex, and then do the prediction inside the loop, so no array storing of the image on submission set. In submission loop: img_to_array, then predict, then put prediction into sample_submission['label']. I think this is the file that has to be submitted again, have to check that."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"scrolled":true},"cell_type":"code","source":"import numpy as np \nimport math\nimport pandas as pd \nimport cv2\nimport matplotlib.pyplot as plt\nfrom PIL import Image\n\nfrom keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img\nfrom keras import layers\nfrom keras.layers import Input, Add, Dense, Activation, ZeroPadding2D, BatchNormalization, Flatten, Conv2D\nfrom keras.layers import AveragePooling2D, MaxPooling2D, Dropout, GlobalMaxPooling2D, GlobalAveragePooling2D\nfrom keras.models import Model, Sequential\nfrom keras.optimizers import Adam\nfrom keras.preprocessing import image\nfrom keras.utils import layer_utils\nfrom keras.utils.data_utils import get_file\nfrom keras.applications import ResNet50\nfrom keras.applications.imagenet_utils import preprocess_input\nimport pydot\nfrom IPython.display import SVG\nfrom keras.utils.vis_utils import model_to_dot\nfrom keras.utils import plot_model\n\nimport keras.backend as K\nK.set_image_data_format('channels_last')\nimport matplotlib.pyplot as plt\nfrom matplotlib.pyplot import imshow\n\n%matplotlib inline\n\nimport time\n\nimport os\nprint(os.listdir(\"../input\"))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"train_labels = pd.read_csv('../input/train_labels.csv', dtype=str)\ntest_labels = pd.read_csv('../input/sample_submission.csv', dtype=str)\n\n#print('train : ','\\n', train_sample.head(5))\n#print('test : ','\\n', test_labels.head(5))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"87213a83d7f9f262cbae6dd88f15bd62ec45edf2"},"cell_type":"markdown","source":"Very early remarks:\n\n1. We have 220,025 images in train_labels, of which 89,117 are labelled as having a cancer pixel in the 32x32 center zone.\n2. There are 57,5k images in sample_sumbissions\n3. Images are of size 96x96x3 (Meaning RBG, and of total size 27,648)\n4. The only information we have is, if there is a cancer cell of not (in the 32x32 center zone, according to the data description). We do not know what it looks like, which pixel is identified as the one being the cancer, and what makes or doesn't make a cancer cell. This will make EDA relatively fast in my opinion, because there isn't much information we are going to be abel to look at; apart from looking at 1 labelled pictures, and visually trying to find what looks like the patterns compared to 0 labelled data.\n"},{"metadata":{"trusted":true,"_uuid":"b60f8b6a66b399818f814587de9fe39222e6f972"},"cell_type":"code","source":"#This image is labelled as having a cancer cell.\nimage = plt.imread('../input/train/c18f2d887b7ae4f6742ee445113fa1aef383ed77.tif')\nplt.imshow(image)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1d49c6561c9904d0c404ae955f89995defac413b"},"cell_type":"code","source":"num_classes = 2\nmy_model = Sequential()\nmy_model.add(ResNet50(include_top=False, weights='imagenet'))\nmy_model.add(Dense(num_classes, activation = 'softmax'))\nmy_model.layers[0].trainable = False","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a145bad784beb3c40496f69a4582e94da504e450"},"cell_type":"code","source":"my_model.compile(optimizer = Adam(lr=0.0001), loss = 'binary_crossentropy', metrics = ['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f961c36a706eea5d3bdd6b5ed86dbda42b7f80ac"},"cell_type":"code","source":"#Ratio of images in train compared to test\nsample_size = 2000\nratio = 0.9 \nsample_train = train_labels[:sample_size]\nsample_test = test_labels[:sample_size]\n\nsize_train = math.ceil(ratio*sample_train.shape[0])\ntrain_df=sample_train[:size_train]\ntest_df=sample_test[size_train+1:]\n\nprint('sample size : ', sample_size, '\\n',\n      'ratio : ', ratio,'\\n',\n      'train size : ', train_df.shape[0],'\\n',\n      'test size : ', test_df.shape[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"374e861cacd8ea448185c0c82332d7776c8a1780"},"cell_type":"code","source":"# used in the reference url: https://medium.com/@vijayabhaskar96/tutorial-on-keras-flow-from-dataframe-1fd4493d237c\ndef append_ext(fn):\n    return fn + '.tif'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"62da69dd9a31eebc55857a466b533bb352cd11f3"},"cell_type":"code","source":"train_df['id']=train_df['id'].apply(append_ext)\ntest_df['id']=test_df['id'].apply(append_ext)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0e8f345eadb5b64900cfd8c018a5a043d4fc5a3e"},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"31f4bc7362c4e96b4b1cd791f4ea03df5c1c15b4"},"cell_type":"code","source":"train_batch_size = 10\nval_batch_size = 10\nvalid_ratio = 0.25\n\ntrain_steps = np.ceil(train_df.shape[0] / train_batch_size)\nval_steps = np.ceil((train_df.shape[0]*valid_ratio) / val_batch_size)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"d9ccc5a777bd56f59a9f2e050ca6acc6ab7bbb52"},"cell_type":"code","source":"data_generator = ImageDataGenerator(rescale = 1./255., validation_split=valid_ratio)\n\ntrain_generator = data_generator.flow_from_dataframe(dataframe = train_df, \n                                                directory = '../input/train/',\n                                               x_col = 'id',\n                                               y_col = 'label',\n                                               subset = 'training',\n                                               batch_size = train_batch_size,\n                                               shuffle = True,\n                                               class_mode = 'categorical',\n                                               target_size = (96, 96))\nvalidation_generator = data_generator.flow_from_dataframe(dataframe = train_df,\n                                                         directory = '../input/train/',\n                                                        x_col = 'id',\n                                                        y_col = 'label',\n                                                        subset = 'validation',\n                                                        batch_size=val_batch_size,\n                                                        shuffle = True,\n                                                        class_mode = 'categorical',\n                                                        target_size = (96, 96))\n\ntest_datagen = ImageDataGenerator(rescale = 1./255.)\n\ntest_generator = test_datagen.flow_from_dataframe(dataframe = test_df,\n                                                directory = '../input/test/',\n                                               x_col = 'id',\n                                               y_col = 'label',\n                                               class_mode = None,\n                                               shuffle = False,\n                                               target_size = (96, 96))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"02ab0857b14ee0456143d4678174b1b43b8ea4e3"},"cell_type":"code","source":"STEP_SIZE_TRAIN=train_generator.n//train_generator.batch_size\nSTEP_SIZE_VALID=validation_generator.n//validation_generator.batch_size\nSTEP_SIZE_TEST=test_generator.n//test_generator.batch_size\n\nprint('step size for : ', '\\n', 'train : ', STEP_SIZE_TRAIN,\n     '\\n', 'valid : ', STEP_SIZE_VALID,\n     '\\n', 'test : ', STEP_SIZE_TEST)\n\nmy_model.fit_generator(generator = train_generator,\n                       steps_per_epoch=STEP_SIZE_TRAIN,\n                       validation_data = validation_generator,\n                       validation_steps=STEP_SIZE_VALID,\n                       epochs = 3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a29a06c87cbbfbb0e6a190e4ec86335e667e8ab8"},"cell_type":"code","source":"evaluation = my_model.evaluate(x= val_img_array, y=val_img_label)\nprint()\nprint (\"Loss = \" + str(evaluation[0]))\nprint (\"Test Accuracy = \" + str(evaluation[1]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"675073fba19507d889965c9e717cecf2b4c8273d"},"cell_type":"code","source":"print('number of images labelled with cancer : ',test_sample[test_sample['label']==1].shape[0],\n      ' out of ', test_sample.shape[0], ' examples')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"df690179fc56b9f02978ad811d094e8ef28a0744"},"cell_type":"code","source":"test_sample.to_csv('test_predictions.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}