{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introduction\n\nYou've seen how to build a model from scratch to identify handwritten digits.  You'll now build a model to identify different types of clothing.  To make models that train quickly, we'll work with very small (low-resolution) images. \n\nAs an example, your model will take an images like this and identify it as a shoe:\n\n![Imgur](https://i.imgur.com/GyXOnSB.png)","metadata":{}},{"cell_type":"markdown","source":"# Data Preparation\nThis code is supplied, and you don't need to change it. Just run the cell below.","metadata":{}},{"cell_type":"code","source":"import numpy as np\nfrom sklearn.model_selection import train_test_split\nfrom tensorflow import keras\n\nimg_rows, img_cols = 28, 28\nnum_classes = 10\n\ndef prep_data(raw):\n    y = raw[:, 0]\n    out_y = keras.utils.to_categorical(y, num_classes)\n    \n    x = raw[:,1:]\n    num_images = raw.shape[0]\n    out_x = x.reshape(num_images, img_rows, img_cols, 1)\n    out_x = out_x / 255\n    return out_x, out_y\n\nfashion_file = \"../input/fashionmnist/fashion-mnist_train.csv\"\nfashion_data = np.loadtxt(fashion_file, skiprows=1, delimiter=',')\nx, y = prep_data(fashion_data)\n\n# Set up code checking\nfrom learntools.core import binder\nbinder.bind(globals())\nfrom learntools.deep_learning.exercise_7 import *\nprint(\"Setup Complete\")","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:02:52.203342Z","iopub.execute_input":"2022-07-24T04:02:52.203577Z","iopub.status.idle":"2022-07-24T04:03:28.614331Z","shell.execute_reply.started":"2022-07-24T04:02:52.203528Z","shell.execute_reply":"2022-07-24T04:03:28.612929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 1) Start the model\nCreate a `Sequential` model called `fashion_model`. Don't add layers yet.","metadata":{}},{"cell_type":"code","source":"from tensorflow import keras\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Flatten, Conv2D\n\n# Your Code Here\nfashion_model = Sequential()\n\nq_1.check()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:03:28.616753Z","iopub.execute_input":"2022-07-24T04:03:28.617304Z","iopub.status.idle":"2022-07-24T04:03:28.650455Z","shell.execute_reply.started":"2022-07-24T04:03:28.617245Z","shell.execute_reply":"2022-07-24T04:03:28.649913Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"q_1.solution()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:03:28.651706Z","iopub.execute_input":"2022-07-24T04:03:28.651962Z","iopub.status.idle":"2022-07-24T04:03:28.661192Z","shell.execute_reply.started":"2022-07-24T04:03:28.651917Z","shell.execute_reply":"2022-07-24T04:03:28.660379Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2) Add the first layer\n\nAdd the first `Conv2D` layer to `fashion_model`. It should have 12 filters, a kernel_size of 3 and the `relu` activation function. The first layer always requires that you specify the `input_shape`.  We have saved the number of rows and columns to the variables `img_rows` and `img_cols` respectively, so the input shape in this case is `(img_rows, img_cols, 1)`.","metadata":{}},{"cell_type":"markdown","source":"https://qiita.com/kenichiro-yamato/items/60affeb7ca9f67c87a17\n\nkeras.layers.**Conv2D**(\n\n  **filters,**\n  **kernel_size,**\n  strides=(1, 1),\n  padding='valid',\n  data_format=None,\n  dilation_rate=(1, 1),\n  **activation=None,**\n  use_bias=True,\n  kernel_initializer='glorot_uniform',\n  bias_initializer='zeros',\n  kernel_regularizer=None,\n  bias_regularizer=None,\n  activity_regularizer=None,\n  kernel_constraint=None,\n  bias_constraint=None\n  \n)\n\n**filters :**\n**整数で、出力空間の次元（つまり畳み込みにおける出力フィルタの数）。**\n「カーネル」のことを「フィルタ」と呼ぶ場合もある\nカーネルは「フィルタ」であり「特徴検出器」である。すべて同じ意味である。\n\n**Conv2D(16, (3, 3)の解説：**\n**「3×3」の大きさのフィルタを16枚使うという意味**です（16種類の「3×3」のフィルタ）。\n「5×5」「7×7」などと、中心を決められる奇数が使いやすいようです。\nフィルタ数は、「16・32・64・128・256・512枚」などが使われる傾向にあるようですが、\n複雑そうな問題ならフィルタ数を多めに、簡単そうな問題ならフィルタ数を少なめで試してみるようです。\n\n**activation=relu の解説**\n：活性化関数「ReLU（Rectified Linear Unit）- ランプ関数」。\nフィルタ後の画像に実施。入力が0以下の時は出力0。入力が0より大きい場合はそのまま出力する。\nactivation: 使用する活性化関数の名前（activationsを参照）\n何も指定しなければ，活性化は一切適用されません\nactivation=\"relu\"は「活性化関数としてReLUを使いなさい」という命令になる。\n**「活性化関数を指定すればモデルの表現力が増す（賢いAIが作れる）から活性化関数を指定しましょう」そして「標準的に使われるのはReLUですよね」といった感じ。**\n\n**input_shape=(28, 28, 1)の解説：**\n**縦28・横28ピクセルのグレースケール（白黒画像）を入力しています。**\n白黒画像なら1,RGBなら3\n","metadata":{}},{"cell_type":"code","source":"# Your code here\nfashion_model.add(Conv2D(12,\n                         activation='relu',\n                         kernel_size=3,\n                         input_shape = (img_rows, img_cols, 1)))\n\n\nfashion_model.summary()\nq_2.check()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:03:28.662766Z","iopub.execute_input":"2022-07-24T04:03:28.663308Z","iopub.status.idle":"2022-07-24T04:03:28.708522Z","shell.execute_reply.started":"2022-07-24T04:03:28.663257Z","shell.execute_reply":"2022-07-24T04:03:28.707726Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"q_2.hint()\nq_2.solution()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:03:28.709691Z","iopub.execute_input":"2022-07-24T04:03:28.710137Z","iopub.status.idle":"2022-07-24T04:03:28.719842Z","shell.execute_reply.started":"2022-07-24T04:03:28.710090Z","shell.execute_reply":"2022-07-24T04:03:28.719025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3) Add the remaining layers\n\n1. Add 2 more convolutional (`Conv2D layers`) with 20 filters each, 'relu' activation, and a kernel size of 3. Follow that with a `Flatten` layer, and then a `Dense` layer with 100 neurons. \n2. Add your prediction layer to `fashion_model`.  This is a `Dense` layer.  We alrady have a variable called `num_classes`.  Use this variable when specifying the number of nodes in this layer. The activation should be `softmax` (or you will have problems later).","metadata":{}},{"cell_type":"markdown","source":"http://marupeke296.com/IKDADV_DL_No15_cnn_experiment.html　\n\n2次元以上の入力データを1次元のデータに変換するのが**Flatten層**です：\n\nFlatten(\n    data_format     = None,\n    **kwargs\n):\n\n引数のdata_formatはConv2Dのとまったく一緒です。これにより画像データはピクセル数次元なデータとなり次の層のニューロンへ送られます。\n\nhttps://keras.io/ja/layers/core/\n\nkeras.layers.**Dense**\n(units, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None\n)\n\nhttps://child-programmer.com/ai/keras/dense/\n\nmodel.add(**Dense**(128, activation=’relu’))\n#コード解説\n：全結合層。出力128。\n活性化関数「ReLU（Rectified Linear Unit）- ランプ関数」。入力が0以下の時は出力0。入力が0より大きい場合はそのまま出力する。","metadata":{}},{"cell_type":"code","source":"# Your code here\nfashion_model.add(Conv2D(20, activation='relu', kernel_size=3))\nfashion_model.add(Conv2D(20, activation='relu', kernel_size=3))\nfashion_model.add(Flatten())\nfashion_model.add(Dense(100, activation='relu'))\nfashion_model.add(Dense(10, activation='softmax'))\n\nfashion_model.summary()\n\nq_3.check()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:03:28.721192Z","iopub.execute_input":"2022-07-24T04:03:28.721660Z","iopub.status.idle":"2022-07-24T04:03:28.885365Z","shell.execute_reply.started":"2022-07-24T04:03:28.721597Z","shell.execute_reply":"2022-07-24T04:03:28.884763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"q_3.solution()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:03:28.888526Z","iopub.execute_input":"2022-07-24T04:03:28.888747Z","iopub.status.idle":"2022-07-24T04:03:28.898357Z","shell.execute_reply.started":"2022-07-24T04:03:28.888706Z","shell.execute_reply":"2022-07-24T04:03:28.897269Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 4) Compile Your Model\nCompile fashion_model with the `compile` method.  Specify the following arguments:\n1. `loss = \"categorical_crossentropy\"`\n2. `optimizer = 'adam'`\n3. `metrics = ['accuracy']`","metadata":{}},{"cell_type":"code","source":"# Your code to compile the model in this cell\nfashion_model.compile(loss='categorical_crossentropy',\n                      optimizer='adam',\n                      metrics=['accuracy'])","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:04:31.972884Z","iopub.execute_input":"2022-07-24T04:04:31.973197Z","iopub.status.idle":"2022-07-24T04:04:32.083708Z","shell.execute_reply.started":"2022-07-24T04:04:31.973128Z","shell.execute_reply":"2022-07-24T04:04:32.082959Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"q_4.solution()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:04:36.713537Z","iopub.execute_input":"2022-07-24T04:04:36.713938Z","iopub.status.idle":"2022-07-24T04:04:36.723469Z","shell.execute_reply.started":"2022-07-24T04:04:36.713877Z","shell.execute_reply":"2022-07-24T04:04:36.722591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 5) Fit The Model\nRun the command `fashion_model.fit`. The arguments you will use are\n1. The data used to fit the model. First comes the data holding the images, and second is the data with the class labels to be predicted. Look at the first code cell (which was supplied to you) where we called `prep_data` to find the variable names for these.\n2. `batch_size = 100`\n3. `epochs = 4`\n4. `validation_split = 0.2`\n\nWhen you run this command, you can watch your model start improving.  You will see validation accuracies after each epoch.","metadata":{}},{"cell_type":"code","source":"# Your code to fit the model here\nfashion_model.fit(x, y, batch_size=100, epochs=4, validation_split=0.2)\n#q_5.check()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:04:40.748517Z","iopub.execute_input":"2022-07-24T04:04:40.748826Z","iopub.status.idle":"2022-07-24T04:04:52.748137Z","shell.execute_reply.started":"2022-07-24T04:04:40.748769Z","shell.execute_reply":"2022-07-24T04:04:52.747241Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"q_5.solution()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:03:34.320005Z","iopub.status.idle":"2022-07-24T04:03:34.320629Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 6) Create A New Model\n\nCreate a new model called `second_fashion_model` in the cell below.  Make some changes so it is different than `fashion_model` that you've trained above. The change could be using a different number of layers, different number of convolutions in the layers, etc.\n\nDefine the model, compile it and fit it in the cell below.  See how it's validation score compares to that of the original model.","metadata":{}},{"cell_type":"code","source":"# Your code below\nsecond_fashion_model = Sequential()\nsecond_fashion_model.add(Conv2D(12,\n                         activation='relu',\n                         kernel_size=3,\n                         input_shape = (img_rows, img_cols, 1)))\n# Changed kernel sizes to be 2\nsecond_fashion_model.add(Conv2D(20, activation='relu', kernel_size=2))\nsecond_fashion_model.add(Conv2D(20, activation='relu', kernel_size=2))\n# added an addition Conv2D layer\nsecond_fashion_model.add(Conv2D(20, activation='relu', kernel_size=2))\nsecond_fashion_model.add(Flatten())\nsecond_fashion_model.add(Dense(100, activation='relu'))\n# It is important not to change the last layer. First argument matches number of classes. Softmax guarantees we get reasonable probabilities\nsecond_fashion_model.add(Dense(10, activation='softmax'))\n\nsecond_fashion_model.compile(loss='categorical_crossentropy',\n                             optimizer='adam',\n                             metrics=['accuracy'])\n\nsecond_fashion_model.fit(x, y, batch_size=100, epochs=4, validation_split=0.2)\n\nq_6.check()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:05:26.683605Z","iopub.execute_input":"2022-07-24T04:05:26.683923Z","iopub.status.idle":"2022-07-24T04:05:40.008405Z","shell.execute_reply.started":"2022-07-24T04:05:26.683872Z","shell.execute_reply":"2022-07-24T04:05:40.007772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"q_6.solution()","metadata":{"execution":{"iopub.status.busy":"2022-07-24T04:05:07.992786Z","iopub.execute_input":"2022-07-24T04:05:07.993066Z","iopub.status.idle":"2022-07-24T04:05:08.000138Z","shell.execute_reply.started":"2022-07-24T04:05:07.993016Z","shell.execute_reply":"2022-07-24T04:05:07.999404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Keep Going\nYou are ready to learn about **[strides and dropout](https://www.kaggle.com/dansbecker/dropout-and-strides-for-larger-models)**, which become important as you start using bigger and more powerful models.\n","metadata":{}}]}