{
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0,
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "00b63778-674d-0a2d-e5b0-8956c5ce1770",
        "_active": false
      },
      "source": "## Introduction\n###Let me start by saying, this is not the best way to classify digits! This notebook is rather meant to be for someone who might not know where to start. As an ml beginner myself, I find it helpful to play with these sorts of commented kernels. Any suggestions for improvement or comments on poor coding practices are appreciated!",
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "d0abbfc5-e04e-98d3-8c5c-37d114daa804",
        "_active": true
      },
      "outputs": [],
      "source": "import pandas as pd\nimport matplotlib.pyplot as plt, matplotlib.image as mpimg\nfrom sklearn.model_selection import train_test_split\nfrom sklearn import svm\n%matplotlib inline"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "11f567ca-77d0-2c41-bbc9-a60396caacea",
        "_active": false
      },
      "source": "## Loading the data\n- We use panda's [read_csv][1]  to read train.csv into a [dataframe][2].\n- Then we separate our images and labels for supervised learning. \n- We also do a [train_test_split][3] to break our data into two sets, one for training and one for testing. This let's us measure how well our model was trained by later inputting some known test data.\n\n### For the sake of time, we're only using 5000 images. You should increase or decrease this number to see how it affects model training.\n\n\n  [1]: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html\n  [2]: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html#pandas.DataFrame\n  [3]: http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html",
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "e8c0e206-f504-253f-8ac6-2c50eff06b58",
        "_active": false,
        "collapsed": false
      },
      "outputs": [],
      "source": "import numpy as np\n\nlabeled_images = pd.read_csv('../input/train.csv')\n\n#np.random.seed = 28\n#rand_image_index=np.random.randint(0,labeled_images.shape[0],5000)\n#images = labeled_images.iloc[rand_image_index,1:]\n#labels = labeled_images.iloc[rand_image_index,:1]\n\nimages = labeled_images.iloc[:,1:]\nlabels = labeled_images.iloc[:,:1]\n\ntrain_images, test_images,train_labels, test_labels = train_test_split(images, labels, train_size=0.8, random_state=0)\ntest_images /= 255\ntrain_images /= 255"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "caa9e676-3656-5509-0aa4-a503437f727f",
        "_active": false
      },
      "source": "## Viewing an Image\n- Since the image is currently one-dimension, we load it into a [numpy array][1] and [reshape][2] it so that it is two-dimensional (28x28 pixels)\n- Then, we plot the image and label with matplotlib\n\n### You can change the value of variable <i>i</i> to check out other images and labels.\n\n\n  [1]: https://docs.scipy.org/doc/numpy/reference/generated/numpy.array.html\n  [2]: https://docs.scipy.org/doc/numpy/reference/generated/numpy.reshape.html",
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "dc43b65b-d3d9-8208-e7a2-88c2535b506f",
        "_active": false
      },
      "outputs": [],
      "source": "\"\"\"i=1\nimg=train_images.iloc[i].as_matrix()\nimg=img.reshape((28,28))\nplt.imshow(img,cmap='gray')\nplt.title(train_labels.iloc[i,0])\n\"\"\""
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "42a5a5bd-83be-7116-7cd5-20cbc8336417",
        "_active": false
      },
      "source": "## Examining the Pixel Values\n### Note that these images aren't actually black and white (0,1). They are gray-scale (0-255). \n- A [histogram][1] of this image's pixel values shows the range.\n\n\n  [1]: http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.hist",
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "b6143ae2-eb4e-a4af-aad3-37fa85b657b7",
        "_active": false
      },
      "outputs": [],
      "source": "#plt.hist(train_images.iloc[i])"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "b7f8578b-9985-4ccc-eab5-82465d7cad8b",
        "_active": false
      },
      "source": "## Training our model\n- First, we use the [sklearn.svm][1] module to create a [vector classifier][2]. \n- Next, we pass our training images and labels to the classifier's [fit][3] method, which trains our model. \n- Finally, the test images and labels are passed to the [score][4] method to see how well we trained our model. Fit will return a float between 0-1 indicating our accuracy on the test data set\n\n### Try playing with the parameters of svm.SVC to see how the results change. \n\n\n  [1]: http://scikit-learn.org/stable/modules/svm.html\n  [2]: http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html\n  [3]: http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC.fit\n  [4]: http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC.score\n  [5]: http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html#sklearn.svm.SVC.score",
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "e35c02d7-4510-3202-a7b4-fb3accde0c5a",
        "_active": false
      },
      "outputs": [],
      "source": "clf = svm.SVC(C=10, gamma=0.01)\nclf.fit(train_images, train_labels.values.ravel())\nclf.score(test_images,test_labels)"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "61729c3a-fc18-d204-1e72-a453af1ff0a3",
        "_active": false
      },
      "source": "## How did our model do?\n### You should have gotten around 0.10, or 10% accuracy. This is terrible. 10% accuracy is what get if you randomly guess a number. There are many ways to improve this, including not using a vector classifier, but here's a simple one to start. Let's just simplify our images by making them true black and white.\n\n- To make this easy, any pixel with a value simply becomes 1 and everything else remains 0.\n- We'll plot the same image again to see how it looks now that it's black and white. Look at the histogram now.",
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "b5f3a212-b832-55af-d228-9b1931dcbc32",
        "_active": false
      },
      "outputs": [],
      "source": "#img=train_images.iloc[i].as_matrix().reshape((28,28))\n#plt.imshow(img,cmap='binary')\n#plt.title(train_labels.iloc[i])"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "edbe1d68-3224-1bf4-b571-a3bd845c3de5",
        "_active": false
      },
      "outputs": [],
      "source": "#plt.hist(train_images.iloc[i])"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "785c75aa-06f7-fce3-b690-038356d4a51c",
        "_active": false
      },
      "source": "## Retraining our model\n### We follow the same procedure as before, but now our training and test sets are black and white instead of gray-scale. Our score still isn't great, but it's a huge improvement.",
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "6f0355ef-d019-c814-11ff-795f81d041f1",
        "_active": false
      },
      "outputs": [],
      "source": "#clf = svm.SVC()\n#clf.fit(train_images, train_labels.values.ravel())\n#clf.score(test_images,test_labels)"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "7ca53ece-d50a-efe5-d242-dea8d089244b",
        "_active": false
      },
      "source": "## Labelling the test data\n### Now for those making competition submissions, we can load and predict the unlabeled data from test.csv. Again, for time we're just using the first 5000 images. We then output this data to a results.csv for competition submission.",
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "8fa0b0aa-1bf3-dcd3-6ef2-fae5ccda55b3",
        "_active": false
      },
      "outputs": [],
      "source": "test_data=pd.read_csv('../input/test.csv')\ntest_data[test_data>0]=1\nresults=clf.predict(test_data)"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "28ec8d56-d3f6-573d-2230-bc5a410868c6",
        "_active": false
      },
      "outputs": [],
      "source": "df = pd.DataFrame({'ImageID': range(1, test_data.shape[0]+1, 1),\n                   'Label': results})\n#df.index.name='ImageId'\n#df.index+=1\n#df.columns=['Label']\ndf.to_csv('results.csv', header=True)"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "097e0f0a-246c-2f96-cd2d-b5cd75f26a65",
        "_active": false
      },
      "outputs": [],
      "source": ""
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "_cell_guid": "242f04e8-ceec-ed24-5658-1d25b5bf1211",
        "_active": false
      },
      "outputs": [],
      "source": ""
    }
  ]
}