{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "39c14b1b-a927-2feb-3aa7-c4d276bda2d6" }, "outputs": [], "source": [ "\n", "#Introduction#\n", "\n", "This is a prototype of a Digit Recognizer using A specific Fourier Transform, a connected component transform, some Kpi's from each training image and a feed forward Neural Nework with a hidden Layer.\n", "\n", "In the following chunks I'll describe step by step the process.\n", "\n", "#Data Import#\n", "\n", "```{r data input, warning=FALSE, cache=TRUE, include=FALSE}\n", "library(ggplot2)\n", "library(proto)\n", "library(readr)\n", "train <- data.frame(read_csv(\"data/train.csv\"))\n", "```\n", "\n", "#Binarization of the input#\n", "\n", "```{r cache=TRUE}\n", "#Visualization \n", "set.seed(71)\n", "n1=nrow(train)\n", "data<-train\n", "par(mfrow=c(10,10),mar=c(0.1,0.1,0.1,0.1))\n", "datos<-list()\n", "original<-list()\n", "\n", "l=0\n", "for (k in 1:n1)\n", "{\n", " l=l+1\n", " row <- NULL\n", " for (n in 2:785)\n", " row[n-1] <- train[k,n-1]\n", " matrix1 <- matrix(row,28,28,byrow=FALSE)\n", " matrix2 <- matrix(rep(0,784),28,28)\n", " matrix3 <- matrix(rep(0,784),28,28)\n", " \n", " for (i in 1:28)\n", " for (j in 1:28)\n", " {\n", " matrix3[i,28-j+1]<-matrix1[i,j]\n", " if (matrix1[i,j]>50) matrix2[i,28-j+1]<-1 \n", " }\n", " datos[[l]]<-matrix2\n", " original[[l]]<-matrix3\n", " \n", "\n", "}\n", "\n", "```\n", "\n", "In this two images we can see and example of how the binarization takes place.\n", "```{r}\n", " image(original[[10]], axes=FALSE, col=topo.colors(10))\n", "\n", " image(datos[[10]], axes=FALSE, col=topo.colors(10))\n", "```\n", "As we can see I had to choose a thredhold for every pixel to be a 1 or a 0. After several tryouts, the choosen number was 50 out of 255. This number gave me a good start to binare each image without a major information loss.\n", "\n", "#KPI Calculation#\n", "\n", "\n", "##Sums per row and per column##\n", "In this 2 kpi's we generate per image (hich as 28 pixels by 28 pixels), the sum of pixels per column and row. As an Example we can see the sum of columns and rows for the image above.\n", "```{r}\n", "\n", "##sum per column\n", "\n", "sum1<-matrix(rep(0,28),28)\n", "rows<-list()\n", "\n", "for (k in 1:n1)\n", "{\n", " image<-datos[[k]]\n", " for (i in 1:28)\n", " {\n", " sum1[i]<-0\n", " for (j in 1:28)\n", " {\n", " sum1[i]<-sum1[i]+image[i,j]\n", " \n", " }\n", " }\n", " rows[[k]]<-sum1\n", "}\n", "\n", "##sum per Row\n", "\n", "sum1<-matrix(rep(0,28),28)\n", "columns<-list()\n", "\n", "for (k in 1:n1)\n", "{\n", " image<-datos[[k]]\n", " for (i in 1:28)\n", " {\n", " sum1[i]<-0\n", " for (j in 1:28)\n", " {\n", " sum1[i]<-sum1[i]+image[j,i]\n", " \n", " }\n", " }\n", " columns[[k]]<-sum1\n", "}\n", "matrix(rows[[10]], nrow=1)\n", "matrix(columns[[10]], ncol=1)\n", "\n", "```\n", "\n", "## Dual Images##\n", "In the following 2 KPI's we take each picture and we duplicate the first part of it, like making a mirror of it.\n", "This Kpi will help us identify characters which its first half is the same as the second half. For example, the 0 and the 8 in both directions or the 3 in the vertical mirror.\n", "\n", "As an example we see the original picture and the dual picture of a character 3.\n", "\n", "```{r}\n", "\n", "#Generate dual image by row\n", "dual_row<-list()\n", "for (k in 1:n1)\n", "{\n", " image<-datos[[k]]\n", " dual_image<-image\n", " for (i in 1:28) {\n", " for (j in 1:14) {\n", " dual_image[i,j]<-image[i,j]\n", " dual_image[i,29-j]<-image[i,j]\n", " }\n", " }\n", " dual_row[[k]]<-dual_image\n", "}\n", "\n", "dual_col<-list()\n", "for (k in 1:n1)\n", "{\n", " image<-datos[[k]]\n", " dual_image<-image\n", " for (j in 1:28) {\n", " for (i in 1:14) {\n", " dual_image[i,j]<-image[i,j]\n", " dual_image[29-i,j]<-image[i,j]\n", " }\n", " }\n", " dual_col[[k]]<-dual_image\n", "}\n", "image(datos[[8]], axes=FALSE, col=topo.colors(10))\n", "image(dual_col[[8]], axes=FALSE, col=topo.colors(10))\n", "image(dual_row[[8]], axes=FALSE, col=topo.colors(10))\n", "```\n", "\n", "With this KPI's we are ready now to generate the parameters for the Neural Network.\n", "\n", "\n", "#Generation of parameters#\n", "\n", "1) H30: Sum of pixels at 30% character height as given by Sumcm as calculated using\n", "Equation 1, with c=0.3, m=42 and n=24.\n", "```{r}\n", "H30<-list()\n", "a= round(28/3)\n", "for (k in 1:n1)\n", "{\n", " sum=0\n", " for (j in 1:a)\n", " {\n", " sum=sum+columns[[k]][j]\n", " }\n", " H30[[k]]<-sum\n", "}\n", "\n", "```\n", "\n", "\n", "2. H50: Sum of pixels at 50% character height as given by Sumcm as calculated using\n", "Equation 1, with c=0.5, m=42 and n=24.\n", "```{r}\n", "H50<-list()\n", "a= round(28/2)\n", "for (k in 1:n1)\n", "{\n", " sum=0\n", " for (j in 1:a)\n", " {\n", " sum=sum+columns[[k]][j]\n", " }\n", " H50[[k]]<-sum\n", "}\n", "```\n", "\n", "3. H80: Sum of pixels at 80% character height as given by Sumcm as calculated using\n", "Equation 1, with c=0.8, m=42 and n=24.\n", "\n", "```{r}\n", "H80<-list()\n", "a= round(28/1.25)\n", "for (k in 1:n1)\n", "{\n", " sum=0\n", " for (j in 1:a)\n", " {\n", " sum=sum+columns[[k]][j]\n", " }\n", " H80[[k]]<-sum\n", "}\n", "\n", "\n", "```\n", "4. V30: Sum of pixels at 30% character width as given by Sumcn as calculated using\n", "Equation 2, with c=0.3, m=42 and n=24.\n", "```{r}\n", "V30<-list()\n", "a= round(28/3)\n", "for (k in 1:n1)\n", "{\n", " sum=0\n", " for (j in 1:a)\n", " {\n", " sum=sum+rows[[k]][j]\n", " }\n", " V30[[k]]<-sum\n", "}\n", "\n", "```\n", "5. V50: Sum of pixels at 50% character width as given by Sumcn as calculated using\n", "Equation 2, with c=0.5, m=42 and n=24.\n", "```{r}\n", "V50<-list()\n", "a= round(28/2)\n", "for (k in 1:n1)\n", "{\n", " sum=0\n", " for (j in 1:a)\n", " {\n", " sum=sum+rows[[k]][j]\n", " }\n", " V50[[k]]<-sum\n", "}\n", "\n", "```\n", "6. V80: Sum of pixels at 80% character width as given by Sumcn as calculated using\n", "Equation 2, with c=0.8, m=42 and n=24. \n", "```{r}\n", "V80<-list()\n", "a= round(28/1.25)\n", "for (k in 1:n1)\n", "{\n", " sum=0\n", " for (j in 1:a)\n", " {\n", " sum=sum+rows[[k]][j]\n", " }\n", " V80[[k]]<-sum\n", "}\n", "\n", "```\n", "\n", "7. Hsym: Correlation between an input character sample \u2018I\u2019 with \u2018X\u2019. It should be noted that\n", "\u2018X\u2019 is generated from \u2018I\u2019 where left half of \u2018X\u2019 is same as that of \u2018I\u2019 and right half of \u2018X\u2019\n", "is the mirror image of its left half.\n", "```{r}\n", "#We have to look for the correlation between the matrix datos and dual_row\n", "Hsym<-list()\n", "a= round(28/1.25)\n", "for (k in 1:n1)\n", "{\n", " Hsym[[k]]<-cor(c(datos[[k]]), c(dual_row[[k]]))\n", "}\n", "\n", "```\n", "\n", "8. Vsym: Correlation between an input character sample \u2018I\u2019 with \u2018X\u2019. It should be noted that\n", "\u2018X\u2019 is generated from \u2018I\u2019 where upper half of \u2018X\u2019 is same as that of \u2018I\u2019 and lower half of\n", "\u2018X\u2019 is the mirror image of its upper half.\n", "```{r}\n", "#We have to look for the correlation between the matrix datos and dual_row\n", "Vsym<-list()\n", "for (k in 1:n1)\n", "{\n", " Vsym[[k]]<-cor(c(datos[[k]]), c(dual_col[[k]]))\n", "}\n", "\n", "```\n", "9. Pos: It denotes the \u2018position\u2019 at which the calculated WHT of an input sample shows\n", "highest correlation to the WHT database, containing Walsh Hadamard Transform of each\n", "character in order A-Z and 1-0.\n", "\n", "\n", "Walsh transform\n", "WHT DataBase Generation\n", "\n", "```{r}\n", "library(readbitmap)\n", "library(proto)\n", "library(readr)\n", "library(biclust)\n", "library(bmp)\n", "\n", "numeros<-list()\n", "rotate <- function(x) t(apply(x, 2, rev))\n", "for (k in 1:10) \n", "{\n", "t=k-1\n", "numeros[[k]] <- rotate(abs(binarize(read.bmp(paste(paste('numbers/',t,sep = \"\"),'.bmp',sep = \"\"))[,,1],threshold=100)-1))\n", "}\n", "#image(img[[10]], axes=FALSE, col=topo.colors(100))\n", "\n", "\n", "\n", "library(phangorn)\n", "hm<-list()\n", "#N+1: 0 is in the first position.\n", "\n", "hm[[1]]<-fhm(numeros[[1]])\n", "hm[[2]]<-fhm(numeros[[2]])\n", "hm[[3]]<-fhm(numeros[[3]])\n", "hm[[4]]<-fhm(numeros[[4]])\n", "hm[[5]]<-fhm(numeros[[5]])\n", "hm[[6]]<-fhm(numeros[[6]])\n", "hm[[7]]<-fhm(numeros[[7]])\n", "hm[[8]]<-fhm(numeros[[8]])\n", "hm[[9]]<-fhm(numeros[[9]])\n", "hm[[10]]<-fhm(numeros[[10]])\n", "```\n", "\n", "\n", "I need to compare de correlation of the denoted image with the WHT DataBase.\n", "\n", "```{r}\n", "\n", "library(phangorn)\n", "WHT0<-list()\n", "WHT1<-list()\n", "WHT2<-list()\n", "WHT3<-list()\n", "WHT4<-list()\n", "WHT5<-list()\n", "WHT6<-list()\n", "WHT7<-list()\n", "WHT8<-list()\n", "WHT9<-list()\n", "\n", "correl<-list()\n", "for (k in 1:n1)\n", "{\n", "hm_num<-fhm(datos[[k]])\n", "WHT0[[k]]<-cor(hm_num,hm[[1]])\n", "WHT1[[k]]<-cor(hm_num,hm[[2]])\n", "WHT2[[k]]<-cor(hm_num,hm[[3]])\n", "WHT3[[k]]<-cor(hm_num,hm[[4]])\n", "WHT4[[k]]<-cor(hm_num,hm[[5]])\n", "WHT5[[k]]<-cor(hm_num,hm[[6]])\n", "WHT6[[k]]<-cor(hm_num,hm[[7]])\n", "WHT7[[k]]<-cor(hm_num,hm[[8]])\n", "WHT8[[k]]<-cor(hm_num,hm[[9]])\n", "WHT9[[k]]<-cor(hm_num,hm[[10]])\n", " } \n", "\n", "\n", " WHT0<-c(do.call(\"cbind\",WHT0)) \n", " WHT1<-c(do.call(\"cbind\",WHT1)) \n", " WHT2<-c(do.call(\"cbind\",WHT2)) \n", " WHT3<-c(do.call(\"cbind\",WHT3)) \n", " WHT4<-c(do.call(\"cbind\",WHT4)) \n", " WHT5<-c(do.call(\"cbind\",WHT5)) \n", " WHT6<-c(do.call(\"cbind\",WHT6)) \n", " WHT7<-c(do.call(\"cbind\",WHT7)) \n", " WHT8<-c(do.call(\"cbind\",WHT8))\n", " WHT9<-c(do.call(\"cbind\",WHT9)) \n", " image(datos[[4]], axes=FALSE, col=topo.colors(100))\n", " head(train$label,10)\n", "head(WHT0,10)\n", "head(WHT1,10)\n", "head(WHT2,10)\n", "head(WHT3,10)\n", "head(WHT4,10)\n", "head(WHT5,10)\n", "head(WHT6,10)\n", "head(WHT7,10)\n", "head(WHT8,10)\n", "head(WHT9,10)\n", "\n", "\n", "\n", "#summary(as.factor(sapply(WHT,median)))\n", "\n", "```\n", "\n", "\n", "10. CC: It gives a measure of the number of closed areas in a character. In an image, all the\n", "connected pixels are given same labels. Thus if there are two sets of such pixels, as in \u20188\u2019,\n", "they would be labeled as \u20181\u2019 and \u20182\u2019. Thus the highest value of \u2018label\u2019 gives an idea of\n", "closed areas present in the character.\n", "\n", "We use the connected component transform (Rosenfeld and Pfalz, 1966)\n", "```{r}\n", "library(spatstat)\n", " \n", "CNCP<-list()\n", " \n", "for (k in 1:n1)\n", "{\n", " x2 <- im(datos[[k]], xcol=seq(1,length=28), yrow=seq(1,length=28))\n", " X <- levelset(x2, 0.06)\n", " #plot(X)\n", " Z <- connected(X)\n", " #plot(Z)\n", " # number of components\n", " nc <- length(levels(Z))\n", " # plot with randomised colour map\n", " # plot(Z, col=hsv(h=sample(seq(0,1,length=nc), nc)))\n", " CNCP[[k]]<-nc\n", "}\n", "\n", "\n", "```\n", "\n", "\n", "11. Sumt: This is a secondary feature, calculated by adding the values of all the ten features\n", "evaluated previously. This is an additional feature, used to increase the entropy of the\n", "system, for better recognition\n", "```{r}\n", "\n", "```\n", "\n", " \n", "\n", "13. Data: Dataset of all the calculated kpi's per photo.\n", "\n", "```{r}\n", "\n", "\n", "H30_m <- matrix(unlist(H30), nrow=n1)\n", "H50_m <- matrix(unlist(H50), nrow=n1)\n", "H80_m <- matrix(unlist(H80), nrow=n1)\n", "V30_m <- matrix(unlist(V30), nrow=n1)\n", "V50_m <- matrix(unlist(V50), nrow=n1)\n", "V80_m <- matrix(unlist(V80), nrow=n1)\n", "Hsym_m <- matrix(unlist(Hsym), nrow=n1)\n", "Vsym_m <- matrix(unlist(Vsym), nrow=n1)\n", "Vsym_m[is.na(Vsym_m)] <-0\n", "WHT0_m <- matrix(WHT0, nrow=n1)\n", "WHT1_m <- matrix(WHT1, nrow=n1)\n", "WHT2_m <- matrix(WHT2, nrow=n1)\n", "WHT3_m <- matrix(WHT3, nrow=n1)\n", "WHT4_m <- matrix(WHT4, nrow=n1)\n", "WHT5_m <- matrix(WHT5, nrow=n1)\n", "WHT6_m <- matrix(WHT6, nrow=n1)\n", "WHT7_m <- matrix(WHT7, nrow=n1)\n", "WHT8_m <- matrix(WHT8, nrow=n1)\n", "WHT9_m <- matrix(WHT9, nrow=n1)\n", "CNCP_m <- matrix(unlist(CNCP), nrow=n1)\n", "label<-factor(data$label)\n", "\n", "#Normalize the inputs\n", "\n", "train_final<-data.frame(H30_m,H50_m,H80_m,V30_m,V50_m,V80_m,Hsym_m,Vsym_m,WHT0_m,WHT1_m,WHT2_m,WHT3_m,WHT4_m,WHT5_m,WHT6_m,WHT7_m,WHT8_m,WHT9_m,CNCP_m,label)\n", "\n", "\n", "```\n", "#Train and Test#\n", "```{r train and test }\n", "\n", "library(car)\n", "library(caret)\n", "preObj <- preProcess(x=train_final, method=c(\"center\", \"scale\"))\n", "trainIndex <- createDataPartition(train_final$label, p=.7, list=F)\n", "\n", "numbers.traintotal<-predict(preObj,train_final)\n", "\n", "numbers.train<- predict(preObj, train_final[trainIndex, ])\n", "numbers.test <- predict(preObj, train_final[-trainIndex, ])\n", "\n", "\n", "```\n", "\n", "\n", "\n", "```{r}\n", "\n", "#numbers.train.fs<-numbers.train[,c(2,5,9:20)]\n", "\n", "#my.grid <- expand.grid(.decay = c(0.1,0.5), .size = c(7,8))\n", "#numbers.fit.fs <- train(label ~ ., data = numbers.train.fs,\n", "# method = \"nnet\", maxit = 1000, tuneGrid = my.grid, trace = F, linout = 1) \n", "#numbers.fit <- train(label ~ ., data = numbers.train,\n", "# method = \"nnet\", maxit = 1000, tuneGrid = my.grid, trace = F, linout = 1) \n", "# my.grid2 <- expand.grid(.decay = c(0.1,0.5), .size = c(9,10))\n", "# numbers.fit2 <- train(label ~ ., data = numbers.train,\n", "# method = \"nnet\", maxit = 1000, tuneGrid = my.grid2, trace = F, linout = 1) \n", "\n", "# my.grid3 <- expand.grid(.decay = c(0.1,0.5), .size = c(20,21))\n", " my.grid3 <- expand.grid(.decay = c(0.1,0.5), .size = c(21))\n", " #train_control <- trainControl(method=\"boot\", number=100)\n", " train_control <- trainControl(method=\"repeatedcv\", number=10, repeats=3)\n", " numbers.fit4 <- train(label ~ ., data = numbers.train,\n", " method = \"nnet\", maxit = 1000, trContor=train_control,tuneGrid = my.grid3, trace = F, linout = 1) \n", " \n", "\n", "metric <- \"Accuracy\"\n", "\n", "control <- trainControl(method=\"repeatedcv\", number=10, repeats=3, search=\"random\")\n", "rf_random <- train(label ~ ., data = numbers.train, method=\"rf\", metric=metric, tuneLength=15, trControl=control)\n", "\n", "#http://stats.stackexchange.com/questions/109079/confusion-between-caret-randomforest-predict-results-and-reported-model-perfor\n", "\n", "#https://www.kaggle.com/cnjn22/digit-recognizer/digit-recognizer\n", "\n", "\n", "\n", "first<-predict(numbers.fit4,numbers.test)\n", "second<-predict(rf_random,numbers.test)\n", "table(first,numbers.test$label)\n", "table(second,numbers.test$label)\n", "\n", "```\n", "\n", "\n", "I generated 20 different artificial measures, including : H30: Sum of pixels at 30% character height H50: Sum of pixels at 50% character heightH80: Sum of pixels at 80% character heightV30: Sum of pixels at 30% character widthV50: Sum of pixels at 50% characterV80: Sum of pixels at 80% characterHsym: Correlation between an input character sample \u2018I\u2019 with \u2018X\u2019.Vsym: Correlation between an input character sample \u2018I\u2019 with \u2018X\u2019. 10 variables including the Correlation of each character with a default charater using the Walsh H. Transformation.CC: It gives a measure of the number of closed areas in a character.Given this inputs I generated a Neural Network with 1 hidden layer with 21 neurons. " ] } ], "metadata": { "_change_revision": 0, "_is_fork": false, "kernelspec": { "display_name": "R", "language": "R", "name": "ir" }, "language_info": { "codemirror_mode": "r", "file_extension": ".r", "mimetype": "text/x-r-source", "name": "R", "pygments_lexer": "r", "version": "3.3.3" } }, "nbformat": 4, "nbformat_minor": 0 }