{"cells":[{"metadata":{"_uuid":"67e976faab9af355494b17fa5f4a577b1eec2252"},"cell_type":"markdown","source":"# Machine Learning (ML) - Awesome Tutorials and Basic Text Classification with Movie Reviews\n\n This kernel contains Text classification with Movie Reviews and a topic-wise curated list of Machine Learning and Deep Learning tutorials, articles and other resources. \n\n> #### **Credits**: Thanks to **TensorFlow Team**, **Ujjwal Karn** and other contributers for such wonderful curated collections!\n\n### Here are some of *my kernel notebooks* for **Machine Learning and Data Science** as follows, ***Upvote*** them if you *like* them\n\n> * [Awesome Deep Learning Basics and Resources](https://www.kaggle.com/arunkumarramanan/awesome-deep-learning-resources)\n> * [Data Science with R - Awesome Tutorials](https://www.kaggle.com/arunkumarramanan/data-science-with-r-awesome-tutorials)\n> * [Data Science and Machine Learning Cheetcheets](https://www.kaggle.com/arunkumarramanan/data-science-and-machine-learning-cheatsheets)\n> * [Awesome ML Frameworks and MNIST Classification](https://www.kaggle.com/arunkumarramanan/awesome-machine-learning-ml-frameworks)\n> * [Awesome Data Science for Beginners with Titanic Exploration](https://kaggle.com/arunkumarramanan/awesome-data-science-for-beginners)\n> * [Tensorflow Tutorial and House Price Prediction](https://www.kaggle.com/arunkumarramanan/tensorflow-tutorial-and-examples)\n> * [Practical Machine Learning with PyTorch](https://www.kaggle.com/arunkumarramanan/practical-machine-learning-with-pytorch)\n> * [Data Scientist's Toolkits - Awesome Data Science Resources](https://www.kaggle.com/arunkumarramanan/data-scientist-s-toolkits-awesome-ds-resources)\n> * [Awesome Computer Vision Resources (TBU)](https://www.kaggle.com/arunkumarramanan/awesome-computer-vision-resources-to-be-updated)\n> * [Data Science with Python - Awesome Tutorials](https://www.kaggle.com/arunkumarramanan/data-science-with-python-awesome-tutorials)\n> * [Machine Learning Engineer's Toolkit with Roadmap](https://www.kaggle.com/arunkumarramanan/machine-learning-engineer-s-toolkit-with-roadmap) \n> * [Awesome TensorFlow and PyTorch Resources](https://www.kaggle.com/arunkumarramanan/awesome-tensorflow-and-pytorch-resources)\n> * [Hands-on ML with scikit-learn and TensorFlow](https://www.kaggle.com/arunkumarramanan/hands-on-ml-with-scikit-learn-and-tensorflow)\n> * [Awesome Data Science IPython Notebooks](https://www.kaggle.com/arunkumarramanan/awesome-data-science-ipython-notebooks)\n> * [Machine Learning and Deep Learning - Awesome Tutorials](https://www.kaggle.com/arunkumarramanan/awesome-deep-learning-ml-tutorials)"},{"metadata":{"_uuid":"471063453fe22a507db228edfc51d123286f92d0"},"cell_type":"markdown","source":"# Text Classification with IMDB Movie Reviews from TensorFlow Docs\n\n\nThis notebook classifies movie reviews as *positive* or *negative* using the text of the review. This is an example of *binary*—or two-class—classification, an important and widely applicable kind of machine learning problem. \n\nWe'll use the [IMDB dataset](https://www.tensorflow.org/api_docs/python/tf/keras/datasets/imdb) that contains the text of 50,000 movie reviews from the [Internet Movie Database](https://www.imdb.com/). These are split into 25,000 reviews for training and 25,000 reviews for testing. The training and testing sets are *balanced*, meaning they contain an equal number of positive and negative reviews. \n\nThis notebook uses [tf.keras](https://www.tensorflow.org/guide/keras), a high-level API to build and train models in TensorFlow. For a more advanced text classification tutorial using `tf.keras`, see the [MLCC Text Classification Guide](https://developers.google.com/machine-learning/guides/text-classification/)."},{"metadata":{"trusted":true,"_uuid":"f4e965aab7fdf6215ff3426116484b899b123d5b"},"cell_type":"code","source":"from __future__ import absolute_import, division, print_function\n\nimport tensorflow as tf\nfrom tensorflow import keras\n\nimport numpy as np\n\nprint(tf.__version__)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ef6a2761e1ac0381853cd6164d22fc338322e5f1"},"cell_type":"markdown","source":"## Download the IMDB dataset\n\nThe IMDB dataset comes packaged with TensorFlow. It has already been preprocessed such that the reviews (sequences of words) have been converted to sequences of integers, where each integer represents a specific word in a dictionary.\n\nThe following code downloads the IMDB dataset to your machine (or uses a cached copy if you've already downloaded it):"},{"metadata":{"trusted":true,"_uuid":"fc32852b5f489292602c949a76a9d6db92ad49b7"},"cell_type":"code","source":"imdb = keras.datasets.imdb\n\n(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"36ad7e030828c25dcfb311a199d8b0e4714851d4"},"cell_type":"markdown","source":"The argument `num_words=10000` keeps the top 10,000 most frequently occurring words in the training data. The rare words are discarded to keep the size of the data manageable.\n\n## Explore the data \n\nLet's take a moment to understand the format of the data. The dataset comes preprocessed: each example is an array of integers representing the words of the movie review. Each label is an integer value of either 0 or 1, where 0 is a negative review, and 1 is a positive review."},{"metadata":{"trusted":true,"_uuid":"fe9afd64bd9b8d3b09cf640d25b769c97af4b324"},"cell_type":"code","source":"print(\"Training entries: {}, labels: {}\".format(len(train_data), len(train_labels)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dbad3e2ae203bd5c672dff7d1ce872efc116f083"},"cell_type":"markdown","source":"The text of reviews have been converted to integers, where each integer represents a specific word in a dictionary. Here's what the first review looks like:"},{"metadata":{"trusted":true,"_uuid":"e64ea87b3309e3fd81eb21d3dca74e5757505bc3"},"cell_type":"code","source":"print(train_data[0])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"26cd674b493e06633a6816640c4772d51409530a"},"cell_type":"markdown","source":"Movie reviews may be different lengths. The below code shows the number of words in the first and second reviews. Since inputs to a neural network must be the same length, we'll need to resolve this later."},{"metadata":{"trusted":true,"_uuid":"f63ff4b9b040bf980af00e94525c60fa35359868"},"cell_type":"code","source":"len(train_data[0]), len(train_data[1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"759e97447e824169d54930c13649a86ed3aa0758"},"cell_type":"markdown","source":"### Convert the integers back to words\n\nIt may be useful to know how to convert integers back to text. Here, we'll create a helper function to query a dictionary object that contains the integer to string mapping:"},{"metadata":{"trusted":true,"_uuid":"6f8f45ce4fa615614fff5252453b379da024169d"},"cell_type":"code","source":"# A dictionary mapping words to an integer index\nword_index = imdb.get_word_index()\n\n# The first indices are reserved\nword_index = {k:(v+3) for k,v in word_index.items()} \nword_index[\"\"] = 0\nword_index[\"\"] = 1\nword_index[\"\"] = 2 # unknown\nword_index[\"\"] = 3\n\nreverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n\ndef decode_review(text):\n return ' '.join([reverse_word_index.get(i, '?') for i in text])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"306d7546dfee9f4223eb709a354a3303af2155ab"},"cell_type":"markdown","source":"Now we can use the `decode_review` function to display the text for the first review:"},{"metadata":{"trusted":true,"_uuid":"4c6e7dbc2b8121ad5c3522f2fffafc5bfce30bd3"},"cell_type":"code","source":"decode_review(train_data[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c3eb4a38a49b27576a7257a41ecb654c398a4cfb"},"cell_type":"markdown","source":"## Prepare the data\n\nThe reviews—the arrays of integers—must be converted to tensors before fed into the neural network. This conversion can be done a couple of ways:\n\n* Convert the arrays into vectors of 0s and 1s indicating word occurrence, similar to a one-hot encoding. For example, the sequence [3, 5] would become a 10,000-dimensional vector that is all zeros except for indices 3 and 5, which are ones. Then, make this the first layer in our network—a Dense layer—that can handle floating point vector data. This approach is memory intensive, though, requiring a `num_words * num_reviews` size matrix.\n\n* Alternatively, we can pad the arrays so they all have the same length, then create an integer tensor of shape `max_length * num_reviews`. We can use an embedding layer capable of handling this shape as the first layer in our network.\n\nIn this tutorial, we will use the second approach. \n\nSince the movie reviews must be the same length, we will use the [pad_sequences](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/sequence/pad_sequences) function to standardize the lengths:"},{"metadata":{"trusted":true,"_uuid":"975ebfc6de19282d85a0402068623277a615c155"},"cell_type":"code","source":"train_data = keras.preprocessing.sequence.pad_sequences(train_data,\n value=word_index[\"\"],\n padding='post',\n maxlen=256)\n\ntest_data = keras.preprocessing.sequence.pad_sequences(test_data,\n value=word_index[\"\"],\n padding='post',\n maxlen=256)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2a9924adb578e56d8d53b98e78dd37f1e489901b"},"cell_type":"markdown","source":"Let's look at the length of the examples now:"},{"metadata":{"trusted":true,"_uuid":"b3f7fc5a59410b9aa1d1a41f9b0e784c3b5d41f8"},"cell_type":"code","source":"len(train_data[0]), len(train_data[1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7ff3d37c7df0ce3120dafb281ae8abf662a88841"},"cell_type":"markdown","source":"The larger magnitude contributions have a larger impact on the model's prediction. Negative contributions indicate the feature value for this given example reduced the model's prediction, while positive values contribute an increase in the prediction."},{"metadata":{"trusted":true,"_uuid":"4c00e856f704fdd99a9290b543dfb6a472d83b2b"},"cell_type":"code","source":"print(train_data[0])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"96f104e3c40027fe13422453e351c58f201beb86"},"cell_type":"markdown","source":"## Build the model\n\nThe neural network is created by stacking layers—this requires two main architectural decisions:\n\n* How many layers to use in the model?\n* How many *hidden units* to use for each layer?\n\nIn this example, the input data consists of an array of word-indices. The labels to predict are either 0 or 1. Let's build a model for this problem:"},{"metadata":{"trusted":true,"_uuid":"3d23b47fed86e2d74ac77542bb70b74f33e35e14"},"cell_type":"code","source":"# input shape is the vocabulary count used for the movie reviews (10,000 words)\nvocab_size = 10000\n\nmodel = keras.Sequential()\nmodel.add(keras.layers.Embedding(vocab_size, 16))\nmodel.add(keras.layers.GlobalAveragePooling1D())\nmodel.add(keras.layers.Dense(16, activation=tf.nn.relu))\nmodel.add(keras.layers.Dense(1, activation=tf.nn.sigmoid))\n\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ef1165f71d993b14f2c383fff2482805325029d0"},"cell_type":"markdown","source":"The layers are stacked sequentially to build the classifier:\n\n1. The first layer is an `Embedding` layer. This layer takes the integer-encoded vocabulary and looks up the embedding vector for each word-index. These vectors are learned as the model trains. The vectors add a dimension to the output array. The resulting dimensions are: `(batch, sequence, embedding)`.\n2. Next, a `GlobalAveragePooling1D` layer returns a fixed-length output vector for each example by averaging over the sequence dimension. This allows the model to handle input of variable length, in the simplest way possible.\n3. This fixed-length output vector is piped through a fully-connected (`Dense`) layer with 16 hidden units.\n4. The last layer is densely connected with a single output node. Using the `sigmoid` activation function, this value is a float between 0 and 1, representing a probability, or confidence level."},{"metadata":{"_uuid":"688778b4b4471e900b28c37cc647c1e8eef4139e"},"cell_type":"markdown","source":"### Hidden units\n\nThe above model has two intermediate or \"hidden\" layers, between the input and output. The number of outputs (units, nodes, or neurons) is the dimension of the representational space for the layer. In other words, the amount of freedom the network is allowed when learning an internal representation.\n\nIf a model has more hidden units (a higher-dimensional representation space), and/or more layers, then the network can learn more complex representations. However, it makes the network more computationally expensive and may lead to learning unwanted patterns—patterns that improve performance on training data but not on the test data. This is called *overfitting*, and we'll explore it later."},{"metadata":{"_uuid":"5ed64914aea190fac61be00e83e952daf218be0d"},"cell_type":"markdown","source":"### Loss function and optimizer\n\nA model needs a loss function and an optimizer for training. Since this is a binary classification problem and the model outputs a probability (a single-unit layer with a sigmoid activation), we'll use the `binary_crossentropy` loss function. \n\nThis isn't the only choice for a loss function, you could, for instance, choose `mean_squared_error`. But, generally, `binary_crossentropy` is better for dealing with probabilities—it measures the \"distance\" between probability distributions, or in our case, between the ground-truth distribution and the predictions.\n\nLater, when we are exploring regression problems (say, to predict the price of a house), we will see how to use another loss function called mean squared error.\n\nNow, configure the model to use an optimizer and a loss function:"},{"metadata":{"trusted":true,"_uuid":"6eed0a4d82bcee419e9a7203041d7c4b4e1b0554"},"cell_type":"code","source":"model.compile(optimizer='adam',\n loss='binary_crossentropy',\n metrics=['acc'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"55c32a7877beddc7ae4b207c78e304705547a35b"},"cell_type":"markdown","source":"## Create a validation set\n\nWhen training, we want to check the accuracy of the model on data it hasn't seen before. Create a *validation set* by setting apart 10,000 examples from the original training data. (Why not use the testing set now? Our goal is to develop and tune our model using only the training data, then use the test data just once to evaluate our accuracy)."},{"metadata":{"trusted":true,"_uuid":"464069e7fd96c5fe93bb96189c44854cc118769c"},"cell_type":"code","source":"x_val = train_data[:10000]\npartial_x_train = train_data[10000:]\n\ny_val = train_labels[:10000]\npartial_y_train = train_labels[10000:]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b722719bac464450b269b14aa0aa5ecbc4957c9d"},"cell_type":"markdown","source":"## Train the model\n\nTrain the model for 40 epochs in mini-batches of 512 samples. This is 40 iterations over all samples in the `x_train` and `y_train` tensors. While training, monitor the model's loss and accuracy on the 10,000 samples from the validation set:"},{"metadata":{"trusted":true,"_uuid":"7b96432dfced6becf80d0efdc5b92c20af0aa366"},"cell_type":"code","source":"history = model.fit(partial_x_train,\n partial_y_train,\n epochs=40,\n batch_size=512,\n validation_data=(x_val, y_val),\n verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dc1f35cb53acf2315f9a3fcf2ef2875f4683d87f"},"cell_type":"markdown","source":"## Evaluate the model\n\nAnd let's see how the model performs. Two values will be returned. Loss (a number which represents our error, lower values are better), and accuracy."},{"metadata":{"trusted":true,"_uuid":"8a5669703803d2c582f416dddab542a7bee48dc3"},"cell_type":"code","source":"results = model.evaluate(test_data, test_labels)\n\nprint(results)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"67d4cce6c7a86ef4078814152dfa9da9e9b890dc"},"cell_type":"markdown","source":"This fairly naive approach achieves an accuracy of about 87%. With more advanced approaches, the model should get closer to 95%."},{"metadata":{"_uuid":"0fc64d43bdd14a6006bddf569ef6734d0d60a7c9"},"cell_type":"markdown","source":"## Create a graph of accuracy and loss over time\n\n`model.fit()` returns a `History` object that contains a dictionary with everything that happened during training:"},{"metadata":{"trusted":true,"_uuid":"b49022767303692a5b74c80c587395c7e30207fb"},"cell_type":"code","source":"history_dict = history.history\nhistory_dict.keys()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f406c646add493f2d7ec1f2a9aa257d499f82f74"},"cell_type":"markdown","source":"There are four entries: one for each monitored metric during training and validation. We can use these to plot the training and validation loss for comparison, as well as the training and validation accuracy:"},{"metadata":{"trusted":true,"_uuid":"b5a4f9172937b694b326a88ccf64d5987b8c5ba2"},"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nacc = history_dict['acc']\nval_acc = history_dict['val_acc']\nloss = history_dict['loss']\nval_loss = history_dict['val_loss']\n\nepochs = range(1, len(acc) + 1)\n\n# \"bo\" is for \"blue dot\"\nplt.plot(epochs, loss, 'bo', label='Training loss')\n# b is for \"solid blue line\"\nplt.plot(epochs, val_loss, 'b', label='Validation loss')\nplt.title('Training and validation loss')\nplt.xlabel('Epochs')\nplt.ylabel('Loss')\nplt.legend()\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"71ef0cd92f7b5e6db10bb192818c8a316f8cacf3"},"cell_type":"code","source":"plt.clf() # clear figure\n\nplt.plot(epochs, acc, 'bo', label='Training acc')\nplt.plot(epochs, val_acc, 'b', label='Validation acc')\nplt.title('Training and validation accuracy')\nplt.xlabel('Epochs')\nplt.ylabel('Accuracy')\nplt.legend()\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"39364e4e2d817986ebd0dcb6c3e1464c73d93fa6"},"cell_type":"markdown","source":"\nIn this plot, the dots represent the training loss and accuracy, and the solid lines are the validation loss and accuracy.\n\nNotice the training loss *decreases* with each epoch and the training accuracy *increases* with each epoch. This is expected when using a gradient descent optimization—it should minimize the desired quantity on every iteration.\n\nThis isn't the case for the validation loss and accuracy—they seem to peak after about twenty epochs. This is an example of overfitting: the model performs better on the training data than it does on data it has never seen before. After this point, the model over-optimizes and learns representations *specific* to the training data that do not *generalize* to test data.\n\nFor this particular case, we could prevent overfitting by simply stopping the training after twenty or so epochs. Later, you'll see how to do this automatically with a callback."},{"metadata":{"_uuid":"599e8592df3d123dc3ebaf1b2235d4fa6ae2c98a"},"cell_type":"markdown","source":"## License\n\n##### Copyright 2019 The TensorFlow Authors."},{"metadata":{"trusted":true,"_uuid":"605430b9b0097f0e46b31ca36ec7f5f022f5ce00"},"cell_type":"code","source":"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# https://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"56d85a0a0519a9cb5a4bfdbc171f0a6f126e4e39"},"cell_type":"code","source":"#@title MIT License\n#\n# Copyright (c) 2017 François Chollet\n#\n# Permission is hereby granted, free of charge, to any person obtaining a\n# copy of this software and associated documentation files (the \"Software\"),\n# to deal in the Software without restriction, including without limitation\n# the rights to use, copy, modify, merge, publish, distribute, sublicense,\n# and/or sell copies of the Software, and to permit persons to whom the\n# Software is furnished to do so, subject to the following conditions:\n#\n# The above copyright notice and this permission notice shall be included in\n# all copies or substantial portions of the Software.\n#\n# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL\n# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER\n# DEALINGS IN THE SOFTWARE.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"# Machine Learning (ML) and Deep Learning - Awesome Tutorials\n\n## Contents\n- [Introduction](#general)\n- [Interview Resources](#interview)\n- [Artificial Intelligence](#ai)\n- [Genetic Algorithms](#ga)\n- [Statistics](#stat)\n- [Useful Blogs](#blogs)\n- [Resources on Quora](#quora)\n- [Resources on Kaggle](#kaggle)\n- [Cheat Sheets](#cs)\n- [Classification](#classification)\n- [Linear Regression](#linear)\n- [Logistic Regression](#logistic)\n- [Model Validation using Resampling](#validation)\n - [Cross Validation](#cross)\n - [Bootstraping](#boot)\n- [Deep Learning](#deep)\n - [Frameworks](#frame)\n - [Feed Forward Networks](#feed)\n - [Recurrent Neural Nets, LSTM, GRU](#rnn)\n - [Restricted Boltzmann Machine, DBNs](#rbm)\n - [Autoencoders](#auto)\n - [Convolutional Neural Nets](#cnn)\n- [Natural Language Processing](#nlp)\n - [Topic Modeling, LDA](#topic)\n - [Word2Vec](#word2vec)\n- [Computer Vision](#vision)\n- [Support Vector Machine](#svm)\n- [Reinforcement Learning](#rl)\n- [Decision Trees](#dt)\n- [Random Forest / Bagging](#rf)\n- [Boosting](#gbm)\n- [Ensembles](#ensem)\n- [Stacking Models](#stack)\n- [VC Dimension](#vc)\n- [Bayesian Machine Learning](#bayes)\n- [Semi Supervised Learning](#semi)\n- [Optimizations](#opt)\n\n\n\n## Introduction\n\n- [Machine Learning Course by Andrew Ng (Stanford University)](https://www.coursera.org/learn/machine-learning)\n\n- [In-depth introduction to machine learning in 15 hours of expert videos](http://www.dataschool.io/15-hours-of-expert-machine-learning-videos/)\n\n- [An Introduction to Statistical Learning](http://www-bcf.usc.edu/~gareth/ISL/)\n\n- [List of Machine Learning University Courses](https://github.com/prakhar1989/awesome-courses#machine-learning)\n\n- [Machine Learning for Software Engineers](https://github.com/ZuzooVn/machine-learning-for-software-engineers)\n\n- [Dive into Machine Learning](https://github.com/hangtwenty/dive-into-machine-learning)\n\n- [A curated list of awesome Machine Learning frameworks, libraries and software](https://github.com/josephmisiti/awesome-machine-learning)\n\n- [A curated list of awesome data visualization libraries and resources.](https://github.com/fasouto/awesome-dataviz)\n\n- [An awesome Data Science repository to learn and apply for real world problems](https://github.com/okulbilisim/awesome-datascience)\n\n- [The Open Source Data Science Masters](http://datasciencemasters.org/)\n\n- [Machine Learning FAQs on Cross Validated](http://stats.stackexchange.com/questions/tagged/machine-learning)\n\n- [Machine Learning algorithms that you should always have a strong understanding of](https://www.quora.com/What-are-some-Machine-Learning-algorithms-that-you-should-always-have-a-strong-understanding-of-and-why)\n\n- [Difference between Linearly Independent, Orthogonal, and Uncorrelated Variables](http://terpconnect.umd.edu/~bmomen/BIOM621/LineardepCorrOrthogonal.pdf)\n\n- [List of Machine Learning Concepts](https://en.wikipedia.org/wiki/List_of_machine_learning_concepts)\n\n- [Slides on Several Machine Learning Topics](http://www.slideshare.net/pierluca.lanzi/presentations)\n\n- [MIT Machine Learning Lecture Slides](http://www.ai.mit.edu/courses/6.867-f04/lectures.html)\n\n- [Comparison Supervised Learning Algorithms](http://www.dataschool.io/comparing-supervised-learning-algorithms/)\n\n- [Learning Data Science Fundamentals](http://www.dataschool.io/learning-data-science-fundamentals/)\n\n- [Machine Learning mistakes to avoid](https://medium.com/@nomadic_mind/new-to-machine-learning-avoid-these-three-mistakes-73258b3848a4#.lih061l3l)\n\n- [Statistical Machine Learning Course](http://www.stat.cmu.edu/~larry/=sml/)\n\n- [TheAnalyticsEdge edX Notes and Codes](https://github.com/pedrosan/TheAnalyticsEdge)\n\n- [Have Fun With Machine Learning](https://github.com/humphd/have-fun-with-machine-learning)\n\n- [Twitter's Most Shared #machineLearning Content From The Past 7 Days](http://theherdlocker.com/tweet/popularity/machinelearning)\n\n\n\n## Interview Resources\n\n- [41 Essential Machine Learning Interview Questions (with answers)](https://www.springboard.com/blog/machine-learning-interview-questions/)\n\n- [How can a computer science graduate student prepare himself for data scientist interviews?](https://www.quora.com/How-can-a-computer-science-graduate-student-prepare-himself-for-data-scientist-machine-learning-intern-interviews)\n\n- [How do I learn Machine Learning?](https://www.quora.com/How-do-I-learn-machine-learning-1)\n\n- [FAQs about Data Science Interviews](https://www.quora.com/topic/Data-Science-Interviews/faq)\n\n- [What are the key skills of a data scientist?](https://www.quora.com/What-are-the-key-skills-of-a-data-scientist)\n\n- [The Big List of DS/ML Interview Resources](https://towardsdatascience.com/the-big-list-of-ds-ml-interview-resources-2db4f651bd63)\n\n\n\n## Artificial Intelligence\n\n- [Awesome Artificial Intelligence (GitHub Repo)](https://github.com/owainlewis/awesome-artificial-intelligence)\n\n- [UC Berkeley CS188 Intro to AI](http://ai.berkeley.edu/home.html), [Lecture Videos](http://ai.berkeley.edu/lecture_videos.html), [2](https://www.youtube.com/watch?v=W1S-HSakPTM)\n\n- [MIT 6.034 Artificial Intelligence Lecture Videos](https://www.youtube.com/playlist?list=PLUl4u3cNGP63gFHB6xb-kVBiQHYe_4hSi), [Complete Course](https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/)\n\n- [edX course | Klein & Abbeel](https://courses.edx.org/courses/BerkeleyX/CS188x_1/1T2013/info)\n\n- [Udacity Course | Norvig & Thrun](https://www.udacity.com/course/intro-to-artificial-intelligence--cs271)\n\n- [TED talks on AI](http://www.ted.com/playlists/310/talks_on_artificial_intelligen)\n\n\n\n## Genetic Algorithms\n\n- [Genetic Algorithms Wikipedia Page](https://en.wikipedia.org/wiki/Genetic_algorithm)\n\n- [Simple Implementation of Genetic Algorithms in Python (Part 1)](http://outlace.com/miniga.html), [Part 2](http://outlace.com/miniga_addendum.html)\n\n- [Genetic Algorithms vs Artificial Neural Networks](http://stackoverflow.com/questions/1402370/when-to-use-genetic-algorithms-vs-when-to-use-neural-networks)\n\n- [Genetic Algorithms Explained in Plain English](http://www.ai-junkie.com/ga/intro/gat1.html)\n\n- [Genetic Programming](https://en.wikipedia.org/wiki/Genetic_programming)\n\n - [Genetic Programming in Python (GitHub)](https://github.com/trevorstephens/gplearn)\n \n - [Genetic Alogorithms vs Genetic Programming (Quora)](https://www.quora.com/Whats-the-difference-between-Genetic-Algorithms-and-Genetic-Programming), [StackOverflow](http://stackoverflow.com/questions/3819977/what-are-the-differences-between-genetic-algorithms-and-genetic-programming)\n\n\n\n## Statistics\n\n- [Stat Trek Website](http://stattrek.com/) - A dedicated website to teach yourselves Statistics\n\n- [Learn Statistics Using Python](https://github.com/rouseguy/intro2stats) - Learn Statistics using an application-centric programming approach\n\n- [Statistics for Hackers | Slides | @jakevdp](https://speakerdeck.com/jakevdp/statistics-for-hackers) - Slides by Jake VanderPlas\n\n- [Online Statistics Book](http://onlinestatbook.com/2/index.html) - An Interactive Multimedia Course for Studying Statistics\n\n- [What is a Sampling Distribution?](http://stattrek.com/sampling/sampling-distribution.aspx)\n\n- Tutorials\n\n - [AP Statistics Tutorial](http://stattrek.com/tutorials/ap-statistics-tutorial.aspx)\n \n - [Statistics and Probability Tutorial](http://stattrek.com/tutorials/statistics-tutorial.aspx)\n \n - [Matrix Algebra Tutorial](http://stattrek.com/tutorials/matrix-algebra-tutorial.aspx)\n \n- [What is an Unbiased Estimator?](https://www.physicsforums.com/threads/what-is-an-unbiased-estimator.547728/)\n\n- [Goodness of Fit Explained](https://en.wikipedia.org/wiki/Goodness_of_fit)\n\n- [What are QQ Plots?](http://onlinestatbook.com/2/advanced_graphs/q-q_plots.html)\n\n- [OpenIntro Statistics](https://www.openintro.org/stat/textbook.php?stat_book=os) - Free PDF textbook\n\n\n\n## Useful Blogs\n\n- [Edwin Chen's Blog](http://blog.echen.me/) - A blog about Math, stats, ML, crowdsourcing, data science\n\n- [The Data School Blog](http://www.dataschool.io/) - Data science for beginners!\n\n- [ML Wave](http://mlwave.com/) - A blog for Learning Machine Learning\n\n- [Andrej Karpathy](http://karpathy.github.io/) - A blog about Deep Learning and Data Science in general\n\n- [Colah's Blog](http://colah.github.io/) - Awesome Neural Networks Blog\n\n- [Alex Minnaar's Blog](http://alexminnaar.com/) - A blog about Machine Learning and Software Engineering\n\n- [Statistically Significant](http://andland.github.io/) - Andrew Landgraf's Data Science Blog\n\n- [Simply Statistics](http://simplystatistics.org/) - A blog by three biostatistics professors\n\n- [Yanir Seroussi's Blog](https://yanirseroussi.com/) - A blog about Data Science and beyond\n\n- [fastML](http://fastml.com/) - Machine learning made easy\n\n- [Trevor Stephens Blog](http://trevorstephens.com/) - Trevor Stephens Personal Page\n\n- [no free hunch | kaggle](http://blog.kaggle.com/) - The Kaggle Blog about all things Data Science\n\n- [A Quantitative Journey | outlace](http://outlace.com/) - learning quantitative applications\n\n- [r4stats](http://r4stats.com/) - analyze the world of data science, and to help people learn to use R\n\n- [Variance Explained](http://varianceexplained.org/) - David Robinson's Blog\n\n- [AI Junkie](http://www.ai-junkie.com/) - a blog about Artificial Intellingence\n\n- [Deep Learning Blog by Tim Dettmers](http://timdettmers.com/) - Making deep learning accessible\n\n- [J Alammar's Blog](http://jalammar.github.io/)- Blog posts about Machine Learning and Neural Nets\n\n- [Adam Geitgey](https://medium.com/@ageitgey/machine-learning-is-fun-80ea3ec3c471#.f7vwrtfne) - Easiest Introduction to machine learning\n\n- [Ethen's Notebook Collection](https://github.com/ethen8181/machine-learning) - Continuously updated machine learning documentations (mainly in Python3). Contents include educational implementation of machine learning algorithms from scratch and open-source library usage\n\n\n\n## Resources on Quora\n\n- [Most Viewed Machine Learning writers](https://www.quora.com/topic/Machine-Learning/writers)\n\n- [Data Science Topic on Quora](https://www.quora.com/Data-Science)\n\n- [William Chen's Answers](https://www.quora.com/William-Chen-6/answers)\n\n- [Michael Hochster's Answers](https://www.quora.com/Michael-Hochster/answers)\n\n- [Ricardo Vladimiro's Answers](https://www.quora.com/Ricardo-Vladimiro-1/answers)\n\n- [Storytelling with Statistics](https://datastories.quora.com/)\n\n- [Data Science FAQs on Quora](https://www.quora.com/topic/Data-Science/faq)\n\n- [Machine Learning FAQs on Quora](https://www.quora.com/topic/Machine-Learning/faq)\n\n\n\n## Kaggle Competitions WriteUp\n\n- [How to almost win Kaggle Competitions](https://yanirseroussi.com/2014/08/24/how-to-almost-win-kaggle-competitions/)\n\n- [Convolution Neural Networks for EEG detection](http://blog.kaggle.com/2015/10/05/grasp-and-lift-eeg-detection-winners-interview-3rd-place-team-hedj/)\n\n- [Facebook Recruiting III Explained](http://alexminnaar.com/tag/kaggle-competitions.html)\n\n- [Predicting CTR with Online ML](http://mlwave.com/predicting-click-through-rates-with-online-machine-learning/)\n\n- [How to Rank 10% in Your First Kaggle Competition](https://dnc1994.com/2016/05/rank-10-percent-in-first-kaggle-competition-en/)\n\n\n\n## Cheat Sheets\n\n- [Probability Cheat Sheet](http://static1.squarespace.com/static/54bf3241e4b0f0d81bf7ff36/t/55e9494fe4b011aed10e48e5/1441352015658/probability_cheatsheet.pdf),\n[Source](http://www.wzchen.com/probability-cheatsheet/)\n\n- [Machine Learning Cheat Sheet](https://github.com/soulmachine/machine-learning-cheat-sheet)\n\n\n\n## Classification\n\n- [Does Balancing Classes Improve Classifier Performance?](http://www.win-vector.com/blog/2015/02/does-balancing-classes-improve-classifier-performance/)\n\n- [What is Deviance?](http://stats.stackexchange.com/questions/6581/what-is-deviance-specifically-in-cart-rpart)\n\n- [When to choose which machine learning classifier?](http://stackoverflow.com/questions/2595176/when-to-choose-which-machine-learning-classifier)\n\n- [What are the advantages of different classification algorithms?](https://www.quora.com/What-are-the-advantages-of-different-classification-algorithms)\n\n- [ROC and AUC Explained](http://www.dataschool.io/roc-curves-and-auc-explained/) ([related video](https://youtu.be/OAl6eAyP-yo))\n\n- [An introduction to ROC analysis](https://ccrma.stanford.edu/workshops/mir2009/references/ROCintro.pdf)\n\n- [Simple guide to confusion matrix terminology](http://www.dataschool.io/simple-guide-to-confusion-matrix-terminology/)\n\n\n\n\n## Linear Regression\n\n- [General](#general-)\n\n - [Assumptions of Linear Regression](http://pareonline.net/getvn.asp?n=2&v=8), [Stack Exchange](http://stats.stackexchange.com/questions/16381/what-is-a-complete-list-of-the-usual-assumptions-for-linear-regression)\n \n - [Linear Regression Comprehensive Resource](http://people.duke.edu/~rnau/regintro.htm)\n \n - [Applying and Interpreting Linear Regression](http://www.dataschool.io/applying-and-interpreting-linear-regression/)\n \n - [What does having constant variance in a linear regression model mean?](http://stats.stackexchange.com/questions/52089/what-does-having-constant-variance-in-a-linear-regression-model-mean/52107?stw=2#52107)\n \n - [Difference between linear regression on y with x and x with y](http://stats.stackexchange.com/questions/22718/what-is-the-difference-between-linear-regression-on-y-with-x-and-x-with-y?lq=1)\n \n - [Is linear regression valid when the dependant variable is not normally distributed?](https://www.researchgate.net/post/Is_linear_regression_valid_when_the_outcome_dependant_variable_not_normally_distributed)\n- Multicollinearity and VIF\n\n - [Dummy Variable Trap | Multicollinearity](https://en.wikipedia.org/wiki/Multicollinearity)\n \n - [Dealing with multicollinearity using VIFs](https://jonlefcheck.net/2012/12/28/dealing-with-multicollinearity-using-variance-inflation-factors/)\n\n- [Residual Analysis](#residuals-)\n\n - [Interpreting plot.lm() in R](http://stats.stackexchange.com/questions/58141/interpreting-plot-lm)\n \n - [How to interpret a QQ plot?](http://stats.stackexchange.com/questions/101274/how-to-interpret-a-qq-plot?lq=1)\n \n - [Interpreting Residuals vs Fitted Plot](http://stats.stackexchange.com/questions/76226/interpreting-the-residuals-vs-fitted-values-plot-for-verifying-the-assumptions)\n\n- [Outliers](#outliers-)\n\n - [How should outliers be dealt with?](http://stats.stackexchange.com/questions/175/how-should-outliers-be-dealt-with-in-linear-regression-analysis)\n\n- [Elastic Net](https://en.wikipedia.org/wiki/Elastic_net_regularization)\n - [Regularization and Variable Selection via the\nElastic Net](https://web.stanford.edu/~hastie/Papers/elasticnet.pdf)\n\n\n\n## Logistic Regression\n\n- [Logistic Regression Wiki](https://en.wikipedia.org/wiki/Logistic_regression)\n\n- [Geometric Intuition of Logistic Regression](http://florianhartl.com/logistic-regression-geometric-intuition.html)\n\n- [Obtaining predicted categories (choosing threshold)](http://stats.stackexchange.com/questions/25389/obtaining-predicted-values-y-1-or-0-from-a-logistic-regression-model-fit)\n\n- [Residuals in logistic regression](http://stats.stackexchange.com/questions/1432/what-do-the-residuals-in-a-logistic-regression-mean)\n\n- [Difference between logit and probit models](http://stats.stackexchange.com/questions/20523/difference-between-logit-and-probit-models#30909), [Logistic Regression Wiki](https://en.wikipedia.org/wiki/Logistic_regression), [Probit Model Wiki](https://en.wikipedia.org/wiki/Probit_model)\n\n- [Pseudo R2 for Logistic Regression](http://stats.stackexchange.com/questions/3559/which-pseudo-r2-measure-is-the-one-to-report-for-logistic-regression-cox-s), [How to calculate](http://stats.stackexchange.com/questions/8511/how-to-calculate-pseudo-r2-from-rs-logistic-regression), [Other Details](http://www.ats.ucla.edu/stat/mult_pkg/faq/general/Psuedo_RSquareds.htm)\n\n- [Guide to an in-depth understanding of logistic regression](http://www.dataschool.io/guide-to-logistic-regression/)\n\n\n\n## Model Validation using Resampling\n\n- [Resampling Explained](https://en.wikipedia.org/wiki/Resampling_(statistics))\n\n- [Partioning data set in R](http://stackoverflow.com/questions/13536537/partitioning-data-set-in-r-based-on-multiple-classes-of-observations)\n\n- [Implementing hold-out Validaion in R](http://stackoverflow.com/questions/22972854/how-to-implement-a-hold-out-validation-in-r), [2](http://www.gettinggeneticsdone.com/2011/02/split-data-frame-into-testing-and.html)\n\n\n\n- [Cross Validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics))\n - [How to use cross-validation in predictive modeling](http://stuartlacy.co.uk/2016/02/04/how-to-correctly-use-cross-validation-in-predictive-modelling/)\n - [Training with Full dataset after CV?](http://stats.stackexchange.com/questions/11602/training-with-the-full-dataset-after-cross-validation)\n \n - [Which CV method is best?](http://stats.stackexchange.com/questions/103459/how-do-i-know-which-method-of-cross-validation-is-best)\n \n - [Variance Estimates in k-fold CV](http://stats.stackexchange.com/questions/31190/variance-estimates-in-k-fold-cross-validation)\n \n - [Is CV a subsitute for Validation Set?](http://stats.stackexchange.com/questions/18856/is-cross-validation-a-proper-substitute-for-validation-set)\n \n - [Choice of k in k-fold CV](http://stats.stackexchange.com/questions/27730/choice-of-k-in-k-fold-cross-validation)\n \n - [CV for ensemble learning](http://stats.stackexchange.com/questions/102631/k-fold-cross-validation-of-ensemble-learning)\n \n - [k-fold CV in R](http://stackoverflow.com/questions/22909197/creating-folds-for-k-fold-cv-in-r-using-caret)\n \n - [Good Resources](http://www.chioka.in/tag/cross-validation/)\n \n - Overfitting and Cross Validation\n \n - [Preventing Overfitting the Cross Validation Data | Andrew Ng](http://ai.stanford.edu/~ang/papers/cv-final.pdf)\n \n - [Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation](http://www.jmlr.org/papers/volume11/cawley10a/cawley10a.pdf)\n\n - [CV for detecting and preventing Overfitting](http://www.autonlab.org/tutorials/overfit10.pdf)\n \n - [How does CV overcome the Overfitting Problem](http://stats.stackexchange.com/questions/9053/how-does-cross-validation-overcome-the-overfitting-problem)\n\n\n\n\n- [Bootstrapping](https://en.wikipedia.org/wiki/Bootstrapping_(statistics))\n\n - [Why Bootstrapping Works?](http://stats.stackexchange.com/questions/26088/explaining-to-laypeople-why-bootstrapping-works)\n \n - [Good Animation](https://www.stat.auckland.ac.nz/~wild/BootAnim/)\n \n - [Example of Bootstapping](http://statistics.about.com/od/Applications/a/Example-Of-Bootstrapping.htm)\n \n - [Understanding Bootstapping for Validation and Model Selection](http://stats.stackexchange.com/questions/14516/understanding-bootstrapping-for-validation-and-model-selection?rq=1)\n \n - [Cross Validation vs Bootstrap to estimate prediction error](http://stats.stackexchange.com/questions/18348/differences-between-cross-validation-and-bootstrapping-to-estimate-the-predictio), [Cross-validation vs .632 bootstrapping to evaluate classification performance](http://stats.stackexchange.com/questions/71184/cross-validation-or-bootstrapping-to-evaluate-classification-performance)\n\n\n\n\n## Deep Learning\n\n- [fast.ai - Practical Deep Learning For Coders](http://course.fast.ai/)\n\n- [fast.ai - Cutting Edge Deep Learning For Coders](http://course.fast.ai/part2.html)\n\n- [A curated list of awesome Deep Learning tutorials, projects and communities](https://github.com/ChristosChristofidis/awesome-deep-learning)\n\n- [Lots of Deep Learning Resources](http://deeplearning4j.org/documentation.html)\n\n- [Interesting Deep Learning and NLP Projects (Stanford)](http://cs224d.stanford.edu/reports.html), [Website](http://cs224d.stanford.edu/)\n\n- [Core Concepts of Deep Learning](https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/)\n\n- [Understanding Natural Language with Deep Neural Networks Using Torch](https://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/)\n\n- [Stanford Deep Learning Tutorial](http://ufldl.stanford.edu/tutorial/)\n\n- [Deep Learning FAQs on Quora](https://www.quora.com/topic/Deep-Learning/faq)\n\n- [Google+ Deep Learning Page](https://plus.google.com/communities/112866381580457264725)\n\n- [Recent Reddit AMAs related to Deep Learning](http://deeplearning.net/2014/11/22/recent-reddit-amas-about-deep-learning/), [Another AMA](https://www.reddit.com/r/IAmA/comments/3mdk9v/we_are_google_researchers_working_on_deep/)\n\n- [Where to Learn Deep Learning?](http://www.kdnuggets.com/2014/05/learn-deep-learning-courses-tutorials-overviews.html)\n\n- [Deep Learning nvidia concepts](http://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/)\n\n- [Introduction to Deep Learning Using Python (GitHub)](https://github.com/rouseguy/intro2deeplearning), [Good Introduction Slides](https://speakerdeck.com/bargava/introduction-to-deep-learning)\n\n- [Video Lectures Oxford 2015](https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPoTuPcqmOV53Fu), [Video Lectures Summer School Montreal](http://videolectures.net/deeplearning2015_montreal/)\n\n- [Deep Learning Software List](http://deeplearning.net/software_links/)\n\n- [Hacker's guide to Neural Nets](http://karpathy.github.io/neuralnets/)\n\n- [Top arxiv Deep Learning Papers explained](http://www.kdnuggets.com/2015/10/top-arxiv-deep-learning-papers-explained.html)\n\n- [Geoff Hinton Youtube Vidoes on Deep Learning](https://www.youtube.com/watch?v=IcOMKXAw5VA)\n\n- [Awesome Deep Learning Reading List](http://deeplearning.net/reading-list/)\n\n- [Deep Learning Comprehensive Website](http://deeplearning.net/), [Software](http://deeplearning.net/software_links/)\n\n- [deeplearning Tutorials](http://deeplearning4j.org/)\n\n- [AWESOME! Deep Learning Tutorial](https://www.toptal.com/machine-learning/an-introduction-to-deep-learning-from-perceptrons-to-deep-networks)\n\n- [Deep Learning Basics](http://alexminnaar.com/deep-learning-basics-neural-networks-backpropagation-and-stochastic-gradient-descent.html)\n\n- [Stanford Tutorials](http://ufldl.stanford.edu/tutorial/supervised/MultiLayerNeuralNetworks/)\n\n- [Train, Validation & Test in Artificial Neural Networks](http://stackoverflow.com/questions/2976452/whats-is-the-difference-between-train-validation-and-test-set-in-neural-networ)\n\n- [Artificial Neural Networks Tutorials](http://stackoverflow.com/questions/478947/what-are-some-good-resources-for-learning-about-artificial-neural-networks)\n\n- [Neural Networks FAQs on Stack Overflow](http://stackoverflow.com/questions/tagged/neural-network?sort=votes&pageSize=50)\n\n- [Deep Learning Tutorials on deeplearning.net](http://deeplearning.net/tutorial/index.html)\n\n- [Neural Networks and Deep Learning Online Book](http://neuralnetworksanddeeplearning.com/)\n\n- Neural Machine Translation\n\n - [Introduction to Neural Machine Translation with GPUs (part 1)](https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-with-gpus/), [Part 2](https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-2/), [Part 3](https://devblogs.nvidia.com/parallelforall/introduction-neural-machine-translation-gpus-part-3/)\n \n - [Deep Speech: Accurate Speech Recognition with GPU-Accelerated Deep Learning](https://devblogs.nvidia.com/parallelforall/deep-speech-accurate-speech-recognition-gpu-accelerated-deep-learning/)\n\n\n\n- Deep Learning Frameworks\n\n    - [Torch vs. Theano](http://fastml.com/torch-vs-theano/)\n \n - [dl4j vs. torch7 vs. theano](http://deeplearning4j.org/compare-dl4j-torch7-pylearn.html)\n   - [Deep Learning Libraries by Language](http://www.teglor.com/b/deep-learning-libraries-language-cm569/)\n   - [Theano](https://en.wikipedia.org/wiki/Theano_(software))\n \n - [Website](http://deeplearning.net/software/theano/)\n \n - [Theano Introduction](http://www.wildml.com/2015/09/speeding-up-your-neural-network-with-theano-and-the-gpu/)\n \n - [Theano Tutorial](http://outlace.com/Beginner-Tutorial-Theano/)\n \n - [Good Theano Tutorial](http://deeplearning.net/software/theano/tutorial/)\n \n - [Logistic Regression using Theano for classifying digits](http://deeplearning.net/tutorial/logreg.html#logreg)\n \n - [MLP using Theano](http://deeplearning.net/tutorial/mlp.html#mlp)\n \n - [CNN using Theano](http://deeplearning.net/tutorial/lenet.html#lenet)\n \n - [RNNs using Theano](http://deeplearning.net/tutorial/rnnslu.html#rnnslu)\n \n - [LSTM for Sentiment Analysis in Theano](http://deeplearning.net/tutorial/lstm.html#lstm)\n \n - [RBM using Theano](http://deeplearning.net/tutorial/rbm.html#rbm)\n \n - [DBNs using Theano](http://deeplearning.net/tutorial/DBN.html#dbn)\n \n - [All Codes](https://github.com/lisa-lab/DeepLearningTutorials)\n \n - [Deep Learning Implementation Tutorials - Keras and Lasagne](https://github.com/vict0rsch/deep_learning/)\n\n - [Torch](http://torch.ch/)\n \n - [Torch ML Tutorial](http://code.madbits.com/wiki/doku.php), [Code](https://github.com/torch/tutorials)\n \n - [Intro to Torch](http://ml.informatik.uni-freiburg.de/_media/teaching/ws1415/presentation_dl_lect3.pdf)\n \n - [Learning Torch GitHub Repo](https://github.com/chetannaik/learning_torch)\n \n - [Awesome-Torch (Repository on GitHub)](https://github.com/carpedm20/awesome-torch)\n \n - [Machine Learning using Torch Oxford Univ](https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/), [Code](https://github.com/oxford-cs-ml-2015)\n \n - [Torch Internals Overview](https://apaszke.github.io/torch-internals.html)\n \n - [Torch Cheatsheet](https://github.com/torch/torch7/wiki/Cheatsheet)\n \n - [Understanding Natural Language with Deep Neural Networks Using Torch](http://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/)\n\n - Caffe\n - [Deep Learning for Computer Vision with Caffe and cuDNN](https://devblogs.nvidia.com/parallelforall/deep-learning-computer-vision-caffe-cudnn/)\n\n - TensorFlow\n - [Website](http://tensorflow.org/)\n \n - [TensorFlow Examples for Beginners](https://github.com/aymericdamien/TensorFlow-Examples)\n \n - [Stanford Tensorflow for Deep Learning Research Course](https://web.stanford.edu/class/cs20si/syllabus.html)\n \n - [GitHub Repo](https://github.com/chiphuyen/tf-stanford-tutorials)\n \n - [Simplified Scikit-learn Style Interface to TensorFlow](https://github.com/tensorflow/skflow)\n \n - [Learning TensorFlow GitHub Repo](https://github.com/chetannaik/learning_tensorflow)\n \n - [Benchmark TensorFlow GitHub](https://github.com/soumith/convnet-benchmarks/issues/66)\n \n - [Awesome TensorFlow List](https://github.com/jtoy/awesome-tensorflow)\n \n - [TensorFlow Book](https://github.com/BinRoot/TensorFlow-Book)\n \n - [Android TensorFlow Machine Learning Example](https://blog.mindorks.com/android-tensorflow-machine-learning-example-ff0e9b2654cc)\n \n - [GitHub Repo](https://github.com/MindorksOpenSource/AndroidTensorFlowMachineLearningExample)\n - [Creating Custom Model For Android Using TensorFlow](https://blog.mindorks.com/creating-custom-model-for-android-using-tensorflow-3f963d270bfb)\n - [GitHub Repo](https://github.com/MindorksOpenSource/AndroidTensorFlowMNISTExample) \n\n\n\n- Feed Forward Networks\n\n - [A Quick Introduction to Neural Networks](https://ujjwalkarn.me/2016/08/09/quick-intro-neural-networks/)\n \n - [Implementing a Neural Network from scratch](http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/), [Code](https://github.com/dennybritz/nn-from-scratch)\n \n - [Speeding up your Neural Network with Theano and the gpu](http://www.wildml.com/2015/09/speeding-up-your-neural-network-with-theano-and-the-gpu/), [Code](https://github.com/dennybritz/nn-theano)\n \n - [Basic ANN Theory](https://takinginitiative.wordpress.com/2008/04/03/basic-neural-network-tutorial-theory/)\n \n - [Role of Bias in Neural Networks](http://stackoverflow.com/questions/2480650/role-of-bias-in-neural-networks)\n \n - [Choosing number of hidden layers and nodes](http://stackoverflow.com/questions/3345079/estimating-the-number-of-neurons-and-number-of-layers-of-an-artificial-neural-ne),[2](http://stackoverflow.com/questions/10565868/multi-layer-perceptron-mlp-architecture-criteria-for-choosing-number-of-hidde?lq=1),[3](http://stackoverflow.com/questions/9436209/how-to-choose-number-of-hidden-layers-and-nodes-in-neural-network/2#)\n \n - [Backpropagation in Matrix Form](http://sudeepraja.github.io/Neural/)\n \n - [ANN implemented in C++ | AI Junkie](http://www.ai-junkie.com/ann/evolved/nnt6.html)\n \n - [Simple Implementation](http://stackoverflow.com/questions/15395835/simple-multi-layer-neural-network-implementation)\n \n - [NN for Beginners](http://www.codeproject.com/Articles/16419/AI-Neural-Network-for-beginners-Part-of)\n \n - [Regression and Classification with NNs (Slides)](http://www.autonlab.org/tutorials/neural13.pdf)\n \n - [Another Intro](http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html)\n\n\n\n- Recurrent and LSTM Networks\n - [awesome-rnn: list of resources (GitHub Repo)](https://github.com/kjw0612/awesome-rnn)\n \n - [Recurrent Neural Net Tutorial Part 1](http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-1-introduction-to-rnns/), [Part 2](http://www.wildml.com/2015/09/recurrent-neural-networks-tutorial-part-2-implementing-a-language-model-rnn-with-python-numpy-and-theano/), [Part 3](http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/), [Code](https://github.com/dennybritz/rnn-tutorial-rnnlm/)\n \n - [NLP RNN Representations](http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/)\n \n - [The Unreasonable effectiveness of RNNs](http://karpathy.github.io/2015/05/21/rnn-effectiveness/), [Torch Code](https://github.com/karpathy/char-rnn), [Python Code](https://gist.github.com/karpathy/d4dee566867f8291f086)\n \n - [Intro to RNN](http://deeplearning4j.org/recurrentnetwork.html), [LSTM](http://deeplearning4j.org/lstm.html)\n \n - [An application of RNN](http://hackaday.com/2015/10/15/73-computer-scientists-created-a-neural-net-and-you-wont-believe-what-happened-next/)\n \n - [Optimizing RNN Performance](http://svail.github.io/)\n \n - [Simple RNN](http://outlace.com/Simple-Recurrent-Neural-Network/)\n \n - [Auto-Generating Clickbait with RNN](https://larseidnes.com/2015/10/13/auto-generating-clickbait-with-recurrent-neural-networks/)\n \n - [Sequence Learning using RNN (Slides)](http://www.slideshare.net/indicods/general-sequence-learning-with-recurrent-neural-networks-for-next-ml)\n \n - [Machine Translation using RNN (Paper)](http://emnlp2014.org/papers/pdf/EMNLP2014179.pdf)\n \n - [Music generation using RNNs (Keras)](https://github.com/MattVitelli/GRUV)\n \n - [Using RNN to create on-the-fly dialogue (Keras)](http://neuralniche.com/post/tutorial/)\n \n - Long Short Term Memory (LSTM)\n \n - [Understanding LSTM Networks](http://colah.github.io/posts/2015-08-Understanding-LSTMs/)\n \n - [LSTM explained](https://apaszke.github.io/lstm-explained.html)\n \n - [Beginner’s Guide to LSTM](http://deeplearning4j.org/lstm.html)\n \n - [Implementing LSTM from scratch](http://www.wildml.com/2015/10/recurrent-neural-network-tutorial-part-4-implementing-a-grulstm-rnn-with-python-and-theano/), [Python/Theano code](https://github.com/dennybritz/rnn-tutorial-gru-lstm)\n \n - [Torch Code for character-level language models using LSTM](https://github.com/karpathy/char-rnn)\n \n - [LSTM for Kaggle EEG Detection competition (Torch Code)](https://github.com/apaszke/kaggle-grasp-and-lift)\n \n - [LSTM for Sentiment Analysis in Theano](http://deeplearning.net/tutorial/lstm.html#lstm)\n \n - [Deep Learning for Visual Q&A | LSTM | CNN](http://avisingh599.github.io/deeplearning/visual-qa/), [Code](https://github.com/avisingh599/visual-qa)\n \n - [Computer Responds to email using LSTM | Google](http://googleresearch.blogspot.in/2015/11/computer-respond-to-this-email.html)\n \n - [LSTM dramatically improves Google Voice Search](http://googleresearch.blogspot.ch/2015/09/google-voice-search-faster-and-more.html), [Another Article](http://deeplearning.net/2015/09/30/long-short-term-memory-dramatically-improves-google-voice-etc-now-available-to-a-billion-users/)\n \n - [Understanding Natural Language with LSTM Using Torch](http://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/)\n \n - [Torch code for Visual Question Answering using a CNN+LSTM model](https://github.com/abhshkdz/neural-vqa)\n \n - [LSTM for Human Activity Recognition](https://github.com/guillaume-chevalier/LSTM-Human-Activity-Recognition/)\n \n - Gated Recurrent Units (GRU)\n \n - [LSTM vs GRU](http://www.wildml.com/2015/10/recurrent-neural-network-tutorial-part-4-implementing-a-grulstm-rnn-with-python-and-theano/)\n \n - [Time series forecasting with Sequence-to-Sequence (seq2seq) rnn models](https://github.com/guillaume-chevalier/seq2seq-signal-prediction)\n\n\n\n\n- [Recursive Neural Network (not Recurrent)](https://en.wikipedia.org/wiki/Recursive_neural_network)\n\n - [Recursive Neural Tensor Network (RNTN)](http://deeplearning4j.org/recursiveneuraltensornetwork.html)\n \n - [word2vec, DBN, RNTN for Sentiment Analysis ](http://deeplearning4j.org/zh-sentiment_analysis_word2vec.html)\n\n\n\n- Restricted Boltzmann Machine\n\n - [Beginner's Guide about RBMs](http://deeplearning4j.org/restrictedboltzmannmachine.html)\n \n - [Another Good Tutorial](http://deeplearning.net/tutorial/rbm.html)\n \n - [Introduction to RBMs](http://blog.echen.me/2011/07/18/introduction-to-restricted-boltzmann-machines/)\n \n - [Hinton's Guide to Training RBMs](https://www.cs.toronto.edu/~hinton/absps/guideTR.pdf)\n \n - [RBMs in R](https://github.com/zachmayer/rbm)\n \n - [Deep Belief Networks Tutorial](http://deeplearning4j.org/deepbeliefnetwork.html)\n \n - [word2vec, DBN, RNTN for Sentiment Analysis ](http://deeplearning4j.org/zh-sentiment_analysis_word2vec.html)\n\n\n\n- Autoencoders: Unsupervised (applies BackProp after setting target = input)\n\n - [Andrew Ng Sparse Autoencoders pdf](https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf)\n \n - [Deep Autoencoders Tutorial](http://deeplearning4j.org/deepautoencoder.html)\n \n - [Denoising Autoencoders](http://deeplearning.net/tutorial/dA.html), [Theano Code](http://deeplearning.net/tutorial/code/dA.py)\n \n - [Stacked Denoising Autoencoders](http://deeplearning.net/tutorial/SdA.html#sda)\n\n\n\n\n- Convolutional Neural Networks\n\n - [An Intuitive Explanation of Convolutional Neural Networks](https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/)\n \n - [Awesome Deep Vision: List of Resources (GitHub)](https://github.com/kjw0612/awesome-deep-vision)\n \n - [Intro to CNNs](http://deeplearning4j.org/convolutionalnets.html)\n \n - [Understanding CNN for NLP](http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/)\n \n - [Stanford Notes](http://vision.stanford.edu/teaching/cs231n/), [Codes](http://cs231n.github.io/), [GitHub](https://github.com/cs231n/cs231n.github.io)\n \n - [JavaScript Library (Browser Based) for CNNs](http://cs.stanford.edu/people/karpathy/convnetjs/)\n \n - [Using CNNs to detect facial keypoints](http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/)\n \n - [Deep learning to classify business photos at Yelp](http://engineeringblog.yelp.com/2015/10/how-we-use-deep-learning-to-classify-business-photos-at-yelp.html)\n \n - [Interview with Yann LeCun | Kaggle](http://blog.kaggle.com/2014/12/22/convolutional-nets-and-cifar-10-an-interview-with-yan-lecun/)\n \n - [Visualising and Understanding CNNs](https://www.cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf)\n\n\n\n\n## Natural Language Processing\n\n- [A curated list of speech and natural language processing resources](https://github.com/edobashira/speech-language-processing)\n\n- [Understanding Natural Language with Deep Neural Networks Using Torch](http://devblogs.nvidia.com/parallelforall/understanding-natural-language-deep-neural-networks-using-torch/)\n\n- [tf-idf explained](http://michaelerasm.us/post/tf-idf-in-10-minutes/)\n\n- [Interesting Deep Learning NLP Projects Stanford](http://cs224d.stanford.edu/reports.html), [Website](http://cs224d.stanford.edu/)\n\n- [NLP from Scratch | Google Paper](https://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/35671.pdf)\n\n- [Graph Based Semi Supervised Learning for NLP](http://graph-ssl.wdfiles.com/local--files/blog%3A_start/graph_ssl_acl12_tutorial_slides_final.pdf)\n\n- [Bag of Words](https://en.wikipedia.org/wiki/Bag-of-words_model)\n\n - [Classification text with Bag of Words](http://fastml.com/classifying-text-with-bag-of-words-a-tutorial/)\n \n\n\n- [Topic Modeling](https://en.wikipedia.org/wiki/Topic_model)\n\n - [LDA](https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation), [LSA](https://en.wikipedia.org/wiki/Latent_semantic_analysis), [Probabilistic LSA](https://en.wikipedia.org/wiki/Probabilistic_latent_semantic_analysis)\n \n - [What is a good explanation of Latent Dirichlet Allocation?](https://www.quora.com/What-is-a-good-explanation-of-Latent-Dirichlet-Allocation)\n \n - [Awesome LDA Explanation!](http://blog.echen.me/2011/08/22/introduction-to-latent-dirichlet-allocation/). [Another good explanation](http://confusedlanguagetech.blogspot.in/2012/07/jordan-boyd-graber-and-philip-resnik.html)\n \n - [The LDA Buffet- Intuitive Explanation](http://www.matthewjockers.net/2011/09/29/the-lda-buffet-is-now-open-or-latent-dirichlet-allocation-for-english-majors/)\n \n - [Difference between LSI and LDA](https://www.quora.com/Whats-the-difference-between-Latent-Semantic-Indexing-LSI-and-Latent-Dirichlet-Allocation-LDA)\n \n - [Original LDA Paper](https://www.cs.princeton.edu/~blei/papers/BleiNgJordan2003.pdf)\n \n - [alpha and beta in LDA](http://datascience.stackexchange.com/questions/199/what-does-the-alpha-and-beta-hyperparameters-contribute-to-in-latent-dirichlet-a)\n \n - [Intuitive explanation of the Dirichlet distribution](https://www.quora.com/What-is-an-intuitive-explanation-of-the-Dirichlet-distribution)\n \n - [Topic modeling made just simple enough](https://tedunderwood.com/2012/04/07/topic-modeling-made-just-simple-enough/)\n \n - [Online LDA](http://alexminnaar.com/online-latent-dirichlet-allocation-the-best-option-for-topic-modeling-with-large-data-sets.html), [Online LDA with Spark](http://alexminnaar.com/distributed-online-latent-dirichlet-allocation-with-apache-spark.html)\n \n - [LDA in Scala](http://alexminnaar.com/latent-dirichlet-allocation-in-scala-part-i-the-theory.html), [Part 2](http://alexminnaar.com/latent-dirichlet-allocation-in-scala-part-ii-the-code.html)\n \n - [Segmentation of Twitter Timelines via Topic Modeling](http://alexperrier.github.io/jekyll/update/2015/09/16/segmentation_twitter_timelines_lda_vs_lsa.html)\n \n - [Topic Modeling of Twitter Followers](http://alexperrier.github.io/jekyll/update/2015/09/04/topic-modeling-of-twitter-followers.html)\n\n\n\n- word2vec\n\n - [Google word2vec](https://code.google.com/archive/p/word2vec)\n \n - [Bag of Words Model Wiki](https://en.wikipedia.org/wiki/Bag-of-words_model)\n \n - [word2vec Tutorial](https://rare-technologies.com/word2vec-tutorial/)\n \n - [A closer look at Skip Gram Modeling](http://homepages.inf.ed.ac.uk/ballison/pdf/lrec_skipgrams.pdf)\n \n - [Skip Gram Model Tutorial](http://alexminnaar.com/word2vec-tutorial-part-i-the-skip-gram-model.html), [CBoW Model](http://alexminnaar.com/word2vec-tutorial-part-ii-the-continuous-bag-of-words-model.html)\n \n - [Word Vectors Kaggle Tutorial Python](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-2-word-vectors), [Part 2](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-3-more-fun-with-word-vectors)\n \n - [Making sense of word2vec](http://rare-technologies.com/making-sense-of-word2vec/)\n \n - [word2vec explained on deeplearning4j](http://deeplearning4j.org/word2vec.html)\n \n - [Quora word2vec](https://www.quora.com/How-does-word2vec-work)\n \n - [Other Quora Resources](https://www.quora.com/What-are-the-continuous-bag-of-words-and-skip-gram-architectures-in-laymans-terms), [2](https://www.quora.com/What-is-the-difference-between-the-Bag-of-Words-model-and-the-Continuous-Bag-of-Words-model), [3](https://www.quora.com/Is-skip-gram-negative-sampling-better-than-CBOW-NS-for-word2vec-If-so-why)\n \n - [word2vec, DBN, RNTN for Sentiment Analysis ](http://deeplearning4j.org/zh-sentiment_analysis_word2vec.html)\n\n- Text Clustering\n\n - [How string clustering works](http://stackoverflow.com/questions/8196371/how-clustering-works-especially-string-clustering)\n \n - [Levenshtein distance for measuring the difference between two sequences](https://en.wikipedia.org/wiki/Levenshtein_distance)\n \n - [Text clustering with Levenshtein distances](http://stackoverflow.com/questions/21511801/text-clustering-with-levenshtein-distances)\n\n- Text Classification\n\n - [Classification Text with Bag of Words](http://fastml.com/classifying-text-with-bag-of-words-a-tutorial/)\n\n- [Language learning with NLP and reinforcement learning](http://blog.dennybritz.com/2015/09/11/reimagining-language-learning-with-nlp-and-reinforcement-learning/)\n\n- [Kaggle Tutorial Bag of Words and Word vectors](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-1-for-beginners-bag-of-words), [Part 2](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-2-word-vectors), [Part 3](https://www.kaggle.com/c/word2vec-nlp-tutorial/details/part-3-more-fun-with-word-vectors)\n\n- [What would Shakespeare say (NLP Tutorial)](https://gigadom.wordpress.com/2015/10/02/natural-language-processing-what-would-shakespeare-say/)\n\n- [A closer look at Skip Gram Modeling](http://homepages.inf.ed.ac.uk/ballison/pdf/lrec_skipgrams.pdf)\n\n\n\n## Computer Vision\n- [Awesome computer vision (github)](https://github.com/jbhuang0604/awesome-computer-vision)\n\n- [Awesome deep vision (github)](https://github.com/kjw0612/awesome-deep-vision)\n\n\n\n\n## Support Vector Machine\n\n- [Highest Voted Questions about SVMs on Cross Validated](http://stats.stackexchange.com/questions/tagged/svm)\n\n- [Help me Understand SVMs!](http://stats.stackexchange.com/questions/3947/help-me-understand-support-vector-machines)\n\n- [SVM in Layman's terms](https://www.quora.com/What-does-support-vector-machine-SVM-mean-in-laymans-terms)\n\n- [How does SVM Work | Comparisons](http://stats.stackexchange.com/questions/23391/how-does-a-support-vector-machine-svm-work)\n\n- [A tutorial on SVMs](http://alex.smola.org/papers/2003/SmoSch03b.pdf)\n\n- [Practical Guide to SVC](http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf), [Slides](http://www.csie.ntu.edu.tw/~cjlin/talks/freiburg.pdf)\n\n- [Introductory Overview of SVMs](http://www.statsoft.com/Textbook/Support-Vector-Machines)\n\n- Comparisons\n\n - [SVMs > ANNs](http://stackoverflow.com/questions/6699222/support-vector-machines-better-than-artificial-neural-networks-in-which-learn?rq=1), [ANNs > SVMs](http://stackoverflow.com/questions/11632516/what-are-advantages-of-artificial-neural-networks-over-support-vector-machines), [Another Comparison](http://www.svms.org/anns.html)\n \n - [Trees > SVMs](http://stats.stackexchange.com/questions/57438/why-is-svm-not-so-good-as-decision-tree-on-the-same-data)\n \n - [Kernel Logistic Regression vs SVM](http://stats.stackexchange.com/questions/43996/kernel-logistic-regression-vs-svm)\n \n - [Logistic Regression vs SVM](http://stats.stackexchange.com/questions/58684/regularized-logistic-regression-and-support-vector-machine), [2](http://stats.stackexchange.com/questions/95340/svm-v-s-logistic-regression), [3](https://www.quora.com/Support-Vector-Machines/What-is-the-difference-between-Linear-SVMs-and-Logistic-Regression)\n \n- [Optimization Algorithms in Support Vector Machines](http://pages.cs.wisc.edu/~swright/talks/sjw-complearning.pdf)\n\n- [Variable Importance from SVM](http://stats.stackexchange.com/questions/2179/variable-importance-from-svm)\n\n- Software\n\n - [LIBSVM](https://www.csie.ntu.edu.tw/~cjlin/libsvm/)\n \n - [Intro to SVM in R](http://cbio.ensmp.fr/~jvert/svn/tutorials/practical/svmbasic/svmbasic_notes.pdf)\n \n- Kernels\n - [What are Kernels in ML and SVM?](https://www.quora.com/What-are-Kernels-in-Machine-Learning-and-SVM)\n \n - [Intuition Behind Gaussian Kernel in SVMs?](https://www.quora.com/Support-Vector-Machines/What-is-the-intuition-behind-Gaussian-kernel-in-SVM)\n \n- Probabilities post SVM\n\n - [Platt's Probabilistic Outputs for SVM](http://www.csie.ntu.edu.tw/~htlin/paper/doc/plattprob.pdf)\n \n - [Platt Calibration Wiki](https://en.wikipedia.org/wiki/Platt_scaling)\n \n - [Why use Platts Scaling](http://stats.stackexchange.com/questions/5196/why-use-platts-scaling)\n \n - [Classifier Classification with Platt's Scaling](http://fastml.com/classifier-calibration-with-platts-scaling-and-isotonic-regression/)\n\n\n\n\n## Reinforcement Learning\n\n- [Awesome Reinforcement Learning (GitHub)](https://github.com/aikorea/awesome-rl)\n\n- [RL Tutorial Part 1](http://outlace.com/Reinforcement-Learning-Part-1/), [Part 2](http://outlace.com/Reinforcement-Learning-Part-2/)\n\n\n\n## Decision Trees\n\n- [Wikipedia Page - Lots of Good Info](https://en.wikipedia.org/wiki/Decision_tree_learning)\n\n- [FAQs about Decision Trees](http://stats.stackexchange.com/questions/tagged/cart)\n\n- [Brief Tour of Trees and Forests](http://statistical-research.com/a-brief-tour-of-the-trees-and-forests/)\n\n- [Tree Based Models in R](http://www.statmethods.net/advstats/cart.html)\n\n- [How Decision Trees work?](http://www.aihorizon.com/essays/generalai/decision_trees.htm)\n\n- [Weak side of Decision Trees](http://stats.stackexchange.com/questions/1292/what-is-the-weak-side-of-decision-trees)\n\n- [Thorough Explanation and different algorithms](http://www.ise.bgu.ac.il/faculty/liorr/hbchap9.pdf)\n\n- [What is entropy and information gain in the context of building decision trees?](http://stackoverflow.com/questions/1859554/what-is-entropy-and-information-gain)\n\n- [Slides Related to Decision Trees](http://www.slideshare.net/pierluca.lanzi/machine-learning-and-data-mining-11-decision-trees)\n\n- [How do decision tree learning algorithms deal with missing values?](http://stats.stackexchange.com/questions/96025/how-do-decision-tree-learning-algorithms-deal-with-missing-values-under-the-hoo)\n\n- [Using Surrogates to Improve Datasets with Missing Values](https://www.salford-systems.com/videos/tutorials/tips-and-tricks/using-surrogates-to-improve-datasets-with-missing-values)\n\n- [Good Article](https://www.mindtools.com/dectree.html)\n\n- [Are decision trees almost always binary trees?](http://stats.stackexchange.com/questions/12187/are-decision-trees-almost-always-binary-trees)\n\n- [Pruning Decision Trees](https://en.wikipedia.org/wiki/Pruning_(decision_trees)), [Grafting of Decision Trees](https://en.wikipedia.org/wiki/Grafting_(decision_trees))\n\n- [What is Deviance in context of Decision Trees?](http://stats.stackexchange.com/questions/6581/what-is-deviance-specifically-in-cart-rpart)\n\n- [Discover structure behind data with decision trees](http://vooban.com/en/tips-articles-geek-stuff/discover-structure-behind-data-with-decision-trees/) - Grow and plot a decision tree to automatically figure out hidden rules in your data\n\n- Comparison of Different Algorithms\n\n - [CART vs CTREE](http://stats.stackexchange.com/questions/12140/conditional-inference-trees-vs-traditional-decision-trees)\n \n - [Comparison of complexity or performance](https://stackoverflow.com/questions/9979461/different-decision-tree-algorithms-with-comparison-of-complexity-or-performance)\n \n - [CHAID vs CART](http://stats.stackexchange.com/questions/61230/chaid-vs-crt-or-cart) , [CART vs CHAID](http://www.bzst.com/2006/10/classification-trees-cart-vs-chaid.html)\n \n - [Good Article on comparison](http://www.ftpress.com/articles/article.aspx?p=2248639&seqNum=11)\n \n- CART\n\n - [Recursive Partitioning Wikipedia](https://en.wikipedia.org/wiki/Recursive_partitioning)\n \n - [CART Explained](http://documents.software.dell.com/Statistics/Textbook/Classification-and-Regression-Trees)\n \n - [How to measure/rank “variable importance” when using CART?](http://stats.stackexchange.com/questions/6478/how-to-measure-rank-variable-importance-when-using-cart-specifically-using)\n \n - [Pruning a Tree in R](http://stackoverflow.com/questions/15318409/how-to-prune-a-tree-in-r)\n \n - [Does rpart use multivariate splits by default?](http://stats.stackexchange.com/questions/4356/does-rpart-use-multivariate-splits-by-default)\n \n - [FAQs about Recursive Partitioning](http://stats.stackexchange.com/questions/tagged/rpart)\n \n- CTREE\n\n - [party package in R](https://cran.r-project.org/web/packages/party/party.pdf)\n \n - [Show volumne in each node using ctree in R](http://stackoverflow.com/questions/13772715/show-volume-in-each-node-using-ctree-plot-in-r)\n \n - [How to extract tree structure from ctree function?](http://stackoverflow.com/questions/8675664/how-to-extract-tree-structure-from-ctree-function)\n \n- CHAID\n\n - [Wikipedia Artice on CHAID](https://en.wikipedia.org/wiki/CHAID)\n \n - [Basic Introduction to CHAID](https://smartdrill.com/Introduction-to-CHAID.html)\n \n - [Good Tutorial on CHAID](http://www.statsoft.com/Textbook/CHAID-Analysis)\n \n- MARS\n\n - [Wikipedia Article on MARS](https://en.wikipedia.org/wiki/Multivariate_adaptive_regression_splines)\n \n- Probabilistic Decision Trees\n\n - [Bayesian Learning in Probabilistic Decision Trees](http://www.stats.org.uk/bayesian/Jordan.pdf)\n \n - [Probabilistic Trees Research Paper](http://people.stern.nyu.edu/adamodar/pdfiles/papers/probabilistic.pdf)\n\n\n\n## Random Forest / Bagging\n\n- [Awesome Random Forest (GitHub)**](https://github.com/kjw0612/awesome-random-forest)\n\n- [How to tune RF parameters in practice?](https://www.kaggle.com/forums/f/15/kaggle-forum/t/4092/how-to-tune-rf-parameters-in-practice)\n\n- [Measures of variable importance in random forests](http://stats.stackexchange.com/questions/12605/measures-of-variable-importance-in-random-forests)\n\n- [Compare R-squared from two different Random Forest models](http://stats.stackexchange.com/questions/13869/compare-r-squared-from-two-different-random-forest-models)\n\n- [OOB Estimate Explained | RF vs LDA](https://stat.ethz.ch/education/semesters/ss2012/ams/slides/v10.2.pdf)\n\n- [Evaluating Random Forests for Survival Analysis Using Prediction Error Curve](https://www.jstatsoft.org/index.php/jss/article/view/v050i11)\n\n- [Why doesn't Random Forest handle missing values in predictors?](http://stats.stackexchange.com/questions/98953/why-doesnt-random-forest-handle-missing-values-in-predictors)\n\n- [How to build random forests in R with missing (NA) values?](http://stackoverflow.com/questions/8370455/how-to-build-random-forests-in-r-with-missing-na-values)\n\n- [FAQs about Random Forest](http://stats.stackexchange.com/questions/tagged/random-forest), [More FAQs](http://stackoverflow.com/questions/tagged/random-forest)\n\n- [Obtaining knowledge from a random forest](http://stats.stackexchange.com/questions/21152/obtaining-knowledge-from-a-random-forest)\n\n- [Some Questions for R implementation](http://stackoverflow.com/questions/20537186/getting-predictions-after-rfimpute), [2](http://stats.stackexchange.com/questions/81609/whether-preprocessing-is-needed-before-prediction-using-finalmodel-of-randomfore), [3](http://stackoverflow.com/questions/17059432/random-forest-package-in-r-shows-error-during-prediction-if-there-are-new-fact)\n\n\n\n## Boosting\n\n- [Boosting for Better Predictions](http://www.datasciencecentral.com/profiles/blogs/boosting-algorithms-for-better-predictions)\n\n- [Boosting Wikipedia Page](https://en.wikipedia.org/wiki/Boosting_(machine_learning))\n\n- [Introduction to Boosted Trees | Tianqi Chen](https://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf)\n\n- Gradient Boosting Machine\n\n - [Gradiet Boosting Wiki](https://en.wikipedia.org/wiki/Gradient_boosting)\n \n - [Guidelines for GBM parameters in R](http://stats.stackexchange.com/questions/25748/what-are-some-useful-guidelines-for-gbm-parameters), [Strategy to set parameters](http://stats.stackexchange.com/questions/35984/strategy-to-set-the-gbm-parameters)\n \n - [Meaning of Interaction Depth](http://stats.stackexchange.com/questions/16501/what-does-interaction-depth-mean-in-gbm), [2](http://stats.stackexchange.com/questions/16501/what-does-interaction-depth-mean-in-gbm)\n \n - [Role of n.minobsinnode parameter of GBM in R](http://stats.stackexchange.com/questions/30645/role-of-n-minobsinnode-parameter-of-gbm-in-r)\n \n - [GBM in R](http://www.slideshare.net/mark_landry/gbm-package-in-r)\n \n - [FAQs about GBM](http://stats.stackexchange.com/tags/gbm/hot)\n \n - [GBM vs xgboost](https://www.kaggle.com/c/higgs-boson/forums/t/9497/r-s-gbm-vs-python-s-xgboost)\n\n- xgboost\n\n - [xgboost tuning kaggle](https://www.kaggle.com/khozzy/rossmann-store-sales/xgboost-parameter-tuning-template/log)\n \n - [xgboost vs gbm](https://www.kaggle.com/c/otto-group-product-classification-challenge/forums/t/13012/question-to-experienced-kagglers-and-anyone-who-wants-to-take-a-shot/68296#post68296)\n \n - [xgboost survey](https://www.kaggle.com/c/higgs-boson/forums/t/10335/xgboost-post-competition-survey)\n \n - [Practical XGBoost in Python online course (free)](http://education.parrotprediction.teachable.com/courses/practical-xgboost-in-python)\n \n- AdaBoost\n\n - [AdaBoost Wiki](https://en.wikipedia.org/wiki/AdaBoost), [Python Code](https://gist.github.com/tristanwietsma/5486024)\n \n - [AdaBoost Sparse Input Support](http://hamzehal.blogspot.com/2014/06/adaboost-sparse-input-support.html)\n \n - [adaBag R package](https://cran.r-project.org/web/packages/adabag/adabag.pdf)\n \n - [Tutorial](http://math.mit.edu/~rothvoss/18.304.3PM/Presentations/1-Eric-Boosting304FinalRpdf.pdf)\n\n\n\n## Ensembles\n\n- [Wikipedia Article on Ensemble Learning](https://en.wikipedia.org/wiki/Ensemble_learning)\n\n- [Kaggle Ensembling Guide](http://mlwave.com/kaggle-ensembling-guide/)\n\n- [The Power of Simple Ensembles](http://www.overkillanalytics.net/more-is-always-better-the-power-of-simple-ensembles/)\n\n- [Ensemble Learning Intro](http://machine-learning.martinsewell.com/ensembles/)\n\n- [Ensemble Learning Paper](http://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/springerEBR09.pdf)\n\n- [Ensembling models with R](http://amunategui.github.io/blending-models/), [Ensembling Regression Models in R](http://stats.stackexchange.com/questions/26790/ensembling-regression-models), [Intro to Ensembles in R](http://www.vikparuchuri.com/blog/intro-to-ensemble-learning-in-r/)\n\n- [Ensembling Models with caret](http://stats.stackexchange.com/questions/27361/stacking-ensembling-models-with-caret)\n\n- [Bagging vs Boosting vs Stacking](http://stats.stackexchange.com/questions/18891/bagging-boosting-and-stacking-in-machine-learning)\n\n- [Good Resources | Kaggle Africa Soil Property Prediction](https://www.kaggle.com/c/afsis-soil-properties/forums/t/10391/best-ensemble-references)\n\n- [Boosting vs Bagging](http://www.chioka.in/which-is-better-boosting-or-bagging/)\n\n- [Resources for learning how to implement ensemble methods](http://stats.stackexchange.com/questions/32703/resources-for-learning-how-to-implement-ensemble-methods)\n\n- [How are classifications merged in an ensemble classifier?](http://stats.stackexchange.com/questions/21502/how-are-classifications-merged-in-an-ensemble-classifier)\n\n\n\n## Stacking Models\n\n- [Stacking, Blending and Stacked Generalization](http://www.chioka.in/stacking-blending-and-stacked-generalization/)\n\n- [Stacked Generalization (Stacking)](http://machine-learning.martinsewell.com/ensembles/stacking/)\n\n- [Stacked Generalization: when does it work?](http://www.ijcai.org/Proceedings/97-2/011.pdf)\n\n- [Stacked Generalization Paper](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.56.1533&rep=rep1&type=pdf)\n\n\n\n## Vapnik–Chervonenkis Dimension\n\n- [Wikipedia article on VC Dimension](https://en.wikipedia.org/wiki/VC_dimension)\n\n- [Intuitive Explanantion of VC Dimension](https://www.quora.com/Explain-VC-dimension-and-shattering-in-lucid-Way)\n\n- [Video explaining VC Dimension](https://www.youtube.com/watch?v=puDzy2XmR5c)\n\n- [Introduction to VC Dimension](http://www.svms.org/vc-dimension/)\n\n- [FAQs about VC Dimension](http://stats.stackexchange.com/questions/tagged/vc-dimension)\n\n- [Do ensemble techniques increase VC-dimension?](http://stats.stackexchange.com/questions/78076/do-ensemble-techniques-increase-vc-dimension)\n\n\n\n\n## Bayesian Machine Learning\n\n- [Bayesian Methods for Hackers (using pyMC)](https://github.com/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers)\n\n- [Should all Machine Learning be Bayesian?](http://videolectures.net/bark08_ghahramani_samlbb/)\n\n- [Tutorial on Bayesian Optimisation for Machine Learning](http://www.iro.umontreal.ca/~bengioy/cifar/NCAP2014-summerschool/slides/Ryan_adams_140814_bayesopt_ncap.pdf)\n\n- [Bayesian Reasoning and Deep Learning](http://blog.shakirm.com/2015/10/bayesian-reasoning-and-deep-learning/), [Slides](http://blog.shakirm.com/wp-content/uploads/2015/10/Bayes_Deep.pdf)\n\n- [Bayesian Statistics Made Simple](http://greenteapress.com/wp/think-bayes/)\n\n- [Kalman & Bayesian Filters in Python](https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python)\n\n- [Markov Chain Wikipedia Page](https://en.wikipedia.org/wiki/Markov_chain)\n\n\n\n\n## Semi Supervised Learning\n\n- [Wikipedia article on Semi Supervised Learning](https://en.wikipedia.org/wiki/Semi-supervised_learning)\n\n- [Tutorial on Semi Supervised Learning](http://pages.cs.wisc.edu/~jerryzhu/pub/sslicml07.pdf)\n\n- [Graph Based Semi Supervised Learning for NLP](http://graph-ssl.wdfiles.com/local--files/blog%3A_start/graph_ssl_acl12_tutorial_slides_final.pdf)\n\n- [Taxonomy](http://is.tuebingen.mpg.de/fileadmin/user_upload/files/publications/taxo_[0].pdf)\n\n- [Video Tutorial Weka](https://www.youtube.com/watch?v=sWxcIjZFGNM)\n\n- [Unsupervised, Supervised and Semi Supervised learning](http://stats.stackexchange.com/questions/517/unsupervised-supervised-and-semi-supervised-learning)\n\n- [Research Papers 1](http://mlg.eng.cam.ac.uk/zoubin/papers/zglactive.pdf), [2](http://mlg.eng.cam.ac.uk/zoubin/papers/zgl.pdf), [3](http://icml.cc/2012/papers/616.pdf)\n\n\n\n\n## Optimization\n\n- [Mean Variance Portfolio Optimization with R and Quadratic Programming](http://www.wdiam.com/2012/06/10/mean-variance-portfolio-optimization-with-r-and-quadratic-programming/?utm_content=buffer04c12&utm_medium=social&utm_source=linkedin.com&utm_campaign=buffer)\n\n- [Algorithms for Sparse Optimization and Machine Learning](http://www.ima.umn.edu/2011-2012/W3.26-30.12/activities/Wright-Steve/sjw-ima12)\n\n- [Optimization Algorithms in Machine Learning](http://pages.cs.wisc.edu/~swright/nips2010/sjw-nips10.pdf), [Video Lecture](http://videolectures.net/nips2010_wright_oaml/)\n\n- [Optimization Algorithms for Data Analysis](http://www.birs.ca/workshops/2011/11w2035/files/Wright.pdf)\n\n- [Video Lectures on Optimization](http://videolectures.net/stephen_j_wright/)\n\n- [Optimization Algorithms in Support Vector Machines](http://pages.cs.wisc.edu/~swright/talks/sjw-complearning.pdf)\n\n- [The Interplay of Optimization and Machine Learning Research](http://jmlr.org/papers/volume7/MLOPT-intro06a/MLOPT-intro06a.pdf)\n\n- [Hyperopt tutorial for Optimizing Neural Networks’ Hyperparameters](http://vooban.com/en/tips-articles-geek-stuff/hyperopt-tutorial-for-optimizing-neural-networks-hyperparameters/)\n\n\n\n\n"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","collapsed":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"markdown","source":"## Credits (Reference)\n\n> * [Text Classification](https://www.tensorflow.org/tutorials/keras/basic_text_classification)\n> * [GitHub Deep Learning Topic](https://github.com/topics/deep-learning)\n> * [Ujjwal Karn](https://github.com/ujjwalkarn/Machine-Learning-Tutorials)\n\n## License\n\n[![CC0](http://mirrors.creativecommons.org/presskit/buttons/88x31/svg/cc-zero.svg)](https://creativecommons.org/publicdomain/zero/1.0/)\n\n### Please ***UPVOTE*** my kernel if you like it or wanna fork it.\n\n##### Feedback: If you have any ideas or you want any other content to be added to this curated list, please feel free to make any comments to make it better.\n#### I am open to have your *feedback* for improving this ***kernel***\n###### Hope you enjoyed this kernel!\n\n### Thanks for visiting my *Kernel* and please *UPVOTE* to stay connected and follow up the *further updates!*"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}