Beginner's Guide: Some Useful Machine Learning Websites(Part 1)

I have been away for quite some time now. The reason is I was quite busy with my academic thesis and general academic pressure. Now I do feel I can regularly post in my blog.

So today's topic that I choose is Machine Learning. I have done my thesis on neural networks. There were also 2 optional courses( Machine Learning and Pattern Recognition) I undertook for better understanding machine learning as a whole. Machine learning is a hot topic of research these days. All the top varsities are doing extensive studies and research on various aspects of machine learning. Some are exploring its application in real life situations while others are delving deep into the mechanisms of the algorithms.

When I first started doing my undergraduate thesis, I hardly knew anything about machine learning. As a result I had to search through internet for websites that would easily and extensively explain the basic mechanisms of machine learning, specifically neural networks.

I found 2 sources quite helpful in understanding how machine learning works and how to implement them into python code. I was able to read research papers with greater understanding after going through the contents of these 2 websites. It also helped me in my academic studies. I will describe first one for today's post.

Adventures In Machine Learning:

I think before mindlessly coding for machine learning models, one must learn the theory to understand what one is actually coding. Adventures in machine learning is one of the best sources out there to introduce newbies to the vast world of machine learning. The articles also come with basic implementation of the theories. It helps greatly with both academic courses and thesis activities.

Some of the articles I had to study extensively are given below. They helped me build a strong basis in learning about machine learning for both my academic course and my thesis.

• Neural Networks Tutorial – A Pathway to Deep Learning: This is the first article of this source I studied. The articles are rather large and does require quite a bit of your time and effort. For the first timers this may even seem overwhelming. So I advise you bookmark the article and read it part by part. Trying to read the whole article at once will only hurt your brain. You may feel frustrated and at the end of reading the whole article at one go you may not even learn anything. Too much information will just jumble up everything. That's why you need to read it part by part taking your time. And do read it multiple times for clear understanding. Reading it only once may not convey much information.
  So this article deals with the basics of neural networks and an example step by step implementation in python. It describes at length both the feed forward and back propagation methods with necessary theories to help with your academic studies. I found the back-propagation portion quite useful because it clarified many confusion about back-propagation algorithm when studied the academic materials. The python implementation also clears ones understanding of implementation and will help you code your first neural network model with comfort. After reading this article, if you sit to code a neural network model, you will know why you're coding, what you're coding. You won't be just blindly copy pasting stuff from github or any other sources :P
  

  

  Neural Network with 1 hidden layer, 2 output and 3 input nodes
  

  
  

• Stochastic Gradient Descent: This article talks about 3 types gradient descent algorithms at length. The discussion includes their characteristics and  their advantages/disadvantages. A detailed python implementation is provided to understand the algorithms step by step. These greatly helped clear many confusions I had about this seemingly difficult algorithm. I suggest you read the first article mentioned above before jumping into this article. The first article gives you an expressive idea about neural networks and this one discusses an algorithm to improve the performance of neural network.
• NOTE: So now you may feel exhausted reading 2 previous VERY big articles. It may take some time to both finish reading them and understanding the concepts of neural networks. But I assure you it's worth it. In academia, you always have to spend time and effort on such large and detailed articles. You may want to take some time off before moving on to other articles so that the knowledge you learned here will sit with you in future. You may also want to revise them( I suggest you do) to keep the knowledge fresh in memory.
• Improve Neural Networks: So let's dive in further into the vast world of neural networks. For beginner models you will see high performance BUT as you start learning more complex models, accuracy becomes a BIG issue. You then feel frustrated because the performance isn't improving. In most cases the reason behind low accuracy of the model for beginners can be attributed to lack of implementing some basic concepts. 2 such concepts have been included in this articles. Their definition, description and expressive implementation in python have been provided for our convenience. It also discusses about the process of choosing better variables to improve neural networks along with a step by step implementation. This article is shorter than the other 2. So hopefully, reading it won't exhaust you :p
• Neural Networks and TensorFlow: If you read the previous 3 articles and understand them, you get a pretty good idea about various aspects of neural networks. Till now the codes are given in python without using any special library for neural network so that the readers can view the step by step implementation of neural network for better understanding. However, if you want to use neural networks in projects, assignments and thesis, coding in simple python is quite time consuming and as the size of the code grows due to the increasing complexity of the model, the chances of making mistakes increase dramatically. Hence special python libraries for using neural networks are used. One such example is TensorFlow. This article is a general, detailed and easy-to-follow guideline of using TensorFlow to implement neural nets. The number of codes becomes much smaller and easier. So this article is a must, if you wish to code neural nets for real life projects/ assignments.
• Convolutional Neural Network in TensorFlow: The previous neural networks you learned do not perform well when you're using large 2D/3D colored image data-sets. In such a case CNN is a must. This article talks about the basic theory about CNN at length. This helped me greatly when I had to study about CNN for both my thesis and academic course. Then comes step by step detailed implementation in TensorFlow. So this article is like  a full package to know all the important things about CNN. Since this article is quite long and there are many new things to learn, I suggest divide the article and read part by part taking time and effort to better understand. Do revise the article so that you don't forget the mechanism of this wonderful neural network.
• CNN in Keras: You have learned TensorFlow and CNN. If you understand them, this article would be a piece of cake. This is the last article from this website, I'm going to talk about. Now you may think why I should recommend you this article since this article is about implementing CNN using another python library. You may think, "isn't TensorFlow enough to implement neural networks?" My answer to your thought is: "Yes." BUT Keras implements neural networks using fewer code than TensorFlow. This come in handy when you have to make many NN models and work with them. In such a case, you may find working with TensorFlow rather annoying. In my thesis, I have to use NN for many datasets. I found Keras truly convenient. Then you may ask 'why learn TensorFlow at all?' My answer would be:'The course teacher may want you to implement something in TensorFlow because Keras might be too short and your teacher may not allow it.' In many of my academic courses, I had to implement code in C++ even though I could have easily used Java/Python. The same goes for using TensorFlow instead of Keras. Other than that you can simply jump to learning Keras. Also Keras has given code to include a diverse array of dataset in your code but in TensorFlow you have to include dataset using your own written code.

That's it for today. In the next article (Part 2) I'm going to review another of my favorite online resource regarding neural networks.