DISCLAIMER: I am still an amateur in this field and have a lot to accomplish before deeming myself a professional. If there are errors in my understanding of concepts, I would be grateful to be corrected through your suggestions and inputs. Moreover, constructive criticism is always appreciated. Thank you 🙂
From the preliminary days of introduction to computers, the concept of designing an algorithm that can efficiently identify patterns in data, like humans but with great accuracy, efficiency and complexity, tingled my interest. However, this interest evolved into a genuine involvement and a potential career path when I was required to look into the core of this topic during my presidential term at the maximum demanding Science and Information Technology Club.
As the president, I undertook an oath and responsibility to answer the queries posted by the fellows of my club to the best of my ability. This meant that the least I could contribute to the growth of my fellows was to be up-to-date on the current affairs in the field of our interest. And, since artificial intelligence and machine learning was the talk of the town amongst the students who involved themselves in the field of technical science, it would have been negligent to have no knowledge on the know-hows of the mathematics and computations that guided machine learning.
This is where I started my journey to machine learning, specifically neural network development.
Having a clear understanding on the area of use of any concept provides us with an outlook of the potential future that it may take, and a thorough pass through the application of machine learning certainly assisted me in accomplishing that. Additionally, it also provided me with a tool to devise a potential future plan that I could involve myself into after having conquered the core understanding of the subject.
Using these limited information, I held my first open-discussion seminar as the president in order to get other interested individuals involved in the journey alongside me.
OPEN DISCUSSION SEMINAR I
POSTER PRESENTATION, DISCUSSION AND PLAN FORMULATION
This seminar was the initial open-discussion podium for all of the individuals caught in the hype of machine learning to start looking at its depth. Since there had not been any genuine participation in this field within my institution till date, I took the assistance of a poster to help present all of my learnings and findings of the applications to the attendees.
Simultaneously, I publicly announced my commitment to involve myself in machine learning and invited others to do the same. Doing so, made sure that us interested individuals are motivated to work towards the end-goal without slacking off on the mathematical and the computational part of it.
Essentially, if one is open about their involvement in learning, they are likely to be questioned about the topic and the core concepts surrounding it. Hence, the individual becomes more likely to not skip parts of learning that could have been overlooked if the involvement was kept secretive.
Thus, I exploited this emotion of us as human beings for the benefit of us all.
After ending the session for non-interested members, seven of us committed individuals shifted our discussion to the procedures that we needed to follow for our journey ahead. The discussion primarily constituted of the initial plan:
- Should we start by reading a book?
- Should we jump into a TensorFlow tutorial available in YouTube and learn the core concept from YouTubers as we go?
- Should we look into various courses devised for beginners?
Eventually we settled for the beginners route.
We planned to start with a Machine Learning course in Coursera by Andrew Ng. It required us to code using Octave or MATLAB. But we as beginners with limited time at hand would want to know the core concept and not waste our time in learning a new language altogether. So, we moved on with our search for courses in Machine Learning that promoted the use of python, a widely popular language at present. This is when we discovered the five part course on Deep Neural Network Learning by the same tutor.
We finalized that it would not be possible for us to move through it at the same rate, as we have different learning rates similar to that of different models. Hence, we continued at our own pace and decided to have an end of year session to reflect on our learnings.
JOURNEY FROM THEN
After a few weeks, some students dropped their courses, some moved out of the school, some changed their major and left the club, and some shifted their focus to a different topic in computer science. Essentially, only two of us were headed towards the deep end of deep learning.
Personally, the first few weeks of the course were extremely mathematical, confusing and challenging for me (someone with only the beginners’ level understanding of calculus). However, through continuous learning from external sources and rewinding of the course, I was able to understand the mathematics that guided the forward propagation and backward propagation of a neural network.
Initially, the activation functions confused me; and their usage whilst in application was still unknown to me. But as weeks passed by, I was given a notebook within the course which I could utilize to procedurally learn the concepts as I go, without having to write all the components of a program by myself.
Even though most of the codes and mathematics behind the functions utilized for the training had been in place for the majority of the course materials, the euphoric feeling within me of training a model and displaying the results in real time was unmatched.
After the completion of part one, I was completely involved in the mathematics of the topic, and by then, programming of individual elements came easy to me. Furthermore, the calculus I had been practicing, alongside, was starting to have an impact on my ability to use various activation functions as I wished.
Confidently, I embarked in creating my very first custom neural network using Kaggle dataset.
This was not easy. The activation function I chose didn’t work properly. The cost functions in each epoch were fluctuating left right and center. And the final curve for the cost function was all over the place.
Essentially, I was introduced to the brutality of the real life experience [using demonstration purpose data from Kaggle] on the very first attempt of training a model. This strengthened my inner voice that echoed, “Complex Neural Network Creation is not going to be a piece of cake.”
However, I strived to complete it unaware of the hindrances that awaited me in the future. The processing capability of my computer completely reduced the rate of my progress to null. I had to halt my progress for 3 months since my device did not agree to work with any notebook. All in all, I had to wait till the end of the term to clear all my data from the laptop and have a new OS installed.
This delayed my progress by a few months. However as soon as I had a fairly new [feeling] laptop I resumed my progress from where I left. It was helpful that I was able to refer back to the course materials, and other notebooks posted by fellow Kaggle mates to have my first custom neural network running, with 80% training accuracy and 75% testing accuracy.
In this period of technological advancement, where there are individuals involved in the field of machine learning with higher mental capability and physical processing ability than that of me and my computer, it came as no surprise that my custom neural network was nowhere near comparable to them. But, this did not dishearten me, the progress I was able to make, given such a short time period of active involvement, I was fully satisfied with the first neural network I had in front of me.
OPEN DISCUSSION SESSION II
INFORMAL POST-PRESIDENTIAL TERM DISCUSSION
Since our progress was not smooth, the discussion between the final two of us happened after my presidential term ended. By this time, we were already starting our first semester term at our respective universities. Given the distance and geographical complexities of our new study place, meeting physically was not possible. Hence, we resorted to an informal video call to present our learnings and the custom neural network we were able to prepare. The discussion was very open. Having only two participants, we were honest on our failures and struggles. And our experience facilitated us to give wise recommendations to each other.
I was fundamentally suggested to look into feature engineering which I was completely unaware till then. With this informal exchange, we concluded our commitment [promises] to each other, wished the best for our journey ahead in this field, and parted ways.
FROM THEN TILL DATE
It has been long since the final informal meeting, and my last involvement in creating a neural network. The first semester in a new environment of the university was hectic. I, alongside my university mates, were involved in the physical building of engineering samples, required for the course completion. Hence, most of our project hours were spent on these physical meetings with the teacher and labor in the workshop house within the university. The adaptation difficulty paired with time restrictions, completely disengaged me from this field of unlimited possibilities that I was excited about.
However, recently, I have regained the pace and confidence.
It has been about a month since I prepared a neural network on another Kaggle data set with fewer neurons but higher hidden layers, and managed to achieve 92% training accuracy and 90% testing accuracy. This boosted my confidence on my ability to retain information, and simultaneously, improve the numbers.
Hence, currently I am involved in feature engineering using the Titanic Competition Data Set as suggested by my companion, and have been studying papers prepared on various regularization techniques: Dropout Regularization and Batch Normalization, to start the second part of the Deep Neural Network Course On Hyperparameter Tuning, Regularization and Optimization.