When learning a new concept in this highly technical field, I believe that building things from scratch not only strengthens your programming skills but also allows you to get a bottom-up and fundamental understanding of how the algorithm actually works. One thing to remember is that in Machine Learning, everything we do is inherently mathematical. If you do not understand the mathematics behind the algorithm, you will not be able to efficiently deliver the key insights of the results to those who are non-technical — who, might I add, you deal with just as much as those who are technical.
While this should be common sense, let me raise a little disclaimer from now: you building the algorithm from scratch should not replace highly optimized libraries to do the specific task. Rather, you building out the algorithm should act as a complement to learning the mathematics and seeing it solve in real-time, step by step. If you have never built a neural network before, you will likely not be able to understand the underlying mechanisms of the API calls within the libraries of PyTorch or TensorFlow.
I truly believe that if you are learning a new algorithm, learning about how it works is a great first start, but learning how to build it yourself really allows you to get lost in the beauty of why it works, which in my opinion, is where most of the fun can be found in this field.