GANs and other generative machine learning algorithms are still hyped and work fantastically with images, texts, and sounds. They’re able not only to generate data for fun but solve important theoretical issues and boost production ML pipelines. Unfortunately, typical today’s practical use case is limited to “fine-tune pre-trained StyleGAN2 for zombies generation”. And what’s worse, almost no one cares to explain why do we need generative modeling in the real world and where the roots of such need are coming from.
The following two articles aim to bridge the gap between modern cool stuff and slightly forgotten old-school mathematical modeling, where there were no big datasets and powerful neural networks. After grasping this material, you will be able to:
- Understand why generative machine learning is a more powerful data modeling paradigm that was with us for centuries
- Learn and implement every modern generative model faster, knowing the scientific fundamentals and needs for such a process
- Formulate novel scenarios for generative modeling and get additional value for the R&D processes or to the final product
As always, the source code for all the experiments, you can find on my Github.
Let’s remember physics classes. The very first topics you were taught were force, motion, time, velocity, acceleration in the simple 1D cases. Even before learning differentiation or integration in calculus classes, you could easily calculate the velocity of the object based on time and related positions. The same strategy could be applied to more complicated mechanics as spring force, pendulum, multi-dimensional mechanics, etc. You only substitute formulas for the appropriate ones and re-do the calculus routines. The simplified process of such modeling would be the following:
- Identify what object is moving, what are the positions, the origin of the coordinate system, initial conditions
- Find the appropriate model that describes the behavior of the given object under the given conditions
- Solve the equation of the model with the given conditions and find velocity, acceleration, or another variable
- Analyze the solution and its validity
For example, in the case of a pendulum system (on illustrations above), you can define the object dynamics model as an equilibrium between its kinetic and potential energy in a Lagrangian, and if you solve it for a single degree of freedom (the angle of the oscillation), you get the equation of the motion trajectory, which you can solve for different conditions.
And now the coolest part:
The same as we sample faces, cats, and songs with GANs, we could sample complex motions of physical objects movements just via solving equations. For centuries. It literally got us to the moon. Without gigabytes of data and GPUs for deep neural nets.
I bet that they didn’t teach you this angle in the physics class. If you have the exact formula for this pendulum, you have your “pendulum-GAN”: you just need to sample length, gravity, amplitude, etc and insert them into the formula: this way you can generate as many pendulums as you want. The only difference that GAN has some quasi-random vector as the input and the formula is a black-box neural net trained with data. Below are a couple of illustrations of sampled trajectories and the code you can find here.