The model isn’t doing so hot.
But, if we give it something to work off of with the same method we did before, then the algorithm starts to get it!
It’s not perfect by any means, but it is MUCH better than the iteration before. In fact, I personally think Sam the Salamander is doing pretty awesome!
Ok, but what if we want it to orient spatially as well?
Accounting for topology and space
The way we can get the generated image to rotate is by changing the perception.
If you recall, before we base the perception of the cell on the other cell around it. We then estimate the alive-ness or dead-ness gradient (or overall change) for each of the cells around it, through something called Sobel filters.
Basically we edit the features (kernels) to reflect the angle at which the image is being rotated by to allow the cells to proliferate into the image at any orientation. Though in the real world, its unlikely we’ll know exactly how many degrees the object has been rotated by, computers make this easy 🙃.
Because of how easy computers make life, we can generate images like these, where Sam looks like he’s climbing a tree, walking forward, just awkwardly staring, and more.
Though this method is technically quite a hefty assumption (as its unlikely we’ll know exactly how much we want a desired image to rotate by), using an exact measurement to circulate about the axes is a pretty good fix for now!
And there you have it! By combining these methods in our model, we can generate Sam and his friends no problem! 👇🏾
So cool!
But remember how I said we never used the previously remaining channels for everything?
Well, if we want to make numbers that classify themselves, that’s exactly what we’re going to need to use.
I’ll get this out of the way right now:
- MINST Digit: Modified National Institute of Standards and Technology; just the most common handwritten number digit that is used for all things machine learning and CNN when it comes to numbers and classification 🙂
Ok. Moving on.
So we had 10 whole channels that we didn’t use on Sam, and Sam likes numbers, so he wanted us to save the digits to be used on his friends (named 0 through 9).
We use these 10 channels that were unused to create labels for all 10 MINST Digits: 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9. The labels represent each of the 10 numbers that the generated digit could be.
And then, to train the model, the label with the highest correctness probability will be used to label each of the cells!
However, this time we don’t have predefined convolutional weights (just image stuff to work off of), and we instead allow the network to define parameters for weights on its own (independence 💪🏾).
Another difference is that in the last one, both the dead and alive cells were doing work, but this model is more realistic, and the dead cells are actually dead, and don’t preform computation. Therefore, when updating, we only consider the alive cells for the update.
So, after training, the model gives us this:
Isn’t it just so fun to see our numbers pass the boundaries of mutation and undergo a midlife crisis?
I think so.
Anyway, some of the 1s are trying to be 2s and the 3s are trying to be 8s, so we need to quickly optimize this model.
The reason this is happening is because when the model predicts the value of each cell, it generates a probability inside of the last 10 spots of the perception vector which helps to label the cell.
However, the cross entropy loss function being used doesn’t do so hot when two neighboring cells with the same state to have different probability numbers, because entropy = disorder/randomness. This means that some of the cells can’t stabilize into their predicted identity, and the midlife crisis ensues. If you want to, you can look at the graph, and see for yourself!
So, the obvious solution to resolve this disagreement would be to just swap out the loss function for something better, right?
Right.
So by changing the loss function to L2 loss, this disorder error no longer occurs, and we get a more stabilized result like this one:
The cells finally converge into a label, and (for the most part), decide on who they are!
What the L2 loss actually did is stabilize how large the states were, and minimize the update as the cells start to agree and label as a MNIST value.
By visualizing these internal values, we can actually view this process in action, and watch the dynamical (changing) system destabilize in real time!
And with that, we slap on some nice HTML, and we get an interactive MNIST Cellular Automata Network! Check it out!
Pretty sweet that I can go from 1 ↔ 2 ↔ 3 ↔ 4 ↔ 5 ↔ 6 ↔ 7 ↔ 8 ↔ 9…
So aside from making me jump up and down with joy when my model works, what can cellular automata and self organizing neural networks actually even do.
Well, of course they have huge implications in simulating biology, and understanding how regeneration works.
Aside form that, they’ll allow us to run even more complex simulations regarding biology, which I’m currently working on.
It involves previous work I was doing with myocytes…
It’ll be some interesting research!
What would you do if you could simulate stem cells?
Leave your comments in the reply section and smash 50 likes on this article if you liked it!
Here are my top picks:
Before you go…
My name is Okezue Bell, and I’m investing my time in researching and developing myself (in the super interesting applied biology space)! Be sure to contact me more if you want to collaborate, invite me, have an opportunity, or talk more (or any other engagement with me you can think of!):
LinkedIn: https://www.linkedin.com/in/okezue-a-…
Personal Website: https://www.okezuebell.com
Twitter: https://twitter.com/okezuebell
Also, I recommend tuning in to my monthly newsletter, non-scientific blog (on website), and letters (in newsletter site)! I post new updates there at least every month!