Using the Location Information
It was very simple till now. This is the part where it becomes tricky. How to integrate the additional location information into the input of the NN? There are two approaches to this.
Adding an Additional Layer
The images will come in three-layer RGB format, the trick is to add an additional fourth layer that contains the location information.
This is essentially what you do-
- Pro: The whole ConvNet gets to see the location data and learns well.
- Con: ConvNets are not good at dealing with such information, so this approach might not be very effective.
Concatenating Information after Feature Extraction by ConvNets, and before the FC Layers
In architectures like ResNet50, there are several fully connected (FC) layers trailing the convolutional layers. The goal is to put tabular data into those FC layers and let those last FC layers learn from the location information.
The architecture will look similar to this-