Model Optimization using Quantization
Data science models based on deep neural networks become huge in terms of the number of parameters and weights. This effectively increases the model size which has a direct impact on model load and inference time. In this post, I will be exploring model quantization to decrease model load time and model inference time for various Image/Text models. More specifically, I would like to benchmark model performance and in specific I want to compare FP32 weights vs FP16/INT8 weights for some of the internal layers in the Neural Network. My first attempt is to use tflite and try optimizing the model. I will use the galaxy image classification data to see if there is any drop in accuracy when compared to the base-model. for this purpose, I will use mobilenet and train it with custom head for classifying a set of 100-200 images. From the research paper at https://arxiv.org/abs/2104.11849, I understand that quantization of mobile nets might not give us per...