Emergency vs Non-Emergency Vehicle Classification
Motivation
Recently I participated in JanataHack: Computer Vision Hackathon hosted by Analytics Vidhya. The aim of the competition was to create a binary image classifier that could differentiate the Non — Emergency Vehicles eg. private owned vehicles . from the emergency vehicles (police vehicles , ambulances, etc).
Originally I had used keras at that time of submission. But then I decided to implement a pytorch version to see if there was any advantage in using pytorch as most top participants has used fastai which used pytoch in backend.
Why Is this important ?
Fatalities due to traffic delays of emergency vehicles such as ambulance & fire brigade is a huge problem. In daily life, we often see that emergency vehicles face difficulty in passing through traffic.
So differentiating a vehicle into an emergency and non emergency category can be an important component in traffic monitoring as well as self drive car systems as reaching on time to their destination is critical for these services.
Data Description
There are total of 1646 in the train set and 706 in the test set.
There are two .csv files which contain the train and test image names.
Class Balance
Class imbalanced.
0 965
1 681Here 0 stands for Non-Emergency vehicles and 1 stands for Emergency Vehicles.
There are about 284 more images in class 0.
This didn’t create any problem with so I didn’t try to balance them in the augmentation stage.
Looking at the labels and the problem statement we can conclude that this is a binary classification problem
EDA.
One advantage of having I small dataset is that we can skim through all the images and see if there are any outliers ,mislabeled samples.
By doing this I found 6 images which is miss classified and some images contains only dashboards.
Miss labelled Images.
by removing these images my accuracy increased the val_score by 2%. It went from 94% to 96% and some times 97% the model I used was resnet50. The same implementation in keras only got me at 94%.
Now I would like to take the time and appreciate pytorch here. I really don’t know why this is happening maybe someone can help me with this.
Images of dashboards.
By removing these images the accuracy become more stable (less oscillations).
One think to note here I was able to remove these dashboard images because I didn’t find any similar images in the test data.
Data Pipeline.
The Model
Here I have made use of the same model used in the tutorial 4 and 5. I have added a batchnorm layer after every conb2d layer.
I also tried adding an dropout layer but it had no effect so I removed it.
Experimentation with weight decay and gradient clipping
Using simple fit function. Experimenting with weight decay and gradient clipping
Keeping everything constant and varying only one parameter eg. weight decay or gradient clipping. We can observe who the parameter will effect the learning
Selecting Weight Decay
from the above table we can see that weight decay of 1e-3 and 1e-4 produce the best val_score but wd of 1e-4 has less over-fitting than 1e-3
Selecting Weight Gradient Clipping
In the above table gradient clipping of 0.3 with weight decay
results in a val_score of 78.7%.
Now checking for Weight decay = 1e-4
when weight decay = 1e-4 the results are different. here weight the gradient clipping of 0 gives the best result. with less over-fitting.
So from the above experiment I decided to go with weight decay of 1e-4 and gradient clipping of 0.
Plotting the loss and accuracy plots.
There is very less over-fitting and the val_accuracy reaches its peak val at 90%. here again I would like to add when I had created a custom model in keras the height val_score I was able to achieve was 83% changing the framework got we an increase of 7%. One
more thing the size of the mode, using pytorch I am able to use a model having more than 3 Conv2d layers without over-fitting. But in keras I could only use 2 layers not more than that anything higher or lower would just add to the training cost without improving the accuracy.
Using The Pretrained Models:
I made use of two model architectures resnet and densenet. One thing to not the densenet models produce almost similar results to resnet models with lower epochs and most important the saved model takes half the memory space.
Model Definition.
For resnet50.
loss and accuracy plots.
here one can see lot of over-fitting and now improvement in val_score. I decided to try using the cyclic scheduler training strategy here’s the result.
I still need to do more experiment with this method but as one can see. I have reduce the overfitting to some extend but the val_accuracy is still low.
Using Densenet169
If you look at the loss and accuracy plots. The over-fitting has decreased. the val accuracy is better but this was done without the cyclic scheduler.
Using early stopping the training can be stopped at 5 epochs.
Saving and loading the model.
Creating a simple web app using streamlit.
Note the web app only accepts jpg images.
Conclusion
I was able to get a 200 rank out of 10000 so I made it in the top 2% using the above model. My final rank in the competition was 147 using efficientnet in keras but sadly I couldn’t find implementation in the models.
To get a higher rank one would need to use ensemble approaches which is out of scope of the is blog.
all the code will be available in my github repo: https://github.com/evilc3/EmergencyVehicleDetector don’t forget to star my repo. if you like it.
The entire notebook : https://colab.research.google.com/drive/13En-V2A-w2o4uXuDZk0ypktxzX9joXIY?usp=sharing
Web app link : https://emervehicledetector.herokuapp.com/
For any one trying to create a web app add the following line to your requirements.txt file -f https://download.pytorch.org/whl/torch_stable.html this makes sure that you are using the cpu version.Else heroku downloads the gpu and cpu version which exceeds the 500mb limit throwing an slug size too large to know more about the content on the requirement file check the repo.
If anyone can help me with this it will be really nice.If you want to try the web app you can check my repo. is present there in the web_app folder.
