Rishabh Shah

9 – Nilamber Grandeur, Bhayli,Vadodara 391410, Gujarat, India

+91-8849923141 | rishabhshah096@gmail.com |

EDUCATION
Vellore Institute of Technology (VIT), Vellore, Tamil Nadu, India Jul 2014 – May 2018

Bachelor of Technology (B.Tech) - Computer Science & Engineering - GPA: 7.8/10.0

OBJECTIVE
-------------------------------

WORK EXPERIENCE
1. RSA Instruments Pvt. Ltd., Baroda, India Aug’19 – present

Product Development head and Lead Computer Vision Engineer Aug’19 - Present

Developing actuators which can reduce the number of human touch points in manufacturing unit, it is all done with the help of
installing cameras and sensors at the right place.

PROJECTS
Cartesian Robot to segregate waste without human contact
(Sep’19 - present)

Objective is to build an object detection and recognition system which can easily differentiate between different kinds of waste
materials and objects, this should be customisable according to the different customer’s need.

Object detection algorithm Faster RCNN has been implemented and has been deployed on GPU to provide with the fastest
prediction considering fps. There are a number of different processes after detection and recognition which is like converting this
output into input suitable for the servo motors so that they move the robotic arm accordingly across X and Y axis.

Technologies:
-Image Processing – Surrounding environment analysis
-Image feature extraction: Object tracking, object detection, Texture
-Machine learning and deep learning-based models: MLP, FR- CNN, Yolov3

-Python libraries OpenCV, TensorFlow, Keras, Scikit-learn , PyQT(UI),Flask

2. Continental Automotive Components Pvt. Ltd., Bengaluru, India Jan 2018 – Jul 2019

Artificial Intelligence and Robotics (AIR) – Graduate Engineer Trainee (Jul’18 – Jul’19)
Intern (Jan’18 – Jul’18)

Develop new POCs/prototypes for in-cabin monitoring of passengers in Level –3,4,5 autonomous vehicles using cameras and
sensors. Machine learning and Computer vision being the core part of developing the solutions related to camera.

PROJECTS

Robust Face Anti-Spoofing for facial recognition systems in an automobile - Hyundai Motor Corp (client).
(Jan'18 – Sep'18)
Main objective was to prevent facial recognition from detecting faces of 2D attacks like printed face photographs or images of your
face from the mobile

Generated a literature review for the algorithms which included both traditional computer vision algorithms and the sophisticated
ones which deployed neural network for final classification, for the client’s background survey.

Final algorithm which was developed, looked into different features like optical flow, Local Binary Patterns(variants), Blobs and
SIFT. Features were selected carefully to complement one another with the help of techniques like Fisher Ratio and finally the face
images features were passed onto different classifiers like Multilayer Perceptron Network and SVM.
Different approaches like that of Variational Auto encoder were used so that we could predict the false face as soon as that live
face is enrolled.

The metrics used for gaining insights into the algorithm were False Acceptance Rate (FAR) and False Rejection Rate (FRR).

The algorithm worked successfully for both NIR and RGB camera hence making robust to different scenarios and hardware.

Accuracy for the algorithm was 97.35 % in given test conditions and scenarios.

Technologies:
-Image Processing – Surrounding environment analysis (e.g. Day or Night)
-Image feature extraction: Object tracking, Optical flow map generation, Key point extraction, Texture
-Machine learning and deep learning-based models: MLP, CNN, Auto-encoder and Variational auto encoders

-Python libraries OpenCV, TensorFlow, Keras,Scikit-learn

Voice recognition system for personalization features within the automobile

(Oct’18 - Jan’19)
Worked towards POC for voice recognition system, with the condition being the speakers will not be saying same hook line like
“Alexa” or “Ok google”, which made the system text independent. The algorithm was based on deep learning architectures like
VGG and ResNet. Different machine learning techniques were also implemented to find out the lower bound for every result and
compare it with deep learning architecture results. It also needed a lot of speech pre-processing since the test conditions had more
of background noise and cluttering of vehicles, measured with the help of signal to noise ratio.

Opensource dataset like ”Voxceleb YouTube” dataset were also used to make the network robust and prevent it from overfitting.

Number of True matches and False matches in the test dataset was taken as a metric.

Accuracy for the algorithm was 95% in given test conditions and scenarios.

Technologies:
-Signal Processing – Surrounding environment analysis (e.g. Car honking, child crying, people laughing)
-Feature extraction: Mel Frequency Cepstral Coefficient (MFCC), Pitch, Loudness
-Deep learning/ Machine learning models based on voice: Gaussian mixture models, CNN, LSTM

-Python libraries TensorFlow, Keras , Scikit-learn, Librosa

Driver distraction detection through the analysis of driver’s actions – Passive safety and Sensorics (PSS)
(internal client) (Jan’19 - April’19)
The goal to achieve was that detection of certain kinds of actions from driver which are considered to be not allowed while driving
which includes like talking/texting over cell phone, drinking any fluid, head direction.

There were three parts to it first being object detection to detect the certain kind of decided objects, the head pose estimation and
movement of hands. The dataset used was made in similar lines to Kaggle state farm driver distraction dataset.

The deep learning architectures like Faster RCNN were used for detection of the specific object in ROI. To understand whether the
networks were detection right regions a visualization technique called Class Activation Maps were used. This architecture results
were combined with that of head pose estimation and the hand motion estimation to provide final output related to distractedness of
the driver.

The accuracy of the model was 83% which considering the state-of-the-art methods was good enough.

Technologies:
-Deep learning/Machine learning models: Faster RCNN, YoloV3 and MobileNet-SSD
-Python libraries TensorFlow, Keras , Scikit-learn, OpenCV

Impart emotion into speech generated from the chatbot’s Text-to-Speech model
(April’19 - Jul-’19)
Worked towards POC for converting emotion from generated speech from neutral to angry and neutral to sad/sorrow.
The research and development were based on two theories one where the voice was converted after it was generated and the
other option was to convert directly into certain emotion. The former approach was done through with the help Cycle-GAN and the
later one required some tweaks in Tacotron model. The cycle GAN was trained on custom dataset for a particular set of text. The
results obtained on the cycle-GAN were really positive.

Technologies:
-Deep learning/Machine learning models: Cycle-GAN,Tacotron(Includes LSTM and CNN)
-Python libraries TensorFlow, Keras , Scikit-learn,Librosa,NLTK

ACADEMIC PROJECTS
Visual Question Answering Bot for Blind (Final Year project) (Dec’18- April’18)

It is a chat-bot which answers your queries related to the image which is being shown to it. It uses the standard VGG-16 to compute
the feature vector for an image and this feature vector is concatenated with the text feature vector generated by an LSTM network
over the caption describing it.The network was trained on cloud through Floydhub,on custom built dataset of a house. There were
images of the house from inside and the description about those images were provided, so if the blind were to enter this house,
he/she would have every information about it.

The dataset used is MS-COCO for pre-trained weights and analysis, which is really huge and has an image and description about it
in every format. The chat bot answers the question in an objective way since making it generative would be a bit difficult but it was
working fine with an accuracy of 72%.The scores generated are based on the custom-built dataset and not on MS-COCO.

Technologies:
-Deep learning/Machine learning models: CNN, LSTM
-Python libraries TensorFlow, Keras , Scikit-learn, OpenCV, NLTK

Sentiment analysis for Amazon food reviews (March 2017 – May 2017)

Kaggle - Amazon food review dataset has been used to identify the sentiment of the comment and logistic regression classifier is
used to train the system. NLP(NLTK-library) has been implemented to tokenize and stem a particular comment for better training of
classifier. TF-IDF method was used to generate the feature vector out of the sentences used to review a particular product. The
classifier used was SVM since it the comments were either positive, negative or neutral.

Technologies:
-Machine learning models: SVM
-Python libraries Scikit-learn, NLTK

Anomaly detection in Human Actions (Feb 2017 – May 2017)

Aim of this project was to identify thieves and other criminals directly through cameras at public places. Implemented two CNNs
one focuses on the optical flow through series of images (Temporal CNN) and the other one works upon analysis of each image
(Spatial CNN) in a video frame. The training of neural net was done on UCF-101 dataset. Since there were not enough proper
videos the dataset was custom built (with data augmentation) and should not be used for real time testing but it did give result of
87% accuracy at classifying right actions. Gaussian mixture model (using soft clustering) was used to identify the features which
are closely related and the ones that had less probability were classified as anomalous.

Technologies:
-Deep learning/Machine learning models: GMM, CNN
-Python libraries TensorFlow, Keras , OpenCV

EDMATE – A chatbot that solves all the student queries regarding the college (July 2017 – Nov 2017)

The chatbot was trained on custom built university-based questions and answers. The major research went into the types of questions
asked and how many possible ways a similar question can be asked hence it was also augmented with some general conversations
so that the chat seemed seamless. A classifier was trained to classify each sentence into what category of question was asked and
then a Bidirectional LSTM encoder –decoder mode with attention mechanism was used to provide a suitable answer.

Accuracy of the classifier was 89% which was based on our custom test cases, while the LSTM accuracy was 77% since the
dataset was not perfect enough to provide the best result.

Technologies:
-Deep learning/Machine learning models: RNN-LSTM
-Python libraries TensorFlow, Keras ,NLTK,Spacy

Eventivity – An app that solves all the student queries regarding the college (Sept 2015 – Nov 2015)
An android application that could provide you with a list of events happening around the campus and provide you with list of
projects/activities happening.

Technologies:
-Java, XML, Android Studio, SQLite

CERTIFICATIONS
-Deep Learning Specialization by deeplearning.ai – Coursera
-Machine Learning Course by Stanford University - Coursera
-Machine learning with Python and R – edx
-3D drawing and Animation with MAYA
-Web App development using Google polymers – Google Developer Group conference

PUBLICATION
An Intelligent Chat-Bot Using Natural Language Processing

Link: http://ijer.in/publication/v6/063.pdf
International Journal of Engineering Research - ISBN: 978-81-932091-0-3
A question-answering chat-bot based on Long Short-Term Memory networks (LSTM) and K-means clustering algorithm.

ACTIVITIES & ACHIEVEMENTS

-Chief Networking Officer (CNO) - Spartans Association of Youth (Non-profit organization) (Jul’16 - April’17)

Connected the organization to different early stage startups from colleges and tier-2 cities within the country and provide them with
a medium to connect with college students and engage their user base. Turned this NPO startups oriented and created a culture to
embrace each kind of innovation.

-Guest Speaker for Pattern recognition and its application workshop – Vellore Institute of Technology (April’19)

An interactive session on how Convolutional Neural Network functions and how its applications are changing the automobile
industry

-Runners up in “TCS – Digital Impact square” Hackathon (Sept’17)

Developed a solution to analyse and maintain the e-toilets launched by the Government of India with the help of IoT and Data
analysis.

-Session on “How to tune hyperparameters of CNN with Bayesian Optimization” Continental – Open AIR (Jan’19)

The session was about looking into deep learning with a probabilistic approach and how it could help in providing with better results
regarding

TECHNOLOGIES
C, C++, Python, Java, PHP, HTML, XML, CSS, JavaScript, D3.js, SQL, MATLAB, Arduino, TensorFlow, Keras.

Android Studio, Visual Studio, Hadoop, Google Cloud Platform, Amazon Web services, PyCharm, Eclipse, Git.