Problem Statement: The core problem we are trying to solve is to help future dog owners find a dog who is a good fit for their lifestyle and family environment.
Solution: A user friendly app that helps connect future dog owners with dogs available for adoption.
Features:
Help the user search for dogs based on certain features such as size and color.
Find similar dogs by uploading a picture of a dog the user is interested
Connect the dog with the user by allowing the user to chat with a persona of the dog.
App Demo
Dog Search With Features
Every candidate has a bunch of tags associated with her.
When a user types in text in search box it is compared to available tags.
When a user types in text in search box it is compared to available tags.
Tags for the following picture can be:Retriever,Black
Dog Search With Images
To get the embeddings we use EfficientNet.
This model was finetuned on Stanford Dogs dataset.
Workflow:
Generate embedding for query image.
Run a similarity search over existing embeddings using FAISS.
Return results in descending order of similarity.
EfficientNet Architecture
Chatbot:
To build the chatbot we tried 3 different models:
BERT
GPT2
GPT2 DoubleHead
Chatbot demo
Chatbot
1. BERT
Masked Language Model
Made up of only the Encoder with stacked transformer blocks
Bidirectional language model
BERT Embeddings
BERT Architecture
2. GPT2
Auto-regressive model (A word is predicted using words from its left context only)
Made up of only the Decoder with stacked transformer blocks