Seasons Of Code
Generating a human pose dataset using PC games • Rohit Jena
WnCC - Seasons of Code
Seasons of Code is a programme launched by WnCC along the lines of the Google Summer of Code. It provides one with an opprtunity to learn and participate in a variety of interesting projects under the mentorship of the very best in our institute.
List of Running Projects
- Browser Based PDF manager
- Resume Script Generator
- Physicc : A Simple Physics Engine
- Image Colorization
- Language Model Based Syntax Autocompletion in a Text Editor
- Computer vision based web app
- Cribbit Cribbit (Open for PGs Only)
- Techster Texter
- Language Detection
- Book Tracker
- ResoBin - Not the bin we deserve but the bin we need!
- Moodify
- Agree to disagree
- Unscripted
- Watson (World's smartest assistant in your pocket)
- IITinder
- BriefKing
- Meta Learning - Learning to Learn
- Break free of the matrix, by building one!
- Procedurally Generated Infinite Open World
- Introduction to App Development
- PAC MAN
- Introduction to Web Development
- (Un)Clear
- Goal ICPC
- Traffic congestion modelling and rendering
- PyRated
- Tools for Data Science
- Machine Learning Based Metropolitan Air Pollution Estimation
- Audio controlled drone
- NLPlay with Transformers
- DIY FaceApp
- A Deep Dive into CNNs
- Competitive Coding
- Snake AI
- Facial Recognition App
- Gaming meets AI !!!
- R(ea)L Trader
- Computational Geometry
- Deep reinforcement learning - 2048 AI
- Reinforcement Learning to Finance
- Developing Hybrid ANN-Statistical Model for Robust Stock Market Prediction
- Si-Phy
- Astronomical Data-modelling and Interpretation
- Visual Perception for Self Driving Cars
- Convolutional Neural Networks and Applications
- Quantum Computing Algorithms
- Algorithm Visualizer
- Anime Club IITB Website using Django
- Machine Learning in Browser
Generating a human pose dataset using PC games
This project will focus on getting human pose estimates in games to generate a dataset using no manual annotations or labelling.
Deep networks are very data hungry in this age. Annotating lots of data is very tedious, expensive, and inefficient.
However, a lot of ground truth data can be easily generated by using the rendering of video games to extract specific information like semantic segmentation, depth maps, etc. The project will focus on getting human pose estimates in games to generate a “in-the-wild” dataset using no manual annotations or labelling.
This will be done by injecting specialized code into the DirectX rendering API. We’ll further test the effectiveness of the dataset on real images to see if such a dataset can provide benefits in training.
Timeline
| Week | Task |
|---|---|
| Week1 | Understand the main paper, and what pose estimation is |
| Week2 | Download a free game and start exploring the DirectX API |
| Week 3, 4 | Extract the pose information from pre-renders |
| Week 5, 6 | Cleaning up the dataset, and testing a small DNN to predict pose |
| Week 7, 8 | Try on one more game and start testing on real datasets |