WnCC - Seasons of Code
Seasons of Code is a programme launched by WnCC along the lines of the Google Summer of Code. It provides one with an opprtunity to learn and participate in a variety of interesting projects under the mentorship of the very best in our institute.
List of Running Projects
- Browser Based PDF manager
- Resume Script Generator
- Physicc : A Simple Physics Engine
- Image Colorization
- Language Model Based Syntax Autocompletion in a Text Editor
- Computer vision based web app
- Cribbit Cribbit (Open for PGs Only)
- Techster Texter
- Language Detection
- Book Tracker
- ResoBin - Not the bin we deserve but the bin we need!
- Agree to disagree
- Watson (World's smartest assistant in your pocket)
- Meta Learning - Learning to Learn
- Break free of the matrix, by building one!
- Procedurally Generated Infinite Open World
- Introduction to App Development
- PAC MAN
- Introduction to Web Development
- Goal ICPC
- Traffic congestion modelling and rendering
- Tools for Data Science
- Machine Learning Based Metropolitan Air Pollution Estimation
- Audio controlled drone
- NLPlay with Transformers
- DIY FaceApp
- A Deep Dive into CNNs
- Competitive Coding
- Snake AI
- Facial Recognition App
- Gaming meets AI !!!
- R(ea)L Trader
- Computational Geometry
- Deep reinforcement learning - 2048 AI
- Reinforcement Learning to Finance
- Developing Hybrid ANN-Statistical Model for Robust Stock Market Prediction
- Astronomical Data-modelling and Interpretation
- Visual Perception for Self Driving Cars
- Convolutional Neural Networks and Applications
- Quantum Computing Algorithms
- Algorithm Visualizer
- Anime Club IITB Website using Django
- Machine Learning in Browser
If you remember the game in the image above, your childhood was awesome :p
In this project, we aim to build an intelligent agent “Pac-Man” that uses Reinforcement Learning to ace the game I’m sure all of us have played.
No. of mentees: 4
Reinforcement Learning (RL) is a field of Artificial Intelligence where an agent learns by interacting with its environment and receiving a reward/penalty for its actions. RL has recently started receiving a lot more attention, owing to the famous victory by an RL agent over the world champion in the game of “Go”. This project aims to get people familiar with the field and get started with a hands-on project that will require them to understand and implement some of the popular RL algorithms. We will start with building a simple GUI, and will then try out algorithms like SARSA and Q-learning for training agents in basic environments. We will then move on to Deep RL techniques for handling Pac-Man which involve the use of neural networks for training the agents.
Nothing as such right now. If you’re interested, you might want to read the first chapter of the book “Introduction to Reinforcement Learning” by Sutton and Barto to get a feel of RL. The book is available freely online and a simple google search should do :)
Watching the video lectures of the Reinforcement Learning Specialisation on Coursera by The University of Alberta can be a good starting point for getting the hang of RL.
Sutton and Barto (http://incompleteideas.net/book/RLbook2018trimmed.pdf) Coursera (https://www.coursera.org/specializations/reinforcement-learning?) Enjoy watching this beautiful documentary about AlphaGo by DeepMind - (https://www.youtube.com/watch?v=WXuK6gekU1Y)
The mentees will work collaboratively towards the project, with individually defined goals for everyone at a regular frequency.
We will be happy to mentor 4 students for this project. Familiarity with Python is essential. We would also use an ML framework such as PyTorch or Keras, but don’t worry if do not have experience working with them, you can spend some extra time on it along with the reading material during the initial phase of the project. Familiarity with probability theory will help you grasp the material better. You should mention your motivation for taking up this project and your past experience with programming in the proposal. Also, mention a rough approach for the project based on your understanding of the problem.
Tentative Project Timeline
|Week Number||Tasks to be Completed|
|Week 1||Basics of RL and some basic implementations. Python beginners should also start learning about the ML libraries. The goal for this week from the project side will be to have a simple GUI from openai gym ready and running for visualization.|
|Week 2-3||Understanding RL algorithms and implementing them on basic examples such as grid-world and mountain car. Training the agent based on these algorithms and observing the results.|
|Week 4-5||Understanding the technique of Deep Reinforcement Learning, implementing them, and comparing the results with other available implementations.|
|Week 6||Wrapping things up, documenting the work done and exploring possible extensions to the work done.|
|1||Getting started with the theory of RL. Getting the GUI ready|
|2||Understanding simple RL concepts and algorithms like MDP planning, SARSA and Q Learning|
|3||Implementing the basic applications of RL algorithms to get a hang of them|
|4||Understanding and implementing Deep RL algorithm for learning for Pac Man|
|5||Finishing up the implementation by fine-tuning the hyperparameters and removing bugs|