Page Not Found
Page not found. Your pixels are in another canvas.
Sitemap
A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Pages
Projects
My Github hosted projects
MPC ML Task List
A Model-Predictive-Control style list of ML tasks to do
Posts
Fastai Transforms for DonkeyCar
Fastai transforms don’t seem to help DonkeyCar training
Effect of Dropout Layers in a VAE
Dropout layers make VAE output worse
Endgame Speculation
My speculation on how Avengers Endgame could go
Stable Baselines Algorithms
Table of algorithms in the Stable Baselines repo
First Real DAgger Results
Using DAgger to improve MaLPi’s training, while MaLPi is driving
Initial DAgger Results
Using DAgger to improve MaLPi’s training
First (Second) Fully Autonomous Full Lap
One of MaLPi’s first fully autonomous laps
First attempts at Hyperparameter Optimization
Hyperparameter optimization using hyperopt on racetrack data.
Normalizing image data before training - GRU version
Testing image normalization when using a GRU.
Normalizing image data before training - LSTM version
Testing image normalization when using an LSTM.
Normalizing image data before training
I had completely forgotten to normalize the images I’m feeding into MaLPi’s network, so I thought I’d try to be a bit more formal about it than my usual.
Getting a Keras LSTM layer to work on MaLPi
Training on batch sizes and/or sequence lengths longer than one, while still being able to run one image at a time on the robot.
Experimenting with OpenAIs Baselines code
I forked Open AI’s baseline code and made a few changes. This was my first full run before I started playing around with the model architecture.
Single Battery
Eliminated a battery! (maybe)
State of the Hardware v2
I’m about to make large changes to MaLPi so I wanted to document the current state of the hardware.
Motors
What have I gotten myself into?!?! Motors and controllers and a breadboard, oh my!
Lego Chassis
Some progress on the hardware front.
Endurance Test
Test how long the PowerGen battery can run MaLPi on a single charge.
MaLPi Intro
MaLPi (Machine Learning Pi)
publications
Horde: A Scalable Real-time Architecture for Learning Knowledge from Unsupervised Sensorimotor Interaction
Using RL value functions to encode semantic knowledge, specifically by a robot.
Off-Policy Actor-Critic
Off-Policy AC with linear state features. Includes elegibility traces.
Human-level control through deep reinforcement learning
One of the first deep reinforcement learning papers.
Deep Reinforcement Learning with Double Q-learning
Improved Q-value estimation by reducing overestimates of Deep Q-networks.
Reinforcement Learning: An Introduction (2nd Edition)
In-progress second edition of an RL textbook.
Attention and Augmented Recurrent Neural Networks
Overview (with references) of attention and several types of augmentation for RNNs.
Reinforcement Learning with Unsupervised Auxiliary Tasks
Increase speed of a Reinforcement Learning system with auxiliary task.
Questions and Intuition for Tackling Deep Learning Problems
Five questions to ask about your deep learning project.
A simple neural network module for relational reasoning
Relationships between objects.
One Model To Learn Them All
A single ML model used for very different tasks.
Eligibility Traces
Notes on using Eligibility Traces with neural networks
Deep Learning in Robotics: A Review of Recent Research
A long review of the use of DL in robotics
An Overview of Multi-Task Learning in Deep Neural Networks
A long review of the use of DL in robotics
Pre-training Neural Networks with Human Demonstrations for Deep Reinforcement Learning
Pre-train using supervised learning on human provided demonstations.
Mastering the game of Go without human knowledge
AlphaGo Zero, all RL self-play.
Tensorizing LSTMs
Tensorizing LSTMs to make them wider and deeper without adding parameters and with minimal extra compute costs.
Pretraining Deep Actor-Critic Reinforcement Learning Algorithms With Expert Demonstrations
Pretraining Deep Actor-Critic Reinforcement Learning Algorithms With Expert Demonstrations.
World Models
Unsupervised learning of image encoding and dynamics model.
Unsupervised Predictive Memory in a Goal-Directed Agent
Unsupervised training of a memory that is used for prediction of state and reward.
Learning Real-World Robot Policies by Dreaming
Unsupervised learning of image encoding, dynamics and reward models.
Sample-Efficient Deep RL with Generative Adversarial Tree Search
Learned dynamics model with a GAN for image generation and MCTS for planning.
Unicorn: Continual learning with a universal, off-policy agent
Continual learning with a universal, off-policy agent.
On the link between conscious function and general intelligence in humans and machines
Access consciousness and it’s relation to general intelligence