Using Double DQN and Sensor Information for Autonomous Driving

(E-pub Ahead of Print)

Author(s): Ramesh Raghavan*, Dinesh Chander Verma, Tarun Bansal

Journal Name: International Journal of Sensors, Wireless Communications and Control

Become EABM
Become Reviewer
Call for Editor


Background: In most of the research the autonomous driving problem is either solved with policy gradients or DQN. In this paper, we have tried to eliminate the problem of policy gradients which is high variance and an overestimation of values in DQN. We have used DDQN as it has low variance, and it solves the problem of overestimation in DQN.

Aim: The main aim of this paper is to propose a framework for an autonomous driving model that takes in raw sensor information as input data and predicts actions as output from the model which could then be used for simulating the car.

Objective: The main objective of this paper is to use DDQN and Discretization technique to solve the autonomous driving problem and get better results even with a continuous action space.

Methods: To solve the bridge between self-driving cars and reinforcement learning we have used Double Deep QNetworks as this could help to prevent the overestimation of values by decoupling the selection from the evaluation. Also, to solve the problem of continuous action space we have used the discretization technique in which variables are grouped into bins and each bin is assigned a value in such a way that the relationship between the bins is preserved.

Result: The experimental results showed improved performance of the agent. The agent was tested for different conditions like curve roads and traffic, which showed the agent can drive at different conditions as well. We have also illustrated how DDQN performed well over policy gradients just by adding a simple discretization technique to make the action space discrete and overcoming the issue of overestimation of q-values.

Conclusion: We design the gym environment and reward function for DDQN to work. We have also used CARLA as a virtual simulator for training purposes. Finally, we have demonstrated that our agent was able to perform well in different cases and conditions. As a reminder note, we can improve our agent to also work for following traffic light rules and other road safety measures.

Keywords: Deep Reinforcement Learning, DDQN, Artificial Intelligence, Autonomous Driving, CARLA, overestimation.

Rights & PermissionsPrintExport Cite as

Article Details

Published on: 14 January, 2021
(E-pub Ahead of Print)
DOI: 10.2174/2210327911666210114121049
Price: $95

Article Metrics

PDF: 207