Mobile robot navigation based on Q-learning technique

Lazhar Khriji, Farid Touati, Kamel Benhmed, Amur Al-Yahmedi

Research output: Contribution to journalArticle

23 Citations (Scopus)

Abstract

This paper shows how Q-learning approach can be used in a successful way to deal with the problem of mobile robot navigation. In real situations where a large number of obstacles are involved, normal Q-learning approach would encounter two major problems due to excessively large state space. First, learning the Q-values in tabular form may be infeasible because of the excessive amount of memory needed to store the table. Second, rewards in the state space may be so sparse that with random exploration they will only be discovered extremely slowly. In this paper, we propose a navigation approach for mobile robot, in which the prior knowledge is used within Q-learning. We address the issue of individual behavior design using fuzzy logic. The strategy of behaviors based navigation reduces the complexity of the navigation problem by dividing them in small actions easier for design and implementation. The Q-Learning algorithm is applied to coordinate between these behaviors, which make a great reduction in learning convergence times. Simulation and experimental results confirm the convergence to the desired results in terms of saved time and computational resources.

Original languageEnglish
Pages (from-to)45-51
Number of pages7
JournalInternational Journal of Advanced Robotic Systems
Volume8
Issue number1
Publication statusPublished - Mar 2011

Fingerprint

Mobile robots
Navigation
Learning algorithms
Fuzzy logic
Data storage equipment

Keywords

  • Behaviors based navigation
  • Fuzzy logic and reinforcement learning
  • Mobile robot

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Artificial Intelligence

Cite this

Mobile robot navigation based on Q-learning technique. / Khriji, Lazhar; Touati, Farid; Benhmed, Kamel; Al-Yahmedi, Amur.

In: International Journal of Advanced Robotic Systems, Vol. 8, No. 1, 03.2011, p. 45-51.

Research output: Contribution to journalArticle

@article{b08eee3cbfb04165ba97c1f50df16ee0,
title = "Mobile robot navigation based on Q-learning technique",
abstract = "This paper shows how Q-learning approach can be used in a successful way to deal with the problem of mobile robot navigation. In real situations where a large number of obstacles are involved, normal Q-learning approach would encounter two major problems due to excessively large state space. First, learning the Q-values in tabular form may be infeasible because of the excessive amount of memory needed to store the table. Second, rewards in the state space may be so sparse that with random exploration they will only be discovered extremely slowly. In this paper, we propose a navigation approach for mobile robot, in which the prior knowledge is used within Q-learning. We address the issue of individual behavior design using fuzzy logic. The strategy of behaviors based navigation reduces the complexity of the navigation problem by dividing them in small actions easier for design and implementation. The Q-Learning algorithm is applied to coordinate between these behaviors, which make a great reduction in learning convergence times. Simulation and experimental results confirm the convergence to the desired results in terms of saved time and computational resources.",
keywords = "Behaviors based navigation, Fuzzy logic and reinforcement learning, Mobile robot",
author = "Lazhar Khriji and Farid Touati and Kamel Benhmed and Amur Al-Yahmedi",
year = "2011",
month = "3",
language = "English",
volume = "8",
pages = "45--51",
journal = "International Journal of Advanced Robotic Systems",
issn = "1729-8806",
publisher = "Vienna University of Technology",
number = "1",

}

TY - JOUR

T1 - Mobile robot navigation based on Q-learning technique

AU - Khriji, Lazhar

AU - Touati, Farid

AU - Benhmed, Kamel

AU - Al-Yahmedi, Amur

PY - 2011/3

Y1 - 2011/3

N2 - This paper shows how Q-learning approach can be used in a successful way to deal with the problem of mobile robot navigation. In real situations where a large number of obstacles are involved, normal Q-learning approach would encounter two major problems due to excessively large state space. First, learning the Q-values in tabular form may be infeasible because of the excessive amount of memory needed to store the table. Second, rewards in the state space may be so sparse that with random exploration they will only be discovered extremely slowly. In this paper, we propose a navigation approach for mobile robot, in which the prior knowledge is used within Q-learning. We address the issue of individual behavior design using fuzzy logic. The strategy of behaviors based navigation reduces the complexity of the navigation problem by dividing them in small actions easier for design and implementation. The Q-Learning algorithm is applied to coordinate between these behaviors, which make a great reduction in learning convergence times. Simulation and experimental results confirm the convergence to the desired results in terms of saved time and computational resources.

AB - This paper shows how Q-learning approach can be used in a successful way to deal with the problem of mobile robot navigation. In real situations where a large number of obstacles are involved, normal Q-learning approach would encounter two major problems due to excessively large state space. First, learning the Q-values in tabular form may be infeasible because of the excessive amount of memory needed to store the table. Second, rewards in the state space may be so sparse that with random exploration they will only be discovered extremely slowly. In this paper, we propose a navigation approach for mobile robot, in which the prior knowledge is used within Q-learning. We address the issue of individual behavior design using fuzzy logic. The strategy of behaviors based navigation reduces the complexity of the navigation problem by dividing them in small actions easier for design and implementation. The Q-Learning algorithm is applied to coordinate between these behaviors, which make a great reduction in learning convergence times. Simulation and experimental results confirm the convergence to the desired results in terms of saved time and computational resources.

KW - Behaviors based navigation

KW - Fuzzy logic and reinforcement learning

KW - Mobile robot

UR - http://www.scopus.com/inward/record.url?scp=79952386510&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79952386510&partnerID=8YFLogxK

M3 - Article

VL - 8

SP - 45

EP - 51

JO - International Journal of Advanced Robotic Systems

JF - International Journal of Advanced Robotic Systems

SN - 1729-8806

IS - 1

ER -