Virtual to Real Reinforcement Learning for Autonomous Driving

Xinlei Pan, Yurong You, Ziyan Wang and Cewu Lu

Abstract

Reinforcement learning is considered as a promising direction for driving policy learning. However, training autonomous driving vehicle with reinforcement learning in real environment involves non-affordable trial-and-error. It is more desirable to first train in a virtual environment and then transfer to the real environment. In this paper, we propose a novel realistic translation network to make model trained in virtual environment be workable in real world. The proposed network can convert non-realistic virtual image input into a realistic one with similar scene structure. Given realistic frames as input, driving policy trained by reinforcement learning can nicely adapt to real world driving. Experiments show that our proposed virtual to real (VR) reinforcement learning (RL) works pretty well.

Session

Spotlights

Files

PDF iconPaper (PDF)
MP4 iconVideo (MP4)

DOI

10.5244/C.31.11
https://dx.doi.org/10.5244/C.31.11

Citation

Xinlei Pan, Yurong You, Ziyan Wang and Cewu Lu. Virtual to Real Reinforcement Learning for Autonomous Driving. In T.K. Kim, S. Zafeiriou, G. Brostow and K. Mikolajczyk, editors, Proceedings of the British Machine Vision Conference (BMVC), pages 11.1-11.13. BMVA Press, September 2017.

Bibtex

            @inproceedings{BMVC2017_11,
                title={Virtual to Real Reinforcement Learning for Autonomous Driving},
                author={Xinlei Pan, Yurong You, Ziyan Wang and Cewu Lu},
                year={2017},
                month={September},
                pages={11.1-11.13},
                articleno={11},
                numpages={13},
                booktitle={Proceedings of the British Machine Vision Conference (BMVC)},
                publisher={BMVA Press},
                editor={Tae-Kyun Kim, Stefanos Zafeiriou, Gabriel Brostow and Krystian Mikolajczyk},
                doi={10.5244/C.31.11},
                isbn={1-901725-60-X},
                url={https://dx.doi.org/10.5244/C.31.11}
            }