Energy management control strategy for plug-in fuel cell electric vehicle based on reinforcement learning algorithm

LIN Xin-you; XIA Yu-tian; WEI Shen-shen

doi:10.13374/j.issn2095-9389.2018.10.15.001

Volume 41 Issue 10

Oct. 2019

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Engineering > 2019 > 41(10): 1332-1341

LIN Xin-you, XIA Yu-tian, WEI Shen-shen. Energy management control strategy for plug-in fuel cell electric vehicle based on reinforcement learning algorithm[J]. Chinese Journal of Engineering, 2019, 41(10): 1332-1341. doi: 10.13374/j.issn2095-9389.2018.10.15.001

Citation:

LIN Xin-you, XIA Yu-tian, WEI Shen-shen. Energy management control strategy for plug-in fuel cell electric vehicle based on reinforcement learning algorithm[J]. Chinese Journal of Engineering, 2019, 41(10): 1332-1341. doi: 10.13374/j.issn2095-9389.2018.10.15.001

Citation:

PDF( 9215 KB)

Energy management control strategy for plug-in fuel cell electric vehicle based on reinforcement learning algorithm

doi: 10.13374/j.issn2095-9389.2018.10.15.001

LIN Xin-you^{1), 2)
,
,},
XIA Yu-tian¹⁾,
WEI Shen-shen¹⁾

1).
College of Mechanical Engineering and Automation, Fuzhou University, Fuzhou 350002, China
2).
Key Laboratory of Fluid Power and Intelligent Electro-Hydraulic Control, Fuzhou University, Fuzhou 350002, China

More Information

Corresponding author: LIN Xin-you, E-mail: linxinyoou@fzu.edu.cn
Received Date: 2018-10-15
Publish Date: 2019-10-01

Abstract

Abstract

To cope with the increasingly stringent emission regulations, major automobile manufacturers have been focusing on the development of new energy vehicles. Fuel-cell vehicles with advantages of zero emission, high efficiency, diversification of fuel sources, and renewable energy have been the focus of international automotive giants and Chinese automotive enterprises. Establishing a reasonable energy management strategy, effectively controlling the vehicle working mode, and reasonably using battery energy for hybrid fuel-cell vehicles are core technologies in domestic and foreign automobile enterprises and research institutes. To improve the equilibrium between fuel-cell hydrogen consumption and battery consumption and realize the optimal energy distribution between fuel-cell systems and batteries for plug-in fuel-cell electric vehicles (PFCEVs), considering vehicles as the environment and vehicle control as an agent, an energy management strategy for the PFCEV based on reinforcement learning algorithm was proposed in this paper. This strategy considered the immediate return and future cumulative discounted returns of a fuel-cell vehicle's real-time energy allocation. The vehicle simulation model was built by Matlab/Simulink to carry out the simulation test for the proposed strategy. Compared with the rule-based strategy, the battery can store a certain amount of electricity, and the integrated energy consumption of the vehicle was notably reduced under different mileages. The energy consumption in 100 km was reduced by 8.84%, 29.5%, and 38.6% under 100, 200, and 300 km mileages, respectively. The hardware-in-loop-test was performed on the D2P development platform, and the final energy consumption of the vehicle was reduced by 20.8% under urban dynamometer driving schedule driving cycle. The hardware-in loop-test results are consistent with the simulation findings, indicating the effectiveness and feasibility of the proposed energy management strategy.
- fuel-cell vehicle,
- reinforcement learning,
- energy management,
- Q_learning algorithm,
- control strategy

FullText(HTML)

References(19)

References

[1]	徐梁飛, 華劍鋒, 包磊, 等. 燃料電池混合動力客車等效氫耗優化策略. 中國公路學報, 2009, 22(1): 104 doi: 10.3321/j.issn:1001-7372.2009.01.017 Xu L F, Hua J F, Bao L, et al. Optimized strategy on equivalent hydrogen consumption for fuel cell hybrid electric bus. China J Highway Transport, 2009, 22(1): 104 doi: 10.3321/j.issn:1001-7372.2009.01.017
[2]	Yun H T, Liu S D, Zhao Y L, et al. Energy management for fuel cell hybrid vehicles based on a stiffness coefficient model. Int J Hydrogen Energy, 2015, 40(1): 633 doi: 10.1016/j.ijhydene.2014.10.135
[3]	Oldenbroek V, Verhoef L A, Van Wijk A J M. Fuel cell electric vehicle as a power plant: Fully renewable integrated transport and energy system design and analysis for smart city areas. Int J Hydrogen Energy, 2017, 42(12): 8166 doi: 10.1016/j.ijhydene.2017.01.155
[4]	楊偉斌, 陳全世, 田光宇, 等. 插電式混合動力汽車換擋規律及轉矩分配策略. 機械工程學報, 2013, 49(14): 91 https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB201314015.htm Yang W B, Chen Q S, Tian G Y, et al. Shifting schedule and torque distribution strategy for the plug-in hybrid electric vehicle. J Mech Eng, 2013, 49(14): 91 https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB201314015.htm
[5]	Ahmadi L, Croiset E, Elkamel A, et al. Effect of socio-economic factors on EV/HEV/PHEV adoption rate in Ontario. Technol Forecast Social Change, 2015, 98: 93 doi: 10.1016/j.techfore.2015.06.012
[6]	Montazeri-Gh M, Mahmoodi-K M. An optimal energy management development for various configuration of plug-in and hybrid electric vehicle. J Cent South Univ, 2015, 22(5): 1737 doi: 10.1007/s11771-015-2692-6
[7]	秦大同, 趙新慶, 蘇嶺, 等. 插電式混合動力汽車變參數能量管理策略. 中國公路學報, 2015, 28(2): 112 doi: 10.3969/j.issn.1001-7372.2015.02.014 Qin D T, Zhao X Q, Su L, et al. Variable parameter energy management strategy for plug-in hybrid electric vehicle. China J Highway Transport, 2015, 28(2): 112 doi: 10.3969/j.issn.1001-7372.2015.02.014
[8]	Lin C C, Peng H, Jeon S, et al. Control of a hybrid electric truck based on driving pattern recognition//Proceedings of the 2002 Advanced Vehicle Control Conference. Hiroshima, 2002: 9
[9]	林歆悠, 孫冬野. 基于工況識別的混聯式混合動力客車控制策略研究. 中國機械工程, 2012, 23(7): 869 doi: 10.3969/j.issn.1004-132X.2012.07.023 Lin X Y, Sun D Y. Development of control strategy for a series-parallel hybrid electric city bus based on roadway type recognition. China Mech Eng, 2012, 23(7): 869 doi: 10.3969/j.issn.1004-132X.2012.07.023
[10]	朱元, 吳志紅, 田光宇, 等. 基于馬爾可夫決策理論的燃料電池混合動力汽車能量管理策略. 汽車工程, 2006, 28(9): 798 doi: 10.3321/j.issn:1000-680X.2006.09.003 Zhu Y, Wu Z H, Tian G Y, et al. An energy management strategy for fuel cell hybrid electric vehicle based on Markov decision process. Autom Eng, 2006, 28(9): 798 doi: 10.3321/j.issn:1000-680X.2006.09.003
[11]	石英喬, 何彬, 曹桂軍, 等. 燃料電池混合動力瞬時優化能量管理策略研究. 汽車工程, 2008, 30(1): 30 doi: 10.3321/j.issn:1000-680X.2008.01.007 Shi Y Q, He B, Cao G J, et al. A study on the energy management strategy for fuel cell electric vehicle based on instantaneous optimization. Autom Eng, 2008, 30(1): 30 doi: 10.3321/j.issn:1000-680X.2008.01.007
[12]	王欽普, 杜思宇, 李亮, 等. 基于粒子群算法的插電式混合動力客車實時策略. 機械工程學報, 2017, 53(4): 77 https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB201704012.htm Wang Q P, Du S Y, Li L, et al. Real-time energy management strategy for plug-in hybrid electric bus on particle swarm optimization algorithm. J Mech Eng, 2017, 53(4): 77 https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB201704012.htm
[13]	林歆悠, 馮其高, 張少博. 等效因子離散全局優化的等效燃油瞬時消耗最小策略能量管理策略. 機械工程學報, 2016, 52(20): 102 https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB201620014.htm Lin X Y, Feng Q G, Zhang S B. Global optimal discrete equivalent factor of equivalent fuel consumption minimization strategy based energy management strategy for a series-parallel plug-in hybrid electric vehicle. J Mech Eng, 2016, 52(20): 102 https://www.cnki.com.cn/Article/CJFDTOTAL-JXXB201620014.htm
[14]	Hu Z Y, Li J Q, Xu L F, et al. Multi-objective energy management optimization and parameter sizing for proton exchange membrane hybrid fuel cell vehicles. Energy Convers Manage, 2016, 129: 108 doi: 10.1016/j.enconman.2016.09.082
[15]	Xu L F, Mueller C D, Li J Q, et al. Multi-objective component sizing based on optimal energy management strategy of fuel cell electric vehicles. Appl Energy, 2015, 157: 664 doi: 10.1016/j.apenergy.2015.02.017
[16]	Han J H, Park Y, Kum D. Optimal adaptation of equivalent factor of equivalent consumption minimization strategy for fuel cell hybrid electric vehicles under active state inequality constraints. J Power Sources, 2014, 267: 491 doi: 10.1016/j.jpowsour.2014.05.067
[17]	Murgovski N, Johannesson L M, Sj?berg J. Engine on/off control for dimensioning hybrid electric powertrains via convex optimization. IEEE Trans Veh Technol, 2013, 62(7): 2949 doi: 10.1109/TVT.2013.2251920
[18]	Zheng C H, Oh C E, Park Y I, et al. Fuel economy evaluation of fuel cell hybrid vehicles based on equivalent fuel consumption. Int J Hydrogen Energy, 2012, 37(2): 1790 doi: 10.1016/j.ijhydene.2011.09.147
[19]	Huang Y J, Wang H, Khajepour A, et al. Model predictive control power management strategies for HEVs: a review. J Power Sources, 2017, 341: 91 doi: 10.1016/j.jpowsour.2016.11.106