A Deep Q-Learning Network for ship stowage planning problem

• Autorzy: Shen Y. , Zhao N. , Xia M. , Du X.

Identyfikatory

DOI

10.1515/pomr-2017-0111

• Języki publikacji: EN

Abstrakty

EN

Ship stowage plan is the management connection of quae crane scheduling and yard crane scheduling. The quality of ship stowage plan affects the productivity greatly. Previous studies mainly focuses on solving stowage planning problem with online searching algorithm, efficiency of which is significantly affected by case size. In this study, a Deep Q-Learning Network (DQN) is proposed to solve ship stowage planning problem. With DQN, massive calculation and training is done in pre-training stage, while in application stage stowage plan can be made in seconds. To formulate network input, decision factors are analyzed to compose feature vector of stowage plan. States subject to constraints, available action and reward function of Q-value are designed. With these information and design, an 8-layer DQN is formulated with an evaluation function of mean square error is composed to learn stowage planning. At the end of this study, several production cases are solved with proposed DQN to validate the effectiveness and generalization ability. Result shows a good availability of DQN to solve ship stowage planning problem.

Słowa kluczowe

EN

Deep Q-Leaning Network (DQN); container terminal; ship stowage plan; markov decision process; value function approximation; generalization

• Wydawca: Gdańsk University of Technology, Faculty of Ocean Engineering & Ship Technology
• Czasopismo: Polish Maritime Research
• Rocznik: 2017
• Tom: S 3
• Strony: 102--109
• Opis fizyczny: Bibliogr. 25 poz., rys., tab.

Twórcy

autor

Shen Y.

• Scientific Research Academy, Shanghai Maritime University, China

autor

Zhao N.

• Logistics Engineering College Shanghai Maritime University Shanghai201306 China

autor

Xia M.

• Scientific Research Academy, Shanghai Maritime University, China

autor

Du X.

• Logistics Engineering College Shanghai Maritime University Shanghai201306 China

Bibliografia

• 1. M. Omar, S. S. Supadi. 2012. Integrated models for shipping a vendor’s final production batch to a single buyer under linearly decreasing demand for consignment policy. Sains Malaysiana 41.3: 367-370.
• 2. C. Mi, Z. W. Zhang, Y. F. Huang, Y. Shen, 2013. A fast automated vision system for container corner casting recognition. Journal of Marine Science and TechnologyTaiwan, 24(1): 54-60. DOI: 10.6119/JMST-016-0125-8
• 3. X. P. Rui, X. T. Yu, J. Lu, et al. 2016. An algorithm for generation of DEMs from contour lines considering geomorphic features. Earth Sciences Research Journal, 20(2): G1-G9, 20(2):G1-G9.
• 4. Y. Shen, 2016. An Anti-Collision Method of Slip Barrel for Automatic Ship Loading in Bulk Terminal. Polish Maritime Research, 23(s1).
• 5. C. Mi, Y. Shen, W. J. Mi, Y. F. Huang, 2015. Ship Identification Algorithm Based on 3D Point Cloud for Automated Ship Loaders. Journal of Coastal Research, 2015(SI.73): 28-34. DOI: 10.2112/SI73-006.
• 6. C. Mi, Z. W. Zhang, X. He, Y. F. Huang, W. J. Mi, 2015. Two-stage classification approach for human detection in camera video in bulk ports, Polish Maritime Research, 22(SI.1):163-170. DOI: 10.1515/pomr-2015-0049
• 7. C. Mi, H. W. Liu, Y. F. Huang, W. J. Mi, Y. Shen, 2016. Fatigue alarm systems for port machine operators. Asia Life Sciences, 25(1): 31-41.
• 8. Yifan S, Ning Z, Weijian M. 2016. Group-Bay Stowage Planning Problem for Container Ship. Polish Maritime Research, 23(s1).
• 9. Mengjue X., Ning Z, Weijian M. 2016. Storage Allocation in Automated Container Terminals: the Upper Level. Polish Maritime Research, 23(s1).
• 10. C. Mi, X. He, H. W. Liu, Y. F. Huang, W. J. Mi, 2014. Research on a Fast Human-Detection Algorithm for Unmanned Surveillance Area in Bulk Ports. Mathematical Problems in Engineering. DOI: 10.1155/2014/386764
• 11. D. S. Todd, P. Sen, 1997. A Multiple Criteria Genetic Algorithm for Containership Loading International Conference on Genetic Algorithms, East Lansing, Mi, Usa, July. DBLP, 674-681.
• 12. N. Zhao, W. J. Mi, 2008. Robust approach in stowage planning at contianer terminals. IEEE proceeding of the 4th International Conference on Intelligent Logistic System, 191-204.
• 13. A. Moura, J. Oliveira, C. Pimentel, 2013. A Mathematical Model for the Container Stowage and Ship Routing Problem. Journal of Mathematical Modelling and Algorithms in Operations Research, 12(3): 217-231.
• 14. M. Avriel, M. Penn, 1993. Exact and approximate solutions of the container ship stowage problem. Computers & Industrial Engineering, 25(1-4):271-274.
• 15. J. J. Shields, 1984. Containership Stowage: A ComputerAided Preplanning System. Marine Technology, 21(4): 370-383.
• 16. A. Imai, T. Miki, 1989. A heuristic algorithm with expected utility for an optimal sequence of loading containers into a containerized ship. Journal of Japan Institute of Navigation, 80: 117–124 (in Japanese).
• 17. A. Imai, E. Nishimura, K. Sasaki, S. Papadimitriou, 2001. Solution comparisons of algorithms for the containership loading problem. Proceedings of the International Conference on Shipping: Technology and Environment, available on CD-ROM.
• 18. A. Imai, E. Nishimura, K. Sasaki, S. Papadimitriou, 2001. Solution comparisons of algorithms for the containership loading problem. Proceedings of the International Conference on Shipping: Technology and Environment, available on CD-ROM.
• 19. A. Haghani, E. I. Kaisar, 2001. A model for designing container loading plans for containerships. In: 80th Transportation Research Board Annual Meeting, Washington, DC, USA.
• 20. J. F. Álvarez, 2006. A heuristic for Vessel planning in a reach stacker terminal. Journal of Maritime Research Jmr, 3(1): págs. 3-16.
• 21. K. H. Kim, 1994. Analysis of rehandles of transfer crane in a container yard. APORS-Conference, 3: 357-365.
• 22. K. H. Kim. 1997. Evaluation of the number of rehandles in container yards. Computers & Industrial Engineering, 32: 701–711.
• 23. K. H. Kim, Y. M. Park, K. R. Ryu, 2000. Deriving decision rules to locate export containers in container yards. European Journal of Operational Research, 124: 89-101.
• 24. K. H. Kim, J. S. Kang, K. R. Ryu, 200. A beam search algorithm for the load sequencing of outbound containers in port container terminals. OR Spectrum, 26: 93-116.
• 25. Y. Lee, J. Kang, K. R. Ryu, K. H. Kim, 2005. Optimization of Container Load Sequencing by a Hybrid of Ant Colony Optimization and Tabu Search, Natural Computation Lecture Notes in Computer Science, 3611, 1259-1268.

Uwagi

PL

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2018).