Nowadays the equipment used on ships is becoming more and more sophisticated, safety of shipping depends heavily not only on trustworthiness of high-tech products, but also on mariners’ competence. It is necessary to find methods to enable seafarers to keep track with technical developments. Simulators as a tool combined with a properly developed course curriculum and qualified instructors provide an appropriate method not only for training, but also for measuring, assessing and evaluating individual mariner performance in order to test levels of competency and proficiency. According to Nieri (1995, p. 1/6), “The development of performance-based assessment tools would utilize the widely-recognized advantages of simulators and part-task trainers to generate a scenario in which the license candidate must demonstrate his knowledge of system relationships, knowledge of operational procedures, monitor systems and situations, respond to unexpected occurrences, identify and diagnose problems, and manage personnel, in a dynamic world“. This paper discusses the usage of new capabilities of the NTPRO 4000 full mission ship handling simulator in the assessment and evaluation processes at Lithuanian Maritime Academy.