Wyniki wyszukiwania - BazTech

1

Design optimization of obstacle avoidance of intelligent building steel bar by integrating reinforcement learning and BIM technology

Chai Hong, Guo Junchao

Archives of Civil Engineering

|

2024

|

Vol. 70, nr 1

621--634

EN

In promoting the construction of prefabricated residential buildings in Yunnan villages and towns, the use of precast concrete elements is unstoppable. Due to the dense arrangement of steel bars at the joints of precast concrete elements, collisions are prone to occur, which can affect the stress of the components and even pose certain safety hazards for the entire construction project. Because the commonly used the steel bar obstacle avoidance method based on building information modeling has low adaptation rate and cannot change the trajectory of the steel bar to avoid collision, a multi-agent reinforcement learning-based model integrating building information modeling is proposed to solve the steel bar collision in reinforced concrete frame. The experimental results show that the probability of obstacle avoidance of the proposed model in three typical beam-column joints is 98.45%, 98.62% and 98.39% respectively, which is 5.16%, 12.81% and 17.50% higher than that of the building information modeling. In the collision-free path design of the same object, the research on the path design of different types of precast concrete elements takes about 3–4 minutes, which is far less than the time spent by experienced structural engineers on collision-free path modeling. The experimental results indicate that the model constructed by the research institute has good performance and has certain reference significance.

2

Predictive Modelling of a Honeypot System Based on a Markov Decision Process and a Partially Observable Markov Decision Process

Wang Lidong, Mosher Reed, Duett Patti, Falls Terril

Applied Cybersecurity & Internet Governance

|

2023

|

Vol. 2, No. 1

1--14

EN

A honeypot is used to attract and monitor attacker activities and capture valuable information that can be used to help practice good cybersecurity. Predictive modelling of a honeypot system based on a Markov decision process (MDP) and a partially observable Markov decision process (POMDP) is performed in this paper. Analyses over a finite planning horizon and an infinite planning horizon for a discounted MDP are respectively conducted. Four methods, including value iteration (VI), policy iteration (PI), linear programming (LP), and Q-learning, are used in the analyses over an infinite planning horizon for the discounted MDP. The results of the various methods are compared to evaluate the validity of the created MDP model and the parameters in the model. The optimal policy to maximise the total expected reward of the states of the honeypot system is achieved, based on the MDP model employed. In the modelling over an infinite planning horizon for the discounted POMDP of the honeypot system, the effects of the observation probability of receiving commands, the probability of attacking the honeypot, the probability of the honeypot being disclosed, and transition rewards on the total expected reward of the honeypot system are studied.

3

Markov Decision Process based Model for Performance Analysis an Intrusion Detection System in IoT Networks

Kalnoor Gauri, Gowrishankar

Journal of Telecommunications and Information Technology

|

2021

|

nr 3

42--49

EN

In this paper, a new reinforcement learning intrusion detection system is developed for IoT networks incorporated with WSNs. A research is carried out and the proposed model RL-IDS plot is shown, where the detection rate is improved. The outcome shows a decrease in false alarm rates and is compared with the current methodologies. Computational analysis is performed, and then the results are compared with the current methodologies, i.e. distributed denial of service (DDoS) attack. The performance of the network is estimated based on security and other metrics.

4

Piecewise Affine Dynamical Models of Petri Nets – Application to Emergency Call Centers

Allamigeon Xavier, Boyet Marin, Gaubert Stéphane

Fundamenta Informaticae

|

2021

|

Vol. 183, nr 3-4

169--201

EN

We study timed Petri nets, with preselection and priority routing. We represent the behavior of these systems by piecewise affine dynamical systems. We use tools from the theory of nonexpansive mappings to analyze these systems. We establish an equivalence theorem between priority-free fluid timed Petri nets and semi-Markov decision processes, from which we derive the convergence to a periodic regime and the polynomial-time computability of the throughput. More generally, we develop an approach inspired by tropical geometry, characterizing the congestion phases as the cells of a polyhedral complex. We illustrate these results by a current application to the performance evaluation of emergency call centers in the Paris area. We show that priorities can lead to a paradoxical behavior: in certain regimes, the throughput of the most prioritary task may not be an increasing function of the resources.

5

Control design for untimed Petri nets using Markov Decision Processes

Daoui C., Lefebvre D.

Operations Research and Decisions

|

2017

|

Vol. 27, No. 4

27--43

EN

Design of control sequences for discrete event systems (DESs) has been presented modelled by untimed Petri nets (PNs). PNs are well-known mathematical and graphical models that are widely used to describe distributed DESs, including choices, synchronizations and parallelisms. The domains of application include, but are not restricted to, manufacturing systems, computer science and transportation networks. We are motivated by the observation that such systems need to plan their production or services. The paper is more particularly concerned with control issues in uncertain environments when unexpected events occur or when control errors disturb the behaviour of the system. To deal with such uncertainties, a new approach based on discrete time Markov decision processes (MDPs) has been proposed that associates the modelling power of PNs with the planning power of MDPs. Finally, the simulation results illustrate the benefit of our method from the computational point of view.

6

Bottom-up learning of hierarchical models in a class of deterministic POMDP environments

Itoh H., Fukumoto H., Wakuya H., Furukawa T.

International Journal of Applied Mathematics and Computer Science

|

2015

|

Vol. 25, no. 3

597--615

EN

The theory of partially observable Markov decision processes (POMDPs) is a useful tool for developing various intelligent agents, and learning hierarchical POMDP models is one of the key approaches for building such agents when the environments of the agents are unknown and large. To learn hierarchical models, bottom-up learning methods in which learning takes place in a layer-by-layer manner from the lowest to the highest layer are already extensively used in some research fields such as hidden Markov models and neural networks. However, little attention has been paid to bottom-up approaches for learning POMDP models. In this paper, we present a novel bottom-up learning algorithm for hierarchical POMDP models and prove that, by using this algorithm, a perfect model (i.e., a model that can perfectly predict future observations) can be learned at least in a class of deterministic POMDP environments.

7

Optimal maintenance of a series production system with two multi-component subsystems and an intermediate buffer

Zhou Y., Zhang Z.

Eksploatacja i Niezawodność

|

2015

|

Vol. 17, no. 2

314--325

EN

Intermediate buffers often exist in practical production systems to reduce the influence of the breakdown and maintenance ef subsystems on system production. At the same time, the effects of intermediate buffers also make the degradation process of the system more difficult to model. Some existing papers investigate the performance evaluation and maintenance optimisation of a production system with intermediate buffers under a predetermined maintenance strategy structure. However, only few papers pay attention to the property of the optimal maintenance strategy structure. This paper develops a method based on the Markov decision process to identify the optimal maintenance strategy for a series-parallel system with two multi-component subsystems and an intermediate buffer. The structure of the obtained optimal maintenance strategy is analysed, which shows that the optimal strategy structure cannot be modelled by a limited number of parameters. However, some useful properties of the strategy structure are obtained, which can simplify the maintenance optimisation. Another interesting finding is that a large buffer capacity cannot always bring about high average revenue even through the cost of holding an item in the buffer is much smaller than the production revenue per item.

PL

W systemach produkcyjnych często stosuje się bufory pośrednie w celu zmniejszenia wpływu awarii i konserwacji podsystemów na system produkcji. Jednocześnie, oddziaływanie buforów pośrednich utrudnia modelowanie procesu degradacji systemu. Istnieją badania dotyczące oceny funkcjonowania i optymalizacji utrzymania systemów produkcyjnych wykorzystujących bufory pośrednie przy założeniu wcześniej określonej struktury strategii utrzymania ruchy. Jednak tylko nieliczne prace zwracają uwagę na własności optymalnej struktury strategii utrzymania ruchu. W przedstawionej pracy opracowano opartą na procesie decyzyjnym Markowa metodę określania optymalnej strategii utrzymania ruchu dla układu szeregowo-równoległego z dwoma podsystemami wieloskładnikowymi oraz buforem pośrednim. Przeanalizowano strukturę otrzymanej optymalnej strategii utrzymania i wykazano, że struktury takiej nie można zamodelować przy użyciu ograniczonej liczby parametrów. Jednak odkryto pewne przydatne właściwości struktury strategii, które mogą ułatwić optymalizację utrzymania ruchu. Innym interesującym odkryciem było to, że duża pojemność bufora nie zawsze daje wysoką średnią przychodów mimo iż koszty przechowywania obiektu w buforze są znacznie mniejsze niż przychody z produkcji w przeliczeniu na jeden obiekt.

8

An example of a technical object operation process model describing influence of engine damages on the operation process course

Landowski B.

Journal of Polish CIMAC

|

2014

|

Vol. 9, no 3

33--42

EN

The authors of this paper present, a method for building a model of technical objects operation whose sequence of successive operation states, their duration times, incomes and costs connected with the objects being in these states and the way a given state is reached depend not only on the object current state but also on other factors. The model describes, among others, influence of damages of the powertrain, being a compression-ignition internal combustion piston engine, of the analyzed object on the operation process of that object. The analysed technical object is a means of transport (a vehicle). A simplified computing model has been presented in order to illustrate the discussion. The model of operation process was built basing on the analysis of spaces of states and operation events concerning technical objects used in a real transportation system. In result of identification of the analyzed system and its multi-state process of technical objects operation, operation states and possible transitions between these states, significant for this research, have been determined. Sets of source data indispensable for the model assessment and its initial verification were obtained on the basis of experimental tests with the use of passive experiment method from a real research object. In order to perform mathematical modelling of the technical object operation process the Markov decision process was applied.

9

Example of applying markov decision process o model vehicle maintenance process

Landowski B.

Journal of KONES

|

2013

|

Vol. 20, No. 4

209--218

EN

The subject of this paper are selected issues connected with modelling, prognosis and control of the process of operation and maintenance of a certain category of technical objects. The research object is a system of transport means operation. Assumptions for building a model of the operation process state changes of objects used in a complex system of operation and maintenance have been presented. The operation process model was built on the basis of an analysis of the space of states and events concerning technical objects, which are used in the analysed real transportation system. Source data necessary to make assumptions for the model development and its initial verification, has been obtained by means of the passive experiment method from a real system of transport means operation. An example of Markov decision process use for modelling and analysis of a public city transportation system operation process has been presented. Determination of values of indices characterizing the analysed process has been carried out basing on computer simulation of Markov decision process, being a mathematical model of technical objects operation process. A computing model has illustrated the entire study. Values of the model parameters have been estimated basing on the results of preliminary tests conducted in a real system of transport means operation. The model presented in this work has been built in such a way that it can be used for a wide category of problems connected with technical objects operation.

10

Stochastyczny model obciążenia sieci dla sterowania obliczeniami rozproszonymi

Onderka Z.

Studia Informatica

|

2012

|

Vol. 33, nr 3A

79-90

PL

Artykuł prezentuje modele: pierwszy dla aplikacji typu manager-worker i dla heterogenicznej sieci komputerowej oraz drugi model opisujący dynamikę zmian obciążenia sieci komunikacyjnej. Dla tych modeli przedstawiono problem stochastycznego sterowania alokacją zadań. Sformułowano również problem MDP (aag. Markov Decision Control Problem) dła optymalnej alokacji zadań w sieci komputerowej.

EN

The paper preseated the model describing the dynamics of the background load for the corninunication network. The problem of stochastic control for the task allocation is formulated. Than the Markov Decision Control Problem of optimal task distribution in eomputer network is formulated. The idea of the open-loop and closed-loop control based on the stochastic forecast are presented.

11

Application of Markov decision process as a mathematical model of operation and maintenance process

Landowski B.

Zeszyty Naukowe / Akademia Morska w Szczecinie

|

2010

|

nr 24 (96)

12-16

EN

This paper presents an example of applying Markov decision process to model and analyse the bus operation and maintenance process within an urban transport system and to forecast the influence of the operation and maintenance strategies realised for the technical objects on the transport system behaviour. Setting the values of the indices describing the process under the analysis is performed on the basis of a computerised simulation of the Markov decision process, being a mathematical model of the technical objects operation and maintenance process. In order to simulate the operation and maintenance process (Markov decision process) and to evaluate the influence of the decisions being made on the course and effectiveness of the process being realised within the study object a simulation algorithm has been elaborated and a computer calculation program has been written. The model presented herein has been created in such a way to assure that it shall be possible to use it in as extensive as possible class of the problems regarding the operation and maintenance of the technical objects.

PL

W artykule przedstawiono przykład zastosowania decyzyjnego procesu Markowa do modelowania i analizy procesu eksploatacji autobusów komunikacji miejskiej oraz prognozowania wpływu realizowanych strategii eksploatacji obiektów technicznych na zachowanie się systemu transportowego. Wyznaczanie wartości wskaźników charakteryzujących analizowany proces realizowane jest na podstawie komputerowej symulacji decyzyjnego procesu Markowa, będącego matematycznym modelem procesu eksploatacji obiektów technicznych. W celu symulacji procesu eksploatacji (decyzyjnego procesu Markowa) i oceny wpływu podejmowanych decyzji na przebieg i efektywność procesu realizowanego w obiekcie badań opracowano algorytm symulacji i napisano komputerowy program obliczeniowy. Przedstawiony w pracy model został skonstruowany w taki sposób, by zapewnić możliwość jego wykorzystania w jak najszerszej klasie problemów związanych z eksploatacją obiektów technicznych.

12

Applying the Markov decision processes to model changes to the maintenance states of an object

Landowski B.

Journal of KONES

|

2010

|

Vol. 17, No. 3

263-270

EN

Operation and maintenance of technical objects is related to occurrence of various events, the effects of which affect the operation and maintenance process course, and particularly the courses of their use and service processes. Occurrence of those events may be of both determined and random nature. Assessing, analysing and forecasting the operation and maintenance process course, in complex technical systems, are connected with the problems of modelling the operation and maintenance processes of technical objects. Those processes are random ones which depend on one another. The paper presents an example of the Markov decision process to model changes of the analysed operation and maintenance states of technical objects. The investigation object is a real operation and maintenance system of urban transport buses in a middle-sized agglomeration (about 400 k residents). Supporting a decision maker, in the process of making decisions concerning performance of the operation and maintenance process of the means of transport, may be carried out by analysing the results of the investigations of the operation and maintenance process model. The investigations of that type are to determine values of the selected measures of technical and economic efficiency of the process being carried out for the estimated values of the model parameters. The values of the model parameters were estimated on the basis of the analysis of the results of the investigations performed in the analysed system of urban bus transport. A change to the values of the model parameters may reflect a change of influence of internal and external factors on behaviour of the system and the operation and maintenance process of the means of transport being carried out in it. Mathematical models of the operation and maintenance processes are intrinsically simplified, therefore practical conclusions resulting from investigating those models should be formulated carefully.

13

Wieloagentowe systemy decyzyjne

Kałuski J.

Zeszyty Naukowe. Organizacja i Zarządzanie / Politechnika Śląska

|

2009

|

z. 49

133-143

PL

W pracy przedstawiono definicje i podstawowe pojęcia Sieci Semantycznych Web oraz Wieloagentowych Systemów Informatycznych i Decyzyjnych. Omówiono ich genezę i rozwój. Pokazano dwa przykłady szczegółowe tych systemów.

EN

In the work some definitions and notions of the Semantic Web and Multi-Agent Systems are presented. The first part of the work contains some characteristic of concept and theory applied to semantic networks. The planning, grouping time and space, closing on semantic networks algorithm in two practical examples of One-and Three-Agent Systems are done.

14

Intelligent control system for HSM

Vallejo A., Morales-Menendez R., Elizalde-Siller H.

Journal of Automation Mobile Robotics and Intelligent Systems

|

2009

|

Vol. 3, No. 3

54-63

EN

Next-generation of High-Speed Machining (HSM) systems demand advanced features such as intelligent control under uncertainty. This requires, in turn, an efficient administration and optimization of all system's resources towards a previously identified objective. This work presents an optimization system based on Markov Decision Process (MDP), where an intelligent control guides the actions of the operator in peripheral milling processes. Early results suggest that MDP framework can cope with this application, yielding several benefits, which are discussed in detail. Future work will address the full integration of the developed optimization scheme within a commercial machining center.

15

Analityczna metoda wyznaczania niektórych macierzy zdyskontowanego decyzyjnego łańcucha Markowa

Kałuski J.

Zeszyty Naukowe. Organizacja i Zarządzanie / Politechnika Śląska

|

2000

|

z. 2

53-69

PL

W pracy przedstawiono analityczną metodę wyznaczania ergodycznej i różnicowych macierzy decyzyjnego łańcucha Markowa ze skończoną liczbą stanów z zadanym współczynnikiem dyskonta. Znany w literaturze wynik dla pełnych oczekiwanych dochodów takiego łańcucha zinterpretowano w aspekcie otrzymanych macierzy. Przedstawione wyniki dają możliwość wyróżnienia składowej stałej i zmiennej pełnego oczekiwanego dochodu. Metodę zilustrowano na dwóch prostych przykładach. W oparciu o zaprezentowaną metodę zaproponowano nowy wskaźnik jakości dla optymalizacji omawianego łańcucha Markowa.

EN

In the work the analytical method to calculate the ergodic and difference matrices of finite state discounted Markov decision processes is presented. On the basis well - known literature the result for overall discounted value, this one in interpretation of the calculated matrices is shown. The obtained results gives a possibility to distinguish the constant andvariable parts of the overall discounted value. The presented analitical method is illustrated by two simple examples. New performance index to discounted optimal Markov control problem is proposed.