A correctly estimated component of fetal heart rate signal (FHR) – so called baseline – is a precondition for proper recognition of acceleration and deceleration patterns. A number of various algorithms for estimating the FHR baseline was proposed so far. However, there is no reference standard enabling their objective evaluation, and thus no methodology of comparing the different algorithms still exists. In this paper we propose a method for evaluation of automatically determined baseline in reference to a set of experts, based on ten separate groups of signals comprising typical variability patterns observed in the fetal heart rate. As it was proposed earlier , the given algorithm is evaluated based on the characteristic patterns detected using the obtained baseline, instead of direct analysis of the baseline shape. For the purpose of quantitative assessment of the estimated baseline a new synthetic inconsistency coefficient was applied. The proposed methodology enabled to evaluate eleven well-known algorithms. We believe that the method will be a valuable tool for assessment of the existing algorithms, as well as for developing the new ones.