Bi-rotary milling head is the primary component of multiple-axis machine tool toward the multiply machining operation. The machining performance is greatly related to the structure characteristics and positioning precisions of the swivel head. This study was aimed at developing a bi-rotary milling head module, which is composed of a direct drive motor, cross roller bearings and motorized spindle unit. In order to evaluate the machining stability at the design phase, the dynamic characteristics of the rotary milling were first analyzed with finite element method. Especially, the variations of the dynamic characteristics of the spindle tool with the changing of the titling configuration of swivel axis were examined. In order to consider the accurate presentation of a spindle tool system and swivel mechanism, the bearings in the rolling components were also included in the finite element model and simulated with surface contact elements with adequate contact stiffness. The dynamic frequency response function of the spindle tool at different swinging positions were predicted for comparisons, which were further used to calculate the machining stability based on the machining mechanics. The current results show that the feeding direction and swinging positions of rotary milling head have a significant influence on the dynamic characteristics and machining ability of the spindle tool. The variations of the cutting depth with the swinging of A axis fall in the range of 11% to 40%, depending on the feeding direction and swinging angle. The analysis results are expected to clearly demonstrate the variation of the machining performance of the spindle tool under different milling configurations. The devised model and modeling approach can be applied to develop a five axis milling machine with desired dynamic and machining performance.