With traveling professions and nomadic lifestyles on the increase there is a growing need for comprehensive mobile networking within the Next Generation Network, and the security, reliability and technical independency of the Short Message Service (SMS) is becoming more and more indispensable for business processes. Since competitiveness goes hand-in-hand with this business capability, and methods to determine the quality of this service are becoming equally indispensable. This paper presents possible methods to evaluate the “Completion Rate of SMS” and “End-to-End Delivery Time” and combinations of them (new metric in this paper). A software tool, QoSCalc(SMS), has been developed in collaboration with the authors in order to verify whether service providers are abiding by the Service Level Agreements. The tool has been tested in a real measurement environment and the results are presented graphically here and interpreted.