Active - Passive: On Preconceptions of Testing

• Autorzy: Brzeziński K. M.
• Języki publikacji: EN

Abstrakty

EN

In telecommunications and software engineering, testing is normally understood to be essentially active: a tester is said to stimulate, control, and enforce. Passive testing does not fit this paradigm and thus remains the niche research subject, which bears on the scope and depth of the obtained results. It is argued that such limited understanding of testing is one of its many community-bound preconceptions. It may be acceptable in the current engineering approach to testing, but can and should be challenged in order to converge on the core concepts of the proposed science of testing ("testology"). This methodological work aims at establishing that there are no fundamental reasons for admitting the dominant role of the active element in testing. To show this, external (also extratechnical) areas are consulted for insight, direct observations, and metaphors. The troublesome distinction between (passive) testing and monitoring, as well as unclear relations between testing and measurements, are also addressed.

Słowa kluczowe

EN

behavior; development; metrology; monitoring; passive testing; reactive systems; scientific method

• Wydawca: Instytut Łączności - Państwowy Instytut Badawczy
• Czasopismo: Journal of Telecommunications and Information Technology
• Rocznik: 2011
• Tom: nr 3
• Strony: 63--73
• Opis fizyczny: Bibliogr. 57 poz., rys.

Twórcy

autor

Brzeziński K. M.

• Institute of Telecommunications, Warsaw University of Technology, Nowowiejska st 15/19, 00-665 Warsaw, Poland,

Bibliografia

• [1] K. M. Brzeziński, “On common meta-linguistic aspects of intrusion detection and testing”, Int. J. Inform Assurance and Secur. (JIAS), vol. 2, no. 3, pp. 167–178, 2007.
• [2] K. M. Brzeziński, “A joint meta-linguistic taxonomy of intrusion detection and testing/verification”, in Proc. 2nd Int. Worksh. Secure Informa. Syst. SIS’07, Wisła, Poland, 2007.
• [3] K. M. Brzeziński, “Towards the methodological harmonization of passive testing across ICT communities”, in Engineering the Computer Science and IT, S. Soomro, Ed. In-Tech, 2009, pp. 143–168.
• [4] K. M. Brzeziński, “On conceptual struggles over ‘testing’”, in Proc. XIV Poznańskie Warsztaty Telekomunikacyjne PWT 2010, Poznań, Poland, 2010.
• [5] K. M. Brzeziński, “Standards are signs”, in Proc. 15th EURAS Ann. Standardization Conf., Lausanne, Switzerland, 2010, pp. 43–60.
• [6] Software testing. Vocabulary. BS 7925-1. British Standards Institution, 1998.
• [7] Standard Glossary of Terms Used in Software Testing, version 2.0. ISTQB (Glossary Working Party), 2007.
• [8] Certified Tester. Foundation Level Syllabus. ISTQB, 2007.
• [9] Certified Tester. Advanced Level Syllabus. ISTQB, 2007.
• [10] A. Spillner, T. Linz, and H. Schaefer, Software Testing Foundations. Rocky Nook, 2007.
• [11] G. Bath and J. McKay, The Software Test Engineer’s Handbook. Rocky Nook, 2008.
• [12] C. F. Tschudin, “On the structuring of computer communications”, Ph.D. dissertation, University of Geneve, 1993.
• [13] VIM, International vocabulary of metrology – Basic and general concepts and associated terms. Joint Committee for Guides in Metrology (JCGM), 2008, vol. JCGM 200.
• [14] R. R. Hoffman, Metaphor in Science. Lawrence Erlbaum Associates, Inc., 1980, pp. 393–423.
• [15] S. Purao, C. Y. Baldwin, A. Hevner, V. C. Storey, J. Pries-Heje, B. Smith, and Y. Zhu, “The sciences of design: observations on an emerging field”, Harvard Business School, Working Paper 09-056, 2008.
• [16] K. Naik and P. Tripathy, Software Testing and Quality Assurance:Theory and Practice. Wiley, 2008.
• [17] L. Cacciari and O. Rafiq, “Controllability and observability in distributed testing”, Inform. Software Technol., vol. 41, no. 11-12,pp. 767–780, 1999.
• [18] C. Sunshine, “Formal techniques for protocol specification and verification”, Computer, vol. 12, no. 9, pp. 20–27, 1979.
• [19] IEEE Standard Glossary of Software Engineering Terminology. IEEE Std 610-12. IEEE, 1990.
• [20] Guide to the Software Engineering Body of Knowledge. SWEBOK, IEEE, 2004.
• [21] L. Frantzen and J. Tretmans, “Model-based testing of environmental conformance of components”, in Formal Methods of Components and Objects (FMCO’06), LNCS 4709. Springer, 2007, pp. 1–25.
• [22] E. Brinksma and J. Tretmans, “Testing transition systems: an annotated bibliography”, in Proc. MOVEP 2000, Nantes, France, 2000, pp. 187–195.
• [23] E. Brinksma, W. Grieskamp, and J. Tretmans, “Summary”, in Perspectives of Model-Based Testing of Dagstuhl Seminar Proceedings, E. Brinksma, W. Grieskamp, and J. Tretmans, Eds., no. 04371. IBFI, 2005.
• [24] M. Utting, A. Pretschner, and B. Legeard, “A taxonomy of model-based testing”, Univ. of Waikato, Hamilton, New Zealand, Working Paper 04/2006, 2006.
• [25] J. Ryser, S. Berner, and M. Glinz, “On the state of the art in requirements-based validation and test of software”, Tech. Rep. IFI-98.12, Univ. of Zurich, May 1998.
• [26] Conformance Testing Methodology and Framework. ISO/IEC 9646. ISO/IEC, n.d., vol. 1–7.
• [27] MTS; The Testing and Test Control Notation version 3. ETSI ES 201873, ETSI, n.d.
• [28] K. M. Brzeziński, “Intrusion detection as passive testing: linguistic support with TTCN-3”, in DIMVA, LNCS 4579. Lucerne: Springer, 2007, pp. 79–88.
• [29] K. M. Brzeziński, D. Mastalerz, and R. Artych, “Practical support of testing activities: the PMM Family”. COST 247 WG3 Internal Report, IT P.W., LTiV Tech. Rep. 965, 1996.
• [30] K. M. Brzeziński, “Weryfikacja i testowanie”, Przegląd Telekomunikacyjny i Wiadomości Telekomunikacyjne, no. 4, pp. 139–140, 2010 (in Polish).
• [31] K. M. Brzeziński, “Testowanie w cyklu życia systemu: nieregularności meta-standaryzacji”, in Krajowe Sympozjum Telekomunikacji i Teleinformatyki (KSTiT), Warszawa, 2009 (in Polish).
• [32] M.-C. Gaudel, “Formal methods and testing: hypotheses, and correctness approximation”, Formal Methods, pp. 2–8, 2005.
• [33] K. M. Brzeziński, A. Gumieniak, and P. Jankowski, “Passive testing for reverse engineering: specification recovery”, in Proc. IASTED Int. Conf. Paral. Distrib. Comput. Netw. PDCN 2008, Innsbruck, Austria, 2008, pp. 27–32.
• [34] K. Popper, Conjectures and Refutations: The Growth of Scientific Knowledge. London: Routledge, 1963.
• [35] L. Heerink and E. Brinksma, “Validation in context”, in Proc. 15th IFIP Int. Symp. Protocol Specification, Testing and Verification PSTV 1995, Warsaw, Poland, 1995, pp. 221–236.
• [36] Framework on Formal Methods in Conformance Testing. ITU-T Z500. ITU-T, May 1997.
• [37] M. Heidelberger, “Experimentation and instrumentation”, in Encyclopedia of Philosophy, D. M. Borchert, Ed. Thomson Gale, 2006, vol. 10, pp. 12–20.
• [38] J. S. Mill, Of Observation and Experiment. Routledge and Kegan Paul, 1974.
• [39] J. A. Arnedo, A. R. Cavalli, and M. N´u˜nez, “Fast testing of critical properties through passive testing”, in Proc. IFIP Int. Conf. Testing Commun. Syst. TestCom 2003, Sophia Antipolis, France, 2003, pp. 295–310.
• [40] R. Dssouli and R. Fournier, “Communication software testability”, in Protocol Test Systems III. North-Holland, 1991, pp. 45–55.
• [41] K. Svozil, “Extrinsic-Intrinsic Concept and Complementarity”, in Inside Versus Outside, H. Atmanspacher and G. J. Dalenoort, Eds. Berlin: Springer, 1994, pp. 273–288.
• [42] P. G. Neumann, “Cause of AT&T network failure”, Risks Dig., vol. 9, no. 62, 1990.
• [43] M. Boahene, “Information systems development methodologies: are you being served?” in Proc. 10th Australasian Conf. Information Syst., Wellington, New Zealand, 1999, pp. 88–99.
• [44] Webster’s Encyclopedic Unabridged Dictionary of the English Language. Gramercy Books, 1996.
• [45] R. Wvong, “A new methodology for OSI conformance testing based on trace analysis”, Master’s thesis, University of British Columbia, 1990.
• [46] G. von Bochmann and O. B. Bellal, “Test result analysis with respect to formal specifications”, in Proc. 2nd. Int. Worksh. Protocol Test Syst., Berlin, Germany, 1989, pp. 272–294.
• [47] D. Lee, A. N. Netravali, K. K. Sabnani, B. Sugla, and A. John, “Passive testing and applications to network management”, in Int. Conf. Netw. Protoc. ICNP’97, Atlanta, USA, 1997, pp. 113–122.
• [48] J. Tretmans, “Testing concurrent systems: a formal approach”, in 10th Int. Conf. CONCUR’99, Eindhoven, The Netherlands, 1999, pp. 46–65.
• [49] K. M. Brzeziński, “Towards Practical Passive Testing”, in Proc. IASTED Int. Conf. Paral. Distrib. Comput. Netw. PDCN 2005, Innsbruck, Austria, 2005, pp. 177–183.
• [50] M. Dilman and D. Raz, “Efficient reactive monitoring”, IEEE J. Selec. Areas Commun., vol. 20, no. 4, pp. 668–676, 2002.
• [51] L. Mari, “The role of determination and assignment in measurement”, Measurement, vol. 21, no. 3, pp. 79–90, 1997.
• [52] Evaluation of Measurement Data – Guide to the Expression of Uncertainty in Measurement. GUM. Joint Committee for Guides in Metrology (JCGM), 2008, vol. JCGM 100.
• [53] “Metrology for Information Technology (IT)”. NISTIR 6025. NIST, White paper, 1997.
• [54] Conformity Assessment – Vocabulary and General Principles. ISO/IEC 17000, ISO/IEC, 2004.
• [55] A. C. Baratto, “Measurand: a cornerstone concept in metrology”, Metrologia, vol. 45, pp. 299–307, 2008.
• [56] R. M. Olejnik, “Kod pocztowy jako przykład metrologicznej skali nominalnej”, Pomiary Automatyka Robotyka, no. 7–8, pp. 186–188, 2004 (in Polish).
• [57] T. S. E. Maibaum, “The epistemology of validation and verification testing”, in Proc. 17th IFIP Int. Conf. TestCom 2005, Montreal, Canada, 2005, pp. 1–8.