Shin Yoo firstname.lastname@example.org Office: E3-1 Room 2405
This course is concerned with a broad range of software testing techniques, with a heavy emphasis on automation, tools, and frameworks, as well as the research outputs behind them. The topic will include, but are not limited to: black box testing/combinatorial testing, random testing, concepts of coverage, structural testing, mutation testing, regression testing, testability transformation, automated debugging, etc.
- Strong programming skills: you are required to actively contribute to group and individual project, which involves serious implementation. There will be also a number of hands-on sessions where we will program together during the class.
- Unix/Linux-savvy: you should be familiar with the usual build tools and Unix/Linux command line environments.
- Git-aware: you will be required to submit a github repository as part of your project deliverable.
- Ideally, CS350 Introduction to Software Engineering.
- Also, ideally, a laptop you can bring to the classroom. If this becomes a problem, let me know.
- Coursework: 30%
- Midterm Exam: 30%
- Project: 30%
- Class Participation: 10%
To be announced
We do not have a textbook per se, and the course will be based on slides and other reading material that are deemed appropriate. However, if you want to get broader sense for some of the topics dealt by this course, I recommend the following books and publications.
- Paul Ammann and Jeff Offutt. Introduction to Software Testing (2nd Ed.)
- Andreas Zeller. Why Programs Fail (2nd Ed.)
- Y. Jia and M. Harman. An analysis and survey of the development of mutation testing. IEEE transactions on software engineering, 37(5):649–678.
- P. McMinn. Search-based software test data generation: A survey. Software Testing, Verification and Reliability, 14(2):105–156, June 2004.
I have a few trips I am obliged to make in Spring 2019:
- IEEE International Conference on Software Testing, Verification and Validation (ICST 2019) (22-27 April), for which I am the Steering Committee Chair.
- ACM International Conference on Software Engineering (ICSE 2019) (25-31 May), for which our group is presenting a paper. I am also part of the organisation team for ICSE 2020, which is in Seoul.
- I am committed as an external reviewer for a PhD defence in University of Oslo, Norway, on 2nd April.
We will do our best to minimise the disruption to the learning experience. The following schedule are tentative, and may change.
- 26 Feb: Introduction & Testing Fundamentals
- 28 Feb: Black Box Testing & Combinatorial Interaction Testing
- 05 Mar: Testing Finite State Machines
- 07 Mar: Control and Data Flow
- 12 Mar: Random and Adaptive Random Testing
- 14 Mar: Search Based Test Data Generation
- 19 Mar: Mutation Testing
- PIT: a practical mutation testing tool for Java
- 21 Mar: Mutation Testing Hands-on with Code Defender
- 26 Mar: Regression Testing
- 28 Mar: Fault Localisation
- 02 Apr: No Lecture (Lecturer Away for a PhD Defense)
- 04 Apr: No Lecture (Lecturer Away for a PhD Defense)
- 09 Apr: Untestable Programs and Metamorphic Testing
- 12 Apr: Testing Deep Neural Networks
- Midterm Exam (exact date & time to be announced)
- 23 Apr: No Lecture (ICST 2019)
- 25 Apr: No Lecture (ICST 2019)
- 30 Apr: Web Testing Automation Hands-on
- 02 May: Random Testing & Delta Debugging Hands-on
- 07 May: Group Project Presentation
- 09 May: Paper Reading
- 14 May: Paper Reading
- 16 May: Paper Reading
- 21 May: Paper Reading
- 23 May: Paper Reading
- 28 May: No Lecture (ICSE 2019)
- 30 May: No Lecture (ICSE 2019)
- 04 Jun: Project Presentation
- 06 Jun: No Lecture (Memorial Day)
Team Project Deliverables
- A. Machiry, R. Tahiliani, and M. Naik. Dynodroid: An input generation system for android apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013, pages 224–234, New York, NY, USA, 2013. ACM.
- J. Zhang, J. Chen, D. Hao, Y. Xiong, B. Xie, L. Zhang, and H. Mei. Search-based inference of polynomial metamorphic relations. In Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering, ASE ’14, pages 701–712, New York, NY, USA, 2014. ACM.
- R. Abraham and M. Erwig. Mutation operators for spreadsheets. IEEE Transactions on Software Engineering, 35(1):94–108, 2009.
- S. Elbaum, G. Rothermel, and J. Penix. Techniques for improving regression testing in continuous integration development environments. In Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2014, pages 235–245, New York, NY, USA, 2014. ACM.
- M. Harman and P. McMinn. A theoretical and empirical analysis of evolutionary testing and hill climbing for structural test data generation. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA 2007), pages pp. 73–83. ACM Press, July 2007.
- J. Ding, X. Kang, and X. H. Hu. Validating a deep learning framework by metamorphic testing. In 2017 IEEE/ACM 2nd International Workshop on Metamorphic Testing (MET), pages 28–34, May 2017.
- Q. Zhu, A, Panichella, and A. Zaidman. An Investigation of Compression Techniques to Speed up Mutation Testing. In 2018 IEEE International Conference on Software Testinv, Validation, and Verification (ICST 2018), to appear.
- L. Mariani, C. Monni, M. Pezzè, O. Riganelli and R. Xin. Localizing Faults in Cloud Systems. In 2018 IEEE International Conference on Software Testinv, Validation, and Verification (ICST 2018), to appear.
- C. Timperley, A. Afzal, D. Katz, J. M. Hernandez and C. Le Goues. Crashing simulated planes is cheap: Can simulation detect robotics bugs early?. In 2018 IEEE International Conference on Software Testinv, Validation, and Verification (ICST 2018), to appear.
- J. Bell, O. Legunsen, M. Hilton, L. Eloussi, T. Yung, D. Marinov. DeFlaker: Automatically Detecting Flaky Tests. In 2018 International Conference on Software Engineering (ICSE 2018), to appear.