diff --git a/Assignments/A4/Assig4.pdf b/Assignments/A4/Assig4.pdf new file mode 100644 index 0000000000000000000000000000000000000000..f77f47bb1b2e6e70f5da3b844e602816df79174b Binary files /dev/null and b/Assignments/A4/Assig4.pdf differ diff --git a/Assignments/A4/Assig4.tex b/Assignments/A4/Assig4.tex new file mode 100644 index 0000000000000000000000000000000000000000..9073aadec368ff3202ac7dc9913825479163fa6a --- /dev/null +++ b/Assignments/A4/Assig4.tex @@ -0,0 +1,118 @@ +\documentclass[12pt]{article} + +\usepackage{graphicx} +\usepackage{paralist} +\usepackage{amsfonts} +\usepackage{hyperref} + +\oddsidemargin 0mm +\evensidemargin 0mm +\textwidth 160mm +\textheight 200mm +\renewcommand\baselinestretch{1.0} + +\pagestyle {plain} +\pagenumbering{arabic} + +\newcounter{stepnum} + +\title{Assignment 4} +\author{COMP SCI 2ME3, SFWR ENG 2AA4} + +\begin {document} + +\maketitle + +The purpose of this assignment is to design and specify a module (or modules) to +store the state of a game of Battleship. The modules cover the Model portion of +the Model View Controller design pattern. The rules for Battleship can be found +at the following web-page: + +~\newline +\href +{https://www.thespruce.com/how-to-play-battleship-411069}{https://www.thespruce.com/how-to-play-battleship-411069} +~\newline + +Your assignment is to write a module that stores the state of the game board and +the status of the game. You do not need to worry about modules that display +graphics, or control the game play, or determine the strategies of a computer +opponent, etc. + +Bonus marks are available for students that both implement and test their specification. + +\section *{Deadlines} + +\begin{itemize} +\item Specification: due 11:59 pm Apr 3 +\item Code and Test Suite (Bonus): due 11:59 pm Apr 3 +\end{itemize} + +\section *{Step \refstepcounter{stepnum} \thestepnum} + +Submit a report showing your design specification of the MIS for the game state +module. If your specification requires additional modules, you should include +their MISes as well. It is up to you to determine your modules interface; that +is, you decide on the exported constants, access programs, exceptions etc. You +also determine your state variables and specify the semantics for your access +program calls. When working on your design, you may want to consider the +following points: + +\begin{itemize} + +\item You probably want a game state for each of the two players. This would + make the module an abstract data type, not an abstract object. +\item The state of the game board with the battleships could be modelled as a two dimensional + sequence. +\item A separate board could be kept for each player to track the shots they + have taken. +\item You will need a routine to initialize the board. +\item You will need to be able to tell if the placement of a ship in the initial + configuration is valid or not. +\item You will need to be able to determine whether a shot is valid or not. +\item You will need to be able to inspect the state of any cell of the game + board. +\item You will need to be able to tell when a ship has been sunk. +\item You need to be able to tell when the game is over. +\end{itemize} + +\section *{Bonus Step \refstepcounter{stepnum} \thestepnum} + +Submit Java or Python code that matches the specification given in the previous +step. You should also submit code that tests your module(s) using PyUnit or +JUnit. You are free to document your source code as you see fit. That is, you +can use doxygen, javadoc or pydoc, or you can use regular code comments. You +code should include a makefile, with rules \texttt{make test} and \texttt{make + doc} (if appropriate). + +\noindent Your submission of code and test cases should be to our git repo. + +\subsubsection*{Notes} + +\begin{enumerate} +\item Your git repo is organized with the following directories at the + top level: {\tt A1}, {\tt A2}, {\tt A3}, and {\tt A4}. +\item Inside the {\tt A4} folder you will start with initial stubs of the files + and folders that you need to use. Please do not change the names or locations + of any of these files or folders. The structure of your project files and + folders should look like this: +\begin{itemize} +\item A4 +\begin{itemize} +\item[$*$] Makefile +\item report +\begin{itemize} +\item report.tex +\item report.pdf +\end{itemize} +\item src +\end{itemize} +\end{itemize} +\item Please put your name and macid at the top of each of your source + files. +\item Your program must work in the ITB labs on mills. +\item \textbf{Any changes to the assignment specification will be announced in + class. It is your responsibility to be aware of these changes. Please + monitor all pushes to the course git repo.} +\end{enumerate} + +\end {document} \ No newline at end of file diff --git a/Assignments/A4/MarkingScheme.xlsx b/Assignments/A4/MarkingScheme.xlsx new file mode 100644 index 0000000000000000000000000000000000000000..e869d2825845602a0f2d9eb7c7c6a623b72ec2ac Binary files /dev/null and b/Assignments/A4/MarkingScheme.xlsx differ diff --git a/Lectures/Figures/CompleteCoveragePrinciple.png b/Lectures/Figures/CompleteCoveragePrinciple.png new file mode 100644 index 0000000000000000000000000000000000000000..e3fcbbd3e4c1122f6f57cbeccfc6da854290f6b8 Binary files /dev/null and b/Lectures/Figures/CompleteCoveragePrinciple.png differ diff --git a/Lectures/Figures/CompleteCoveragePrinciple.ppt b/Lectures/Figures/CompleteCoveragePrinciple.ppt new file mode 100644 index 0000000000000000000000000000000000000000..2c0d43f5f37027a0b137922ff3c541793575e2a5 Binary files /dev/null and b/Lectures/Figures/CompleteCoveragePrinciple.ppt differ diff --git a/Lectures/Figures/ControlGraphRules.png b/Lectures/Figures/ControlGraphRules.png new file mode 100644 index 0000000000000000000000000000000000000000..d159c6e12a7f57123181b3ac548dcfa2e1d1f04c Binary files /dev/null and b/Lectures/Figures/ControlGraphRules.png differ diff --git a/Lectures/Figures/ControlGraphRules.ppt b/Lectures/Figures/ControlGraphRules.ppt new file mode 100644 index 0000000000000000000000000000000000000000..04f03306a2355a1a186cf058a8423fd07cfd8031 Binary files /dev/null and b/Lectures/Figures/ControlGraphRules.ppt differ diff --git a/Lectures/Figures/EuclidsAlgorithm.png b/Lectures/Figures/EuclidsAlgorithm.png new file mode 100644 index 0000000000000000000000000000000000000000..84fe1f286f6dcbf619f1d32ef91cb032fba4711f Binary files /dev/null and b/Lectures/Figures/EuclidsAlgorithm.png differ diff --git a/Lectures/Figures/EuclidsAlgorithm.ppt b/Lectures/Figures/EuclidsAlgorithm.ppt new file mode 100644 index 0000000000000000000000000000000000000000..05ce624c9f4cfea5501c78ae7000062981be3d8d Binary files /dev/null and b/Lectures/Figures/EuclidsAlgorithm.ppt differ diff --git a/Lectures/Figures/Example.png b/Lectures/Figures/Example.png new file mode 100644 index 0000000000000000000000000000000000000000..8e4b0bf8737b4395c3498b8963bbb994b51d4fa2 Binary files /dev/null and b/Lectures/Figures/Example.png differ diff --git a/Lectures/Figures/Example.ppt b/Lectures/Figures/Example.ppt new file mode 100644 index 0000000000000000000000000000000000000000..04379a529ad122ad61b744fcb9903a327998a500 Binary files /dev/null and b/Lectures/Figures/Example.ppt differ diff --git a/Lectures/Figures/ExampleStatementCoverage.png b/Lectures/Figures/ExampleStatementCoverage.png new file mode 100644 index 0000000000000000000000000000000000000000..98b8bf23d646858d2101a9193125651924d92eaa Binary files /dev/null and b/Lectures/Figures/ExampleStatementCoverage.png differ diff --git a/Lectures/Figures/ExampleStatementCoverage.ppt b/Lectures/Figures/ExampleStatementCoverage.ppt new file mode 100644 index 0000000000000000000000000000000000000000..46511fa1c1714bfcdab060b5caf39fd2fe7b16d3 Binary files /dev/null and b/Lectures/Figures/ExampleStatementCoverage.ppt differ diff --git a/Lectures/Figures/FunctionTableTesting.png b/Lectures/Figures/FunctionTableTesting.png new file mode 100644 index 0000000000000000000000000000000000000000..af1f93161f737479ded3afdcb7ab2f04b67229a2 Binary files /dev/null and b/Lectures/Figures/FunctionTableTesting.png differ diff --git a/Lectures/Figures/FunctionTableTesting.ppt b/Lectures/Figures/FunctionTableTesting.ppt new file mode 100644 index 0000000000000000000000000000000000000000..9d7bbd557b1ac94f6519ffa4d4f8daede9671d91 Binary files /dev/null and b/Lectures/Figures/FunctionTableTesting.ppt differ diff --git a/Lectures/Figures/Inheritance.png b/Lectures/Figures/Inheritance.png new file mode 100644 index 0000000000000000000000000000000000000000..e5defc036ec86f2e83a0527b468a491fef21f7e4 Binary files /dev/null and b/Lectures/Figures/Inheritance.png differ diff --git a/Lectures/Figures/Inheritance.ppt b/Lectures/Figures/Inheritance.ppt new file mode 100644 index 0000000000000000000000000000000000000000..45964a3a50206bcf01b9e51155e6de340b77a007 Binary files /dev/null and b/Lectures/Figures/Inheritance.ppt differ diff --git a/Lectures/Figures/Simplification.png b/Lectures/Figures/Simplification.png new file mode 100644 index 0000000000000000000000000000000000000000..19c715fed88dee10790ced6aef0c2166a2065f46 Binary files /dev/null and b/Lectures/Figures/Simplification.png differ diff --git a/Lectures/Figures/Simplification.ppt b/Lectures/Figures/Simplification.ppt new file mode 100644 index 0000000000000000000000000000000000000000..44e71fa8996ea8cf008563ecdf9a9fe533062315 Binary files /dev/null and b/Lectures/Figures/Simplification.ppt differ diff --git a/Lectures/Figures/TestingAFunctModule.png b/Lectures/Figures/TestingAFunctModule.png new file mode 100644 index 0000000000000000000000000000000000000000..ed63241fc0ac3a43051d1010612f3a05377cb6db Binary files /dev/null and b/Lectures/Figures/TestingAFunctModule.png differ diff --git a/Lectures/Figures/TestingAFunctModule.ppt b/Lectures/Figures/TestingAFunctModule.ppt new file mode 100644 index 0000000000000000000000000000000000000000..b252d7db86376417b0325f9e7e08bd35dd3ff544 Binary files /dev/null and b/Lectures/Figures/TestingAFunctModule.ppt differ diff --git a/Lectures/Figures/UsesRelation.png b/Lectures/Figures/UsesRelation.png new file mode 100644 index 0000000000000000000000000000000000000000..1290e0d073d167454fcd2574d465d57941bb9ae3 Binary files /dev/null and b/Lectures/Figures/UsesRelation.png differ diff --git a/Lectures/Figures/UsesRelation.ppt b/Lectures/Figures/UsesRelation.ppt new file mode 100644 index 0000000000000000000000000000000000000000..bc6c45b5258ed401105d6d20b92613516666e012 Binary files /dev/null and b/Lectures/Figures/UsesRelation.ppt differ diff --git a/Lectures/Figures/WeaknessConditionCoverage.png b/Lectures/Figures/WeaknessConditionCoverage.png new file mode 100644 index 0000000000000000000000000000000000000000..d363baf43f9fe04cfd84d78f131fff7586e550bf Binary files /dev/null and b/Lectures/Figures/WeaknessConditionCoverage.png differ diff --git a/Lectures/Figures/WeaknessConditionCoverage.ppt b/Lectures/Figures/WeaknessConditionCoverage.ppt new file mode 100644 index 0000000000000000000000000000000000000000..1b91b98fbe8db6244354e4df719fa949a24913dc Binary files /dev/null and b/Lectures/Figures/WeaknessConditionCoverage.ppt differ diff --git a/Lectures/Figures/WeaknessEdgeCoverage.png b/Lectures/Figures/WeaknessEdgeCoverage.png new file mode 100644 index 0000000000000000000000000000000000000000..54121b5f80ebe6c3f024922106ae16b5f7dfa142 Binary files /dev/null and b/Lectures/Figures/WeaknessEdgeCoverage.png differ diff --git a/Lectures/Figures/WeaknessEdgeCoverage.ppt b/Lectures/Figures/WeaknessEdgeCoverage.ppt new file mode 100644 index 0000000000000000000000000000000000000000..db34e8432608201bff5b43fb454ba3003dc0a0f8 Binary files /dev/null and b/Lectures/Figures/WeaknessEdgeCoverage.ppt differ diff --git a/Lectures/Figures/WeaknessStatementCoverage.png b/Lectures/Figures/WeaknessStatementCoverage.png new file mode 100644 index 0000000000000000000000000000000000000000..b0a85fdec068da6b9fa66dff5d6277bbf7946e02 Binary files /dev/null and b/Lectures/Figures/WeaknessStatementCoverage.png differ diff --git a/Lectures/Figures/WeaknessStatementCoverage.ppt b/Lectures/Figures/WeaknessStatementCoverage.ppt new file mode 100644 index 0000000000000000000000000000000000000000..905e6dfeea8d0bba9a401b51616d6bfecce644eb Binary files /dev/null and b/Lectures/Figures/WeaknessStatementCoverage.ppt differ diff --git a/Lectures/L29_IntroductionToVerification/IntroductionToVerification.pdf b/Lectures/L29_IntroductionToVerification/IntroductionToVerification.pdf index 2e0279765900bd6b6817805c57f45d7321630e6d..582892df0625339412ddbb449a0075d8a4962629 100644 Binary files a/Lectures/L29_IntroductionToVerification/IntroductionToVerification.pdf and b/Lectures/L29_IntroductionToVerification/IntroductionToVerification.pdf differ diff --git a/Lectures/L29_IntroductionToVerification/IntroductionToVerification.tex b/Lectures/L29_IntroductionToVerification/IntroductionToVerification.tex index 844842628556d278e2924170f4a3344fb3f29a51..58a57f61e14e1b73c79a8620ae42f93b2328dc1b 100755 --- a/Lectures/L29_IntroductionToVerification/IntroductionToVerification.tex +++ b/Lectures/L29_IntroductionToVerification/IntroductionToVerification.tex @@ -461,295 +461,4 @@ Avoids potential ArrayIndexOutOfBoundsException Exception %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -\begin{frame} -\frametitle{Does our Engineering Analogy Fail?} - -\begin{itemize} - -\item \structure{If a bridge can hold 500 kN, can it hold 499 kN?} -\item \structure{If our software works for the input 500, will it work for 499?} - -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Verification in Engineering} - -\begin{itemize} - -\item Example of bridge design -\item One test assures infinite correct situations -\item In software a small change in the input may result in significantly different behaviour -\item There are also chaotic systems in nature, but products of engineering - design are usually stable and well-behaved -% consider the examples of weather, bifurcation in engineering, chaos theory for traffic, etc. -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Modified Version Works for 500, but not 499} - -\includegraphics[scale=0.5]{../Figures/BinarySearch.png} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Testing and Lack of ``Continuity''} - -\begin{itemize} - -\item Testing samples behaviours by examining ``test cases'' -\item Impossible to extrapolate behaviour of software from a finite set of test cases -\item No continuity of behaviour - it can exhibit correct behaviour in - infinitely many cases, but may still be incorrect in some cases - -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Goals of Testing} - -\begin{itemize} - -\item \structure{If our code passes all test cases, is it now guaranteed to be - error free?} -\item \structure{Are 5000 random tests always better than 5 carefully selected - tests?} - -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Goals of Testing} - -\begin{itemize} - -\item To show the \structure{presence} of bugs (Dijkstra, 1972) -\item If tests do not detect failures, we cannot conclude that software is defect-free -\item Still, we need to do testing - driven by sound and systematic principles -\begin{itemize} -\item Random testing is often not a systematic principle to use -% look at the example from the text, or divide by zero within a circular range -\item Need a test plan -\end{itemize} -\item Should help isolate errors - to facilitate debugging - -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Goals of Testing Continued} - -\begin{itemize} -\item Should be repeatable -\begin{itemize} -\item Repeating the same experiment, we should get the same results -\item Repeatability may not be true because of the effect of the execution environment on testing -\item Repeatability may not occur if there are uninitialized variables -\item Repeatability may not happen when there is nondeterminism -\end{itemize} - -\item Should be accurate -\begin{itemize} -\item Accuracy increases reliability -\item Part of the motivation for formal specification -\end{itemize} - -\item \structure{Is a \structure{successful} test case one that passes the test, or one - that shows a failure?} - -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Test Plan} - -\begin{itemize} - -\item \structure{Given that no single verification technique can prove - correctness, the practical approach is to use ALL verification techniques. - Is this statement True or False?} - -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Test Plan} - -\begin{itemize} - -\item Testing can uncover errors and build confidence in the software -\item Resources of time, people, facilities are limited -\item Need to plan how the software will be tested -\item You know in advance that the software is unlikely to be perfect -\item You need to put resources into the most important parts of the project -\item A risk analysis can determine where to put your limited resources -\item A risk is a condition that can result in a loss -\item Risk analysis involves looking at how bad the loss can be and at the - probability of the loss occurring - -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Test Plan} - -\begin{itemize} - -\item Risks cannot be eliminated, but the development process can reduce the probability of loss associated with risks -to an ``acceptable'' level -\item One approach to risk analysis is FMEA - Failure Mode Effect Analysis -\item Consider the capstone project of the autonomous rescue robots -\begin{itemize} -\item Largest risk, robot fails during final demonstrations -\item Test to improve reliability -\item Test results of great interest to IBM judges -\item Think about test cases, think about testing environment versus final environment -\end{itemize} -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{White Box Versus Black Box Testing} -\begin{itemize} -\item White box testing is derived from the program's internal structure -\item Black box testing is derived from a description of the program's function -\item Should perform both white box and black box testing -\item Black box testing -\begin{itemize} -\item Uncovers errors that occur in implementing requirements or design specifications -\item Not concerned with how processing occurs, but with the results -\item Focuses on functional requirements for the system -\item Focuses on normal behaviour of the system -\end{itemize} -\end{itemize} -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{White Box Testing} - -\begin{itemize} -\item Uncovers errors that occur during implementation of the program -\item Concerned with how processing occurs -\item Evaluates whether the structure is sound -%\item Focuses on the nonfunctional requirements for the system -\item Focuses on abnormal or extreme behaviour of the system -\end{itemize} - -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Dynamic Versus Static Testing} -\begin{itemize} -\item Another classification of verification techniques, as previously discussed -\item Use a combination of dynamic and static testing -\item Dynamic analysis -\begin{itemize} -\item Requires the program to be executed -\item Test cases are run and results are checked against expected behaviour -\item Exhaustive testing is the only dynamic technique that guarantees program -validity -\item Exhaustive testing is usually impractical or impossible -\item Reduce number of test cases by finding criteria for choosing representative test cases -\end{itemize} -\end{itemize} -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Static Testing Continued} -\begin{itemize} -\item Static analysis -\begin{itemize} -\item Does not involve program execution -\item Testing techniques simulate the dynamic environment -\item Includes syntax checking -\item Generally static testing is used in the requirements and design stage, where there is no code to execute -\item Document and code walkthroughs -\item Document and code inspections -\end{itemize} -\end{itemize} -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Manual Versus Automated Testing} -\begin{itemize} -\item Manual testing -\begin{itemize} -\item Has to be conducted by people -\item Includes by-hand test cases, structured walkthroughs, code inspections -\end{itemize} -\item Automated testing -\begin{itemize} -\item The more automated the development process, the easier to automate testing -\item Less reliance on people -\item Necessary for \structure{regression testing} -\item Test tools can assist, such as Junit, Cppunit, CuTest etc. -\item Can be challenging to automate GUI tests -\item Test suite for Maple has 2 000 000 test cases, run on 14 platforms, every night, automated reporting -\end{itemize} -\end{itemize} -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - -\begin{frame} -\frametitle{Automated Testing at MapleSoft} -\begin{itemize} -\item Three steps -\begin{itemize} -\item Write the problem description -\item result := solver(problem) -\item assert(result == expected) -\end{itemize} -\item Assert writes out code to reproduce any failures -\item Track failures -\begin{itemize} -\item Source code management (like CVS or Subversion) -\item Database of test cases, functions called -\item Database of source files, functions defined -\item Database of 40 days of timings and resources used -\end{itemize} -\item Automatically sends an e-mail to the programmer and his/her boss -\end{itemize} -\end{frame} - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% - \end{document} \ No newline at end of file diff --git a/Lectures/L30_IntroToVerificationContd/IntroToVerificationContd.pdf b/Lectures/L30_IntroToVerificationContd/IntroToVerificationContd.pdf new file mode 100644 index 0000000000000000000000000000000000000000..0f941d1d05f23588e45832ec96db2e45d0d822b0 Binary files /dev/null and b/Lectures/L30_IntroToVerificationContd/IntroToVerificationContd.pdf differ diff --git a/Lectures/L30_IntroToVerificationContd/IntroToVerificationContd.tex b/Lectures/L30_IntroToVerificationContd/IntroToVerificationContd.tex new file mode 100755 index 0000000000000000000000000000000000000000..a49f88aea7f30c09318149d90bf29d861a8ed5dc --- /dev/null +++ b/Lectures/L30_IntroToVerificationContd/IntroToVerificationContd.tex @@ -0,0 +1,449 @@ +%\documentclass[t,12pt,numbers,fleqn,handout]{beamer} +\documentclass[t,12pt,numbers,fleqn]{beamer} + +\usepackage{pgfpages} +\usepackage{hyperref} +\hypersetup{colorlinks=true, + linkcolor=blue, + citecolor=blue, + filecolor=blue, + urlcolor=blue, + unicode=false} +\urlstyle{same} + +\usepackage{hhline} +\usepackage{booktabs} +\usepackage{multirow} +\usepackage{multicol} +\usepackage{array} +\usepackage{listings} + +\usepackage{amssymb} +\usepackage{amsmath} + +\useoutertheme{split} %so the footline can be seen, without needing pgfpages + +% \pgfpagesuselayout{resize to}[letterpaper,border +% shrink=5mm,landscape] %if this is uncommented, the hyperref links do not work + +\mode<presentation>{} + +\input{../def-beamer} + +\newcommand{\topic}{30 Introduction to Verification Continued (Ch.\ 6)} + +\input{../titlepage} + +\begin{document} + +\input{../footline} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Introduction to Verification Continued} + +\begin{itemize} +\item Partially based on slides by Dr.\ Wassyng, Ghezzi et al +\item Administrative details +\item Approaches to verification +\item Goals of testing +\item Test plan +\item Types of test - white box, versus black box, manual versus automated, etc. +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Administrative Details} + +\begin{itemize} + +\item Investigating 9 academic integrity cases for A2 + +\item A3 deadlines +\begin{itemize} +\item Part 2 - Code: due 11:59 pm Mar 20 +\item {Part 1 spec available in repo} +\end{itemize} + +\item A4 +\bi +\item Your own design and specification +\item Due April 3 at 11:59 pm +\ei + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Approaches to Verification} + +\begin{itemize} + +\item \structure{What are some approaches to verification?} +\item \structure{How can we categorize these approaches?} + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Approaches to Verification} + +\begin{itemize} + +\item Experiment with behaviour of product +\begin{itemize} +\item Sample behaviours via testing +\item Goal is to find ``counter examples'' +\item \structure{Dynamic} technique +\item Examples: unit testing, integration testing, acceptance testing, white box + testing, stress testing, etc. +\end{itemize} +\item Analyze product to deduce its adequacy +\begin{itemize} +\item Analytic study of properties +\item \structure{Static} technique +\item Examples: Code walk-throughs, code inspections, correctness proof, etc. +\end{itemize} + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Does our Engineering Analogy Fail?} + +\begin{itemize} + +\item \structure{If a bridge can hold 500 kN, can it hold 499 kN?} +\item \structure{If our software works for the input 500, will it work for 499?} + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Verification in Engineering} + +\begin{itemize} + +\item Example of bridge design +\item One test assures infinite correct situations +\item In software a small change in the input may result in significantly different behaviour +\item There are also chaotic systems in nature, but products of engineering + design are usually stable and well-behaved +% consider the examples of weather, bifurcation in engineering, chaos theory for traffic, etc. +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Modified Version Works for 500, but not 499} + +\includegraphics[scale=0.5]{../Figures/BinarySearch.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Testing and Lack of ``Continuity''} + +\begin{itemize} + +\item Testing samples behaviours by examining ``test cases'' +\item Impossible to extrapolate behaviour of software from a finite set of test cases +\item No continuity of behaviour - it can exhibit correct behaviour in + infinitely many cases, but may still be incorrect in some cases + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Goals of Testing} + +\begin{itemize} + +\item \structure{If our code passes all test cases, is it now guaranteed to be + error free?} +\item \structure{Are 5000 random tests always better than 5 carefully selected + tests?} + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Goals of Testing} + +\begin{itemize} + +\item To show the \structure{presence} of bugs (Dijkstra, 1972) +\item If tests do not detect failures, we cannot conclude that software is defect-free +\item Still, we need to do testing - driven by sound and systematic principles +\begin{itemize} +\item Random testing is often not a systematic principle to use +% look at the example from the text, or divide by zero within a circular range +\item Need a test plan +\end{itemize} +\item Should help isolate errors - to facilitate debugging + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Goals of Testing Continued} + +\begin{itemize} +\item Should be repeatable +\begin{itemize} +\item Repeating the same experiment, we should get the same results +\item Repeatability may not be true because of the effect of the execution environment on testing +\item Repeatability may not occur if there are uninitialized variables +\item Repeatability may not happen when there is nondeterminism +\end{itemize} + +\item Should be accurate +\begin{itemize} +\item Accuracy increases reliability +\item Part of the motivation for formal specification +\end{itemize} + +\item \structure{Is a \structure{successful} test case one that passes the test, or one + that shows a failure?} + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Test Plan} + +\begin{itemize} + +\item \structure{Given that no single verification technique can prove + correctness, the practical approach is to use ALL verification techniques. + Is this statement True or False?} + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Test Plan} + +\begin{itemize} + +\item Testing can uncover errors and build confidence in the software +\item Resources of time, people, facilities are limited +\item Need to plan how the software will be tested +\item You know in advance that the software is unlikely to be perfect +\item You need to put resources into the most important parts of the project +\item A risk analysis can determine where to put your limited resources +\item A risk is a condition that can result in a loss +\item Risk analysis involves looking at how bad the loss can be and at the + probability of the loss occurring + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Test Plan} + +\begin{itemize} + +\item Risks cannot be eliminated, but the development process can reduce the probability of loss associated with risks +to an ``acceptable'' level +\item One approach to risk analysis is FMEA - Failure Mode Effect Analysis +\item Consider the capstone project of the autonomous rescue robots +\begin{itemize} +\item Largest risk, robot fails during final demonstrations +\item Test to improve reliability +\item Test results of great interest to IBM judges +\item Think about test cases, think about testing environment versus final environment +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{White Box Versus Black Box Testing} +\begin{itemize} +\item White box testing is derived from the program's internal structure +\item Black box testing is derived from a description of the program's function +\item Should perform both white box and black box testing +\item Black box testing +\begin{itemize} +\item Uncovers errors that occur in implementing requirements or design specifications +\item Not concerned with how processing occurs, but with the results +\item Focuses on functional requirements for the system +\item Focuses on normal behaviour of the system +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{White Box Testing} + +\begin{itemize} +\item Uncovers errors that occur during implementation of the program +\item Concerned with how processing occurs +\item Evaluates whether the structure is sound +%\item Focuses on the nonfunctional requirements for the system +\item Focuses on abnormal or extreme behaviour of the system +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Dynamic Versus Static Testing} +\begin{itemize} +\item Another classification of verification techniques, as previously discussed +\item Use a combination of dynamic and static testing +\item Dynamic analysis +\begin{itemize} +\item Requires the program to be executed +\item Test cases are run and results are checked against expected behaviour +\item Exhaustive testing is the only dynamic technique that guarantees program +validity +\item Exhaustive testing is usually impractical or impossible +\item Reduce number of test cases by finding criteria for choosing representative test cases +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Static Testing Continued} +\begin{itemize} +\item Static analysis +\begin{itemize} +\item Does not involve program execution +\item Testing techniques simulate the dynamic environment +\item Includes syntax checking +\item Generally static testing is used in the requirements and design stage, where there is no code to execute +\item Document and code walkthroughs +\item Document and code inspections +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Manual Versus Automated Testing} +\begin{itemize} +\item Manual testing +\begin{itemize} +\item Has to be conducted by people +\item Includes by-hand test cases, structured walkthroughs, code inspections +\end{itemize} +\item Automated testing +\begin{itemize} +\item The more automated the development process, the easier to automate testing +\item Less reliance on people +\item Necessary for \structure{regression testing} +\item Test tools can assist, such as Junit, Cppunit, CuTest etc. +\item Can be challenging to automate GUI tests +\item Test suite for Maple has 2 000 000 test cases, run on 14 platforms, every night, automated reporting +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Automated Testing at MapleSoft} +\begin{itemize} +\item Three steps +\begin{itemize} +\item Write the problem description +\item result := solver(problem) +\item assert(result == expected) +\end{itemize} +\item Assert writes out code to reproduce any failures +\item Track failures +\begin{itemize} +\item Source code management (like CVS or Subversion) +\item Database of test cases, functions called +\item Database of source files, functions defined +\item Database of 40 days of timings and resources used +\end{itemize} +\item Automatically sends an e-mail to the programmer and his/her boss +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Fault Testing} +\begin{itemize} +\item Common analogy involves planting fish in a lake to estimate the fish population +\item T = total number of fish in the lake (to be estimated) +\item N = fish stocked (marked) in the lake +\item M = total number of fish caught in lake +\item M' = number of marked fish caught +\item T = (M - M')*N/M' +\item Artificially seed faults, discover both seeded and new faults, estimate the total number of faults +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Fault Testing Continued} +\begin{itemize} +\item Method assumes that the real and seeded faults have the same distribution +\item Hard to seed faults +\begin{itemize} +\item By hand (not a great idea) +\item Independent testing by two groups and obtain the faults from one group for use by the other +\end{itemize} +\item Want most of the discovered faults to be seeded faults +\item If many faults are found, this is a bad thing +\item The probability of errors is proportional to the number of errors already found +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\end{document} \ No newline at end of file diff --git a/Lectures/L31_OverviewOfTesting/OverviewOfTesting.pdf b/Lectures/L31_OverviewOfTesting/OverviewOfTesting.pdf new file mode 100644 index 0000000000000000000000000000000000000000..1ef63887ec8c8e20dcf2d56447131a367cbd20ce Binary files /dev/null and b/Lectures/L31_OverviewOfTesting/OverviewOfTesting.pdf differ diff --git a/Lectures/L31_OverviewOfTesting/OverviewOfTesting.tex b/Lectures/L31_OverviewOfTesting/OverviewOfTesting.tex new file mode 100755 index 0000000000000000000000000000000000000000..6f97459309e81ae6c24c8998976aef80bd7690d6 --- /dev/null +++ b/Lectures/L31_OverviewOfTesting/OverviewOfTesting.tex @@ -0,0 +1,266 @@ +%\documentclass[t,12pt,numbers,fleqn,handout]{beamer} +\documentclass[t,12pt,numbers,fleqn]{beamer} + +\usepackage{pgfpages} +\usepackage{hyperref} +\hypersetup{colorlinks=true, + linkcolor=blue, + citecolor=blue, + filecolor=blue, + urlcolor=blue, + unicode=false} +\urlstyle{same} + +\usepackage{hhline} +\usepackage{booktabs} +\usepackage{multirow} +\usepackage{multicol} +\usepackage{array} +\usepackage{listings} + +\usepackage{amssymb} +\usepackage{amsmath} + +\useoutertheme{split} %so the footline can be seen, without needing pgfpages + +% \pgfpagesuselayout{resize to}[letterpaper,border +% shrink=5mm,landscape] %if this is uncommented, the hyperref links do not work + +\mode<presentation>{} + +\input{../def-beamer} + +\newcommand{\topic}{31 Overview of Testing (Ch.\ 6) DRAFT} + +\input{../titlepage} + +\begin{document} + +\input{../footline} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Overview of Testing} + +\begin{itemize} +\item Administrative details +\item Test plan +\item Types of test +\begin{itemize} +\item White box versus black box +\item Manual versus automated +\item Static versus dynamic +\item etc. +\end{itemize} +\item Testing phases +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Administrative Details} + +\begin{itemize} + +\item Today's slide are partially based on slides by Dr.\ Wassyng + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Test Plan} + +\begin{itemize} + +\item Testing can uncover errors and build confidence in the software +\item Resources of time, people, facilities are limited +\item Need to plan how the software will be tested +\item You know in advance that the software is unlikely to be perfect +\item You need to put resources into the most important parts of the project +\item A risk analysis can determine where to put your limited resources +\item A risk is a condition that can result in a loss +\item Risk analysis involves looking at how bad the loss can be and at the probability of the loss occuring + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Test Plan} + +\begin{itemize} + +\item Risks cannot be eliminated, but the development process can reduce the probability of loss associated with risks +to an ``acceptable'' level +\item One approach to risk analysis is FMEA - Failure Mode Effect Analysis +\item Consider the capstone project of the autonomous rescue robots +\begin{itemize} +\item Largest risk, robot fails during final demonstrations +\item Test to improve reliability +\item Test results of great interest to IBM judges +\item Think about test cases, think about testing environment versus final environment +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{White Box Versus Black Box Testing} +\begin{itemize} +\item White box testing is derived from the program's internal structure +\item Black box testing is derived from a description of the program's function +\item Should perform both white box and black box testing +\item Black box testing +\begin{itemize} +\item Uncovers errors that occur in implementing requirements or design specifications +\item Not concerned with how processing occurs, but with the results +\item Focuses on functional requirements for the system +\item Focuses on normal behaviour of the system +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{White Box Testing} + +\begin{itemize} +\item Uncovers errors that occur during implementation of the program +\item Concerned with how processing occurs +\item Evaluates whether the structure is sound +%\item Focuses on the nonfunctional requirements for the system +\item Focuses on abnormal or extreme behaviour of the system +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Dynamic Versus Static Testing} +\begin{itemize} +\item Another classification of verification techniques, as previously discussed +\item Use a combination of dynamic and static testing +\item Dynamic analysis +\begin{itemize} +\item Requires the program to be executed +\item Test cases are run and results are checked against expected behaviour +\item Exhaustive testing is the only dynamic technique that guarantees program +validity +\item Exhaustive testing is usually impractical or impossible +\item Reduce number of test cases by finding criteria for choosing representative test cases +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Static Testing Continued} +\begin{itemize} +\item Static analysis +\begin{itemize} +\item Does not involve program execution +\item Testing techniques simulate the dynamic environment +\item Includes syntax checking +\item Generally static testing is used in the requirements and design stage, where there is no code to execute +\item Document and code walkthroughs +\item Document and code inspections +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Manual Versus Automated Testing} +\begin{itemize} +\item Manual testing +\begin{itemize} +\item Has to be conducted by people +\item Includes by-hand test cases, structured walkthroughs, code inspections +\end{itemize} +\item Automated testing +\begin{itemize} +\item The more automated the development process, the easier to automate testing +\item Less reliance on people +\item Necessary for \structure{regression testing} +\item Test tools can assist, such as Junit, Cppunit, CuTest etc. +\item Can be challenging to automate GUI tests +\item Test suite for Maple has 2 000 000 test cases, run on 14 platforms, every night, automated reporting +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Automated Testing at Maple} +\begin{itemize} +\item Three steps +\begin{itemize} +\item Write the problem description +\item result := solver(problem) +\item assert(result == expected) +\end{itemize} +\item Assert writes out code to reproduce any failures +\item Track failures +\begin{itemize} +\item Source code management (like CVS or Subversion) +\item Database of test cases, functions called +\item Database of source files, functions defined +\item Database of 40 days of timings and resources used +\end{itemize} +\item Automatically sends an e-mail to the programmer and his/her boss +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Fault Testing} +\begin{itemize} +\item Common analogy involves planting fish in a lake to estimate the fish population +\item T = total number of fish in the lake (to be estimated) +\item N = fish stocked (marked) in the lake +\item M = total number of fish caught in lake +\item M' = number of marked fish caught +\item T = (M - M')*N/M' +\item Artificially seed faults, discover both seeded and new faults, estimate the total number of faults +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Fault Testing Continued} +\begin{itemize} +\item Method assumes that the real and seeded faults have the same distribution +\item Hard to seed faults +\begin{itemize} +\item By hand (not a great idea) +\item Independent testing by two groups and obtain the faults from one group for use by the other +\end{itemize} +\item Want most of the discovered faults to be seeded faults +\item If many faults are found, this is a bad thing +\item The probability of errors is proportional to the number of errors already found +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\end{document} + diff --git a/Lectures/L32_WhiteBoxTesting/WhiteBoxTesting.pdf b/Lectures/L32_WhiteBoxTesting/WhiteBoxTesting.pdf new file mode 100644 index 0000000000000000000000000000000000000000..55b8ca8c35fc970f58674d50206c29902c80d885 Binary files /dev/null and b/Lectures/L32_WhiteBoxTesting/WhiteBoxTesting.pdf differ diff --git a/Lectures/L32_WhiteBoxTesting/WhiteBoxTesting.tex b/Lectures/L32_WhiteBoxTesting/WhiteBoxTesting.tex new file mode 100755 index 0000000000000000000000000000000000000000..7d2cc5a09cd45f36765ff177d2f6add388d51d1a --- /dev/null +++ b/Lectures/L32_WhiteBoxTesting/WhiteBoxTesting.tex @@ -0,0 +1,373 @@ +%\documentclass[t,12pt,numbers,fleqn,handout]{beamer} +\documentclass[t,12pt,numbers,fleqn]{beamer} + +\usepackage{pgfpages} +\usepackage{hyperref} +\hypersetup{colorlinks=true, + linkcolor=blue, + citecolor=blue, + filecolor=blue, + urlcolor=blue, + unicode=false} +\urlstyle{same} + +\usepackage{hhline} +\usepackage{booktabs} +\usepackage{multirow} +\usepackage{multicol} +\usepackage{array} +\usepackage{listings} + +\usepackage{amssymb} +\usepackage{amsmath} + +\useoutertheme{split} %so the footline can be seen, without needing pgfpages + +% \pgfpagesuselayout{resize to}[letterpaper,border +% shrink=5mm,landscape] %if this is uncommented, the hyperref links do not work + +\mode<presentation>{} + +\input{../def-beamer} + +\newcommand{\topic}{32 White Box Testing (Ch.\ 6) DRAFT} + +\input{../titlepage} + +\begin{document} + +\input{../footline} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{White Box Testing} + +\begin{itemize} +\item Administrative details +\item Nonfunctional testing +\item Functional testing +\item Testing phases +\item Theoretical foundations of testing +\item Complete coverage principle +\item Statement coverage +\item Edge coverage +\item Condition coverage +\item Path coverage +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Administrative Details} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\begin{frame} +\frametitle{Fault Testing} +\begin{itemize} +\item Common analogy involves planting fish in a lake to estimate the fish population +\item T = total number of fish in the lake (to be estimated) +\item N = fish stocked (marked) in the lake +\item M = total number of fish caught in lake +\item M' = number of marked fish caught +\item T = (M - M')*N/M' +\item Artificially seed faults, discover both seeded and new faults, estimate the total number of faults +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Fault Testing Continued} +\begin{itemize} +\item Method assumes that the real and seeded faults have the same distribution +\item Hard to seed faults +\begin{itemize} +\item By hand (not a great idea) +\item Independent testing by two groups and obtain the faults from one group for use by the other +\end{itemize} +\item Want most of the discovered faults to be seeded faults +\item If many faults are found, this is a bad thing +\item The probability of errors is proportional to the number of errors already found +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Nonfunctional System Testing} +\begin{itemize} +\item Stress testing: Determines if the system can function when subject to large volumes +\item Execution: Determines if the system achieves the desired level of proficiency in production status (performance) +\item Recovery: Determines if the system has the ability to restart operations after integrity has been lost +\item Operations: Determines if the operating procedures and staff can properly execute the system (documentation) +\item Compliance (to process): Determines if the system has been developed in accordance with information technology +standards, procedures and guidelines +\item Security: Determines if the system can protect confidential information +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Functional System Testing} +\begin{itemize} +\item Requirements: Determines if the system can perform its function correctly and that the correctness can be +sustained over a continuous period of time +%\item Regression: Determines if changes to the system do not invalidate previous positive testing results +\item Error Handling: Determines the ability of the system to properly process incorrect transactions +\item Manual Support: Determines that the manual support procedures are documented and complete, where manual support +involves procedures, interfaces between people and the system, and training procedures +\item Inter-systems: Determines the that interconnections between systems function correctly +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Functional System Testing Continued} +\begin{itemize} +\item Control: Determines if the processing is performed in accordance with the intents of management +\begin{itemize} +\item Includes data validation, file integrity, audit trail, backup and recovery, documentation and other aspects +related to integrity +\item Controls are designed to reduce risks +\end{itemize} +\item Parallel: Determines the results of the new application are consistent with the processing of the previous +application or version of the application +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Testing Phases} +\begin{enumerate} +\item Unit testing +\item Integration testing +\item System testing +\item Acceptance testing +\end{enumerate} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Theoretical Foundations Of Testing: Definitions} +\begin{itemize} +\item P (program), D (input domain), R (output domain) +\bi +\item P: D $\rightarrow$ R (may be partial) +\ei +\item Correctness defined by OR $\subseteq$ D $\times$ R +\bi +\item P(d) correct if $\langle$ d, P(d) $\rangle \in$ OR +\item P correct if all P(d) are correct +\ei +\item Failure +\bi +\item P(d) is not correct +\item May be undefined (error state) or may be the wrong result +\ei +\item Error (Defect) +\bi +\item Anything that may cause a failure +\bi +\item Typing mistake +\item Programmer forgot to test ``x=0'' +\ei +\ei +\item Fault +\bi +\item Incorrect intermediate state entered by program +\ei +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Definitions Continued} +\begin{itemize} +\item Test case t +\bi +\item An element of D +\ei +\item Test set T +\bi +\item A finite subset of D +\ei +\item Test is successful if P(t) is correct +\item Test set successful if P correct for all t in T +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Theoretical Foundations of Testing} + +\begin{itemize} +\item Desire a test set $T$ that is a finite subset of $D$ that will uncover all errors +\item Determining and ideal $T$ leads to several undecideable problems +\item No algorithm exists to state if a test set will uncover all possible errors +\item No algorithm exists to derive a test set that would prove program correctness +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Empirical Testing} + +\begin{itemize} +\item Need to introduce empirical testing principles and heuristics as a compromise between the impossible and the +inadequate +\item Find a strategy to select \structure{significant} test cases +\item Significant means the test cases have a high potential of uncovering the presence of errors +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Complete-Coverage Principle} + +\begin{itemize} +\item Try to group elements of $D$ into subdomains $D_1$, $D_2$, ..., $D_n$ where any element of each $D_i$ is likely +to have similar behaviour +\item $D = D_1 \cup D_2 \cup ... \cup D_n$ +\item Select one test as a representative of the subdomain +\item If $D_j \cap D_k = \emptyset$ for all $j \neq k$, (partition), any element can be chosen from each subdomain +\item Otherwise choose representatives to minimize number of tests, yet fulfilling the principle +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Complete-Coverage Principle} + +\includegraphics[scale=0.5]{../Figures/CompleteCoveragePrinciple.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{White-box Coverage Testing} + +\begin{itemize} +\item (In)adequacy criteria - if significant parts of the program structure are not tested, testing is inadequate +\item Control flow coverage criteria +\begin{itemize} +\item Statement coverage +\item Edge coverage +\item Condition coverage +\item Path coverage +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Statement-Coverage Criterion} + +\begin{itemize} +\item Select a test set $T$ such that every elementary statement in $P$ is executed at least once by some $d$ in $T$ +\item An input datum executes many statements - try to minimize the number of test cases still preserving the desired +coverage +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Example} + +\includegraphics[scale=0.5]{../Figures/ExampleStatementCoverage.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Weakness of the Criterion} + +\includegraphics[scale=0.6]{../Figures/WeaknessStatementCoverage.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Edge-Coverage Criterion} + +\begin{itemize} +\item Select a test set $T$ such that every edge (branch) of the control flow is exercised at least once by some $d$ +in $T$ +\item This requires formalizing the concept of the control graph and how to construct it +\begin{itemize} +\item Edges represent statements +\item Nodes at the ends of an edge represent entry into the statement and exit +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Control Graph Construction Rules} + +\includegraphics[scale=0.4]{../Figures/ControlGraphRules.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Simplification} + +A sequence of edges can be collapsed into just one edge\\ +~\newline + +\includegraphics[scale=0.5]{../Figures/Simplification.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Example: Euclid's Algorithm} + +\includegraphics[scale=0.5]{../Figures/EuclidsAlgorithm.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Weakness} + +\includegraphics[scale=0.4]{../Figures/WeaknessEdgeCoverage.png}\\ +\uncover<2->{\structure{Do not discover the error ($<$ instead of $\leq$)}} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\end{document} + diff --git a/Lectures/L33_WhiteBoxTestingContinued/CondCovExample.java b/Lectures/L33_WhiteBoxTestingContinued/CondCovExample.java new file mode 100644 index 0000000000000000000000000000000000000000..e83ce3e23f13acc860371cbb4c74191b111f9081 --- /dev/null +++ b/Lectures/L33_WhiteBoxTestingContinued/CondCovExample.java @@ -0,0 +1,14 @@ +if c1 and c2 then + st; +else + sf; + +// equivalent to + +if c1 then + if c2 then + st; + else + sf; +else + sf; diff --git a/Lectures/L33_WhiteBoxTestingContinued/TestPointT.java b/Lectures/L33_WhiteBoxTestingContinued/TestPointT.java new file mode 100644 index 0000000000000000000000000000000000000000..52bca3144b7aa762fc9ba353f1ef02c084878cb8 --- /dev/null +++ b/Lectures/L33_WhiteBoxTestingContinued/TestPointT.java @@ -0,0 +1,69 @@ +import org.junit.*; +import static org.junit.Assert.*; +public class TestPointT +{ + private static double ADMISS_ERR_CONSTRUCTOR = 0; + private static double ADMISS_ERR_DIST = 1e-20; + @Test + public void testConstructorForx() + { + assertEquals(23, new PointT(23, 38).xcoord(), ADMISS_ERR_CONSTRUCTOR); + } + @Test + public void testConstructorFory() + { + assertEquals(38, new PointT(23, 38).ycoord(), ADMISS_ERR_CONSTRUCTOR); + } + @Test (expected=InvalidPointException.class) + public void testForExceptionNegx() + { + PointT p = new PointT(-10, 0); + } + @Test (expected=InvalidPointException.class) + public void testForExceptionNegy() + { + PointT p = new PointT(0, -10); + } + @Test (expected=InvalidPointException.class) + public void testForExceptionMaxx() + { + PointT p = new PointT(Constants.MAX_X+1, 0); + } + @Test (expected=InvalidPointException.class) + public void testForExceptionMaxy() + { + PointT p = new PointT(0, Constants.MAX_Y+1); + } + @Test + public void testDistNormal() + { + double x = Constants.MAX_X/2.0; + double y = Constants.MAX_Y/2.0; + PointT p = new PointT(x, y); + assertEquals(Math.sqrt(x*x + y*y), p.dist(new PointT(0, 0)), ADMISS_ERR_DIST); + } + @Test + public void testDistLargestDiagonal() + { + double x = Constants.MAX_X; + double y = Constants.MAX_Y; + PointT p = new PointT(x, y); + assertEquals(Math.sqrt(x*x + y*y), p.dist(new PointT(0, 0)), ADMISS_ERR_DIST); + } + @Test + public void testDistAlongEdge() + { + double x = Constants.MAX_X; + double y = Constants.MAX_Y; + PointT p = new PointT(x, y); + assertEquals(Constants.MAX_X, p.dist(new PointT(0, Constants.MAX_Y)), ADMISS_ERR_DIST); + } + @Test + public void testDistZero() + { + double x = Constants.MAX_X/2.0; + double y = Constants.MAX_Y/2.0; + PointT p = new PointT(x, y); + assertEquals(0, p.dist(p), ADMISS_ERR_DIST); + } +} diff --git a/Lectures/L33_WhiteBoxTestingContinued/WhiteBoxTestingContinued.pdf b/Lectures/L33_WhiteBoxTestingContinued/WhiteBoxTestingContinued.pdf new file mode 100644 index 0000000000000000000000000000000000000000..e993057c35f57ad9430d540d85d0c48d7b8954bd Binary files /dev/null and b/Lectures/L33_WhiteBoxTestingContinued/WhiteBoxTestingContinued.pdf differ diff --git a/Lectures/L33_WhiteBoxTestingContinued/WhiteBoxTestingContinued.tex b/Lectures/L33_WhiteBoxTestingContinued/WhiteBoxTestingContinued.tex new file mode 100755 index 0000000000000000000000000000000000000000..5f60a938b0eef2d98c25813182d9bfc5c8d1b6b4 --- /dev/null +++ b/Lectures/L33_WhiteBoxTestingContinued/WhiteBoxTestingContinued.tex @@ -0,0 +1,213 @@ +%\documentclass[t,12pt,numbers,fleqn,handout]{beamer} +\documentclass[t,12pt,numbers,fleqn]{beamer} + +\usepackage{pgfpages} +\usepackage{hyperref} +\hypersetup{colorlinks=true, + linkcolor=blue, + citecolor=blue, + filecolor=blue, + urlcolor=blue, + unicode=false} +\urlstyle{same} + +\usepackage{hhline} +\usepackage{booktabs} +\usepackage{multirow} +\usepackage{multicol} +\usepackage{array} +\usepackage{listings} + +\usepackage{amssymb} +\usepackage{amsmath} + +\useoutertheme{split} %so the footline can be seen, without needing pgfpages + +% \pgfpagesuselayout{resize to}[letterpaper,border +% shrink=5mm,landscape] %if this is uncommented, the hyperref links do not work + +\mode<presentation>{} + +\input{../def-beamer} + +\newcommand{\topic}{32 White Box Testing Continued (Ch.\ 6) DRAFT} + +\input{../titlepage} + +\begin{document} + +\input{../footline} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{White Box Testing Continued} + +\begin{itemize} +\item Administrative details +\item White box testing +\begin{itemize} +\item Edge coverage +\item Condition coverage +\item Path coverage +\end{itemize} +\item Black Box Testing +\begin{itemize} +\item Informal Example +\item Formal using MIS for PointT +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Administrative Details} + +\begin{itemize} + +\item Today's slide are partially based on slides by Dr.\ Wassyng +\item Assignment 5 +\begin{itemize} +\item Implementation due by midnight March 30 +\item If there is a new specification, it is due by 4pm March 31 (under door of ITB/167 is fine for submission) +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Edge-Coverage Criterion} + +\begin{itemize} +\item Select a test set $T$ such that every edge (branch) of the control flow is exercised at least once by some $d$ +in $T$ +\item This requires formalizing the concept of the control graph and how to construct it +\begin{itemize} +\item Edges represent statements +\item Nodes at the ends of an edge represent entry into the statement and exit +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Control Graph Construction Rules} + +\includegraphics[scale=0.4]{../Figures/ControlGraphRules.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Simplification} + +A sequence of edges can be collapsed into just one edge\\ +~\newline + +\includegraphics[scale=0.5]{../Figures/Simplification.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Weakness} + +\includegraphics[scale=0.4]{../Figures/WeaknessEdgeCoverage.png}\\ +\uncover<2->{\structure{Do not discover the error ($<$ instead of $\leq$)}} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +%\frametitle{Alternative View of Condition Coverage} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} +\noindent \lstinputlisting{CondCovExample.java} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Condition-Coverage Criterion} + +\begin{itemize} +\item Select a test set $T$ such that every edge of $P$s control flow is traversed and all possible values of the +constituents of compound conditions are exercised at least once +\item This criterion is finer than edge coverage +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Weakness} + +\includegraphics[scale=0.5]{../Figures/WeaknessConditionCoverage.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Path-Coverage Criterion} + +\begin{itemize} +\item Select a test set $T$ that traverses all paths from the initial to the final node of $P$s control flow +\item It is finer than the previous kinds of coverage +\item However, number of paths may be too large, or even infinite (see while loops) +\item Loops +\begin{itemize} +\item Zero times (or minimum number of times) +\item Maximum times +\item Average number of times +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{The Infeasibility Problem} + +\begin{itemize} +\item Syntactically indicated behaviours (statements, edges, etc.) are often impossible +\item Unreachable code, infeasible edges, paths, etc. +\item Adequacy criteria may be impossible to satisfy +\begin{itemize} +\item Manual justification for omitting each impossible test case +\item Adequacy ``scores'' based on coverage - example 95 \% statement coverage +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Further Problem} + +\begin{itemize} +\item What if the code omits the implementation of some part of the specification? +\item White box test cases derived from the code will ignore that part of the specification! +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\end{document} + diff --git a/Lectures/L34_BlackBoxTesting/BlackBoxTesting.pdf b/Lectures/L34_BlackBoxTesting/BlackBoxTesting.pdf new file mode 100644 index 0000000000000000000000000000000000000000..b54d28a7d406a62fb78cd749506ed226d496cf37 Binary files /dev/null and b/Lectures/L34_BlackBoxTesting/BlackBoxTesting.pdf differ diff --git a/Lectures/L34_BlackBoxTesting/BlackBoxTesting.tex b/Lectures/L34_BlackBoxTesting/BlackBoxTesting.tex new file mode 100755 index 0000000000000000000000000000000000000000..06f10a1399e5f39d980de02a6d35becbd9b8624f --- /dev/null +++ b/Lectures/L34_BlackBoxTesting/BlackBoxTesting.tex @@ -0,0 +1,259 @@ +%\documentclass[t,12pt,numbers,fleqn,handout]{beamer} +\documentclass[t,12pt,numbers,fleqn]{beamer} + +\usepackage{pgfpages} +\usepackage{hyperref} +\hypersetup{colorlinks=true, + linkcolor=blue, + citecolor=blue, + filecolor=blue, + urlcolor=blue, + unicode=false} +\urlstyle{same} + +\usepackage{hhline} +\usepackage{booktabs} +\usepackage{multirow} +\usepackage{multicol} +\usepackage{array} +\usepackage{listings} + +\usepackage{amssymb} +\usepackage{amsmath} + +\useoutertheme{split} %so the footline can be seen, without needing pgfpages + +% \pgfpagesuselayout{resize to}[letterpaper,border +% shrink=5mm,landscape] %if this is uncommented, the hyperref links do not work + +\mode<presentation>{} + +\input{../def-beamer} + +\newcommand{\topic}{34 Black Box Testing (Ch.\ 6) DRAFT} + +\input{../titlepage} + +\begin{document} + +\input{../footline} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Black Box Testing} + +\begin{itemize} +\item Administrative details +\item Black Box Testing +\begin{itemize} +\item Formal using RegionT +\item Function tables +\end{itemize} +\item Testing boundary conditions +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Administrative Details} + +\begin{itemize} + +\item Today's slide are partially based on slides by Dr.\ Wassyng + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\begin{frame} +\frametitle{Black Box Testing Example} + +The program receives as input a record describing +an invoice. (A detailed description of the format of +the record is given.) The invoice must be inserted +into a file of invoices that is sorted by date. The invoice +must be inserted in the appropriate position: If other +invoices exist in the file with the same date, then the +invoice should be inserted after the last one. Also, +some consistency checks must be performed: The program +should verify whether the customer is already in a +corresponding file of customers, whether the customerÕs +data in the two files match, etc. \\ + +~\newline + +{\structure{What test cases would satisfy the complete-coverage principle?}} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Invoice Example Test Cases} + +\begin{enumerate} +\item An invoice whose date is the current date +\item An invoice whose date is before the current date (This might be even forbidden by law) This case, in turn, can be +split into the two following subcases: +\begin{enumerate} +\item An invoice whose date is the same as that of some existing invoice +\item An invoice whose date does not exist in any previously recorded invoice +\end{enumerate} +\item Several incorrect invoices, checking different types of inconsistencies +\end{enumerate} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Systematic Black-Box Techniques} + +\begin{itemize} +\item Testing driven by logic specifications +%\item Decision table based testing +\item Function table based testing +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Test Cases from MIS for PointT} + +\begin{tabular}{| l | l | l | l |} +\hline +\textbf{Routine name} & \textbf{In} & \textbf{Out} & \textbf{Exceptions}\\ +\hline +PointT & real, real & PointT & InvalidPointException\\ +\hline +xcoord & ~ & real & ~\\ +\hline +ycoord & ~ & real & ~\\ +\hline +dist & PointT & real & ~\\ +\hline +\end{tabular} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame}[allowframebreaks] +\frametitle{TestPointT.java} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} +\noindent \lstinputlisting{TestPointT.java} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Function Table-Based Testing} + +\begin{itemize} + +\item Boundaries are obvious in table predicates +\item Make test cases that exercise between and on boundaries +\item Coverage already aided by function table ``rules'' + +\end{itemize} + +\includegraphics[scale=0.5]{../Figures/FunctionTableTesting.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Testing Boundary Conditions} + +\begin{itemize} + +\item Testing criteria partition input domain in classes, assuming that behavior is ``similar'' for all data within a +class +\item Some typical programming errors, however, just happen to be at the boundary between different classes +\begin{itemize} +\item Off by one errors +\item $<$ instead of $\leq$ +\item equals zero +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Criterion} + +\begin{itemize} + +\item After partitioning the input domain D into several classes, test the program using input values not only +``inside'' the classes, but also at their boundaries +\item This applies to both white-box and black-box techniques +\item In practice, use the different testing criteria in combinations +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{The Oracle Problem} + +\begin{itemize} + +\item Given input test cases that cover the domain, what are the expected outputs? +\item Oracles are required at each stage of testing to tell us what the right answer is +\item Black-box criteria are better than white-box for building test oracles +\item Automated test oracles are required for running large amounts of tests +\item Oracles are difficult to design - no universal recipe +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{The Oracle Problem Continued} + +\begin{itemize} + +\item Determining what the right answer should be is not always easy +\begin{itemize} +\item Air traffic control system +\item Scientific computing +\item Parallel testing can approximate an oracle +\item Properties of the expected values can be easier than stating the expected output +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Module Testing} + +\begin{itemize} + +\item Scaffolding needed to create the environment in which the module should be tested +\item Stubs - a module used by the module under test +\item Driver - module activating the module under test +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\end{document} + diff --git a/Lectures/L34_BlackBoxTesting/TestPointT.java b/Lectures/L34_BlackBoxTesting/TestPointT.java new file mode 100644 index 0000000000000000000000000000000000000000..52bca3144b7aa762fc9ba353f1ef02c084878cb8 --- /dev/null +++ b/Lectures/L34_BlackBoxTesting/TestPointT.java @@ -0,0 +1,69 @@ +import org.junit.*; +import static org.junit.Assert.*; +public class TestPointT +{ + private static double ADMISS_ERR_CONSTRUCTOR = 0; + private static double ADMISS_ERR_DIST = 1e-20; + @Test + public void testConstructorForx() + { + assertEquals(23, new PointT(23, 38).xcoord(), ADMISS_ERR_CONSTRUCTOR); + } + @Test + public void testConstructorFory() + { + assertEquals(38, new PointT(23, 38).ycoord(), ADMISS_ERR_CONSTRUCTOR); + } + @Test (expected=InvalidPointException.class) + public void testForExceptionNegx() + { + PointT p = new PointT(-10, 0); + } + @Test (expected=InvalidPointException.class) + public void testForExceptionNegy() + { + PointT p = new PointT(0, -10); + } + @Test (expected=InvalidPointException.class) + public void testForExceptionMaxx() + { + PointT p = new PointT(Constants.MAX_X+1, 0); + } + @Test (expected=InvalidPointException.class) + public void testForExceptionMaxy() + { + PointT p = new PointT(0, Constants.MAX_Y+1); + } + @Test + public void testDistNormal() + { + double x = Constants.MAX_X/2.0; + double y = Constants.MAX_Y/2.0; + PointT p = new PointT(x, y); + assertEquals(Math.sqrt(x*x + y*y), p.dist(new PointT(0, 0)), ADMISS_ERR_DIST); + } + @Test + public void testDistLargestDiagonal() + { + double x = Constants.MAX_X; + double y = Constants.MAX_Y; + PointT p = new PointT(x, y); + assertEquals(Math.sqrt(x*x + y*y), p.dist(new PointT(0, 0)), ADMISS_ERR_DIST); + } + @Test + public void testDistAlongEdge() + { + double x = Constants.MAX_X; + double y = Constants.MAX_Y; + PointT p = new PointT(x, y); + assertEquals(Constants.MAX_X, p.dist(new PointT(0, Constants.MAX_Y)), ADMISS_ERR_DIST); + } + @Test + public void testDistZero() + { + double x = Constants.MAX_X/2.0; + double y = Constants.MAX_Y/2.0; + PointT p = new PointT(x, y); + assertEquals(0, p.dist(p), ADMISS_ERR_DIST); + } +} diff --git a/Lectures/L35_Analysis/Analysis.pdf b/Lectures/L35_Analysis/Analysis.pdf new file mode 100644 index 0000000000000000000000000000000000000000..3cda1182d723d51b38fecaca179da1b44d6833fc Binary files /dev/null and b/Lectures/L35_Analysis/Analysis.pdf differ diff --git a/Lectures/L35_Analysis/Analysis.tex b/Lectures/L35_Analysis/Analysis.tex new file mode 100755 index 0000000000000000000000000000000000000000..cd8da138be1f5fb95564bd8321862f1db7bb5484 --- /dev/null +++ b/Lectures/L35_Analysis/Analysis.tex @@ -0,0 +1,390 @@ +%\documentclass[t,12pt,numbers,fleqn,handout]{beamer} +\documentclass[t,12pt,numbers,fleqn]{beamer} + +\usepackage{pgfpages} +\usepackage{hyperref} +\hypersetup{colorlinks=true, + linkcolor=blue, + citecolor=blue, + filecolor=blue, + urlcolor=blue, + unicode=false} +\urlstyle{same} + +\usepackage{hhline} +\usepackage{booktabs} +\usepackage{multirow} +\usepackage{multicol} +\usepackage{array} +\usepackage{listings} + +\usepackage{amssymb} +\usepackage{amsmath} + +\useoutertheme{split} %so the footline can be seen, without needing pgfpages + +% \pgfpagesuselayout{resize to}[letterpaper,border +% shrink=5mm,landscape] %if this is uncommented, the hyperref links do not work + +\mode<presentation>{} + +\input{../def-beamer} + +\newcommand{\topic}{35 Analysis (Ch.\ 6) DRAFT} + +\input{../titlepage} + +\begin{document} + +\input{../footline} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Analysis} + +\begin{itemize} +\item Administrative details +\item The Oracle problem +\item Module testing +\item Integration testing +\item Testing OO programs +\item Testing concurrent and real-time systems +\item Analysis +\begin{itemize} +\item Code walk throughs and inspections +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Administrative Details} + +\begin{itemize} + +\item Today's slide are partially based on slides by Dr.\ Wassyng + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{The Oracle Problem} + +\begin{itemize} + +\item Given input test cases that cover the domain, what are the expected outputs? +\item Oracles are required at each stage of testing to tell us what the right answer is +\item Black-box criteria are better than white-box for building test oracles +\item Automated test oracles are required for running large amounts of tests +\item Oracles are difficult to design - no universal recipe +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{The Oracle Problem Continued} + +\begin{itemize} + +\item Determining what the right answer should be is not always easy +\begin{itemize} +\item Air traffic control system +\item Scientific computing +\item Parallel testing can approximate an oracle +\item Properties of the expected values can be easier than stating the expected output +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Module Testing} + +\begin{itemize} + +\item Scaffolding needed to create the environment in which the module should be tested +\item Stubs - a module used by the module under test +\item Driver - module activating the module under test +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Testing a Functional Module} + +\includegraphics[scale=0.5]{../Figures/TestingAFunctModule.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Integration Testing} + +\begin{itemize} +\item Big-bang approach +\begin{itemize} +\item First test individual modules in isolation +\item Then test integrated system +\end{itemize} +\item Incremental approach +\begin{itemize} +\item Modules are progressively integrated and tested +\item Can proceed both top-down and bottom-up according to the USES relation +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Integration Testing and USES relation} + +\begin{itemize} +\item If integration and test proceed bottom-up only need drivers +\item Otherwise if we proceed top-down only stubs are needed +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Example} + +\begin{center} +\includegraphics[scale=0.35]{../Figures/Example.png} +\end{center} + +\begin{itemize} +\item $M_1$ USES $M_2$ and $M_2$ IS\_COMPOSED\_OF \{$M_{2,1}$, $M_{2,2}$\} +\item Case 1 +\begin{itemize} +\item Test $M_1$ providing a stub for $M_2$ and a driver for $M_1$ +\item Then provide an implementation for $M_{2,1}$ and a stub for $M_{2,2}$ +\end{itemize} +\item Case 2 +\begin{itemize} +\item Implement $M_{2,2}$ and test it by using a driver +\item Implement $M_{2,1}$ and test the combination of $M_{2,1}$ and $M_{2,2}$ (i.e.\ $M_2$) by using a driver +\item Finally implement $M_1$ and test it with $M_2$ using a driver for $M_1$ +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Testing OO Programs} + +\begin{itemize} +\item New issues +\begin{itemize} +\item Inheritance +\item Genericity +\item Polymorphism +\item Dynamic binding +\end{itemize} +\item Open problems still exist +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Inheritance} + +\includegraphics[scale=0.5]{../Figures/Inheritance.png} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{How to Test Classes of the Hierarchy} + +\begin{center} +\includegraphics[scale=0.5]{../Figures/UsesRelation.png} +\end{center} + +\begin{itemize} +\item ``Flattening'' the whole hierarchy and considering every class as totally independent component +\item This does not exploit incrementality +\item Finding an ad-hoc way to take advantage of the hierarchy +\item Think about testing PointT.java and PointMassT.java +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{A Sample Strategy} + +\begin{itemize} +\item A test that does not have to be repeated for any heir +\item A test that must be performed for heir class X and all of its further heirs +\item A test that must be redone by applying the same input data, but verifying that the output is not (or is) changed +\item A test that must be modified by adding other input parameters and verifying that the the output changes accordingly +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Testing Concurrent and Real-time Systems} + +\begin{itemize} +\item Nondeterminism inherent in concurrency affects repeatability +\item For real-time systems, a test case consists not only of input data, but also of the times when such data are supplied +\item Considerable care and detail when testing real-time systems +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Analysis Versus Testing} + +\begin{itemize} +\item Testing characterizes a \structure{single} execution +\item Analysis characterizes a \structure{class} of executions; it is based on a \structure{model} +\item They have complementary advantages and disadvantages +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Informal Analysis Techniques and Code Walkthroughs} + +\begin{itemize} +\item Recommended prescriptions +\begin{itemize} +\item Small number of people (three to five) +\item Participants receive written documentation from the designer a few days before the meeting +\item Predefined duration of the meeting (a few hours) +\item Focus on the \structure{discovery} of errors, not on fixing them +\item Participants: designer, moderator, and a secretary +\item Foster cooperation; no evaluation of people +\item Experience shows that most errors are discovered by the designer during the presentation, while trying to explain the design to other people +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\begin{frame} +\frametitle{Informal Analysis Techniques Code Inspection} + +\begin{itemize} +\item A reading technique aiming at error discovery +\item Based on checklists +\begin{itemize} +\item Use of uninitialized variables +\item Jumps into loops +\item Nonterminating loops +\item Array indexes out of bounds +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Correctness Proofs} + +\begin{itemize} +\item Formal program analysis is a verification aid that may enhance program reliability +\item Mathematically prove that the program's semantics implies its specification +\item Can use pre and post conditions +\item Tabular expressions can be proven to match between specification of requirements and a specification of the design +\item In many cases verification can be automated, at least partially +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Model Checking} + +\begin{itemize} +\item Correctness verification, in general, is an undecidable problem +\item Model checking is a rather recent verification technique based on the fact that most interesting system +properties become decidable (algorithmically verifiable) when the system is modelled as a finite state machine +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Model Checking Continued} + +\begin{itemize} +\item Describe a given system - software or otherwise - as an FSM +\item Express a given property of interest as a suitable formula +\begin{itemize} +\item Does a computation exist that allows a process to enter a critical region? +\item Is there a guarantee that a process can access shared resources? +\end{itemize} +\item Verify whether the system's behaviour does indeed satisfy the desired property +\begin{itemize} +\item This step can be performed automatically +\item The model checker either provides a \structure{proof} that the property holds or gives a \structure{counter +example} in the form of a test case that exposes the system's failure to behave according to the property +\end{itemize} +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Why so Many Approaches to Testing and Analysis?} + +\begin{itemize} +\item Testing versus (correctness) analysis +\item Formal versus informal techniques +\item White-box versus black-box techniques +\item Techniques in the small/large +\item Fully automatic versus semi-automatic techniques (for undecidable problems) +\item ... +\end{itemize} + +View all of these as complementary + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Debugging} + +\begin{itemize} +\item The activity of locating and correcting errors +\item It can start once a failure has been detected +\item The goal is closing the gap between a fault and a failure +\begin{itemize} +\item Memory dumps, watch points +\item Intermediate assertions can help +\item Tools like gdb, valgrind, etc. +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\end{document} + diff --git a/Lectures/L36_DesignPatterns/DesignPatterns.pdf b/Lectures/L36_DesignPatterns/DesignPatterns.pdf new file mode 100644 index 0000000000000000000000000000000000000000..dce7a90cd5746053c6547703d8f689a4b6a39bad Binary files /dev/null and b/Lectures/L36_DesignPatterns/DesignPatterns.pdf differ diff --git a/Lectures/L36_DesignPatterns/DesignPatterns.tex b/Lectures/L36_DesignPatterns/DesignPatterns.tex new file mode 100755 index 0000000000000000000000000000000000000000..60dcd1329866f1fd195b25c341b75368d1b1f666 --- /dev/null +++ b/Lectures/L36_DesignPatterns/DesignPatterns.tex @@ -0,0 +1,272 @@ +%\documentclass[t,12pt,numbers,fleqn,handout]{beamer} +\documentclass[t,12pt,numbers,fleqn]{beamer} + +\usepackage{pgfpages} +\usepackage{hyperref} +\hypersetup{colorlinks=true, + linkcolor=blue, + citecolor=blue, + filecolor=blue, + urlcolor=blue, + unicode=false} +\urlstyle{same} + +\usepackage{hhline} +\usepackage{booktabs} +\usepackage{multirow} +\usepackage{multicol} +\usepackage{array} +\usepackage{listings} + +\usepackage{amssymb} +\usepackage{amsmath} + +\useoutertheme{split} %so the footline can be seen, without needing pgfpages + +% \pgfpagesuselayout{resize to}[letterpaper,border +% shrink=5mm,landscape] %if this is uncommented, the hyperref links do not work + +\mode<presentation>{} + +\input{../def-beamer} + +\newcommand{\topic}{36 Design Patterns DRAFT} + +\input{../titlepage} + +\begin{document} + +\input{../footline} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Design Patterns} + +\begin{itemize} +\item Administrative details +\item Debugging +\item Verifying other qualities +\item Design patterns +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Administrative Details} + +\begin{itemize} + +\item Today's slide are partially based on slides by Dr.\ Wassyng and on van Vliet (2000) + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Debugging} + +\begin{itemize} +\item The activity of locating and correcting errors +\item It can start once a failure has been detected +\item The goal is closing the gap between a fault and a failure +\begin{itemize} +\item Memory dumps, watch points +\item Intermediate assertions can help +\item Tools like gdb, valgrind, etc. +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Verifying Performance} + +\begin{itemize} +\item Worst case analysis versus average behaviour +\item For worst case focus on proving that the system response time is bounded by some function of the external +requests +\item Standard deviation +\item Analytical versus experimental approaches +\item Consider verifying the performance of a pacemaker +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Verifying Reliability} + +\begin{itemize} +\item There are approaches to measuring reliability on a probabilistic basis, as in other engineering fields +\item Unfortunately there are some difficulties with this approach +\item Independence of failures does not hold for software +\item Reliability is concerned with measuring the probability of the occurrence of failure +\item Meaningful parameters include +\begin{itemize} +\item Average total number of failures observed at time $t$: $AF(t)$ +\item Failure intensity: $FI(T)=AF'(t)$ +\item Mean time to failure at time $t$: $MTTF(t) = 1/FI(t)$ +\end{itemize} +\item Time in the model can be execution or clock or calendar time +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Design Patterns} + +\begin{itemize} + +\item Christopher Alexander (1977, buildings/towns): +\begin{itemize} +\item ``Each pattern describes a problem which occurs over and over again in our environment, and then describes the +core of the solution to that problem, in such a way that you can use this solution a million times over, without ever +doing it the same way.'' +\end{itemize} +\item Design reuse (intended for OO) +\item Solution for recurring problems found in design of many systems +\item Transferring knowledge from experienced to novice designers +\item A design pattern is a recurring structure of communicating components that solves a general design problem within a particular context +\item Design patterns consist of multiple modules, but they do not constitute an entire system architecture +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{UML Diagram of Measurable Interface} + +\includegraphics[scale=0.65]{../Figures/UMLDataSetMeasurableInterface.pdf} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{UML Diagram of Measurer Interface} + +\includegraphics[scale=0.6]{../Figures/UMLDataSetMeasurerInterface.pdf} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Model View Controller (MVC)} + +\begin{itemize} + +\item Separate computational elements from I/O elements +\item Three components +\begin{enumerate} +\item Model encapsulates the system's data as well as the operations on the data +\item View displays the data from the model components, possibly multiple view components +\item Controller handles input actions +\end{enumerate} +\item The controller may or may not depend on the state of the model +\item The controller depends on model state when menu items are enabled or disabled depending on the state of the model +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Design Pattern Properties} + +\begin{itemize} + +\item A pattern addresses a recurring design problem that arises in specific design situations and presents a solution to it +\item A pattern must balance a set of opposing forces +\item Patterns document existing, well-proven design experience +\item Patterns identify and specify abstractions above the level of single components (modules) +\item Patterns provide a common vocabulary and understanding for design principles +\item Patterns are a means of documentation +\item Patterns support the construction of software with defined properties, including non-functional requirements, such as flexibility and maintainability +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Describing Patterns} + +\begin{itemize} + +\item Context: the situation giving rise to a design pattern +\item Problem: a recurring problem arising in that situation, and +\item Solution: a proven solution to that problem + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{The Proxy Pattern (from van Vliet (2000))} + +\begin{itemize} + +\item Context: A client needs services from another component. Though direct access is possible, this may not be the best approach +\item Problem: We do not want to hard-code access to a component into a client. Sometimes, such direct access is +inefficient; in other cases it may be unsafe. This inefficiency or insecurity is to be handled by additional control +mechanisms, which should be kept separate from both the client and the component to which it needs access. +\item Solution: The client communicates with a representative rather than the component itself. This representative, the +\structure{proxy}, also does and pre- and postprocessing that is needed. + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Command Processor Pattern} + +\begin{itemize} + +\item Context: User interfaces which must be flexible or provide functionality that goes beyond the direct handling of +user functions. Examples are undo facilities or logging functions +\item Problem: We want a well-structured solution for mapping an interface to the internal functionality of a system. +All `extras' which have to do with the way user commands are input, additional commands such as undo and redo, and any +non-application-specific processing of user commands, such as logging, should be kept separate from the interface to +the internal functionality. + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Command Processor Pattern Continued} + +\begin{itemize} + +\item Solution: A separate component, the \structure{command processor}, takes care of all commands. The command +processor component schedules the execution of commands, stores them for later undo, logs them for later analysis, and +so on. The actual execution of the command is delegated to a supplier component within the application. + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\end{document} + diff --git a/Lectures/L37_ReviewForFinal/ReviewForFinal.pdf b/Lectures/L37_ReviewForFinal/ReviewForFinal.pdf new file mode 100644 index 0000000000000000000000000000000000000000..9d2b7fce3067d16463c7ec00388ba0ee284ecfe8 Binary files /dev/null and b/Lectures/L37_ReviewForFinal/ReviewForFinal.pdf differ diff --git a/Lectures/L37_ReviewForFinal/ReviewForFinal.tex b/Lectures/L37_ReviewForFinal/ReviewForFinal.tex new file mode 100755 index 0000000000000000000000000000000000000000..fa5cb74f5787690792d3e677fdb0921d24700d12 --- /dev/null +++ b/Lectures/L37_ReviewForFinal/ReviewForFinal.tex @@ -0,0 +1,291 @@ +%\documentclass[t,12pt,numbers,fleqn,handout]{beamer} +\documentclass[t,12pt,numbers,fleqn]{beamer} + +\usepackage{pgfpages} +\usepackage{hyperref} +\hypersetup{colorlinks=true, + linkcolor=blue, + citecolor=blue, + filecolor=blue, + urlcolor=blue, + unicode=false} +\urlstyle{same} + +\usepackage{hhline} +\usepackage{booktabs} +\usepackage{multirow} +\usepackage{multicol} +\usepackage{array} +\usepackage{listings} + +\usepackage{amssymb} +\usepackage{amsmath} + +\useoutertheme{split} %so the footline can be seen, without needing pgfpages + +% \pgfpagesuselayout{resize to}[letterpaper,border +% shrink=5mm,landscape] %if this is uncommented, the hyperref links do not work + +\mode<presentation>{} + +\input{../def-beamer} + +\newcommand{\topic}{37 Review for Final DRAFT} + +\input{../titlepage} + +\begin{document} + +\input{../footline} + +\lstset{language=java,breaklines=true,showspaces=false,showstringspaces=false,breakatwhitespace=true} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Review for Final Exam} + +\begin{itemize} +\item Administrative details +\item Design patterns +\item Topics on the exam +\item Structure of the exam +\item Advice on exam preparation +\begin{itemize} +\item Time management before the exam +\item Time management during the exam +\item How to study +\end{itemize} +\item Questions? Feedback? Comments? +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Administrative Details} + +\begin{itemize} + +\item Today's slide are partially based on slides by Dr.\ Wassyng and on van Vliet (2000) + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Model View Controller (MVC)} + +\begin{itemize} + +\item Separate computational elements from I/O elements +\item Three components +\begin{enumerate} +\item Model encapsulates the system's data as well as the operations on the data +\item View displays the data from the model components, possibly multiple view components +\item Controller handles input actions +\end{enumerate} +\item The controller may or may not depend on the state of the model +\item The controller depends on model state when menu items are enabled or disabled depending on the state of the model +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Design Pattern Properties} + +\begin{itemize} + +\item A pattern addresses a recurring design problem that arises in specific design situations and presents a solution to it +\item A pattern must balance a set of opposing forces +\item Patterns document existing, well-proven design experience +\item Patterns identify and specify abstractions above the level of single components (modules) +\item Patterns provide a common vocabulary and understanding for design principles +\item Patterns are a means of documentation +\item Patterns support the construction of software with defined properties, including non-functional requirements, such as flexibility and maintainability +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Describing Patterns} + +\begin{itemize} + +\item Context: the situation giving rise to a design pattern +\item Problem: a recurring problem arising in that situation, and +\item Solution: a proven solution to that problem + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{The Proxy Pattern (from van Vliet (2000))} + +\begin{itemize} + +\item Context: A client needs services from another component. Though direct access is possible, this may not be the best approach +\item Problem: We do not want to hard-code access to a component into a client. Sometimes, such direct access is +inefficient; in other cases it may be unsafe. This inefficiency or insecurity is to be handled by additional control +mechanisms, which should be kept separate from both the client and the component to which it needs access. +\item Solution: The client communicates with a representative rather than the component itself. This representative, the +\structure{proxy}, also does and pre- and postprocessing that is needed. + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Command Processor Pattern} + +\begin{itemize} + +\item Context: User interfaces which must be flexible or provide functionality that goes beyond the direct handling of +user functions. Examples are undo facilities or logging functions +\item Problem: We want a well-structured solution for mapping an interface to the internal functionality of a system. +All `extras' which have to do with the way user commands are input, additional commands such as undo and redo, and any +non-application-specific processing of user commands, such as logging, should be kept separate from the interface to +the internal functionality. + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Command Processor Pattern Continued} + +\begin{itemize} + +\item Solution: A separate component, the \structure{command processor}, takes care of all commands. The command +processor component schedules the execution of commands, stores them for later undo, logs them for later analysis, and +so on. The actual execution of the command is delegated to a supplier component within the application. + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Topics on the Final Exam} + +\begin{itemize} + +\item All of them +\item From ``introduction to software engineering'' to ``design patterns'' +\item Greater emphasis on the material since the midterm, such as +\begin{itemize} +\item Specification +\item Verification +\end{itemize} + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Types of Questions} + +\begin{itemize} + +\item Short answer +\item Complete the specification +\item Write small bits of code (C, OCaml or Java) +\item Design a module +\item List the test cases for coverage +\item Build the control flow graph +\item ... + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Time Management} + +\begin{itemize} + +\item Time management before the exam +\begin{itemize} +\item Make a schedule +\item Optimize the reward for spending your time and energy +\item Work smarter not harder +\item Schedule time for rest +\end{itemize} +\item Time management during the exam +\begin{itemize} +\item Divide time proportional to value of question +\item Start with what you know best +\item Leave nothing blank +\end{itemize} + +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{How to Study?} + +\begin{itemize} + +\item Better if an active, rather than a passive, process +\item Do questions +\begin{itemize} +\item From midterm, assignments +\item From the textbook +\item From other books +\item Make up your own +\begin{itemize} +\item MIS for an ADT that you have studied +\item MIS for 2C03 assignments +\item Questions from last year's final +\item Post questions to WebCT +\end{itemize} +\end{itemize} +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Questions?} + +\begin{itemize} + +\item Software quality? +\item Software principles? +\item Module decomposition? +\item MIS? +\item Parnas tables? +\item Fault seeding +\item White box testing? +\item Analysis? +\item ... +\end{itemize} + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\end{document} +