diff --git a/Lectures/L09_VerificationAndValidation/VerificAndValid.pdf b/Lectures/L09_VerificationAndValidation/VerificAndValid.pdf index db80664505414b427fad8f13e6b0fb06b4a15344..35806ff56838a6ea6e20dbfaf201aaf5f744eebb 100644 Binary files a/Lectures/L09_VerificationAndValidation/VerificAndValid.pdf and b/Lectures/L09_VerificationAndValidation/VerificAndValid.pdf differ diff --git a/Lectures/L09_VerificationAndValidation/VerificAndValid.tex b/Lectures/L09_VerificationAndValidation/VerificAndValid.tex index c6feab507caa69a396d9af127c895ab92bd08b19..5f26cc9ff80c7a92407812699f7738e27966f4b1 100755 --- a/Lectures/L09_VerificationAndValidation/VerificAndValid.tex +++ b/Lectures/L09_VerificationAndValidation/VerificAndValid.tex @@ -79,7 +79,7 @@ \item Enough issues, but shallow 1/2 \item Enough issues and deep (not surface) 2/2 \ei -\item Issues are due 2 days after your colleague assigns your issue +\item Issues are due 2 days after being assigned \ei \item Reading week next week, no 741 classes \item Work on your VnV plan during reading week diff --git a/Lectures/L10_VnVContinued/VnVContinued.pdf b/Lectures/L10_VnVContinued/VnVContinued.pdf index 9235e865a020732399e7a09222dcb2f7dc7adaa3..58c9c330eb21e2c21d9932c8dedc409289ce4bc4 100644 Binary files a/Lectures/L10_VnVContinued/VnVContinued.pdf and b/Lectures/L10_VnVContinued/VnVContinued.pdf differ diff --git a/Lectures/L10_VnVContinued/VnVContinued.tex b/Lectures/L10_VnVContinued/VnVContinued.tex index 16c745bd9be51a5e6129df2ca8776682ab37f562..26f6a5af1eab0f23c50a4aafd939f058fe54c8db 100755 --- a/Lectures/L10_VnVContinued/VnVContinued.tex +++ b/Lectures/L10_VnVContinued/VnVContinued.tex @@ -46,7 +46,7 @@ \item Start recording \item Administrative details \item Questions? -\item Finish what started last day +%\item Finish what started last day \item Nonfunctional software testing \item Theoretical foundations of testing \item Complete coverage principle @@ -64,24 +64,37 @@ \frametitle{Administrative Details} \bi -\item As the GitHub repo owner +\item Drasil projects look at \href{http://learnyouahaskell.com/} {Learn you a + Haskell for Great Good} \ei + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Admin Details: VnV Presentations} + \bi -\item Add your reviewers as collaborators -\item When your project is ready for review +\item Not everyone will do VnV presentations +\item Select 1 or 2 of the following: \bi -\item Assign your reviewers an issue for - them to create issues -\item Assign the instructor to review -\ei +\item \textbf{Specific} functional system test cases +\item \textbf{Specific} nonfunctional system test cases, such as +\bi +\item Performance profile +\item Usability testing \ei -\item As a GitHub reviewer +\item SRS verification plan +\item Automated testing and verification tools \bi -\item Assigned 2 colleagues (see \texttt{Repos.xlsx} in repo) -\item Provide at least 5 issues on their SRS +\item Profiling tools +\item Continuous integration +\item Code coverage +\item Linters \ei -\item Reading week, no 741 classes -\item V\&V template in repo -\item Adding a V\&V checklist to repo +\ei +\item We would like a variety of topics presented +\item If you are uncertain of your specific presentation plan, please ask \ei \end{frame} @@ -89,42 +102,45 @@ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \begin{frame} -\frametitle{Administrative Details: Report Deadlines} -~\newline -\begin{tabular}{l l l} -System VnV Plan & Week 08 & Oct 28\\ -MG + MIS & Week 10 & Nov 25\\ -Final Documentation & Week 14 & Dec 9\\ -\end {tabular} + \frametitle{Admin Details: Proof of Concept Presentations} \bi -\item The written deliverables will be graded based on the repo contents as of -11:59 pm of the due date -\item If you need an extension, please ask -\item Two days after each major deliverable, your GitHub issues will be due -\item Domain expert code due 1 week after MIS deadline +\item Deepen your understanding by jumping into implementation +\item Identify a risk with your code and implement enough to show you can + resolve it +\item Looking for an actual demo with running code +\item Presentation +\bi +\item Explicitly identify your risk +\item Run your code +\item Discuss your implementation +\ei +\item Simplify as much as necessary +\item Do not use this code in your actual implementation \ei \end{frame} -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \begin{frame} -\frametitle{Administrative Details: Presentations} - +\frametitle{Administrative Details: Report Deadlines} ~\newline -\begin{tabular}{l l l} -Syst.\ VnV Present & Week 07 & Week of Oct 21\\ -MG + MIS Syntax Present & Week 9 & Week of Nov 4\\ -MIS Semantics Present & Week 11 & Week of Nov 18\\ -Unit VnV or Impl.\ Present & Week 12/13 & Week of Nov 28\\ +\begin{tabular}{l l} +%\textbf{SRS} & Oct 8\\ +\textbf{System VnV Plan} & Oct 29\\ +MG + MIS (Traditional) & Nov 19\\ +Drasil Code and Report (Drasil) & Nov 19\\ +Final Documentation & Dec 9\\ \end {tabular} \bi -\item Informal presentations with the goal of improving everyone's written - deliverables -\item Domain experts and secondary reviewers (and others) will ask questions - (listed in Repos.xlsx file) +\item The written deliverables will be graded based on the repo contents as of +11:59 pm of the due date +\item If you need an extension for a written deliverable, please ask +\item You should inform your primary and secondary reviewers of the extension +\item Two days after each major deliverable, your GitHub issues will be due +%\item Domain expert code due 1 week after MIS deadline \ei \end{frame} @@ -132,40 +148,61 @@ Unit VnV or Impl.\ Present & Week 12/13 & Week of Nov 28\\ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \begin{frame} -\frametitle{Administrative Details: Presentation Schedule} +\frametitle{Admin Details: Presentation Schedule} \bi -\item Syst V\&V Plan Present +\item Syst V\&V Plan Present (15 min) \bi -\item \textbf{Monday: Deema, Peter} -\item \textbf{Thursday: Sharon, Ao} +\item Thurs, Oct 22: Ting-Yu, Mohamed, Naveen, Liz, Salah \ei -\item MG + MIS Syntax Present +\item Proof of Concept Demonstrations (15 min) \bi -\item Monday: Deema, Bo -\item Thursday: Sasha +\item Mon, Oct 26: Tiago, Mohamed, Xuanming, Parsa, Gaby +\item Mon, Nov 2: Sid, Shayan, Leila, Xingzhi, Liz +\item Thurs, Nov 12: Salah, John \ei -\item MIS Syntax + Semantics Present +\item MG Present (10 minutes) \bi -\item Monday: Zhi, Peter -\item Thursday: Sharon, Ao +\item Thurs, Nov 12: John, Tiago, Leila, Xuanming, Andrea \ei -\item Unit VnV Plan or Impl.\ Present +\item MIS Present \bi -\item Monday: Bo, Sasha -\item Thursday: Zhi, Peter, Ao +\item Mon, Nov 16: Shayan, Parsa, Gaby, Sid, Xingzhi +\ei +\item Drasil Project Present (20 min each) +\bi +\item Thurs, Nov 26: Andrea, Naveen, Ting-Yu \ei \ei \end{frame} -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +\begin{frame} +\frametitle{Presentation Schedule Continued} + +\bi +\item Test or Impl.\ Present (15 min each) +\bi +\item Mon, Nov 30: John, Salah, Liz, Xingzhi, Leila +\item Thurs, Dec 3: Shayan, Naveen, Sid, Gaby, Seyed +\item Mon, Dec 7: Ting-Yu, Xuanming, Mohamed, Andrea, Tiago +\ei +\item 4 presentations each +\item If you will miss a presentation, please trade with someone else +\ei + +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \begin{frame} \frametitle{Questions?} \begin{itemize} -\item Questions about SRS? +%\item Questions about SRS? \item Questions about V\&V? +\item Questions about PoC? \end{itemize} \end{frame} @@ -272,6 +309,30 @@ Unit VnV or Impl.\ Present & Week 12/13 & Week of Nov 28\\ %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +\begin{frame} +\frametitle{Description Rather Than Specification} +\begin{itemize} +\item Test cases are often phrased as Expected = Calculated +\item In scientific software you generally should not test for equality +\bi +\item Absolute error within tolerance +\item Relative error within tolerance +\item If comparing matrices or vectors, consider using norms of residual +\ei +\item Even a specific tolerance often doesn't make sense in a scientific context +\item Often your plan should be to \textbf{describe} the error rather than + \textbf{prescribe} +\bi +\item Plot of error versus problem size, or condition number, or ... +\item Consider summarizing multiple tests with the infinity norm of the relative + error (or similar) +\ei +\item Your description plan is part of your V\&V plan! +\end{itemize} +\end{frame} + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + \begin{frame} \frametitle{White Box Versus Black Box Testing} \begin{itemize}