-
Pavle Arezina authoredPavle Arezina authored
Code owners
Assign users and groups as approvers for specific file changes. Learn more.
TestPlan.tex 23.53 KiB
\documentclass[12pt, titlepage]{article}
\usepackage{booktabs}
\usepackage{tabularx}
\usepackage{hyperref}
\hypersetup{
colorlinks,
citecolor=black,
filecolor=black,
linkcolor=red,
urlcolor=blue
}
\usepackage[round]{natbib}
\title{SE 3XA3: Test Plan\\Gifitti}
\author{Team \#2,Gifitti
\\ Nicolai Kozel kozeln
\\ Riley McGee mcgeer
\\ Pavle Arezina arezinp
}
\date{\today}
\input{../Comments}
\begin{document}
\maketitle
\pagenumbering{roman}
\tableofcontents
\listoftables
\listoffigures
\begin{table}[bp]
\caption{\bf Revision History}
\begin{tabularx}{\textwidth}{p{3cm}p{2cm}X}
\toprule {\bf Date} & {\bf Version} & {\bf Notes}\\
\midrule
October 22 & 1.0 & Adding sections 1,2, and 7\\
October 24 & 1.1 & Adding functional req tests\\
October 25 & 1.2 & Adding non functional req tests.\\
October 30 & 1.3 & Adding Unit tests, and automated functional tests\\
November 30 & 1.4 & Final edits for revision 1\\
\bottomrule
\end{tabularx}
\end{table}
\newpage
\pagenumbering{arabic}
This document describes the test plan for the Gifitti application developed for 3XA3 at McMaster University.
\section{General Information}
\subsection{Purpose}
The purpose of this testing plan is to establish a set of tests that will test the product in its entirety to ensure that it fulfills the intended purpose. This will be accomplished by verifying that Gifitti satisfies the different functional and non-functional requirements that were assigned to it. Having test plans for any product is essential to be able to understand how well the product is satisfying the client’s needs and if there is room for improvement.
\subsection{Scope}
This testing plan will utilize different testing methods such as automated and user created as well as various techniques such as black box and white box testing, to establish if the project has any need for improvement. Two different products will be analyzed through these tests, the Proof of Concept and the first iteration of the final product. The Proof of Concept will be tested to ensure that a basic representation of the product will be demonstrated. The first iteration of the final product will be tested to verify the requirements.
\subsection{Acronyms, Abbreviations, and Symbols}
\begin{table}[hbp]
\caption{\textbf{Table of Abbreviations}} \label{Table}
\begin{tabularx}{\textwidth}{p{3cm}X}
\toprule
\textbf{Abbreviation} & \textbf{Definition} \\
\midruleGIF & Graphics Interchange Format\\
BMP & Bitmap\\
JPEG & Joint Photographic Experts Group\\
PNG & Portable network Graphics\\
TIFF & Tag Image File Format\\
\bottomrule
\end{tabularx}
\end{table}
\begin{table}[!htbp]
\caption{\textbf{Table of Definitions}} \label{Table}
\begin{tabularx}{\textwidth}{p{3cm}X}
\toprule
\textbf{Term} & \textbf{Definition}\\
\midrule
GIF & A lossless format for image files that supports both animated and static images.\\
Sprite Sheet &A series of images (usually animation frames) combined into a larger image (or images).\\
Dialog Class & Specifies the base class used for displaying dialog boxes on the screen.\\
File Path & A path, the general form of the name of a file or directory, specifies a unique location in a file system. \\
Bitmap & A representation in which each item corresponds to one or more bits of information, especially the information used to control the display of a computer screen.\\
Frame & One of the many still images which compose a complete moving picture.\\
Pixel & A minute area of illumination on a display screen, one of many from which an image is composed.\\
TIFF & A common format for exchanging raster graphics (bitmap) images between application programs, including those used for scanner images.\\
JPEG & A commonly used method of lossless compression for digital images, particularly for those images produced by digital photography.\\
PNG & A raster graphics file format that supports lossless data compression. \\
\bottomrule
\end{tabularx}
\end{table}
\subsection{Overview of Document}
This testing plan is broken up into distinct parts. Under the heading Plan, the basic information will be given on the product and the tests. System Test Description will contain the specific tests for the functional requirements stated for the product. These tests will broke down into what type of tests they are and the results they achieve depending on their specific input. Testing for non-functional requirements will follow the same format where input will be given and the output will be measured for all of the provided non-functional requirements. Under tests for proof of concept, the same format will be utilized as the functional testing but it will not be testing
the requirements for the project, but for the goals of the proof of concept. Furthermore there will be tests to compare Gifitti to the
original product it was based on and unit testing plans to ensure correct output is achieved through proper internal functions.
\section{Plan}
\subsection{Software Description}
Gifitti is a software product that allows the common user to be able to manipulate GIFS for their entertainment or commercial needs.
With an intuitive design, it allows a person who has never done any kind of graphical editing to be able to manipulate a GIF to achieve
the person's particular design. This product can also serve a commercial purpose in allowing the quick production of GIFs.
\subsection{Test Team}
The team to implement the test plan for the project will be Pavle Arezina, Riley McGee, Nicolai Kozel.
\subsection{Automated Testing Approach}
This test plan will utilize automated testing for verification that the GIF manipulation functionality changes the GIF properly. That is to say that the exported GIF image will match what is created by the user of our product.
\subsection{Testing Tools}
The only tool to be utilized to test this product will be the Microsoft Unit Test Framework that is native with Visual Studio.
\subsection{Testing Schedule}
The testing schedule is listed on the Gantt Chart which can be found \href{https://gitlab.cas.mcmaster.ca/kozeln/Gifitti/tree/master/ProjectSchedule}{ here.}
\section{System Test Description}
Testing allows the developers to detect errors that the test cases cover. With the functional and non-functional requirements the developer knows the expected results and can evaluate the results of tests accordingly. Pass or fail values are assigned to executed tests discreetly unless the test is to have an error tolerance for passing.
\subsection{Tests for Functional Requirements}
Functional requirements prescribe what services the software should provide. They capture the intended software effects on the environment and applicability conditions. These tests ensure that the functional requirements of Gifitti are fulfilled or the tests will help to discover any functional requirements not satisfied by Gifitti.
\subsubsection{Open GIF}
\paragraph{The User is able to open a GIF from a specified location}
\begin{enumerate}
\item{Select Properly Formatted GIF- id1\\}
Type: Manual Functional.
Initial State: Program loaded; no GIF Loaded.Input: File path for GIF.
Output: System loads gif into memory, displays it to the user in the gif view.
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Select Open}
\item{From the Open dialog specify a path to a known gif image}
\item{After the image is loaded verify that it is being displayed, and resides in system memory}
\\
\end{enumerate}
\item{No File Selected in File Dialog-id2\\}
Type: Manual Functional.
Initial State: Program loaded; no GIF Loaded.
Input: None.
Output: No image loaded.
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Select Open}
\item{Select open option with no file path specified}
\item{Verify program remains open, and no image is loaded}
\\
\end{enumerate}
\item{Close File Dialog-id3\\}
Type: Manual Functional.
Initial State: Program loaded; no GIF Loaded.
Input: None.
Output: No image loaded.
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Select Open}
\item{Close the file dialog}
\item{Verify program remains open, and no image is loaded}
\\
\end{enumerate}
\item{Open Random Non-GIF File-id4\\}
Type: Manual Functional.
Initial State: Program loaded; no GIF Loaded.
Input: File that is not a GIF.
Output: No image loaded.
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Select Open}
\item{Try and select a file that is not a GIF or specify a file path to a known file}
\item{Verify program remains open, no image is loaded, and an error message is displayed to the user}
\\
\end{enumerate}
%
%End of Open Requirement
\end{enumerate}
\subsubsection{Save GIF}
\paragraph{The user is able to save a GIF to a specified location}
\begin{enumerate}
\item{Save a GIF to a Known Location-id1\\}
Type: Manual Functional.\\
Initial State: Program loaded; GIF Loaded.\\Input: File path, GIF file.\\
Output: GIF file saved to specified location.\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Open a GIF}
\item{Select "Save As..."}
\item{Specify a known system file path and a saved image name}
\item{Verify the loaded and saved GIFs are identical}
\\
\end{enumerate}
\item{Save GIF to a Non-existent Location- id2\\}
Type: Manual Functional.\\
Initial State: Program loaded; GIF Loaded.\\
Input: File path, GIF file.\\
Output: GIF file not saved to specified location.\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Open a GIF}
\item{Select "Save As..."}
\item{Specify a system file path known not to exist and a saved image name}
\item{Verify the user is informed that the file path does not work and the system remains running}
\\
\end{enumerate}
\item{Save GIF to Opened Location- id3\\}
Type: Manual Functional.\\
Initial State: Program loaded; GIF Loaded.\\
Input: File path, GIF file.\\
Output: GIF file saved to specified location.\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Open a GIF}
\item{Modify the GIF}
\item{Select the save image option}
\item{Verify new GIF is saved over the originally opened GIF}
\\
\end{enumerate}
%
%End of save Requirement
\end{enumerate}
\subsubsection{GIF Start, Stop, Modify Length}
\paragraph{The user is able to start and stop GIFs as well as shorten the GIF and change the playback speed.}
\begin{enumerate}
\item{Play Stop GIF-id1\\}
Type: Manual Functional.\\
Initial State: Program loaded; GIF Loaded.\\
Input: GIF\\
Output: None.\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Open a GIF}
\item{Select Play Option}
\item{Verify the GIF is iterating over frames as expected}
\item{Select Play option again, verify no change occurs}
\item{Select Stop}
\item{Verify GIF stops playing}\item{Select Stop again, verify no change occurs}
\\
\end{enumerate}
\item{GIF shorten-id2\\}
Type: Manual Functional.\\
Initial State: Program loaded; GIF Loaded.\\
Input: GIF\\
Output: Shortened GIF.\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Open a GIF}
\item{Shorten gif frames in use}
\item{Play the GIF to verify it is correct}
\item{Export the GIF}
\item{Verify that the exported GIF represents the one playing, and is shorter than the original}
\\
\end{enumerate}
\item{Automated GIF Shortening-id3\\}
Type: Automated Functional.\\
Initial State: Image reading module driver \\
Input: GIF; Expected output GIF;\\
Output: Expected output GIF \\
How test will be performed:
\begin{enumerate}
\item{Test will load GIF from a known location}
\item{Test will subset the GIF to a known frame set}
\item{The GIF will then be compared to an expected GIF in a frame-by-frame manner}
\item{Each frame is to be cast to a bitmap then compared in a pixel-by-pixel manner}
\item{Pass if 100 percent of all pixels match}
\end{enumerate}
\item{GIF speed-id4\\}
Type: Manual Functional.\\
Initial State: Program loaded; GIF Loaded.\\
Input: GIF\\
Output: GIF inputted with different frame rate.\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Open a GIF}
\item{Increase frame hold time}
\item{Play the GIF to verify it is slower than original}
\item{Export the GIF}
\item{Verify that the exported GIF represents the one playing, and is slower, with respect to frame rate, than the original}
\\
\end{enumerate}
%
%End of GIF Start, Stop, Modify Length Requirement
\end{enumerate}
\subsubsection{Save All Frames in a GIF to Another Known Format}
\paragraph{The user is able to save a GIF frame by frame to PNG, JPEG, BMP and TIFF formats}
\begin{enumerate}
\item{Save PNG Frames to Known Location-id1\\}
Type: Manual Functional.\\
Initial State: Program loaded; GIF Loaded.\\
Input: Folder path, GIF file.\\
Output: PNG Frames in specified folder named corresponding to frame.\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Open a GIF}
\item{Choose to export the image as a PNG set}
\item{Select the folder where the PNGs will be dumped}\
\item{Verify the folder fills with GIF frames as PNGs}
\item{Ensure a 3rd party program can load the PNGs. Software such as Photoshop or MS paint is satisfactory}
\\
\end{enumerate}
\item{Save JPEG frames to known location-id2\\}
This test is identical to id1 with PNG replaced with JPEG
\item{Save BMP frames to known location-id3\\}
This test is identical to id1 with PNG replaced with BMP
\item{Save TIFF frames to known location-id4\\}
This test is identical to id1 with PNG replaced with TIFF
\end{enumerate}
%
%End of Save all frames in a GIF to another known format Requirement
\subsubsection{Help Context}
\paragraph{Verifies and validates the Help context}
\begin{enumerate}
\item{Help Context-id1\\}
Type: Manual Functional.\\
Initial State: Program loaded\\
Input: None\\
Output: None\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Select Help Context}
\item{Goes through all Help Context options, verify spelling for professionality}
\item{Close help context, verify program remains in same state as launch put it in}
\\
\end{enumerate}
%
%End of Help Requirement
\end{enumerate}
\subsubsection{Resize}
\paragraph{Verifies a GIF can be exported and resized}
\begin{enumerate}
\item{resizing-id1\\}
Type: Manual Functional.\\
Initial State: Program loaded\\
Input: GIF, Width, Height\\
Output: Resized GIF\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Load a GIF}
\item{Click Resize}
\item{Enter a width and height value}
\item{Choose save as and specify a file path.}
\item{Verify the GIF was exported successfully and resized accordingly.}
\\
\end{enumerate}
\item{resizing-id2\\}
Type: Manual Functional.\\Initial State: Program loaded\\
Input: GIF, Width, Height\\
Output: Resized GIF\\
How test will be performed:
\begin{enumerate}
\item{Launch the program}
\item{Load a GIF}
\item{Click Resize}
\item{Enter boundary cases for width and height values.}
\item{Verify these boundary cases do not cause the program to crash.}
\\
\end{enumerate}
%
%End of Help Requirement
\end{enumerate}
\subsection{Tests for Non-functional Requirements}
Non-functional requirements constrain how the functional requirements and services should be provided. They are related to the quality of the product, such as the safety of utilizing the software or how accurate it is. These tests ensure that the non-functional requirements of Gifitti are fulfilled.
\subsubsection{Performance Requirements}
\paragraph{Speed}
\begin{enumerate}
\item{speed-01\\}
Type: Manual Structural
Initial State: Program loaded; GIF loaded.
Input: GIF with number of frames on the magnitude of 100's of frames.
Output: Subsection of frames chosen to be exported.
How test will be performed: Create a timer inside the program to output the time it takes to output the files, to a debug.txt file. The time should be
less than MAX\_EXPORT\_TIME.
\item{speed-02\\}
Type: Manual Structural
Initial State: Program loaded;
Input: GIF.
Output: NA
How test will be performed: Load a GIF into the program. The time to load the GIF and start playback should be less than MAX\_UI\_LOAD. MAX\_UI\_LOAD is derived from the original system, GIF Viewer, so make sure this test is performed on the same system as the benchmark MAX\_UI\_LOAD is acquired from.
\end{enumerate}
\paragraph{Precision}
\begin{enumerate}
\item{precision-01\\}
Type: Manual Structural
Initial State: Program loaded; GIF loaded.
Input: GIF
Output: Subsection of frames chosen to be exported.
How test will be performed: Export the first 5 frames of the GIF. Navigate to the save location and verify that the number of frames saved is equal to 5.
\end{enumerate}
\paragraph{Safety Critical}
\begin{enumerate}
\item{safety-01\\}
Type: Manual Structural
Initial State: Program loaded; GIF loaded.
Input: GIF
Output: NA
How test will be performed: Export a subsection of frames from the GIF to a folder. Try to export to the same folder again. The program should ask if you are okay with overwriting the existing files.
\item{safety-02\\}
Type: Manual Structural
Initial State: Program loaded; GIF loaded.
Input: GIF
Output: NA
How test will be performed: Create a small partition of the HDD and fill it so there is no available space. Try to export frames to this location. The program should
not continue the operation and inform the user that disk space needs to be cleared before continuing.
\item{robustness-03\\}
Type: Manual Structural
Initial State: Program loaded; GIF loaded.
Input: GIF, Frame Range
Output: Selected frames if the input is a valid range.
How test will be performed: Attempt to enter invalid input to the frame selection text boxes and record the result. Program should function normally (and as expected) for all entries. This includes negative numbers, letters or other characters, and invalid ranges.
\end{enumerate}
\section{Tests for Proof of Concept}
The proof of concept is a demonstration of the feasibility of the software project. Ensuring that the Proof on Concept is producing the intended results can only be achieved through tests specifically designed to root out any of the errors present in the build of the proof of concept. By successfully testing the proof of concept and achieving the intended goals, it is shown that the software-to-be is a possible entity.
\subsection{Opening a GIF file for playback}
\paragraph{Open GIF}
\begin{enumerate}
\item{OpenGif-01\\}
Type: Manual Functional
Initial State: Program must be in normal state (form window is open and playback window is blank).
Input: GIF File
Output: GIF is shown in playback window of the form.
How test will be performed: Click open button, select a file of type .gif, and verify that the GIF loads and begins to playback within the form window.
\end{enumerate}
\subsection{Saving a GIF file's frames}
\paragraph{Save GIF}
\begin{enumerate}
\item{SaveGif-01\\}
Type: Manual Functional
Initial State: Program must be in playback state (form window is open and playback window is playing GIF).
Input: GIF File
Output: GIF's frames are saved as .bmp in folder specified.
How test will be performed: Click save frames button, select a folder, and verify that the GIF's frames are saved to the specified folder as .bmp.
\end{enumerate}
\section{Comparison to Existing Implementation}
\subsection{Graphics/UI}
\begin{enumerate}
\item GIF playback resolution is the same or better than Gif Viewer.
\item Program has the same button scheme as Gif Viewer (Open button to open file, Extract Frames button to save frames).
\item Program has a help menu available that is similar to Gif Viewer, but is available at all times.
\item Program's color scheme and design resembles Gif Viewer.
\end{enumerate}
\subsection{Performance}
\begin{enumerate}
\item GIF playback is at the same smoothness/framerate or better than Gif Viewer.
\item Opening and saving a file takes the same amount of time or less than Gif Viewer.
\end{enumerate}
\section{Unit Testing Plan}
The unit testing plan is the devised set of tests that the code and the system modules must undergo regardless of the system integration level. Unit testing allows the project to have white box testing on the individual system components where each unit test can be executed via testing frameworks in C Sharp.
\\
\\
The testing framework being utilized is the native Visual Studio unit testing framework: Microsoft.VisualStudio.TestTools.UnitTesting.
\subsection{Unit testing of internal functions}
Unit testing will be done on system methods that return a value, or modify known objects in an expected way. A metric for passing is to be used on all methods with an error tolerance, such as manipulation of a GIF (where slight variations can exist in files that present the same). A method with no expected error tolerance should be implemented as discreetly pass XOR fail. All unit test inputs must fall in a known domain where the expected output can be assumed. No stubs will be required for unit testing of functions. Image reading is needed as a driver for any method using GIFs or other image types. Any written unit test must utilize the C Sharp attributes for Unit Testing provided by the framework. This ensures the system does not build the testing package into the consumer version of the software. We plan to have unit testing cover approximately 50 percent of our major methods due to project time constraints.
\subsection{Unit testing of output files}
To test the validity of outputted GIF images, the testing system will hold image resources of input frames, input GIFs, as well as the expected GIF outputted. A known GIF should be manipulated and then compared to the expected GIF in a frame-by-frame matter. To exclude meta-data issues associated with GIF images, each frame should be cast to a known image format such as a bitmap, then compared pixel-by-pixel. Frames should have a 100 percent pixel match for the expected GIF to pass. A method for casting Frames to bitmaps and then comparing the pixels, should be added to the testing package prior to testing GIF manipulation. This function can then be verified by simply sending the same image(s) as parameters.
%\bibliographystyle{plainnat}
\section{Bibliography}
Microsoft Visual Studio Unit Testing Framework, Microsoft Incorporated; \\\href{https://msdn.microsoft.com/en-us/library/dd264975.aspx}{https://msdn.microsoft.com/en-us/library/dd264975.aspx} \\\\
Questionnaire Template, Chin, J.P., Diehl, V.A., Norman, K.L.; \\\href{http://garyperlman.com/quest/quest.cgi?form=QUIS}{ http://garyperlman.com/quest/quest.cgi?form=QUIS}
\newpage
\section{Appendix}
This section contains symbolic parameters for this document and the usability survey that will be delivered
to a focus group upon initial completion of the application.
\subsection{Symbolic Parameters}
The definition of the test cases will call for SYMBOLIC\_CONSTANTS.
Their values are defined in this section for easy maintenance.
\begin{enumerate}
\item MAX\_UI\_LOAD: 3 Seconds
\item MAX\_EXPORT\_TIME: 10 Seconds
\end{enumerate}
\subsection{Usability Survey Questions}The survey will be delivered in the same format as the questionnaire for User Interface Satisfaction. This
questionnaire is composed of various questions pertaining to several sub categories on a 0-9 scale. This includes the screen,
terminology and system information, learning, and system capabilities. It also allows the user to list the most
positive and negative aspects of the program. The questionnaire can be found at \href{http://garyperlman.com/quest/quest.cgi?form=QUIS}{garyperlman.com}
\end{document}\documentclass[12pt, titlepage]{article}