Figuring out how to balance resources for a software testing process can be overwhelming. Software components have different levels of complexity, which require different processing times. Different skill levels of resources, further modify the time to process. The order in which the components are scheduled provides additional complexity. Rework loops increase the level of difficulty. This is a multi-dimensional problem that can only be guessed at without simulation.
This article is only an example of how simulation and modeling can be applied to software development and testing. These techniques apply to many other software activities.
The real question we should be trying to solve is what’s the balance of resources that will minimize our cost and shorten our delivery time.
- How to Find the Software Testing Model
- How the Model Was Built
- Model Simulation & Output Report
- Model Optimization
- How to Import Raw Data
How to Find the Software Testing Model
Before we move further, let’s download the example model.
Download the Software Testing example model.
The example model needs ProcessModel to open and run. Make sure that you have ProcessModel installed on your computer. If you do not have ProcessModel, Request a Trial.
Once you have the model downloaded, open it in ProcessModel and let’s start reviewing how the model is built.
How the Model Was Built
1. Most of the objects in the model are color-coded.
1. All the activities are in light yellow.
2. All the resources are colored and numbered.
3. The squares in the light blue, green, or red background are highlighted texts that describe specific paths.
4. The objects at the bottom of the model, the people running, are used to attach the different optimizations files, which we will use later (below).
2. Activities have a separate input queue before them. For someone familiar with ProcessModel, this will look odd, since traditionally the input queue is defined in the activity by default. Here the input queue is separated so that we can define action logic in the input queue to capture resources, using the Excel files. Otherwise, Action Logic in the input queue, would not have been possible. The input queue in the activity itself is set to zero.
3. All activities have zero in the Time field. This is because all the times are accomplished using the action logic using attribute values, arriving with the entities.
4. Software Components are introduced into the model using a scheduled arrival. Opening the scheduled arrival, you can review all the arrivals having action logic. The attributes provide the processing time at each station.
5. The two Excel icons attach the two Excel files that will be needed to prepare and import user data into the model.
6. The three resources on the left have unique scenario parameters set as the quantity, p_NumbTester, p_NumbDesigner and so on. Scenario parameters provide a way to change the number of resources during experimentation.
7. Open the Attributes, Variables & Scenarios window by clicking Insert \ Attributes, Variables & Scenarios.
1. Entity Attributes:
1. a_TimeIN is used to record the time the Software Component enters the input queue.
2. a_Time_*Act-Name* provides the processing time for each entity at the activity name. The attribute is defined in scheduled arrivals.
3. a_ErrorRate_*Act-Name* The error rate attribute records potential errors on this software component.
2. Variables:
1. v_EntCheck# This is an internal variable to control the selection of resources.
2. v_*Act-Name*_TimeInQ Used to store the time software components have spent in the input queue, waiting.
3. Scenario Parameters:
1. p_Numb*Res-Name* is used to control the number of resources available for an experiment.
2. p_CostPer*Res-Name* Hr is used to control the hourly cost associated with a particular type of resource.
8. Notice there is action logic in the input queue and the action logic of the activities. Because we will be using the Excel files, we do not have to write or change this logic.
At this point, it is a good idea to go through the basic model flow to understand what is happening within the model. A software component comes in, goes to the input queue of the Design Test Cases, where the component must wait for a resource to become available. The software component then moves into the actual capacity. After capturing the proper resource and performing the processing time, the component moves to the input queue of Design Test Classes, and so on.
Model Simulation & Output Report
Now that we have a grasp of the model flow and what makes it work. Let’s simulate the model, and then view the Output Report.
Click the Simulate button to start the simulation.
You should now see a new window with the movement of the components animated. Spend some time reviewing where the animation to see where the components are getting stuck and so on.
On simulation end, click No, do not review the old Output Report, and then click the Output Report icon from the toolbar.
In this report, you can review the resource utilization, total system cost and the number of components (entities) approved (processed).
Model Optimization
Now that we have a grasp of the model and know how the output report looks like. Let’s try optimizing the model. We have already discussed the objects at the end of the model (the graphics of the people running), are used to attach the optimization files we have already created for this model.
Let’s try one of the 5 optimization files and see how it works. Once we have gone through one example, reviewing the rest should be simple.
Optimize for lowest cost and time
For the optimization of the model we will use SimRunner, which is an optimization tool built for ProcessModel.
Before optimization can work, the model must be simulated, then immediately open SimRunner.
Click Tools \ SimRunner to start the optimization tool. Once SimRunner starts, click File \ Open Project, browse to the folder where the Software Testing model is saved. Since we are optimizing for the lowest cost and time, locate the file named ‘Opt for lowest cost_Time.opt,’ and then click Open.
Click the Define objectives on the left. Review the Response statistics selected for objective function. We are optimizing for minimizing the number of resources, minimizing the total system cost, and minimizing the duration of the project.
Note that the v_TotalHours has an objective function of 100. In order to change the hours to be significant in comparison with the total cost, we had to add a waiting factor of 100.
Click Define inputs on the left. Review the Macros selected as Input Factors. We are using the scenario parameters we talked about above to change the number of resources assigned to activities. SimRunner will adjust the quantities of resources to find the target objective function. In this example, the number of testers can be as low as two and as high as six with a starting point of four, while the developers and designers can be a min of two and a maximum of six with the starting point of three. The default value is where the simulation would start experimenting first.
Now, click on the Optimize Model text, and then click Seek optimum on the left. Click Run to start the optimization process. In this model, we have set each experiment to run with ten replications. Each experiment will be compared by the objective function, the most valuable ones will rise to the top.
From the experiments above, we can see that the overall cost has gone up from 12,099.17 to 12,209.104, the total hours form 45.23 to 31.244 while using 5 testers, 4 designers, and 5 developers.
Review the other optimization experiments using the same instructions as above, by opening a different project.
How to Import Raw Data
You’ve just learned how the model flows, how to view the output report, and how to optimize to achieve a specific goal. Now we need to learn how to import your project data.
Software Component Scheduler
Right-click on the Excel icon with the text Software Component Scheduler. Click the third option from the bottom to open the Scheduled Arrivals with Table v-x-x workbook. This allows us to do two things:
1. It allows the scheduling of the order to process software components; high complexity first, low complexity first, intermingled, etc.
2. It provides an easy way to enter the time for each software component at each stage of processing, based on complexity.
To learn in detail on how this worksheet works, see Scheduled Arrivals with Table Input.
1. Column A / Time: Enter in the following date for the start of each software component: 11/04/2019 12:01:00 AM.
We are using this date because we are not using shifts and off-shift time. The model will be running for a continuous time and because the date will provide the exact starting time for when the simulation starts.
2. Column B / Qty: Add quantity of components for each level of complexity.
3. Column C / Codes: For this example our codes are the complexity level. However, you could change the timing based on characteristics of the components to be tested. In this example we will use just the complexity level.
4. Copy all of the time attributes from the model. An easy method of transferring attributes from the model to the Excel Input Sheet is to export the model file and select the Attributes tab. Then select all the Time attributes. Copy the selected attributes. Place the cursor in cell D6 of the Schedule Input sheet.
Paste, using Transpose to switch from vertical to horizontal orientation.
5. Goto the worksheet called Table (Time).
6. Paste the attributes again, into column A3.
7. Paste your codes into cell C2.
8. Add your base times and times/distributions based on each level of complexity.
The base time is the amount of time used, irrespective of the level of complexity. This is administrative work performed before you start the actual testing. If there is no administrative work then change the base time to zero.
The time values can be entered as an integer value or a distribution. Generally, the times will be a distribution that describes how long it takes to perform an activity, for a given complexity level. In the example below, you will notice that design test cases is very simple for level one requiring from twelve to twenty minutes, where as a level five takes two to four hours to plan. You will change these to match the time requirements for your system.
9. Go back to the Ent (Software_Component) worksheet.
10. Click the Populate Data button to populate the values into the columns.
Why is the populate button important to you? It’s important because switching the order or alternating the order of components can be accomplish in seconds with updated times for each component.
11. Go back to ProcessModel, export the data and close the export file. Click Tools \ Export Data. Click in positive for all prompts.
This is done to prepare the models to be read into ProcessModel.
12. You should now see the Software TestingData worksheet open for you automatically. Close this worksheet.
13. Return to Scheduled Arrivals with Table v-x-x workbook and click the Update* button to import the new data into ProcessModel.
* Before clicking the Update button, close all dialogs in ProcessModel.
Software Planner
The resource planner allows quick reassignment of resources and setting up priorities when resources have multiple responsibilities. When information changes on this sheet, automation updates the model.
Right click on the Excel icon with text Software Planner, click the third option from the bottom to open the Software Planner vx-x workbook. Once the workbook opens, we can define times, error rates, skill and assign resources.
The first four sheets will be used to setup, the fifth worksheet could be changed on an ongoing basis.
1. We are simulating uninterrupted hours for this model. This means that limited changes will be made to the Shifts worksheet.
If this was our first time using the worksheet, we will click the Setup Workbook button.
2. On the second worksheet, Time Definition, define the skill levels for each resource type (developer, designer, and tester) could have. In this example, the resources are defined numerically, one equals a tester, two equals a designer, and three equals a developer.
Then define the time multipliers. In this example, a resource with a skill level of three would take 40% less time (0.6 * time defined in the attribute) than the skill level one person for designing test cases. You can change these multipliers to match your requirements. ADD You could have different skill levels for developers assigning Most accomplished developers to concentrate on the most difficult problems first. For this example, we will continue with the three skill levels.
3. The third worksheet, Error Rate.
Define the percent error expected from each resource skill type for each activity. In this example, the worksheet is set up so that a skill level three person will have a 1% error rate on design test cases.
4. The fourth worksheet, Skills Portfolio.
Define the skill level of each resource for every activity. For example, at the activity Design Test Cases, all Skill Levels have the capability to do this job.
5. The fifth worksheet, Resource Assignment.
Use the color-coding to see which resource has what skill for which activity, and then assign the resources to a particular activity by using a priority number. For example, Tester will work on Designing Test Cases, Execute Tests, and Evaluate Test Results, with the priority being designing test cases. The higher the number, the higher the priority.
For example, To assign all resources to work on Design Test Cases, enter a priority in cell C7 and C8. Under this scenario, the Tester would be the first choice, the Developer the second choice and the Designer would be the third choice.
6. Click the Import Information to ProcessModel * button to import all the changes to your model.
* Before clicking the Import Information to ProcessModel button, close all dialogs in ProcessModel.
This important because, making change to resource assignments when many other factors are included, becomes difficult. All the changes are automated and performed within seconds.
The concept of this model and training can be applied to many software testing and development scenarios. Adding new activities, changing the names of the activities or starting a new model from scratch is easily accomplished with the two Excel files in minutes.
You can probably think of some additional changes that you might want to try to reduce the time and cost to complete the defined work. For example: is it possible to have higher-skilled workers participate in lower-skilled areas when they are not busy? Or how about mixing the scheduling of software component complexity levels. You might schedule 1 difficult, 2 hard, 4 medium, etc, then repeat the pattern. Each of these changes takes only minutes to prepare and test. You might have other ideas you want to try. Have fun and give us your feedback.
You might have a lot of ideas you want to try. Go ahead and have fun. We would love to hear your feedback.
Leave a Reply