Simrunner is a tool to intelligently optimize models towards an improvement goal. For every optimization project, SimRunner employs advanced optimization algorithms to enhance multiple factors at the same time, based on a validated model, an objective function to assess system performance, and a set of factors that SimRunner can modify to enhance system performance.
Where Do I Begin?
Start with a validated model
Once you complete and validate your simulation model, you are ready to begin an optimization project. If you are not working with a valid model, you don’t need to perform an optimization until the output from the simulation is valid.
Identify your simulation type
It is important to properly identify the simulation type—terminating or non-terminating. What does it mean to refer to a simulation as terminating or non-terminating? A terminating system stops when some key event occurs like the end of the day. When you come back the next day, you start fresh again. A non-terminating system is not necessarily a system that never stops production, rather it is a system that resumes from the point it left off. For both terminating and nonterminating simulations, you need to determine the appropriate run length and number of replications. For non-terminating simulations it is also necessary to determine the warm-up period.
Determine if model is a true candidate for optimization
Not every simulation model is built with the express purpose of optimizing some particular element. Many simulation models are built to demonstrate the relationships that exist between various elements of your system. If optimization is appropriate, define the objective function—the output statistics used to measure the performance of proposed solutions.
Use simulation to identify and examine potential solutions
Simulation has always been trial and error when it comes to optimization. We have some sort of optimization method that we apply to the model and we examine the model’s output statistics to see if we achieved the desired outcome. This is not a bad approach if you have one decision variable you are trying to optimize—but what if you are trying to optimize multiple decision variables at once? Interaction becomes very complex and requires more advanced optimization methods like SimRunner.
Define scenario parameters
When you build your model, you must define a scenario parameter for any variable you want to optimize. This provides SimRunner with a series of values it can change as it seeks to optimize your model. In SimRunner, these values are called factors. For information on how to add and modify Scenario Parameters see Scenario Parameters.
Screen factors
Part of the simulation process is to evaluate the relationships that exist between model elements, or factors. Often, you will take the time to adjust some part of your model to find that the adjustment has no impact on system performance. Factor screening is the process of identifying which model elements (factors) do not affect the output of your model, and narrowing your search to include only those factors that affect the model’s output. Be discriminant in your selections.
Relax and wait
While some models contain relatively few factors that you can quickly optimize, others contain many. In a previous inventory reduction project, a high-end computer took approximately 24 hours to compute what it estimated to be the optimal value for the model. Although it took a long time to produce this result, the net savings were tremendous.
Consider the results
While there is no promise that SimRunner will identify the optimal solution to your process, it is possible. SimRunner will, however, find better solutions than you would likely get with your own trial and error experimentation. The surest way to know the optimal solution to any model is to run an infinite number of replications of all possible inputs. Since this is not an option, take into consideration the number of experiments you are able to perform and act accordingly.
General Procedure
The following is an overview of the process you will use to perform an optimization of your system.
Step 1: Create, verify, and validate
The most important preparation you can make for an optimization project is a validated model. It is not enough to simply create a model—it will profit you nothing if the model does not reflect the real operation. Once you validate the model, you are ready to begin.
Step 2: Build a project
With your model prepared for evaluation, create a new SimRunner project and identify the response statistic you wish to target. Using these response statistics, define an objective function by which to gauge system performance. SimRunner will use this objective function to measure system improvement. Next, select the input factors you will allow SimRunner to use as it determines how best to achieve system improvement. When you optimize the model, SimRunner tests each input factor to seek the combination of factors that will result in the greatest improvement of model performance.
Step 3: Run experiments
Once you select the input factors and define the objective function, you can use SimRunner to automatically conduct a series of experiments on your model. SimRunner runs your model for you and tests a variety of possible combinations of values. After it completes the tests, SimRunner lists the test results in order of the most to the least successful combination of factor values.
Step 4: Evaluate suggestions
The fourth step is to consider and evaluate SimRunner’s suggestions. This is crucial because SimRunner will often identify several candidate solutions to your problem and you may, for reasons not addressed in the model, prefer one solution over another. You may also wish to make additional model runs (replications) and look at confidence intervals to further evaluate SimRunner’s list of possible solutions.
Step 5: Apply solution
Once you identify the solution that best fits your needs, implement the solution.
Pitfalls
If you follow the general procedure stated previously, your chances of success are very good. Typically, projects fail because the:
Model is not valid
Analysis considers insignificant factors
Analysis ignores significant factors
Objective Function is inappropriately formulated
Test results are not scrutinized
The SimRunner Interface
The SimRunner interface provides you with easy access to every step necessary to create an optimization project. The project building process is divided into three phases, each containing a series of steps necessary to complete the phase. As you move from phase to phase (displayed at the top of the dialog), a list of steps for the phase appears in the left pane.
Set Up a Project
The first thing you must do to create an optimization project is to start a ProcessModel simulation of your model. You don’t need to complete it. The simulation just needs to be started in order to load the data required by SimRunner. After ending the simulation and returning to the modeling window, click the Tools menu and select SimRunner.
The model data will be loaded automatically.
Define Objectives
The objective of your project is the final outcome you want to achieve. SimRunner measures your progress toward this goal using an objective function. An objective function is composed of response statistics, a min/max or target range, and a specific weight you wish to apply to each response statistic.
What is an Objective Function?
An expression used to quantitatively evaluate a simulation model’s performance. By measuring various performance characteristics and weighting them, an objective function is a single measure of how well a system performs. SimRunner allows you to include many different performance characteristics in one objective function. For example, if you want an objective function to include a measure of total entities processed and resource utilization you could measure how well a certain simulation scenario ran by measuring Z , where: Z = (Total Processed) + (Resource Utilization)
Response Category
The response category is the type of statistic you wish to use to evaluate your model. Response categories include model elements such as locations, entities, resources, and variables.
Response Statistics
By clicking on one of the Response Categories, you will see a list of Response Statistics. Simply put, response statistics are those values you wish to improve. Once you define your targeted improvements, you are ready to define how you want them to perform—the objective for the response statistic.
Objective for Response Statistic
The objective for the response statistic refers to the way in which you want to effect change for that item. If you are trying to increase the overall output of a system, you would maximize the response statistic. Likewise, you could minimize the statistic or target a specific range within which you want the result. Finally, enter the reward or weight for each response statistic—larger numbers signify greater rewards in situations that require more than one objective.
Max Check this option if you want to maximize the final value of this statistic.
Min Check this option if you want to minimize the final value of this statistic.
Target Range Check this option to enter a specific target range within which you want the final result.
Statistic’s weight Weights serve as a means of load balancing for statistics that might bias the objective function. Since most simulation models produce a variety of large and small values, it is often necessary to weight these values to ensure that the objective function does not unintentionally favor any particular statistic. For example, suppose that a run of your model returns a throughput of .72 and an average WIP of 4.56. If you maximize throughput and minimize WIP by applying the same weight to both ( W 1 = W 2 ), you will bias the objective function in favor of WIP:
Maximize [(W 1 )*(Throughput)] = .72
Minimize[(W 2 )*(WIP)] = 4.56
In this case, since you want to ensure that both statistics carry equal weight in the objective function, you will apply a weight of 6.33 ( W 1 =6.33) to throughput and 1.0 ( W 2 =1.0) to WIP to make them of equal weight in the objective function.
Maximize[(W 1 )*(Throughput)] = 4.56
Minimize[(W 2 )*(WIP)] = 4.56
In situations where it is necessary to favor one statistic over another, balancing the statistics first will make it easier to control the amount of bias you apply. For example, if you apply a weight of 12.67 ( W 1 =12.67) to throughput and 1.0 ( W 2 =1.0) to WIP, the objective function will consider throughput to be twice as important as WIP (adapted from Harrell, Ghosh, and Bowden 2000).
Typically, you will need to experiment with your model to identify the weight ratio necessary to balance statistics.
Response statistics selected for objective function
After you define the objective for the response statistic, you may click the Add button to include the statistic as part of the objective function. SimRunner combines the statistics into a linear combination and displays the updated objective function for the project.
Objective Function
The objective function is an expression used to quantitatively evaluate a simulation model’s performance. By measuring various performance characteristics and taking into consideration how you weigh them, SimRunner can measure how well your system operates. However, SimRunner knows only what you tell it via the objective function. For instance, if your objective function measures only one variable, Total_Throughput, SimRunner will attempt to optimize that variable. If you do not include an objective function term to tell it that you also want to minimize the total number of operators used, SimRunner will assume that you don’t care how many operators you use. Since the objective function can include multiple terms, be sure to include all of the response statistics about which you are concerned.
SimRunner’s capacity to include many different response statistics in an objective function gives it tremendous capability. For example, the objective function below signifies that you wish to maximize the total ovens and cooktops processed while minimizing the total resource cost. The numeric weighting factors indicate that maximizing Total Ovens Processed is the most important, followed by maximizing Total Cooktops Processed, then minimizing Total Resource Cost.
Perhaps the best way to define your objective function is in terms of cost or profit. When possible, this allows you to use a simple, single response statistic like maximize profit without concern over assigning meaningful weights to multiple response statistics.
SimRunner’s objective function is calculated by multiplying the result of each objective by its weighting factor, and then adding each product together. Maximized and Target Range objectives are positive and minimized objectives are negative.
For example, if you maximize the number of entities processed, minimize their cost, and minimize the number of resources used, the formula would be as follows:
The following example shows how to achieve a throughput target level. Suppose you want to target a production rate of 300 to 325 units per day. In the target range fields, assign a range of 300 to 325 and enter a weight of 1.
Target Range
The target range in SimRunner is a means of applying bounds to the output values. SimRunner calculates the mean of the range you specify. The closer an entity is to that mean, the higher the value that element of the objective function returns.
If the value returned is below the mean of the objective function, the formula is:
Objective Function Element = Weight * (Value – Min)
If the value returned is above the mean of the objective function, the formula is:
Objective Function Element = Weight * (-1) * (Value – Max)
If the value returned is equal to the mean of the objective function, the formula is:
Objective Function Element = Take any of the above (above or below mean) calculated value
For example, suppose you want to assign a target range to the total number of bicycles processed. You want your objective function to return a value nearest to 400 and your acceptable target range is 300 to 500. Assume your weighting factor is 1. If the number of bikes processed was 380, SimRunner would calculate the value of that element of the objective function as 1 * (380 – 300) = 80. If the number of bikes processed was 420, the value of that element of the objective function would be 1 * (-1) * (420 – 500) = 80. If the number of bikes processed was 400, the value of that element of the objective function would be 1 * 400 = 400.
The key to using the target range is that it must be a RANGE of possible values. If you defined the target range as 400 to 400, the range of possible values would be 0. Therefore you would never get a value besides 0 out of that element of the objective function.
The objective function, in its entirety, is the sum of all elements of the objective function. It calculates the value of each element by multiplying the weight by the returned value. Then, if you are minimizing that element, it is multiplied by -1. All of those elements are added together to calculate the total objective function value.
Define Inputs
In every system, there are controllable and uncontrollable factors that determine the outcome of the process. Controllable factors include staffing, equipment, schedules, and facilities. Uncontrollable factors refer to such things as arrival rates.
Often, relationships exist between various controllable factors. How do you identify these relationships and use them to seek the best value for your objective function?— SimRunner. SimRunner allows you to target controllable model factors and determine which combination of values for those factors will elicit the behavior you desire.
With each test it performs, SimRunner examines the results to see if the results will produce the effect you require from the objective function. For information about how SimRunner performs these tests and evaluates data.
Scenario Parameters listed in model
All Scenario Parameters defined in your model are displayed in SimRunner as Macros. In this manual macros will consistently be referred to as Scenario Parameters. SimRunner will use these Scenario Parameters to improve the value of your objective function. Typically, these are the controllable factors of your model.
Scenario Parameter Properties
Scenario Parameter properties describe the basic attributes of each Scenario Parameter used in your project.
Data Type: the data type refers to the numeric type of the data (integer or real) SimRunner will use. Typically, you will use integers to represent the number of resources (e.g., people) and real numbers to represent time values or percentages (e.g., machine processing times or product mix).
Default Value: Refers to SimRunner’s initial setting for the Scenario Parameter—the value SimRunner will use to analyze the model before conducting the optimization.
Lower and Upper Bound: The limits (constraints) within which this value must fall during subsequent tests.
Scenario Parameters Selected as Input Factors: Displays a list of all Scenario Parameters you selected for use as input factors. Typically, you will want to use as few inputs as possible—only those you anticipate will affect the value of the objective function.
Define Scenario Parameters
Before you can use a model element as an input factor, you must define a Scenario Parameter for that element. Scenario Parameters allow SimRunner to control the value of the factor and feed it back into the model.
Variables
If you want to use a variable from your model as an input factor, you must define a corresponding Scenario Parameter and set the variable equal to the Scenario Parameter at some point in your model. To do this, you can:
• Enter the name of the Scenario Parameter in the variable’s “initial value” field.
• Set the variable equal to the Scenario Parameter in the model’s processing logic.
Resource & Activity Capacities
In order to use a resource or activity capacity as an input factor, you will need to define a Scenario Parameter that represents the capacity or number of resources and place this Scenario Parameter in the appropriate field for the resource or activity. For instance, if you have a resource named Operator_1, you can create a Scenario Parameter named Number_Of_Operators to represent the “number of units” for the operator. When you run SimRunner, the Scenario Parameter Number_Of_Operators will appear on the list of input factors. Testing this factor will change the number of units of Operator_1 in the model.
SimRunner will not recognize any Scenario Parameter containing non-numeric characters (e.g., “N”, “)”, or “e+”) as a valid input from the scenario parameters “text” field.
Analyze Model
The first step in conducting any analysis of your simulation model’s output is to make sure that your model produces output with the degree of accuracy you desire. If all of the aspects of your model are deterministic (with no random variance in any part of the model), you will get a precise output statistic with one replication. In most models, however, random variance is present in several places. The result of using random inputs in the model is that the output is also random. Because of this, you should think of a simulation experiment as a sampling technique. Since you don’t know the exact, true value or population statistic, the best you can do is collect a sample large enough to make a good estimate—you must run multiple replications. ref: Analyze Model.
In general, with each additional replication you run, you generate a more accurate estimate of the population statistic your are trying to measure in the objective function. For example, suppose you wish to maximize the utilization of a given resource by changing several factors in the model. For each experiment, you may need to run ten replications to get an accurate estimate of the utilization for a specific combination of settings. If your estimate isn’t accurate enough, you will not be able to tell if improvements to a system are the result of changes or simply an effect of random variance. SimRunner helps you select the appropriate number of replications for both terminating and nonterminating systems.
For non-terminating systems, SimRunner’s Objective Function Time Series graph shows the value of the objective function as it changed during the simulation based on several replications. At the very beginning of a simulation run, the simulation model is “empty.” There are no entities in the system and often all the statistics are set to zero. This is an artificial condition and, for non-terminating simulations, does not reflect reality. When you run a non-terminating simulation, you must run it long enough for the operation to stabilize—to reach steady state. The warm-up time is how long it takes the simulation to reach steady state. The end of the warm-up time is the point at which you want to begin sampling statistical data—data representative of how the system normally operates.
When you define a warm-up period, the model runs for a specific period of time at the beginning of each replication before results are recorded.
How do you know how long the warm-up period should be? SimRunner uses an approach based on Welch’s graphical method. SimRunner displays a time series graph of the objective function’s current value and moving average based on several replications of the simulation. As the objective function begins to stabilize, the moving average graph will appear to “flatten out” around a specific value or range. This behavior signals the end of the warm-up period.
It is quite possible that you may be unable to determine the warm-up period from just one look at the graph. If you are unable to make this determination, run the model again with a longer run length.
Define Experiment
In the case of a non-terminating system, the warm-up period is the amount of time the model must run for the objective function to exhibit statistical regularity—to reach a steady-state. This occurs when the distribution of the objective function values is the same from one time period to the next. SimRunner helps you identify the length of the warm-up period.
Experimental Parameters
Simulation Run Length: The total duration of the simulation. If you run a non-terminating simulation, the run length should be sufficient to allow the model to warm up and reach steady state. Start with a large initial value for the run length.
Output Recording Time Interval (period): SimRunner records output statistics during regular intervals of time. This sampling technique provides you with an observation of the output statistic for each time interval. The
refers to the length of each time interval. The recording time interval should be large enough to ensure that simulated events occur, and that the model produces an output value for each response statistic listed as part of the objective function. For example, given a job arrives to a queue every 30 minutes, you do not want to select a recording time interval of less than 30 minutes if the average time jobs wait in the queue is part of the objective function. Be sure to set the recording time interval to observe at least one observation during each interval.
Number of Test Replications: The number of replications needed to conduct the analysis. The number of test replications should be five or more depending on how long you have to wait for results.
Percent Error in Objective Function Estimate: Given that you can only estimate the true value of the objective function, you have to decide how accurate your estimate needs to be. Entering a 10 indicates that you wish to estimate the required number of replications needed to obtain an estimate of the average value of the objective function to within 10% of its true value.
Confidence Level: The confidence level (90%, 95%, or 99%) used to approximate the number of replications needed to estimate the value of the objective function to within the stated “percent error” of its true value. The higher the confidence level, the larger the number of replications needed will be.
Conduct Analysis
Start Analysis
Run: Begin analyzing the model.
Final Report: Produces an analysis report which describes the results of each experiment.
Analysis Status: Displays the completion status of the analysis.
Warm-up: Detection for steady-state estimates.
Moving Average Window: Allows you to adjust the number of periods used to compute the moving average plot. The plot is automatically redrawn.
Estimate required number of replications
Warm-up Periods: Enter the number of time periods you wish to use as the warm-up period when estimating the required number of replications. Periods refer to the frequency with which you recorded data from the model. For example, if your output recording time interval is four hours, you may find that five periods will suffice (20 hours).
No. of Replications: This field is automatically updated as the warm-up periods field is defined.
Introduction
Often, the reason for building a simulation model of a system is to answer questions such as: “What are the optimal settings for to minimize (or maximize) ?” You can think of the simulation model as a black box that imitates the actual system. When you present inputs to the black box, the box produces outputs that estimate how the actual system will respond. In the question above, the first blank represents the input(s) to the simulation model that you control. (These inputs are often called decision variables or factors.) The second blank represents the performance measure(s) of interest, computed from the output response of the simulation model when you set the factors to specific values (see example below).
In the question, “What is the optimal number of material handling devices needed to minimize the time that workstations wait for material?”, the input factor is the number of material handling devices and the performance measure is the time that workstations wait (computed from the output response of the simulation model). The goal is to locate the optimal value for each factor that minimizes or maximizes the performance measure of interest.
Example
The following diagram shows the relationship between an optimization algorithm and a simulation model (adapted from Harrell, Ghosh, and Bowden 2000).
If you were to evaluate all combinations of the different values for the input factors (X 1 , X 2 , X 3, …) and plot the output response generated by the simulation model, you would create what is called a response surface of the output for that model. Think of the response surface as a mountainous region with many peaks and valleys. In the case of a maximization problem, you wish to climb to the highest peak. In the case of a minimization problem, you wish to descend to the lowest valley.
Optimization Concepts
To optimize functions or the output from simulation models, you must apply some kind of method to evaluate different combinations of input factors until you arrive at what you consider the optimal solution. If you take several different input/output combinations (i.e., function evaluations), you should be able to establish which variables cause improvement in the output response and try a few more function evaluations. By ascending the “peak” (in the case of a maximizing problem), you will work your way to the top.
Once you climb to a peak (i.e., reach a local optimum), how do you determine whether your local optimum is the global optimum—the very best? How do you know that there is no better combination of inputs? The surest way would be to try every possible combination of input factors. This is not feasible for most problems. What you can do, however, is try to find and compare several local optimums. This will greatly increase your chance of finding the best, or global, solution.
In many situations, it is impossible to formulate a single equation that precisely describes a system’s behavior. For this reason, techniques such as simulation are used to model and study these systems. Although simulation itself does not optimize, it acts as a function to tell you (based on your model of the system) what results you will achieve from a given set of input factors. SimRunner conducts a variety of tests to seek ideal operation levels for your model.
When setting up your optimization, you can do several things to greatly improve SimRunner’s performance:
Limit the number of input factors
Don’t include input factors that have relatively little impact on model results. Although SimRunner can handle multiple factors, each factor you add increases the amount of time necessary to optimize the model. So, the fewer the number of factors, the faster you will get results.
Set good, tight bounds
It takes a lot longer to find an answer between 1 and 1000 than between 1 and 10. The tighter the bounds you specify for your input factors, the faster you will get results.
Formulate a good objective function
Since SimRunner uses the objective function to evaluate a solution’s performance, it is absolutely necessary that you include the right output responses in the objective function. This allows SimRunner to test several combinations of factors to see which arrangement will improve the desired aspect of the system’s performance. Remember also that your answers are only as precise as the system’s randomness allows. If your model gives results that are accurate only to ± 5, SimRunner’s solutions will be equally accurate ( ± 5).
SimRunner Optimization Techniques
SimRunner intelligently and reliably seeks the optimal solution to your problem based on the feedback from your simulation model by applying some of the most advanced search techniques available today. The SimRunner optimization method is based upon Evolutionary Algorithms (Goldberg 1989, Fogel 1992, and Schwefel 1981).
Evolutionary Algorithms are a class of direct search techniques based on concepts from the theory of evolution. The algorithms mimic the underlying evolutionary process in that entities adapt to their environment in order to survive. Evolutionary Algorithms manipulate a population of solutions to a problem in such a way that poor solutions fade away and good solutions continually evolve in their search for the optimum. Search techniques based on this concept have proven to be very robust and have solved a wide variety of difficult problems. They are extremely useful because they provide you with not only a single, optimized solution, but with many good alternatives.
Set Options
Optimization Options
Optimization Profile: SimRunner provides three optimization profiles: Aggressive, Moderate, and Cautious. The optimization profile is a reflection of the number of possible solutions SimRunner will examine. For example, the cautious profile tells SimRunner to consider the highest number of possible solutions—to cautiously and more thoroughly conduct its search for the optimum. As you move from aggressive to cautious, you will most often get better results because SimRunner examines more results—but not always. Depending on the difficulty of the problem you are trying to solve, you may get solutions that are equally good. If you are pressed for time or use relatively few input factors, a more aggressive approach—the aggressive profile—may be appropriate.
The optimization profile affects the number of solutions SimRunner will evaluate before it converges. An aggressive profile generally converges quickly, but will have a lower probability of finding the optimum.
Convergence Percentage: With every experiment, SimRunner tracks the objective function’s value for each solution. By recording the best and the average results produced, SimRunner is able to monitor the progress of the algorithm. Once the best and the average are at or near the same value, the results converge and the optimization stops. The convergence percentage controls how close the best and the average must be to each other before the optimization stops. A high percentage value will stop the search early, while a very small percentage value will run the optimization until the points converge.
Advanced Options
• Options: The following options are available if selected on the Advanced Options menu. Select Options \ Advanced to enable Max and Min generation selection.
• Max No. of Generations: The most iterations SimRunner will use to conduct the analysis. This impacts the maximum time allocated to the optimization search. Very high values allow SimRunner to run until it satisfies the convergence percentage.
• Min No. of Generations: The fewest iterations SimRunner will use to conduct the analysis—this impacts the maximum time allocated to the optimization search. Typically, you will set the minimum number of generations to 1; however, if you specify a very large value, you can force SimRunner to continue to seek the optimal solution even after it satisfies the convergence percentage.
Simulation Options
Disable Animation: Uncheck to activate the animation during run time. If you enable the animation, it will take longer to conduct the analysis but will not affect the model’s results.
Number ofReplications per Experiment: The number of times the model will run in order to conduct the experiment—to estimate the value of the objective function for a solution. (Typically, you will never conduct an optimization with only one replication.)
Warm-up Time: The amount of time the model must run before it reaches steady-state.
Run Time: The total run length of the model.
Confidence Level: If you specify more than one replication, SimRunner will compute and display a confidence interval based on the mean value of the objective function and its sample variance. The confidence level allows you to specify a 90%, 95%, or 99% confidence interval. Although you cannot determine the true mean value of the objective function for a given experiment (you can only estimate it), the confidence interval provides a range that hopefully includes the true mean. The higher the value you select for the confidence level, the more confident you can be that the interval contains the true mean. However, note that a 99% confidence interval is generally wider that a 90% confidence interval given a fixed number of replications.
Typically you want SimRunner to run the algorithm until it fully converges. Do this by specifying a very low value for convergence percentage, such as 0.01, and, if running in advance mode, a very high value for maximum number of generations, such as 99999, and a value of 1 for minimum number of generations.
Seek Optimum
Run: Begin optimizing the model.
Stop: Halt the optimization.
Performance Plot: Plots the best objective function value found throughout the optimization process.
Final Report: The final report contains several of the best solutions found. SimRunner sorts these solutions to allow you to evaluate them further before you make your final decision. These results can be saved to a file using the Export Optimization Data option found in the File menu.
Convergence Status
Phase 1 and Phase 2: SimRunner sometimes uses a genetic algorithm before using the evolution strategies algorithm based on the characteristics of the problem being solved. Phase 1 displays the convergence status of the genetic algorithm. Phase 2 displays the convergence status of the evolution strategies algorithm.
Generation: The current generation. Each generation represents a collection of experiments performed on the model.
Experiment: The current simulation experiment.
Response Plot
Independent variable 1 & 2: Select the input factors you wish to use to produce a partial response surface of the model’s output.
Update Chart: Updates the chart to reflect the data you selected.
Edit Chart: Accesses the controls you will need to change the appearance of the Surface Response Plot.
File Menu
New: Starts a new project.
Open Model: Opens an existing simulation model.
Open Project: Opens an existing optimization project.
Save: Saves the current optimization project to its original path and filename.
Save As: Allows you to save the project under a different name or in a different location.
Export Optimization Data: Exports the data found in the optimization grid to a comma-delimited file of your choice.
Recently Opened Files: Displays a list of all recently opened files.
Exit: Quits SimRunner.
Options Menu
Clear Optimization Data: Clears Optimization Data from the data grid.
Clear Analysis Data: Clears Analysis Data from the data grid.
Advanced: Provides an option to control Minimum and Maximum Generations. This affects the Setup Project \ Set Options screen, providing additional optimization control to the user.
Help Menu
Help Topics: Provides access to the help system.
About SimRunner: Shows the version of SimRunner being used.
Getting Started
In this chapter, you will create a new project from a validated model that contains Scenario Parameters and examine how the model might perform more optimally.
Set Up Project
Step 1: Simulate Model
1. From ProcessModel, load your model and begin a simulation (you can end the simulation as soon as it starts).
2. Start SimRunner by clicking the Tools menu and selecting SimRunner.
The example used in this chapter is of the Bicycle Manufacturing demo model available within ProcessModel.
To learn by going through an example, see Case Study 5 in the Self Teaching Guide.
Step 2: Define Objectives
1. Select the response category and statistic for each item you wish to monitor.
2. Click Add (the down arrow button). The statistic will appear in the selected response statistics window. You can remove portions of the function by selecting the up arrow button.
3. Enter the objective for the response statistic (Max, Min, Target range) and the weight to apply to the statistic. Select the Update button to apply the changes to the objective function.
4. Repeat this process to add any other response statistics. If you wish to remove a statistic, select the statistic to remove, then click the up arrow button. Once you have selected all the statistics you will use, click Next.
Step 3: Define Inputs
1. Select a Scenario Parameter you will use in the project from the Scenario Parameters list. Scenario.
What are termed Scenario Parameters in ProcessModel are termed Macros in SimRunner.
2. Click the down arrow button to add the Scenario Parameter to the input factors list.
3. Select the numeric type, then enter the default value, the lower bound, and the upper bound. Select Update to reflect the changes in the Scenario Parameter selected.
4. Repeat this process to add any other response statistics. If you need to change the properties for a Scenario Parameter that has already been added, select the Scenario Parameter to change, make the Scenario Parameter property change (Integer, Real, value and bounds), then click Update . If you wish to remove a statistic, select the statistic to remove, then click the up arrow button. Once you have selected all the statistics you want use, click Next.
Step 4: Set Options
1. Select the optimization profile and enter the convergence percentage you wish to use.
2. Select the confidence level.
3. Enable or disable the animation as necessary and specify the number of replications per experiment you wish to run.
4. Enter the warm-up time and run time for the model.
5. Click Next or Seek Optimum to continue.
Step 5: Seek Optimum
1. Click Run to start the optimization.
2. A plot of the optimization will be displayed automatically. But it can also be opened manually anytime following the optimization by clicking the Performance Plot button.
3. View the optimization data.
4. Click the Final Report button to display a written summary of the optimization.
Step 6: Response Plot
1. Enter the independent variables (Scenario Parameter input factors) you wish to use in the response plot and click Update chart.
2. If you wish to modify the appearance of the graph, click Edit Chart. The following dialog will appear.
Analyze Model
Step 1: Define Experiment Parameters
1. Enter the simulation run length and the output recording time interval (period).
2. Enter the number of test replications
3. Enter the percent of error in the objective function estimate and select a confidence level.
4. Click Next to continue.
Step 2: Conduct Analysis
1. Click Run to begin the analysis.
2. If you wish to adjust the number of periods used to compute the moving average (for steady state estimates), adjust the value for the Moving Average Window . The moving average plot will update to match the current settings.
3. Enter the warm-up time. SimRunner will suggest the number of replications you should run for the optimization project.
4. Click the Final Report button to view the analysis report.
SimRunner uses both genetic and evolution strategies algorithms with its primary algorithm being evolution strategies. The specific design of these algorithms for SimRunner is based on the work of Dr. Royce Bowden and other experts. Evolutionary algorithms have been extensively evaluated by academics and have proved reliable in optimizing simulated systems.
Evolutionary algorithms
An evolutionary algorithm is a numerical optimization technique based on simulated evolution. Solutions generated from the evolutionary algorithm must adapt to their environment in order to survive. Since each potential solution returns a specific result, you must establish an objective function to measure the performance of each solution. If the returned value falls within the acceptable range defined by the objective function, SimRunner will continue to examine the value as it searches for the optimal solution.
Examples
To help you better understand how SimRunner’s optimization process works, consider the following analogy. If you and a group of explorers found yourselves on the slopes of a mountain, in the dark, with nothing but radios and altimeters, how would you find the summit?
The first step would be to establish the current altitude of everyone in your group by recording each person’s altimeter reading.
Next, you would direct your group members to wander out in any direction for various distances, then stop and review their new altimeter readings. At the second reading, you find that some of your group has moved higher and some have moved lower.
By comparing the results of each altimeter reading, you can determine the general direction in which to proceed. Since you want to climb to the summit, you proceed in the general direction that had the highest reading (in this case, 2). Again, after everyone moves various distances, you stop to take another altimeter reading.
By taking the average of the group’s new altimeter readings, you can see that the overall reading increased. This confirms that you are moving in the right direction. Just to be sure, you repeat the process and take another reading.
As you will notice from the example, your group is beginning to converge upon a single point. The more you repeat the data collection process, the closer the group will get to one another, and the more certain you will be that you are at or near the summit. Just to be on the safe side, however, you even send a few members of your group out to remote parts of the terrain to be sure that there is nothing higher than the point you already identified.
Once your average altimeter reading is equal to the best in your group, you have converged upon the summit and can be confident that you have achieved your goal.
Conceptually, this is how SimRunner works—only the terminology is different. Instead of altimeters, explorers, and tests, you will use objective functions, input factors, and replications. Each time the explorers move, SimRunner calls this a generation. The concepts are the same.
Not an exhaustive search
In spite of what some may think, you don’t want an algorithm that will do an exhaustive search. In a previous inventory reduction project, modelers estimated the number of possible solutions to be around 9.38 x 10^37 —it was a big project. To perform an exhaustive search for the optimum value would take a lifetime. More than likely, you will never get the answer.
What you want is an algorithm that can efficiently explore the response surface (the output from the model) and focus on those areas returning good answers—without evaluating everything.
Suggested Readings
If you would like more information on optimization, see the following references:
Bowden, R. O. and J. D. Hall. 1998. Simulation Optimization Research and Development. Proceedings of the 1998 Winter Simulation Conference 1693-1698.
Hall, J. D. and R. O. Bowden. 1997. Simulation Optimization by Direct Search: A Comparative Study. Sixth International Industrial Engineering Research Conference 298-303.
Hall, J. D.; R. O. Bowden; and J. M. Usher. 1996. Using Evolution Strategies and Simulation to Optimize a Pull Production System. Journal of Materials Processing Technology 61:47-52.
Goldberg, D. 1989. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison Wesley, Massachusetts.
Schwefel, H. P. 1981. Numerical Optimization of Computer Models. Chichester: John Wiley and Son.
