Design of Experiments (DOE)

Skill level: Expert/specialty

Description

Designed experimentation is the manipulation of controllable factors (independent variables) at different levels to see their effect on a response (dependent variable).

Design of experiments (DOE) is used to solve problems from complex processes or systems where there are many factors influencing the outcome and where it is impossible to isolate one factor or variable from the others. Usually, it implies interaction between the variables that impacts the outcome.

One of the most complex and powerful tools in problem solving, DOE requires following a rigorous methodology when preparing the experiment, when executing the experiment, and when analyzing it.

Benefits

  • The designed experiment gives a mathematical model relating the variables and responses
  • The model can be easily optimized
  • Extremely flexible, robust statistical analysis tool that can be run with special statistical software packages
  • Very structured methodology that can be applied in any field or business

How to Use

  • Step 1.  Map the process under study and determine the key variables and settings.
  • Step 2.  Define the number of factors and level to be included in the experiment and how you can run the experiment (full factorial, fractional factorial).
  • Step 3.  Run the experiment in the order of the DOE and collect data from the output(s) or response(s).
  • Step 4.  Enter results from the output(s) into the statistical application used to create the DOE and run the DOE analysis.
  • Step 5.  Study results and draw conclusions.

Relevant Definitions

Full factorial: All factors at the set level are included.

Fractional factorial: Some factors at a set level are not included and will be inferred by the analysis.

Example

The marketing team of a consulting firm specializing in website design and online advertisements wants to determine the best way to increase the website conversion rate for its customers. There are many variables in a web page that can be changed, but the team agrees that the following factors, which change an ad’s effectiveness, should be considered:

  • Color scheme or palette
  • Location (upper left corner, center left, or lower left corner)
  • Size (4 or 6 square inches)
  • Type (static or dynamic)

The team designs an experiment including these factors (full factorial) in real time with real users under a special monitoring and measuring system.  Then, it measures the conversion rate on the ad in real time over a two-week period.

After the two-week period, the data are analyzed to determine if one factor had more influence and to see if there was an interaction between color scheme and location, as suspected by the team. The analysis of the experiment reveals, without a statistical doubt, that the “high key” scheme combined with a dynamic ad caught customers’ attention more frequently than all other variables.  Location did not rank as a significant factor; therefore, the ad can be placed in the most convenient location on the page according to priorities, earned value, or any other web indexes in usage.

Schematic of the process under study and key factors for the experimentation (DOE).
Ad = advertisement as a pop-up image display in the page that the customer is currently viewing.

Design_Of_Experiments_Figure1

Note: The design of the experiment (table) and the analysis of the data are beyond the scope of this tool. For more information, consult advanced statistical books or related material for design of experiments.

 

« Back to Glossary Index