How to decide the proper Beta Risk for our Process

How to decide the proper Beta Risk for our Process

Let’s start with a refresher…

Beta Risk in Measurement Systems is also known as Consumer’s Risk.

It is so called because Beta Risk is the proportion of Defectives Parts assessed by the measurement system as Ok Parts and therefore sent to the internal or external costumer. It is the proportion of defective parts not detected by our measurement system. In every Measurement System, there is always a Beta Risk greater than zero. But what is the proper one? It is always the typical 10% (sometimes 20%) mentioned in the technical literature?

The question is… Is Beta the actual Consumer’s Risk? Is Beta the proportion of NOK parts actually sent to the next step of the process or to the costumer?

Indeed it is not! Beta is not the proportion of NOK parts sent to the costumer out of the total. It is the proportion of NOK parts not detected by the Measurement System from the total NOK parts produced. So the consumer’s Risk depends on Beta, but also depends on the proportion of NOK parts produced by the process.

How to decide a proper Beta Risk

In the real world, Beta is always a Risk which implies a cost. A cost sometimes driven by

sample size, sometimes by technology reasons, …

The correct approach for Beta is taking in account not only Beta but also  along with the proportion of NOK parts produced by the process, as follows:

Proportion of NOK parts sent to the costumer = Beta x p

Where:

- Beta: Number of Nok parts assessed as OK / Total number of Nok parts

- p: proportion of Nok parts produced by the process

NOTE: Beta and p are estimated parameters, therefore it is very important  that a significant and representative sample size has to be taken to estimate those parameters.

From what has been said before we have to decide what is the Risk we are assuming of sending Nok parts to the costumer and then decide Beta and p that always mean cost.

Example

Imagine we are talking  about a characteristic that is classified as Significant Characteristic by our costumer, so a Ppk of 1,33 (Long Term capability) is the maximum Risk (minimum value of Ppk) our costumer is allowing us to have for that characteristic.

That means the proportion of Nok parts would be 34 parts per million. Expressed as a proportion, it would be: 0,000034

Imagine our process is performing with a proportion of 1 Defect out of 1000 parts, which is p=0,001.

If we work out Beta using the formula from above: Beta = 0,000034/0,001=0,034 -> 3,4%

So in this example we would need our measurement system to have a Beta Risk of 3,4%, which is better (lower) than the “typical” value of 10%.

That would be the case if the process is running and it is not possible to improve the proportion of defective parts the process is producing, which was p. At least if it would be not possible to improve immediately, so we would have to protect our costumer with a Measurement System performing at least with a Beta of 3,4%. Once the costumer is protected, we can analyze the problem, find out the root causes, and improve it with no pressure.

Once the process is improved, imagine an improvement of 90% of defects, then at the end we would have an improved process with p=0,0001 (1 defect out of 10000 parts). Then the Beta allowed would be: Beta=0,000034 / 0,0001 = 34%.

Once the process is improved, we would not need so much effort on the measurement system anymore.

FINAL CONCLUSION:

Beta Risk depends on the actual Risk our Costumer is allowing us to assume and how our process is performing (process capability).

What about Beta in Hypothesis testing?

What about Beta in Hypothesis testing?

The main purpose of this blog is to clarify Beta Risk concept and how it works and interpret its value. This post is intended for Black Belts so as to have clear and complete understanding of how to deal with hypothesis testing results in different situations.

Alpha Risk and its relationship with p-Value is better understood than Beta. Nevertheless, let’s see a refresher as Beta and Alpha are both working when using hypothesis testing methods.

Let’s put an example. In this first example we are going to reject the Null Hypotesis because p-value is less than Alpha risk.

Example:

We are considering changing suppliers for a part that we currently purchase from a supplier that charges us a premium for the hardening process. 

The proposed new supplier has provided us with a sample of their product.  They have stated that they can maintain a given characteristic of 5 on their product.

We want to test the samples and determine if their claim is accurate.

Statistical Problem:

H0: μN.S. = 5

Ha: μN.S. ≠ 5

Set Risk Levels and choose test:

  1-Sample t Test (population Standard Deviation unknown, comparing a sample mean to a target).

  α = 0.05       β = 0.10

We have a set of samples as shown below...

We are going to use 1 sample t-test to test our Null hypothesis: mean = 5 because we can use normal distribution and we do not know the historical standard deviation from this new supplier.

One-Sample T: Values

Test of μ = 5 vs ≠ 5

Variable  N    Mean   StDev  SE Mean       95% CI                    T    P-Value

Values    9  4,7889  0,2472   0,0824  (4,5989; 4,9789)  -2,56  0,034

Conclusion:

As the P-Value is less than our criteria Alpha, which was 0,05 (5%), we reject the Null. This means that by rejecting that the supplier has a mean of 5, we are accepting that the mean could be 5 with a probability of 0,034. This probability is considered low, as our criteria of what is low and high was 5% (Alpha). So we reject the Null hypothesis and say that there is enough statistical evidence to say that with a 95% of Confidence Level the new supplier is not performing with an average of 5.

As we reject the Null we do not care about Beta, which is the risk of being wrong when we fail to reject the Null Hypotesis.

The correct statement is: “we do not have enough statistical evidence to reject the Null hypothesis which was mean = 4,9, so have to accept it”.

In this case, there are two posibilities for what is truly happening. One is that they are different and due to the sample size we are not able to detect some amount of difference. We are saying that we simply have too much risk (Beta risk) for a certain difference.

The other possibility is that they are not different.

Let’s see how Beta works so to have a more clear view about what could be happening. For this pupose we have to use Beta as we are failing to reject. One could think that Beta works in a similar way as Alpha does. Then we could work out using Minitab Power and Beta, which is 1-power using the data we have. So the result using power and sample size would be: (see next result).

Power and Sample Size

1-Sample t Test

Testing mean = null (versus ≠ null)

Calculating power for mean = null + difference

α = 0,05  Assumed standard deviation = 0,2472

            Sample

Difference    Size     Power

    0,1111      24  0,559395

Beta = 1-0,559 = 0,441

By saying that Beta is 0,441, we are saying that if the populatin mean was the one from the sample which has a difference from the target of 0,1111 we would fail to reject that difference in the 44,1% of the cases.

This has nothing to do with the real problem as we simply do not know which is the population mean, we cannot say that the population mean is the one from the sample.

The way Beta has to be used in this case is as we have been provided with a certain sample we just decide which is our risk, the risk we assume to be reasonable, let’s put the tipical one, which is Beta=10%. In this case, using Minitab:

Power and Sample Size

1-Sample t Test

Testing mean = null (versus ≠ null)

Calculating power for mean = null + difference

α = 0,05  Assumed standard deviation = 0,2472

Sample

  Size  Power  Difference

    24    0,9    0,170852

Interpretation

The statement would be that with a sample sample of 24 and a standard deviation of 0,2472 the difference we can detect with a 10% of risk is 0,171.

Conclusion

If a difference of 0,171 and/or a risk of 10% is not enough for us, we had to put the new one in Minitab and take enough samples.

Imagine using the same example, that a difference of 0,1 is not allowed, so we want to ensure at least that difference with a 10% of risk (Beta risk). Therefore:

Power and Sample Size

1-Sample t Test

Testing mean = null (versus ≠ null)

Calculating power for mean = null + difference

α = 0,05  Assumed standard deviation = 0,2472

            Sample  Target

Difference    Size   Power  Actual Power

       0,1          67     0,9      0,903663

We had to have taken at least 67 samples to detect a difference of 0,1 with a 10% of risk.

Imagine that even a 10% of risk is too high for us due to the cost it implies, so only a 1% of risk is allowed.

With Beta=0,01 , Power=0,99

Power and Sample Size

1-Sample t Test

Testing mean = null (versus ≠ null)

Calculating power for mean = null + difference

α = 0,05  Assumed standard deviation = 0,2472

            Sample  Target

Difference    Size   Power  Actual Power

       0,1        115    0,99      0,990392

We would need a sample size of 115 to ensure a difference of 0,1 with a risk of 1%.

SUMMARY:

§From the sample we know the estimation for standard deviation and mean, so we can know the propability of having at least a certain value for the population parameter, for example the mean (this probalitility is p-value).

§Using the statement from above, the only thing we know about the population (thanks to a sample) is the probability distribution of the sample.

§Beta is the risk of not detecting a certain difference when the population changes. As we do not know what is the value of the population parameter, we cannot use the difference between the sample value and the target as certain to work out a “real” Beta. The sample value can only be used to work out a distribution which means we have infinite values of differences and infinite probabilities of that values, so infinite values for Beta as well.

§The way to use Beta is to decide which is the risk we could assume of not detecting a certain difference if there was indeed that difference. Then it gives us a sample size. Or the other way around, when we are provided with a certain sample, which is the difference and the risk we are assuming.

§Beta cannot be used to accept or reject the Null Hypothesis. It has to be used to decide what sample size to take or if we have enough sample size.