ASTM E 141 Standard Practice For Acceptance of Evidence Based On The Results of Probability Sampling
ASTM E 141 Standard Practice For Acceptance of Evidence Based On The Results of Probability Sampling
--`,````,``,,,,,,````,``,`````,-`-`,,`,,`,`,,`---
3.1.1 Equal Complete Coverage Result, n—the numerical
courts, commissions, etc. will be able to verify whether such characteristic (u) of interest calculated from observations made
procedures have been applied. The methods may not give least by drawing randomly from the frame, all of the sampling units
uncertainty at least cost, they should however furnish a covered by the frame.
reasonable estimate with calculable uncertainty. 3.1.1.1 Discussion—Locating the units and evaluating them
1.3 This practice is primarily intended for one-of-a-kind are supposed to be done in exactly the same way and at the
studies. Repetitive surveys allow estimates of sampling uncer- same time as was done for the sample. The quantity itself is
tainties to be pooled; the emphasis of this practice is on denoted u. The equal complete coverage result is never actually
estimation of sampling uncertainty from the sample itself. The calculated. Its purpose is to serve as the objectively defined
parameter of interest for this practice is effectively a constant. concrete goal of the investigation. The quantity u may be the
Thus, the principal inference is a simple point estimate to be population mean, ( Ȳ), total (Y), median (M), the proportion
used as if it were the unknown constant, rather than, for (P), or any other such quantity.
example, a forecast or prediction interval or distribution 3.1.2 frame, n—a list, compiled for sampling purposes,
devised to match a random quantity of interest. which designates all of the sampling units (items or groups) of
1.4 This standard does not purport to address all of the a population or universe to be considered in a specific study.
safety concerns, if any, associated with its use. It is the 3.1.2.1 Discussion—The list may cover a specific shipment
responsibility of the user of this standard to establish appro- or lot, all households in a county, a state or country; for
priate safety and health practices and determine the applica- example, any population of interest. Every sampling unit in the
bility of regulatory limitations prior to use. frame (1) has a unique serial number, which may be preas-
2. Referenced Documents signed or determined by some definite rule, (2) has an
address—a complete and clear instruction (or rules for its
2.1 ASTM Standards:
formulation) as to where and when to make the observation or
E 105 Practice for Probability Sampling of Materials2
evaluation, (3) is based on physically concrete clerical mate-
E 122 Practice for Choice of Sample Size to Estimate a
rials such as directories, dials of clocks or of meters, ledgers,
Measure of Quality for a Lot or Process2
maps, aerial photographs, etc., referred to in the addresses.
3.1.3 sample, n—a group of items, observations, test results,
1
This practice is under the jurisdiction of ASTM Committee E11 on Quality and or portions of material, taken from a larger collection of such
Statistical Methods and is the direct responsibility of Subcommittee E11.10 on items; it provides information for decisions concerning the
Sampling and Data Analysis. larger collection.
Current edition approved August 15, 1991. Published November 1991. Origi-
nally published as E 141 – 59 T. Last previous edition E 141 – 69 (1975).
2
Annual Book of ASTM Standards, Vol 14.02.
Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959, United States.
i51
~yi 2 ȳ!2/n~n 2 1!. (2)
that u = 6.58 does not equal u = 6.74. This is because u is a
ratio of two overall averages while u is the average of 20
There are n − 1 degrees of freedom in this standard error. ratios. For an example with k = 2, average 1⁄3 and 3⁄5 and
When the observations are: compare to (1 + 3)/(3 + 5).
81.6, 78.7, 79.7, 78.3, 80.9, 79.5, 79.8, 80.3, 79.5, 80.7 5.2.5 Procedures—must be described in written form and
then ȳ = 79.90 and se(ȳ) = .32. As this example illustrates, should cover the following matters; (1) parties interested in
formula (2) is correct when k replaces n and subsample collecting data should agree on the importance of knowing u
estimates are used in place of observations. and its definition including measurement methods, (2) the
5.2.4.2 Example 2 on the Finite Population Correction frame shall be carefully and explicitly constructed; N shall be
(fpc)—Multiplying se ( ȳ) by =1 2 n/N is always correct well established, (3) random numbers (or a certifiably honest
when the goal of the survey is to estimate the finite population physical random device) shall dictate selection of the sample.
mean (u = Ȳ). Using the previous data and if N = 50, then There will be no substitution of one sampling unit for another.
se(ȳ) becomes se(ȳ) = .28 after applying the fpc. If random The method of sample selection shall permit calculation of a
measurement error exists in the observations, then u8 based on standard error of the estimate (4) the use of replicate sub-
a reference measurement method may be a more appropriate samples is recommended (see section 5.24.2.2); an audit
survey goal than u (see section 4.1.4.1). If so, then se(ȳ) would subsample should be selected and processed and any depar-
be further adjusted upward by an amount somewhat less than tures from prescribed measurement methods and location
the downward adjustment of the fpc. Both of these adjustments instructions noted (see 5.2.2). A report should list u and its
are often numerically so small that these adjustments may be standard error with the degrees of freedom in the se( u).
omitted—leaving se(ȳ) of (2) as a slight overestimate.
5.2.4.3 Example 3—If the quantity of interest is (a) a 6. Adequacy of Sample Size
proportion or (b) a total and the sample is simple random then 6.1 Deciding on Increasing Sample Size: Choice of sample
the above formulas are still applicable. A proportion is the size should be made carefully in accordance with Practice
mean of zeroes and ones, while the total is a constant times the E 122 or on a comparable basis. Since procedures for setting
mean. Thus: sample size are based on judgements of the variability to be
(a) when u is taken to be the population proportion (u = P) encountered, there is a possibility that the standard error as
then; calculated from the data could greatly exceed that anticipated.
u 5 p 5 (yi/n 5 a/n (3) It may happen that the time period of interest for the population
has passed or for some other reasons it is not possible to take
where:
more observations and thus the following discussion should be
a is the number of units in the sample with the attribute, and ignored. Otherwise, a decision may be made to increase the
se~p! 5 =p~1 2 p!/~n 2 1! (4) number of replicate subsamples or even to carry out a census
of the universe. Such decisions must be made strictly indepen-
(b) when u = the population total (u = Y) then dent of knowledge of u, for example, in adversarial settings one
party may feel the size of u is inappropriate and will seek to
u 5 Nȳ and se ~u! 5 Nse ~ȳ! (5)
have it changed. Therefore experimental protocols along with
the standard error should be reviewed prior to announcement of
If a simple random sample of size n = 200 has a = 25 items the estimate u. Once all parties are satisfied (methods are
with the attribute then the conclusion is u = .125 and sound, standard error adequate) then the estimate can be
se(u) = .023 on 199 degrees of freedom. furnished.
5.2.4.4 Example 4. If u is a parameter other than a mean or 6.2 Increasing Sample Size by Calculating Costs and
if the sample design is complex, then replicate subsamples Losses: To assume that u = u is to make a judgement that the
should be used in the sample design. Denote the k separate cost of decreasing se(u) by increasing sample size is greater
estimates as ui, i = 1, 2, ..., k and denote by u the estimate based than the risks stemming from u not equal to u. If n is to be
on the whole sample. The average of the ui will be close to, but increased it is necessary to understand the survey costs as well
in general not equal to u. The standard error of u is calculated as the costs of inaccuracies in u. Survey costs are determined
as: through ordinary cost accounting procedures. In judging the
se ~u! 5 Œ( k
i51
~ui 2 u!2/k~k 2 1! (6)
seriousness of inaccuracy in u one needs to imagine losses
entailed if u were one standard error below u and above u.
Calculate these two losses and divide the average by two. This
--`,````,``,,,,,,````,``,`````,-`-`,,`,,`,`,,`---
7.5.1.4 Alternating multiplications with divisions will help 1 235.78A 11 3.85 21 3.40
2 19.21 12 3.76 22 3.38
avoid rounding errors. Probabilities for a = 1, a = 2, etc. can 3 9.22 13 3.69 23 3.36
now be obtained recursively by at most two multiplications and 4 6.62 14 3.64 24 3.34
two divisions. For the case of A = 31, n = 200 and N = 800, 5 5.51 15 3.59 25 3.33
6 4.90 16 3.54 26 3.32
Pr(a = 0) = .0001096913978. Multiplying by (31/570) and 7 4.53 17 3.51 27 3.30
(200/1) brings us to Pr(a = 1); further multiplying by (30/571) 8 4.28 18 3.48 28 3.29
and (199/2) gets us to Pr(a = 2); and, finally, multiplying by 9 4.09 19 3.45 29 3.28
10 3.96 20 3.42 30 3.27
(29/572) and (198/3) produces Pr(a = 3) = .02087. Adding 40 3.20
these four (4) results gives .02841 which is above .025 and 50 3.16
requires us to go on to the case of A = 32. ` 3.00
A
7.5.1.5 A lower bound on A could also be found by trial and Note—When used to calculate an exact three sigma interval this value is
ta/2(n) = 235.80 for an exact a/2 = .001349898 and n = 1.
error after setting A = 3, A = 4, and so forth until the probabil-
ity of 3 or more, first exceeds a. When we set A = 3 the
(15)
probability of a = 3 becomes .0154 so that A = 3 can be ruled
out by the evidence at the a = .0154 level of significance. 8. Review of Important Points
However, when we set A = 4 the probability of getting a = 3 or
8.1 Probability sampling as practiced in accord with 5.2 is a
a = 4 is found to be .0503 and so A(L) = 3.5 becomes the lower
procedure by which one obtains a result from a selected set of
bound.
sampling units that will agree, within calculable limits of
7.5.1.6 Using the above formula for Pr(a = 0) it is now variation, with the equal complete coverage result.
possible to furnish an upper bound on the population propor- 8.2 The equal complete coverage result may or may not be
tion when zero proportion is observed, that takes account of the acceptable evidence. Whether it is acceptable depends on many
size of the finite population. We take the example of n = 20 considerations such as definitions, method of test, care exer-
with a = 0 and suppose N = 100. For A = 14 Pr(a = 0) is found cised in the testing, completeness of the frame, and on other
to be (80/100)(79/99)—(67/87) = .03413, while points not to be settled by statistical theory since these points
Pr(a = 0) = .0443 for A = 13, but Pr(a = 0) = .0574 when belong to the subject matter, and are the same whether one uses
A = 12. Thus an upper 5 % bound on A is set at A(U) = 12.5 sampling or not. Mistakes, whether in testing, counting, or
when N = 100 or the upper bound on the finite population weighing will affect the result of a complete coverage just as
proportion becomes .125. such mistakes will affect the sample result.
7.6 Extreme Security Limits: The extreme variation of an 8.3 When the audit subsample shows that there was reason-
--`,````,``,,,,,,````,``,`````,-`-`,,`,,`,`,,`---
estimate (from a probability sample) can often be placed at an able conformity with prescribed procedures and when the
interval of three standard deviations above or below the sample known instances of departures from the survey plan can be
result. When the sample is of sufficient size, only 27 out of shown to have no appreciable effect on the estimate, then the
10,000 intervals so calculated would not be expected to cover value u should be used.
the universe value. Table 1 shows values for ta/2 where
a = .0027, and gives some idea of the effect of having to 9. Keywords
estimate the standard deviation rather than using previous 9.1 Probability sampling; sampling unit; sampling frame;
knowledge of it. equal complete coverage; replicate subsamples; audit sub-
7.6.1 Example 9. For the percent condition estimate, ex- sample; sampling distribution; estimate; standard error; popu-
treme security limits would be set at: lation parameter; finite population correction; sample size
79.90 2 ~4.09 3 .32! to 79.90 1 ~4.09 3 .32! adequacy; confidence limits; skewness; hypergeometric
or from 78.59 to 81.21. distribution.
(1) Cochran, William G., Sampling Techniques, Third Edition, John Wiley Wiley and Sons, New York, NY, 1986.
and Sons, New York, NY, 1977. (5) Mahalanobis, P. C., “Recent Experiments in Statistical Sampling in the
(2) Deming, W. Edwards, Sample Design in Business Research, John Indian Statistical Institute,” Journal of the Royal Statistical Society,”
Wiley and Sons, Inc., New York, NY, 1960. 109:325–370, 1946.
(3) Dixon, W. J., and Massey, F. J, Introduction to Statistical Analysis,
Fourth Edition, McGraw-Hill, Inc., New York, NY, 1983. (6) Yates, Frank, Sampling Methods for Censuses and Surveys, Fourth
(4) Lehmann, E. L, Testing Statistical Hypotheses, Second Edition, John Edition, MacMillan Publishing Co., Inc., New York, NY, 1981.
ASTM International takes no position respecting the validity of any patent rights asserted in connection with any item mentioned
in this standard. Users of this standard are expressly advised that determination of the validity of any such patent rights, and the risk
of infringement of such rights, are entirely their own responsibility.
This standard is subject to revision at any time by the responsible technical committee and must be reviewed every five years and
if not revised, either reapproved or withdrawn. Your comments are invited either for revision of this standard or for additional standards
and should be addressed to ASTM International Headquarters. Your comments will receive careful consideration at a meeting of the
responsible technical committee, which you may attend. If you feel that your comments have not received a fair hearing you should
make your views known to the ASTM Committee on Standards, at the address shown below.
This standard is copyrighted by ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959,
United States. Individual reprints (single or multiple copies) of this standard may be obtained by contacting ASTM at the above
address or at 610-832-9585 (phone), 610-832-9555 (fax), or [email protected] (e-mail); or through the ASTM website
(www.astm.org).
--`,````,``,,,,,,````,``,`````,-`-`,,`,,`,`,,`---