Copyright is owned by the Author of the thesis.  Permission is given for 
a copy to be downloaded by an individual for the purpose of research and 
private study only.  The thesis may not be reproduced elsewhere without 
the permission of the Author. 
 



ON SOME ASPECTS OF SECOND ORDER 

RESPO NSE SURFACE METHODOLOGY. 

A thes is presented in partial fulfil men t of the 

requir ements for the degre e of M,Sc. in Mathematics 

at Massey University . 

Vernon John Thomas 

1972 



Abstract 

A unified development of the theoretical basis of 

response sur fucc methodology , particul~rly as it 

applies to second ord~r re3ponse surfaces , in 

presented. ,\ ri. 6orous justification of th0 V(.lrious 

tests of hypothesis usually used is s~ven , as ~el l 

as a convcnic~t means of mukins tccts on who]c 

factors , rather than o~ ter~s of a ~iven dcgrcr , 

a s is c ustomary at present . Finally , the super-

- impos ition of so~e e lc~e~tary clas~~ ficati on 

ii 

dcsier:s on a 1'€1:;po:,s,· s-.trface :i.es:i.s,1 is consi.:::crcd , 



iii 

Ackno wlc dreme nt 

I would like to take this opportuni ty of express ing 

my grat itude to Dr B. S . liei r f or his co~mc n ts and 

e ncouragcxc nt in the pre paration of this t h esis. 



iv 

Table of contents 

Abst ract ii 

hcknowledgement iii 

Table of contents iv 

1. Response surfa ce methodology 1 

Introduction 1 

The model 2 

Estimation 5 

Hypothesis testing 7 

Further topics 13 

2. Historical development 16 

3• Form of XTX for second order designs 23 

Non-quadratic effects 23 

Quadratic terms when orthogonality 

does not hold 28 

Special forms for Q 30 

Rotatable designs 32 

4. Overlaid experimental designs 34 

Introduction 34 

General experimental design model 35 

One-way classification 37 

Two-way classification 39 

Regression model dependent on 

classification 42 

5• Summary 

6. References 

~ 
I 



1 

1 • Response Surfarc .cthodology. 

Introduction 

Response surface methodology seeks t o estimate , 

by regression methods , that linear combination of 

previously specified gradu~t~ng functions of a number 

of inde pendent ~ariubles ~hich provides , in some scns e t 

t he best fit to an obs e rved response. 

While t h e technique s of fittinc are identical 

with , or closely related to , th ose of multiple linear 

regress io n , the emphasis is Elightly different , in 

that coc~ide rabl e stress i~ laid on the desicn as~cct 

of th ~ rroble~ . It is assu:1.cid t hat ~he 2. evels oi' the 

indepcn-:ie:it vari~0lc::.; rtay be p~e-sps~iL.ed at will , 

wi thir. k· oad 1 imi ts . The spac f! de fin cd on the in,ie pendent 

vari able~., , und v;ltLl!'"" the:~t: limits , ~~s terme1 the 

regio n of opera, ility. Th e ~ub-cpace o~ this r ecio~ , 

i n 1•1hicJ-: cstir::a:es o f resl'or::;c are of interes: to the 

experi menter , i 3 tcr~cd the r~gion of interest • 

Typically , a number of experiments are carried 

out , a ccnrdi~~ to so~e previously decided experimental 

p l an . Each experi men t c onsists of t he ~easurement o f 

an obser ved response at a point defined by so~e 

combinati on of the in dependent variables . I n sorr.e 

c ases , sequent ial designs are used - that i s , t he 

c u r ve fitted to date is used as informatio n to assis t 

in th e specifi cation of the combination of independen t 

va r iables t o be used i n the next experiment. 

The bas ic va ri ab ility of the obs e rved response 



2 

is measured by replication of experimental points, 

or by the residual error of the observed response 

from the fitted surface. This latter error can arise 

from a true observational error or from inadequate 

specification of the model, whereas the error based 

on point replication estimates true experi~ental error 

only • For this reason, when point replication is used, 

the residual error may be used to teat model adequacy. 

The model 

The model is developed by assuming p independent 

variables, given by 

and k pre-specified vector graduating functions of 

these variables, ~iven by 

The observational response is assumed (or known) 

to be 

y ;')+£ 

where e is a random variate with zero mean, and the 

so-called "true response" Y) is given by the exact 

relationship 

T 
1J = ~~ ( 1. 1) 

where~ is a vector of unknown coefficients. The 

measurement of an observed y, for some known~, is 

termed an experiment. The values of E arising from 

different experiments are assumed to be statistically 



3 

independent, wi th constant , unknown , varia nce 2 
a • 

The aim of the sequenc e of experiments i.s to 

e s timate ~ by~, and , from thi s , to es timate the response 

at any point of t he rccion of interes t by 

A 'I' 
y == X b 

To achieve thi s , n c xpcrjments ur e co ndu cte d , 

at the point s ~ , u=1 , • • • n 1 y ielding n obs e rved 
- u 

r esronses 

No·.v let 
rp 

Z - ( ~
1 

••• «; ) ... of diEien .<:~:; on nxp 
- - n 

X - x( ~ ) 
~ u - - u 

X :... ( x 
-1 

,T x, of di~ension nxk - n 

s o that l is t he observ e d value of the tru e r esponse X~-

Properly f.:peaki.nt,; 1 2 i s th e dcsi gr: rr:n t rix , s::.Lc e 

....-. ......... 
~ di.::· ter:;:::..r:e:::; x. !:owc "...~e r , on c e ~ i~ c !".:.osc:~r; , o c co r di!:t; 

to some desicn criteri on , it is co nv enient to r e f e r to 

X us th e dcsicn ~atrix , since all operations are in 

t errr:s of x. 

ln the v3s t ma jority of urpli cations , ~ consi s t s 

of all powers of the~, se para te ly or togethe r , up to 

some maximum degreed. The desi gn is then r eferred 

to as a dth order design. Thus , for a second order 

des ign 

For th~s type of ~esigu , it is conveni ent to 

use the subscripts occ urring in the correoponding 



element of ~ t o identify the elements of ~, th us , 

for second or de r designs , 

4 

In ccncral , for a dth order de sjgn , there wi ll 

be ( p:d ) coefficients. 

;'ii.thin tl: is fr ar.1cw o!'1; , !:.ft i.s .:1 i;ent:>ral dth 

order polyno~ial in p variables , 

The exceptions to this k ind of polynomial are 

of t~o ty !-~5 • In th e first type , the clements of x 

are I ot l!O\•;0r::; of tr.e elem<.:nts of ~ . For cxamr,le , 

V. J . Box (1 968) considcrdd the functions ;i vcn by 

x. :::. exp ( ~ .) 
J.. J 

as well as other no n-poly~omial functions , 

The seco~d ty pe occurs when certa~n of the tcr~s 
m 

of the polyno;;1~a::.. ·-5//J c.:.rnr: o t be estir~Dtcd , and must , 

t herefo:--e , be o:ni tt c::.. For ex~1--::ple , in ti:e bivc,,riute 
,......, 

Cilse ( p:=2) , i: ..::. ~-:pcci fied t.Le 1,oints of a 3x5 ff.lctorio.l 

design , r.cc essarily the pol;y 11or:1ial elc1N:nts of x nn,:., t 

be n. sul.Jsct of 

2 2 . " ? '"' L. ? :;:, 7 li ;:, !.i. 
1 , c;1 , ~ 2 ,E. ,i, ~.2 , ~~ ~2 ,c; 2 ,~,i( ~ ,~-.~~ ' ~ 2 ~s~~:, s/~~, ~/. _: ,e:~ t2 

?. I .., 
which o::iits the co rnbi,w.tion::, ~ 1, ~ ~ , CJ.nd c;' ~{ 2 , ·,:ho3e 

coeffici en ts cannot be es t imated because an i nsufficien t 

nu mber of levels of ~, was used , Similarly only two 

coeffici ents of degree five or higher may be es timQ ted 

fro m this design . In practice it i s unlikely tha t an 

attempt would be made to estimate the co e fficient s 

of ~1~~ or ~ ~~~- If i t were , and if the factorial 



5 

were unreplicated , an exac t fit would be obtained . 

Esti'.'cntion 

Methods, " cul~i~utin; in the estimates ~ and y , 

rnuy be divided into desig n proce d~rcs and e~ti~ati on 

procedures, Desigc rroccdur es are those used to specify 

,..., 
~ and hecce X. Disc~ssio~ 0!1 methods of selec ting the 

design is outside the .s c ofc of thi s t!-lesis , Bs tir--:atio!1 

procedur es arc those which, given X &nd y , se2k to 
~ 

estimate b , In general , ~ is assu~e d to be a line3r 

co ~b inat ion o f tte o bs8rvc d responses , of th e form 

b - T·-,· - ~ 

wh 0re '1:' depends only on X (not, fer exa::-.p l e , 0,1 ~ : . 

The co::..110J,cct csti,n2.tor a!'it::cc froc1 r.1in~1::ization 

of t.e s u m o~ squar es of the 
A ,.,-,, . 

y - y • k·nis 
u u 

is 

kno~n as the l east squ~ree csti~utcr 1 and ic , in fact , 

id~ntical to that obtained ~hen E ia assu~ed to ha\c a 

nor~al di~tribution , a nd ~axi~um l~kelitood e3ti~ation 

us ei . 

The quantity to be mini~izcd is 

T 
( y-Xb) (y-Xb) 

._ - - -
Differentiation with respect to~ a nd equation to 

zero yi elds 

f rom w:1i ch 

b = ( X
1

rX)- 1XTy 
-.. - -

or T = (XTX )- 1XT 

(1. 2 ) 



6 

. d . t t t XTX . . 7 If x'l'x . . 1 provi e~ 11a 1s non- singu~ar. . is singu ar , 

a gene r alized inverse may be used , but is unnecessary 

in the present case. 

This s traight-for&urd esti~ator haG many desirabJe 

prop~rties. Tn pc1 ·t:i.c:J.la;:· , 

E(12_) =.: o?:o-'1xTX .§ :: I?, ( 1 • 3) 

Vnr(b) = (XTX)- 1XT Var (r) X(XTX)- 1 

fro:n t~,c 2.ss 1;r.1:r_:.tions abcut e . !kne e the cstj ·:- a tor 

is unbiaccd r~a~ (1.3). !t can also he shc~n tha t 

(~. 4) gives the mini~u~ variance aris ine fro ~ an unbiased 

linear es ti~ator . 

Fin::-1] y 

for a .:::·b:. tr.:!.r -· ~ ' no:. necessarily or: c of the x • TlL1:_; 
- '..l 

rr. '1 
the vario:J. s v~rianccs c an ca~ily be derived fro~ (X~X)- · • 

ThiE l e3ds naturally to the cor.cept of a rotatnble 

cie~;.:..gn , ·>:hich is a polynomial design for which Va r(y) 

dei;erld s or.ly or. o
2 

and ~ T~. That is , Var(y) is invariant ~ ,.., 
under orthoconal rot ati on of th e ~ -axes , 

It should be emphasized that the estimator defined 

above is not the only linear estimat or possible . In 

particular , in conditions where the specified model 

(1.1) is inadequate , that is , where y contains other 

t erms than those- in a li.near combinat ion of the specified 

~, a different estima tor may a~sist in c ompensa ting , 



7 

in s ome degree , for thi s inadequacy i a t the e xpe nse 

of greate r variance • 

Hypoth osi~ t csti~g 

From thi .s point h:ypotL csis tc~tin6 will b e 

c o ,,s:: .~.-.: red 1 .:.J.:-id tL c ,L"i6 i ti0nn] as sun j ,t ion ih~ t tr:o 

€ are no r ~ally dis tribut ed ~il l be re quired. 

Ifo w l e t 

;fo te tha t bot :i :'. D- nd ~-: nro i de:npo t r.nt r:iat r iccs , nx n, 

-:- t {Y ' y ~ v ) - 1 '{ ~} tr 3 r .. \., , -"- . ,. 
:ix:: 

= t {()''.,:' ..,) - 1·/!:,f } r \ \,. .\. ~1,. .J .. 

p:,:r: 

sin~e co~pn tible ~atrices c o~mu tc ~~ dc r the trac e 

tr N = tr I = p 
pxp 

tr M = tr I - tr i\ = n-p 
nxn 

Th e r es i.dual su~ of squares (1.2) i s equal, on 

expans:ion, to 

,.. c- E T 'l',_., 
;::;0 = Y y-y l'. 0 - - - -

T 
= y My -

It is necessary to r ecall a th eore~ on the 

distribution of quadra tic forms ( see , for example , 

Gray bill (1 961) ). 

(1.5) 



8 

Theorem : I f y - N(µ , o2I), then yTAy/ o 2 i s distributed - - - -
as X.' 2 

( k ,),. ) , where X-' 2 
represents the non-central 

h . ' ' . t . ' t . d ' 1 'I'A c J-squarea uis riou ion, a n A=--2 µ µ , if, an d 
20 ~ -

onJy if, A i s an idempotn~t matrix and tr A - k . 

In the pr,.:-::;cnt 1,i t llil tion , i - r: (): ~, cl:;:) and hence 

SSE 
a?.. 

~ ~ 

'I'hus .SSE/ a"- ta.s a c en tral X'-
di~tribution with n-p aecrces of f reedom, 

The second ter~ in (1.5) is the sum of cquares 

a c ~ounte~ fer by the recression , a~d is 
m 

SSR ==- ( 1. Nl 

3y a proce s s of r eascnint si~i l ar to th3t for 

~SE , it is an easy matter to establish that 

wh ere 
1 rr m 

' -· - fe_ ~X1.''X f?_ I\ -~ -, .: • 'J 

..... r_ - -
cC'f 

1 rp m 
p., -vl. " P.. 

:.: --? f:::; I, .A j.:: 
2l) '- - ~ 

Agai n fro~ t he theory of quadr~tic for~s , a 

necessary and su ff i c ient c ondition for 

to be indepenirnt is th a t AB:O . 

and 

Hence , sinc e rN=O , SSE and SS H are independen t 

F ::: SSR/ SSE 
p n - p 

ha s a non-c entr a l F-distribution with p and n-p de grees 

f f d 1 t 1 . t t - 1- A TXTY f1. o ree om anc non-c e n r a i y parame er 2 r,:. · · ~ · 
20 

Thus F may be used to t es t the hypothesis that ~ =-=Q.• 

In r e sponse surface design it is us ua l to further 

subdivide S.S'E by . taking·advant age·o r point r eplication. 

As a pre limi nary, suppose that t he model 



9 

specific a tion (1.1) i s incorrect a nd tha t while the 

model 

T n - X J:l . I - ~ 1 t::1 

has be en assu~ed 1 th e true model is 

Using x
1 

and X~ in ~n obvious ~ay, 

r 'I' )-1T b = ,x1x1 x1l 

In these c ircu~st2n~es 

and bis a tiascd e,.; Lima tor of A _...,.·he t ' '".J+-.....,~ v r:::::1 • -- ~I,. .... L, .......... ~ 

\'iil E.;c .. ( 1951)) rt?d ::.-c~Llt:r<:s th e o:t c:, t of tJ; e bic:s, 

and 

Pu t :inz; 

--~ 
02 

! . I v ('•'T. , ,- 1,,T 
·-, ::. - ;,1 ' ·1 ;·1' ·'•1 

':1 
ss~ : . .- y· ;.~y ,-

How e ver, int~ , p~csent c ase 

and thu s 

= 1 A T). T , )' 11.. ..1 O 
- ~ t::'- 2 ' 2···1 ---.~ 2 r 
20 '- - C. 

in general , and the F-tcs t described above is no longer 

available. 

Suppose, however , that point replic a tion has 



10 

been use d, and that r di s tinct points have been included 

in the design, the sth of them n times , s=1 , ... r , and 
s 

r E
6

~ 1 n
5 

= n , Also, let y
5 

be the group mean of the 

y-valucs measur ed at the sth dist inct point, 

Without l oss of 5enerality , th e poin ts may be 

arranged in such a way tha t then points in the sth 
s 

group arc together in the land X matrices, 

Now define 

K = I -

0 

0 

1 
- J n n r r 

- I - J 

where J is then xn ~atrix with al~ unit elements, 
n s s 

s 
so tlw. t , without poir. t repljcation, n =1, a~d Ka 0, 

s 

No, K2 =K , hence K is idempotenti and tr K = n-r, 

,-, 
Al s o, since .=. , 2-r. d hence x

1 
and x

2
, consj_.st of r croup~, 

If the y-vulue~ are standardized by 

-where y
5 

i s the gr oup mean 

= Kf 

containing y , then 
u 

Now SSW , the sum of squares within groups of 

observations at the same point , is given by 

s s w = ~ T f = l TKl = ~ T K € 

and since ~ ~ N( Q, o
2 r) , and tr K = n-r, 

SS\'/ 2 
2 - X. ( n-r) 

a 



11 

by the theor em quoted for SSE. Also , SSW and SSR arc 

ind epe ndent, since 

Now cons ide r SSF ( for sum of squares due to 

l a ck of fit) , de fin ed by 

.SSF -- ssr~ - ss 1;: 

= y'r ( M -I-'. )y ~ 1 ::.. 

(M1-K) 2 
') 

K2 Now = ML - 1~
1 

K - KM + .., 1 I 

= M1-K 

tr (M 1-K) - tr ~-l 1 - tr }; 

= r - p 

Henc8 1 from the t nA cr e m, 

w:1ere . i A -K ) ( X ~ A A + z 2122) 
I !I- I 

Thi s require~ , reasonably enouch , r>p. 

Finally, SS~= T,, h ~· X ()'TX )- 1X J i,1y we re H1::;_ ,., '1,., ,A , 
,..,,. . I I I 

Now note that , where Lis an ar b1.trary matr ix, 

T T 
= E(tr t LI)= E(tr L]l ) 

= tr [LE(X1~ 1+X2~2+ f )(X 1~ 1+X2~2+€ )TJ 

= tr [L(X1~1 +Xz~2 ) (X 1~1+X2~2? +c/L] 

T T 2 
= tr P- X LX{i. + o tr L 



12 

whe r e 

(

~ 1 ) 
~ = 

~ 2 

Fr o m thi s , the expected va lues of the vario~s 

sums of squares are readily deri ved • Th e a nalysis l s 

given in t able 1. 

TabJ e 1 

Basic rcspon.se t1urfaCe A! :0 1
/ 

Sourc e 

Regressio:1 

Lac k of f::t 

Error within 
replic ~ ted ~oints 

Total 

of squu.rcG D? 

rn 
:: y' X t 

- 1 -
p 

SSF by subtraction r-p 

n- r S.S~'i T,. 
= l.. J\ l_ 

c· " rn T 
u.J ~ = 1. y n 

? 1 ',' T 
- _11,. -v~ ·· ·, p., 

0 + t:: ,, . , ,. ' /-::! 
}:; - I ._ 

2 1 ~-=1 
V 1'., . . a_ 

C •-- _ .... , : 1•'-· ,-.,_ ,._ r -p - L L C. - i:..:.,. 

Note that withou t replication n~r , and if ~ 2=J , 

t ab l e re duces to the simpler f or~ d e rive i earlier • 

While the above argument establishes the theoreti c al 

j ustification for the use of the F- tests , the test of t he 

wh ole regression is , in practice , of little use . 

Howe ver , it i s perfectly general , and n ot dependent on 

a polynomial specific a t i on of!· In the event that a 

polynomial i s used , the SSR is ordinarily broken down 

into the c lassification shown in table 2 , 



Table 2 

Conventional ANOV for regression 

c oefficient~ in polynomial model 

l-'.e a n , {!>
0 

Linear terr.is 

Second order ter~s 

Third order t~rms 

dth cr ct ·,r t erms 

DF 

1 

p 

1 
2p(p+1) 

1 7~( p+1) (p+2) 
t:;, 

p 
r, d- 1 

- '1 i l.= 

13 

~hil3 thi s is suitab l~ fo r establ i shin; t he true 

de~r0e of t h e poly no~ial , it 

th e i!r.porta:1 ::: e I in the finc1 l res1)0:1 ,:.c I of a p r,rticuL ... r 

~ - Sec tion 3 of thi s thesis cons iders the strac tur c of 
rp 

X£X, for the second order polyno~ial ~odcl , in some 

det ai l , in ord e r to f o.c ili ta te te sts a i med at es ta blj_sh:;.nc 

the importance of particulor elements of ~ 

Furt her topic s 

In field experiment s , each experiment usually 

c ons ists of a plot of ground , In most circumstances , 

th e number of such plots which can be assume d t o 

represent e ssen tlally the same external conditions 

is quite limite d, 



14 

In order to control this t ype of environmental 

variation, a block s tructure may be superimpoced on 

the response s urface design , yielding a mode l of the 

for:n 
m 

( 1. 6 ) 

where ~ is t he b]ock e ff ect ass ocia ted with the wth 
VI 

block , with r~ =0 · 
VI 

w 
De: ::; icns inc lu d in!__; such block s tr:.ic b:re:, were 

introduc ed by DcPa un (1956) and elnbor~ted by Box and 

Eu:itcr ( 1-757) in U:e co.se of rotat;,tle desir;i,0 • ~'he s e 

variatio:-1 . 

A n,~tura.l ext c:-::.s:i en of this type cf de sic:-, 

t o conside r the possibility of supcri~pos icg a ~urth er 

trcot~ent e ff ec t, ~t~ch , ~n pr3ctice , could rcprc5ent 

T n =, 0c +T +:· P, 
- , w v~t~ 

where n~~ T ~s the vth treaL~e nt ef~cct , As far as 
V 

trc &tment ~ are c o~cerncd , su ch a model is identical 

to the analysis of covariunce model , ~h ich uses the 

r egression variables! to reduce variation in the 

response , major interest being focus sed on the 

superimposed treati:.cnt e f fe c ts . A response s urface 

approac h would h ave equal interes t in both parts of 

the fitt ed model . 

Pursuing thi s line of enquiry further , section 4 

of this thesis conside rs the implications of combinin~ 

var iou s classi ficat ion designs with a respor.s e surfnce 

design. 



15 

One obvious extension of the rr.ociel de5cribed 

by (1 . 6) i s to allow~ to vary with th e block , giving 

a model of the for~ 

In ~any appl ic nt ic~s the question of the doGree 

t1f corrc.sror.dence bct·.v ::0 en t.he :iud ividual r•2cre--si ons 

A &nd the overall r egressi on ~ is of conside rable i~portaccc. ~ -.. ; 

Section 4 a~~ o considers , briefly, thi s aspect of 

r e sponse i, 1,rf<lce met:2c dology. 



16 

2 Histo!"ical n~ve looment 

After a small number of r e l ated papers in t he 

ninetean-fortien , Box and tilson (1951) laid the 

founcb tion fo1· lat, r 'nor :~ 0n response .surface nr.alj· s is . 

They ~ere pri~~rjly co~cerccd with a sequential 

series of cxperi~nnt~ to determine the ~nxirr.uc 

or ~ini~~m point 0f ~ ~uadratic r espor.~c surfhCC • 

Thei!" :tpy,roach v:o..t; to fit ~ L.near r.iodel over the 

rrgi on o~ lntcr~Rt nn~ mak~ 2dditional experiments 

order o~fccts , over 3o~e ~~~1~ sub-re:ion , wore 

ncgliciL~~ , nnd the~ fit a second order ~odel . They 

also introducLd th e concept of the aljas ~~trix 

to rr.ec:.sure ~he :1ias ari.sinc; f1·c::-: ti,c use of an 

inadcqu3te ;.;Ccial , ~~c dcsi5~~ ttey c onsidered nre 

kr.0·11:1 a:-; ccr.tr.:Jl co:r,s-osi t e ct•::;..;icnc.; 1 cor.'.·istir,g 

as ttey do of a c~peri~position of two or ~ore 

centrally cy~~etric des i~n~, usually a cuboidal 

(or fact oriol) cte~iEn a~d a simplex dc~ign (a type 

of design w~ich varies each variable in turn , .setting 

a ll others t o the c entral level) . A~ example of 

a three- ~ay design (that i s , one involving (
1

, ~ ~ , 
C. 

and ~ ~> or this type is 
;J 

<±2 , 0 , 0) {0 , ±2 , 0) and ( 0 , 0 ,±2) 

together with replicated central points (O , O, O) . 

These i deas were furL her develored by Box (1 952) 



who used rotationo to mini~ jze quadr a tic bias in 

linear mode l s . 

Elfving ( 1952 ) c o~sidcred th e two-variable 

s um of the v~rian~cc o~ th~ coeffic::..~nt.3 , 

LlfvinG ' s ~ap2r , and ltat of Dox 3nd ~ilcon 

to ~ore than t~o 6i~2nsiots , and ua8j fi sher ' s 

e s t ab l i_ s l: i. n c c on :~ i c.: c :-i. c e r- e: :J ; c r. s f c r t L :2 s o 1. ,_;_ t i o !': 

of a .set of si;;:ulJ.:.;~~ ·~o u.:; cq_iJ_ut~o11.s , a:1~ ~I)rlicu 

i' . v DJ.S to t:1c pro':.:: c.: ~- 8 .:· t h c 

stutio~ ~ry pci~t on 

surface . 

Eox ( 1951,F. ) h.:1.3 a corr: '~.c nt on a "con.:~ :iei:cc 

c one " of an esti::.atc d vector ·shier: , i1 t:-,is case , 

is the vector of steepest ascent o f a response 

surface , as used by Box and t ilson (1 95 1) . 

Hunter (195~ , 1956) discussed , in general 

ter~s , th e problem of findins a s t ationary point 

o n a response su~face , and poin t ed out that a 

ecneral secon d order response surface cou! d be 

trans for med to a c anonical form 

17 



18 

Box ( 1954a) a nd Davi ec (1954) g~v e g e ner a l 

surv eys o f the then curre nt s tate of re s ponse surface 

methodology . 

po1 yncr:.iiJ. l re 0 rcb:.d.:;n , w:i.t l; one ind8per,:i.e:1t v~r ii.ibl e, 

s to~cd th a t, for ~ny ~r bil~nry spucin~ of experimental 
rp 

point ~ 1 it is al ha y s poss ~blc to obtnin t h e sbmc X~ X 

rna tr ix , ucinc no t ~ere th an d+ 1 di s t i~ct cxperi~cn t a l 

J e ,·els of ~ . ! 1 e L•1cr: con.sic.icrcd ho·.·1 t.r:cse points :r.ay 

b e sel0cted in such a wo y as t o mi nimi ze lhe varia nce 

of t ha t coe:~fic i-2:1t ,,!1 ic h t ac th e n,a xir::ur.1 vu r i ,1ncc , 

Th i s criterion is knoTn a3 the • ini~ax v2riancc 

cr ~terio:1 . Gue~t (1 95~) obt • i~ei general for~~ ae 

wi th a uni. fo r ::i src.1.ci:is -

of respo~3c s ur f ace~ in t he ~io l d of chc~islry . 

De~au n ( ~95G) w& s th e first t o apply me th ods 

of bloc~i~G t o c entral c o~~osite des i g~s 1 wi t h a 

r ather c urs or y su r ve y of t he poss i bilities . Hi s 

ideas were e xtende d by Box a n d gunt e r (1 957) , who 

consid er e d rot a t Db l e desig ns in ge n eral, and mad e 

an extens iv e s t~dy of central composi te de s igns 

in pa rticula r. Box and Hunter ' s paper give s what 

is proba bly the bes t s um~a r y of the class ical 

approa ch to respp ns e s urfac e experi mental desien­

Thc fir s t di s cus sions on re s ponse surface 



19 

methods to appear in textbook~ were Davies , mentioned 

above , De Baun and Schneider ( 1958) 1 ·,:ho described 

particular applicat ions , and Plackett (1 960) , who 

st;;r.marized the e.J.rly opt:i r.um ori ented work, in his 

\·'.any of t he pa; f'n; thu t appeared in thu late 

1 9 '.; 0 ' s G n d car l y t o n. i d d l e 1 9 GO ' s 1:: c r -=: l y 1 i s t 

particular dc~i6ns or cl~sscs of dc ~is~s . Hartley 

(19~ 9) ccnBi~eretl th~ cmal1cst co~positc dusi~n s 

for ~·J.::iL.g :p.,.;.:i~ratic !'espc,,n~e surfaces , b.::.tsed 

o tr~nafor~atio~ g r o~r t o cenerate pci~t sets 

'. . B C.i J_ i':'l (~ 11 f3 J O r: S • · C X 

s i rrplcx de~ic~~ to derive s0cond ord~r ~0slsns 

f r c ~ first or der dcsic~s - The re~u:tiL~ dc~iGn A 

Das a~d Kar asi~han (1 962) de veloped quadratic 

de ci ic ns from balanced incc~p:ete b!ock desiens , 

Draper (1 960a) and HerzbcrG (1 967a) gave rather 

similar methods for ecnerating secocd order dc s iGns 

bas e d on permuting point sets and building up designs 

in p dimensi ons from designs in p-1 dimensions , 

Then , together , ( Draper and He rzber g (1968)) they 

developed methods based on c,mposite desiEns with 



more than one fractional factorial . Das (1961) 

considered second order and third order designs 

derived from factoria l s . 

c..v 

Third order designs arP not, properly speaking , 

within the scope of this thesis , however , third 

order designs h-~vc beer. d0•1elored b:,• Gardiner , 

Grandasc , and Ha~er (1959) 9 Draper (1 960b 1 1960c 1 

1 9G1b 1 1962), and Eerzberc ( 1964). 

DsBaun (1959) and fox and Beh~ken (196Gb) 

con~idcred des1t;n~; i1: v:hich , for rei:wou; derendine; 

on tl:e context of t.hc ~~:rer::.::icnt , ench factor ir; 

li~ited to only three leveJ~. 9rapcr and Stor-c~~n 

( 1968) cxt-:r-dc:i tr.is ;,:ork to the cr .. S!' ·,.here f,o:r,·~ 

factorG are restricted to t~o lcvclc ~ni otlcr~ 

oc velcfc l cyJ.ir:C:::·.ical dccii[;i, ~; , ir: \\r.:icL one factor 

wac sel a t a pred8Ler~incd n~~b~r of levels , hu~ 

A :r.ore co!';.1Jic~,ted three-factor dcsicr: , 1J:,in5 

ttc ~ro~crties of dodecatcdror:s , was developed 

'cy Herr.::.i.nson ct a) . ( 1964). 

Bose and Carter (1959) used corrplex numter 

properties to examine so~e of the ctaracteristic6 

of two-factor designs . 

Missing values were considered by Jraper 

(1 961a) and the effects of point replica tion by 

Box (1 959) and Dykstra (1959 , 1960) . 

Kitagawa (1959) extended the early work on 



21 

sequent ial experiments , Umland nnd Smith (1959) 

gave an interesting exumple of the use of LaGranGe 

multipliers in fitting second order responcc surfaces 

under a second order co~Gtraint , and Box and :~dwell 

(1962) 6av0 a use ful ~u~~ury of the tffcct of 

tra n, ror::-,2-.tior:,; o: the im:cpendcr:t v~:r~Dblcs , 

!'.un'...~1- (1 966 ) . Eo\':c\'~r , t he.1r :-!:~r;l.~ ~.1~'. was on 

v. r.:re o::ii t, led , 

u r:u.~.l-c:l~ o: r ... :t1:€..:".s -.-,~. ~ c: .. c ar :it1c..1·c~ t:.c e:ff'ect cf 

and :::., vid .:.r! J. !.rer.s (-: '):;9) , In R P:::) thesis ~'('} r:~ 

Their work w~s extended a nd related ~ore 

directly t o respo~sc surface methods in an important 

paper by Box and Drap e r (1959) , who cor.sider ed the 

pro~lc~ of estimating a response by 

T 
y = X b 

- 1 -

where !
1 

includes nl l terms up to de~ree d1 , whe n 

in fact the true ·:nodel is 



22 

'1' T T 
= ;{ - A :.: X , ~ , + X ~~ -, 

....... r:;.. - - , - t::.. - ,..:. 

where~ inclu de s all terms up to degree ct
2

, with 

!ilizatio:; cl bins nni 01 var i:-:nce , i11tc·gr .:1.Le~ ov er 

the re6ion cf iLt erc~t . Their • ~i n conclu~icn was 

tlu t bi'.l. s considc.::--atic,::s were l::.kely to l:0ve 3. 

o: re~ro~~e su rf~ce ~ c thodolo~; furttar Jn ttis 

t tes i s , 



23 

3. For~ of XTX for second order de~igns 

~on - quadratic e f fect0 

In th e analysis of re~ro~sc surface rccults , 

+' ~ne on y di fficult cnloul~lior: ~tep iG t~e inversion 

will be pnid to tte f o~ffi of this matrix , 

The cencrnl ue~ o nJ order ~ode: is civen by 
p p -:, p -1 p 

l"') ::c /3 ~, + z.": f3 ~~ -- ·I I: /3 , ;~'.- + l L /3, ; ~<_. ~.; 
\J . ~ J. l . 1 ~~ .L . 1 . . 1 ,. , -· ,.; 

l:::, _L -- l::: J=l+ ~ 

(3.1) 

whi ch suc~ s ~ts n ~aLural pa rtition of S into elc~entc 

related to the c onstnr.t term , thv:;<! relat ed to ti1e 

li r: car tc.::·:r.~- , tr, ()~~ 1· c 'l.EJted to the ciun.ci.rcJ.ti c :er;:;s 
-:, 
'-

( th n tis , lcr~~ involv~!g ~ -) ~nd these rel3ted to 
l 

~nteract ion tcr~s (tt at is , tho s~ 

'Ece i:-:-:-,ro'iuct:ior: of zero ::; , or bloc:-~ f z'-ro:::, , 

into S •;1iJ l r:ot " o:--!_i.y s~~pli fy t~~ inv~r0io~ cf S , 

but will cnat lc t ests of hy~othe~es on orthogonal 

eroups of co0ffici-~:,i.::; to tc 1H.--de without ref:tti"i; 

the par~-.rr.0iers . 

So~c progress in th is direct ion ~ay be ~ade 

by measuring the vari ab l eG ~ . , •.. ~ f rom t heir 
I p 

respectiv e means over the design. Using the subscript 

u to r a nee o ver the actual sample points , and making 

thi s t rans formation , 

r ~. = o 
u'=-i u i = 1, .•• p 

-. 
Thi s has th e i mrr.~d i a t e effect of making the 

linear effects orthogoLal to the constant te r m. 



Within thi s fram ework , attention will be 

c onfined t o de s igns for which all odd mom en ts , 

tha t i s , mo me nt s which include at least on e odd 

24 

power , about the mean o f the independent variables 

~, up to t he fourth :nor.:e nts , arc z.ero , Such des igns 

include al l symmetric tlesicns , ~hat is , designs 

for which the inc:usion of a point containing a 

co-ord inate ( , implie s the inclu:c, i on c f u.u othen:ise 
J.. 

simi l a r point with co- or dinat e - ~ . , which ~ay be 
l 

.._h · .- · r & ,.., "'.c same poin"' i "7 ·= ··-· • 
l 

Thi s restriction st ill inc ludes ull rotatable 

desi~ns , an d , in particular , alJ. central co~po·ite 

designs , This l atte r type of deLli[n ha~ r eceived 

by f a r th e Greates t attenti on i n the l iteratu r e , 

As noted in se c tion 2 of thi s thes i s , they are 

for med by th e supc ri ~posi tion of cuboida l designs , 

simpl ex or " star" designs , and c entre points , 

For sy~~etric designs , f or every ter m ~ . ~ . 1 <.-, J.U '-.J U 

eithe r ~ - or t; . = O , or the;re exists a nother 
J.U JU 

term ~ - ~ - = - ~- c; . . 
J.V JV 1.U JU 

I' J:: 3 =L &2 t:: =0 , 
u ., i u u ., iu ; ju 

He!lce r. ~. ~ . =0 , 
U J.U JU 

Si:niJ.arly 

Thi s re qu irement that odd rn offient s be zero is 

ne c essary for rot a t abi lity and convenie nt for 

or th ogonality , and in most c ases does not restric t 

the c hoice of design t o any significant degr ee . 



25 

The effect of the r estriction is t o ffiake 

t he linear a nd interaction termE orthogonal to each 

o ther and to all other terms . Hence the p a rt s 

of S corresponding to t h e s e terms are diagona l, Thu s 

it is possible to give th e e s ti~a tes for A. ~nd r' l 

/3 .. ( i/j) imr.-1e di a tely as 
lJ 

b. 
l 

b . ._ ~ ~ ,,, ,: 2 ~ 2 
ij - ~~ iu~ juYu1 t ~i ~ j u 

However , the qu adr a tic t e r~s a r c s t il l 

i/j 

non-orthog ona l to eact oth e r a nd to the constant 

term, Suppose now tha t the qua dr a tic func tions 

:::> 
~ :- a r e replaced by a tl'..;cJ. dr a t i c po }yr,o rr: i 11 l 

l 

The values of 0. a nd~ ~ ~a y be s el e ct ed lo i~prov c 
l 

orthogona lity. The or tiogonality c o ndi ti o n~ ur e 

giv e n in t ab l e 3 . 

'Po.':::le 3 

Orthogona li ty c onditions on 0i , µi 

To achi e v e o:--thogo .:-: ,l l i t 
Requirement 

of <;iu with 

1. Constant term re. =O 
U J..U 

2 . Linear terms t {. ~- =0 
U 1U JU 

3. In t eraction ter r:is 'f,( ~ t. :.: 0 
u iu ju ku 

j/k 

4. 2 
Other quadratic terms r.c. ~. =o 

U J..U JU 



26 

Using the rest r iction on odd moments , nnd 

expanding S, , condition 3 is autow.a tically satisfied . 
lU 

whence 

Condition 1 gives 

2 r~. -+np. . = 0 
U J.U l 

- _1, i-2 
µ i - nu'>iu 

Condition 2 is automatically satisfied for i/j . 

'.'ihcn i=j , 

or 

Hence 

and 

9 '<' F 2 = 0 
i i:i " iu 

l; iu 

e. = o 
l 

::> 1 ? = ,F -· - - 1 $. '­
"-,iu n u"'iu 

Tr.us f;. . is ort.:.oGcr.:il to :ill tr,r:::s c:~cc1 t 
.l u 

( 3 . 2) 

t hose in C . . ~o cnsi::·•1 th s ortl.cc::or.,, lit:,: for j/j , 
JU 

condition 4 ~ust be satisfied , or 

r ,:2 s: 2 
u '->iu., jc: 

This i3 ~quiva]cnt to 

i/j 

? the r<.'1.·u.:.r c:::ci:t :rut ~ ­
iu 

(3 . 3) 

and ~ ~ have zero covariance , in a ny of the : r (i-1) 
JU 

c ombinat i ons . Since n may be adjns~ed , by !,he addition 

of centre points , which do not affect a ny of the 

su:n:r.ations , if 

r~~ r ~~ ;r~~ ~~ 
U lU U~ JU U ~ lU'-> Ju 

i s an integer greater than n , t he desjcn may be 

made orthogona l by the addition of centre points , 

t hus increasing n to satisfy (3 . 3) . 

Consider , for exampJe , t he three-way central 

c omposi te design mentioned earlier , with t he size 

o f the simplex part of t he design made ee~eral . 



The dec:;icn i s 

ancl 

}-:: $ 2 
u.,iu -

? 
8 + 2~'--

r >~ ~2: 
u "> J_U JU 

.:.: 8 

n = 111- + n 
C 

i =1, 2 , 3 

i/j 

where n is the number of centrnl po~nts (O , O, O). 
C 

Condition 4 then requires 

') ? ?. ? L $:"- r ~~ ;r $ · ~ 
u ~iu u 4 j u u 4 ~~ ., ju 

27 

if the desicn is to be made ort~ogonal , ~~is re~uircs 

r 2 2 that (4 + o ) be an 

for thi s i1:Lci;:er arc ;,O , 32 1 31. , a!1d t:,c, forth , If 

a nd nu~b~rs of pointH ~ust both be t akc!1 i~to 

considcr:.tion, 

·_,; h c r: condition I. is s :1th; f .:. e d , 

The onl y e~tirr~te of the c ocffic i cn~ ~ in 

(3. 1) tha ~ is altr r cd by the tr&nsfcrmntio~ to S i1,;. ' 

is tha t f or ~ 0 • UGins t he ~r ~ns f ormation , the ..., 

- 1 A element of S corresponding to ~ O is 1/n, H~nce , 

in t he tra nsformed mode l , fro~ the for mula ~ 

bo =; tYu :: Y 
Thu s , test s of hypothes e s on t he transformed b

0 

are , in fact , t ests on the sampl e mean , 

The estimates an j regression sums of squares 

f or the situation in which condition 4 is me t 



28 

a r e g iv en i n t able 4 , The only part of th e t ab l e 

t ha t docs no t apply in t he general c ase is that 

f or /3 . .. 
l J. 

Table 1+ 

Estimate s and r egression ~urns of squares 

when individual co e fficients are to be t es t ed 

Coo f f i c ieut Estima t e s . s . D. F. 

f:,
0 

(;nean) 

{!, i 

/3ii 

(3 ij ( i/ j ) 

b , >- ., 
l. ;-: -.. s ,. J U 

.. .c. ·"· I.A. 

o . . r r . .. ... 
- .. , , ':, ., .,-.1, . 
.;_.~ LA _:..... U. 

b. _z. .,:. . • - ✓-· 
lJ U'-:, .:..t/ -. .,~l 'J. 

In the e 01ent tbi t the con d.:_ tion l c~di r.E; to 

o r thogo c nl~ ty tet~ecn different quadrati: tcr~s 

1 

1 

1 

: 

doeG not hold , it W.il~. be n eccs.3ary to inve rt t hat 

submat r ix of S t hat pertains t o the quadrat ic terms . 

De n ote t his su~~ntrix by Q, That i s , Q is t h e submatrix 

whose ( i , j ) th element is 1.:(. s- . The 
U J.U J U 

e l ements of b wi l l b e c a l led 

corresponding 

and t h ose of t he appropr iat e par t o f the t ran sformed 

X matr i x wil l be cal led z. That i s , the (u, i )t h 



element of Z will be Siu• 

Now 

Q = ZTZ 

~Q = Q-1zT1 

29 

In order to te s t a s ubse t of the /3 . . , t ogeth~r 
l. l. 

with their associated effects on the constant term , 

it is necessary to r efit the model . To do this, 

one must specify which coefficients, of the /3 . . , are 
l. l. 

to be tested , which are taken to be already fitted , 

and which are to be ignored, These l a tter are accounted 

for by deleting t he corresponding columns of Zand 

thereafter ignoring them , Thu s , without loss of 

generali ty , those to be ignor ed may be disregarded 

entirely, assuming that {3
11

, ,,. ~ pp consist O!lly 

of those to be tested and those considered already 

fit ted , Assume that the firs t p
1 

elements of ~Q 

have been fitted, and that t he last p- p
1 

are to 

be t ested , Now assume that Q, Zand ~Qare appropriately 

partitioned, That is, 

The reduction in the residua l sum of squares 

arising from fitting EQ is 

~~ZTl = lTZQ-1ZTl 

and that from fitting 

T T 
!>Q1z1r 

}2Q1 alone is 

T -1 T 
= l. Z1Q11z1l 

Hence the improvement from fitting 2Q
2 

is 

lT(ZQ-1 ZT - z,Q;~z;)1 (3.4) 



30 

Representing Q- 1 
by P, with suitable partitioning , 

and using the formula for the inverse of a partitioned 

matrix, 

Als o 

T T T T ZPZ = z
1

P
11

z
1

+2Z
1

P
12

z
2

+z
2

P
22

z
2 

since 
T 

p12=P21• 

The bracketed expression in (3.4) now becomes 

Hence the improvement in the residual sum of squares 

When £Qz c ons i sts of a single element, bii' say , 

this r educes to 

2 
b . . /p .. 

l. l. l. l. 

where p .. is the ith diagonal element of P, This 
l. l. 

enable s a test to be made o f the hypothesis b .. =0, 
l. l. 

in the presence of the other quadratic coefficients , 

Special forms for Q 

The above a nalysis covers the c ase of general 

Q, However 9 in many cases it will be found that Q 

can be put into the form 

T 
Q=b,.+tr 

where L\ is easily inverted (usually diagonal) and 



31 

y is s ome vector. This pattern arises particularly 

in the case of permutation designs (which include 

central composite designs based on full factorials). 

These designs are such that if, for each point each 

co-ordinat e ~- is divided by the scale fact o r (I,~ )i , 
l. U l.U 

then for any particular point, every permutation of 

these standardized co-ordinates exists in the design. 

Thus, if, using~~ for the standa rdi zed co-ordinates, 
1U 

there exists a point 

then for every permu ta tion of these values , there 

exists a point ( which may be the same point if the 

permuted co-ordina tes a r e equal) whose co-or dina tes 

are these permuted values, This ar r a n geme nt h a s the 

effect that I( r;,. , i/j, has the for m cI~~ r,t,,~ 
U l.U JU U 1U U JU 

where c is a cons tant, indepen de nt of i or j, Thus 

2 2 T r is proportional to (I~1 ... r,~ ) • Then the _ u u u pu 

ith diagonal element of the diagonal matrix is 

r.t.,~ -Cc+1)(t~~ )
2 

i=1, 
U l.U n U l.U 

• • • p 

The inverse of this special form of Q is 

·- readily calculated as 

-1 -1 111..-1 
Q = fl --u y Tyfs1 

µ ~ -
T 11. -1 -1 

where µ is the scalar 1+y u y, thus Q has the 

same form as Q• 

If fl is block di agonal, with blocks ~ , s=1, ••• r 
6 

-1 and the corresponding blocks of Q are P, then 
s 

p = 6,-1 _ 1D. -1 T ~-1 
s s µ s l s'f..s s 



which has inverse 

where 

p-1 
s 

T ~-1 
µs=Ys s fs• 

32 

Hence, usi ng (3.5) 3 the improve ment from fitting 

the sth block, in the presence of th e other coefficients 

is 

b T A b + ( T ) 2/( T I\. -1 'r A -1 ) 
- Qs J..l s - Qs EQs[s 1+? ~ l -'{su. s ! s 

and if/). is diagonal, the improveruant from fitting 

b .. in the pres ence of the remaining quadrati c 
1.1. 

coefficients is 

2 2 2 
b .. [d .. +y . /(1+ r; y ./d . . )] 

1. i i 1. - 1 j Ii- J J J 

using d .. for the diagonal elements of~. 
l. l. 

Rota table desi ~ns 

The conditions for a second or der design 

to be rotatable are (Box and Hunter (1 957)) th a t 

all t he moments containing an o dd po we r be zero, 

and t hat the two kinds of standardized fourth moment 

each be constant . Al so , the relations hip 

r;.$'.2 c 2 
A _ 0 u'-:>i uc.,ju 

4 - r,f/ r:E,~ 
U 1.UU JU 

must hold for all i, j, and 1. Thus A4 is the 

basic parameter for the design. 

In the present notation these conditions 

become, using (3.2) 

q .. 
J. l. 

2 A 2 2 
= ~~iu = <3 4- 1 )(Eu<;iu) /n 

q .. 
l.J 

= E~. , . = (.A4-1)E~~ Es~ /n 
U l.U JU U l.UU JU 



Now 9 using the notation of the previous 

subsection, 

and 

from which 

X4-1 -1 

Y = <->~o:~2 
n u 1u 

~ = 

1 
µ 

n 

= 

(

o:~2 >2 
u½1u 

0 

. 

Evidently, from the definition of A4 and 

equation (3.3), orthogonality is achieved if A4=1, 

which would imply y=O. 

The (i,j) th element of Q- 1 
is 

n t h.. >-4-1 
-- J..J -
2.X " 2 4 (L~~ ) A4( p+2)-p 

U J..U 
L~~ :~~ } 
U l.UU JU 

where & .. is the Kronecker delta and has the value 
J..J 

33 

(3.6) 

1 if i=j, and otherwise zero . From (3.6) the diagonal 

elements 

and 

-1 
of Q are 

p .. = 
J..1 

n 

nCA4(p+1)-p+1J 

2>.4c>.4 (p+2)-pJ 

1 

i:~2 
u'-:>juyu 

LF2 
u'>ju 

from which the effect of fitting b .. is easily 
1. J.. 

2 
calculated by b .. 1 . 

1.J.. pii 



4. Overlaid Experimental Designs 

Introduction 

As noted in section 1, the control of error 

by blocking ha s been considered by a number of 

authors. The design requi rements in this case are 

recapitula ted below, 

34 

It is natural to ex tend this to the t wo-way 

class ification situation, both with and without 

interaction, Further extension, to multiple classification 

models is likely to make the scheme unwieldy in 

pra ctice, but is co nce ptually straightforward, 

Another natural development is to assume 

that more accurate information may be wanted on 

the classific a tion part of the de sign than on the 

regression part, In thi s situation, a split-plot 

arrangement might be us e d, with closely rel a ted 

sub-plots containing representatives of each of 

the classification treatments, and each whole plot 

concerned bearing only one combination of the 

regression treatments, Alternatively, the emphasis 

may be placed on the regression part of the model, 

All these designs are generalizations of the 

analysis of covariance model, except that the 

regression aspect is fully analysed. 



General experimental design model 

In the succeeding discussion, the overall 

mean will be assumed to be part of the regression 

model rather than the experimental design or 

qualitative model. 

The design matrix for these compound designs 

will be represented by the partitioned mat rix 

W =(DX) 

where Xis the regressi on design matrix discussed 

earlier, and Dis the superimposed experimental 

desi g n matrix. 

Suppos e now that D, (mxr), is an arbitrary 

desi g n matrix , with th e imposed constraint that 

the sum of the r effects is zero (in orde r to 

include the overal l mean in the regr ession model). 

This constraint can now be used, as in normal 

experimental design, to repararneterize the 

qualitative part of the model in order to make 

all the effects orthogonal to the mean . This mea ns 

that, where j is an mx1 vector, all of whose 
-m 

elements are unity, 

where Dis the reparameterized design matrix , 

35 

Now generate a design in which the whole 

design matr ix Dis repeated n times, each repetition 

corresponding to ??e poi~t of some regression 

design with matrix x. If the rows of X are represented 



T 
by ~1 I 

Now 

and 

T 
~n' the overall design now has the form 

w = 

T 
~1 

D I 

I 
I T 
I X _____ , __ -1_ 

I 
I 

I 
I 
I 
I 
I 
I 
I 
I 

T 
X ~n 

T 
X 
~n 

Cons i der now the s ubmatrix 

W.= 
l. 

D 

T 
\V. W. = 

l. ). 

= 

= 

I 
I 

I 
I • 
I 'I' 

X. 
-J. 

( D . T) = J X. 
-m-i 

(D j x~) 
-m-i 

36 

hence, with this arrangement, qualitative and 

quantitative effects are orthogonal , Thus, any 

experimenta l design in which the effects may be made 

orthogonal to the mean may be combined with an 

arbitrary rsponse surface mode l in such a way that 



37 

qualitative and quantitative effect s may be estima ted 

separately. 

The simplest way to analyse any of these 

"full replication" designs, is to analyse the 

qualitative model, regarding the quantita t ive points 

as a furt he r classi fic a tion effect, akin to 

replicat ions , t hen sum over the qualitative model 

before fitting the regressic n . An i ntera ction 

between elements of the quali tati ve de sign, and 

the replications arising from the qu a ntitative 

model would ind i cate that the regression model 

was dependen t on the classific a tion, and varied 

accordi ng t o the classification model effects 

involved. 

However, the above strategy of r epea ting 

the entire design could well be extravagant in 

experime ntal points. In pra ctice, more co~pact 

desi gns are pos s ible. 

In the material that follows, the X desi gns 

are assumed to satis fy the constraint r equiring 

ze r o odd moments . 

One-way classification 

The simplest design is the one-way classification , 

with des ign matrix, before reparameterization, 

{ An ° ) 
D = \o 1. ... 

Jn 
r 

( 4 . 1 ) 



with n 1+ ••• +nr = n. The design matrix, after 

re paramet erization, becomes 

T 
where n ~ 
property 

D = 

= (n1 • •• nr). 

that 1;D=O• 

Thi s has the required 

38 

Now introduce the subscript w, to range over the 

classification effects, giving independent regression 

variables~- , i=1, ••• p, w=1, ••• r, u=1, ••• n. The 
l. WU W 

full design matrix, using D defined above , is W=(D X). 

Under thes e conditions, the require ment for 

orthogonality between the linear term and the ith 

regression variable, and the wth classification effect 

is 
n 

w 
I: { -

U=1 iwu 

~ n 
n w 

w I: I: f 
n w=1 U=1 l.WU 

= 0 

However, b~ the requirement of z e ro o d d moments, 

the latter term must be zero. Hence the classification 

effects are orthogonal to the linear effects if 
n 

w 

u~1siwu - 0 for all i, w 

The equivalent for the regression interaction 

effects is derived in an identical way, and gives 
n 

" I:
1

~. >. = 0 for all iFj, all w u= "='1.wu .,JWU 

For quadratic effects, the requirement is 
n 

w 2 
E c;. = 

U:1 1.WU 

r n n w 
_w E E c.2 
n w=1 u=1',iwu 

which is identical to the proportionate variance 



requirement derived in a more intuitive manner by 

Box and Hunter (1957). 

Two-way classification designs 

In the t wo-way clas s ification, without 

classification interaction, the one-way conditions 

must be satisfied for each of the classifications. 

In addition, if the two sets of class i fication 

39 

effects are to be ortho gonal (using v as the subscript 

for t he second classification) 

n = n n /n 
WV WV 

is required. 

If a classi fication interaction t erm is 

included in the model, a further condition is 

requir ed to ensure tha t this interaction is ort hogonal 

to th e quadratic term in the regression . 

In this cas e the model needs to be 

reparameterized in such a way that the main effects 

and claGsification interaction are orthogonal to 

the mean. The reparameteri zat ion is more complex, 

and is not readily expressible in matrix notation. 

Table 5 summarizes the distinct elements of the design 

matrix for a particular combina tion (w,v). 

In this form the columns of the matrix may be 

added readily, to show that each column sum is zero, 

and hence that each eff~~t is orthogonal to the 

overall mean. The condition for orthogonality between 



40 

main effects is conceptually straightforward and is 

obtained from: 

Sum over rows of table (Number of similar rows x 

w-treatment effect xv-treatment effect)= 0 

which, noting the cocmon factor (n +n )( n +n ) is 
W WV V WV 

n 1 
(n +n )(n +n )(~ - -) = 0 

w WV V WV n n n 
VI V 

Table 5 

Summary of reparameterized design matrix for 

the two-way classification with interaction c ase . 

From Number w V ( w,v) 
of similar effect effect effect 

rows 

(n +n )x (n +n )x n X 
w WV V WV WV 

Both 1 1 1 1 1 1 1 1 
n -- -- +-

& w V WV n n n n n n n 
w V WV w V 

w but 1 1 1 1 1 
n -n -- -(- --) 

not V w WV n n n n n 
w w 

V but 1 1 1 1 1 
n -n -- -(- --) 

not w V WV n n n n n 
V V 

Other 1 1 1 n-n -n +n --terms w V WV n n n 

Hence, as before, the orthogonality requirement, for 

main effects, is •, 

n = n n /n 
WV WV 

The condition for orthogonality of thew-effect 

n 



and the interaction, obtained in th e same way, is 
n 

n (n +n )(1 - __!!,,;!_) = 0 
WV w WV n n n 

W V 

which leads to the same condition as before. 

41 

In discussing orthogonality between classification 

effects and regression effects, it is convenient to 

define 

~ 2 
,,./,_ = S· ' ~- ~. ' or ~l.WVU rwvu iwvu iwvu½Jwvu 

as required. Also, use the dot notation to denote 

summation , thus for example<\) = r rl, • 
••. wvuTwvu 

Now the orthogonality requirements can be 

derived quite quickly from table 5, and summarized 

as: 

For orthog on a l i ty of regressi on 
effects with : 

w - effects: ~ <1?w. • -
w 

effects: 1 
V - n<\,.v.-

V 

14, = 0 n ••• 

1cp 0 = n ••• 

w, v interaction: 1 A- _ 1<'\? _ 14> + let, = O 
n 't' wv . n w. • n • v. n ••• 

WV W V 

or, using the main effects conditions in the interaction 

condition, 

1 <p - l<p 
n wv. n ••• 

WV 

The main effects conditions are implied by this 

condition, which is thus the requirement for orthogonality 

of regression effects and classification effects. In 

terms of the original regression variables, this 

condition becomes 

l:~. 
U l.WVU 

>::~2 
u iwvu 

= r~. {. = o 
U l.WVU JWVU 
n 

= ..:!!:!. rr.r>2 
n wvu'->iwvu 



42 

As an illustration, consider the case in whi c ~ 

the whole of a response surface design is repeated 

for each w1 v combination. In this case the first 

two condi tions above are automatically satisfied. 

Also 

I:~~ = K s a 
U ',1WVU t y 

is cons tant, and since n is constant (equal to 
WV 

the size of the re gressi on design) the last condi tion 

reduc es to 
n 

K. ::= WV (~ K) 
n n 

WV 

wh~ch is also sa t isfied. Thus the design satisfies 

the conditiocs, in agreement with the general r esult 

on the full replication model derived earlier. 

Re gre ss ion mod e l d ependent on cl assi fication 

A natural extens i o n of the class ification model 

is to allow the re gression coefficients to vary with 

the classification effect. Thus~ is r epl aced by a 

serie s A , w=1, ••• r. To achieve orthogonality with ~w 

this scheme, the full replication ty pe of desi gn is 

required. 

The simplest and most natural way to proceed, is 

to fit the regressions separately, and combine the 

results for an overall analysis of variance 

afterward. 



5• Summary 

After a general de velopment of the theory 

behind response surface methodology, with particular 

reference to polynomial models and rotatable designs , 

section 1 of this thesi s gives a rigorous development 

of the justification of the standard F-tests used. 

In particular, the l ack of fit tes t, using an error 

estimate based on point replication, is justified. 

Section 2 surveys the literatu re relating to 

response surfaces, with the exception of recent 

bias-oriented work . The e ~phas is i s on theore tical 

developme nt, rather tha n on applications. 

In section 3 the analysis of second order 

designs i s considered in some detail, the aim 

being to provide a means of testing hypotheses 

about individua l coefficients. This separation 

is acheived for the slightly restricted c ase in 

which the d epe ndent variables have zero odd moments 

about their means . Conditions for orthogonality 

between different quadratic terms are developed. 

Methods are derived for testing subsets of the 

quadratic terms when orthogonality does not hold, 

and, in particular, the case in which the part 

of the sums of squares and cross-products matrix 

relating to t?e quadrat~? terms has a particular form 

is considered. Finally, the ideas developed are 



44 

applied to rotatable designs. 

Section 4 considers the combination of response 

surface designs with ordinary qu a litative experimental 

designs. The first design considered is a general 

one enabling the combination of any response surface 

design with any experimental design in which the mean 

is orthogonal to the tre a tment effects. Since the 

de sign described is like ly to be extravagant in 

experimental points, considera tion is given to 

orthogonality conditions for general one- and two-way 

classification designs . In particular, the two-way 

classification with interaction is studied in some 

detail. Finally, brief me ntion is ma de of the 

possibility of regression coefficients differing 

with treatments. 



45 

6. References 

ANDERSON, R.L. (1953) "Recent advances in finding 

best operating conditions." J.Am.Statist.Assoc 48 

789-798 

BOSE, R,C. & CARTER, R,L , (1 959) "Complex representation 

in the construction of rot atable designs•" 

Ann.Math.Statist. ~ 771-780 

BOSE, R,C. & DRAPER, N,R. (1 959) "Second order 

rotatable designs in three dimensions," 

Ann.Math.Stati st. 30 1097-1112 

BOX, G.E.P, (1952) " Multi factor designs of first order." 

Bio me trika 39 49-57 

(1 954a) Discussion on the symposium on interval 

estimat ion. J. R. Statist.Soc. B 16 211-212 

(1 954b) "The exploration and exploitation of 

r esponse surfaces: some general considerations 

and examples." Biometrics 12. 16-60 

(1959) Answer to query: "Replication of non­

centre points in the rotatable and near -rotatable 

central composite design." Bi ometrics .J.:?. 133 

BOX, G,E,P. & BEHNKEN, D.W. (1960a ) "Simplex-sum 

designs : a class of second order rotatable 

designs derivable from those of first order." 

Ann.Math.Statist. 2.1 838-864 

(1960b) 11 S01:1e new \hree level designs for the 

study of quantitative variables." Technometrics 

2 455-475, erratum 3 576 - -



BOX , Q.E.P. & DRAPER, N.R. (1959) "A basis for the 

selection of a response surface design." 

J.Am.Statist .Assoc . 2,!±, 622-654 

BOX, Q.E.P. & HUNTER, J . S . ( 1954) "A confidence 

46 

region for the solution of a set of simultaneous 

equations with an application to experimental 

design. " Biometrika i:!_ 190-199 

(1 957) " Multifactor experimental designs for 

exploring response s urfaces." 

Ann.Math.Statist 28 195-241 

BOX , G.E . P. & 'l'ID'/,ELL, P . W. (1962) " Transformati ons 

of the indepe!1dent variable." 

Technometrics ~ 531-550 

BOX, G. E .P. & WILSON, K.B. (1951) "On the experimental 

attainme nt of optimum conditions." 

J. R.Statist.Soc . B 12. 1-45 

BOX, G.E .P. & YOrLE, P .V. (1955) "The exploration 

~nd exploitation of response surfaces : an 

exampl e of the link between the fitted surface 

and the basic mechanism of the system." 

Biometrics 11 287-32~ 

BOX , M. J . (1968) "The occurrence of replications in 

opt imal designs of experiments to estimate 

parameters in non-linear models." 

J.R.Statist 0 Soc. 30 290-302 

CHERNOFF, H. (1 953) " Locally optimal designs for 

estimating parameters." 

Ann.Math.Statist. 24 586-602 



47 

DAS, M.N. (1961) "Construction of rotatable designs 

from factorial desi gns." 

J.Indian Soc. Agric. Statist • .1l 169-194 

(1963) "On the construction of second order 

rotatable de signs throu gh balanced incomplete 

bloc k designs with blocks of unequal size . 11 

Bull.Calcutta Stat. Assoc. ~ 31-46 

DAS, M.N. & NARAS H'.HAN , V.L. (1 962) "Construction of 

rotatable de signs through balanced incomplete 

block designs." Ann . Math . Statist. 33 1421-1439 

DAVID, H.A . & ARENS, B. E . (1 959) "Optimal spacing in 

regressi on analysis ." 

Ann.Math.Statist . 30 1072-1081 

DAVIES, o.L. (1 954) The design and analysis of 

indust ri al expe ri~ents. Hafner , New York , Ch 11, 495 

DEBAUN, R.M. ( 1956) "Block effects in the determination 

of optimum conditions." Bi ometric s ~ 20-22 

(1 959 ) "Response surface designs for three factors 

at three levels." Techno me trics 1 1-8 

DEBAUN, R.M. & SCHNEIDER, A.M. (1 958) " Experience s 

with respons e surface designs." Inv.Chew (ed.) 

Experimental designs in industry 

John Wiley & Sons, New York, p235 

DE LA GARZA, A. (1954) "Spacing of information in 

polynomial regression." 

Ann.Math.Statist. 25 123-130 



48 

DRAPER, N.R. (1 960a ) "Second order rotatable design in 

four or more dimensions." 

Ann.Math.Statist . 31 23-33 

(1960b ) "Third order rotatable design in four 

di mens ions." Ann.Nath.Statist. 22 865- 874 

(1 960c ) "Third order rotatable design in three 

dimensions." Ann.Math.Statist. 22 875- 877 

(1 961a ) "Missing values in re sp onse surface 

designs." Technometrics l 389 

(1 961b) "Third order rotatable designs in thr e e 

di mensi ons: s ome specific designs ." 

Ann. Ma th.,Statist . 32 910-9 13 

(1 962) " Third order rotatable designs in three 

di mens ions : ara lysis." Te chno me t r ics i 219-234 

DRAPER , N.R. & HE RZBERG , A, M. (1 968 ) " Furthe r second 

order rotatable designs ." 

Ann.Math, S tati s t. 39 1995-2001 

DRAPER, N.R . & STONEViAN , D. M. (1968 ) " Response surface 

designs for factors at t wo and thre e levels and 

at two and four levels." Technometrics .lQ 177-192 

DYKSTRA, o. (1950) "Partial duplication of factorial 

experiments." Technometrics .l 63-75 

(1960) "Partial duplication of response surface 

designs." Technometrics £ 185-195 

EHRENFELD, s. (1955) "On the efficier.cy of experimental 

designs." Ann. Math. Statist. 26 247-255 



49 

ELFVING, G, (1952) "Optimum allocation in linear 

regression theory," Ann.Ma th,Statist. 23 255-262 

FOLKS, J.L. (1958 ) "Comparison of designs for 

exploration of response relationships ," 

Ph.D. Thesis, Iowa State College 

GARDINER, D,A,, GRANDAGE, A,H .E, & HADER , R,J. (1959) 

"Third order rotatable designs for exploring 

response surfaces," 

Ann,Math.Statist. 30 1082-1096 

GRAYBILL, F,A . (1961) Linear statistical models , Vol 1 

McGraw-Hill 

GRILICHES , z. (1957) "Specification bias in estimates 

of production functions," Farm Economics 39 8-20 

GUEST, P,G. (1958) "The spacing of observations in 

polynomial re gress ion," 

Ann,Math,Statist. 29 294-299 

HARTLEY, H,O, (195 9) "Sma llest composite design for 

quadratic response surface ," Biometrics ..!L 611-624 

HERMANSO N, H,P., GATES, C.E., CHAPMAN, J,W, & FARNHAM , 

R.S. (;964) "An agronomically useful three-factor 

response surface design based on dodecahedron 

symmetry," Ann.Math.Statist. 22. 445-446 

HERZBERG, A,M, (1964) "Two third order rotatable designs 

in four dimensions." Ann.Math.Statist. 35 445-446 

(1966) "Cylindrically rotatable designs." 

Ann,Math.Statist. 37 242-247 



50 

HERZBERG, A.M. (1967a) "A method for the construction 

of second order rotatable designs ink dimensions." 

Ann. Math .S tatist . 38 177-180 

(1967b) "Cylindrically rotatable designs of types 

1, 2, and 3. 11 Ann.Math.Statist. 38 167-176 

HILL, W.J. & HUNTER, W. G. (1 966) "A review of response 

surface methodology: a literature survey." 

Technometrics ~ 571-590 

HOEL, P.G. (1 958) "Efficiency problems in polynomial 

regression." Ann.Math.Statist. 29 1134-1145 

HUNTER, J.S, (1954) "Sea rching for o p timum co n ditions," 

Trans ,N,Y.Acad,Sci,, Series ].2 17 

(1956) "Statistica l methods for determining 

optimum conditions," 

Trans.Tenth Annual Convention A.s. q .c. 

p415 

KIEFER, J. (1 958) "On the nonrandomized optimality 

and randomized non-optimality of symmetric 

desi gns ." Ann.Math.Statist. 29 675-699 

( 1959) "Optimum experimental designs,;, 

J,R.Statist,Soc. B ~ 272-319 

KIEFER , J. & WOLFOWITZ , J. (1959) "Optimum designs in 

regression problems." 

Ann.Math.Statist. 30 271-294 

KITAGAWA, T. (1959) "Successive process of statistical 

inference applied to linear regression analysis 

and its specialization to response surface analysis." 

Bull.Math,Statist. 8 80-114 



51 

PLACKETT , R. L. (1 960) Principles of r egression analysis . 

Oxford U.P. 

UMLAND, A.W . & SMITH, W.N . (1959) ''The use of LaGrange 

multipliers with response surfaces.' ' 

Tochnometrics 1 289