Optimal Designs and Linear Regression in Statistical Analysis

2014 2014 9 9 20 20

THE GOAL OF THIS TALK 2 What' " s good" configurat ion of points on the sphere ? S

OPTIMAL DESIGNS

WHATS OPTIMAL DESIGN? (by Wikipe dia)

LINEAR REGRESSION positive a be 2 Let Borel measure on the sphere . S Let be a real finite - dimensiona Hilbert l space of H 2 2 " - functions" on with the , = inner product L S , , : ( ) ( ) , . f g f x g x d f g H 2 S 2 space a By of " - functions" we , mean space a of genuine S L 2 2 functions which given with is embedding an in ( , ). L

GAUSS-MARKOV MODEL consider We linear a f regression + model = + H + 2 ( ) ( ) ( ) ( ), Y x f x f x x x S 1 1 l l where , , are a basis of , ( a is ) = noise such that f x 1 l = 2 x y = ( ) , 0 ( ) ( ) , E x E x y 0 x y 2 , , , unknown. are 1 l Can we given a take number of p observatio locations n 2 , , such that s ' i are " well - estimated ? " x x S 1 p

OPTIMAL DESIGN search We for a with rational N weights, meaning, design p = N : , . i N x i N i = 1 minimizes i The least square estimate i d 2 ( ( ) ( )) , Y x f x i i is covariance the and matrix N of given by ( ) ( : ). 2 ) = ), = d 1 M M Cov( ( ( ) ( ) f x f x i j well be should ) 1 M To require that - estimable, ( be . small

Theorem (S., to appear in Sugaku) 2 2 Let space a be of continuous - functions on . H L S supportnes full with measure Borel positive a be Let s. with = M M Then there a is design rational weights s.t. ( ) ( ). 2 2 If is the space Pol ( ) of polynomial s M of degree , then H S t 2 t = smallest has ) Kiefer : ) 1 uniform the measure ( ( (1960). = means )" = on = 2 2 M M " ( ) ( { | , Pol ( } ) Pol ( . ) fg f g S S 2 t t

CUBATURE AND HILBERT IDENTITY

WHATS CUBATURE? 1 n Let (normalize the be ..., , { Let 1 = v V measure of subset finite a be } uniform d) on , . S 1 n R and : . v S V 0 N Definition ((Spherical) Cubature) The pair ( , is ) called a if , t V cubature f ormula of strength N = k (1) ( ) ( ) ( ) f u d v f v k k 1 n S = 1 S s 1 n every Pol This is of if , t index for polynomial every for holds (1) ( ). f t 1 n homogeneou polynomial Hom ( ). f tS spherical a If cubature is of index 2 then , it antipodali is of index 2, ..., 2 . 2 t t So, it is of strength 2 if t points its have the property. ty

JUST FOR READERS INFORMATION - 1 dimensiona cubature l (quadratur is e) tied with orthogonal polynomial s. Theorem (Gauss, 1814) = Let [ , and ] probabilit a be y measure. a b Let ( orthogonal an be ) polynomial of degree and , n , , roots. its p x x x 1 n n Then we quadrature a get of degree = 2 1 n Christoffel Christoffel Number , ) ( ) ( i x Number c b n = ( ) n p f x d f x i ( ' n ) p x 1 i a 1 i n where a is n constant. c later... topic this back to come shall We

EXAMPLE 1 The following spherical a is cubature of index 2 on . S 1 0 , 1 {( cos = ( , ) ( , ) f u u d f x x 1 2 1 2 4 1 ) ( , u u S 1 2 ) = ( , ), , 0 ( , 1 u )} x u x 1 2 = Use the polar coordinate system : sin . 1 2 2 1 1 sin 2 1 2 2 = d 2 (cos ) = + = u d 1 2 1 2 4 2 2 ) ( , 0 u u S 1 2 0 1 0 , 1 {( 2 = x 1 4 ) ( , , 0 ( ), 0 , 1 {( 1 )} x x 1 2 1 = = Similarly, we have ( 0 ). u u d x x 1 2 1 2 4 1 ) ( , u u S 1 2 ) ( , ), , 0 ( 1 )} x x 1 2 simple A way of understand look to is CF ing at . Hilbert identities

WHATS HILBERT IDENTITY? Definition (Hilbert Identity) r n ( ) 2 2 r + + representa A tion of as a sum of x x x 1 1 i n n = 1 i (with positive coefficien real ts) For example, look Liouville' at identity s (1859) : 1 i 2 2 2 2 + + + = + + 2 4 4 ( 6 ) {( ) ( ) } x x x x x x x x 1 2 3 4 i j i j 4 j shown be can This noting by + that = + + + 4 4 4 2 2 4 ( ) ( ) 2 12 2 a b a b a a b b counting and number the of times given a monomial occurs on each side.

WARINGS PROBLEM Liouville identity t this used make o the first advance on Waring' problem s Lagrange' since Four s - Square Theorem. Theorem (Liouville, 1859) at of sum a is integer Every n most 4th 53 powers of integers. = + Proof. Write 6 where , m 0 . 5 n t t 4 4 2 2 = = Writing and we , have m x x y i i i j = = 1 1 i j 4 2 2 2 2 = + + + + 2 ( 6 1 = ) . n t y y y y 1 2 3 4 i i i i i copies ) 5 4 By Liouville' identity, s this is a sum of ( y of 1 t 4 and 4 summands 12 of the form ( ) . y i j i k

HILBERTS THEOREM Mathematic ians in the mid of 19th century, Liouville like (1859) and Schur Hilbert (1909), destroyed gave their similar cottage formulae industry, degrees for solving up to 10. problem. Waring Theorem (Hilbert, 1909) For any positive integers , there , integer an are ( , ) , 0 n r N n r rational numbers integers and 0, such that , k k j ( , ) N n r = k 2 2 + + = + + 2 r r ( ) ( ) x x x x 1 1 1 n k k k n n 1

Reznick extensivel discussed y aesthetic meanings of Hilbert identities famous his in paper "Sums o f even pow ers of rea l linear f orms" (Memoirs of the AMS, Vol.96, 1992, No.463) Fundamental Problem what To extent can we reduce number the ( , ) of 2 powers? th r N n r Find explicit constructi ons of Hilbert identities . connection A with cubature is key!

REZNICKS THEOREM Theorem (Reznick, 1992) The following equivalent are . = k N = (i) ( ) ( ) f u d f v k k 1 n S 1 spherical a is cubature of index = + j 2 . r N 2 2 + + = + + 2 r r (ii) ( ) ( ) c x x v x v x 2 , n 1 1 1 r n k k k n n 1 k 2 n j r = = whe re , : ( , , . ) n c v v v 2 , n 1 r k k k + = 1 2 1 j The identity of equivalent is (ii) isometric an to embedding of n N classical finite - dimensiona Banach l spacs 2 l l 2 r

Reznick Theorem establishe a s bound for the number of N + 2 2 + r 2 - powers th of linear forms identities in for ( ) . r x nx 1 Theorem (Stroud (1971), Delsarte et al. .(1977)) + 1 n r ( ( ) ) = n R dim Hom N r 1 n + 1 n r then ther , Proof. If a is e nonzero form of h degree r N = 1 n = 2 such that ) Now, putting we , have ( 1 , 0 . f h h v k N k N = k = = 0 ( ) ( ) . 0 f u d f v k k 1 n S 1

HOW GOOD IS THIS BOUND? + 7 2 1 = = = For example, if , 2 then , 7 28 . r n 7 1 identity An of Reznick (1992) exactly has terms 28 : 7 = i geometric/ its 2 = + 2 4 12 ( ) ( ) x x x x + 1 3 i i i i 7 1 indication an As of combinator significan ial ce, coordinate the take 0 , 0 , 0 , 1 , 0 , 1 , 1 ( 0 , 1 , 1 , 0 , 0 , 0 , 1 ( coefficien the 0 , 0 , 1 , 0 , 1 , 1 , 0 ( ), 1 , 1 , 0 , 0 , 0 , 1 , 0 ( ), s of terms, 7 of ts 0 , 1 , 0 , 1 , 1 , 0 , 0 ( ), 1 , 0 , 0 , 0 , 1 , 0 , 1 ( ), ignoring 1 , 0 , 1 , 1 , 0 , 0 , 0 ( ), ). : ), This is the Fano plane of equivalent or 2, order ly, 2 - (7,3,1).

MY ORIGINALS 1. " positive On cubature rules on the simplex isometric and embeddings ". (With Yuan Xu). 2. " Remarks Hilbert on identities , isometric embeddings and , invariant cubature". (With Hiroshi Nozaki).

COMBINATORIAL DESIGN , ( : ) , ( 1 = u n k S 1 , 0 2 2 + + = , : ) 1 k , u u u k u S v 1 n n i = : n d d v ( , ) S n ( , ) k Definition (Combinatorial t-design) subset A of ( , a is ) - ( , if , ) n V S k n t design on S k 1 V = ( ) ( ) f u d f v ( , ) S k n v V homogeneou every for polynomial s Hom ( ( , )). f S k n t definition This Victoir to due is (2004), Delsarte (1973).

EXAMPLE (2-DESIGN) = Let : ( , , ) . v v v V 1 7 := i i i 0 , 0 , 0 , 1 , 0 , 1 , 1 {( ), V 1 7 1 = i 0 , 0 , 1 , 0 , 1 , 1 , 0 ( ), = v iv 1 2 i 7 7 0 , 1 , 0 , 1 , 1 , 0 , 0 ( ), 1 7 2 = = / 35 u u d 1 , 0 , 1 , 1 , 0 , 0 , 0 ( ), 1 2 3 2 7 , 3 ( S ) 0 , 1 , 1 , 0 , 0 , 0 , 1 ( ), 2 u = Note 1 that = 1 i on ( 3 , ). u = i S k n 1 , 1 , 0 , 0 , 0 , 1 , 0 ( ), 1 1 7 7 1 1 , 0 , 0 , 0 , 1 , 0 , 1 ( )} = 2 = v v 1 1 i i 7 7 7 ) 1 , 3 , 7 ( - 2 ( design) 1 7 1 2 = u = = d u d / 35 1 1 3 1 7 , 3 ( S ) ) 7 , 3 ( S

CUBATURE ON THE SIMPLEX , , : 1 = n u u R T . 1 = ( ) = n n n 0 | , 0 , , u u u 1 n i 1 i The Weyl group of type say , ( contains ), B W B n n n R subgroup a of permutatio all ns of coordinate s of a vector in . K Proposition 1 (S. .-Xu, 2012) n T cubature a are there Assume of index on t + p p q 1 = i S t = p V = + ( ) ( ) ( , ) i i w f u du f f n k K | ( , | ) n S k n | | u u T i + K 1 ( , ) 1 k n i i i 1 n i i where normalized the is n w constant, and - t designs on ( , . ) n S k i i Then the following formula index has also = i i V : + p p q 1 = p = + ( ) ( ) ( . ) i i w f u du f f n k K | | n | | u u T i + K 1 1 V i i 1 n i i

Proposition 2 (S. .- -Xu, 2012) Assume that ( ) T 1 N = = , ) i n ( ) ( Hom . w f u du f f n i t n u u T 1 i 1 n Then, ( ) N w = = 1 n ( ) L ( , ) Hom , n i f u d f f S 2 t L 2 1 n | | S ( ) 1 i i i ( ) ( ) = = n T where for every , , , : , , and , u u u u u u 1 B 1 n n a is L subgroup of ( ) of sign all changes of coordinate s of a vector. W n By Propositio further by (and 2 and 1 ns generalizi them), ng we get many new spherical cubature Hilbert so (and identities : )

Theorem (Nozaki-S., ., 2012) 7 56 28 7 2 = + + 3 6 6 6 120 ( ) ( ) ( 2 ) 2 ( ) x x x x x x x x x + + + + + 2 3 4 2 3 i i i i i i i i i = Shatalov (2001) gave a evaluate to table = = minimum the N N min n N n = 2 = 3 6 for which ( ) ( ) for given : x x n i ki i 1 1 1 i k i n n N 3 3 11 4 4 23 5 5 41 6 6 63 7 7 8 8 120 9 9 457 <481 91 <114 Hardin- Sloane (1998) Delsarte- Goethals- Seidel (1977) Source Ditkin (1948) Stroud (1967) Stroud (1967) Reznick (1992) Shatalov (2001)

TRANSLATING CF INTO IDENTITIES Bajnok Theorem 1 n spherical no is There cubature of index 8 on with S ( ) W B points from the orbits of the form ( , , , 0 , ) 0 , . 1 1 n k k Bajnok Theorem (Identity Version) ( , 0 { with ) + i n n a x a ) 1 4 2 representa no is There tion of convex a as combinatio n x i + 8 of ( }. a x 1 1 ( ) . 4 4 4 2 6 2 Proof. The ratio of coefficien the + x a ts of , : 3 is 2 in x x x x ix 1 2 1 2 }. 1 + 8 But it' 5 s : 2 in any ( ) , , 0 { a x a 1 1 n n i

COMING BACK TO OPTIMAL DESIGNS

GENERALIZING MEAN-VALUE THM following The generalize Seymour s - Zaslavsky Theorem (1984). Theorem (S. ., 2014+) Let be a path - connected topologic space, al space a H of continuous Let . support. full with measure a be 2 ( - integrable functions ) on L such that , Then there a is finite subset of V 1 V = ( ) ( , ) v . f u d f f H v V Can we relax the full - support condition? How small could | be? | V

Theorem (S., 2014+) Let be a path - connected topologic space, al space a be of continuous H Let . positive a be 2 - functions on support. full with measure M L with = M Then there a is design rational weights s.t. ( ) ( ). rotational is then , ) If ly - invariant and is M the space Pol ( H 2 t with 1 there a is measure ' smallest' ( : ) Kiefer (1960) etc. What about other models? constructi Give ons of optimal designs. collaborat last The or is...

TIME!!

WHATS CUBATURE? S for 1 Problem Compute . ( , ) continuous a function on . f x y d f S 1 approximat We integral the e an weighte by sum d of 1 function v the finitely at alues many specified points on . S = + + example, For consider ( , ) . 2 f x y x y = + + ( , ) 2 f x y x y = 4 f d 1 S = + + + ) 1 + + {( 1 f 0 ) 2 f (( 0 2 )} ) 0 , 1 ( A = + { ( ) ( )} A B y x + { ( ) ( )}. f d f B f C ) 0 , 1 ( C B 1 S 1 S important is It choose to a good pointset.

INVARIANT CUBATURE subgroup a : of orthogonal the group ( ) G O n Definition (Invariant Spherical Cubature) Definition (Invariant Spherical Cubature) is ), spherical A cubature of G index say , ( , - invariant, if t V G (i) a is V union of - orbits; constant a takes (ii) on each orbit. For example, 1 = ( ) , 0 ( ( ) f u d f v 4 1 u S 0 , 1 {( ), 1 )} v 1 a is - invariant spherical cubature of index 2 on . D S 2

BAJNOK THEOREM (DETAILED VERSION) Theorem ( Theorem (Bajnok Bajnok, 2007) , 2007) There is no - invariant spherical cubature of index 8 B n B 1 n on consisting of orbits of the form , 1 ( , 0 , 1 , ) 0 , . S n Points of the form , 1 ( , 0 , 1 , B ) 0 , are of corner vectors the Weyl group of type : The fundamenta = roots l of the k group n say , , = , are , B 1 n n and , 1 : for 1 . e e e + 1 k k k n n The such that is k iff . corner vector p p k l k l

Theorem (Nozaki Theorem (Nozaki- -S., 2011/2012) S., 2011/2012) Let be a finite irreducibl reflection e group Then there . in G 1 n is no - invariant cubature of index on that consists of G t S orbits the i of corner vec the G if = tors, in the following cases : = ( ) ii 6 ; t A 1 n ( ) iii 8 , ; t if G B D n n = ( ) 10 ; t if G E 6 = ( ) 12 , , ; iv t if G F H E 4 3 7 = ( ) vi 16 ; v t if G E 8 = ( ) 24 . t if G H 4

Key ingredients of the proof Key ingredients of the proof Theorem ( Theorem (Sobolev Sobolev, 1962) , 1962) Let subgroup a be of O ( Then the ). following equivalent are : G n ( ) is 1 n (i) , a - invariant spherical cubature of degree on . V G t S G = (ii) ( ) ( ) , 0 Harm ( ) 1 , v v i t i v V Molien- -Poincar Poincar Series) Theorem ( Theorem (Molien Series) = G Let be a finite reflection group in Let . dim Harm ( ) . G integers q i i = Let 1 exponents the be of G Then, . m m m 1 2 n n 1 = i = i = i q x i + 1 m 1 x i 1 2

GENERALIZING VICTOIRS IDEA ( 1 , 0 2 2 + + B , , : ) , u u u u k u 1 , 1 n n i = : , , 1 , , K k k k k k 1 f i j = : , , , ( , ) Y y y y S k n k k k i 1 f i B ( , ) S k n f = i S ( = : i K y n 1 , ) y k n B i k i Definition ( Definition (t t- -wise balanced design) wise balanced design) a is B The set - balanced wise design, B if t 1 B f = ( ) ( ) ( , ) y Hom ( ( , )) f x dx f f S k n f K t i ( , ) S k n = 1 i i y = 1 i

PROOF OF REZNICKS THEOREM Assume that ( ) N = = n R ( ) ( , ) ( ) Hom f v d f v f v 2 k k r 1 n v S 1 k 2 r = Then, for ( ) , we , have f v x v N = i 2 2 r r r 2 r = = , , ( ) , ( ). x v x v d v x x u d u 1 i i 1 1 n n v S u S 1 This equivalent is, ly, N = i 2 r r = , , , x v x x i i c 2 , n r 1 + 2 n j r 2 r = = where . c u d 2 , n 1 r + = 1 2 j 1 j 1 n u S