Lambda Calculus
---------------

Last time, we introduced the simply-typed lambda calculus and its type
system.  We proved a type safety theorem, showing that "well-typed
programs don't go wrong".  However, we noted that we couldn't find a
type t such that  OMEGA = (\f:??. (f f)) (\f:??. (f f))  is a
well-typed expression of type e.  Today we will confirm that, in
contrast to IMP and the untyped lambda calculus, the simply-typed
lambda calculus does not admit (well-typed) expressions whose
evaluation diverges.


* Small-step, Call-by-value *

First, let's review the small-step, call-by-value operational
semantics for the lambda calculus:


(Arith)  i1 bop i2 -> i     (i == i1 bop i2)

                 e1 -> e1'
(ArithL) -------------------------
          e1 bop e2 -> e1' bop e2

                 e2 -> e2'
(ArithR) -------------------------
          v1 bop e2 -> v1 bop e2'

(IfT)    if i then e1 else e2 -> e1     (i != 0)

(IfF)    if 0 then e1 else e2 -> e2

                             e -> e'
(IfC)    ------------------------------------------------
          if e then e1 else e2 -> if e0' then e1 else e2

             e1 -> e1'
(Oper)   -----------------
          e1 e2 -> e1' e2

             e2 -> e2'
(Arg)    -----------------
          v1 e2 -> v1 e2'

(Beta)   (\x:t. e1) v2 -> e1[v2/x]


* Large-step, Call-by-value *

We can also give a large-step, call-by-value operational semantics for
the lambda calculus.


(Int)   i => i

         e1 => i1   e2 => i2
(Arith) ---------------------     (i == i1 bop i2)
          e1 bop e2 => i

             e => i   e1 => v
(IfT)   ---------------------------     (i != 0)
         if e then e1 else e2 => v

             e => 0   e2 => v
(IfF)   ---------------------------
         if e then e1 else e2 => v

(Abs)   \x:t e' => \x:t. e'

         e1 => \x:t e'   e2 => v2   e'[v2/x] => v
(Beta)  ------------------------------------------
                        e1 e2 => v


Trivial Exercise: 
What changes are necessary to give a large-step, call-by-name
operational semantics for the lambda calculus?

As with IMP, we took the small-step semantics as the definition of
what it means for a lambda calculus program to evaluate, so we are
obligated to prove that the large-step semantics coincide with the
small-step semantics.  That is we (and by we, I mean you) are
obligated to prove that:

Theorem:
1. e => v'    implies   e ->* v'
2. e ->* v'   implies   e => v'
Proof: Excercise

Having this theorem in hand, we can jump back and forth between the
small-step and the large-step semantics depending on which is more
convenient for the task at hand.  In particular, note that if we wish
to prove some theorem about the evaluation of e to a value, we can
either use
 I) e ->* v,
    which will likely incur an induction on n (the number of steps)
    followed by induction on the (9 rules for the) small-step
    relation, or
II) e => v,
    which will only incur an induction on the (6 rules for the)
    large-step relation.

But, let's move on to proving that the evaluation of all well-typed
lambda calculus expressions must terminate.


* Proof of Termination *

To be concrete, we state the theorem we are hoping to prove:

Theorem [Normalization]
If e is a closed expression of type t ( |- e : t ),
then e is normalizable ( e ->* v / e => v).

Given everything we've seen up to this point, we would be well-advised
to attempt proving this theorem by induction.  Unfortunately, a direct
application of induction will fail.  Let's see why:

Proof Attempt of Normalization:
 Proceed by induction on the structure of the expression e.
 Case (e = e1 e2): Note that we have the derivation:

    |- e1 : ta -> t  |- e2 : ta
   -----------------------------
           |- e1 e2 : t

  Apply induction to e1; applying the result to |- e1 : ta -> t, we
  conclude that e1 => v1.  Applying Preservation and Canonical Forms,
  we conclude that v1 = \x:ta. e'.

  Apply induction to e2; applying the result to |- e2 : ta, we
  conclude that e2 => v2.

  Now, we want to show that

            e1 => \x:t e'   e2 => v2   e'[v2/x] => v
   (Beta)  ------------------------------------------
                           e1 e2 => v

  But e'[v2/x] is not a sub-expression of e.  
XXX

We will run into similar problems attempting to induct on 
 * the size of the expression e
   because the size of e'[v2/x] may be larger than the size of e,
 * the structure of the type t,
   because the type ta -> t is not a sub-type of t (!! Note: we mean
   "sub-type" in the syntactic sense, not in the subtyping sense. !!),
 * the size of the type t,
   because the size of the type ta -> t is larger than the size of t,
 * the structure of the derivation |- e : t,
   because the derivation of |- e'[v2/x] : t, which exists by the
   Substitution Lemma, is not a sub-derivation of |- e : t,
 * the size of the derivation of |- e : t,
   because the size of the derivation of |- e'[v2/x] : t may be larger
   than the size of the derivation of |- e : t,

To make progress, we need to introduce another proof technique: a
logical relations argument.  This technique will provide us with a
strong enough induction hypothesis to complete our proof.


* Logical Relations *

Logical relations arguments start by defining a family of relations,
indexed by types.  In our case, the relations are unary:

      V[int] = { v | |- v : int } = { i }
 V[t1 -> t2] = { v | |- v : t1 -> t2 and
                     All v' in V[t1]. v v' in E[t2] }
        E[t] = { e | |- e : t and e => v and v in V[t] }

Note that V[t] subset E[t] subset { e | |- e : t }.

Notice that we've rigged the definition so that E[t] includes the
well-typed expressions of type t that terminate and, futhermore, if it
terminates with a lambda-abstraction, that abstraction terminates when
applied to a value.  This gets us around the trickyness with
application, but we'll need to show that we haven't left out any
well-typed expressions.

Lemma:
If e in E[t], 
then e is normalizable ( e => v / e ->* v ).
Proof:
Immediate from the definition of E[t].

Now, we want to prove that every closed expression of type t is a
member of E[t].  We'll want to proceed by induction on the 
derivation |- e : t.  The one remaining obstacle is the fact that in
the (Abs) case, our only sub-derivation is of the form  x:t1 |- e' : t2.  
This problem is easily solved by strengthening the lemma; rather than
prove a statement for all closed expressions, we prove the statement
for all compatible substitutions that close an open expression of type t.

We extend V[] to typing contexts in the following manner:

G[.] = { {} }
G[x:t,G] = { g[x |-> v] | g in G[G] and v in V[t] }

With this definition in hand, we prove our lemma.

Lemma:
If G |- e : t and g in G[G],
then g(e) in E[t].
Proof:
Note that  |- g(e) : t  by the Substitution Lemma.
Proceed by induction on the derivation G |- e : t.
 Case (Var): Note that e is a variable (with e = x) and we have the
  derivation: 

   (Var) G1,x:t,G2 |- x : t

  Note that  g(x) = v in V[t] subset E[t].
 Case (Abs): Note that e is an abstraction (with e = \x:t1. e' 
  and t = t1 -> t2) and we have the derivation:

              G,x:t1 |- e' : t2
   (Abs) ---------------------------
          G |- \x:t1. e' : t1 -> t2

  Note that  g(\x:t1. e') = \x:t1. g(e')  (!! The BVC ensures that x
  isn't in the domain of G or g. !!)
  We have  \x:t1. g(e') => \x:t1. g(e').
  It remains to show that  \x:t1. g(e') in V[t1 -> t2].

  Consider arbitrary  v' in V[t1].
  We are required to show that  (\x:t1. g(e')) v' in E[t2].
  Note that

          |- \x:t1. g(e') : t1 -> t2   |- v' : t1
   (App) -----------------------------------------
                 |- (\x:t1. g(e')) v' : t2

  Apply induction to  G,x:t1 |- e' : t2 and g[x |-> v'] in G[G,x:t1];
  we conclude that  g[x |-> v'](e') in E[t2].
  Hence,  g[e |-> v'](e') => v''  and  v'' in V[t2].
  Furthermore,

           \x:t1. g(e') => \x:t1. g(e')   v' => v'   g(e')[v'/x] => v''
   (Beta) --------------------------------------------------------------
                             (\x:t1. g(e')) v' => v''

  and v'' in V[t2].
  Hence,  (\x:t1. g(e')) v' in E[t2].

  Therefore,  \x:t1. g(e') in V[t1 -> t2].
  Hence,  g(\x:t1. e') in E[t1 -> t2].
 Case (App): Note that e is an application (with e = e1 e2) and we
  have the derivation:

          G |- e1 : ta -> t   G |- e2 : ta
   (App) ----------------------------------
                   G |- e1 e2 : t

  Note that  g(e1 e2) = g(e1) g(e2).
  
  Apply induction to  G |- e1 : ta -> t and g in G[G]; 
  we conclude that  g(e1) in E[ta -> t].
  Therefore,  g(e1) => v1  and  v1 in V[ta -> t].
  By Canonical Forms, we conclude that  v1 = \x:ta. e'.

  Apply induction to  G |- e2 : ta  and  g in G[G]; 
  we conclude that  g(e2) in E[ta].
  Therefore,  g(e2) => v2  and  v2 in V[ta].

  Note that  \x:ta. e' in V[ta -> t]  and  v2 in V[ta] 
  implies that  (\x:ta. e') v2 in E[t].
  Hence,  (\x:ta. e') v2 => v'  and  v' in V[t].
  Therefore, we have the derivation:

           \x:ta. e' => \x:ta. e'   v2 => v2   e'[v2/x] => v'
   (Beta) ----------------------------------------------------
                          (\x:ta. e') v2 => v'

  Finally,

           g(e1') => \x:ta. e'   g(e2) => v2   e'[v2/x] => v'
   (Beta) ---------------------------------------------------
                           g(e1) g(e2) => v'

  and v' in V[t].
  Hence, g(e) = g(e1) g(e2) in E[t].
 Case (Int,Binop,If): ...
QED

Corollary:
If |- e : t, 
then e in E[t].

Exercise: 
Prove the Normalization Theorem for the call-by-name operational
semantics.

Exercise: 
Pierce proves normalization using the following predicates:
      E'[int] = { e | |- e : int and e ->* v }
 E'[t1 -> t2] = { e | |- e : t1 -> t2 and e ->* v and
                      All e' in E'[t1]. e e' in E'[t2] }
Show that these definitions are equivalent to the ones above; i.e.,
prove that forall t, E[t] = E'[t].
Hint: There is an easy proof and a hard proof.


* Extensions *

Chapter 11 of Pierce demonstrates a number of simple extensions to the
simply-typed lambda calculus.  Time permitting, we'll look at:

+ Unit +

  t := ... | Unit
  e := ... | #u

+ Product / Tuples / Record +

  t := ... | t1 * t2
  e := ... | <t1, t2> | e.1 | e.2

  t := ... | t1 * ... * tn
  e := ... | <t1, ..., tn> | e.i

  t := ... | {l1:t1, ..., ln:tn}
  e := ... | {l1 = t1, ..., ln = tn} | e.l

+ Sum / Variants +

  t := ... | t1 + t2
  e := ... | inl e | inr e | case e of inl x => e1 | inr y => e2

  t := ... | [l1:t1,...,ln:tn]
  e := ... | [l=e] | case e of [l1=x1] => e1 | ... | [ln=xn] => en

+ General recursion +

  e := ... | fix e


References:

Pierce: Ch 12, Ch 11