Lambda Calculus
---------------

In IMP, we had a language with no functions.  With the lambda
calculus, we'll have a language with (almost) all functions.


* Abstract Syntax *

  x in Var
  i in Z
bop in Binop := + | * | -
  e in Exp   := i | e1 bop e2 | if e then e1 else e2 
              | x | \x. e | e1 e2

x     : Variable expression, refers to variable defined/bound by
        surrounding context 
\x. e : Lambda-abstraction expression, defines a new function with
        argument variable x and body e
e1 e2 : Application expression, applies the function e1 to the
        argument e2

Note: The "pure untyped lambda calculus" excludes integer, binop, and
conditional expressions.  Developed by Church and co in the 1920s and
1930s, it is a formal system (logic) in which all computation is
reduced to the basic operations of function definition and function
application.


* Small-step, structured operational semantics *

Recall that for SOS we describe an abstract machine that takes a
configuration and steps to a new configuration.  For the lambda
calculus:
1) our configurations will simply be an expression e.  
2) our step relation will take one configuration (expression) to
   another: e1 -> e2.  
3) our terminal configurations will be a value v, a particular form of
   expressions described by:
   v in Val := i | \x. e     (!! Val subset of Exp !!)

Call-by-value (CBV)

(Arith)  i1 bop i2 -> i     (i == i1 bop i2)

                 e1 -> e1'
(ArithL) -------------------------
          e1 bop e2 -> e1' bop e2

                 e2 -> e2'
(ArithR) -------------------------
          v1 bop e2 -> v1 bop e2'

(IfT)    if i then e1 else e2 -> e1     (i != 0)

(IfF)    if 0 then e1 else e2 -> e2

                             e -> e'
(IfC)    ------------------------------------------------
          if e then e1 else e2 -> if e' then e1 else e2

             e1 -> e1'
(Oper)   -----------------
          e1 e2 -> e1' e2

             e2 -> e2'
(Arg)    -----------------
          v1 e2 -> v1 e2'

(Beta)   (\x. e1) v2 -> e1[v2/x]     (!! substitution of v2 for x in e1 !!)


Note: The rules above define one evaluation system for the lambda
calculus, termed "call-by-value" (because we "call" a function with
it's argument reduced to a "value").  There is a second evaluation
system for the lambda calculus, termed "call-by-name" where we omit
the (Arg) rule and use a modified (Beta) rule:

(Beta')   (\x. e1) e2 -> e1[e2/x]     (!! substitution of e2 for x in e1 !!)

None of the results discussed today (or next time) depend crucially on
a call-by-value evaluation (though, the proofs change in minor ways),
so we'll stick with call-by-value for the time being.

Defining substitution is tricky; we'll look at it in some detail
below.  For now, we'll use our intuition and examples that avoid the
tricky cases.

Example:

(\x.\f. f x) 3 (\y. (y + 1)) ==
(((\x. (\f. (f x))) 3) (\y. (y + 1)))     (!! explicitly parenthesized !!)
-> ((\f. (f 3)) (\y. (y + 1)))
-> ((\y. (y + 1)) 3)
-> 3 + 1
-> 4

The power (and, for some, the appeal) of the lambda calculus comes
from the ability to express functions that return (or accept) other
functions easily.

IDENTITY == \x. x
COMPOSE == \f. \g. (\x. f (g x))
INC == \y. y + 1
TWICE == \h. (COMPOSE h h)

But, this power comes at a price.  Recall that every IMP program
either (a) evaluates to a terminal configuration <skip;s>, or (b)
diverges.  With (this formulation of) the lambda calculus, we have
both of those options:

a) (TWICE INC) 2 ->* 4
b) OMEGA = (\f. (f f)) (\f. (f f))
   OMEGA -> OMEGA -> OMEGA -> ...

but we also have another option:

c) INC IDENTITY -> (\x. x) + 1 
   where no evaluation rule applies, but neither do we have a terminal
   configuration.  We could technically describe the situation as
   "having an irreducible, non-value configuration", but we normally
   describe it simply as "getting stuck" or "going wrong".

Our goal today is to introduce a type-system and prove that
"well-typed programs don't get stuck/go wrong".

But first, we need to understand substitution.


* Substitution *

Why is substitution problematic?  

(x (\x. x))[z / x] =?= z (\x. z)
                   --> unwanted substitution

(y (\x. (x y)))[x / y] =?= (x (\x. (x x)))
                       --> variable capture

Substitution is tricky because we want it to be smart (avoiding
unwanted substitution and variable capture), but we want to build it
out of stupid components (like your editor's replace command).

Substitution is also tricky because even though we are dealing with
_abstract_ syntax, it is syntax all the same.  

We first need to distinguish between free and bound variables --
variable capture occurs when we substitute an expression into a
context where its variables are bound.

FV[x] = {x}
FV[\x. e] = FV[e] - {x}
FV[e1 e2] = FV[e1] U FV[e2]
FV[i] = {}
FV[e1 bop e2] = FV[e1] U FV[e2]
FV[if e then e1 else e2] = FV[e] U FV[e1] U FV[e2]

BV[x] = {}
BV[\x. e] = {x}
BV[e1 e2] = BV[e1] U BV[e2]
BV[i] = {}
BV[e1 bop e2] = BV[e1] U BV[e2]
BV[if e then e1 else e2] = BV[e] U BV[e1] U BV[e2]

Following Pierce, we adopt the "BVC":

Bound Variable Convention: 
Expressions that differ only in the names of bound variables are
equivalent in all contexts.

Therefore, we consider
 \x. x == \y. y == \z. z
and
 \x. y x == \z. y z 
but
 \x. y x <> \y. y y

This notion of equivalence is known as alpha-equivalence.  We could be
more formal, following any of a number of different paths, but we will
elide the details here.  (See Gunter or Barendregt for more details.)

With the BVC in hand, we define substitution as:

       x[e'/x] = e'
       y[e'/x] = y     (y != x)
 (e1 e2)[e'/x] = e1[e'/x] e2[e'/x]
(\y. e1)[e'/x] = \y. e[e'/x]     (y != x and y not in FV[e'])

Note that in the last clause, we can _always_ choose an
alpha-equivalent expression to satisfy the side conditions:

(\x. x)[(\y. y)/x] == (\z. z)[(\y. y)/x] = \z. z 

(\y. x y)[y/x] = (\z. x z)[y/x] = \z. y z


* Simply-Typed Lambda Calculus *

Let's revisit the abstract syntax of the lambda calculus and add
types:

  t in Typ   := int | t1 -> t2
  e in Exp   := i | e1 bop e2 | if e then e1 else e2 
              | x | \x:t. e | e1 e2

The type "int" is called a ground type and the types "t1 -> t2" are
called higher types.


* Type system *

Inference rules are just as convenient for defining when an expression
is well-typed as they are for defining when an expression evaluates to
another.  To do so, we will define a judgement

  G |- e : t     read     "in (typing) context G, expression e has type t"

Typing contexts record the types of free variables.  A typing context
is a list x1:t1,...,xn:tn of pairs of variables and types such that
the variables xi are distinct.


(Int)   G |- i : int


         G |- e1 : int   G |- e2 : int
(Binop) -------------------------------
             G |- e1 bop e2 : int

         G |- e : int   G |- e1 : t   G |- e2 : t
(If)    ------------------------------------------
              G |- if e then e1 else e2 : t

(Var)   G1,x:t,G2 |- x : t

             G,x:t1 |- e : t2
(Abs)   --------------------------     (!! BVC !!)
         G |- \x:t1. e : t1 -> t2

         G |- e1 : ta -> tr   G |- e2 : ta
(App)   -----------------------------------
                  G |- e1 e2 : tr


Example:

 x:int,y:int |- y : int
---------------------------------
 x:int |- \y:int. y : int -> int
--------------------------------------------
 |- \x:int. \y:int. y : int -> (int -> int)    |- 1 : int
----------------------------------------------------------
 |- (\x:int. \y:int. y) 1 : int -> int


* Type Safety *

Remember, we want "well-typed programs don't go wrong".  Formally, we
want a type safety theorem:

Theorem [Type Safety]:
If e is a closed expression of type t ( |- e : t ),
then forall e' such that e ->* e',
it is the case that either 
(A) e' is a value (say, v') and |- v' : t, or
(B) exists e'' such that e' -> e''.

Note that the two alternatives (A) and (B) are closely related to the
two options (a) and (b) discussed above, but that the type safety
theorem explicitly excludes (c): "going wrong" isn't an option.

You'll use essentially the same statement of Type Safety throughout
this course, though we'll change the specific syntax, typing judgment,
and step relation as we enrich the language.

The now standard method of proving a Type Safety Theorem is to prove
two lemmas:

Lemma [Preservation]:
If e -> e' and e is a closed exp of type t ( |- e : t ),
then e' is a closed exp of type t ( |- e' : t ).

Lemma [Progress]:
If e is a closed exp of type t ( |- e : t ),
then either
(A') e is a value, or
(B') exists e' such that e -> e'.

Exercise: 
Prove the Type Safety Theorem using the Preservation and Progress
Lemmas.

[[ Substitution Lemma ]]

Lemma [Preservation]:
If e -> e' and e is a closed exp of type t ( |- e : t ),
then e' is a closed exp of type t ( |- e' : t ).

Proof of Preservation:
 Proceed by induction on the evaluation step e -> e'.
 Case (Oper): Note that e is an application (with e = e1 e2 
  and e' = e1' e2) and we have the step:

              e1 -> e1'
   (Oper) -----------------
           e1 e2 -> e1' e2

  Hence, the derivation |- e : t must take the form

          |- e1 : ta -> t   |- e2 : ta
   (App) ------------------------------
                  |- e1 e2 : t

  Apply induction to e1 -> e1'; applying the result to |- e1 : ta -> t,
  we conclude that |- e1' : ta -> t.  Thus, |- e' : t, namely:

          |- e1' : ta -> t   |- e2 : ta
   (App) -------------------------------
                  |- e1' e2 : t

 Case (Arg): Similar to (Oper).
 Case (Beta): Note that e is an application (with e = (\x:ta. e1) v2
   and e' = e1[v2/x]) and we have the step:

    (Beta) (\x:ta. e1) v2 -> e1[v2/x]

  Hence, the derivation |- e : t must take the form

               x:ta |- e1 : t
   (Abs) --------------------------
          |- (\x:ta. e1) : ta -> t    |- v2 : ta
   (App) ----------------------------------------
                   |- (\x:ta. e1) v2 : t

  We would like to combine the facts  x:ta |- e1 : t  and  |- v2 : ta
  to conclude that  |- e1[v2/x] : t.  Intuitively, this should work
  because every (free) occurence of x in e1 is expected to have the
  type ta (i.e, uses  (Var) x:ta |- x : ta  in the derivation), and
  could be replaced by v2 which has type ta (i.e., use  |- v2 : ta  at
  the corresponding point in the derivation).  We formalize this as a
  Substitution Lemma (see below), and conclude that |- e1[v2/x] : t.

  Thus, |- e' : t, namely |- e1[v2/x] : t.
 Case (ArithL,ArithR,IfC): Similar to (Oper).
 Case (Arith): Note that e is a binop (with e = i1 bop i2 and e' = i)
  and we have the step: 

   (Arith) i1 bop i2 -> i     (i == i1 bop i2)

  Hence, the derivation |- e : t must take the form

            |- i1 : int   |- i2 : int
   (Binop) ---------------------------
               |- i1 bop i2 : int

  and t = int.  Thus, |- e' : t, namely:

   (Int) |- i : int

 Case (IfT): Note that e is a condional (with e = if i then e1 else e2 
  and e' = e1) and we have the step

   (IfF) if i then e1 else e2 -> e1     (i != 0)

  Hence, the derivation |- e : t must take the form

         |- i : int   |- e1 : t   |- e2 : t
   (If) ------------------------------------
            |- if i then e1 else e2 : t

  Thus, |- e' : t, namely: |- e1 : t.
 Case (IfF): Similar to (IfT).
QED

Lemma [Substitution]:
If  x:t' |- e : t  and  |- e' : t',
then  |- e[e'/x] : t.

Proof: 
Exercise.

[[ Canonical Forms ]]

Lemma [Progress]:
If e is a closed exp of type t ( |- e : t ),
then either
(A') e is a value, or
(B') exists e' such that e -> e'.

Proof of Progress:
Proceed by induction on the typing derivation |- e : t.
 Case (Var): Impossible.
 Case (Int,Abs): Immediate, as e is a value.
 Case (App): Note that e is an application (with e = e1 e2) and we
  have the derivation:

    |- e1 : ta -> t  |- e2 : ta
   -----------------------------
           |- e1 e2 : t

  Apply induction to |- e1 : ta -> t.
  If there exists e1' such that e1 -> e1' (i.e, (B')),
  then there exists e' such that e -> e', namely:

              e1 -> e1'
   (Oper) -----------------
           e1 e2 -> e1' e2

  Otherwise, e1 is a value (i.e., (A')); hence, e1 = v1.
  Furthermore, |- v1 : ta -> t implies that v1 is a lambda
  abstraction; hence, v1 = \x:ta. e1'.  (!! This implication, which is
  easy to verify for the simply-typed lambda calculus, is usually
  extracted as it's own Canonical Forms Lemma; see below. !!)

  Apply induction to |- e2 : ta.
  If there exists e2' such that e2 -> e2' (i.e, (B')),
  then there exists e' such that e -> e', namely:

             e2 -> e2'
   (Arg) -----------------
          v1 e2 -> v1 e2'

  Otherwise, e2 is a value (i.e., (A')); hence, e2 = v2.

  Hence, there exists e' such that e -> e', namely:

   (Beta) (\x:ta. e1') v2 -> e1'[v2/x]

 Case (Binop,If): Similar to (App).
QED

Lemma [Canonical Forms]:
(1) If v is a closed value of type int, 
    then v = i.
(2) If v is a closed value of type t1 -> t2 ( |- v : t1 -> t2),
    then v = \x:t1. e'.
Proof: Straightforward.


* Wrap Up *

We've introduced the simply-typed lambda calculus and its type system.
We've proven type safety, showing that "well-typed programs don't go
wrong".  Unfortunately, we've given up something in the process.  We
wanted to rule out getting stuck, but we were o.k. with divergent
evaluations.  Recall our divergent example:

OMEGA = (\f:??. (f f)) (\f:??. (f f))

You can check that there is no type t such that OMEGA is a closed exp
of type t.  If we have an interpreter that only executes well-typed
terms, then we can't evaluate OMEGA, even though it doesn't go wrong.
We'll see next time that in ruling out option (c), we've also ruled
out option (b), leaving exactly one option for well-typed lamdba
calculus programs: (a) evaluation to a terminal value.

References:

Pierce: Ch 5, Ch 8, Ch 9
Gunter: Ch 2
Winskell: Ch 11