Growing the Language:

Let's begin by growing the simply-typed lambda calculus in fairly
benign ways.  We'll start by adding support for unit and void,
products and sums, as well as things like integer operations (e.g

  t ::= unit | void | t1 -> t2 | t1 * t2 | t1 + t2

  v ::= x | () | \x:t.e | (v1,v2) | inl_t v | inr_t v

  e ::= v | e1 e2 | (e1, e2) | #1 e | #2 e | inl_t e | inr_t e |
        case e of inl(x) => e1 | inr(x) => e2

with (CBV) evaluation rules as follows:

  (\x:t.e) v  ->  e[v/x]

  #1 (v1,v2)  ->  v1
  #2 (v1,v2)  ->  v2

  case inl_t(v) of inl(x) => e1 | inr(x) => e2  -> e1[v/x]
  case inr_t(v) of inl(x) => e1 | inr(x) => e2  -> e2[v/x]  

plus the rules to evaluate left-most inner most expressions (outside
of a lambda).  The new typing rules are as follows:

  G |- () : unit

  G |- e1 : t1   G |- e2 : t2
  ---------------------------
  G |- (e1,e2) : t1 * t2

  G |- e : t1 * t2
  ----------------  (i = 1,2)
  G |- #i e : ti

  G |- e : t1
  ----------------------
  G |- inl_t2(e) : t1+t2

  G |- e : t2
  ----------------------
  G |- inr_t1(e) : t1+t2

  G |- e : t1+t2   G,x1:t1 |- e1 : t   G,x2:t2 |- e2 : t
  ------------------------------------------------------
  G |- case e of inl(x1) => e1 | inr(x2) => e2 : t

Notice that by stamping the inl and inr terms with the other type in
the sum, the rules remain syntax-directed.  That is, if we left of the
_t2 on inl_t2(e) then we would know that e : t1, but would have to
somehow infer t2 from the surrounding context.  Nonetheless, in what
follows, I will often omit this extra information and just write
"inl(e)" or "inr(e)".  

Notice also that there's a unit introduction rule, but not elimination
rule.  From a logical perspective, unit corresponds to "true".
Dually, there no introduction rule for void which represents the empty
type (i.e, "false").  One could imagine introducing a rule such as the
following:

  G1,x:void,G2 |- e : t

reflecting that we can never get into a situation where a variable is
bound to a value of void type.  This corresponds to a proof by
contradiction.  However, this is only really valid in a CBV
interpretation of the language (once we add in non-termination) and
there's another way to get at contradiction (or negation) through
continuations.

The denotational semantics can be extended in a fairly straightforward
fashion as well (exercise!).

It's also possible to extend the proof that every well-formed
expression terminates to this expanded language.  Of course, 
if we add support for recursive functions, this property will
break.  But let's put that off for a bit and talk about 
control constructs such as exceptions and continuations...

---------------------------------------------------------------------
The language defined so far has no support for exceptions.  However,
if we want to write a function that is only partially defined,
we can always use a sum type.  For instance, an integer division
operation could be given the type:

  div : int * int -> (int + unit)

with the idea that:

  div (x,0) = inr_int()
  div (x,y) = inr_unit(z)  where z = z/y

Note that any caller of div is going to have to do a pattern match
to eliminate the "+" type.  This corresponds to the usual situation
in C where we return an error code or the real value, except that
we use a *disjoint union* to make sure that the values are separate
from the error codes.  

Now suppose we wished to pretend like we had support for exceptions
in the language.  In particular, suppose we add two new forms of
expression:

  e ::= ... | throw | try e catch e'

with the intended semantics that throw transfers control to the
nearest (dynamically) enclosing try/catch handler.  One way to
realize this is to translate the language with throw and
try/catch to the language without it using sums. 

We begin by defining a translation on types:

  V[unit] = unit
  V[void] = void
  V[t1*t2] = V[t1]*V[t2]
  V[t1+t2] = V[t1]+V[t2]
  V[t1->t2] = V[t1] -> C[t2]

  C[t] = V[t] + unit

The idea is that values can't raise an exception, but computations
(the body of functions) can.  When they raise an exception we are
really just returning inr() as the result.  When a function doesn't
raise an exception, we are returning inl(v) as the result, where
v is the value of the function.  

Now we can define an expression translation:

  E[()] = inl()
  E[x]  = inl(x)
  E[throw] = inr()
  E[try e catch e'] = 
   case E[e] of
     inl(x) => inl(x)
   | inr(_) => E[e']
  E[\x:t.e] = inl(\x:T[t].E[e])
  E[e1 e2] = 
    case E[e1] of
      inl(v1) => (case E[e2] of
                   inl(v2) => v1 v2
                 | inr(_) => inr())
    | inr(_) => inr()
  E[(e1,e2)] = 
    case E[e1] of
      inl(v1) => (case E[e2] of
                   inl(v2) => inl(v1,v2)
                 | inr(_) => inr())
    | inr(_) => inr()
  E[#i e] = 
    case E[e] of
      inl(v1) => #i v1
    | inr(_) => inr()

and so on.  Notice how we've ensured that if e : t, then
E[e] : C[t] = V[t] + unit.  

Of course, writing this translation is somewhat awkward and
lets us see the inefficiencies in the resulting code because
we're doing all sorts of case analysis at every sub-expression.
(Note:  we can significantly simplify the translation by
first simplifying the code to A-normal form, aka Monadic
style, where all non-trivial computations must be bound to
a variable.  This will isolate the case analysis to essentially
the monadic "let" or "bind".  More on this later.)

Regardless, this is one way to model the control aspects of a language
with exceptions: simply translate them to sums.  How might we 
construct a model that more directly represents the intended
control aspects?  

One approach is to introduce the notion of a stack in the
abstract machine.  First, let's do this in a context without
exceptions.  Our new abstract machine states will be a pair
(S,e) of a stack and an expression.  The stack will be a
list of frames defined as follows:

  F ::= [] e | v [] | ([],e) | (v,[]) | #i [] | inl [] | inr [] |
        case [] of inl(x) => e1 | inr(x) => e2

Each frame has a "hole" denoted by [] in it.  I will write 
F[e] to denote the expression obtained by plugging the hole
in the frame F with the expression e.  The frames are designed
to represent what we should do with the result of evaluating
a sub-expression.  For instance, when we have a compound term
like:

  e1 e2

we want to first evaluate e1 to a value v1 and *then evaluate
v1 e2*.  So we represent what we want to do after evaluating
e1 by the frame "[] e2".  Here are the rewriting rules for
our new abstract machine.  The key reduction rules are the
same and have no effect on the stack:

  (S, (\x:t.e) v) -> (S, e[v/x])   

  (S, #i (v1,v2)) -> (S, vi)

  (S, case inl(v) of inl(x)=>e1 | inr(y)=>e2) -> (S, e1[v/x])

  (S, case inr(v) of inl(x)=>e1 | inr(y)=>e2) -> (S, e2[v/x])

The next few rules deconstruct a compound expression by pushing
an appropriate frame on the stack:

  (S, v e) -> ((v [])::S, e)

  (S, e1 e2) -> (([] e2)::S, e1)    (e1 not a value)

  (S, (v,e)) -> ((v,[])::S, e)

  (S, (e1,e2)) -> (([],e2)::S, e1)  (e1 not a value)

  (S, #i e) -> ((#i [])::S, e)

  ...

Finally, there is a rule for popping the top frame off the stack:

  (F::S, v) -> (S, F[v])

The terminal configurations of this abstract machine are of the
form (nil, v).  Notice that this abstract machine is tail-recursive
in the sense that there are no hypotheses for the rewriting rules.
That is, the derivations are all flat -- we don't need a stack
at the meta-level to implement the langauge because we've refied
this aspect in the abstract machine itself.  Also notice that
contrary to popular belief, a function call does *not* push
a frame on the stack.  

What's the relation between the stack-based machine and our
original rules?  Given a configuration (S,e) where S = 
Fn::Fn-1::...::F2::F1::nil, then we write S[e] for the
term obtained by F1[F2[...[Fn-1[Fn[e]]]...]].  Then I claim
that:

  (S,e) -> (S',e') implies S[e] -> S'[e'] or S[e] = S'[e']

Now that we've reified stacks in the abstract machine we can
start to play around with them.  In particular, we can add
support for catch/throw as follows:

  F ::= try [] catch e'

  (S, try e catch e') -> ((try [] catch e')::S, e)

  (S1 @ (try [] catch e')::S2, throw) -> (S2, e')   (no try/catch in S1)

So in one transition step, we pop off all of the frames
until we find the nearest enclosing try/catch handler
if any.  Notice that now we have two terminal configurations:

  (nil, v)

  (S1, throw)   (no try/catch in S1)

Finally, it's worth remarking that we can actually represent
stack frames using lambda terms.  In particular, each of the
"holes" can be filled with a variable which is abstracted as
part of the frame:

    F ::= \x.x e | \x.v x | \x.(x,e) | \x.(v,x) | \x.#i x | ...

The process of "filling" a hole in a frame now is just a
function application:  F[e] = F e

And stacks can be represented as compositions of frames with
nil represented as the identity function:

   nil = \x.x
   F :: S = F o S

So in some sense, our abstract machine can be viewed as a
configuration consisting of a current expression e coupled
with a function representing what we're supposed to do when
we finish evaluating e.  That function is called a continuation.

What is the type of the stack?  Well, the type of the empty
stack (\x.x) is t->t where t is the type of the whole program.
The type of the stack F::S if t'->t where S : t''->t and 
F : t'->t''.  So in some sense, every stack/continuation is
something of the form t'->t where t is the answer type of
the program.  So a configuration:

  (S,e)

is well formed when S : t'->t and e : t'.  The whole thing
then has type S[e] : t.

---------------------------------------------------------------
First-class continuations:

Now that we see a continuation is really just a stack, and
now that we've reified the stack in the abstract machine,
we can make stacks first-class objects:

  v ::= ... | cont(S)

  e ::= ... | letcc x in e | throw e1 e2

A cont(S) value is a captured stack.  The letcc operation
binds x to a copy of the current stack.  The throw operation
takes a cont(S) and installs it as the current stack with
e2's value as the thing we return to the stack:

  (S, letcc x in e) -> (S, e[cont(S)/x])

  (S, throw cont(S') v) -> (S', v)

Typing wise, it's useful to introduce a new constructor:

  t ::= ... | cont(t)

with the idea that cont(S) is a stack that expects a t
value to be "returned" to it, at which point it will return
a value of the answer type of the whole program.  Technically,
we need to adjust all of the typing rules to keep track of
the answer type.  

Logically, we can think of a continuation as something that
has type t->void (think t implies false).  The idea is that
a continuation is like a function, except that when we call
it, it doesn't return.  Rather it returns to some alternate
universe.  So, we can say that it returns nothing (void).

Interestingly, the addition of continuations to the language,
in spite of the fact that they provide some twisted control,
does not effect termination.  That is, our extended language
still has the property that every well-typed expression will
eventually terminate.