Lambda Calculus --------------- In IMP, we had a language with no functions. With the lambda calculus, we'll have a language with (almost) all functions. * Abstract Syntax * x in Var i in Z bop in Binop := + | * | - e in Exp := i | e1 bop e2 | if e then e1 else e2 | x | \x. e | e1 e2 x : Variable expression, refers to variable defined/bound by surrounding context \x. e : Lambda-abstraction expression, defines a new function with argument variable x and body e e1 e2 : Application expression, applies the function e1 to the argument e2 Note: The "pure untyped lambda calculus" excludes integer, binop, and conditional expressions. Developed by Church and co in the 1920s and 1930s, it is a formal system (logic) in which all computation is reduced to the basic operations of function definition and function application. * Small-step, structured operational semantics * Recall that for SOS we describe an abstract machine that takes a configuration and steps to a new configuration. For the lambda calculus: 1) our configurations will simply be an expression e. 2) our step relation will take one configuration (expression) to another: e1 -> e2. 3) our terminal configurations will be a value v, a particular form of expressions described by: v in Val := i | \x. e (!! Val subset of Exp !!) Call-by-value (CBV) (Arith) i1 bop i2 -> i (i == i1 bop i2) e1 -> e1' (ArithL) ------------------------- e1 bop e2 -> e1' bop e2 e2 -> e2' (ArithR) ------------------------- v1 bop e2 -> v1 bop e2' (IfT) if i then e1 else e2 -> e1 (i != 0) (IfF) if 0 then e1 else e2 -> e2 e -> e' (IfC) ------------------------------------------------ if e then e1 else e2 -> if e' then e1 else e2 e1 -> e1' (Oper) ----------------- e1 e2 -> e1' e2 e2 -> e2' (Arg) ----------------- v1 e2 -> v1 e2' (Beta) (\x. e1) v2 -> e1[v2/x] (!! substitution of v2 for x in e1 !!) Note: The rules above define one evaluation system for the lambda calculus, termed "call-by-value" (because we "call" a function with it's argument reduced to a "value"). There is a second evaluation system for the lambda calculus, termed "call-by-name" where we omit the (Arg) rule and use a modified (Beta) rule: (Beta') (\x. e1) e2 -> e1[e2/x] (!! substitution of e2 for x in e1 !!) None of the results discussed today (or next time) depend crucially on a call-by-value evaluation (though, the proofs change in minor ways), so we'll stick with call-by-value for the time being. Defining substitution is tricky; we'll look at it in some detail below. For now, we'll use our intuition and examples that avoid the tricky cases. Example: (\x.\f. f x) 3 (\y. (y + 1)) == (((\x. (\f. (f x))) 3) (\y. (y + 1))) (!! explicitly parenthesized !!) -> ((\f. (f 3)) (\y. (y + 1))) -> ((\y. (y + 1)) 3) -> 3 + 1 -> 4 The power (and, for some, the appeal) of the lambda calculus comes from the ability to express functions that return (or accept) other functions easily. IDENTITY == \x. x COMPOSE == \f. \g. (\x. f (g x)) INC == \y. y + 1 TWICE == \h. (COMPOSE h h) But, this power comes at a price. Recall that every IMP program either (a) evaluates to a terminal configuration , or (b) diverges. With (this formulation of) the lambda calculus, we have both of those options: a) (TWICE INC) 2 ->* 4 b) OMEGA = (\f. (f f)) (\f. (f f)) OMEGA -> OMEGA -> OMEGA -> ... but we also have another option: c) INC IDENTITY -> (\x. x) + 1 where no evaluation rule applies, but neither do we have a terminal configuration. We could technically describe the situation as "having an irreducible, non-value configuration", but we normally describe it simply as "getting stuck" or "going wrong". Our goal today is to introduce a type-system and prove that "well-typed programs don't get stuck/go wrong". But first, we need to understand substitution. * Substitution * Why is substitution problematic? (x (\x. x))[z / x] =?= z (\x. z) --> unwanted substitution (y (\x. (x y)))[x / y] =?= (x (\x. (x x))) --> variable capture Substitution is tricky because we want it to be smart (avoiding unwanted substitution and variable capture), but we want to build it out of stupid components (like your editor's replace command). Substitution is also tricky because even though we are dealing with _abstract_ syntax, it is syntax all the same. We first need to distinguish between free and bound variables -- variable capture occurs when we substitute an expression into a context where its variables are bound. FV[x] = {x} FV[\x. e] = FV[e] - {x} FV[e1 e2] = FV[e1] U FV[e2] FV[i] = {} FV[e1 bop e2] = FV[e1] U FV[e2] FV[if e then e1 else e2] = FV[e] U FV[e1] U FV[e2] BV[x] = {} BV[\x. e] = {x} BV[e1 e2] = BV[e1] U BV[e2] BV[i] = {} BV[e1 bop e2] = BV[e1] U BV[e2] BV[if e then e1 else e2] = BV[e] U BV[e1] U BV[e2] Following Pierce, we adopt the "BVC": Bound Variable Convention: Expressions that differ only in the names of bound variables are equivalent in all contexts. Therefore, we consider \x. x == \y. y == \z. z and \x. y x == \z. y z but \x. y x <> \y. y y This notion of equivalence is known as alpha-equivalence. We could be more formal, following any of a number of different paths, but we will elide the details here. (See Gunter or Barendregt for more details.) With the BVC in hand, we define substitution as: x[e'/x] = e' y[e'/x] = y (y != x) (e1 e2)[e'/x] = e1[e'/x] e2[e'/x] (\y. e1)[e'/x] = \y. e[e'/x] (y != x and y not in FV[e']) Note that in the last clause, we can _always_ choose an alpha-equivalent expression to satisfy the side conditions: (\x. x)[(\y. y)/x] == (\z. z)[(\y. y)/x] = \z. z (\y. x y)[y/x] = (\z. x z)[y/x] = \z. y z * Simply-Typed Lambda Calculus * Let's revisit the abstract syntax of the lambda calculus and add types: t in Typ := int | t1 -> t2 e in Exp := i | e1 bop e2 | if e then e1 else e2 | x | \x:t. e | e1 e2 The type "int" is called a ground type and the types "t1 -> t2" are called higher types. * Type system * Inference rules are just as convenient for defining when an expression is well-typed as they are for defining when an expression evaluates to another. To do so, we will define a judgement G |- e : t read "in (typing) context G, expression e has type t" Typing contexts record the types of free variables. A typing context is a list x1:t1,...,xn:tn of pairs of variables and types such that the variables xi are distinct. (Int) G |- i : int G |- e1 : int G |- e2 : int (Binop) ------------------------------- G |- e1 bop e2 : int G |- e : int G |- e1 : t G |- e2 : t (If) ------------------------------------------ G |- if e then e1 else e2 : t (Var) G1,x:t,G2 |- x : t G,x:t1 |- e : t2 (Abs) -------------------------- (!! BVC !!) G |- \x:t1. e : t1 -> t2 G |- e1 : ta -> tr G |- e2 : ta (App) ----------------------------------- G |- e1 e2 : tr Example: x:int,y:int |- y : int --------------------------------- x:int |- \y:int. y : int -> int -------------------------------------------- |- \x:int. \y:int. y : int -> (int -> int) |- 1 : int ---------------------------------------------------------- |- (\x:int. \y:int. y) 1 : int -> int * Type Safety * Remember, we want "well-typed programs don't go wrong". Formally, we want a type safety theorem: Theorem [Type Safety]: If e is a closed expression of type t ( |- e : t ), then forall e' such that e ->* e', it is the case that either (A) e' is a value (say, v') and |- v' : t, or (B) exists e'' such that e' -> e''. Note that the two alternatives (A) and (B) are closely related to the two options (a) and (b) discussed above, but that the type safety theorem explicitly excludes (c): "going wrong" isn't an option. You'll use essentially the same statement of Type Safety throughout this course, though we'll change the specific syntax, typing judgment, and step relation as we enrich the language. The now standard method of proving a Type Safety Theorem is to prove two lemmas: Lemma [Preservation]: If e -> e' and e is a closed exp of type t ( |- e : t ), then e' is a closed exp of type t ( |- e' : t ). Lemma [Progress]: If e is a closed exp of type t ( |- e : t ), then either (A') e is a value, or (B') exists e' such that e -> e'. Exercise: Prove the Type Safety Theorem using the Preservation and Progress Lemmas. [[ Substitution Lemma ]] Lemma [Preservation]: If e -> e' and e is a closed exp of type t ( |- e : t ), then e' is a closed exp of type t ( |- e' : t ). Proof of Preservation: Proceed by induction on the evaluation step e -> e'. Case (Oper): Note that e is an application (with e = e1 e2 and e' = e1' e2) and we have the step: e1 -> e1' (Oper) ----------------- e1 e2 -> e1' e2 Hence, the derivation |- e : t must take the form |- e1 : ta -> t |- e2 : ta (App) ------------------------------ |- e1 e2 : t Apply induction to e1 -> e1'; applying the result to |- e1 : ta -> t, we conclude that |- e1' : ta -> t. Thus, |- e' : t, namely: |- e1' : ta -> t |- e2 : ta (App) ------------------------------- |- e1' e2 : t Case (Arg): Similar to (Oper). Case (Beta): Note that e is an application (with e = (\x:ta. e1) v2 and e' = e1[v2/x]) and we have the step: (Beta) (\x:ta. e1) v2 -> e1[v2/x] Hence, the derivation |- e : t must take the form x:ta |- e1 : t (Abs) -------------------------- |- (\x:ta. e1) : ta -> t |- v2 : ta (App) ---------------------------------------- |- (\x:ta. e1) v2 : t We would like to combine the facts x:ta |- e1 : t and |- v2 : ta to conclude that |- e1[v2/x] : t. Intuitively, this should work because every (free) occurence of x in e1 is expected to have the type ta (i.e, uses (Var) x:ta |- x : ta in the derivation), and could be replaced by v2 which has type ta (i.e., use |- v2 : ta at the corresponding point in the derivation). We formalize this as a Substitution Lemma (see below), and conclude that |- e1[v2/x] : t. Thus, |- e' : t, namely |- e1[v2/x] : t. Case (ArithL,ArithR,IfC): Similar to (Oper). Case (Arith): Note that e is a binop (with e = i1 bop i2 and e' = i) and we have the step: (Arith) i1 bop i2 -> i (i == i1 bop i2) Hence, the derivation |- e : t must take the form |- i1 : int |- i2 : int (Binop) --------------------------- |- i1 bop i2 : int and t = int. Thus, |- e' : t, namely: (Int) |- i : int Case (IfT): Note that e is a condional (with e = if i then e1 else e2 and e' = e1) and we have the step (IfF) if i then e1 else e2 -> e1 (i != 0) Hence, the derivation |- e : t must take the form |- i : int |- e1 : t |- e2 : t (If) ------------------------------------ |- if i then e1 else e2 : t Thus, |- e' : t, namely: |- e1 : t. Case (IfF): Similar to (IfT). QED Lemma [Substitution]: If x:t' |- e : t and |- e' : t', then |- e[e'/x] : t. Proof: Exercise. [[ Canonical Forms ]] Lemma [Progress]: If e is a closed exp of type t ( |- e : t ), then either (A') e is a value, or (B') exists e' such that e -> e'. Proof of Progress: Proceed by induction on the typing derivation |- e : t. Case (Var): Impossible. Case (Int,Abs): Immediate, as e is a value. Case (App): Note that e is an application (with e = e1 e2) and we have the derivation: |- e1 : ta -> t |- e2 : ta ----------------------------- |- e1 e2 : t Apply induction to |- e1 : ta -> t. If there exists e1' such that e1 -> e1' (i.e, (B')), then there exists e' such that e -> e', namely: e1 -> e1' (Oper) ----------------- e1 e2 -> e1' e2 Otherwise, e1 is a value (i.e., (A')); hence, e1 = v1. Furthermore, |- v1 : ta -> t implies that v1 is a lambda abstraction; hence, v1 = \x:ta. e1'. (!! This implication, which is easy to verify for the simply-typed lambda calculus, is usually extracted as it's own Canonical Forms Lemma; see below. !!) Apply induction to |- e2 : ta. If there exists e2' such that e2 -> e2' (i.e, (B')), then there exists e' such that e -> e', namely: e2 -> e2' (Arg) ----------------- v1 e2 -> v1 e2' Otherwise, e2 is a value (i.e., (A')); hence, e2 = v2. Hence, there exists e' such that e -> e', namely: (Beta) (\x:ta. e1') v2 -> e1'[v2/x] Case (Binop,If): Similar to (App). QED Lemma [Canonical Forms]: (1) If v is a closed value of type int, then v = i. (2) If v is a closed value of type t1 -> t2 ( |- v : t1 -> t2), then v = \x:t1. e'. Proof: Straightforward. * Wrap Up * We've introduced the simply-typed lambda calculus and its type system. We've proven type safety, showing that "well-typed programs don't go wrong". Unfortunately, we've given up something in the process. We wanted to rule out getting stuck, but we were o.k. with divergent evaluations. Recall our divergent example: OMEGA = (\f:??. (f f)) (\f:??. (f f)) You can check that there is no type t such that OMEGA is a closed exp of type t. If we have an interpreter that only executes well-typed terms, then we can't evaluate OMEGA, even though it doesn't go wrong. We'll see next time that in ruling out option (c), we've also ruled out option (b), leaving exactly one option for well-typed lamdba calculus programs: (a) evaluation to a terminal value. References: Pierce: Ch 5, Ch 8, Ch 9 Gunter: Ch 2 Winskell: Ch 11