Compilers
Table driver parser
PDA, if we can push the start symbol then pop it off at the end we know we're done. Generic table parsing routine can use the table to parse the input.
the grammar for (1+2+3) and (1+2+3)+4 is
non-deterministic without arbitrary look ahead. This is why change a grammar from left recursive(non-terminals on the left side) to right recursive.
Back end of compiler is responsible for
optimization, and code gen
Lexical Analysis (scanning)
Take stream of individual chars, convert to tokens. Detect lexical errors(badly formed literals, illegal ch ars) discard white space, discard comments.
CFG contains
Terminal symbols (Tokens generated by lexical analyzer), non-terminal symbols, production rules, start symbol.
symbol table built during _____ _______
semantic analysis
Comparing item separated in the program or counting items will end up being a ______ ____-
semantic check.
Lexical analyzer generator
mechanically converts a set of regular expressions (and their action routines) into a program.
interpreters advantage over compilers
more flexible, better diagnostics, easier portability, take less space.
Why not regular expressions for syntactic analysis?
Because they do not have enough power being finite, we need to use CFGs because they can recognize nesting that is too deep for DFAs (input is more complex than O(n).
what does LL(1) mean?
Can we automatically construct a parser from the CFG that will create an unambiguous parse tree from any given string, or tell us the string is unparseable. Do a left to right scan of input, do a Leftmost derivation(right recursion), using 1 token of look ahead(look at the next token without consuming it). Top-down O(n) size of input string
Semantic analysis covers stuff we can't cover in cfg and isn't convenient
Comparing argument lists in function calls to that given in the function's declaration. (This isn't context-free) Ensuring a function has at least one return statement. (inconvenient with CFG, return could be anywhere in the function)
Interpreter Vs Compiler
Compiler- entire code analyzed, more efficient done once. Catches lots of errors. interpreter- one line at a time, less efficient done every time. More flexible.
Syntactic analysis (Parsing)
Context free, Take stream of tokens, and convert into a parse tree. Captures the hierarchical structure of the program (declarations definitions, blocks, statements, expressions, BUT NOT undeclared identifiers, mismatched function calls) syntactic errors are about form, NOT structure.
Semantic analysis
Context sensitive stuff,
Why we use regular expressions to do lexical scanning?
Correspondence between REs and finite automaton, and finite automaton are convertible into scanning program in mechanical way.
Front end of the compiler or the first phases
Discovers its meaning, comprised of lexical, syntactic and semantic anlysis steps.
Role of the semantic analyzer
Enforce all static semantic rules Annotate teh parse tree with any info required by intermediate code gen(clarifications: resolution of overloaded operations),(Dynamic semantic checks)
semantic content of scalar
Its Type and Value
LR(k) Parsing
Left-to-right scan of input, producing Rightmost derivation using k tokens of lookahead. Bottom-up = use production rules right to left. Use stack more sophisticated. First, k tokens in the input stream form the lookahead Shift:push the first input token onto the stack reduce: select some grammar rule e.g. X>A B C; pop C, B, A off the stack; and push X onto the stack. If parser makes it to the point where it can shift EOF marker onto t he stack, it has accepted the input as valid 3 cases in reducing that can occur: 1). top few items do not match the RHS of any rule of grammar: so we should shift next token onto stack 2). Top few items match the RHS of one rule of the grammar: reduce, popping off those matching items and pushing the LHS of the rule 3). Top few items match the RHS of multiple rules of the grammar: the grammar is ambiguous we don't know which reduction to make (there are more than 1 rule in an entry for the parse table) this is a reduce/reduce ambiguity shift/reduce conflicts: you could shift or reduce dangling else reduce/reduce conflict: the grammar itself is not ambiguous, the problem is we only get 1 token of look ahead A-> B c d A-> E c f B-> x y E-> x y shifting and reducing is done via a DFA, the DFA is applied to the contents of the stack, not parsing the CFA.
Improved CFG to make none ambiguous
Lower precedence comes first to enforce order of operations(hierarchy, all add ops come before mul ops in production rules, unless they are in parenthesis), to enforce associativity by having left recursion for that rule (expr->expr add_op term) expr->term | expr add_op term term-> factor | term mul_op factor factor->id | number | - factor | (expr) add_op-> + | - mul_op-> * | /
(In predictive parsing to decide if we can do predict parsing) To predict which production rule to use, we need 3 things for every non-terminal X:
NULLABLE(X): does x ever derive e? FIRST(X): which terminals can appear first in a string derived from X? FOLLOW(X): which terminals can appear immediately after X? given these we can construct the parse table for the CFG The parse table tells us which rule to use when we are trying to parse a given non-terminal and are seeing a given terminal every box has at most 1 rule meaning the CFG for LL(1) is non-ambiguous, boxes with no value mean parsing that results in error
Parsing complexity of regular expressions
O(n) where n is the length of the string you're trying to parse. Because the acceptance or rejection of an input string in a DFA will take no more transitions than the length of the input. (you consume a symbol on every transition)
Parsing complexity of Context free
O(n^3) where n is length of tokens you're trying to parse
Last phases of compilation do what?
Producing the corresponding target code, This is the bac k end. Comprised of optimization of intermediate code, target code generation.
patterns of token classes as
Regular expressions, easy to write, lots of theory to help with processing
Undeclared identifiers mismatch function calls are a _____ error. Why?
Semantic error because they require context.
Compiler translates _______ into equivalent _______ then ____ _____
Source program into equivalent target program and then (compiler) goes away
Target code generation
Translates intermediate form into the target language, hard to generate good code.
RE
no recursion, although it may apear, S->aS
non-deterministic without arbitrary lookahead
While the grammar is not ambiguous its non-deterministic without arbitrary look ahead. (1+2+3) vs. (1+2+3)+4 expr-> expr add_op term expr-> term term-> term mul_op factor term-> factor solution: expr-> term in the first case but expr-> expr add_op term in the second case.
The output of syntactic analyzer phase is:
a Parse tree, which represents the structure of a particular input token stream as determined by the languages grammar rules, unique for a given steam of tokens (if not, the grammar is ambiguous and needs to be fixed)
Recursive descent parser
a parser written for each non-terminal uses the parse table slighty more efficient
As static semantic phase runs it transforms the parse tree to an
abstract syntax tree, with only essential info, uses a symbol. Many compilers use AST as intermediate form to hand off to back end for code gen (lower level language) other compilers tree walk AST and generate different intermediate form.
Action and Goto table
analysis of grammar results in action and goto table when you're in a given state and you see a specific token whether you should shift or reduce it by a certain rule, then use a goto
Turing complete
any real-world general purpose computer or language can simulate the computational aspects of any other computer language.
interpretation in c happens when
at run time looks through the special characters in printf(%d, %s) to see what comes after
Code improvement happens when?
can happen right after semantic analysis, the earlier the better improvement, you cannot change the result.
dynamic semantic rules are enforced by the
compiler inserting specific code to perform the check. Example no divide by zero, if denom 0 compiler gets angry
compiling is a ______ analysis of the ________ ______
complete analysis of source code.
Parse Tree (Concrete syntax tree)
completely shows how a sequence of tokens was derived using CFG.
examples of static semantic checks
declaration of identifier before use no use of identifier in an inappropriate context correct number/ type of paras in subroutines distinct constant labels on the branches of switch non-void return type function returns a value
improved cfg
eliminate explicit alternation (expr->expr add_op term expr -> term) eliminate left recursion and do right recursion(we did left recursion originally for associativity) move recursion from left to right(but generates identical strings): (left recursion) X->Xa X->B replaced by: (right recursion) X->BX' X'->aX' X'->e
interpretation is translate ______ ______
every time
What does FLEX do?
flex takes your formal description of token categories and pukes out a c program that you can compile
How was first compiler written
in assembly and machine code.
All FA are finite the language accepted by FA can be
infinite, additionally each string in the language itself is finite.
CFGs can refer to themselves, the definition of recursive, unlike REs which have
left or right recursion exclusively
in a tree leave of an expression are always
literals or ids
C doesn't have any dynamic semantic checks
the hardware checks (divide by 0)
Java has a lot of static and dynamic semantic checks
to catch bugs and malicious code (buffer overflows, SQL injection)
Why would you write assembly?
to touch certain parts of the hardware, or for performance
Dynamic semantic checks (enforced at runtime)
variables not used in expression unless they have been assigned a value pointers are deference arrays subscripts within bounds arithmetic expressions don't over flow these depend on info that isn't known until run time.
P-code (byte code)
very simple language that is easy to translate to machine code
formal notation(CFGS, and REs) is good because
we can process them mechanically, which means we can write programs to manipulate them.
Self hosting compiler
written in its own language(c compiler written in c), compiles first time by bootstrapping(knows enough to get to the next level).