Compiler Design: Syntactic and Semantic Analysis

232 Pages·2013·3.16 MB·English

by  Reinhard Wilhelm

Reinhard Wilhelm
Helmut Seidl
Sebastian Hack
Compiler
Design
Syntactic and Semantic Analysis
Compiler Design



Reinhard Wilhelm Helmut Seidl
Sebastian Hack
Compiler Design
Syntactic and Semantic Analysis
HelmutSeidl ReinhardWilhelm,SebastianHack
FakultätfürInformatik FBInformatik
TechnischeUniversitätMünchen UniversitätdesSaarlandes
Garching,Germany Saarbrücken,Germany
ISBN978-3-642-17539-8 ISBN978-3-642-17540-4(eBook)
DOI10.1007/978-3-642-17540-4
LibraryofCongressControlNumber:2013938370
©Springer-VerlagBerlinHeidelberg2013
Thisworkissubjecttocopyright. Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broadcasting,
reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,
1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer. Violations
areliabletoprosecutionundertheGermanCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnotim-
ply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotective
lawsandregulationsandthereforefreeforgeneraluse.
Printedonacid-freepaper.
SpringerispartofSpringerScience+BusinessMedia
www.springer.com
ToMargret,Hannah,Eva,andBarbara
R.W.
ToKerstinandAnna
H.S.
ToKerstin,Charlotte,andMarlene
S.H.
Preface
Compilers for high-levelprogramming languagesare largeand complex software
systems. They have, however, several distinct propertiesby which they favorably
differfrommostothersoftwaresystems. Theirsemanticsis(almost)welldefined.
Ideally, completely formalor at least precise descriptions existboth of the source
and the target languages. Often, additional descriptions are providedof the inter-
faces to the operating system, to programming environments, to other compilers,
andtoprogramlibraries.
The task of compilation can be naturally decomposed into subtasks. This de-
composition results in a modular structure which, by the way, is also reflected in
thestructureofmostbooksaboutcompilers.
As early as the 1950s, it was observed that implementing application systems
directlyasmachinecodeisbothdifficultanderror-prone,andresultsin programs
whichbecomeoutdatedasquicklyasthecomputersforwhichtheyhavebeende-
veloped. High-level machine-independent programming languages, on the other
hand,immediatelymadeitmandatorytoprovidecompilers,whichareabletoauto-
maticallytranslatehigh-levelprogramsintolow-levelmachinecode.
Accordingly,thevarioussubtasksofcompilationhavebeensubjecttointensive
research. For thesubtasks oflexical andsyntactic analysis of programs, concepts
likeregularexpressions,finiteautomata,context-freegrammarsandpushdownau-
tomatahavebeenborrowedfromthetheoryofautomataandformallanguagesand
adapted to the particular needs of compilers. These developments have been ex-
tremely successful. In the case of syntactic analysis, they led to fully automatic
techniquesforgeneratingtherequiredcomponentssolelyfromcorrespondingspec-
ifications,i.e.,context-freegrammars. Analogousautomaticgenerationtechniques
would be desirable for further components of compilers as well, and have, to a
certainextent,alsobeendeveloped.
Thecurrentbookdoesnotattemptto beacookbookfor compilerwriters. Ac-
cordingly,therearenorecipessuchas“inordertoconstructacompilerfromsource
language X into target language Y, take ...”. Instead, our presentation elabo-
rates on the fundamental aspects such as the technical concepts, the specification
vii
viii Preface
formalismsforcompilercomponentsandmethodshowtosystematicallyderiveim-
plementations. Ideally,thisapproachmayresultinfullyautomaticgeneratortools.
The book is written for students of computer science. Some knowledgeabout
an object-oriented programming languagesuch as JAVA and very basic principles
of a functionallanguagesuch as OCAML or SML arerequired. Knowledgeabout
formallanguagesorautomataisuseful,butisnotmandatoryasthecorresponding
backgroundisprovidedwithinthepresentation.
OrganizationoftheBook
For the new edition of the book Wilhelm/Maurer: Compiler Construction, we
decided to distribute the contents over several volumes. The first volume Wil-
helm/Seidl: Compiler Design – Virtual Machines speaks about what a compiler
does, i.e., which correspondence it realizes between source program and target
program. This is exemplified for an imperative, a functional, a logic-based and
an object-orientedlanguage. For each language, a correspondingvirtual machine
(inearliereditionsalsocalledabstractmachine)isdesigned,andimplementations
are provided for each construct of the programming language by means of the
instructionsofthevirtualmachine.
The subsequent three volumes then describe how the compilation process is
structuredandorganized.Eachcorrespondstoonephaseofcompilation,namelyto
syntacticandsemanticanalysis,toprogramtransformation,andtocodegeneration.
Thepresentvolumeisdedicatedtotheanalysisphase,realizedbythefront-endof
thecompiler. Wealso brieflydiscusshowthecompilationprocessasawholecan
beorganizedinto phases, which tasks should beassigned to differentphases, and
whichtechniquescanbeusedfortheirimplementation.
Furthermaterialrelatedtothisbookisprovidedonthewebpage: http://www2.
informatik.tu-_muenchen.de/~seidl/compilers/
Acknowledgement
Besides the helpers from former editions, we would like to thank Michael Jacobs
andJörgHerterforcarefullyproofreadingthechapteronsyntacticanalysis. When
revisingthedescriptionoftherecursivedescentparser,wealsoweresupportedby
ChristophMallonandhisinvaluableexperiencewithpracticalparserimplementa-
tions.
Wehopethatourreaderswillenjoythepresentvolumeandthatourbookmay
encourage them to realize compilers of their own for their favorite programming
languages.
SaarbrückenandMünchen,August2012 ReinhardWilhelm,HelmutSeidl
andSebastianHack
Contents
1 TheStructureofCompilers. . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 CompilationSubtasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 LexicalAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 TheScreener . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 SyntacticAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 SemanticAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6 Machine-IndependentOptimization . . . . . . . . . . . . . . . . . . . 7
1.7 GenerationoftheTargetProgram . . . . . . . . . . . . . . . . . . . . 8
1.8 SpecificationandGenerationofCompilerComponents . . . . . . . 9
2 LexicalAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1 TheTaskofLexicalAnalysis . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 RegularExpressionsandFiniteAutomata . . . . . . . . . . . . . . . 12
2.2.1 WordsandLanguages . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 ALanguageforSpecifyingLexicalAnalyzers. . . . . . . . . . . . . 27
2.3.1 CharacterClasses . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.3.2 NonrecursiveParentheses . . . . . . . . . . . . . . . . . . . . . 28
2.4 ScannerGeneration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4.1 CharacterClasses . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4.2 AnImplementationoftheuntil-Construct . . . . . . . . . . . 30
2.4.3 SequencesofRegularExpressions . . . . . . . . . . . . . . . 32
2.4.4 TheImplementationofaScanner . . . . . . . . . . . . . . . . 34
2.5 TheScreener . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.5.1 ScannerStates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.5.2 RecognizingReservedWords . . . . . . . . . . . . . . . . . . 38
2.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.7 BibliographicNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3 SyntacticAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.1 TheTaskofSyntacticAnalysis . . . . . . . . . . . . . . . . . . . . . . 43
3.2 Foundations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.2.1 Context-FreeGrammars . . . . . . . . . . . . . . . . . . . . . . 46
3.2.2 ProductivityandReachabilityofNonterminals . . . . . . . . 52
3.2.3 PushdownAutomata . . . . . . . . . . . . . . . . . . . . . . . . 57
ix
x Contents
3.2.4 TheItem-PushdownAutomatontoaContext-FreeGrammar 59
3.2.5 first-andfollow-Sets . . . . . . . . . . . . . . . . . . . . . . . . 64
3.2.6 TheSpecialCasefirst1andfollow1 . . . . . . . . . . . . . . . 71
3.2.7 PureUnionProblems . . . . . . . . . . . . . . . . . . . . . . . 74
3.3 Top-DownSyntaxAnalysis . . . . . . . . . . . . . . . . . . . . . . . . 77
3.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.3.2 LL.k/: Definition,Examples,andProperties. . . . . . . . . 79
3.3.3 LeftRecursion. . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.3.4 StrongLL.k/-Parsers . . . . . . . . . . . . . . . . . . . . . . . 87
3.3.5 Right-RegularContext-FreeGrammars . . . . . . . . . . . . 89
3.3.6 Recursive-DescentParsersforRLL.1/-Grammars . . . . . 92
3.3.7 TreeGenerationbyMeansofRecursive-DescentParsers. . 96
3.3.8 ErrorHandlingwithinRLL.1/-Parsers . . . . . . . . . . . . 97
3.4 Bottom-upSyntaxAnalysis . . . . . . . . . . . . . . . . . . . . . . . . 101
3.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.4.2 LR.k/-Parsers. . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.4.3 LR.k/: Definition,Properties,andExamples. . . . . . . . . 112
3.4.4 ErrorHandlinginLR-Parsers . . . . . . . . . . . . . . . . . . 124
3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
3.6 BibliographicNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4 SemanticAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.1 TheTaskofSemanticAnalysis . . . . . . . . . . . . . . . . . . . . . . 139
4.1.1 RulesforValidityandVisibility . . . . . . . . . . . . . . . . . 144
4.1.2 CheckingtheContext-Conditions . . . . . . . . . . . . . . . . 147
4.1.3 OverloadingofIdentifiers . . . . . . . . . . . . . . . . . . . . 154
4.2 TypeInference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4.2.1 Syntax-DirectedTypeInference . . . . . . . . . . . . . . . . . 165
4.2.2 Polymorphism . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
4.2.3 SideEffects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
4.2.4 TypeClasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
4.3 AttributeGrammars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
4.3.1 TheSemanticsofanAttributeGrammar . . . . . . . . . . . . 184
4.3.2 SomeAttributeGrammars . . . . . . . . . . . . . . . . . . . . 185
4.4 TheGenerationofAttributeEvaluators . . . . . . . . . . . . . . . . . 191
4.4.1 Demand-DrivenAttributeEvaluation . . . . . . . . . . . . . . 191
4.4.2 StaticPrecomputationsofAttributeEvaluators . . . . . . . . 193
4.4.3 Visit-OrientedAttributeEvaluation . . . . . . . . . . . . . . . 200
4.4.4 Parser-DirectedAttributeEvaluation . . . . . . . . . . . . . . 206
4.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
4.6 BibliographicNotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
1
The Structure of Compilers
Ourseriesofbookstreatsthecompilationofhigherprogramminglanguagesintothe
machinelanguagesofvirtualorrealcomputers. Suchcompilersarelarge,complex
softwaresystems. Realizinglargeandcomplexsoftwaresystemsisadifficulttask.
What is so special about compilers such that they can even be implemented as a
project accompanying a compiler course? One reason is that the big task can be
naturallydecomposedintosubtaskswhichhaveclearlydefinedfunctionalitiesand
cleaninterfacesbetweenthem. Anotherreasonisautomation: severalcomponents
ofcompilersneednotbeprogrammedbyhand,butcanbedirectlygeneratedfrom
specificationsbymeansofstandardtools.
The general architecture of a compiler, to be described in the following, is a
conceptualstructureoftheprocess. Itidentifiesthesubtasksofcompilingasource
languageintoatargetlanguageanddefinesinterfacesbetweenthecomponentsre-
alizingthesubtasks. Theconcretearchitectureofthecompileristhenderivedfrom
this conceptual structure. Several components might be combined if the realized
subtasksallowthis. Ontheotherhand,acomponentmayalsobesplitintoseveral
componentsiftherealizedsubtaskisverycomplex.
Afirstattempttostructureacompilerdecomposesthecompilerintothreecom-
ponentsexecutingthreeconsecutivephases:
1. Theanalysisphase,realizedbythefront-end. Itdeterminesthesyntacticstruc-
ture of the source program and checks whether the static semantic constraints
aresatisfied. Thelattercontainthetypeconstraintsinlanguageswithstatictype
systems.
2. Theoptimizationandtransformationphase,performedbythemiddle-end. The
syntactically analyzed and semantically checked program is transformed by
semantics-preservingtransformations.Thesetransformationsmostlyaimatim-
provingtheefficiencyoftheprogrambyreducingtheexecutiontime,themem-
ory consumption, or the consumed energy. These transformations are inde-
pendent of the target architecture and mostly also independent of the source
language.
3. Thecodegenerationandthemachine-dependentoptimizationphase,performed
by the back-end. The program is now translated into an equivalent program
R.Wilhelm,H.Seidl,S.Hack,CompilerDesign,DOI10.1007/978-3-642-17540-4_1, 1
(cid:2)c Springer-VerlagBerlinHeidelberg2013