| | |
Lexical Analyzer
Please support our C++ advertiser: Intel Parallel Studio Home
![]() |
Well, let's remember C language (it's almost so called expression language) lexical grammar...
Here it is expression part:
Add some punctuators (~5-6) and recognize some identifiers as keywords - and you have full C language scanner...
Here it is expression part:
•
•
•
•
C language expression tokens::
Identifiers:: all but sizeof
<letter|_>[<letter>|_|<digit>]...
----------------------------------
Operators and punctuators:: one of
[ ] ( )
:
? .
+ - * / % ^ & | ~
! = < > += -= *= /= %=
^= &= |= << >> >>= <<= == !=
<= >= && || ++ -- , ->
sizeof
-----------------
Literals:: one of
integer-literal
character-literal
floating-literal
string-literal
-----------------
Whitespaces:: one of
blanks
horizontal tab
vertical tab
newlines
formfeed
comment
-------
This very minute? It's impossible. The scanner for C expressions is a program with ~1000 lines of source code (of course, it's a rough estimate). Probably, it's possible to write more compact (and fast) scanner with special methods (based on automata theory, for example).
Better start from a simple grammar. For example, consider simplest arithmetical expressions:
Tokens:: one of
identifier
decimal-integer-literal
Operators:: one of
+ - / *
Punctuators:: one of
( )
Try to implement a scanner for this lexical grammar.
Input: source text string (or char array)
Output: next token code
Better start from a simple grammar. For example, consider simplest arithmetical expressions:
Tokens:: one of
identifier
decimal-integer-literal
Operators:: one of
+ - / *
Punctuators:: one of
( )
Try to implement a scanner for this lexical grammar.
Input: source text string (or char array)
Output: next token code
Follow Salem's advice!
I have wrote a scanner for simple C expressions (~300 LOCs) but I will post the source (may be
) only after you will try to do it yourself.
What's a bliss to write parsers for automat grammatics!..
Some remark: fortunately, a scanner does not know operator precedence (let syntax parser knows).
I have wrote a scanner for simple C expressions (~300 LOCs) but I will post the source (may be
) only after you will try to do it yourself.What's a bliss to write parsers for automat grammatics!..
Some remark: fortunately, a scanner does not know operator precedence (let syntax parser knows).
![]() |
Similar Threads
- Help with lexical analyzer program (C++)
- Why are there so many languages? (Computer Science)
- what is the best compiler (Computer Science)
Other Threads in the C++ Forum
- Previous Thread: Segmentation fault during execution of code.
- Next Thread: Just a simple question
| Thread Tools | Search this Thread |
api array arrays beginner binary bitmap c++ c/c++ calculator char char* class classes coding compile compiler console conversion convert count data database delete desktop developer directshow dll dynamiccharacterarray email encryption error file forms fstream function functions game generator getline google graph homeworkhelper iamthwee ifstream input int integer java lib linkedlist linux list loop looping loops map math matrix memory multiple news node number numbertoword output parameter pointer problem program programming project proxy python random read recursion recursive reference return rpg sorting string strings struct template templates test text tree unix url vector video visualstudio win32 windows winsock word wordfrequency wxwidgets






