Write A program that reads a statement of c++ langauge and prints a table of tokens and classes (keyword, identifier
Stuugie 50
What have you got so far?
this question from compiler design .
we need to know the keywords, identifiers, reserved words...etc in a written statement in programming language like c++ and type them in table
Stuugie 50
Sorry but I need to clarify because of the differences between your 1st and 2nd post. Are you supposed to write some code or just get the list of keywords?
Ancient Dragon 5,243
I need c++ source code to identify the keywords in any statement typed in the program
like :
for (int i = 0 ; i < 10 ; i++)
for , int : keywords
i : identifier
< , + : oprator
Schol-R-LEA 1,087
OK, so what you actually need is a lexical analyzer, also called a tokenizer from it's main mode of operation, which is to break a stream of characters into tokens. There are several discussions of this topic on DaniWeb, if you do a search on 'tokenizer'. You might want to check out this post, for example, as well as the StateMachine class in my own Suntiger Algol compiler.
import java.util.ArrayList;
import java.util.regex.*;
public class Lexer {
public static enum TokenType {
// Token types cannot have underscores
NUMBER("-?[0-9]+"), BINARYOP("[*|/|+|-]"), WHITESPACE("[ \t\f\r\n]+");
public final String pattern;
private TokenType(String pattern) {
this.pattern = pattern;
}
}
public static class Token {
public TokenType type;
public String data;
public Token(TokenType type, String data) {
this.type = type;
this.data = data;
}
@Override
public String toString() {
return String.format("(%s %s)", type.name(), data);
}
}
public static ArrayList<Token> lex(String input) {
// The tokens to return
ArrayList<Token> tokens = new ArrayList<Token>();
// Lexer logic begins here
StringBuffer tokenPatternsBuffer = new StringBuffer();
for (TokenType tokenType : TokenType.values())
tokenPatternsBuffer.append(String.format("|(?<%s>%s)", tokenType.name(), tokenType.pattern));
Pattern tokenPatterns = Pattern.compile(new String(tokenPatternsBuffer.substring(1)));
// Begin matching tokens
Matcher matcher = tokenPatterns.matcher(input);
while (matcher.find()) {
if (matcher.group(TokenType.NUMBER.name())!= null) {
tokens.add(new Token(TokenType.NUMBER, matcher.group(TokenType.NUMBER.name())));
continue;
}
else if (matcher.group(TokenType.BINARYOP.name()) != null) {
tokens.add(new Token(TokenType.BINARYOP, matcher.group(TokenType.BINARYOP.name())));
continue;
}
else if (matcher.group(TokenType.WHITESPACE.name()) != null)
continue;
}
return tokens;
}
public static void main(String[] args) {
String input = "11 + 22 - 33";
// Create tokens and print them
ArrayList<Token> tokens = lex(input);
for (int i = 0 ; i < tokens.size(); i++)
System.out.println(tokens.get(i));
}
}
I have an error in th following part and I can't solve th problem with this error , so if you can do it, help me.
while (matcher.find()) {
if (matcher.group(TokenType.NUMBER.name())!= null) {
tokens.add(new Token(TokenType.NUMBER, matcher.group(TokenType.NUMBER.name())));
continue;
}
else if (matcher.group(TokenType.BINARYOP.name()) != null) {
tokens.add(new Token(TokenType.BINARYOP, matcher.group(TokenType.BINARYOP.name())));
continue;
}
else if (matcher.group(TokenType.WHITESPACE.name()) != null)
continue;
}
Ancient Dragon 5,243
what is the error?
cannot recognize the method group(String)
Ancient Dragon 5,243
cannot recognize the method group(String)
Does matcher have such a method?
