0

Hi All,

Just starting my online C programming class and my textbook states "In a computer language, a token is the smallest unit of the language that has a unique meaning. Thus, the reserved words, programmer-created identifiers, and all special mathematical symbols, such as + and -, are considered tokens of the C language."

Since the text previously states that the three type of C identifiers consist of keywords/reserved words, standard identifiers and programmer-created identifiers, then I'm presuming that both keywords and standard identifiers are also tokens, correct? And that punctuation marks ("()", "{}", ";" etc.), even though they represent attributes of the language, that they are considered to be part of the grammatical structure and are outside the definition of tokens, per se?

Regards,
Paul

Edited by aeinstein: corrected misquote

3
Contributors
5
Replies
7
Views
6 Years
Discussion Span
Last Post by aeinstein
0

> are outside the definition of tokens
No. The Standard defines token as either of:
keyword
identifier
constant
string-literal
punctuator
and punctuator in turn is either of
[ ] ( ) { } . ->
++ -- & * + - ~ !
/ % << >> < > <= >= == != ^ | && ||
? : ; ...
= *= /= %= += -= <<= >>= &= ^= |=
, # ##
<: :> <% %> %: %:%:

0

TYVM for your reply nezachem. So the text is wrong in regards to mathematical operators being considered as tokens?

Paul

0

All operators (and thus all mathematics operators), are tokens. Your compiler help should have a list of them.

0

Thanks for your replies nezachem and Adak, very helpful and very much appreciated.

Paul

PS: My apologies nezachem, I totally missed the quoted part of your reply, which is why I asked for clarification.

Edited by aeinstein: n/a

This question has already been answered. Start a new discussion instead.
Have something to contribute to this discussion? Please be thoughtful, detailed and courteous, and be sure to adhere to our posting rules.