0

I'm feeling a little dumb when it comes to the editor that we are using, and I was wondering if anyone knew of any good online tutorials for an introduction to tokenizers.

3
Contributors
3
Replies
5
Views
5 Years
Discussion Span
Last Post by Airshow
0

Yeah, not what I need. I'm trying to make all these changes to the tokenizer used by the editor and it's confusing me because I have no experience with how a tokenizer works. :(

2

Dani,

I looked at the code behind the editor a couple of months ago and have to agree, it ain't the easiest to follow. I felt it needed to be taken apart and put back together in a more organised fashion.

If you want to take a step back from the actual code and remind yourself of the concepts and teminology in this area, then Wikipedia's entry on Lexical Analysis seems pretty good (with the usual provisos for Wikis). Some of the links may also prove fruitful.

Edited by Airshow

Votes + Comments
Thanks
This topic has been dead for over six months. Start a new discussion instead.
Have something to contribute to this discussion? Please be thoughtful, detailed and courteous, and be sure to adhere to our posting rules.