About tokens and lexemes

Ben Scholzen (Oct 21, 2009)
Talk at ZendCon 2009 Uncon (English - US)

Rating: 4 of 5

A Parser is an integral part when building a Domain Specific Language or file format parser, such as our example usage case: the Ical format. This session will cover the general concept about tokenizing and parsing into a datastructure, as well as going into depth about how to keep the memory footprint and runtime low with the help of a stream-tokenizer.

 
Comments closed.

Comments

Rating: 4 of 5

Sep 1, 2009, 10:12 by Anonymous

a big +1!

Rating: 3 of 5

Sep 21, 2009, 19:46 by Anonymous

Ehh, not so much, but good luck. - Supavisah

Rating: 4 of 5

Sep 24, 2009, 14:37 by Anonymous

Not something that's often talked out

Rating: 5 of 5

Sep 28, 2009, 15:52 by guilhermeblanco

Lexical analysis/parsing is a rare subject covered in conferences, but it is an amazing area that most applications take advantage in any way. Good examples are wiki format and markdown.

If it could be turned into a more generic solution, it'll be an amazing talk!

Rating: 4 of 5

Oct 9, 2009, 19:18 by bjori

So, thats new. Not a topic that gets a lot of attention from "the hippies". - I like it!

Rating: 5 of 5

Oct 9, 2009, 21:19 by dcousineau

Definitely would like a talk on this. For some reason I readily understand LL(k) parsers (loved my finite automata course) however for some reason have a mental block around tokenizers (or I'm thinking of them the right way and my lazy side is rebelling). I'd love to see some code-by-example or other examples of a simple system working.

Rating: 5 of 5

Oct 12, 2009, 14:45 by Anonymous

Sounds interesting.

Rating: 4 of 5

Oct 12, 2009, 16:40 by Anonymous

Sounds good to me.

Rating: 4 of 5

Oct 14, 2009, 13:13 by mwillbanks

+1

Rating: 5 of 5

Oct 17, 2009, 19:15 by Anonymous

+1

© Joind.in 2010