Yes I agree, my code includes a good tokenizer, not a simple word splitter.

		JPLeRouzic 28 days ago \| parent \| context \| favorite \| on: Ask HN: How are Markov chains so different from ti... Yes I agree, my code includes a good tokenizer, not a simple word splitter.