--- !ruby/object:RI::ClassDescription 
attributes: []

class_methods: []

comment: 
- !ruby/struct:SM::Flow::P 
  body: A simple HTML tokenizer. It simply breaks a stream of text into tokens, where each token is a string. Each string represents either &quot;text&quot;, or an HTML element.
- !ruby/struct:SM::Flow::P 
  body: This currently assumes valid XHTML, which means no free &lt; or &gt; characters.
- !ruby/struct:SM::Flow::P 
  body: "Usage:"
- !ruby/struct:SM::Flow::VERB 
  body: "  tokenizer = HTML::Tokenizer.new(text)\n  while token = tokenizer.next\n    p token\n  end\n"
constants: []

full_name: HTML::Tokenizer
includes: []

instance_methods: []

name: Tokenizer
superclass: Object
