Login Register

tokenize

tokenize(src, filename = '-', lineno = 1)

Class Public methods

Tokenizes the Ruby program and returns an array of strings.

p Ripper.tokenize("def m(a) nil end")
   # => ["def", " ", "m", "(", "a", ")", " ", "nil", " ", "end"]

Links:

https://github.com/ruby/ruby/blob/099f6676a1f896b1ab0eda1ba9cf3534952fe7d0/ext/ripper/lib/ripper/lexer.rb#L20

doc_ruby_on_rails

2015-05-08 14:06:01

Comments

Leave a Comment

Please login to continue.

Popular Articles

tokenize.tokenize() tokenize.tokenize(readline) The tokenize() generator requires one argument, readline, which must be a callable object which provides the same inte

token_dom_tokens token_dom_tokens(first, last=None) Instance Public methods Alias for: token_DOM

tokenize tokenize(str) Class Public methods Returns a token stream for str, for testing

Token class Token Represents a Token. Constants EOF_TYPE NAME_TYPE NUMBER_TYPE STRING_TYPE OPERATOR_TYPE PUNCTUATION_TYPE Pro

token token() Instance Public methods