class Prism::Translation::Ripper

This class provides a compatibility layer between prism and Ripper. It functions by parsing the entire tree first and then walking it and executing each of the Ripper callbacks as it goes. To use this class, you treat Prism::Translation::Ripper effectively as you would treat the Ripper class.

Note that this class will serve the most common use cases, but Ripper’s API is extensive and undocumented. It relies on reporting the state of the parser at any given time. We do our best to replicate that here, but because it is a different architecture it is not possible to perfectly replicate the behavior of Ripper.

The main known difference is that we may omit dispatching some events in some cases. This impacts the following events:

on_assign_error
on_comma
on_ignored_nl
on_ignored_sp
on_kw
on_label_end
on_lbrace
on_lbracket
on_lparen
on_nl
on_op
on_operator_ambiguous
on_rbrace
on_rbracket
on_rparen
on_semicolon
on_sp
on_symbeg
on_tstring_beg
on_tstring_end

Constants

BINARY_OPERATORS: A list of all of the Ruby binary operators.
EVENTS: This array contains name of all ripper events.
KEYWORDS: A list of all of the Ruby keywords.
PARSER_EVENTS: This array contains name of parser events.
PARSER_EVENT_TABLE: This contains a table of all of the parser events and their corresponding arity.
SCANNER_EVENTS: This array contains name of scanner events.
SCANNER_EVENT_TABLE: This contains a table of all of the scanner events and their corresponding arity.

Attributes

column [R]

The current column in bytes of the parser.

filename [R]

The filename of the source being parsed.

lineno [R]

The current line number of the parser.

source [R]

The source that is being parsed.

Public Class Methods

lex (src, filename = "-", lineno = 1, raise_errors: false)

Source

# File lib/prism/translation/ripper.rb, line 71
def self.lex(src, filename = "-", lineno = 1, raise_errors: false)
  result = Prism.lex_compat(src, filepath: filename, line: lineno, version: "current")

  if result.failure? && raise_errors
    raise SyntaxError, result.errors.first.message
  else
    result.value
  end
end

Tokenizes the Ruby program and returns an array of an array, which is formatted like [[lineno, column], type, token, state]. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, use the raise_errors keyword to raise a SyntaxError for an error in src.

require "ripper"
require "pp"

pp Ripper.lex("def m(a) nil end")
#=> [[[1,  0], :on_kw,     "def", FNAME    ],
     [[1,  3], :on_sp,     " ",   FNAME    ],
     [[1,  4], :on_ident,  "m",   ENDFN    ],
     [[1,  5], :on_lparen, "(",   BEG|LABEL],
     [[1,  6], :on_ident,  "a",   ARG      ],
     [[1,  7], :on_rparen, ")",   ENDFN    ],
     [[1,  8], :on_sp,     " ",   BEG      ],
     [[1,  9], :on_kw,     "nil", END      ],
     [[1, 12], :on_sp,     " ",   END      ],
     [[1, 13], :on_kw,     "end", END      ]]

new (source, filename = "(ripper)", lineno = 1)

Source

# File lib/prism/translation/ripper.rb, line 482
def initialize(source, filename = "(ripper)", lineno = 1)
  if source.is_a?(IO)
    @source = source.read
  elsif source.respond_to?(:gets)
    @source = +""
    while line = source.gets
      @source << line
    end
  else
    @source = source.to_str
  end

  @filename = filename
  @lineno = lineno
  @column = 0
  @result = nil
end

Create a new Translation::Ripper object with the given source.

parse (src, filename = "(ripper)", lineno = 1)

Source

# File lib/prism/translation/ripper.rb, line 45
def self.parse(src, filename = "(ripper)", lineno = 1)
  new(src, filename, lineno).parse
end

Parses the given Ruby program read from src. src must be a String or an IO or a object with a gets method.

sexp (src, filename = "-", lineno = 1, raise_errors: false)

Source

# File lib/prism/translation/ripper.rb, line 393
def self.sexp(src, filename = "-", lineno = 1, raise_errors: false)
  builder = SexpBuilderPP.new(src, filename, lineno)
  sexp = builder.parse
  if builder.error?
    if raise_errors
      raise SyntaxError, builder.error
    end
  else
    sexp
  end
end

Parses src and create S-exp tree. Returns more readable tree rather than Ripper.sexp_raw. This method is mainly for developer use. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, returning nil in such cases. Use the raise_errors keyword to raise a SyntaxError for an error in src.

require "ripper"
require "pp"

pp Ripper.sexp("def m(a) nil end")
  #=> [:program,
       [[:def,
        [:@ident, "m", [1, 4]],
        [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil, nil, nil, nil]],
        [:bodystmt, [[:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]

sexp_raw (src, filename = "-", lineno = 1, raise_errors: false)

Source

# File lib/prism/translation/ripper.rb, line 428
def self.sexp_raw(src, filename = "-", lineno = 1, raise_errors: false)
  builder = SexpBuilder.new(src, filename, lineno)
  sexp = builder.parse
  if builder.error?
    if raise_errors
      raise SyntaxError, builder.error
    end
  else
    sexp
  end
end

Parses src and create S-exp tree. This method is mainly for developer use. The filename argument is mostly ignored. By default, this method does not handle syntax errors in src, returning nil in such cases. Use the raise_errors keyword to raise a SyntaxError for an error in src.

require "ripper"
require "pp"

pp Ripper.sexp_raw("def m(a) nil end")
  #=> [:program,
       [:stmts_add,
        [:stmts_new],
        [:def,
         [:@ident, "m", [1, 4]],
         [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil]],
         [:bodystmt,
          [:stmts_add, [:stmts_new], [:var_ref, [:@kw, "nil", [1, 9]]]],
          nil,
          nil,
          nil]]]]

tokenize (...)

Source

# File lib/prism/translation/ripper.rb, line 90
def self.tokenize(...)
  lex(...).map { |token| token[2] }
end

Tokenizes the Ruby program and returns an array of strings. The filename and lineno arguments are mostly ignored, since the return value is just the tokenized input. By default, this method does not handle syntax errors in src, use the raise_errors keyword to raise a SyntaxError for an error in src.

p Ripper.tokenize(“def m(a) nil end”) # => [“def”, “ ”, “m”, “(”, “a”, “)”, “ ”, “nil”, “ ”, “end”]

Public Instance Methods

error? ()

Source

# File lib/prism/translation/ripper.rb, line 505
def error?
  result.failure?
end

True if the parser encountered an error during parsing.

parse ()

Source

# File lib/prism/translation/ripper.rb, line 510
def parse
  result.comments.each do |comment|
    location = comment.location
    bounds(location)

    if comment.is_a?(InlineComment)
      on_comment(comment.slice)
    else
      offset = location.start_offset
      lines = comment.slice.lines

      lines.each_with_index do |line, index|
        bounds(location.copy(start_offset: offset))

        if index == 0
          on_embdoc_beg(line)
        elsif index == lines.size - 1
          on_embdoc_end(line)
        else
          on_embdoc(line)
        end

        offset += line.bytesize
      end
    end
  end

  result.magic_comments.each do |magic_comment|
    on_magic_comment(magic_comment.key, magic_comment.value)
  end

  unless result.data_loc.nil?
    on___end__(result.data_loc.slice.each_line.first)
  end

  result.warnings.each do |warning|
    bounds(warning.location)

    if warning.level == :default
      warning(warning.message)
    else
      case warning.type
      when :ambiguous_first_argument_plus
        on_arg_ambiguous("+")
      when :ambiguous_first_argument_minus
        on_arg_ambiguous("-")
      when :ambiguous_slash
        on_arg_ambiguous("/")
      else
        warn(warning.message)
      end
    end
  end

  if error?
    result.errors.each do |error|
      location = error.location
      bounds(location)

      case error.type
      when :alias_argument
        on_alias_error("can't make alias for the number variables", location.slice)
      when :argument_formal_class
        on_param_error("formal argument cannot be a class variable", location.slice)
      when :argument_format_constant
        on_param_error("formal argument cannot be a constant", location.slice)
      when :argument_formal_global
        on_param_error("formal argument cannot be a global variable", location.slice)
      when :argument_formal_ivar
        on_param_error("formal argument cannot be an instance variable", location.slice)
      when :class_name, :module_name
        on_class_name_error("class/module name must be CONSTANT", location.slice)
      else
        on_parse_error(error.message)
      end
    end

    nil
  else
    result.value.accept(self)
  end
end

Parse the source and return the result.

Private Instance Methods

compile_error (msg)

Source

# File lib/prism/translation/ripper.rb, line 3498
def compile_error(msg)
end

This method is called when the parser found syntax error.

dedent_string (string, width)

Source

# File lib/prism/translation/ripper.rb, line 3513
def dedent_string(string, width)
  whitespace = 0
  cursor = 0

  while cursor < string.length && string[cursor].match?(/\s/) && whitespace < width
    if string[cursor] == "\t"
      whitespace = ((whitespace / 8 + 1) * 8)
      break if whitespace > width
    else
      whitespace += 1
    end

    cursor += 1
  end

  string.replace(string[cursor..])
  cursor
end

This method is provided by the Ripper C extension. It is called when a string needs to be dedented because of a tilde heredoc. It is expected that it will modify the string in place and return the number of bytes that were removed.

warn (fmt, *args)

Source

# File lib/prism/translation/ripper.rb, line 3489
def warn(fmt, *args)
end

This method is called when weak warning is produced by the parser. fmt and args is printf style.

warning (fmt, *args)

Source

# File lib/prism/translation/ripper.rb, line 3494
def warning(fmt, *args)
end

This method is called when strong warning is produced by the parser. fmt and args is printf style.