Using parser combinators to collate lines of text

Question

I'm trying to parse a text file using parser combinators. I want to capture the index and text in a class called Example. Here's a test showing the form on an input file:

object Test extends ParsComb with App {
  val input = """
0)
blah1
blah2
blah3
1)
blah4
blah5
END
"""
  println(parseAll(examples, input))
}

And here's my attempt that doesn't work:

import scala.util.parsing.combinator.RegexParsers

case class Example(index: Int, text: String)

class ParsComb extends RegexParsers {
  def examples: Parser[List[Example]] = rep(divider~example) ^^ 
                                          {_ map {case d ~ e => Example(d,e)}}
  def divider:  Parser[Int]           = "[0-9]+".r <~ ")"    ^^ (_.toInt)
  def example:  Parser[String]        = ".*".r <~ (divider | "END") 
}

It fails with:

[4.1] failure: `END' expected but `b' found

blah2

^

I'm just starting out with these so I don't have much clue what I'm doing. I think the problem could be with the ".*".r regex not doing multi-line. How can I change this so that it parses correctly?

Zane XY · Accepted Answer · 2012-06-20 13:17:05Z

5

What does the error message mean?

According to your grammar definition, ".*".r <~ (divider | "END"), you told to the parser that, an example should followed either by a divider or a END. After parsing blah1, the parser tried to find divider and failed, then tried END, failed again, there're no other options available, so the END here was the last alternative of the production value, so from the parser's perspective, it expected END, but it soon found, the next input was blah2 from the 4th line.

How to fix it?

Try to be close to your implementation, the grammar in your case should be:

examples ::= {divider example}
divider  ::= Integer")"
example  ::= {literal ["END"]}

and I think parsing "example" into List[String] makes more sense, anyway, it's up to you.

The problem is your example parser, it should be a repeatable literal.

So ,

class ParsComb extends RegexParsers {
  def examples: Parser[List[Example]] = rep(divider ~ example) ^^ { _ map { case d ~ e => Example(d, e) } }
  def divider: Parser[Int] = "[0-9]+".r <~ ")" ^^ (_.toInt)
  def example: Parser[List[String]] = rep("[\\w]*(?=[\\r\\n])".r <~ opt("END"))
}

the regex (?=[\\r\\n]) means it's a positive lookahead and would match characters that followed by \r or \n.

the parse result is:

[10.1] parsed: List(Example(0,List(blah1, blah2, blah3)), Example(1,List(blah4, blah5)))

If you want to parse it into a String(instead of List[String]), just add a transform function for example: ^^ {_ mkString "\n"}

edited Jun 20, 2012 at 13:17

answered Jun 20, 2012 at 10:46

Zane XY

2,8291 gold badge24 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Luigi Plinge Over a year ago

Thanks, great answer - please post more to StackOverflow!

senia · Accepted Answer · 2012-06-20 06:18:08Z

2

Your parser can't process newline character, your example parser eliminates next divider and your example regex matches divider and "END" string.

Try this:

object ParsComb extends RegexParsers { 
  def examples: Parser[List[Example]] = rep(divider~example) <~ """END\n?""".r ^^ {_ map {case d ~ e => Example(d,e)}} 
  def divider: Parser[Int] = "[0-9]+".r <~ ")\n" ^^ (_.toInt) 
  def example: Parser[String] = rep(str) ^^ {_.mkString}
  def str: Parser[String] = """.*\n""".r ^? { case s if simpleLine(s) => s}

  val div = """[0-9]+\)\n""".r
  def simpleLine(s: String) = s match {
    case div() => false
    case "END\n" => false
    case _ => true
  }

  def apply(s: String) = parseAll(examples, s)
}

Result:

scala> ParsComb(input)
res3: ParsComb.ParseResult[List[Example]] =
[10.1] parsed: List(Example(0,blah1
blah2
blah3
), Example(1,blah4
blah5
))

answered Jun 20, 2012 at 6:18

senia

38.1k6 gold badges93 silver badges134 bronze badges

1 Comment

Luigi Plinge Over a year ago

Thanks - I will have a play with this later to ensure I understand what's going on!

Daniel C. Sobral · Accepted Answer · 2012-06-25 13:42:50Z

1

I think the problem could be with the ".*".r regex not doing multi-line.

Exactly. Use the dotall modifier (strangely called "s"):

def example:  Parser[String]        = "(?s).*".r <~ (divider | "END")

edited Jun 25, 2012 at 13:42

answered Jun 20, 2012 at 17:07

Daniel C. Sobral

298k88 gold badges508 silver badges688 bronze badges

Collectives™ on Stack Overflow

Using parser combinators to collate lines of text

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related