Composing scala parser combinators at runtime

Question

I've seen several other SO posts relevant here, suggesting that inheritance using traits is the only way out of this problem, but I can't see how to use that here.

I'm writing an assembler which has a directive that allows you to change the model of processor, thereby affecting which set of opcodes are parsable. I have two parser classes, one handling all the directive keywords, and one handling the instructions. (There will be more, for the different processor models). When the 'cpu' directive is parsed, that chooses the appropriate instruction parser. Here's a very cut-down illustration:

import scala.util.parsing.combinator.JavaTokenParsers

class ComposingParser {

    sealed abstract class Statement {
    }
    case class Dinst(value: String) extends Statement
    case class Keyword(value: String) extends Statement

    class InstructionParser extends JavaTokenParsers {
        def directOpcode: Parser[Statement] = j | ldlp | pfix

        private def j: Parser[Dinst]     = """(?i)J""".r     ^^ ( x => Dinst(x.toUpperCase) )
        private def ldlp: Parser[Dinst]  = """(?i)LDLP""".r  ^^ ( x => Dinst(x.toUpperCase) )
        private def pfix: Parser[Dinst]  = """(?i)PFIX""".r  ^^ ( x => Dinst(x.toUpperCase) )
    }

    class KeywordParser extends JavaTokenParsers {
        def program: Parser[Statement] = keys | instruction // the main, top-level parser

        def keys: Parser[Keyword] = start | end

        private def start: Parser[Keyword]  = """(?i)START""".r  ^^ ( x => Keyword(x.toUpperCase) )
        private def end: Parser[Keyword]  = """(?i)END""".r  ^^ ( x => Keyword(x.toUpperCase) )

        private def instruction: Parser[Statement] = {
            val ip = new InstructionParser // will be dynamically instantiating different parsers for different instruction sets so can't use traits
            ip.directOpcode
            // Error:(46, 16) type mismatch;
            // found   : ip.Parser[ComposingParser.this.Statement]
            // required: KeywordParser.this.Parser[ComposingParser.this.Statement]
            // ip.directOpcode
        }
        // I can't use traits as in https://stackoverflow.com/questions/2650254/scala-how-to-combine-parser-combinators-from-different-objects
        // I can't see how to apply the solution from https://stackoverflow.com/questions/40166258/reuse-parser-within-another-parser-with-scala-parser-combinators
        // Is it possible to convert an 'ip.Parser[Statement]' into a 'KeywordParser.this.Parser[Statement]' ?
    }
}

directOpcode and instruction both return Parser[Statement], why can't I combine them like this? Can self-type annotations help here? Thanks in advance for any assistance you can provide... (or illustrations showing how the solutions posted in the other SO posts cited might help).

Why exactly can you not use traits?

Andrey Tyukin
– Andrey Tyukin

2018-07-05 21:51:36 +00:00
Commented Jul 5, 2018 at 21:51 — Andrey Tyukin
– Andrey Tyukin, Commented Jul 5, 2018 at 21:51

Andrey Tyukin · Accepted Answer · 2018-07-05 21:55:29Z

1

From the question, it's not obvious why you "cannot use traits". At least your concrete example does work with traits just fine:

// uses `$ivy`-imports, either run with Ammonite, or remove the import and
// compile using SBT.
import $ivy.`org.scala-lang.modules:scala-parser-combinators_2.12:1.1.1`
import scala.util.parsing.combinator.JavaTokenParsers

class ComposingParser {

    sealed abstract class Statement {
    }
    case class Dinst(value: String) extends Statement
    case class Keyword(value: String) extends Statement

    trait InstructionParser extends JavaTokenParsers {
        def directOpcode: Parser[Statement] = j | ldlp | pfix

        private def j: Parser[Dinst]     = """(?i)J""".r     ^^ ( x => Dinst(x.toUpperCase) )
        private def ldlp: Parser[Dinst]  = """(?i)LDLP""".r  ^^ ( x => Dinst(x.toUpperCase) )
        private def pfix: Parser[Dinst]  = """(?i)PFIX""".r  ^^ ( x => Dinst(x.toUpperCase) )
    }

    trait InstructionParser2 extends JavaTokenParsers {
        def directOpcode2: Parser[Statement] = j | ldlp | pfix

        private def j: Parser[Dinst]     = """(?i)J""".r     ^^ ( x => Dinst(x.toUpperCase) )
        private def ldlp: Parser[Dinst]  = """(?i)LDLP""".r  ^^ ( x => Dinst(x.toUpperCase) )
        private def pfix: Parser[Dinst]  = """(?i)PFIX""".r  ^^ ( x => Dinst(x.toUpperCase) )
    }

    class KeywordParser extends JavaTokenParsers 
    with InstructionParser 
    with InstructionParser2 {
        def program: Parser[Statement] = keys | instruction // the main, top-level parser

        def keys: Parser[Keyword] = start | end

        private def start: Parser[Keyword]  = """(?i)START""".r  ^^ ( x => Keyword(x.toUpperCase) )
        private def end: Parser[Keyword]  = """(?i)END""".r  ^^ ( x => Keyword(x.toUpperCase) )

        private def instruction: Parser[Statement] = {
            if (math.random < 0.5) directOpcode else directOpcode2 
        }
    }
}

answered Jul 5, 2018 at 21:55

Andrey Tyukin

45.1k4 gold badges65 silver badges103 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Matt Gumbley Over a year ago

Thank you Andrey, your example has shown me exactly how to solve this - you must use traits. I couldn't, because my parser classes had some extra state passed in via constructor arguments - I forgot this in my example above, and have now refactored the real code to place this state in another trait. Once all instruction parsers are traits, and all mixed into the main parser class, everything works. I didn't realise that there's more to the hierarchy of Parser[Statement] than meets the eye - is it a path-dependent type? Many thanks!

Andrey Tyukin Over a year ago

@MattGumbley Yes, it's path dependent. In your code, the compiler can only infer that the type of ip.directOpCode is ip.Parser[Statement], and this depends on value ip.

Collectives™ on Stack Overflow

Composing scala parser combinators at runtime

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related