7

I am looking for a SQL Library that will parse an SQL statement and return some sort of Object representation of the SQL statement. My main objective is actually to be able to parse the SQL statement and retrieve the list of table names present in the SQL statement (including subqueries, joins and unions).

I am looking for a free library with a license business friendly (e.g. Apache license). I am looking for a library and not for an SQL Grammar. I do not want to build my own parser.

The best I could find so far was JSQLParser, and the example they give is actually pretty close to what I am looking for. However it fails parsing too many good queries (DB2 Database) and I'm hoping to find a more reliable library.

4 Answers 4

6

I doubt you'll find anything prewritten that you can just use. The problem is that ISO/ANSI SQL is a very complicated grammar — something like more than 600 production rules IIRC.

Terence Parr's ANTLR parser generator (Java, but can generate parsers in any one of a number of target languages) has several SQL grammars available, including a couple for PL/SQL, one for a SQL Server SELECT statement, one for mySQL, and one for ISO SQL.

No idea how complete/correct/up-to-date they are.

http://www.antlr.org/grammar/list

Sign up to request clarification or add additional context in comments.

6 Comments

I don't actually need a complete Object representation of the SQL statement, what I really need is to extract the names of the tables present in the SQL statement. So, as long as I can somehow get that from an existing library and as long as it correctly parses most queries, I would be happy. Writing my own parser right now is not an optin as I don't have the time for that :/
your not gonna be writing your own parser. your gonna be using a preexisting one. YACC is another option.
antlr will let you write a code snippet to a java file every time it encounters a table element in the query. you will use this generated java file to parse the sql, and if you wrote the code snippet and header / footer correctly in antlr, then you will just have a list of tables.
i dont see how you can get a list of table names without a complete representation. if your queries are really simplistic, then maybe you can just fake it with regular expressions.
what I meant was that I don't need a very flexible library. As long as it would understand most of my queries (DB2) and would give me the names of the tables present in the query, I would be happy.
|
5

You needn't reinvent the wheel, there is already such a reliable SQL parser library there, (it's commerical, not free), and this article shows how to retrieve the list of table names present in the SQL statement (including subqueries, joins and unions) that is exactly what you are looking for.

http://www.dpriver.com/blog/list-of-demos-illustrate-how-to-use-general-sql-parser/get-columns-and-tables-in-sql-script/

This SQL parser library supports Oracle, SQL Server, DB2, MySQL, Teradata and ACCESS.

1 Comment

This does exactly what I am looking for. However, as I said in my question, I can only consider free libraries (although I'm under the impression that's not going to happen :/) as this is just a prototype. Nonetheless, thank you! It's good to know that at least there's a good commercial library that does exactly what I need, and maybe later I would be allowed to use it.
5

You need the ultra light, ultra fast library to extract table names from SQL (Disclaimer: I am the owner)

Just add the following in your pom

<dependency>
 <groupId>com.github.mnadeem</groupId>
 <artifactId>sql-table-name-parser</artifactId>
 <version>0.0.1</version>

And do the following

new TableNameParser(sql).tables()

For more details, refer the project

Comments

0

Old question, but I think this project contains what you need:

Data Tools Project - SQL Development Tools

Here's the documentation for the SQL Query Parser.

Also, here's a small sample program. I'm no Java programmer so use with care.

package org.lala;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.charset.Charset;
import java.util.Iterator;
import java.util.List;

import org.eclipse.datatools.modelbase.sql.query.QuerySelectStatement;
import org.eclipse.datatools.modelbase.sql.query.QueryStatement;
import org.eclipse.datatools.modelbase.sql.query.TableReference;
import org.eclipse.datatools.modelbase.sql.query.ValueExpressionColumn;
import org.eclipse.datatools.modelbase.sql.query.helper.StatementHelper;
import org.eclipse.datatools.sqltools.parsers.sql.SQLParseErrorInfo;
import org.eclipse.datatools.sqltools.parsers.sql.SQLParserException;
import org.eclipse.datatools.sqltools.parsers.sql.SQLParserInternalException;
import org.eclipse.datatools.sqltools.parsers.sql.query.SQLQueryParseResult;
import org.eclipse.datatools.sqltools.parsers.sql.query.SQLQueryParserManager;
import org.eclipse.datatools.sqltools.parsers.sql.query.SQLQueryParserManagerProvider;

public class SQLTest {

    private static String readFile(String path) throws IOException {
        FileInputStream stream = new FileInputStream(new File(path));
        try {
            FileChannel fc = stream.getChannel();
            MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0,
                    fc.size());
            /* Instead of using default, pass in a decoder. */
            return Charset.defaultCharset().decode(bb).toString();
        } finally {
            stream.close();
        }
    }

    /**
     * @param args
     * @throws IOException
     */
    public static void main(String[] args) throws IOException {
        try {
            // Create an instance the Parser Manager
            // SQLQueryParserManagerProvider.getInstance().getParserManager
            // returns the best compliant SQLQueryParserManager
            // supporting the SQL dialect of the database described by the given
            // database product information. In the code below null is passed
            // for both the database and version
            // in which case a generic parser is returned

            SQLQueryParserManager parserManager = SQLQueryParserManagerProvider
                    .getInstance().getParserManager("DB2 UDB", "v9.1");
            // Sample query
            String sql = readFile("c:\\test.sql");
            // Parse
            SQLQueryParseResult parseResult = parserManager.parseQuery(sql);
            // Get the Query Model object from the result
            QueryStatement resultObject = parseResult.getQueryStatement();
            // Get the SQL text
            String parsedSQL = resultObject.getSQL();
            System.out.println(parsedSQL);

            // Here we have the SQL code parsed!
            QuerySelectStatement querySelect = (QuerySelectStatement) parseResult
                    .getSQLStatement();
            List columnExprList = StatementHelper
                    .getEffectiveResultColumns(querySelect);
            Iterator columnIt = columnExprList.iterator();
            while (columnIt.hasNext()) {
                ValueExpressionColumn colExpr = (ValueExpressionColumn) columnIt
                        .next();
                // DataType dataType = colExpr.getDataType();
                System.out.println("effective result column: "
                        + colExpr.getName());// + " with data type: " +
                                                // dataType.getName());
            }
            List tableList = StatementHelper.getTablesForStatement(resultObject);
            // List tableList = StatementHelper.getTablesForStatement(querySelect);
            for (Object obj : tableList) {
                TableReference t = (TableReference) obj;
                System.out.println(t.getName());
            }
        } catch (SQLParserException spe) {
            // handle the syntax error
            System.out.println(spe.getMessage());
            @SuppressWarnings("unchecked")
            List<SQLParseErrorInfo> syntacticErrors = spe.getErrorInfoList();
            Iterator<SQLParseErrorInfo> itr = syntacticErrors.iterator();
            while (itr.hasNext()) {
                SQLParseErrorInfo errorInfo = (SQLParseErrorInfo) itr.next();
                // Example usage of the SQLParseErrorInfo object
                // the error message
                String errorMessage = errorInfo.getParserErrorMessage();
                String expectedText = errorInfo.getExpectedText();
                String errorSourceText = errorInfo.getErrorSourceText();
                // the line numbers of error
                int errorLine = errorInfo.getLineNumberStart();
                int errorColumn = errorInfo.getColumnNumberStart();
                System.err.println("Error in line " + errorLine + ", column "
                        + errorColumn + ": " + expectedText + " "
                        + errorMessage + " " + errorSourceText);
            }
        } catch (SQLParserInternalException spie) {
            // handle the exception
            System.out.println(spie.getMessage());
        }
        System.exit(0);
    }
}

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.