Create a "temporary" table in spark-SQL, not spark-scala?

Question

I have loaded a table from an input file.

CREATE TABLE MyTable (
    ID INT,
    VALUE FLOAT,
    RATE  INT
...

LOAD DATA LOCAL INPATH 'MYPATH' INTO TABLE MyTable;

Now I'd like to create a new based on this one

DerivedTable = 
    SELECT ID, VALUE*RATE AS Total
    FROM MyTable
    WHERE VALUE IS NOT NULL;

Then I'm going to use this table as a source for other tables and for outputs.

What is a correct Sql (or Hive) way to create this "temporary" table? This should work in spark-sql?

PS: I know how to do that in spark-shell. But that is not what I'm looking for.

Alper t. Turker · Accepted Answer · 2017-12-07 00:42:20Z

6

You can:

CREATE TEMPORARY VIEW DerivedTable AS (
   SELECT ID, VALUE*RATE AS Total
   FROM MyTable
   WHERE VALUE IS NOT NULL);

answered Dec 7, 2017 at 0:42

Alper t. Turker

35.3k9 gold badges89 silver badges118 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1