Exploding Array of Struct using HiveQL

Question

CREATE TABLE IF NOT EXISTS Table2
(
USER_ID BIGINT,
PURCHASED_ITEM ARRAY<STRUCT<PRODUCT_ID: BIGINT,TIMESTAMPS:STRING>>
) ROW FORMAT
 DELIMITED FIELDS TERMINATED BY '-'
 collection items terminated by ','
 map keys terminated by ':'
 LINES TERMINATED BY '\n'
 STORED AS TEXTFILE
 LOCATION '/user/rj/output2';

Below is the data in Table2

1345653-110909316904:1341894546,221065796761:1341887508

I can explode the above data by using this below query and it works fine for above data-

SELECT  * FROM (select user_id, prod_and_ts.product_id as product_id,
prod_and_ts.timestamps as timestamps FROM table2 LATERAL VIEW
explode(purchased_item) exploded_table as prod_and_ts) prod_and_ts;

And I will get output like this which is fine-

1345653                                110909316904     1341894546
1345653                                221065796761     1341887508

But in some cases I have data in the table below like this, multiple timestamp appended by pound sign for same product_id-

1345653-110909316904:1341894546#1341885695,221065796761:1341887508#1341885453

And I need output like this for above data using the HiveQL query-

1345653                                110909316904     1341894546
1345653                                110909316904    1341885695
1345653                                221065796761     1341887508
1345653                                221065796761    1341885453

Is this possible to do this somehow?

Any suggestions will be appreciated.?

P.S I ask this question few days back, but in that case data was different and now data is totally different and I need similar output.

user2265478 · Accepted Answer · 2013-09-26 12:52:42Z

4

You can use the function regexp_replace or regex_extract to get only the product id. Try this:

SELECT  * FROM (select user_id, prod_and_ts.product_id as product_id,
regex_replace(prod_and_ts.timestamps, "#\\d*", "")  as timestamps FROM table2 LATERAL VIEW
explode(purchased_item) exploded_table as prod_and_ts) prod_and_ts;

answered Sep 26, 2013 at 12:52

user2265478

1639 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Exploding Array of Struct using HiveQL

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related