Hive - regexp_replace function for multiple strings

Question

I am using hive 0.13! I want to find multiple tokens like "hip hop" and "rock music" in my data and replace them with "hiphop" and "rockmusic" - basically replace them without white space. I have used the regexp_replace function in hive. Below is my query and it works great for above 2 examples.

drop table vp_hiphop;
create table vp_hiphop as
select userid, ntext,
       regexp_replace(regexp_replace(ntext, 'hip hop', 'hiphop'), 'rock music', 'rockmusic') as ntext1
from  vp_nlp_protext_males
;

But I have 100 such bigrams/ngrams and want to be able to do replace efficiently where I just remove the whitespace. I can pattern match the phrase - hip hop and rock music but in the replace I want to simply trim the white spaces. Below is what I tried. I also tried using trim with regexp_replace but it wants the third argument in the regexp_replace function.

drop table vp_hiphop;
create table vp_hiphop as
select  userid, ntext,
        regexp_replace(ntext, '(hip hop)|(rock music)') as ntext1
from  vp_nlp_protext_males
;

Jeremy Beard · Accepted Answer · 2015-02-18 04:11:53Z

3

You can strip all occurrences of a substring from a string using the TRANSLATE function to replace the substring with the empty string. For your query it would become this:

drop table vp_hiphop;
create table vp_hiphop as
select  userid, ntext,
        translate(ntext, ' ', '') as ntext1
from  vp_nlp_protext_males
;

answered Feb 18, 2015 at 4:11

Jeremy Beard

2,7251 gold badge21 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Hive - regexp_replace function for multiple strings

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related