Python: How to aggregate values in DataFrame

Question

I have a dataframe with below values

Text                                State
This is a beutiful day              California
But I am stuck with code            New York
It's too hard                       California
Can somebody please help me         Florida
I am new to python                  Florida
How should I solve this problem     New York
Day is turning bad                  New York
I am getting exhaused               California
Need some help                      New York

I need to calculate sentiment analysis on the text state wise. How may I aggregate the text state wise and then do sentiment analysis.

df = df.groupby(df.columns.difference(['Text']))
result = pd.DataFrame(df['State'].unique(), columns=df.columns)

I am trying to do it this way to aggregate text first but its not working. Need some suggestions for aggregation and then how to perform sentiment analysis using loop in the dataframe.

sidm · Accepted Answer · 2017-10-16 04:29:12Z

1

You can group the text using lambda function which takes the text and joins using the delimiter provided.

`delimiter = ' '
df2 =  df.groupby('State')['Text'].apply(lambda x: "%s" % delimiter.join(x)).reset_index()
print (df2)`

Adding reset.index() would convert into your required dataframe

edited Oct 16, 2017 at 4:29

answered Oct 16, 2017 at 1:25

sidm

961 silver badge4 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

cs95 Over a year ago

apply(delimiter.join) is enough.

Alex Over a year ago

Thanks, it worked. Could you also suggeest how should I do sentiment analysis on each row of the 'df2' dataframe now.

Alex Over a year ago

and how to preprocess and lemmitize the text column to remove all the stopwords, punctuation and digits

Collectives™ on Stack Overflow

Python: How to aggregate values in DataFrame

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related