How to count word frequency from a Pandas Dataframe- Python

Question

I currently created a Pandas Dataframe from a dictionary. The Dataframe looks something like:

      URL         TITLE
0   /xxxx.xx   Hi this is word count
1   /xxxx.xx   Hi this is Stack Overflow
2   /xxxx.xx   Stack Overflow Questions

I want to add a new column to this table, which lists the number of frequency the word "Stack Overflow" appears. So for example, it would be like:

      URL         TITLE                          COUNT
0   /xxxx.xx   Hi this is word count               0
1   /xxxx.xx   Hi this is Stack Overflow           1
2   /xxxx.xx   Stack Overflow Questions            1

The count function does not seem to work for dictionaries, but only for strings. Would there bean easy way to do this?

Can you show the dictionary and the code for creating the table — Moses Koledoye
– Moses Koledoye, Commented Mar 29, 2017 at 7:48
"Stack Overflow" is not a word, it is two words. What if the string contains "Overflow Stack" or "Notstack Overflow"? — tobias_k
– tobias_k, Commented Mar 29, 2017 at 7:55
@Jan Yes it is a dictionary that I converted into a pandas dataframe. I will try to further clarify the question. — Elizabeth
– Elizabeth, Commented Mar 29, 2017 at 7:58

Jan · Accepted Answer · 2017-03-29 08:02:32Z

3

Assuming this is actually a pandas dataframe, you could do:

import pandas as pd

table = {   'URL': ['/xxxx.xx', '/xxxx.xx', '/xxxx.xx'], 
            'TITLE': ['Hi this is word count', 'Hi this is Stack Overflow', 'Stack Overflow Questions']}

df = pd.DataFrame(table)
df['COUNT'] = df.TITLE.str.count('Stack Overflow')
print(df)

This yields:

                       TITLE       URL  COUNT
0      Hi this is word count  /xxxx.xx      0
1  Hi this is Stack Overflow  /xxxx.xx      1
2   Stack Overflow Questions  /xxxx.xx      1

answered Mar 29, 2017 at 8:02

Jan

43.3k11 gold badges57 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Raymond Hettinger · Accepted Answer · 2017-03-29 15:59:24Z

1

The count() method on dataframes is good at counting occurrences of a single values such as "Stack Overflow".

To do frequency analysis of multiple values, consider using collection.Counter(data) and its .most_common(k) method.

answered Mar 29, 2017 at 15:59

Raymond Hettinger

229k67 gold badges405 silver badges504 bronze badges

Collectives™ on Stack Overflow

How to count word frequency from a Pandas Dataframe- Python

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related