Remove rows from pandas dataframe if string has 'only numbers'

Question

I am looking to remove all rows from the df that have ONLY numbers in the string

Here is an extract of the dataframe

                         qid    question_stemmed                                   target   question_length total_words
149952  1d53c9c017999b4f77e2    8430397824532987451912384179815150754023741609...   0              241              3

Is there a way i can do that?

I tried the below, but it will remove all rows that have numbers in the string (along with any other datatype). However, i am looking to see if i can remove all 'numeric ONLY' rows.

df['question_stemmed'] = df[df['question_stemmed'].str.contains(r'[^a-z]')]

Appreciate any help here

df = df[~df['question_stemmed'].str.match(r'\d+$')] or df = df[~df['question_stemmed'].str.match(r'[0-9]+$')] — Wiktor Stribiżew
– Wiktor Stribiżew, Commented Jul 15, 2020 at 7:07
Thanks @WiktorStribiżew i got a better solution for a larger problem from you on the other post :) — Shalin
– Shalin, Commented Jul 15, 2020 at 21:07

cs95 · Accepted Answer · 2020-07-15 02:08:39Z

5

If we're only worrying about ASCII digits 0-9:

df = df[~df['question_stemmed'].str.isdigit()]

If we need to worry about unicode or digits in other languages:

df = df[~df['question_stemmed'].str.isnumeric()]

Pandas methods internally call the corresponding python methods. See What's the difference between str.isdigit, isnumeric and isdecimal in python? for an explanation of how these functions work.

answered Jul 15, 2020 at 2:08

cs95

406k106 gold badges744 silver badges797 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Shalin Over a year ago

thanks , but for some strange, reason, it is assigning the value of qid to the question_stemmed column. it must be a very silly error. Trying to figure out what is causing this.. Any idea here?

cs95 Over a year ago

@Shalin this doesn't do any column assignment, so the issue is likely elsewhere in your code. Not sure as I can't see it...

Shalin Over a year ago

it was my error. I applied this df['question_stemmed'] = df[~df['question_stemmed'].str.isnumeric()] hence it was reassigning. works fine now. Thanks @cs95

Collectives™ on Stack Overflow

Remove rows from pandas dataframe if string has 'only numbers'

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related