Check if ENTIRE pandas object column is a string

Question

How can I check if a column is a string, or another type (e.g. int or float), even though the dtype is object?

(Ideally I want this operation vectorised, and not applymap checking every row...)

import io
# American post code
df1_str = """id,postal
1,12345
2,90210
3,"""
df1 = pd.read_csv(io.StringIO(df1_str))
df1["postal"] = df1["postal"].astype("O")  # is an object (of type float due to the null row 3)

# British post codes
df2_str = """id,postal
1,EC1
2,SE1
3,W2"""
df2 = pd.read_csv(io.StringIO(df2_str))
df2["postal"] = df2["postal"].astype("O")  # is an object (of type string)

Both df1 and df2 return object when doing df["postal"].dtype

However, df2 has .str methods, e.g. df2["postal"].str.lower(), but df1 doesn't.
Similarly, df1 can have mathematical operations done to it, e.g. df1 * 2

This is different to other SO questions. who ask if there are strings inside a column (and not the WHOLE column). e.g:

Try: datascience.stackexchange.com/questions/60955/… However, I usually use pd.to_numeric without huge performance issues — David Erickson
– David Erickson, Commented Jan 17, 2021 at 1:52
It would be cool if the pandas devs included an optional argument in df.info() for this in the future. It could show the value counts of each data type within each column and again would be an optional argument. — David Erickson
– David Erickson, Commented Jan 17, 2021 at 2:26

Pablo C · Accepted Answer · 2021-01-17 02:13:57Z

7

You can use pandas.api.types.infer_dtype:

>>> pd.api.types.infer_dtype(df2["postal"])
'string'
>>> pd.api.types.infer_dtype(df1["postal"])
'floating'

From the docs:

Efficiently infer the type of a passed val, or list-like array of values. Return a string describing the type.

edited Jan 17, 2021 at 2:13

answered Jan 17, 2021 at 2:07

Pablo C

4,7612 gold badges10 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Check if ENTIRE pandas object column is a string

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related