Pandas Dataframe Issue Converting Column dtype

Question

I have a simple pandas dataframe with a column:

col = [['A']]
data = [[1.0],[2.3],[3.4]]
df = pd.DataFrame.from_records(data, columns=col)

This creates a dataframe with one column of type np.float64, which is what I want.

Later in the process, I want to add another column of type string.

df['SOMETEXT'] = "SOME TEXT FOR ANALYSIS"

The dtype of this column is coming though as dtype of object, but I need it to be type string. So I do the following:

df['SOMETEXT'] = df['SOMETEXT'].astype(str)

If I look at the dtype again, I get the same dtype for that column: object.

I have a process down my workflow that is dtype sensitive and I need the column to be a string.

Any ideas?

array = df.to_records(index=False) # convert to numpy array

The dtypes on the array still carry the object dtype, but the columns should be a string.

Possible duplicate of How to convert column with dtype as object to string in Pandas Dataframe — languitar
– languitar, Commented Mar 14, 2017 at 12:38

chrisaycock · Accepted Answer · 2017-03-14 12:52:04Z

3

In pandas, all strings are object type. It confused me too when I first started.

Once in NumPy, you can cast the string:

In [24]: array['SOMETEXT'].astype(str)
Out[24]: 
array(['SOME TEXT FOR ANALYSIS', 'SOME TEXT FOR ANALYSIS',
       'SOME TEXT FOR ANALYSIS'], 
      dtype='<U22')

edited Mar 14, 2017 at 12:52

answered Mar 14, 2017 at 12:37

chrisaycock

38.1k15 gold badges94 silver badges128 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

code base 5000 Over a year ago

ya, but when you export the DF to an numpy array, it carries the object type and doesn't convert the data to a string.

code base 5000 Over a year ago

i forgot a piece of information, please see the updated question.

languitar Over a year ago

That's just the internal implementation. Have a look at one of the answers of the potential duplicate for an explanation.

languitar Over a year ago

It won't work, as this is not intended. There is an explanation why this is of type object for strings.

Collectives™ on Stack Overflow

Pandas Dataframe Issue Converting Column dtype

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related