Replacing values in DataFrame with None

Question

When creating a Pandas DataFrame with None values, they are converted to NaN:

> df = pd.DataFrame({'a': [0, None, 2]})
> df

      a
0   0.0
1   NaN
2   2.0

Same thing if I set a value to None by index:

> df = pd.DataFrame({'a': [0, 1, 2]})
> df["a"].iloc[1] = None
> df

      a
0   0.0
1   NaN
2   2.0

However, if I do a replace, weird things start happening:

> df = pd.DataFrame({'a': [0, 1, 2, 3]})
> df["a"].replace(1, "foo")

    a
0   0
1   'foo'
2   2
3   3

> df["a"].replace(2, None)

    a
0   0
1   1
2   1
3   3

What is going on here?

df["a"].replace({1:None})

BENY
– BENY

2020-05-14 19:39:30 +00:00
Commented May 14, 2020 at 19:39 — BENY
– BENY, Commented May 14, 2020 at 19:39

Zesty Dragon · Accepted Answer · 2020-05-14 19:40:25Z

1

s = pd.Series([10, 'a', 'a', 'b', 'a'])
s.replace({'a': None})
0      10
1    None
2    None
3       b
4    None
dtype: object

s.replace({'a': None}) is equivalent to s.replace(to_replace={'a': None}, value=None, method=None):

When value=None and to_replace is a scalar, list or tuple, replace uses the method parameter (default ‘pad’) to do the replacement. So this is why the ‘a’ values are being replaced by 10 in rows 1 and 2 and ‘b’ in row 4 in this case. The command s.replace('a', None) is actually equivalent to s.replace(to_replace='a', value=None, method='pad'):

  s.replace('a', None)
    0    10
    1    10
    2    10
    3     b
    4     b
    dtype: object

answered May 14, 2020 at 19:40

Zesty Dragon

5714 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Quang Hoang · Accepted Answer · 2020-05-14 19:39:21Z

According to the doc string

When ``value=None`` and `to_replace` is a scalar, list or
tuple, `replace` uses the method parameter (default 'pad') to do the
replacement. So this is why the 'a' values are being replaced by 10
in rows 1 and 2 and 'b' in row 4 in this case.
The command ``s.replace('a', None)`` is actually equivalent to
``s.replace(to_replace='a', value=None, method='pad')``

If you want to actually replace with None, pass a dict:

>>> s = pd.Series([10, 'a', 'a', 'b', 'a'])

When one uses a dict as the `to_replace` value, it is like the
value(s) in the dict are equal to the `value` parameter.
``s.replace({'a': None})`` is equivalent to
``s.replace(to_replace={'a': None}, value=None, method=None)``:

>>> s.replace({'a': None})
0      10
1    None
2    None
3       b
4    None
dtype: object

Collectives™ on Stack Overflow

Replacing values in DataFrame with None

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related