PHP convert unicode spaces to ascii spaces

Question

So I'm having a problem where I believe what's happening is I'm receiving data that uses some unicode spaces and some ascii spaces, such that certain strings that appear the same are not equivalent, for example, "water resistant" != "water resistant". These strings appear differently in my database, however, with the weird characters you normally see when there's a multibyte character: "waterÂ resistantÂ" and "Â water resistant".

I would like a way to make all spaces be ascii spaces, or if easier, all spaces be multibyte spaces.

I've tried using preg_replace, but then the strings no longer read like valid multibyte strings anymore. (Multibyte characters in the strings will appear as garbage).

preg_replace('/[\pZ\pC]/',' ',$field);

I've also tried using mb_ereg_replace, but it had no effect.

mb_ereg_replace('/[\pZ\pC]/',' ',$field)

stackoverflow.com/questions/1497885/…

marekful
– marekful

2013-11-20 19:55:58 +00:00
Commented Nov 20, 2013 at 19:55 — marekful
– marekful, Commented Nov 20, 2013 at 19:55

Rob Evans · Accepted Answer · 2013-11-22 15:19:24Z

9

You can find and replace them with standard ascii spaces if you wanted via:

$string = str_replace("\xc2\xa0", "\x20", $string);

answered Nov 22, 2013 at 15:19

Rob Evans

7,0085 gold badges42 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

aequalsb Over a year ago

i consider this solution the best -- simple and functional. worked for me in my WordPress content_save_pre() filter to kill non-breaking spaces where a user typed two consecutive spaces in whatever content editor they were using (like Word) -- which converts one of the spaces to a non-breaking space to preserve 2-spaces. since we're not using typewriters, it's absurd to 2-space -- besides, it's type flow hell in a browser

Sithu Over a year ago

This did not work for me, but @Kai's answer did work.

Kai · Accepted Answer · 2013-11-20 20:25:12Z

5

It looks like preg_replace('/[\pZ\pC]/u',' ',$field); works (forgot the u at the end of the regex)

answered Nov 20, 2013 at 20:25

Kai

3,8231 gold badge20 silver badges33 bronze badges

1 Comment

rinogo Over a year ago

This works for me, but note that it seems to be a little more aggressive than may be desired. The regex provided also matches a "standard" ASCII space. So, if you're trying to replace only Unicode non-breaking spaces (e.g. with a non-space character), this will replace more characters than you intend.

Joren · Accepted Answer · 2013-11-20 19:54:34Z

2

I think you're looking for utf8_decode($field).

answered Nov 20, 2013 at 19:54

Joren

3,37327 silver badges49 bronze badges

2 Comments

Mat Carlson Over a year ago

Agreed - That works for me with ISO-8859-1 to UTF-8 for my database.

Kai Over a year ago

If I call utf8_decode($field), the field will still appear with garbage characters when displayed on the webpage. I also need to fix the spaces problem before I save to the database, because otherwise it will store duplicates of "water resistant" with various white spaces, rather than just a single entry "water resistant".

Lorenz Meyer · Accepted Answer · 2013-11-20 20:40:34Z

0

Those spaces that you call unicode spaces are non-breaking spaces ( what & nbsp; stands for ).

When saving the data, you have to clean it first. Replace all non-breaking spaces by normal spaces, replace double spaces by single and finally trim the string.

answered Nov 20, 2013 at 20:40

Lorenz Meyer

20.1k23 gold badges83 silver badges130 bronze badges

Collectives™ on Stack Overflow

PHP convert unicode spaces to ascii spaces

4 Answers 4

2 Comments

1 Comment

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

2 Comments

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related