Ruby encoding question

Question

I'm saving scraped data to a web app, and here's a sample param:

400\xB0F.

This is the 'degree' character from a website, but when I put that into my model I get the dreaded invalid byte sequence in UTF-8 error.

Since it's coming from the web I thought I might try some client side encoding, so javascript turns that into: 400%B0F. This can at least get saved by ActiveRecord with no issue, but Rails seems to be escaping it again on the way out so those entities aren't decoded by the browser, so my show method shows the entire encoded string.

Where should I be cleaning up my input data, and what methods might be the best to use for unpredictable input?

Thanks!

It looks like the website is encoded with CP-1252. I'm not sure how you'd go about decoding this though, sorry. — Hugh
– Hugh, Commented Aug 5, 2011 at 5:13
I would look up how to use the String.encode method to effect conversions among encodings if I were you. — Ryanmt
– Ryanmt, Commented Aug 5, 2011 at 5:56

Sam Ruby · Accepted Answer · 2011-08-05 23:54:17Z

1

Years ago I had, and solved, this very same problem in builder. Take a look at the to_xs method: http://builder.rubyforge.org/classes/String.html#M000007

You can require builder, and use it directly (you might want to pass false to escaping or you will get entity escaped output). Either that, or simply steal and adapt the source.

Update: here is the original, standalone, library: http://intertwingly.net/stories/2005/09/28/xchar.rb

edited Aug 5, 2011 at 23:54

answered Aug 5, 2011 at 23:43

Sam Ruby

4,35024 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Alex Mcp Over a year ago

Oh wow, an answer from -the- Sam Ruby. I'm all a-nerd-flutter! ;-)

user880173 · Accepted Answer · 2011-08-05 08:52:38Z

0

Perhaps you can use a binary form (like for upload file) with enctype="multipart/form-data" in form tag. Like this, you can use this data as a binary data ?

It's depends perhaps of waht you do with this data.

answered Aug 5, 2011 at 8:52

user880173

Comments

Alex Mcp · Accepted Answer · 2011-08-05 22:13:26Z

0

URI.unescape was the trick, after I encoded it client-side

answered Aug 5, 2011 at 22:13

Alex Mcp

19.4k12 gold badges62 silver badges94 bronze badges

Collectives™ on Stack Overflow

Ruby encoding question

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related