Erlang binary strings by default

Question

I am writing an erlang module that has to deal a bit with strings, not too much, however, I do some tcp recv and then some parsing over the data.

While matching data and manipulating strings, I am using binary module all the time like binary:split(Data,<<":">>) and basically using <<"StringLiteral">> all the time.

Till now I have not encounter difficulties or missing methods from the alternative( using lists) and everything is coming out quite naturally except maybe for adding the <<>>, but I was wondering if this way of dealing with strings might have drawbacks I am not aware of.

Any hint?

Ben · Accepted Answer · 2012-04-03 00:52:05Z

5

As long as you and your team remember that your strings are binaries and not lists, there are no inherent problems with this approach. In fact, Couch DB took this approach as an optimization which apparently paid nice dividends.

answered Apr 3, 2012 at 0:52

Ben

6,0591 gold badge27 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

dsmith · Accepted Answer · 2012-04-03 17:04:09Z

You do need to be very aware of how your string is encoded in your binaries. When you do <<"StringLiteral">> in your code, you have to be aware that this is simply a binary serialization of the list of code-points. Your Erlang compiler reads your code as ISO-8859-1 characters, so as long as you only use Latin-1 characters and do this consistently, you should be fine, But this isn't very friendly to internationalization.

Most application software these day should prefer a unicode encoding. UTF-8 is compatible with your <<"StringLiteral">> for the first 128 codepoints, but not for the second 128, so be careful. You might be surprised what you see on your UTF-8 encoded web applications if you use <<"StrïngLïteral">> in your code.

There was an EEP proposal for binary support in the form of <<"StrïngLïteral"/utf8>>, but I don't think this is finalized.

Also be aware that your binary:split/2 function may have unexpected results in UTF-8 if there is a multi-byte character that contains the IS0-8859-1 byte that to are splitting on.

Some would argue that UTF-16 is a better encoding to use because it can be parsed more efficiently and can be more easily split by index, if you are assuming or verify that there are no 32-bit characters.

The unicode module should be use, but tread carefully when you use literals.

I GIVE CRAP ANSWERS · Accepted Answer · 2012-04-02 22:16:29Z

3

The only thing to be aware of is that a binary is a slice of bytes, whereas a list is a list of unicode codepoints. In other words, the latter is naturally unicode whereas the former requires you to do some sort of encoding, usually UTF-8.

To my knowledge, there is no drawbacks to your method.

answered Apr 2, 2012 at 22:16

I GIVE CRAP ANSWERS

18.9k3 gold badges46 silver badges48 bronze badges

Comments

Hynek -Pichi- Vychodil · Accepted Answer · 2012-04-03 09:40:31Z

2

Binaries are very efficient structures to store strings. If they are longer than 64B they are also stored outside process heap so they are not object of GC (still GC'ed by ref counting when last ref lost). Don't forget use iolists for concatenation them to avoid copying when performance matter.

answered Apr 3, 2012 at 9:40

Hynek -Pichi- Vychodil

26.2k5 gold badges55 silver badges75 bronze badges

Collectives™ on Stack Overflow

Erlang binary strings by default

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related