C++ Null characters in string?

Question

I want to read a txt file and convert two cells from each line to floats.

If I first run:

someString = someString.substr(1, tempLine.size());

And then:

std::stof(someString)

it only converts the first number in 'someString' to a number. The rest of the string is lost. When I handled the string in my IDE I noticed that copying it and pasting it inside quotation marks gives me "\u00005\u00007\u0000.\u00007\u00001\u00007\u00007\u0000" and not 57.7177.

If I instead do:

std::string someOtherString = "57.7177"

std::stof(someOtherString)

I get 57.7177.

Minimal working example is:

int main() {

std::string someString = "\u00005\u00007\u0000.\u00007\u00001\u00007\u00007\u0000";
float someFloat =  std::stof(someString);
return 0;
}

Same problem occurs using both UTF-8 and -16 encoding.

What is happening and what should I do differently? Should I remove the null-characters somehow?

@tadman: Its' UTF16BE. The 5 and the 7 aren't the values 5 and 7, they're the characters 5 and 7. — Mooing Duck
– Mooing Duck, Commented Jul 9, 2020 at 16:11
@MooingDuck Oh, I misread that as \u0005 instead of \0000 and then 5. I think that's the answer then! — tadman
– tadman, Commented Jul 9, 2020 at 16:12
The trend is to use UTF-8 everywhere but on some ill-configured operating systems (old Windows computers) it is still not the case. Without any minimal reproducible example your question stays unclear — Basile Starynkevitch
– Basile Starynkevitch, Commented Jul 13, 2020 at 6:06

Mooing Duck · Accepted Answer · 2020-07-09 16:15:14Z

3

"I want to read a txt file"

What is the encoding of the text file? "Text" is not a encoding. What I suspect is happening is that you wrote code that reads in the file as either UTF8 or Windows-1250 encoding, and stored it in a std::string. From the bytes, I can see that the file is actually UTF16BE, and so you need to read into a std::u16string. If your program will only ever run on Windows, then you can get by with a std::wstring.

You probably have followup questions, but your original question is vague enough that I can't predict what those questions would be.

answered Jul 9, 2020 at 16:15

Mooing Duck

67.6k19 gold badges107 silver badges169 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

C++ Null characters in string?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related