Converting char * to double while reading from binary file in C++?

Question

I am writing a binary I/O for storing data in my application.

For illustration consider I want to store a double array of size 10 to the file.

Now since it is not guaranteed that double uses 8 bytes on all platforms, the reader of the file needs to be be modified a bit. Although I am using Qt I think the problem is mainly in the way data read in char * is translated into double. The data read is almost zero.

For example, 1 is read as 2.08607954259741e-317.

Why is every double being read as zero even thought it is not?

void FileString::SaveBinary()
{
    QFile *file = new QFile(fileName);
    if (!file->open(QFile::WriteOnly))
    {
        QString err = file->errorString();
        QString *msgText = new QString("Could not open the file from disk!\n");
        msgText->append(err);
        QString *msgTitle = new QString("ERROR: Could not open the file!");
        emit errMsg(msgTitle, msgText, "WARNING");
        delete file;
        return;
    }
    QDataStream out(file);
    QString line = "MyApp";
    out << line;
    line.setNum(size);//size = 10
    out << line;
    line.setNum(sizeof(double));
    out << line;

    for(int i = 0; i < size; i++)
    {
        out << array[i];
    }

    if(out.status() != QDataStream::Ok)
    {
        qCritical("error: " + QString::number(out.status()).toAscii());
    }
    file->close();
    delete file;
}

void FileString::ReadBinary()
{
    bool ok = false;
    QString line = "";
    QFile *file = new QFile(fileName);
    if (!file->open(QFile::ReadOnly))
    {
        QString err = file->errorString();
        QString *msgText = new QString("Could not open the file from disk!\n");
        msgText->append(err);
        QString *msgTitle = new QString("ERROR: Could not open the file!");
        emit errMsg(msgTitle, msgText, "WARNING");
        delete file;
        return;
    }

    QDataStream in(file);
    in >> line;
    if(line.simplified().contains("MyApp"))
    {
        in >> line;
        size = line.simplified().toInt();
        if(size == 10)
        {
            int mysize = 0;
            in >> line;
            mysize = line.simplified().toInt();
            if(1)//this block runs perfect
            {
                for(int i = 0; i < size; i++)
                {
                    in >> array[i];
                }

                if(in.status() == QDataStream::Ok)
                    ok = true;
                }
            }
            else if(1)//this block reads only zeros
            {
                char *reader = new char[mysize + 1];
                int read = 0;
                double *dptr = NULL;
                for(int i = 0; i < size; i++)
                {
                    read = in.readRawData(reader, mysize);
                    if(read != mysize)
                    {
                        break;
                    }

                    dptr = reinterpret_cast<double *>(reader);//garbage data stored in dptr, why?
                    if(dptr)
                    {
                        array[i] = *dptr;
                        dptr = NULL;
                    }
                    else
                    {
                        break;
                    }
                }


                if(in.status() == QDataStream::Ok)
                    ok = true;
                delete[] reader;
            }
        }
    }

    if(!ok || (in.status() != QDataStream::Ok))
    {
        qCritical("error : true" + " status = " + QString::number((int) in.status()).toAscii());
    }
    file->close();
    delete file;
}

EDIT:

Contents of the generated file

   & M y A p p   1 . 1 8 . 3 . 0    1 0    8?ð      @       @      @      @      @      @      @       @"      @$

That is supposed to contain:

MyApp 1.18.3.010812345678910

"MyApp 1.18.3.0" "10" "8" "12345678910"

That is a statement, not a question. Please make sure you have a question so that we do not need to guess what the question may be. Also, please share your file content, etc. — László Papp
– László Papp, Commented Jan 11, 2014 at 10:02
@Lazlo sorry I did not make it clear. Please see the updated question. — Cool_Coder
– Cool_Coder, Commented Jan 11, 2014 at 10:05
You could have provided a much smaller example to illustrate the problem. — Ixanezis
– Ixanezis, Commented Jan 11, 2014 at 10:08
@Lazlo added the contents, not sure whether anything can be deduced from it though.... — Cool_Coder
– Cool_Coder, Commented Jan 11, 2014 at 10:58

Community · Accepted Answer · 2017-05-23 12:29:16Z

1

What do you expect to read if sizeof double on read platform differs from sizeof double on write platform?

Suppose sizeof double on your write platform was 10. Then you stored a sequence of 10 bytes in a file that represents a 10-byte double. Then, if sizeof double on your read platform is 8, you would try to parse bits of an 10-byte double into an 8-byte and that would obviously end up with garbage.

Here's a more intuitive example with ints: If you a have a 2-byte integer number, say 5. If you store it in binary file, you'll get a sequence of 2 bytes: 00000000 00000101. Then, if you try to read the same number as a 1-byte int, you'll manage to read only the first byte, which is 00000000 and get just zero as a result.

Consider using strings to save doubles for portability https://stackoverflow.com/a/6790009/817441

edited May 23, 2017 at 12:29

CommunityBot

11 silver badge

answered Jan 11, 2014 at 10:35

Ixanezis

1,69115 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Ixanezis Over a year ago

I address that his doubles are read as garbage. The fact that doubles 'are nearly zero' is just a special case of that. The OP is not only aware of 8-10 bit differences, but it is really so in his code. I believe the answer explains exactly what OP wants, because if the question was really about 'why is what I've read is close to zero?', then it would have been formulated as 'How a doubles represented in memory?'. AFAICT, you haven't devoted much time in OP's question and code investigation.

Cool_Coder Over a year ago

@Lazlo that is exactly what I am talking about! I am reading/writing currently on my laptop. So atleast on the same machine the code should work. Lxanezis do you agree?

Ixanezis Over a year ago

If that is true, then if(mysize == sizeof(double)) should had been evaluated true. As I can learn from the code, the problem appears in another branch of the code, when mysize != sizeof(double). This is why I was pretty sure sizes do not match and it leads to incorrect double reading.

Cool_Coder Over a year ago

it does evaluate to true. And the code in that block runs perfectly. But for debugging purpose if I set it to false so that the next bock runs then I getting the zero problem. So the created binary file is definitely correct.

Community · Accepted Answer · 2020-06-20 09:12:55Z

1

Note that in your original code sizeof(double) could work instead of the hard-coded string, but it will not as long as to migrate to a different architecture with a different double size on it.

As a side note if you are worried about the performance of the double to string conversion, you may have more problems when your users or you would like to move to embedded later. I have just run some conversions in a loop, and it is not that bad on my old laptop either. Here is my very poor benchmark result:

time ./main 

real    0m1.244s
user    0m1.240s
sys     0m0.000s

I would like to point it out again that it is an old laptop.

for the code:

#include <QString>

int main()
{
    for (int i = 0; i < 1000000; ++i)
        QString::number(5.123456789012345, 'g', 15);
    return 0;
}

So, instead of the non-portable direct write, I would suggest to use the following method:

QString QString::number(double n, char format = 'g', int precision = 6) [static]

Returns a string equivalent of the number n, formatted according to the specified format and precision. See Argument Formats for details.

Unlike QLocale::toString(), this function does not honor the user's locale settings.

http://doc-snapshot.qt-project.org/qdoc/qstring.html#number-2

Having discussed all this theretically, I would be writing something like this if I were you:

void FileString::SaveBinary()
{
    QFile *file = new QFile(fileName);
    if (!file->open(QFile::WriteOnly))
    {
        QString err = file->errorString();
        QString *msgText = new QString("Could not open the file from disk!\n");
        msgText->append(err);
        QString *msgTitle = new QString("ERROR: Could not open the file!");
        emit errMsg(msgTitle, msgText, "WARNING");
        delete file;
        return;
    }
    QDataStream out(file);
    QString line = QString::number(myDouble);
    out << line;

    for(int i = 0; i < size; i++)
    {
        out << array[i];
    }

    if(out.status() != QDataStream::Ok)
    {
        qCritical("error: " + QString::number(out.status()).toAscii());
    }
    file->close();
    delete file;
}

One portable option could be to use long double, but of course that would increase the computation at other places, so depending on the scenario, it may or may not be an option.

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Jan 11, 2014 at 11:31

László Papp

53.5k40 gold badges118 silver badges142 bronze badges

5 Comments

Cool_Coder Over a year ago

For my application I am already using text files. The text is generated using QString::number(). However when there are many million double variables involved the time for translation is large. Also the text file size becomes large. That is why I wanted to add another file format for binary I/O. It looks like using raw binary files consisting of double variables is not portable at all. So I will add a warning message when the binary file is opened in an incompatible PC.

Cool_Coder Over a year ago

You benchmarked by converting the double to a QString only upto 6 (default) decimal digits. Try for 15 digits then you will understand the bottleneck.

Cool_Coder Over a year ago

Well you are benchmarking incorrectly still! By passing 15 digit to QString::number() do you expect it to print to the file with precision of 15 digits? Instead you should use QString::number(5.123, 'g', 15); This requires 1270ms on my laptop. That is why I downvoted.

Cool_Coder Over a year ago

@Lazlo I definitely appreciate you spending time for me. But just look where you diverted our discussion to? Don't you see why I am not converting double to QString now? I told this already to you, but you wouldn't listen. My question was pretty simple, "how to read/write double variables in a portable manor". Ixanezis answered exactly what I wanted to know. But still as an appreciation for helping me I will give you an upvote, please don't get me wrong here...

László Papp Over a year ago

@Cool_Coder: my name is "Laszlo". :) Yes. the other explain the high-level concept as well. I just wished to provide a bit more concrete answer with benchmark and code included, as well as alternative approaches I believe progammers.stackexchange.com is better for only high-level dicussions. Have you seen the "long double" at the end added in a subsequent edit? Also, fwiw, sizeof(double) write works here, and IMO should. I hope the benchmark is also useful. :) Not requesting to select my answer, just explaining.

Collectives™ on Stack Overflow

Converting char * to double while reading from binary file in C++?

2 Answers 2

4 Comments

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related