Are MapByteBuffer faster than BufferReader?

Question

I am trying to extract specific data from a text file. I using bufferreader to read the file line by line so i can proccess the line.

Below are code that i written:

File CAD = new File (filename);

    try (FileReader fileReader = new FileReader(CAD)) 
    {
        String bn = CAD.getName();
        String[] bn2 = bn.split("\\.");
        BOARD_NAME= bn2[0];
        StringBuilder PACKAGE = new StringBuilder(),
        GRAPHIC = new StringBuilder(),
        PADS_DIA = new StringBuilder(),
        PADS = new StringBuilder(),
        COMPONENTS = new StringBuilder();
        try (BufferedReader bufferedReader = new BufferedReader(fileReader))
        {
            String line;

            while ((line = bufferedReader.readLine()) != null) 
            {
                if (line.startsWith("J!")) 
                {
                    String[] units = line.split("!");
                    UNITS = units[8];

                } 

                else if (line.startsWith("A!REFDES!COMP_CLASS!")) 
                {

                    while (!"A!NET_NAME!REFDES!PIN_NUMBER!PIN_NAME!PIN_GROUND!PIN_POWER!".equals(line = bufferedReader.readLine())) 
                    {
                        if (line.startsWith("S!")) 
                        {
                            String[] data = line.split("!");
                            COMPONENTS.append(data[1])
                                .append("!")
                                .append(data[8])
                                .append("!")
                                .append(data[10])
                                .append("!")
                                .append(data[11])
                                .append("!")
                                .append(data[12])
                                .append("\n");


                        }
                    }


                } 

                else if (line.startsWith("A!PAD_NAME")) 
                {

                    do 
                    {   

                        if (line.startsWith("S!") && line.contains("00001")) 
                        {
                            String[] data = line.split("!");
                            if ("CIRCLE".equals(data[6])) 
                            {
                                PADS.append(data[1])
                                    .append("!C!")
                                    .append(data[7])
                                    .append("!")
                                    .append(data[8])
                                    .append("!TOP")
                                    .append("\n");

                            } 

                            else if (data[6].equals("")) 
                            {
                                PADS.append(data[1])
                                    .append("!0!")
                                    .append(data[7])
                                    .append("!")
                                    .append(data[8])
                                    .append("!TOP")
                                    .append("\n");
                            }

                            else 
                            {
                                PADS.append(data[1])
                                    .append("!R!")
                                    .append(data[7])
                                    .append("!")
                                    .append(data[8])
                                    .append("!TOP")
                                    .append("\n");
                            }
                        } 

                        else if (line.startsWith("S!") && line.contains("BOTTOM")) 
                        {
                            String[] data = line.split("!");
                            if ("CIRCLE".equals(data[6])) 
                            {
                                PADS.append(data[1])
                                    .append("!C!")
                                    .append(data[7])
                                    .append("!")
                                    .append(data[8])
                                    .append("!BOTTOM")
                                    .append("\n");
                            } 

                            else if (data[6].equals("")) 
                            {
                                PADS.append(data[1])
                                    .append("!0!")
                                    .append(data[7])
                                    .append("!")
                                    .append(data[8])
                                    .append("!BOTTOM")
                                    .append("\n");
                            }    

                            else 
                            {
                                PADS.append(data[1])
                                    .append("!R!")
                                    .append(data[7])
                                    .append("!")
                                    .append(data[8])
                                    .append("!BOTTOM")
                                    .append("\n");
                            }


                        } 

                        else if (line.startsWith("S!") && line.contains("~DRILL")) 
                        {

                            String[] data = line.split("!");
                            PADS_DIA.append(data[1])
                                .append("!")
                                .append(data[6])
                                .append("\n");

                        }

                    }while (!"A!GRAPHIC_DATA_NAME!GRAPHIC_DATA_NUMBER!RECORD_TAG!GRAPHIC_DATA_1!GRAPHIC_DATA_2!GRAPHIC_DATA_3!GRAPHIC_DATA_4!GRAPHIC_DATA_5!GRAPHIC_DATA_6!GRAPHIC_DATA_7!GRAPHIC_DATA_8!GRAPHIC_DATA_9!SUBCLASS!SYM_NAME!REFDES!".equals(line = bufferedReader.readLine()));           
                } 

                if (line.startsWith("A!SYM_NAME!"))                
                {
                    int k =0,j=0;
                    String[] data1 = line.split("!");
                    for (int i =0; i<data1.length;i++)
                    {
                        if (data1[i].equals("PIN_NUMBER"))
                        {
                            k=i;
                        }
                        else if (data1[i].equals("PIN_ROTATION"))
                        {
                            j=i;
                        }
                    }
                    while(!"A!VIA_X!VIA_Y!PAD_STACK_NAME!NET_NAME!TEST_POINT!".equals(line = bufferedReader.readLine()))
                    {  
                        if (line.startsWith("S!")) 
                        {
                            String[] data = line.split("!");
                            if (data[8].equals("")||data[8]==null)
                            {}
                            else
                            {
                                PACKAGE.append(data[1])
                                    .append("!")
                                    .append(data[k])
                                    .append("!")
                                    .append(data[k+1])
                                    .append("!")
                                    .append(data[k+2])
                                    .append("!")
                                    .append(data[k+3])
                                    .append("!")
                                    .append(data[k+4])
                                    .append("!")
                                    .append(data[j])
                                    .append("\n");                      
                            }
                        }
                    }


                }

                else if (line.startsWith("A!CLASS!SUBCLASS!GRAPHIC_DATA_NAME!"))
                {          

                    while (!"A!SUBCLASS!PAD_SHAPE_NAME!GRAPHIC_DATA_NAME!GRAPHIC_DATA_NUMBER!RECORD_TAG!GRAPHIC_DATA_1!GRAPHIC_DATA_2!GRAPHIC_DATA_3!GRAPHIC_DATA_4!GRAPHIC_DATA_5!GRAPHIC_DATA_6!GRAPHIC_DATA_7!GRAPHIC_DATA_8!GRAPHIC_DATA_9!PAD_STACK_NAME!REFDES!PIN_NUMBER!".equals(line = bufferedReader.readLine()))
                    {
                        if (line.startsWith("S!BOARD GEOMETRY!OUTLINE")) 
                        {
                            //to find the diamension x and y of board
                            String data[]=line.split("!");
                            double valuex = Double.parseDouble(data[6]);
                            double valuey = Double.parseDouble(data[7]);
                            PANEL_X.add(valuex);
                            PANEL_Y.add(valuey);

                        }

                        else if (line.startsWith("S!REF DES"))
                        {
                            String data[] = line.split("!");
                            if (data[2].equals("ASSEMBLY_TOP")||data[2].equals("DISPLAY_TOP"))
                            {
                                GRAPHIC.append(data[12])
                                    .append("!")
                                    .append("TOP")
                                    .append("\n");
                            }

                            else if(data[2].equals("ASSEMBLY_BOTTOM")||data[2].equals("DISPLAY_BOTTOM"))
                            {
                                GRAPHIC.append(data[12])
                                   .append("!")
                                   .append("BOTTOM")
                                   .append("\n"); 
                            }
                        }
                    }


                    double max_x = max(PANEL_X);
                    double min_x = min(PANEL_X);
                    double max_y = max(PANEL_Y);
                    double min_y = min(PANEL_Y);
                    double diamensionx = max_x - min_x;
                    double diamensiony = max_y - min_y;
                    MIN_X = min_x;
                    MIN_Y = min_y;
                    BOARD_X = diamensionx;
                    BOARD_Y = diamensiony;
                    BOARD_Z = 62.9921;

                }

                ArrayList<String> $TEMP_LIST = new ArrayList<>(asList(GRAPHIC.toString().split("\n")));
                HashSet<String> graphic = new HashSet<>($TEMP_LIST);
                GRAPHIC_LIST = new ArrayList<>(graphic);
                PACKAGE_LIST = new ArrayList<>(asList(PACKAGE.toString().split("\n")));
                PADS_LIST = new ArrayList<>(asList(PADS.toString().split("\n")));
                PADS_DIA_LIST = new ArrayList<>(asList(PADS_DIA.toString().split("\n")));
                COMPONENTS_LIST = new ArrayList<>(asList(COMPONENTS.toString().split("\n")));           
            }

        }                
        fileReader.close();
        }

On my code,still got many if else condition. When i try to extract from a file size about 20MB , it take time like 30 seconds. But when i try to read a file size about 200MB , it take very long time, like more than 30 minutes.My friends suggest me to use MappedByteBuffer to proccess faster.

I am still new in Java. So do MappedBuffer perform faster than BufferReader in this case? If yes , how can i process the data line by line?

You can read millions of lines a second with BufferedReader. Your problem lies elsewhere. — user207421
– user207421, Commented Oct 1, 2015 at 8:19
ya.. i knew that..so i wondering is it too much if else condition make BufferReader perform badly. — Jian. Eoh
– Jian. Eoh, Commented Oct 1, 2015 at 8:30
That question doesn't begin to make sense. The performance of the BufferedReader has nothing whatsoever to do with the if-else's, and whatever other reading technology you use, you will still have the if-else's, unless you find a way to improve them, in which case BufferedReader would benefit just as much as any other technology. And MappedByteBuffer doesn't support reading lines. — user207421
– user207421, Commented Oct 1, 2015 at 8:40
One thing you could do is to read the file into a string buffer first before you being to manipulate the content. Also look at what you are doing with the content and areas where you can improve. If you deem the buffered reader is giving an issue, then check the Java NIO 2 package which does Native file reads as opposed to going through the JVM. — dinukadev
– dinukadev, Commented Oct 1, 2015 at 8:45
@dinukadev Pre-reading the file would only add more latency and waste memory. — user207421
– user207421, Commented Oct 1, 2015 at 8:47

Kayaman · Accepted Answer · 2015-10-01 07:55:39Z

1

I doubt your issue is with BufferedReader, it's fast enough for almost all purposes. Using MappedByteBuffer does allow you to read the contents (or parts of it) of a file into memory all at once, but then you're dealing with a ByteBuffer instead of lines of text, and it's unlikely that you'd get a significant performance improvement with it.

If you remove the contents of the while loop and just read the lines without any processing, you'll notice that a file of 20MB is read in an instant.

An obvious (minor) improvement is to move the line.split("!") into the scope where it's actually needed. I'm not sure if the code you're showing is your actual code (probably not since you spelled split() as splits(), but in the example code you would only need to split the line if it starts with cga. Without seeing your actual code it's hard to point out all the locations that make it slow (you could use a profiler of course).

As a last suggestion, if you're writing Java professionally, you might want to avoid non-Java things like $ in variable names, and opening braces on their own lines (Java adheres to 1TBS). It's helpful to other Java developers who may read your code.

answered Oct 1, 2015 at 7:55

Kayaman

73.8k6 gold badges91 silver badges128 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jian. Eoh Over a year ago

i have put my code inside the question. Are there any thing make the proccess time slow? 30 secs still acceptable but i dont't know why it would take more than 30 min to proccess a larger file.

Kayaman Over a year ago

Well, it's not very readable. The first thing I would do is refactor the code so that most of the if clauses would become of the form if(something) { output.addAll(mySomeMethodProcessing(input)); }. That way you'll get the if logic clearer and you can give the methods a descriptive name (at the moment your code is pretty much unreadable). Joop Eggen's suggestion about setting the initial capacity is a good idea too, so the StringBuilders and ArrayLists don't need to perform unnecessary growing.

Michael · Accepted Answer · 2015-11-09 15:16:20Z

1

Without profiling this is difficult to answer correctly. One thing which might be worth trying:

You use String.split for each line. For every call of split, Java compiles a Pattern:

public String[] split(String regex, int limit) {
    return Pattern.compile(regex).split(this, limit);
}

So it may be better to compile the Pattern you use only once at the beginning:

Pattern pattern = Pattern.compile("!");
Pattern patternNL = Pattern.compile("\n");

and use this to split your strings:

String[] parts  = pattern.split(line);
String[] singleLines = patternNL.split(lines);

answered Nov 9, 2015 at 15:16

Michael

111 bronze badge

Comments

Joop Eggen · Accepted Answer · 2015-10-01 08:37:24Z

0

Not necessarily faster; only if your text file becomes in the order of 100 MB, you might consider such a thing.

Better do other optimizations first.

Like new StringBuilder(10_000). Also for BufferedReader.

answered Oct 1, 2015 at 8:37

Joop Eggen

110k8 gold badges89 silver badges142 bronze badges

Collectives™ on Stack Overflow

Are MapByteBuffer faster than BufferReader?

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related