How to replace characters in a very large string using java causing large memory?

Question

I have a large String in which I need to replace a few string pathern.

I have written code as below and I am getting very large memory usage in deployed environment.

I have read online to see if there is any more optimal way this can be re-written but could not find conclusive answer.

Can anyone give any suggestions for code below.

    String response = "@very #large  & string % STUFF";
    System.out.println(response);
    String[] SPECIAL_CHARACTERS = {"&","%","#","STUFF"};
    for(int count = 0;count<SPECIAL_CHARACTERS.length;count++)
    {
        if(response.contains(SPECIAL_CHARACTERS[count]))
        {
            response = response.replace(SPECIAL_CHARACTERS[count],"");
        }
    }
    System.out.println(response);

Ahh, the bothersome immutability of Java/.NET world. stackoverflow.com/questions/279507/what-is-meant-by-immutable — PTwr
– PTwr, Commented Jun 4, 2020 at 11:58
What is the source of the string (for example Web Request), and what is the destination of the output(for example File / database)? — samabcde
– samabcde, Commented Jun 4, 2020 at 12:05
I am getting data from database having these special characters and then in java code replacing it and returning it in REST API response. I can't do replacement in DB sql due to performance consideration. — satish
– satish, Commented Jun 4, 2020 at 12:32

Stephen C · Accepted Answer · 2020-06-04 13:22:49Z

2

I have a large String in which I need to replace a few characters.

I would avoid getting into that situation. Stream your data! (And how come you have a "very large" string stored in the database? That doesn't sound good.)

If you can't, this is the most memory efficient way to do what you are doing:

   int len = response.length();
   for (int i = 0; i < len; i++) {
       char ch = response.charAt(i);
       switch (ch) {
           case '&': case '%': case '#': case '@':
               break;
           default:
               System.out.print(ch);
       }
   }
   System.out.println();

In some circumstances, it may be better to use a StringBuilder so that you can do a single write operation.

   int len = response.length();
   StringBuilder temp = new StringBuilder(len);
   for (int i = 0; i < len; i++) {
       char ch = response.charAt(i);
       switch (ch) {
           case '&': case '%': case '#': case '@':
               break;
           default:
               temp.append(ch);
       }
   }
   System.out.println(temp.toString());

but that allocates more memory.

edited Jun 4, 2020 at 13:22

answered Jun 4, 2020 at 13:17

Stephen C

723k95 gold badges849 silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

satish Over a year ago

Thanks. Actually I missed mentioning that the text to be replaced can be a string (thus I created a String array). I am trying to think how would this code snippet would work in that case as I have to write a generic code to replace a specific text. I have updated my question to reflect same. In this case, I guess I would need to use String.replace() function which causes memory issue. Do you think same?

Stephen C Over a year ago

You can easily modify the snippets to handle that. You might end up replacing the switch with a sequence of if else statements or a loop. But I have answered the question that you asked. Avoid using String.replace() because each time you call it you copy the entire string.

Stephen C Over a year ago

The key to modifying the snippets will be to understand how they work. Read the code. It is pretty obvious when you think about it.

Lajos Arpad · Accepted Answer · 2020-06-04 12:07:16Z

1

I would solve this kind of problem by reading packets from the input and append them to the output. So, assuming that the very large String is called input, I would always read easily manageable packets from input into temp, do the algorithm of replacements for temp and then append temp to output, which should be initialized before your cycles as empty String.

The problem you encounter is that you are always replacing your whole String. So, instead of replacing response each time, do all the replacements on smaller packets and add the result to a String variable.

answered Jun 4, 2020 at 12:07

Lajos Arpad

80.4k42 gold badges123 silver badges234 bronze badges

2 Comments

satish Over a year ago

I am not not sure I am following it. A code sample would be handy.

Lajos Arpad Over a year ago

@satish in order to do that I would need to test it before posting it here, but I'm lazy to do so. If you want, you can try implementing it and you may ask questions when you are stuck.

Collectives™ on Stack Overflow

How to replace characters in a very large string using java causing large memory?

2 Answers 2

3 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related