You have to first escape the backslash because it's a literal (yielding \\), and then escape it again because of the regular expression (yielding \\\\). So, Try:

 s.replaceAll("'", "\\\\'");

output:

You\'ll be totally awesome, I\'m really terrible

Use replace()

 s = s.replace("'", "\\'"); 

output:

You\'ll be totally awesome, I\'m really terrible

Let's take a tour of String#repalceAll(String regex, String replacement)

You will see that:

An invocation of this method of the form str.replaceAll(regex, repl) yields exactly the same result as the expression

Pattern.compile(regex).matcher(str).replaceAll(repl)

So lets take a look at Matcher.html#replaceAll(java.lang.String) documentation

Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.

You can see that in replacement we have special character $ which can be used as reference to captured group like

System.out.println("aHellob,aWorldb".replaceAll("a(\\w+?)b", "$1"));
// result Hello,World

But sometimes we don't want $ to be such special because we want to use it as simple dollar character, so we need a way to escape it.
And here comes \, because since it is used to escape metacharacters in regex, Strings and probably in other places it is good convention to use it here to escape $.

So now \ is also metacharacter in replacing part, so if you want to make it simple \ literal in replacement you need to escape it somehow. And guess what? You escape it the same way as you escape it in regex or String. You just need to place another \ before one you escaping.

So if you want to create \ in replacement part you need to add another \ before it. But remember that to write \ literal in String you need to write it as "\\" so to create two \\ in replacement you need to write it as "\\\\".

So try

s = s.replaceAll("'", "\\\\'");

Or even better

to reduce explicit escaping in replacement part (and also in regex part - forgot to mentioned that earlier) just use replace instead replaceAll which adds regex escaping for us

s = s.replace("'", "\\'");

This doesn't say how to "fix" the problem - that's already been done in other answers; it exists to draw out the details and applicable documentation references.

When using String.replaceAll or any of the applicable Matcher replacers, pay attention to the replacement string and how it is handled:

Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.

As pointed out by isnot2bad in a comment, Matcher.quoteReplacement may be useful here:

Returns a literal replacement String for the specified String. .. The String produced will match the sequence of characters in s treated as a literal sequence. Slashes (\) and dollar signs ($) will be given no special meaning.

You could also try using something like StringEscapeUtils to make your life even easier: http://commons.apache.org/proper/commons-lang/javadocs/api-2.6/org/apache/commons/lang/StringEscapeUtils.html

s = StringEscapeUtils.escapeJava(s);

You can use apache's commons-text library (instead of commons-lang):

Example code:

org.apache.commons.text.StringEscapeUtils.escapeJava(escapedString);

Dependency:

compile 'org.apache.commons:commons-text:1.8'

OR

<dependency>
   <groupId>org.apache.commons</groupId>
   <artifactId>commons-text</artifactId>
   <version>1.8</version>
</dependency>