No answers after a few days, so thought I'd have a go. Found it trickier than expected, so the answer's pretty long, but I'm reasonably confident it works. Should be able to trim it a fair bit more.

Python 3.8, 431 405 410 bytes

import re,itertools as I
B,M,C,P="\-]^"
s=sorted;x=s({*input()})
d=[ord(x)-i for i,x in enumerate(x)];l=[];i=0
for _,g in I.groupby(d):n=len([*g]);l+=[[x[i]+M+x[i+n-1]],x[i:i+n]][n<4];i+=n
o="".join(s(l,key=lambda i:2*(M in i[::2])+(P in i)-3*(i[0]==M<C not in x)-2*(C in i)))
print("["+re.sub(".",lambda m:B*((x:=m.group())==B or x==P==o[0]or x==M<M*2==o[(i:=m.start()):i+2]in o[1:-1]or x==C in o[1:])+x,o)+C)

Try it online!

Explanation

The overall approach is to:

1. Split the letters into ranges of length at least 4 and individual characters. Note that ranges of length 3 are never necessary (eg. "[abc]" is the same length as "[a-c]" and "[]\\[]" is the same length "[[-\]]") and are sometimes worse (e.g. "]-./" is shorter as "[]./-]" than as "[]\--/]" or "[--/\]]").

2. Sort the letters and ranges, so "]" goes first, otherwise "-" goes first, otherwise "-" goes last, and with "^" as late as posssible. There is no conflict between putting "]" first and "-" last since "]--" is not a valid range.

3. Escape as necessary: \ always, ] unless it's first, ^ if it's first, and - if it's in the middle (which will only happen if it's the beginning of a range).

Extra test cases

"-^"      => "[-^]"
"-4]"     => "[]4-]"
"abcde^"  => "[a-e^]"
"XYZ[\\]" => "[X-\]]"
"-./0]"   => "[]\--0]"
"+*,-^"   => "[\^*--]"
"^_`a"    => "[\^-a]"