two-step awk procedure
I have a procedure that works for your example but have had to re-interpret the rules and it is possible it may not work for every case you wish to process if my rules are wrong (apologies if so).
interpretation
The process I was aiming for is:
If the first-opened { style block on a line is closed (}) before another is opened, a new line begins immediately following the closure of that block.
An @media rule can only be at the start of a line.
for a block containing other blocks, and so ending with }}, a new line always follows the double closure.
overview of approach
The procedure uses two awk procedures, the first receiving its input from the data file, the second taking its input from the output of the first. I will first explain their workings separately before giving the combined procedure, joined using a pipe.
first awk step
This first awk step modifies the input line to take account of rule 3 above (that }} can only be at the end of a line). This is achieved by setting the awk field separator to a double closure using FS="}}". A loop is used to print each field (followed by the }} which is otherwise lost) causing awk to separate the fields onto their own lines with its default output record separator ORS (a new line):
awk 'BEGIN{FS="}}"} {for(i=1;i<NF;i++) print $i"}}"; print $NF}' mini.css
This first awk step results in the following output (this does not need to be saved to a file as it will be 'piped' into the second awk procedure later):
.class{margin:0px}.class1,.class2{margin 0px}@media{.class{color:blue}.class1,.class2{color:red}}
@media{.classA.classB,.classC{margin:0px}}
@media{.classD,.classE{color:blue}.class1,.class2{color:red}}
@media only screen and (min-width: 1441px){.classX(color:blue}}
@media only screen and (min-width: 1441px){.class{color:blue}.class1,.class2{color:red}}
@media only screen and (min-width: 1441px){.classA.classB,.classC{margin:0px}}
@media only screen and (inverted-colors){.classD,.classE{color:blue}.class1,.class2{color:red}.classV{color:red}.classR{color:red}.classU{color:red}.classS{color:red}.classT{color:red}}
.classNew{margin: 10px}
Note at this stage, all lines, except the last, terminate with }} and }} never exists internally in a line. rule 3 above is satified by this step.
second awk step
The second awk step receives the above output from the first step via a pipe. It performs two functions:
Firstly, the procedure checks to see if the first opening block { on a line is closed } before another is opened. If it is closed, a new line is introduced for the remainder of the line, unless, for the specific case where a second } occurs after only one opening { as in the third line of the output above @media only screen and (min-width: 1441px){.classX(color:blue}} (this may have been a typo in your original file where the ( following .classX was intended to be {). The check will become irrelevant but harmless if it was a typo and is corrected. (see closing note for correction).
The check is made by dividing the input line into fields separated by {. The field separator is set in a BEGIN block. At the same time the output field separator is set to the same opening brace, to replace { when the fields are printed:
BEGIN{FS="{"; OFS="{"}
Now, if there is an occurrence of } in field 2, then it closes the first block before another was opened and so a line break is inserted after the closure using substitution:
$2~"}"{if (NF>2) sub("}","}\n",$2);}
Note the use of the condition pattern $2~"}", which only applies the action if } appears in field 2. This satifies rule 1 above.
Secondly, the procedure examines each field after the first to see if it contains a media querie, if @ is present a substitution is made to insert a line break before it so that the media querie begins a new line (rule 2).
A final action block in the second awk procedure prints the modified line.
The second procedure in its entirety follows (note this won't function without the output of the first procedure being piped into it or supplied as a file:
awk 'BEGIN{FS="{"; OFS="{"} $2~"}"{if (NF>2) sub("}","}\n",$2);} {for(i=2;i<=NF;++i) sub("@","\n@",$i)} {print}'
whole procedure
Here are the two procedures combined by pipe:
awk 'BEGIN{FS="}}"} {for(i=1;i<NF;i++) print $i"}}"; print $NF}' mini.css | awk 'BEGIN{FS="{"; OFS="{"} $2~"}"{if (NF>2) sub("}","}\n",$2);} {for(i=2;i<=NF;++i) sub("@","\n@",$i)} {print}'
Note the data file, named mini.css in this case, is the argument to the first awk procedure. The output from the first procedure is piped into the second.
test results
(Tested on Mac Terminal using GNU Awk 5.2.0)
This is the output of the single line example data in the question, saved as mini.css, and processed using the combined awk scripts above under whole procedure:
.class{margin:0px}
.class1,.class2{margin 0px}
@media{.class{color:blue}.class1,.class2{color:red}}
@media{.classA.classB,.classC{margin:0px}}
@media{.classD,.classE{color:blue}.class1,.class2{color:red}}
@media only screen and (min-width: 1441px){.classX(color:blue}}
@media only screen and (min-width: 1441px){.class{color:blue}.class1,.class2{color:red}}
@media only screen and (min-width: 1441px){.classA.classB,.classC{margin:0px}}
@media only screen and (inverted-colors){.classD,.classE{color:blue}.class1,.class2{color:red}.classV{color:red}.classR{color:red}.classU{color:red}.classS{color:red}.classT{color:red}}
.classNew{margin: 10px}
note regarding possible typo
If this part of the original data:
@media only screen and (min-width: 1441px){.classX(color:blue}}
was intended to be:
@media only screen and (min-width: 1441px){.classX{color:blue}}
then the if (NF>2) in the second procedure becomes redundant. The working joined procedures, would then be:
awk 'BEGIN{FS="}}"} {for(i=1;i<NF;i++) print $i"}}"; print $NF}' mini.css | awk 'BEGIN{FS="{"; OFS="{"} $2~"}"{sub("}","}\n",$2);} {for(i=2;i<=NF;++i) sub("@","\n@",$i)} {print}'
