Oracle SQL (as opposed to PL/SQL) doesn't have a boolean type, so TRUE isn't recognised in the SQL statement (standalone, or if it happens to be within a PL/SQL block).
You can use a dummy expression that always evaluates to true, e.g.:
INSERT WHEN (1=1) THEN
INTO ...
Select ...
Quick SQL Fiddle demo.
Unless this is a 'fixed' part of a larger INSERT ALL construct, the clause seems rather redundant if you know it always has to be true; if you aren't evaluating something real, the WHEN clause isn't adding anything useful. If you want to use it as a flag to control the behaviour then your stringtest approach would probably be easier since you can change that setting in one place - in the declare section - rather than hunting through and changing all the individual WHEN clauses.
insert into some table select ... where stringest = 'yes'