You would have to iterate over the fields and replace the quote character in each. Stéphane shows how to do this using awk in their answer.
$ mlr --csv --fs pipe -N put 'for (k,v in $*) { $[k] = sub(v, "^\047", "") }' file
foo|012|that's nice|bar
This uses Miller (mlr) to read the input as a header-less CSV data set (with pipes as field delimiters). For each record, the put expression iterates over all fields and removes the first character from them if it is a single quote (047 in octal).
Another approach with Miller which uses apply() to apply a function that deletes an initial single quote from a field. The function is applied to each field in each record.
$ mlr --csv --fs pipe -N put '$* = apply($*, func(k,v) { return { k: sub(v, "^\047", "") } })' file
foo|012|that's nice|bar