The code is working properly. You have mixed up your error messages. The code should be:
#if defined(TEST)
#error "DEFINED"
#else
#error "NOT DEFINED"
#endif
UPDATE: The question was edited after the time of this answer to correct the mixed up error messages.