Short Answer: Yes
Long Answer:
HMAC is Hash-based message authentication code. You should HMAC anything which you want to authenticate, or in other words, anything which you want to protect against being modified.
Although the RFC standard is more complicated, it may make sense to think of HMAC as a salted hash.
e.g. hmac(message, key) = hash(message + key)
- You can only recreate the same hmac with an identical message and key.
- You can't recreate the same hmac if the key is identical but the message differs.
- You can't recreate the same hmac if the message is identical but the key differs.
An attacker (who doesn't have the HMAC key) cannot modify part of the HMAC message without invalidating the existing HMAC. It really does depend on your data format and your usage of that data to determine what should be included in the HMAC message and HMAC key. But assuming you are using the HMAC to authenticate the decryption, then you should always include in the HMAC message anything that the decryption depends on. The symmetric key is typically used as the HMAC key.
In your quote, the poster says the IV and the algorithm should also be hashed. Consider a file/database format consisting of
ALGORITHM + IV + CIPHERTEXT + HMAC
If you only HMAC the ciphertext, an attacker would be able to modify the algorithm or IV (corrupting the file) without affecting the validity of the HMAC. This is bad because you can end up with a corrupted encrypted file with a valid HMAC. Decryption will proceed as normal because your software will think everything is ok. The result is a totally garbled decryption, but the point is that your software is broken because it returned the wrong output when decrypting and didn't give any errors. This can be classed as a 'security risk' if your application tries to do something with that erroneous data because it assumes it is correct. It is not a security risk in the sense that it makes the underlying encryption weaker or easier to crack. HMAC and symmetric encryption are two totally different technologies doing different things. The point of using a HMAC is that you can assume that the decryption layer is returning data which is 100% correct.
In the above example the ALGORITHM is a dynamic piece of data which I used to explain "algorithm agility" in the OPs quote. It defines what encryption algorithm was used. The point is that it is dynamic so it needs to be read from somewhere rather than hardcoded. This fact makes it a dependency of the decryption so it should be included in the HMAC message. However, if you always use some static algorithm then it should be assumed by (hardcoded in to) your decryption code and there is no need to store this data anyway. There is no need to include static data in the HMAC message because it has no affect on the decryption.
An example of a file format which uses a static algorithm is the open source AES-256 Crypt File Format. The algorithm is consistent and so it is always assumed. It actually uses 2 HMACs for speed reasons. 1 to authenticate the IV and keys, and the 2nd to authenticate the encrypted data part.