A hardware security module (HSM) is a digital key vault. Keys are generated inside the HSM and cannot be taken out. Its tamper-proof housing, a multitude of sensors, and different hardware and software layers make sure nobody can get to the keys. Only applications that have the correct access credentials to the HSM can use these keys.
Storing of keys in an HSM is just the start. One has to make sure that they can only be used by adhering to certain rules attached to every key. This makes it impossible for corrupted or hacked applications (or admins) to use the keys, dramatically reducing the risk of having your assets stolen.