A bit of both. A person's personality stems from how they emotionally react to a given situation. Without a soul's potency, there are no emotions. They would be unfeeling and unable to understand the feelings of those around them.

No more than a tenth of potency can be safely removed. A person will be unaffected by that. With less than half of it removed, a person can still recover, but it may take years of relearning how to understand and cope with emotions until they are the same again...

More than half and it is irreversible, unless I were to die.