There’s no justification or reason to grant the rights of free speech to those who would deny it to others.

A threat of violence indicates that the threatening party doesn’t respect the right of the other party to speak freely. There’s no free speech when you’re in fear for your life, or dead.