Norman and Dick inform him that the only thing wrong with this town is subversives like him trying to undermine the American way of life and backstabbing their own neighbors.

"Numbers," Dick says, taking Norman's place, looking at him with an annoyed, resigned sort of displeasure, weight on one hip and arms folded, mouth in a long line and head canted to one side, "Don't make this harder than it has to be. I don't want to have to do this."

It's clearly a last warning before the setup at the short side of the table is brought into the picture.