The speaker doesn't allow anyone to label him as either poor, or sad, or unhappy. Nobody's position in this world is secure. Therefore, nobody should estimate other people's state. In reality, everything is relative and we aren't capable of objectivity. The speaker, however, defends his right to "hope for heaven", although the heaven might mean nothing to someone else.