Remove an empty directory with ssh

Remove an empty directory with ssh

Why rm dirname doesn't work ? It says it's a directory, well that I know :) but I want to delete it not even recursively as it is empty.