When using apt-get remove or aptitude remove the configuration files of the packages uninstalled are not deleted.

Now you can manually do it yourself with apt-get purge , or, you can use the dpkg command to do it in a nice oneliner.

dpkg --get-selections | grep deinstall | sed 's/deinstall/\lpurge/' | dpkg --set-selections; dpkg -Pa