
apt-mark for example doesn't include the xserver-xorg package, while the other file does.

Very few packages still fall through the cracks, although I suspect these are actually installed by the user, either right after the installation through the language localization setup or e.g. Using aptitude: comm -23 <(aptitude search '~i !~M' -F '%p' | sed "s/ *$//" | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) Using apt-mark: comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u) Both yield the exact same output on my machine and are more precise than all solutions proposed up until now (July 6, 2014) in this question.

You can use either of these two one-liners.
