Find all user-installed packages

"Look at these files,

'/var/log/installer/initial-status.gz' -- your primary installation

this file date would be your installation date (i think)
'/var/log/dpkg.log' update timeline (this is what you want)
'/var/log/apt/term.log' -- things apt updated on your system
'/var/cache/apt/archives/' will contain the deb packages downloaded for installation

Update: use the following two steps for exact list of new installs:

execute: grep -w install /var/log/dpkg.log > full-list.log
Look at lines beyond the /var/log/installer/initial-status.gz timestamp

Since you want to get a clean installation on another system with these packages, you could even copy the 'deb' files from the 'cache/apt/archives' path to that of the new installation and get them installed in one shot (without downloading them again)."

"Based on the information above, I wrote a short Python script to list packages that were manually installed. See this link.

Feel free to use it although I assume no responsibility for it.
However, feedback and suggestions are always welcome."

"This is a hack-job, but it completely works.

First, go to http://releases.ubuntu.com/maverick/ (or whatever version of Ubuntu you're using) and grab the *.manifest file that is associated with the version of Ubuntu you're using.

Then, run the following script (replacing <manifest file>, angle brackets and all, with the path to the file you downloaded. You can always append > output to the end to make a file dump.

diff --suppress-common-lines <(sed 's/ .*//' <manifest file>) <(dpkg --get-selections | sed 's/[ \t].*//') | grep '>' | sed 's/[>] //'"

"Thanks geekosaur, nice code. I used it but it took a while to figure out how to get it working. Here's how I did it in Ubuntu 11.10—it works in the bash terminal:

comm -13 \
<(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort) \
<(comm -23 \
<(dpkg-query -W -f='${Package}\n' | sed 1d | sort) \
<(apt-mark showauto | sort) \
) > user-installed-packages

Then to add a tab—\t—and install on each line:

sed 's/$/\tinstall/' user-installed-packages >uip

Then on the new machine:

sudo dpkg --set-selections < uip

And to install the packages:

sudo apt-get dselect-upgrade"

"Another way to do this is by determining what has been installed based on your ""tasks"" which determine the base packages to install according to your initial needs.

tasksel --list-tasks

At the very least you'd have server. However, you may choose to have more. For each of those tasks you have installed, you can get a list of packages that are installed the following command does it all in one line (broken down for clarity) for the ones I have chosen in my installation:

(tasksel --task-packages server ; \
tasksel --task-packages openssh-server ; \
tasksel --task-packages lamp-server) | sort | uniq

A generic approach to the above would be:

(for a in $( tasksel --list-tasks | grep ""^i"" | awk '{ print $2 }' ) ; \
do tasksel --task-packages $a; done) | sort | uniq

Now use apt-cache depends --installed -i --recurse <packagename> | grep -v ""^ "" to get a list of dependencies used by all the packages defined in the task. This can be done in one line as follows

apt-cache depends --installed -i --recurse \
$(for a in $( tasksel --list-tasks | \
grep ""^i"" | \
awk '{ print $2 }' ) ; \
do tasksel --task-packages $a; done) | grep -v ""^ "" | sort | uniq

The following lists all the packages that are installed in your system (not including dependencies).

dpkg --get-selections | grep ""[[:space:]]install"" | awk '{print $1}'

Now use the comm command to find the ones that are in the second list only (i.e. ignore those that are in both files and just the first file)

comm -13 <(apt-cache depends --installed -i --recurse \
$(for a in $( tasksel --list-tasks | \
grep ""^i"" | \
awk '{ print $2 }' ) ; \
do tasksel --task-packages $a; done) | grep -v ""^ "" | sort ) \
<( dpkg --get-selections | grep ""[[:space:]]install"" | \
awk '{print $1}' | sort)"