Program to Queue Files for Copy/Move/Delete in linux?

"Since the script given by Josh Arenberg might have some deadlocking issues (which I did not experience so far, but also have not investigated), I have written up something on my own. It should not have deadlocking problems. It also works for any shell command, not just cp.

Contents of ~/bin/q

#!/bin/bash

#this waits for any PIDs to finish
anywait(){

for pid in ""$@""; do
while kill -0 ""$pid"" 2&>1 >/dev/null; do
sleep 0.5
done
done
}

PIDFILE=~/.q.pid

#open PIDFILE and aquire lock
exec 9>>$PIDFILE
flock -w2 9 || { echo ""ERROR: flock() failed."" >&2; exit 1; }

#read previous instances PID from PIDFILE and write own PID to PIDFILE
OLDPID=$(<$PIDFILE)
echo $$>$PIDFILE

#release lock
flock -u 9

#wait for OLDPID
anywait $OLDPID

#do stuff
""$@""

#afterwards: cleanup (if pidfile still contains own PID, truncate it)
flock -w2 9 || { echo ""ERROR: flock() failed."" >&2; exit 1; }
if [ $(<$PIDFILE) == $$ ]; then
truncate -s0 $PIDFILE
fi
flock -u 9

It creates a chain of processes, each waiting for the previous one. If a process in the middle of the chain crashes while waiting (unlikely but not impossible), the chain is broken and both parts run in parallel. The same happens if one of the processes is killed.

Usage like this:

q $COMMAND $ARGS

or even

q $COMMAND $ARGS; $ANOTHER_COMMAND $MORE_ARGS

Test e.g. by typing

q sleep 10 &
q echo blubb &

and finding that after 10 seconds blubb is printed."

Create a list of files and use SCP to perform the copy. The nice thing in linux is you can add to your text file using echo.

"I found this Unix Batch System project called Task Spooler which let's you queue up tasks.

Arch linux
Ubuntu

Or you can do sudo apt-get install task-spooler

Once installed, you can simply put ts (or tsp for Ubuntu/Debian systems) in front of any regular shell command to queue it up.

This page has lot of examples about it's usages: www.ostechnix.com/add-linux-commands-queue-execute-one-one/ or you watch a video here: www.youtube.com/watch?v=wv8D8wT20ZY

I verified it and it seems to be working as expected for all my cp commands."