L
L
Luther Yezhov2017-06-05 15:39:23
linux
Luther Yezhov, 2017-06-05 15:39:23

How can a parallel program call be converted to serial through a task queue?

Given: the assembly make -j8gives a significant increase in speed during compilation, but when the ld linker is running, the memory is eaten up to the point that swap is almost completely filled.
Required: to find a way to call linkers not in parallel, but sequentially, keeping compilers running in parallel.
Maybe there are ready-made solutions or do you have to make a request queue manager to the linker yourself?
OS: Linux (CentOS 7)
UPD:
Solution in the comments, use flock The
following script was born:

neck
#!/bin/bash
NAME=`basename $0`
DIR=`dirname $0`
[email protected]
PATH=${PATH//$DIR/''}

if [ $NAME == 'neck' ]
then
  exit 0
fi

function get_origin {
  ORIGIN=`which --skip-alias --skip-functions $NAME 2>&1`
  local res=$?

  if [ $res -ne 0 ]
  then
    echo $res
    echo "Origin file $NAME not found"
    exit 1
  fi
}

function get_lock_file {
  LOCK_FILE="/tmp/${ORIGIN//\//_}.lock"
}

get_origin
get_lock_file
flock $LOCK_FILE $NAME $ARGS
exit $?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
M
Maxim Moseychuk, 2017-06-05
@Louter

If you have one target (one program or library), then the linker will be called once anyway.
And if we have several goals, then you can specify the dependency between them, then they will be collected in turn.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question