Finally figured out the problem
I need to run the salloc with first with
salloc -N 1 --qos interactive --time 02:00:00 --gpus=4 --constraint gpu --account=
So all the 4 gpus are visible,
and then i run the script once the node is allocated.
srun -n 4 python dist.py