No progress after submitting job (custom library)?

Dear VF Team,

I submitted the job for ligand preparation using input_files contain smi files ( ~60) and edited the all.ctrl file as per the documentation. I checked the status of my job. there is no progress. I am not sure whether I should prepare or modify existing todo.all file in VFLP, since it is showing total number of ligand collections: 1144227 in job status.

Could you please suggest me needful ?.

My job status
Number of jobfiles in the workflow/jobfiles/main folder: 5
Number of joblines in the batch system: 5
Number of joblines in the batch system currently running: 0

  • Number of joblines in queue “smp” currently running: 0
    Number of joblines in the batch system currently not running: 5
  • Number of joblines in queue “smp” currently not running: 5
    Number of cores/slots currently used by the workflow: 0

Total number of ligand collections: 1144227
Number of ligand collections completed: 0
Number of ligand collections in state “processing”: 0
Number of ligand collections not yet started: 0

Thank you
-Vas

Hi all,

I deleted pre-existing old files in todo.all.****, except todo.all. and then submitted job again. I overcome the previous error. Now i am getting following errors.

Starting the (re)filling of the todolists of the queues.

Before (re)filling the todolists the queue 10-1-1 had 0 ligands todo distributed in 0 collections.

The ligand-collections/todo/todo.all (if existent) did not meet the requirements for continuation (trial 1).
The ligand-collections/todo/todo.all (if existent) did not meet the requirements for continuation (trial 2).
The ligand-collections/todo/todo.all (if existent) did not meet the requirements for continuation (trial 3).
The ligand-collections/todo/todo.all (if existent) did not meet the requirements for continuation (trial 4).
The ligand-collections/todo/todo.all (if existent) did not meet the requirements for continuation (trial 5).
prepare-todolists.sh: line 453: [: AAAA_00000: integer expression expected

  • Warning: Could not get the length of collection AAAA_00000. Found value is: AAAA_00000. Exiting.
    Error was trapped
    Error in bash script /tmp/slurm/job14610212/slurm_script
    Error on line 273
    Environment variables

QT_PLUGIN_PATH=/usr/lib64/kde4/plugins:/usr/lib/kde4/plugins
OLDIAGS=/opt/dell/onlinediags/oldiags/bin
VF_QUEUES_PER_STEP=1
INCLUDE=/usr/local/intel/17/compilers_and_libraries_2017/linux/include
LMOD_DIR=/usr/local/lmod/lmod/libexec
__LMOD_STACK_MPIFORT=bXBpZm9ydA==
LMOD_COLORIZE=yes
VF_ERROR_SENSITIVITY=normal
XAUTHORITY=/home/srp106/.Xauthority
MPICXX=mpicxx
BASH_FUNC_module()=() { eval $($LMOD_CMD bash “$@”);
[ $? = 0 ] && eval $(${LMOD_SETTARG_CMD:-:} -s sh)
}
BASH_FUNC_ml()=() { eval $($LMOD_DIR/ml_cmd “$@”)
}
OLDPWD=/home/srp106/software/VFPREP/VFLP-develop/tools
_=/usr/bin/env

  • Trying to stop this queue and causing the jobline to fail…

                  *** Final Job Information ***
    

======================================================================

Starting time: Sat Apr 11 03:13:23 EDT 2020
Ending time: Sat Apr 11 03:15:23 EDT 2020

                 *** Final Job Information ***

======================================================================

Dear VF_team.

After fixing the openbabel path variable in my path, I end up the following error.

prepare-todolists.sh: line 453: [: AAAA_00000: integer expression expected

  • echo ’ * Warning: Could not get the length of collection AAAA_00000. Found value is: AAAA_00000. Exiting.’
  • Warning: Could not get the length of collection AAAA_00000. Found value is: AAAA_00000. Exiting.
  • exit 1
  • clean_up
  • cp /dev/shm/srp106/VFLP/t/1/prepare-todolists/todo.all …/…/workflow/ligand-collections/todo/todo.all.locked
  • mv …/…/workflow/ligand-collections/todo/todo.all.locked …/…/workflow/ligand-collections/todo/todo.all
  • rm -r /dev/shm/srp106/VFLP/t/1/prepare-todolists/
  • kill ‘#!’
  • true
    ++ error_response_std 273
    ++ echo ‘Error was trapped’
    Error was trapped
    ++ echo ‘Error in bash script /tmp/slurm/job14614596/slurm_script’
    Error in bash script /tmp/slurm/job14614596/slurm_script
    ++ echo ‘Error on line 273’
    Error on line 273
    ++ echo ‘Environment variables’
    Environment variables
    ++ echo ----------------------------------

I would like share job.out file to investigate on this. Please have a look once and suggest me needful to get rid of this error.
Thanks for your time and consideration.

https://drive.google.com/open?id=1AHsvhY8M2h8pto0a3ZoanmBOGVK5GB8Y

Hi VF Team,

I executed the scripts in my folder, that causing my previous errors. So i made slurm script for HPC to submit the job. Now it is causing the following errors. Please suggest me the needful.

[srp106@hpc4 VFLP-develop]$ tail -f vplp.o14620990
grep: templates/all.ctrl: No such file or directory
grep: templates/all.ctrl: No such file or directory

cp: cannot create regular file ‘…/workflow/job-files/main/1.job’: No such file or directory
Error was trapped which is a nonstandard error.
Error in bash script vf_start_jobline.sh
Error on line 95

Im not very experienced with the code so best is to wait for an answer of Christoph. However, in two of your reactions i see that the lenght of the collection cannot be found. Did you use the prepare command?

Hi Kaneki,
Thanks for reply, I used the prepare command. Looks like that is causing the error. I am not sure why it is causing this error. I check the folder permission to read the files. everything looks fine.

I will wait on Christoph to respond on this.

  • ‘[’ AAAA_00000 -eq AAAA_00000 ‘]’
    prepare-todolists.sh: line 453: [: AAAA_00000: integer expression expected
  • echo ’ * Warning: Could not get the length of collection AAAA_00000. Found value is: AAAA_00000. Exiting.’
  • Warning: Could not get the length of collection AAAA_00000. Found value is: AAAA_00000. Exiting.
  • exit 1

  • clean_up

  • cp /dev/shm/srp106/VFLP/t/1/prepare-todolists/todo.all …/…/workflow/ligand-collections/todo/todo.all.locked

  • mv …/…/workflow/ligand-collections/todo/todo.all.locked …/…/workflow/ligand-collections/todo/todo.all

  • rm -r /dev/shm/srp106/VFLP/t/1/prepare-todolists/

  • kill ‘#!’

  • true
    ++ error_response_std 267
    ++ echo ‘Error was trapped’
    Error was trapped
    ++ echo ‘Error in bash script /tmp/slurm/job14633205/slurm_script’
    Error in bash script /tmp/slurm/job14633205/slurm_script
    ++ echo ‘Error on line 267’
    Error on line 267
    ++ echo ‘Environment variables’
    Environment variables
    ++ echo ----------------------------------

  • echo AAAA_00000

  • no_to_add=AAAA_00000

  • ‘[’ AAAA_00000 -eq AAAA_00000 ‘]’
    prepare-todolists.sh: line 453: [: AAAA_00000: integer expression expected

  • echo ’ * Warning: Could not get the length of collection AAAA_00000. Found value is: AAAA_00000. Exiting.’

  • Warning: Could not get the length of collection AAAA_00000. Found value is: AAAA_00000. Exiting.
  • exit 1
  • clean_up
  • cp /dev/shm/srp106/VFLP/t/1/prepare-todolists/todo.all …/…/workflow/ligand-collections/todo/todo.all.locked
  • mv …/…/workflow/ligand-collections/todo/todo.all.locked …/…/workflow/ligand-collections/todo/todo.all
  • rm -r /dev/shm/srp106/VFLP/t/1/prepare-todolists/
  • kill ‘#!’
  • true
    ++ error_response_std 267
    ++ echo ‘Error was trapped’
    Error was trapped
    ++ echo ‘Error in bash script /tmp/slurm/job14633205/slurm_script’
    Error in bash script /tmp/slurm/job14633205/slurm_script
    ++ echo ‘Error on line 267’
    Error on line 267
    ++ echo ‘Environment variables’
    Environment variables
    ++ echo ----------------------------------

Judging by your error messages, your todo.all file misses the collection length. That is you have a file
AAAA_00000
AAAA_00001
.
.
.
when the tool expects
AAAA_00000 100
AAAA_00001 23
.
.
.
On the other hand, it is not clear to me from the documentation if there should not just be one file per collection, in which case the number would be obsolete. Apperently the tool will still require a “1” in every line or it will give your errror message.

Julian

HI Julian,

Thanks for reply. I agree with you,

In ~/tools/templates folder, I found that todo.all file which have only one column to specify the combination of letters i.e AAAE_XXXXX

Looks like After preparation of folders (with shell script), I supposed to have integers as 2nd Column in todo.all file ( ~/workflow/ligand-collections/todo/todo.all) . Correct me, if i am wrong.

From error, prepare-todolists.sh script is also expecting integer as 2nd column. I am not quite sure … where the problem is ?.

My input-files folder contain only .smi files only.

Try to append your todo.all file manually so that there is an integer <collection_size> behind every collection. Then call vf_prepare_folders That should remove the error you experience

Thank you ! it works after fixing 2nd column with integer. but end-up with following error.

nailgun/tools/google-java-format/
nailgun/tools/google-java-format/LICENSE
nailgun/tools/google-java-format/README.txt
nailgun/tools/google-java-format/google-java-format-1.6-all-deps.jar
Error: Could not find or load main class com.martiansoftware.nailgun.NGServer
Caused by: java.lang.ClassNotFoundException: com.martiansoftware.nailgun.NGServer


lihui1

Julian

5m

Dear Julian,
I have a similar problem. I changed the todo.all file in the tools/templates folder to look like this:
AAAA_00000 1
AAAA_00001 1
AAAA_00002 1

is that all right?
In the documentation given, I did not see a tutorial to perform the vf_prepare_todolists.sh file, and none of the documents mentioned. So,when should I execute it, after vf_prepare_folders. sh or after vf_start_jobline.sh? And I donot find another sh file to execute it .

Looking forward to your reply
Thank you

In VFLP, I use the following commands, in order from top to bottom:

./vf_prepare_folders.sh
./vf_start_jobline.sh 1 8 ./templates/template2.slurm.sh false 2
./vf_prepare_todolists.sh 1 8
./vf_continue_all.sh ./templates/template2.slurm.sh 2

It turned out that under the tempfile path, there is a controlfile file and a directory of 1-1 / packages / chemaxon.
In the / workflow / ligand-collections / todo directory, there is a todo.all-> todo.all.0000, and all todo.all. * File sizes become 0.
And there is an additional directory 1/1 /, which contains 1-1-1,1-1-2 , 1-1-8. Each 1-1-1 * file is between 140,000 and 150,000 lines.However, I put in a smi file with a total molecular number of about 400,000.
The vf_prepare_todolists.sh took a lot of time to execute. But after submitting vf_continue_all.sh, squeue did not find any job.

Please point out my mistake
Looking forward to your reply
Thank you

this is the error in the /workflow/output-files/jobs/ and there is nothing in the /workflow/output-files/queue

  • VF_QUEUE_NO_3=8
  • VF_QUEUE_NO=1-1-8
  • cp ‘/public/home/lihui1/VirtualFlow/TEMP/lihui1/VFLP/t/1-1/1-1-8/workflow/output-files/queues/1/1/queue-1-1-8.’ …/workflow/output-files/queues/1/1/
    cp: cannot stat '/public/home/lihui1/VirtualFlow/TEMP/lihui1/VFLP/t/1-1/1-1-8/workflow/output-files/queues/1/1/queue-1-1-8.
    ’: No such file or directory
  • true
    srun: error: a12r4n12: task 0: Exited with exit code 1
    srun: Terminating job step 1559341.0
  • let exit_code=1
  • ‘[’ 1 == 1 ‘]’
  • error_response_std 294
  • echo ‘Error was trapped’
    Error was trapped
  • echo ‘Error in bash script /opt/gridview/slurm/spool_slurmd/job1559341/slurm_script’
    Error in bash script /opt/gridview/slurm/spool_slurmd/job1559341/slurm_script
  • echo ‘Error on line 294’
    Error on line 294
  • echo ‘Environment variables’
    Environment variables
  • echo ----------------------------------

  • env

###############################################################

jchemsuite/lib/com.chemaxon-alchemist-configbuilder.jar

  • [[ none == \n\o\n\e ]]
  • echo ’ Error: The ChemAxon license file was not specified, but is required…’
    Error: The ChemAxon license file was not specified, but is required…
  • error_response_std 198
  • echo ‘Error was trapped’
    Error was trapped
    ++ basename /public/home/lihui1/VirtualFlow/VFLP/tools/…/workflow/job-files/sub/one-step.sh
  • echo ‘Error in bash script one-step.sh’
    Error in bash script one-step.sh
  • echo ‘Error on line 198’
    Error on line 198
  • echo ‘Environment variables’
    Environment variables

HI lihui,

Looks like you didn’t specify the ChemAxon license file in all.ctrl
From the error : The ChemAxon license file was not specified, but is required.

/public/home/lihui1/VirtualFlow/TEMP/lihui1/VFLP/t/1-1/1-1-8/workflow/output-files/queues/1/1/queue-1-1-8 and in /workflow/output-files/queues/1/1/queue-1-1-8.*
Second error : it is not creating the queue file i.e queue-1-1-8* in above folders.
Reason : Unknown, I do have same problems

Hi vas2201,
Thanks for you reply.
I tried to apply for a license on ChemAxon, but I didn’t get any response. Can I not use Chemaxon, just use openbabel for ligand preparation? It should be feasible in theory.

Hi Lihui,

FYI, if you look at scripts and output closely , if the code exit with error one because of known/unknown reason, it can terminate the all your jobs.

for example 1
srun: error: a12r4n12: task 0: Exited with exit code 1
srun: Terminating job step 1559341.0

example 2

  • let exit_code=1
  • ‘[’ 1 == 1 ‘]’
  • error_response_std 294
  • echo ‘Error was trapped’
    Error was trapped