Setting and Running VFVS on AWS

VFVS VFVS 1
#1

Hi all,
We are thinking of screening the 1.4B Enamine library on one of our targets. In that effort, we are trying to set up the cluster on AWS cluster.

The cluster we have set up is of this:

  1. Head node: m5.2xLarge 8vCPUS 32Gb (has 8 cores)
  2. Compute node: c5.24xlarge 96 vCPUS 200GB Memory (has 96 cores)
  3. Job scheduler installed: SLURM

The number of compute nodes can be increased. So, how would one set many of the parameters in the all.ctrl files with the above configuration in mind. Here are the parameters that I am not able to understand, and I would appreciate if you could give some clarity. This is the first time I am setting up a cluster and using it on this scale.

  1. partition=test ( I first got an error say partition name test couldn’t be found; After googling, I used sinfo and found that under PARTITION was compute*. So, I changed it to compute
  2. steps_per_job=1
  3. cpus_per_step=1
  4. queues_per_step=1
  5. cpus_per_queue = 1

How to set these parameters ( as an example using the configuration of cluster we have here)

I also tried to run the tutorial from scratch and ran into problems.The specific error I am getting is

sbatch: error: Memory specification can not be satisfied
sbatch: error: Batch job submission failed: Requested node configuration is not available

Hope to hear from you as to how to deal with these things.

Best regards,
Vishwa

1 Like
#3

Dear Vishwa,

Your problem seems to be specific to SLURM and AWS. VirtualFlow runs on a SLURM cluster (whether a local cluster or in the cloud like on AWS), but the SLURM cluster needs to be up and running properly. For help on setting up SLURM in general and on AWS, it will be best if you get directly in touch with their support teams, since we cannot provide support for these platforms.

But what I can say is that when we tested it last year on AWS, it worked without problems. But in the meantime, they might have changed some settings and options.

Good luck,
Christoph