Created by: Xirider
Continuation of https://github.com/facebookresearch/metaseq/pull/476 . As metaseq-internals unification PR was not merged, a few other features got added to metaseq-internal's sweep and slurm. I brought these here into metaseq.
Note: Gpu tests are broken in main since (https://github.com/facebookresearch/metaseq/pull/497), so I tested manually