Created by: lilisierrayu
Patch Description Tis patch fixes a bug when using metaseq_internal.eval for generation task. Currently, sequence_generator returns a {"token": 3d torch.Tensor, "score": 3d torch.Tensor}, while metaseq_internal.eval expects List[List[{"token": 1d torch.Tensor, "score": 1d torch.Tensor}]].
Testing steps Test on any generation task.
CUDA_VISIBLE_DEVICES=0,1 python -m metaseq_internal.eval.gpt3_eval --results-dir {out_dir} --model-name {model_name} --tasks ni_v2__task1393_superglue_copa_text_completion --distributed-port 1234 --distributed-world-size 2 --fsdp --n 2