Skip to content
GitLab

Add script to measure generation benchmarks for different models sizes

Merged Administrator requested to merge kchakrabarty/generationbenchmarks into main

Created by: KUNAL1612

Patch Description Added a script that takes in one/many model sizes as input and outputs generation statistics. Addresses #220 (closed). Also allows for custom data to be passed as input.

Testing steps Run this with multiple model sizes to observe WPS counts. Can take multiple models as input and report output for each.

Usage :

python generation_benchmarks.py --model_name 2.7b 13b --padding_size 32

NB: model checkpoint sizes for 66b are huge and the script will require sufficient storage and time tor run.