Community Integration：Using OPT with Colossal-AI

Created by: binmakeswell

🚀 Feature Request

Motivation

Thank you for your outstanding contribution to open OPT weights.

We have recently integrated OPT with the Colossal-AI system and would like to build a more formal integration with Metaseq.

Colossal-AI is a unified deep learning system for the big model era, which integrates many efficient techniques like multi-dimensional tensor parallelism, sequence parallelism, heterogeneous memory management, etc.

By using Colossal-AI, we could help users to efficiently and quickly deploy large AI model training and inference, reducing large AI model budgets and scaling down the labor cost of learning and deployment.

Pitch

Like the integration with Hugging Face and Alpa community, we want to contribute to the use and deployment of OPT models with Colossal-AI, making less hardware resources for a more efficient and simple large-scale training and inference possible.

We have provided an example of OPT and Colossal-AI, with a corresponding blog. A detailed tutorial will be added very soon. We can provide a draft PR for you to finish this integration and other necessary steps or content. We are also willing to assist you in any other way we can.

Thank you for your review and look forward to your reply.