Ever wondered how much it would cost to deploy an LLM (Llama 2 or Mistral 7b) on AWS for production? AdithyaSK tested it out by deploying it himself, and we’ll be sharing the details in the blog post. The Production LLama 2 AWS Environment Cost comes to $1.5k per month.
Read related article where we compare Mistral 7B vs LLaMa.
AWS Environment Cost Calculations
1 endpoint x 1 instance per endpoint x 24 hours per day x 30 days per month = 720.00 SageMaker Real-Time Inference hours per month 720.00 hours per month x $2.03 per hour instance cost = $1,461.60 (monthly On-Demand cost).
Total cost for Inference (monthly): $1,461.60
Here is the link to the AWS Cost Estimation Report: AWS Cost Estimate
Read related articles: