llama Data List

라마3(Llama3) 란 | 데이터카페

DDataDeveloper

2024-05-09 14:22:00

/ 목차 /

라마3(Llama3) 란
성능비교

라마3(Llama3) 란
- Meta Ai에서 개발한 대규모 언어 모델(LLM)입니다. 메타에서 오픈소스로 공개 하였으며 상업적용도로 사용 가능하며 누구나 무료로 이용할 수 있는 모델입니다. 

성능비교


  
    　
    Meta Llama 3 8B
    Gemma 7B - It Measured
    Mistral 7B Instruct Measured
  


  
    MMLU 5-shot
    68.4
    53.3
    58.4
  
  
    GPQA 0-shot
    34.2
    21.4
    26.3
  
  
    HumanEval 0-shot
    62.2
    30.5
    36.6
  
  
    GSM-8K 8-shot, CoT
    79.6
    30.6
    39.9
  
  
    MATH 4-shot, CoT
    30
    12.2
    11
  



  
    　
    Meta Llama 3 70B
    Gemini Pro 1.5 Published
    Claude 3 Sonnet Published
  


  
    MMLU 5-shot
    82
    81.9
    79
  
  
    GPQA 0-shot
    39.5
    41.5
    38.5
  
  
    HumanEval 0-shot
    81.7
    71.9
    73
  
  
    GSM-8K 8-shot, CoT
    93
    91.7
    92.3
  
  
    MATH 4-shot, CoT
    50.4
    58.5
    40.5

	Meta Llama 3 8B	Gemma 7B - It Measured	Mistral 7B Instruct Measured
MMLU 5-shot	68.4	53.3	58.4
GPQA 0-shot	34.2	21.4	26.3
HumanEval 0-shot	62.2	30.5	36.6
GSM-8K 8-shot, CoT	79.6	30.6	39.9
MATH 4-shot, CoT	30	12.2	11

	Meta Llama 3 70B	Gemini Pro 1.5 Published	Claude 3 Sonnet Published
MMLU 5-shot	82	81.9	79
GPQA 0-shot	39.5	41.5	38.5
HumanEval 0-shot	81.7	71.9	73
GSM-8K 8-shot, CoT	93	91.7	92.3
MATH 4-shot, CoT	50.4	58.5	40.5

Comment