Mixtral 8x7B: Sparse MoE Innovation

• Released: 2023
• Architecture: 8 experts, 2 active
• Effective: 12.9B parameters
• Performance: Exceeds Llama 70B
• Speed: 2.5x faster than dense 70B
• Efficiency: Revolutionary performance-per-token