MiniMax 2.5 vs Llama 3.1 vs DeepSeek-R1 Comparison
Sitepoint did a comparison of local coding models to see which ones are best for coding. They run the models through 4 coding tasks:
- Generating a complete, correct python function from a one-shot prompt.
- Given a code block with a deliberate bug, let the LLM find it, explain it, and fix it.
- Given a poorly structured block of code with key code smells, produce a refactored version that preserves correct behavior but fixed readability and performance
- Provide three related files, answer questions about the cross-file dependencies
Almost all of them require a NVidia Blackwell 6000 or better to run since more require at least 50gb of RAM to run. Here were their results:
| Dimension | MiniMax 2.5 | Llama 3.1 405B | DeepSeek-R1 |
|---|---|---|---|
| Best Task Category | Code refactoring | Function generation & multi-file context | Bug detection & debugging |
| Avg Tokens/sec (dual RTX 3090) | 17.5 | 7.8 | 9.8 |
| Min VRAM Requirement | ~46 GB (dual GPU + partial CPU offload) | ~48 GB+ (dual GPU + heavy CPU offload) | ~44 GB (dual GPU + heavy CPU offload) |
| Composite Rank Across 4 Tasks | 1st or 2nd on 3 of 4 tasks | 1st on 2 of 4 tasks; highest peak quality | 1st on debugging; competitive elsewhere |