Skip to main content
Lab Grimoire
TW EN
Coffee
software

Magika

Google's open-source AI file type detection — the system behind Gmail, Drive, Safe Browsing, and VirusTotal. Uses deep learning to read full file content patterns instead of magic bytes, catching disguised malicious files. ~99% accuracy across 200+ formats, ~5ms per file on CPU, multi-language SDK (Python, Rust, JS, Go). Main limitation: encrypted/compressed files get low confidence, and it's ~100-1000x slower than `file`.

Best For

Security engineers building file upload processing, automated scanning pipelines, or anything that needs to correctly identify what a file actually is — not what its extension or header claims. If you've ever worried about an attacker renaming an ELF binary to have a .jpg extension and slipping it past your filter, this is the tool that addresses that.

How I Actually Use It

For CLI usage, the first thing I do with any suspicious or unknown file is magika -i -s file — this gives me the MIME type and the confidence score in one shot. A confidence score below 0.5 is my signal to treat the file as potentially encrypted or compressed, or as something in Magika's long-tail training coverage.

In Python scanning scripts, I initialize Magika() once and reuse it. After the model loads, each file takes about 5ms. For batch processing a few hundred uploads, that's fast enough. I use the confidence threshold to route files: high confidence mode for security-critical paths (mark anything below threshold as unknown), medium confidence for general scanning, and best-guess when a downstream process can't handle null values.

The --jsonl output piped through jq is useful for quickly spotting files where the detected type doesn't match the claimed extension — that's usually worth a second look.

One thing to know upfront: brew install magika installs the Rust CLI only. If you need the Python API (from magika import Magika), you need a separate pip install magika or pipx install magika. This caught me off guard the first time.

Where It Is Strong

  • ~99% accuracy trained on ~100 million samples. The model identifies formats that would require hand-written rules in libmagic, and adding new formats only requires collecting training data rather than writing rules.
  • Catches disguised files: Changing magic bytes fools file; it doesn't fool full-content statistical analysis. This is the core reason to pick Magika over traditional tools in security contexts.
  • ~5ms per file after model load, fast enough for batch processing. The Rust CLI starts faster than the Python version for shell script usage.
  • Google production-validated: Gmail, Drive, Safe Browsing, VirusTotal all run this. That's a higher operational confidence bar than most open-source security tools clear.
  • Multi-language SDK: Python for application integration, Rust CLI for scripts and CI/CD (no Python dependency), JavaScript/TypeScript for Node, Go WIP.
  • Three confidence modes: high confidence (strict, lower false positive rate), medium confidence (broader coverage), best-guess (always returns a result). The Python API returns the raw score (0-1 float) so you can build your own threshold logic.

Where It Fails

  • Encrypted and compressed content returns low confidence. Once you encrypt something, the statistical features disappear; Magika gives you a low-confidence guess. You'll need a separate handling path for these files.
  • ~5ms is 100-1000x slower than file: For microsecond-latency-sensitive pipelines (real-time packet inspection, etc.), this isn't the right tool.
  • The Python version requires onnxruntime. Not a problem on standard servers, but relevant for edge devices or restricted environments. v1.0.2 removed the numpy direct dependency, which helps.
  • 200+ types is good coverage, but unusual or domain-specific formats have less training data and lower accuracy. Test with your own files before committing to it.

Pricing, Difficulty, and Risk

Open-source, Apache 2.0. No licensing concerns for commercial use. Installation is brew install magika (Rust CLI) or pip install magika (Python). Beginner-friendly — no configuration needed to get started.

Risk profile is low: the model runs locally, no network calls, data doesn't leave your machine. The ONNX Runtime is a known dependency with a good security track record.

Verdict

If you're handling files from untrusted sources and you care about catching disguised malicious files, Magika is the straightforward choice. The Google production validation and 15k stars aren't just marketing — the tool genuinely works. Just remember that brew install magika gets you the Rust CLI, not the Python module, and don't expect it to replace file in latency-sensitive scenarios.

Not the right fit if your workload is dominated by encrypted/compressed archives, or if you're deploying to edge hardware where ONNX Runtime is a problem.

Source

https://github.com/google/magika