other
0.77B parameters
Commercial OK
Multimodal
Reviewed November 2026

Florence-2 Large

770M-parameter unified vision foundation model with a DaViT image encoder and BART-style seq2seq decoder. One model, one set of weights — handles captioning, OCR, region/grounding, segmentation, and dense detection via task-prompt tokens. Trained on FLD-5B (5.4B annotations over 126M images).

License: mit·Context: 4,096 tokens
BLK · VERDICT

Our verdict

OP · Eruo Fredoline|UPDATED NOV 8, 2026
Not tested by usEditorial assessment from specifications and published reports.
unrated

Absurd value for 770M params and the most under-rated vision model Microsoft has shipped. Use it when you need many vision tasks on cheap hardware and a chat interface is not the point.

Overview

770M-parameter unified vision foundation model with a DaViT image encoder and BART-style seq2seq decoder. One model, one set of weights — handles captioning, OCR, region/grounding, segmentation, and dense detection via task-prompt tokens. Trained on FLD-5B (5.4B annotations over 126M images).

Strengths

  • One 770M checkpoint for caption / detailed-caption / OCR / OCR-with-region / grounding / detection / segmentation
  • The fine-tuned Florence-2-large-ft (same 0.77B size) beats Flamingo (80B) on COCO caption CIDEr, Ferret (13B) on RefCOCO and GIT2 (5.1B) on TextVQA, per the model card: https://huggingface.co/microsoft/Florence-2-large
  • MIT license, no usage restrictions
  • Tiny by VLM standards — runs in <2GB VRAM at FP16, viable on CPU and edge devices
  • Task-prompt API: <CAPTION>, <OD>, <OCR>, <REFERRING_EXPRESSION_SEGMENTATION>, etc.

Weaknesses

  • Not a conversational VLM — pure task-prompt, no free-form chat
  • Caption outputs are short and factual; verbose narration weaker than Qwen2.5-VL
  • OCR is good for English print but lags GOT-OCR2 on formulas, complex tables, CJK
  • Trust-remote-code required in transformers — extra friction for locked-down deployments

Quantization variants

Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.

QuantizationFile sizeVRAM required
FP161.6 GB2 GB

Get the model

HuggingFace

Original weights

huggingface.co/microsoft/Florence-2-large

Source repository — direct quantization required.

Hardware that runs this

Cards with enough VRAM for at least one quantization of Florence-2 Large.

Compare alternatives

Models worth comparing

Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.

Step up
More capable — bigger memory footprint
Step down
Smaller — faster, runs on weaker hardware
No verdicted models in the next tier down yet.

Frequently asked

What's the minimum VRAM to run Florence-2 Large?

2GB of VRAM is enough to run Florence-2 Large at the FP16 quantization (file size 1.6 GB). Higher-quality quantizations need more.

Can I use Florence-2 Large commercially?

Yes — Florence-2 Large ships under the mit, which permits commercial use. Always read the license text before deployment.

What's the context length of Florence-2 Large?

Florence-2 Large supports a context window of 4,096 tokens (about 4K).

Does Florence-2 Large support images?

Yes — Florence-2 Large is multimodal and accepts text + vision inputs. Vision support requires a runner that handles its image-conditioning architecture.

Source: huggingface.co/microsoft/Florence-2-large

Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.

Related — keep moving

Before you buy

Verify Florence-2 Large runs on your specific hardware before committing money.