MMDeepResearch Model Rankings — Sortable Table

Overall score

Overall: one-number summary (0–100; higher is better). It combines three groups of metrics: FLAE (report quality), TRACE (citation grounding), and MOSAIC (visual grounding) using 0.2·FLAE + 0.5·TRACE + 0.3·MOSAIC.

FLAE — report quality

FLAE evaluates writing quality using three dimensions:

• READ (Readability): how clear, coherent, and easy to read the report is.
• INSH (Insightfulness): whether the report provides meaningful analysis and helpful takeaways (not just restating facts).
• STRU (Structural Completeness): whether the report is well-structured (sections, coverage of requested items, completeness).

Why it matters: This captures how readable and complete the report is—but good writing alone doesn’t guarantee correct grounding.

TRACE — citation grounding

TRACE checks whether cited sources truly support the claims and whether the report stays faithful to the task (including visual requirements):

• VEF (Visual Evidence Fidelity): whether claims that depend on the images actually match what’s in the provided visuals. This is treated as a strict “prompt-faithfulness” check in TRACE (pass/fail thresholded).
• CON (Consistency): whether claims align with their cited sources (not contradicted or mismatched).
• COV (Coverage): how much of the report’s important content is properly supported with citations (few missing/uncited key claims).
• FID (Textual Fidelity): how faithfully a claim matches the cited content—penalizing issues like over‑specific claims, unsupported details, or incorrect cause/effect direction.

Why it matters: High TRACE usually means claims are verifiable and citations are disciplined (fewer “looks supported but isn’t”).

MOSAIC — visual grounding

MOSAIC evaluates whether statements that reference images (charts, tables, diagrams, photos) match the visuals:

• SEM (Visual‑Semantic Alignment): whether the text that references an image matches the image at a meaning/semantic level.
• ACC (Visual Data Interpretation Accuracy): whether the model correctly reads precise visual details (numbers, labels, table cells, chart values).
• VQA (Complex Visual QA / Reasoning): how well the model answers harder, multi‑step questions that require reasoning over visual evidence (e.g., comparisons, trends, derived quantities).

Why it matters: A report can sound plausible yet be wrong if it misreads a chart/table or fails multi-step visual reasoning.

Row tags

Each row also includes:

• Modality:
  • Single‑Modal = text-only.
  • Multimodal = text + images.
  • Deep Research = an agent-style system built for multi-step research/reporting.
• Retrieval:
  • Offline = no web access.
  • Web Search = can browse/search the web.
  • Agent = tool-using research agent workflow.

Quick interpretation tips

• High FLAE + low TRACE: well-written, but citations may not truly support claims.
• High Coverage (Cov.) + low Fidelity (Fid.): lots of citations, but they may be mismatched or over-stretched.
• High Sem. + low Acc.: matches the “gist” of an image but gets exact numbers/details wrong.
• Low Vef.: likely misinterprets the provided visual evidence or violates key visual requirements.