如何让AI真正“读懂”人心，理解对话背后的真实意图？

先来查看下面这个案例⬇️

在这张相亲对话截图中，暴露了多模态大模型（MLLMs）的两大认知错误：

1、全局上下文理解不足

• 模型只听到了“可以做朋友”的表面善意，却完全忽略了“头巾=不是基督徒=不约会”这一核心的、隐含偏见的前提逻辑链。

2、推理路径简单，缺乏深度逻辑推理

• 在判断男士情绪时，模型可能完全依赖了“可以做朋友”这句看似积极的话，忽视了潜在的视觉、听觉线索：男士可能出现的嘴角下垂/眼神回避、女士说"可能不行"后的3秒沉默、人类社交中的矛盾信号。

这些缺陷导致模型无法全面捕捉隐藏在多模态信息中的深层逻辑，从而做出片面甚至错误的判断。为了解决这些问题，我们推出了 HumanOmniV2 ，一个专为突破多模态推理瓶颈而设计的全新模型。（文末查看HumanOmniV2分析全流程）

HumanOmniV2 的一大核心创新在于引入了强制性上下文总结机制。

在生成最终答案之前，模型必须首先输出一个<context>标签内的上下文概括。这种结构化的设计确保模型不会跳过任何关键信息，而是系统性地分析视觉、听觉和语言信号，构建出完整的场景背景。

为实现这一目标，我们要求模型在推理之前，首先在一个 <context> 标签内输出其对上下文信息的概括。这种强制性的上下文理解能够有效避免模型跳过关键的多模态输入信息，并在随后的推理过程中提供全面的全局背景支持。

例如，在上述相亲对话案例中，模型会主动识别出男士佩戴头巾这一视觉线索，并结合后续对话内容，推断出可能涉及的文化或宗教背景，从而避免因忽略隐含偏见而导致错误判断。

除了结构化的上下文理解，HumanOmniV2 还通过一套 LLM 驱动的多维度奖励机制来引导模型的学习过程。这套机制包括：

• 上下文奖励（Context Reward）：评估模型是否准确捕捉了多模态输入的整体语境；

• 格式奖励（Format Reward）：确保模型输出符合结构化要求；

• 准确性奖励（Accuracy Reward）：提升模型回答的正确率；

• 逻辑奖励（Logical Reward）：激励模型使用反思、归纳、演绎等高级推理方式。

这些奖励机制协同作用，使得 HumanOmniV2 在训练过程中不断优化其对多模态信息的理解能力，同时鼓励其发展出更复杂的推理路径。

为了让模型具备更强的逻辑推理能力，我们在训练策略上也进行了深度优化。基于 Qwen2.5-Omni-Thinker 架构，我们对 GRPO（Group Relative Policy Optimization）方法进行了多项改进：

• 引入令牌级损失（Token-level Loss），解决长序列训练中常见的不平衡问题；

• 移除问题级归一化项 ，避免不同难度样本之间的权重偏差；

• 应用动态 KL 散度机制，在训练初期鼓励探索，在后期稳定收敛，全面提升模型的泛化能力和训练稳定性。

此外，我们还构建了一个高质量的全模态推理训练数据集，其中包含大量图像、视频和音频任务，并附带详细的上下文总结和推理路径标注。这些数据为模型的冷启动训练和强化学习阶段提供了坚实基础。

为了更准确地评估模型在理解人类意图方面的能力，我们推出了全新的评测基准——IntentBench 。该基准包含 633 个视频和 2689 个相关问题，紧密关联视频中的听觉和视觉线索，要求模型不仅要理解全局上下文，还需关注细致观察和复杂社会关系的推理。与以往侧重于感知层面的测试集（如 Daily-Omni 和 WorldSense）相比，IntentBench 更聚焦于评估 AI 对人类行为动机、情感状态和社会互动的深层理解能力。

为了验证 HumanOmniV2 在理解和推理人类复杂意图方面的能力，我们将其在多个主流全模态基准上进行了系统评估。结果显示，HumanOmniV2 在所有测试任务中均表现出色，尤其在新构建的 IntentBench 基准上取得了令人瞩目的成绩。

在 Daily-Omni 上，HumanOmniV2 达到了 58.47% 的准确率，在 WorldSense 上则达到了 47.1% ，而在我们全新推出的、更加注重“深层意图理解”的 IntentBench 上，模型更是取得了高达 69.33% 的准确率。

这些性能提升，得益于我们在模型架构、训练策略以及数据构建上的多项创新设计。HumanOmniV2 通过引入强制上下文总结机制、LLM驱动的多维度奖励体系，以及基于 GRPO 的优化训练方法，在多模态推理方向上进行了有益探索。这一进展标志着 AI 在理解和建模人类复杂意图方面迈出关键一步，为后续研究与应用提供了重要参考。

现阶段 HumanOmniV2 已开源，如果你也对多模态推理感兴趣，欢迎前往以下链接查看论文、了解训练数据及原代码。

➡️ Github

https://github.com/HumanMLLM/HumanOmniV2

➡️ Arxiv

https://arxiv.org/abs/2506.21277

➡️ ModelScope

https://modelscope.cn/models/iic/humanomniv2

➡️ HuggingFace

https://huggingface.co/PhilipC/HumanOmniV2

➡️ IntentBench

https://huggingface.co/datasets/PhilipC/IntentBench

 

Q：Why does the woman sitting closest to the man roll her eyes?

A. She is offended because she feels accused of wrongdoing.

B. The woman is checking her phone.

C. She is admitting that she agrees with what the man is saying.

D. The woman is frustrated with the man because she misunderstands what he's saying.

<context>The woman on the left is wearing a black hat, a denim jacket, and a light blue top, with a noticeable necklace. She is smiling and looking at the paper, occasionally nodding and using gestures to assist in her expression. The woman in the middle is wearing a black hat, a denim jacket, and a light blue top, with a noticeable necklace. She is smiling and looking at the paper, occasionally nodding and using gestures to assist in her expression. The man on the right is wearing a red and black striped hat, glasses, and a black T-shirt, with his arms crossed. He is smiling and looking at the woman in the middle. The three are sitting on a white sofa, with a relaxed and friendly atmosphere, and the background includes a lamp and a pillow.

The conversation involves a lot of fantasy elements, such as "mystery school," "spells," and "giving people poison." The tone of the woman in the middle is rapid and high-pitched, while the man on the right responds in a low and gentle tone. The conversation includes many exaggerated expressions (such as "Oh my God" and "You really eased me into that"), with a clear sense of humor and playfulness.The man on the right does not directly refute the content of the conversation but uses laughter and other responses to ease the atmosphere.</context ><think>Okay, I'm trying to understand why the woman on the left rolled her eyes. I need to look at what she's doing, what the others are saying, and how they're saying it.

First, let me focus on her. I see her roll her eyes, and she's also smiling and looking at a piece of paper.She seems to be nodding and using her hands to help explain something. This makes me think she's not really upset or angry. Her body language, the smile, and the way she's interacting with the paper and her gestures all point to her being engaged, not defensive or frustrated.

Now, let me listen to what they're saying. The man on the right is talking about "giving people poison." That sounds pretty serious. But then, I hear the woman in the middle laugh. She also says things like "You really eased me into that," which sounds like she's making light of the situation. The woman on the left is also smiling and seems to be taking it all in. So, the conversation, despite the "poison" comment, doesn't seem to be a serious argument. It feels more like a playful or exaggerated discussion.

So, putting it all together:

*The eye-rolling happens right when the man mentions "giving people poison." But her smile and the overall friendly vibe of the scene make me think it's not a serious accusation. It feels more like a playful or exaggerated reaction.

*I don't see her looking at a phone; she's clearly focused on the paper and the conversation.

*She's not denying anything or agreeing too strongly; her eye-rolling and smile seem to be a way to handle the potentially sensitive topic.

*And it doesn't seem like she's confused; the conversation flows, and everyone seems to be getting along.

Therefore, I think her eye-rolling is more of an exaggerated, playful reaction to a potentially sensitive topic, rather than a sign of actual anger or confusion.</think><answer>C</answer>

? 文末福利

你认为 HumanOmniV2 可以用在哪些场景中？

欢迎评论区留言，点赞排名第一可获得定制POLO

活动截止时间：2025年7月11日17:00

推荐阅读

ThinkSound：你的专业音效师

如何用泡面预算，喂饱一个手机AI