opensource/mateclaw
2
0
Fork 0
mirror of https://gitee.com/mateos/mateclaw.git synced 2026-09-13 03:13:41 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

fix(dingtalk): forward voice messages by reading recognition from stream payload

Browse Source 
The stream SDK delivers voice messages as ChatbotMessage with msgtype=audio
and the server-side ASR result already filled into MessageContent.recognition
(same shape as WeCom's voice.content). The adapter's handleStreamMessage
only read msg.getText(), which is null for audio events, so the message
landed in handleWebhook with no msgtype, fell through to the default text
branch, found null content, and got dropped at 'Empty message content,
ignoring'. From the user's side: send a voice, nothing happens, no log of
the attempt.

Two surgical edits:
- handleStreamMessage now checks getContent().getRecognition() first; if
  present and non-blank, builds payload {msgtype: audio, audio: {recognition}}
  before falling back to the existing text path. The earlier comment about
  richText being handled inside handleWebhook was wrong — picture and
  richText also need their fields propagated through the payload Map; left
  a TODO for them.
- handleWebhook gains an explicit case 'audio' branch that pulls text out
  of audio.recognition and pushes it onto contentParts.
- ChannelMessage.inputMode now reflects 'voice' when msgtype=audio,
  mirroring feishu's behavior so downstream code (memory-extraction
  filters, voice-themed system prompts) can tell text vs voice turns apart.

No STT call required — DingTalk transcribes server-side and ships text in
the webhook, so this is a 0-network, 0-config fix.
This commit is contained in:
matevip 2026-04-28 11:13:13 +08:00
parent acf6eccb3a
commit 5bef83a156
1 changed files with 20 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
mateclaw-server/src/main/java/vip/mate/channel/dingtalk/DingTalkChannelAdapter.java
View File

@ -166,11 +166,19 @@ public class DingTalkChannelAdapter extends AbstractChannelAdapter implements St
payload.put("sessionWebhook", msg.getSessionWebhook());
// 消息内容
if (msg.getText() != null) {
// 钉钉服务端已经把语音转写好放在 MessageContent.recognition 里（跟
// 企业微信 voice.content 一个模式），不需要 STT。优先读 recognition；
// 否则读 text.content。其他复杂类型（picture / richText）暂由
// handleWebhook 内部处理 —— 但 stream 模式下我们目前没把那些类型
// 的字段塞进 payload，是个遗留待修项（picture / richText 同样会掉消息）。
String recognition = msg.getContent() != null ? msg.getContent().getRecognition() : null;
if (recognition != null && !recognition.isBlank()) {
payload.put("msgtype", "audio");
payload.put("audio", Map.of("recognition", recognition));
} else if (msg.getText() != null) {
payload.put("msgtype", "text");
payload.put("text", Map.of("content", msg.getText().getContent() != null ? msg.getText().getContent() : ""));
}
// richText 等复杂类型暂由 handleWebhook 内部处理
handleWebhook(payload);
} catch (Exception e) {
@ -385,6 +393,15 @@ public class DingTalkChannelAdapter extends AbstractChannelAdapter implements St
textContent = textBuilder.toString().trim();
}
}
} else if ("audio".equals(msgtype)) {
// 钉钉服务端已经把语音转写好放在 audio.recognition 里。这跟企业微信
// 的 voice.content 是一个模式 —— webhook 自带 ASR 文本，0 STT 调用。
Map<String, Object> audioBody = (Map<String, Object>) payload.get("audio");
String recognition = audioBody != null ? (String) audioBody.get("recognition") : null;
if (recognition != null && !recognition.isBlank()) {
textContent = recognition.trim();
contentParts.add(MessageContentPart.text(textContent));
}
} else {
// 默认 text 消息
Map<String, Object> msgBody = (Map<String, Object>) payload.get("text");
@ -424,6 +441,7 @@ public class DingTalkChannelAdapter extends AbstractChannelAdapter implements St
.content(content)
.contentType(contentParts.stream().anyMatch(p -> "image".equals(p.getType())) ? "image" : "text")
.contentParts(contentParts)
.inputMode("audio".equals(msgtype) ? "voice" : "text")
.timestamp(LocalDateTime.now())
.rawPayload(payload)
.build();