ESP32S3+IDF环境下websocket连接阿里百炼实时多模态大模型

汇干环 · 3 天前

ESP32S3+IDF环境下websocket连接阿里百炼实时多模态大模型

首先贴一个阿里百炼实时多模态大模型的文档（找对这个文档花了我不少时间）：
大模型服务平台百炼控制台
文档给出了一个 websocket python 示例，我们要做的是参考并移植到C环境下：

# pip install websocket-client
import json
import websocket
import os
API_KEY=os.getenv("DASHSCOPE_API_KEY")
API_URL = "wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=qwen-omni-turbo-realtime"
headers = [
"Authorization: Bearer " + API_KEY
]
def on_open(ws):
print(f"Connected to server: {API_URL}")
def on_message(ws, message):
data = json.loads(message)
print("Received event:", json.dumps(data, indent=2))
def on_error(ws, error):
print("Error:", error)
ws = websocket.WebSocketApp(
API_URL,
header=headers,
on_open=on_open,
on_message=on_message,
on_error=on_error
)
ws.run_forever()

复制代码

根据这个示例，我们得出需要以下两个参数：

API_URL = "wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=qwen-omni-turbo-realtime"
headers = [ "Authorization: Bearer " + API_KEY]

所以改写进ESP32-idf环境：

#include "esp_websocket_client.h"
void websocket_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data);
void websocket_init(void)
{
esp_websocket_client_config_t ws_cfg = {
.uri = "wss://dashscope.aliyuncs.com/api-ws/v1/realtime?model=qwen-omni-turbo-realtime", // 使用带鉴权参数的URL
.transport = WEBSOCKET_TRANSPORT_OVER_SSL, // 明确使用SSL传输
.reconnect_timeout_ms = 10000, // 重连超时设置
};
client = esp_websocket_client_init(&ws_cfg);
esp_websocket_client_append_header(client, "Authorization", "Bearer sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXX");
esp_websocket_register_events(client, WEBSOCKET_EVENT_ANY, websocket_event_handler, NULL);
esp_websocket_client_start(client);
}
void websocket_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data)
{
esp_websocket_event_data_t *data = (esp_websocket_event_data_t *)event_data;
switch (event_id) {
case WEBSOCKET_EVENT_CONNECTED:
ESP_LOGI("WS", "连接成功");
vTaskDelay(4000 / portTICK_PERIOD_MS); // 延迟2秒再启动任务
break;
case WEBSOCKET_EVENT_DATA:
printf("收到数据: %s\n", (char*)data->data_ptr);
break;
case WEBSOCKET_EVENT_DISCONNECTED:
ESP_LOGI("WS", "连接断开，尝试重连...");
break;
}
}

复制代码

连接成功后得到以下打印：

I (8740) WS: 连接成功
收到数据: {"event_id":"event_R6ALkF8t1bCgkkNymZvvT","type":"session.created","session":{"object":"realtime.session","model":"qwen-omni-turbo-realtime","modalities":["text","audio"],"instructions":"...model instructions here...","voice":"Chelsie","input_audio_format":"pcm16","output_audio_format":"pcm16","input_audio_transcription":{"model":"gummy-realtime-v1"},"turn_detection":{"type":"server_vad","threshold":0.5,"prefix_padding_ms":300,"silence_duration_ms":800,"create_response":true,"interrupt_response":true},"tools":[],"tool_choice":"auto","temperature":0.8,"max_response_output_tokens":"inf","id":"sess_CFP8GFog5dp8RGOrhoJY3"}}

连接成功！

来源：程序园用户自行投稿发布，如果侵权，请联系站长删除
免责声明：如果侵犯了您的权益，请联系站长，我们会及时删除侵权内容，谢谢合作！

账号		自动登录	找回密码
密码			立即注册