Alibaba Cloud Model Studio:CosyVoice speech synthesis Python SDK

User guide: For model introductions and selection recommendations, see Speech synthesis - CosyVoice.

• You have activated Alibaba Cloud Model Studio and obtained an API key. Configure the API key as an environment variable instead of hard coding it in your code to prevent security risks that can result from code exposure.

  Note

  When you need to provide temporary access permissions to third-party applications or users, or want to strictly control high-risk operations such as accessing or deleting sensitive data, we recommend that you use temporary authentication tokens.

  Compared with long-term API keys, temporary authentication tokens provide higher security. They have shorter validity periods of 60 seconds, which makes them suitable for temporary call scenarios and effectively reduces the risk of API key leakage.

  Usage: In your code, replace the API key with the temporary authentication token to perform authentication.

• Install the latest version of the DashScope SDK.

Text length limits

• Non-streaming calls (Synchronous call or Asynchronous call): The text length in a single request cannot exceed 2,000 characters.

• Streaming calls: The text length in a single request cannot exceed 2,000 characters, and the total text length cannot exceed 200,000 characters.

SSML support

The Speech Synthesis Markup Language (SSML) feature is currently available only for some voices of the cosyvoice-v2 model. You can check the voice list to confirm SSML support. The following conditions must be met:

• You must use DashScope SDK 1.23.4 or a later version.

• Only synchronous call and asynchronous call (that is, the call method of the SpeechSynthesizer class) are supported. Streaming call (that is, the streaming_call method of the SpeechSynthesizer class) are not supported.

• The usage is the same as for standard speech synthesis. You can pass the text containing SSML to the call method of the SpeechSynthesizer class.

Synchronous call

You can submit a single speech synthesis task and obtain the full result at once without using a callback function. No intermediate results are streamed.

You can instantiate the SpeechSynthesizer class, bind the request parameters, and call the call method to synthesize the audio and obtain the binary data.

The length of the sent text cannot exceed 2,000 characters. For more information, see the call method of the SpeechSynthesizer class.

# coding=utf-8

import dashscope
from dashscope.audio.tts_v2 import *

# If you have not configured the API key in your environment variables, replace "your-api-key" with your own API key.
# dashscope.api_key = "your-api-key"

# Model
model = "cosyvoice-v2"
# Voice
voice = "longxiaochun_v2"

# Instantiate SpeechSynthesizer and pass request parameters such as model and voice in the constructor.
synthesizer = SpeechSynthesizer(model=model, voice=voice)
# Send the text to be synthesized and get the binary audio data.
audio = synthesizer.call("What is the weather like today?")
print('[Metric] Request ID: {}, First package delay: {} ms'.format(
    synthesizer.get_last_request_id(),
    synthesizer.get_first_package_delay()))

# Save the audio to a local file.
with open('output.mp3', 'wb') as f:
    f.write(audio)

Asynchronous call

You can submit a single speech synthesis task and stream the intermediate results through a callback. The synthesized result is obtained through the callback functions in ResultCallback.

You can instantiate the SpeechSynthesizer class, bind the request parameters and the ResultCallback interface, call the call method to start synthesis, and receive the result in real time through the on_data method of the ResultCallback interface.

The length of the sent text cannot exceed 2,000 characters. For more information, see the call method of the SpeechSynthesizer class.

# coding=utf-8

import dashscope
from dashscope.audio.tts_v2 import *

from datetime import datetime

def get_timestamp():
    now = datetime.now()
    formatted_timestamp = now.strftime("[%Y-%m-%d %H:%M:%S.%f]")
    return formatted_timestamp

# If you have not configured the API key in your environment variables, replace "your-api-key" with your own API key.
# dashscope.api_key = "your-api-key"

# Model
model = "cosyvoice-v2"
# Voice
voice = "longxiaochun_v2"


# Define the callback interface.
class Callback(ResultCallback):
    _player = None
    _stream = None

    def on_open(self):
        self.file = open("output.mp3", "wb")
        print("Connection established: " + get_timestamp())

    def on_complete(self):
        print("Speech synthesis complete. All results received: " + get_timestamp())

    def on_error(self, message: str):
        print(f"An error occurred during speech synthesis: {message}")

    def on_close(self):
        print("Connection closed: " + get_timestamp())
        self.file.close()

    def on_event(self, message):
        pass

    def on_data(self, data: bytes) -> None:
        print(get_timestamp() + " Binary audio data length: " + str(len(data)))
        self.file.write(data)


callback = Callback()

# Instantiate SpeechSynthesizer and pass request parameters such as model and voice in the constructor.
synthesizer = SpeechSynthesizer(
    model=model,
    voice=voice,
    callback=callback,
)

# Send the text to be synthesized and receive the binary audio data in real time in the on_data method of the callback interface.
synthesizer.call("What is the weather like today?")
print('[Metric] Request ID: {}, First package delay: {} ms'.format(
    synthesizer.get_last_request_id(),
    synthesizer.get_first_package_delay()))

Streaming calls

You can submit text multiple times within a single speech synthesis task and receive the synthesized result in real time through a callback.

1. Instantiate the SpeechSynthesizer class.

   Instantiate the SpeechSynthesizer class and bind the request parameters and the ResultCallback interface.

2. Stream the text.

   Stream the text by calling the streaming_call method of the SpeechSynthesizer class multiple times to send the text to be synthesized to the server in segments.

   While you send the text, the server returns the synthesized result to the client in real time through the on_data method of the ResultCallback interface.

   The length of the text fragment sent in each call to the streaming_call method (the text parameter) cannot exceed 2,000 characters, and the total length of all text sent cannot exceed 200,000 characters.

3. You can end the process.

   End the process by calling the streaming_complete method of the SpeechSynthesizer class to end the speech synthesis.

   This method blocks the current thread until the on_complete or on_error callback of the ResultCallback interface is triggered. After the callback is triggered, the thread is unblocked.

   Make sure to call this method. Otherwise, the final part of the text may not be converted to speech.

# coding=utf-8
#
# pyaudio installation instructions:
# For macOS, run the following commands:
#   brew install portaudio
#   pip install pyaudio
# For Debian/Ubuntu, run the following commands:
#   sudo apt-get install python-pyaudio python3-pyaudio
#   or
#   pip install pyaudio
# For CentOS, run the following commands:
#   sudo yum install -y portaudio portaudio-devel && pip install pyaudio
# For Microsoft Windows, run the following command:
#   python -m pip install pyaudio

import time
import pyaudio
import dashscope
from dashscope.api_entities.dashscope_response import SpeechSynthesisResponse
from dashscope.audio.tts_v2 import *

from datetime import datetime

def get_timestamp():
    now = datetime.now()
    formatted_timestamp = now.strftime("[%Y-%m-%d %H:%M:%S.%f]")
    return formatted_timestamp

# If you have not configured the API key in your environment variables, replace "your-api-key" with your own API key.
# dashscope.api_key = "your-api-key"

# Model
model = "cosyvoice-v2"
# Voice
voice = "longxiaochun_v2"


# Define the callback interface.
class Callback(ResultCallback):
    _player = None
    _stream = None

    def on_open(self):
        print("Connection established: " + get_timestamp())
        self._player = pyaudio.PyAudio()
        self._stream = self._player.open(
            format=pyaudio.paInt16, channels=1, rate=22050, output=True
        )

    def on_complete(self):
        print("Speech synthesis complete. All results received: " + get_timestamp())

    def on_error(self, message: str):
        print(f"An error occurred during speech synthesis: {message}")

    def on_close(self):
        print("Connection closed: " + get_timestamp())
        # Stop the player.
        self._stream.stop_stream()
        self._stream.close()
        self._player.terminate()

    def on_event(self, message):
        pass

    def on_data(self, data: bytes) -> None:
        print(get_timestamp() + " Binary audio data length: " + str(len(data)))
        self._stream.write(data)


callback = Callback()

test_text = [
    "The streaming text-to-speech SDK ",
    "can convert input text ",
    "into binary speech data. ",
    "Compared to non-streaming speech synthesis, ",
    "the advantage of streaming synthesis is its real-time performance, ",
    "which is much stronger. Users can hear ",
    "nearly synchronized speech output while ",
    "inputting text, which greatly improves the ",
    "interactive experience and reduces user waiting time. ",
    "It is suitable for scenarios that call large ",
    "language models (LLMs) to perform ",
    "speech synthesis by streaming ",
    "text input.",
]

# Instantiate SpeechSynthesizer and pass request parameters such as model and voice in the constructor.
synthesizer = SpeechSynthesizer(
    model=model,
    voice=voice,
    format=AudioFormat.PCM_22050HZ_MONO_16BIT,  
    callback=callback,
)


# Stream the text to be synthesized. Receive the binary audio data in real time in the on_data method of the callback interface.
for text in test_text:
    synthesizer.streaming_call(text)
    time.sleep(0.1)
# End the streaming speech synthesis.
synthesizer.streaming_complete()

print('[Metric] Request ID: {}, First package delay: {} ms'.format(
    synthesizer.get_last_request_id(),
    synthesizer.get_first_package_delay()))

The SpeechSynthesizer class

SpeechSynthesizer is imported through from dashscope.audio.tts_v2 import * and provides key interfaces for speech synthesis.

def call(self, text: str, timeout_millis=None)

def streaming_call(self, text: str)

def streaming_complete(self, complete_timeout_millis=600000)

def get_last_request_id(self)

def get_first_package_delay(self)

def get_response(self)

Callback interface (ResultCallback)

For asynchronous or streaming calls, the server uses callbacks to return key process information and data to the client. You need to implement the callback methods to handle the information or data returned by the server.

You can import it using from dashscope.audio.tts_v2 import *.

class Callback(ResultCallback):
    def on_open(self) -> None:
        print('Connection successful')
    
    def on_data(self, data: bytes) -> None:
        # Implement the logic to receive the synthesized binary audio result.

    def on_complete(self) -> None:
        print('Synthesis complete')

    def on_error(self, message) -> None:
        print('An exception occurred: ', message)

    def on_close(self) -> None:
        print('Connection closed')


callback = Callback()

def on_open(self) -> None

def on_event( self, message: str) -> None

def on_complete(self) -> None

def on_error(self, message) -> None

def on_data(self, data: bytes) -> None

def on_close(self) -> None

If you encounter an error, see Error messages for troubleshooting.

If the problem is still not resolved, join the developer group to provide feedback on the problem and include the Request ID for further investigation.

The default supported voices are listed in the following table. If you need a more personalized voice, you can use the voice cloning feature to create a custom voice for free. For more information, see Use a cloned voice for speech synthesis.

• cosyvoice-v2 voice list

  Important

  The following is the cosyvoice-v2 voice list. When using these voices, you must set the model request parameter to cosyvoice-v2. Otherwise, the call will fail.

  Scenario	Voice	Voice characteristics	Audio sample (right-click to save)	voice parameter	Language	SSML
  Customer service	Long Yingcui	Serious male		longyingcui	Chinese, English	✅
  	Long Yingda	Cheerful high-pitched female		longyingda	Chinese, English	✅
  	Long Yingjing	Low-key and calm female		longyingjing	Chinese, English	✅
  	Long Yingyan	Righteous and stern female		longyingyan	Chinese, English	✅
  	Long Yingtian	Gentle and sweet female		longyingtian	Chinese, English	✅
  	Long Yingbing	Sharp and assertive female		longyingbing	Chinese, English	✅
  	Long Yingtao	Gentle and calm female		longyingtao	Chinese, English	✅
  	Long Yingling	Gentle and empathetic female		longyingling	Chinese, English	✅
  Voice assistant	YUMI	Formal young female		longyumi_v2	Chinese, English	✅
  	Long Xiaochun	Intellectual and positive female		longxiaochun_v2	Chinese, English	✅
  	Long Xiaoxia	Calm and authoritative female		longxiaoxia_v2	Chinese, English	✅
  Livestreaming and e-commerce	Long Anran	Lively and textured female		longanran	Chinese, English	✅
  	Long Anxuan	Classic female livestreamer		longanxuan	Chinese, English	✅
  Audiobooks	Long Sanshu	Calm and textured male		longsanshu	Chinese, English	✅
  	Long Xiu	Knowledgeable male storyteller		longxiu_v2	Chinese, English	✅
  	Long Miao	Rhythmic female		longmiao_v2	Chinese, English	✅
  	Long Yue	Warm and magnetic female		longyue_v2	Chinese, English	✅
  	Long Nan	Wise young male		longnan_v2	Chinese, English	✅
  	Long Yuan	Warm and healing female		longyuan_v2	Chinese, English	✅
  Social companion	Long Anrou	Gentle best friend female		longanrou	Chinese, English	✅
  	Long Qiang	Romantic and charming female		longqiang_v2	Chinese, English	✅
  	Long Han	Warm and devoted male		longhan_v2	Chinese, English	✅
  	Long Xing	Gentle girl-next-door		longxing_v2	Chinese, English	✅
  	Long Hua	Energetic and sweet female		longhua_v2	Chinese, English	✅
  	Long Wan	Positive and intellectual female		longwan_v2	Chinese, English	✅
  	Long Cheng	Intelligent young male		longcheng_v2	Chinese, English	✅
  	Long Feifei	Sweet and delicate female		longfeifei_v2	Chinese, English	✅
  	Long Xiaocheng	Magnetic low-pitched male		longxiaocheng_v2	Chinese, English	✅
  	Long Zhe	Awkward and warm-hearted male		longzhe_v2	Chinese, English	✅
  	Long Yan	Warm and gentle female		longyan_v2	Chinese, English	✅
  	Long Tian	Magnetic and rational male		longtian_v2	Chinese, English	✅
  	Long Ze	Warm and energetic male		longze_v2	Chinese, English	✅
  	Long Shao	Positive and upwardly mobile male		longshao_v2	Chinese, English	✅
  	Long Hao	Emotional and melancholic male		longhao_v2	Chinese, English	✅
  	Long Shen	Talented male singer		kabuleshen_v2	Chinese, English	✅
  Children's voice	Long Jielidou	Sunny and mischievous male		longjielidou_v2	Chinese, English	✅
  	Long Ling	Childish and stiff female		longling_v2	Chinese, English	✅
  	Long Ke	Innocent and well-behaved female		longke_v2	Chinese, English	✅
  	Long Xian	Bold and cute female		longxian_v2	Chinese, English	✅
  Dialect	Long Laotie	Straightforward Northeastern dialect male		longlaotie_v2	Chinese (Northeastern), English	✅
  	Long Jiayi	Intellectual Cantonese female		longjiayi_v2	Chinese (Cantonese), English	✅
  	Long Tao	Positive Cantonese female		longtao_v2	Chinese (Cantonese), English	✅
  Poetry recitation	Long Fei	Passionate and magnetic male		longfei_v2	Chinese, English	✅
  	Li Bai	Ancient male poet		libai_v2	Chinese, English	✅
  	Long Jin	Elegant and gentle male		longjin_v2	Chinese, English	✅
  News broadcast	Long Shu	Calm young male		longshu_v2	Chinese, English	✅
  	Bella2.0	Precise and capable female		loongbella_v2	Chinese, English	✅
  	Long Shuo	Knowledgeable and capable male		longshuo_v2	Chinese, English	✅
  	Long Xiaobai	Calm female announcer		longxiaobai_v2	Chinese, English	✅
  	Long Jing	Typical female announcer		longjing_v2	Chinese, English	✅
  	loongstella	Confident and crisp female		loongstella_v2	Chinese, English	✅
  Overseas marketing	loongeva	Intellectual British English female		loongeva_v2	British English	❌
  	loongbrian	Calm British English male		loongbrian_v2	British English	❌
  	loongluna	British English female		loongluna_v2	British English	❌
  	loongluca	British English male		loongluca_v2	British English	❌
  	loongemily	British English female		loongemily_v2	British English	❌
  	loongeric	British English male		loongeric_v2	British English	❌
  	loongabby	American English female		loongabby_v2	American English	❌
  	loongannie	American English female		loongannie_v2	American English	❌
  	loongandy	American English male		loongandy_v2	American English	❌
  	loongava	American English female		loongava_v2	American English	❌
  	loongbeth	American English female		loongbeth_v2	American English	❌
  	loongbetty	American English female		loongbetty_v2	American English	❌
  	loongcindy	American English female		loongcindy_v2	American English	❌
  	loongcally	American English female		loongcally_v2	American English	❌
  	loongdavid	American English male		loongdavid_v2	American English	❌
  	loongdonna	American English female		loongdonna_v2	American English	❌
  	loongkyong	Korean female		loongkyong_v2	Korean	❌
  	loongtomoka	Japanese female		loongtomoka_v2	Japanese	❌
  	loongtomoya	Japanese male		loongtomoya_v2	Japanese	❌

Features, billing, and rate limiting

Troubleshooting

If a code error occurs, see the information in Error codes for troubleshooting.

pip uninstall websocket-client
pip uninstall websocket
pip install websocket-client

Permissions and authentication

More questions

A: See the GitHub QA.