MediaCodec的使用(音频编码一)
准备工作
我们准备一个pcm文件即可,采样率16000,单声道
如果可以使用ffplay如下命令正常播放,那么音频格式是没有问题的。- ffplay -f s16le -ar 16000 -ch_layout mono -i hello.pcm
复制代码 将文件推送到download目录
- adb -s adb-4128230924000237-Nl4jPZ._adb-tls-connect._tcp push hello.pcm /sdcard/Download/
复制代码 Android10+是无法之间操作Download目录的文件的,即使你获取到读写权限。
读写权限的前世今生
- android.Manifest.permission.READ_EXTERNAL_STORAGE,
- android.Manifest.permission.WRITE_EXTERNAL_STORAGE,
复制代码 这两个已经完全过时了在Android13+- READ_EXTERNAL_STORAGE is deprecated (and is not granted) when targeting Android 13+. If you need to query or interact with MediaStore or media files on the shared storage, you should instead use one or more new storage permissions: READ_MEDIA_IMAGES, READ_MEDIA_VIDEO or READ_MEDIA_AUDIO. Toggle info (Ctrl+F1)
复制代码 Android10+
shared storage 使用MediaStore
但是上面两个还应用于视频、音频、照片
Android13++
上面两个权限完全过时了,视频、音频、照片使用单独的权限
当然国内的魔改手机在Android13+仍有可能使用上面的两个权限- READ_MEDIA_IMAGES, READ_MEDIA_VIDEO or READ_MEDIA_AUDIO
复制代码 我现在这台手机是android11,访问Download目录,所有用不上上面的两个权限。
但是如果是android10以下,还是需要的。
所以将hello.pcm只能通过SAF访问或者MANAGE_EXTERNAL_STORAGE.
SAF
- @Composable
- internal fun MediaCodecScreen(
- navHostController: NavHostController,
- snackBarHostState: SnackbarHostState,
- ) {
- val context: Context = LocalContext.current
- val coroutineScope: CoroutineScope = rememberCoroutineScope()
- val launcher: ManagedActivityResultLauncher, Uri?> = rememberLauncherForActivityResult(
- contract = ActivityResultContracts.OpenDocument()
- ) { uri: Uri? ->
- uri?.apply {
- Log.i(TAG, "MediaCodecScreen -> uri: $this")
- context.contentResolver.openInputStream(this)?.use { inputStream ->
- val bytes = ByteArray(1024)
- inputStream.read(bytes)
- Log.i(TAG, "MediaCodecScreen -> data: ${bytes.joinToString()}")
- }
- }
- }
- Column(
- modifier = Modifier.fillMaxSize()
- ) {
- Text(
- text = "发送信息",
- Modifier
- .padding(top = 10.dp)
- .background(color = Color.Black, shape = RoundedCornerShape10)
- .padding(all = 5.dp)
- .clickable {
- launcher.launch(arrayOf("*/*"))
- },
- color = Color.White
- )
- }
- }
复制代码 建议在android10+访问shared storage尽量使用MediaStore,如果不使用的话还是很危险的,如果生成一个文件到Download,用户重新卸载安装那你就不能访问这个文件了,会崩溃的,每个app都相当于一个匿名用户,重新安装那你的匿名身份就变更了,当然文件就不能访问了。
当然在android10~android12如果访问视频、音频、照片还是要读写权限的。
还要注意MediaStore相当于是一个数据库,他加快了检索文件的速度。当你使用MeidaStore的时候,不要仅仅是插入一条记录,还有把文件的二进制数据写入到MediaStore.
还有尽量不要使用uri转file,file转uri.不符合现代android的开发。
当然在app的私有目录无论是外部存储还是内部存储都可以使用java 的 File因为他和shared storage没有任何关系。
PCM 编码 MP3
编码函数如下:- internal suspend fun pcmToMp3(
- context: Context,
- pcmUri: Uri,
- mp3Uri: Uri,
- ): Unit = suspendCancellableCoroutine { continuation ->
-
- val mediaCodecList = MediaCodecList(MediaCodecList.ALL_CODECS)
- val mp3Encoders: List<MediaCodecInfo> = mediaCodecList.codecInfos.filter { it.isEncoder }.filter {
- it.supportedTypes.contains(element = MediaFormat.MIMETYPE_AUDIO_MPEG)
- }
- if (mp3Encoders.isEmpty()){
- if (continuation.isActive){
- continuation.resume(Unit) // 不支持的话可以使用lame软编码
- }
- return@suspendCancellableCoroutine
- }
-
- mp3Encoders.forEach { mediaCodecInfo ->
- if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
- Log.i(TAG, "pcmToAac -> name: ${mediaCodecInfo.name}, canonicalName: ${mediaCodecInfo.canonicalName}, isAlias: ${mediaCodecInfo.isAlias}, isVendor: ${mediaCodecInfo.isVendor}, isHardwareAccelerated: ${mediaCodecInfo.isHardwareAccelerated}, isEncoder: ${mediaCodecInfo.isEncoder}, isSoftwareOnly: ${mediaCodecInfo.isSoftwareOnly}, supportedTypes: ${mediaCodecInfo.supportedTypes.joinToString()}")
- } else {
- Log.i(TAG, "pcmToAac -> name: ${mediaCodecInfo.name}, isEncoder: ${mediaCodecInfo.isEncoder}, supportedTypes: ${mediaCodecInfo.supportedTypes.joinToString()}")
- }
- }
- val mediaFormat: MediaFormat = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_MPEG, 16000, 1).apply {
- setInteger(MediaFormat.KEY_BIT_RATE, 128000)
- }
- val mediaCodec: MediaCodec = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_MPEG)
- mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
- val pcmInputStream: InputStream = context.contentResolver.openInputStream(pcmUri)!!
- val mp3OutputStream: OutputStream = context.contentResolver.openOutputStream(mp3Uri)!!
- val bytes = ByteArray(1024 * 8)
- mediaCodec.setCallback(object : MediaCodec.Callback(){
- override fun onError(
- codec: MediaCodec,
- e: MediaCodec.CodecException
- ) {
- Log.e(TAG, "onError name: ${codec.name}, thread: ${Thread.currentThread()}, error: ${e.message}", e)
- }
- override fun onInputBufferAvailable(
- codec: MediaCodec,
- index: Int
- ) {
- val inputBuffer: ByteBuffer = codec.getInputBuffer(index) ?: return
- val size: Int = pcmInputStream.read(bytes, 0, inputBuffer.limit()) // 或许需要去 min(bytes.size, inputBuffer.limit())
- Log.i(TAG, "onInputBufferAvailable -> name: ${mediaCodec.name}, index: $index, thread: ${Thread.currentThread()}, size: $size, limit: ${inputBuffer.limit()}, position: ${inputBuffer.position()}")
- if (size > 0) {
- inputBuffer.put(bytes, 0, size)
- codec.queueInputBuffer(index, 0, size, System.nanoTime() / 1000, 0)
- } else {
- codec.queueInputBuffer(index, 0, 0, System.nanoTime() / 1000, MediaCodec.BUFFER_FLAG_END_OF_STREAM)
- }
- }
- override fun onOutputBufferAvailable(
- codec: MediaCodec,
- index: Int,
- info: MediaCodec.BufferInfo
- ) {
- Log.i(TAG, "onOutputBufferAvailable -> name: ${codec.name}, index: $index, info: ${info.size}, thread: ${Thread.currentThread()}")
- val outputBuffer: ByteBuffer = codec.getOutputBuffer(index) ?: return
- outputBuffer.get(bytes, 0, info.size)
- mp3OutputStream.write(bytes, 0, info.size)
- codec.releaseOutputBuffer(index, false)
- if (info.flags == MediaCodec.BUFFER_FLAG_END_OF_STREAM){
- Log.i(TAG, "onOutputBufferAvailable -> == 编码结束...") // todo
- pcmInputStream.close()
- mp3OutputStream.close()
- if (continuation.isActive){
- Log.i(TAG, "pcmToMp3 -> 解码完成 resume before...")
- continuation.resume(Unit)
- Log.i(TAG, "pcmToMp3 -> 解码完成 resume after...")
- }
- }
- }
- override fun onOutputFormatChanged(
- codec: MediaCodec,
- format: MediaFormat
- ) {
- Log.i(TAG, "onOutputFormatChanged -> name: ${codec.name}, format: ${format.getString(MediaFormat.KEY_MIME)}")
- }
- })
- Log.i(TAG, "pcmToAac -> before start...")
- mediaCodec.start()
- Log.i(TAG, "pcmToAac -> after start...")
- }
复制代码 调用如下- @Composable
- internal fun MediaCodecScreen(
- navHostController: NavHostController,
- snackBarHostState: SnackbarHostState,
- ) {
- val context: Context = LocalContext.current
- val coroutineScope: CoroutineScope = rememberCoroutineScope()
- val launcher: ManagedActivityResultLauncher, Uri?> = rememberLauncherForActivityResult(
- contract = ActivityResultContracts.OpenDocument()
- ) { uri: Uri? ->
- uri?.apply {
- Log.i(TAG, "MediaCodecScreen -> uri: $this")
- val audioExtractManager = AudioExtractManager()
- val contentValues: ContentValues = contentValuesOf(
- MediaStore.Audio.Media.DISPLAY_NAME to "hello.mp3",
- MediaStore.Audio.Media.MIME_TYPE to MediaFormat.MIMETYPE_AUDIO_MPEG,
- MediaStore.Audio.Media.RELATIVE_PATH to Environment.DIRECTORY_MUSIC
- )
- val mp3Uri: Uri = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) {
- context.contentResolver.insert(MediaStore.Audio.Media.EXTERNAL_CONTENT_URI, contentValues)
- } else {
- FileProvider.getUriForFile(context, "${context.packageName}.provider", File(
- Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_MUSIC),
- "hello.mp3"
- ))
- } ?: return@apply
- Log.i(TAG, "MediaCodecScreen -> mp3Uri: $mp3Uri")
- coroutineScope.launch {
- Log.i(TAG, "MediaCodecScreen -> before ${Thread.currentThread()}")
- audioExtractManager.pcmToMp3(context, this@apply, mp3Uri)
- Log.i(TAG, "MediaCodecScreen -> after ${Thread.currentThread()}")
- }
- }
- }
- Column(
- modifier = Modifier.fillMaxSize()
- ) {
- Text(
- text = "开发者",
- Modifier
- .padding(top = 10.dp)
- .background(color = Color.Black, shape = RoundedCornerShape10)
- .padding(all = 5.dp)
- .clickable {
- val intent = Intent(Settings.ACTION_APPLICATION_DEVELOPMENT_SETTINGS)
- context.startActivity(intent)
- },
- color = Color.White
- )
- Text(
- text = "发送信息",
- Modifier
- .padding(top = 10.dp)
- .background(color = Color.Black, shape = RoundedCornerShape10)
- .padding(all = 5.dp)
- .clickable {
- launcher.launch(arrayOf("*/*"))
- },
- color = Color.White
- )
- }
- }
复制代码 PCM 编码 adts aac
说到adts,它是包含header的aac裸流,在某些设备上是无法正确播放的比如ios.
参考: adts aac
7字节的头
7字节的头其一共7 * 8=56bit位。分为 各占28bit- adts_fixed_header();
- adts_variable_header();
复制代码 我们一位一位的看.
adts_fixed_header()
syncword: 前12位是固定的都是1, 代表adts header开始的标识
ID:1bit, MPEG Version: 0 for MPEG-4,1 for MPEG-2
Layer:2bits, always: '00'
protection_absent:1bit, Warning, set to 1 if there is no CRC and 0 if there is CRC
一般是0表示: no CRC
profile:2bits, 表示使用哪个级别的AAC,如01 Low Complexity(LC) -- AAC LC
profile的值等于 Audio Object Type的值减1.
profile = MPEG-4 Audio Object Type - 1
sampling_frequency_index: 4bits, 采样率下标,可以参考上面的链接查看
Private bit, 1bit, 设置为0即可
channel_configuration: 占3bits, 声道数下标,可以参考上面的链接查看
原创性,1bit: 设为1表示音频为原创,设为0则表示非原创。
家用性,1bit:设为1表示音频为家用,设为0则表示非家用。
总共28bit
adts_variable_header
版权标识位(Copyright ID bit):1bit, 中央注册版权标识符的下一比特位。采用LSB(最低有效位优先)顺序滑动遍历比特串时,将当前比特值填入此字段,到达末尾时循环回到起始位置(环形缓冲区结构)。
版权标识起始位(Copyright ID start):1bit, 1表示当前帧的版权标识位是起始位,设为0则表示非起始位。
aac_frame_length:13bits, 一个ADTS帧的长度包括ADTS头和AAC原始流。
frame length, this value must include 7 or 9 bytes of header length:
aac_frame_length = (protection_absent == 1 ? 7 : 9) + size(AACFrame)
protection_absent=0时, header length=9bytes
protection_absent=1时, header length=7bytes
adts_buffer_fullness:11bit,
计算公式- max_bit_reservoir = minimum_decoder_input_size - mean_bits_per_RDB; // for CBR
- // bit reservoir state/available bits (≥0 and <max_bit_reservoir); for the i-th frame.
- bit_reservoir_state[i] = (int)(bit_reservoir_state[i - 1] + mean_framelength - framelength[i]);
- // NCC is the number of channels.
- adts_buffer_fullness = bit_reservoir_state[i] / (NCC * 32);
复制代码 编码函数如下:- private fun addAdtsHeader(packet: ByteArray, packetLen: Int, sampleRate: Int, channels: Int) {
- val profile = 2 // AAC LC
- val freqIdx = when(sampleRate){
- 96000 -> 0
- 88200 -> 1
- 64000 -> 2
- 48000 -> 3
- 44100 -> 4
- 32000 -> 5
- 24000 -> 6
- 22050 -> 7
- 16000 -> 8
- 12000 -> 9
- 11025 -> 10
- 8000 -> 11
- else -> 4 // 默认44100
- }
- /**
- 0000 0x00
- 0001 0x01
- 0010 0x02
- 0011 0x03
- 0100 0x04
- 0101 0x05
- 0110 0x06
- 0111 0x07
- 1000 0x08
- 1001 0x09
- 1010 0x0A
- 1011 0x0B
- 1100 0x0C
- 1101 0x0D
- 1110 0x0E
- 1111 0x0F
- */
- val chanCfg = channels // CPE = 1, mono = 1
- packet[0] = 0xFF.toByte() // 1111 1111 1字节
- packet[1] = 0xF9.toByte() // 1111 1001 2字节 id 1 for MPEG-2, layer = 00 protection_absent = 1
- // [01]00 0000 profile 2bits
- // [0110 00]00 freqIdx 4bits
- // [0100 000]0 Private bit
- // channel 3 bits 001 右移两位 0
- // [0100 0000] channel的1位0加上来 // 3字节
- packet[2] = ((profile - 1 shl 6) + (freqIdx shl 2) + (chanCfg shr 2)).toByte()
- // 0000 0000
- // channel 0000 0001 左移6位 01
- // [01]00 0000 channel 处理完成 // 26
- // Originality、Home两bit Copyright ID bit, Copyright ID start 2bit 合并到 packetLen,所有其一共17bits
- // 0x7FFF
- // 0000 0000 0000 000[0 00011|111 1111 1111] packetLen 右边移 11 ->
- // 0000 0000 0000 0000 0000 00[00 0011] packetLen 右边移 11 ->
- // [0100 0011]
- packet[3] = (((chanCfg and 3) shl 6) + (packetLen shr 11)).toByte() // 4字节
- packet[4] = (packetLen and 0x7FF shr 3).toByte() // [111 1111 1]111
- // [111]0 0000 后面即 0xFFC
- packet[5] = ((packetLen and 7 shl 5) + 0x1F).toByte() // 0001 1111
- // 1111 1100
- packet[6] = 0xFC.toByte() // 7字节
- }
复制代码 使用案例- internal suspend fun pcmToAac(context: Context, pcmUri: Uri, aacUri: Uri): Unit = suspendCancellableCoroutine { continuation ->
- // 编码器
- // aac 3gpp audio/amr-wb audio/flac
- // h264
- // 解码器 解码器那可就太多了
- val mediaCodecList = MediaCodecList(MediaCodecList.ALL_CODECS)
- val aacEncoders: List<MediaCodecInfo> = mediaCodecList.codecInfos.filter { it.isEncoder }.filter {
- it.supportedTypes.contains(element = MediaFormat.MIMETYPE_AUDIO_AAC)
- }
- Log.i(TAG, "pcmToAac -> aacEncoders: ${aacEncoders.joinToString { it.name }}")
- if (aacEncoders.isEmpty()){
- // 不支持硬件解码
- Log.i(TAG, "pcmToAac -> 不支持硬件编码...") // 使用软解码
- if (continuation.isActive){
- continuation.resume(Unit)
- }
- return@suspendCancellableCoroutine
- }
- // 可惜 aac 仅支持软解码
- Log.i(TAG, "pcmToAac -> 支持硬件编码")
- val mediaFormat: MediaFormat = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, 16000, 1).apply {
- setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC)
- setInteger(MediaFormat.KEY_BIT_RATE, 128000)
- }
- val mediaCodec: MediaCodec = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC)
- mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
- val pcmInputStream: InputStream = context.contentResolver.openInputStream(pcmUri)!!
- val aacOutputStream: OutputStream = context.contentResolver.openOutputStream(aacUri)!!
- val bytes = ByteArray(1024 * 8)
- mediaCodec.setCallback(object : MediaCodec.Callback() {
- override fun onError(
- codec: MediaCodec,
- e: MediaCodec.CodecException
- ) {
- Log.e(
- TAG,
- "onError name: ${codec.name}, thread: ${Thread.currentThread()}, error: ${e.message}",
- e
- )
- }
- override fun onInputBufferAvailable(
- codec: MediaCodec,
- index: Int
- ) {
- Log.i(
- TAG,
- "onInputBufferAvailable -> name: ${mediaCodec.name}, index: $index, thread: ${Thread.currentThread()}"
- )
- val inputBuffer: ByteBuffer = codec.getInputBuffer(index) ?: return
- val size: Int = pcmInputStream.read(bytes, 0, inputBuffer.limit())
- if (size > 0) {
- inputBuffer.put(bytes, 0, size)
- codec.queueInputBuffer(index, 0, size, System.nanoTime() / 1000, 0)
- } else {
- codec.queueInputBuffer(
- index,
- 0,
- 0,
- System.nanoTime() / 1000,
- MediaCodec.BUFFER_FLAG_END_OF_STREAM
- )
- }
- }
- override fun onOutputBufferAvailable(
- codec: MediaCodec,
- index: Int,
- info: MediaCodec.BufferInfo
- ) {
- Log.i(
- TAG,
- "onOutputBufferAvailable -> name: ${codec.name}, index: $index, info: ${info.size}, thread: ${Thread.currentThread()}"
- )
- val outputBuffer: ByteBuffer = codec.getOutputBuffer(index) ?: return
- val aacData = ByteArray(info.size + 7)
- addAdtsHeader(aacData, aacData.size, 16000, 1)
- outputBuffer.get(aacData, 7, info.size)
- aacOutputStream.write(aacData)
- codec.releaseOutputBuffer(index, false)
- if (info.flags == MediaCodec.BUFFER_FLAG_END_OF_STREAM) {
- Log.i(TAG, "onOutputBufferAvailable -> == 编码结束...") // todo
- pcmInputStream.close()
- aacOutputStream.close()
- if (continuation.isActive) {
- Log.i(TAG, "pcmToAac -> 解码完成 resume before...")
- continuation.resume(Unit)
- Log.i(TAG, "pcmToAac -> 解码完成 resume after...")
- }
- }
- }
- override fun onOutputFormatChanged(
- codec: MediaCodec,
- format: MediaFormat
- ) {
- Log.i(
- TAG,
- "onOutputFormatChanged -> name: ${codec.name}, format: ${
- format.getString(MediaFormat.KEY_MIME)
- }"
- )
- }
- })
- Log.i(TAG, "pcmToAac -> before start...")
- mediaCodec.start()
- Log.i(TAG, "pcmToAac -> after start...")
- }
- @Suppress("SameParameterValue")
- private fun addAdtsHeader(packet: ByteArray, packetLen: Int, sampleRate: Int, channels: Int) {
- val profile = 2 // AAC LC
- val freqIdx = when(sampleRate){
- 96000 -> 0
- 88200 -> 1
- 64000 -> 2
- 48000 -> 3
- 44100 -> 4
- 32000 -> 5
- 24000 -> 6
- 22050 -> 7
- 16000 -> 8
- 12000 -> 9
- 11025 -> 10
- 8000 -> 11
- else -> 4 // 默认44100
- }
- /**
- 0000 0x00
- 0001 0x01
- 0010 0x02
- 0011 0x03
- 0100 0x04
- 0101 0x05
- 0110 0x06
- 0111 0x07
- 1000 0x08
- 1001 0x09
- 1010 0x0A
- 1011 0x0B
- 1100 0x0C
- 1101 0x0D
- 1110 0x0E
- 1111 0x0F
- */
- val chanCfg = channels // CPE = 1, mono = 1
- packet[0] = 0xFF.toByte() // 1111 1111 1字节
- packet[1] = 0xF9.toByte() // 1111 1001 2字节 id 1 for MPEG-2, layer = 00 protection_absent = 1
- // [01]00 0000 profile 2bits
- // [0110 00]00 freqIdx 4bits
- // [0100 000]0 Private bit
- // channel 3 bits 001 右移两位 0
- // [0100 0000] channel的1位0加上来 // 3字节
- packet[2] = ((profile - 1 shl 6) + (freqIdx shl 2) + (chanCfg shr 2)).toByte()
- // 0000 0000
- // channel 0000 0001 左移6位 01
- // [01]00 0000 channel 处理完成 // 26
- // Originality、Home两bit Copyright ID bit, Copyright ID start 2bit 合并到 packetLen,所有其一共17bits
- // 0x7FFF
- // 0000 0000 0000 000[0 00011|111 1111 1111] packetLen 右边移 11 ->
- // 0000 0000 0000 0000 0000 00[00 0011] packetLen 右边移 11 ->
- // [0100 0011]
- packet[3] = (((chanCfg and 3) shl 6) + (packetLen shr 11)).toByte() // 4字节
- packet[4] = (packetLen and 0x7FF shr 3).toByte() // [111 1111 1]111
- // [111]0 0000 后面即 0xFFC
- packet[5] = ((packetLen and 7 shl 5) + 0x1F).toByte() // 0001 1111
- // 1111 1100
- packet[6] = 0xFC.toByte() // 7字节
- }
复制代码 来源:程序园用户自行投稿发布,如果侵权,请联系站长删除
免责声明:如果侵犯了您的权益,请联系站长,我们会及时删除侵权内容,谢谢合作! |