奇怪的 WebRTC SinkID 问题

【问题标题】：Weird WebRTC SinkID issue奇怪的 WebRTC SinkID 问题
【发布时间】：2020-07-31 02:14:10
【问题描述】：

我有 8 条虚拟音频线路。我正在尝试将不同的音频/视频播放器映射到不同的虚拟线路，因为我们将其用于远程口译。我的问题是当一个新的 WebRTC 出现时，所有玩家都默认使用最新的 sinkID 类型。接收器 ID 报告它们没有更改，但所有音频都开始流经最近设置的接收器 ID。有谁知道这是浏览器限制还是什么？

只有当我在人们连接时按需创建音频播放器时才会发生这种情况。如果我在每个人都连接后设置 ID，则没有问题。我什至尝试将所有 sinkID 重置为每次新连接时应该使用的，但音频仍然全部流经最新音频播放器上的 SinkID。

我附上了一张关于一切外观的视图。在附件中，Dave 将附加到第 4 行的 SinkID。这很好用。如果另一个口译员加入他加禄语，他们也可以在第 4 行听到。问题是，如果有人加入让我们说西班牙语，当我在西班牙语（第 3 行）为播放器设置接收器 ID 时，所有音频都通过 SinkID 听到第 3 行，即使第 4 行人员的 SinkID 仍设置为第 4 行。我认为这可能是 Chrome 错误。我正在使用 OpenTok (tokbox)，一些 PHP 并不需要显示来管理显示的语言。我附上了管理所有这些的 iFrame 代码。 

<!DOCTYPE html>
<html>
<head>
    <title>IRIS Remote Player</title>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <script src="../vendor/jquery/jquery.min.js"></script>
    <script src="/socket.io/socket.io.js"></script>
    <link href="/vendor/bootstrap/css/bootstrap.min.css" rel="stylesheet">
    <script src="https://static.opentok.com/v2/js/opentok.js"></script>
    <link href="https://cdn.jsdelivr.net/gh/gitbrent/bootstrap4-toggle@3.6.1/css/bootstrap4-toggle.min.css" rel="stylesheet">
    <script src="https://cdn.jsdelivr.net/gh/gitbrent/bootstrap4-toggle@3.6.1/js/bootstrap4-toggle.min.js"></script>
    <meta name="lastLoaded" content="2020-04-21 11:34:57">
    <script> var EnglishOn; </script>
<style>
  .slow  .toggle-group { transition: left 0.7s; -webkit-transition: left 0.7s; }
  .fast  .toggle-group { transition: left 0.1s; -webkit-transition: left 0.1s; }
  .quick .toggle-group { transition: none;      -webkit-transition: none; }
</style>    
</head>
<body>
<button type="button" class="btn btn-lg btn-danger form-control" id="connect">Connect Remote Interpreters</button>
<div class="container-fluid">
    <div class="row live" style="display:none;background:black;color:white;padding-bottom:5px">
        <div class="col-sm-2"><input type="checkbox" checked data-width="100%" id="AllAudio" data-toggle="toggle" data-offstyle="danger" data-onstyle="success" data-on="Live" data-off="Break" data-style="quick"></div>
        <div class="col-sm-2"><input type="checkbox" checked data-width="100%" id="Admins" data-toggle="toggle" data-offstyle="danger" data-onstyle="success" data-on="Admins Live" data-off="Admins Off" data-style="quick"></div>
    </div>

    <div class="row live" style="display:none;background:#fff;color:black">
<div class="col-sm-2" id="PL"><input data-style="quick" type="checkbox" class="Interpretation AudioControl SwitchPL" data-width="100%" id="floorPolish" data-lang="PL" data-toggle="toggle" checked  data-offstyle="danger" data-onstyle="success" data-on="Polish" data-off="Floor"><br />Floor<br /><audio autoplay controls muted id="englishPolish" class="englishPL"></audio></div><div class="col-sm-2" id="RU"><input data-style="quick" type="checkbox" class="Interpretation AudioControl SwitchRU" data-width="100%" id="floorRussian" data-lang="RU" data-toggle="toggle" checked  data-offstyle="danger" data-onstyle="success" data-on="Russian" data-off="Floor"><br />Floor<br /><audio autoplay controls muted id="englishRussian" class="englishRU"></audio></div><div class="col-sm-2" id="ES"><input data-style="quick" type="checkbox" class="Interpretation AudioControl SwitchES" data-width="100%" id="floorSpanish" data-lang="ES" data-toggle="toggle" checked  data-offstyle="danger" data-onstyle="success" data-on="Spanish" data-off="Floor"><br />Floor<br /><audio autoplay controls muted id="englishSpanish" class="englishES"></audio></div><div class="col-sm-2" id="TL"><input data-style="quick" type="checkbox" class="Interpretation AudioControl SwitchTL" data-width="100%" id="floorTagalog" data-lang="TL" data-toggle="toggle" checked  data-offstyle="danger" data-onstyle="success" data-on="Tagalog" data-off="Floor"><br />Floor<br /><audio autoplay controls muted id="englishTagalog" class="englishTL"></audio></div><div class="col-sm-2" id="EN"><input type="checkbox" data-style="quick" class="AudioControl" data-width="100%" id="floorEnglish" data-toggle="toggle" data-offstyle="danger" data-onstyle="success" data-on="English" data-off="Floor"><br />Floor<br /><audio autoplay controls muted id="englishEnglish" class="englishEN"></audio></div>
    </div>
</div>




<script>
var audioSource;
var vars = {};
var floorSource;

$('#connect').click(function() {

    $('input:checkbox#Admins').change(function(){
        if ($(this).is(':checked')) {
            $('.ADMIN').each(function() {
                var adminID = $(this).attr("id");
                var thisID = adminID.replace("userID", "");
                $("#"+thisID).prop('muted', false);
                $('#'+thisID).prop('volume', 1);                    
            });         
        } else {
            $('.ADMIN').each(function() {
                var adminID = $(this).attr("id");
                var thisID = adminID.replace("userID", "");
                $("#"+thisID).prop('muted', true);
                $('#'+thisID).prop('volume', 0);                    
            });             
        }   
    });
    $('input:checkbox.AudioControl').change(function(){
        var LanguageCode = $(this).attr("data-lang");
        var LanguageName = $(this).attr("data-on");
        if ($(this).is(':checked')) {
            $("#english"+LanguageName).prop('muted', true);
            $('#english'+LanguageName).prop('volume', 0);   
            $("."+LanguageCode).prop('muted', false);
            $('.'+LanguageCode).prop('volume', 1);
            setTimeout(function(){ 
                if ($('#Admins').is(':checked')) {} else {
                    $('.ADMIN').each(function() {
                        var adminID = $(this).attr("id");
                        var thisID = adminID.replace("userID", "");             
                        $("#"+thisID).prop('muted', true);
                        $('#'+thisID).prop('volume', 0);    
                    }); 
                }   
            }, 200);
        } else {
            //Activate Floor
            $("#english"+LanguageName).prop('muted', false);
            $('#english'+LanguageName).prop('volume', 1);   
            $("."+LanguageCode).prop('muted', true);
            $('.'+LanguageCode).prop('volume', 0);
            setTimeout(function(){ 
                if ($('#Admins').is(':checked')) {} else {
                    $('.ADMIN').each(function() {
                        var adminID = $(this).attr("id");
                        var thisID = adminID.replace("userID", "");             
                        $("#"+thisID).prop('muted', true);
                        $('#'+thisID).prop('volume', 0);
                    });                 
                }       
            }, 200);                
        }
    });

    $('input:checkbox#AllAudio').change(function(){
        if ($(this).is(':checked')) {
            $('.Interpretation').bootstrapToggle('on')
            $('#floorEnglish').bootstrapToggle('off')
        } else {
            $('.AudioControl').bootstrapToggle('off')       
        }
    }); 

    $("#connect").hide();
    $("#subscriber").show();
    $(".live").show();
playFloor('Polish','PL');
        playFloor('Russian','RU');
        playFloor('Spanish','ES');
        playFloor('Tagalog','TL');
            playFloor('English', 'EN');

    function playFloor(language, code) {
        const audio = document.getElementById('english'+language);
        const constraints = {
            audio: {deviceId: floorSource},
            video: false
        };
        function handleSuccess(stream) {
          const audioTracks = stream.getAudioTracks();
          console.log('Got stream with constraints:', constraints);
          console.log('Using audio device: ' + audioTracks[0].label);
          stream.oninactive = function() {
            console.log('Stream ended');
          };
          window.stream = stream; // make variable available to browser console
          audio.srcObject = stream;
        }

        function handleError(error) {
          alert('navigator.MediaDevices.getUserMedia error: ', error.message, error.name);
        }

        newSinkID = vars[code];

        navigator.mediaDevices.getUserMedia(constraints).then(handleSuccess).catch(handleError);


        //const MyEnglishAudio = document.getElementById("englishPlayer");
        audio.setSinkId(newSinkID);
        console.log("Setting Audio to: "+newSinkID);

        if(code == "EN") {  
            $("#englishEnglish").prop('muted', false);
            $('#englishEnglish').prop('volume', 1); 
            console.log("Setting English Volume on "+floorSource);
        }       
    }
        function handleRemoteError(error) {
          if (error) {
            alert(error);
          }
        }




    var apiKey = 'HIDEEN';
    var sessionId = 'HIDDEN';
    var token = 'HIDDEN';

    socket = io.connect('/', {
        secure: true,
        'reconnection': true,
        'reconnectionDelay': 1000,
        'reconnectionAttempts': Infinity
    });


    socket.on('disconnect', function () {
            console.log("Disconnected from the server"); 
            //location.reload(true);            
        }); 

     socket.on('send', function (msg) {
         fromSocket = msg.split(" ");
         to_meeting_id = fromSocket[0];
         to_lang = fromSocket[1];
         action = fromSocket[2];
         who = fromSocket[3];

    if(to_meeting_id == '17') {      
        console.log("Action by "+who+": "+action+" for "+to_lang);
        var info = msg.split(" ");
        console.log("0: "+info[0]+" 1: "+info[1]+" 2: "+info[2]+" 3: "+info[3]+" 4:"+info[4]+" 5: "+info[5]+" 6: "+info[6]+" 7: "+info[7]);

        if(action == "english") {
            $('#floorEnglish').bootstrapToggle('on');
            $('.Switch'+info[7]).bootstrapToggle('off');
            useraudio = document.getElementById(who);
            MynewSinkID = vars['EN'];
            //useraudio.setSinkId(MynewSinkID);
            useraudio.setSinkId(MynewSinkID)
            .then(() => {
              console.log('successfully set the audio output device - Note Unmuting '+who);
                    $("#"+who).prop('muted', false);
                    $('#'+who).prop('volume', 1);
                    $('#'+who).css("background-color", "green");                    
            })
            .catch((err) => {
              console.error('Failed to set the audio output device ', err);
            });
        }
        if(action == "englishOff") {
            $('#floorEnglish').bootstrapToggle('off');
            $('.Switch'+info[6]).bootstrapToggle('on');
            // Set Sink Audio Back
            useraudio = document.getElementById(who);
            MynewSinkID = vars[info[6]];
            //useraudio.setSinkId(MynewSinkID);
            useraudio.setSinkId(MynewSinkID)
            .then(() => {
              console.log('successfully set the audio output device');
                    $('#'+who).css("background-color", "white");      
            })
            .catch((err) => {
              console.error('Failed to set the audio output device ', err);
            });         
        }   
            if(action == "privacyOn") {
                $('#floor'+info[4]).bootstrapToggle('off');
            }
            if(action == "privacyOff") {
                $('#floor'+info[4]).bootstrapToggle('on');
            }
    }

    }); 

    console.log("Starting Session");
      var session = OT.initSession(apiKey, sessionId);
        var AllUsers = [];
        i = 0;
      // Subscribe to a newly created stream
      session.on('streamCreated', function streamCreated(event) {
        console.log("New stream in the session: " + event.stream.streamId);
        var subscriberOptions = {
          insertMode: 'append',
          insertDefaultUI: false,
          subscribeToAudio: true,
          showControls: true,
          width: 400,
          height: 100,
          subscribeToVideo: false
        };

        console.log("STARTING LOG OF EVENT");
        console.log(event);
        console.log("END LOG OF EVENT");
        parms = event.stream.connection.data;
        //alert(parms)
        newparms = parms.split(",");
        ConnectingID = newparms[0];
        Channel = newparms[1];
        subscriber = session.subscribe(event.stream, subscriberOptions, handleRemoteError);
        console.log("Connecting to event stream: "+event.stream);
        //console.log("My Lang: "+MyLang+" VS "+Channel);
        item = {}
        item ["Channel"] = Channel;
        item ["RemoteUserID"] = ConnectingID;
        item ["RemoteStream"] = event.stream;
        // See if we have seen this user before. If so, remove them from the array so we can readd them after with new stream info
        jQuery.each(AllUsers, function(i, val) {
           if(val.RemoteUserID == ConnectingID) // delete index
           {
              //AllUsers.splice(i, 1); // Remove previous streams by this user
          }
        });
        AllUsers.push(item);
        console.log(AllUsers);

    subscriber.on('videoElementCreated', (event) => {
        //console.log("Info "+event.target.stream.connection.data);
        var connectionData = event.target.stream.connection.data;
        var info = connectionData.split(",")
        console.log("User ID: "+info[0]);
        console.log("Lang: "+info[1]);
        $('#userID'+info[0]).remove();
        $.ajax({
            type: 'POST',
            timeout: 10000,
            url: '/ajax/who.php',
            dataType: 'html',
            data: { "user_id": info[0] },
            cache: false,               
            error: function (jqXHR, exception) {
                var msg = '';
                if (jqXHR.status === 0) {
                    msg = 'Can not reach network.\n Verify you have internet.';
                } else if (jqXHR.status == 403) {
                    msg = 'Your five digit code did not match. ';
                } else if (jqXHR.status == 404) {
                    msg = 'Requested page not found. [404]';
                } else if (jqXHR.status == 500) {
                    msg = 'Internal Server Error [500].';
                } else if (exception === 'parsererror') {
                    msg = 'Requested JSON parse failed.';
                } else if (exception === 'timeout') {
                    msg = 'Time out error.';
                } else if (exception === 'abort') {
                    msg = 'Ajax request aborted.';
                } else {
                    msg = 'Uncaught Error.\n' + jqXHR.responseText;
                }
                alert("Local Server Request: "+msg);                        
            },
            success: function (data) {
                $('#'+info[1]).append(data);
                document.getElementById(info[1]).appendChild(event.element);
                setTimeout(function(){          
                    if (data.indexOf('ADMIN') > -1) {
                        if ($('#Admins').is(':checked')) {} else {
                            $("#"+info[0]).prop('muted', true);
                            $('#'+info[0]).prop('volume', 0);
                        }
                    }
                }, 200);
            }
        });         

        console.log("Video Element Created");
        console.log(event);
        dynamicSinkID = vars[info[1]];
        //alert(dynamicSinkID);
        event.element.setAttribute("class", "InterpreationSource "+info[1]);
        event.element.setAttribute("data-lang", info[1]);
        event.element.setAttribute("id", info[0]);
        event.element.setAttribute("controls", "controls");
        event.element.setAttribute("width", "300");
        event.element.setAttribute("height", "54");
      if (typeof event.element.sinkId !== 'undefined') {
        event.element.setSinkId(dynamicSinkID)
        .then(() => {
          console.log('successfully set the audio output device');
        })
        .catch((err) => {
          console.error('Failed to set the audio output device ', err);
        });
      } else {
        console.warn('device does not support setting the audio output');
      }



      // FIX FOR CHROME BUG SETTING LAST PLAYER CREATED AS THE SINKID FOR All - FAILED
      /*
      setTimeout(function(){
            $('.InterpreationSource').each(function() {

                var playerID = $(this).attr("id");
                var lang_code = $(this).attr("data-lang"); // Get language of each player
                var FixID = vars[lang_code]; // Get SinkID it's supposed to be
                var theFix = document.getElementById(playerID);
                console.log("Fixing "+playerID+" with "+FixID);
                theFix.setSinkId(FixID)
                .then(() => {
                  console.log('successfully set the audio output device - Note Unmuting '+playerID);    
                })
                .catch((err) => {
                  console.error('Failed to set the audio output device ', err);
                });                 
            });     

        }, 2000);
        */      
      // END FIX BUG FOR CHROME

    });         

      });

    session.connect(token, function (error) {
        if(error) {
            alert(error);
        }
    }); 



});


if (!navigator.mediaDevices || !navigator.mediaDevices.enumerateDevices) {
  console.log("enumerateDevices() not supported.");
}

// List cameras and microphones.

navigator.mediaDevices.enumerateDevices()
.then(function(devices) {
  devices.forEach(function(device) {
    //console.log(device.kind + ": " + device.label +" id = " + device.deviceId);
    if(device.kind == "audiooutput" && device.label == "Line 5 (Virtual Audio Cable)") {
        vars['EN'] = device.deviceId;
        console.log(device.kind + ": " + device.label +" id = " + device.deviceId);
    }
    if(device.kind == "audiooutput" && device.label == "Line 1 (Virtual Audio Cable)") {
        vars['PL'] = device.deviceId;
        console.log(device.kind + ": " + device.label +" id = " + device.deviceId);
    }
    if(device.kind == "audiooutput" && device.label == "Line 2 (Virtual Audio Cable)") {
        vars['RU'] = device.deviceId;
        console.log(device.kind + ": " + device.label +" id = " + device.deviceId);
    }
    if(device.kind == "audiooutput" && device.label == "Line 3 (Virtual Audio Cable)") {
        vars['ES'] = device.deviceId;
        console.log(device.kind + ": " + device.label +" id = " + device.deviceId);
    }
    if(device.kind == "audiooutput" && device.label == "Line 4 (Virtual Audio Cable)") {
        vars['TL'] = device.deviceId;
        console.log(device.kind + ": " + device.label +" id = " + device.deviceId);
    }
    if(device.kind == "audioinput" && device.label == "Microphone Array (Synaptics Audio)") {
        floorSource = device.deviceId;
        console.log(device.kind + ": " + device.label +" id = " + device.deviceId);
    }   
  });
})
.catch(function(err) {
  console.log(err.name + ": " + err.message);
});


</script>
</body>
</html>

***** 编辑 ***** 看看我的问题可能与https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/discuss-webrtc/vrw44ZGE0gs/2YJ6yUEjBgAJ

WebRTC 在内部混合音轨，所有音轨的音频实际上作为单个音频流传送到 Chrome。对音量等属性的更改会发送到 WebRTC 以在混合之前应用。因此，不幸的是，这意味着该混音器的输出只能定向到单个音频设备。

以上适用于和媒体元素。

但是，Chrome 中提供了一种解决方法，即通过 WebAudio 进行渲染。为此，仍然需要从混合流中“提取”音频，即使它不一定要发送到特定设备或已静音。 然后，您可以按照此示例克隆远程轨道并通过 WebAudio 进行渲染。

所以现在我只需要弄清楚如何做到这一点。

【问题讨论】：

  • 什么语言？什么浏览器？什么是“玩家”？也许显示一些代码？
  • 我已经更新了我的示例。希望现在这些信息已经足够了。

标签： html google-chrome audio webrtc


【解决方案1】：

所以我想出了如何解决这个问题。我创建了一个函数并使用以下代码通过 WebAudio 循环 WebRTC 音频：

    function addNewStream(event, dynamicSinkID, lang) {     
        var mediaStream = event.element.srcObject;
        var remote_stream1 = mediaStream.clone();
        test_audio_context1 = new AudioContext();
        webaudio_source1 = test_audio_context1.createMediaStreamSource(remote_stream1);
        webaudio_ms1 = test_audio_context1.createMediaStreamDestination();
        webaudio_source1.connect(webaudio_ms1);
        test_output_audio1 = new Audio();
        test_output_audio1.srcObject  = webaudio_ms1.stream;
        test_output_audio1.play();
        test_output_audio1.setSinkId(dynamicSinkID);
        test_output_audio1.setAttribute("controls", "controls");
        document.getElementById(lang).appendChild(test_output_audio1);
        test_output_audio1.setAttribute("controls", "controls");
        test_output_audio1.setAttribute("width", "300");
        test_output_audio1.setAttribute("height", "54");                          
    }

【讨论】：

  • 您能否详细说明您是如何实现这一目标的？我在与多个参与者进行语音聊天时遇到了类似的问题。我想允许每个参与者的音频被路由到一个专用的输出。但是，一旦我更改了一个参与者的输出，所有参与者都会被路由到同一个接收器。
  • 您只是在克隆上述描述中的事件源。然后设置你想要的 sinkid。 Lang 可以从我上面的例子中省略。复制源时，可以将原始静音并设置新的 sinkid。在您创建的克隆上设置 sinkid 只会更改它，而不会更改其他任何内容，因为您使用该 sinkid 创建它。
猜你喜欢
  • 2015-02-27
  • 2018-06-26
  • 2011-08-13
  • 2011-08-17
  • 2011-02-02
  • 2021-10-10
  • 2014-04-21
  • 2010-12-30
  • 2014-03-21
相关资源
最近更新 更多
热门标签
Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode