Unable to use microphone (affects Duolingo, Google Translate, and other sites)

[MGRussell]

Latest status (this comment edited by @bsclifton)

This feature relies on a Speech API key that we would need to create and pay for on a per-use basis. As captured below, @tomlowenthal has looked at this.

At the moment, this is on hold. We have some other features requiring this speech to text API also. But we haven't taken any action as creating a token and installing will have a cost associated with it.

Original issue details by @MGRussell

This issue was previously opened in the browser-laptop repo here and was archived without resolution with numerous OS/versions chiming in. I am reopening it here, and reporting with Windows 7 with Brave Version 0.61.51 Chromium: 73.0.3683.75 (Official Build) (64-bit). This may be a related issue which mentions that Webspeech API is disabled within Brave.

Description

The browser does not function properly with Duolingo. The site asks for permission to use the microphone and everything seems to work fine, but it seems no input is ever being sent from brave to the site. The site only officially supports chrome and so this is something that would need to be remedied on Brave's end, if the browser is to be capable of using Duolingo.

Steps to Reproduce

The browser fails on any tests from Duolingo that asks the user to send voice data, 100% of the time.

Website problems only:

The issue does not resolve when disabling brave shields. This issue is not present on the latest version of Chrome.

[honza-zidek]

Hey guys, for me it's a show stopper for Brave! Please do something with it - the issue was first reported on 03.10.2016 in #4476, so 2.5 years ago!

[rebron]

@honza-zidek Feedback noted.

[Brave-Matt]

+1 -- a couple users reporting on community:
https://community.brave.com/t/twitter-videos-duolingo-audio/61126/6

[estebanhst]

I have the same problem with Brave
Version 0.64.77 Chromium: 74.0.3729.169 (Official Build) (64-bit)
on Mac OS Mojave 10.14.4

Please, as some users have said, this is an issue that has persisted for years. There is no problem with the website while using Chrome.

[gnarfle]

I've also noticed this. On mac weirdly when I start a question on duolingo it instantly fails. It's not like it doesn't get the audio, it just somehow immediately marks it as wrong before I can even speak. Works fine in chrome.

[bsclifton]

Trying to reproduce here - I've created an account (bsclifton if anyone wants to add me) and am working through Spanish exercises 😄Still haven't hit a microphone one, but when I do I have some thoughts on what to try

[bsclifton]

I'm on macOS Mojave 10.14.5 and was shown this the first time a microphone prompt came up (after the web notification was shown)

After clicking OK and then clicking the microphone icon as reported here, it fails instantly:

[honza-zidek]

That's good that you reproduced it, after several years since the error was first reported (the microphone non-working in Brave was first reported on 03.10.2016 in #4476). Now there might be a chance someone will address the issue :) (Well, I admit it is a sarcarsm... but I somehow cannot understand how such a bug could have been ignored by the Brave team for so long time).

[bsclifton]

@honza-zidek I'm sorry about that - there are many issues we personally find important and would like to fix (like this one), but it's all a matter of relative prioritization and having enough time ☹️

After reproducing, I tried toggling a few flags on brave://flags, but didn't notice any difference. @honza-zidek can you confirm: you're supposed to press (and hold) the microphone button and then let go after you finish saying the word?

That is how I tried it and it did not work. I verified that the site has proper permissions in settings.

Just today, we accepted a pull request by @jumde to disable field trials (we were using a test config, which is less than ideal) #4551. Our next Nightly build will include that patch and I'm interested if that has any impact. Will try that and report back

[honza-zidek]

you're supposed to press (and hold) the microphone button and then let go after you finish saying the word?

You are supposed to press the microphone, then release it, then say the phrase.
If I remember correctly, I think the way you describe works in Mondly. You may also try creating an account in Mondly or Memrise or other language learning app - they also have some speaking exercises.

The easiest way to try it is using Chrome for Duolingo.

However, I think that it is a more general issue. As I understand from #4476, microphone has never worked in Brave, has it? It seems not only Duolingo specific issue.

[bsclifton]

@honza-zidek good call - just tried and verified in Chrome...

Click once is supposed to enable it- you say the words, then push stop when done.

Microphone definitely works (in Brave) in other situations. For example, Google Hangouts worked even in the old Electron-fork browser. Will report back any other findings 😄

[bsclifton]

Confirmed that with the new Nightly (0.68.2), this still does not work ☹️ Will need some more investigation

[simonhong]

Below test script throws network error. I think this is related with Google speech API.

var recognizer = new webkitSpeechRecognition();
recognizer.lang = "en-US";
recognizer.onerror = function(event) {
  console.log(event.error)
};
recognizer.start();

[simonhong]

Recording in duoringo and voice search in google.com give same below log. Aborting with error SpeechRecognitionErrorCode::kNetwork

[25606:40711:0626/155300.097507:VERBOSE1:speech_recognizer_impl.cc(542)] Device parameters: format: 1, channel_layout: 2, channels: 1, sample_rate: 48000, frames_per_buffer: 128, effects: 64, mic_positions: , hw_cap.min_frames_per_buffer: 128, hw_cap.max_frames_per_buffer: 4096
[25606:40711:0626/155300.097612:VERBOSE1:speech_recognizer_impl.cc(566)] SpeechRecognizerImpl starting audio capture.
[25606:40711:0626/155300.097642:VERBOSE1:speech_recognizer_impl.cc(586)] SRI::output_parameters: format: 1, channel_layout: 2, channels: 1, sample_rate: 16000, frames_per_buffer: 1600, effects: 0, mic_positions:
[25606:40711:0626/155300.097676:VERBOSE1:speech_recognizer_impl.cc(615)] SRI::input_parameters: format: 1, channel_layout: 2, channels: 1, sample_rate: 48000, frames_per_buffer: 4800, effects: 64, mic_positions: , hw_cap.min_frames_per_buffer: 128, hw_cap.max_frames_per_buffer: 4096
[25606:40711:0626/155300.548512:VERBOSE1:speech_recognition_engine.cc(380)] Downstream complete success: 0 response_code: 403
[25606:40711:0626/155300.548606:VERBOSE1:speech_recognition_engine.cc(841)] Aborting with error SpeechRecognitionErrorCode::kNetwork
[25606:40711:0626/155300.550244:VERBOSE1:speech_recognizer_impl.cc(835)] SpeechRecognizerImpl closing audio capturer source.
[25606:40711:0626/155300.551116:VERBOSE1:speech_recognizer_impl.cc(729)] SpeechRecognizerImpl canceling recognition.

SpeechRecognitionEngine interacts with google service -https://www.google.com/speech-api/full-duplex/v1 for speech recognition.
See https://cs.chromium.org/chromium/src/content/browser/speech/speech_recognition_engine.cc?sq=package:chromium&dr=C&g=0&l=34

cc @rebron I think this issue needs google's speech recognition service.

[bsclifton]

Per discussion with @simonhong, this is also failing in Chromium

To use this, you need to request keys:
https://www.chromium.org/developers/how-tos/api-keys

And we'd want to proxy the call on our side, of course

[rebron]

cc: @tomlowenthal for api-key

[simonhong]

@jumde Whenever when user tries to use voice recognition service such as clicking mic button in google.com, brave will send request to google service - https://cs.chromium.org/chromium/src/content/browser/speech/speech_recognition_engine.cc?sq=package:chromium&dr=C&g=0&l=34

[tomocrafter]

same problem here, I cannot use microphone with Duolingo, Google Translate on Brave Browser.

[tildelowengrimm]

This Google Speech API topic spans multiple issues on this repo. I'm working on it in brave/internal#608. When I resolve things there, I'll come back here with the next technical steps.

[deondon]

+1 for this issue with Google Doc Dication Feature.
It's the only thing I still use Chrome for everything else is now Brave.
There was an audio issue with clicking that has been fixed so thanks for that!

[aploftus]

any movement on this? this is a huge accessibility hindrance for voice-to-text.

[sleazypikachu]

has anyone made any progess with this?

[AdalDesign]

For the sake of communicating the importance of this, here's my personal, friendly bump. Thank you all for your hard work!

[iefremov]

@AdalDesign we are actively working on it. Hopefully will soon start testing in Nightly.

[igroglaz]

Please fix it, folks ❤️ Very annoying thing to switch to Chrome only to use GT

[mkdizajn]

Second this - this is a huge deal breaker!
👎 On desktop - linux, Ubuntu 22 LTS
👍 On mobile - android - works as expected

p.s. one thing that can be of help for developers maybe:

speech to text works in brave - while dictating in word @ microsoft365.com

[arabnia]

Any updates?

[snowbound]

Sorry to say but based on my years of experiences with Brave and submitted issues, I do not hold out hope for any bugs getting fixed in Brave.

[bsclifton]

Hi folks - unfortunately, there hasn't really been any update since the post in 2019
#3725 (comment)

Google Chrome and Microsoft Edge work because they have the backend services to handle the audio transcription. We could enable the service for our Google API key, but this has a cost to us per minute of audio transcribed. This is a service called Cloud Speech-to-Text API that Chromium based browsers can use.

I've re-raised internally to help find closure. There is not a bug in the code here @snowbound. We could disable these Speech to Text APIs completely and that would mask the problem. The sites wouldn't show the microphone icon, etc.

It's important to realize that even if we got these services for free, this is sending customer audio to Google services. I don't think we have any guarantee for what happens once the audio leaves your machine. There are privacy concerns that would need to be resolved before it could be enabled.

[mkdizajn]

Hej @bsclifton, how do you explain that speech to text Works @ microsoft365.com site?

[bsclifton]

@mkdizajn I'm not sure- do you have an example? I have office 365 subscription. How can I reproduce a transcribe with microphone? I can try to see what is being done.

It's possible they are not using the browser's speech to text API and instead have something else implemented on that website specifically

[mkdizajn]

Hey @bsclifton - couple comments before:

#3725 (comment)

I've captured in gif example of successful speech > text on 365 site - very easy - click of the button, standard browser enable access to speech and that's it.

So my point is - if that works on 365 site (while not on others) - issue is then per site - and not browser as a whole!

Kindly, Kres

[derde]

The microphone doesn't work with microsoft teams (you don't have a microphone ,they say). But not to worry, sometimes it does work. Oh wait, it's broken again. So it's the site, not the browser. Except it always works outside of Brave. No, it's the browser.

[bsclifton]

OK folks - thanks for your patience

There are a few folks (@boocmp @iefremov with help from @evq) that have been working on a private and secure implementation of Speech To Text here at Brave - it's in a good working state as far as development goes. Before we look at bringing the online and wiring the browser up to this service, there is some cleanup that needs to take place and also more testing (including load testing).

I don't know an ETA for that unfortunately - maybe @iefremov can share an update as more information becomes available.

[spidy0x0]

Congratulations on the milestone. Hopefully, it gets deployed soon 🫰🏽

[NayamAmarshe]

Eagerly waiting for this! I have an app that uses speech recognition but I have to warn Brave and Firefox users that their browsers aren't supported.

[MichaelMF3]

I have been working on creating a Laravel/JS application which uses Speech To Text so that users can speak to an AI prompt for both accessibility and use as a language learning platform. I'm experiencing the same issue as stated by the user @NayamAmarshe where it doesn't work on Firefox or Brave.

Do you have an estimate for release for this? I can understand if there's specific hardening steps I need to put into the code such as with CORS errors, but based on this thread I don't believe that to be the issue here. This however does work on Chrome which is what brave is based on.

[jhubbardsf]

@MichaelMF3 I've been following this thread years since I started building my own text to speech library. But my mic works fine in google meetings/google voice (still in Brave). Does your microphone work in other websites or just not the app you're building? It's been so long so since I've looked at this issue and now I'm rethinking it.

[alicercedigital]

We need this feature as well. Without it, I still have to open Chrome.

[MichaelMF3]

Applications such as zoom meetings, voice recorder apps work. Things like the microphone button on google translate do not unless I use chrome instead. It seems this issue is specific to speech to text functions and not microphone usage globally. @jhubbardsf I would say that if google is not working properly then that is a sizable issue.

[MichaelMF3]

Apparently the Speech to Text API is missing from Brave and something about ublock origins is the cause. I got a descriptive error message after installing a popular addon but it won't let me post it here for some reason. Extension with the error message: Voice In Speech To Text Dictation.

[Maximon9]

It's been almost 6 months since this update.

OK folks - thanks for your patience

There are a few folks (@boocmp @iefremov with help from @evq) that have been working on a private and secure implementation of Speech To Text here at Brave - it's in a good working state as far as development goes. Before we look at bringing the online and wiring the browser up to this service, there is some cleanup that needs to take place and also more testing (including load testing).

I don't know an ETA for that unfortunately - maybe @iefremov can share an update as more information becomes available.

I was wondering if there were any new updates on this.

[bbondy]

We ran into some roadblocks with the previous approach but are currently looking into Lingvanex speech recognition and LLM services for this. There's still hope ;)