AR Live Captioning Glasses provide real-time accessibility. Improve inclusion and communication for the deaf. Discover more!
With AR captioning glasses, you may enter a new era of communication, where real-time transcription brings discussions to life before your eyes. In this digital age, technology has revolutionized the way we communicate and interact with the world around us. One such groundbreaking innovation is the development of Augmented Reality (AR) Live Captioning Glasses and Hearing Aid Vision. These cutting-edge devices are transforming the lives of individuals with hearing impairments, providing them with enhanced communication experiences and breaking down barriers.
Have you ever found it difficult to converse in social settings—bars, restaurants, and business meetings—due to hearing loss? Wouldn’t it be great if real life had captions, allowing you to see a transcript of discussions in real time right in front of your eyes? This is no longer a far-fetched idea, but rather something made attainable by augmented reality (AR) glasses.
There are automatic captions everywhere in today’s environment. They may be found on YouTube, social networking sites, and video communication services such as Google Meet, Zoom, and FaceTime. People with hearing loss who wish to follow a TV show or comprehend a video chat might benefit greatly from closed captioning.
AR is a digitally augmented rendition of a real-world environment that incorporates digital visual components, audio, or other sensory cues. Live-captioning glasses employ augmented reality to project a captioning service into your actual life.
AR captioning glasses can help you with everyday communication, which means you no longer have to rely on body language and speechreading to fill in any missing terms in conversation and can communicate with more comfort. Simply put on the captioning glasses and you’ll be able to “see” the conversion taking place in your range of vision, exactly as when watching television with captions.
Although captioned glasses have been available for a few years, early versions were aesthetically “chunky” and would not have been pleasant to wear for long periods of time. They are becoming more lightweight, and some even include new revolutionary capabilities other than captioning. This means they may be worn in a variety of settings, including talks, the cinema, theatre, lectures, meetings, and so on.
As you’ll see below, numerous captioning glasses are now “under development,” with some available for testing right now. When imagining the new world of captioned glasses, here’s a fast overview table of some of the greatest, most promising technological businesses, followed by more extensive information about each product.
We’ll also keep this page updated when new firms and models enter the market—so bookmark it!
| XRAI Glass | Vuzix Ultralite | Xander Glasses | Transcribe Glass | LEION Hey AR | |
|---|---|---|---|---|---|
| Weight | 2.6 oz* | 1.3 oz | Not available | Target: 1 oz | 2.8 oz |
| Live speech-to-text captioning | ✓ | ✓ | ✓ | ✓ | ✓ |
| Translation | ✓ | ✓ | In future | ✓ | |
| Removable shades | ✓** | ✓ | Upon request | Clip device on shades | ✓ |
| Works w/ prescription lenses | ✓ | ✓ | ✓ | ✓ | ✓ |
| AI Assistant / ChatGTP | ✓ | ✓ | ✓ | ||
| Conversation recorder | ✓ | ✓ | In future | ✓ | ✓ |
| Wireless phone connection | ✓*** | ✓ | Not required | ✓ | ✓ |
| Android & iOS compatible | ✓ | ✓ | Not required | ✓ | ✓ |
| Available for ordering | ✓ | ✓(for OEMs) | Oct. 2023 | Kick starter | |
| Price | $455 | TBD | $95† | $699-899†† |
The following is a quick reference table of the features present in the AR captioning glasses. *XRAI glasses start at 2.6 oz in weight; **Only available in TCL NXTWEAR S; ***In TCL RayNeo X2 versions scheduled for Q3 2023; TranscribeGlass target price; Kickstarter pricing.
XRAI App and Glass of AR Live Captioning glasses
Prices begin at €406.00 (about $455) plus app subscription.
Dan Scarfe and Mitchell Feldman, two British men, established XRAI Glass in the United States. The term XRAI (pronounced x-ray) is a combination of XR (mixed reality) and AI (artificial intelligence). Scarfe got the concept for the XRAI Glass AR smart glasses after observing his 96-year-old grandpa straining to converse across the dinner table one Christmas Day owing to hearing loss. Scarfe’s objective was to “break down communication barriers and promote communication between everyone.”
The XRAI software works both with and without the company’s glasses, which means you may benefit from live captioning of speech via captioned glasses that use the XRAI software or simply by using the free XRAI app on your phone or tablet. The XRAI software’s initial iteration, released in November 2022, enabled the XRAI Glass captioning glasses. The software’s most current second edition now supports three types of glasses:
- XREAL Air (comfort-adjustable), T
- CL NXTWEAR S (with detachable sunglasses),
- Rokid Max (with myopia/nearsightedness adjustment for eyeglass users).
These glasses’ directional microphones take up the voice of the person in front of you and digest the information quickly. Only you can see a live transcript of the discussion shown on the inside of the glasses in the center of the screen. If numerous persons are speaking at the same time, the app’s AI will recognize each voice and label it separately on the screen.
XRAI Glass collaborates with Lensology to ensure that all XRAI-powered glasses are compatible with prescription lenses. Simply inform Lensology what type of glasses you have and submit them your prescription; they will send you an insert to put inside the glasses.
To function, the glasses must be hooked into a smartphone, which requires the XRAI Glass software, which is available for both Android and iOS. The software is subscription-based, with three options: free Essentials, Premium, and Ultimate, which provide access to more features.
XRAI software has the following extra features:
- Translation: Additional AI suppliers, such as Microsoft and Deepgram, have joined Amazon to give accurate subtitles and translation into 76 languages and 140 dialects, with and without an internet connection.
- Conversation recorder: The programme saves your talks, which you may retrieve whenever you want. According to XRAI, it does not have access to this data, and users have complete authority and responsibility over how it is used or shared.
- Personal Assistant: The XRAI Glass has a ChatGPT-powered personal assistant that allows the user to ask complicated questions and receive answers in the form of closed captions.
Though the current variants of XRAI Glass-compatible glasses require a wire connection to your phone, the wireless RayNeo X2 glasses are set to be available in the coming months. The battery and CPU will be incorporated within the spectacles.
XRAI Glass has received several awards, including the “Technology For Good Award” – Global Business Tech Awards (2023), the “Startup Tech Company of the Year” – National Technology Awards (2023), the “Best Assistive Technology App” – Assistive Technology Awards (2023), and the “Smart Tech Award – Highly Commended” – Disability Smart Tech Awards (2023).
Smart glasses Vuzix Ultralite (for OEMs) for AR Live Captioning Glasses
Vuzix, located in Rochester, New York, is a prominent supplier of wearable display technology, virtual reality (VR), and augmented reality (AR). Vuzix has created an original equipment manufacturer (OEM) platform, which means that other firms may purchase it and incorporate it into their own products.
According to Vuzix CEO Paul Travers, the business plans to “make the dream of sleek and fashionable smart glasses a reality for the broader market.” He goes on to say that the new technology platform would be made available to third-party OEMs in order for them to create a variety of new applications: “The Vuzix Ultralite OEM solution will enable leading consumer technology firms to accelerate the deployment of AR smart glasses solutions to the masses.”
They exhibited its Vuzix Ultralite smart glasses platform at last year’s Consumer Electronics Show (CES 2023). The glasses are intended to be a full wireless solution that utilizes information from Android and iOS mobile phones for anything from live voice captioning to other digital AR applications such as language translation, visual/contextual search, text notifications, and directions. A developer’s kit will be available to add further features, such as step count and heart rate, allowing you to track your health throughout the day.
The glasses use superior monocular waveguide optics and small display monitors (about the size of a pencil eraser) to provide a clean transparent picture over the wearer’s real-world perspective. They have enough battery power to last two days on a single charge, according to reports. They are also smaller and lighter than earlier AR glasses, weighing less than 1.5 ounces. Vuzix products are also prescription-ready and extremely customizable, with “virtually endless configuration options around size, colour, form factor, and optical setup to set your solution apart.”
When you put on the glasses, they link to your phone through Bluetooth, eliminating the need to take your phone out of your pocket or purse. All critical information from your smartphone is presented on the glasses’ display screen. The idea is that you’ll be able to keep your head up and your hands free all day.
The company has received CES awards for innovation from 2005 to 2023, as well as other wireless technology innovation awards.
Vuzix Ultralite glasses are not yet on the market, however you may contact Vuzix to learn more about their Vuzix Ultralite OEM platform (also see XanderGlasses below).
XanderGlasses of AR Live Captioning Glasses
Alex Westner and Marilyn Morgan Westner co-founded Xander, a group of audio professionals, engineers, and researchers. Xander utilizes augmented reality to deepen in-person talks. They have collaborated with Vuzix, and their first product, Xander captioning glasses for those with hearing loss, is based on the Vuzix Shield model.
Alex Westner spent 20 years in the audio business after getting an MS in acoustics and audio processing from the MIT Media Lab. He worked with industry giants Gibson, Mitsubishi Research Labs, and iZotope. After being diagnosed with a visual issue, he got fascinated by sensory replacement and “the most profound human problem related to audio”: assisting those who have difficulty hearing. XanderGlasses evolved as an assistive technology to help persons with hearing loss interpret noises and conversation more clearly.
XanderGlasses include a comfortable fit and an ergonomic design. They are easy to use and may be worn right away, eliminating the need to link them with your smartphone or install an app. Simply press a button to turn them on, and several noise-cancelling microphones surrounding the frames gather up clear audio from the dialogue around you and feed it directly to the lens as speech-bubble subtitles.
Xander claims that their glasses provide an average of 90% accuracy in captions in noisy circumstances such as packed restaurants. In less noisy environments, accuracy is significantly higher.
The glasses charge through a USB-C interface, and the battery life is projected to be 6-7 hours of talking time.
XanderGlasses employ a cloud/offline hybrid technology. The glasses may provide the maximum accuracy when connected to their cloud service through WiFi. If you don’t want to connect to WiFi or are in an area where there isn’t a good signal, the glasses will instead utilise an offline, built-in speech-to-text algorithm that apparently still achieves excellent accuracy. XanderGlasses does not keep your interactions, whether online or offline, in accordance with SOC 2 standard compliance regulations.
The firm is actively developing a translation tool that will be made available in the near future.
In case you need to use them in places that demand safety glasses, the glasses include tough lenses that fulfil ANSI Z87.1 and EN166/170 safety regulations. Prescription inserts will also be available, as will clip-on sunglasses for outdoor use. Because the glasses are software-based, they will be upgraded and enhanced over time.
PHOTO Of xanderglases
The price of XanderGlasses is presently unknown, but the business expects it will be equivalent to the cost of a typical pair of prescription hearing aids. According to the corporation, accessibility for all—regardless of poverty or wealth—is vital to them, and they are actively collaborating with organisations and agencies that may assist individuals in paying for their products. They’re also collaborating with US veterans’ organisations and audiologists to test and enhance the XanderGlasses, all while helping hard-of-hearing veterans reclaim their feeling of connection with their families, friends, and communities.
On August 15, 2023, XanderGlasses will be available only in the Boston region. It will be a local event where folks may try on and buy the glasses. If all goes well, they want to have XanderGlasses available in North America by early October, and then worldwide.
In addition to winning a Catalyst Award from the National Academy of Medicine, Xander won the CTA Foundation’s Eureka Park Accessibility Contest at CES 2023.
Xander has operational prototypes, albeit they are not yet for sale. You may join a waiting list to be informed on their progress while they strive to enhance their design.
TranscribeGlass of AR Live Captioning Glasses
Currently in beta testing, with a target price of $95
When a classmate dropped out of school because he couldn’t afford accessible accommodations, Madhav Lavakare of New Delhi, India, came up with the idea of captioning glasses. Lavakare set out to solve the problem.
TranscribeGlass focuses on affordability as well as comfort, making it an accessible and cheap assistive technology solution for persons who have hearing loss or difficulty interpreting spoken communication. TranscribeGlass is not a set of glasses, but rather a heads-up display that clamps onto the frames of your spectacles, or a pair of empty frames (included in the box) if you do not use glasses.
The gadget connects to a companion mobile app, which will be accessible on both iOS and Android, from which you may choose your preferred speech-to-text software or live captioning source. The captions are then delivered over Bluetooth Low Energy to the TranscribeGlass heads-up display. The captions are layered in real-time over the user’s vision of the surrounding real-world environment, and the caption font size and location may be adjusted. The software also allows you to examine the transcript history.
Lavakare and his colleagues are hard at work polishing and perfecting the TranscribeGlass design. The device’s target weight is less than 1 oz, and the target price is $95.
TranscribeGlass Beta is TranscribeGlass’s initial restricted release, and it costs $55. There is a beta queue on the website where early adopters may reserve the gadget and agree to contribute feedback as testers. Feedback from testers will be critical in identifying faults in the product and mobile app as well as enhancing the design.
TranscribeGlass was “Highly Commended” in the Accessibility category of the Tech4Good Awards 2021 and received the AssisTech Foundation (ATF) Award for “Best AT Start Up for Innovation in 2021.”
CEO Tom Pritsky and intern Desiree recently attended the 2023 Hearing Loss Association of America (HLAA) meeting, where they showed TranscribeGlass.
LEION Hey AR glasses
Kickstarter reward bundles range in price from $699 to $899.
The Leion Hey AR Glasses were produced by Beijing LLVision Technology Co., Ltd to enrich your everyday experiences by delivering real-time transcription and translation capabilities, as well as integrated chat functions. These glasses, created for the deaf, are wireless all-in-one AR and AI communication enhancers with live voice transcription in milliseconds. They are iOS and Android compatible.
The Leion Hey AR Glasses employ two microphones for noise reduction, which may be tuned to improve radio reception. According to the business, in a 70 dB noise setting, the accuracy rate of sound transcription at 5 metres may approach 85%, and the audio material is translated into text and displayed in 500 milliseconds.
PHOTO OF LEON HEY
They are lightweight (2.8 ounces), so you may wear them for lengthy periods of time without discomfort. They incorporate a Binocular Light Waveguide Fusion Display, which provides good view of anything in front of your eyes without inducing dizziness. They are also weather resistant, so you may wear them in any weather without fear of things breaking.
When you first use the Leion Hey Glasses, you must connect them to your phone through Bluetooth and WiFi using the companion app. Then you sign up as a user, go through the basic setup, and link your glasses with your smartphone. This is only necessary once because the glasses remember the settings for future usage. You have complete control over all functions.
These glasses provide three essential tasks by utilising AR and AI:
- Conversation mode: This is where you’ll discover the live captioning feature, which, as predicted, converts voice into text that appears on the screen. These smart glasses also record your chats with others as well as the messages you dictate.
- Leion is in translation mode. Hey, Smart Glasses can recognise and translate more than 90 languages. To utilise the translation tool, open the app and choose the language in which the individual is speaking. It will interpret the voice and turn it to text on the screen within the glasses in front of you.
- ChatGTP: The ChatGTP capability allows these glasses to work as an AI assistant, allowing you to ask ChatGTP questions and receive answers on the screen.
The Leion Hey AR Glasses charge through a USB-type C-cable and offer four kinds of sunglasses attachments that link magnetically to the front of the glasses, allowing you to use them in bright circumstances. You may also see an optometrist to get your glasses customised with prescription lenses.
Visit the Leion Hey AR glasses Kickstarter page for additional details.
Will Google (re)launch its AR glasses?
Google Glass smart glasses have been in development for some years, and a public retail version was introduced in 2014. Google Glass allowed users to view material and do operations such as taking phone calls, browsing images, and checking messages without using their hands. The technique needs a long battery life, enhanced picture recognition, and a large amount of data. They eventually ceased selling the smartphone on March 15, 2023.
However, further innovations may be on the horizon. Google CEO Sundar Pichai revealed new prototype AR glasses at the Google I/O developer conference in May 2022, with the goal of breaking down communication barriers by providing “subtitles for the world.” Google Translate is embedded into these glasses, and a business video demonstrates AR glasses translating languages and captioning audio in real time. Though this product is not yet ready, it is one to keep an eye on!
Captioning apps
While we’re on the subject of glasses that can deliver real-time captioning, it seems appropriate to highlight several popular captioning applications for your smartphone. Furthermore, because certain smart glasses, such as Vuzix and TranscribeGlass, work in tandem with your smartphone and/or your favourite speech-to-text app, the software on your phone may have a direct influence on your AR glasses’ captioning capabilities.
Various smartphone apps can rapidly and efficiently convert voice to text, allowing you to benefit from live captioning while on the road.
Look into:
- XRAI Glass (iOS and Android)
- NALscribe: live captions (only for Apple iOS)
- Google Live Caption-AI Voice to Text (Only for Android)
- Google Live Transcribe and Notification (Only for Android)
- Ava (Android and iOS)
- Hearing Aid – Live Captions (Only for Apple iOS)
- Otter: Voice Notes Transcription (Android and iOS)
- Android and Apple iOS Speech to Text Transcription
The Captioning Glasses of the Future
It’s reasonable to anticipate that more of this sort of technology will enter the market and become widespread. With AI programmes like OpenAI and Visual ChatGTP rapidly expanding in capacity, there will undoubtedly be a greater need for wearable AI-compatible solutions in the future. Furthermore, with various third parties creating applications for Vuzix and other micro-display platform technologies, we may anticipate increased market rivalry.
