Our first roundtable discussion focussed on large language models, and what that really means for Speechmatics and its customers. Leading the discussion was Trevor Back, CPO at Speechmatics, with 10+ years experience in machine-learning and AI.

Here are 5 key takeaways that came out of the roundtable. 

1. What is a large language model? 

A large language model is an AI model that's trained on vast amounts of text data. It learns patterns in language and can generate human-like text, answer questions, summarize text, and more. OpenAI's GPT series are examples of such models.  

Large Language models are good at understanding multiple different phrases or ways of saying different topics. They can go into transcripts and seek any line that refers to a particular topic like ‘safety’ for example and find any other lines that refer to that. This is extremely helpful in Contact Centers, where you can examine what the most regular complaints are and gather feedback based on these findings. Seizing the potential of LLMs allows your business to explore abundant interactions with transcripts, leading to valuable advantages. 

2. How do LLMs impact Speechmatics? 

The Speechmatics team have recently been exploring summarization across multiple transcripts. By recording internal meetings, processing these transcripts and summarizing discussions, we have been able to gather key takeaways, action items, company blockers and any broader themes and gaining a lot of company insight. The stacking of LLM’s, using it not only for transcripts but also the summarization, can get you a lot of additional gains. Check out a quick demo of the summarization feature in our portal.  

3. Security issues and data retention 

Utilizing a closed API that’s offered, comes with all the standard cloud-based security issues. Trevor explains that Speechmatics is discussing internally hosting open-source language model on premises. Still a lot of discussions to be had but Trevor makes note, ‘’I think there's opportunities there to solve some security issues by having an open-source model on prem.’’  

In certain countries, there are lots of challenges around data retention and LLMs. There’s a lot more opportunity now with more people bringing these models on in data centers, it’s not just OpenAI anymore. The EU has a lot more regulation around it than the United States. When it comes to transcriptions in a different language, Speechmatics has got you covered.  

4. Use Cases 

LLMs and their stacking potential create numerous opportunities across industries. Here are a few use cases Speechmatics wants to emphasize.  

• Media (content creation, advanced archiving, robust monitoring) 

• EdTech (extracting course content from audio, faster note-taking/revision)  

• Calls (summarize outcomes, extract resolutions, highlight outstanding issues) 

• And more!  

5. Friction Points & the Future

Extracting industry-specific points 

During the roundtable, a question was raised about extracting industry-specific points. Trevor responded that topic detection allows you to choose any topic you desire. There’s an opportunity here to leverage the language model’s capabilities to generate topics by providing it with internal documents and key terms from your custom dictionary. This approach enhances accuracy and demonstrates the flexibility of LLMs.  

Where is the source? 

ChatGPT is based on the LLM architecture, although when providing information, it does not provide the source. This was a friction point that was raised in the round table. Pointing to the source is extremely important for customers, although nobody has necessarily solved this issue. This is something that Speechmatics is thinking about as well as a different model that can go back into the information and pull out where it thinks that summarization came from. 

Labelling speaker names 

Currently transcripts are not labelled by the speaker. Through building internal tools and playing with LLMs, labelling transcripts by the speaker has been found to significantly enhance transcription results and insights. It's not just about, giving labels to speakers, it's also about giving more context before the transcript and the prompt of person A from Company X, and person B from company Y. There’s a lot of little tricks to help the diarization problem. 

Previously building a new machine learning feature, could take years' worth of research and years' worth of building. Large language models enable the rapid development of complex systems within a few days, if desired.  

The ever-growing possibilities and benefits of LLMs hold immense potential for driving substantial gains in your business. Speechmatics is well-positioned to support you in leveraging the capabilities of large language models to propel your organization forward. 

We were delighted with our first customer roundtable. We love hearing directly from our customers learning about their pain points and talking through the possibilities of speech and AI. We'll be doing more of these in the future, so stay tuned for more insights from our customers. If you'd like to talk about how LLMs can be used in your business, look out for more events we'll be running, or get in touch.