[Automatic captions by Autotekst using OpenAI Whisper V3. May contain recognition errors.]
We have worked for a long time with transcription here at the university, both since we had made the dictation that you saw earlier, but also because we have lost the text of video from lectures, and we have had a conscious relationship with that at the university for many years.
Fortunately, they came in the fall of 2022.
Technology that revolutionized transcription.
Now we have set up a common, interactive, hybrid show.
So I think I'll just let you get started.
There you go.
Then you can just start organizing.
I will talk with Tor and Magnus at the end about Autotext, both about the news and the experience of using Nationalbibliotekets models.
Does that sound okay?
Yes, we have good signals on that.
It's a bit of logistics, because I'm at the cabin, and we're dependent on Starlink and the satellite there, which is constantly switching between functions, since it's a bit of an empty space.
But we're betting that it will be fine.
Autotext is in many cases the best solution available for Norwegian spoken text, and it is based on Whisper from OpenAI.
It was quite revolutionary when it came out, and it's the only open AI that you can build on your own.
You don't have access to the training data, and it's no secret that they have had some freedom when they have made these models.
The results are available as open source models, and then you can use them yourself, and also, as the National Library has done, build on them and train further.
This is something we do through Autotext UUNO, which is a web application that will be demonstrated later, where you load up your own media files.
which is available to employees and students from customers of this solution.
In addition, we have created what is called Autotext VMS, which means that you can get things done automatically through Panopto, so that it just happens by itself.
It is also available in Net Schema through Net Schema Dictaphone.
It is not directly available in TSD now, but it is natural to see a solution there as well, or at least it has benefited from the experience with Autotext.
It is not yet decided how it will be.
This is something that is available for the sector in Norway.
In practice, all non-commercial actors can order it.
We have customers for Autotext UNO, which is the service that has been the longest, and about half of Autotext DMS, i.e.
that the institution has it directly connected to SIN.
And there has been a lot of use, and together over 50,000 hours that have been transcribed so far.
If you want to order it, there are some websites on the UiO website that you can order it from.
What was very exciting and happened in spring was that the National Library was finished with a work I had been doing for a long time.
And earlier, even if it had come for Norwegian, it was good.
is still not as good as for other languages, such as English.
Many of the problems with Norwegian is that it is the limited amount of training data that is available to train up models for AI-based things, or also for the earlier approaches, where you also managed to get pretty good at English, but not so good at Norwegian.
Nationalbiblioteket is in a unique position, because they have access to more or less everything
of publicised data, from throwing around and so on.
This is a unique approach, and the way these AI models work, where there is a difference between the training and the results, makes it possible to make the end result accessible to everyone, and make it completely open, even if we can't access the training data they have used.
In contrast to when these big AI-actors have done this and taken liberties, this is something that some national libraries have been allowed to do.
It is very nice that we can do that, and that they are allowed to publish it for us, so that we can take it further.
They launched this in February, and we got it out in early April.
They have used many years of work on this, and have done an impressive work that I do not know of having been done elsewhere in the same way.
It has given big improvements for Norwegian.
It has become much easier to understand dialects.
And not least, it is much better support to get new Norwegian out.
Previously, it would have been easy sometimes to come out in Swedish, or that there were mixtures or different things.
It's not perfect, but it's much better now.
with the new models at the National Library.
So in addition to a lot of manual work, they have used a lot of computing power on this, which they largely borrowed or sponsored, and it is a size order that would be quite costly to do these training jobs if you had to pay for it.
But there they have done that job.
If you want to do fine adjustments, for example,
to include examples of specific words and expressions for an industry or a field of expertise, you will be able to handle less training.
So that's something we might consider doing afterwards, where we would then continue to build on what the National Library has done.
But we haven't done that yet.
Then we thought we'd do a little demo of Autotext.
I'll get you connected, and then Tor can take over the screen.
Yep.
So... This is Autotext.
I'll just put it down a bit.
Autotext.uo.no.
As Oddmund said, there are three versions of this website.
Autotext.vms, which is for opto integration, and NetSchema.
This is where you can log in with Feide users, either as an UiO user or one of our other customers.
And then you can either click here or drag your own file in to transcribe it.
I have made a file that I am going to transcribe.
First you get a few different options.
First, you can load up many files at once if you want to do a batch processing of
several files with exactly the same parameters.
But here we only take one.
And as standard, we have set it to recognized language.
This works quite well if you have represented language in the first 30 seconds of the video or the sound file.
Otherwise, we recommend choosing a language if you know what language it is.
And if we then
If we choose a bookmark, for example, we see that the pre-selected next choice we have, which is a language model, and here are the models of these national libraries that Madmun talked about.
As standard, we have OpenAI's version 3 model and version 2 model.
Version 2 model was the one we drove until now in February, when we changed some systems.
Version 3 we have used at the same time as the National Library.
Here we have two models from the National Library.
We have one that is a verbal team,
that we haven't put a suffix on, but that they've called semantic.
The difference is that the verb team tries to transcribe whatever you say, word for word, so that if I cut a little in the sentence or something, they'll start picking it up.
It can be useful for interviews, that kind of thing.
While the verb team finds more meaning and writes it down to what you actually intended to say, to a greater extent.
And because we have taken advantage of another piece of this, another open source code called WhisperX, we can also differentiate between speakers.
So if there are more people talking in the same recording, it will try to differentiate them from each other.
Also very useful in, for example, interviews.
Then we can just send it to texting.
Then it will first load up.
If it's not that much load right now, we can see that it's transcribing.
It's not a very long shot, so it takes a very short time.
It will try to cut out all the possible silence it finds in the video, and that also reduces the production time, but because we use quite powerful cards, like the Dagpinn
Referred to.
Because this is on Fox.
This is on Fox, yes.
I forgot to say that.
Before we had a dedicated machine with IVGPUs.
Now we run all this on Fox, as others work in a completely dedicated project.
with access to red data.
And it doesn't actually use very, very much GPU power, so it uses parts of H100 cards, so-called MIGs, or multi-instance GPUs.
But that was it.
And then we can preview it.
Let's see if I can... Yes, that might be fine.
Yes.
And now the bookmark output.
We can also try to get the output in Nynorsk.
I can just queue it up while I show the video.
Because I know that one of the things with the National Library's models is that it has become much better in Nynorsk.
Earlier we said that they had to take it to bookmark, then they had to go to GPTU and UNO and translate it to Nynorsk, because the quality was so bad for what came to Nynorsk from Whisper.
I'm not really an expert on Nynorsk, but it's so much better now that I think you can read through it and correct it a bit more.
I think it's going to be fine.
Let's see.
Here we have a bit of a mess, you know.
How many of you use the job gathering?
15% We would like to offer a GPT service for employees and students in the first year This is a presentation from last year
We have classified Paul as speaker 0.
If we go a little earlier back in the video.
Yes, that's fine.
Yes, correct.
That's what I saw.
Let's see if I get... Do I not get captions at the top?
That's a bit stupid.
Yeah, cool.
If I do that, I get captions.
This is interesting.
The recordings have not been delayed.
Svein Støhlen was out and said, look what we got here at the University of Oslo.
He is an early researcher himself and knows this with sound recordings.
Very punctual.
We are a little shocked.
Then we got this, the special team, as we call it.
This was really cool.
This was actually from last year, so already then we were aware that we were doing a certain amount of work on this, but we hadn't gotten any
But in February, they launched all these models, and then we could fully use them in production.
We have talked to them.
We have been to a number of presentations, because we have received support from Bufir to make this.
We have talked to them, but we have not started any concrete cooperation with them.
This is how it works, and then you can finally... There, I think.
Then you can download these files, because now I've shown you that you can preview them, you can also delete them, of course.
And then you can download them either in plain text, which is a bit like what we see here in the preview,
two selected subtext formats, SRT and VTT.
VTT is mostly used on the web, and SRT is a slightly older format that you can use in, for example, VLC, video, that kind of thing.
And that's really what I'm going to show, because it's not much worse than that, at least for the user.
Yes, I agree.
I just have to ask, so the National Library had Whisper, you got hold of it, and then they were going to teach it new Norwegian, and then they measured in all the books they had at the library?
Yes, not quite, but they measured it with transcribed sound recordings.
Like NRK and NO, for example?
Yes, from three sources.
If you want to call it that.
Stortinget, that is, recordings from Stortinget.
They are open and transcribed.
Good at dialects, very good at dialects.
A lot of dialects.
NRK, very high quality.
Good at dialects, good at dialects.
And finally, a whole heap of audiobooks.
Where it is well transcribed, of course, since it is read from a text.
I don't remember the exact number of hours, but there were tens of thousands of hours divided into these.
They did a lot of language cleaning to get such good quality.
And you have really noticed that.
For example, the amount of garbage that it finds in periods of silence has dropped significantly with the National Library's models, because they just put in training so that it would ignore silence and not just find things.
Something Alpenai wasn't that good at.
Very strange.
I don't think we'll ever talk about it, because it was so strange what the data machine came up with in periods.
It's very important to look through the recordings to see what happened to the data machine, and then a lot of shit came out of it.
A great example was a lecture.
here from UiO, where something went wrong with the microphone, so that the right and left channel had gone out of phase.
So if you listened to it, it sounded fine, maybe something was wrong, but the machine saw it as total silence.
45 minutes recording or something, and just found a whole heap of nonsense and dirty language and terrible stuff.
We don't experience that now.
I picked up the dictaphone and wanted to show you something, and then something happened, and I put it in my pocket.
And then it went up in my pocket.
It was a bit loud, so I couldn't let it go and send it in.
But then it just came in a different language.
In a very incomprehensible language.
It was Greek or something.
I have a few more slides, Tor, if you can... Yes.
Then we're over with you.
I'll see if I can get you up again here.
There you go.
Can you hear me?
Good.
One of the points is that users now have to relate to several models.
Often it's enough to just choose a language, but sometimes you need, as Tor was saying, to choose between other alternatives.
And in order to get the best possible results, the users need to have a conscious relationship to it, because models have something to say.
In terms of the models that have been used in the last few weeks after we got more accessories, we have...
There's a lot going on with the Norwegian semantic model, and I think that's good, and it's partly because we're using the verbal model.
It's still relatively early, but you should have a certain relationship to that.
There are other places where people have reacted to the semantic model taking too much freedom.
Not many have complained about that in Norway, but it varies a bit.
And it would like to help, like all these AI models, to be good, but it's not always the first year what you actually meant when someone tries to map things and so on.
Some statistics from the National Library, from their first presentation, show that these models are much better, and that is also the experience.
The most common goal for the quality of this is something that
called word error rate, and semantic models that write a little about typically have a higher word error rate than more correct word models, while this semdist goal, which they also had with them, tries to look at equality in meaning and not just equality word for word.
They also had some numbers for different dialects, and there it is also much better than it was.
Thor has been involved in most of this.
We also have better synchronization than before of generated text to video, which is also a side effect of this VisperX.
And then we were a little bit into that with EduCloud Research, which makes us more scalable now than we were before.
We have a completely different architecture, and where we also want to take into account the old ways to keep
and have even more flexibility.
But in general, the fact that we have been able to take advantage of research makes us have a lot more freedom to use newer models and have more flexibility.
The plan is to use Bluemaster again to have even more redundancy.
When EduCloud stops, Autotext also stops.
It does not stop to receive things, but it stops to process things.
Now it has happened a little more often than we like in the period after Easter.
Among other things, a bit of a long-term problem with storage.
So that's why it's good to have more possibilities.
But in general, it's very good and works very well.
There are also some things we need to do, and there is a lot of burden in terms of prioritizing queues of different types.
A little better statistics.
One thing that would make it better is also to find a way to get automatic recognition with model selection.
As it is now, if you set auto, you have already bound yourself to one model.
We can get to it in other ways, but that's something we have to consider.
That's what I had.
You've already asked a few questions.
We're running out of time.
I don't know if we have time for a few more questions.
Absolutely.
Bluemaster, have you started to... It's been thrown out as if everyone knew what it was.
That's right.
Bluemaster is...
The last dedicated machine we had... Down to the right of the figure, down to the right of the... Yes, the one down here.
It was still a powerful machine, but with a special IBM Power architecture, so it was a bit difficult to maintain, and because it was the only processing node we had, we couldn't download and update it without downloading the entire text.
Now we have moved on to transcribing on EduCloud with Fox, which is run by not only us, but a large team here at Vue.
So that does a lot of work.
However, Bluemaster is a very powerful machine that does not have as much use as transcription for the moment, or is difficult to use like many others.
So we have started working with it and getting it back again in some kind of shared service.
It can still transcribe thousands of hours of footage per month, so it's an asset.
There were actually two questions I had.
One is, what kind of information does Visper use?
Do you use the OpenAI thing, or do you use Visper CPP?
I used Visper CPP with great success so far.
The National Library's model was incredibly much better at handling best parents in Nordic dialect form.
The original was unusable, and the new one was fantastic.
But I'm also wondering, are there any good Norwegian models that are able to distinguish speakers, so you can identify who is speaking?
So, two questions, just like that.
Which implementation of Whisper do we use, and are there any models that are good at distinguishing speakers in Norwegian?
We use WhisperX, which is a framework for cutting out sound, segmenting speakers, etc., which again uses the FasterWhisper version.
I've also used WhisperCPP, which works very well on Mac, for example, so you don't need a lot of calculations to do that.
Faster Whisper works much faster than the original version.
Combined with WhisperX, which cuts out silence, things go much, much faster.
If there are any models that are particularly good at distinguishing in Norwegian, I haven't found any.
The one WhisperX uses is one called Pianote.
Pianote Segmentation Diarise, I think.
Yes, can we say that?
Tusen takk.