jespah
 
  3  
Reply Wed 22 Sep, 2010 07:01 am
@Butrflynet,
Butrflynet wrote:
Quote:
Maybe Jespah will stumble upon the thread and enlighten me.


Earlier, I sent her a PM with your question and a link.


I'll try, but I last worked for the company in '06. When they were bought by Nuance, Nuance scrapped the legal product I was working on. I was the SME (Subject Matter Expert). So, heh, nice to see the spelling and pronunciation of terms like res ipsa loquitur is most likely in the legal product. Formatted citations are me (and my friend Alan, who coded them -- he was also let go in The Great Purge of 2006), too.

I suspect that those are many of the differences, e. g. footnoting, citations, etc., plus probably some specific legal terminology, when you buy the legal versus the regular product. Plus even with their scanning technology it may not be easy for the software to figure out how to pronounce all the pseudo semi-Latin there is in the law.

An 85% recognition rate is good and high. But it's midwestern men (I do believe you fit the bill, er, Bill) who get the best recognition scores. East coast women like me (and my boss) would routinely score in the mid-60s to low-70s back in '04. The boss's boss was from Hyderabad -- he often scored in the 20s. Our soft-spoken male coworker from Michigan used to be scored in the mid- to high-90s. We used to score by computer and by hand then, and match the two, see how the computer did versus what we observed. Plus we had a transcriptionist who would type as best as she could and we'd compare the computer versus Melissa.

But that information is all a half-decade old by now. A 99% recognition rate is awesome but I doubt most people get that, even now. And that still guarantees that one word out of every 100 will be mistyped, dropped, doubled or an extra one added. This is not awful and most transcriptionists are comparable if not worse but it is not perfect.

I'd also caution that, even with small issues like the radio, people often put on the "dictation voice", speaking more slowly and carefully, and deliberately choosing their words, avoiding ums and pauses, and in general behaving better, but not the way they normally do. Even if you don't think you're doing that, you might be. Fortunately, the product does work better with training -- but you should be training it a bit with sloppy speech, too, as that is more of a fact of life than the "dictation voice" is.
OCCOM BILL
 
  1  
Reply Wed 22 Sep, 2010 03:36 pm
@maxdancona,
maxdancona wrote:
Ironically you are using Dragon as an example to argue against my point that having a domain specific language model (i.e. a version just for legal text) won't help accuracy.
No, I specifically told you I am not trying to argue with you; other than the FACT that I am getting much better results than you predicted. I've little doubt that you know what you're talking about and my question re: the legal specific software is how much, coupled with the distinct possibility that the area's of law practiced at this firm may actually benefit from it learning our style only, rather than anticipating legal jargon that we don't, or rarely use.

maxdancona wrote:
I think the problem is that you don't understand the process of turning sound into text. It is not nearly as straightforward as you suggest. The big problems isn't the difference between "than" and "that". You haven't understood that that process is a statistical one, and the statistics make all the difference.
I would agree that I don't understand the process, and I appreciate your assistance in helping me to understand it better.

maxdancona wrote:
Here's an interesting experiment-- read the following into your speech recognizer.

"The deafened ant was remanded into the custody of the sherriff".

I suspect if you put an unnatural exaggerated pause in the correct place, you might get the desired sentence. The reason is that in your language model, the phrase "deafened ant" is not very statistically likely, whereas the word defendant is rather likely to be near the word remanded.
Nope; it wouldn’t reproduce that with 5 tries. Defendant and remanded work flawlessly... but it wouldn't reproduce "deafened" until I added "ear" to a sentence; so your point is well made. Thanks!
OCCOM BILL
 
  1  
Reply Wed 22 Sep, 2010 03:42 pm
@JTT,
JTT wrote:

Plus Bill, 600 or 1000 bucks doesn't seem like a ton of money for a law office. Law offices make their money by cranking out paper.
I gather you've never managed a business. That's not a lot of money, to any business; but the opportunity to spend a like amount presents itself constantly and if you go around spending indiscriminately you could run virtually any business into the ground. On the other hand; if the software negates the need for one full-time transcription employee; it might be worth 40 to 60 times that much every year. Think of it like this: Every dollar spent comes out of someone's paycheck, so every dollar should be justified in some way or other... even if it's just a morale booster.
0 Replies
 
OCCOM BILL
 
  1  
Reply Wed 22 Sep, 2010 03:43 pm
@Butrflynet,
Butrflynet wrote:
I'm also not sure if there is a size limit on the various vocabulary and statistical modules in the Standard version that is not in the Professional (Legal) version. Haven't looked into that.
Definitely worth looking into, thanks again.
0 Replies
 
OCCOM BILL
 
  1  
Reply Wed 22 Sep, 2010 03:45 pm
@jespah,
jespah wrote:
An 85% recognition rate is good and high. But it's midwestern men (I do believe you fit the bill, er, Bill) who get the best recognition score. East coast women like me (and my boss) would routinely score in the mid-60s to low-70s back in '04. The boss's boss was from Hyderabad -- he often scored in the 20s. Our soft-spoken male coworker from Michigan used to be scored in the mid- to high-90s.
Apparently I must fit it's prefered profile because it was pretty solid right out of the gate and seems to have impoved already. High 90's is as good as my secretary (or me for that matter) prior to proofing. And early testing indicates it out-spells us all. Wink


jespah wrote:
I'd also caution that, even with small issues like the radio, people often put on the "dictation voice", speaking more slowly and carefully, and deliberately choosing their words, avoiding ums and pauses, and in general behaving better, but not the way they normally do. Even if you don't think you're doing that, you might be. Fortunately, the product does work better with training -- but you should be training it a bit with sloppy speech, too, as that is more of a fact of life than the "dictation voice" is.
Interesting caution/advice. I KNOW I've been speaking as crisply as I can in testing, and just assumed the dictator should develope a habit of doing so until it becomes second nature (already is for the guy who's been talking to a tape-recorder for 20+ years.) Interesting that you'd suggest teaching it a more natural rythym. Do you figure a trained Dragon wouldn't have a preference? Or are you merely pointing out that it isn't necesarily necesary?
ossobuco
 
  1  
Reply Wed 22 Sep, 2010 04:14 pm
@maxdancona,
That's something that I wonder about. When I knew to read italian moderately well - some time ago now and my ital reading has gone downhill - I could still screw up reading at least one thing while parsing one of their newspapers online, and so, when I did follow them, I also followed the english version, if there was one, of x newspaper, to check myself. A phrase or two could mess up the context for me, not always, but once in a while.

0 Replies
 
maxdancona
 
  1  
Reply Wed 22 Sep, 2010 05:33 pm
@OCCOM BILL,
Quote:
No, I specifically told you I am not trying to argue with you; other than the FACT that I am getting much better results than you predicted.


Bill (also not trying to argue) but I am very interested in how much better your results are. The only way to calculate this is to measure the accuracy rate. So try this, read 500 words into the thing, and then count how many errors it makes.

The accuracy will be 100 - #words with mistakes/500 * 100.

This non attempted argument would be more interesting if we had a real figure to work with.



Butrflynet
 
  1  
Reply Wed 22 Sep, 2010 07:45 pm
@maxdancona,
One of the initial training exercises in the default set up of the customized user dictionary is to read the Kennedy inaugural address into the mike so DNS can learn your voice. So, let's use that for this test.

By the way, in the fourth paragraph is the word heirs. This is an example of just one of the words in English that have very similar acoustics but different meanings. When said aloud, heirs could be airs, errors, or heirs when converted to text. It is these types of words that require further investment in the training and customization of DNS to your unique voice to make the first draft of the converted text coherent.

MS Word says it contains 1,382 words. Here's the text:

Quote:
Vice President Johnson, Mr. Speaker, Mr. Chief Justice, President Eisenhower, Vice President Nixon, President Truman, reverend clergy, fellow citizens:

We observe today not a victory of party, but a celebration of freedom -- symbolizing an end, as well as a beginning -- signifying renewal, as well as change. For I have sworn before you and Almighty God the same solemn oath our forebears prescribed nearly a century and three-quarters ago.

The world is very different now. For man holds in his mortal hands the power to abolish all forms of human poverty and all forms of human life. And yet the same revolutionary beliefs for which our forebears fought are still at issue around the globe -- the belief that the rights of man come not from the generosity of the state, but from the hand of God.

We dare not forget today that we are the heirs of that first revolution. Let the word go forth from this time and place, to friend and foe alike, that the torch has been passed to a new generation of Americans -- born in this century, tempered by war, disciplined by a hard and bitter peace, proud of our ancient heritage, and unwilling to witness or permit the slow undoing of those human rights to which this nation has always been committed, and to which we are committed today at home and around the world.

Let every nation know, whether it wishes us well or ill, that we shall pay any price, bear any burden, meet any hardship, support any friend, oppose any foe, to assure the survival and the success of liberty.

This much we pledge -- and more.

To those old allies whose cultural and spiritual origins we share, we pledge the loyalty of faithful friends. United there is little we cannot do in a host of cooperative ventures. Divided there is little we can do -- for we dare not meet a powerful challenge at odds and split asunder.

To those new states whom we welcome to the ranks of the free, we pledge our word that one form of colonial control shall not have passed away merely to be replaced by a far more iron tyranny. We shall not always expect to find them supporting our view. But we shall always hope to find them strongly supporting their own freedom -- and to remember that, in the past, those who foolishly sought power by riding the back of the tiger ended up inside.

To those people in the huts and villages of half the globe struggling to break the bonds of mass misery, we pledge our best efforts to help them help themselves, for whatever period is required -- not because the Communists may be doing it, not because we seek their votes, but because it is right. If a free society cannot help the many who are poor, it cannot save the few who are rich.

To our sister republics south of our border, we offer a special pledge: to convert our good words into good deeds, in a new alliance for progress, to assist free men and free governments in casting off the chains of poverty. But this peaceful revolution of hope cannot become the prey of hostile powers. Let all our neighbors know that we shall join with them to oppose aggression or subversion anywhere in the Americas. And let every other power know that this hemisphere intends to remain the master of its own house.

To that world assembly of sovereign states, the United Nations, our last best hope in an age where the instruments of war have far outpaced the instruments of peace, we renew our pledge of support -- to prevent it from becoming merely a forum for invective, to strengthen its shield of the new and the weak, and to enlarge the area in which its writ may run.

Finally, to those nations who would make themselves our adversary, we offer not a pledge but a request: that both sides begin anew the quest for peace, before the dark powers of destruction unleashed by science engulf all humanity in planned or accidental self-destruction.

We dare not tempt them with weakness. For only when our arms are sufficient beyond doubt can we be certain beyond doubt that they will never be employed.

But neither can two great and powerful groups of nations take comfort from our present course -- both sides overburdened by the cost of modern weapons, both rightly alarmed by the steady spread of the deadly atom, yet both racing to alter that uncertain balance of terror that stays the hand of mankind's final war.

So let us begin anew -- remembering on both sides that civility is not a sign of weakness, and sincerity is always subject to proof. Let us never negotiate out of fear, but let us never fear to negotiate.

Let both sides explore what problems unite us instead of belaboring those problems which divide us.

Let both sides, for the first time, formulate serious and precise proposals for the inspection and control of arms, and bring the absolute power to destroy other nations under the absolute control of all nations.

Let both sides seek to invoke the wonders of science instead of its terrors. Together let us explore the stars, conquer the deserts, eradicate disease, tap the ocean depths, and encourage the arts and commerce.

Let both sides unite to heed, in all corners of the earth, the command of Isaiah -- to "undo the heavy burdens, and [to] let the oppressed go free."¹

And, if a beachhead of cooperation may push back the jungle of suspicion, let both sides join in creating a new endeavor -- not a new balance of power, but a new world of law -- where the strong are just, and the weak secure, and the peace preserved.

All this will not be finished in the first one hundred days. Nor will it be finished in the first one thousand days; nor in the life of this Administration; nor even perhaps in our lifetime on this planet. But let us begin.

In your hands, my fellow citizens, more than mine, will rest the final success or failure of our course. Since this country was founded, each generation of Americans has been summoned to give testimony to its national loyalty. The graves of young Americans who answered the call to service surround the globe.

Now the trumpet summons us again -- not as a call to bear arms, though arms we need -- not as a call to battle, though embattled we are -- but a call to bear the burden of a long twilight struggle, year in and year out, "rejoicing in hope; patient in tribulation,"² a struggle against the common enemies of man: tyranny, poverty, disease, and war itself.

Can we forge against these enemies a grand and global alliance, North and South, East and West, that can assure a more fruitful life for all mankind? Will you join in that historic effort?

In the long history of the world, only a few generations have been granted the role of defending freedom in its hour of maximum danger. I do not shrink from this responsibility -- I welcome it. I do not believe that any of us would exchange places with any other people or any other generation. The energy, the faith, the devotion which we bring to this endeavor will light our country and all who serve it. And the glow from that fire can truly light the world.

And so, my fellow Americans, ask not what your country can do for you; ask what you can do for your country.

My fellow citizens of the world, ask not what America will do for you, but what together we can do for the freedom of man.

Finally, whether you are citizens of America or citizens of the world, ask of us here the same high standards of strength and sacrifice which we ask of you. With a good conscience our only sure reward, with history the final judge of our deeds, let us go forth to lead the land we love, asking His blessing and His help, but knowing that here on earth God's work must truly be our own.
0 Replies
 
jespah
 
  1  
Reply Thu 23 Sep, 2010 06:41 am
@OCCOM BILL,
OCCOM BILL wrote:

.....
jespah wrote:
I'd also caution that, even with small issues like the radio, people often put on the "dictation voice", speaking more slowly and carefully, and deliberately choosing their words, avoiding ums and pauses, and in general behaving better, but not the way they normally do. Even if you don't think you're doing that, you might be. Fortunately, the product does work better with training -- but you should be training it a bit with sloppy speech, too, as that is more of a fact of life than the "dictation voice" is.
Interesting caution/advice. I KNOW I've been speaking as crisply as I can in testing, and just assumed the dictator should develope a habit of doing so until it becomes second nature (already is for the guy who's been talking to a tape-recorder for 20+ years.) Interesting that you'd suggest teaching it a more natural rythym. Do you figure a trained Dragon wouldn't have a preference? Or are you merely pointing out that it isn't necesarily necesary?


Well, what I'm saying is (and I'm probably writing it badly) that people have a tendency to do the dictation voice for a while and then they get sloppy. But the product is trained to listen for you being all clear and stuff. What I think you may be doing is training it to listen for you being as perfect as you can possibly be -- which is fine if you want to dictate that way from now until Kingdom Come. If not, if you slip up, you have trained it one way, but have gone in another direction.

I don't honestly know what the best solution to that is, by the way. E. g. do you train it in close-to-perfection and attempt to maintain that standard forever more, or do you train it with normal speech and make it harder for the product to do its thing, so that it won't have any drop-off? We were told to train it with normal speech but, as you know, it's not always easy to talk to a microphone the way you and I would speak to one another.

BTW -- as an aside -- you shoulda seen the Legal Language Model when I first saw it. Whoever put it together was a Star Trek fan and, yes, the word Klingon was in there, among many other Trek trivialities, such as the names of actors, characters, species, etc., all mixed among normal words like bicycle and legal words like plaintiff. Spent months cleaning that crap out, and replacing it with, among other things, dog breeds, makes of cars, types of guns and ammunition, and every possible synonym for the word street (e. g. road, lane, etc.).
JTT
 
  1  
Reply Thu 23 Sep, 2010 06:26 pm
@jespah,
Quote:
Well, what I'm saying is (and I'm probably writing it badly) that people have a tendency to do the dictation voice for a while and then they get sloppy. But the product is trained to listen for you being all clear and stuff. What I think you may be doing is training it to listen for you being as perfect as you can possibly be -- which is fine if you want to dictate that way from now until Kingdom Come. If not, if you slip up, you have trained it one way, but have gone in another direction.


If one uses a normal conversational voice, Jespah, does DS/DD get used to and type natural phonological reductions, ie. I'm gonna ask you a new question; Ya'll hafta speak louder; Wadja say?; Wenizee comin'?; ...
jespah
 
  1  
Reply Fri 24 Sep, 2010 04:39 am
@JTT,
Not sure -- like I said, it's been a while since I worked there. But I do know that it tries to recognize accents, but of course not every single possible variant of an accent is in there.
0 Replies
 
Butrflynet
 
  1  
Reply Fri 24 Sep, 2010 11:00 am
@JTT,
Quote:
If one uses a normal conversational voice, Jespah, does DS/DD get used to and type natural phonological reductions, ie. I'm gonna ask you a new question; Ya'll hafta speak louder; Wadja say?; Wenizee comin'?; ...


That's part of the training investment needed to increase accuracy.

At first, it will translate that acoustically. By going back and correcting the words, it will learn to correlate that set of acoustics with the taught way of spelling it. If you don't use the program to correct it and correct it later with a spell checker, DNS will never improve and will always repeat the same error of interpretation. You also teach it how to recognize accents for words and phrases by recording the pronunciation of the word or phrase in your voice and teaching it how to interpret that set of acoustics.

0 Replies
 
OCCOM BILL
 
  1  
Reply Fri 24 Sep, 2010 01:06 pm
@maxdancona,
Will do so soon. I'm curious too, but it's been a busy week.
0 Replies
 
OCCOM BILL
 
  1  
Reply Fri 24 Sep, 2010 01:15 pm
@jespah,
The main "dictator" around here speaks to a machine as casually as he would to me, after 20 years practice. I hope the desktop stuff is considerably more powerful than the stuff I put on my phone because: He talks too damned fast. As well as it understood me; it failed miserably when I handed it to him. Could be it's already used to my tone or style or some such thing, but it was comical reading what it thought he said aloud.

Klingon: Laughing
0 Replies
 
 

Related Topics

 
  1. Forums
  2. » Dragon Speak
  3. » Page 4
Copyright © 2024 MadLab, LLC :: Terms of Service :: Privacy Policy :: Page generated in 0.03 seconds on 12/26/2024 at 06:00:29