[Athen] Searching Video Lectures - MIT lecture search engine

[Gaeir Dietrich]

We have looked at other similar products, and while they do a fairly decent
job of indexing material, they are not even close to creating accurate
word-for-word transcripts.

If all you want is to know the kinds of words being spoken five minutes into
an audio file, it works. If you actually want to know what is being said, it
does not work.

******************************************************
Gaeir (rhymes with "fire") Dietrich
High Tech Center Training Unit of the
California Community Colleges
De Anza College, Cupertino, CA
www.htctu.net
408-996-6043
-----Original Message-----
From: athen-bounces at athenpro.org [mailto:athen-bounces at athenpro.org] On
Behalf Of Kathleen Cahill
Sent: Wednesday, January 09, 2008 8:36 AM
To: Access Technologists in Higher Education Network
Cc: ITACCESS at LISTSERV.EDUCAUSE.EDU
Subject: Re: [Athen] Searching Video Lectures - MIT lecture search engine

Hi all;

I have been in touch with Jim Glass, the researcher for this project to 
inquire about any plans to make the software available in the future.  
I'll post when I find something out.

Thanks,
Kathy

_______________________

Kathleen Cahill
MIT ATIC (Adaptive Technology) Lab
77 Mass. Ave. 7-143
Cambridge MA 02139
(617) 253-5111
kcahill at mit.edu


Saroj Primlani wrote:

> Have you all heard about this?  I can't find information on the speech

> recognition engine.  If this is viable it would a major solution to our

> problems, we really need to investigate this.  

> http://web.mit.edu/newsoffice/2007/lectures-tt1107.html

>

> Article in Technology Review

> http://www.technologyreview.com/Infotech/19747/page1/

>

> Monday, November 26, 2007

> Searching Video Lectures

> A tool from MIT finds keywords so that students can efficiently review

> lectures.

> By Kate Greene

> Researchers at MIT have released a video and audio search tool that solves

> one of the most challenging problems in the field: how to break up a

lengthy

> academic lecture into manageable chunks, pinpoint the location of

keywords,

> and direct the user to them. Announced last month, the MIT Lecture Browser

> website gives the general public detailed access to more than 200 lectures

> publicly available though the university's OpenCourseWare initiative. The

> search engine leverages decades' worth of speech-recognition research at

MIT

> and other institutions to convert audio into text and make it searchable.

>

> The Lecture Browser arrives at a time when more and more universities,

> including Carnegie Mellon University and the University of California,

> Berkeley, are posting videos and podcasts of lectures online. While this

> content is useful, locating specific information within lectures can be

> difficult, frustrating students who are accustomed to finding what they

need

> in less than a second with Google.

>

> "This is a growing issue for universities around the country as it becomes

> easier to record classroom lectures," says Jim Glass, research scientist

at

> MIT. "It's a real challenge to know how to disseminate them and make it

> easier for students to get access to parts of the lecture they might be

> interested in. It's like finding a needle in a haystack."

>

> The fundamental elements of the Lecture Browser have been kicking around

> research labs at MIT and places such as BBN Technologies in Boston,

Carnegie

> Mellon, SRI International in Palo Alto, CA, and the University of Southern

> California for more than 30 years. Their efforts have produced software

> that's finally good enough to find its way to the average person, says

> Premkumar Natarajan, scientist at BBN. "There's about three decades of

work

> where many fundamental problems were addressed," he says. "The technology

is

> mature enough now that there's a growing sense in the community that it's

> time [to test applications in the real world]. We've done all we can in

the

> lab." 

>

> A handful of companies, such as online audio and video search engines

Blinkx

> and EveryZing (which has licensed technology from BBN) are making use of

> software that converts audio speech into searchable text. (See "Surfing TV

> on the Internet" and "More-Accurate Video Search".) But the MIT

researchers

> faced particular challenges with academic lectures. For one, many

lecturers

> are not native English speakers, which makes automatic transcription

tricky

> for systems trained on American English accents. Second, the words favored

> in science lectures can be rather obscure. Finally, says Regina Barzilay,

> professor of computer Science at MIT, lectures have very little

discernable

> structure, making them difficult to break up and organize for easy

> searching. "Topical transitions are very subtle," she says. "Lectures

aren't

> organized like normal text."

>

> To tackle these problems, the researchers first configured the software

that

> converts the audio to text. They trained the software to understand

> particular accents using accurate transcriptions of short snippets of

> recorded speech. To help the software identify uncommon words--anything

from

> "drosophila" to "closed-loop integrals"--the researchers provided it with

> additional data, such as text from books and lecture notes, which assists

> the software in accurately transcribing as many as four out of five words.

> If the system is used with a nonnative English speaker whose accent and

> vocabulary it hasn't been trained to recognize, the accuracy can drop to

50

> percent. (Such a low accuracy would not be useful for direct transcription

> but can still be useful for keyword searches.)

>

> The next step, explains Barzilay, is to add structure to the transcribed

> words. Software was already available that could break up long strings of

> sentences into high-level concepts, but she found that it didn't do the

> trick with the lectures. So her group designed its own. "One of the key

> distinctions," she says, "is that, during a lecture, you speak freely; you

> ramble and mumble." 

>

> To organize the transcribed text, her group created software that breaks

the

> text into chunks that often correspond with individual sentences. The

> software places these chunks in a network structure; chunks that have

> similar words or were spoken closely together in time are placed closer

> together in the network. The relative distance of the chunks in the

network

> lets the software decide which sentences belong with each topic or

subtopic

> in the lecture. 

>

> The result, she says, is a coherent transcription. When a person searches

> for a keyword, the browser offers results in the form of a video or audio

> timeline that is partitioned into sections. The section of the lecture

that

> contains the keyword is highlighted; below it are snippets of text that

> surround each instance of the keyword. When a video is playing, the

browser

> shows the transcribed text below it.

>

> Barzilay says that the browser currently receives an average of 21,000

hits

> a day, and while it's proving popular, there is still work to be done.

> Within the next few months, her team will add a feature that automatically

> attaches a text outline to lectures so users can jump to a desired

section.

> Further ahead, the researchers will give users the ability to make

> corrections to the transcript in the same way that people contribute to

> Wikipedia. While such improvements seem straightforward, they pose

technical

> challenges, Barzilay says. "It's not a trivial matter, because you want an

> interface that's not tedious, and you need to propagate the correction

> throughout the lecture and to other lectures." She says that bringing

people

> into the transcription loop could improve the accuracy of the system by a

> couple percentage points, making user experience even better. 

>

> Copyright Technology Review 2007

> _________________________________ 

> Saroj Primlani 

> Coordinator of University IT Accessibility 

> Office of Information Technology

> 919 513 4087 

> http://ncsu.edu/it/access 

>

>   



_______________________________________________
Athen mailing list
Athen at athenpro.org
http://athenpro.org/mailman/listinfo/athen_athenpro.org