Monthly Archives: April 2013

Second Language Testing (Guest Speakers Abbas Mousavi and Jill Robbins, 4/30/13)

Jill Robbins and Abbas Mousavi from Second Language Testing, Inc spoke to us about second language testing and assessment. Dr. Robbins has worked on developing approaches to teach language, and Dr. Mousavi specializes in proficiency assessment (and is somewhat of an expert in Arabic calligraphy).

Their slideshows are available here.

NOTES (thanks to Megan Chao):

Language Testing

  • Field in applied linguistics

  • Determine growth/achievement by which success of a student can be evaluated

  • Way of quantifying an unobservable ability

  • Measure of usefulness of a particular teaching methodology, curriculum, approach, etc

  • Example: assessments

    • Formative (during study)

    • Summative (at the end of study)

  • Humongous tree showing purposes of language testing

    • Attainment (past)

      • Achievement

        • General

        • Progress

        • Mastery

        • Diagnostic

      • Proficiency

      • Knowledge

    • Prognostic (future)

      • Selection

        • Entrance

        • Readiness

        • Aptitude

        • Competition

      • Placement

  • Scoring is either criterion or norm-referenced

    • Criterion-referenced: Marking/interpreting test based on a well-defined criterion level of ability

    • Norm-referenced: Compare students against each other (i.e. on a curve)

  • LANGUAGE TESTS MUST BE RELIABLE, VALID, AND PRACTICAL

    • Practicality: includes financial/logistic aspects, enough room/time/money, etc

    • Reliability: consistency of test scores across different times, test forms, graders, and other measurements

      • Taking the time on a different day

      • Different version of test

      • Different types of questions (multiple choice, essay, short answer, etc)

      • Different graders (grading on different days)

  • Validity: appropriateness of test/parts as measure of what purported to measure

    • A test may be valid for one purpose but not for other: it must “defend its name”

    • A math test should test math and not english

  • Using arrows + targets as an example!

    • Arrows go everywhere and wildly miss the center of the target, scattering in various degrees of horribly wrong -> test is neither reliable nor valid, and therefore irrelevant

    • Arrows all converge on one point which is decidedly not the center of the target -> test is reliable but invalid

    • Arrows all hit the center of the target -> reliable and valid

    • A test must be reliable before it can be valid

    • A valid test is by definition at least sort of reliable

 

Applying Native Language Learning Standards to Development of Assessments

  • First, some background on the Choctaw Language:

    • only 1% of speakers are actually fluent at the elementary school level

  • Five “goals” of SLTI (Second Language Testing, Inc) program:

    • Communication

      • Interpretive: listening/reading/viewing

      • Interpersonal: speaking/writing

      • Presentational

    • Culture

    • Connections

      • Transfer/apply knowledge to other disciplines of learning

    • Comparisons

      • Developing insights into the nature of language/culture

    • Communities

      • Participate in bilingual communities

  • What can be observed in testing context?

  • Miccosukee: only tribe w/o peace treaty w/ United States

    • SLTI worked on

      • Revision of Standards

      • Standards-based Assessments

      • Curriculum Maps Aligned to Standards

    • Didn’t encourage writing, so mostly oral standards

      • Can identify/say/segment/blend various units of speech sounds to make multi-syllable words

      • Follow simple directions

      • Sing the color song

      • Compare past/current culture

  • Where will Native languages be in 20 years??

UMD Trip Summary

Lots of ling happened! Notes (big thanks to Megan Chao) available here.

At Marie Mount Hall
At Marie Mount Hall

Thanks to all our presenters:

Yakov Kronrod Introduction (and general organization)
Phillip Resnik Machine Translation (history)
Josh Falk Sentiment Analysis (movie reviews)
Mike Fetters Minimalism
Tara Mease Infant Language (Baby Lab tour)
Alix Kowalski and Chris Heffner Aphasia
Josh Falk (again) Phono-Poetry (stress patterns of Finnish)
Alexander Williams Semantics (implicitness)
Ellen Lau EEG and MEG (lab tour and discussion)

Linguistics of ASL (Guest Speaker Pamela Toman, 4/23/13)

Pamela Toman, a data scientist at Booz Allen Hamilton with training in linguistics, spoke to us this Tuesday about the linguistics of American Sign Language, including its phonological, syntactical, and social aspects.

NOTES:

Pamela’s posted a resource page/outline of her lecture here (note that we did not discuss everything). Here’s a summary of stuff we talked about not on her site:

Oralism – belief that deaf people should learn to function in a hearing world. That means making them learn lip reading and speaking. Back in the 1960s and earlier, people even believed that sign language would harm one’s spoken language, and so they banned signing in schools. On a side note, people didn’t ban signing in black schools, which is why Black Sign Language is significantly different and older than American Sign Language.

Sapir-Worf Hypothesis – may be in play here, although probably not. Because of sign language’s emphasis on space, maybe deaf people have better spatial reasoning?

Sociolinguistics – We talked about how hearing parents raising deaf children can be a problem, starving children of language.

UMD Visit Details

UMD permission slips are ready at last! Please print out the slip, fill it in, and give it to me by this Friday (the sooner the better). Mr. Ostrander has asked me to point out that, although your absence will be excused, this is not a field trip or school-sponsored activity – e.g., if you trip and break your leg, you’re responsible.

We will meet at the attendance office at 7:35 – so just check in with your first period teacher. Please do not be late, otherwise we miss the bus and bad things ensue.

We’ll take the C2 to UMD (7:49 – 8:15) and back (2:25 – 2:56). Please bring school ID, and money/SmarTrip just in case. Our stop is University and Colesville (we’ll walk over together from the attendance office), and the other is just north of Marie Mount Hall, which houses the linguistics department.

Lunch will be provided.

Time permitting, we’ll very quickly run over to the dairy (also nearby) after the lectures, grab some ice cream, and be back at the bus stop by 2:20. Bring money if you want ice cream, of course.

Please bring your phones, and wear red! A notepad and an umbrella might also be useful.

Tentative schedule for the day (check later post for updated version):
Time Topic Speaker
8:30 – 8:45 Introduction Yakov Kronrod
8:45 – 9:15 Minimalism Mike Fetters
9:15 – 9:45 Machine Translation Philip Resnik
9:45 – 10:15 Sentiment Analysis John Falk
10:15 – 11:00 Infant Language Tara Mease
11:00 – 11:00 Aphasia Alix Kowalski and Chris Heffner
11:30 – 12:00 Lunch and Psycholinguistics Colin Phillips
12:00 – 1:00 Phonology Mike Key and Josh Falk
1:00 – 1:30 Semantics Alexander Williams
1:30 – 2:00 EEG and MEG Ellen Lau

NACLO Round II Results

NACLO Round II results are out! Congratulations to Samuel Zbarsky, who’s an alternate for the US team, and Michelle Noh, who received an award for best solution of problem 1. Sam was on the US team last year, and received an honorable mention at the IOL.

We had six Round II qualifiers from Blair. In order of final ranking: Sam Zbarsky (#11), Victor Xu (#15), Alan Du (#24), Michelle Noh (#78), Daniela Ganelin (#82), and Daniel Amir (#115). Combined with a homeschooled student who qualified from our site, Blair was 5th in the nation for number of qualifiers.

Check back next year for 2014 NACLO information!

Linguistic Illusions (Guest Speaker Colin Phillips, 4/9/13)

Colin Phillips, a professor at the UMD Linguistics Department spoke to us about linguistic illusions. He specializes in psycholinguistics and is “an authority on diverse topics such as sentence processing, pronoun/anaphora resolution, real-time language processing, and many others.”

He asked that we read a paper that introduces some grammatical illusions and selective fallibility of the brain’s language processor. It was accurate several years ago, but “some more recent findings undermine certain conclusions”. Partial outline available here, thanks to Alan. To see some grammatical illusions explained in nontechnical language, look here.

Slideshow is available here – note that we did not get to everything.

NOTES (thanks to Megan Chao):

  • There is often ambiguity in language (e.g. HISTORIC CHURCH + TOILETS sign)
  • Examples of visual illusions: we are inclined to see a certain effect (e.g. curved instead of straight lines); of course this can also occur in language
  • Ideally, language should be robust even in noisy environments
  • This leads to examples of confusing japanese sentences where verbs come at the end:
    • ジョンが マリーに トムが お店で ミルクを 買ったと 言った
      • John-nom / Mary-dat / Tom-nom / store-at / milk-acc / buy-decl / told
      • Translation: “John told Mary that Tom bought milk at the store.”
    • ジョンが マリーに りんごを 食べた 犬を あげた
      • John-nom / Mary-dat / apple-acc / ate / dog-acc / gave
      • [Did not catch the translation in time]
  • Electrical/magnetic brain activity
    • It takes us about 250-400 milliseconds to access words in language
    • That makes about 3-5 word accessed per second
    • At each word, the brain does the following to understand the language
      • Visual/acoustic processing
      • Phoneme recognition
      • Word recognition
      • Syntactic analysis
      • Semantic interpretation
    • But, there is a computational bottleneck: updating interpretation at each word requires much more time than is available
  • Models
    • Iron chef model
    • McDonalds’ model (quality may have to be sacrificed)
    • Julia Child’s cooking show model
  • Tower of Pisa: visual illusion where identical pictures look misaligned side-by-side

Dr. Lin’s NLP Study Group (4/13/13)

We will be meeting Dr. Lin from 1-3 on 4/13 at Quince Orchard Library. As a reminder, last time’s “homework” was to:

  • Look at this Java package. Try to get it running and tap into the public sample Twitter stream. (Sam’s created a package to help).
  • Download and play w/ the Stanford NLP tools. Play with some POS tagging, NE tagging, parsing, etc. Learn the API.
  • Think of interesting project ideas