The Nemesysco Scandal – A Report

1
The Nemesysco Scandal A Report
John J Ohala Professor Emeritus, Linguistics, UCB
2
Outline
1. Nemesysco, their beginnings, their products,
and how they promote them.
2. The Eriksson Lacerda paper re Nemesysco
3. Press coverage of Nemesysco of their
threats to the publisher of the Eriksson
Lacerda paper.
4. Coverage in the Blogosphere.
5. Who is the guy behind all this?
6. What can be done?
3
1. Nemesysco, their beginnings, their products
and how they promote them.
4
Starting around 1997 or 2000 (its not clear
when) an Israeli firm, Nemesysco, started
promoting and selling devices that were said to
be able to detect emotion and stress in a speaker
via an analysis of their voice. It used an
analysis techmique called layered voice
analysis or LVA for short.
Their claim Nemesyscos Layered Voice
Analysis (LVA) technology detects and measures
the emotional content of human speech, captured
live or extracted from recorded audio. LVA
identifies various types of stress, cognitive
processes and emotional reactions which together
comprise the ?emotional signature? of an
individual at a given moment, based solely on the
properties of his or her voice . The technology
detects minute, involuntary changes in the voice
reflective of various types of brain activity. By
utilizing a wide range spectrum analysis to
detect minute changes in the speech waveform, LVA
detects anomalies in brain activity and
classifies them in terms of stress, excitement,
deception, and varying emotional states,
accordingly. This way, LVA detects what we call
?brain activity traces,? using the voice as a
medium. The information that is gathered is then
processed and analyzed to reveal the speaker?s
current state of mind.
Segue to their professionally desgined website
(live).
5
In their promotions they emphasize their
technique is patented. Everyone should know
that it is not that difficult to get something
patented all you need to do is follow the format
dictated by the patenting agencies. (Aside
even I have a patent look it up!) As far as I
can tell, there are two US patents awarded to
Amir Liberman. There are other patents filed in
other countries.
6
Further elaboration of their claims
LVA Analysis Process  LVA has two basic formulas
comprised of unique signal processing algorithms
that extract more than 120 emotional parameters
from each voice segment. These are further
classified into nine major categories of basic
emotions. Depending on the goal of the analysis,
up to eight final analysis formulas can be
applied to the emotional parameter data. These
include Lie stress analysis, Arousal level,
Attention level, Emotional level, Conflict level,
Deception patterns match, and additional methods
for veracity assessment
Just to note one of many inconsistencies
Elsewhere on their website they insist that their
products are NOT lie detectors.
7
Window dressing to make the customrer think they
are experts in speech.
The Human Speaking Mechanism  The human speaking
mechanism is one of the most complicated
procedures the human body is capable of, due to
the number of muscles and physical apparatus
involved, and the ways in which they need to be
synchronized in perfect timing.   Initially, the
brain apprehends a given situation and the
possible implications of whatever will be said.
Then when a person decides to speak, air is
pushed upward from the lungs into the vocal
cords. This causes the vocal cords to vibrate at
a specific frequency and produce sound. The
vibrated air continues to flow up toward the
mouth where it is manipulated by the tongue,
teeth and lips to produce sound streams which we
interpret as words or phrases.  The brain
closely monitors all of these procedures,
ensuring that the sound emitted is what was
intended, is intelligible, and is at a volume
that can be heard by the intended listener. Due
to this constant cerebral monitoring, every
"event" that passes through the brain will leave
a trace on the speech flow. LVA technology
ignores what your subject is saying (i.e., the
specific content) and focuses only on changes in
brain activity that are reflected in the voice.
In other words, what is critical is not what
your subject is saying, but how he or she says
it.
8
Nemesysco has spiffy computer interfaces
9
A Figure from the patent of 17 Jan 2007.
10
To those unschooled in the basics of speech
communication, Nemesysco describes their methods
in language that is bound to impress them it
sounds scientific and authoritative.
SENSE Analysis Process The SENSE technology is
comprise of 4 sub-processes 1. The vocal
waveform is analyzed to measure the presence of
local micro-high frequencies, low frequencies and
changes in their presence within a single voice
sample. 2. A precise frequency spectrum of the
vocal input is sampled and analyzed. 3. The
parameters gathered by the previous steps are
used to create a baseline profile for the
subject. 4. The new voice segments to be tested
are compared with the subject's baseline profile,
and the analysis is generated.
11
Nemesysco cites a number of studies supposedly
giving evidence of the efficacy of their products.
As documented in the blogs about Nemesysco, a
number of these studies are worthless, being
conference papers or posters at conferences where
there was no reviewing process, or where the
authors have some connection to or financial
interest in promoting Nemesyscos products.
But lets examine one study where the above
discounting factors are not involved.
12
One document cited on the Nemsysco website in
support of one of their early products called
Vericator (one thing Nemesysco does well is
coining clever names for itself and its products)
is said to report that "A three-year study by
Air Force Research Laboratory (AFRL) Information
Directorate engineers has concluded that several
features of voice stress analysis are effective
for detecting when a person is answering
questions under stress..."
13
(No Transcript)
14
(No Transcript)
15
And, one might ask, why cite this study when on
their own website, Nemesysco declares
Layered Voice Analysis LVA is not a "Voice
Stress" analysis technology, nor does it use any
previously known method for detecting voice
stress. As such, LVA does not perform
"micro-tremors" analysis in the voice.
16
Further suspicious omission
In reviewing the literature on VSA, this very
same report mentions
17
My own take on this study As for the officer
quoted in the report, Michael G. Adsit, Criminal
Investigator, and the authors of the report to
the US Dept of Justice, Darren Haddad, Sharon
Walter, Roy Ratley, Megan Smith, none of them
really know much about the motor control of the
speech apparatus. They dont control the vast
literature in this area. (They may have
expertise in other domains speech processing,
speech compression.)
18
Finally, the Nemesysco list of studies neglects
to mention two studies that did attempt to
implement proper controls
1. Hollien, Harry James D. Harnsberger
(2006-03-17), "Voice Stress Analyzer
Instrumentation Evaluation" (pdf), CIFA Contract
FA 4814-04-0011, http//www.clas.ufl.edu/users/
jharns/Research20Projects/UF_Report_03_17_2006.pd
f
device showed significant sensitivity to the
presence of stress or deception in the speech
samples tested. The true positive and false
positive rates were parallel to a great extent
2. Damphousse, Kelly R. (March 2008). "Voice
Stress Analysis Only 15 Percent of Lies About
Drug Use Detected in Field Test". NIJ Journal
(National Institute of Justice)
Does VSA work? As our findings revealed, the two
VSA programs that we tested had approximately a
50-percent accuracy rate in detecting deception
about drug use in a field (i.e., jail)
environment however, the mere presence of a VSA
program during an interrogation may deter a
respondent from answering falsely..
19
2. The Eriksson Lacerda paper re Nemesysco (
one other similar product named Diogenes)
20
Then came the paper by Anders Eriksson and
Francisco Lacerda
Charlatanry in forensic speech science A
problem to be taken seriously The International
Journal of Speech, Language and the Law
(Publisher Equinox), IJSLL vol 14.2 2007
169193.
Papers published in IJSLL are peer-reviewed.
21
Francisco Lacerda, Professor of Phonetics at
Stockholm University
Anders Eriksson Professor of Phonetics Dept. of
Philosophy, Linguistics and Theory of
Science University of Gothenburg
22
Eriksson, A. Lacerda, F. (2007)"Charlantry in
forensic speech science A problem to be taken
seriouslyIntl J. Speech Lang. Law 14, 169 173.
ABSTRACT
A lie detector which can reveal lie and deception
in some automatic and perfectly reliable way is
an old idea we have often met with in science
fiction books and comic strips.  This is all very
well.  It is when machines claimed to be lie
detectors appear in the context of criminal
investigations or security applications that we
need to be concerned.  In the present paper we
will describe two types of "deception" or "stress
detectors" (euphemisms to refer to what quite
clearly is known as "lie detectors")." Both types
of detection are claimed to be based on voice
analysis but we found no scientific evidence to
support the manufacturers claims.  Indeed, our
review of scientific studies will show that these
machines perform at chance level when tested for
reliability." Given such results and the absence
of scientific support for the underlying
principles it is justified to view the use of
these machines as charlatanry and we argue that
there are serious ethical and security reasons to
demand that responsible authorities and
institutions should not get involved in such
practices.
23
Their paper reviewed existing literature but did
not report the results of a controlled study,
unlike those of Hollien Harnsberger (2006) or
Damphousse(2008). Rather, as concerned the
Nemesysco product, they they did something more
fundamental and ultimately more important to
inform the knowledgeable speech tech community
they examined the publically accessible patent
for the device. What they found
To say that there is absolutely no scientific
basis for the claims made by the LVA proponents
is an understatement. The ideas on which the
products are based are simply complete nonsense.
They also offer what we may characterize as a
sociological account of Nemesyscos economic
success
While, as we have seen, the voice stress
detectors are not of any real use as the lie or
stress detectors they are claimed to be, they
have certainly not been without success in other
areas. One such area is making money for the
vendors.
24
The statistics are based upon what is defined as
thorns and plateaus which has no relevance at all
for voice analysis and is moreover dependent on
how the signal is sampled.
25
Explanation
This figure (from the US Patent) is a simple
digitization of the incoming signal.
The program computes statistics on the relative
incidence of thorns and plateaus and comes up
with a number that is said to show the speakers
emotional state.
The problem (as noted by Eriksson Lacerda) is
that these statistics will vary with sampling
rate, quantization of the sampling, background
noise, etc. Indeed, it could come up with a
number just by sampling traffic noise or bird
song.
26
From the US patent
This last statement, of course, is utter,
unadulterated, nonsense. A student incorporating
this in an undergraduate phonetics/speech science
course would get an F!
27
From the US patent
And yet the Nemesysco website declares their
device is not lie detection system.
28
An examination of the description of the method
in the American patent documents confirms the
suspicion that the method is pure nonsense,
perhaps best described as statistics based on
digitization artefacts. Eriksson Lacerda
29
The legal controversy
The use of words such as charlatanry, fraud
and, perhaps, the inclusion of reports from a
Swedish reporter who interviewed Amir Liberman,
the CEO of Nemesysco, revealing that he had no
scientific credentials in speech analysis or in
psychology or psychiatry, -- in fact no higher
education except in marketing -- were grounds for
Nemesysco threatening to sue Equinox (the IJSLL
publisher) and the authors for libel, defamation
of character, etc.
30
Charlantry in forensic speech science A problem
to be taken seriously Anders Eriksson, Francisco
Lacerda NOTE FROM PUBLISHER December 4 2008 In
the December 2007 Edition of the International
Journal of Speech, Language and the Law, an
article was published which made serious
allegations concerning Mr Amir Liberman and
Nemesysco Limited. We have received complaints
from Mr Liberman and Nemesysco Limited about the
content of this article and particularly that the
allegations made against them in it were highly
defamatory, containing many inaccuracies and
misleading statements. In addition, they complain
that it was prepared without reference to them
and without giving them an opportunity to comment
upon it. The Journal accepts that Mr Liberman and
Nemesysco Limited were not asked to assist in the
preparation of the article and further that they
were not invited to comment on the content of the
article prior to its publication where, in view
of the content of the article, it would have been
appropriate to invite them to do so. We have
agreed to publish a letter from Mr Liberman and
Nemesysco Limited setting out their objections to
the article in more detail in a future issue of
the journal. The article will no longer be made
available in electronic form through the Equinox
website.Janet JoyceManaging Director
31
3. The response to this controversy in the press
and blogosphere.
32
BBC
33
Wed December 17, 2008 Behavioral screening --
the future of airport security?
By Dana Rosenblatt CNNTEL AVIV, Israel (CNN) --
Keep your shoes and belts on Waiting in long
airport security lines to pass through metal
detectors may soon be a thing of the
past. Security experts say focus is shifting
from analyzing the content of carry-ons to
analyzing the content of passengers' intentions
and emotions. Nemesysco, another Israeli-based
technology company, believes the key to a
person's emotions and intentions lies in their
voice. The company's patented LVA, or Layered
Voice Analysis, technology can pick up verbal
cues from a passenger who may pose a
threat. Unlike a polygraph test, which checks for
lies, Nemesysco's systems work as an "emotion
detector," says Nemesysco CEO Amir Liberman. In
other words, it's not what passengers say, but
how they say it. Nemesysco's devices use a series
of patented signal-processing algorithms that can
differentiate between a "normal" voice and
a"'stressed" voice. If emotional stress is
detected, officials can determine if the
passenger should be taken aside for further
questioning.
34
CNN (continued)
The system works on the premise that all voices
have a certain frequency, and any deviation of
that baseline frequency can indicate
stress. Liberman says it takes approximately five
to 10 seconds for their system to capture a
"normal" voice in casual conversation, which
establishes a baseline. Their system then
measures changes from the baseline voice that
signify an increase in stress, excitement,
anticipation, hesitation or other emotions that
can indicate a potential terrorism threat. A
computer processes the voice patterns and then
flashes words such as "high risk," "medium risk,"
"excited" and "highly stressed." Through his
system, Liberman says, he "can see what's going
on in your brain." Versions of Nemesysco's system
already have been successfully tested at Moscow
Domodedovo International Airport, where officials
used it to target criminals and drug traffickers.
A version was recently implemented at another
major international airport which Liberman
declined to identify. Layered Voice Analysis also
has been used to test for insurance fraud and on
the TV program "Big Brother Australia."
35
From a New Yorker piece HELLO, HAL. Will we
ever get a computer we can really talk to? by
John Seabrook. JUNE 23, 2008 There is a small
market for voice-based lie detectors, which are
becoming a popular tool in police stations around
the country. Many are made by Nemesysco, an
Israeli company, using a technique called
layered voice analysis to analyze some hundred
and thirty parameters in the voice to establish
the speakers pyschological state. The academic
world is skeptical of voice-based lie detection,
because Nemesysco will not release the algorithms
on which its program is based after all, they
are proprietary. Layered voice analysis has
failed in two independent tests. Nemesyscos
American distributor says thats because the
tests were poorly designed. (The company played
Roger Clemenss recent congressional testimony
for me through its software, so that I could see
for myself the Rockets stress levels leaping.)
Nevertheless, according to the distributor more
than a thousand copies of the software have been
soldat fourteen thousand five hundred dollars
eachto law-enforcement agencies and, more
recently, to insurance companies, which are using
them in fraud detection.
36
http//www.guardian.co.uk/news/2007/sep/09/letters
.theobserver
Letter sent to the on-line version of The
Guardian re their uncritical coverage of
Nemesyscos LVA voice stress detector
The truth is out there The technology referred
to 'Lie detectors target benefit claim cheats'
(News last week) has been subject to considerable
testing in the academic research community. It
has been found that the results produced by such
systems do not exceed the level of chance. But
these devices will probably save Harrow Council
money because if people are told their speech is
being monitored by a gadget that detects lies,
they are more likely to tell the truth.Anders
Eriksson, Professor of Phonetics, University of
GothenburgPaul Foulkes, Reader in Linguistics,
University of YorkProfessor Peter French,
forensic speech scientist, JP French Associates,
YorkFrancis Nolan, Professor of Phonetics,
University of Cambridge
37
It is important to note that neither the
publisher nor Nemesysco disputed the scientific
evaluation of the product.
The sticking points in the published paper were
the words charlatanry fraud and inclusion of
the statements from a Swedish journalist who
interviewed Amir Liberman and reported that he
had no academic or other scientific credentials
which would lend credence to the claims made for
the Nemesysco products.
Given the plaintiff-friendly libel laws in the UK
(it has been characterized as a country that
invites libel tourism), the publisher did what
Francis Nolan characterized as what a person
facing a mugger with a gun would do comply with
the muggers demands or face devastating
consequences. One of the possibly positive
outcomes of the Nemesysco scandal is that some
MPs and the UK media are urging a change in these
libel laws and have cited the Nemesysco case as
an example of why this is necessary.
38
4. Coverage in the Blogosphere.
39
The response in the blogosphere has been harsh
toward Nemesysco and toward the IJSLL
publisher, Equinox (but I dont think they fully
understand the jeopardy faced by publishers given
the lop-sided libel laws in the UK). I append to
this presentation a number of links to blogs and
other websites covering and commenting on this
scandal.
A purely personal judgment by me take it or
leave it -- the blogs did a far better job of
investigating and publicizing this scandal than
the established press or TV networks did CNN,
BBC, New Yorker, etc. I tried to get the New
York Times interested in this. Result, so far
NOTHING. I also emailed our (Califs) senator,
Diane Feinstein. Result NOTHING. If anyone
wants to research this scandal go to the blogs.
40
5. Who is the guy behind all this?
41
This controversy, partly fought in a newspaper,
caught the interest of a journalist, Arne
Lapidius, who was working in Israel for the
Swedish daily Expressen. After some research he
managed to locate Mr Liberman, a 32 year old (in
2004) businessman in a small office in the town
of Natania. The business appeared to be a one-man
operation. Mr Lapidus interviewed Mr Liberman
about his academic background and was told that
he basically had none. He has no degree (never
had time to get one, he explains) but has taken
some courses in marketing at an Israeli open
university. As we have explained above, the LVA
is a simple program written in rather
amateurishly used Visual Basic. Eriksson
Lacerda
42
Amir Liberman
43
6. What can be done?
44
Observation this scandal demeans the speech
tech industry.
Question How can the speech tech industry
police itself?
Suggested answer the major speech tech
organization (ISCA, ASA, ASHA, etc.) have to
issue statements emphasizing that claims made
about what speech tech systems can do have to be
backed up by controlled scientific studies
published in reputable peer-reviewed journals.
45
Observation in matters of food and drug safety
and efficacy, in the US, there is the FDA that is
a watchdog and gatekeeper on products and
processes.
Can the speech tech industry lobby for a similar
such body to pass on the efficacy of products in
their domain.
For that matter, why should such a body pass on
ANY technical product?
Consider, for example, that the NY Times recently
reported on a device marketed by a UK company,
the ADE 651 a hand-held "remote portable
substance detector" that is claimed to be able to
detect from a distance the presence and location
of various explosives, drugs, and other
substances. The device has been sold to a number
of countries in the Middle and Far East,
including Iraq, for as much as 60,000 per unit.
The Iraqi government is said to have spent 52m
(85m) on the devices. However, investigations by
the BBC and other organisations have reported
that the device is little more than a "glorified
dowsing rod" with no ability to perform its
claimed functions. from Wikipedia
46
Useful Links.
47
The Nemesysco saga in the News and Blogosphere
http//balneus.wordpress.com/2009/02/17/science-jo
urnals-commercial-censorship-law-and-social-securi
ty-benefits/
https//antipolygraph.org/cgi-bin/forums/YaBB.pl?n
um1230840076/4
https//antipolygraph.org/blog/?paged2
https//antipolygraph.org/blog/?p245
Stockholm University account of the
event http//www.su.se/english/about/news-and-eve
nts/scientists-threatened-with-legal-action-1.1149
A statement released by Nemesysco with critical
comments by Lacerda attached http//www.ling.su.s
e/staff/frasse/LVA_technology2009/NemesyscoRespons
etoLacerda_commented.pdf
48
http//languagelog.ldc.upenn.edu/nll/?p1390
The actual US patent http//www.freepatentsonline
.com/6638217.pdf
A brief report by the Science Insider (AAAS
website) http//blogs.sciencemag.org/scienceinsid
er/2009/05/royal-swedish-a.htmlmore
Neuroskeptic (scroll down to last
article) http//neuroskeptic.blogspot.com/2009_02
_01_archive.html
An account in Nature News (web news from the
prestigious British science journal,
Nature) http//www.nature.com/news/2009/090216/fu
ll/news.2009.99.html
The Ministry of Truth blog provides important
evidence that the scientific studies validating
LVA have been done by individuals with a conflict
of interest, including financial ties, to firms
promoting or distributing Nemesysco
products http//www.ministryoftruth.me.uk/2009/03
/18/purnells-lie-detector-indecent-disclosures/
49
http//www.ministryoftruth.me.uk/2009/03/13/purnel
ls-lie-detector-some-things-the-guardian-didnt-men
tion/
Much overlap with previous links but adds
discussion about legal and ethical aspects of the
use of Nemesyscos device and of their threats to
quash scientific research which doesnt boost
their sales http//balneus.wordpress.com/2009/02/
17/science-journals-commercial-censorship-law-and-
social-security-benefits/
LanguageLog (by Mark Liberman) http//languagelog.
ldc.upenn.edu/nll/?p1390
Language Log (by Mark Liberman) on the history of
VSA http//itre.cis.upenn.edu/myl/languagelog/ar
chives/001149.html
I like this one, Sceptiphrenia an occasional
blog on science, scepticism, and
philosophy http//sceptiphrenia.wordpress.com/cat
egory/voice-risk-analysis/
50
Thank you.
51
Slides not used
52
Controversial device analyzes passengers' voices
Tuesday, January 10, 2006 CNN no author
by-line
The GK-1 voice analyzer, created by Israeli firm
Nemesysco, requires passengers to don headphones
at a console and answer "yes" or "no" into a
microphone to questions about their travel
plans. The manufacturers say the device, which
will cost between 10,000 to 30,000, will
usually be able to pick up uncontrollable tremors
in the voice that give away liars or those with
something to hide. "When you are very stressed,
very excited, very confused or you have some
hidden agenda, then different messages go to the
voice which are not controlled," Nemesysco CEO
Amir Liberman told CNN. "Our software is capable
of extracting those out to build a profile and
then make the decision. Those that fail the
screening are led away for more in-depth
questioning and, if necessary, searches.
Liberman says the device has proved highly
successful in tests, but admits that the results
can sometimes be difficult to interpret with
around 12 percent of passengers likely to show
stress even when they have nothing to hide. The
detectors are also likely to raise objections
from civil liberties groups already upset over
intrusions on privacy from current security
measures.