Language Technology

1
Language Technology Empowerment of the Bhashas

• Udaya Narayana Singh
• Central Institute of Indian languages, Mysore

LTT-IT VALEDICTORY ADDRESS
2
Two faces of language

• SOCIAL
• GRAMMATICAL

• Language is a human construct, undoubtedly
  which is why the saying Language is species
  specific and species general is so apt.
• Lets turn the question around To what extent is
  man a construct of language?
• To a great extent will be the answer for we
  know that language shapes our world view.
• Language, then, has two-fold existence
  grammatical and social, and man has two others
  individual and social.
• If we remove the common factor then, is language
  an individual entity?

3
Why did man create language-related technology?

• As a social instrument, language acts as the
  prime vehicle for encoding, accessing and
  disseminating information.
• In all such acts, the attempt of the human
  aggregate is to win over the curse of rapid
  fading which Hockett describes in detail.
• In devising all our technologies all through,
  mans effort has been to ensure that words or
  texts do not get lost
• This applies to the most primitive to the most
  sophisticated language-related technologies, from
  the discovery of metrics and power of rote to
  stylus or pen to be used by scribes.
• Also valid for the modern-day technologies like
  printing, radio, recording, filming to computers.
• Rather, texts be preserved (encoded) in a manner
  that they become easily retrievable.

4
Arresting collective knowledge

• Initially, it was thought that it was possible
  to arrest knowledge by depending entirely on
  compositional structure of texts and rote memory
  but a time came when its futility was evident.
• It was also thought necessary to organize
  public debates and dialogues to both test
  application of accumulated knowledge of a given
  school of thought and also to sharpen it.

5
One technology replaces another
In expanding our knowledge bases the real
problems came when those who created and
perpetrated knowledge also began to realize its
immense power and tried to ensure that texts are
limited to clans. This is also the time when
individual authors began ascribing their names
and also names of their sponsors in each text
marking the beginning the IPR in this part of the
world.
6

• What began as a sustained and sustainable effort
  in keeping knowledge in public space became
  activities in privatizing and monopolizing
  knowledge. Politicization of the varna became a
  menace.

On the other hand, the opposing trend of
liberating knowledge also ran parallel to this
limiting trends.
7
Lessons to be learnt

• From the history of creation, expansion
  distribution of knowledge, there is an important
  lesson to be learnt.
• In the world that has become complicated with
  IPRs, market restrictions, patenting, and
  monopolistic tendencies, one cannot make
  everything open and available to all no doubt.
• Even Universal Digital Library projects or
  liberating facilities like internet suffer their
  own restrictions.
• But developers of Indian languages technologies
  could ensure in the model of development and
  distribution they choose that big players and
  software giants do not monopolize our own
  language tools.
• Even when we share resources, we must make sure
  that benefit goes to end-users of the tools

Contd/
8

• Be that as it may, events such as LTT-IT make us
  think how such benefits can accrue to all?
• What could we do to ensure that their spread
  percolates down to all layers of society?
• These questions are specially important for us at
  CIIL.
• By setting up an institution with such focus as
  ours, the Government has committed itself to
  ensuring that such technologies change ordinary
  lives.

9
Challenges before Indian languages

• The emphasis was on opening up of greater number
  of contexts that remained locked firmly for those
  who were conversant only with indigenous
  languages.
• This included contexts like mass media, school
  education, legal proceedings and official work.
• As we could see, each one has a wide application,
  and success that may be possible to achieve in
  any context will benefit millions of people in
  one stroke.

• Here I remember with fond memory what the likes
  of Shri Vavilala Gopalakrishnaiah, in whose
  memory this Seminar is dedicated, had stood for
  - hand in hand with stalwarts in linguistics like
  Chatterji, Katre or TPM
• What were these enthusiasts of Bhasha tradition
  trying to tell us?
• Their entire thrust was on creation of conditions
  that make it possible for our languages to
  compete with international modes of
  communication.

Contd./-
10

• What do we require to bring in this kind of
  change in the use of our own languages like
  Telugu, Bangla or Marathi? We need first of all
  the political will, and once this will is
  demonstrated, it must be backed up with hard work
  by linguists language technology groups.
• But one could easily see that for any
  sophisticated tools to be created, we must be
  first equipped with some basic tools that would
  contribute to creation and maintenance of
  standards and frames of reference.
• What I mean by basic tools here are much needed
  every-day objects like a comprehensive
  dictionary, a set of technical glossaries,
  special-purpose lexicons, thesauri, style manuals
  and a reference grammar which are all projects
  that must be done by linguists.
• While implementation of our policy decisions on
  promotion of Indian languages need not wait for
  these tools to be developed, those who were
  equipped with the knowledge and techniques that
  would construct such facilities were expected to
  go on developing them on a parallel track.
• No doubt that these tools required a concerted
  attempt to develop and refine.

11
Sensitivity required

• True these are activities that couldnt be
  performed without thorough knowledge of
  linguistics and the given language structure.
• But then they would also require a rare
  sensitivity towards variations that existed in
  ones own language. One must show the respect to
  the speech variety a child inherits from her
  mother.
• I know there are many who would not like
  deviations from what is called the Standard.
• Nevertheless, we have all witnessed the power
  demonstrated by many such varieties in the way
  natural history of our languages had allowed them
  to be developed.
• Standards, as we know, emerge, and are not made
  to design. And yet, the history of language
  standardization has had a curious path so far.

12
Respect for Speech Varieties and linguistic
minorities

• I know some of my colleagues would like to see
  all members of our speech community speak in the
  same way.
• But then, we know we cannot wish speech variation
  away, nor can we undermine linguistic minorities
  who have chosen to live with the majority in
  harmony.
• Lets ask ourselves Isnt it wonderful to live
  in a plural world? Yes, it is.
• What kind of space is this plural world? Is it
  a world of hope, or a world full of despair?

• Is it a world in which only rivers flow or is it
  a place where the mountain blows it counch loud
  and clear? Can there be a place for both, the
  stationery and the hyper-mobile, the young and
  the old, or the aged and the agile?

13
The Plural World

• Didnt our poets say
• The rivers run swift with a song, breaking
  through all barriers. But the mountain stays and
  remembers, and follows her with his love.
  (Tagore, 1913. The Gift, in The Crescent Moon, In
  Das, S.K., ed. 1994 153)

• Even when we talk of streams, and claim to be in
  the mainstream, or at least, swear by it, where
  are the other streams those numerous torrents
  and tributaries?
• Do the edges, the periphery, have a story to
  tell? Many may wonder Those who have by now got
  into a habit of staying only in their cocoons,
  can they narrate a tale of places far and wide?

14
Managing Multilingualism

• Let me further ask Do we live in a world where
  the alphabet begins with an I and ends with
  another I where there is no scope for
  othering? Shouldnt there a scope for you?
• Arent we forgetting the fact that the creative
  moment comes only when the I and the you begin to
  mould each other?
• Let I and you merge into a we.
• Talking about us, in India, we have had a rich
  and long experience of living together.
• Those who ruled at the regional as well as
  supra-regional levels have also had interesting
  experiences in managing our multilingualism.

15
Tamed and Wicked Problems

• No doubt there have been both tamed and wicked
  problems.
• While tamed and tamable problems have have had
  their resolutions, there are many wicked problems
  that are still dodging any attempt to solve,
  however well-meaning the attempts may be.
• Coupled with that, there have been interesting
  interplay of different formations.
• The politics of planning and execution of
  policies have been as interesting as the
  socio-political forces that any effort of this
  gigantic size has to learn to deal with.

16
Enormity of Indian Experience

• Many other developing nations are now trying to
  learn from the Indian experience in managing our
  pluri-cultural and multilingual scenario.
• For us, the time has now come to document many of
  these problems in the area of language management
  and consider situations across our space which
  pose problems for social engineers.
• However, any attempt to prepare an archaeology of
  living traditions of these large number of
  languages in the developing and under-developed
  world has to remember the enormity of this task
  and the interesting theoretical challenges they
  throw up for our disciplines and scholars.
• Ultimately, even this documentation has to depend
  on technology we would choose.

17
Linguistic fanaticism on the rise

• The last five decades in the life of our nation
  has seen numerous language problems in different
  parts of the country.
• Some problems have arisen from denial of basic
  language rights to minor and minority speech
  communities while others sprang up from control
  over scarce resources.
• While some communities have been linguistically
  tolerant to others, some have demonstrated
  prejudices.
• Studies have revealed interesting linguistic
  attitudes that a given sub-group has towards
  others within a speech community.
• But all these things happen as we do not realize
  two things (a) All said and done, South Asia IS
  a LINGUISTIC AREA, and there are greater rewards
  in exploring the nature of this convergence and
  (b) Language Technology can now enable us to
  efficiently manage such a vast plural state, and
  also to go from one language to the other in
  strategically important ways.

18
Problems facing our endangered languages

• Compare this scenario with a recent prediction by
  a bio-mathematician called Pagel that, out of
  6,000 odd distinct human languages spoken all
  over the world, only 10 will survive the first
  half of this new century.
• Given that the developing and under-developed
  countries house close to 90 of this stupendous
  figure, it portrays a bleak picture for many of
  our languages and their speakers.
• With these languages, their rich cultural
  heritage as much as they are preserved through
  their speech behavior will also disappear.
• I need not elaborate further that there is a
  serious problem here for our endangered languages
  as well as for their speakers, and they need our
  urgent attention.

19
Lets try to understand the Spread of the problem
in India

• The Scheduled Tribes alone account for 67.76
  millions representing 8.08 of our population
  living mainly in the forest and hilly regions
  (1991 Census)
• More than 70 are in Madhya Pradesh, Maharashtra,
  Orissa, Bihar, Andhra Pradesh, West Bengal and
  Gujarat.
• In addition, there are a large number of minor
  languages spoken by small segments of population
  that do not belong to the reserved category.
• We not only need special provisions for their
  protection from social injustices and all forms
  of exploitation, we also need concrete plans for
  development with safeguards including promotion
  of educational and economic interests
• I think a concrete Language Development Plan is
  needed.

20
What could we do in India?

• There is a general feeling among those who do not
  understand the Indian polity and the
  administrative set up of the country that we do
  not have a mechanism in place to protect and
  promote minor and minority languages.
• The trouble is that they often point to small
  countries like Nepal where both in Constitutional
  provisions and in Universal Education documents
  these issues are specifically mentioned.
• In case of India, the sheer size of the country
  and complexity of the administrative set up are
  such that it cannot be compared with other
  nation-states in this respect.

21
Divided Responsibilities Whose baby is it?

• 5th 6th Schedule of the Constitution Article
  224 made special provision the Home Ministry to
  implement
• Special representation for the STs in the Lok
  Sabha and State assemblies till Jan 2010 (Art
  330, 332, 334)
• Under Articles 164 and 338, separate State-level
  and National Commission at the Centre was set up
  in 1999, but under Ministry of Tribal Welfare.
• Then there is Commission for Linguistic
  Minorities (Allahabad) under the Ministry of
  Social Justice Empowerment

• Grant-in-Aid scheme under Article 275(1) was also
  created
• Protection of Civil Rights Act, 1955 and the
  Scheduled Castes, Scheduled Tribes (Prevention of
  Atrocities) Act 1989 were enacted
• Planning Commission took a landmark step by
  opening 43 Special Multi-purpose Tribal Blocks
  (SMPTBs) during 2nd Five Year Plan
• Later, under 4th Plan, many projects set up in
  Andhra Pradesh, Bihar, Madhya Pradesh and Orissa,
  and a separate Tribal Development Agency was
  established.

22
A lot more needs to be done with concerted focus,
esp. in prevention of land alienation from tribal
to no-tribal, review of National Forest Policy
and Forest (Conservation) Act 1980, etc

• The Fifth Five Year Plan marked a shift in the
  approach when the Tribal Sub Plan (TSP) for
  direct benefit of the STs was launched
• In 1987, the Tribal Cooperative Marketing
  Development Federation (TRIFED) was set up to
  provide marketing help
• The GIA scheme covers 376 NGOs working on them
  some on languages and cultures, each getting
  about 90 grant.

BUT ARE ALL THESE ENOUGH?
23
Possible areas we could move in
Cultural documentation Dictionaries (general
purpose) Thesauri Specialized/Technical
Glossary Literacy books Primers Style
Manuals Promote initial Literary Attempts
Language Technology for them will follow this
swolid foundation