Anna the avatar translates platform announcements into Swiss-German sign language. | Image: University of East Anglia/Universität Zürich, Institut für Computerlinguistik/trainslate

Sign languages are natural languages. They are developed and used in a linguistic community just like spoken languages. In Switzerland, there are three in use: Swiss-German, French and Italian sign languages.

Modern linguistic research into sign languages began in the 1960s with studies in the USA and the Netherlands. In Switzerland, Penny Boyes Braem founded the Center for Sign Language Research in Basel in 1982 as a private, non-profit organisation. “Back then, no other institution in Switzerland was prepared to promote research into these languages, which were looked down on by some”, she explains. “For linguists, however, the descriptions in these languages, which are produced and perceived visually, are highly interesting because they often shed new light on traditional language theories”. For example, in sign language, visual iconicity is present on all levels – in other words, there is a visible relationship to linguistic expressions. This casts a ‘long shadow’ over the linguistic principle that the relationship between the linguistic sign (i.e. words) and their meaning is arbitrary in all human languages. For example, the words for the concept ‘tree’ are very different in unrelated languages. But in many sign languages of the world, the signs for ‘tree’ offer a pictorial aspect of the form of a tree.

Compulsory sign language

Today, sign language is being investigated at several institutions in Switzerland, such as at the University of Applied Sciences of Special Needs Education (HfH), at the University of Zurich and at the Zurich University of Applied Sciences in Winterthur (ZHAW). These projects need researchers who have themselves mastered sign language. One of them is Katja Tissi of the HfH. She has been deaf since birth and learnt her sign language from her older sister, who is also deaf. “As a child I felt bad when I used sign language”, she recalls. Until 1980, Swiss experts concentrated on sending the hearing-impaired to hearing and speech training sessions, and they communicated mostly via lip-reading. But on a visit to the USA, Tissi discovered that sign language was a subject of scientific research over there. “When I saw that sign language was recognised, it opened up whole new worlds to me, and gave me self-confidence”.

Sign language research has profited from developments in computers and multimedia. What is of central importance is image recognition. “Just like people who can hear, more and more hearing-impaired people are using the Internet and social media” explains Penny Boyes Braem. In order to communicate via the Internet, the hearing-impaired often produce video clips in sign language. That way, they can be easily identified – unlike the users of spoken language, who can remain anonymous thanks to their use of written communication. This is why researchers are developing technologies to recognise signs on video, and then to have these translated back into gestures by a completely anonymous avatar. One initial step in this direction is the ‘Smile’ project (Scalable Multimodal sign language technology for sIgn language Learning and assessmEnt). It is being run by the Idiap Research Institute in Martigny in collaboration with the HfH and the University of Surrey (UK). This project is developing automatic sign language recognition technology that will offer learners feedback on their use of Swiss-German sign language.

Translating announcements

Automatic machine translation also plays a role in the doctoral research of Sarah Ebling at the University of Zurich. Hearing-impaired people cannot understand the platform announcements given at train stations. So Ebling has developed a system that enables the announcements to be shown by an avatar in Swiss-German sign language (DSGS) on a smartphone.

Further research areas include the signals of the hands and the face as well as the cognitive processes involved when using sign languages. “Various studies have shown that coordinating a manual gesture with a non-manual component is a great challenge for adult learners who can hear. We still know far too little about the functioning of this modality-specific language acquisition”, explains Tobias Haug, the director of studies and a researcher at HfH. In order to investigate this, the HfH is planning a project for a so-called ‘learner corpus’ in DSGS. “The goal of this learner corpus is to collect data on learners over a certain period of their DSGS acquisition. We want to find out, for example, the typical difficulties they encounter when learning a sign language”.

Astrid Tomczak-Plewka is a freelance journalist
in Bern.