What the Thing is an Open Sourced App that Uses Image Recognition to Teach Words in New Languages

What the Thing is an Open Sourced App that Uses Image Recognition to Teach Words in New Languages

We may earn a commission for purchases made using our links.

One of the underrated perks of being multilingual is being able to remember the names of objects in languages other than your primary language, which really helps you in a pinch when the words are really just there at the tip of your tongue. For example, if I forget the word “water” in English, I can always refer to it as “पानी”, “पाणी”, “પાણી”, “پانی” or even “l’eau”. Once there, it’s just a simple matter of translating the word back to recall the name of the object.

Reddit user /u/vigzmv took this skill and made an Android application out of it. Meet What the Thing, an Android app that helps you teach new words in other languages.

/u/vigzmv is primarily a Javascript developer with no knowledge of Java. What the Thing is built using React Native, an interesting choice that supplements this particular developer’s existing knowledge of Javascript to make an Android app without knowing Java or C++.

What the Thing is inspired by Thing Translator, a web app that lets you point your phone or even laptop (since it is a web app) at different objects to hear it in a different language. Thing Translator was made as part of Google’s AI Experiments project, and makes use of Google’s Cloud Vision and Translate APIs. What the Thing takes the concept and makes an Android app out of it without using the conventional Android development tools. What the Thing also uses Clarifai‘s concept recognition and Yandex‘s language translation.

What the Thing is open source and distributed as code. You need to compile the app before you can install it. The developer promises to eventually release a signed APK for installation, but for now, you can try out Thing Translator’s web app on your phone to get a taste of it.

What do you think of What the Thing? Let us know your thoughts in the comments below!