Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
jiteshkprasad 
posted an update 13 days ago
Post
90
Okay so, we at
udgamlabs just released Lekhani, a Hindi transliterator that is 0.87 MB in total. You type Hindi in English letters, like "kaise ho", and it writes कैसे हो.

The whole thing is a 0.81M-parameter model at 4 bits, a 200,000-word list and a 5 KB JavaScript runner with no dependencies. It runs in the browser or in Node, so nothing you type leaves your device.

On the Aksharantar Hindi test it gets 65.9% of words exactly right, 7 points more than IndicXlit's model on its own (58.9%), which has 14x more parameters. IndicXlit's full package is still ahead at 68.7%, but its word list alone takes 358 MB of memory, about 400 times the size of all of Lekhani. I think for a web page or an app, that difference matters a lot.

It reads one word at a time, so "main" and "mein" can come out the same, and a few common words still come out wrong. The card has the full table, the other open models we measured and where it falls short.

It's Apache-2.0 for the model and runner, and CC0 for the word list. If you type Hindi in English letters, try it and tell me which words it gets wrong.

Model: udgamlabs/Lekhani
Try it: https://udgamlabs.com/models/lekhani
Code: https://github.com/udgamlabs/lekhani
Kreout

The tiny browser bundle is a compelling part of this. One integration case I’d add is mixed input: transliterated Hindi beside a URL, an email address or user_id. An explicit way to leave those spans untouched would make it easier to put this into a text field without surprising the user. That behavior would be useful to document alongside word-level accuracy.

·
jiteshkprasad

@Kreout You are right, and thanks for this. I just checked, right now tr.text() changes every run of Latin letters, so name@example.com comes out as नाम@एग्जाम्पल.कॉम, and a URL or a user_id gets changed the same way. In a real text field that would surprise people.

For now, if you are building with it, you can keep URLs, emails and IDs as they are on your side and call tr.word() only on the words. The runner is one small JS file and it is all open, so it is easy to add that rule. And I agree this belongs on the card, next to the accuracy.
dipankarsarkar

Your main → में miss comes from the word list, not the missing context.

I ran translit.js on it. The model alone puts मैं first (-0.524 vs -0.605).
में sits one frequency band higher (list -4.06 vs -7.64), and at λ=0.1 that 0.36 beats the 0.08 margin.

kal is the other shape. कल and काल share a band (-9.44 each), so the list can't vote and the model's काल wins.

On the 9,991 Aksharantar test words (I get your 65.9% exactly; the model alone is 56.2%):

  • the list fixes 1,315 words and breaks 351
  • of the misses where the right answer is in the 8 candidates: 1,022 are outside the 200k list, 800 are the list pulling toward a more common wrong word, 224 are same-band ties
  • λ and the out-of-list offset are already tuned. The best grid point I found is 66.1%

The number I'd add to the card: top-2 is 79.0%, top-3 is 82.6%.
One runner-up chip buys 13 points, more than any λ or offset setting I tried.

Did you try a two-word prior for main/mein, or does that break the 0.87 MB budget?

·
jiteshkprasad

@dipankarsarkar Thanks for running it, and your 65.9% and 56.2% match ours exactly. I did not know the main → में miss comes from the word list and not from the missing context. So our card is not fully right there, the model alone gets मैं, and it is the list that pulls it to में.

And the top-2 point is a good one. tr.word() already gives all 8 candidates, so a keyboard can show the runner-up as a chip, and by your count that takes it to 79.0%.

On your question, no, we have not tried a two-word prior. Context from the previous word was on our list of possible fixes, but we never tested it, so I do not know yet what it would cost in size. Everything is open, the model, the 200,000-word list and the runner, so anyone who wants to try it on top can.
In this post