Rendered at 16:16:47 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
Frieren 3 hours ago [-]
Very interesting, now I really wonder about Spanish, and Mandarin Chinese. There is so much written literature in Spanish that I would guess that much of it has been used to train models. For Chinese is a similar case but it has the interesting side effect to be a very different language than English. (English has germanic roots, but most of its words are latin based so not so far away from Spanish).
ahartmetz 5 hours ago [-]
Would have been interesting to include German and Dutch. The German training corpus must be much bigger while the language is roughly the same "distance" from English as other Germanic languages except Dutch. Dutch is most similar to English, but likely has a smaller training corpus than German.
vpaulus 10 hours ago [-]
Finnish is in a totally different language family than Swedish-Norwegian-Danish. However Viking performs the same result. I am wondering the outcome when using Viking for Hungarian, as it is a near relative to Finnish.
silfverstrom 8 hours ago [-]
Viking was trained on Finnish as well, so that's why it gets good results.
I ran a quick check on Hungarian, Viking didn't perform well at all(about 2x the tokens of English).