5/27/22, 8:18 PM Integrating autoconversion: Facebook’s path from Zawgyi to Unicode - Engineering at Meta Last year, to support Myanmar’s transition to Unicode, we removed Zawgyi as an interface language option for new F and other policy-violating content weren’t going to trip over Zawgyi content and began work on integrating font conv country continue its transition to Unicode, we are announcing that we’ve implemented font converters in Facebook a Unicode converter will continue to allow people transitioning to Unicode to read posts, messages, and comments eve will detail the technical challenges involved in integrating these converters, including how we differentiate Zawgyi tex how to convert between the two, as well as some lessons we learned along the way. Facebook post without conversion (left) and with it (right). Why Unicode? Unicode was designed as a global system to allow everyone in the world to use their own language on their devices. B Which means the people using those devices are now dealing with compatibility issues across platforms, operating sy content producers in Myanmar often post in both Zawgyi and Unicode in a single post, not to mention English or othe combined renderings; requires twice as many code points to represent only a subset of the script; and vowel code po which leads to search and comparison problems, even within a single document. This makes any kind of communicati Facebook supports Unicode because it offers support and a consistent standard for every language. In Myanmar, in p It allows people in Myanmar to use our apps and services in languages other than Burmese. Zawgyi supports en spoken in Myanmar, like Shan and Mon. It offers a normalized form for languages in Myanmar, which helps us protect the people who use our apps by det tools. It makes it more efficient for us to review reports of potentially harmful content on Facebook, and content review encoded. A three-pronged approach When we first started looking at Myanmar encoding, our top priority was making sure our systems that detect harmfu goals for that in this blog post. The same challenges (such as multiple code points and combined renderings) that ma classifiers and AI systems to effectively detect policy-violating content. Fortunately, we are not the only company working on this issue, and we were able to use Google’s open source myanm major upgrade, in terms of accuracy of detection and conversion, over the regex-based library we had been using. Ab content into Unicode before going through our classifiers. Implementing autoconversion across our products was not encoding detection, device encoding detection, and conversion — had its own challenges. https://engineering.fb.com/2019/09/26/android/unicode-font-converter/ 2/7