• Skip to main content
  • Skip to secondary menu
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Crypto Currency
  • Technology
  • Contact
NEO Share

NEO Share

Sharing The Latest Tech News

  • Home
  • Artificial Intelligence
  • Machine Learning
  • Computers
  • Mobile
  • Crypto Currency

Japanese Manga Translation Via Multimodal Context-Aware Framework

December 31, 2020 by systems

Synced

A new machine translation method from Japanese start-up Mantra Inc., Yahoo Japan and the University of Tokyo enables global manga fans to enjoy immediate translations of their favourite Japanese comics. The researchers say this is the first comprehensive system for fully automated manga translation from the original Japanese into English or Chinese.

Comics are an integral component of many cultures. US superhero sagas, Korean webtoons and Chinese manhua all have unique styles and characteristics. Most comics however remain readily available only in their domestic markets, due largely to the high cost of translation. To expand the reach of these creative works, the researchers developed an automated system using machine translation (MT) technology.

Manga presents two important problems for MT: context-awareness and multimodal translation. Format-specific challenges include the comics’ unstructured mashup of text and images and the lack of corpora or benchmarks to train and evaluate models.

The team proposes a multimodal context-aware translation framework, which they say is the first to incorporate context information obtained from an image into manga translation. The framework extracts three context types that are useful for multimodal context-aware translation: scene, reading order, and visual information.

Proposed manga translation framework

The proposed method automatically builds a manga parallel corpus from pairs of original manga texts extracted using computer vision techniques and their translations. The researchers also created a multilingual manga dataset, effectively the first manga translation benchmark, and have made it publicly available.

Proposed framework of parallel corpus construction

Given a Japanese manga page as input, the system recognizes Japanese texts and translates them into the target language using the trained neural machine translation model. An image inpainting model meanwhile removes the original Japanese texts from the page, and the translated texts are rendered on the cleaned page with optimized font size and location.

Results of fully automatic manga translation from Japanese to English and Chinese

By integrating text recognition, machine translation, and image processing into a unified system, the proposed method achieves impressive automatic image-to-image manga translation and has established a foundation for further research in this area of MT.

The paper Towards Fully Automated Manga Translation is on arXiv.

Filed Under: Artificial Intelligence

Primary Sidebar

Stay Ahead: The Latest Tech News and Innovations

Cryptocurrency Market Updates: What’s Happening Now

Emerging Trends in Artificial Intelligence: What to Watch For

Top Cloud Computing Services to Secure Your Data

The Future of Mobile Technology: Recent Advancements and Predictions

Footer

  • Privacy Policy
  • Terms and Conditions

Copyright © 2025 NEO Share

Terms and Conditions - Privacy Policy