Projects

RUBRIC-MQMACL Industry 2025

Span-level LLM-as-judge for machine translation. Rubric-style prompting reduces label bias and handles near-perfect translations; an updated version of GEMBA-MQM.

FALCON / H-FALCONTechRxiv 2025 · WMT 2025

Document-level MT evaluation grounded in functional linguistics. FALCON defines pragmatic, discourse-level criteria; H-FALCON refines them for reliable human annotation.

DDPEMNLP 2026

Discourse dependency, a metric-free, source-side measure of how far back a segment must look in its document to resolve its entities and pronouns.

FACETWMT 2026

Reference-free submission to the WMT26 Automated Translation Quality Evaluation Task. Evaluation is split into Fluency, Accuracy, and Consistency passes, each given only the context its error type needs.

MultiAgentMTWMT 2025

Translate–Post-edit–Proofread pipeline of LLM agents guided by RUBRIC-MQM error spans. Submission to the WMT25 General MT Task.

ESA CounterfactualEMNLP 2026

18,420 expert English–Korean annotations comparing coherent documents with documents mixed from different systems, testing whether document-level presentation changes judgments.

Doc-APEarXiv 2026

Document-level automatic post-editing with proprietary and open-weight LLMs, covering quality, use of context, robustness, and cost.

Korean SacreBLEUHumEval @ACL 2022

Into-Korean meta-evaluation resources for testing how tokenization (word, morpheme, subword, character, jamo) changes the human correlation of SacreBLEU.