Alon Kipnis - Detecting Some Human Edits of AI-Generated Text with Information Theory and Higher Criticism
We address the question of whether a given article is the output of a generative language model or perhaps includes some significant edits by a different author, possibly a human. For this problem, we develop a detection method that involves many perplexity tests for the origin of individual sentences, combining these multiple tests into a global test of significance using Higher Criticism (HC). As a by-product, we can identify sentences or other text chunks suspected as generated by a different mechanism than the language model.