Large language models often answer the same question differently depending on whether they are queried generatively or discriminatively. This inconsistency can be addressed through the consensus game; a signaling game between a generator and a discriminator whose regularized equilibrium is used to rank candidate answers. In this thesis we formally describe the consensus game as an extensive-form game with imperfect information, compute its strategic form and its set of pure Nash equilibria, and prove a characterization of its weak sequential equilibria, showing that the unregularized game does not rule out implausible solutions. We give a detailed derivation of the piKL-Hedge algorithm and of the strategy update rule regularized by Kullback–Leibler divergence. To the best of our knowledge, we are the first to evaluate the method on natural language inference and the first to do so for Slovene, using the SI-NLI dataset with the GaMS models under zero-shot, one-shot and three-shot prompts.
|
