The thesis presents a novel system that enhances NPCs utilizing an LLM in games with contextual awareness of their surroundings, offering dynamic, environment-sensitive interactions. Traditionally, NPCs rely on pre-scripted dialogue and lack awareness of their environment, limiting their responsiveness to player actions. Our system addresses this by capturing a panoramic image of the NPC’s surroundings and applying semantic segmentation to identify objects and their spatial positions. We generate a structured JSON representation of the NPC’s environment by combining object locations with segmentation information. This data is provided as context to a LLM, enabling NPCs to incorporate spatial knowledge into their conversations with players. The result is more immersive gameplay, where the NPC can reference nearby objects, landmarks, and environmental features during interactions, enhancing believability and engagement. The thesis discusses the technical implementation of this system, demonstrating how integrating visual perception into NPCs can transform in-game dialogues and interactions.
|
