Title: Learning How to Value
Abstract: In formal theories of decision-making, we often take an agent's preferences as given and criticize them only when they fail to be coherent in themselves - such as an intransitive preference for love over money, money over fun, but fun over love. This raises a question: is there a possible formal critique of preferences beyond internal coherence? In particular, consider cases where intuitively, increased domain knowledge leads to new values, such as when being forced to look at a lot of Minimalist paintings for a course leads to a new interest and appreciation for those paintings. On their face, these cases look like descriptive knowledge about the environment shaping preferences in a rational, or at least comprehensible, fashion. In this talk, I will argue that we can indeed give a cogent account of this process using tools from reinforcement learning. The key is allowing descriptive facts to shape the state space, the division of the world and the agent's actions into coarse-grained possibilities. I will show that because of a connection between state spaces and latent preferences, allowing the world to shape the agent's state space can lead to changes in what she can and does care about.