Our research combines the principled paradigm of rational agent design based on decision theory with formal definitions of the emotional states and personality of an artificial intelligent agent. We view the emotional states as the agent’s decision-making modes, predisposing the agent to make its choices in a specific, yet rational, way. Change of the emotional state, say due to an external stimulus, invokes a transformation of the agent’s decision-making behavior. We define personality as consisting of the agent’s emotional states together with the specification of transitions taking place among the states. To model the personalities and emotional states of other agents and humans, we additionally provide a definition of a personality models of other agents. Our definition allows the personality models to be learned over the course of multiple interactions with the users and other agents.