Researchers evaluated whether LLMs update their estimates of user preferences in line with optimal Bayesian inference by testing them in a simplified flight recommendation task. The setup compared LLM behavior with a Bayesian assistant that followed the optimal Bayesian strategy.
In the task, LLMs acted as assistants with a simulated user for five rounds. In each round, three flight options were shown to both the user and the assistant. Each flight was defined by a departure time, a duration, a number of stops, and a cost.
Each simulated user had a set of preferences for those features. For each feature, the user could have a strong or weak preference for high or low values, or no preference at all.
The Bayesian assistant maintained a probability distribution representing its estimate of the user’s preferences and used Bayes’ rule to update that distribution as new information from the user became available. The controlled setting made it possible to measure how far LLMs deviated from that strategy.
The assistant’s goal was to recommend the flight that matched the user’s choice. At the end of each round, the user told the assistant whether it had chosen correctly and provided the correct answer.
Source: research.google.
Companies can share verified announcements through Newz9’s international press release submission page.
