{"id":1163228,"date":"2026-03-03T13:59:12","date_gmt":"2026-03-03T21:59:12","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/research\/?post_type=msr-blog-post&p=1163228"},"modified":"2026-03-03T13:59:14","modified_gmt":"2026-03-03T21:59:14","slug":"actions-speak-louder-than-prompts-rethinking-how-llms-reason-over-graph-data","status":"publish","type":"msr-blog-post","link":"https:\/\/www.microsoft.com\/en-us\/research\/articles\/actions-speak-louder-than-prompts-rethinking-how-llms-reason-over-graph-data\/","title":{"rendered":"Actions Speak Louder Than Prompts: Rethinking How LLMs Reason Over Graph Data"},"content":{"rendered":"\n

By\u202fBen Finkelshtein (opens in new tab)<\/span><\/a>\u202f(University of Oxford),\u202fSilviu Cucerzan<\/a>,\u202fSujay Kumar Jauhar<\/a>, and\u202fRyen W. White<\/a>\u202f(Microsoft) <\/p>\n\n\n\n

<\/p>\n\n\n\n

Think about the last time you opened a shared document at work. Behind that simple action lies a complex network of relationships: the colleagues who edited the file before you, the team site on which it is stored, the related documents those collaborators have touched, and the organizational structure connecting all of it. Such collaborative platforms are built on graphs \u2013 rich networks of people, content, and activity. A fundamental challenge in making them intelligent is understanding what each node in that graph represents. Should this document be flagged as sensitive? Which files should surface in a colleague’s feed? Does this sharing pattern look anomalous? <\/p>\n\n\n\n

These are all instances of\u202fnode classification<\/em>: given an entity embedded in a network of relationships, the goal is to assign it a meaningful label. It\u2019s a problem that extends far beyond human collaboration to applications such as fraud detection in financial networks, product categorization in e\u2011commerce, and road traffic congestion prediction. And it’s a problem where large language models (LLMs) are increasingly being applied. <\/p>\n\n\n\n

The appeal for using LLMs is clear. Graph neural networks (GNNs), the traditional tool for this task, must be trained per dataset, don’t transfer across domains, and struggle with the rich textual information that real-world nodes often carry \u2013 lengthy document content, detailed product descriptions, user profiles. By contrast LLMs offer a compelling alternative with their broad world knowledge and flexible reasoning capabilities. Yet despite a surge of interest, the field has lacked a principled understanding of\u202fhow<\/em>\u202fLLMs should interact with graph data,\u202fwhen<\/em>\u202fdifferent approaches work best, and\u202fwhy<\/em>. <\/p>\n\n\n\n

Our new study, “Actions Speak Louder than Prompts (opens in new tab)<\/span><\/a>,” \u2013 which will appear as an oral presentation at the upcoming ICLR 2026 conference \u2013 aims to fill that gap. We conducted one of the largest controlled evaluations of LLMs for graph inference to date, spanning 14 datasets across four domains, multiple structural regimes, and a range of model sizes and capabilities. The result is a set of practical, actionable insights for people building systems that combine language models with structured data \u2013 whether in collaborative platforms, social networks, e-commerce, or beyond. <\/p>\n\n\n\n

It’s not just what you ask; it’s how you let the model work <\/strong><\/h2>\n\n\n\n

When most people think about applying LLMs to a problem, they think about\u202fprompting<\/em>\u202f\u2013 crafting the right instructions and feeding the relevant information directly into the model’s context window. This is indeed the most common approach in the LLM-for-graphs literature: serialize a node’s neighborhood into text, describe the labels, and ask the model to classify. <\/p>\n\n\n\n

However, prompting is only one way an LLM can interact with a graph. To paint a more complete picture of LLM interaction paradigms, we systematically compared three fundamentally different strategies: <\/p>\n\n\n\n