Designing LLM-Based Assistants for Automated Financial Stock Analysis: A Multi-Agent RAG Approach

Abstract

This thesis explores the use of Large Language Models (LLM) as AI assistants capable of generating high-quality company stock reports for financial analysts and retail investors. As financial analysis traditionally demands interpretive depth, structured reasoning, and domain-specific language, this thesis investigates whether LLMs can emulate the narrative sophistication and insightfulness of expert-written reports, such as those from Morningstar Inc. A multi-agent architecture was designed,inspired by MarketSenseAI, featuring agents for processing fundamentals, market news, and price dynamics, coordinated by a central Signal Agent to generate comprehensive reports.Retrieval-Augmented Generation (RAG), prompt engineering, and embedding-based document segmentation were applied to reduce hallucinations and enhance factual reliability. The models,GPT-4o, GPT-4o-mini, o3, and o4-mini, were evaluated against Morningstar reports using a hybrid method: a Likert-style LLM-as-a-Judge scoring system assessing clarity, comprehensiveness,insightfulness, and actionability, accuracy in the form of comparing specific metrics in the generated reports against the reports from Morningstar, and a real-world backtest of buy/hold/sell recommendations over 3-, 6-, and 12-month periods. Results suggest that, with proper architecture and prompting, LLMs can produce structured, interpretable, and actionable investment narratives,though barriers remain in data availability and evaluation fairness, keeping it from achieving the level of a Morningstar analyst. This study contributes a practical framework for AI-driven financial reporting and highlights broader implications for integrating generative AI into information-intensive domains.