Building Polyfolio: An AI Copilot for Portfolio Tracking and Prediction Markets

Portfolio tracking and prediction markets usually live in separate tools. You check your holdings in one app, browse Polymarket in another, and do mental math to imagine “what if I moved 20% of my portfolio into this bet?” Tax and compliance implications? That’s another layer of guesswork.

We wanted a single place where you could ask questions in plain language and get answers grounded in real data — with citations, confidence scores, and clear reasoning.

What We Built

Polyfolio is an AI copilot embedded directly into Ghostfolio, the open-source wealth management platform. It’s not a separate chatbot; it’s a floating chat panel in the bottom-right corner of the app. You stay in your portfolio context and ask things like:

• How is my portfolio doing year to date?
• Give me a full financial health checkup
• What’s trending on Polymarket right now?
• How would my portfolio perform if I put all my money on the “Will Jesus Christ return before 2027?” prediction market?

Yes, that last one is real. It’s a live Polymarket market, and the agent can find it, model the scenario, and explain tax and compliance implications.

Architecture in Plain English

The stack is a monorepo: Ghostfolio (Angular + NestJS + Prisma + PostgreSQL + Redis) plus a Python FastAPI sidecar that runs a LangGraph agent.

When you send a message, it flows like this:

1. Angular chat widget → POST to /api/v1/agent/chat (same-origin via NestJS proxy)
2. NestJS → Proxies the request to the Python agent service
3. FastAPI + LangGraph → Routes intent, selects tools, runs them, validates outputs
4. Tools → Call Ghostfolio’s REST API and Polymarket’s Gamma API
5. SSE stream → Events (thinking, tool_call, tool_result, token, done) stream back to the UI in real time

The agent uses a 7-node LangGraph topology: router, tool executor, validator, orchestrator, synthesizer, clarifier, and error handler. For multi-step queries like “full health checkup,” the orchestrator chains 2–3 tools and the synthesizer combines the results into one coherent answer.

The Tools

We built six tools, each with Pydantic-validated inputs and a shared ToolResult contract:

• Portfolio Performance Analyzer — Returns, allocation drift, benchmark comparison
• Transaction Categorizer — Groups activities by BUY, SELL, DIVIDEND, FEE, INTEREST, LIABILITY
• Capital Gains Tax Estimator — FIFO cost basis, short-term vs long-term classification
• Asset Allocation Advisor — Compares current allocation to target, flags concentration risk
• Compliance Checker — Wash sales, pattern day trading, concentration rules
• Prediction Markets Explorer — Browse, search, analyze, simulate, and run reallocation scenarios on Polymarket

The prediction markets tool is the bounty feature. It connects to Polymarket’s Gamma API, enriches data with implied probabilities, liquidity grades, and spread analysis, and can model “what if I reallocated X% of my portfolio into this market?” with win/lose outcomes, expected value, tax estimates, and compliance flags.

What We Learned

1. Don’t Let the LLM Do Math

For financial logic, the LLM should not compute. Tax estimates, FIFO cost basis, allocation percentages — those must be deterministic code. The model’s job is to route, narrate, and synthesize. The tools do the math. That keeps outputs verifiable and testable.

2. Citations Build Trust

Every claim in the response should trace back to a data point. We extract citations from tool outputs and attach them to the final answer. Users see “Sources (3)” and can expand to see exactly where each number came from. Confidence scores (high/medium/low) reflect data quality. That transparency matters for financial use cases.

3. Tool Calling Is Non-Negotiable for Determinism

Structured tool calls with validated schemas prevent the model from inventing numbers. We use OpenAI’s function-calling for routing and enforce strict input validation before any API call. Invalid inputs return structured errors (INVALID_TIME_PERIOD, INVALID_TAX_YEAR, etc.) instead of exceptions. The agent never guesses; it either has tool output or it deflects.

4. Multi-Step Orchestration Is Where It Gets Interesting

Single-tool queries are straightforward. The real value is when the user asks something like “full health checkup” and the agent chains portfolio analysis, allocation advice, and compliance checks, then synthesizes cross-cutting insights. That’s where the orchestrator earns its keep.

5. Prediction Markets + Portfolio Analytics Fit Well Together

Crypto-native users already think in event markets. Combining traditional portfolio tracking with Polymarket in one interface felt natural. The “Jesus return” scenario is a good demo because it’s memorable and shows fuzzy market resolution, scenario modeling, and tax/compliance integration in one query.

6. Out-of-Scope Handling Matters

When someone asks “What’s the weather in New York?”, the agent doesn’t hallucinate or force-fit a financial tool. It deflects politely and lists what it can do. That guardrail is important for trust and safety.

Try It Live

Polyfolio is running in production:

https://ghostfolio-production-61c8.up.railway.app

Sign in with a Security Token (there’s a demo portfolio pre-loaded). Open the chat FAB in the bottom-right and try the prompts above. The Jesus market query is a good one to show off the prediction markets integration.

What’s Next

We’re considering a frontend refactor to make UI updates easier, expanding the prediction markets tool with more actions, and packaging the Polymarket integration as an upstream contribution to Ghostfolio. The codebase is open; the agent, tools, and graph are all in the repo.

Built with Ghostfolio, LangGraph, FastAPI, Angular, and Polymarket’s Gamma API. Deterministic where it matters, transparent by design.