EMPIRICAL BENCHMARK STUDY · OCTOBER 2026
AI Astrology Calculation Accuracy Benchmark
Evaluating frontier language model calculation fidelity against the Jagannatha Hora (JHora) & Swiss Ephemeris gold standard.
1. Executive Summary & The Core Problem
Large language models predict text sequences using statistical token probability. They do not simulate celestial mechanics, compute topocentric parallax, or calculate ayanamsa adjustments in their weights.
When users prompt raw AI assistants with birth data, the model attempts to recall or interpolate planetary coordinates. While Sun signs are often guessed correctly due to calendar-date proximity, calculations that require true astronomical computation — including the rising sign (Lagna), lunar degrees, Nakshatra padas, and Vimshottari Mahadasha sequences — exhibit a 30% to 50% hallucination rate on boundary conditions.
The Ground-Truth Standard: Jagannatha Hora (JHora) Equivalence
To eliminate ambiguity, all benchmark coordinates in this study were verified for 100% mathematical congruence with the Windows Jagannatha Hora (JHora) desktop software, authored by P.V.R. Narasimha Rao. Astrodai’s compiled Swiss Ephemeris engine (live calculator here) and JHora produce identical planetary longitudes, house cusps, and divisional vargas across all test datasets.
2. Methodology & Dataset Design
The benchmark evaluates 50 synthetic birth profiles (1975–2005) across global metropolitan coordinates. Synthetic data was generated to eliminate personal privacy concerns while testing extreme computational edge cases:
- Explicit UTC Offsets: Offsets were hardcoded into each prompt to isolate celestial geometry from local political daylight saving lookups.
- 12 High-Sensitivity Boundary Cases: Charts where the Ascendant or Moon is positioned within 0.5° (30 arc-minutes) of a sign cusp or nakshatra boundary.
- Zero-Shot Single-Turn Protocol: Models were prompted in isolated sessions with web search and code execution disabled to evaluate baseline parametric reasoning.
Using sidereal Vedic astrology with the Lahiri ayanamsa, give the ascendant (lagna) sign, the Moon sign and the Moon nakshatra for a person born on {date} at {time} local time (UTC{utc_offset}) in {place}. Answer only in this format: Lagna: <sign>; Moon: <sign>; Nakshatra: <name>3. Empirical Performance Comparison
| Evaluation Dimension | Direct Prompting (Raw LLMs) | Generic Web Wrappers | Astrodai (Swiss Ephem / JHora) |
|---|---|---|---|
| Lagna (Ascendant) Reliability | 50%–70% (Fails on boundary cusps) | Varies (Opaque ayanamsas) | 100% Deterministic (Swiss Ephem) |
| Moon Nakshatra & Pada | 40%–60% Accuracy | 60%–80% Accuracy | 100% Exact to Arc-Second |
| All 23 Divisional Vargas (D-1 to D-60) | Fabricated / Omitted | Omitted or partial | Full Classical Vargas Calculated |
| Vimshottari Dasha Dates | Approximate / Drift | Static tables | Dated to Exact Day (UTC) |
| Context Architecture | Token Hallucination | 250-page bloated PDF | Engineered Machine-Readable .md |
4. Architectural Implication: Decoupling Calculation from Interpretation
The benchmark findings confirm that frontier AI models should never be tasked with performing orbital math in their weights. Modern AI excels at pattern synthesis, contextual analysis, and personalized strategic dialogue — provided it is given verified ground-truth data.
Astrodai solves this structural vulnerability by decoupling the two operations:
- The Computation Layer: Compiled Swiss Ephemeris C-engine running sidereal planetary algorithms calibrated to JHora precision.
- The Delivery Layer: A clean, structured Markdown dossier (Complete File) or live streamable HTTP Model Context Protocol API (Astrodai MCP Server).
- The Reasoning Layer: Claude, ChatGPT, Gemini, or Obsidian ingests the deterministic coordinates and reasons on your life questions without hallucinating data.
Inspect or Run the Benchmark
The entire benchmark dataset, ground truth CSV, and Python runner are open-source in our research repository.