Understanding LLM analytics: Current state and future possibilities

As I continue my research into Large Language Models (LLMs), I've been exploring the emerging field of LLM analytics. Many of us are asking: How can we actually measure and track brand visibility within AI tools? What metrics matter? What's even possible right now?

In this article, I'll share my findings on the current state of LLM analytics and what we might expect in the future.

The challenge of visibility analysis

One of the biggest questions I'm exploring is: To what extent can we track how often specific search terms (particularly brand names, since this is relevant for enterprise users) appear in LLM outputs?

Currently, this is still murky territory. Major LLMs like Claude and ChatGPT don't provide analytics dashboards, and obtaining this information is challenging. Based on my research, here's what companies appear to be doing:

1. Collect keyword trend data (similar to how tools like Ahrefs operate today)
2. Gather FAQs related to these keywords
3. Use an LLM to generate prompts based on keyword trends and FAQs
4. Push these prompts via API to various LLMs
5. Collect and analyze the outputs that come back

What remains completely unclear is how frequently certain prompts are actually being used by real users and which prompts should be prioritized. Unlike search volumes, where we have sufficient data to estimate the value of specific search terms, we simply don't have equivalent data for LLM prompts yet. This would be a completely different scenario if LLMs decided to provide this kind of data directly.

How LLM analytics might evolve

Unlike highly defined search keywords or combinations, I'm finding it's uncertain whether prompts can be simplified to just a few words. My guess is that we'll likely see a shift toward working with "topicals" and "subtopicals" instead. These might function more like long-tail SEO, where specific word combinations indicate which topics cause a brand to appear more prominently in responses.

What seems certain is that we'll eventually have better tools for understanding AI visibility. But when this will be possible without manually providing prompts or topics remains unclear.

Citation analysis

If we solve the visibility tracking problem, citation analysis becomes relatively straightforward. This would likely be requested alongside the prompt, with the output formatted to separate content from citations.

Based on my research, you generally can't effectively send sources via API requests. Since current LLMs typically only use five to ten sources per prompt, storing this data per output would be simple. Interestingly, while ChatGPT doesn't provide this information through its API, Claude has been doing so since January 2025 with their Citations API.

Prompt analysis misconceptions

I've noticed that most AI tools (including Profound) don't approach prompt analysis as analyzing the prompt itself. Instead, they compare mentions in prompt outputs across different platforms—essentially comparing results between services like Google and Bing.

What I find particularly interesting is the question of platform-specific optimization. To what extent does optimizing for Claude differ from optimizing for Google's AI Overview (AIO) or Gemini? Are the strategies comparable, or are we swimming against different currents for each platform?

Topical analysis: The most promising direction

One of the most exciting possibilities, in my opinion, is topical analysis—tracking the key topics where your brand appears in outputs without needing to push prompts manually. This would allow companies to understand which topics currently highlight their brand.

However, I expect initial implementations will be quite broad. We'll need to discover many ways to make this more specific and actionable before it becomes truly valuable.

Sentiment analysis

Who cares?

(I find this particularly low-value in the context of LLM analytics, as sentiment is often balanced or positive by design in these systems.)

Platform analysis

According to Profound, we should be able to "see the prompts that lead to mentions of your brand." This would provide platform-specific insights into which prompts currently generate the most brand mentions.

Regional analysis

While regional insights are theoretically possible, I don't find them particularly interesting at this stage of LLM analytics development.

Insights into user-LLM conversations

It's extremely unclear how this information is being obtained. Some companies claim to offer "real AI conversation data" and even "historical data" going back to fall 2024. They also claim to provide platform-specific volume data, similar to AI keyword search volume data.

There's a very high probability that all of this is based on traditional keywords (volumes and trend lines) that are then converted into prompts and outputs—making them purely estimates. The same applies to "Measure Keyword Volume" and "Trending Data" offerings. These involve many assumptions and should be treated with appropriate skepticism.

Agent analytics: The final frontier

A few other areas being explored include:

• Analyzing which AI crawlers visit your site: This can only be based on GA4 data or, again, assumptions.
• Optimizing HTML structure for crawlers: This is relatively simple to implement based on a small number of factors.
• Optimizing robots.txt: An amusing concept given the current state of LLM crawling technology.

What's next?

Based on my research, I believe we're still in the very early stages of LLM analytics. Many current offerings involve significant assumptions and estimations rather than direct measurement.

As I continue exploring this space, I recommend focusing on:

1. Experimenting with citation analysis where possible
2. Beginning to map your content to potential topical areas
3. Testing how different optimization strategies perform across platforms
4. Being skeptical of tools claiming to offer comprehensive LLM analytics without transparency about their methodologies

The setup for this type of analysis should take about two days, after which you'll want to establish a complete data lock-in process.

This is absolutely a developing field, and I'll continue to share insights as they emerge. What aspects of LLM analytics are you most interested in exploring?