AI Observability by OpenObserve

Does this integrate with products like Hermes? I tend to use this to as the main agent coordinator for my projects but it then also delegat3s to opencode. It would be amazing if this integrated with hermes so that it can monitor usage across all of my projects including the tasks that are delegated. I like the idea of b3ing able to better visualize and analyze usage directly, it is often the hidden layer that is missing and so telling when things are either not working or costing more than expected. Great work.

Hey PH - Eli from the GTM team at OpenObserve here! We've worked tons of different use cases and migrations, so if there are any questions feel free to drop them below!

We run OpenObserve self-hosted at Feature1, so congrats from a user. What I like about AI Observability: agent sessions land next to the logs, metrics and traces we already keep there, on OpenTelemetry, instead of in a separate LLM tracing tool with its own bill. Following a failure from the model call into the backend and the database is the part standalone tools cannot do. Looking forward to putting our own agent runs through it.congrats! @prabhat_sharma5

Congrats on the updates! 🎉🎉Some minor feedback, last I checked (~ month ago) it only allows integration via manual integration or custom harnesses, which is great of course for many cases.However in case is with an existing harness, such as OpenClaw, Hermes, and LobeHub, I didn't see an easy way to easily integrate this feature.Ideally would leverage corresponding model's API or webhooks to allow all uses of inference to be hooked up even in cases where the harness isn't cooperative.Maybe this is all solved now, but Iast I checked was an issue as much work is done via existing or well known harnesses.Please keep it up, really great improvements over the last year!!

I love OpenObserve because of its very friendly to use with great UI. I will like to know if there is test package that can be used as a proof of concept with my team?

How long does it take to go from a fresh install to seeing my first agent trace? What does the instrumentation actually look like?

@hengfei_yang for someone who wants to configure evaluation pipeline, how easy is it going to be? I am also looking to score the responses, say hallucination, risky and etc. With the agentic workflows we have lots of tool calls and that takes a while so how could we trace these tool calls and improve them?

Can I see what AI agents are doing when they interact with each other, and trace where things go wrong if one of them starts going off the rails?

Cool service! Can I break everything down by user and see statistics for each user? We have many agents working within a single user session, and that would be very useful for unit economics.

Is the trace approach applicable to multiple agents, or is it designed for single agent flows only?Btw, Congratulations @hengfei_yang & team @OpenObserve ✌️🎊.

Hi Product Hunt community, I'm Jake and one of the team members at OpenObserve. We're so excited to hear your feedback and questions today. Please drop your message here and let us know what you think.

Does ai observability share the same storage and query layer as the APM/Infra monitoring engine?

Hey Product Hunt, DevRel at OpenObserve here.We've been building AI Observability into OpenObserve for the last few months, and it's finally live.Everyone is shipping agents, but most tools only tell you the request went through, not whether the answer was any good. a hallucination still returns 200 OK. that's the gap we're closing.how?- trace every agent, tool call and model request- score quality on live traffic, and run experiments before you ship- send weak traces to a human, then turn them into eval datasetsMore importantly it's one unified platform. LLM traces sit next to your logs and infra, OpenTelemetry native, self host or cloud, priced per GB not per span.It's in beta and we'd love practitioner feedback. if you're running agents in prod, what's painful for you today?thanks for taking a look.

If an agent uses many models in one session, can I see how much costs for each model, and total cost for the session?

Hi Product Hunt, Simran here from the OpenObserve team.One thing we kept running into while working with AI workloads: an LLM call can succeed while the agent still fails.That’s the gap we wanted to solve with AI Observability.Instead of looking at LLM calls in isolation, we wanted to see the entire session: LLM calls, tool calls, tokens, cost, latency, evaluations, and the infrastructure underneath it.And because it’s built into OpenObserve, those AI traces live alongside your existing logs, metrics, and application traces. No second observability stack just for your AI workloads.It’s also fully OpenTelemetry-native, so you can bring telemetry from the frameworks and instrumentation you’re already using.If you’re already running AI workloads in production, I’d be curious to hear what you’re actually using today to debug them.