LLMonitor

Maker

📌

👋 Hi Product Hunt, My name is Vince, and together with my friend Hugh we've been building LLMonitor, an open-source observability toolkit for builders of AI apps. If you’re using OpenAI or LangChain, the integration takes seconds to start recording data. Main Features: ⚡ Log full queries & errors 💸 Metrics into costs, tokens & latency 👨‍👩‍👧‍👦 Track users and their usage 💬 Record chat conversations & user feedback 🕹️ Prompt playground to iterate on prompts 🐾 Traces to easily debug agents 🧪 Evaluations of outputs quality 🧑‍💻 You can Self-host it on your own servers. Or use our cloud-hosted version (free up to 1k events /day.) I started building LLMonitor as an internal tool at my previous startup. We had no insight into how users interacted with our chatbots and needed a way to monitor chats and LLM queries easily.  If you’re building a conversational app powered by AI, we’d love to hear what you think and how we can make it better! Also, we'd be reaaally happy if you can support us with a ⭐ star on GitHub: https://github.com/llmonitor/llm... Thankss 🙏

Report

1yr ago

Raju Singh

@vincelwt Hey Vince. Congrats on the launch. For sure AI apps need a lot of quality safeguarding. This tool seems to be a great aid to developers for that. Keep up the good work!

Report

1yr ago