Helicone AI

Open-source LLM Observability for Developers

5.0•16 reviews•

1K followers

Open-source LLM Observability for Developers

5.0•16 reviews•

1K followers

Visit website

Automation tools

•

AI Metrics and Evaluation

Helicone is the open-source platform for logging, monitoring, and debugging your AI applications. Free to start. 1-line integration to access usage tracking, LLM metrics, prompt management and more. See a list of integrations at docs.helicone.ai

Free

Launch tags:

Open Source•Analytics•Developer Tools

Launch Team / Built With

Justin Torre

Helicone AI

Maker

📌

💡 Hi Product Hunt! 👋👋 Justin here, CEO of Helicone with my co-founder Cole (@cole_gottdank). We are extremely excited to launch Helicone on Product Hunt! 🧊 Background story Before launching Helicone, we developed several projects with GPT-3/GPT-J, including valyr chat, airapbattle.com, tabletalk.ai, dreamsubmarine.com, and debateai.org. For each project, we used a beta version of Helicone which gave us instant visibility into user engagement and result quality issues. As we talked to more builders and companies, we realized they were spending too much time building in-house solutions like this and that existing analytics products were not tailored for LLM workloads. 🧊 Helicone’s differentiators Other startups (LangChain) are working on similar products like LangSmith with high learning curves, rigid pricing structure and closed source, or are slow-moving giants like Datadog which isn’t exactly purpose-built for Large Language Models. 🧊 Our goals Our goal is simple. As a team of developers, we realized that we wanted a tool that “just works”, and was dead easy to start. We focus a ton on the developer experience, by allowing developers to access all observability and monitoring features with just a header. We aim to help developers surface insights or errors, so they can focus on problem-solving faster. 🧊 What’s new with this launch ⭐ Improved dashboard & request table: we make it faster to segment your data on the dashboard, and view responses and relevant monitoring metrics such as cost, latency, and time to first token (TTFT). ⭐ Introducing Sessions: trace your AI agent and RAG chatbot, debug them or refine your responses in different context in our Playground. ⭐ Introducing Prompts & Experiments: iterate on your prompts, experiment with a different dataset or model, and evaluate your responses with Scores. ⭐ Scalability & Reliability: since the beginning, we’ve made significant progress towards 100% log coverage, so all your logs will be captured. Thanks for supporting us on the launch! We are always excited to hear your feedback. Feel free to message us or “submit an issue” on GitHub, and we will respond/build it. Github: github.com/Helicone/helione Docs: docs.helicone.ai Happy developing! ✌️ And thank you @mwseibel for hunting us!

Report

12mo ago

Max Savonin

@cole_gottdank @mwseibel @justintorre Helicone's focus on simplifying LLM observability and monitoring sounds like a valuable solution for developers working with large language models (LLMs). The emphasis on ease of use and developer experience aligns well with the current needs in this field. Here are some questions for further exploration: - Which LLMs (e.g., GPT-3, Jurassic-1 Jumbo) currently does Helicone support for integration? - How does Helicone ensure the security and privacy of sensitive user data processed by LLMs? - Does Helicone offer integrations with popular LLM development frameworks or platforms? As a developer with experience in LLMs and data analysis, I'm interested in Helicone's technology. Here are some potential areas for collaboration: - Contributing to expanding Helicone's support for a wider range of LLMs. - Developing functionalities for advanced error detection and performance analysis of LLM outputs. - Collaborating on further security features and data anonymization techniques for LLM integrations. I'd love to learn more about Helicone's technical architecture and explore opportunities to contribute to its development. Additionally, I'll check out the updated Helicone features and explore the GitHub repository!

Report

12mo ago

Joseph Abraham

SaaS for Greater Good

@cole_gottdank @mwseibel @justintorre Cheers to the team behind this! Wishing you all the best on this new adventure. What are you most excited about with this launch?

Report

12mo ago

Cole Gottdank

Helicone AI

Maker

@mwseibel @justintorre @kjosephabraham Thanks, Joseph! We're thrilled to have your support. For this launch, I'm excited about sharing Helicone with the open-source community and our new Sessions feature. I believe the future of AI workflows is agentic, so having the ability to track your AI agent through its entire lifecycle is a necessity. The approach we took to building Sessions is unlike others, but it follows our ethos of focusing on a seamless developer experience. How about you? Any feature that caught your eye?

Report

12mo ago

Cole Gottdank

Helicone AI

Maker

@mwseibel @justintorre @max_savonin1 Hi Max, thank you for the kind words! To answer your questions: - We have support for all LLMs! To fully integrate with Helicone, all that is required is changing your baseURL to proxy through us! All of the docs to connect to the LLM providers are available here: https://docs.helicone.ai/getting... - Helicone is fully SOC2 compliant and we follow the best security practices. For use cases where users may have strict compliance, we offer the ability to omit request and response bodies. We also have product-grade self hosting support through our Helm chart or Docker solutions. - Yes! We have tons of integrations with platforms like LangChain, LlamaIndex, LiteLLM, DeepInfra, OpenWebUI, DifyAI and many more. All of these are available on our docs! To learn about Helicone's architecture, you can read my blog I did on it here: https://upstash.com/blog/impleme... All contributions to our open-source repository would be greatly appreciated! Thanks for your question Max!

Report

12mo ago