No more diving into complex DOM structures or writing fragile XPath expressions. Just specify what data you are scraping from the web with natural language-like queries, and AgentQL handles the rest.

Hey Product Hunters! 👋 I am Shuhao, cofounder of the AgentQL team. Allow me to introduce AgentQL, an AI-powered semantic framework designed to enable AI agents to seamlessly interact with the web, using natural language. That's a lot of big words. Let's break down what AgentQL can do: 🕹️ Give it the command to find "price", and it will retrieve all the price data on an e-commerce page you spend less time building rigid selectors 🖥️ Take the same code to another site, and it can work just as well >> you spend less time maintaining fragile scripts 📝. Need to interact with sites as part of your work flow? Simply use natural language, tell AgentQL to click, scroll_page or fill. So why did we decide to build AgentQL? Traditional web automation struggles with identifying elements consistently through UI changes. When LLMs and agents first emerged, we envisioned a web infrastructure tailored for agentic interaction, which excels in handling these small but breaking changes, so we decided to tackle this challenge head-on. How did we go about this? At the heart of our framework is the AgentQL Query, a language crafted to describe locate web elements within a structured schema. We’ve integrated robust DOM processing with advanced prompt engineering, creating a powerful combination that dynamically generates context-aware prompts. By doing so, we overcome the fragility of static XPath or attribute selector-based scripts, enabling a more resilient and flexible web interaction infrastructure. Would you like to join our journey to an agentic future? Our early users have reported real productivity boosts with AgentQL, unlocking use cases that were previously too costly or complex to develop. Your feedback and questions will be invaluable to us. 👀 Please check out our technology at AgentQL.com 💬 Connect with our community on Discord and X.com Thank you for being a part of our story!

@shuhao_zhang Congratulations to the AgentQL team on their release! Great job - up and running in seconds🦾

@shuhao_zhang Congrats on your launch day! Wishing you great success and new opportunities. What challenges did you overcome to get here?

@shuhao_zhang AgentQL's focus on natural language for web interaction with AI agents is a fascinating concept! The ability to handle dynamic website changes and overcome the limitations of traditional selectors is valuable. Here are some questions for further exploration: - How does AgentQL ensure the accuracy of its natural language parsing and element identification? How easy is it to maintain AgentQL queries when websites undergo significant changes? - Can AgentQL integrate with existing web scraping or automation frameworks? Can it learn and improve its understanding of user intent over time? - What security measures are in place to prevent unintended automation behavior or data breaches? Potential for Collaboration: As a developer with experience in AI and web automation, I'm interested in learning more about AgentQL's technology. Here are some potential areas for collaboration: - Contributing to further development of AgentQL's NLP capabilities for more robust understanding of user intent. - Exploring ways to leverage ML models for improved element identification and dynamic adaptation to website changes. - Collaborating on building a robust security framework for secure and controlled web interaction with AgentQL. I'd love to learn more about AgentQL's technical architecture and explore opportunities to contribute to its development.

@shuhao_zhang @max_savonin1 Hi, Max! I may be able to help with some of these questions-- they're really great questions, btw! - How does AgentQL ensure the accuracy of its natural language parsing and element identification? How easy is it to maintain AgentQL queries when websites undergo significant changes? We do regular benchmarking to ensure our results are the best for accuracy and reproducibility, trading off against other dimensions. We also iterate against a set of real-world use cases to further improve AgentQL. We've found that providing context for certain terms is critical for accuracy and stabilization of results. As for maintaining AgentQL queries as websites undergo changes, we've found that we're generally able to handle most cases (even testing against some historic / archived versions of websites where available) so long as the elements exist, but obviously, if the structure changes too significantly, e.g. the elements or data no longer exists or the steps of a workflow are completely shuffled, the query wouldn't be able to adapt to this. - Can AgentQL integrate with existing web scraping or automation frameworks? Can it learn and improve its understanding of user intent over time? We currently provide an extension to the Playwright Python SDK, and are able to extend Page objects directly to run queries, interact, or provide some other utility functions. Most other frameworks that are leveraging Playwright where you still have access to the underlying Page object should be compatible. As for learning and understanding user intent, AgentQL itself is not personalized and doesn't adapt to usage, but we've provided tools like the aforementioned context to help disambiguate results. - What security measures are in place to prevent unintended automation behavior or data breaches? We currently view ourselves as a tool to enable developers, as it's simply a product which helps interpret data and provide elements for programmable interaction on web. We're curious to see the ingenuity of the community and believe our technology can be used to unlock a lot of good use-cases, but we will definitely remain vigilant and remain open to changing our position on this as we grow! Hope that helps, and definitely open to connecting to explore possible collaborations!

@antonikozelski Thanks so much! We're excited to hear that the onboarding is smooth for you. Appreciate the support!

AgentQL is probably the easiest web data extraction tool that I have ever used. While it doesn't require all the back and forth editing to choose the right web elements, it still offers the ability to use code to process the data extracted ! Give it a try and you won't regret!

Thanks for giving us a shot and the kind words! We're really excited to see positive engagement from the community!

Thank you for your support, Shawn!