Neko Nik

Which IDE do you prefer?

I am developing a web scraping project (70% done) that offers users a range of configurable options. These options would encompass tasks such as JS rendering with running JS scripts, taking page screenshots, parsing data into clean markdown format, form filling, and performing clicks. With proxies and parallel scraping. The question I am contemplating is whether it would be advisable to create a platform where users can perform web scraping and parsing in a single location. Furthermore, I am considering whether to provide users with the flexibility to submit their HTML data, which would then be parsed and returned in markdown format. Additionally, the option of generating embeddings from the data could be made available. This approach would enable users to select the specific tasks they wish to perform, allowing for a more customized experience. I am seeking feedback and suggestions regarding this concept.