Hello Product Hunters,
I started DataScraping.co around a year back by launching the desktop app "Data Scraping Studio" on product hunt (https://www.producthunt.com/tech...) as a point and click CSS selector scraping app with amazing capability to run multiple agents simultaneously and over 5000 users downloaded the application and uses now in their business.
Over the time we noticed, one of the major ask from community was the online solution and REST API. To integrate and use the scraped data in existing system without installing the desktop app or to use across operating systems. And we keep recommending our user to try (YC Backed) Kimonolabs, but then suddenly they shutdown their platform on March, 16 leaving 100k+ users leaving behind on short notice.
Today, I'm very excited to launch the online web scraper with the same capabilities and power of cloud along with REST API to get the same Kimono(or even better) experience back. Go to our website and signup to get 1000 pages credit free every month to test the features and you can go PRO anytime if you like it!
I’ll be available all the day to answer any comments, feedback you have, really looking forward to it!.
@vikashrathee IMO scheduled, cloud scrapers are the way to go! Was a BIG fan of Kimono, but what did it for me with them was the ability to get an RSS feed from my extraction and a super reliable Google Sheets Add-on that would update when the scraper would run. Still can't find anything similar! Do you/ will you implement something similar? 😁
@sandrojazzar Hey Sandro, thank you for the comment. Yes we do have RSS feed in plan and will be adding that in coming weeks. Google spreadsheet is not the priority right now but yes we will see this if more user requests the same.
For now there is an alternative using the Zapier can post the data to Google Spreadsheet from webhook (we do support webhook). You can give a try. https://zapier.com/zapbook/zaps/...
@marcelp81@sandrojazzar
No they don't have that to automate I guess. What they explained is manual import each time in this article.
https://support.scrapinghub.com/...
So better would be to use Zapier to post the data automatically in spreadsheet each time the scraping job completes using any tool of your choice.
@nick88msn
- Save money : Import.io costs $299 monthly for 50k pages while datascraping.co charges $130 only.
- Unlimited agents : Import.io support only 30 runs storage, while in datascraping.co you can create any number of agents and upto 30 days historical data for any number of jobs run during 30 days.
- Sub accounts : Create sub account for your team to share the same account with role based security.
- Crawl with Limit : Advance limit option to crawl data in parts instead all in one go without modifying the input lists.
- Modify data using JavaScript - Currently available for desktop app, will be added for hosted app soon (https://www.datascraping.co/doc/...)
- Dynamic input : Extract many website using a single agent. (https://www.datascraping.co/doc/...)
- Password protect site crawling : Easy form submission to crawl password protected websites (https://www.datascraping.co/doc/...)
And lots more...
unfortunately it doesn't work for me. I've tried to use it on https://web.archive.org/web/2016... to extract data from a lost site. data is inside a div that appears on hover over images.
my agent is 20fce2dc52
@hadifarnoud Hi Hadi, I just copied your agent in admin account to test and found that, the data was extracted but was not visible in table due to space and line breaks. I just enabled the clean, trim function in your account and you are good to re-run your agent now.
Here is the screenshot I tested this in my account. And your question is posted here to help others to understand what went wrong. (https://www.datascraping.co/doc/...)
idm.in
idm.in
idm.in
idm.in
Retime
idm.in
Retime
idm.in