What is the best web scraping open source tool?

- Quora Page 1 sur 7

What is the best web scraping open source tool?

Related Questions

Promoted by JetBrains What are the fastest open source scraping tools?

Learn how ReSharper helps .NET & web developers in Visual Studio. Is there any open source scraping tool?
Eliminate errors and code smells, refactor and navigate your code, comply with coding
standards, & more. What is web scraping and what tools are needed for
it?
Download at jetbrains.com
What is the fastest open source scraping software?

11 Answers What do I need to know to get into web scraping?

Is there any web scraping tool in which I can set my

Vlad Ionescu, Robotic Automation, Web Automation and Screen Scraping own rules?
Written Oct 31
Not open source, but free for individuals and SMEs - UiPaths Community Edition . How do I use .NET for a free web scraping tool?

The software does much more than Web Scraping - you can crawl, extract screen What are my best options for web scraping?

information , create macros and deploy software robots to automate any Windows
What is web scraping? Is C-Sharp the best language
tasks or applications. to use for this?

For resources related to web scraping please check www.scraping.pro - it has indepth How do I use dot net & the free web scraping tool?
reviews of almost all relevant solutions.

2.8k Views View Upvotes

Related Questions More Answers Below

What are the fastest open source scraping tools?

Is there any open source scraping tool?

What is web scraping and what tools are needed for it?

What is the fastest open source scraping software?

What do I need to know to get into web scraping?

Vicky Rathee, Founder for DataScraping.co - Data Scraping Studio

Written Jan 11, 2016
Give a try to Data Scraping Studio (Freeware)

Use their point and click chrome extension to create a web scraping agent by clicking on
HTML element you want to extract. It will create CSS selectors for those element and show
the extracted preview instantly and you can go on advance mode to extract
TEXT/ATTR/HTML or REGEX.

Also, you can download the on page output in CSV, TSV or JSON format.

https://www.quora.com/What-is-the-best-web-scraping-open-source-tool 04/04/2017
What is the best web scraping open source tool? - Quora Page 2 sur 7

Once done with the setup save the scraping agent to execute with Data Scraping Studio
desktop app with more advanced features like batch URL crawling or large data
extraction like 100 or million of web pages, scheduling, input from CSV etc. with amazing
capability to execute multiple web scraping job in parallel.

2.7k Views

Promoted by Diffbot
Turn websites into data.
Never write another web scraper again. Automatically extract content from any website. No rules
required.

Learn More at diffbot.com

Olle Andreasson, Fullstack creative mind going all in with javascript,

music, art and gaming.
Written May 31, 2016
Ive been working in the web scraping industry for three years now. We have been scaling
from just a small startup with very few people to a multi millon dollar buisness, its all based
on web scraping where I am responsible for the whole backend pipeline. With this
experience behind me i hope i can answed this question in a good way.

To start with, it of cource depends of what language you are using when youre scraping. At
my company were using node.js which has multiple open source projects availabe to use on
npm . Back when I stared scraping the most popular one where cheerio , but now days i
think there are tools built on top of cheerio like x-ray .

What we did at my company was to build our own tool on top of cheerio too, but unfort the
investors has decided this tool will not be open source.

Anyhow, if you want to look into how to use cheerio on different sites, i use it for my side
project get some internet . Where i scrape over 20 different sites every day and present the
content in a nice UI where you can search and see whats trending.

The code for the scraping is using cheerio it can be found here .

Good luck!

1.8k Views View Upvotes

Katie Lundsgaard, Success at Kimono Labs

Written Jan 31, 2015
You should check out Kimono for web scraping (Turn websites into structured APIs from
your browser in seconds ). While its not open source, it is completely free. Kimono makes
a bookmarklet or a Chrome extension that lets you scrape almost any page. Its really easy to
use, and you can update your data whenever you tell it to. You can get your output in JSON,
CSV, and RSS formats, and Kimono will even host your data for you.

https://www.quora.com/What-is-the-best-web-scraping-open-source-tool 04/04/2017
What is the best web scraping open source tool? - Quora Page 3 sur 7

We also just released a new modify results feature (Write JavaScript functions to transform
your API results ) which will let you operate on your data with javascript! This is great if
you need to clean or modify your data before it comes back to you.

I should admit that I work for Kimono, but the product is really awesome.

5.6k Views View Upvotes

Paul Black, I use web scraping tools a lot.

Written Apr 8, 2016
There're plenty of web scraping tools now. I used other tools before but it's not free. I have
free version but there's limited functions. Then I switch to Octoparse and it has all functions
that all the other tools has. And it's free, so whatever.

Here's the link to download it. Octoparse free download and version history I hope it will
be useful for you. I used it to scrape information from Amazon and eBay. And it worked
pretty awesome.

Now, Octoparse has been used by hundreds of thousands of individuals and companies in
the world. They come from various fields, ranging from News industry, E-commerce, Real
Estate, Finance, and so on. The use o f Octoparse is essentially a form of data mining. Items
like market pricing, auction details, contact information, weather reports, or any other list
of data can all be sought in Octoparses efforts.

I'm one of its biggest customers and get lots of discount of their data services. I think the
logo looks pretty cute. Octoparse is capable of extracting any visible data from websites
effortlessly. You dont even need to know any coding knowledge.

How to Extract Information from Yellow Page Websites

How to Extract Data from eBay

How to Extract Tweets from Twitter

How to Extract Data from Facebook

BTW, I think the logo looks pretty cute.

https://www.quora.com/What-is-the-best-web-scraping-open-source-tool 04/04/2017
What is the best web scraping open source tool? - Quora Page 4 sur 7

And here's some pictures of the tool. The result looks good.

4.1k Views View Upvotes

Arijit De, Web Applications, Rails3, HTML5, RubyGems, Gadget Freak,

Written Aug 28, 2012
For Node.js framework, Zombie and CasperJS would be good choice. Zombie is based on
PhantomJS (headless scraping browser kit - very fast). An outdated but still can be used
would be pjscrape javascript library.
In the ruby world, try nokogiri and mechanize, they will serve your purpose.

3.8k Views View Upvotes

Jett Gamboa, CTO at Incuventure

https://www.quora.com/What-is-the-best-web-scraping-open-source-tool 04/04/2017
What is the best web scraping open source tool? - Quora Page 5 sur 7

Written Mar 1, 2013

For those who use Python, I find webscraping (https://code.google.com/p/webscr... ) easy
to use.

A feature I like about it is its ability to cache recently "accessed" urls so you don't hit the
server all the time. It also has a facility to simulate site navigation (e.g. button or link clicks)

6.4k Views View Upvotes

Straw Bot, former internet marketer

Updated Mar 21
If all these tools are too complicated for you and all you want is the information not the
hassle.

I suggest you out source it to an expert.

Check out this website

http://filecat.site/web-scraping/

353 Views

Robert Woges, Business Analyst

Written Jun 20, 2015
There were many tools used before now. But those tools weren't as effective as in these days.
There are many new things and updates have been done in recent times. So We should
change our trends and ideas often that will make us standard as ever. Here i could get 5 best
tools 5 Excellent Open Source Tools for Web Developers These tools will definitely help
you. Do check that out.

2.1k Views View Upvotes

Duraid Shaihob, Head of Growth @Marketizator

Written Mar 12, 2015
import.io is a great tool for web scraping and data extraction. It's not open source as far as
I know but it's completely free.

You can use it in your web browser or download the desktop app. And it can even export
your data to Google Sheets or Plot.ly to create awesome visualizations like this one:

2.3k Views View Upvotes

Ati Jain
Written Aug 11, 2015
Hi,
Here I suggesting you the best tool and best company for scraping all the details available
on any website called as ''NETUCON". They developed there own tool Please have a look.

Netucon company based at Ahmedabad (India) provides the ultimate solution to its
customers and software development services with innovation and creativity. Founded by a
highly experienced team of IT, and management professionals, Netucon understands the
various requirements of client for technology and business, thus, provides the solutions to
our valued clients, both in India as well as in abroad.

They are the team of IT professionals having more than 12+ years of development
experience with 91% of Success ratio .

They developed this LinkedIn connection Creator this LCC is useful for scraping CEOs:
Seeking connections to different CEOs, Creating B2B contacts, Creating B2C contacts, Lead

https://www.quora.com/What-is-the-best-web-scraping-open-source-tool 04/04/2017
What is the best web scraping open source tool? - Quora Page 6 sur 7

Generators, Digital Marketers, Bloggers who post their blogs on LinkedIn and so on..

For more details have a look

LinkedIn Connection Creator(LCC).docx

They are also expert in providing services like :

1. .net development projects
(Microsoft .NET Framework 1.1/2.0/3.0/3.5/4.0/4.5)
2. E commerce Integration
(Yelp, Just dial,Amazon Integration ,E bay Integration, Shopify Integration, Volusion
Integration)
3. Web Data Scraping
(Yelp scraping, Amazon scraping,LinkedIn, Government websites, Social networking sites
scraping and so on)
4. Quick Book Integration
5. Accounting Software Integration
6. Custom Website Development
7. ERP Development
8. Data entry
9. Data mining
10. Lead Generation on LinkedIn, Twitter and Face book.
11. BPO:- Data Processing
12. Digital Marketing and so on

You can read more about Netucon here: Home

They also have developers you can hire to do the job for you; their Skype is mayankm_wg

1.7k Views

Related Questions

Is there any web scraping tool in which I can set my own rules?

How do I use .NET for a free web scraping tool?

What are my best options for web scraping?

What is web scraping? Is C-Sharp the best language to use for this?

How do I use dot net & the free web scraping tool?

What is currently the best technology stack for web scraping?

Can you recommend free scraping tool for Web 2.0?

I found a good software tool for a web scrape. Which is the best software tool available in
the market?

I want the open source web scraping tool. What are some suggestions on this?

Which tool do you use for web scraping?

What are some of the best web scraping tools in Java?

Are there any free open source scraping tools that scrape from others sites?

What should I look for when choosing a web scraping tool?

What is web scraping and is Python the best language to use for this?

What are some great sources to learn web scraping?

https://www.quora.com/What-is-the-best-web-scraping-open-source-tool 04/04/2017
What is the best web scraping open source tool? - Quora Page 7 sur 7

https://www.quora.com/What-is-the-best-web-scraping-open-source-tool 04/04/2017