BY

D.VIKAS(17311A05L1)
M.RACHANA(17311A05N5)
 The advent of internet and smartphones has been good to the e-commerce
industry. With millions of customers and billions of dollars at stake, the
market has started seeing the many customers. Which in turn has led to rise
of e-commerce aggregator platforms which collect and show you the
information regarding your products from across multiple portals? For
example when planning to buy a smartphone and you would want to see the
prices at different platforms at a single place. What does it take to build such
an aggregator platform? Here’s my small take on building an e-commerce
site scraper.
 It contains the main part: First, we have to fetch all the HTML DOM elements
in the page and find the feasible parts of dom where the actual content about
the products information is placed and combines all the data together from
all other sources and displays to the user based on the product searched by
user dynamically.
 The following information needs to be extracted from the page:
 Product Name
 Product price
 Product discount
 Product image
 There are many software tools available that can be used to
customize web-scraping solutions. This software may
attempt to automatically recognize the data structure of a
page or provide a recording interface that removes the
necessity to manually write web-scraping code, or some
scripting functions that can be used to extract and transform
content, and database interfaces that can store the scraped
data in local databases. Some web scraping software can also
be used to extract data from an API directly.
SOFTWARE REQUIRENENTS

Javascript tools:

 Greasemonkey
 Node.js
 PhantomJS – scripted, headless browser used for
automating
 web page interaction.
 jQuery

Web crawling frameworks

 These can be used to build web scrapers.
 Scrapy
HARDWARE REQUIRMENTS

 512 MB of RAM(min)
 P-IV processor or higher
• The advent of internet and smartphones has been good to
the e-commerce industry. With millions of customers and
billions of dollars at stake, the market has started seeing the
many customers.
• Which in turn has led to rise of e-commerce aggregator
platforms which collect and show you the information
regarding your products from across multiple portals.
• For example when planning to buy a smartphone and you
would want to see the prices at different platforms at a
single place. What does it take to build such an aggregator
platform? Here’s my small take on building an e-commerce
site scraper.
 Web scraping is used for contact scraping, and as a component of
applications used for web indexing, web mining and data mining,
online price change monitoring and price comparison, product
review scraping (to watch the competition), gathering real estate
listings, weather data monitoring, website change detection,
research, tracking online presence and reputation, web
mashup and, web data integration.
 Web pages are built using text-based mark-up languages
(HTML and XHTML), and frequently contain a wealth of useful data in
text form.
 However, most web pages are designed for human end-users and
not for ease of automated use. Because of this, tool kits that scrape
web content were created. A web scraper is an Application
Programming Interface (API) to extract data from a web site.
Companies like Amazon AWS and Google provide web scraping
tools, services and public data available free of cost to end users.
 Web scraping can be done manually by a software user, the
term typically refers to automated processes implemented
using a bot or web crawler. It is a form of copying, in which
specific data is gathered and copied from the web, typically
into a central local database or spreadsheet, for
later retrieval or analysis.
 The content of a page may be parsed, searched, reformatted,
its data copied into a spreadsheet, and so on.
https://www.promptcloud.com/blog/customer-
spotlight-dandan-price-comparison-startup/
https://towardsdatascience.com/https-medium-
com-hiren787-patel-web-scraping-applications-
a6f370d316f4