Comparison shopping website

A comparison shopping website, sometimes called a price comparison website, comparison shopping agent, shopbot or comparison shopping engine, is a vertical search engine that shoppers use to filter and compare products based on price, features, and other criteria. Most comparison shopping sites aggregate product listings from many different retailers but do not directly sell products themselves. In the United Kingdom, these services made between £780m and £950m in revenue in 2005^[1].

History

The first widely recognized comparison-shopping agent was BargainFinder developed by then Andersen Consulting (now Accenture) and its SmartStore center in 1995 for an experiment. The first commercial shopping agent, called Jango, was produced by Netbot, a Seattle startup company founded by University of Washington professors Oren Etzioni and Daniel S. Weld; Netbot was acquired by the Excite portal in late 1997. Junglee, a Bay-area startup, also pioneered comparison shopping technology and was soon acquired by Amazon.com. Other early comparison shopping agents included pricewatch.com and killerapp.com. Most of them were price comparison for computer related products and hence did not attract much public attention.

Around 2006, the price comparison websites found their way to emerging markets including finder.com.au^[2][3] in (Australia) and finder.com ^[4][5] in the (United States), co-founded by Fred Schebesta^[6] and Frank Restuccia. While in 2010 South-East Asia became a burgeoning region for many new comparison websites starting with CompareXpress in Singapore in 2010, and in the following years companies including Baoxian (China) and AskHanuman (Thailand) followed.^[7] The market is expected to grow a lot in the upcoming years as ecommerce is becoming more familiar in this region. The Middle East is also seeing its fair share of a burgeoning e-commerce ecosystem, with players such as Pricena.com present in multiple countries.

As of 2013, the market for more data-driven price comparison sites was growing, as several venture capital firms made large investments in price comparison sites with big-data oriented platforms, including FindTheBest, Askhanuman, Priceza and the Singaporean price comparison startup Save 22.^{[8][9][10][11]}

Comparison shopping agent

In the early development stage from 1995 to 2000, comparison shopping agents included not only price comparison but also rating and review services for online vendors and products. For example, services like Bizrate.com provided ratings for online vendors. Today, websites like Epinion.com provide review and rating services for products. Altogether, there were three broad categories of comparison shopping services.^[12]

Later, through mergers and acquisitions, many services were consolidated. As a result, shopping.com, PriceGrabber and shopzilla have become the top three comparison shopping agents since 2000.

Comparison shopping agents may be considered early examples of the Semantic Web, but early systems used wrappers to extract structured information about products from Web pages. Wrapper construction requires extensive programming and results in a fragile system, since they must be reprogrammed when an online store changes its layout. Modern comparison shopping systems get most of their data from relational data feeds generated by retailers. While this typically yields more robust results, it requires retailer cooperation and may produce less comprehensive listings.

Shopping

In the late 1990s, as more people gained access to the internet, a range of shopping portals were built that listed retailers for specific product genres. Retailers listed paid the website a fixed fee for appearing. These were little more than an online version of the Yellow Pages. As technology has improved, a newer "breed" of shopping Web portals is being created that are changing both the business model and the features and functionality offered. These sites do not "aggregate" data-feeds provided from the retailers, they search and retrieve the data directly from each retailer site. That allows for a much more comprehensive list of retailers and the ability to update the data in real time.

Generic portals and search engines launched similar services and companies that stood to benefit from increased internet shopping (especially credit card and delivery firms) launched similar sites.

Services

Through 1998 and 1999, various firms developed technology that searched retailers websites for prices and stored them in a central database. Users could then search for a product, and see a list of retailers and prices for that product. Advertisers did not pay to be listed, but paid for every click on a price. Streetprices, founded in 1997, has been a very early company in this space; it invented price graphs and email alerts in 1998.^[13] These useful services let users see the high and low price of any product graphed over time, and request email alerts when a product's price drops to the price the user wants.

Technology

Price comparison sites can collect data directly from merchants. Retailers who want to list their products on the website then supply their own lists of products and prices, and these are matched against the original database. This is done by a mixture of information extraction, fuzzy logic and human labour.

Comparison sites can also collect data through a data feed file. Merchants provide information electronically in a set format. This data is then imported by the comparison website. Some third party businesses are providing consolidation of data feeds so that comparison sites do not have to import from many different merchants. Affiliate networks such as LinkShare, Commission Junction or TradeDoubler aggregate data feeds from many merchants and provide them to the price comparison sites. This enables price comparison sites to monetize the products contained in the feeds by earning commissions on click through traffic. Other price comparison sites like PriceGrabber have deals with merchants and aggregate feeds using their own technology.

In recent years, many off the shelf software solutions^[14] have been developed that allow website owners to take price comparison websites' inventory data to place retailer prices (context adverts) on their blog or content only website. In return the content website owners receive a small share of the revenue earned by the price comparison website. This is often referred to as the revenue share^[15] business model.

Another approach is to crawl the web for prices. This means the comparison service scans retail web pages to retrieve the prices, instead of relying on the retailers to supply them. This method is also sometimes called 'scraping' information. Some, mostly smaller, independent sites solely use this method, to get prices directly from the websites that it is using for the comparison.

Yet another approach is to collect data is through crowdsourcing. This lets the price comparison engine collect data from almost any source without the complexities of building a crawler or the logistics of setting up data feeds at the expense of lower coverage comprehensiveness. Sites that use this method rely on visitors contributing pricing data. Unlike discussion forums, which also collect visitor input, price comparison sites that use this method combine data with related inputs and add it to the main database though collaborative filtering, artificial intelligence, or human labor. Data contributors may be rewarded for the effort through prizes, cash, or other social incentives. Wishabi, a Canadian-based price comparison site, is one example that employs this technique in addition to the others mentioned.

However, some combination of these two approaches is most frequently used. Some search engines are starting to blend information from standard feeds with information from sites where product stock-keeping units (SKUs) are unavailable.

Similar to search engine technology, price comparison sites are now spawning "comparison site optimisation" specialists, who attempt to increase prominence on the comparison sites by optimising titles, prices and content. However, this does not always have the same effect, due to the differing business models in price comparison.

Functionality and performance

Comparison shopping websites implement algorithms for shopping search comparison. Shopping search comparison (SSC) is composed of two different technologies: page-wise search and site-wise search.

In page-wise search a phrase, such as a product name, is searched over an index of pages. When the phrase is found, the URLs of the pages in which the phrase was found are returned to the user in the user’s browser along with pictures of the products found.

In site-wise search, several product names are searched not over an index of pages, but over an index of sites. To perform a site-wise search the SSC engine must search all pages in every site in its index and return the sites that have pages where one of the several product names occur. Site-wise search is more computer-intensive because multiple products are searched over multiple pages on multiple sites. The result, although costly in terms of computing power, is that a list of products may be searched and found at a single website – for example at an online merchant.

Empirical projects that assessed the functionality and performance of page-wise SSC engines (AKA bots) exist. These studies demonstrate that no best or parsimonious shopping bot exists with respect to price advantage.^[16][17]

Price data collection

Some price comparisons sites use web scraping technology or robots to extract price from the online stores to display in price comparison table, while others use an affiliation API call to display price comparison of products.

Mobile

Mobile comparison shopping is a growing subset of comparison sites/applications. Due to the nuances of mobile application development, different product strategies have been pursued. SMS-based products allow users to find product prices using SMS-based interaction (example: TextBuyIt by Amazon), mobile web applications let users browse mobile optimized websites (Example: Google Product Search Mobile). At the heavier end, native client applications installed on the device offer features such as bar code scanning (Example: Barnes & Noble iPhone app).^[19]

Pricing History

In addition to comparison between stores, some sites also provide price history information. This information can be used to determine, e.g., that certain products go on sale regularly, or that certain stores never discount specific product categories. Seeing this information across multiple stores can facilitate price matching or price protection.

Business models

Price comparison sites typically do not charge users anything to use the site. Instead, they are monetized through payments from retailers who are listed on the site. Depending on the particular business model of the comparison shopping site, retailers either pay a flat fee to be included on the site, pay a fee each time a user clicks through to the retailer web site, or pay every time a user completes a specified action—for example, when they buy something or register with their e-mail address. Comparison shopping sites obtain large product data feeds covering many different retailers from affiliate networks such as LinkShare and Commission Junction. There are also companies that specialize in data feed consolidation for the purpose of price comparison and that charge users for accessing this data. When products from these feeds are displayed on their sites they earn money each time a visitor clicks through to the merchant's site and buys something. Search results may be sorted by the amount of payment received from the merchants listed on the website.^[20] large price comparison sites.^[21]

Google Panda and price comparison

Like most websites, price comparison websites partly rely on search engines for visitors. The general nature of Shopping focused price comparison websites is that, since their content is provided by retail stores, content on price comparison websites is unlikely to be absolutely unique. The table style layout of a comparison website could be considered by Google as "Autogenerated Content and Roundup/Comparison Type of Pages".^[22] As of the Google Panda, Google seems to have started considering these Roundup/Comparison type of pages low quality.^[23]

Niche players

Due to large affiliate network providers providing easily accessible information on large amounts of similar products from multiple vendors, in recent years small price comparison sites have been able to use technology that was previously only available to large price comparison sites.^[21]

The rise in popularity of video games expanded the price comparison market beyond physical goods. Niche websites for comparing the prices of virtual goods in video games and digital downloads have emerged in recent years.. Development of the niche market continued as the comparison market grew in size throughout the mid 2010's. By 2015, the niche market had expanded to cover a variety of goods and services, including the likes of specialist cycling equipment, craft beer and legal services.

In addition to comparing tangible goods, service price comparison sites expanded. This include insurance, credit card, phone bill, and money transfer comparison sites. Examples are MoneySuperMarket.com, Money.co.uk and TheMoneyCloud.com.

See also

• Discovery shopping
• Gasoline price website

References

• Doorenbos, B; Etzioni, O; Weld, D. A scalable comparison-shopping agent for the World-Wide Web, AGENTS '97 Proceedings of the first international conference on Autonomous agents CM New York, NY, USA 1997
• Clark, D. Shopbots become agents for business change, IEEE Computer, Vol. 33, Issue 2, 2000