Ad blocking

Ad filtering or ad blocking is removing or altering advertising content in a webpage. Advertising can exist in a variety of forms including pictures, animations, embedded audio and video, text, or pop-up windows and can employ autoplay of audio and video. It is a known problem with most web browsers, including Firefox, that restoring sessions often plays multiple embedded ads at once.^[1] All browsers offer some solution to the problem, either by targeting technologies (Adobe Flash/Shockwave, Windows Media Audio files, etc.) that are used to deliver ads, targeting URLs that are the source of ads, or targeting behavioural characteristic of ads (such as the use of HTML5 autoplay of both audio and video).

Benefits

To users, the benefits of ad blocking include quicker loading and cleaner looking Web pages free from advertisements, lower resource waste (bandwidth, CPU, memory, etc.), and privacy benefits gained through the exclusion of the tracking and profiling systems of ad delivery platforms. Blocking ads can also save substantial amounts of energy.^[2][3]

Another important aspect is improving security; according to some research, online advertising subjects users to a higher risk of infecting their devices than surfing porn sites.^[4] In a high-profile case malware was distributed through advertisements provided to YouTube by a malicious customer of Google's Doubleclick.^[5][6]

Users who pay for total transferred bandwidth ("capped" or pay-for-usage connections) including most mobile users worldwide, have a direct financial benefit from filtering an ad before it is loaded. Streaming audio and video, even if they are not presented to the user interface, can rapidly consume gigabytes of transfer especially on a faster 4G connection. In Canada, where users without a data plan often pay C$0.50/megabyte ($500/gigabyte) for at least the first 50-100MB exceeding their data allowance, the cost of tolerating ads can be quite high. Even fixed connections are often subject to usage limits, especially the faster connections (100Mbit/s and up) which can quickly saturate a network if filled by streaming media. "The extent of unlimited bandwidth plans is often grossly over-estimated by US and European users and advertisers. This problem affects other countries, especially those with bandwidth limitations on their global Internet connections, or those that have poor regulatory or effective monopoly providers."

To advertisers, the benefits include not angering or annoying users into blocking, defaming or boycotting their products or websites. Though few advertisers actually "intend" to anger end users, the prevalence of invasive advertising (tracking cookies, malware, etc.), bandwidth consumption, bloated search engine results and the end user usually paying for the bandwidth advertisements use, advertisers almost unanimously anger their end users. Very sophisticated filtering and anti-spam techniques can involve active defenses which can shut down an advertiser's domains or brokers, ban them from searches or target them for other countermeasures. Some countries have even considered banning the use of certain ports, e.g. South Korea's proposed ban on port 25 used by SMTP.^[7] Future countermeasures would be likely to include bans on ads South Koreans are unlikely to want or even ad brokering services. Ad substituting is also a legal and common practice already, for instance in Canadian cable TV where regulations permit showing a Canadian channel with Canadian ads instead of a US channel with US ads, where both are broadcasting the show simultaneously - this practice has spread to the web with some cable Internet providers uniformly substituting foreign ads for local ones, for which they receive a share of the revenue. Avoiding national, provider or technological interference with their ads is a priority for advertisers and especially brokers of advertising, to whom it could be fatal.

Methods

One method of filtering is simply to block (or prevent autoplay of) Flash animation or image loading or Windows audio and video files. This can be done in most browsers easily and also improves security and privacy. This crude technological method is refined by numerous browser extensions. Every internet browser handles this task differently, but, in general, one alters the options, preferences or application extensions to filter specific media types. An additional add-on is usually required to differentiate between ads and non-ads using the same technology, or between wanted and unwanted ads or behaviors.

The more advanced filters allow fine-grained control of advertisements through features such as blacklists, whitelists, and regular expression filters. Certain security features also have the effect of disabling some ads. Some antivirus software can act as an ad blocker.

Filtering by intermediaries such as providers or national governments is increasingly common. See below especially re provider ad substitution and national root DNS.

Browser integration

Modern web browsers block unsolicited pop-up ads automatically. Current versions of Konqueror^[8] and Internet Explorer^[9] also include content filtering support.

Content filtering can be added to Firefox, Chromium-based browsers, Opera, Safari and other browsers with extensions such as AdBlock, Adblock Plus and uBlock, and a number of sources provide regularly updated filter lists. Adblock Plus is included on Maxthon by default.^[10]

Another method for filtering advertisements uses CSS rules to hide specific HTML and XHTML elements.

External programs

A number of external applications offer ad filtering as a primary or additional feature. A traditional solution is to customize an HTTP proxy (or web proxy) to filter content. These programs work by caching and filtering content before it is displayed in a user's browser. This provides an opportunity to remove not only ads but also content which may be offensive, inappropriate, or simply junk. Popular proxy software which blocks content effectively include Netnanny, Privoxy, Squid, and some content-control software. The main advantage of the method is freedom from implementation limitations (browser, working techniques) and centralization of control (the proxy can be used by many users).

Proxies are very good at filtering but have several limitations compared to browser based solutions:

• difficulty to filter https/SSL traffic
• full webpage context is not available to the filter
• difficulty to filter JavaScript-generated content

Hosts file and DNS manipulation

The chip receiver (or "bit bucket") from a UNIVAC key punch.

Most operating systems, even those which are aware of the Domain Name System (DNS), still offer backwards compatibility with a locally-administered list of foreign hosts. This configuration, for historical reasons, is stored in a flat text file that by default contains very few hostnames and their associated IP addresses. Editing this hosts file is simple and effective because most DNS clients will read the local hosts file before querying a remote DNS server. Storing blackhole entries in the hosts file prevents the browser from accessing an ad server by manipulating the name resolution of the ad server to a local or nonexistent IP address (127.0.0.1 or 0.0.0.0 are typically used for IPv4 addresses). While simple to implement, these methods are also very easy to circumvent. One method to circumvent this form of ad filtering is to load ads from servers with hard coded IP addresses, thus skipping name resolution altogether. Another method to evade this form of filtering is to load the advertisements from a server which also serves the main content; blocking name resolution of this server would also block the useful content of the site.

Further information: hosts file.

Using a DNS sinkhole by manipulating the hosts file exploits the fact that most operating systems store a file with IP address, domain name pairs which is consulted by most browsers before using a DNS server to look up a domain name. By assigning the loopback address to each known ad server, the user directs traffic intended to reach each ad server to the local machine or to a virtual blackhole of /dev/null or bit bucket.

DNS cache

This method operates by filtering and changing records of a DNS cache. On most operating systems the domain name resolution always goes via DNS cache. By changing records within the cache or preventing records from entering the cache, programs are allowed or prevented from accessing domain names. The external programs monitor internal DNS cache and import DNS records from a file.^[11] As a part of the domain name resolution process, a DNS cache lookup is performed before contacting a DNS server. Thus its records take precedence over DNS server queries. Unlike the method of modifying a Hosts file, this method is more flexible as it uses more comprehensive data available from DNS cache records.

DNS filtering

Advertising can be blocked by using a DNS server which is configured to block access to domains or hostnames which are known to serve ads by spoofing the address.^[12] Users can choose to use an already modified DNS server or set up a dedicated device such as a Raspberry Pi themselves. ^[13]

Manipulating DNS is a widely employed method to manipulate what the end user sees from the internet but can also be deployed locally for personal purposes.

China runs its own root DNS and the EU has considered the same. Google has required their Google Public DNS be used for some applications on its Android devices. Accordingly, DNS addresses / domains used for advertising may be extremely vulnerable to a broad form of ad substitution whereby a domain that serves ads is entirely swapped out with one serving more local ads to some subset of users. This is especially likely in countries, notably Russia, India and China, where advertisers often refuse to pay for clicks or page views. DNS-level blocking of domains for non-commercial reasons is already common in China.^[14]

Hardware Devices

Devices such as AdTrap^[15] use their own hardware which one connects to their network to block internet advertising.

By external parties and internet providers

Internet providers, especially mobile operators, frequently offer proxies designed to reduce network traffic. Even when not targeted specifically at ad-filtering, these proxy-based arrangements will block many types of advertisements that are too large or bandwidth-consuming, or that are otherwise deemed unsuited for the specific internet connection or target device.

Many internet operators block some form of advertisement and inject their own.

Limitations

As with most content filters, blocking advertisements presents the possibility that desirable content will be blocked in error, called a false positive.

Economic consequences for online business

One consequence of widespread ad blocking is decreased revenue to a website sustained by advertisements,^[16] where this blocking can be detected.

A number of website operators, such as Reddit, who use online advertisements to fund the hosting of their websites, argue that the use of ad-blocking software risks cutting off their revenue stream. While some websites have successfully implemented subscription and membership based systems for revenue, the majority of websites today rely on online advertising to function.

Advertiser offensive countermeasures and justifications

Some websites have taken counter-measures against ad-blocking software, such as attempting to detect the presence of ad blockers and informing users of their views, or outright preventing users from accessing the content unless they disable the ad-blocking software. There have been several arguments supporting^[17] and opposing^[18] the assertion that blocking ads is wrong.^[19]

Advertiser Network Blocking

The PIPCU maintains an 'Infringing Website List' (IWL), a portal for digital advertisers to be informed of sites containing infringing content with the intention that they cease advertising on them.^[20] Sites are identified as infringing by rights holders for the and the list is not made available to the public.^[21] As of August 12, a freedom of information request from TorrentFreak revealed:^[22] 74 domains are subject to the advertiser blocking programme, of which of October 2014 only 2 domains^[23] had ever been removed from the list. 83 advertising companies with a UK presence are currently participating.

Working with the media and advertiser industry body, the Audit Bureau of Circulations (ABC) created a technology portal called 'Project Sunblock'.^[24] If the PIPCU do not receive a response from the website operators, the host or registrar^[25] of an allegedly infringing site, the site is added to the IWL via the Sunblock portal, which is then passed along to participating advertising networks for blacklisting. From June 2014 this technology allowed replacing the adverts of websites believed to be offering pirate content with warnings from the PIPCU.^[26]

Piracy warning replacing adverts

Parties involved in project sunblock^[24]

A freedom of information request requesting the contents of the list and whether it contains technology from the controversial^[27] advertising company Phorm was refused under section 30:^[28]

This is an ongoing investigation and disclosure to the public domain would raise the profile of those sites unlawfully providing copyright material. This would enable individuals to visit the sites highlighted and unlawfully download copyright material and increase the scale of the loss. In the case of advertisers, public identification would increase the risk of harm to them by way of cyber attack or other means.

—City of London Police

See also

References