International IAB/ABC Spiders and Bots list

Posted on: Wednesday 28 September 2016

Share this

Established in partnership with the IAB US and launched in 2006, this is a list of known robotic user agents that is updated each month and shared with subscribers. Media owners can apply the list to any analytics, ensuring all known spiders and robots to date can be recognised and separated.

Did you know that ABC UK has had to adjust claimed web traffic figures by up to 60% due to robot activity?

By using the International IAB/ABC Spiders and Bots list you are taking a key step to eliminate non-human traffic being counted in your web analytics. IAB Europe member companies can deliver further confidence to their clients in how their digital advertising budget is being spent by accessing and implementing this list to their web analytics reporting.

According to a comScore research: 51% of internet “users” and 3-10% of ad impressions reach are non-humans– i.e. robots and spider. Meaning brands advertising and communicating online could be wasting significant budget.

How to ensure these figures don’t relate to your business?

ABC UK manages access to the International IAB/ABC Spiders and Robots List for European based companies. IAB Europe is working with ABC UK to make this data available to you, this is a globally applicable list created by the merging of the IAB US and ABC UK’s data. Implementing the list will significantly reduce the risk of counting invalid non-human traffic.

By subscribing to ABC’s Spiders & Robots service you can get:

1. Access to the ABC website ( and its restricted technical area, which holds:

  • The International IAB/ABC Spiders and Robots List

  • The International IAB/ABC List of Valid Browser

  • The ABC IP Address Exclusion List

2. A “Steps to Implementation” document

  • Support in implementing and updating the list

  • Information on the management of the centrally updated lists reflecting monthly changes that are brought to the attention of IAB US, ABC UK or the Policy Board.

IAB UK members get a special discount on the subscription rate. For more information contact

Share this

Related content

Twitter reinforces immediate & intimate connection with premium content

IAB UK Digital Upfronts 2018: Twitter reinforces immediate & intimate connection with premium content

Having been a partner of the IAB Digital Upfronts since their inception, Twitter know how to host an engaging session. This year was no different with...

Learn more
The IAB & Mind Ideas Hack

The IAB & Mind Ideas Hack

Learn more

IAB UK says get involved!

IAB UK today invited the industry to roll up its collective sleeves and get involved at its inaugural non-conference, ‘Nonference’, on November 1st.

... Learn more

IAB UK Moves forward with Engage

The IAB UK has announced the date for Engage, its flagship conference, which is set to take place on Thursday 7 June. 


Learn more

Looking to boost your skills?

Lasting two hours each, our virtual training modules have been designed to fit into your day while you work from home and provide you with the opportunity to learn new skills and keep your team up-to-date with latest best practice.