Last Updated on January 19, 2024 by theadmin

Are Rotating Residential or Datacenter Proxies Enough ?

In an ideal world, we’d all have our own dedicated network of super fast, highly secure proxies dedicated for our use only.  In reality though this is extremely expensive and unless you’ve got a pretty huge budget then it’s pretty much unrealistic for most of us.   The majority of us have to compromise somehow, perhaps on proxy type, speed or where the proxies are hosted – even with these limitations you can still spend serious amounts on getting even a half decent proxy infrastructure.  In this article we’ll explain some of the important points to consider when choosing any sort of proxy service to use.  Hopefully it will help you decide between the myriad of options and whether you’re better with a rotating residential proxy or sticking to datacenter proxies.

The major problem that faces most people is scale, proxies are normally needed to support automated tools and these can ‘burn’ through servers very quickly indeed.  One of the classic mistakes that newcomers tend to make is to overuse their proxies by not configuring their software properly.  This can get very costly when your expensive residential proxy servers get their IPs blacklisted one by one.   Although it’s not complicated, people often lose sight of the basics – that is the red flags that sites look for in detecting the IP address of a proxy server.

Here’s some of the important ones:

• Large number of queries coming from suspicious location
• Large number of queries coming in too quickly.
• Large number of queries from same web browsers
• Large number of duplicate/suspicious queries.

That is not a definitive list, but it’s the sort of things that many websites look out for when checking for proxies.   When you think about it, they’re common sense especially if you consider the queries from the other side of the coin.   First of all remember that especially in commercial setups, the proxy is there to make you look like a legitimate customer with each connection.  If you’re trying to stock up on the latest Jordan’s to fill your eBay inventory then routing your internet connection through a proxy server based in Lagos, Nigeria is not a wise move.  Firstly the country has a poor reputation for online fraud and secondly an E-commerce site set up to service the US market is not going to be interested in Nigerian based traffic.

Also remember you need many addresses for most task, apart from very simple web scraping or data extraction purposes.  This is why using a rotating proxy is pretty much essential now, unless you want to spend literally hours switching the addresses manually.   A decent rotating proxy should do this for you automatically, and as long as the proxy pool contains the right IP addresses then you will be fine.

It’s important to stress this point though – it doesn’t matter how premium the proxy network is or how many are assigned per proxy.  If the IP rotation is being filled with addresses from the wrong location you’re going to be wasting your time.  A few decent addresses is way better than a million rotating blacklisted ones with unlimited bandwidth.  The example of Nigeria is perhaps an obvious one, but you can face the same problem by bombarding a US server with proxies rotating through United Kingdom IP addresses too.  It’s vital to pick the right sort of addresses for the task required.  For instance, residential IPs are expensive, so they’re rarely essential for simple web scraping tasks.   If you do want to look like a home user in real time then make sure your location is correct, some rotating residential proxies will sometimes even switch between countries.

It’s not always important, but if you want addresses that are less likely to be geo-targeted specifically then mobile proxies will be worth considering.  Again though IP addresses even from a mobile network are going to look suspicious if they’re all from a completely different country from your target.   Many mobile proxies are full of ranges from obscure providers – paying big money for a rotating proxy network based on a Russian mobile phone network is useless for posting Craigslist ads.  Location coverage is key in many tasks where as you can be a bit more flexible in other areas like social media.

Useful Checklist for Choosing a Decent Proxy Provider

Rotating Proxy FAQ
What's the Best Proxy to Use?	If possible use private residential proxies that are specifically designed for your task e.g. Instagram proxies, Twitter proxies.
What is Proxy Scraping ?	Using proxies to access popular sites repeatedly without being blocked or results tapered to your real IP address.
What is a Proxy Pool?	These are the advanced proxies which instead of using static IP addresses rotate through a pool of different IP addresses.
What are Residential Proxies?	Proxies which have residential IP addresses registered for home users. Normally only allocated to ISPs.
Do I need a VPN ?	Main difference between a proxy and VPN is the VPN encrypts all your communication which means activity can't be logged and is more secure.

Indeed most commercial servers are already programmed to detect high risk countries and flag them for potential fraud. It’s not entirely fair but e-commerce sites have learnt to their cost where most fraudulent transactions come from and often simply block them.  Indeed many entrepreneurs resell in these countries at hugely inflated prices simply because items are so hard to obtain legitimately.   If you start using addresses which suffer IP bans due to location you’re at best wasting your money, at worse losing valuable accounts – it will be no difference than using the free proxy lists.   Any number of these addresses in your proxy rotation can cause issues whether they’re rotated automatically, manually or via an API call. The technology behind those dedicated proxies is important but not as much as using the right rotating IP addresses for the task.

So How Do I Choose the Best Rotating Proxy Service ?

The most basic answer is of course, simply by trial and error.  The best proxies depend largely on the purpose you’re using them for – a network of proxies may be fabulous for Ticketmaster but may be completely blocked out by Google or Instagram. You might be lucky and find extremely cheap, generic proxies that work brilliantly for you – although this is more likely for scraping and research tools than with commercially focused software. In most circumstances though you’ll find the best proxy providers will specifically group and manage their proxy resources for specific purposes. If a block of IP addresses is constantly used to access a specific website by hundreds of paying customers then they’ll quickly get blacklisted. However if the proxies are carefully monitored and rotated properly they can often be used indefinitely for a specific purpose. Much will depend on the professionalism of your proxy service, the IPs, the proxy API and how you use all these elements.

This is why you’ll see the best proxy networks will provide ‘sneaker proxies’ or ‘instagram proxies’ and so on. Large blocks of undefined proxies used for any purpose and not rotated properly soon become useless for any sites.  If you have a very specific need for these servers it’s worth checking this before you buy rotating proxies from anyone.  The very best proxy service providers will steer you towards something suitable (per use cases) depending on your scale and specific task.   Indeed Luminati actually have a feature in their proxy manager interface walled ‘the waterfall’ which will start tasks with cheap data center proxies and then use a proxy rotator to select other addresses if their blocked.  This is designed to provide the cheapest option and pricing plans that work so you don’t end up paying for static residential IPs every request to service simple web scraping needs.  It’s an impressive feature for proxy services but still leaves the risk of getting accounts flagged as you test out the cheaper IP options.  They also charge on bandwidth used so you’ll need to configure the settings correctly to ensure that all these concurrent connections don’t end up actually costing more.

There’s no better alternative in my opinion to simply speaking to an experienced, honest proxy provider who will guide you to the right proxy services for your geo targeting and IP pool needs. There are a few proxy providers who really know what they’re doing and make a real effort to protect their resources. After all although anyone can create simple proxies based in datacentres very easily, residential proxies are much more difficult to obtain and cost lots of money to replace. These are proxies with residential IP addresses which are much less likely to be flagged or blocked by ordinary websites.

Beware Cheap Rotating Proxies with Residential Addresses

Indeed any company with these sorts of resources will be very careful to configure the other aspects which are likely to raise red flags. For example creating unique user agents and properly relaying the native referrer source from each individual IP address.

These companies have also discovered that the best way to protect their proxy resources is to ensure that they control the rotation of the servers and addresses too. Although most automated software allows you to switch IP addresses automatically there’s no guarantee that this will be configured correctly. By setting up backconnect rotating proxies, the provider can control how often each IP address is used. You can read more about backconnect rotating proxies on this site too.

There is another question that you may want to ask yourselves – where do these addresses come from?  We’ve stressed that it’s difficult to get hold of residential IP addresses as they’re distributed by ISPs to home users.  So how do the likes of Microleaves and Luminati manage to get hold of millions of these addresses.  Well there’s only one real method to build up those sort of numbers and that’s to get them from ordinary users themselves.  Microleaves for example is well known for rather nasty adware which installs itself on your computer which can be controlled via a browser plugin.  This also gives them access to your internet connection and residential IP address.  Luminati offer a free VPN and some other applications in exchange for using your connection too.

Ultimately, you will to a certain extent get what you pay for with proxies. You can of course find free proxies online but they are totally useless for most commercial tasks simply because they’re completely overloaded. Also a fair proportion of them are set up by cyber criminals and identity thieves to steal data so you should never use them for any commercial transactions, or anything involving usernames or passwords.

There is however another reason why many providers use rotating proxies and that’s because they don’t actually own any IP addresses of their own.  Yes, there’s actually a thriving market in reselling other peoples connections to provide what looks like a professional service which supplies unlimited residential proxies (yet is actually little more than a reseller with a fancy website).  These addresses are bought from a pool of addresses and connections created from a variety of means – here’s just a brief example.

• Application Vendors who’s API provides access to User’s internet connections.
• VPN services which supply ‘free options’ which share the host’s connections
• Malware and Virus farmed addresses from hacked accounts and computers (beware free proxy list api offers as this is where many addresses originate from)

It’s big business now and particularly residential addresses from normal internet connections have a significant value.  If you’ve done any research into how to make residential proxies then you’ll appreciate how difficult and expensive it is to create them.  They’re much harder to obtain and run than standard datacenter proxy solutions.  If the pricing plan seems too low you’re probably better avoiding them altogether as there’s a large chance that the IP addresses actually are hacked or abused internet connections shared by thousands.

Unfortunately, nowadays residential IP addresses are increasingly required as many sites block access from the cheap and available commercial IP address ranges. There is no way to obtain any volume of these without spending a lot of money, but they’re increased cost is definitely worth it.

There are actually only a handful of proxy providers who offer a reliable and extensive network of proxies. Indeed many companies merely lease or resell other peoples proxies and addresses too.  The best ones will answer your questions and will usually offer a brief trial for you to test.  Be aware that most can’t offer long trials as they are often abused and the IP addresses blocked.

Storm Proxies

These guys are a great place to start, one of the oldest proxy providers, honest, knowledgeable and reliable rotating proxy providers.