Last Updated on January 19, 2024 by theadmin
Are Your Proxies Working as Intended
Many of us have probably sat down at our desks, after having invested a small fortune in automated software like for grabbing sneakers, tickets or some other precious merchandise. We know that we won’t get far using our PC and single IP address so have invested in a whole bunch of rotating residential proxies so are feeling quietly confident that we are prepared.
Indeed even those two steps put you several steps ahead of the competition and there’s every chance you’ll succeed in your venture. Except you don’t, your sneakers never get to the basket, your Craigslist adverts just get your account banned and those precious concert tickets simply never appear. In fact you might as well have saved your money and failed like the noobs through a single desktop connection!
This article will hopefully give a little insight into how proxies are actually blocked. It happens to us all, in fact if all this stuff was easy to do then everyone would be doing it. However it’s important not to complicate these situations, the secret is to identifying a combination that works for you and then watch the profits flow in.
There is lots of hype and lots of misleading advice when it comes to choosing proxies for your chosen market. Most people quickly realise that using free proxies is both risky, pointless and pretty stupid too. Yet choosing the right proxies for any particular task can be incredibly confusing mostly because it’s a very fluid situation and changes all the time. Some of the proxy companies which were incredible one year will often simply stop working over night, it’s wise to never get to over invested in any method long term.
Basic Proxy Blocking Techniques
The problem of using proxies is actually simple and is fairly similar across all sorts of markets. We need clean, secure proxies with IP addresses which let us use the service unhindered. We need to look like unique connections not multiple requests from the same person. So it’s important to know how this can fail i.e. how do the services block my proxies.
― The Art of War
“If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not the enemy, for every victory gained you will also suffer a defeat. If you know neither the enemy nor yourself, you will succumb in every battle.”
Sounds a little pretentious I know, but it’s quite a relevant quote for much of us trying to make our living online. Most of the servers are trying to block the use of proxies, so knowing what methods they use can certainly help you select some winners.
So how do the companies like Nike, Ticketmaster and Supreme detect all these proxies? First of all let’s remember any ‘fail’ is not always the fault of the proxies, most security services use a range of of measures to detect unwanted traffic. It makes sense to look at lots of information in order to block this traffic not just a simple proxy detection script.
Most services use a system which rates the probability of any given connection using a proxy, where perhaps 0 = completely clear and 1= it’s definitely a proxy. This allows the intelligence company to build up an assessment based on whatever evidence they can gather and also allows the company to decide the threshold at which they block a connection.
Static IP Address Blacklists (Banned Proxies)
This is without the doubt the biggest factor in how these systems vet inbound connections. Creating and checking against blacklists sounds simple but is an extremely effective method of detecting proxies. The value obviously depends on the quality of the information used in compiling these lists. There are thousands of these lists floating around online – some are very accurate but many are completely out of date and incorrect. The problem with blacklists like this is that they have to kept up to date, IP addresses change owners and use all the time and should be reflected in these lists.
The most sophisticated and advanced companies maintain their own databases, compiled by a variety of means. There are problems using this method but if your proxies IP addresses are on any of these lists then there’s a good chance that they’re going to get blocked by your target servers.
Dynamic Lists
If these companies just relied on a few static lists, choosing proxies would be much simpler. However in reality, there’s many more methods that can be used which create individual dynamic lists. So for example if you use mis-configure your Bot to use the same address over and over again whilst using Ticketmaster then that IP address is going to be added to a list. Normal customers don’t try and get thirty tickets, and by attempting this then you’ll inadvertently got your address blocked.
User behaviour is at the core of these lists, activity can be analysed in great detail when people are using the website. This is why it’s so important to be careful especially when using any automated software – configure them carefully and ensure that delays and settings which mimic human like behavior are used.
If you think this is easily avoidable and you’d never do this you’re probably right. However consider the dozens of other people using the same IP addresses on a rotating proxies from your provider – what if one of them does this? It’s quite easy for a range of IP addresses to become completely useless with a target server very quickly which is why scale is so important.
The proxy provider with 200 residential IP addresses which they rotate carefully is in a very vulnerable position. With the best will in the world, these addresses can be blocked or blacklisted very quickly on a specific platform. It happens all the time, which is why scale is so important.
Scale – Large IP Networks make Great Proxies
I like little, specialized companies especially in IT. You normally find people who know what they’re doing and the company is run well. However all it takes is a few careless/dishonest customers to wreck a large number of IP addresses especially if they’re using Bots. The risks can definitely be minimized, checking with customers, supplying support – careful rotation of IP addresses etc but scale helps solve these issues too.
The largest companies like Luminati have millions of IP addresses which they can route your traffic through. Sure some might get blocked but with those odds, most tasks can be completed successfully. Check out the Luminati Waterfall system for minimizing costs too, it’s highly innovative and a great idea!
The Datacenter Dilemma
Finally the last point I want to mention is the usual problem of what sort of IP addresses should our proxies use. The cheapest and fastest proxies use datacentre IP ranges which are easy to get and don’t need any sort of special routing. However for many applications they don’t seem to work very well anymore. Using a datacentre proxy will almost certainly move the scale upwards in your proxy probability score. For certain platforms or smaller scale efforts then this might still work, but be prepared to risk your accounts.
Most people now look straight for rotating residential proxies because these IP addresses are much more trusted than datacentre ones. If you need to look like a residential user then use the same IP address that we all use from our home connections. Using reserved dedicated addresses for your own use only sounds appealing but in reality it can be very expensive. It’s cheaper to use proxies that are automatically rotated for you and shared with other users – they usually work just as well – you can try them at our friends in RotatingProxies.com
