Importing IP2Proxy data into Redis and querying with PHP

The aim of this guide is to demonstrate how to import IP2Proxy data in csv form into Redis and then query the data in a PHP web page.

First of all, you will need to download the IP2Proxy csv file. You can use any of the PX packages from PX1 to PX4.
Download Free LITE version at https://lite.ip2location.com/
Download commercial version at https://www.ip2location.com/download

Extract out the file from the downloaded zipped file.

Important Note

We will not cover installation of Redis, Perl or PHP in this guide. We will assume you have already setup Redis on the localhost and are using PHP via Apache (also on the localhost). For this example, we are using a Debian Linux.

More info can be found at the following URLs if you need assistance with installations:
PHP: http://php.net/manual/en/install.unix.debian.php
Redis: http://redis.io/download
Perl Redis Library: http://search.cpan.org/~dams/Redis/lib/Redis.pm
PHP Redis Library: http://pecl.php.net/package/redis

We will be using the Perl Redis library to do mass import of CSV data into Redis and then querying the data via PHP using the PHP Redis library.

Importing the csv data into Redis

As mentioned earlier, we will be using a Perl script to perform the import of the CSV data into Redis as we need to format our CSV data into an easier to store form.

Create a new Perl file called importredis.pl and paste the following code into it:

use strict;
use Redis;
 
my $filename = 'IP2PROXY-IP-COUNTRY.CSV'; # PX1
 
my $data = '';
my $dbname = 'PX1';
my $client = Redis->new( server => 'localhost:6379', password => 'your_redis_password' );
$client->del($dbname);
open IN, "<$filename" or die;
 
my $counter = 0;
my $line = '';
while (<IN>)
{
    $line = $_;
    $line =~ s/[\r\n]+//; # remove EOL
 
    if ($line =~ /^"([^"]+)","([^"]+)","([^"]+)","([^"]+)"$/)
    {
        $counter++;
        my $ipfrom = $1;
        my $ipto = $2;
        my $countrycode = $3;
        my $countryname = $4;
        my $datastr = '"' . $ipfrom . "|" . $ipto . "|" . $countrycode . "|" . $countryname . '"';
        $datastr =~ s/([\@\%\$])/\\\1/g;
        $data .= ', ' . $ipfrom . ' => ' . $datastr;
 
        if ($counter % 5000 == 0)
        {
            print "Running line $counter\n";
            &runMe($data);
            $data = '';
        }
    }
}
 
if ($data ne '')
{
    &runMe($data);
    $data = '';
}
close IN;
 
sub runMe
{
    my $mydata = shift;
    eval('$client->zadd($dbname' . $mydata . ');');
    warn $@ if $@;
}

 

use strict;
use Redis;
 
my $filename = 'IP2PROXY-IP-PROXYTYPE-COUNTRY.CSV'; # PX2
 
my $data = '';
my $dbname = 'PX2';
my $client = Redis->new( server => 'localhost:6379', password => 'your_redis_password' );
$client->del($dbname);
open IN, "<$filename" or die;
 
my $counter = 0;
my $line = '';
while (<IN>)
{
    $line = $_;
    $line =~ s/[\r\n]+//; # remove EOL
 
    if ($line =~ /^"([^"]+)","([^"]+)","([^"]+)","([^"]+)","([^"]+)"$/)
    {
        $counter++;
        my $ipfrom = $1;
        my $ipto = $2;
        my $proxytype = $3;
        my $countrycode = $4;
        my $countryname = $5;
        my $datastr = '"' . $ipfrom . "|" . $ipto . "|" . $proxytype . "|" . $countrycode . "|" . $countryname . '"';
        $datastr =~ s/([\@\%\$])/\\\1/g;
        $data .= ', ' . $ipfrom . ' => ' . $datastr;
 
        if ($counter % 5000 == 0)
        {
            print "Running line $counter\n";
            &runMe($data);
            $data = '';
        }
    }
}
 
if ($data ne '')
{
    &runMe($data);
    $data = '';
}
close IN;
 
sub runMe
{
    my $mydata = shift;
    eval('$client->zadd($dbname' . $mydata . ');');
    warn $@ if $@;
}

 

use strict;
use Redis;
 
my $filename = 'IP2PROXY-IP-PROXYTYPE-COUNTRY-REGION-CITY.CSV'; # PX3
 
my $data = '';
my $dbname = 'PX3';
my $client = Redis->new( server => 'localhost:6379', password => 'your_redis_password' );
$client->del($dbname);
open IN, "<$filename" or die;
 
my $counter = 0;
my $line = '';
while (<IN>)
{
    $line = $_;
    $line =~ s/[\r\n]+//; # remove EOL
 
    if ($line =~ /^"([^"]+)","([^"]+)","([^"]+)","([^"]+)","([^"]+)","([^"]+)","([^"]+)"$/)
    {
        $counter++;
        my $ipfrom = $1;
        my $ipto = $2;
        my $proxytype = $3;
        my $countrycode = $4;
        my $countryname = $5;
        my $regionname = $6;
        my $cityname = $7;
        my $datastr = '"' . $ipfrom . "|" . $ipto . "|" . $proxytype . "|" . $countrycode . "|" . $countryname . "|" . $regionname . "|" . $cityname . '"';
        $datastr =~ s/([\@\%\$])/\\\1/g;
        $data .= ', ' . $ipfrom . ' => ' . $datastr;
 
        if ($counter % 5000 == 0)
        {
            print "Running line $counter\n";
            &runMe($data);
            $data = '';
        }
    }
}
 
if ($data ne '')
{
    &runMe($data);
    $data = '';
}
close IN;
 
sub runMe
{
    my $mydata = shift;
    eval('$client->zadd($dbname' . $mydata . ');');
    warn $@ if $@;
}

 

use strict;
use Redis;
 
my $filename = 'IP2PROXY-IP-PROXYTYPE-COUNTRY-REGION-CITY-ISP.CSV'; # PX4
 
my $data = '';
my $dbname = 'PX4';
my $client = Redis->new( server => 'localhost:6379', password => 'your_redis_password' );
$client->del($dbname);
open IN, "<$filename" or die;
 
my $counter = 0;
my $line = '';
while (<IN>)
{
    $line = $_;
    $line =~ s/[\r\n]+//; # remove EOL
 
    if ($line =~ /^"([^"]+)","([^"]+)","([^"]+)","([^"]+)","([^"]+)","([^"]+)","([^"]+)","([^"]+)"$/)
    {
        $counter++;
        my $ipfrom = $1;
        my $ipto = $2;
        my $proxytype = $3;
        my $countrycode = $4;
        my $countryname = $5;
        my $regionname = $6;
        my $cityname = $7;
        my $isp = $8;
        my $datastr = '"' . $ipfrom . "|" . $ipto . "|" . $proxytype . "|" . $countrycode . "|" . $countryname . "|" . $regionname . "|" . $cityname . "|" . $isp . '"';
        $datastr =~ s/([\@\%\$])/\\\1/g;
        $data .= ', ' . $ipfrom . ' => ' . $datastr;
 
        if ($counter % 5000 == 0)
        {
            print "Running line $counter\n";
            &runMe($data);
            $data = '';
        }
    }
}
 
if ($data ne '')
{
    &runMe($data);
    $data = '';
}
close IN;
 
sub runMe
{
    my $mydata = shift;
    eval('$client->zadd($dbname' . $mydata . ');');
    warn $@ if $@;
}

 

Run the Perl script by calling the below command in command prompt:
perl importredis.pl

Querying the IP2Proxy data from a PHP web page

Now, create a PHP file called test.php in your website.

Paste the following PHP code into it and then run it in the browser:

<?php
$ip = '8.8.8.8';
 
function queryIP2Proxy($myip) {
    $redis = new Redis();
    $redis->connect('127.0.0.1'); // port 6379 by default
    if ($redis->auth('your_redis_password')) {
        $ipnum = sprintf("%u", ip2long($myip));
 
        $result = $redis->zRevRangeByScore('PX1', $ipnum, 0, array('limit' => array(0, 1)));
 
        if (!empty($result)) {
            $result = $result[0];
            $arr = explode("|", $result);
 
            if ($arr[1] >= $ipnum) {
                echo "is proxy<br>\n";
                echo "country code: " . $arr[2] . "<br>\n";
            echo "country name: " . $arr[3] . "<br>\n";
            }
            else {
                echo "Not proxy<br>\n";
            }
        }
        else {
            echo "Not proxy<br>\n";
        }
    }
    else {
        echo "Incorrect password\n";
    }
    $redis->close();
}
 
queryIP2Proxy($ip);
?>

 

<?php
$ip = '8.8.8.8';
 
function queryIP2Proxy($myip) {
    $redis = new Redis();
    $redis->connect('127.0.0.1'); // port 6379 by default
    if ($redis->auth('your_redis_password')) {
        $ipnum = sprintf("%u", ip2long($myip));
 
        $result = $redis->zRevRangeByScore('PX2', $ipnum, 0, array('limit' => array(0, 1)));
 
        if (!empty($result)) {
            $result = $result[0];
            $arr = explode("|", $result);
 
            if ($arr[1] >= $ipnum) {
                echo "is proxy<br>\n";
                echo "proxy type: " . $arr[2] . "<br>\n";
            echo "country code: " . $arr[3] . "<br>\n";
            echo "country name: " . $arr[4] . "<br>\n";
            }
            else {
                echo "Not proxy<br>\n";
            }
        }
        else {
            echo "Not proxy<br>\n";
        }
    }
    else {
        echo "Incorrect password\n";
    }
    $redis->close();
}
 
queryIP2Proxy($ip);
?>

 

<?php
$ip = '8.8.8.8';
 
function queryIP2Proxy($myip) {
    $redis = new Redis();
    $redis->connect('127.0.0.1'); // port 6379 by default
    if ($redis->auth('your_redis_password')) {
        $ipnum = sprintf("%u", ip2long($myip));
 
        $result = $redis->zRevRangeByScore('PX3', $ipnum, 0, array('limit' => array(0, 1)));
 
        if (!empty($result)) {
            $result = $result[0];
            $arr = explode("|", $result);
 
            if ($arr[1] >= $ipnum) {
                echo "is proxy<br>\n";
                echo "proxy type: " . $arr[2] . "<br>\n";
            echo "country code: " . $arr[3] . "<br>\n";
            echo "country name: " . $arr[4] . "<br>\n";
            echo "region name: " . $arr[5] . "<br>\n";
            echo "city name: " . $arr[6] . "<br>\n";
            }
            else {
                echo "Not proxy<br>\n";
            }
        }
        else {
            echo "Not proxy<br>\n";
        }
    }
    else {
        echo "Incorrect password\n";
    }
    $redis->close();
}
 
queryIP2Proxy($ip);
?>

 

<?php
$ip = '8.8.8.8';
 
function queryIP2Proxy($myip) {
    $redis = new Redis();
    $redis->connect('127.0.0.1'); // port 6379 by default
    if ($redis->auth('your_redis_password')) {
        $ipnum = sprintf("%u", ip2long($myip));
 
        $result = $redis->zRevRangeByScore('PX4', $ipnum, 0, array('limit' => array(0, 1)));
 
        if (!empty($result)) {
            $result = $result[0];
            $arr = explode("|", $result);
 
            if ($arr[1] >= $ipnum) {
                echo "is proxy<br>\n";
                echo "proxy type: " . $arr[2] . "<br>\n";
            echo "country code: " . $arr[3] . "<br>\n";
            echo "country name: " . $arr[4] . "<br>\n";
            echo "region name: " . $arr[5] . "<br>\n";
            echo "city name: " . $arr[6] . "<br>\n";
            echo "isp: " . $arr[7] . "<br>\n";
            }
            else {
                echo "Not proxy<br>\n";
            }
        }
        else {
            echo "Not proxy<br>\n";
        }
    }
    else {
        echo "Incorrect password\n";
    }
    $redis->close();
}
 
queryIP2Proxy($ip);
?>

 

 

 

Was this article helpful?

Yes No

Related Articles