Intro

Good news! IP2Proxy Go Package version 2.3.0 has been modified to support concurrent proxy queries using the IP2Proxy BIN database files. This means you can write more powerful and faster codes to get your task done sooner. Below is our simple example to demonstrate concurrency with the IP2Proxy Go Package.

Example code

Below we have the full example code where we launch 20 threads. Each thread will then query proxy server info using the IP2Proxy Go Package for 1000 random IPs. Random IPs and multiple threads present a more realistic usage scenario in today’s multi-threaded environment.

package main

import (
  "fmt"
  "strings"
  "strconv"
  "time"
  "github.com/ip2location/ip2proxy-go"
  crypto_rand "crypto/rand"
  "encoding/binary"
  math_rand "math/rand"
  "sync"
)

func main() {
  maxthreads := 20
  respond := make(chan string, maxthreads)
  var wg sync.WaitGroup
  wg.Add(maxthreads)
  
  for i := 1; i <= maxthreads; i++ {
    go doQuery(respond, &wg)
  }
  
  wg.Wait()
  close(respond)
  
  for queryResp := range respond {
    fmt.Printf("Got Response:\t %s\n", queryResp)
  }
}

func doQuery(respond chan<- string, wg *sync.WaitGroup) {
  defer wg.Done()
  
  db, err := ip2proxy.OpenDB("./IP2PROXY-IP-PROXYTYPE-COUNTRY-REGION-CITY-ISP-DOMAIN-USAGETYPE-ASN-LASTSEEN-THREAT-RESIDENTIAL-PROVIDER.BIN")
  if err != nil {
    return
  }
  
  var ips []string
  var totaltime int64 = 0
  finalresults := ""
  maxtests := 1000
  
  ip := ""
  for x := 0; x < maxtests; x++ {
    ip = getIp()
    ips = append(ips, ip)
  }
  
  for index, _ := range ips {
    start := time.Now()
    all, err := db.GetAll(ips[index])
    
    if err != nil {
      fmt.Print(err)
      return
    }
    
    end := time.Now()
    duration := end.Sub(start)
    totaltime = totaltime + duration.Nanoseconds()
    
    finalresults += fmt.Sprintf("IP: %s\n", ips[index])
    finalresults += fmt.Sprintf("ModuleVersion: %s\n", ip2proxy.ModuleVersion())
    finalresults += fmt.Sprintf("PackageVersion: %s\n", db.PackageVersion())
    finalresults += fmt.Sprintf("DatabaseVersion: %s\n", db.DatabaseVersion())
    finalresults += fmt.Sprintf("isProxy: %s\n", all["isProxy"])
    finalresults += fmt.Sprintf("ProxyType: %s\n", all["ProxyType"])
    finalresults += fmt.Sprintf("CountryShort: %s\n", all["CountryShort"])
    finalresults += fmt.Sprintf("CountryLong: %s\n", all["CountryLong"])
    finalresults += fmt.Sprintf("Region: %s\n", all["Region"])
    finalresults += fmt.Sprintf("City: %s\n", all["City"])
    finalresults += fmt.Sprintf("ISP: %s\n", all["ISP"])
    finalresults += fmt.Sprintf("Domain: %s\n", all["Domain"])
    finalresults += fmt.Sprintf("UsageType: %s\n", all["UsageType"])
    finalresults += fmt.Sprintf("ASN: %s\n", all["ASN"])
    finalresults += fmt.Sprintf("AS: %s\n", all["AS"])
    finalresults += fmt.Sprintf("LastSeen: %s\n", all["LastSeen"])
    finalresults += fmt.Sprintf("Threat: %s\n", all["Threat"])
    finalresults += fmt.Sprintf("Provider: %s\n", all["Provider"])
    finalresults += fmt.Sprintln("=========================================================")
  }
  
  db.Close()
  
  totaltime64 := float64(totaltime / 1000000)
  maxtests64 := float64(maxtests)
  finalresults += fmt.Sprintf("Total time: %vms\n", totaltime64)
  finalresults += fmt.Sprintf("Average time: %vms\n", (totaltime64 / maxtests64))
  
  respond <- finalresults
}

func getIp() string {
  var b [8]byte
  _, err := crypto_rand.Read(b[:])
  
  if err != nil {
    panic("Cannot seed math/rand package with cryptographically secure random number generator")
  }
  math_rand.Seed(int64(binary.LittleEndian.Uint64(b[:])))
  
  var s []string
  p := math_rand.Perm(255)
  for _, r := range p[:4] {
    // fmt.Println(r)
    s = append(s, strconv.Itoa(r))
  }
  ip := strings.Join(s, ".")
  
  return ip
}

NOTE: The ModuleVersion function is called under the ip2proxy object as it is related to the main object and not the specific database object.

The main function

In our main function, we are creating a buffered channel so we can get values from our 20 threads. We are also creating a WaitGroup so we can collect results from all 20 threads before we output them.

To spawn a new thread, we call our doQuery function with the go keyword. Both the channel and the WaitGroup are passed as parameters to the doQuery function so that they can return their results and notify the WaitGroup when the function is done processing.

The doQuery function

In this function, we creating an instance of the DB object by calling the ip2proxy.OpenDB function. This DB object called db functions independently from the db object in the other threads. The db object is the one performing the proxy queries.

After instantiating the db object, we generate a list of random IPs for our tests using the getIp function. Then we feed the test IPs to the db.GetAll method which will return all the relevant proxy information. We will append the results into a string. Each loop will check the proxy data for the current IP in the loop then keep appending the results. When the loop is done, we will call db.Close to close the db object.

Once all tests have concluded, the result string is returned to the calling code via the channel. Our code also calculates the average time taken per query in that particular thread.

Conclusion

Any decent package should be usable in a multi-threaded environment and our example above clearly demonstrates that the IP2Proxy Go Package is now capable of that. We hope that this new ability will enable you to create even better codes for your users.