View Categories

Enrich CSV Data with IP2Location BIN in Apache NiFi

hexasoft
Updated on July 10, 2026

8 min read

Enrich CSV Data with IP2Location BIN in Apache NiFi

Intro #

Apache NiFi is a visual dataflow automation platform. It lets you build data pipelines by connecting processors on a canvas instead of writing a full custom ETL program. NiFi can read data from files, APIs, queues, databases, Kafka, HTTP endpoints, and many other systems. It can then transform, route, enrich, and write the data to another destination.

In this tutorial, we will use NiFi to read a CSV file containing IP addresses, enrich each row using IP2Location DB26 geolocation data, and output a new enriched CSV file.

The enrichment is handled by the IP2Location NiFi Record Bundle:

https://github.com/ip2location/nifi-ip2location-record-bundle

The bundle provides:

StandardIP2LocationDatabaseService

IP2LocationEnrichRecord

The Controller Service opens the IP2Location DB26 BIN file. The processor reads each CSV record, looks up the IP address, and adds geolocation fields such as country, region, city, latitude, longitude, ZIP code, time zone, ISP, domain, usage type, district, ASN, and other DB26 fields.

We will expose the NiFi UI through Caddy:

Browser

  ↓ HTTPS 443

Caddy reverse proxy

  ↓ local HTTPS 8443

Apache NiFi

The final NiFi flow will be:

GetFile

  ↓

IP2LocationEnrichRecord

  ↓

QueryRecord

  ↓

PutFile

IP2LocationEnrichRecord adds the IP2Location result as a nested record. QueryRecord then flattens the nested result into normal CSV columns.

Prerequisites #

This tutorial assumes:

Server OS: Debian 13

NiFi version: Apache NiFi 2.9.0

Java version: JDK 21

Reverse proxy: Caddy

Database: IP2Location DB26 BIN

NAR file: nifi-ip2location-record-bundle from GitHub Releases

Example domain used in this tutorial:

nifi.example.com

Replace it with your real domain.

Example paths:

/opt/nifi                         Apache NiFi installation

/opt/nifi/extensions/ip2location  Custom IP2Location NAR directory

/opt/ip2location                  IP2Location BIN database directory

/var/lib/nifi-data/in             Input CSV directory

/var/lib/nifi-data/out            Output CSV directory

Step 1: Point your domain to the server #

Create an A record in your DNS provider:

nifi.example.com → YOUR_SERVER_PUBLIC_IP

Make sure these ports are open in your cloud firewall or security group:

80/tcp

443/tcp

You do not need to expose NiFi port 8443 publicly because Caddy will proxy to it locally.

Step 2: Install Java 21, Caddy, and required tools #

Install required packages:

💻 
sudo apt update
sudo apt install -y openjdk-21-jdk curl unzip jq caddy

Check Java:

💻 
java -version

Confirm the Java path:

💻 
readlink -f /usr/bin/java

You should see something like:

/usr/lib/jvm/java-21-openjdk-amd64/bin/java or /usr/lib/jvm/java-21-openjdk-arm64/bin/java

The correct JAVA_HOME is usually:

/usr/lib/jvm/java-21-openjdk-amd64 or /usr/lib/jvm/java-21-openjdk-arm64

You can confirm it with:

💻 
dirname "$(dirname "$(readlink -f /usr/bin/java)")"

Step 3: Create the NiFi user and directories #

Create a dedicated NiFi system user:

💻 
sudo useradd --system --home /opt/nifi --shell /usr/sbin/nologin nifi

Create required directories:

💻 
sudo mkdir -p /opt/nifi
sudo mkdir -p /opt/ip2location
sudo mkdir -p /var/lib/nifi-data/in
sudo mkdir -p /var/lib/nifi-data/out

Step 4: Download and install Apache NiFi #

Set the NiFi version, then download and extract NiFi:

💻 
NIFI_VERSION=2.9.0
cd /tmp
curl -fLO "https://archive.apache.org/dist/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-bin.zip"
unzip "nifi-${NIFI_VERSION}-bin.zip"

Install it under /opt/nifi:

💻 
sudo rm -rf /opt/nifi
sudo mv "nifi-${NIFI_VERSION}" /opt/nifi
sudo mkdir -p /opt/nifi/extensions/ip2location

Set ownership:

💻 
sudo chown -R nifi:nifi /opt/nifi
sudo chown -R nifi:nifi /opt/ip2location
sudo chown -R nifi:nifi /var/lib/nifi-data
sudo chown -R nifi:nifi /opt/nifi/extensions

Step 5: Configure NiFi for local HTTPS behind Caddy #

Edit NiFi properties:

💻 
sudo nano /opt/nifi/conf/nifi.properties

Set or update these values:

nifi.properties 
nifi.web.https.host=localhost
nifi.web.https.port=8443
nifi.web.proxy.host=nifi.example.com,nifi.example.com:443

Replace:

nifi.example.com

with your actual domain.

Set a sensitive properties key. You can generate one with:

💻 
openssl rand -base64 32

Then set it in nifi.properties:

nifi.properties 
nifi.sensitive.props.key=PASTE_A_LONG_RANDOM_VALUE_HERE

The important idea is:

NiFi listens only on localhost:8443.

Caddy is the only public entry point.

Step 6: Set a known NiFi username and password #

Set a single-user login:

💻 
sudo -u nifi /opt/nifi/bin/nifi.sh set-single-user-credentials admin 'ChangeThisStrongPassword123!'

Use a stronger password for a real server.

Step 7: Create the systemd service and fix JAVA_HOME #

Find the Java home path and create the systemd service:

💻 
JAVA_HOME_PATH=$(dirname "$(dirname "$(readlink -f /usr/bin/java)")")
echo "$JAVA_HOME_PATH"

sudo tee /etc/systemd/system/nifi.service > /dev/null <<EOF
[Unit]
Description=Apache NiFi
After=network.target

[Service]
Type=forking
User=nifi
Group=nifi
Environment="JAVA_HOME=${JAVA_HOME_PATH}"
Environment="PATH=${JAVA_HOME_PATH}/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
ExecStart=/opt/nifi/bin/nifi.sh start
ExecStop=/opt/nifi/bin/nifi.sh stop
Restart=on-failure
TimeoutStartSec=300
TimeoutStopSec=120

[Install]
WantedBy=multi-user.target
EOF

Reload systemd:

💻 
sudo systemctl daemon-reload
sudo systemctl enable nifi

Start NiFi:

💻 
sudo systemctl start nifi

Check status:

💻 
sudo systemctl status nifi

Check the NiFi log:

💻 
sudo tail -f /opt/nifi/logs/nifi-app.log

NiFi can take a minute or two to fully start.

Step 8: Confirm NiFi is listening locally #

Check port 8443:

💻 
sudo ss -tulpn | grep 8443

You should see NiFi listening on localhost, for example:

127.0.0.1:8443

or:

[::1]:8443

You should not need public access to port 8443.

Step 9: Configure Caddy as HTTPS reverse proxy #

Edit the Caddyfile:

💻 
sudo nano /etc/caddy/Caddyfile

Use this configuration:

Caddyfile 
nifi.example.com {
    redir / /nifi/

    reverse_proxy https://localhost:8443 {
        header_up X-ProxyScheme https
        header_up X-ProxyHost {host}
        header_up X-ProxyPort 443

        transport http {
            tls_insecure_skip_verify
            tls_server_name localhost
        }
    }
}

 Replace:

nifi.example.com

with your actual domain.

This configuration lets Caddy get a public HTTPS certificate for your domain, then proxy traffic to NiFi running locally on https://localhost:8443.

Validate Caddy:

💻 
sudo caddy validate --config /etc/caddy/Caddyfile

Reload Caddy:

💻 
sudo systemctl reload caddy

Check Caddy:

💻 
sudo systemctl status caddy

Now open:

https://nifi.example.com/nifi

You should see the NiFi login page with a valid HTTPS certificate from Caddy.

Step 10: Install the IP2Location NiFi NAR #

Download the latest NAR from GitHub Releases:

💻 
NAR_URL=$(curl -s https://api.github.com/repos/ip2location/nifi-ip2location-record-bundle/releases/latest \
  | jq -r '.assets[] | select(.name | endswith(".nar")) | .browser_download_url' \
  | head -n 1)
echo "$NAR_URL"

Download it into the NiFi extension directory:

💻 
sudo curl -L "$NAR_URL" -o "/opt/nifi/extensions/ip2location/$(basename "$NAR_URL")"
sudo chown -R nifi:nifi /opt/nifi/extensions/ip2location

Tell NiFi to load NARs from this directory:

💻 
sudo bash -c 'grep -q "^nifi.nar.library.directory.ip2location=" /opt/nifi/conf/nifi.properties \
  || echo "nifi.nar.library.directory.ip2location=/opt/nifi/extensions/ip2location" >> /opt/nifi/conf/nifi.properties'

Restart NiFi:

💻 
sudo systemctl restart nifi

Watch the log:

💻 
sudo tail -f /opt/nifi/logs/nifi-app.log

After restart, these components should be available in the NiFi UI:

StandardIP2LocationDatabaseService

IP2LocationEnrichRecord

Step 11: Install the IP2Location DB26 BIN database #

If you don’t have a BIN file yet, you can either register for the free IP2Location LITE BIN databases or subscribe to the paid IP2Location BIN databases. Upon login, you can download the zipped file containing the BIN file you want, then extract that BIN file and upload it to the server.

For our example, we’ll be using the IP2Location DB26 IPv6 BIN file. We’ll copy the BIN file to the server on the home directory after renaming it to DB26.BIN.

On the server:

💻 
sudo mv ~/DB26.BIN /opt/ip2location/
sudo chown nifi:nifi /opt/ip2location/DB26.BIN

Confirm NiFi can read it:

💻 
sudo -u nifi ls -lh /opt/ip2location/DB26.BIN

Step 12: Create a sample CSV input file #

Create a test CSV file:

💻 
sudo nano /var/lib/nifi-data/in/orders.csv

Add:

orders.csv 
order_id,ip,amount
A1001,8.8.8.8,19.90
A1002,1.1.1.1,29.90
A1003,208.67.222.222,39.90
A1004,invalid_ip,9.90

Set ownership:

💻 
sudo chown nifi:nifi /var/lib/nifi-data/in/orders.csv

Step 13: Configure the IP2Location Controller Service #

In the NiFi UI, open:

Controller Services

Add a new Controller Service:

StandardIP2LocationDatabaseService

Configure it:

IP2Location BIN File:

/opt/ip2location/DB26.BIN

Database Access Method:

File I/O

Lookup Cache Size:

10000

Enable the StandardIP2LocationDatabaseService.

Step 14: Create the CSVReader Controller Service #

Add a new Controller Service:

CSVReader

Configure it:

Schema Access Strategy:

Use String Fields From Header

CSV Format:

RFC 4180

Enable the CSVReader.

This makes NiFi use the CSV header row as field names.

Step 15: Create the intermediate JsonRecordSetWriter #

Add a new Controller Service:

JsonRecordSetWriter

Configure it:

Schema Write Strategy:

Do Not Write Schema

Enable the JsonRecordSetWriter.

The IP2Location processor adds a nested ip2location object, so JSON is a good intermediate format before flattening back to CSV.

Step 16: Create the JsonTreeReader for QueryRecord #

Add another Controller Service:

JsonTreeReader

Configure it:

Schema Access Strategy:

Infer Schema

Enable the JsonTreeReader.

This reader will be used by QueryRecord to read the enriched JSON content.

Step 17: Create the final CSVRecordSetWriter #

Add another Controller Service:

CSVRecordSetWriter

Configure it:

Schema Access Strategy:

Inherit Record Schema

Schema Write Strategy:

Do Not Write Schema

Include Header Line:

true

Enable the CSVRecordSetWriter.

This writer will produce the final enriched CSV output.

Step 18: Add the GetFile processor #

Drag a processor onto the canvas and choose:

GetFile

Configure it:

Input Directory:

/var/lib/nifi-data/in

Keep Source File:

false

This processor reads files from the input directory.

Step 19: Add the IP2LocationEnrichRecord processor #

Add another processor:

IP2LocationEnrichRecord

Configure it:

IP2Location Database Service:

StandardIP2LocationDatabaseService

Record Reader:

CSVReader

Record Writer:

JsonRecordSetWriter

IP Field Path:

/ip

Output Field Name:

ip2location

Return Fields:

all

Include Status:

true

Include Empty Values:

false

Non-OK Lookup Strategy:

Keep Record

This processor reads each CSV row, gets the IP address from the /ip field, performs a local DB26 lookup, and adds the result under the ip2location field.

Step 20: Add QueryRecord to flatten the enriched result #

Add another processor:

QueryRecord

Configure it:

Record Reader:

JsonTreeReader

Record Writer:

CSVRecordSetWriter

Add a dynamic property.

Property name:

enriched_csv

Property value:

enriched_csv 
SELECT
  order_id,
  ip,
  amount,

  RPATH_STRING(ip2location, '/status') AS ip2location_status,
  RPATH_STRING(ip2location, '/country_short') AS country_code,
  RPATH_STRING(ip2location, '/country_long') AS country_name,
  RPATH_STRING(ip2location, '/region') AS region_name,
  RPATH_STRING(ip2location, '/city') AS city_name,
  RPATH_DOUBLE(ip2location, '/latitude') AS latitude,
  RPATH_DOUBLE(ip2location, '/longitude') AS longitude,
  RPATH_STRING(ip2location, '/zipcode') AS zip_code,
  RPATH_STRING(ip2location, '/timezone') AS time_zone,
  RPATH_STRING(ip2location, '/isp') AS isp,
  RPATH_STRING(ip2location, '/domain') AS domain,
  RPATH_STRING(ip2location, '/netspeed') AS net_speed,
  RPATH_STRING(ip2location, '/iddcode') AS idd_code,
  RPATH_STRING(ip2location, '/areacode') AS area_code,
  RPATH_STRING(ip2location, '/weatherstationcode') AS weather_station_code,
  RPATH_STRING(ip2location, '/weatherstationname') AS weather_station_name,
  RPATH_STRING(ip2location, '/mcc') AS mcc,
  RPATH_STRING(ip2location, '/mnc') AS mnc,
  RPATH_STRING(ip2location, '/mobilebrand') AS mobile_brand,
  RPATH_DOUBLE(ip2location, '/elevation') AS elevation,
  RPATH_STRING(ip2location, '/usagetype') AS usage_type,
  RPATH_STRING(ip2location, '/addresstype') AS address_type,
  RPATH_STRING(ip2location, '/category') AS category,
  RPATH_STRING(ip2location, '/district') AS district,
  RPATH_STRING(ip2location, '/asn') AS asn,
  RPATH_STRING(ip2location, '/as') AS as_name,
  RPATH_STRING(ip2location, '/asdomain') AS as_domain,
  RPATH_STRING(ip2location, '/asusagetype') AS as_usage_type,
  RPATH_STRING(ip2location, '/ascidr') AS as_cidr

FROM FLOWFILE

The dynamic property name becomes a relationship named:

enriched_csv

Step 21: Add the PutFile processor #

Add another processor:

PutFile

Configure it:

Directory:

/var/lib/nifi-data/out

Conflict Resolution Strategy:

replace

Step 22: Connect the processors #

Create these connections:

Processors connection

For a simple tutorial, auto-terminate unused relationships such as:

IP2LocationEnrichRecord failure

QueryRecord original

QueryRecord failure

PutFile success

PutFile failure

For production, route failure relationships to a separate error-handling flow.

Step 23: Run the flow #

Start the processors in this order:

PutFile

QueryRecord

IP2LocationEnrichRecord

GetFile

Wait a few seconds, then check the output directory:

💻 
sudo ls -lh /var/lib/nifi-data/out
sudo cat /var/lib/nifi-data/out/orders.csv

The output should look similar to this:

orders.csv 
order_id,ip,amount,ip2location_status,country_code,country_name,region_name,city_name,latitude,longitude,zip_code,time_zone,isp,domain,net_speed,idd_code,area_code,weather_station_code,weather_station_name,mcc,mnc,mobile_brand,elevation,usage_type,address_type,category,district,asn,as_name,as_domain,as_usage_type,as_cidr
A1001,8.8.8.8,19.90,OK,US,United States of America,California,Mountain View,37.386051177978516,-122.08384704589844,94035,-07:00,Google LLC,google.com,T1,1,650,USCA0746,Mountain View,,,,32.0,DCH,A,IAB19-11,Santa Clara County,15169,Google LLC,google.com,DCH,8.8.8.0/24
A1002,1.1.1.1,29.90,OK,AU,Australia,Queensland,Brisbane,-27.467540740966797,153.02809143066406,4000,+10:00,APNIC and CloudFlare DNS Resolver Project,cloudflare.com,T1,61,07,ASXX0016,Brisbane,,,,16.0,CDN,A,IAB19-11,Brisbane,13335,CloudFlare Inc,cloudflare.com,CDN,1.1.1.0/24
A1003,208.67.222.222,39.90,OK,US,United States of America,California,San Jose,37.33938980102539,-121.89495849609375,95134,-07:00,Cisco OpenDNS LLC,opendns.com,T1,1,408,USCA0993,San Jose,,,,24.0,CDN,A,IAB19-11,Santa Clara County,36692,Cisco OpenDNS LLC,opendns.com,CDN,208.67.222.0/24
A1004,invalid_ip,9.90,INVALID_IP_ADDRESS,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Actual values depend on your DB26 BIN file version.

The invalid IP row should still appear because we configured:

Non-OK Lookup Strategy = Keep Record

Its ip2location_status may show a non-OK status such as:

INVALID_IP_ADDRESS

Conclusion #

In this tutorial, we installed Apache NiFi on Debian 13, configured JAVA_HOME correctly for the NiFi systemd service, placed NiFi behind Caddy for trusted HTTPS access, installed the IP2Location NiFi Record Bundle, configured an IP2Location DB26 BIN file, and built a CSV enrichment flow.

The final pipeline reads CSV input, performs local IP2Location DB26 lookups, flattens the geolocation fields, and writes the enriched result as a CSV file.

There are quite a few steps to follow but we’ve broken them down into easily digestible parts so that you have a clear idea of what needs to be done. Now, you can adapt the flow to fit your production needs.

Updated on July 10, 2026
Apache IP2Location
Scroll to Top