Tuesday, April 30, 2013

Visualize IP Connections - a FastCGI and Geolocation Solution

As a result of the Internet, computers have connections to many destinations, even while not used or when no internet browser is active. Open TCP/UDP connections can be the result of legitime background processes/applications like Skype, Google Drive, Dropbox but also can be a sign of unwanted malware or other. 

I personally have the habit to check these connections from time to time and see if nothing is out of the ordinary.

My usual procedure is to open a Terminal and do:

 netstat -an | grep "tcp\|udp"  

and go over the ip adresses, when I see a suspicious ip address/port number I use a service like http://www.ip2location.com to check the ip address origin. 

Now this is a bit of a tedious process and I wanted to make it automated in one way or another. So I decided to display that information on a Google Map with my current location as the center and lines reaching out to all locations I have open connections to.

All I needed was a URL I could submit all those 'netstat' IP adresses to and get a nice Google Map back with markers that represent the open connections.  

Something like:


Next I can auto build the URL with the actual IP addresses with a bash line as follow:

 for i in `netstat -an | grep 'tcp\|udp' | awk '{print $5}' | grep -v "*.*" | grep -v "192.168" | grep -v "127.0.0"`;do a=$a$i,;done;echo ips=$a  

And for the impatient with a Mac and Chrome, the following bash is the result and produces such a map;

 a="";for i in `netstat -an | grep 'tcp\|udp' | awk '{print $5}' | grep -v "*.*" | grep -v "192.168" | grep -v "127.0.0"`;do a=$a$i,;done;open /Applications/Google\ Chrome.app 'http://locations.keymolen.com/cgi-bin/locations.fcgi?ips='$a  

Solution: FastCGI & Geolocation Database.

A bit old school but perfectly legit for this occasion I decided to write some C++ CGI code and use the free IP Geolocation Database IP2LOCATION - Lite.

Using FastCGI in a C++ application turns out to be pretty easy.

First you need to install the FastCGI libraries and the header files, so download the latest version from http://www.fastcgi.com/ and compile & install. I used version 2.4.1 and had to add the header "#include <stdio.h>" at the top of the file libfcgi/fcgio.cpp to get it compiled on my Linux box.

My FastCGI Code looks as follow:
1:  void doCGI() {  
2:       keymolen::IPDB db;  
3:       db.load(path);  
4:       //Start the CGI loop  
5:       FCGX_Init();  
6:       while (true) {  
7:            FCGX_Request* request = new FCGX_Request;  
8:            FCGX_InitRequest(request, 0, 0);  
9:            int rc = FCGX_Accept_r(request);  
10:           if (rc < 0) {  
11:                 std::ostringstream os;  
12:                 os << "Content-type: text/html\n";  
13:                 os << "\n";  
14:                 os << "<b>Error! </b>";  
15:                 os << rc;  
16:                 os << "\n\n\n";  
17:                 std::string resp = os.str();  
18:                 FCGX_PutS(resp.c_str(), request->out);  
19:                 FCGX_FFlush(request->out);  
20:                 FCGX_Finish_r(request);  
21:                 delete request;  
22:                 usleep(100);  
23:                 continue;  
24:            }  
25:            const char* method = FCGX_GetParam("REQUEST_METHOD", request->envp);  
26:            const char* path_info = FCGX_GetParam("PATH_INFO", request->envp);  
27:            const char* cl = FCGX_GetParam("CONTENT_LENGTH", request->envp);  
28:            const char* req_full = FCGX_GetParam("QUERY_STRING", request->envp);  
29:            const char* remoteip = FCGX_GetParam("REMOTE_ADDR", request->envp);  
30:            std::vector<std::string> ips;  
31:            ips.push_back(remoteip);  
32:            if (req_full) {  
33:                 const char *req = strstr(req_full, "ips=");  
34:                 if (req != 0) {  
35:                      req += 4;  
36:                      parseIps(ips, req);  
37:                 }  
38:            }  
39:            std::string resp = makeHTML(db, ips);  
40:            FCGX_PutS(resp.c_str(), request->out);  
41:            FCGX_FFlush(request->out);  
42:            FCGX_Finish_r(request);  
43:            delete request;  
44:       }  
45:  }  

  • Line 5 
    • Init FastCGI, have to be done only once.
  • Line 7,8,9
    • Prepare a new incoming http request and wait until it arrives (line 9)
  • Line 10-24
    • Error handling
  • Line 25-29
    • FastCGI provide a set of parameters, I only used "QUERY_STRING" and "REMOTE_ADDRESS", I left the others in as example.
  • Line 30-38
    • Parse the Query String wich, by our own invented convention, has the following format: 
      • /locations.fcgi?ips=<ip 1>,<ip 2>, ..., <ip n>
  • Line 39
    • Generate the HTML code (the code that contains the MAP)
  • Line 40
    • Write the HTML back to the browser
  • Line 41-43
    • Finalize the request 

Another thing you have to keep in mind if working with CGI's is that you are responsible for the HTTP headers.

See the generation of the HTML code:
1:  std::string makeHTML(keymolen::IPDB &db, std::vector<std::string> &ips)  
2:  {  
3:     std::ostringstream os;  
4:     os << "Content-type: text/html\n";  
5:     os << "\n";  
6:     os << "<!DOCTYPE html>";  
7:     os << "<html>";  

Notice the Content-type: text/html and the two new lines (\n).

The above CGI code reads and parse the HTTP requests and send HTML code as an answer.

Our example reads a comma separated set of IP addresses and puts them in the vector std::vector<std::string> ips; Now we need to do a lookup in the IP Geolocation database in order to find the latitude and longitude for every IP address so we can place that neat markers on a Google Map.

The ip2location-lite database is a comma separated value text file and has 1.681.819 lines (records). 

We have several options here, and I identified a few:
  1. Do a search in the text file for every lookup
  2. Load the file in a SQL database and query per request
  3. Load and keep the whole file in Memory
I went for option 3, it is imo a fast and least complicated way to handle the search. So I decided to load and parse the file at the first request and put all in a std::map<unsigned int, IPDBRecord*> where the unsigned int represents the IP address. 

Load the DB in memory:
1:       //"16777216","16777471","AU","AUSTRALIA","QUEENSLAND","SOUTH BRISBANE","-27.483330","153.016670","-","+10:00"  
2:       void IPDB::load(std::string path)  
3:       {  
4:            std::ifstream db_file;  
5:            db_file.open(path.c_str(), std::ios::in);  
6:            std::string line;  
7:            while (!db_file.eof())  
8:            {  
9:                 std::getline(db_file, line);  
10:                 //Process the line  
11:                 IPDBRecord* r = new IPDBRecord();  
12:                 r->lon = atol(parseString(line.c_str(), 7).c_str());  
13:                 r->lat = atol(parseString(line.c_str(), 8).c_str());  
14:                 _db[atoi(parseString(line.c_str(), 1).c_str())] = r;  
15:            }  
16:            db_file.close();  
17:       }  

The code reads line by line and parses the longitude and latitude out of the line at position 7 and 8. The values are stored in the map by: _db[atoi(parseString(line.c_str(), 1).c_str())] = r; where the 1st field (the start of the IP range) as the Key.

Usage & live demo:

I did put the FastCGI online and if you have a MAC and Chrome installed, feel free to test it with the following command from Terminal:

 a="";for i in `netstat -an | grep 'tcp\|udp' | awk '{print $5}' | grep -v "*.*" | grep -v "192.168" | grep -v "127.0.0"`;do a=$a$i,;done;open /Applications/Google\ Chrome.app 'http://locations.keymolen.com/cgi-bin/locations.fcgi?ips='$a  

Chrome (if installed at: /Applications/Google\ Chrome.app, otherwise change the path) should open and a map should load and show you your current connections.

For linux or others, change the command to your needs; at the end a browser need to load the following URL:


Will show a connection between you and the Google and Microsoft website

The full sources can be downloaded at:  https://github.com/brunokeymolen/locations


  1. Very Nice Work Bruno. Thx. You just need to change in ipdb.cpp this:
    69 r->lon = atol(parseString(line.c_str(), 7).c_str());
    70 r->lat = atol(parseString(line.c_str(), 8).c_str());


    69 r->lon = atof(parseString(line.c_str(), 7).c_str());
    70 r->lat = atof(parseString(line.c_str(), 8).c_str());
    because atol coverts to long but to get precision we need double so is better atof.

    But great work ..


  2. Indeed, thats a mistake.
    good catch, thanks!