Internet2 Network- Visible Backbone Raw Data Access

There are two ways to access the raw XML data collected from the routers. There's a simple HTML browsing interface and there's a programatic interface using SOAP and CGI.

You'll need to know how the data is stored for either access method. There's a directory structure based on the time and and date the data was collected. The top level is the year. Next comes the month, day, hour, and minutes the data was collected. The files in /2003/4/21/13/10 were collected April 21 2003, at 1:10 pm. Each file is compressed with GZIP.

Once you have the data you'll need to parse it. It was stored as an array with each element being the response from one of the routers. The array was stored using the perl Storable module. I like XML::Simple (and then Data::Dumper). Here's an example of how you could access the data:

# Read in the data structure stored by the MCP.
%rawStrings = %{retrieve($dataFileName)};
for($i=0;$i<$count;$i++) {
        $hash_ref[$i]= XMLin($rawStrings{$ipList[$i]);
}
$ipList is an array of router names. You can find the current set in abilene.rtr. $count is the number of elements in the array.

Raw HTML Data Access

SOAP/CGI Access

You can access the stored data from an application using SOAP and/or an HTTP library. The following small perl app shows a SOAP interface which allows you to browse the directory structure: 
#!/usr/bin/perl

  use SOAP::Lite;
  use Data::Dumper;

  $soapResult = SOAP::Lite
    -> uri('http://vn.grnoc.iu.edu/VN')
    -> proxy('http://vn.grnoc.iu.edu/cgi-bin/vn-soap.cgi')
    -> list($ARGV[0])
    -> result;


foreach $entry  (@$soapResult) {
        print $entry,"\n";
}

Once you know the path to the data you can fetch the file with an HTTP library: 

#!/usr/bin/perl

use LWP::Simple;

$file = get("http://vn.grnoc.iu.edu/xml" . $ARGV[0]);

open(FILE11,"> data.gz");
print FILE11 $file;
sleep 2;
close(FILE11);

Finally, here's an example showing the two small apps running: 

[gcbrowni@loadrunner soap]$ ./soapc.pl /
2003
[gcbrowni@loadrunner soap]$ ./soapc.pl /2003
3
4
[gcbrowni@loadrunner soap]$ ./soapc.pl /2003/4
1
2
3
19
20
21
[gcbrowni@loadrunner soap]$ ./soapc.pl /2003/4/21
10
11
12
13
[gcbrowni@loadrunner soap]$ ./soapc.pl /2003/4/21/13
10
[gcbrowni@loadrunner soap]$ ./soapc.pl /2003/4/21/13/10
show_interfaces.gz
show_ipv6_neighbors.gz
show_isis_adjacency_extensive.gz
show_isis_hostname.gz
show_isis_interface.gz
show_isis_routes.gz
show_isis_spf_log.gz
show_isis_statistics.gz
show_mld_group.gz
show_mld_interface.gz
show_mld_statistics.gz
show_mpls_cspf.gz
show_mpls_interface.gz
abilene.rtr.gz
[gcbrowni@loadrunner soap]$ ./soapf.pl /2003/4/21/13/10/show_isis_routes.gz
[gcbrowni@loadrunner soap]$


spectrum