Root Zone File ? Splitting, extracting etc

[Yofie]

I have access to all the zone files, but I am not sure what to do with them after I DL the files. I have used the DRT splitter, but the files it outputs look almost like a different language.

So does anybody know a decent tool that I can break down the lists and search them how I wish with the info provided in the zone file? Like domain name... let's say length or by keyword or LLLL only etc.

I would also like to be able to search by Expire Date and Creation date etc.

I am aware of the DomainPunch.com tool, but I tried it and was not happy with it. Do you know of any other tools? Any more info on how to split and search the files would be helpful

 

[Theo]

No Windows application will do this. All the semi-free services that extract any type of usable info from the zone files, store the raw data in a database and query it based on all sorts of criteria. Most likely in Perl or PHP, definitely running on linux.

 

[jdk]

There is a Windows application I have used in the past that did just that and very quick. I will have to go thru my old emails as I have since upgraded computers and do not have it installed on my new machine.

I used it to compare two different zone files from different dates and display the differences to see which domains dropped, etc. I am sure there were more features, but that is all I used it for.

 

[Yofie]

I used it to compare two different zone files from different dates and display the differences to see which domains dropped, etc. I am sure there were more features, but that is all I used it for.

This sounds like the Dnware.com Zone File Comparer. I did check that out, but it didn't have enough "features" for me

I guess I will have to have one built. :yes:

 

[jdk]

This was freeware/shareware.

 

[accomatic]

What kind of tasks do you need to be performed over zone files?

Keep in mind that zone files contain only domains and their nameservers relationships, nothing more nothing less. If you need more data, such as whois info, IP and hosting data, expiry dates etc. you'll have to gather that data from other sources.

Zone files, especially .COM, are rather huge (1.3GB compressed), so you either need a lot of computer resources to process it or quality software written in low-level language such as C for effectiveness and speed. Putting data into conventional SQL database is not a good option either.

Send me a PM if you need custom software developed.

 

[Yofie]

What kind of tasks do you need to be performed over zone files?

Keep in mind that zone files contain only domains and their nameservers relationships, nothing more nothing less. If you need more data, such as whois info, IP and hosting data, expiry dates etc. you'll have to gather that data from other sources.

Zone files, especially .COM, are rather huge (1.3GB compressed), so you either need a lot of computer resources to process it or quality software written in low-level language such as C for effectiveness and speed. Putting data into conventional SQL database is not a good option either.

Send me a PM if you need custom software developed.

Zone files contain... Domain Name, Creation Date, Expire Date and Name Server from my understanding. Just what I need. :yes:

I have found a developer, so we will see how it goes. If this one falls threw, I can contact you

 

[accomatic]

Zone files contain... Domain Name, Creation Date, Expire Date and Name Server from my understanding.

BIND Zone Files contain just domain names and their nameservers. No creation and expiration dates to be found there, you'll have to batch query whois services to get it.

 

[Yofie]

BIND Zone Files contain just domain names and their nameservers. No creation and expiration dates to be found there, you'll have to batch query whois services to get it.

BIND?

I am pretty sure I have the Root zone files. Which should contain the Creation Date and expire date... How would they know when a domain would expire?

 

[katherine]

Here is a typical .com zone file:

;File start: 66129
; The use of the Data contained in Verisign Inc.'s aggregated
; .com, and .net top-level domain zone files (including the checksum
; files) is subject to the restrictions described in the access Agreement
; with Verisign Inc.

$ORIGIN COM.
@ IN SOA A.GTLD-SERVERS.NET. NSTLD.VERISIGN-GRS (
1208118093 ;serial
1800 ;refresh every 30 min
900 ;retry every 15 min
604800 ;expire after a week
900 ;minimum of 15 min
)
$TTL 518400
NS A.GTLD-SERVERS.NET.
NS G.GTLD-SERVERS.NET.
NS H.GTLD-SERVERS.NET.
NS C.GTLD-SERVERS.NET.
NS I.GTLD-SERVERS.NET.
NS B.GTLD-SERVERS.NET.
NS D.GTLD-SERVERS.NET.
NS L.GTLD-SERVERS.NET.
NS F.GTLD-SERVERS.NET.
NS J.GTLD-SERVERS.NET.
NS K.GTLD-SERVERS.NET.
NS E.GTLD-SERVERS.NET.
NS M.GTLD-SERVERS.NET.
$TTL 172800
ASGARD NS NS13.WORLDNIC
ASGARD NS NS14.WORLDNIC
IDU NS NS57.1AND1
IDU NS NS58.1AND1
HAL9K NS NS57.1AND1
HAL9K NS NS58.1AND1
HEDONISM NS NS1.ILIIT
HEDONISM NS NS2.ILIIT
CATAWBA NS NS2.COMPORIUM.NET.
CATAWBA NS NS4.COMPORIUM.NET.
STENSETH NS NS2.COMPORIUM.NET.
STENSETH NS NS4.COMPORIUM.NET.
CUBICULUM NS NS1.VCSWEB
CUBICULUM NS NS1.TWISTED4LIFE
CUBICULUM NS NS2.AFRAID.ORG.
CUBICULUM NS TIGGER.CUBICULUM
CUBICULUM NS NS0.VCSWEB
CUBICULUM NS NS1.EDITDNS.NET.
CUBICULUM NS NS2.EDITDNS.NET.
CUBICULUM NS NS3.EDITDNS.NET.
CUBICULUM NS NS4.EDITDNS.NET.
OFFICES NS NS1.FASTPARK.NET.
OFFICES NS NS2.FASTPARK.NET.
BWCO NS NCNOC.NCREN.NET.
BWCO NS NS2.OAR.NET.
BWCO NS REGGAE.NCREN.NET.
ARMORY NS ALDEBARAN.ARMORY
ARMORY NS NS.CHIME
ARMORY NS NS.ARMORY
CICHLID NS NS2.MYDYNDNS.ORG.
CICHLID NS NS1.MYDYNDNS.ORG.

The file begins with a header then the list of names begins. The first relevant entry is in bold:
ASGARD NS NS13.WORLDNIC
ASGARD NS NS14.WORLDNIC
It means domain ASGARD(.com) uses name servers NS13.WORLDNIC.COM & NS13.WORLDNIC.COM. There is one line per name server entry. Most names have 2 name servers so they appear twice.
To sum up the only information you will get is the list of all registered names and associated name servers. Creation and expiry dates can be queried at registry level or from registrar whois information (which is not always consistent)
PS: the domains with no name servers are missing from the zone.

That's it

PPS: Bind is DNS server software.

 

[Yofie]

Thank you for all the info!

All I can say is Darn it! I was hoping for Domain, Name Server, Creation Date and Expire date....

I wonder how the Registry knows when to "remove" a domain. I guess that's why I was thinking the creation and expire dates would be in the zone files.

 

[katherine]

The registry has the creation and expiry date in their systems, it's just that the info is not in the zone files.

 

[Yofie]

The registry has the creation and expiry date in their systems, it's just that the info is not in the zone files.

Darn them! Why take out the "good" info! :upset:

Now I have to rethink on what I want to do.....

 

[accomatic]

Darn them! Why take out the "good" info! :upset:

Now I have to rethink on what I want to do.....

Verisign publishes zone files, and that is good, because most other zones are not available at all.

If you need to know creation and expiration dates you have to:
1. parse zone file(s), extract domain names;
2. query whois servers and extract dates.

Massive whois scraping is a non-trivial task.