For Sale processing zone files

[Joe826]

Anyone actually done this? I know most people say exody or an alike service is the easiest way to go, and i'm sure it is, but I got access to the zone files I think it sounds fun. Anyone had any experience with this?

 

[Joe826]

How long does it take you to parse through an update? I'm thinking of just doing it for fun. But you're right, is the info you get from the fresh logs much different from what you can get from exody? Does the time difference really matter that much? Maybe i'll have to just check it out for myself .

Thanks for the reply.

 

[BobBob is verified member.]

I have done this daily for the last 2 years. I wrote a C++ program to parse the files. I can parse the org zone file in about 4 minutes, the net in about 6 minutes and the com in about 10-12 minutes (my machine is a P3-1100 MHz with 1500 MB (1.5 GB) ram). Then I import into a foxpro database. This takes the longest time. After the data are in the database, the rest is cake.


-Bob

 

[DotDuDe]

linux + mysql + perl dbi and perl scripts.. I would say the database checking for the dropped names takes the most of the time.. I will consider to re-writing it to C/C++ if I will have a good chance of catching drops, I will probably go for it:

Linux, MySQL, MySQL C++ API and C or Perl scripts

 

[netznutz]

I filter most of the zonefile junk during the first
read using certain patterns (<4 dashes, <30chars...).
Then I 'diff' against 'yesterdays' zonefile and get
all newly dopped domains.

Am I right thinking that now that RGP is in effect,
it is sufficient to get the zonefiles once in 30 days
as long as I only look after 'manual snappable'
domains?

 

[clemzonguy]

Isn't processing the zone file in vain now if there will be no more "unexpected drops" due to the mandatory RGP and deletion periods?

 

[DotDuDe]

but still you need to get the names that are about to drop anyway....

 

[morel]

What information does the the zone file contain? Specifically, does the zone file give the expiry date of each domain? It would be really cool if you could sort all domains in existence by expiry date. But I think someone once said that that information is not available, though that was awhile ago... perhaps it has been changed along with the new whois format?

 

[Dale Hubbard]

Can anyone help me with access to the zone files? I need them only once every 30 days and can grab them via ftp and big pipe. Am willing to make token payment every month. Only need com and net. Thanks!

 

[miloman]

If you are going to process the zonefiles:

- They update twice a day
- Never the same time of the day, within certain ranges
- Learn a few unix commands and you can get the zonefile processing in minutes.

Make your script do this:
- Check for file modification
- Download files
- Compare yesterday zones, to today
- Save results
- Flatfiles are the fastest way.

* Learn the structure of the zonefiles *
zcat com.zone.gz | more

Good luck