.com Zone File Query - http://www.ZFbot.com

[kengreenwood]

That's my point though. Focusing on specific nameservers isn't an accurate model. For example: Simple research would reveal that domains pointed to ns51.domaincontrol.com are parked with GoDaddy.com. But that's not the case with 100% certainty. I did a lot of testing and found plenty of legitimate websites that are pointed to that nameserver. I know that I can use the parking name servers for the clear cut parking sites like sedo, namedrive, etc... but it's the companies like 1and1, godaddy, etc that do parking but don't have clear cut rules for how they park domains

[jmcc]

Originally Posted by kengreenwood

That's my point though. Focusing on specific nameservers isn't an accurate model. For example: Simple research would reveal that domains pointed to ns51.domaincontrol.com are parked with GoDaddy.com. But that's not the case with 100% certainty. I did a lot of testing and found plenty of legitimate websites that are pointed to that nameserver. I know that I can use the parking name servers for the clear cut parking sites like sedo, namedrive, etc... but it's the companies like 1and1, godaddy, etc that do parking but don't have clear cut rules for how they park domains

Beyond the main parking companies, it comes down to crafting a regexp for the specific advert feeds. The set of regexps also has to be continually updated. From experience, it involves reading through a lot of HTML and looking for common fragments of code that indicate a potentially parked page and then confirming with a browser. GD tends to have a specific parking string, often relating to the parker's account, as do many of the larger hosters. It is the smaller ones that are the problem and take up the most time. The parked pages on the large hosters where the hoster itself is parking them tend to use a common feed and format. Then there are the microsites that obfuscate the advert feeds - they really make things interesting. However if you approach this on a page by page process, it is extremely difficult. Like breaking a code, you need a decent bit of ciphertext to work on first.

Regards...jmcc

[aZooZa]

jmcc said:

"This is where it stops being about simple domain and DNS problems and enters the realm of search engine algorithm development."

Ken, this is quite right. You're going to have to investigate each site which will be a huge task. By the time you have analysed 70 million .com sites, the data will probably be out-of-date. If you have a large infrastructure whereby you could split the load, you might find it viable, but you're going to need quite a few machines; i.e. CPU time, RAM and bandwidth.

I know you're doing a little preprocessing with Unix. This would be the time to do this as you can use 'lynx --dump' and then parse the output with grep and regexp. You'll need a set of rules for each parking site to match against, and again, this is going to be resource intensive.

I'm being a little simplistic here but I do believe you're at a watershed if you want to add accurate parking stats without staying with simple NS-based assumptions.

[sdsinc]

Many people park domains but use their own servers so they can park the name and still receive mail for the domain name. By resolving domain names to IP addresses you could determine if the IP belongs to a parking provider ie Sedo etc.

[HomerJ]

I'm almost done with the portfolio search - It's going to be a major upgrade and it will allow users to load up their entire portfolio of domains and get a complete result set of matches - begins with, ends with and exact.

absolutely incredible! my hat's off to you


just a quick note though: not sure if I am the only one but I seem to be experiencing some trouble with the export function. If I click the export and choose to open the file with excel, my download window pops up and it says 'starting' but the download bar never shows any progress and it kinda just freezes.

when I choose to save file, i save to my desktop then when i click to open i get the following popup:

The file you are trying to open, 'zfbot - domain export.xls', is in a different format than specified by the file extension. Verify that the file is not corrupted and is from a trusted source before opening the file. Do you want to open the file now?

if I click 'yes' then the file opens, but still this is a bit odd

[Acro]

Originally Posted by sdsinc

Many people park domains but use their own servers so they can park the name and still receive mail for the domain name. By resolving domain names to IP addresses you could determine if the IP belongs to a parking provider ie Sedo etc.

Unfortunately this is not fool-proof because domains that are hosted for the purpose of email, stats etc. and forwarded to a parking page, resolve to the hosting server's IP.

[sdsinc]

Right, this method is for domains where the A record is set to the parking provider, not forwarded domains.

[kengreenwood]

OK - I loaded up the change. I have a secondary table within the zfbot database that holds the nameservers that are known to be used for parking websites. Here's the list I have... (I don't have the extension on it cause I didn't need it and I wanted to conserve some space)

NS.BUYDOMAINS
NS1.BUYDOMAINS
NS1.THIS-DOMAIN-FOR-SALE
NS2.BUYDOMAINS
NS2.THIS-DOMAIN-FOR-SALE
NS53.DOMAINCONTROL.COM
NS54.DOMAINCONTROL.COM
NS1.DSREDIRECTION
NS2.DSREDIRECTION
NS1.DOTZUP
NS2.DOTZUP
NS1.DYNADOT
NS2.DYNADOT
NS1.FABULOUS
NS2.FABULOUS
NS1.FASTPARK
NS2.FASTPARK
NS1.GOLDKEY
NS2.GOLDKEY
NS1.HITFARM
NS2.HITFARM
NS1.HOSTINGNET
NS2.HOSTINGNET
NS.IPARKING
NS2.IPARKING
NS1.PARKED
NS2.PARKED
NS1.PARKINGPANEL
NS2.PARKINGPANEL
NS1.PARKINGSPA
NS2.PARKINGSPA
NS1.PARKLOGIC
NS1.EXPIREDDOMAINS
NS1.SEDOPARKING
NS2.SEDOPARKING
NS1.SMARTNAME
NS2.SMARTNAME
NS1.TRAFFICZ
NS2.TRAFFICZ
PARKING1.MDNSSERVICE
NS1.WHYPARK

I'll keep adding to the list when I find more culprits... Here's an example query that shows parked domains starting with 'dnforum' ... quite a few actually...

http://www.zfbot.com/#/search=dnforu...=1;part=0;go=f

[EG.domains]

Bodis:

NS1.BODIS
NS2.BODIS

[james2002]

I visited your site and it is great.

Good work.

[kengreenwood]

Originally Posted by EG.domains

Bodis:

NS1.BODIS
NS2.BODIS

Thanks for the tip on that... keep 'em coming. I added a bunch more myself.

Originally Posted by james2002

I visited your site and it is great.

Good work.

Thanks James...

[kengreenwood]

Some updates - I got rid of the chatroom for now... was getting in the way and no one was using it... just fluff anyway.

If you uncheck the "View Domain Stats" checkbox on the upper right, you will see a couple new columns show up in the data grid - hosting company and primary name server.

Nice info to have...

Here's an example (using the deep linking and auto-run flags in the URL... pretty cool to be able to do stuff like this in Flash):

http://www.zfbot.com/#/search=dnforu...0;stats=f;go=t

[kengreenwood]

Added .name domains - another 250 thousand domains. Just over 106 million in total now.

Now have .com, .net, .org, .biz, .us, .mobi, .name ... just waiting on response about .info

[Albert Tai]

Hey ken could you reply to my PM

[kengreenwood]

Originally Posted by Albert Tai

Hey ken could you reply to my PM

Just sent some info to you Al... hope all is well.

[WeBuyThe.Com]

Hi everyone - I've been trying to reach ken greenwood for a while and now his voice mail is full. Does anyone here know how can reach him? Thanks

[aZooZa]

Originally Posted by david261

It's nice one. what you think about 2gbhosting.com

Spam.

[kengreenwood]

Oooooook. Finally... I've taken the time to create the registration/portfolio match. You can now register, log on, add your domains one by one or in bulk.... and the system will match the newly added domains every half hour and the existing domains once a day. I'm working on creating email alerts for specific queries... No one but you can see your domains/matches so you don't need to worry about security. Make sure you check your spam email folder as the registration email may end up there... Here are some snapshots:







The functionality is pretty self explanatory but if anyone needs assistance, let me know.

Keep in mind that I may end up turning this additional functionality into a pay for subscription/service at some point so use it now while it's free!

One thing to note - if you have a large portfolio or some short highly valuable key word domains, the initial run will end up creating a very large number of matches (ie... i have writers.us and it returns almost 10,000 matches on it's own) ... i'm still tuning up the query that pulls those back .... those with large match counts will see some lengthy query runs until i'm done tuning. Just make sure you use the filters if you're gonna pull back "All Matches". I added a limit of 50,000 for the match results for the time being...

Also - here is an excel macro that cleans up and adds subtotals and hyperlinks to the results from the portfolio export:

http://www.zfbot.com/ZFBOT.bas

Download it and import it into excel... after opening the spreadsheet, hit SHIFT-X to run it.

[Acro]

The killer app for 2009 is packed with features beyond any imagination. You should definitely make the extra service for pay for portfolios - say - over 25 domains.

[Johnn]

Originally Posted by david261

It's nice one. what you think about 2gbhosting.com

Have a nice vacation and stop spamming or your account will be terminated.