Building a search engine... - DNForum - Domain Sales, Domain Forum, Domain Appraisals, Domain Registrars

**DotComGod** · 03-07-2005, 05:56 PM

Does anyone know of software you can buy to create a search engine.

What I mean by that is not a script to start a search engine but software that can spider other sites on its own to build the results page and pages in the engine.

-=DCG=-

**Darren06** · 03-07-2005, 06:02 PM

Maybe this? http://www.pjltechnology.com/linkssuite.htm

**fayqing** · 03-07-2005, 06:13 PM

please try phpdig. useful spider search engineer.
maybe you should have very strong php knowledge.
i also can set up one for you, if you like.
such as gg like. but not so powerful.

should be 10,000-100,000 per realm. over this rank, it should be slow down.

maybe next end week, i can give your demo that i just created one for myself.

**DotComGod** · 03-07-2005, 06:16 PM

Quote:

Originally Posted by Darren06

Maybe this? http://www.pjltechnology.com/linkssuite.htm

Seems like it spiders certain engines and I want to build a large search engine.

But I do appreaciate the searching.

Seem like will be a high end commercial package but I cant find it.

-=DCG=-

**DNjet** · 03-10-2005, 07:26 AM

I don't know if this one is of interest.
http://www.webwizguide.info/asp/samp...ine_script.asp

Also if you go over to doneright they will custom build your needs reasonable , they quoted me $500 to build a custom SE script once but I never did the project. They do some very nice work if you look at their products.

http://www.donerightscripts.com

**stevey** · 03-10-2005, 12:18 PM

http://www.questronixsoftware.com/Pr...olsSpider.aspx try this, i just downloaded it, in the settings you can get it to just spider 1 website or you can change it to follow the links so it will basicly spider the whole net if you let it also you can set it to read the robots.txt and obay it on ppl's sites or ignore there robots.txt, you can also set it to just follow internal links :santa: hope its what your looking for

**DotComGod** · 03-10-2005, 12:36 PM

Thanks to all, will try these.

-=DCG=-

**theparrot** · 03-10-2005, 04:05 PM

Quote:

Originally Posted by DotComGod

Seems like it spiders certain engines and I want to build a large search engine.

But I do appreaciate the searching.

Seem like will be a high end commercial package but I cant find it.

-=DCG=-

How large do you want it? To do a large search engine most likely requires colocated hardware on its own interconnect and lots of RAM, diskspace and bandwidth. The market for such is pretty small, so I am not sure there would be an off the shelf package for it. At least for what I would view as a large search engine. There are some companies that sell the toolkit for making one, but it is far from an off the shelf solution. They also wind up needing tweeking as new ways to spoof and flood it are found, and to improve the ranking relivancy tools, so they are also not really set and forget types of things at this scale. All that being said, I do have a toolkit for making a search engine archived around someplace around here from some work I did like 7 years ago.

**DNGeeks** · 03-10-2005, 04:14 PM

http://searchenginewatch.com/resources/
http://searchenginewatch.com/resourc...le.php/2156591

A part of the serverwatch.com family of sites. This should give you all the resources you need.

**cyphix** · 03-19-2005, 06:34 AM

Shouldn't be that hard to build really.... sure you could get what you wanted created easy enough.

**FusionX** · 03-19-2005, 08:10 PM

For large projects, Nutch is the only viable search engine that is both opensource and really powerful. http://www.nutch.org