The Little Search Engine That Could

[thartley]

If this first set of computers work well (i.e., we can cluster then and they can crawl the Internet) then we'll be getting another batch and another rack will need to be built.

I can just imagine the electronic buzz you will be sleeping to.  Good luck!  heh! 

[Hunter]

Hey! that's a beautiful sound! That, and the whine of a modem making a connection. To bad those aren't on sound machines.

[thartley]

I love the "What is the Acorn?" graphic thingy on the homepage!  Caught my eye this evening.   

[thartley]

How did the trip to Bangkok go with the computer shopping?  I am only now catching up on the blog...

Maybe you can update us on this while you sleep?   

[KitKat]

How did the trip to Bangkok go with the computer shopping?  I am only now catching up on the blog...

Maybe you can update us on this while you sleep?   

Terri, just a suggestion, you might take a look at PK's Acorn Blog. He tells a little something about his experience with ordering computers.  It sounds like  he'll have to put the computers aside for now, until he gets this renovation completed.  

[thartley]

Yeah, I read the blog right before I posted that.  Wasnt sure if he piggybacked computer browsing/shopping with the last trip into Bangkok.... window shopping.

Didnt know if I had missed something.  Building first, computers later.  Got it.

 

[Ginafish]

I thought he went shopping to Bangkok for computers too... Hmmm.. maybe it's in the works.

I'm so glad he'll have plenty of space for them now!

[KitKat]

Yep, window shopping for computers, that's what he did. lol

Sorry, I sometimes get his e-mails mixed up with his blogs. lol

[thartley]

Yep, window shopping for computers, that's what he did. lol

Sorry, I sometimes get his e-mails mixed up with his blogs. lol

Wow!  You get really good e-mails! 

[PK]

Thanks for asking, thartley.

We went to Bangkok and finally saw the equipment we need to buy but decided to save our money until after the Learning Center is up and running. Renovation projects tend to go WAY over budget so we should probably not spend any cash on ACORN right now. So ACORN is not dead. It's just on hiatus until we have a good place to put it.

[PK]

UPDATE: Acorn's new location is wired and a few days ago we received the first shipment of hardware. After some minor tweaking we were able to get everything inside the racks and working.

Until we have a crawler ready to go the equipment is hosting Acorn's temporary site and Folding for Isara's Folding@home Team.

Visit Acorn's LIVE Site - http://acorn.isara.org

We've also figured out a better way of crawling millions of websites. We'll have more about that later.

[Jane]

Ohh la la I like the little blue man hiding under a red curtain!!! Yay- sounds like it's coming along!

[Soulessaint]

UPDATE: Acorn's new location is wired and a few days ago we received the first shipment of hardware. After some minor tweaking we were able to get everything inside the racks and working.

Until we have a crawler ready to go the equipment is hosting Acorn's temporary site and Folding for Isara's Folding@home Team.

Visit Acorn's LIVE Site - http://acorn.isara.org

We've also figured out a better way of crawling millions of websites. We'll have more about that later.

Very good, PK! Gbya

[KitKat]

Whohoo!    We now have 2 live webcams! 

Ginafish is really going to be busy! 

[thartley]

If Ginafish snaps enough pics a la webcam, she could put together another slideshow, only have them on a hard transition instead of a fade, and set the time for each pic to .5 seconds....and we could have VIDEO!!!!!


 

[ZeroG]

This is the start of an article PK sent me.

Search engines are as critical to Internet use as any other part of the network infrastructure, but they differ from other components in two important ways. First, their internal workings are secret, unlike, say, the workings of the DNS (domain name system). Second, they hold political and cultural power, as users increasingly rely on them to navigate online content.

When so many rely on services whose internals are closely guarded, the possibilities for honest mistakes, let alone abuse, are worrisome. Further, keeping search-engine algorithms secret means that further advances in the area become less likely. Much relevant research is kept behind corporate walls, and useful methods remain largely unknown.

If that doesn't motivate you, nothing will. Acorn is a better name than Nutch, however, they have the technology. But we have the spirit. This is a GREAT combination.

 

[PK]

And here's something from the end of that same article:

One type of nonprofit in particular that we hope to see is a PSE (public search engine), a search site that is as usable as any commercial one, but that operates without advertising or commercial engagement. These engines will help make good on Nutch's promise to make search results more transparent to users. Conversely, they will make for-profit engines easier to spot if they adjust rankings for commercial gain.

A PSE might get its funds through donations from users, corporations, or foundations, just as public broadcasting channels do. It's worth noting that PSEs do not need to process a huge percentage of search queries to be successful. Their existence will help ensure that search users always have a good alternative (one that doesn't exist today).

We can do this!!

[KitKat]

  This article is awesome. Just what we needed to hear. People are wanting an open source search engine and Acorn is the right one.

Acorn will need lots of help, but we can do it! Someone's going to do it, why not us?

I know Isara's Acorn will be blessed because our motive is right. 

[Soulessaint]

And here's something from the end of that same article:

One type of nonprofit in particular that we hope to see is a PSE (public search engine), a search site that is as usable as any commercial one, but that operates without advertising or commercial engagement. These engines will help make good on Nutch's promise to make search results more transparent to users. Conversely, they will make for-profit engines easier to spot if they adjust rankings for commercial gain.

A PSE might get its funds through donations from users, corporations, or foundations, just as public broadcasting channels do. It's worth noting that PSEs do not need to process a huge percentage of search queries to be successful. Their existence will help ensure that search users always have a good alternative (one that doesn't exist today).

We can do this!!

   YES, WE CAN!!!!  Gbya

[ZeroG]

I found this thread using the search engine we will use for Acorn.
It was a test, and I tried it on another site while PK gets the
the real Linux server running. I only crawled Isara.org, but it
shows real progress towards our goal.

Temporary Search of Isara

Give it a try by searching for your favorite thread.

[thartley]

Hey, that thing worked!     I searched for something that had been posted to recently and something long dead.  Didnt know if that mattered, but worked like a charm. Great stuff!

[KitKat]

  Great news, keep up the good work.  As Gina says "Isara rocks". 

[PK]

Acorn just completed a 2,000,000 URL crawl which resulted in a little over 2 million pages indexed. Sounds like a lot but it's only a small fraction of the 2 billion pages Google has. But we must start somewhere.

You can test it out here.
http://acorn.no-ip.org:8080

[PK]

Update:
For those who don't visit the Acorn Blog, here's a quick update.

Acorn is growing more and more each day. Hundreds of thousands of URLs are added daily and new features are slowly being hammered out.

- We have nearly 5 million URLs in our index with hopes of doubling that by the end of August.

- We've added a "Did you mean?" spelling feature to search results.

- If people are not satisfied with the Acorn results they can easily search Google or Yahoo, for the same query, by just clicking one link.

- On the result pages, any bad (or questionable) links can be reported by users. This will keep the index as clean as possible.

- An upgrade of our current Internet Connection has been requested. The upgrade will include a static (fixed) IP address. We are waiting to learn if our ISP will donate the upgrades.

- Thanks to Isara member, G Wiz, two of our server nodes will have a 400% upgrade in SDRAM. And the next server we build will have 1.5gb of DDR2 RAM. The more RAM Acorn has the faster it will be. Thanks, G Wiz!


The plan is to get Acorn Online by the end of August. It all depends on whether we get the static IP address. Keep your fingers crossed.
-pk

[KitKat]

  Keep on going with Acorn! Great job!