A programmer wants to take on Google Search, starting with two PCs that sit in his laundry room. Ryan Pearce, who worked in both enterprise software and game development roles, was curious about what it takes to build a search engine. According to Fast Company, Pearce built two search engines: Searcha Page and its incognito variant, Seek Ninja. These two engines both have a database with over 2 billion entries. Although it’s less than half a percent of Google’s 400-billion-strong search index, it’s still a considerable number, and he expects to double it in the next six months or so.
This self-hosted search engine sits in Pearce’s home — but instead of going with a low-powered Raspberry Pi or a mini-PC that most people use when self-hosting, he opted for a used 32-core AMD EPYC 7532, which was one of the best workstation CPUs when it launched in 2020. After all, if you want to create a massive search engine, you’d need something punchier than an entry-level CPU. Nevertheless, since that processor is already five years old at the time of writing, you can now easily get a used copy for less than $200. “I could have gotten another chip for the same price, which would have had twice as many threads, but it would have produced too much heat,” the programmer told Fast Company.
It’s for this reason that the two servers sit inside his laundry room. Pearce initially had them installed in his bedroom, but the heat generated by the system prevented him from sleeping comfortably. So, he moved them to his utility room, with one device sitting on a ledge, and the other supporting unit propped on a couple of small stools. The two devices are plugged into an extension cord, and the network cable that connects them to the internet is routed through a hole he drilled into the wall. There was also a makeshift vent that allowed the heat generated by the machines to escape from his home. “The heat hasn’t been absolutely terrible,” Pearce said to the publication. “But if the door is closed for too long, it is a problem.”
The secret to how one programmer can run a massive search engine is AI — but it’s not the kind of AI that most people would think of. Pearce does not employ front-facing AI tools that summarize search results and do the thinking for you. Instead, he uses machine learning algorithms to expand the keywords used in the query and help understand the user's context. This enables search engines to deliver relevant results with a fraction of the resources that Google has.
Pearce said that he’s building his search engine piece by piece, and that he’s already written around 150,000 lines of code. However, he says that he’s done so much more than that, and that he has iterated over 500,000 lines of code. Most of these changes were made to reduce the code’s reliance on large language models and make it work without relying on AI. This technique enabled him to create a complex system and then lock in modules that work by removing the variable of AI.
Ryan said that he’s considering moving the search engine out of his home and into a data center-like facility. However, since he dislikes cloud services, it will likely be at a location near his house, where he can easily access the servers and work on them on-site as necessary. But, in the meantime, Pearce said that he’s doing affiliate-style advertising to help him achieve that goal.
Follow Tom's Hardware on Google News, or add us as a preferred source, to get our up-to-date news, analysis, and reviews in your feeds. Make sure to click the Follow button!
Get Tom's Hardware's best news and in-depth reviews, straight to your inbox.