How Perfect Search is Unique

How is Perfect Search Different than Other Enterprise Search Vendors?

1.Our Sizzling Speed is a Result of Software Innovation.

Most improvements to indexing and software speed have come from hardware advances. Some companies are bringing back massive servers that almost resemble the mini mainframes of a few decades ago in order to have the computing power to handle demanding processes such as indexing and querying massive data sets, or calculating complex joins across multiple tables.

Perfect Search has invented a new way to do the core processes of search. This patented innovation is more efficient; it dramatically slashes the horsepower required to create an index or to execute a query. This means that Perfect Search is lightning fast even on commodity hardware, and will perform even better on high end systems.

2.Disk-based Index

Most traditional search engines work hard to have their index cached in RAM. As long as the index fits in RAM, performance is adequate. But if the size of the index exceeds the amount of RAM, the query has to access the index stored on the hard disk.

The time to read a record on a disk drive can dominate the time needed to compare keys once the record is available. The time to read a record from a disk drive involves a seek time and a rotational delay. The seek time may be 0 to 20 or more milliseconds, and the rotational delay averages about 1/2 the rotation period. For a 7200 RPM drive, the rotation period is 8.333 milliseconds. For a drive such as the Seagate ST3500320NS, the track-to-track seek time is 0.8 milliseconds and the average reading seek time is 8.5 milliseconds.

For traditional search engines, query speed performance dramatically drops if the query has to access the index stored on the disk drive. For this purpose, traditional search engines will recommend additional servers to allow them to partition their index across the additional RAM available. For example, assume a commodity server has 8 GBs of RAM. Traditional search engines will allow an index up to 16 GBs to be stored on this server, assuming that only half of the index will need to be cached at any one time. For an index of 160 GBs, then there would need to be 10 commodity servers, each handling a 1/10th partition of the 160 GB master index.

Because Perfect Search’s patented technology approaches the core process of search differently, Perfect Search is able to store the index on disk and still have zippy performance that is comparable to or better than traditional search. For super-charged speed, Perfect Search can also cache the index in RAM and have orders of magnitude faster query times.

Perfect Search's patented vortex search approach minimizes seeks, thus reducing the cost of each server and delivering millisecond response time using commodity hard drives.

A disk-based index also gives the ability to scale using the lower cost disk drives that have much greater capacity than the more expensive RAM. In the above example, the 160 GB index would store easily on an average disk drive, requiring only 1 commodity server, rather than the 10 servers that would be required by those search engines that require that their index be stored in cache.

3.Elision

To the inverted indexes and b-trees of traditional search , Perfect Search adds an important elision enhancement. While watching a combine slice straw and chaff to separate it from wheat, Dillon Inouye ( one of our founders) had an epiphany about how to eliminate the straw and chaff in huge amounts of data, leaving small molecules of relevant information.

Dillon discussed the idea with Ron Millett, Perfect Search's Chief Scientist. Ron saw how an earlier solution called hyperspace 1 that he had invented could be enhanced to another level. Ron Millett said, “At that moment, I understood that focusing on elision of the irrelevant hierarchies, combined with a mathematical and semantic method, would enable the processing of the growing deluge of information with a fraction of the traditional overhead. Dillon and I both then realized that the efficiency of this approach could be quantum improvement in both speed of indexing and also speed of retrieval in a search system, giving a higher quality results and using far less computer resources.”

4.A Molecular Approach

Without losing source meaning, the Perfect Search system reduces information to a hierarchy of indexes and accelerators-- small bundles of molecules arranged in “vortexes”. Each can “spin” a query to the precise information matching the user's query. The system then “unwraps” the bundles of information molecules and delivers a relevant, on-target answer.

5.Avoiding Runaway Index Size

Historically, one of the ways to increase query speed has been to increase the number of patterns in the index. This allows for faster queries, but also increases the size of the index. If you really push the number of patterns, you can have a combinatorial explosion, resulting in an index that is many times larger than the original data set. Search engines have had to walk a fine line between having an index that is complex enough to allow for adequate query performance but not being too big, exceeding the memory available in cache.

Perfect Search rewrites this tradeoff. The molecular approach allows blistering index speed, sizzling queries, and modest index size, all at the same time.

6.Hyper-Federation

Commonly, federated search acts as a portal delivering a roll-up of “federated content.” A query is transformed into the syntax of other retrieval systems, issued, and then collated into a combined result list and sent to the user's browser for review.

This saves the user the time and bother of sending a query to multiple systems one at a time. However, each system responds differently, and traditional federating systems may exhibit some sluggish response times.

This may be addressed by adding hardware and implementing more aggressive result caching. Unfortunately, however, Using multiple servers and then scaling them when performance degrades is too expensive, too complicated, and too unpredictable for most organizations.

The problem is exacerbated by disappearing documents, sometimes called “broken connectors.” An index from a remote system may point to a document that has been deleted or is no longer available, and users do not understand a link without the source content.

The bottom line is that in most deployments, federated search is an unhappy, lackluster compromise.

Perfect Search and Hyper-Federation

Hyper-federation deals with the problem of information retrieval in a fundamentally new way. The idea behind hyper-federation is to tackle the problems of the amount of information, the different types of data, and the cost of infrastructure head on.

Hyper-federation processes content from multiple sources and creates a single, optimized index. Source data can be retained in a repository or in the search system itself. The low cost of storage and advanced content processing eliminates the need for time-consuming round trips between the user and the source.

Perfect Search has developed a method that permits indexing multiple, sometimes geographically separate databases with a single search engine. The innovations at Perfect Search permit high-speed retrieval, drastically reduced hardware footprints, and extensive customization.

Perfect Search is one of the few organizations able to handle billions of documents and other instances of digital information using two or three servers instead of racks of cutting-edge devices.

7.Additional Query Terms Can Speed-up Query Speeds

As Perfect Search molecules are processed, adding additional search terms often actually speeds up the search. This especially works when one of the added terms is a low frequency term. With traditional search engines, adding more terms almost always slows down the query. Perfect Search’s ability also enables personalized and context sensitive “invisible” search terms to be included in the search. Term proximity, metadata fields, and key document paragraphs can all receive specialized relevance weighting. Word form and database field semantic analysis adds further relevance boosts. The rapid indexing operations enable data mining preprocessing. The system also makes use of document length and date (freshness). Users may enter one or more search terms or enter longer phrases or segments of text. Fuzzied queries for structured data are also supported.