(Sync|Async)+ MPI Search Engines

Abstract

We propose a parallel MPI search engine that is capable of automatically switching between asynchronous message passing and bulk-synchronous message passing modes of operation. When the ob- served query traffic is small or moderate the standard multiple mana- gers/workers thread based model of message passing is applied for pro- cessing the queries. However, when the query traffic increases a round- robin based approach is applied in order to prevent from unstable behav- ior coming from queries demanding the use of a large amount of resources in computation, communication and disk accesses. This is achieved by (i) a suitable object-oriented multi-threaded MPI software design and (ii) an “atomic” organization of the query processing which allows the use of a novel control strategy that decides the proper mode of operation.