Managing community membership information in a small-world grid

Abstract

As the Grid matures the problem of resource discovery across communities, where resources now include computational services, is becoming more critical. The number of resources available on a world-wide grid is set to grow exponentially in much the same way as the number of static web pages on the WWW. We observe that the world-wide resource discovery problem can be modelled as a slowly evolving very-large sparse-matrix where individual matrix elements represent nodes’ knowledge of one another. Blocks in the matrix arise where nodes offer more than one service. Blocking effects also arise in the identification of sub-communities in the Grid. The linear algebra community has long been aware of suitable representations of large, sparse matrices. However, matrices the size of the world-wide grid potentially number in the billions, making dense solutions completely intractable. Distributed nodes will not necessarily have the storage capacity to store the addresses of any significant percentage of the available resources. We discuss ways of modelling this problem in the regime of a slowly changing service base including phenomena such as percolating networks and small-world network effects.