Co-allocation with communication considerations in multi-cluster systems

Abstract

Processor co-allocation can be of performance benefit. This is because breaking jobs into components reduces overall cluster fragmentation. However, the slower inter-cluster communication links increase job execution times. This leads to performance deterioration which can make co-allocation unviable. We use intra-cluster to inter-cluster communication speed ratio and job communication intensity to model the job execution time penalty due to co-allocation. We then study viability of co-allocation in selected job and system based instances. We also study performance variation with selected job stream parameters. We observe that co-allocation is viable so long as the execution time penalty caused is relatively low. We also observe that the negative performance effect due to co-allocation is felt by the entire job stream rather than only the (few) co-allocated jobs. Finally, we observe that for every value of communication time penalty, there is a job size s*, where if all jobs whose size is greater than s* are co-allocated, we get the best performance.