Hostname: page-component-74656f9884-j7cmm Total loading time: 0 Render date: 2026-11-10T05:53:29.031Z Has data issue: false hasContentIssue false

Partitioning strategies for distributed association rule mining

Published online by Cambridge University Press:  07 July 2006

FRANS COENEN
Affiliation:
Department of Computer Science, University of Liverpool, Liverpool L69 3BX, UK. E-mail: frans@csc.liv.ac.uk, phl@csc.liv.ac.uk
PAUL LENG
Affiliation:
Department of Computer Science, University of Liverpool, Liverpool L69 3BX, UK. E-mail: frans@csc.liv.ac.uk, phl@csc.liv.ac.uk
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

In this paper a number of alternative strategies for distributed/parallel association rule mining are investigated. The methods examined make use of a data structure, the T-tree, introduced previously by the authors as a structure for organizing sets of attributes for which support is being counted. We consider six different approaches, representing different ways of parallelizing the basic Apriori-T algorithm that we use. The methods focus on different mechanisms for partitioning the data between processes, and for reducing the message-passing overhead. Both ‘horizontal’ (data distribution) and ‘vertical’ (candidate distribution) partitioning strategies are considered, including a vertical partitioning algorithm (DATA-VP) which we have developed to exploit the structure of the T-tree. We present experimental results examining the performance of the methods in implementations using JavaSpaces. We conclude that in a JavaSpaces environment, candidate distribution strategies offer better performance than those that distribute the original dataset, because of the lower messaging overhead, and the DATA-VP algorithm produced results that are especially encouraging.

Information

Type
Research Article
Copyright
© 2006 Cambridge University Press