Infinispan Distribution 5.2.6.Final-redhat-2

org.infinispan.distexec.mapreduce
Class MapReduceManagerImpl

java.lang.Object
  extended by org.infinispan.distexec.mapreduce.MapReduceManagerImpl
All Implemented Interfaces:
MapReduceManager

public class MapReduceManagerImpl
extends Object
implements MapReduceManager

Default implementation of MapReduceManager.

This is an internal class, not intended to be used by clients.

Since:
5.2
Author:
Vladimir Blagojevic

Nested Class Summary
static class MapReduceManagerImpl.IntermediateCompositeKey<V>
          IntermediateCompositeKey
 
Method Summary
protected
<KIn,VIn,KOut,VOut>
Set<KOut>
combine(MapCombineCommand<KIn,VIn,KOut,VOut> mcc, org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut> collector)
           
protected
<KIn> Set<KIn>
filterLocalPrimaryOwner(Set<KIn> nodeLocalKeys, DistributionManager dm)
           
 ExecutorService getExecutorService()
          ExecutorService provided for local task execution
 void init(EmbeddedCacheManager cacheManager, CacheLoaderManager cacheLoaderManager, ExecutorService asyncTransportExecutor)
           
protected
<KIn> Set<KIn>
loadAllKeysFromCacheLoaderUsingFilter(Set<KIn> filterOutSet)
           
protected
<KIn,KOut> KOut
loadValueFromCacheLoader(KIn key)
           
protected
<KIn,VIn,KOut,VOut>
org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut>
map(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
           
<KIn,VIn,KOut,VOut>
Set<KOut>
mapAndCombineForDistributedReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
          Invoked when MapCombineCommand arrives to a target Infinispan node and returns set of intermediate keys that needs to be reduced.
<KIn,VIn,KOut,VOut>
Map<KOut,List<VOut>>
mapAndCombineForLocalReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
          Invoked when MapCombineCommand arrives to a target Infinispan node and returns map of resulting values that need to be reduced.
<T> Map<Address,List<T>>
mapKeysToNodes(DistributionManager dm, String taskId, Collection<T> keysToMap, boolean useIntermediateCompositeKey)
          Maps Map/Reduce task intermediate or input keys to nodes on Infinispan cluster
<KOut,VOut>
Map<KOut,VOut>
reduce(ReduceCommand<KOut,VOut> reduceCommand)
          Invoked when ReduceCommand arrives to a target Infinispan node.
protected  CacheLoader resolveCacheLoader()
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

init

public void init(EmbeddedCacheManager cacheManager,
                 CacheLoaderManager cacheLoaderManager,
                 ExecutorService asyncTransportExecutor)

getExecutorService

public ExecutorService getExecutorService()
Description copied from interface: MapReduceManager
ExecutorService provided for local task execution

Specified by:
getExecutorService in interface MapReduceManager
Returns:
ExecutorService for local tasks

mapAndCombineForLocalReduction

public <KIn,VIn,KOut,VOut> Map<KOut,List<VOut>> mapAndCombineForLocalReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
                                                    throws InterruptedException
Description copied from interface: MapReduceManager
Invoked when MapCombineCommand arrives to a target Infinispan node and returns map of resulting values that need to be reduced.

Unlike MapReduceManager.mapAndCombineForDistributedReduction(MapCombineCommand) implementations should return a map Map> which should be ready for reduction at master Infinispan node

Specified by:
mapAndCombineForLocalReduction in interface MapReduceManager
Parameters:
mcc - MapCombineCommand sent from MapReduceTask
Returns:
a map Map> which should be ready for reduction at master Infinispan node
Throws:
InterruptedException

mapAndCombineForDistributedReduction

public <KIn,VIn,KOut,VOut> Set<KOut> mapAndCombineForDistributedReduction(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
                                               throws InterruptedException
Description copied from interface: MapReduceManager
Invoked when MapCombineCommand arrives to a target Infinispan node and returns set of intermediate keys that needs to be reduced.

Specified by:
mapAndCombineForDistributedReduction in interface MapReduceManager
Parameters:
mcc - MapCombineCommand sent from MapReduceTask
Returns:
a set of intermediate keys to be reduced distributively
Throws:
InterruptedException

reduce

public <KOut,VOut> Map<KOut,VOut> reduce(ReduceCommand<KOut,VOut> reduceCommand)
                      throws InterruptedException
Description copied from interface: MapReduceManager
Invoked when ReduceCommand arrives to a target Infinispan node. Implementations should return a map of reduced output keys and values to be returned to invoker of MapReduceTask

Specified by:
reduce in interface MapReduceManager
Parameters:
reduceCommand - ReduceCommand sent from MapReduceTask
Returns:
map of reduced output keys and values returned to MapReduceTask
Throws:
InterruptedException

map

protected <KIn,VIn,KOut,VOut> org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut> map(MapCombineCommand<KIn,VIn,KOut,VOut> mcc)
                                                                                              throws InterruptedException
Throws:
InterruptedException

combine

protected <KIn,VIn,KOut,VOut> Set<KOut> combine(MapCombineCommand<KIn,VIn,KOut,VOut> mcc,
                                                org.infinispan.distexec.mapreduce.MapReduceManagerImpl.CollectableCollector<KOut,VOut> collector)
                     throws Exception
Throws:
Exception

loadAllKeysFromCacheLoaderUsingFilter

protected <KIn> Set<KIn> loadAllKeysFromCacheLoaderUsingFilter(Set<KIn> filterOutSet)

loadValueFromCacheLoader

protected <KIn,KOut> KOut loadValueFromCacheLoader(KIn key)

resolveCacheLoader

protected CacheLoader resolveCacheLoader()

mapKeysToNodes

public <T> Map<Address,List<T>> mapKeysToNodes(DistributionManager dm,
                                               String taskId,
                                               Collection<T> keysToMap,
                                               boolean useIntermediateCompositeKey)
Description copied from interface: MapReduceManager
Maps Map/Reduce task intermediate or input keys to nodes on Infinispan cluster

Specified by:
mapKeysToNodes in interface MapReduceManager
Parameters:
dm - distribution manager to use for locating keys on hash wheel
taskId - id of the map/reduce task
keysToMap - list of input keys to locate in the cluster
useIntermediateCompositeKey - if true use composite keys for shared intermediate cache
Returns:
map where each key is an Address in the cluster and value are the keys mapped to that Address

filterLocalPrimaryOwner

protected <KIn> Set<KIn> filterLocalPrimaryOwner(Set<KIn> nodeLocalKeys,
                                                 DistributionManager dm)

Infinispan Distribution 5.2.6.Final-redhat-2

Copyright © 2013 JBoss, a division of Red Hat. All Rights Reserved.