| 程序包 | 说明 |
|---|---|
| org.apache.hadoop.hdfs.protocol | |
| org.apache.hadoop.hdfs.server.namenode | |
| org.apache.hadoop.mapred |
A software framework for easily writing applications which process vast
amounts of data (multi-terabyte data-sets) parallelly on large clusters
(thousands of nodes) built of commodity hardware in a reliable, fault-tolerant
manner.
|
| org.apache.hadoop.net |
Network-related classes.
|
| 限定符和类型 | 类和说明 |
|---|---|
class |
DatanodeInfo
DatanodeInfo represents the status of a DataNode.
|
| 限定符和类型 | 方法和说明 |
|---|---|
Node |
DatanodeInfo.getParent()
Return this node's parent
|
| 限定符和类型 | 方法和说明 |
|---|---|
void |
DatanodeInfo.setParent(Node parent) |
| 限定符和类型 | 类和说明 |
|---|---|
class |
DatanodeDescriptor
DatanodeDescriptor tracks stats on a given DataNode,
such as available storage capacity, last update time, etc.,
and maintains a set of blocks stored on the datanode.
|
| 限定符和类型 | 方法和说明 |
|---|---|
protected void |
BlockPlacementPolicyWithNodeGroup.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode) |
protected void |
BlockPlacementPolicyDefault.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode)
After choosing a node to place replica, adjust excluded nodes accordingly.
|
| 限定符和类型 | 方法和说明 |
|---|---|
protected int |
BlockPlacementPolicyWithNodeGroup.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Find other nodes in the same nodegroup of localMachine and add them
into excludeNodes as replica should not be duplicated for nodes
within the same nodegroup
|
protected int |
BlockPlacementPolicyWithNodeGroup.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Find other nodes in the same nodegroup of localMachine and add them
into excludeNodes as replica should not be duplicated for nodes
within the same nodegroup
|
protected int |
BlockPlacementPolicyDefault.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Add localMachine and related nodes to excludedNodes
for next replica choosing.
|
protected int |
BlockPlacementPolicyDefault.addToExcludedNodes(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes)
Add localMachine and related nodes to excludedNodes
for next replica choosing.
|
protected void |
BlockPlacementPolicyWithNodeGroup.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode) |
protected void |
BlockPlacementPolicyWithNodeGroup.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode) |
protected void |
BlockPlacementPolicyDefault.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode)
After choosing a node to place replica, adjust excluded nodes accordingly.
|
protected void |
BlockPlacementPolicyDefault.adjustExcludedNodes(HashMap<Node,Node> excludedNodes,
Node chosenNode)
After choosing a node to place replica, adjust excluded nodes accordingly.
|
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes)
choose local node of localMachine as the target.
|
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes)
choose local node of localMachine as the target.
|
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalNode(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyWithNodeGroup.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseLocalRack(DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRandom(int numOfReplicas,
String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRandom(int numOfReplicas,
String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseRandom(String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected DatanodeDescriptor |
BlockPlacementPolicyDefault.chooseRandom(String nodes,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxNodesPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyWithNodeGroup.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyWithNodeGroup.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
protected void |
BlockPlacementPolicyDefault.chooseRemoteRack(int numOfReplicas,
DatanodeDescriptor localMachine,
HashMap<Node,Node> excludedNodes,
long blocksize,
int maxReplicasPerRack,
List<DatanodeDescriptor> results,
boolean avoidStaleNodes) |
DatanodeDescriptor[] |
BlockPlacementPolicy.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas nodes for writer to replicate
a block with size blocksize
If not, return as many as we can.
|
DatanodeDescriptor[] |
BlockPlacementPolicy.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas nodes for writer to replicate
a block with size blocksize
If not, return as many as we can.
|
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas data nodes for writer
to re-replicate a block with size blocksize
If not, return as many as we can.
|
DatanodeDescriptor[] |
BlockPlacementPolicyDefault.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas data nodes for writer
to re-replicate a block with size blocksize
If not, return as many as we can.
|
abstract DatanodeDescriptor[] |
BlockPlacementPolicy.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas data nodes for writer
to re-replicate a block with size blocksize
If not, return as many as we can.
|
abstract DatanodeDescriptor[] |
BlockPlacementPolicy.chooseTarget(String srcPath,
int numOfReplicas,
DatanodeDescriptor writer,
List<DatanodeDescriptor> chosenNodes,
HashMap<Node,Node> excludedNodes,
long blocksize)
choose numOfReplicas data nodes for writer
to re-replicate a block with size blocksize
If not, return as many as we can.
|
LocatedBlock |
FSNamesystem.getAdditionalBlock(String src,
String clientName,
HashMap<Node,Node> excludedNodes)
The client would like to obtain an additional block for the indicated
filename (which is being written-to).
|
LocatedBlock |
FSNamesystem.getAdditionalBlock(String src,
String clientName,
HashMap<Node,Node> excludedNodes)
The client would like to obtain an additional block for the indicated
filename (which is being written-to).
|
| 限定符和类型 | 方法和说明 |
|---|---|
Node |
JobTracker.getNode(String name)
Return the Node in the network topology that corresponds to the hostname
|
static Node |
JobTracker.getParentNode(Node node,
int level) |
Node |
JobTracker.resolveAndAddToTopology(String name) |
| 限定符和类型 | 方法和说明 |
|---|---|
Collection<Node> |
JobTracker.getNodesAtMaxLevel()
Returns a collection of nodes at the max level
|
| 限定符和类型 | 方法和说明 |
|---|---|
static Node |
JobTracker.getParentNode(Node node,
int level) |
| 限定符和类型 | 类和说明 |
|---|---|
class |
NodeBase
A base class that implements interface Node
|
| 限定符和类型 | 字段和说明 |
|---|---|
protected Node |
NodeBase.parent |
| 限定符和类型 | 方法和说明 |
|---|---|
Node |
NetworkTopology.chooseRandom(String scope)
randomly choose one node from scope
if scope starts with ~, choose one from the all nodes except for the
ones in scope; otherwise, choose one from scope
|
Node |
NetworkTopology.getNode(String loc)
Given a string representation of a node, return its reference
|
protected Node |
NetworkTopologyWithNodeGroup.getNodeForNetworkLocation(Node node) |
protected Node |
NetworkTopology.getNodeForNetworkLocation(Node node)
Return a reference to the node given its string representation.
|
Node |
NodeBase.getParent()
Return this node's parent
|
Node |
Node.getParent()
Return this node's parent
|
| 限定符和类型 | 方法和说明 |
|---|---|
List<Node> |
NetworkTopology.getLeaves(String scope)
return leaves in scope
|
| 限定符和类型 | 方法和说明 |
|---|---|
void |
NetworkTopologyWithNodeGroup.add(Node node)
Add a leaf node
Update node counter & rack counter if necessary
|
void |
NetworkTopology.add(Node node)
Add a leaf node
Update node counter & rack counter if neccessary
|
boolean |
NetworkTopology.contains(Node node)
Check if the tree contains node node
|
int |
NetworkTopology.getDistance(Node node1,
Node node2)
Return the distance between two nodes
It is assumed that the distance from one node to its parent is 1
The distance between two nodes is calculated by summing up their distances
to their closest common ancestor.
|
protected Node |
NetworkTopologyWithNodeGroup.getNodeForNetworkLocation(Node node) |
protected Node |
NetworkTopology.getNodeForNetworkLocation(Node node)
Return a reference to the node given its string representation.
|
static String |
NodeBase.getPath(Node node)
Return this node's path
|
boolean |
NetworkTopologyWithNodeGroup.isOnSameNodeGroup(Node node1,
Node node2)
Check if two nodes are on the same node group (hypervisor) The
assumption here is: each nodes are leaf nodes.
|
boolean |
NetworkTopology.isOnSameNodeGroup(Node node1,
Node node2)
Return false directly as not aware of NodeGroup, to be override in sub-class
|
boolean |
NetworkTopologyWithNodeGroup.isOnSameRack(Node node1,
Node node2) |
boolean |
NetworkTopology.isOnSameRack(Node node1,
Node node2)
Check if two nodes are on the same rack
|
protected boolean |
NetworkTopology.isSameParents(Node node1,
Node node2)
Compare the parents of each node for equality
To be overridden in subclasses for specific NetworkTopology
implementations, as alternative to overriding the full
NetworkTopology.isOnSameRack(Node, Node) method. |
void |
NetworkTopologyWithNodeGroup.pseudoSortByDistance(Node reader,
Node[] nodes)
Sort nodes array by their distances to reader
It linearly scans the array, if a local node is found, swap it with
the first element of the array.
|
void |
NetworkTopologyWithNodeGroup.pseudoSortByDistance(Node reader,
Node[] nodes)
Sort nodes array by their distances to reader
It linearly scans the array, if a local node is found, swap it with
the first element of the array.
|
void |
NetworkTopology.pseudoSortByDistance(Node reader,
Node[] nodes)
Sort nodes array by their distances to reader
It linearly scans the array, if a local node is found, swap it with
the first element of the array.
|
void |
NetworkTopology.pseudoSortByDistance(Node reader,
Node[] nodes)
Sort nodes array by their distances to reader
It linearly scans the array, if a local node is found, swap it with
the first element of the array.
|
void |
NetworkTopologyWithNodeGroup.remove(Node node)
Remove a node
Update node counter and rack counter if necessary
|
void |
NetworkTopology.remove(Node node)
Remove a node
Update node counter & rack counter if neccessary
|
void |
NodeBase.setParent(Node parent)
Set this node's parent
|
void |
Node.setParent(Node parent)
Set this node's parent
|
protected static void |
NetworkTopology.swap(Node[] nodes,
int i,
int j) |
| 限定符和类型 | 方法和说明 |
|---|---|
int |
NetworkTopology.countNumOfAvailableNodes(String scope,
Collection<Node> excludedNodes)
return the number of leaves in scope but not in excludedNodes
if scope starts with ~, return the number of nodes that are not
in scope and excludedNodes;
|
| 构造器和说明 |
|---|
NodeBase(String name,
String location,
Node parent,
int level)
Construct a node from its name and its location
|
Copyright © 2009 The Apache Software Foundation