Balancer (Hadoop 1.2.2-SNAPSHOT API)

java.lang.Object
- org.apache.hadoop.hdfs.server.balancer.Balancer

所有已实现的接口:

Configurable, Tool
```
public class Balancer
extends Object
implements Tool
```
The balancer is a tool that balances disk space usage on an HDFS cluster when some datanodes become full or when new empty nodes join the cluster. The tool is deployed as an application program that can be run by the cluster administrator on a live HDFS cluster while applications adding and deleting files.
SYNOPSIS
```
 To start:
      bin/start-balancer.sh [-threshold ]
      Example: bin/ start-balancer.sh 
                     start the balancer with a default threshold of 10%
               bin/ start-balancer.sh -threshold 5
                     start the balancer with a threshold of 5%
 To stop:
      bin/ stop-balancer.sh
 
```
DESCRIPTION
The threshold parameter is a fraction in the range of (0%, 100%) with a default value of 10%. The threshold sets a target for whether the cluster is balanced. A cluster is balanced if for each datanode, the utilization of the node (ratio of used space at the node to total capacity of the node) differs from the utilization of the (ratio of used space in the cluster to total capacity of the cluster) by no more than the threshold value. The smaller the threshold, the more balanced a cluster will become. It takes more time to run the balancer for small threshold values. Also for a very small threshold the cluster may not be able to reach the balanced state when applications write and delete files concurrently.
The tool moves blocks from highly utilized datanodes to poorly utilized datanodes iteratively. In each iteration a datanode moves or receives no more than the lesser of 10G bytes or the threshold fraction of its capacity. Each iteration runs no more than 20 minutes. At the end of each iteration, the balancer obtains updated datanodes information from the namenode.
A system property that limits the balancer's use of bandwidth is defined in the default configuration file:
```
 
   dfs.balance.bandwidthPerSec
   1048576
   Specifies the maximum bandwidth that each datanode 
 can utilize for the balancing purpose in term of the number of bytes 
 per second. 
 
 
```
This property determines the maximum speed at which a block will be moved from one datanode to another. The default value is 1MB/s. The higher the bandwidth, the faster a cluster can reach the balanced state, but with greater competition with application processes. If an administrator changes the value of this property in the configuration file, the change is observed when HDFS is next restarted.
MONITERING BALANCER PROGRESS
After the balancer is started, an output file name where the balancer progress will be recorded is printed on the screen. The administrator can monitor the running of the balancer by reading the output file. The output shows the balancer's status iteration by iteration. In each iteration it prints the starting time, the iteration number, the total number of bytes that have been moved in the previous iterations, the total number of bytes that are left to move in order for the cluster to be balanced, and the number of bytes that are being moved in this iteration. Normally "Bytes Already Moved" is increasing while "Bytes Left To Move" is decreasing.
Running multiple instances of the balancer in an HDFS cluster is prohibited by the tool.
The balancer automatically exits when any of the following five conditions is satisfied:
1. The cluster is balanced;
2. No block can be moved;
3. No block has been moved for five consecutive iterations;
4. An IOException occurs while communicating with the namenode;
5. Another balancer is running.
Upon exit, a balancer returns an exit code and prints one of the following messages to the output file in corresponding to the above exit reasons:
1. The cluster is balanced. Exiting
2. No block can be moved. Exiting...
3. No block has been moved for 5 iterations. Exiting...
4. Received an IO exception: failure reason. Exiting...
5. Another balancer is running. Exiting...
The administrator can interrupt the execution of the balancer at any time by running the command "stop-balancer.sh" on the machine where the balancer is running.

字段概要

字段
限定符和类型	字段和说明
`static int`	`ALREADY_RUNNING`
`static int`	`ILLEGAL_ARGS`
`static int`	`IO_EXCEPTION`
`static int`	`MAX_NO_PENDING_BLOCK_ITERATIONS`
`static int`	`MAX_NUM_CONCURRENT_MOVES` The maximum number of concurrent blocks moves for balancing purpose at a datanode
`static int`	`NO_MOVE_BLOCK`
`static int`	`NO_MOVE_PROGRESS`
`static int`	`SUCCESS`

方法概要

方法
限定符和类型	方法和说明
`Configuration`	`getConf()` return this balancer's configuration
`static void`	`main(String[] args)` Run a balancer
`int`	`run(String[] args)` main method of Balancer
`void`	`setConf(Configuration conf)` set this balancer's configuration

从类继承的方法 java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- 字段详细资料
  - MAX_NUM_CONCURRENT_MOVES
```
public static final int MAX_NUM_CONCURRENT_MOVES
```
    The maximum number of concurrent blocks moves for balancing purpose at a datanode
    
    另请参阅:
    常量字段值
  - MAX_NO_PENDING_BLOCK_ITERATIONS
```
public static final int MAX_NO_PENDING_BLOCK_ITERATIONS
```
    另请参阅:
    常量字段值
  - SUCCESS
```
public static final int SUCCESS
```
    另请参阅:
    常量字段值
  - ALREADY_RUNNING
```
public static final int ALREADY_RUNNING
```
    另请参阅:
    常量字段值
  - NO_MOVE_BLOCK
```
public static final int NO_MOVE_BLOCK
```
    另请参阅:
    常量字段值
  - NO_MOVE_PROGRESS
```
public static final int NO_MOVE_PROGRESS
```
    另请参阅:
    常量字段值
  - IO_EXCEPTION
```
public static final int IO_EXCEPTION
```
    另请参阅:
    常量字段值
  - ILLEGAL_ARGS
```
public static final int ILLEGAL_ARGS
```
    另请参阅:
    常量字段值
- 方法详细资料
  - main
```
public static void main(String[] args)
```
    Run a balancer
    
    参数:
    args -
  - run
```
public int run(String[] args)
        throws Exception
```
    main method of Balancer
    
    指定者:
    
    run 在接口中 Tool
    
    参数:
    args - arguments to a Balancer
    
    返回:
    exit code.
    
    抛出:
    
    any - exception occurs during datanode balancing
    
    Exception
  - getConf
```
public Configuration getConf()
```
    return this balancer's configuration
    
    指定者:
    
    getConf 在接口中 Configurable
  - setConf
```
public void setConf(Configuration conf)
```
    set this balancer's configuration
    
    指定者:
    
    setConf 在接口中 Configurable

类 Balancer

字段概要

方法概要

从类继承的方法 java.lang.Object

字段详细资料

MAX_NUM_CONCURRENT_MOVES

MAX_NO_PENDING_BLOCK_ITERATIONS

SUCCESS

ALREADY_RUNNING

NO_MOVE_BLOCK

NO_MOVE_PROGRESS

IO_EXCEPTION

ILLEGAL_ARGS

方法详细资料

main

run

getConf

setConf