Quantcast
Channel: Hakuna MapData! » shell
Viewing all articles
Browse latest Browse all 4

Recreating HBase table without violating region starting keys

$
0
0

Today, I was using completebulkload to load larger amount of data into HBase. To keep this process well-balanced over the entire cluster (and thus faster), I was loading data into a table with pre-created regions. Since my bulkloading process failed in the middle a couple of times due to some misconfiguration, I needed to truncate the table each time.

I have noticed that a command like

hbase(main):017:0> truncate 'table'

will disable, drop and recreate the table with the same name settings (number of column familiers, compression, ttl, blocksize etc), but it does not maintain the region boundaries. It means that if you have a table with some number of regions and then truncate it, the table will be recreated with a single one region e.g.

hbase(main):018:0> create 't1', 'f1', {SPLITS => ['10', '20', '30', '40']}
hbase(main):019:0> truncate 't1'
Truncating 't1' table (it may take a while):
 - Disabling table...
 - Dropping table...
 - Creating table...

If you look at http://${hbase-master}:60010/table.jsp?name=t1, you will find that truncating makes table t1 empty and reduces the number of regions to single one.

Actually, I needed a functionality to truncate a table, but not violate the region and their starting keys (since I have choosen them so carefully earlier to balance the load over all region servers). I have implemented simple script for this purpose:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
include Java
 
import org.apache.hadoop.hbase.client.HTable
import org.apache.hadoop.hbase.util.Bytes
import org.apache.hadoop.hbase.client.HBaseAdmin
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.hbase.HTableDescriptor
 
table_name = ARGV[0]
 
table = HTable.new(table_name)
region_start_keys = table.getStartKeys()
table.close()
 
admin = HBaseAdmin.new(Configuration.new())
table_descriptor = admin.getTableDescriptor(Bytes.toBytes(table_name))
admin.disableTable(table_name)
admin.deleteTable(table_name)
admin.createTable(table_descriptor, region_start_keys)

You may use it in a following way:

$ hbase org.jruby.Main region-keys.rb t1

The table should be successfully recreated with the same region boundaries, so that it will look the same as before, but it will be just empty.


Viewing all articles
Browse latest Browse all 4

Trending Articles