You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/main/asciidoc/_chapters/architecture.adoc
+54Lines changed: 54 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2542,6 +2542,60 @@ All the settings that apply to normal compactions (see <<compaction.parameters>>
2542
2542
The exceptions are the minimum and maximum number of files, which are set to higher values by default because the files in stripes are smaller.
2543
2543
To control these for stripe compactions, use `hbase.store.stripe.compaction.minFiles` and `hbase.store.stripe.compaction.maxFiles`, rather than `hbase.hstore.compaction.min` and `hbase.hstore.compaction.max`.
2544
2544
2545
+
[[ops.fifo]]
2546
+
===== FIFO Compaction
2547
+
2548
+
FIFO compaction policy selects only files which have all cells expired. The column family *MUST* have non-default TTL.
2549
+
Essentially, FIFO compactor only collects expired store files.
2550
+
2551
+
Because we don't do any real compaction, we do not use CPU and IO (disk and network) and evict hot data from a block cache.
2552
+
As a result, both RW throughput and latency can be improved.
2553
+
2554
+
[[ops.fifo.when]]
2555
+
===== When To Use FIFO Compaction
2556
+
2557
+
Consider using FIFO Compaction when your use case is
2558
+
2559
+
* Very high volume raw data which has low TTL and which is the source of another data (after additional processing).
2560
+
* Data which can be kept entirely in a a block cache (RAM/SSD). No need for compaction of a raw data at all.
2561
+
2562
+
Do not use FIFO compaction when
2563
+
2564
+
* Table/ColumnFamily has MIN_VERSION > 0
2565
+
* Table/ColumnFamily has TTL = FOREVER (HColumnDescriptor.DEFAULT_TTL)
2566
+
2567
+
[[ops.fifo.enable]]
2568
+
====== Enabling FIFO Compaction
2569
+
2570
+
For Table:
2571
+
2572
+
[source,java]
2573
+
----
2574
+
HTableDescriptor desc = new HTableDescriptor(tableName);
Although region splitting is still supported, for optimal performance it should be disabled, either by setting explicitly `DisabledRegionSplitPolicy` or by setting `ConstantSizeRegionSplitPolicy` and very large max region size.
2596
+
You will have to increase to a very large number store's blocking file (`hbase.hstore.blockingStoreFiles`) as well.
2597
+
There is a sanity check on table/column family configuration in case of FIFO compaction and minimum value for number of blocking file is 1000.
0 commit comments