Skip to content

Commit 756924e

Browse files
committed
FLUME-2108. Corrections to CHANGELOG for 1.4.0 release.
(Mike Percy)
1 parent 3664472 commit 756924e

File tree

1 file changed

+4
-96
lines changed

1 file changed

+4
-96
lines changed

CHANGELOG

Lines changed: 4 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -1,72 +1,42 @@
11
Release Notes - Flume - Version v1.4.0
22

33
** New Feature
4-
* [FLUME-917] - Thrift Support
54
* [FLUME-924] - Implement a JMS source for Flume NG
65
* [FLUME-997] - Support secure transport mechanism
76
* [FLUME-1170] - Create a framework for transaction listeners
8-
* [FLUME-1249] - Add start/stop script
9-
* [FLUME-1485] - FileChannel should support checksum
107
* [FLUME-1502] - Support for running simple configurations embedded in host process
118
* [FLUME-1516] - FileChannel Write Dual Checkpoints to avoid replays
129
* [FLUME-1632] - Persist progress on each file in file spooling client/source
13-
* [FLUME-1669] - Add support for columnar event serializer in HDFS
14-
* [FLUME-1715] - OpenTSDB Sink
10+
* [FLUME-1735] - Add support for a plugins.d directory
1511
* [FLUME-1894] - Implement Thrift RPC
1612
* [FLUME-1917] - FileChannel group commit (coalesce fsync)
17-
* [FLUME-1941] - Support defaults or inheritance in configs
18-
* [FLUME-1988] - Add Support for Additional Deserializers for SpoolingDirectorySource
19-
* [FLUME-1998] - FlumeRESTClient - A HTTP Client for Interacting with REST Servers
20-
* [FLUME-1999] - GooglePlusSource - An EventDrivenSource for for Streaming In Data from Google+
21-
* [FLUME-2000] - EmailSource - An EventDrivenSource for Streaming In Events from POP/IMAP Mailboxes
22-
* [FLUME-2001] - TwitterSource - An EventDrivenSource for Streaming In Data from Twitter
2313
* [FLUME-2004] - Need to capture metrics on the Flume exec source such as events received, rejected, etc.
2414
* [FLUME-2010] - Support Avro records in Log4jAppender and the HDFS Sink
2515
* [FLUME-2048] - Avro container file deserializer
16+
* [FLUME-2070] - Add a Flume Morphline Solr Sink
2617

2718
** Improvement
28-
* [FLUME-841] - Generate javadoc from NG build
29-
* [FLUME-890] - Add unit tests for flume ng channels
30-
* [FLUME-1033] - Create system tests for flume
31-
* [FLUME-1071] - Evaluate performance of AVRO source with multiple clients
32-
* [FLUME-1076] - SInk batch sizes vary wildy
33-
* [FLUME-1081] - Load testing tool for Avro source
19+
* [FLUME-1076] - Sink batch sizes vary wildy
3420
* [FLUME-1100] - HDFSWriterFactory and HDFSFormatterFactory should allow extension
35-
* [FLUME-1141] - Avro compression unit test should fail if compression does not work
36-
* [FLUME-1159] - Add reconfigurability support to components
37-
* [FLUME-1178] - Improve performance of SyslogTcpSource
38-
* [FLUME-1209] - Support file name extension for HDFS sink
39-
* [FLUME-1237] - Refactor configuration and add more tests.
40-
* [FLUME-1274] - Clarify semantics of uncaught runtime exceptions thrown from configure() method
41-
* [FLUME-1278] - Flume hot-reconfiguration specification
42-
* [FLUME-1413] - Document of Channel (especially FileChannel) and Transaction
43-
* [FLUME-1497] - Make the TestFileChannel.testInOut() more deterministic
44-
* [FLUME-1521] - Document the StressSource
4521
* [FLUME-1571] - Channels should check for positive capacity and transaction capacity values
4622
* [FLUME-1586] - File Channel should support verifying integrity of individual events.
4723
* [FLUME-1652] - Logutils.getLogs could NPE
4824
* [FLUME-1661] - ExecSource cannot execute complex *nix commands
4925
* [FLUME-1677] - Add File-channel dependency to flume-ng-node's pom.xml
5026
* [FLUME-1699] - Make the rename of the meta file platform neutral
5127
* [FLUME-1702] - HDFSEventSink should write to a hidden file as opposed to a .tmp file
52-
* [FLUME-1722] - License file corrections
53-
* [FLUME-1735] - Add support for a plugins.d directory
5428
* [FLUME-1740] - Remove contrib/ directory from Flume NG
5529
* [FLUME-1745] - FlumeConfiguration Eats Exceptions
5630
* [FLUME-1756] - Avro client should be able to use load balancing RPC
5731
* [FLUME-1757] - Improve configuration of hbase serializers
58-
* [FLUME-1759] - make file and dirname of flume log tweakable in log4j
59-
* [FLUME-1760] - an extra hdfs. prefix feels redundant in HDFS sink configuration
6032
* [FLUME-1762] - File Channel should recover automatically if the checkpoint is incomplete or bad by deleting the contents of the checkpoint directory
61-
* [FLUME-1764] - File_roll type Sink how to achieve according to multiple files dynamically generated directory, and the file on the corresponding directory
6233
* [FLUME-1768] - Multiplexing channel selector should allow optional-only channels
6334
* [FLUME-1769] - Replicating channel selector should support optional channels
6435
* [FLUME-1770] - Flume should have serializer which supports serializer the headers to a simple string
6536
* [FLUME-1777] - AbstractSource does not provide enough implementation for sub-classes
6637
* [FLUME-1790] - Commands in EncryptionTestUtils comments require high encryption pack to be installed
6738
* [FLUME-1794] - FileChannel check for full disks in the background
6839
* [FLUME-1800] - Docs for spooling source durability changes
69-
* [FLUME-1801] - Configurable batch size for spooling client
7040
* [FLUME-1808] - ElasticSearchSink is missing log4.properties
7141
* [FLUME-1821] - Support configuration of hbase instances to be used in AsyncHBaseSink from flume config
7242
* [FLUME-1847] - NPE in SourceConfiguration
@@ -83,7 +53,6 @@ Release Notes - Flume - Version v1.4.0
8353
* [FLUME-1926] - Optionally timeout Avro Sink Rpc Clients to avoid stickiness
8454
* [FLUME-1940] - Log a snapshot of Flume metrics on shutdown
8555
* [FLUME-1945] - HBase Serializer allow key from regular expression group
86-
* [FLUME-1961] - Enhancements to the Channel interface
8756
* [FLUME-1976] - JMS Source document should provide instruction on JMS implementation jars
8857
* [FLUME-1977] - JMS Source connectionFactory property is not documented
8958
* [FLUME-1992] - ElasticSearch dependency is marked optional
@@ -97,47 +66,24 @@ Release Notes - Flume - Version v1.4.0
9766
* [FLUME-2049] - Compile ElasticSearchSink with elasticsearch 0.90
9867
* [FLUME-2062] - make it possible for HBase sink to deposit event headers into corresponding column qualifiers
9968
* [FLUME-2063] - Add Configurable charset to RegexHbaseEventSerializer
100-
* [FLUME-2070] - Add a Flume Morphline Solr Sink
69+
* [FLUME-2076] - JMX metrics support for HTTP Source
10170
* [FLUME-2093] - binary tarball that is created by flume's assembly shouldn't contain sources
10271
* [FLUME-2100] - Increase default batchSize of Morphline Solr Sink
10372
* [FLUME-2105] - Add docs for MorphlineSolrSink
10473

10574
** Bug
106-
* [FLUME-981] - Unify sink failure behavior, throwing EventDeliveryException
107-
* [FLUME-1000] - Properties validator misses an invalid config
108-
* [FLUME-1086] - duplicate lib in Flume's lib dir
109-
* [FLUME-1093] - HDFSEventSink gives no indication of failure when Hadoop jars aren't in the classpath
11075
* [FLUME-1110] - HDFS Sink throws IllegalStateException when flume-daemon shuts down
111-
* [FLUME-1115] - project "flume-ng-node" should depends on "flume-recoverable-memory-channel"
112-
* [FLUME-1152] - flume-ng script has issue with hadoop 0.23 script
11376
* [FLUME-1153] - flume-ng script is missing some agent options in help output
114-
* [FLUME-1161] - Logging output says name:null after initiating agent shutdown
115-
* [FLUME-1165] - Separate the component factories into pools which hold state and factories which simply create the new components
11677
* [FLUME-1175] - RollingFileSink complains of Bad File Descriptor upon a reconfig event
117-
* [FLUME-1203] - Hbase Sink needs to check for partial failures.
118-
* [FLUME-1206] - Hbase sink should pull in dependencies based on profiles
119-
* [FLUME-1233] - HDFS Sink has problem with %c escape sequence in bucket path
12078
* [FLUME-1262] - Move doc generation to a different profile
121-
* [FLUME-1283] - Flume HDFS Sink can't generate file name containing space and : which mentioned by %c
12279
* [FLUME-1285] - FileChannel has a dependency on Hadoop IO classes
123-
* [FLUME-1291] - FileChannel log close calls interrupt on current thread and does not kill worker thread immediately
12480
* [FLUME-1296] - Lifecycle supervisor should check if the monitor service is still running before supervising
125-
* [FLUME-1308] - HDFS Sink throws DFSOutputStream when exception when maxOpenFiles=1
126-
* [FLUME-1321] - BasicTransactionSemantics should never throw from close()
127-
* [FLUME-1378] - Sink falls into infinite lifecycle with EventDeliveryException
128-
* [FLUME-1403] - File channel throws logFile is null exception in channel.take
129-
* [FLUME-1411] - Add average events per second params to MBeans
130-
* [FLUME-1481] - Custom component may not support dynamic configuration
13181
* [FLUME-1511] - Scribe-source doesn't handle zero message request correctly.
132-
* [FLUME-1551] - Create a home for custom Flume components
133-
* [FLUME-1623] - Add more info about failover sink processor
13482
* [FLUME-1676] - ExecSource should provide a configurable charset
13583
* [FLUME-1688] - Bump AsyncHBase version to 1.4.1
13684
* [FLUME-1709] - HDFS CompressedDataStream doesn't support serializer parameter
13785
* [FLUME-1720] - LICENSE file contain entry for protobuf-<version>.jar, however proper artifact name is protobuf-java-<version>.jar
138-
* [FLUME-1721] - SpoolDirectorySource stop should shutdown executor
13986
* [FLUME-1731] - SpoolableDirectorySource should have configurable support for deleting files it has already completed instead of renaming
140-
* [FLUME-1732] - Build is failing due to netty problems
14187
* [FLUME-1741] - ElasticSearch tests leave directory data/elasticsearch/nodes/ lying around
14288
* [FLUME-1748] - HDFS Sink should check if the thread is interrupted before performing any HDFS operations
14389
* [FLUME-1755] - Load balancing RPC client has issues with downed hosts
@@ -158,7 +104,6 @@ Release Notes - Flume - Version v1.4.0
158104
* [FLUME-1803] - Generated dist tarball is missing flume-ng-embedded-agent
159105
* [FLUME-1804] - JMS source not included in binary dist
160106
* [FLUME-1805] - Embedded agent deps should be specified in dependencyManagement section of pom
161-
* [FLUME-1815] - File Channel auto-recovery when checkpoint is bad should be optional
162107
* [FLUME-1818] - Support various layouts in log4jappender
163108
* [FLUME-1819] - ExecSource don't flush the cache if there is no input entries
164109
* [FLUME-1820] - Should not be possible for RPC client to block indefinitely on close()
@@ -190,19 +135,16 @@ Release Notes - Flume - Version v1.4.0
190135
* [FLUME-1911] - Add deprecation back to the legacy thrift code
191136
* [FLUME-1916] - HDFS sink should poll for # of active replicas. If less than required, roll the file.
192137
* [FLUME-1918] - File Channel cannot handle capacity of more than 500 Million events
193-
* [FLUME-1919] - Consider adding areYouOK() RPC calls to Avro & Thrift RPC mechanisms
194138
* [FLUME-1922] - HDFS Sink should optionally insert the timestamp at the sink
195139
* [FLUME-1924] - Bug in serializer context parsing in RollingFileSink
196140
* [FLUME-1925] - HDFS timeouts should not starve other threads
197141
* [FLUME-1929] - CheckpointRebuilder main method does not work
198142
* [FLUME-1930] - Inflights should clean up executors on close.
199143
* [FLUME-1931] - HDFS Sink has a commons-lang dependency which is missing in pom
200144
* [FLUME-1932] - no-reload-conf command line param does not work
201-
* [FLUME-1936] - Configure max rolls for under-replicated blocks
202145
* [FLUME-1937] - Issue with maxUnderReplication in HDFS sink
203146
* [FLUME-1939] - FlumeEventQueue must check if file is open before setting the length of the file
204147
* [FLUME-1943] - ExecSource tests failing on Jenkins
205-
* [FLUME-1944] - parameterising the dependency variables in pom file.
206148
* [FLUME-1948] - plugins.d directory(ies) should be separately overridable, independent of FLUME_HOME
207149
* [FLUME-1949] - Documentation for sink processor lists incorrect default
208150
* [FLUME-1955] - fileSuffix does not work with compressed streams
@@ -233,65 +175,31 @@ Release Notes - Flume - Version v1.4.0
233175
* [FLUME-2054] - Support Version Info on Windows and fix failure of TestVersionInfo
234176
* [FLUME-2057] - Failures in FileChannel's TestEventQueueBackingStoreFactory on Windows
235177
* [FLUME-2060] - Failure in TestLog.testReplaySucceedsWithUnusedEmptyLogMetaDataFastReplay test on Windows
236-
* [FLUME-2068] - File Channel issue - recovering from BadCheckpoint exception on Windows
237178
* [FLUME-2072] - JMX metrics support for HBase Sink
238179
* [FLUME-2081] - JMX metrics support for SpoolDir
239180
* [FLUME-2082] - JMX support for Seq Generator Source
240181
* [FLUME-2083] - Avro Source should not start if SSL is enabled and keystore cannot be opened
241182
* [FLUME-2098] - Make Solr sink depend on the CDK version of morphlines
242183

243184
** Documentation
244-
* [FLUME-1014] - Agent Config docs are out of date
245-
* [FLUME-1022] - Update Flume NG Getting Started Guide to mention pulling from trunk instead of flume-728
246-
* [FLUME-1144] - Update Flume User Guide with a section on Flume properties such as flume.called.from.service
247-
* [FLUME-1592] - Remove appendTimeout from HDFS sink docs
248-
* [FLUME-1594] - Document JMX options
249185
* [FLUME-1621] - Document new MemoryChannel parameters in Flume User Guide
250-
* [FLUME-1624] - Docs could be more consistent when giving examples of types
251-
* [FLUME-1833] - Embedded Agent docs are missing from website
252186
* [FLUME-1910] - Add thrift RPC documentation
253187
* [FLUME-1953] - Fix dev guide error that says sink can read from multiple channels
254188
* [FLUME-1962] - Document proper specification of lzo codec as lzop in Flume User Guide
255189
* [FLUME-1979] - Wrong propname for connection reset interval in avro sink
256190
* [FLUME-2030] - Documentation of Configuration Changes JMSSource, HBaseSink, AsyncHBaseSink and ElasticSearchSink
257191

258192
** Task
259-
* [FLUME-813] - Build Flume site from Maven
260-
* [FLUME-1051] - Configuration System
261-
* [FLUME-1618] - Make Flume NG build and tests work with Hadoop 2.0
262193
* [FLUME-1686] - Exclude target directories & Eclipse files from rat checks
263-
* [FLUME-2076] - JMX metrics support for HTTP Source
264194
* [FLUME-2094] - Remove the deprecated - Recoverable Memory Channel
265195

266196
** Sub-task
267-
* [FLUME-731] - Create NG internal diagrams
268-
* [FLUME-732] - Create NG developer code tour docs
269-
* [FLUME-733] - Define NG master interfaces and classes
270-
* [FLUME-738] - Create NG system configuration component
271-
* [FLUME-749] - Add support for monitoring APIs
272-
* [FLUME-1056] - AvroSource configuration
273-
* [FLUME-1057] - AvroSink configuration
274-
* [FLUME-1058] - Exec source configuration
275-
* [FLUME-1059] - failover sink configuration
276-
* [FLUME-1060] - Hdfs sink configuration
277-
* [FLUME-1061] - jdbc channel configuration
278-
* [FLUME-1062] - Memory Channel configuration
279-
* [FLUME-1063] - Miscellanous changes related to imports and minor changes related to config changes
280-
* [FLUME-1064] - Channel selector configuration
281-
* [FLUME-1065] - netcat source configuration
282-
* [FLUME-1066] - PseudoTxnMemoryChannel configuration
283-
* [FLUME-1067] - RollingFileSink configuration
284-
* [FLUME-1068] - syslog source configuration
285-
* [FLUME-1069] - FileChannel configuration
286197
* [FLUME-1626] - Support Hbase security in Hbase sink
287198
* [FLUME-1630] - Flume configuration code could be improved
288199
* [FLUME-1674] - Documentation / Wiki
289200
* [FLUME-1896] - Implement Thrift RpcClient
290201
* [FLUME-1897] - Implement Thrift Sink
291202
* [FLUME-1898] - Implement Thrift Source
292-
* [FLUME-1989] - Source Code and Unit Tests for org.apache.flume.serialization.RegexDelimiterDeSerializer
293-
* [FLUME-1990] - Source Code and Unit Tests for org.apache.flume.serialization.CharSequenceDelimiterDeSerializer
294-
* [FLUME-1991] - Documentation for New Deserializers for SpoolingDirectorySource
295203
* [FLUME-2102] - Update LICENSE file for Flume 1.4.0
296204

297205
----

0 commit comments

Comments
 (0)