Skip to content

Commit 79ab880

Browse files
openargusopenargus
committed
Merge branch 'v5.0.0'
2 parents 013d845 + affaff7 commit 79ab880

File tree

479 files changed

+344118
-121625
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

479 files changed

+344118
-121625
lines changed

‎.gitignore‎

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ argus-clients.xcodeproj/xcuserdata/
2020
.devel
2121
.nfs*
2222

23+
*.a
2324
*.o
2425
*.swp
2526
Makefile
@@ -35,11 +36,10 @@ include/argus_config.h
3536
/common/scanner.c
3637
/common/tokdefs.h
3738
/common/version.c
38-
/lib/*.a
3939
/autom4te.cache
4040
/argus-clients-*.tar.gz
4141
/lib/argus-clients.spec
42-
/pkg/win7/argus-clients.wxs
42+
/pkg/win/argus-clients.wxs
4343
/pkg/osx/BUILDROOT
4444
/examples/radhcp/radhcp-query
4545
/examples/rahisto/rahisto-delsql
@@ -64,3 +64,15 @@ include/argus_config.h
6464
/examples/rasql_udf_cidr/config.h
6565
/examples/rasql_udf_cidr/rasql_udf_cidr.la
6666
/examples/rasql_udf_cidr/rasql_udf_cidr_la-rasql_udf_cidr.lo
67+
68+
# radhcp intermediates
69+
/examples/radhcp/example-libargus_dhcp
70+
71+
# pythonlib
72+
/pythonlib/argusWgan.py
73+
/pythonlib/argusWgan_wrap.c
74+
/pythonlib/build/
75+
/pythonlib/swig.stamp
76+
/pythonlib/argusPython.py
77+
/pythonlib/argusPython_wrap.c
78+

‎AUTHORS‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
11
#
2-
# Argus Client Software. Tools to read, analyze and manage Argus data.
3-
# Copyright (c) 2000-2022 QoSient, LLC
2+
# Argus-5.0 Client Software. Tools to read, analyze and manage Argus data.
3+
# Copyright (c) 2000-2024 QoSient, LLC
44
# All rights reserved.
55
#
6-
# $Id: //depot/argus/clients/AUTHORS#7 $
7-
# $DateTime: 2016/06/01 15:17:28 $
8-
# $Change: 3148 $
6+
# $Id: //depot/gargoyle/clients/AUTHORS#4 $
7+
# $DateTime: 2014/05/14 00:30:13 $
8+
# $Change: 2825 $
99

1010
Argus and its client software are written by Carter Bullard <[email protected]>
11+
and Eric Kinzie <[email protected]>.
1112

1213
Many contributions of fixes, bug reports and design have been submitted to the
1314
argus developers mailing list over the years, please see the ./CREDITS file for

‎CHANGES‎

Lines changed: 148 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,45 +1,165 @@
11
/*
2-
* Argus Client Software. Tools to read, analyze and manage Argus data.
3-
* Copyright (c) 2000-2022 QoSient, LLC
2+
* Argus-5.0 Client Software. Tools to read, analyze and manage Argus data.
3+
* Copyright (c) 2000-2024 QoSient, LLC
44
* All rights reserved.
55
*
6-
* This program is free software; you can redistribute it and/or modify
7-
* it under the terms of the GNU General Public License as published by
8-
* the Free Software Foundation; either version 2, or (at your option)
9-
* any later version.
10-
11-
* This program is distributed in the hope that it will be useful,
12-
* but WITHOUT ANY WARRANTY; without even the implied warranty of
13-
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14-
* GNU General Public License for more details.
15-
6+
* This program is free software, released under the GNU General
7+
* Public License; you can redistribute it and/or modify it under the terms
8+
* of the GNU General Public License as published by the Free Software
9+
* Foundation; either version 3, or any later version.
10+
*
11+
* Other licenses are available through QoSient, LLC.
12+
* Inquire at [email protected].
13+
*
14+
* This program is distributed WITHOUT ANY WARRANTY; without even the
15+
* implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
16+
* See the * GNU General Public License for more details.
17+
*
1618
* You should have received a copy of the GNU General Public License
1719
* along with this program; if not, write to the Free Software
18-
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. */
20+
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
1921
*
2022
*/
2123

2224

23-
CHANGES - This file attempts to capture the changes between argus-clients-3.0.8.4
24-
and argus-clients-3.0.8.5. There are some significant changes, so if I
25-
missed anything, please be kind.
25+
Argus-5.0 Data Support
26+
27+
Argus-5.0 is a public version of commercial argus, that provides enhanced
28+
functionality and performance. Argus-5.0 is designed to be completely
29+
backward compatible with the previous versions of open source argus and
30+
its program names, configuration, etc should be similar when possible.
31+
32+
Product specific changes are provided with each product. When no
33+
specific changes are mentioned, gargoyle should be perceived as
34+
completely compatible, but enhanced.
35+
36+
37+
Argus Data Support
38+
39+
Argus-5.0 represents a major change in argus data. The format
40+
was completely updated to support 128-bit uuid argus source id's,
41+
As a result there is no forward compatibility between argus-3.0 programs and
42+
argus-5.0 data. Argus-5.0 programs are backward compatible, however, so
43+
you can read and process your data with argus-5.0 programs.
44+
45+
We have made provisions for argus-5.0 programs to generate argus-3.x formatted
46+
data, but that is configurable, and not the default behavior. So there is
47+
explicit conversion that will be going on under the covers. There is no
48+
compelling justification to convert your argus-3.x data, allowing you to leave
49+
the original data unmodified.
50+
51+
Argus-5.0.0 and its clients provides for extended modes of transport of argus
52+
data, earlier version of argus-clients cannot read these transport formats.
53+
54+
Argus-clients-5.0 provides extended capabilities for reading flow-tools
55+
originated data, and bro/zeek data and converting them to argus-5.0 data
56+
formats.
57+
58+
Argus-clients-5.0.0 is backward compatible with all prior releases of
59+
argus data, fixing a large number of bugs, with regard to data represenation
60+
and processing.
61+
62+
63+
Architecture
64+
65+
The client programs evolved quite a bit between argus-2.0, argus-3.0 and argus-5.0.
66+
With the addition of large scale argus data collection and distribution, using
67+
radium(), and rastream(). Argus-clients-5.0 extends this architecture, providing
68+
the ability to collect, distribute, archive, analyze, and process network flow
69+
data, for argus data, flow-tools data, netflow v5-8 partial sflow data processing,
70+
and zeek data conversion.
71+
72+
radium(), and rastream() are the principal programs that have been added to the
73+
ra* family of programs. radium can connect to multiple sources of argus data,
74+
whether they are streams or files of data, and can write out data to multiple targets,
75+
supporting independant access control, authentication, and filtering per target.
76+
What this means is that you can build a argus data distribution tree, to collect,
77+
process and redistribute argus data.
78+
79+
Rastream() is known as a stream block processors (SBP). You want to collect data
80+
from a set of argus data stream sources, and the data just keeps coming in.
81+
When/how can you stop to process the data, say for real-time indexing, search
82+
and/or processing? In the database world this is called 'stream block processing'.
83+
rastream() reads in argus data, and output the data into a set of files
84+
that make up a native OS filesystem based archive. rastream() extends
85+
this capability by implementing a wa hold buffer to allow for input sorting,
86+
and then based on command line options, rastream() can call scripts against the
87+
files after a time period or an event. We use rastream() to periodically commit
88+
data to an information system for indexing, searching, processing, compressing,
89+
and then archive. Say every 5 minutes, on the second, rastream() will close
90+
completed input files and then spawn any number of processes against those files.
91+
92+
With these programs, we have collected data from as many as 5K argus data
93+
sources, and managed the data in a set of argus data respositories.
94+
95+
Argus-clients-3.0.6 provides new capabilities in this area, allowing
96+
radium to "serve up" files that are generated by rastream().
97+
See the manpages for radium.1 and ra.1.
98+
99+
100+
The clients distribution has been restructred in argus-5.0.0. It is organized
101+
into argus client Core Programs, and Examples.
102+
103+
Client Core Programs
104+
ra - principal program that read, process, filter, and print argus data.
105+
racount - no basic changes.
106+
racluster - complete rewrite of argus aggregation strategies,
107+
and replaces ragator.
108+
109+
radium - argus record collection and distribution program.
110+
111+
ranonymize - updated for new data types.
112+
rasort - ported.
113+
114+
ratop - massive rewrite. Completely new program.
115+
116+
Client Example Programs
117+
These programs provide examples in key areas of argus data processing
118+
and management.
119+
120+
argus data environment
121+
ratop - realtime argus data processing environment (curses based)
122+
provides vi() like functionality for streaming and file based flow data,
123+
supporting printing, searching, editing, sorting, writing argus data.
124+
125+
argus data processing
126+
raconvert - ascii to binary data record conversion
127+
raevent - non flow data printing
128+
rafilteraddr - high performance filtering
129+
ralabel - semantic enhancement / metadata tagging
130+
rastream - stream block processing
131+
rastrip - data compression
26132

133+
analytics
134+
rahisto - frequency distribution analysis for argus data metrics
27135

28-
Argus MetaData Support
136+
graphing
137+
ragraph - time series graphing (rrd-tool based)
138+
raplot - general plotting (gnuplot based)
29139

30-
Argus MetaData Support
140+
storage management
141+
ramysql - mysql based utilities
142+
rasql - read native argus data from mysql database tables.
143+
rasqlinsert - insert and read argus data from/to mysql data tables.
144+
rasqltimeindex - generate argus data file time indexes for searching.
31145

32-
In 3.0.8.5 we added JSON format support for the label. In the clients, we have
33-
a configuration variable for the rarc file that can cause any client to convert
34-
the label to JSON. By default, this is turned off.
146+
forensics
147+
radump - decode captured user data
148+
ragrep - regular expression matching from captured user data
149+
raservices - user data analysis to determine used protocol
35150

36-
All argus clients support the '-M label="regex"' option, which search the contents
37-
of the label string in flow records, and processes those records that match.
38-
Knowing that the format is JSON may impact the format of your regex.
151+
reporting
152+
radark - scanner detection and reporting
153+
rascan - scanner detection and reporting
154+
rahosts - IP address inventory reporting
155+
raips - IP address inventory reporting
156+
rapath - print topology information derived from argus data
157+
rapolicy - continuous access control policy verification
158+
raports - application port usage
159+
rarpwatch - arpwatch driven using argus data
160+
ratimerange - argus data file time span
39161

40-
When you aggregate flow records, the labels get merged, and the resulting structures
41-
will be stored either as legacy or JSON.
162+
development
163+
ratemplate - ra client development template when using the argus clients library.
42164

43-
When you read either formats, the clients will retain the original format for 3.0 Argus.
44165

45-
In Argus 4.0, we intend to shift all metadata to JSON, so this is a transition strategy.

‎CREDITS‎

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,27 @@
11
/*
2-
* Argus Client Software. Tools to read, analyze and manage Argus data.
3-
* Copyright (c) 2000-2022 QoSient, LLC
2+
* Argus-5.0 Client Software. Tools to read, analyze and manage Argus data.
3+
* Copyright (c) 2000-2024 QoSient, LLC
44
* All rights reserved.
55
*
6-
* This program is free software; you can redistribute it and/or modify
7-
* it under the terms of the GNU General Public License as published by
8-
* the Free Software Foundation; either version 2, or (at your option)
9-
* any later version.
10-
11-
* This program is distributed in the hope that it will be useful,
12-
* but WITHOUT ANY WARRANTY; without even the implied warranty of
13-
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14-
* GNU General Public License for more details.
15-
6+
* This program is free software, released under the GNU General
7+
* Public License; you can redistribute it and/or modify it under the terms
8+
* of the GNU General Public License as published by the Free Software
9+
* Foundation; either version 3, or any later version.
10+
*
11+
* Other licenses are available through QoSient, LLC.
12+
* Inquire at [email protected].
13+
*
14+
* This program is distributed WITHOUT ANY WARRANTY; without even the
15+
* implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
16+
* See the * GNU General Public License for more details.
17+
*
1618
* You should have received a copy of the GNU General Public License
1719
* along with this program; if not, write to the Free Software
18-
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. */
20+
* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
1921
*
2022
*/
2123

24+
2225
CREDITS
2326

2427
To my colleagues who contributed so much to the theory of Argus and its

‎ChangeLog‎

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,32 @@
1-
Sun Jun 5 15:34:56 EDT 2016
2-
Mods for racluster.conf strategy for using "none" to force no aggregation
1+
Sat Nov 26 15:18:51 EST 2016
2+
Fix for rasqlinsert stack problems.
33

4-
Wed Jun 1 14:45:00 EDT 2016
5-
Many, many, many, many changes to fix bugs and to support new features from argus and argus use.
6-
Consider this a major bug release from 3.0.8 clients but is basically backward compatible.
7-
Primary changes are in ratop.1, rabins.1, ralabel.1, ra.1 and radium.1.
4+
Mon Oct 17 12:03:30 EDT 2016
5+
Change to ratop.1 to reenable lsof event correlation.
6+
Default flow key for aggregation and comparison needs to use the sid, rather than the srcid, so that the event originated flows (evt0) can match the observed flows (i.e. en0).
7+
8+
Sat Oct 1 20:18:54 EDT 2016
9+
Added support for inf in events and management records.
10+
Added filters for inf and sid, and cleaned up the configuration for argus
11+
12+
Tue Sep 27 20:07:59 EDT 2016
13+
Added rasqlcheckconf to import an argus.conf file into a database table,
14+
then to compare the contents with the database, replacing the contents
15+
if there is a difference, and its configured to "-M replace" the table
16+
17+
Thu Sep 15 14:41:58 EDT 2016
18+
Changes to support ekinzie mods
19+
20+
Sun Oct 4 15:23:19 EDT 2015
21+
Added control channel to argus_output.c for ratop remote coupling.
22+
Allows to send to ratop 'highlight' 'search' 'filter' etc directives,
23+
so that another application can use ratop() as a display.
24+
25+
Tue Oct 21 16:40:33 EDT 2014
26+
Added l2tp encapsulation parsing
27+
28+
Mon Oct 13 11:12:51 EST 2014
29+
Added vxlan encapsulation parsing
830

931
Tue Aug 26 09:10:03 EDT 2014
1032
Fixes for pkg'ing.
@@ -235,6 +257,18 @@ Wed May 2 18:28:05 EDT 2012
235257
Fixes for memory leaks in Aggregators
236258
Modified label and user data buffer processing for buffer overflow issues.
237259

260+
Thu Jun 21 15:26:04 EDT 2012
261+
Fix country code aggregation issues
262+
263+
Fri May 11 12:47:25 EDT 2012
264+
Mods for objective-c client linking - move argus_int.h definitions into argus_util.c
265+
Mods to bring gargoyle and argus into sync with label processing.
266+
- artificial limit to 4 values per attribute to handle argus-events
267+
268+
Wed May 2 18:28:05 EDT 2012
269+
Fixes for memory leaks in Aggregators
270+
Modified label and user data buffer processing for buffer overflow issues.
271+
238272
Wed Apr 18 09:04:30 EDT 2012
239273
Create argus-3.0.7.1 after branching code to argus-3.0.6
240274
Fix raconvert so it can read from stdin and write to stdout.

0 commit comments

Comments
 (0)