This repository was archived by the owner on Apr 3, 2019. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 1
mycallmax/LogQuery
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
---------------------------------------------------------------------- Distributed Log query system: ---------------------------------------------------------------------- ---------------------------------------------------------------------- Build: ---------------------------------------------------------------------- Makefile is provided to build the source files (.java files) - Hit 'make' to build the class files ---------------------------------------------------------------------- Starting the service: ---------------------------------------------------------------------- Use the below command to initiate the grep service "java LogQueryServer <port_number> <logfile_path>" <port_number>: TCP port number <logfile_path>: Path where log file resides eg. 'java LogQueryServer 1111 /tmp/logs/machine.1.log' Note: Make sure that grep services are initialized for all nodes. The distributed grep client will have the node addresses and port numbers initialized with the code. Note: To ensure that this can be run on EWS machines in the lab, we have used the below EWS nodes to participate in our distributed grep system i.p port 130.126.112.146 1111 130.126.112.117 1122 130.126.112.148 1133 130.126.112.146 1144 --------------------------------------------------------------------- Starting the query: --------------------------------------------------------------------- A query can be initiated from any node using the below command "java LogQueryClient <grep_arg1> <grep_arg2> ... <grep_argN>" <grep_argN> : Nth argument for the grep command (all grep arugments are appliable) eg. 'java LogQueryClient -E "Hello.*World"' Note: - A unit test suite is created to perform different tests and verify the results.The different tests cover "rare", "frequent" and "somewhat frequent" patterns for one/all/some machines. The script can be used with varying log file sizes. To avoid typing password each time the unit test is run, please follow the instruction in the following link to enable SSH connection without typing password: http://www.linuxproblem.org/art_9.html - Add the node list with port numbers and log file paths to the unit test script before executing the script 'make unittest' or './LogQueryUnitTest.sh' ---------------------------------------------------------------------- Log Generator ---------------------------------------------------------------------- Log generator can be use to create sample log files that matches the real world logs. The logging is done using random generator to ensure 'frequent', 'rare' and 'somewhat frequent' patterns in the log file Logs can be generated using below command "java LogGenerator <log_path> <file_size> <machine_id> <log_path> : The path that you store the output log file <file_size> : The size of log file to be generated <machine_id>: The node for which the log is generated. A logical id can be assigned for each machine
About
Distributed Log Query Systems
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published