[Dxspider-support] Kind of log parser
Niels
pd9q at packet-radio.net
Wed Dec 30 19:14:05 CET 2020
Hi,
I am writing some kind of log parser in Bash. It generates some info
about the connected station over the last month, and some returning
users. Number of connect per users. Now I have two questions. Since I
am only a very,very,very small node, I am looking for a large log file.
My log file of a month is about 50Mb. I would like a log file of the Big
Boys. Is there anyone who would like to make it available for download?
I would like to use this to tweak and speed up my script. At this point,
the script reads and processes approximately 865,000 lines in 255ms. I
also have 1 line perl in the script I would like to change to Bash, but
I cannot find a good solution for this line.
# Change the epoch time to readable text.
g=$(echo "$e" | perl -pe 's/(\d+)/localtime($1)/e')
"$e" look like this
1606784607 G7VJA connected from 81.174.245.245
1606817130 WA1UHA connected from 73.34.224.142
1606841436 PA1RBZ-2 connected from 84.17.47.9
1606926662 G0RQL connected from 81.174.171.24
1606993290 PD9Q connected from 188.207.110.139
1607248295 IW3HER connected from 185.138.39.139
1607257552 W4OA connected from 162.223.105.204
With the above perl code it` look like this.
Tue Dec 1 02:03:27 2020 G7VJA connected from 81.174.245.245
Tue Dec 1 11:05:30 2020 WA1UHA connected from 73.34.224.142
Tue Dec 1 17:50:36 2020 PA1RBZ-2 connected from 84.17.47.9
Wed Dec 2 17:31:02 2020 G0RQL connected from 81.174.171.24
Thu Dec 3 12:01:30 2020 PD9Q connected from 188.207.110.139
Sun Dec 6 10:51:35 2020 IW3HER connected from 185.138.39.139
Sun Dec 6 13:25:52 2020 W4OA connected from 162.223.105.204
I am looking for a bash option (awk, sed (dont think sed can do this)).
Thanks for the time.... Hoop that someone can help me with a log file
of the perl code
The output of the script look like this at the moment.
** Output Generated on 12-30-2020 19:06:50
** Log file : /spider/local_data/log/2020/12.dat
** Files size : 49M
** Lines in the log file : 866023
** Times of inbound connects : 83
** Connected Stations over a month
==============================================================
Tue Dec 1 02:03:27 2020 G7VJA connected from 81.174.245.245
Tue Dec 1 11:05:30 2020 WA1UHA connected from 73.34.224.142
Tue Dec 1 17:50:36 2020 PA1RBZ-2 connected from 84.17.47.9
Wed Dec 2 17:31:02 2020 G0RQL connected from 81.174.171.24
Thu Dec 3 12:01:30 2020 PD9Q connected from 188.207.110.139
Sun Dec 6 10:51:35 2020 IW3HER connected from 185.138.39.139
Sun Dec 6 13:25:52 2020 W4OA connected from 162.223.105.204
Sun Dec 6 13:44:40 2020 PJ4EL connected from 190.4.70.236
Tue Dec 8 19:16:32 2020 LA3SHA connected from 84.208.137.133
Wed Dec 9 19:54:36 2020 EA1GFY connected from 84.76.91.185
Thu Dec 10 04:02:56 2020 WB4QOJ connected from 108.226.130.238
Sun Dec 13 04:32:27 2020 YS1RR connected from 190.53.13.104
Tue Dec 15 20:21:38 2020 ON1GL connected from 91.179.152.167
Wed Dec 16 19:04:05 2020 GE8DNH connected from 198.46.81.55
Wed Dec 16 22:33:37 2020 WB8JAY connected from 76.189.232.252
Sat Dec 19 13:35:05 2020 KD8MJL connected from 172.58.140.178
Sat Dec 19 14:00:52 2020 KD8MJJ connected from 24.236.102.184
Wed Dec 23 13:17:40 2020 PE1KDA connected from 213.126.72.11
Sat Dec 26 09:36:24 2020 S50U connected from 86.61.79.218
Sat Dec 26 23:51:11 2020 N5XL connected from 64.201.126.41
Tue Dec 29 19:36:36 2020 PE1OWG connected from 77.175.251.204
Wed Dec 30 11:57:23 2020 PF7X connected from 212.238.138.128
==============================================================
** End Connected Stations
** Last Connected Stations - Return Connects
==============================================================
Wed Dec 30 16:29:10 2020 PF7X connected from 212.238.138.128
Wed Dec 30 13:03:40 2020 WB4QOJ connected from 172.58.6.242
Wed Dec 30 08:53:48 2020 N5XL connected from 64.201.126.41
Tue Dec 29 00:42:23 2020 WB8JAY connected from 76.189.232.252
Sat Dec 26 20:01:44 2020 G7VJA connected from 81.174.245.245
Sat Dec 26 15:51:02 2020 GE8DNH connected from 198.46.81.55
Sat Dec 19 14:02:07 2020 KD8MJL connected from 172.58.140.178
Sat Dec 12 13:19:32 2020 EA1GFY connected from 84.76.91.185
Thu Dec 10 04:52:43 2020 W4OA connected from 162.223.105.204
Tue Dec 8 19:24:07 2020 LA3SHA connected from 84.208.137.133
Mon Dec 7 04:08:16 2020 PJ4EL connected from 190.4.70.236
==============================================================
** End last Connected Stations
Number Call Times
======================================
1 EA1GFY 2
2 G0RQL 1
3 G7VJA 19
4 GE8DNH 12
5 IW3HER 1
6 KD8MJJ 1
7 KD8MJL 2
8 LA3SHA 2
9 N5XL 3
10 ON1GL 1
11 PA1RBZ 1
12 PD9Q 1
13 PE1KDA 1
14 PE1OWG 1
15 PF7X 3
16 PJ4EL 4
17 S50U 1
18 W4OA 20
19 WA1UHA 1
20 WB4QOJ 2
21 WB8JAY 3
22 YS1RR 1
Script Runtime 251ms
real 0m0.258s
user 0m0.152s
sys 0m0.064s
73 Niels PD9Q
https://packet-radio.net
http://dx.packet-radio.nl
telnet://dx.packet-radio.nl 7300
More information about the Dxspider-support
mailing list