[Dxspider-support] Top users command ?

[Boudewijn (Bob) Tenty]

Very handy, thanks Niels.

73,

Bob/Boudewijn VE3TOK

On 2023-03-26 09:50, Niels PD9Q via Dxspider-support wrote:
> Hi,
>
> Way back I wrote a ugly bash script. On the other hand, it works quite nicely. The output will look something like this.
>
> ** Output Generated on 03-26-2023 15:41:23
> ** Log file : /spider/local_data/log/2023/03.dat
> ** Files size : 67M
> ** Lines in the log file : 1184821
> ** Total of inbound connects : 321
> ** Total users : 29
> ** Connected Stations over a month
> =====================================================================================
> Wed  Mar  1   01:21:31  2023  IW1QLH-34  connected  from (removed ip-address)
> Wed  Mar  1   02:22:02  2023  IW1QLH-35  connected  from
> Wed  Mar  1   03:22:32  2023  IW1QLH-32  connected  from
> Wed  Mar  1   08:24:31  2023  IW1QLH-33  connected  from
> <snip>
> Mon  Mar  13  19:14:55  2023  EC3BH-5    connected  from
> Tue  Mar  14  07:52:04  2023  HB9VQQ     connected  from
> Sat  Mar  18  16:27:45  2023  HB9BB      connected  from
> Tue  Mar  21  09:59:46  2023  EA3CV-1    connected  from
> Tue  Mar  21  18:32:09  2023  EA3CV-22   connected  from
> Sat  Mar  25  01:24:47  2023  IZ3WQE     connected  from
> =====================================================================================
> ** End Connected Stations
>
> ** Last Connected Stations - Return Connects
> =====================================================================================
> Sun  Mar  26  04:55:11  2023  YV5GRB     connected  from  (removed ip-address)
> Sat  Mar  25  17:13:39  2023  ON3SCA     connected  from
> Sat  Mar  25  11:37:45  2023  IW1QLH-35  connected  from
> Fri  Mar  24  12:44:59  2023  PA3DRQ     connected  from
> <snip>
> Thu  Mar  9   16:31:12  2023  PA5ZL      connected  from
> Thu  Mar  9   14:04:38  2023  HB9CU      connected  from
> Sat  Mar  4   00:50:58  2023  PJ7FM      connected  from
> Fri  Mar  3   06:15:20  2023  HB9BQP     connected  from
> Thu  Mar  2   12:21:30  2023  PH4E       connected  from
> =====================================================================================
> ** End last Connected Stations
>
> Nr  Call       Times
> 1   BI4MPH     5
> 2   EA3CV-1    1
> 3   EA3CV-22   1
> 4   EC3BH-1    2
> 5   EC3BH      4
> 6   EC3BH-5    1
> 7   G1DIF      1
> 8   G7VJA      2
> 9   HB9BB      7
> 10  HB9BQP     7
> 11  HB9BZA     2
> 12  HB9CU      4
> 13  HB9VQQ     7
> 14  IW1QLH-32  32
> 15  IW1QLH-33  32
> 16  IW1QLH-34  30
> 17  IW1QLH-35  27
> 18  IZ3WQE     1
> 19  LB4NH      2
> 20  M0YOL      37
> 21  ON3SCA     6
> 22  PA2WCB     62
> 23  PA3DRQ     8
> 24  PA5ZL      2
> 25  PH4E       5
> 26  PH9M       1
> 27  PJ7FM      2
> 28  SN5Y       1
> 29  YV5GRB     29
> Script Runtime 327ms
>
> Copy and past in a file you like and chmod +x (filename)
>
> #!/bin/bash
> ts=$(date +%s%N)
> # Path to the log dir
> path=/spider/local_data/log
> # Get Month and Year.
> month=$(date +%m)
> year=$(date +%Y)
> # Get the log file.
> file="$path"/"$year"/"$month".dat
> if [ -f "$file" ]; then
>     a="$path"/"$year"/"$month".dat
> else
>     echo "$file does not exist. Error, check the path."
> fi
> # Get some info about the log file
> size=$(du -sh "$path"/"$year"/"$month".dat | awk '{ print $1 }')
> line=$(wc -l "$a")
> # Get the connected from.
> b=$(sed -n '/connected from/p' "$a")
> # Remove some mess.
> c=$(echo "$b" | sed -e '/\>^DXProt^\</d' -e 's/\>^DXCommand^\</ /g')
> # Remove connecteds from localhost (Nobody have to see that i`m connecting 1000 times a day) Remove sk1/sk0.
> d=$(echo "$c" | sed -e '/127.0.0.1/d' -e '/SK0MMR/d' -e '/SK1MMR/d' -e '/PD9Q/d')
> # First connect from call... uniq on the second column.
> e=$(echo "$d" | awk -F '[ .]' '!uniq[$2]++')
> # Last connect from call sort bottom to top / uniq on the second column.
> f=$(echo "$d" | sort -t: -k 1nr,1 | awk -F '[ .]' '!uniq[$2]++')
> # Change the epoch time to reable text (can find any good option in bash so perl it is).
> g=$(echo "$e" | perl -pe 's/(\d+)/localtime($1)/e')
> h=$(echo "$f" | perl -pe 's/(\d+)/localtime($1)/e')
> # Echo to a file, need it to compare.
> echo "$g" > first
> echo "$h" > last
> grep -vxFf first last >last.tmp && mv last.tmp last
> i=$(cat first)
> j=$(cat last )
> # ** Count times connected
> # Get only the calls
> m=$(echo "$d" | awk -F' ' '{ print $2}')
> #
> count_total=$(echo "$m" | wc -l)
> # Count the uniq calls, add the times off connection and sort
> n=$(echo "$m" | awk -- '{for (i = 1; i <= NF; i++) wc[$i] +=1}; END {for (w in wc) print w, wc[w]}' | sort)
> count_call=$(echo "$n" | wc -l)
> # Add line number
> o=$(echo "$n" |awk -v ln=1 '{print ln++  " "  $0 }')
> # Add tab between numbers and words
> p=$(echo "$o" | awk -v OFS='\t\t' '{ print $1, $2, $3 }')
>
> div================================================================
> div=$div$div
>
> width=85
>
> printf "\n** Output Generated on %(%m-%d-%Y %H:%M:%S)T\n" $(date +%s)
> printf "** Log file : "$a"\n"
> printf "** Files size : "$size"\n"
> printf "** Lines in the log file : "$line""
> printf "\n** Total of inbound connects : "$count_total""
> printf "\n** Total users : "$count_call"\n"
> printf "** Connected Stations over a month\n"
> printf "%$width.${width}s\n" "$div"
> printf "$i\n" | column -t
> printf "%$width.${width}s\n" "$div"
> printf "** End Connected Stations\n\n"
>
> printf "** Last Connected Stations - Return Connects\n"
> printf "%$width.${width}s\n" "$div"
> printf "$j\n" | column -t
> printf "%$width.${width}s\n" "$div"
> printf "** End last Connected Stations\n"
>
>
> header="\n%-0s %5s %11s\n"
> width=38
> printf "$header" "Nr" "Call" "Times"
> printf "$p\n" | column -t
> echo "Script Runtime $((($(date +%s%N) - $ts)/1000000))ms"
>
>  73 Niels PD9Q / PI1LAP-1
>
> Op 26-3-2023 om 05:32 schreef Tim Tuck via Dxspider-support:
>> Hi all,
>>
>> I want to turn on registration on my node but I'd like to pre-register my top users in the process to minimise any pain.
>>
>> The question then is... who are my top users ?
>>
>> I couldn't find a command to show me any form of database of who has connected to and used my node over the past month other than parsing the log files for the connected statements.
>>
>> So before I go do some scripting around the log files I thought I'd ask if anyone has done this and can share  :)
>>
>> Of course I could be missing something obvious so I stand by for a face palm moment!
>>
>> thanks
>>
>> Tim
>>
>>
>
> _______________________________________________
> Dxspider-support mailing list
> Dxspider-support at tobit.co.uk
> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support

-- 
There is nothing permanent except change
  
-Heraclitus