[Dxspider-support] Is dedupe really working?
IZ2LSC
iz2lsc.andrea at gmail.com
Sat Feb 1 21:48:25 GMT 2025
Please wait a second.
if my dupage = 1800
this should mean that the same spot is not accepted and forwarded for 30
min.
So why Kin you have the same spot every minute:
sh/dx 0TEST
7070.0 0TEST 1-Feb-2025 1931Z test ignore <EA3CV>
7070.0 0TEST 1-Feb-2025 1930Z test ignore <EA3CV>
7070.0 0TEST 1-Feb-2025 1928Z test ignore <EA3CV>
I expect to see only the first spot, and no more dupes for the next 30 min.
Or am I misunderstanding the meaning of $dupeage and $timegranularity ?
How I can accomplish the goal of not accepting dupes if they are sent
within 30 min?
And for dupes I mean the same spotter, dx, qrg, comment ?
Thanks
Andrea
iz2lsc
-->
On Sat, Feb 1, 2025 at 9:04 PM Dirk Koopman via Dxspider-support <
dxspider-support at tobit.co.uk> wrote:
> A slightly edited version done on a decent screen and not a mobile phone
> with auto correct...
>
> Edits underlined
>
> On 01/02/2025 18:15, Dirk Koopman via Dxspider-support wrote:
>
> The problem I have is that I *cannot* see what you are seeing. And
> further more, * if you are getting all these dupes you would be seeing
> multiple spots on your console*.
>
> What I suggest you do is to stop your node, delete your
> /spider/local_cmd/dupefile then restart your node.
>
> Startup a console and type set/debug *dxdupe*. In another shell window
> type ‘watchdbg dxdupe’.
>
> Then wait. In amongst all the dupe add lines, once a minute you will see a
> line with the count of dupe records there are. If you have $Spot::dupe
> age set to 10 minutes, after that time has passed deletions should start to
> occur and the count will settle down to a slowly rising count (caused by 3
> hours of announces) and then will stabilise to a number which will be in
> the lowish hundreds (as it’s a contest weekend). On normal days I see
> something between 120 - 160 ish.
>
>
>
> You will see something like this (one minute's worth):
>
> 19:26:38 (dxdupe) DXDupe::add key: XZ32TO|IK4ZIF|7182|1738438200||EUDXC
> time: 1936Z
> 19:26:45 (dxdupe) DXDupe::add key: XEA4DEI|KB8OTK|24917|1738438200||
> time: 1936Z
> 19:26:50 (dxdupe) DXDupe::add key:
> XK1LZ|TF1A|14200|1738438200||59INICELANDCQCO time: 1936Z
> 19:26:50 (dxdupe) DXDupe::add key: XK1LZ|TF1A|14200|1738438200|5/9 IN
> ICELAND time: 1936Z
> 19:27:05 (dxdupe) DXDupe::add key: XPA4O|RX7T|7015|1738438200|| time:
> 1937Z
> 19:27:06 (dxdupe) DXDupe::add key: XWA0MN|N0UR|28044|1738438200||MNQPSTE
> time: 1937Z
> 19:27:06 (dxdupe) DXDupe::add key: XWA0MN|N0UR|28044|1738438200|MNQP STE
> time: 1937Z
> 19:27:16 (dxdupe) DXDupe::add key:
> XPE9GHZ/B|PA2DOL|5760815|1738438200||JO21GXTRJO11WM4 time: 1937Z
> 19:27:16 (dxdupe) DXDupe::add key:
> XPE9GHZ/B|PA2DOL|5760815|1738438200|JO21GX(TR)JO11W time: 1937Z
> 19:27:22 (dxdupe) DXDupe::add key:
> XAI0M|W2VTV|28420|1738438200||MNOLMSTED time: 1937Z
> 19:27:22 (dxdupe) DXDupe::add key: XAI0M|W2VTV|28420|1738438200|MN
> OLMSTED time: 1937Z
> 19:27:29 (dxdupe) DXDupe::add key: XF5IN|RX7T|7016|1738438200|| time:
> 1937Z
> 19:27:34 (dxdupe) DXDupe::add key: XWA0JGT|AE9LL|14245|1738438200|| time:
> 1937Z
> 19:27:35 (dxdupe) DXDupe::add key:
> XYQ5A|UB3LBU|3573|1738438200||THEYEAROFHOMAGE time: 1937Z
> 19:27:35 (dxdupe) DXDupe::add key: XYQ5A|UB3LBU|3573|1738438200|THE YEAR
> OF HOM time: 1937Z
> 19:27:36 (dxdupe) DXDupe::del key: XDK2NI|UA3BL|3740|1738437600||CONTEST
> time: 1926Z
> 19:27:36 (dxdupe) DXDupe::del key: XHA6QU|HA5KFU|1837|1738437600|EU TEST
> HA6QU H time: 1926Z
> 19:27:36 (dxdupe) DXDupe::del key: XK3MTO|K5YDD|18128|1738437600||POTA
> time: 1927Z
> 19:27:36 (dxdupe) DXDupe::del key: XOM6TX|SP5BK|3731|1738437600|| time:
> 1927Z
> 19:27:36 (dxdupe) DXDupe::del key: XTG9AHM|AD4YQ|24985|1738437600|| time:
> 1927Z
> 19:27:36 (dxdupe) DXDupe::del key:
> XHA6QU|HA5KFU|1837|1738437600||EUTESTHA6QUHU04 time: 1926Z
> 19:27:36 (dxdupe) DXDupe::del key: XNQ1B|N3KN|21049|1738437600||CW time:
> 1926Z
> 19:27:36 (dxdupe) DXDupe::del key: XPV2K|PY2OF|14248|1738437600|CQ TEST
> time: 1926Z
> 19:27:36 (dxdupe) DXDupe::del key: XPV2K|PY2OF|14248|1738437600||CQTEST
> time: 1926Z
> 19:27:36 (dxdupe) DXDupe::del key: XVE3GJP|N5MEF|14250|1738437600|| time:
> 1927Z
> 19:27:36 (dxdupe) DXDupe::del key: XSQ6PLE|DK1HJG|3586|1738437600||RTTY
> time: 1926Z
> 19:27:36 (dxdupe) DXDupe::del key: XEC1A|KC2GOW|21092|1738437600||RTTY
> time: 1927Z
> 19:27:36 (dxdupe) DXDupe::del key: XPA3DUU|LZ2HA|7058|1738437600||RTTY
> time: 1926Z
> 19:27:36 (dxdupe) DXDupe::per_minute number of records 258
>
> It is normal to see two entries where there is a comment on the spot and
> the "compressed and normalised" version is different from the raw comment.
> "Normalised" means: capitalise all letters and remove everything that is
> not a letter or a digit.
>
> The "number of records" is the current number of records in left in
> dupefile + the number of records it has just deleted. Just me being lazy,
> I'll change it to be the correct number at the time of printing. This
> instance has been running here since about 21h00 last night so should be
> representative of this level of activity.
>
> A swift "grepdbg dxdupe per minute", just now, yields:
>
> 01Feb2025 at 19:26:36 (dxdupe) DXDupe::per_minute number of records 256
> 01Feb2025 at 19:27:36 (dxdupe) DXDupe::per_minute number of records 258
> 01Feb2025 at 19:28:36 (dxdupe) DXDupe::per_minute number of records 263
> 01Feb2025 at 19:29:36 (dxdupe) DXDupe::per_minute number of records 271
> 01Feb2025 at 19:30:36 (dxdupe) DXDupe::per_minute number of records 262
> 01Feb2025 at 19:31:36 (dxdupe) DXDupe::per_minute number of records 251
> 01Feb2025 at 19:32:36 (dxdupe) DXDupe::per_minute number of records 246
> 01Feb2025 at 19:33:36 (dxdupe) DXDupe::per_minute number of records 244
> 01Feb2025 at 19:34:36 (dxdupe) DXDupe::per_minute number of records 247
> 01Feb2025 at 19:35:36 (dxdupe) DXDupe::per_minute number of records 239
> 01Feb2025 at 19:36:36 (dxdupe) DXDupe::per_minute number of records 234
> 01Feb2025 at 19:37:36 (dxdupe) DXDupe::per_minute number of records 249
> 01Feb2025 at 19:38:36 (dxdupe) DXDupe::per_minute number of records 256
> 01Feb2025 at 19:39:36 (dxdupe) DXDupe::per_minute number of records 248
> 01Feb2025 at 19:40:36 (dxdupe) DXDupe::per_minute number of records 260
> 01Feb2025 at 19:41:36 (dxdupe) DXDupe::per_minute number of records 259
> 01Feb2025 at 19:42:36 (dxdupe) DXDupe::per_minute number of records 264
>
> This is a more "normal" level:
>
> 30Jan2025 at 11:40:12 (dxdupe) DXDupe::per_minute number of records 92
> 30Jan2025 at 11:41:12 (dxdupe) DXDupe::per_minute number of records 100
> 30Jan2025 at 11:42:26 (dxdupe) DXDupe::per_minute number of records 110
> 30Jan2025 at 11:44:05 (dxdupe) DXDupe::per_minute number of records 100
> 30Jan2025 at 11:44:45 (dxdupe) DXDupe::per_minute number of records 109
> 30Jan2025 at 11:45:15 (dxdupe) DXDupe::per_minute number of records 112
> 30Jan2025 at 11:46:12 (dxdupe) DXDupe::per_minute number of records 119
> 30Jan2025 at 11:47:12 (dxdupe) DXDupe::per_minute number of records 123
> 30Jan2025 at 11:48:12 (dxdupe) DXDupe::per_minute number of records 123
> 30Jan2025 at 11:49:12 (dxdupe) DXDupe::per_minute number of records 124
> 30Jan2025 at 11:50:12 (dxdupe) DXDupe::per_minute number of records 125
> 30Jan2025 at 11:51:12 (dxdupe) DXDupe::per_minute number of records 125
> 30Jan2025 at 11:52:12 (dxdupe) DXDupe::per_minute number of records 113
> 30Jan2025 at 11:53:12 (dxdupe) DXDupe::per_minute number of records 119
> 30Jan2025 at 11:54:12 (dxdupe) DXDupe::per_minute number of records 115
> 30Jan2025 at 11:55:12 (dxdupe) DXDupe::per_minute number of records 115
> 30Jan2025 at 11:56:12 (dxdupe) DXDupe::per_minute number of records 115
> 30Jan2025 at 11:57:12 (dxdupe) DXDupe::per_minute number of records 111
> 30Jan2025 at 11:58:12 (dxdupe) DXDupe::per_minute number of records 108
> 30Jan2025 at 11:59:12 (dxdupe) DXDupe::per_minute number of records 105
> 30Jan2025 at 12:00:12 (dxdupe) DXDupe::per_minute number of records 99
> 30Jan2025 at 12:01:12 (dxdupe) DXDupe::per_minute number of records 99
> 30Jan2025 at 12:02:12 (dxdupe) DXDupe::per_minute number of records 109
> ...
>
> Further, just now on my test machine, I have done the same as you:
>
> dx IZ2ABC 7090 test 2
> DX de G7BRN: 7090.0 IZ2ABC test 2 15 1946Z 14
> G7BRN de GB7TLH 1-Feb-2025 1946Z dxspider >
> dx IZ2ABC 7090 test 2
> Sorry, this is a duplicate
> G7BRN de GB7TLH 1-Feb-2025 1946Z dxspider >
> dx IZ2ABC 7090 test 2
> Sorry, this is a duplicate
> G7BRN de GB7TLH 1-Feb-2025 1946Z dxspider >
>
> So, as far as I am concerned - just now at least - I believe deduping for
> spots is working.
>
> 73 Dirk G1TLH
>
>
> _______________________________________________
> Dxspider-support mailing list
> Dxspider-support at tobit.co.uk
> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.tobit.co.uk/pipermail/dxspider-support/attachments/20250201/2c098560/attachment-0001.htm>
More information about the Dxspider-support
mailing list