[Dxspider-support] Self-spotting and de-duplication

Fri Jan 17 09:17:37 GMT 2025

If every time a spot is ignored is also added again to the dupe list, this
is a loop. Maybe is this the bug?

1737101764^(pc11) PCPROT: Duplicate Spot  IZ2LSC-97: PC61
7006.0|JF4IQS/1|1737101700|JF4IQS|IZ2LSC-97 ignored

view_dupes JF4IQS/1
|JF4IQS/1|JF4IQS|433060||1737100440|POTAJP1149FMHIK = 17-Jan-2025 0754Z
expires 17-Jan-2025 0824Z
|JF4IQS/1|JF4IQS|7005||1737100800|POTAJP1149CWHIK = 17-Jan-2025 0801Z
expires 17-Jan-2025 0831Z
|JF4IQS/1|JF4IQS|7005|[-POTA-] JP-114 = 17-Jan-2025 0801Z expires
17-Jan-2025 0831Z
|JF4IQS/1|JI2OJV|7005||1737101580|POTAJP1149CWHIK = 17-Jan-2025 0814Z
expires 17-Jan-2025 0844Z
|JF4IQS/1|JI2OJV|7005|[-POTA-] JP-114 = 17-Jan-2025 0814Z expires
17-Jan-2025 0844Z
*|JF4IQS/1|JF4IQS|7005||1737101700|POTAJP1149CWHIK = 17-Jan-2025 0816Z
expires 17-Jan-2025 0846Z*

-->

On Thu, Jan 16, 2025 at 11:01 PM IZ2LSC <iz2lsc.andrea at gmail.com> wrote:

> Kin,
> when the hash is expired, is it removed from the file?
> I can see hash already expired and still present in the file.
> Not sure if this is expected.
>
> Andrea
>
> -->
>
>
> On Thu, Jan 16, 2025 at 8:12 PM Björn Ekelund via Dxspider-support <
> dxspider-support at tobit.co.uk> wrote:
>
>> Thanks. For human spots it makes perfect sense to include the comment.
>>
>> Björn SM7IUN
>>
>> On Thu, Jan 16, 2025 at 7:06 PM Kin <ea3cv at cronux.net> wrote:
>>
>>> Björn,
>>>
>>>
>>>
>>> All this is for non-RBN spots.
>>>
>>> I don't remember now the treatment of RBN spots.
>>>
>>>
>>>
>>> Kin
>>>
>>>
>>>
>>> *De:* Björn Ekelund <bjorn at ekelund.nu>
>>> *Enviado el:* jueves, 16 de enero de 2025 18:26
>>> *Para:* Kin <ea3cv at cronux.net>
>>> *CC:* The DXSpider Support list <dxspider-support at tobit.co.uk>
>>> *Asunto:* Re: [Dxspider-support] Self-spotting and de-duplication
>>>
>>>
>>>
>>> Kin,
>>>
>>>
>>>
>>> this was very clear and very helpful.
>>>
>>> Thank you for spending the time to do it.
>>>
>>> I am sure others appreciate it too.
>>>
>>>
>>>
>>> In my opinion, all comments should be ignored when determining dupes for
>>> skimmer spots.
>>>
>>>
>>>
>>> Björn SM7IUN
>>>
>>>
>>>
>>> On Thu, Jan 16, 2025 at 10:28 AM Kin <ea3cv at cronux.net> wrote:
>>>
>>> Morning Björn,
>>>
>>>
>>>
>>> *>So timegranularity defines a window within which spots are considered
>>> to have *
>>>
>>> *>the same time, Does this then effectively define the minimum
>>> respotting time?*
>>>
>>>
>>>
>>> Yes, timegranularity effectively defines the minimum respotting time. It
>>> determines the size of the time window within which spots are considered
>>> to have the same timestamp. For example, if timegranularity is set to
>>> 600 seconds (10 minutes), any spots reported within the same 10-minute
>>> interval will be grouped together, effectively preventing additional
>>> spots from being considered separate unless they occur in a different
>>> interval.
>>>
>>>
>>>
>>> *>Or is this the function of dupage? If it is, then I do not see the
>>> purpose of quantizing *
>>>
>>> *>time into timegranularity steps.*
>>>
>>>
>>>
>>> timegranularity is about standardizing time representation, while dupage
>>> enforces the minimum time between duplicate spots.
>>>
>>>
>>>
>>> *>And how can 600 be 4 minutes? Is time counted in steps of 0.4s?*
>>>
>>> *>Does this mean that dupage 3600 is 24 minutes?*
>>>
>>>
>>>
>>> *>And what is the function of qrggranularity? My empirical experiments
>>> say that I need *
>>>
>>> *>to move 10kHz away for DXSpider to not consider a new spot a dupe. Not
>>> 25kHz.*
>>>
>>>
>>>
>>> I hope that the following examples will clarify how it works.
>>>
>>>
>>>
>>> *>All I'm trying to do is to set up my node so that *
>>>
>>>
>>>
>>> *>1. a new spot of the same station on the same QRG after more than 3
>>> minutes is never considered a dupe*
>>>
>>>
>>>
>>> $Spot::timegranularity = 60
>>>
>>> $Spot::dupage = 180
>>>
>>>
>>>
>>> You should know that a duplicate is the sum of the fields
>>> spotted+spotter+qrg+time+comments, which will generate a hash like this:
>>> |IZ0ARL|IK2JTS|7075||1737012600|DTMBAI1823RM. For this reason, just
>>> using spotted+qrg+time is not sufficient to determine duplication.
>>>
>>>
>>>
>>> *>2. a new spot of a station 1kHz away from the previous spot is never
>>> considered a dupe.*
>>>
>>>
>>>
>>> $Spot::qrggranularity = 1
>>>
>>>
>>>
>>> *>3. self spots are never ever blocked*
>>>
>>>
>>>
>>> $Spot::minselfspotqrg = 0
>>>
>>>
>>>
>>>
>>>
>>> Example
>>>
>>>
>>>
>>> * Variables
>>>
>>>
>>>
>>> $Spot::timegranularity = 600    --> 10 min
>>>
>>> $Spot::qrggranularity = 25      --> 25 kHz
>>>
>>> $Spot::dupage = 3600            --> 1 h
>>>
>>>
>>>
>>> * Spot
>>>
>>>
>>>
>>> DX de IK2JTS:     7065.0  IZ0ARL       DTMBA I1823RM
>>> 0731Z (1737012660)
>>>
>>>
>>>
>>>
>>>
>>> Adjusts to the nearest multiple (either downward or upward)
>>>
>>>
>>>
>>> *1737012660*/600 = 2895021.1 --> 2895021 (round to the nearest integer)
>>>
>>> 2895021*600 = *1737012600*
>>>
>>>
>>>
>>> *7065.0*/25 = 282.6 --> 283 (round to the nearest integer)
>>>
>>> 283×25 = *7075*
>>>
>>>
>>>
>>> * dupefile
>>>
>>>
>>>
>>> |IZ0ARL|IK2JTS|*7075*||*1737012600*|DTMBAI1823RM = 16-Jan-2025 0731Z
>>> expires 16-Jan-2025 0831Z
>>>
>>>
>>>
>>> This means that time is fixed in 10-minute intervals, and frequency is
>>> fixed in 25 kHz steps.
>>>
>>>
>>>
>>>
>>>
>>> * Comparison with different values for the same spot:
>>>
>>>
>>>
>>> DX de IK2JTS:     *7065.0*  IZ0ARL       DTMBA
>>> #I-1823RM                  *0735Z* (*1737012900*)
>>>
>>>
>>>
>>> Scenario 1:
>>>
>>>
>>>
>>> $Spot::timegranularity = 600
>>>
>>> $Spot::qrggranularity = 25
>>>
>>> $Spot::dupage = 3600
>>>
>>>
>>>
>>> |IZ0ARL|IT9JZK|*7075*||*1737013200*|DTMBAI1823RM = 16-Jan-2025 *0735Z*
>>> expires 16-Jan-2025 *0835Z*
>>>
>>>                        (*0740Z*)
>>>
>>>
>>>
>>> Scenario 2:
>>>
>>>
>>>
>>> $Spot::timegranularity = 60
>>>
>>> $Spot::qrggranularity = 1
>>>
>>> $Spot::dupage = 300
>>>
>>>
>>>
>>> |IZ0ARL|IT9JZK|*7065*||*1737012900*|DTMBAI1823RM = 16-Jan-2025 *0735Z*
>>> expires 16-Jan-2025 *0740Z*
>>>
>>>                       (*0735Z*)
>>>
>>>
>>>
>>>
>>>
>>> I'm not sure if I’ve managed to explain myself clearly this time. This
>>> is my interpretation after analyzing the code, from someone who is not
>>> an expert. I think the person who can help us best is Dirk, the creator
>>> of this. And sorry because my English is terrible
>>>
>>>
>>>
>>> Kin EA3CV
>>>
>>>
>>>
>>> _______________________________________________
>> Dxspider-support mailing list
>> Dxspider-support at tobit.co.uk
>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.tobit.co.uk/pipermail/dxspider-support/attachments/20250117/c632be77/attachment-0001.htm>