[Dxspider-support] Self-spotting and de-duplication

Kin ea3cv at cronux.net
Wed Jan 15 10:10:09 GMT 2025 

Hi Andrea,
 
I have been looking at the dupefile, and I see no inconsistencies in the timings. It could be that there is a bug in the code, I don't know.
 
If you want to see the contents of dupefile, you can use this script: view_duples.pl
 
https://github.com/EA3CV/dxspider_info/blob/main/view_dupes.pl
 
Kin EA3CV
 
 
De: IZ2LSC <iz2lsc.andrea at gmail.com> 
Enviado el: miércoles, 15 de enero de 2025 8:46
Para: Kin <ea3cv at cronux.net>
CC: The DXSpider Support list <dxspider-support at tobit.co.uk>; Simon Ravnič <s53zo at t-2.net>
Asunto: Re: [Dxspider-support] Self-spotting and de-duplication
 
Kin, 
even without changing any parameters from default, some spots are wrongly classified as dupes. Please consider that I have seen dupes detected after 3 hrs from the previous spot (dupeage is 1hr by default) on a different QRG and with different comments.
 
 
Andrea
 
-->
 
 
Il giorno mar 14 gen 2025 alle ore 17:48 Kin <ea3cv at cronux.net <mailto:ea3cv at cronux.net> > ha scritto:
Andrea,
 
If you change any variables related to dupes, you should delete the /spider/local_data/dupefile and then restart the node.
If there is an entry in dupefile, it is possible that it will match even if you have changed the variable.
 
Kin EA3CV
 
 
De: IZ2LSC <iz2lsc.andrea at gmail.com <mailto:iz2lsc.andrea at gmail.com> > 
Enviado el: martes, 14 de enero de 2025 9:09
Para: The DXSpider Support list <dxspider-support at tobit.co.uk <mailto:dxspider-support at tobit.co.uk> >
CC: Kin <ea3cv at cronux.net <mailto:ea3cv at cronux.net> >; Simon Ravnič <s53zo at t-2.net <mailto:s53zo at t-2.net> >
Asunto: Re: [Dxspider-support] Self-spotting and de-duplication
 
Even changing the set/var $Spot::dupage doesn't produce the expected result.

You can refer to this thread: https://mailman.tobit.co.uk/pipermail/dxspider-support/2024-September/018992.html
 
73
Andrea, IZ2LSC
 
 
 
-->
 
 
Il giorno mar 14 gen 2025 alle ore 08:28 Kin via Dxspider-support <dxspider-support at tobit.co.uk <mailto:dxspider-support at tobit.co.uk> > ha scritto:
It is advisable not to touch Stop.pm. 
It would be enough to do:
 
set/var $Spot::dupage = XXXX
set/var $Spot::timegranularity = ZZZZ
 
The default values I have are:
 
$Spot::dupage = 3600
$Spot::timegranularity = 600
 
Regards,
 
Kin EA3CV
 
 
De: Dxspider-support <dxspider-support-bounces at tobit.co.uk <mailto:dxspider-support-bounces at tobit.co.uk> > En nombre de Simon Ravnic via Dxspider-support
Enviado el: lunes, 13 de enero de 2025 23:58
Para: The DXSpider Support list <dxspider-support at tobit.co.uk <mailto:dxspider-support at tobit.co.uk> >
CC: Simon Ravnič <s53zo at t-2.net <mailto:s53zo at t-2.net> >
Asunto: Re: [Dxspider-support] Self-spotting and de-duplication
 
There is an easy way to change it locally. In perl/Spot.pm change
 
dupage = 120;            
timegranularity = 120;   
 
to allow a dupe spot every 120 seconds. 
 
The issue that remains is that such post is visible only on the original node. It appears that it gets deduped on your partner nodes and is therefore not propagated to the network.
 
With the new contesting rules where more and more of them allow self-spotting I believe the network should accommodate to this fact.
 
73
Simon, S53ZO
S50DXS sysop
 
On 13 Jan 2025, at 22:05, IZ2LSC via Dxspider-support <dxspider-support at tobit.co.uk <mailto:dxspider-support at tobit.co.uk> > wrote:
 
Back to the main topic, 

Based on tests I did in the past, the logic behind the deduplication is not clear, or at least I cannot understand it.
Sometime ago I already wrote about this topic in the list, but no one answered.
I have seen many time spots classified as dupe even if a very long time has passed from last spot.
I also tried to change some parameters on the cluster configuration, but with no luck.
I end up by clearing on my cluster the dupe file every hour, to make the deduplication less aggressive.
 
73
Andrea, IZ2LSC
-->
 
 
Il giorno lun 13 gen 2025 alle ore 19:37 Björn Ekelund via Dxspider-support <dxspider-support at tobit.co.uk <mailto:dxspider-support at tobit.co.uk> > ha scritto:
Don't worry, my toes may be large but they can handle a good stepping. 
 
When I put ham radio aside around 1990 for family and career reasons, DX clusters did 
not really exist so when I returned in 2016 I had already missed the early days of clusters 
where all spots were made by humans and the terminal bell went off every time a spot arrived.
 
I can understand there may be hams wanting to stay with that. 
I enjoy CW for pretty much the same nostalgic reasons.
 
I'm of course perfectly fine with there being cluster nodes set up and optimized 
for this type of old school usage. As long the cluster software does not enforce it 
or prevents more modern ways.  
 
However, the way I see it, a casual user actually has ever more reason to embrace a 
more modern cluster usage. The casual user does not spend endless hours by the radio 
so he should have good reasons to maximize his productivity and/or fun. 
 
Having a software (like SpotCollector or HRDLog) monitor the cluster and present 
maps or lists or graphs with needed DXCC band slots, friends, event stations, etc. is a great 
way to do exactly this. 
 
And when you have a piece of software to collect your spots, information overload is no longer 
an issue and you can tap into as many information sources as you like; RBN, PSKReporter, IRC, etc. 
To get even more productive and have more fun. 
 
But there are of course different definitions of fun. People do a lot of things that are difficult or 
uncomfortable for fun. 
 
So perhaps I should have used a bit more respectful language when describing the practices of 
the early days of clusters.
 
Björn SM7IUN
 
On Mon, Jan 13, 2025 at 6:00 PM Rene Olsen via Dxspider-support <dxspider-support at tobit.co.uk <mailto:dxspider-support at tobit.co.uk> > wrote:
Hi.

To be honest I am not quite sure how to respond to this, without stepping on someones toes. 
But here it goes.

You are aware that the DX-Cluster system is also used by casual users, who couldn't care 
less about bandmaps, contests and what else?

DX-Clusters have been used just fine for 30+ years, without the use of RBN.

> A cluster node without skimmer spots is of very little value.

That is some statement. So what you are saying, is that all DX-Clusters without RBN feed 
might as well close down? I mean since they are of very little value.

RBN is probably a good thing for some. But the user can turn it OFF if the user don't want it.

I don't know how much RBN spots are used on various DX-Clusters. I can only speak for 
OZ5BBS-7. I have an average of maybe 45-55 users on a daily basis. Not many of them have 
enabled the RBN feed.

I doubt that the casual user of a DX-Cluster wants to get a DX spot of some W station spotting 
itself 930 times during the CQWW contest.

Its like everything revolves about contesting. Thats not the case.

Many users who use DX-Clusters, couldn't care less about contests.

I am not saying that DX-Clusters shouldn't evolve, and we should be stuck at how it was 30 
years ago. But, it should be done with the casual DX-Cluster user in mind as well.

I hope I didn't step on too many toes here :-)

Vy 73 de Rene / OZ1LQH


On 12 Jan 2025 at 22:51, Björn Ekelund via Dxspider- wrote:

> A cluster node without skimmer spots is of very little value.
> The reverse beacon network produces about 12 million spots on a
> big contest weekend. This is consolidated and de-duped into perhaps half a
> million.
> With this in mind, what makes a few thousand self spots a problem?
> 
> The days of watching the telnet feed scroll by are long gone.
> Today computers turn the spot flow into bandmap items and lists of wanted
> stations.
> The cluster connection is just a data feed. Not a user interface.
> 
> Björn SM7IUN
> 
> On Sun, Jan 12, 2025 at 9:41PM Rene Olsen via Dxspider-support <
> dxspider-support at tobit.co.uk <mailto:dxspider-support at tobit.co.uk> > wrote:
> 
> > Hi.
> >
> > Does this mean that we can get selfspots every 3 minutes from 5000+ users
> > or whatever?
> >
> > If thats the case it is just stupid.
> >
> > If it is really the case, it makes one wonder if it is even worth running
> > a spider node anymore. It
> > will totally ruin the entire idea of DX clusters. Which is not to allow
> > some OZ station or
> > whatever, to send self spot every 3 minutes 24/7.
> >
> > Just my opinion.
> >
> > Vy 73 de Rene / OZ1LQH
> >
> > On 12 Jan 2025 at 18:54, Björn Ekelund via Dxspider- wrote:
> >
> > > Some of the default settings of DXSpider seem optimized for users staring
> > > at the telnet feed.
> > >
> > > In my scripts/startup I have added
> > >
> > > set/var $RBN::respottime = 180
> > > set/var $Spot::minselfspotqrg  0
> > >
> > > A cluster node should never suppress a self spot. The minimum self
> > spotting
> > > periodicity is set
> > > by the contest's rules and the cluster should never interfere with this.
> > > Typical minimum periods for
> > > self spotting in contests are between 3 and 10 minutes.
> > >
> > > If DXSpider does suppress self spots under certain conditions I would
> > like
> > > to know how to disable this.
> > >
> > > Björn SM7IUN
> > >
> > > On Sun, Jan 12, 2025 at 4:35PM Keith, G6NHU via Dxspider-support <
> > > dxspider-support at tobit.co.uk <mailto:dxspider-support at tobit.co.uk> > wrote:
> > >
> > > > Now that self spotting is allowed for a trial period in all RSGB HF
> > > > contests, I´ve been asked about self-spotting and submitting duplicate
> > > > spots by a couple of my users.
> > > >
> > > > Some time ago I set minselfspotqrg to 0 to allow self spotting but
> > what is
> > > > the situation regarding duplicates because I understand some logging
> > > > software can now self spot every few minutes.  I don´t necessarily
> > agree
> > > > with this but it´s going to happen more and more.
> > > >
> > > > What´s the period during which the cluster will reject a self spot as a
> > > > dupe please and is there a variable that can be tweaked to adjust it?
> > > >
> > > > Thanks,
> > > >
> > > > 73 Keith.
> > > > _______________________________________________
> > > > Dxspider-support mailing list
> > > > Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk> 
> > > > https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
> > > >
> > >
> >
> >
> >
> >
> > _______________________________________________
> > Dxspider-support mailing list
> > Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk> 
> > https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
> >
> 




_______________________________________________
Dxspider-support mailing list
Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk> 
https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
_______________________________________________
Dxspider-support mailing list
Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk> 
https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
_______________________________________________
Dxspider-support mailing list
Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk> 
https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
 
_______________________________________________
Dxspider-support mailing list
Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk> 
https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.tobit.co.uk/pipermail/dxspider-support/attachments/20250115/08d3c754/attachment-0001.htm>

More information about the Dxspider-support mailing list