<div dir="ltr">Thanks Dirk,<div><br></div><div>First of all, let me apologize for posting in the wrong forum. I was not aware of the existence of a tech reflector. </div><div><br></div><div>Thank you for an extensive and well argued response. I also respect and appreciate the integrity you express.</div><div><br></div><div>All points well taken and I am the first to agree that the "distance" metric used by Jose's algorithm is essentially wrong (just as </div><div>the check partial algorithm in most loggers) in that it does not measure the true Levenshtein distance (yes, I am also a scientist) </div><div>between the characters.</div><div><br></div><div>The popularity of the CT1BOH method comes mainly from the fact that spots are never suppressed and it is left to the human being </div><div>(not the software!) to judge the questionable spots in a contest situation. The reason for this is that during major contests, </div><div>rare stations with small antennas in areas with poor skimmer coverage may very well be caught by just a single skimmer. Since scoring </div><div>in contests typically is multiplicative, that rare station could be hundreds of times more valuable than the station next door. </div><div><br></div><div>Such rare stations spotted by a single skimmer are not forwarded by a DXSpider with default settings and most likely also not by </div><div>CC cluster node but they are forwarded by an ARC6 node. It is as simple as that.</div><div><br></div><div>I am on the development team of <a href="http://www.dxlog.net/">http://www.dxlog.net/</a> which is the second or third largest contest logger (depending on how you count) </div><div>behind N1MM and there is nothing holy about the current implementation or code. We would happily modify its spot processing to fit </div><div>anything helpful developed in DXSpider. Already today DXLog interprets some unique characteristics of DXSpider spots such as </div><div>the Q:# comment elements and the RTT mode acronym. </div><div><br></div><div>I fully agree on resource needs. I run three nodes in my server closet. The CC Cluster and AR Cluster nodes share a reasonably </div><div>powerful Windows machine. The DXSpider node runs in a microscopic Ubuntu container on my Linux server. </div><div><br></div><div>If DXSpider was not my absolute preference, I would have never made this post.</div><div><br></div><div>So, in summary, I am a log program author and I am willing to engage constructively. </div><div><br></div><div>Having set up my DXSpider node only weeks ago, I am still learning. Where can I best learn about the options for </div><div>machine readable outputs? And where do I sign up to the technology reflector?</div><div><br></div><div>Björn SM7IUN</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Den fre 14 jan. 2022 kl 13:44 skrev Dirk Koopman via Dxspider-support <<a href="mailto:dxspider-support@tobit.co.uk">dxspider-support@tobit.co.uk</a>>:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>Björn<br>
<br>
Firstly, could I gently point out that DXSpider is being used by
more nodes than any other software put together - by quite a large
margin. And this has been the case since well before ARC6 became
available (1997 in my case). It is a continuing source of
irritation that I am called "to account" for why I change
something (particularly in spot formatting) by various logging
programs that *insist* on trying parse the output that is designed
for humans, instead of using the various machine readable outputs
that available in DXSpider (which, incidentally. I would be quite
willing to modify or add to). As yet, no logging program author
has *ever* engaged with me constructively to do something better.<br>
<br>
Having said that, on the substantive question that you have asked.
The answer is a definite no. I am not going to substantially
change it and, I am not about to turn DXSpider into an RBN or
PSKReporter amplifier. Or at least not in the lightly edited and
wasteful form that ARC6 does it. <br>
<br>
As an aside, I am doing some preliminary research on integrating
PSKReporter into the current RBN system. But it will not do it in
the same way the RBN (input) interface does it at the moment
(because the limited number of connections that PSKReporter
allows); therefore the RBN module, in its current internal form,
will change with it. An announcement will be made about that in
due course - probably in a couple of months - work is at a
preliminary research stage at the moment whilst I deal with the
remaining security and integrity of spot reporting issues. <br>
<br>
The issue here is the bandwidth required to both receive this data
and then to explode it to interested users. The SNR of that data
is, in my eyes, unacceptably low (says he politely). And your
justification of ARC6's method frankly does not stand up to any
scrutiny, when I challenge you (or anyone else) to parse this data
by eye - in real time - make a contact (at reasonable WPM) and NOT
MISS ANYTHING ELSE that might arrive in the meantime. The only way
this information *might* become useful is if a(other) program
parses and concentrates it in some way. I am aware that these
programs may do other things, like operate your radio, even run
the contact automatically. No skill required. <br>
<br>
I have a further problem with CT1BOH's method (as documented), as
the algorithm for determining the distance between a suspect spot
and one that passed before is quite CPU intensive and is based on
*characters*, not the *underlying* morse code which is, itself, a
decode of the raw signal. As I have some form in this area (in the
day job) I struggle to understand the justification for relying on
the statistical analysis of third order data in order to determine
what the raw signal might actually have been, while maintaining
that that method is inherently better than DXSpider's. One is
tying both hands behind one's back and wiggling a little finger -
on one hand - whilst looking over the other shoulder and squinting
at it in the mirror. <br>
<br>
ARC6, as I understand it, has to run on quite a beefy CPU, in
multiple threads to process all of this. DXSpider runs perfectly
happily on DigitalOcean's smallest droplet (1 CPU and 1GB RAM) at
about 10% CPU with 1000+ connected users during CQWW CW. And that
matters to me. As does the bandwidth that all this consumes.
DXSpider's system weeds out clearly rubbish spots, but does not
try to guess what they might have been. That is my cost/benefit
analysis - for what it is worth.<br>
<br>
At its heart, CT1BOH's/ARC6's way is just another statistical
method. At the same level as, but different to, mine. You pays yer
money (cost free in my case) and yer takes yer choice! My method
gets you a pretty decent signal but for about 3% of the RBN input,
at a fraction of one CPU. You / logging programs may not like the
format, but that is a detail that could be discussed. <br>
<br>
Arguably, this sort of analysis should be done much nearer to, or
preferably at, groups of skimmers that are reasonably near to each
other - and with the raw signal. That way some usable diversity
might become available and make all this multiple spotting - 25 or
more spots (2000+ bytes) for one strong station's CQ call -
redundant. It would make everyone's life a lot easier and put the
CPU power where it needs to be, instead of delegating it to
dxcluster nodes and ultimately "logging" programs. <br>
<br>
Since DXSpider is a human user oriented program - at least in its
default output - I asked questions of various DXers I know and
came up with the format that it now has. I can spend some time
defending it, if someone wants to dispute the way that it works or
what it looks like. But please - if you want to do that - don't
use this mailing list to do so. Either email me directly or join
the cluster-tech mailing list and do it there. Even better, get
the logging program authors to join in. I am getting a little
tired of trying to deal with things like this second or third
hand. <br>
<br>
73 + HNY<br>
<br>
Dirk G1TLH<br>
<br>
On 14/01/2022 07:37, Joaquin . via Dxspider-support wrote:<br>
</div>
<blockquote type="cite">
<div dir="auto">
<pre dir="ltr" style="max-height:999999px;font-size:24px;line-height:32px;background-color:rgb(248,249,250);border:none;padding:10px 0.14em 10px 0px;margin-top:0px;margin-bottom:0px;font-family:"google sans",roboto,"helvetica neue",arial,sans-serif;width:361.091px;white-space:pre-wrap;color:rgb(32,33,36)"><span style="max-height:999999px">I would bet on keeping the current structure of DXSpider and expanding it with some more qualifier, but it must be third-party applications that should be adapted to spider.
Regarding the possibility of receiving all the spots, that is, Q = 1, this could be implemented with a "set" command.
Kin </span>EA3CV</pre>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">El jue., 13 ene. 2022 21:54,
Björn Ekelund via Dxspider-support <<a href="mailto:dxspider-support@tobit.co.uk" rel="noreferrer" target="_blank">dxspider-support@tobit.co.uk</a>>
escribió:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Around 2013 the world renowned contester Jose
CT1BOH invented a scheme for RBN spot
<div>quality assessment that was later implemented in AR
Cluster Version 6. </div>
<div><br>
</div>
<div>Unlike the Q: method in DXSpider, the method never
suppresses spots but instead tags </div>
<div>them with a quality assessment.</div>
<div><br>
</div>
<div>Since contest loggers like Win-Test and DXLog make use
of these tags, ARC6 nodes have </div>
<div>been popular with contesters. It however seems it is
increasingly difficult to get ARC6 nodes </div>
<div>to run since after the demise of the author since the
software only exists in binary form </div>
<div>and Microsoft's updates to the .NET runtime keep
killing nodes.</div>
<div><br>
</div>
<div>Being the only free cluster software in active
development, DXSpider seems to be becoming the </div>
<div>backbone of the DX cluster.</div>
<div><br>
</div>
<div>Jose's scheme adds a tag in the rightmost end of the
comment field, V for validated, ? for unknown, </div>
<div>B for busted (with an optional corrected call included
within parentheses), and Q for QSY.</div>
<div><a href="http://lists.contesting.com/_cq-contest/2013-11/msg00085.html" rel="noreferrer noreferrer" target="_blank">http://lists.contesting.com/_cq-contest/2013-11/msg00085.html</a><br>
</div>
<div><a href="http://reversebeacon.blogspot.com/2013/08/testing-spot-quality-filters.html" rel="noreferrer noreferrer" target="_blank">http://reversebeacon.blogspot.com/2013/08/testing-spot-quality-filters.html</a><br>
</div>
<div><br>
</div>
<div>You can see it live by connecting to any ARC6 cluster
node and typing </div>
<div>"SET DX EXTENSION SKIMMERQUALITY". You can see an
example below. There is also filtering </div>
<div>options but these are typically not used for contesting
since you do not want to miss spots by </div>
<div>single skimmers.</div>
<div><br>
</div>
<div>I (and surely many other contesters) would love to see
these tags available also from DXSpider. </div>
<div>Particularly since it seems the end of life for ARC6 is
approaching.</div>
<div><br>
</div>
<div>I believe that most of the logic required to determine
the tags is already there and used to determine </div>
<div>the Q:# elements. Supporting this use case would of
course also require allowing single skimmer spots </div>
<div>through, since they are the spots tagged with ?.</div>
<div><br>
</div>
<div>Björn SM7IUN<br>
</div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace">DX de S50ARX-#: 14035.0 RO22NY
CW 21 dB 27 WPM CQ V 1119Z<br>
DX de BG4GOV2-#: 7017.0 JH6LDY CW 6 dB 17 WPM CQ
? 1119Z<br>
DX de MM0ZBH-#: 14007.5 EA1EYL CW 26 dB 20 WPM
CQ V 1119Z<br>
DX de OH6BG-#: 14035.1 RO22NY CW 27 dB 27 WPM
CQ V 1119Z<br>
DX de LZ4UX-#: 14029.0 SQ6IS CW 6 dB 19 WPM CQ
? 1119Z<br>
DX de HB9JCB-#: 21031.3 DL0AET CW 6 dB 22 WPM
CQ (DL0AA) B 1119Z<br>
DX de HB9CAT-#: 14025.6 UA9MA CW 12 dB 32 WPM
CQ V 1119Z<br>
DX de UA0S-#: 14035.0 RO22NY CW 3 dB 28 WPM CQ
V 1119Z<br>
DX de OH4KA-#: 7011.0 LY13LY CW 5 dB 26 WPM CQ
V 1119Z<br>
DX de DO4DXA-#: 14007.5 EA1EYL CW 23 dB 20 WPM
CQ V 1119Z<br>
DX de DO4DXA-#: 14024.0 RC6AQ CW 10 dB 23 WPM
CQ Q 1119Z<br>
DX de TF4X-#: 14051.9 ON6KE CW 22 dB 18 WPM
CQ ? 1119Z<br>
DX de OG66X-#: 14051.9 ON6KE CW 30 dB 19 WPM
CQ V 1119Z<br>
DX de EA1URA-#: 14051.9 ON6KE CW 24 dB 19 WPM
CQ V 1119Z<br>
DX de KP2RUM-#: 14052.0 ON6KE CW 12 dB 18 WPM
CQ V 1119Z<br>
DX de R9IR-#: 14037.5 RA4ZA CW 6 dB 20 WPM CQ
V 1119Z<br>
DX de RN4WA-#: 14035.0 RO22NY CW 33 dB 26 WPM
CQ V 1119Z<br>
DX de K5TR-#: 7038.0 K3Y/9 CW 21 dB 15 WPM
CQ V 1119Z<br>
</font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace"><br>
</font></div>
<div><br>
</div>
</div>
_______________________________________________<br>
Dxspider-support mailing list<br>
<a href="mailto:Dxspider-support@tobit.co.uk" rel="noreferrer
noreferrer" target="_blank">Dxspider-support@tobit.co.uk</a><br>
<a href="https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support" rel="noreferrer noreferrer noreferrer" target="_blank">https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support</a><br>
</blockquote>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Dxspider-support mailing list
<a href="mailto:Dxspider-support@tobit.co.uk" target="_blank">Dxspider-support@tobit.co.uk</a>
<a href="https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support" target="_blank">https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support</a>
</pre>
</blockquote>
<br>
</div>
_______________________________________________<br>
Dxspider-support mailing list<br>
<a href="mailto:Dxspider-support@tobit.co.uk" target="_blank">Dxspider-support@tobit.co.uk</a><br>
<a href="https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support" rel="noreferrer" target="_blank">https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support</a><br>
</blockquote></div>