[Dxspider-support] Alternative/additional comment elements in spots
Björn Ekelund
bjorn at ekelund.nu
Fri Jan 14 17:38:52 GMT 2022
Thanks Dirk,
First of all, let me apologize for posting in the wrong forum. I was not
aware of the existence of a tech reflector.
Thank you for an extensive and well argued response. I also respect and
appreciate the integrity you express.
All points well taken and I am the first to agree that the "distance"
metric used by Jose's algorithm is essentially wrong (just as
the check partial algorithm in most loggers) in that it does not measure
the true Levenshtein distance (yes, I am also a scientist)
between the characters.
The popularity of the CT1BOH method comes mainly from the fact that spots
are never suppressed and it is left to the human being
(not the software!) to judge the questionable spots in a contest situation.
The reason for this is that during major contests,
rare stations with small antennas in areas with poor skimmer coverage may
very well be caught by just a single skimmer. Since scoring
in contests typically is multiplicative, that rare station could be
hundreds of times more valuable than the station next door.
Such rare stations spotted by a single skimmer are not forwarded by a
DXSpider with default settings and most likely also not by
CC cluster node but they are forwarded by an ARC6 node. It is as simple as
that.
I am on the development team of http://www.dxlog.net/ which is the second
or third largest contest logger (depending on how you count)
behind N1MM and there is nothing holy about the current implementation or
code. We would happily modify its spot processing to fit
anything helpful developed in DXSpider. Already today DXLog interprets some
unique characteristics of DXSpider spots such as
the Q:# comment elements and the RTT mode acronym.
I fully agree on resource needs. I run three nodes in my server closet. The
CC Cluster and AR Cluster nodes share a reasonably
powerful Windows machine. The DXSpider node runs in a microscopic Ubuntu
container on my Linux server.
If DXSpider was not my absolute preference, I would have never made this
post.
So, in summary, I am a log program author and I am willing to engage
constructively.
Having set up my DXSpider node only weeks ago, I am still learning. Where
can I best learn about the options for
machine readable outputs? And where do I sign up to the technology
reflector?
Björn SM7IUN
Den fre 14 jan. 2022 kl 13:44 skrev Dirk Koopman via Dxspider-support <
dxspider-support at tobit.co.uk>:
> Björn
>
> Firstly, could I gently point out that DXSpider is being used by more
> nodes than any other software put together - by quite a large margin. And
> this has been the case since well before ARC6 became available (1997 in my
> case). It is a continuing source of irritation that I am called "to
> account" for why I change something (particularly in spot formatting) by
> various logging programs that *insist* on trying parse the output that is
> designed for humans, instead of using the various machine readable outputs
> that available in DXSpider (which, incidentally. I would be quite willing
> to modify or add to). As yet, no logging program author has *ever* engaged
> with me constructively to do something better.
>
> Having said that, on the substantive question that you have asked. The
> answer is a definite no. I am not going to substantially change it and, I
> am not about to turn DXSpider into an RBN or PSKReporter amplifier. Or at
> least not in the lightly edited and wasteful form that ARC6 does it.
>
> As an aside, I am doing some preliminary research on integrating
> PSKReporter into the current RBN system. But it will not do it in the same
> way the RBN (input) interface does it at the moment (because the limited
> number of connections that PSKReporter allows); therefore the RBN module,
> in its current internal form, will change with it. An announcement will be
> made about that in due course - probably in a couple of months - work is at
> a preliminary research stage at the moment whilst I deal with the remaining
> security and integrity of spot reporting issues.
>
> The issue here is the bandwidth required to both receive this data and
> then to explode it to interested users. The SNR of that data is, in my
> eyes, unacceptably low (says he politely). And your justification of ARC6's
> method frankly does not stand up to any scrutiny, when I challenge you (or
> anyone else) to parse this data by eye - in real time - make a contact (at
> reasonable WPM) and NOT MISS ANYTHING ELSE that might arrive in the
> meantime. The only way this information *might* become useful is if
> a(other) program parses and concentrates it in some way. I am aware that
> these programs may do other things, like operate your radio, even run the
> contact automatically. No skill required.
>
> I have a further problem with CT1BOH's method (as documented), as the
> algorithm for determining the distance between a suspect spot and one that
> passed before is quite CPU intensive and is based on *characters*, not the
> *underlying* morse code which is, itself, a decode of the raw signal. As I
> have some form in this area (in the day job) I struggle to understand the
> justification for relying on the statistical analysis of third order data
> in order to determine what the raw signal might actually have been, while
> maintaining that that method is inherently better than DXSpider's. One is
> tying both hands behind one's back and wiggling a little finger - on one
> hand - whilst looking over the other shoulder and squinting at it in the
> mirror.
>
> ARC6, as I understand it, has to run on quite a beefy CPU, in multiple
> threads to process all of this. DXSpider runs perfectly happily on
> DigitalOcean's smallest droplet (1 CPU and 1GB RAM) at about 10% CPU with
> 1000+ connected users during CQWW CW. And that matters to me. As does the
> bandwidth that all this consumes. DXSpider's system weeds out clearly
> rubbish spots, but does not try to guess what they might have been. That is
> my cost/benefit analysis - for what it is worth.
>
> At its heart, CT1BOH's/ARC6's way is just another statistical method. At
> the same level as, but different to, mine. You pays yer money (cost free in
> my case) and yer takes yer choice! My method gets you a pretty decent
> signal but for about 3% of the RBN input, at a fraction of one CPU. You /
> logging programs may not like the format, but that is a detail that could
> be discussed.
>
> Arguably, this sort of analysis should be done much nearer to, or
> preferably at, groups of skimmers that are reasonably near to each other -
> and with the raw signal. That way some usable diversity might become
> available and make all this multiple spotting - 25 or more spots (2000+
> bytes) for one strong station's CQ call - redundant. It would make
> everyone's life a lot easier and put the CPU power where it needs to be,
> instead of delegating it to dxcluster nodes and ultimately "logging"
> programs.
>
> Since DXSpider is a human user oriented program - at least in its default
> output - I asked questions of various DXers I know and came up with the
> format that it now has. I can spend some time defending it, if someone
> wants to dispute the way that it works or what it looks like. But please -
> if you want to do that - don't use this mailing list to do so. Either email
> me directly or join the cluster-tech mailing list and do it there. Even
> better, get the logging program authors to join in. I am getting a little
> tired of trying to deal with things like this second or third hand.
>
> 73 + HNY
>
> Dirk G1TLH
>
> On 14/01/2022 07:37, Joaquin . via Dxspider-support wrote:
>
> I would bet on keeping the current structure of DXSpider and expanding it with some more qualifier, but it must be third-party applications that should be adapted to spider.
>
> Regarding the possibility of receiving all the spots, that is, Q = 1, this could be implemented with a "set" command.
>
> Kin EA3CV
>
>
> El jue., 13 ene. 2022 21:54, Björn Ekelund via Dxspider-support <
> dxspider-support at tobit.co.uk> escribió:
>
>> Around 2013 the world renowned contester Jose CT1BOH invented a scheme
>> for RBN spot
>> quality assessment that was later implemented in AR Cluster Version 6.
>>
>> Unlike the Q: method in DXSpider, the method never suppresses spots but
>> instead tags
>> them with a quality assessment.
>>
>> Since contest loggers like Win-Test and DXLog make use of these tags,
>> ARC6 nodes have
>> been popular with contesters. It however seems it is increasingly
>> difficult to get ARC6 nodes
>> to run since after the demise of the author since the software only
>> exists in binary form
>> and Microsoft's updates to the .NET runtime keep killing nodes.
>>
>> Being the only free cluster software in active development, DXSpider
>> seems to be becoming the
>> backbone of the DX cluster.
>>
>> Jose's scheme adds a tag in the rightmost end of the comment field, V for
>> validated, ? for unknown,
>> B for busted (with an optional corrected call included within
>> parentheses), and Q for QSY.
>> http://lists.contesting.com/_cq-contest/2013-11/msg00085.html
>>
>> http://reversebeacon.blogspot.com/2013/08/testing-spot-quality-filters.html
>>
>> You can see it live by connecting to any ARC6 cluster node and typing
>> "SET DX EXTENSION SKIMMERQUALITY". You can see an example below. There is
>> also filtering
>> options but these are typically not used for contesting since you do not
>> want to miss spots by
>> single skimmers.
>>
>> I (and surely many other contesters) would love to see these tags
>> available also from DXSpider.
>> Particularly since it seems the end of life for ARC6 is approaching.
>>
>> I believe that most of the logic required to determine the tags is
>> already there and used to determine
>> the Q:# elements. Supporting this use case would of course also require
>> allowing single skimmer spots
>> through, since they are the spots tagged with ?.
>>
>> Björn SM7IUN
>>
>> DX de S50ARX-#: 14035.0 RO22NY CW 21 dB 27 WPM CQ V
>> 1119Z
>> DX de BG4GOV2-#: 7017.0 JH6LDY CW 6 dB 17 WPM CQ ?
>> 1119Z
>> DX de MM0ZBH-#: 14007.5 EA1EYL CW 26 dB 20 WPM CQ V
>> 1119Z
>> DX de OH6BG-#: 14035.1 RO22NY CW 27 dB 27 WPM CQ V
>> 1119Z
>> DX de LZ4UX-#: 14029.0 SQ6IS CW 6 dB 19 WPM CQ ?
>> 1119Z
>> DX de HB9JCB-#: 21031.3 DL0AET CW 6 dB 22 WPM CQ (DL0AA) B
>> 1119Z
>> DX de HB9CAT-#: 14025.6 UA9MA CW 12 dB 32 WPM CQ V
>> 1119Z
>> DX de UA0S-#: 14035.0 RO22NY CW 3 dB 28 WPM CQ V
>> 1119Z
>> DX de OH4KA-#: 7011.0 LY13LY CW 5 dB 26 WPM CQ V
>> 1119Z
>> DX de DO4DXA-#: 14007.5 EA1EYL CW 23 dB 20 WPM CQ V
>> 1119Z
>> DX de DO4DXA-#: 14024.0 RC6AQ CW 10 dB 23 WPM CQ Q
>> 1119Z
>> DX de TF4X-#: 14051.9 ON6KE CW 22 dB 18 WPM CQ ?
>> 1119Z
>> DX de OG66X-#: 14051.9 ON6KE CW 30 dB 19 WPM CQ V
>> 1119Z
>> DX de EA1URA-#: 14051.9 ON6KE CW 24 dB 19 WPM CQ V
>> 1119Z
>> DX de KP2RUM-#: 14052.0 ON6KE CW 12 dB 18 WPM CQ V
>> 1119Z
>> DX de R9IR-#: 14037.5 RA4ZA CW 6 dB 20 WPM CQ V
>> 1119Z
>> DX de RN4WA-#: 14035.0 RO22NY CW 33 dB 26 WPM CQ V
>> 1119Z
>> DX de K5TR-#: 7038.0 K3Y/9 CW 21 dB 15 WPM CQ V
>> 1119Z
>>
>>
>>
>>
>> _______________________________________________
>> Dxspider-support mailing list
>> Dxspider-support at tobit.co.uk
>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>>
>
> _______________________________________________
> Dxspider-support mailing listDxspider-support at tobit.co.ukhttps://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>
>
> _______________________________________________
> Dxspider-support mailing list
> Dxspider-support at tobit.co.uk
> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.tobit.co.uk/pipermail/dxspider-support/attachments/20220114/6875ab68/attachment-0001.htm>
More information about the Dxspider-support
mailing list