<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Oh do I remember then days! I just filled a dumpster with old
stuff I had held on to for far to long, 8" floppies, modem banks,
ISDN banks, old DEC Alpha processors, and Sun processors, was
amazing how ancient some of the gear was still in storage. No
teletypes though, guess I got rid of all of them.. I was just
thinking the other day, pretty amazing, I went though hell to get a
T1/E1 line at my house long ago, and how I was so ahead of all the
modem users at that point. Now I actually have 10GE lit fiber in my
house, and built a 10GE LAN so I could take advantage of it. Heck
the new DSspider server has multiple 10GE ports connecting it to the
internet, how times change..<br>
<br>
<br>
On the space used by spots, I actually have logs back to 2011 from
when I first used your code I believe, but I copied some of it out,
as I didn't think it was used. Is there any point in putting it
all back into the data directory structure, as like you said space
is not an issue. I just assumed it was clutter and copied and then
removed it.<br>
<br>
I like your idea of a web reg page, that would be nice, and I sure
have no issues loading a LAMP stack or whatever is needed on the
machine, pretty much anything is adjustable with overall ease.\<br>
<br>
Oh one last question, in my crontab on the old node I had the
following:<br>
<br>
#<br>
# Monday 1am local - Get latest FCC data (W1NR updates Sundays)<br>
3 5 * * 1 spawn("cd /tmp && wget -Nq
<a class="moz-txt-link-freetext" href="ftp://ftp.w1nr.net/usdbraw.gz">ftp://ftp.w1nr.net/usdbraw.gz</a> && /spider/perl/create_usdb.pl
usdbraw.gz")<br>
44 5 * * 1 run_cmd("load/usdb")<br>
50 5 * * 1 spawn("rm /tmp/usdbraw.gz")<br>
#<br>
# Friday 1am local - Get latest Keps to be ready for any contest<br>
1 5 * * 5 spawn("cd /tmp && wget -Nq
<a class="moz-txt-link-freetext" href="https://www.amsat.org/amsat/ftp/keps/current/nasabare.txt">https://www.amsat.org/amsat/ftp/keps/current/nasabare.txt</a> &&
/spider/perl/convkeps.pl -p nasabare.txt")<br>
4 5 * * 5 run_cmd("load/keps")<br>
5 5 * * 5 spawn("rm /tmp/nasabare.txt")<br>
#<br>
# Friday 1am local - Get latest cty.dat to be ready for any contest<br>
4 5 * * 5 spawn("cd /spider/data && wget -Nq
<a class="moz-txt-link-freetext" href="https://www.country-files.com/cty/cty.dat">https://www.country-files.com/cty/cty.dat</a>")<br>
6 5 * * 5 spawn("cd /spider/data && wget -Nq
<a class="moz-txt-link-freetext" href="https://www.country-files.com/cty/wpxloc.raw">https://www.country-files.com/cty/wpxloc.raw</a>")<br>
11 5 * * 5 spawn("cd /spider/data &&
/spider/perl/create_prefix.pl")<br>
13 5 * * 5 run_cmd("load/prefixes")<br>
#<br>
# Connect for RBN Spots<br>
* * * * * start_connect('sk0mmr') unless connected('sk0mmr')<br>
* * * * * start_connect('sk1mmr') unless connected('sk1mmr')<br>
#<br>
<br>
<br>
Should I still be running stuff like this? If not, I need to
clean it up, as it's been in there for years.. LOL<br>
<br>
<pre class="moz-signature" cols="72">---
Howard Leadmon
PBW Communications, LLC
<a class="moz-txt-link-freetext" href="http://www.pbwcomm.com">http://www.pbwcomm.com</a>
</pre>
<div class="moz-cite-prefix">On 12/3/2021 9:53 PM, Dirk Koopman
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:e5ad4aa5-15d5-7208-e367-233d5a4e98f3@tobit.co.uk">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div class="moz-cite-prefix">When I wrote the first versions of
the code (in 1997) I had envisaged that people would like to add
their own commands to it, hence cmd and local_cmd directories.
But in those days people were still on mainly on ax25, dialup or
64/128Kb ISDN lines (as I was) and it was up to me to keep
things like prefixes/keps. The concept of usdb and other
databases were not a thing until later. When the mojo branch
happened I split the data into stuff I provided and stuff
sysops would routinely update like prefixes etc. This has caused
problems with updating in the past, so the concept of local_data
happened. If there is a "clash" between a file in data and one
in local_data, then the newest version wins.<br>
<br>
So you can safely leave stuff that is in data alone (and this is
data that I consider I control) and may change from one release
to another via git. In fact, if I do a release I will generally
generate a prefix list from the latest CTY data and include that
with whatever else is in the update. I just works seamlessly and
also provides a "base" set of data on first startup. Stuff that
I consider definitely sysop changeable then gets moved to
local_data from data.<br>
<br>
So don't move anything that is left in data after first startup.
What is left is git controlled defaults. I've fixed the motd and
issue issues. <br>
<br>
As for data files generated. The debug directory is the only one
that is automatically pruned and IIRC keeps 30 days worth by
default. The rest is up to you. I have a record of every spot
received since 1997 (and wwv/wcy/system logs as well) for
GB7DJK. Because they don't really take up much space and having
them has proven useful on more than one occasion for people
doing research and suchlike. Curiously the sqlite version of the
spot data is more than twice as big as the ascii based versions.
And neither is it as quick to access, especially when doing
needle in haystack searches like 'sh/dx g1tlh'. <br>
<br>
The web interface is slowly creeping up my to do list. Once I
have dealt with our random contest self spotter to my
satisfaction, it could even be next. One of the strengths of
the mojo branch is that the networking stack comes with a build
in web server + websocket interface (see <a
class="moz-txt-link-freetext" href="https://mojolicious.org/"
moz-do-not-send="true">https://mojolicious.org/</a>) . I am
starting to modernise the instructions and I could easily
"self-serve" them as static pages - as a start/proof of concept.<br>
<br>
All user data just gets converted into the new users file format
(json) and that includes all registration data. That's another
reason for activating the web server as I have numerous requests
for an "easy" (i.e. low sysop involvement) registration system.
I suspect that, after the shenanigans in this years CQWWs,
registration may well become more widespread. Maybe even (more
or less) mandatory. <br>
<br>
I think you will find that the new version is pretty stable
right now...<br>
<br>
73 Dirk G1TLH<br>
<br>
On 03/12/2021 23:07, Howard Leadmon wrote:<br>
</div>
<blockquote type="cite"
cite="mid:306e9d73-d3a7-46ad-ef88-bf344263e5ea@leadmon.net">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8">
Hello Dirk,<br>
<br>
Thanks for the reply, and input. On the older server it had
two dual-core processors in a physical server, but as I have
lots of free resourcxes on one my my VM hosts, so I figured what
the heck and just virtualized the configuration.<br>
<br>
Now I get that most files were moved, and I had to copy across
the issue and motd files, but do any of the files I see in the
old dir need moved to the new directory? <br>
<br>
Looking at the old node I see:<br>
<br>
$ ls<br>
CVS bands.pl
motd_nor usdbraw.gz.1 user_asc.oooo<br>
badw_regex cty.dat
prefix_data.pl user_asc user_asc.ooooo<br>
badw_regex.es.issue debug
qsl.v1 user_asc.bak users.v3<br>
badw_regex.gb.issue dupefile
spots user_asc.o wcy<br>
badword log
usdb.v1 user_asc.oo wpxloc.raw<br>
badword.issue motd
usdbraw.gz user_asc.ooo wwv<br>
<br>
<br>
Some of that is not over in the new local_data directory, so
curious if anything else needs moved.<br>
<br>
Also is there any recommendation on how long some of the data
files should be kept?<br>
<br>
If I look at the running node, I see under data:<br>
<br>
9768372 ./debug/2021<br>
9768373 ./debug<br>
318079 ./spots/2020<br>
314002 ./spots/2021<br>
632081 ./spots<br>
166 ./wwv/2018<br>
173 ./wwv/2020<br>
171 ./wwv/2019<br>
160 ./wwv/2021<br>
671 ./wwv<br>
80079 ./log/2018<br>
74187 ./log/2020<br>
81871 ./log/2019<br>
534903 ./log/2021<br>
771039 ./log<br>
385 ./wcy/2021<br>
426 ./wcy/2019<br>
431 ./wcy/2020<br>
430 ./wcy/2018<br>
1674 ./wcy<br>
5 ./CVS<br>
<br>
<br>
I just moved all this over to the new node intact, but figured
maybe there is a recommendation you might have.<br>
<br>
I did also enable the RBN code you have in Mojo, and it seems
to be running fine. Wondering if worth trying to toss a web
interface on it, as I for sure have lots of resources available
if needed. The host has two 12core modern Xeon's, with a few
hundred gig of RAM and quite a few TB of storage, so I just
slice out what is needed.<br>
<br>
Finally, and I know many don't do this, and I am sure some
users avoid the node because of it, but I have always required
the user to register to post spots. I am guessing this just
moved across and works, but if there is anything special I need
to be aware of with Mojo in regards to this, by all means please
let me know..<br>
<br>
73's...<br>
<br>
<pre class="moz-signature" cols="72">---
Howard Leadmon - <a class="moz-txt-link-abbreviated moz-txt-link-freetext" href="mailto:howard@leadmon.net" moz-do-not-send="true">howard@leadmon.net</a>
PBW Communications, LLC
<a class="moz-txt-link-freetext" href="http://www.pbwcomm.com" moz-do-not-send="true">http://www.pbwcomm.com</a>
</pre>
<div class="moz-cite-prefix">On 12/3/2021 4:45 PM, Dirk Koopman
via Dxspider-support wrote:<br>
</div>
<blockquote type="cite"
cite="mid:d279d6eb-ac8a-67d1-9251-45f5720c281b@tobit.co.uk">
<meta http-equiv="Content-Type" content="text/html;
charset=UTF-8">
<div class="moz-cite-prefix">On 03/12/2021 17:35, Howard
Leadmon via Dxspider-support wrote:<br>
</div>
<blockquote type="cite"
cite="mid:dd3276f9-b717-49f6-eee7-7312afb7ffa3@leadmon.net">
<meta http-equiv="content-type" content="text/html;
charset=UTF-8">
<br>
With Dirk saying that he was now ready to switch to Mojo
being the main branch, I decided it was time to get off my
backside and setup a new server for DXspider.<br>
</blockquote>
Hurrah! Please encourage all your friends to do the same. <br>
<blockquote type="cite"
cite="mid:dd3276f9-b717-49f6-eee7-7312afb7ffa3@leadmon.net">
<br>
Overall the conversion went fairly easy, and I have the new
Mojo node running and connected back to the original mode, I
did want to check on one thing. I know from this list, that
local_data needed to created, and I know that my motd (and
motd_nor) needed to be moved over to the this directory.<br>
</blockquote>
<br>
I'm glad that it is very easy to do [ed: I may have
exaggerated that a tad]. There are some setup things that
happen on first run of the mojo version. It appears that motd
et al are not moved with the other things. That is changed on
the next release. It will move the issue and motd file across
to local_data automagically.<br>
<br>
<blockquote type="cite"
cite="mid:dd3276f9-b717-49f6-eee7-7312afb7ffa3@leadmon.net">
<br>
What I am curious about is, did I need to move any other
files across to local_data? Do items like the city files
need moved, and scripts updated vs the old spider version?<br>
</blockquote>
See above.<br>
<blockquote type="cite"
cite="mid:dd3276f9-b717-49f6-eee7-7312afb7ffa3@leadmon.net">
<br>
At the moment in local_data I see:<br>
<br>
$ ls local_data<br>
badword debug dxqsl.v1j motd rbn_cache
usdb.v1 users.v3j wwv<br>
cluster.lck dupefile log motd_nor spots
user_json wcy<br>
<br>
<br>
Before I take this public in place of the old version, I for
sure want to make sure everything is working OK, as I know a
bunch of people use it while contesting, so I strive for
100% uptime.<br>
<br>
<i>If any see this, and care to test this sucker out, it's
always appreciated. It can be reached at </i><i><b>dxtest.wb3ffv.us</b></i><i>
on </i><i><b>port 7300</b></i><i> over both IPv4 and
IPv6.</i><br>
<br>
Dirk, if you see this, and recommendations on CPU and
memory for a VM running Spider. As I built this in a VM, I
have it running current Ubuntu, with 4x VCPU's and 8G of
memory, and it seems quick as heck, but then again there is
nothing really connected to it..<br>
</blockquote>
<br>
Well WA9PIE-2 runs in a 1CPU 1GB RAM 25GB Digital Ocean
droplet. But with some recent changes to speed up simple sh/dx
(sh/dx, sh/dx 100) in scripts and bring it back "inline" so,
as a result of going through two CQWW contests, I would
recommend a tad more RAM (say 2GB) to handle forking cmds
(sh/dx g1tlh). 2CPUs is a nice to have, but WA9PIE seems to
run 1200 users in CQWW SSB at about 15-20% CPU usage. But then
the users don't do much except listen and ask for headings. <br>
<br>
I would suggest what you have there is (as Rolls Royce
famously said when asked how much power the engine produced)
adequate :-)<br>
<br>
I suspect a 2GB RPi4 with a 250TB SSD in a USB3 attached UASP
capable caddy will run WA9PIE's load just fine. <br>
<br>
<blockquote type="cite"
cite="mid:dd3276f9-b717-49f6-eee7-7312afb7ffa3@leadmon.net">
<br>
<br>
73's de WB3FFV<br>
<pre class="moz-signature" cols="72">
---
Howard Leadmon
PBW Communications, LLC
<a class="moz-txt-link-freetext" href="http://www.pbwcomm.com" moz-do-not-send="true">http://www.pbwcomm.com</a>
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Dxspider-support mailing list
<a class="moz-txt-link-abbreviated moz-txt-link-freetext" href="mailto:Dxspider-support@tobit.co.uk" moz-do-not-send="true">Dxspider-support@tobit.co.uk</a>
<a class="moz-txt-link-freetext" href="https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support" moz-do-not-send="true">https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support</a>
</pre>
</blockquote>
<br>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Dxspider-support mailing list
<a class="moz-txt-link-abbreviated moz-txt-link-freetext" href="mailto:Dxspider-support@tobit.co.uk" moz-do-not-send="true">Dxspider-support@tobit.co.uk</a>
<a class="moz-txt-link-freetext" href="https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support" moz-do-not-send="true">https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support</a>
</pre>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>