<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Here is a case for using the mojo
branch. AFAIK there is no programmed in 1024 file limit and it
won't use normal select (unless you forget to install EV) so
FD_SIZE isn't relevant. You might well still have to increase the
maximum number of files available to a process. On my syslem '
ulimit -n 2048' works. But it depends on the hard limits set by
the operating system. In Mike's case I think the hard limit is
4096. You'll need to modify the start up for cluster.pl to do the
ulimit -n. A standard place is in /user/default/dxspider, or you
could start the node with 'ulimit -n 2048; /spider/perl/cluster.pl
> /dev/null >2&1'. YMMV.<br>
<br>
Mike is the *only* person I know that seems to be able to run a
real 300+ user node on the "standard" master branch. Everybody
else approaching that (that I know about) is on the mojo branch.
If one is serious about running "super nodes" then the mojo branch
is the way to go because it uses the Mojolicious framewark + EV +
epoll(2) under the hood.<br>
<br>
Dirk G1TLH<br>
<br>
On 14/04/2020 17:10, Joaquin . via Dxspider-support wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAHTVWh1v2vG7cS59F2pAurw4TQjevKRVaOkoMK6GjGSF5CAJew@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div>Hi,<br>
</div>
<div><br>
</div>
I have mounted 3 VM:<br>
The first with version 1.55 0.204.
<div> With 4 GB of RAM 1 CPU</div>
<div> Ubuntu 18.04.3 LTS<br>
The second with version 1.57 148.</div>
<div> With 4 GB of RAM 1 CPU</div>
<div> Debian GNU/Linux 10<br>
The third was the host from which the telnet requests were
made.<br>
<div> With 4 GB of RAM 1 CPU</div>
<div> Debian GNU/Linux 10</div>
<br>
With the sysctl command I have increased the values of the
variables net.core <br>
and net.ipv4 to give more capacity to the OS, on the 3
machines.<br>
<br>
None of the tests has exceeded 1008 established client
sockets.<br>
<font face="monospace"><br>
root@ea3cv_cluster:~# lsof -i tcp<br>
...<br>
...<br>
perl 8872 sysop 1020u IPv4 190250 0t0
TCP ea3cv_cluster:7300-><a
href="http://192.168.1.34:49984" moz-do-not-send="true">192.168.1.34:49984</a>
(ESTABLISHED)<br>
perl 8872 sysop 1021u IPv4 190251 0t0
TCP ea3cv_cluster:7300-><a
href="http://192.168.1.34:49982" moz-do-not-send="true">192.168.1.34:49982</a>
(ESTABLISHED)<br>
perl 8872 sysop 1022u IPv4 190252 0t0
TCP ea3cv_cluster:7300-><a
href="http://192.168.1.34:49980" moz-do-not-send="true">192.168.1.34:49980</a>
(ESTABLISHED)<br>
perl 8872 sysop <b>1023u</b> IPv4 190253
0t0 TCP ea3cv_cluster:7300-><a
href="http://192.168.1.34:49978" moz-do-not-send="true">192.168.1.34:49978</a>
(ESTABLISHED)</font><br>
<br>
<font face="monospace">root@ea3cv_cluster:~# ss -s<br>
Total: 2511 (kernel 4418)<br>
TCP: 1017 (estab <b>1008</b>, closed 0, orphaned 0,
synrecv 0, timewait 0/0), ports 0<br>
<br>
Transport Total IP IPv6<br>
* 4418 - - <br>
RAW 0 0 0 <br>
UDP 3 3 0 <br>
TCP 1017 1017 0 <br>
INET 1020 1020 0 <br>
FRAG 0 0 0 <br>
<br>
sysop@ea3cv-cluster3:~/spider/kin$ ulimit -n<br>
<b>5000</b></font><br>
<br>
<br>
It has noticed a higher speed when going from 2 GB to 4 GB of
RAM, but nothing more.<br>
The swap memory is not occupied.<br>
<br>
<font face="monospace">sysop@ea3cv-cluster3:~/spider/kin$
uptime<br>
11:32:29 up 2:04, 1 user, load average: 1,11, 0,50, 0,18<br>
<br>
Users: <b>1008 </b><--- 1007 external sockets
+ 1 <a href="http://console.pl" moz-do-not-send="true">console.pl</a><br>
Uptime: 0 00:58<br>
Load: 14.5 % <--- Only <a
href="http://cluster.pl" moz-do-not-send="true">cluster.pl</a>
& <a href="http://console.pl" moz-do-not-send="true">console.pl</a><br>
Mem: 73.7 %<br>
<br>
sysop@ea3cv-cluster3:~/spider/kin$ free<br>
total used free shared
buff/cache available<br>
Mem: 4015844 1448060 158784 20520
2409000 2271352<br>
Swap: 2097148 <b>268 </b>2246880</font><br>
<br>
<br>
When the socket limit was reached, if requests kept coming to
the server, <br>
the <a href="http://console.pl" moz-do-not-send="true">console.pl</a>
would crash and all sockets would be lost.<br>
<br>
I have found a possible cause for this limitation: <b>FD_SETSIZE</b><br>
<br>
FD_SETSIZE is normally 1024, so file descriptors over 1024 are
not supported in general.<br>
You can fiddle with the FD_SETSIZE include sizes as you have
done but making changes <br>
like this might impact other programs too which aim to be
POSIX compliant.<br>
<font face="monospace">...<br>
/usr/include/x86_64-linux-gnu/bits/typesizes.h:#define
__FD_SETSIZE 1024<br>
/usr/include/linux/posix_types.h:#define __FD_SETSIZE 1024<br>
...</font><br>
<br>
<a href="https://access.redhat.com/solutions/488623"
moz-do-not-send="true">https://access.redhat.com/solutions/488623</a><br>
<br>
After many tests I confirm that more than 1024 sockets cannot
be established with this configuration.<br>
So far the tests, let's see what the others say.<br>
</div>
<div><br>
Kin, EA3CV<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">El mar., 14 abr. 2020 a las
11:47, Dirk Koopman via Dxspider-support (<<a
href="mailto:dxspider-support@tobit.co.uk"
moz-do-not-send="true">dxspider-support@tobit.co.uk</a>>)
escribió:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div>Firstly<br>
<br>
SIGXCPU<br>
The SIGXCPU signal is sent to a process when it has used
up the CPU for a duration that exceeds a certain
predetermined user-settable value.[16] The arrival of a
SIGXCPU signal provides the receiving process a chance to
quickly save any intermediate results and to exit
gracefully, before it is terminated by the operating
system using the SIGKILL signal.<br>
<br>
[16] refers to the information on getrlimit and setrlimit.
Reading the manual says:<br>
<br>
RLIMIT_CPU<br>
This is the maximum amount of CPU time, in seconds, used
by a process. If this limit is exceeded, SIGXCPU shall be
generated for the process. If the process is catching or
ignoring SIGXCPU, or all threads belonging to that process
are blocking SIGXCPU, the behavior is unspecified. <br>
<br>
This parameter can be altered. But I am wondering whether
you are hitting the limits of the vCPU that is set in the
machine running this VM?<br>
<br>
<tt>Also there is a default limit on the number of sockets
a process may have of 1024. Now, in the old days (and on
Windows boxes with default limits of 64 (!)), it would
just refuse to accept new ones. It (now) *may* be that
this causes this signal. In any case you should increase
the maximum no of sockets the process will take. Do a
"ulimit -n" and you see the default is still 1024. You
will need to increase the limit. <br>
<br>
You should probably also consider this StackOverflow
answer: </tt><tt><a
href="https://stackoverflow.com/questions/410616/increasing-the-maximum-number-of-tcp-ip-connections-in-linux"
target="_blank" moz-do-not-send="true">https://stackoverflow.com/questions/410616/increasing-the-maximum-number-of-tcp-ip-connections-in-linux</a></tt><br>
<br>
Dirk<br>
<br>
<br>
On 14/04/2020 01:29, Michael Carper, Ph.D. via
Dxspider-support wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif">As of a
bit more than a week ago, at about the same time each
day (with max users connected), the node abruptly
restarts. This is totally frustrating.</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:arial,helvetica,sans-serif">
<div><img
src="cid:part12.4CF0FB56.01E4D43D@tobit.co.uk"
alt="image.png" class="" width="542" height="114"><br>
</div>
<div>I've checked all the crontabs... no events
scheduled for that time.</div>
<div><br>
</div>
<div>I look in the linux logfile and see this...</div>
<div><br>
</div>
<div># cat messages<br>
Apr 12 15:21:58 wa9pie-2b init: dxspider main
process (28416) terminated with status 24<br>
Apr 12 15:21:58 wa9pie-2b init: dxspider main
process ended, respawning<br>
Apr 13 16:14:45 wa9pie-2b init: dxspider main
process (18493) terminated with status 24<br>
Apr 13 16:14:45 wa9pie-2b init: dxspider main
process ended, respawning<br>
</div>
<div><br>
</div>
<div>What gives??</div>
<div><br>
</div>
<div>From what I can see, "status 24" is "stop issued
from terminal."</div>
<div><br>
</div>
<div>Frustrated.</div>
<div><br>
</div>
<div>Mike, WA9PIE, VK4EIE</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
Dxspider-support mailing list
<a href="mailto:Dxspider-support@tobit.co.uk" target="_blank" moz-do-not-send="true">Dxspider-support@tobit.co.uk</a>
<a href="https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support" target="_blank" moz-do-not-send="true">https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support</a>
</pre>
</blockquote>
<br>
<br>
</div>
_______________________________________________<br>
Dxspider-support mailing list<br>
<a href="mailto:Dxspider-support@tobit.co.uk" target="_blank"
moz-do-not-send="true">Dxspider-support@tobit.co.uk</a><br>
<a
href="https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support</a><br>
</blockquote>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Dxspider-support mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Dxspider-support@tobit.co.uk">Dxspider-support@tobit.co.uk</a>
<a class="moz-txt-link-freetext" href="https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support">https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support</a>
</pre>
</blockquote>
<br>
</body>
</html>