[Dxspider-support] Node crashed at 18:04

[Keith, G6NHU]

Thanks Dirk.   Here’s the important part of a top command with 873 users connected.


The Pi is bored, it’s got almost nothing to do whatsoever!  The only time there’s any load is if someone does a historical lookup command.   I ran up a DO droplet and reproduced my setup there but the Pi5 was quicker to respond to searches so I didn’t switch over.

As this has never happened before (and I’ve had more users on), I’m not going to worry about it now but it happens again then I’ll look further.  Spider runs as a service though, would it be possible to add the ulimit command in as part of the service commands?

73 Keith
On 23 Mar 2025 at 23:08 +0000, djk via Dxspider-support <dxspider-support at tobit.co.uk>, wrote:
> There is standard limit of 1024 files open at once per process. You can change this in a shell with 'ulimit -n 2048' (for example). There is also way of changing it system wide in systemd (<spit>) but you'll have to research that yourself or start the node in a shell script like:
> #!/bin/sh
> ulimit -n 2048
> /spider/perl/cluster.pl
> Personally, 900+ users on a 4GB RPi x is going some, especially considering power required to run some windows cluster software (and then still not keeping).
> What does your 'top' say when you are running it at this sort of usage?
> Dirk G1TLH
> On 23/03/2025 18:54, Keith, G6NHU via Dxspider-support wrote:
> > I suppose this really is for Dirk.
> >
> > This has never happened before - I came into the shack with a freshly poured shackbeer and noticed my ssh session had closed so I logged back in and saw my uptime was just 21 minutes.
> >
> > Checking the debug log (attached as a .zip), this is what happened in the same timestamp with the actual error that caused the crash being at the end.
> >
> > My cluster is running on a Pi5 with 4Gb RAM and an external Samsung SSD.   I don’t know the exact number of connected users but when I logged back on, there were 938 so I’d imagine the number prior to the crash was around the same.   The node had been up for about a month.
> >
> > “Too many open files” ?
> >
> > 73 Keith
> >
> > 1742753087^###
> > 1742753087^### RINGBUFFER END 501 debug lines written
> > 1742753087^###
> > 1742753087^(trace) writing /spider/local_data/rbn_cache Too many open files
> > 1742753087^(trace) Stack (2): RBN::DXDebug::confess in /spider/perl/RBN.pm line: 926
> > 1742753087^(trace) Stack (3): RBN::RBN::write_cache in /spider/perl/RBN.pm line: 871
> > 1742753087^(trace) Stack (4): main::RBN::per_minute in /spider/perl/cluster.pl line: 892
> > 1742753087^(*) DXSpider Ceasing
> > 1742753087^(*) DXQSL finished
> > 1742753087^(*) RBN:WRITE_CACHE size: 423.804KB time to write: 26 mS
> > 1742753087^(*) DXDupe finishing
> > 1742753087^(*) DXUser finished
> > 1742753087^(cluster) DXSpider v1.57 build 568 (git: mojo/0920a333[r]) using perl v5.36.0 on linux ended
> > 1742753087^(*) bye bye everyone - bye bye
> > 1742753087^###
> > 1742753087^### RINGBUFFER START at line 0 (zero base)
> > 1742753087^###
> >
> > Then it repeats
> >
> > 1742753087^RING: 18:04:47^(trace) writing /spider/local_data/rbn_cache Too many open files
> > 1742753087^RING: 18:04:47^(trace) Stack (2): RBN::DXDebug::confess in /spider/perl/RBN.pm line: 926
> > 1742753087^RING: 18:04:47^(trace) Stack (3): RBN::RBN::write_cache in /spider/perl/RBN.pm line: 871
> > 1742753087^RING: 18:04:47^(trace) Stack (4): main::RBN::per_minute in /spider/perl/cluster.pl line: 892
> > 1742753087^RING: 18:04:47^(*) DXSpider Ceasing
> > 1742753087^RING: 18:04:47^(*) DXQSL finished
> > 1742753087^RING: 18:04:47^(*) RBN:WRITE_CACHE size: 423.804KB time to write: 26 mS
> > 1742753087^RING: 18:04:47^(*) DXDupe finishing
> > 1742753087^RING: 18:04:47^(*) DXUser finished
> > 1742753087^RING: 18:04:47^(cluster) DXSpider v1.57 build 568 (git: mojo/0920a333[r]) using perl v5.36.0 on linux ended
> > 1742753087^RING: 18:04:47^(*) bye bye everyone - bye bye
> > 1742753087^###
> > 1742753087^### RINGBUFFER END 501 debug lines written
> > 1742753087^###
> >
> > _______________________________________________
> > Dxspider-support mailing list
> > Dxspider-support at tobit.co.uk
> > https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
> _______________________________________________
> Dxspider-support mailing list
> Dxspider-support at tobit.co.uk
> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.tobit.co.uk/pipermail/dxspider-support/attachments/20250324/29e02ed2/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: top.jpeg
Type: image/jpeg
Size: 39030 bytes
Desc: not available
URL: <https://mailman.tobit.co.uk/pipermail/dxspider-support/attachments/20250324/29e02ed2/attachment-0001.jpeg>