[Dxspider-support] export_user

[Dirk Koopman]

I can see why you do the "perl user_json", but I have found that the 
mojo Json serialiser is much faster and, much more importantly, stable 
than the master user_asc Storable serialiser based system. I have not 
*had* to do a restore on GB7DJK since I switched. Also, I had noticed 
that over time, some nasty binary blobs would appear in the user_asc 
after an export_user. On restore, these would (usually) be ignored, but 
could be edited out if not. This simply does not happen with the Json 
serialisation. So I would not bother with perl user_json + 
update_sysop.pl. None of the installations I manage or install do 
anything like that. And none of the four nodes that I regularly sysop 
have required a restore - yet.

As a side note: as you may know, mojo runs potentially long lived 
commandsin a separate process.  My favourite example being: sh/dx g1tlh 
:-). The communication between the two processes has to be done using a 
serialiser. The default has been Storable, the original perl function to 
do that job. But for Mojolicious 7+ it won't work reliably for this purpose.

In fact Storable it won't work reliably in DXSpider full stop, but it 
might work for a year or so before one spots problems with the users 
file and then have to restore it with 'perl user_asc'. But with mojo 
"subcommands" like sh/dx, it only worked for a few days (latterly maybe 
8 subcommands in 10)  if I was lucky. JSON was the answer for that and, 
having done some benchmarking, discovered that it was *three* times 
faster when used for serialising user data. So users.v3j and user+json 
was quickly born and I have not had problems since.

I would recommend JSON for any job like this, in any programming language.

Dirk G1TLH

On 07/01/2022 13:34, Joaquin . via Dxspider-support wrote:
> You have confirmed what I thought.
> For a long time I have included in the spider startup this (although 
> it is not always necessary):
>
> cd /spider/local_data
> perl user_json
> cd /spider/perl
> ./update_sysop.pl <http://update_sysop.pl>
> ./cluster.pl <http://cluster.pl>
>
> As for the crontab, I follow your advice to avoid problems.
>
> The funny thing is that what happened to me this time, never happened 
> to me in many tests. The user database was never corrupted.
>
> Thank you very much for your clarifications, advice and great work.
>
> 73 de Kin
>
>
> Translated with www.DeepL.com/Translator 
> <http://www.DeepL.com/Translator> (free version)
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.tobit.co.uk/pipermail/dxspider-support/attachments/20220107/89dbaa6c/attachment-0001.htm>