[Dxspider-support] Help please! user_json is 13Gb!

Keith Maton g6nhu at me.com
Wed Dec 7 12:26:52 GMT 2022 

OK, thank you.  I’ve done that.

Stopped the node, renamed the /spider/data/users.v3j file and copied users.v3 into that folder from my old installation.

Restarted the node and I see that a new users.v3j file has been created in /spider/local_data.

No sign of a .keep file though and I don’t think I’ve ever seen a .backstop file in there.





> On 7 Dec 2022, at 12:08, Dirk Koopman via Dxspider-support <dxspider-support at tobit.co.uk> wrote:
> 
> What I suggest you do is to copy your users.v3 from your old installation. Then stop the node, remove the users.v3j and then restart the node. It will rebuild your users.v3j and also the .keep and .backstop files. 
> 
> 
> On 07/12/2022 12:03, Keith Maton via Dxspider-support wrote:
>> No this was a fresh installation using the script here: https://www.sv5fri.eu/?p=4318424
>> 
>> I then set it up from scratch, didn’t copy any files over from my old install.
>> 
>> 
>> 
>>> On 7 Dec 2022, at 12:01, Dirk Koopman via Dxspider-support <dxspider-support at tobit.co.uk> <mailto:dxspider-support at tobit.co.uk> wrote:
>>> 
>>> Did you upgrade from an 1.55 (master branch) installation?
>>> 
>>> On 07/12/2022 11:35, Keith Maton via Dxspider-support wrote:
>>>> Arse, too late :-(
>>>> 
>>>> Sorry.
>>>> 
>>>> Is this part of the important magic that happens in the early hours of Wednesday mornings?
>>>> 
>>>> 
>>>> 
>>>>> On 7 Dec 2022, at 11:32, Dirk Koopman via Dxspider-support <dxspider-support at tobit.co.uk> <mailto:dxspider-support at tobit.co.uk> wrote:
>>>>> 
>>>>> keep the .keep file, it's there for a reason.
>>>>> 
>>>>> 
>>>>> On 07/12/2022 11:30, Keith Maton via Dxspider-support wrote:
>>>>>> Ahh, our emails crossed. 
>>>>>> 
>>>>>> Here’s my .v3j file.  I’ll blitz all the user_json files now.
>>>>>> 
>>>>>> -rwxrwxr-x 1 sysop spider 8.0M Dec  7 11:28 users.v3j
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On 7 Dec 2022, at 11:25, Dirk Koopman via Dxspider-support <dxspider-support at tobit.co.uk> <mailto:dxspider-support at tobit.co.uk> wrote:
>>>>>>> 
>>>>>>> I'm struggling to understand why your user_json files (any of them) are more than a few tens of MB. My reference site GB7DJK looks like this:
>>>>>>> 
>>>>>>> -rw-rw-r--  1 sysop spider  50M Dec  7 02:30 user_json
>>>>>>> -rw-rw-r--  1 sysop spider  48M Dec  7 02:30 user_json.backstop
>>>>>>> -rw-rw-r--  1 sysop spider  29M Jan 24  2022 user_json.keep
>>>>>>> -rw-rw-r--  1 sysop spider  50M Nov 30 02:30 user_json.o
>>>>>>> -rw-rw-r--  1 sysop spider  49M Nov 23 02:30 user_json.oo
>>>>>>> -rw-rw-r--  1 sysop spider  49M Nov 16 02:30 user_json.ooo
>>>>>>> -rw-rw-r--  1 sysop spider  49M Nov  9 02:30 user_json.oooo
>>>>>>> -rw-rw-r--  1 sysop spider  48M Nov  2 02:30 user_json.ooooo
>>>>>>> 
>>>>>>> -rw-rw-r--  1 sysop spider  81M Dec  7 11:17 users.v3j
>>>>>>> 
>>>>>>> And remember, this site has been on 24hrs / day since about 1998.
>>>>>>> 
>>>>>>> WA9PIE-2's data is a bit bigger as he has more users and more connections:
>>>>>>> 
>>>>>>> -rw-rw-r-- 1 sysop sysop  64M Dec  7 02:30 user_json
>>>>>>> -rw-rw-r-- 1 sysop sysop  62M Dec  7 02:30 user_json.backstop
>>>>>>> -rw-rw-r-- 1 sysop sysop  49M Jan 12  2022 user_json.keep
>>>>>>> -rw-rw-r-- 1 sysop sysop  64M Nov 30 02:30 user_json.o
>>>>>>> -rw-rw-r-- 1 sysop sysop  63M Nov 23 02:30 user_json.oo
>>>>>>> -rw-rw-r-- 1 sysop sysop  63M Nov 16 02:30 user_json.ooo
>>>>>>> -rw-rw-r-- 1 sysop sysop  62M Nov  9 02:30 user_json.oooo
>>>>>>> -rw-rw-r-- 1 sysop sysop  62M Nov  2 02:30 user_json.ooooo
>>>>>>> 
>>>>>>> -rw-rw-r-- 1 sysop sysop 114M Dec  7 11:19 users.v3j
>>>>>>> 
>>>>>>> You can remove any user_json file and you don't need to restart. They are just backups of users.v3j. The user_json files are there to allow you to restore the users.v3j file. 
>>>>>>> 
>>>>>>> So how big is your users.v3j file?
>>>>>>> 
>>>>>>> Dirk G1TLH
>>>>>>> 
>>>>>>> 
>>>>>>> On 07/12/2022 10:08, Keith Maton via Dxspider-support wrote:
>>>>>>>> Right, I stand corrected.  It’s the user_json.o file that’s 13Gb.
>>>>>>>> 
>>>>>>>> -rw-rw-r-- 1 sysop spider   121704448 Dec  7 10:01 user_json
>>>>>>>> -rw-rw-r-- 1 sysop spider     2838642 Dec  7 02:30 user_json.keep
>>>>>>>> -rw-rw-r-- 1 sysop spider 13067960320 Dec  7 08:29 user_json.o
>>>>>>>> -rwxrwxr-x 1 sysop spider     2838642 Dec  4 09:48 user_json.oo
>>>>>>>> 
>>>>>>>>> On 7 Dec 2022, at 09:32, Joaquin via Dxspider-support <dxspider-support at tobit.co.uk> <mailto:dxspider-support at tobit.co.uk> wrote:
>>>>>>>>> 
>>>>>>>>> Sorry, I don't know why I didn't read about the user_json file.
>>>>>>>>> Do an export_user, remove the 13GB one, and reboot.
>>>>>>>>> 
>>>>>>>>> Kin
>>>>>>>>> 
>>>>>>>>> El 07/12/2022 a las 10:29, Joaquin via Dxspider-support escribió:
>>>>>>>>>> Keith,
>>>>>>>>>> 
>>>>>>>>>> Look to see what you have defined in debug: sh/debug, you also have many traces and you can reduce them with unset/debug.
>>>>>>>>>> Or you can limit the number of log files if your FS is small.
>>>>>>>>>> 
>>>>>>>>>> Kin
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> El 07/12/2022 a las 10:23, Keith Maton via Dxspider-support escribió:
>>>>>>>>>>> I spoke too soon.
>>>>>>>>>>> 
>>>>>>>>>>> Woke up this morning to a crashed node, the Pi was still running but when I tried to connect externally to the node, I had no response, when I logged on and issued a /spider/perl/console.pl command, I just had a grey screen.  I’ve checked the debug log and it fell over at 02:29:47 this morning.  The last entry in the debug log is just a normal spot.
>>>>>>>>>>> 
>>>>>>>>>>> I successfully rebooted and it’s all running but I have PRTG monitoring and I just got pinged for low disk space.
>>>>>>>>>>> 
>>>>>>>>>>> When I check, my /local_data/user_json file has grown to 13Gb.
>>>>>>>>>>> 
>>>>>>>>>>> Any idea what’s caused this?  Can I just stop the node, delete the file and restart?  I suspect with less than 50Mb disk space, it’s going to crash soon anyway.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Dxspider-support mailing list
>>>>>>>>>> Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk>
>>>>>>>>>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>>>>>>>>> _______________________________________________
>>>>>>>>> Dxspider-support mailing list
>>>>>>>>> Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk>
>>>>>>>>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> Dxspider-support mailing list
>>>>>>>> Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk>
>>>>>>>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> Dxspider-support mailing list
>>>>>>> Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk>
>>>>>>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Dxspider-support mailing list
>>>>>> Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk>
>>>>>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>>>>> 
>>>>> _______________________________________________
>>>>> Dxspider-support mailing list
>>>>> Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk>
>>>>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Dxspider-support mailing list
>>>> Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk>
>>>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>>> 
>>> _______________________________________________
>>> Dxspider-support mailing list
>>> Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk>
>>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
>> 
>> 
>> 
>> _______________________________________________
>> Dxspider-support mailing list
>> Dxspider-support at tobit.co.uk <mailto:Dxspider-support at tobit.co.uk>
>> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support
> 
> _______________________________________________
> Dxspider-support mailing list
> Dxspider-support at tobit.co.uk
> https://mailman.tobit.co.uk/mailman/listinfo/dxspider-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.tobit.co.uk/pipermail/dxspider-support/attachments/20221207/091d111d/attachment-0001.htm>

More information about the Dxspider-support mailing list