[Dxspider-support] MYSQL duplication/mass insertion on restart
Jason R Begley
jayray at digitalgoat.com
Tue Feb 14 20:28:37 GMT 2012
I don't recall how I set this up, and it was going to be semi-temporary
until I started integrating it into my webpage.. I normally write this
stuff down, but in this case I didn't... Shame on me.. Below is the
output: I also just ran as normal below, and you can see that I ran it
as root and it crashed because it couldnt write to the spot file, then I
restarted and you can see the behavior... it starts re-importing the
spots from the start.
TU!
KY9J
cluster
DXSpider V1.55, build 0.94 (git: 6262a02) started
Copyright (c) 1998-2012 Dirk Koopman G1TLH
loading prefixes ...
US Database not loaded
loading band data ...
loading user file system ...
starting listeners ...
Internal port: localhost 27754 using IntMsg
External Port: 0.0.0.0 7300 using ExtMsg::login
load badwords: Ok
Starting Dupe system
Read in Messages
Read in Aliases
Start WWV
Start WCY
Starting DX Spot system
initialising spot tables
inserted 0 spots from 2011 305
inserted 2589 spots from 2011 306
inserted 5483 spots from 2011 307
inserted 6375 spots from 2011 308
inserted 10836 spots from 2011 309
inserted 15462 spots from 2011 310
inserted 6410 spots from 2011 311
inserted 5983 spots from 2011 312
inserted 5693 spots from 2011 313
^CSpot::init(/spider/perl/Spot.pm:135):
135: $count++;
DB<1> T
. = Spot::init('Spot') called from file `cluster.pl' line 487
DB<1> c
cluster
DXSpider V1.55, build 0.94 (git: 6262a02) started
Copyright (c) 1998-2012 Dirk Koopman G1TLH
loading prefixes ...
US Database not loaded
loading band data ...
loading user file system ...
starting listeners ...
Internal port: localhost 27754 using IntMsg
External Port: 0.0.0.0 7300 using ExtMsg::login
load badwords: Ok
Starting Dupe system
Read in Messages
Read in Aliases
Start WWV
Start WCY
Starting DX Spot system
initialising spot tables
inserted 0 spots from 2011 305
inserted 2589 spots from 2011 306
inserted 5483 spots from 2011 307
inserted 6375 spots from 2011 308
inserted 10836 spots from 2011 309
inserted 15462 spots from 2011 310
inserted 6410 spots from 2011 311
inserted 5983 spots from 2011 312
inserted 5693 spots from 2011 313
inserted 6701 spots from 2011 314
inserted 7236 spots from 2011 315
inserted 13054 spots from 2011 316
inserted 12695 spots from 2011 317
inserted 5313 spots from 2011 318
inserted 4922 spots from 2011 319
inserted 5792 spots from 2011 320
inserted 5531 spots from 2011 321
inserted 5912 spots from 2011 322
inserted 10640 spots from 2011 323
inserted 16769 spots from 2011 324
inserted 6339 spots from 2011 325
inserted 5663 spots from 2011 326
inserted 5909 spots from 2011 327
inserted 6905 spots from 2011 328
inserted 6893 spots from 2011 329
inserted 24155 spots from 2011 330
inserted 25293 spots from 2011 331
inserted 3858 spots from 2011 332
inserted 4547 spots from 2011 333
inserted 4371 spots from 2011 334
inserted 4189 spots from 2011 335
inserted 4835 spots from 2011 336
inserted 9980 spots from 2011 337
inserted 11109 spots from 2011 338
inserted 4060 spots from 2011 339
inserted 4475 spots from 2011 340
inserted 4150 spots from 2011 341
inserted 4883 spots from 2011 342
inserted 4632 spots from 2011 343
inserted 13792 spots from 2011 344
inserted 15284 spots from 2011 345
inserted 2937 spots from 2011 346
inserted 3418 spots from 2011 347
inserted 3809 spots from 2011 348
inserted 3168 spots from 2011 349
inserted 3609 spots from 2011 350
inserted 7476 spots from 2011 351
inserted 6365 spots from 2011 352
inserted 3052 spots from 2011 353
inserted 3360 spots from 2011 354
inserted 3208 spots from 2011 355
inserted 3272 spots from 2011 356
inserted 3574 spots from 2011 357
inserted 4957 spots from 2011 358
inserted 5565 spots from 2011 359
inserted 4777 spots from 2011 360
inserted 4960 spots from 2011 361
inserted 4800 spots from 2011 362
inserted 5484 spots from 2011 363
inserted 6049 spots from 2011 364
inserted 6807 spots from 2011 365
inserted 6118 spots from 2012 1
inserted 6382 spots from 2012 2
inserted 4256 spots from 2012 3
inserted 4313 spots from 2012 4
inserted 3559 spots from 2012 5
inserted 4374 spots from 2012 6
inserted 7540 spots from 2012 7
inserted 9733 spots from 2012 8
inserted 2709 spots from 2012 9
inserted 3014 spots from 2012 10
inserted 2829 spots from 2012 11
inserted 2815 spots from 2012 12
inserted 3661 spots from 2012 13
inserted 6858 spots from 2012 14
inserted 6961 spots from 2012 15
inserted 3881 spots from 2012 16
inserted 3651 spots from 2012 17
inserted 4195 spots from 2012 18
inserted 3910 spots from 2012 19
inserted 4374 spots from 2012 20
inserted 8978 spots from 2012 21
inserted 8964 spots from 2012 22
inserted 4927 spots from 2012 23
inserted 4694 spots from 2012 24
inserted 4438 spots from 2012 25
inserted 4993 spots from 2012 26
inserted 5422 spots from 2012 27
inserted 10769 spots from 2012 28
inserted 9591 spots from 2012 29
inserted 4258 spots from 2012 30
inserted 4179 spots from 2012 31
inserted 3961 spots from 2012 32
inserted 3862 spots from 2012 33
inserted 4721 spots from 2012 34
inserted 8573 spots from 2012 35
inserted 7429 spots from 2012 36
inserted 3281 spots from 2012 37
inserted 3745 spots from 2012 38
inserted 3546 spots from 2012 39
inserted 3396 spots from 2012 40
inserted 3832 spots from 2012 41
inserted 10874 spots from 2012 42
inserted 10885 spots from 2012 43
inserted 3451 spots from 2012 44
inserted 37 spots from 2012 45
adding spot index ix1
adding spot index ix2
657304 spots converted in 5:59
Start Protocol Engines ...
DXChannel KY9J-2 created (1)
reading existing message headers ...
load badmsg: Ok
load forward: Ok
load swop: Ok
reading cron jobs ...
cron: reading /spider/cmd/crontab
cron: adding 1 0 * * 3 DXUser::export("$main::data/user_asc")
cron: adding 5 0 * * * DXDebug::dbgclean()
cron: adding 0 3 * * * Spot::daily()
cron: reading /spider/local_cmd/crontab
cron: adding 0,10,20,30,40,50 * * * * start_connect('wr3d') unless
connected('wr3d')
reading database descriptors ...
doing local initialisation ...
orft we jolly well go ...
queue msg (0)
queue msg (0)
cron: 0 20 14 2 2 -> doing 'start_connect('wr3d') unless connected('wr3d')'
connect 3: timeout set to 60
connect 3: timeout set to 300
CONNECT 3 sort: telnet command: wr3d.dxcluster.net 7300
Connected 3 to wr3d.dxcluster.net 7300
cmd run: connect wr3d
cmd out: connection to WR3D started
connect 3: This is the WR3D (ex N2TLY) DXCluster system using DXSpider
software%0D
connect 3: expecting: "login:" received: "This is the WR3D (ex N2TLY)
DXCluster system using DXSpider software%0D
"
connect 3: This is the WR3D (ex N2TLY) DXCluster system using DXSpider
software%0D
If your terminal doesn't echo characters, please set the 'local%0D
echo' option (this is only likely to affect Microsoft users)%0D
login:
connect 3: expecting: "login:" received: "This is the WR3D (ex N2TLY)
DXCluster system using DXSpider software%0D
If your terminal doesn't echo characters, please set the 'local%0D
echo' option (this is only likely to affect Microsoft users)%0D
login: "
connect 3: got: "login:" sending: "KY9J-2"
connect 3: KY9J-2%0D
%0D
DXChannel WR3D created (2)
<- O WR3D telnet
-> B WR3D 0
-> E WR3D 0
WR3D channel func state 0 -> init
<- I WR3D KY9J-2
<- I WR3D
<- I WR3D *** Connected to WR3D in Baltimore
<- I WR3D PC18^DXSpider Version: 1.56 Build: 0.17 Git: 281b5d6 pc9x^5456^
WR3D channel func state init -> init
DXSpider version 1.56 build 0.17
Do px9x set on WR3D
send_local_config: doing pc9x
-> D WR3D PC92^KY9J-2^72000^A^^5WR3D:83.244.159.195^H99^
-> D WR3D PC92^KY9J-2^72000.01^K^5KY9J-2:5455:0.94^1^0^H99^
-> D WR3D PC20^
<- I WR3D PC92^HA5DI-5^72000^D^^5HG5ADY^H94^
<- I WR3D PC92^HA5DI-5^72000.01^D^^5HG5DI^H94^
<- I WR3D PC92^DB0ERF-5^72000^A^^7DB0TUD:127.0.0.1^H96^
<- I WR3D PC92^WR3D^72000^A^^5KY9J-2:99.119.146.17^H99^
WR3D channel func state init -> init92
Do pc9x set on WR3D
<- I WR3D PC92^WR3D^72000.01^K^5WR3D:5456:0.17^16^5^H99^
<- I WR3D PC22^
WR3D channel func state init92 -> normal
<- I WR3D PC92^G7VJR-5^72001^D^^5N7OD^H95^
<- I WR3D PC92^F1OYP-8^71996^A^^7F1MAC-6:109.190.2.205^H97^
<- I WR3D PC93^PY3NZ-8^72000^*^PY3NZ-8^*^PY3NZ-8 Araucaria DX Group
Dxcluster - telnet 189.42.190.3 8000^H95^
<- I WR3D PC92^IK2DUW-6^72001^D^^5IK2XDE-6^H95^
<- I WR3D PC92^IW5ECF-6^72002^D^^5IK4ICZ-6^H97^
<- I WR3D PC92^IW5ECF-6^72002.01^D^^5WC2L^H95^
<- I WR3D PC41^UT6IC^4^PI4CC^H8^~
<- I WR3D PC51^KY9J-2^WR3D^1^
-> D WR3D PC51^WR3D^KY9J-2^0^
<- I WR3D PC11^14024.0^6O3A^14-Feb-2012^2000Z^Sweet, wrk'd
14.026.60^KI4TWA^K4UGA^H23^~
can't open /spider/data/spots/2012/045.dat Permission denied at
/spider/perl/DXLog.pm line 149
DXLog::write('DXLog=HASH(0x90aec40)',
'Julian::Day=ARRAY(0x90bc928)', '14024.0^6O3A^1329249600^Sweet, wrk\'d
14.026.60^KI4TWA^163^22...') called at /spider/perl/DXLog.pm line 173
DXLog::writeunix('DXLog=HASH(0x90aec40)', 1329249600,
'14024.0^6O3A^1329249600^Sweet, wrk\'d 14.026.60^KI4TWA^163^22...')
called at /spider/perl/Spot.pm line 191
Spot::add(14024.0, '6O3A', 1329249600, 'Sweet, wrk\'d
14.026.60', 'KI4TWA', 163, 226, 'K4UGA', 48, ...) called at
/spider/perl/DXProtHandle.pm line 219
DXProt::handle_11('DXProt=HASH(0x90b3190)', 11,
'PC11^14024.0^6O3A^14-Feb-2012^2000Z^Sweet, wrk\'d 14.026.60^K...',
'WR3D', 'PC11', 14024.0, '6O3A', '14-Feb-2012', '2000Z', ...) called at
/spider/perl/DXProt.pm line 432
DXProt::normal('DXProt=HASH(0x90b3190)',
'PC11^14024.0^6O3A^14-Feb-2012^2000Z^Sweet, wrk\'d 14.026.60^K...')
called at /spider/perl/DXChannel.pm line 722
DXChannel::process() called at cluster.pl line 541
at /spider/perl/DXLog.pm line 149
DXLog::write('DXLog=HASH(0x90aec40)',
'Julian::Day=ARRAY(0x90bc928)', '14024.0^6O3A^1329249600^Sweet, wrk\'d
14.026.60^KI4TWA^163^22...') called at /spider/perl/DXLog.pm line 173
DXLog::writeunix('DXLog=HASH(0x90aec40)', 1329249600,
'14024.0^6O3A^1329249600^Sweet, wrk\'d 14.026.60^KI4TWA^163^22...')
called at /spider/perl/Spot.pm line 191
Spot::add(14024.0, '6O3A', 1329249600, 'Sweet, wrk\'d
14.026.60', 'KI4TWA', 163, 226, 'K4UGA', 48, ...) called at
/spider/perl/DXProtHandle.pm line 219
DXProt::handle_11('DXProt=HASH(0x90b3190)', 11,
'PC11^14024.0^6O3A^14-Feb-2012^2000Z^Sweet, wrk\'d 14.026.60^K...',
'WR3D', 'PC11', 14024.0, '6O3A', '14-Feb-2012', '2000Z', ...) called at
/spider/perl/DXProt.pm line 432
DXProt::normal('DXProt=HASH(0x90b3190)',
'PC11^14024.0^6O3A^14-Feb-2012^2000Z^Sweet, wrk\'d 14.026.60^K...')
called at /spider/perl/DXChannel.pm line 722
DXChannel::process() called at cluster.pl line 541
can't open /spider/data/spots/2012/045.dat Permission denied at
/spider/perl/DXLog.pm line 149
DXLog::write('DXLog=HASH(0x90aec40)',
'Julian::Day=ARRAY(0x90bc928)', '14024.0^6O3A^1329249600^Sweet, wrk\'d
14.026.60^KI4TWA^163^22...') called at /spider/perl/DXLog.pm line 173
DXLog::writeunix('DXLog=HASH(0x90aec40)', 1329249600,
'14024.0^6O3A^1329249600^Sweet, wrk\'d 14.026.60^KI4TWA^163^22...')
called at /spider/perl/Spot.pm line 191
Spot::add(14024.0, '6O3A', 1329249600, 'Sweet, wrk\'d
14.026.60', 'KI4TWA', 163, 226, 'K4UGA', 48, ...) called at
/spider/perl/DXProtHandle.pm line 219
DXProt::handle_11('DXProt=HASH(0x90b3190)', 11,
'PC11^14024.0^6O3A^14-Feb-2012^2000Z^Sweet, wrk\'d 14.026.60^K...',
'WR3D', 'PC11', 14024.0, '6O3A', '14-Feb-2012', '2000Z', ...) called at
/spider/perl/DXProt.pm line 432
DXProt::normal('DXProt=HASH(0x90b3190)',
'PC11^14024.0^6O3A^14-Feb-2012^2000Z^Sweet, wrk\'d 14.026.60^K...')
called at /spider/perl/DXChannel.pm line 722
DXChannel::process() called at cluster.pl line 541
DXChannel WR3D destroyed (2)
Can't call method "year" on an undefined value at /spider/perl/DXLog.pm
line 149 during global destruction.
(in cleanup) Can't call method "year" on an undefined value at
/spider/perl/DXLog.pm line 149 during global destruction.
Can't call method "year" on an undefined value at /spider/perl/DXLog.pm
line 149 during global destruction.
Can't call method "year" on an undefined value at /spider/perl/DXLog.pm
line 149 during global destruction.
#### snip misc garbage
[ky9j at linus perl]$ perl cluster.pl
cluster
DXSpider V1.55, build 0.94 (git: 6262a02) started
Copyright (c) 1998-2012 Dirk Koopman G1TLH
loading prefixes ...
US Database not loaded
loading band data ...
loading user file system ...
starting listeners ...
Internal port: localhost 27754 using IntMsg
External Port: 0.0.0.0 7300 using ExtMsg::login
load badwords: Ok
Starting Dupe system
Read in Messages
Read in Aliases
Start WWV
Start WCY
Starting DX Spot system
initialising spot tables
DBD::mysql::db do failed: Table 'spot' already exists at
/spider/perl/DXSql.pm line 88.
at /spider/perl/DXSql.pm line 88
inserted 0 spots from 2011 305
inserted 2589 spots from 2011 306
inserted 5483 spots from 2011 307
inserted 6375 spots from 2011 308
inserted 10836 spots from 2011 309
inserted 15462 spots from 2011 310
inserted 6410 spots from 2011 311
inserted 5983 spots from 2011 312
inserted 5693 spots from 2011 313
inserted 6701 spots from 2011 314
inserted 7236 spots from 2011 315
inserted 13054 spots from 2011 316
inserted 12695 spots from 2011 317
inserted 5313 spots from 2011 318
inserted 4922 spots from 2011 319
inserted 5792 spots from 2011 320
inserted 5531 spots from 2011 321
inserted 5912 spots from 2011 322
inserted 10640 spots from 2011 323
inserted 16769 spots from 2011 324
inserted 6339 spots from 2011 325
inserted 5663 spots from 2011 326
More information about the Dxspider-support
mailing list