User:WikiSysop/TECH: Difference between revisions

Latest revision as of 10:14, 8 November 2022

MULTIMEDIA CONTAINER

TUNEARCH PROCEDURES

DB CONNECT mysql --user=tunearch_user --password=<password> --database=tunearch_wikidb

DB SELECT SELECT * FROM page where page_title = '<ARTICLE>'; SELECT * FROM smw_ids where smw_id = <SOME_ID;

MYSQLDUMP
On Mac
/usr/bin/mysqldump -u tunearch_user --password=<password> --default-character-set=latin1 tunearch_wikidb -c | /bin/gzip -9 > ~/public_html/tunearchftp/TuneArch-$(date '+%Y%m%d').sql.gz
On Windows
/usr/bin/mysqldump -u tunearch_user --password=<password> --default-character-set=latin1 tunearch_wikidb -c | /bin/gzip -9 > TuneArch-$(date '+%Y%m%d').sql.gz

XMLDUMP /usr/bin/php -n -d register_argc_argv=true /home/tunearch/public_html/w/maintenance/dumpBackup.php --full | /bin/gzip -9 > ~/public_html/tunearchftp/TuneArch-$(date '+%Y%m%d').xml.gz

PHP
/usr/bin/php -n -d register_argc_argv=true
/usr/local/php55/bin/php update.php

STATISTICS
https://tunearch.org/w/api.php?format=xml&action=query&meta=siteinfo&siprop=statistics
RUN JOBS
/usr/bin/php -n -d register_argc_argv=true runJobs.php --maxjobs 5
/usr/local/php55/bin/php -n -d register_argc_argv=true runJobs.php --maxjobs 5
scp tunearch@184.154.160.18:/home/tunearch/backup/TTA-20110430.xml.gz .
password: <password>

UPGRADE
tar xvzf mediawiki-1.17.0.tar.gz -C /path/to/your/wiki/ --strip-components=1

INDEXING
cronjob
/home/tunearch/opt/usr/bin/indexer --quiet --config /home/tunearch/opt/etc/sphinx/sphinx.conf --rotate > /dev/null 2>&1

manual indexing
/home/tunearch/opt/usr/bin/indexer --config /home/tunearch/opt/etc/sphinx/sphinx.conf --all

SEARCH DAEMON
start
/home/tunearch/opt/usr/bin/searchd --config /home/tunearch/opt/etc/sphinx/sphinx.conf

stop
/home/tunearch/opt/usr/bin/searchd --config /home/tunearch/opt/etc/sphinx/sphinx.conf --stop

MOVE BATCH

/usr/bin/php -n -d register_argc_argv=true moveBatch.php ./renfile.txt

RUN JOBS

/usr/bin/php -n -d register_argc_argv=true runJobs.php --maxjobs=2000

SIDEBAR

- Special:RunQuery/LinkedInForm|Tune Connections

Rebuilding everything

The above methods should be able to fix data records in SMW in most cases. However, it is conceivable that some erroneous content of the SMW storage still persists for some reason. In this case, it makes sense to completely delete and reinstall the database structures of SMW before refreshing all data. To completely delete all SMW data, the setup script SMW_setup.php is used with parameter --delete:

php SMW_setup.php --delete

After this, proceed as if re-installing SMW anew by first running php SMW_setup.php again, and then triggering the repair of all data using one of the above methods. The refresh script SMW_refreshData.php can be also used with parameter -f to delete and recreate all data in one step. In this case, it is suggested to first rebuild the records for all properties and types, and to process the remaining data afterwards. So one would run:

php SMW_refreshData.php -ftpv
php SMW_refreshData.php -v

Note that of course only the first run uses -f. On large wikis, the parameters -s and -e can again be used as explained in the previous section.

Not sure if this is the kind of level you are after but this is what I wrote
up for use in my organisation if we need to perform a complete disaster
recovery of our wiki. This has been tested numerous times for both recovery
on the same server, as well as migrating to another server. I haven't tried
between OS though but don't see there would be any issues with that.

Before doing this, Apache2 needs to be configured in the same manner (ie
same directory root etc), correct php version installed and compiled, etc.

A couple of things: wikidb.servername.$DAY is just a gzipped mysql dump of
the database. wikifiles.$DAY.tbz is simply a tar of the entire directory
structure. I have a cron job run daily to backup both.

Create Database
Mysql –u root –p
Create database wikidb;
Exit
Restore Database files
Cd /<backup location>
Gunzip wikidb.servername.$DAY.gz
Mysql –u root –p wikidb < wikidb.servername.$DAY

Ensure database restored
mysql -u root -p wikidb
show tables;
<list of tables (about 35 should appear).

Create Wiki User account in MYSQL
Mysql –u root -p
create user 'wikiuser'@'localhost' IDENTIFIED BY 'password to be used:
IMPORTANT make sure this is also the same as in the LocalSettings.php
file!';
Grant privileges to wikiuser on wikidb database
Mysql –u root –p
GRANT ALL ON wikidb.* TO wikiuser at localhost IDENTIFIED BY 'password to be
used';
Flush privileges;

Restore wiki config files and images
Cd /
Tar –xvjf /<backup location>/wikifiles.$DAY.tbz
Complete restore process complete – to make sure, perform test plan again

Potential things that ** may ** be required are:
/<wiki install>/maintenance/php5 rebuildall.php
Double double check the correct database, username, password are used in the
/<wiki install>/LocalSettings.php file
Make sure apache is set up correctly, and the wiki location set
appropriately in the LocalSettings.php file

CAPTCHA

python captcha.py --font=/usr/share/fonts/truetype/arial.ttf --wordlist=/home/tunearch/opt/dict/words --key=FOO --output=../CAPTCHA --count=100

@@ Line 1: / Line 1: @@
+__NOEDITSECTION__
+<statistics pages="85741" articles="36489" edits="271290" images="2003" users="930" activeusers="3" admins="3" jobs="0"/>
+<br>
+== MULTIMEDIA CONTAINER ==
+<!-- <div style="max-width: 750px"><multimediacontainer /></div> -->
+<!--[[File:Clef-64x64.png||link=|alt=Listen to the featured tunes of the week]]{{#multimediaitem:YouTube| cAluOq7xTQc |<B>Petronella'</B>}}<br>-->
+<!--[[File:Clef-64x64.png||link=|alt=Listen to the tune]]{{#multimediaitem:MP3|PigAnkle-Hutchison.mp3|PIG ANKLE RAG}} -->
+<!--[[File:Clef-64x64.png||link=|alt=Listen to the featured tunes of the week]]{{#multimediaitem:SoundCloud|218721756|<B>WESTERING HOME</B>}}-->
+<!--[[File:YTDevil.jpg|55px|link=|alt=Listen to the featured tunes of the week]]{{#multimediaitem:YouTube|eg0w3XJmq2Q|<B>WESTERING HOME</B>}}-->
 '''TUNEARCH PROCEDURES'''
 <br>
 <br>
-'''MYSQLDUMP'''
-/usr/bin/mysqldump -u tunearch_user --password=AD41f018 --default-character-set=latin1 tunearch_wikidb -c | /bin/gzip -9 > ~/public_html/tunearchftp/TuneArch-$(date '+%Y%m%d').sql.gz
+'''DB CONNECT'''
+mysql --user=tunearch_user --password=<password> --database=tunearch_wikidb
+'''DB SELECT'''
+SELECT * FROM page where page_title = '<ARTICLE>';
+SELECT * FROM smw_ids where smw_id = <SOME_ID;
+'''MYSQLDUMP''' <br>
+'''On Mac'''<br>
+/usr/bin/mysqldump -u tunearch_user --password=<password> --default-character-set=latin1 tunearch_wikidb -c | /bin/gzip -9 > ~/public_html/tunearchftp/TuneArch-$(date '+%Y%m%d').sql.gz
+<br>
+'''On Windows''' <br>
+/usr/bin/mysqldump -u tunearch_user --password=<password> --default-character-set=latin1 tunearch_wikidb -c | /bin/gzip -9 > TuneArch-$(date '+%Y%m%d').sql.gz
 <br>
 <br>
@@ Line 10: / Line 35: @@
 <br>
 <br>
-Error message: Cannot get command line arguments, register_argc_argv is set to false (TODO: Investigate).<br>
+'''PHP'''<br>
 /usr/bin/php -n -d register_argc_argv=true
 <br>
+/usr/local/php55/bin/php update.php
+<br>
+<br>
+'''STATISTICS'''<br>
+https://tunearch.org/w/api.php?format=xml&action=query&meta=siteinfo&siprop=statistics<br>
+'''RUN JOBS'''<br>
+/usr/bin/php -n -d register_argc_argv=true runJobs.php --maxjobs 5
+<br>
+/usr/local/php55/bin/php -n -d register_argc_argv=true runJobs.php --maxjobs 5
 <br>
 scp tunearch@184.154.160.18:/home/tunearch/backup/TTA-20110430.xml.gz .<br>
-password: TI46;DSC;A
+password: <password>
 <br>
 <br>
@@ Line 51: / Line 85: @@
 <br>
+=== MOVE BATCH ===
+/usr/bin/php -n -d register_argc_argv=true moveBatch.php ./renfile.txt
+=== RUN JOBS ===
+/usr/bin/php -n -d register_argc_argv=true runJobs.php --maxjobs=2000
 '''SIDEBAR'''
 ** Special:RunQuery/LinkedInForm|Tune Connections
+'''Rebuilding everything'''
+The above methods should be able to fix data records in SMW in most cases. However, it is conceivable that some erroneous content of the SMW storage still persists for some reason. In this case, it makes sense to completely delete and reinstall the database structures of SMW before refreshing all data.
+To completely delete all SMW data, the setup script SMW_setup.php is used with parameter --delete:
+ php SMW_setup.php --delete
+After this, proceed as if re-installing SMW anew by first running php SMW_setup.php again, and then triggering the repair of all data using one of the above methods.
+The refresh script SMW_refreshData.php can be also used with parameter -f to delete and recreate all data in one step. In this case, it is suggested to first rebuild the records for all properties and types, and to process the remaining data afterwards. So one would run:
+ php SMW_refreshData.php -ftpv
+ php SMW_refreshData.php -v
+Note that of course only the first run uses -f. On large wikis, the parameters -s and -e can again be used as explained in the previous section.
+----
+<pre>
+Not sure if this is the kind of level you are after but this is what I wrote
+up for use in my organisation if we need to perform a complete disaster
+recovery of our wiki. This has been tested numerous times for both recovery
+on the same server, as well as migrating to another server. I haven't tried
+between OS though but don't see there would be any issues with that.
+Before doing this, Apache2 needs to be configured in the same manner (ie
+same directory root etc), correct php version installed and compiled, etc.
+A couple of things: wikidb.servername.$DAY is just a gzipped mysql dump of
+the database. wikifiles.$DAY.tbz is simply a tar of the entire directory
+structure. I have a cron job run daily to backup both.
+Create Database
+Mysql –u root –p
+Create database wikidb;
+Exit
+Restore Database files
+Cd /<backup location>
+Gunzip wikidb.servername.$DAY.gz
+Mysql –u root –p wikidb < wikidb.servername.$DAY
+Ensure database restored
+mysql -u root -p wikidb
+show tables;
+<list of tables (about 35 should appear).
+Create Wiki User account in MYSQL
+Mysql –u root -p
+create user 'wikiuser'@'localhost' IDENTIFIED BY 'password to be used:
+IMPORTANT make sure this is also the same as in the LocalSettings.php
+file!';
+Grant privileges to wikiuser on wikidb database
+Mysql –u root –p
+GRANT ALL ON wikidb.* TO wikiuser at localhost IDENTIFIED BY 'password to be
+used';
+Flush privileges;
+Restore wiki config files and images
+Cd /
+Tar –xvjf /<backup location>/wikifiles.$DAY.tbz
+Complete restore process complete – to make sure, perform test plan again
+Potential things that ** may ** be required are:
+/<wiki install>/maintenance/php5 rebuildall.php
+Double double check the correct database, username, password are used in the
+/<wiki install>/LocalSettings.php file
+Make sure apache is set up correctly, and the wiki location set
+appropriately in the LocalSettings.php file
+</pre>
+CAPTCHA
+python captcha.py --font=/usr/share/fonts/truetype/arial.ttf --wordlist=/home/tunearch/opt/dict/words --key=FOO --output=../CAPTCHA --count=100