Sunday, December 04, 2016

Setup passwordless SSH login

Found a easy to use step-by-step guide for setting up passwordless ssh login (works on Cygwin).

http://www.linuxhowtos.org/Security/SSH%20login%20without%20password.htm

Sunday, May 22, 2016

Caste, Varna & Jati

See my presentation to RAYS at their 2015 conference in San Francisco


Sunday, December 13, 2015

Notes from Upgrading Hindupedia

I just went through a month long process to upgrade Hindupedia's Debian and Mediawiki from an ancient version installed 5 years ago (Debian) and Mediawiki 1.16.5 to the current version.

This was necessitated by Rackspace forcing me to upgrade our underlying virtual server (from the original virtual server to their "next gen server") which caused Hindupedia to constantly crash.  A problem that took them 3 weeks to diagnose!

The new server at Rackspace has very poor performance as compared to the original server (which I found out after the new site went live)...I have had to upgrade to a higher-performance server but the site performs worse than the old one...requiring lots of additional optimizations.  I would move off of Rackspace if it wasn't so time consuming to do so...

Below are my raw notes and links to helpful sites I found along the way from the migration.


 Restrict account creation -- "Autoconfirmed accounts"  
 Optimize MySQL: https://github.com/major/MySQLTuner-perl   
                  http://haydenjames.io/mysql-query-cache-size-performance/  
 Optimize Apache: http://apache2buddy.pl/  
 Spam cleanup: http://webmasters.stackexchange.com/questions/9194/mass-deletion-of-spam-revisions-in-mediawiki/87775#87775  
 Defragment tables: http://stackoverflow.com/questions/26514073/mysql-defragment-innodb-tables  
 Optimize mediawiki: http://techwelkin.com/slow-mediawiki-optimize-to-enhance-performance-part-1  
 Configure security: http://www.rackspace.com/knowledge_center/article/configuring-basic-security-0  
 COnfigure iptables: http://www.gethackingsecurity.com/lockdown-ubuntu-2/  
 Apache caching for mediawiki: https://testbit.eu/using-mod_disk_cache-with-mediawiki/  
 Check page header: wget -S --delete-after -nd http://www.hindupedia.com/en/Main_Page  
 Memcache instructions: http://wiki.theplaz.com/Memcached_on_1and1_with_MediaWiki  
 {  
 ##mods-enabled/mpm_prefork.conf         
     StartServers           15  
     MinSpareServers      2  
     MaxSpareServers     31  
     MaxRequestWorkers    31  
     MaxConnectionsPerChild  10000  
   
   
 ##apache2.conf  
 <IfModule mpm_prefork_module>  
   StartServers     15  
   MinSpareServers    2  
   MaxSpareServers   31  
   MaxClients      31  
   MaxRequestsPerChild  10000  
 </IfModule>  
 }  
 {  
 PageTools.php Line 80  
   
   
         $user = $this->getSkinTemplate()->getSkin()->getUser();  
   
         if ( $user->isLoggedIn() ) {  
         } else {  
         }  
   
 }  
 LinkTitles  
 {  
  don't add links in templates  
  doesn't recognize diacritical marks on characters  
 }  
 Extra space above title:  
 .p-contentnavigation {  
   display:none  
 }  
   
 Google Site Search extn: https://www.mediawiki.org/wiki/Extension:GoogleCustomWikiSearch  
 Last modified date: https://www.mediawiki.org/wiki/Manual:Footer  
   
   
                 <component type="Html"><![CDATA[<div class="socialWrap">  
 <div class="twitter_follow_us">  
 <a name="Follow on Twitter" href="http://www.twitter.com/Hindupedia" title="Follow on Twitter"><img rip-style-borderwidth-backup="" rip-style-borderstyle-backup="" style="" rip-style-bordercolor-backup="" src="/eng/images/follow_on_twitter_icon.png" alt="Follow on Twitter" height="22"></a>  
 </div>  
 <div class="addthis_sharing_toolbox1 addthis_toolbox addthis_default_style"></div>  
 </div>]]></component>  
   
 MAGIC WORD issues: https://gerrit.wikimedia.org/r/#/c/107259/1/includes/MagicWord.php  
   
 { Server setup script  
 #!/bin/bash  
 IP="166.78.121.59"  
 cd /var/www/www.hindupedia.com  
 echo `pwd`  
 rm -rf hindupedia.sql* htdocs.1112015 htdocs_* htdocs.bak* htdocs.mediawiki.1.25 htdocs.new htdocs.tar root.dir.tgz /var/www/www.hindupedia.com/logs/* /var/www/www.hindupedia.com/htdocs/eng/StartProfiler.php &  
 apt-get install -y php5-dev php-pear libcurl3-openssl-dev php5-intl libicu52 icu-devtools libicu-dev php-apc imagemagick  
 yes '' | pear install pear  
 yes '' | pecl install intl  
 pear install MAIL Net_SMTP  
 echo "extension=raphf.so,propro.so,intl.so" >> /etc/php5/apache2/php.ini   
 sed -i "s/\<VirtualHost.*:80\>/VirtualHost $IP:80/" /etc/apache2/sites-available/www.hindupedia.com.conf  
 wget releases.wikimedia.org/mediawiki/1.25/mediawiki-1.25.3.tar.gz  
 scp root@www.hindupedia.com:~/LocalSettings.php /var/www/www.hindupedia.com/htdocs/eng/.   
 /etc/init.d/apache2 restart  
   
 tar -xvzf mediawiki-1.25.3.tar.gz   
 cp -rf mediawiki-1.25.3/* htdocs/eng/.  
 yes '' | pecl install pecl_raphf pecl_http   
 http://www.hindupedia.org/eng/mw-config/  
 http://www.hindupedia.org/eng/mw-config/  
   
 map q O###KKM update incompatible###^[j^i#^[:w^M  
   
 }  
   
 fix upload security: https://www.mediawiki.org/wiki/Manual:Security#Upload_security  
   
 PECL: http://www.mkfoster.com/2009/01/04/how-to-install-a-php-pecl-extensionmodule-on-ubuntu/  
 http://www.gossamer-threads.com/lists/wiki/mediawiki/  
 Skin: Foreground ( http://foreground.thingelstad.com/wiki/Main_Page )  
 Extension: LinkTitles ( https://www.mediawiki.org/wiki/Extension:LinkTitles )  
 Extension: DynamicPageList (3rd party) ( https://www.mediawiki.org/wiki/Extension:DynamicPageList_%28third-party%29 )  
 cpan CPAN App::cpanminus --installdeps  
 cpanm MediaWiki::Bot --installdeps  
 cpanm MediaWiki::Bot   
 cpanm Data::Dumper::Simple   
 cpanm MediaWiki::API   
 [roothp6:/var/www/www.hindupedia.com/htdocs/eng/extensions]$ rm -rf DynamicPageList simplefeed rss BreadCrumbs  
   
 Import dump:   
   
 { Chameleon skin  
 Skin: Chameleon ( https://www.mediawiki.org/wiki/Skin:Chameleon )  
 * To Install  
 cd /var/www/www.hindupedia.com/htdocs/eng  
 curl -sS https://getcomposer.org/installer | php  
 mv composer.phar /usr/local/bin/composer  
 php /usr/local/bin/composer update  
 php /usr/local/bin/composer require mediawiki/chameleon-skin "1.*"  
 apt-get install git node npm  
 curl -sL https://deb.nodesource.com/setup_4.x | sudo -E bash -  
 sudo apt-get install -y nodejs  
 sudo apt-get install -y build-essential  
 add '$wgDefaultSkin='chameleon';' in LocalSettings.php  
 * To install theme 'United'  
 http://bootswatch.com/help/  
 cd /var/www/www.hindupedia.com  
 git clone https://github.com/thomaspark/bootswatch.git  
 npm install -g grunt-cli  
 npm install  
 apt-get autoremove  
 https://www.mediawiki.org/wiki/Skin_talk:Chameleon  
 }  
   
 { Extension: ColList  
 List extensions: https://www.mediawiki.org/wiki/Category:List_extensions  
 https://www.mediawiki.org/wiki/Extension:ColList ## is bad...  
 https://www.mediawiki.org/wiki/Extension:Sort2  
   
 }  
   
 pear list # verify Net_SMTP OK.  
   
 Autolink extension: LinkTitles  
 Fix bug: https://github.com/bovender/LinkTitles/issues/5 (change "/u" to "/" in line 227 in LinkTitles.body.php  
   
 DynamicPageList extension:  
 (Main page tag)  
 <dpl>  
 namespace=Quotes  
 category=Sagely Quotes  
 include=*  
 includemaxlength=450  
 escapelinks=false  
 randomcount=1  
 reset=categories   
 addpagecounter=true  
 mode=userformat  
 resultsheader=__NOTOC__ __NOEDITSECTION__  
 listseparators={{mp-topicheader-h3|Sagely Quote}}\n  
 </dpl>  
 *** But need to fix table width dynamically resizing ***   
   
 *** Need to enable page history  
 i.e., http://www.hindupedia.org/eng/index.php?title=Main_Page&curid=1&action=history  
   
 http://stackoverflow.com/questions/3513773/change-mysql-default-character-set-to-utf-8-in-my-cnf  
 http://alvinalexander.com/blog/post/mysql/list-tables-in-mysql-database  
   
 { PHPMyAdmin for SQL  
  apt-get install phpmyadmin (for pre-requisites)  
  wget https://files.phpmyadmin.net/phpMyAdmin/4.5.1/phpMyAdmin-4.5.1-all-languages.tar.gz  
  http://wiki.phpmyadmin.net/pma/Setup  
   
 }  
   
 { SQL  
 mysql -u root -p <pw>  
 set names utf8;  
 show tables;  
 SHOW TABLE STATUS;  
 SHOW TABLE STATUS WHERE `NAME` LIKE '%page%';  
 show variables like 'char%';  
 show variables like 'collation%';  
 SHOW COLUMNS FROM page;  
 select page_id,page_title from page where page_title like 'Adhik%';  
 show index in page;  
 ALTER TABLE page MODIFY page_title VARCHAR(255) CHARACTER SET utf8;  
   
   
 http://product.hubspot.com/bid/7049/MySQL-and-Unicode-Three-Gotchas  
   
 -----PAGE TABLE STATUS BEFORE UPGRADE-----  
 mysql> SHOW TABLE STATUS WHERE `NAME` LIKE '%page%';  
 +-------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+----------------+---------+  
 | Name       | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time     | Update_time | Check_time | Collation    | Checksum | Create_options | Comment |  
 +-------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+----------------+---------+  
 | page       | InnoDB |   10 | Compact  | 12461 |      127 |   1589248 |        0 |   2162688 | 11534336 |     14232 | 2015-11-01 10:40:06 | NULL    | NULL    | utf8_general_ci |   NULL |        |     |  
 | page_props    | InnoDB |   10 | Compact  |   0 |       0 |    16384 |        0 |      0 | 11534336 |      NULL | 2015-11-01 10:40:07 | NULL    | NULL    | utf8_general_ci |   NULL |        |     |  
 | page_restrictions | InnoDB |   10 | Compact  |   2 |      8192 |    16384 |        0 |    65536 | 11534336 |       3 | 2015-11-01 10:40:07 | NULL    | NULL    | utf8_general_ci |   NULL |        |     |  
 | pagelinks     | InnoDB |   10 | Compact  | 28377 |       56 |   1589248 |        0 |   1589248 | 11534336 |      NULL | 2015-11-01 10:40:07 | NULL    | NULL    | utf8_general_ci |   NULL |        |     |  
 +-------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+----------------+---------+  
 mysql> SHOW COLUMNS FROM page;  
 +-------------------+---------------------+------+-----+----------------+----------------+  
 | Field       | Type        | Null | Key | Default    | Extra     |  
 +-------------------+---------------------+------+-----+----------------+----------------+  
 | page_id      | int(10) unsigned  | NO  | PRI | NULL      | auto_increment |  
 | page_namespace  | int(11)       | NO  | MUL | NULL      |        |  
 | page_title    | varchar(255)    | NO  |   | NULL      |        |  
 | page_restrictions | tinyblob      | NO  |   | NULL      |        |  
 | page_counter   | bigint(20) unsigned | NO  |   | 0       |        |  
 | page_is_redirect | tinyint(3) unsigned | NO  |   | 0       |        |  
 | page_is_new    | tinyint(3) unsigned | NO  |   | 0       |        |  
 | page_random    | double unsigned   | NO  | MUL | NULL      |        |  
 | page_touched   | binary(14)     | NO  |   |        |        |  
 | page_latest    | int(10) unsigned  | NO  |   | NULL      |        |  
 | page_len     | int(10) unsigned  | NO  | MUL | NULL      |        |  
 +-------------------+---------------------+------+-----+----------------+----------------+  
 mysql> select page_id,page_title from page where page_title like 'Adhik%';  
 +---------+----------------------------+  
 | page_id | page_title         |  
 +---------+----------------------------+  
 |  11974 | Adhikamasa         |  
 |  11975 | Adhikamasa,_adhimasa    |  
 |  7128 | Adhikamāsa        |  
 |  6658 | Adhikamāsa,_adhimāsa   |  
 |  11977 | Adhikara          |  
 |  11978 | Adhikara_Nandi       |  
 |  11976 | Adhikarana         |  
 |  6660 | Adhikaraṇa        |  
 |  6659 | Adhikāra         |  
 |  6661 | Adhikāra_Nandi      |  
 |  7126 | Adhikamāsa,_adhimāsa   |  
 |  7117 | Adhikaraṇa        |  
 |  7125 | Adhikāra         |  
 |  7118 | Adhikāra_Nandi      |  
 +---------+----------------------------+  
 -----PAGE TABLE STATUS AFTER UPGRADE-----  
   
 +-------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+----------------+---------+  
 | Name       | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time     | Update_time | Check_time | Collation    | Checksum | Create_options | Comment |  
 +-------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+----------------+---------+  
 | page       | InnoDB |   10 | Compact  | 11883 |      133 |   1589248 |        0 |   2162688 | 111149056 |     14232 | 2015-11-01 10:40:06 | NULL    | NULL    | utf8_general_ci |   NULL |        |     |  
 | page_props    | InnoDB |   10 | Compact  |   0 |       0 |    16384 |        0 |      0 | 111149056 |      NULL | 2015-11-01 10:40:07 | NULL    | NULL    | utf8_general_ci |   NULL |        |     |  
 | page_restrictions | InnoDB |   10 | Compact  |   2 |      8192 |    16384 |        0 |    65536 | 111149056 |       3 | 2015-11-01 10:40:07 | NULL    | NULL    | utf8_general_ci |   NULL |        |     |  
 | pagelinks     | InnoDB |   10 | Compact  | 28765 |       55 |   1589248 |        0 |   1589248 | 111149056 |      NULL | 2015-11-01 10:40:07 | NULL    | NULL    | utf8_general_ci |   NULL |        |     |  
 +-------------------+--------+---------+------------+-------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+-------------+------------+-----------------+----------+----------------+---------+  
 mysql> select page_id,page_title from page where page_title like 'Adhik%';  
 +---------+----------------------------+  
 | page_id | page_title         |  
 +---------+----------------------------+  
 |  11974 | Adhikamasa         |  
 |  11975 | Adhikamasa,_adhimasa    |  
 |  7128 | Adhikamāsa        |  
 |  6658 | Adhikamāsa,_adhimāsa   |  
 |  11977 | Adhikara          |  
 |  11978 | Adhikara_Nandi       |  
 |  11976 | Adhikarana         |  
 |  6660 | Adhikaraṇa        |  
 |  6659 | Adhikāra         |  
 |  6661 | Adhikāra_Nandi      |  
 |  7126 | Adhikamāsa,_adhimāsa   |  
 |  7117 | Adhikaraṇa        |  
 |  7125 | Adhikāra         |  
 |  7118 | Adhikāra_Nandi      |  
 +---------+----------------------------+  
 }  
 // vim: ts=2:sw=2:noet:comments^=\:///  
   

Monday, January 12, 2015

Correlation matrix for my equity portfolio


I got interested in creating a Correlation matrix of my stock portfolio after the folks at Motley Fool Pro did it for their portfolio.  Attached is the calculator if anyone else is interested in replicating the exercise.

https://drive.google.com/file/d/0B4ehPprHSJUsVkY4amg1SW1SSzQ/view?usp=sharing

Monday, August 04, 2014

Journey Through Shastra

Collection of articles / links for research into my talk "Journey through Shastra"

http://www.mallstuffs.com/Blogs/BlogDetails.aspx?BlogId=232&BlogType=Spiritual&Topic=Science%20in%20Hinduism-Einstein%20Theory%20of%20relativity
http://www.gatewayforindia.com/articles/physics.htm
http://www.religiousforums.com/forum/science-religion/84199-scientific-miracles-hindu-scriptures-must-see.html
http://www.tamilbrahmins.com/sociology/11162-knowledge-biology-hindu-scriptures.html
http://www.hinduwebsite.com/hinduintrod1.asp
http://www.hinduwebsite.com/hinduism/hinduscriptures.asp
http://goaayurvedic.com/Achievements%20of%20Ancient%20India.htm
http://www.hinduism.co.za/hinduism.htm
https://www.archive.org/stream/cu31924031220126/cu31924031220126_djvu.txt
http://theindianhistory.org/ancinet-india-articles-information.html
http://www.culturalindia.net/indian-architecture/ancient-architecture/index.html
http://bharatjanani.com/achievements-of-ancient-india/
http://theindianhistory.org/ancinet-india-articles-information.html
http://ambedkarism.wordpress.com/2011/03/22/why-suddenly-the-brahmins-declare-the-vedas-to-be-infallible-and-not-to-be-questioned/
http://www.voiceofdharma.com/books/rig/ch3.htm
http://www.voiceofdharma.com/books/rig/ch4.htm
http://sankhyakarika.webstarts.com/uploads/RIGVEDA.pdf
http://vedamu.org/PageViewerImage.aspx?DivId=1815
http://vedamu.org/Veda.aspx
http://vedamu.org/PageViewerImage.aspx?DivId=2033
http://siddharthssinha.blogspot.com/2013/03/atharva-veda-part-of-four-vedas.html
http://siddharthssinha.blogspot.com/2013/01/sanatana-dharma-hinduism-part-4_26.html
http://siddharthssinha.blogspot.com/2013/01/sanatana-dharma-hinduism-part-4_14.html
http://www.iitk.ac.in/vs/vs/res/downloads/premendra_talk.pdf
http://www.arshabodha.org/HinduDharma/Lctr4.2_Satish_BrahmanaAranyaka.pdf
http://www.arshabodha.org/HinduDharma.html
http://www.wisdomlib.org/hinduism/book/a-history-of-indian-philosophy-volume-1/d/doc6986.html

Friday, July 11, 2014

Setting up a NAS

My Tonido Plug 1 crashed the other day and unfortunately, Tonido does not provide an updated image of Debian.  The stock image of Ubuntu they provide is horribly outdated and Ubuntu doesn't support ARM, so there is no way to get an updated version without spending a week compiling on the plug computer...

So, I decided to setup a virtual NAS server to run on my HTPC.  The applications I wanted to run are:
  • Tonido
  • Deluge
  • Logitech Media Server (for my Squeeze Box)

 

Setup virtual NAS on Oracle Virtual Box

Install Virtual Box from Oracle -- this is a free virtualization platform
Also install the Virtual Box extensions (also from Oracle and free)

 

Attaching a physical drive on Virtual Box

To attach a physical drive, you have to first create the vmdk
C:\Program Files\Oracle\VirtualBox>VBoxManage.exe internalcommands createrawvmdk
 -filename c:\Users\pc\Documents\vRangadevi\PhysicalDrive.vmdk -rawdisk \\.\Phys
icalDrive3
To get Virtual Box to actually use the disk follow these instructions (from here) if you are running a Windows Host (reproduced below)
  1. Close all open programs or documents on any partition on the disk to pass-through
  2. Run DISKPART (command line utility)
  3. Select hard drive carefully using SELECT DISK. disk numbering starts at zero. you can roughly verify your selection with LIST PARTITION if you know what your partitions should look like or LIST DISK to see disk sizes.
  4. Offline the disk using OFFLINE DISK. all volumes will disappear from windows explorer.
  5. ATTRIBUTES DISK CLEAR READONLY
  6. Verify with ATTRIBUTES DISK
You also need to start virtual box as "administrator"

use VBoxManage modifyhd --type=writethrough

Otherwise, if you use the snapshot feature, all future writes will go to the snapshot instead of the drive

Add the vmdk to the virtual NAS inside Virtual Box

 

Install Debian

I decided to create a second VMDK using Virtual Box to attach to my virtual NAS (and put it on my SSD -- not the physical disk that we just created in the previous step)

To install Debian,
  • Download Debian ISO from their site
  • Create a new Virtual Box instance w/ 512 mb ram, 5gb virtual HDD for the O/S partition (this was what my plug computer was setup with earlier). 
  • Boot off of the ISO and install w/ the following options
    • Select "File Server", "SSH server", and "Standard system utilities" options for packages to install
    • Reboot
su root
vi /etc/apt/sources.list (Comment out the reference to the debian cd)
apt-get update
Edit /etc/fstab to enable automount of physical disk

 

Install applications

apt-get install sudo samba deluged deluge-webui openvpn screen vim

wget http://prdownloads.sourceforge.net/webadmin/webmin_1.690_all.deb
dpkg -i webmin_1.690_all.deb
apt-get install -f -y
  • visudoers (to give your account ability to sudo)
  • vi /etc/samba/smb.conf
    • Change workgroup to your workgroup
    • allow hosts = 192.168.1.*
Add permissions for paths in Samba
[root]
    path = /media/disk2part1/
    browseable = yes
    read only = no
    guest ok = no
    force create mode = 0775
    force directory mode = 0775
    valid users =
Add any other users as needed

 

Setup Deluge

Setup deluge to work with a block-list

 

Install SqueezeBox Server

  • Install SqueezeBox server (Logitech media server)
  • Download from http://www.mysqueezebox.com/download
sudo dpkg -i

 

Install Tonido

wget http://www.tonido.com/download.php?TonidoSetup_i686.deb
sudo dpkg -i
sudo apt-get install -f -y

 

Configure OpenSSL

  • Create conf file with details of vpn provider (name it .conf)
  • Create password file & reference in conf file
  • Edit /etc/init.d/openvpn to add "NAME" variable which contains of conf file
  • To add to system boot list
update-rc.d deluge-daemon defaults
invoke-rc.d deluge-daemon start

 

Setup IP Tables

Setup IP tables as doing so will close unnecessary ports and secure the server To autoload at startup: https://wiki.debian.org/iptables

Install vboxadditions

To install vboxadditions, you need to install header files first...

sudo apt-get install linux-headers-3.2.0-4-486 
mount -t iso9660 -o ro /dev/sr0 /media/cdrom0
/media/cdrom0/VBoxLinuxAdditions.run

Force FSCK to check HDD after every reboot

edit /etc/rc.local and add 
touch /forcefsck
echo "-V -- -v" > /fsckoptions

Install Dante proxy

Dante will enable you to proxy internal network traffic through the vpn tunnel (tun0)

Common URLs/ports for reference

Deluge: http://:8112 
Webmin: https://:10000
Logitech Media Server: http://:9001 
Tonido: http://:10001